2 Comments

Summary:

Netflix open-sources Glisten to make it faster and easier for more customers to use Amazon Web Services’ Simple Workflow Service (SWF)

netflixoss

Netflix is at it again. The streaming media giant just released another tool — this one called Glisten — which aims to help more companies make better use of Amazon’s Simple Workflow Service (SWF) and associated APIs. Source code for Glisten, characterized in the Netflix blog, as a “Groovy way to use Amazon’s Simple Workflow Service”  is now available on Github.

As Netflix senior software engineer Clay McCoy wrote, the company ran into problems with its in-memory task system, which could slow to a crawl when Netflix wanted to add features to Asgard, a popular Netflix application for deploying cloud applications.

Amazon addressed this performance issue with a new higher-level Flow Framework API atop SWF. Flow Framework minimizes SWF-based boilerplate code and makes the workflow “look more like ordinary Java code,” McCoy wrote. But there was a problem using that API with Asgard, probably one of the most popular Netflix open-source tools: the new AWS API is AspectJ oriented whereas Asgard is written in Groovy and Grails.

“Since Groovy itself is well suited to the job of hiding boilerplate code we began to wonder if we could use it to get what we wanted from SWF,” McCoy wrote.

McCoy describes Glisten as:

“a lightweight way to use SWF and only requires a dependency on Groovy. Most of your code can still be written in Java if you prefer. Glisten is currently used in Asgard to enable long lived deployment tasks. There is a comprehensive example workflow in the Glisten codebase and documented on the wiki. It demonstrates many SWF [simple workflow] features (timers, parallel tasks, retries, error handling…) along with unit tests.”

As Server Density CEO David Mytton sees it, Glisten is a simple abstraction layer atop SWF, which frees users from having to dive down into complex Java-like code for every job. “It’s an easy way of achieving a specific, common task [but] if you need more complexity you can bypass it and get back into SWF syntax,” Mytton said via email.

But some see a downside. “Netflix continues to embrace AWS features that increase their lock-in. I think they will ultimately regret this strategy,” said Joe Masters Emison, CTO of BuildFax, Asheville, N.C.

Emison’s comment reflects the belief of many companies that the use of base-level AWS compute and storage resources is fine, but the adoption of  higher-level services will chain them to the Amazon cloud. Third-party options for designing and implementing workflows include Opscode Chef, GigaOM Structure Launchpad winner Saltstack, Ansible as well as Puppet (see disclosure) and RightScale CloudFlow.

Both the upside and downside of Amazon’s cloud is the company’s continued delivery of higher-level services atop its basic infrastructure. The more a customer moves up that stack, the harder it will be to move off of AWS should that need arise.

DisclosurePuppet Labs is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.

  1. Mytton accurately explains that Glisten is an abstraction layer, but the article doesn’t make the connection that this abstraction layer can actually insulate from AWS specifics. Glisten already has a non SWF implementation that is used for testing. Open sourcing allows the community to take that further if they wish. The docs have been updated to make this point a little more clear.
    https://github.com/Netflix/glisten/wiki/Basics#glistens-abstraction

    Share
    1. thanks for the clarification

      Share

Comments have been disabled for this post