Netflix is at it again. The streaming media giant just released another tool — this one called Glisten — which aims to help more companies make better use of Amazon’s Simple Workflow Service (SWF) and associated APIs. Source code for Glisten, characterized in the Netflix blog, as a “Groovy way to use Amazon’s Simple Workflow Service” is now available on Github.
As Netflix senior software engineer Clay McCoy wrote, the company ran into problems with its in-memory task system, which could slow to a crawl when Netflix wanted to add features to Asgard, a popular Netflix application for deploying cloud applications.
Amazon addressed this performance issue with a new higher-level Flow Framework API atop SWF. Flow Framework minimizes SWF-based boilerplate code and makes the workflow “look more like ordinary Java code,” McCoy wrote. But there was a problem using that API with Asgard, probably one of the most popular Netflix open-source tools: the new AWS API is AspectJ oriented whereas Asgard is written in Groovy and Grails.
“Since Groovy itself is well suited to the job of hiding boilerplate code we began to wonder if we could use it to get what we wanted from SWF,” McCoy wrote.
McCoy describes Glisten as:
“a lightweight way to use SWF and only requires a dependency on Groovy. Most of your code can still be written in Java if you prefer. Glisten is currently used in Asgard to enable long lived deployment tasks. There is a comprehensive example workflow in the Glisten codebase and documented on the wiki. It demonstrates many SWF [simple workflow] features (timers, parallel tasks, retries, error handling…) along with unit tests.”
As Server Density CEO David Mytton sees it, Glisten is a simple abstraction layer atop SWF, which frees users from having to dive down into complex Java-like code for every job. “It’s an easy way of achieving a specific, common task [but] if you need more complexity you can bypass it and get back into SWF syntax,” Mytton said via email.
But some see a downside. ”Netflix continues to embrace AWS features that increase their lock-in. I think they will ultimately regret this strategy,” said Joe Masters Emison, CTO of BuildFax, Asheville, N.C.
Emison’s comment reflects the belief of many companies that the use of base-level AWS compute and storage resources is fine, but the adoption of higher-level services will chain them to the Amazon cloud. Third-party options for designing and implementing workflows include Opscode Chef, GigaOM Structure Launchpad winner Saltstack, Ansible as well as Puppet (see disclosure) and RightScale CloudFlow.
Both the upside and downside of Amazon’s cloud is the company’s continued delivery of higher-level services atop its basic infrastructure. The more a customer moves up that stack, the harder it will be to move off of AWS should that need arise.
Disclosure: Puppet Labs is backed by True Ventures, a venture capital firm that is an investor in the parent company of this blog, Giga Omni Media. Om Malik, founder of Giga Omni Media, is also a venture partner at True.