GigaOM » Eucalyptus Systems

For developers, the cloud means having to rethink everything they know about making software

Marten Mickos, Guest Contributor — Sun, 19 May 2013 17:30:50 +0000

The paradigm hasn’t changed since the advent of software: Applications run, and platforms are what they run on. But the underlying principles of application design and deployment do change every now and then – sometimes drastically, thanks to quantum-leap developments in infrastructure.

For instance, application design principles changed dramatically when the PC, x86 architecture, and client/server paradigm were born in the ’80s. And it happened again with the advent of the web and open-source technology in the mid ’90s. Whenever such abrupt changes arise, application developers are forced to rethink how they build and deploy their software.

Today, we’re seeing a huge leap in infrastructure capability, this time pioneered by Amazon Web Services. It’s clear that to take full advantage of the new cloud infrastructure, applications that run successfully on AWS must be inherently different than applications that were built to run successfully on a corporate server – even a virtualized one. But there are a number of other particular ways in which today’s (and tomorrow’s) cloud applications will need to be designed differently than in the past. Here are the most crucial ones, and how the ways of the old world have been changed in the new one :

Scaling

In the old world, scaling was accomplished by scaling up – to accommodate more users or data, you simply bought a bigger server.

In the new world, scaling is typically done by scaling out. You don’t add a bigger machine, you add multiple machines of the same sort. In the cloud world, those machines are virtual machines, and their instantiations in the cloud are instances.

Resilience

Before, software was seen as unreliable, and resilience was built into the hardware layer.

Today, the underlying infrastructure – the hardware – is seen as the weak link, and it is up to applications to accommodate for this. There is no guarantee that a virtual machine instance will always function. It can disappear at any moment and the application must be prepared for this.

By way of example, Netflix, arguably the most advanced user of the cloud today, has gone the farthest in adopting this new paradigm. They have a process called ChaosMonkey that randomly kills virtual machine instances from underneath the application workloads. Why on earth do they do this on purpose? Because they are ensuring uptime and resilience: By exposing their applications to random loss of instances, they force application developers to build more resilient apps. Brilliant.

Bursting

In the old world – think accounting and payroll applications – the application workload was reasonably stable and predictable. It was known how many users a system had, and how many records they were likely to process at any given moment.

In the new world, we see variable and unpredictable workloads. Today’s software systems have to reach farther out in the world, to consumers and devices that demand services at unpredictable moments and unpredictable loads. To accommodate such unforeseen fluctuations in individual application workloads required a new software architecture. We now have it in the cloud, but clearly it is still in its infancy.

Software variety

In the past we didn’t have much software variety. Each application was written in one language and used one database. Companies standardized on a single, or at least very few operating systems. The software stack was boringly simple and uniform (at least now in retrospect).

In the new world of cloud, the opposite is happening. Within a single application, many different languages can be used, many different libraries and toolkits can be employed, and many different database products can be used. And because in a cloud you can create and spin up your own image, tailored to your and your application’s specific needs, applications within one company must be able to operate under a spectrum of configurations.

From VM to cloud

Even between the relatively new technology of hypervisors and the modern cloud thinking, there are differences. VMware, the pioneer and leader in virtualization, built its hypervisors to essentially behave the way physical machines did before.

But in the cloud world, the virtual machine is not a representation of a physical server; it’s a representation of units of compute. (Steve Bradshaw wrote about this topic in depth.)

User patience

In the old world, users were taught to be patient. The system may have needed a long time to respond to simple retrieval or update requests, and new features were added slowly to the application (if at all).

In the new cloud world, users have no patience. They hardly tolerate latency or wait times, and they look for improvements in the service every week, if not every day. Evidence of this can be found in self-service IT. Rather than file a ticket with IT and wait for a response several days later, users of IT can self-provision the resources they need.

Do these observations rhyme with what you are experiencing and taking action on in your organization? I look forward to comments and debate on this topic.

Marten Mickos is the CEO of Eucalyptus Systems. He previously served as CEO of MySQL AB, which was acquired by Sun Microsystems. He is a member of the board of directors of Nokia.

Have an idea for a post you’d like to contribute to GigaOm? Click here for our guidelines and contact info.

Photo courtesy of Mike Flippo/Shutterstock.com.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Don’t look to Europe to create an Amazon rival

Ryan Kim — Wed, 17 Oct 2012 11:17:07 +0000

Amazon is trying to be the Amazon of Europe. But could a homegrown cloud player take that title? It’s not going to be easy, said Marten Mickos, the CEO of Eucalyptus Systems at the GigaOM Structure Europe conference.

Mickos said Europe has some advantages thanks to its 500 million Internet users and population density, which helps with latency. And there are pockets of cloud innovation in the Netherlands, Germany, Scandinavia and even Spain. But he said Europe also struggles with its own set of issues.

“But then you have all of the inertia in this continent and resistance to change and a refusal to acknowledge some changes that are happening in the world,” he said.

Competing with Amazon overall is hard work, he said, especially considering that Amazon has been working on the cloud for six years. A new global competitor might be more likely out of China, which has four times the number of Internet users than the US.

“That’s a fundamental number that drives cloud adoption,” Mickos said.

Europe also doesn’t have a history of creating a pan-European competitor. In most cases, companies look to go global instead of focusing on Europe, Mickos said. That’s why you see more American companies with pan-European products.

But with startups and fast-growing companies emerging in Europe, it’s likely to propel home-grown cloud development. But it may be that Europe develops mobile clouds faster, said Mickos.

Check out the rest of our Structure Europe 2012 live coverage here, and a video recording of the session follows below.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Infrastructure Q1: Cloud and big data woo enterprises

Thu, 19 Apr 2012 06:55:04 +0000

With enterprises now open to hybrid clouds, Amazon Web Services finally relaxed its rigid public-cloud-only stance and launched services to support hybrid-cloud deployments in the first quarter of 2012. On the big data front, the Hadoop players realized very few companies have teams of systems engineers to learn MapReduce. This has meant adding support for SQL and integrating Hadoop with existing data-management tools and systems. In other words, Hadoop has grown up and is now being taken seriously by companies like Oracle and Microsoft. This quarterly report examines these trends as well the exciting M&A and IPO news in this arena. It also includes a near-term outlook for the next 12–18 months.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Amazon, Eucalyptus team up for just don’t call it a hybrid cloud

Om Malik — Thu, 22 Mar 2012 07:01:16 +0000

On paper it looks like one of those strategic agreements that large companies sign-up with small startups. But if you look close enough, you will find that by forging a partnership with Santa Barbara, Calif.-based Infrastructure as a service (IaaS) provider, Eucalyptus Systems, Amazon.com’s web services division is making a masterful move to dominate the cloud market. As part of the agreement, Amazon will allow Eucalyptus to use its “api” to connect Amazon’s cloud services with the private clouds set-up by large companies.

From the press release:

….an agreement that enables customers to more efficiently migrate workloads between their existing data centers and AWS while using the same management tools and skills across both environments. As part of this agreement, AWS will support Eucalyptus as they continue to extend compatibility with AWS APIs and customer use cases. Customers can run applications in their existing datacenters that are compatible with popular Amazon Web Services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

Marten Mickos, CEO, Eucalyptus Systems @ the Future of Cloud panel at Structure 2011.

Eucalyptus CEO Marten Mickos who happened to be in New York met with me briefly (I am at our Structure Data conference) and the beaming smile on this said it all. It was clearly a big shot in the arm for his company that started to face competition from many different quarters including the strong efforts of the Open Stack companies.

While the idea of a hybrid ~~crowd~~ cloud– one that marries public clouds such as Amazon Web Services with private or company owned clouds – has gained popularity, Amazon has been steadfast in its views on what it deems to be the cloud. The Seattle-based retailer believes that cloud is essentially what it sells as S3, EC2 and other related services.

However, Amazon might be softening its stance in small doses. Before today’s news, in Janauary 2012 the company announced Storage Gateway, a way to fuse on-premise storage with its S3 service. A month later, the company announced Simple Workflow Service (SWF) that blurred the boundaries between applications that ran on internal clouds and Amazon’s public cloud.

When asked if this was a tactical admission that Amazon needs a hybrid cloud option, Kay Kinton, a company spokesperson denied and added that is is recognition that the

“integration between on-premises infrastructure and the AWS cloud is a use case that’s important to some of our customers. In addition to this integration with Eucalyptus, we’ve steadily been releasing other features that enable this integration between on-premises infrastructure and AWS, including Amazon Virtual Private Cloud (Amazon VPC), DirectConnect, and our Identity and Access Management service that many customers integrate to their on-premise identity systems.”

It seems that large companies are increasingly asking more from Amazon about ways to merge their infrastructures. When I asked if an all-public cloud option was over for large enterprises, an Kinton wrote back in an email:

Not at all and we believe that over time, most enterprises will not run their own datacenters and those that do will have a much smaller footprint. That said, many enterprises today have legacy applications and a good deal of investment in those legacy applications. This type of arrangement provides the added flexibility to more freely move workloads between those existing IT environments and AWS.

Amazon says that “a typical scenario for using this service could be a corporation that uses AWS for disaster recovery and who will need to move workloads between AWS and Eucalyptus frequently for testing purposes.” Other scenarios include companies tapping AWS infrastructure in case of spike in demand for an application that resides on Eucalyptus.

The new agreement doesn’t impact cloud management console makers such as RIghtscale since ”the management system built around AWS is actually assisted by having another compatible implementation of the AWS APIs.”

When I think about the news, the message is pretty clear: Amazon will do whatever it takes to keep growing its lead in the “cloud” market, even if it means partnering with private cloud providers such as Eucalyptus.

The press release:

Eucalyptus and Amazon Web Services partner to bring additional compatibility between AWS and on-premises IT environments

AWS and Eucalyptus to make it easier for customers to migrate applications between on-premises environments and the cloud

SEATTLE, WA. – March 22, 2012 – Amazon Web Services LLC (AWS), an Amazon.com company and the leader in cloud computing (NASDAQ: AMZN), and Eucalyptus Systems, a provider of on-premises Infrastructure as a Service (IaaS) software, today announced an agreement that enables customers to more efficiently migrate workloads between their existing data centers and AWS while using the same management tools and skills across both environments. As part of this agreement, AWS will support Eucalyptus as they continue to extend compatibility with AWS APIs and customer use cases. Customers can run applications in their existing datacenters that are compatible with popular Amazon Web Services such as Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

“We’re pleased to provide customers with the added flexibility to more freely move workloads between their existing IT environments and the AWS cloud,” said Terry Wise, Director of Amazon Web Services Partner Ecosystem. “Enterprises can now take advantage of a common set of APIs that work with both AWS and Eucalyptus, enabling the use of scripts and other management tools across both platforms without the need to rewrite or maintain environment-specific versions. Additionally, customers can leverage their existing skills and knowledge of the AWS platform by using the same, familiar AWS SDKs and command line tools in their existing data centers.”

“We’re excited to be working more closely with AWS,” said Marten Mickos, CEO of Eucalyptus. “The ability to develop against a common set of market-leading APIs, for both on-premise and cloud deployments, is a big benefit for our customers and software partners. This agreement is going to accelerate our roadmap, and help us maintain our compatibility with AWS as both companies continue to innovate.”

“The cloud is becoming an increasingly important foundation for our data center operations, which support more than 600,000 hotel rooms under seven brands,” said Scott Johnson, Vice President of Enterprise Engineering at InterContinental Hotel Group. “As a user of both Amazon Web Services and Eucalyptus, this agreement will provide us with even greater flexibility and scalability for our cloud deployments. We believe the ability to leverage a common set of APIs will significantly transform our operations and provide a business advantage in the competitive hospitality industry.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Infrastructure Q3: OpenStack and flash step into the spotlight

Thu, 13 Oct 2011 20:36:23 +0000

Last quarter we highlighted the fast maturation of the Platform-as-a-Service and big data spaces. Those two trends only picked up speed during the third quarter of 2011. Joining them on the cusp of IT greatness, though, are the OpenStack project and flash storage. The former gathered serious validation from big-name companies, while the latter saw less funding than last quarter but a significant number of product launches. Of course, the third quarter wasn’t all lollipops and rose petals. We saw new computing technologies and delivery models such as tablets wreak havoc on both HP and Cisco, and there are concerns (aren’t there always?) about how the Internet will handle our increased use of streaming video and cloud computing. Unfortunately for HP and Cisco, the latter problem might be an easier fix than the strategic woes facing them. Additional companies mentioned in this report include CloudBees, Rackspace, Engine Yard and Joyent. For a full list of companies, and to read the full report, sign up for a free trial.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Competition for the private cloud heats up

Paul Miller — Thu, 01 Sep 2011 20:00:09 +0000

Despite OpenStack’s continued growth, a combination of product updates and acquisitions from Citrix, Eucalyptus, Red Hat and VMware over the past week demonstrate that the race to become the dominant private cloud provider, as well as win over the enterprise, is far from over. Is one of [...]

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Despite growth in open source, commercial solutions still dominate

Paul Miller — Thu, 21 Jul 2011 23:57:17 +0000

Open-source cloud computing project OpenStack turned one this week, and cloud backup provider Backblaze freely shared detailed specifications for a storage device capable of holding 135 terabytes of data. Open-source options for everything from servers and data centers to clouds and application platforms are growing more robust. [...]

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

NEA’s Sonsini: Cloud infrastructure still king for VC cash

Derrick Harris — Wed, 15 Jun 2011 22:15:42 +0000

Peter Sonsini

Apple’s iCloud has brought consumer-focused cloud computing into the limelight recently, and the trend probably represents a golden opportunity for startups looking to raise venture capital. However, the brunt of investment won’t necessarily go toward companies trying to compete with Apple or launch their own consumer clouds.

According to New Enterprise Associates partner Peter Sonsini, the key to getting VC investment in cloud computing is still selling the infrastructure tools that underlie popular cloud platforms.

During an interview earlier this week, Sonsini told me that although consumer cloud services are hot, much of the investment money still comes from the enterprise software and hardware upon which those clouds are built. Whether it’s Apple, Dropbox or anyone else targeting consumers with new cloud services, the one commonality across the board is that they’ll need to buy some cutting-edge IT products. Not that this should be surprising coming from Sonsini, who was part of VMware’s early executive team and whose investment portfolio reads like a who’s who of the cloud.

As senior director of strategic alliances during VMware’s formative years, he learned early on how important virtualization would come to be. When he got into the VC game, one of his first investments was open-source virtualization startup XenSource (which Citrix bought and which still provides the moniker for Citrix’s virtualization lineup). Among his current companies are PaaS pioneer Engine Yard, private-cloud vendor Eucalyptus Systems, .NET-cloud enabler Apprenda and Hadoop Distributed File System alternative MapR. On Wednesday, virtual-machine-storage specialist Tintri announced that NEA co-led an $18 million Series C round. It’s one of Sonsini’s deals.

I asked Sonsini his feelings on Eucalyptus, which has garnered a lot of criticism (fairly or unfairly) since OpenStack burst onto the scene and stole Eucalyptus’s open-source cloud thunder. He noted that NEA invested in Eucalyptus pre-OpenStack, but that he thinks a strategy adjustment can right the company’s ship to the degree it’s even off-course. These things always come along, he added, and you just have to deal with them. Regardless, NEA is still “very bullish” on private clouds, Sonsini said, calling them a “top five” initiative for most of the CIO types he speaks with.

Apprenda also sells private-cloud software, although it’s targeted toward creating Software-as-a-Service platforms for .NET applications.

Other than cloud and general enterprise infrastructure, Sonsini said other areas of interest for him are big data (hence MapR) and the consumerization of IT.

Feature image courtesy of Flickr user stevendamron.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

RightScale brings Zynga-like hybrid clouds to the masses

Derrick Harris — Tue, 07 Jun 2011 14:55:06 +0000

Cloud-management platform provider RightScale is launching a new service to help customers manage private and hybrid clouds similar to what RightScale customer Zynga does with its vaunted Z Cloud infrastructure. Hybrid cloud computing appears to be the end-game for many companies’ cloud strategies right now, and RightScale’s new MyCloud service might represent an ideal way of thinking about the model.

RightScale’s new offering provides the proverbial single pane of glass through which to manage private-cloud infrastructure based on either Eucalyptus Systems or Cloud.com as well as public cloud-based resources that are managed via RightScale. And although Zynga built its RightScale-based cloud setup from scratch, RightScale CEO Michael Crandell said it offers what might be the ideal model for doing hybrid cloud computing. Rather than focusing on simple cloudbursting, which Crandell calls “naive” in certain situations, Zynga honed in on standardization as a focus of its cloud efforts.

By utilizing standard server templates, Zynga is able to stage, test and launch games in Amazon EC2, then easily migrate them to its private cloud when the time comes by launching a cluster of identically configured servers in-house. Last week, RightScale opened its MultiCloud Marketplace, which is like an app store for infrastructure configurations that features a wide range of templates for everything from web servers to Hadoop clusters. With MyCloud, users can now create their own Zynga-like experience by utilizing Marketplace templates on private cloud infrastructure as well as in the public cloud.

Another use case for MyCloud, Crandell said, might be to spread different components of an application architectures across both public and private resources. Depending on bandwidth costs and the need for high-speed interconnects, it might make sense for certain parts (e.g., an on-demand processing engine) to run in the public cloud while others remain on premise.

Crandell will be present at Structure 2011, being held June 22-23 in San Francisco, to discuss the future of cloud application platforms and what issues they’ll need to address as mainstream interest picks up. MyCloud is currently available for free through an early access program, although more-functional, and paid, Standard and Enterprise versions are on the way.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

The Structure 50: The Top 50 Cloud Innovators

Thu, 19 May 2011 00:01:53 +0000

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.