Wow…this came as a complete and total surprise. VMware has announced that Diane Greene, president and CEO of the hot virtualization company is leaving and will be replaced by Paul Maritz, whose company, Pi Corp., was acquired by EMC Corp. back in February. I am flummoxed by this move since VMware has been on an upswing and despite increased competition, has been pretty bullish about the future.

Greene didn’t give any hint to her departure when I met her at a recent tech gathering. More importantly, when I was hanging out with VMware co-founder (and Greene’s spouse) Mendel Rosenblum at our Structure 08 conference, he pointed to rosy skies ahead. At the bottom of the press release announcing her departure, however, is information indicating that the company might be facing a rough 2008, which explains the sudden change in management.

VMware expects to announce earnings for the quarter ended June 30, 2008 as scheduled on July 22, 2008 at 2pm PDT. On that call Paul will make observations about the second half of 2008. While VMware is not updating guidance for Q2, we expect revenues for the full year of 2008 will be modestly below the previous guidance of 50% growth over 2007.

The markets aren’t too happy — the stock has tanked more than 30 percent already to as low as $36.51 a share. Shares of EMC are taking a pounding as well, falling as much as 13 percent to change hands for $13.18.

I think something big is going on — no CEO and co-founder just up and quits the company. The numbers might be worse than they seem. Did Greene pay the price for the missed numbers or is there something else going on? 

I couldn’t help but notice the fact that Joe Tucci, chairman of VMware’s board and CEO of EMC, was all over the press release announcing Greene’s departure, with not so much as a word from her, Mendel or any of the VMware co-founders. Of course, the fact that Diana’s replacement, Maritz, works for EMC only adds to the mystery.

By way of background, Maritz retired from Microsoft in 2000 and in 2003 started Pi Corp., a software startup focused on building cloud-based solutions for new ways of doing personal information management. Pi Corp. was acquired by EMC last February, and Maritz became president of EMC’s cloud division. Greene and her four co-founders launched VMware back in 1998. The road leading up to the company’s blockbuster IPO and subsequent stock market darling status was a long one.

Update from Stacey: Greene’s departure is likely more politically than financially motivated, according to sources in the virtualization community. They point to friction between Greene and her bosses at EMC, as well as EMC’s worries about Greene as the CEO of a publicly traded company as the reasons behind her departure. Most people suspect we’ll see her at the helm of another startup within a few months. VCs will certainly be calling her — if not today, then tomorrow.

For a look at how open source figures into the story, check out OStatic.

When visiting Israel in the middle of summer, it’s generally not a good idea to go for a walk in the afternoon, even if it is along the sea. The heat and humidity sap your energy, making you feel as if you spent nearly three hours in the gym. But that wasn’t enough to stop me from writing a post about Microsoft buying Powerset for what is rumored to be around $100 million.

I’ve been unable to stop wondering why founder Barney Pell decided to take the money and run — after all, he used to turn blue in the face telling people how superior Powerset’s approach to search was. If it was so superior, Mike Masnick of Techdirt put it best when he wrote that “[T]he exit certainly falls well short of the hype around Powerset. If Powerset was actually seeing any traction at all it never would have agreed to sell at that price.”

To some extent, Mike is right, but I would add another reason: infrastructure, specifically how expensive it is to build. At our Hadoop meet-up earlier this year, Chad Walters, director of engineering at Powerset, noted that their search “requires 100 times more processing than simple keyword searching and indexing (about one second per sentence is required for processing).”

Powerset used some pretty nifty technologies to build out their system, but in order to really scale, they would have needed more money — a lot of it.

And Powerset would have had to scale; there’s no other way to compete with search’s 800-pound gorilla, Google. That’s why Microsoft is building a gigantic data center in the Chicago area focused almost entirely on search. (Which it can now use to help roll out Powerset’s search technology to a larger audience.)

This is an abject lesson for every startup looking to get into the business of search: No matter how good your algorithms are, you still have to deal with the cost of queries, which need to be low enough to be offset by some kind of advertising in order to make a profit. (The conspiracy theorist in me says that if your results are really good you won’t be able to generate enough inventory to serve up ads that bring in the dollars, but maybe I’m just too cynical.)

One of our readers believes that it is possible to build a search engine that surpasses Google’s. Nevertheless, as I’ve noted in the past, “[P]rocess-optimized infrastructure ensures that Google’s cost of executing a query keep going down” — and that allows the company to wring more dollars from the system.

Given all that, Powerset has done a good job of wringing a hundred million from Microsoft. Not that there’s anything wrong with that.

Bonus Link: Don Dodge of Microsoft explains the logic behind the deal.

Virtualization underpins cloud computing by making it possible to separate the software from the hardware. So far the dominant player has been VMware, with about 95 percent of the market, but Simon Crosby, CTO of Citirix and former CTO of open source virtualization company XenSource (which Citrix acquired last year), plans to take that market back. The launch of Microsoft’s Hyper-V hypervisor is part of of his strategy. The rest revolves around services that “play nicely with others,” and free hypervisors embedded into servers and operating systems.

GigaOM: Can you tell me how the launch of Hyper-V affects Citrix Xen products?

Crosby: Our key founding philosophy was fast, free, compatible and ubiquitous hypervisors. Microsoft’s Hyper-V which is compatible with XenServer, is alright when it comes to being fast; it’s 28 bucks, so close to free; and because it’s Microsoft it will be ubiquitous. So for us, it’s good. The problem is it took them too darn long to get it out. Working with Microsoft has been a little bit like having a ring through the nose of the bull. We have a rope tied to that ring because we’re ahead of them on this thing, but when they charge I’m going to get out the way and point them at VMware.

GigaOM: But will Hyper-V compete with the Citrix server virtualization business anyway?

Crosby: You should look forward to interesting announcements of products to add value to Hyper-V. We’re going to sell into that footprint much like Citrix has always extended the use cases of Microsoft products.

GigaOM: What about VMware?

Crosby: We only have a 4 percent or 5 percent share in this market, and the market is significantly overpaying for what they have today, so it’s a very, very interesting time. We’re going to track VMware down with the fast, free, compatible and ubiquitous hypervisor and sell on top of that. We’ve accepted that hypervisors are not the stuff you can charge for. It has taken longer than I thought to get there, and customers have yet to decide, too, if the hypervisors are part of the box or in the operating system. We’ll be wherever we can to create for ourselves the largest possible upsell with other products.

GigaOM: So we’re early on in this game?

Crosby: Virtualization is reorganizing the IT industry. Separating the software from hardware allows more services oriented on the software stack. It creates this huge power vacuum in the industry, and everyone is rushing to fill it. Virtualization becomes a tool for differentiation for a former commodity box maker. Look at our deal with Symantec and its Veritas products. That’s a profoundly important play because, in the enterprise, what you’re about to see is companies entering the virtualization market with a commodity hypervisor and a clear intent to upsell the products.

GigaOM: What will virtualization mean for storage?

Crosby: With Xen, multiple servers will automatically pool, and from these resource pools customers get dynamism and availability. VMware turns the storage industry into a dumb block of boxes, while we have a storage model that allows us to leverage our software to the let the storage infrastructure participate in the value chain, and the storage industry works with us very closely. Microsoft is completely missing from storage.

GigaOM: What about desktop virtualization? Unlike the server side, you guys have a lot of competition ready to pounce.

Crosby: Yes, and that opportunity is of great interest to both of us. Arguably the remote delivery of apps is what we have been doing for 18 years at Citrix. We have always cared about the line of sight between an app in the data center and the desktop. We’re very confident and have opened up the category, but everyone and their dog are in there too. We’re watching 10 to 12 other offerings, but we just see a lot of smoke and not a lot of fire.

GigaOM: After storage, where is virtualization heading next?

Crosby: There’s a lot happening with I/O virtualization and the creation of these fabrics for information flow. Fibre Channel won’t roll over and die, but some of the Ethernet stuff is really interesting. Backing away this thing that has always been proprietary presents interesting opportunities. When you virtualize the resources of a single compute memory you create a new type of system where Xen is the virtualization engine, because it’s not proprietary.

GigaOM: So you’re describing a cloud?

Crosby: To the extent that the clouds are relevant, the largest virtualization effort is Amazon Web Services. Xen will be in every cloud. The only cloud that it won’t be in is Microsoft’s and that will be running Hyper-V. So that’s an interesting path as clouds become an opportunity for some IT functions to be outsourced.

Many entrepreneurs today have their heads in the clouds. They’re either outsourcing most of their network infrastructure to a provider such as Amazon Web Services or are building out such infrastructures to capitalize on the incredible momentum around cloud computing. I have no doubt that this is The Next Big Thing in computing, but sometimes I get a little tired of the noise. Cloud computing could become as ubiquitous as personal computing, networked campuses or other big innovations in the way we work, but it’s not there yet.

Because as important as cloud computing is for startups and random one-off projects at big companies, it still has a long way to go before it can prove its chops. So let’s turn down the noise level and add a dose of reality. Here are 10 reasons enterprises aren’t ready to trust the cloud. Startups and SMBs should pay attention to this as well.

1. It’s not secure. We live in an age in which 41 percent of companies employ someone to read their workers’ email. Certain companies and industries have to maintain strict watch on their data at all times, either because they’re regulated by laws such as HIPAA, Gramm-Leach Bliley Act or because they’re super paranoid, which means sending that data outside company firewalls isn’t going to happen.
2. It can’t be logged. Tied closely to fears of security are fears that putting certain data in the cloud makes it hard to log for compliance purposes. While there are currently some technical ways around this, and undoubtedly startups out there waiting to launch their own products that make it possible to log “conversations” between virtualized servers sitting in the cloud, it’s still early days.
3. It’s not platform agnostic. Most clouds force participants to rely on a single platform or host only one type of product. Amazon Web Services is built on the LAMP stack, Google Apps Engine locks users into proprietary formats, and Windows lovers out there have GoGrid for supporting computing offered by the ServePath guys. If you need to support multiple platforms, as most enterprises do, then you’re looking at multiple clouds. That can be a nightmare to manage.
4. Reliability is still an issue. Earlier this year Amazon’s S3 service went down, and while the entire system may not crash, Mosso experiences “rolling brownouts” of some services that can effect users. Even inside an enterprise, data centers or servers go down, but generally the communication around such outages is better and in many cases, fail-over options exist. Amazon is taking steps toward providing (pricey) information and support, but it’s far more comforting to have a company-paid IT guy on which to rely.
5. Portability isn’t seamless. As all-encompassing as it may seem, the so-called “cloud” is in fact made of up several clouds, and getting your data from one to another isn’t as easy as IT managers would like. This ties to platform issues, which can leave data in a format that few or no other cloud accepts, and also reflects the bandwidth costs associated with moving data from one cloud to another.
6. It’s not environmentally sustainable. As a recent article in The Economist pointed out, the emergence of cloud computing isn’t as ethereal as is might seem. The computers are still sucking down megawatts of power at an ever-increasing rate, and not all clouds are built to the best energy-efficiency standards. Moving data center operations to the cloud and off corporate balance sheets is kind of like chucking your garbage into a landfill rather than your yard. The problem is still there but you no longer have to look at it. A company still pay for the poor energy efficiency, but if we assume that corporations are going to try to be more accountable with regard to their environmental impact, controlling IT’s energy efficiency is important.
7. Cloud computing still has to exist on physical servers. As nebulous as cloud computing seems, the data still resides on servers around the world, and the physical location of those servers is important under many nation’s laws. For example, Canada is concerned about its public sector projects being hosted on U.S.-based servers because under the U.S. Patriot Act, it could be accessed by the U.S. government.
8. The need for speed still reigns at some firms. Putting data in the cloud means accepting the latency inherent in transmitting data across the country and the wait as corporate users ping the cloud and wait for a response. Ways around this problem exist with offline syncing, such as what Microsoft Live Mesh offers, but it’s still a roadblock to wider adoption.
9. Large companies already have an internal cloud. Many big firms have internal IT shops that act as a cloud to the multiple divisions under the corporate umbrella. Not only do these internal shops have the benefit of being within company firewalls, but they generally work hard — from a cost perspective — to stay competitive with outside cloud resources, making the case for sending computing to the cloud weak.
10. Bureaucracy will cause the transition to take longer than building replacement housing in New Orleans. Big companies are conservative, and transitions in computing can take years to implement. A good example is the challenge HP faced when trying to consolidate its data center operations. Employees were using over 6,000 applications and many resisted streamlining of any sort. Plus, internal IT managers may fight the outsourcing of their livelihoods to the cloud, using the reasons listed above.

Cloud computing will be big, both in and outside of the enterprise, but being aware of the challenges will help technology providers think of ways around the problems, and let cloud providers know what they’re up against.

A few minutes after she delivered a speech at our Structure 08 conference in San Francisco, I caught up with Microsoft’s corporate VP of global foundation services, Debra Chrapaty, for a video chat. I think a more appropriate title for her would be Mr. Softie’s Internet Infrastructure Czar. I found her very knowledgeable, engaging and open with her opinions. “We have some new innovations up our sleeve that are going to knock the socks of anything anyone is doing, including our friends down south,” she told me. She didn’t name Google, of course, but we all know who she was talking about.

Her candor was one of the reasons I wanted decided to share the video with you guys. The common theme of the conversation: Microsoft is spending liberally to build out its Internet infrastructure, including upgrading its backbone network and scaling out its data center infrastructure by adding new technologies.

When I asked her exactly how much Microsoft was spending on it, she dodged the question, saying just that it was a big number. This much we do know: Two years ago, the company was spending close to $2 billion on its infrastructure; it has since undertaken the development of six data centers, with parts of two networks already online.

Watch the video to get the full low-down, but if you’re in a hurry, here are some highlights, including her quotes from our conversation.

• “We are building data centers but I don’t want to say not just data centers. We are already on to our second generation data centers. More utilization, better density and more power efficient.” For Chrapaty, power efficiency is not just talk, it’s her mission — she is the driving force behind Microsoft’s server utilization.
• “Infrastructure is a differentiator. I use FedEx as an example. They are world’s most predominant distribution company. It wasn’t that they had a great brand or they had all these plans. No, what they did was find these strategic landing fields where they could get in and out quickly to key distribution points across the globe. It defined their company. I think the same is true in the infrastructure now. Data centers are already becoming a scarce resource.” Google realized this a long time ago; Microsoft is now demonstrating how it can put money to work and build an advantage over others.
• Like Google, Microsoft is taking the design of servers into its own hands. “We are doing some unique things in the mother board designs, server designs, and because we are Microsoft, operating systems.”
• She’s a big champion of container data centers, which essentially act like the trailers on long-haul trucks, optimized and packed with all sorts of gear — servers, switches, storage systems — that’s wheeled in and plugged into the power grid and the network. Sounds like Rackable Systems and Verari are major suppliers of these containers to Microsoft; the company is making extensive use of them in their Chicago facility.
• Her comments indicate that Microsoft has plans to offer managed services to large corporations.
• She lets us know first that they are building a IOWA data center, which is huge for Iowa. Google has one in Iowa too.

In planning for last Wednesday’s Structure 08 conference, we at GigaOM had our heads in the cloud. We aimed to draw attention to the resurgence of hardware underlying the various software and web services that consumers and businesses now use, and hoped to define the emerging set of offerings that comprise cloud computing.

That definition is important. But not as important, I realized, as figuring out which business models will win out. Because while everyone wants to push their own definition of cloud computing, at its heart, cloud computing is about moving, storing and delivering data on demand.

After moderating two panels, watching almost all of the speakers and having numerous conversations, I came away with the belief that most people view cloud computing not as access to computing resources, but access to services ranging from application-specific offerings such as Salesforce.com to compute grids like that of GoGrid or EC2. And when it comes to buying into such data services (be they software, a platform, storage or compute time), there are certain questions that need to asked, among them:

How do I get my data into the cloud? Maybe it’s as simple as calling up Salesforce.com, or a bit more complicated, like using an API to tap into EC2, but to use a cloud you’re going to need bandwidth. Whether it’s figuring out how to measure and appropriately charge people for bandwidth as Google is attempting to do with their structured meta data, or contracting with a CDN to lower latency, the delivery of data in and around the cloud represents one of its biggest costs and is subsequently one of the areas that’s ripest for innovation.

What format does that data need to be in? Different clouds work with different software. Some clouds work with Windows and others are only friendly to the LAMP stack. Various people expressed the idea that the industry would divide along the lines of low-margin, general purpose clouds like Amazon Web Services, and high-margin, special-purpose clouds such as Heroku’s Ruby on Rails testing environment (which is built on AWS). The key is to know what you need from a cloud before investing.

How can I change and move that data? The differing programming languages or operating systems accepted by various clouds are only part of the issue. The still undecided fight will be between proprietary formats such as BigTable and open standards that are truly standard, as in used by many, many developers. It’s a young effort, so there are no set standards yet, but until there are, transferring data kept in the cloud will never be as seamless as the bank analogy pushed by Sun CTO Greg Papadopoulos.

So while I spent most of my time trying to figure out which areas of compute infrastructure can be offered as a service while providing the highest margins or most defensible market share, I should have been keeping my eye on the data, because how providers treat the data will determine how their business models evolve.

What if there were a way to write and run enterprise applications that you could move from cloud to cloud? And what if that application automatically inherited the best things about that cloud without locking you in?

DreamFactory may do just that. And as such, it may represent a new approach to application design: Cloud-opportunistic software.

DreamFactory makes project, document and data collaboration software that runs in the cloud. Its DreamTeam Suite competes with Basecamp, Liquidplanner, Huddle and others. The company has about 1,000 enterprise customers, and a strong partnership with Salesforce.com. Its application can be customized in either VBScript or Javascript, so it’s extensible.

But rather than being tied to a particular cloud, DreamFactory works with many of them. Relying on a rich client that runs as a browser plug-in, DreamFactory’s application only needs the cloud for storage. It can use Salesforce, Webex Connect and Amazon EC2. Quickbase support is just around the corner, with Google BigTable hot on its heels. It will even run on your hard drive.

DreamFactory doesn’t expect much from these clouds. As long as a cloud can store and retrieve data, DreamFactory will take care of things that enterprises demand, like schema, field-level security and multiuser permissions. “The Amazon SimpleDB and S3 in parallel are really commodity services, but if you have a superclient that can do XML, you can emulate enterprise services atop commodity services,” said DreamFactory CTO Bill Appleton.

Big deal, you say: They’re just using the cloud as a hard disk.

To be fair, avoiding cloud lock-in might be a big deal all on its own. The threat of being stuck on a bad cloud was a hot topic at Structure 08 and the subsequent subject of discussion on both the Wall Street Journal and InformationWeek.

But there’s a much more interesting wrinkle to this story than just portability.

DreamFactory isn’t just cloud agnostic, it’s cloud opportunistic. In other words, the software inherits the unique features of the cloud on which it’s running. “When we port our DreamTeam product to a collaborative platform like WebEx, it becomes highly collaborative,” said Appleton.

If you use Salesforce, a username can be tied to an account. If you’re running on WebEx Connect, you can start a video conference. Build it on Amazon, and you lose those advanced services but you can pay using Devpay. And so on. With tomorrow’s specialized clouds, you might be able to book travel or pull in a friend feed, for example.

This is a fascinating twist: A way get a cloud’s special powers without its Kryptonite.

GigaOM got an early look at the company’s forthcoming Monarch application, available soon, which allows data migration across clouds while preserving not only raw data but also applications, relationships and schema as it moves from platform to platform.

“In our world, we have customers that have already bought into a platform,” said Appleton. “Our plan was to be agnostic from day one. In some ways Salesforce has the best platform, but it’s kind of locked inside a CRM.”

Maybe cloud-opportunistic applications can unlock those platforms.

The debate around cloud computing and software-as-a-service (SaaS) has energized industry conversations on the future of software. But in fact what we are witnessing in the software industry today is not a revolution, but an evolution. Customers are most concerned with how to use software to sustain competitive advantage, align IT with the business and deliver the best experience for users without compromise — regardless of delivery option — whether that is SaaS, on-premise software or a combination of the two. That’s why this evolution of software in a services world is so important for the industry to broadly support, and why customers deserve more than all-or-nothing ultimatums. For more, see Refresh the Net.

Other infrastructure-themed stories that may be of interest:

The Long Tail of IT
Subscription Services: The Future of Our Entire Economy
Architecting for Failure
Five Nines is Still Not Enough
Do You Know What Kind of Cloud You’re Using?
Defogging Cloud Computing: A Taxonomy
The Craft: Automation and Scaling Infrastructure
Is Infrastructure the New Marketing Medium?
Achieving Equality is Critical to the Future of the Internet
Why Google Needs its Own Nuclear Plant
The Geography of Internet Infrastructure

One of the questions that I really wanted to get answered at Structure 08 was what the chances of survival are for the myriad of startups out there building their businesses around Amazon’s Web Services. Companies such as RightScale, Hyperic and Soasta depend on both the success of AWS and its shortcomings — the solutions to which they propose to offer. So I sat down with the online retailer’s CTO, Werner Vogels, to see how Amazon viewed this ecosystem. My takeaway? I think most of the these firms are safe.

Vogels said that Amazon built AWS for the company’s internal developers, and as such, didn’t feel the need to wrap services such as dashboards and testing offerings around it. And the company typically doesn’t announce new features for the AWS platform until they’re ready for use. But when it came to persistent storage, he pointed out, they started talking about it as soon as they had a beta, putting startups and other firms planning such a service on notice that Amazon would enter that market.

“We wanted to make sure people had a look at our roadmap,” Vogels said. “Our goal is to be very respectful and recognize the value of the ecosystem.”

Vogels didn’t offer any specific glimpses of Amazon’s roadmap, but he did say the firm listens to the demands of its customers when deciding which services to pursue. He said popular requests involve content delivery network services, backup, small file transfer, large file transfer and visual applications.

He also noted that many enterprises have “accidentally wandered” into the cloud for one-off projects and then stayed there. For those customers he points out that business processes using AWS can be compliant with Sarbanes-Oxley, and that Amazon does work with customers facing regulatory or industry mandates, such as Visa’s PCI requirements to protect cardholder data.

So while Vogels didn’t draw a map showing what AWS has in store for the future, startups planning additional services tied to AWS now have at least the outlines of the company’s plans. They can now follow that outline or chart their own course when it comes to navigating the cloud.

Load balancers are a cornerstone of any big computing application. By spraying traffic across lots of servers, they let companies turn many unreliable machines into one reliable service. But that service has a lot of moving parts, and sometimes they break. If it keeps happening, it may signal that a new class of networking device is needed for the demands of cloud computing.

According to Amazon’s Web Services Developer Connection, a load balancer was deployed in its S3 storage service on June 20, and removed two days later. During that time, it was corrupting bytes of data sent to the S3 storage service when under load.

This isn’t the first time load balancers have been implicated in an outage at Amazon. At O’Reilly’s Velocity conference, conference co-chair Jesse Robbins talked about a “redundant array of inexpensive data centers” as the basis for tomorrow’s computing platforms. Load balancing is what makes this possible.

Storing data in a server is a tried and tested process. We’ve had decades to optimize the way we store and retrieve data, with standards like iSCSI and IDE proven out worldwide. And RAID is pretty reliable these days.

But the cloud’s still figuring out storage. There are competing models: S3, SimpleDB, BigTable and so on. And with more and more applications relying on S3 for their data, outages are bound to be visible and public.

The cloud model puts increased pressure on networking and server equipment, and on vendors to make their components reliable. Load balancers built for enterprise data centers may not be suited for the cloud, just as domestic power generators wouldn’t work for utility companies. This may be one reason Google is reputed to be building its own switches.

Expect clouds to require significantly different kinds of networking equipment. It’s either an opportunity for networking vendors like Cisco and Juniper, or a huge threat to them.

After today’s launch of Microsoft’s server virtualization hypervisor, Citrix, which bought virtualization company XenSource last year, may be asking itself some hard questions. Microsoft’s Hyper-V will compete directly with Citrix’s XenSource products for the data center as well as with products from VMware and startup Virtual Iron.

But Citrix and Microsoft have close enough ties that the move by Redmond into data center virtualization may be akin to your sister stealing your boyfriend. And that could strain their relationship. Industry players have claimed that Citrix may be ready to let Microsoft get away with the theft, and focus instead on the PC virtualization market. Others disagree. I plan to ask Citrix about its Xen business next week when I talk to Simon Crosby, the CTO of Citrix’s virtualization business.

In May it launched a new XenDesktop product for desktop virtualization, and recently saw analysts downgrade its stock or reduce revenue estimates based on slowing sales of its XenServer products. Citrix has also been relatively quiescent when it comes to doing deals while VMware keeps shopping. Speaking at Structure 08 yesterday VMware co-founder Mendel Rosenblum said that competition for virtualization in the data center was inevitable, and VMware is trying to move into management products and ensuring reliability (hence those deals).

Current Hyper-V features appear less competitive than those in products offered by VMware, Citrix and Virtual Iron, but no company can afford to stand still. If Microsoft is on your heels, it makes sense to keep running. Microsoft may arrive late to the race, but its installed base and free downloads mean customers are likely to give it a whirl.

Yesterday, at our Structure08 conference we launched our newest effort, GigaOM Briefings. We launch with our first briefing on Cloud Computing, which has been written by Alistair Croll, whose work you know from our site.

So what are our Briefings? Briefings are downloadable digital reports that contain in-depth, timely and actionable information on current technologies and technology trends. Our aim is to identify the roots of change behind the trends, the impact they will have and the opportunities they present, and to bring such information to our audience in a format that is concise, factually accurate and easy to understand.

We have started Briefings after many of our readers and also our network of sources pointed out that they wanted to know more about a topic, but didn’t have time to research or read long research reports. We could expand our blog posts. We love blogging and we like to think we have taken part in taking it to a higher level of credibility through our editorial values. But blogging demands that we be succinct.

Hence, GigaOm Briefings.

A GigaOm briefing will be a 20-to-30 page report, that gives you facts and insights in a quick-to-consume manner. The knowledge to empower your business. You can find out which companies are important in a sector, who is going to be important in a sector and well, who is going to get crushed. We will release reports as topics arise and warrant them, so stay tuned. Much the way the Zagat guides help you make decisions about where to eat, we hope our reports become your constant companions in your quest to navigate the ever-changing technology landscape.

Our first briefing looks at Cloud Computing is 18 pages long and costs $249 a copy. You can find more details on briefings.gigaom.com. Future briefings will focus on Location-Based Services, The Smart Grid, & Application Delivery Networks.

Nick Carr, author of “Does IT Matter” and “The Big Switch,” helped us kick off the Structure 08 conference yesterday with a short and sweet message about the shift to cloud computing and why we need to think about the ethics of infrastructure. He also pointed out our conference was being held in the week Bill Gates retires from Microsoft. Carr sees it as an end of one era of computing and the start of another. Watch his message on this video:

Cloud computing isn’t as nebulous as its name implies. Thanks to virtualization, one can separate the storage from the servers and the servers from the software—but it’s also about bandwidth. The primary value will be more about moving data from the hardware to the end user. To that end, Google has automated its network and is using structured metadata to track how much it costs the company to move a bit or byte from one geographic area to another, according to Vijay Gill, manager of engineering at Google.

That allows Google to charge users based not just on compute cycles but on their actual costs of moving the data around the world. Gill talked about establishing an auction model for pricing that will reflect that actual costs of moving data. The importance of bandwidth was also highlighted by Lane Patterson, chief technologist from Equinix, who said that a cloud provider that owns its own bandwidth might achieve a competitive advantage.

As cloud computing unfolds it won’t do so only in the U.S., said Dr. Jay Subrahmonia, director of advanced customer solutions for IBM, who points out that developing countries are adopting it because of the speed and flexibility cloud computing offers. Figuring out how to price and value that speed and flexibility is the next big step.

With all the hype about cloud computing, it’s easy to label it as the latest fad, especially when everyone whose application talks Internet is trying to rebrand themselves as a cloud. But the long view shows that this really is an important change, one of several major shifts in computing that have taken place over the last 40 years, each of them driven by costs and shortages.

Once upon a time, computing was expensive. As a result, programmers carried their stacks of punched cards into basements late at night, and ran them on the mainframe. The CPU was always busy; humans were cheap.

When computing became cheap, bandwidth and storage remained expensive. The CPU was idle, but the links were full. This gave us the PC and client-server architectures. A wide range of clients on a variety of networking protocols kept things complicated, and WAN prices meant most network traffic was local.

Eventually, we settled on browsers, HTTP and TCP/IP. This was web computing, with a simple, standard edge and a tiered core. Client-side broadband access and persistent storage were relatively cheap. (Don’t believe they’re cheap? Go into an enterprise and you’ll find their networks and storage systems have plenty of extra capacity. The same is true for the Internet — if you ignore the impact of spam and P2P traffic.)

Now here’s the cloud. It’s driven by five big things, none of which are hype, and all of which are changing the way we compute.

1. Power and cooling are expensive. Today, it costs far more to run computers than it does to buy them in the first place. To save on power, we’re building data centers near dams; for cooling, we’re considering using decommissioned ships. This is about economics and engineering.
2. Demand is global. Storage itself may be cheap, but data processing at scale is hard to do. With millions of consumers using a service, putting data next to computing is the only way to satisfy them.
3. Computing is ubiquitous. We’ve lost our desktop affinity. Most of the devices in the world that can access the Internet aren’t desktops; they’re cell phones. Keeping applications and content on a desktop isn’t just old-fashioned — it’s inconvenient.
4. Applications are built from massive, smart parts. Clouds give developers building blocks they couldn’t build themselves, from storage to authentication to friend feeds to CRM interfaces, letting coders stand on the shoulders of giants.
5. Clouds let us experiment. By removing the cost of staging an environment, a cloud lets a company try new things faster. This is also true of virtualization in general, but by billing on demand the cloud means anyone can experiment.

This truly is a fundamental change in computing, even if its title has been diluted by marketing agendas. We have to be careful not to throw the innovation baby out with the cloud hype bathwater.

Parascale, a Cupertino, Calif-based start-up that has developed a storage file system for a cloud of computers announced that it had attracted $11.37 million in Series A funding from Charles River Ventures and Menlo Ventures. The company recently changed its chief executive and brought in Sajai Krishnan, a former NetApp executive to run the company.

I spoke to Sajai briefly this morning, though I have not had a chance to dig deeply into the company’s technology just yet. The company is going to officially release its software, currently in trials in Fall 2008. It is targeting the streaming media/video industry and others who want to get storage for less than 50 cents a gigabyte. Others like ISPs can use it to set up their own grids and offer competitors to AWS.

What essentially they have developed is software that gets commodity storage drives attached to plain vanilla low cost servers to behave like a giant cloud of storage space, which can be used (and managed) using protocols such as HTTP, FTP and NFS. The company describes its approach as virtual storage grid.

The general idea is not novel, though the company’s commercial rivals (such as Isilon & polyServe (part of HP)) can handles limited number of nodes, an industry euphemism for storage-attached servers. Parascale claims it can handle hundreds of nodes making it easier for the company to handle terabytes of data.

How it works is that a control server – lets call it the brain of the storage cloud – communicates over Gigabit Ethernet connections with storage nodes and makes them all behave like one giant storage cloud. Similar systems from more traditional storage companies would use custom high-speed connection technology like Fiber Channel.
Storage nodes are x86-based Linux servers that support cheap SATA drives. The brain essentially stores the metadata of the files on storage nodes and at all times knows where data has been placed, file versions, and other such information. A software management console helps manage the flow of data in-and-out of the system.

This approach to build high volume storage systems has received a lot of attention, thanks to the success of Google File System and of late, the open source Hadoop platform, championed by Yahoo and Apache Foundation. Our sister blog, OStatic had noted that Hadoop was putting companies like EMC at risk.

Add newcomers likes Parascale to that list, for we have seen many a few open source projects upend proprietary efforts. I am quite bullish on the prospects of open source cloud projects. Still, the company has garnered some positive reviews from early users of its trial software.

In Other Cloud Computing News: