A more technical explanation is that Piccolo uses an in-memory, key-value store and a “global table interface” — as opposed to Hadoop, which utilizes a distributed file system contained within the disk drives of the machines in the cluster — that lets the CPUs access all the data simultaneously, and at high speeds only possible by pulling data straight from RAM. In this fairly long, but genuinely interesting presentation at the OSDI 10 conference, lead developer Russell Power explains how Piccolo works, how it differs from Hadoop and how it has tested far faster than Hadoop on certain workloads. Power compares Piccolo to the Chevy El Camino, which was both efficient and easy to use while also delivering high performance. Here’s a screenshot of Power illustrating Piccolo’s scalability on an Amazon EC2 cluster:

I’m not suggesting Piccolo is going to replace Hadoop, or MapReduce, generally, anytime soon or ever — Hadoop vendor Cloudera today received a strategic investment from U.S. intelligence sector consultant In-Q-Tel, which should hammer home the fact that Hadoop is for real — but Piccolo is worth watching. It certainly wouldn’t be the first academic project in recent memory to make it big; the Marten-Mickos-led cloud-software provider Eucalyptus Systems was a research project from the University of California Santa Barbara when it caught on, and then struck it big with VCs and early adopters.

To learn even more about the future of big data processing and analysis, make sure to attend out Structure Big Data conference March 23 in New York City. You won’t likely hear about the seedling Piccolo project, but you’ll hear plenty about cutting-edge use cases and tactics for the current generation of big data tools.

Related Content From GigaOM Pro (subscription required)

• Big Data, ARM and Legal Troubles Transformed Infrastructure in Q4
• The Incredible, Growing, Commercial Hadoop Market
• In Q3, Big Data Meant Big Dollars

CumuLogic was co-founded by Sun Cloud and Startup Essentials vets Rajesh Ramchandani and Laura Ventura, and touts Java creator James Gosling and former Sun CIO Bill Vass as the leaders of its technical advisory board. According to Ramchandani, he and Ventura were inspired to create a Java PaaS after they left Oracle and started thinking about how Sun could might have expanded its cloud computing efforts into the PaaS space had Oracle not killed Sun’s cloud project upon its acquisition of the company. We’ll never know how close CumuLogic is to what Sun might have done, but the company does appear to have embraced Sun’s legacy of giving users plenty of choice and control.

CEO Sandip Gupta told me that CumuLogic is focused on legacy Java applications, of which companies have written countless numbers over the years. Instead of rewriting applications to fit new platforms and, essentially, giving up application components on which companies might have standardized over the years, CumuLogic wants to give them the flexibility to keep using those components, from application platforms to databases. Further, CumuLogic wants to give customers choice of where to host their PaaS environment by providing a software product that can be installed locally or atop an IaaS cloud. Makara, the cloud software recently acquired by Red Hat, offers the same functionality with regard to deployment, but the companies differ in terms of scope.

CumuLogic also is trying to set itself apart by retaining a degree of IT control over the environment. Gupta explained that CumuLogic gives IT administrators the ability to do things like determine application lifecycles, establish permissions and transition environments from dev-test to production — all while giving end users the self-service, automated experience they expect from a PaaS offering.

As anyone following cloud computing over the past several weeks has noticed, however, options for Java PaaS are proliferating fast.  Among the offerings now supporting Java applications are Amazon’s Elastic Beanstalk, Google App Engine, VMforce, CloudBees, Makara and Windows Azure. But each product differs in terms of where they’re hosted, what frameworks and stacks they support, and whether they also support additional programming languages. Gupta thinks CumuLogic can carve out a niche serving the likely sizable population of companies, service providers and ISVs that want to support legacy Java applications in the cloud, which probably is true for the time being until customers start writing new applications in today’s popular web languages such as Ruby and PHP. If that time comes, he added, CumuLogic is willing to look at expanding its scope.

Java image courtesy of Flickr user Dominic’s pics.

Related content from GigaOM Pro (sub req’d):

• Multi-Language PaaS: Salesforce.com Is Just One Option
  
• Are the Stars Aligning for an Amazon PaaS Offering?
  
• Java-PaaS: A Bevy of Options in the Blink of An Eye

Jevons recently got big play in a New Yorker article arguing that greater overall energy efficiency won’t help solve our energy usage problems, but will instead lead to more ways to use the fruits of low-cost energy consumption, thus leading to more overall consumption. Perhaps as a society that loves to consume without guilt, we’re just really attuned to his message. Anyhow, in applying it to computation, McAfee states:

Rather than wading into the energy debate, though, I want to wonder out loud if computation is like energy. Both are necessary inputs to many productive activities. Both are consumed by every company in every industry. Both are amplifiers of human ability: energy amplifies or replaces our muscles; computation does the same for our brains and senses. Both come from physical things, yet are themselves ethereal; you can hold a lump of coal or a transistor in your hand, but not a joule or a megaFLOP. And the devices that generate both are getting more efficient over time.

If the analogy is a tight one, if Moore’s Law continues to hold true, and if Jevons was right about energy-intensive processes, then one conclusion seems inescapable: the trends visible in both of the graphs above are going to continue. Computers are going to keep getting cheaper, and aggregate demand for them is going to continue to rise.

I can’t argue with that, but I can argue that the beneficiaries of continued compute demand will change over time. Much like the beneficiaries of cheaper, more abundant energy changed from railroads and transport companies to home appliance or air conditioning makers, the beneficiaries for the lower cost of computing will change.

Intel’s x86 chips may give way to ARM or GPUs or something completely different, and I’d also push the argument out for radios and other silicon that enables connectivity since I fundamentally believe that in the future computation and connectivity will be forever linked. That means processors and radios will always be in demand (even if the purveyors and even form factors change) as well as the hardware used in networking, be it fiber or base stations.

Related content from GigaOM Pro (subscription req’d):

• Supercomputers and the Search for the Exascale Grail
• Pushing Processors Past Moore’s Law
• Nvidia Arms Itself: Who Has Most to Lose

Amazon Web Services, which popularized cloud computing with its Elastic Compute Cloud and Simple Storage Service, has moved up the stack from infrastructure to providing Amazon Elastic Beanstalk, its Platform-as-a-Service play. However, Amazon is layering its PaaS offering on top of its other services in a way that’s easily reversed, which means developers can take the easy way out of developing on Beanstalk, or they can peel back the platform to manually provision and tweak their underlying VMs if they want.

Adam Selipsky, VP of Amazon Web Services, says the service was built to address the idea of vendor lock-in and inflexibility that commonly afflicts other platforms for application development. With the first efforts, Amazon is providing a framework for folks to build Java apps on AWS with other programming languages and partnerships to follow. It’s not surprising, given the attention that Platforms-as-a-Service have been getting in the last 12 months or so. At the beginning of last year, Microsoft finally opened up its Azure platform, while a few months later, VMware and Salesforce.com teamed up to offer VMforce, a Java cloud hosted on Salesforce’s infrastructure, and VMware eased into the PaaS market in other ways this year. Google amped up its App Engine offering, tying it to Salesforce and VMware, while smaller providers of Platforms-as-a-Service such as Makara and Heroku were snapped up (by Red Hat and Salesforce respectively).

For Amazon, long the leader in the cloud space, seeing competitors move up the stack and developers taking advantage of those platforms that weren’t necessarily built on AWS infrastructure was a warning. As my colleague Derrick Harris wrote back in May (GigaOM Pro sub req’d):

So, my question is this: If AWS really will be simplifying management within the coming weeks, what are the chances it does so via a PaaS offering of sorts? It would be wise for AWS to leverage its current leads in market and mind share and preempt any serious momentum by PaaS providers. Technically, they’re not competitors yet (to the degree that IaaS and PaaS can vary differently in terms of target audience), but the next generation of PaaS offerings will blur those lines. AWS has the tools to build a holistic PaaS offering, the economies of scale to make it profitable, and the SDKs to cater to specific set of developers. If it does so, the cloud-computing discussion will take on an entirely different tenor as PaaS providers scramble to differentiate themselves from AWS in this area, too.

Amazon’s Beanstalk offering has taken longer to launch than the few weeks Derrick had hoped for, but now it’s here. Amazon’s next move will be expanding beyond Java, something it could do via partnerships with other providers or on its own. Brian White, a developer with AWS, said PHP and Ruby are high on Amazon’s list, but declined to specify how partnerships with other providers might look. When asked about competing with other PaaS providers who host their platforms on AWS infrastructure, Selipsky suggested that perhaps those might become partners for supporting other languages.

Indeed, in its press release on Beanstalk, Amazon included a quote from John Dillon, CEO of Engine Yard, saying the company is working with Amazon to provide a Ruby on Rails offering on Beanstalk. So will Amazon be a giant lumbering down the beanstalk to crush the PaaS competition, and will it lift others up to its height?

Image courtesy of Flickr user Melody.

Related GigaOM Pro Content (sub req’d)

• Multi-Language PaaS: Salesforce.com Is Just One Option
  
• Are the Stars Aligning for an Amazon PaaS Offering?
  
• Java-PaaS: A Bevy of Options in the Blink of An Eye
  

The agreement with Verizon has three implications, with the first being that Bandwidth.com now knows what it will pay to terminate VoIP calls to landlines, which will help it build out a stable cost basis for its business and thus help other VoIP companies it serves do the same. Much like the agreement with the SoundExchange helped Pandora, the online radio station, figure out the costs of providing streaming music to users so it could build out a business model to support itself, this agreement helps Bandwidth.com and other VoIP providers by setting a baseline cost. David Morken, CEO of Bandwidth.com, says he’s in talks with another major wireline provider to sign a similar deal.

The agreement could also help Verizon, which also has a lot of VoIP traffic on its network, pay lower rates to rural telephone companies. Verizon has to pay other telecommunications providers to terminate its VoIP calls, and in some places, it pays almost a half-cent per minute to do so. By publicizing this agreement, Verizon has told the world what it’s charging others, and said what it wants to pay. Expect rural providers to experience pressure from Verizon to bring their VoIP termination fees down to this $0.0007 cent level. From a research note issued today by investment research firm Stifel Nicolaus:

In the absence of reform, we believe Verizon and others are looking to put downward pressure on intercarrier compensation in the marketplace. Verizon is disputing rural carrier collection of access charges for connecting VoIP calls while attempting to negotiate deals, including the one with Bandwidth.com, that move the industry toward lower rates.

The third implication has to do with the somewhat esoteric world of regulation and telecommunications law. The FCC has never ruled on intercarrier compensation rates for VoIP services, because it has never decided if VoIP is a telecommunications service like wireline telephone or an information service like email. Obviously, folks don’t have to pay $0.0007 cents to send an email, although email doesn’t really cross from an IP network to an analog world unless the recipient has dial-up. This private agreement classifies VoIP as an information service and sets a rate — something the FCC hasn’t yet done for VoIP calls. So while the FCC has decades of rate setting and compensation agreements for voice traffic, it has never made the leap to set rates for VoIP. This has left the Verizons and other telcos of the world able to set prices, and the market for VoIP to develop in a way that’s vastly different from the market for email.

So this rather dull-sounding agreement is a big deal for companies like Google, Skype, Twilio and Bandwidth.com, as well as for rural telcos and the FCC. If the FCC decides to make a decision on VoIP interconnection fees rates this year, as some sources have said it will, then Morken says he can let the agreement expire (or not) depending on what the rules say. For Bandwidth.com, which has gone from operating 1 million numbers at the end of 2009 to 17 million numbers today, the agreement is like a hedge against higher prices and regulatory uncertainty that Morken can use to build his business.

With $100 million in annual revenue, up from $85 million the year before, Bandwidth.com is growing well without ever having taken on venture investment. The company, which is profitable, expects to file to go public within the next 18 months, said Morken. We’ve covered the business in December 2009, and I think as an infrastructure provider for VoIP services, Bandwidth.com can sell itself as a credible VoIP platform. Contracts like this only reinforce that legitimacy, although should the FCC declare VoIP an information service, I suppose the contract could become a cost burden other providers wouldn’t have to deal with.

Related GigaOM Pro Content (sub req’d):

• Who Will Profit From Broadband Innovation?
• The New Net-Neutrality Debate: What’s the Best Way to Discriminate?
• When It Comes to Pain at the Pipe, Upstream Is the New Downstream

OpenStack’s storage component, called Object Storage, is now fully functional and ready for industry adoption beyond its use as the foundation of Rackspace’s Cloud Files offerings. According to Scott Hrastar, SVP of technology at Internap, Object Storage is advanced enough that his company was able to implement it with relatively little effort and focus most of its effort on building differentiation into the user-facing aspects of XIPCloud. In fact, Hrastar noted, his team was able to deploy Object Storage without any formal support, relying instead on a few knowledgeable in-house developers and the robust OpenStack community.

That’s good, because OpenStack doesn’t offer formal support — yet. Jonathan Bryce, chairman of the OpenStack Project Oversight Committee and Rackspace Cloud co-founder, said Rackspace has discussed ways to monetize OpenStack, and that support could be a natural fit given Rackspace’s expertise and reputation in technological support. Already, he noted, several Asian companies are providing installation support services, as the project is very popular in Japan and China.

Whether or not Rackspace launched a formal OpenStack support offering, though, the project will continue to mature. With the second release,”Bexar,” set for next month, OpenStack’s computing component will be ready for deployment in small- to mid-size data centers, and Bryce said it will have even more features than expected by this point. This is thanks, in part, to a thriving ecosystem that presently includes more than 40 technology partners and significantly more individual developers. Bryce said OpenStack is still on pace to be ready for use by service providers and webscale data center operators with its “Cactus” release in April, as I reported in November. Bryce added that OpenStack is evaluating how to integrate the Cloudkick technology that Rackspace acquired last month — a difficult proposition because much of Cloudkick’s value comes from being a hosted service — and that the work it has done to help incorporate OpenStack into Canonical’s latest Ubuntu Linux release will pay off in terms of helping the project better understand operating system integration.

The Object Storage readiness is great, but the world really has been waiting for OpenStack Compute since July. When it’s finally production-ready, we’ll see whether it can live up to its hype of not only elevating Rackspace’s cloud offerings, but also providing an open-source seed that can spawn an ecosystem of interoperable, highly advanced cloud offerings both from service providers and within enterprises. Its progress thus far is promising, but it’s a market rife with competition, from Amazon Web Services to VMware.

Related content from GigaOM Pro (sub req’d):

• Defining Internal Cloud Options: From Appistry to VMware
• Why New IaaS Providers Enter at Their Own Risk
• Why OpenStack Has Its Work Cut Out

During the launch event, EMC showed off its VNXe line — for small- to medium-size business customers — which is cheaper and contains nifty features, such as the ability to provision the hardware from an iPad, and a graphical interface that represents the hardware box, so users can determine what’s wrong. EMC “proved” how easy this system is for folks by getting a fourth grader armed with an iPad to troubleshoot the hardware.

EMC also announced a new product using its $2.1 billion Data Domain acquisition for faster sending of backup information, as well as a new version of its Atmos line of software built on its $2.25 billion acquisition of Isilon. For those drowning in data, EMC offered a new version of it VMAX box that’s twice as fast, with more brains inside for processing big data. (It contains a 128-core processor to manage the memory.) The VMAX system also does away with Fibre Channel interconnects, and instead, counts on the better brains and Flash memory to provide the faster speeds.

However cheesy the presentation, EMC is clearly aware of the big trends in the enterprise, and how the market is fragmenting between the high and low end. At the low end, businesses want cheap, easy-to-use and something that resembles the Apple experience. But at the high end, where there may be fewer customers, but they buy a lot of boxes, EMC is aware that speed, scale and big data are changing the way IT views storage. It’s no longer this backroom archive, but a real-time delivery system for information that could be called upon at any moment. So EMC, which said it has shipped over 10 petabytes of Flash memory last year, is trying to deliver the new future of data. It’s always on, and is the new foundation for business decision-making.

For more on how the data deluge is affecting the enterprise and innovative ways startups are tackling their own data problems, come to our Big Data conference held March 23 in NYC.

Related Content from GigaOM Pro (subscription required)

• Why Cloud Storage is So Hot Right Now
• How to Make Cloud Computing Greener
• The Future of Data Center Storage

Well, those are the visions. However, there are major hurdles to implementing these ideas. Here are the road blocks:

Infrastructure Investment

First and foremost, there will be a colossal investment needed for both clean power and electric car infrastructure to make their way onto the market, and both will take a lot of time. Author and professor Vaclav Smil has explained in his recent book that an all-electric U.S. fleet would conservatively need 980 TWh of electricity per year to run, which was 25 percent of the U.S. electricity generation in 2008. Smil thinks utilities wouldn’t realistically be able to build that additional amount of electricity generation within two decades.

In addition, that extra generation would have to come from clean power to be carbon-reductive. As anyone who has followed the utility-scale solar market knows, it takes years for utility-scale solar projects to move from drawing board to supplying electricity. In the case of BrightSource Energy’s inaugural solar thermal project Ivanpah, it has taken over three years to just get regulatory approval, and now here come all the environmental protests.

California’s utilities have struggled to meet the state mandate that says they need to buy 20 percent of their electricity supply from clean power by 2010. Most utilities weren’t likely to make that deadline, but state regulation gives them until the end of 2013 to comply. Meanwhile, the utilities will have to make sure they line up enough contracts or install their own projects to meet the 33 percent goal by 2020. And this is just in California, which has an aggressive state mandate.

Utility scale wind is a more mature market, but wind installations slowed in the U.S. considerably in 2010 due to the slowed economy. In addition, because of intermittency, Smil and other researchers think wind could never be a dominant form of clean power. In fact, it’s far from clear if solar and wind will be able to provide baseload power (provides energy 24/7), and the U.S. will have to rely on other forms of clean power like nuclear, geothermal, and hydro.

From a plug-in vehicle market perspective, perhaps it’s not such a bad thing that clean power will take such a long time to get built out. Because plug-in car adoption will take just as long. Bloomberg New Energy Finance predicts there will be 1.6 million plug-in cars sold by 2015, rising to 7.6 million by 2020. In 2010, the U.S. had about 245 million passenger cars, SUVs, vans, and light trucks.

EV + Clean Power

Clearly, it’s going to take decades for both clean power and electric vehicles to make a sizable dent in the U.S. infrastructure. In the mean time, some utilities and companies are looking at ways to use or sell clean power for electric vehicle projects.

SAP and German utility MVV Energie are starting a pilot project using 30 corporate SAP electric vehicles that will be powered exclusively by the utility’s clean power. MVV Energie will be building and operating the smart charging stations that are capable of filling electric cars exclusively with certified green energy.

Better Place, the electric vehicle infrastructure company, plans to incorporate clean power into its networks, particularly in its launch region in Israel. In 2008, when Better Place CEO and founder Shai Agassi announced the Israel Better Place launch, he said the infrastructure will be powered by “batteries, that get their energy from green sustainable electricity sources.” (We’re thinking that’ll be mostly solar, given Israel’s climate).

Batteries as Aid for the Power Grid

While we’re waiting for EVs to be powered by the sun, electric car batteries could be an aid to getting clean power onto the grid. A network of electric cars could offer the potential of distributed energy storage and grid services like load balancing or frequency regulation.

The power grid works by constantly balancing supply and demand (generation and load) and must be kept at a 60 Hz frequency. That’s a complex and difficult task given today’s grid has little energy storage capacity. So if the frequency goes too high or low, the utility must respond by shifting generation and load. For example, PJM, a regional transmission organization serving a population of 51 million, pings generators to control regulation as often as hundreds of times per day. Electric vehicle batteries could act as the real-time, distributed intelligent frequency regulators, replacing generators.

PJM has a project with the University of Delaware using electric vehicles in a demand response program, but John Gartner, an analyst with Pike Research, says, “We don’t see this as a commercial application until at least 2015.” However, after the issues are resolved, the arrival of electric vehicles will provide greater flexibility for utilities to integrate higher percentages of wind and solar power, says Gartner.

Image courtesy of Better Place.

Related content from GigaOM Pro (sub req’d):

• Car Data As the Next Platform for Innovation
  
• Report: IT Opportunities in Electric Vehicle Management
• Why Microsoft’s Electric Vehicle Deal With Ford Matters

Anant Agarwal of Tilera (far right) at Structure 2010

Tilera, a chip design firm that hopes to build a 200-core processor for hugely parallel compute problems, has raised $45 million in funding from investors that include Artis Capital Management, WestSummit Capital Management and Comerica Bank. The company has raised a total of $109 million, and now counts strategic investors Cisco and Samsung among its backers. We’ve covered Tilera for years, ever since I named them one of the top 5 multicore startups to watch back in 2008, then again when it released its 100-core chip.

From that story:

Tilera scoffs at quad-core machines. The company’s chips already are used by 75 customers, and come with 36, 64, and now 100 cores. Agarwal says, “The core is the new transistor.” By cramming so many cores onto its chips connected by a mesh network of interconnect that allows the cores to communicate without bottlenecks, Bob Doud, director of marketing, says that Tilera can sell its chips to folks wanting faster memcached servers or better performance at web-scale computing. The chips, which provide 1.25 GHz of performance, are no match for Intel’s workhorse Nehalem processor that can top out at 3.3GHz. But Tilera’s chips only burn 33-50 watts instead of 130 watts that top-of-the-line Nehalem silicon can.

Tilera is joining other specialty chipmakers that are betting that the rising demand for computing power can’t be met by x86 processors without consuming too much power. Other chipmakers are repurposing popular chips from the consumer world, such as Nvidia’s emphasis on GPUs or Calxeda and Marvell’s decision to bring ARM-based chips into servers (Nvidia recently joined this camp too). However, Tilera has scorned the more-popular architectures and has built its own design (it uses a RISC architecture), and it hopes to achieve 200 cores by 2013. In a call to chat about the funding, Tilera executives said the company has made progress porting common software used in networking gear and data centers to its chip and that one of the top three webscale data centers is using a server that contains Tilera chips.

Troy Bailey, VP of marketing at Tilera, dismissed the current threat of ARM-based chips, pointing out that the designs from ARM, which are more commonly used in cell phones, are not offering 64-bit processors yet. This limits their ability to run certain types of corporate software. As for other architectures, Bailey believes many different architectures will have a place in different sections of the data center. In the meantime, the $45 million for Tilera will help it pursue its original networking and embedded market while tweaking its basic design for more specific types of workloads inside hyperscale and cloud data centers. It’s a lot of money, but as we’re seeing with funding for companies such as SeaMicro or Calxeda, venture investors think it’s a big opportunity.

For more on Tilera’s position for the cloud, check out the discussion from last year’s Structure conference, where executives from SeaMicro, Tilera, ARM, Dell and VMware discussed where the hardware for computers was heading.

Related content from GigaOM Pro (subscription req’d):

• Supercomputers and the Search for the Exascale Grail
• Pushing Processors Past Moore’s Law
• Nvidia Arms Itself: Who Has Most to Lose

The reality is that volumes are growing for all types of data, not just the unstructured data that might lead businesses to consider any of the emerging NoSQL or non-SQL database technologies. Organizations doing millions of transactions or otherwise producing relational data want products that meet their needs, too, and they want them to be minimally disruptive in terms of learning new technologies and rewriting applications. It’s hard to get less disruptive than keeping the database layer essentially the same.

Presently, most attempts to scale relational databases involves concepts like sharding or implementing a cache system, so Clustrix’s approach of scaling out the database itself by adding nodes is understandably attractive to customers. Thus far, the only other startup making noise around high-performance, scalable SQL is VoltDB, which focuses on online transaction processing. Just like NoSQL projects have proliferated to solve various problems related to unstructured data, Clustrix’s early success suggests that attempts to build scalable SQL databases might start ramping up, too.

An interesting side note to the Clustrix announcement is how it underscores the notion that SQL and NoSQL databases can coexist within the same organization. Clustrix customer AOL recently announced its work with NoSQL startup Membase (and Hadoop vendor Cloudera) around turning large volumes information on user events into a high-speed, targeted ad-serving system.

Image courtesy of Wikipedia Commons contributor Andreas Praefcke.

Related content from GigaOM Pro (sub req’d):

• Report: NoSQL Databases — Providing Extreme Flexibility and Scale
• With Scalable Data Stores Around, Is NoSQL a Non-Starter?
• Webscale Databases: Is Open Source Really Necessary?