Much has been written about the “big data” phenomenon — the petabytes of machine data from computers, sensors and other equipment; social networking data; scientific data — is a rich but unwieldy trove that is available for the taking. The big data problem is that the sheer amount and diversity of this data outmatches the abilities of traditional relational databases like Oracle, SQL Server and DB2 to handle effectively. With the Hadoop distributed data file system and MapReduce processing power, that data can be aggregated. The next step is finding tools to analyze it further.

It’s that analytics problem that has Andy Palmer excited. Palmer is a serial database entrepreneur who co-founded Vertica Systems (now part of Hewlett-Packard) and VoltDB and was a founding board member of Bluefin Lab, CloudSwitch (now part of Verizon), and Recorded Future.

“The real purpose of big data is to enable big analytics. The most compelling companies out there, I think, are those that attack that problem,” Palmer told me this week.”I really do believe that big data is, in and of itself, a tool. The real story is more about big analytics. Once you aggregate the data you then have to ask really hard questions.”

Andy Palmer

The surging interest in data analytics and visualization tools supports his take. Splunk last month filed for its IPO, and Tableau is well on its way.  Another analytics player, QlikView went public last summer, and its stock has doubled since launch, as Derrick Harris reported in GigaOM. All of these companies aim to help users make sense of all that data.

Palmer, who often works with database pioneer Michael Stonebraker, shares Stonebraker’s view that the sheer variety of data formats and the types of operations to be performed on them call for a variety of specialized databases.

There is a real need for database technology that can handle multi-dimensional data arrays — data sets that often come out of astronomy and other scientific research, Palmer said.  ”When you represent data in traditional relational databases, you can compromise the inherent nature of the data. And if you integrate a lot of data together, ultimately that data looks like a large array. Representing an array in a traditional database is really an unnatural act,” he said.

He is backing yet another Stonebraker company, Paradigm4, that is attacking that problem. In the past, the big database powers were able to shoehorn new types of workloads into their relational model. For example,  a decade or so ago, there was a raft of small, innovative object database companies — Object Design, Ontos and others — that built their businesses on the premise that relational databases could not handle objects which did not fit well into the rows-and-columns world of relational databases. Over time, however, the big data base players pushed and shoved at least some object capabilities into their databases, and those smaller companies disappeared.

Palmer and others in the big data world said this won’t happen again — that big data cannot be co-opted the same way — it would be way too expensive and resource intensive for traditional databases to try to churn through all this stuff. That’s why Oracle et al. are coming out with specialized big data products.

And when it comes to big data, the data itself will be meaningless unless the right analytic tools are available to sift through it and there are people who know what questions to ask. Big data, and the big analytics used to make sense of it, will be hot topics at GigaOM’s Structure: Data conference next month in New York City.

Photo courtesy of  Flickr user Il conte di Luna.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Why the big data startup boom will likely be short-lived
• Health care and big data in 2012
• How search can unlock the power of big data

While the area is more famous for the tech luminaries and startups it lost to other regions — Harvard alums Facebook’s Mark Zuckerberg and Microsoft’s Bill Gates being the most famous examples  – it still can claim a roster of impressive tech startups.

As a Silicon Valley-based partner for Boston-based North Bridge Venture Partners‘  Paul Santinelli has studied the differences between the two technology hotbeds up close and come up with a few conclusions. ”Boston is strong in infrastructure, comms [communications], and enterprise software — the kinds of technologies needed to build businesses,” he said in a recent interview.

Silicon Valley — which led the league in VC money with more than $3 billion invested in Q4 2011, according to the PricewaterhouseCoopers/NVCA MoneyTree Report, is much more focused on the consumer markets, Santinelli said.

But in the post-minicomputer, post-PC world, why build a business in Boston? “That’s a question we had to answer in a very real way when we got started,” said David McFarlane, Co-Founder and CEO of Akiban Technologies, a Boston-based NewSQL database startup. Some of the company’s backers wanted it to relocate to Silicon Valley, he said, but Akiban resisted.

“There’s a tremendous amount of talent in the Boston area where there are quite a few database and data integration companies. There are a number of founding architects that came from Object Design, from Archivas, Blue Agave, and Oberon and InterSystems,” he said. Object Design was a pioneer in object-oriented databases; Blue Agave, a demand management specialist, was acquired by I2 Technologies (which was then acquired by JDA Software); Archivas was a storage startup acquired by HDS; InterSystems is the company behind the Cache database (an outgrowth of the  MUMPS database) used by many hospitals and healthcare organizations.

Ori Herrnstadt, McFarlane’s co-founder and Akiban CTO agreed. “The caliber of architects you found here in the database world was unmatched. The Vertica, the Netezza, the Object Design guys were all here,” he said.  (Vertica, Netezza and Object Design ended up at  Hewlett-Packard, IBM and Progress Software respectively.)

Other hot database or storage oriented startups in the Boston area include Kinvey, ParElastic, Ginger.io, Sonian, Hadapt, Cloudant and VoltDB, the latest brainchild of serial database entrepreneur Michael Stonebraker, who backed Informix, INGRES, Streambase and, Vertica.

It doesn’t hurt that MIT, Harvard, Tufts, Boston University, Boston College, Brandeis, Bentley, Babson, UMass/Boston and other colleges are shoehorned into a compact area around the city. Those schools provide a steady stream of young talent to power startups. Another key part of Boston’s deep bench comes from its background as the home of the minicomputer — the mid-range machines that bridged the mainframe and PC eras. Those minicomputer companies — Digital Equipment Corp., Prime Computer, Data General, Wang Labs, ComputerVision — have gone the way of the dodo bird, but left behind an impressive array of technology experience that remains relevant.

Boston’s proximity to east coast financial companies is another plus. Those companies are not only a possible source of investment but a potential customer base, Santinelli said.

Still, as evidenced by the number of local companies snapped up by outside tech giants, the Boston-Cambridge nexus can feel more like a farm team to distant big leaguers. IBM alone has bought 20 local area companies since it purchased Lotus Development Corp. in 1995. IBM’s most recent purchase was Burlington, Mass.-based Emptoris last December.  Oracle (bought Cambridge-based Endeca in October) and others have cherry picked promising startups in the area. There simply aren’t many tech giants based here any more. On the plus side, the well of expertise still runs deep in the area that witnessed the rise (and fall) of the minicomputer era.

Photo courtesy of Flickr user John Stracke

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Facebook’s IPO filing: ideas and implications
• What the VC Industry Upheaval Means For Startups
• Social 2013: The enterprise strikes back

Cray's XK6 supercomputer

It looks like Oracle has some competition when it comes to selling big iron for big data. On Wednesday, Cray, the Seattle-based company best known for building some of the world’s fastest supercomputers, said it’s getting into the big data game. A new division within Cray, called YarcData, will leverage Cray’s experience working within data-intensive environments for customers such as Boeing in order to woo large-enterprises with big data needs.

Cray was short on details in a press release announcing the new division, but new YarcData SVP and GM Arvind Parthasarathi, formerly of Informatica is quoted saying, “YarcData is the nexus of the world’s most advanced technologies from Cray being applied to solve the world’s most challenging Big Data problems.” The natural leap is that Cray will design parallel-processing systems capable of incredible data throughput — something already required in the supercomputing space, where incredible processing capacity would be wasted without a steady data stream — but that will support today’s popular big data tools (e.g., Hadoop, analytic databases and predictive analytics software).

This type of system could be very valuable for organizations such as banks and intelligence agencies that want to run big data workloads as fast as possible — even process streaming data in real time– and the deep pockets to pay for Cray’s presumably pricey systems. Despite the fact that big-data framework Hadoop gained popularity in part because it’s designed to run on commodity hardware, there’s always a place for high-end hardware when milliseconds really do matter, and there’s something to be said for pre-configured systems that take the guesswork out of building a big data environment, as I explained recently in a piece for GigaOM Pro (sub req’d).

Cray isn’t alone in pushing this high-performance, enterprise-focused big data vision, though. Oracle made a splash in October when it announced a Big Data Appliance that marries Hadoop, R, NoSQL and other technologies to the high-end hardware Oracle obtained when it bought Sun Microsystems. IBM also has an extensive big data software portfolio complemented by a systems business that includes supercomputers, as well. And although it doesn’t have an HPC pedigree like the others, Teradata has years of experience building systems optimized for analytics.

Cray won’t likely become a household name in the big data world, and its notoriously secretive customers might never divulge what they’re using its analytics products for, but there certainly is a market — however small — for super-big, super-fast and super-expensive data.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Buying into big data appliances
• The importance of putting the U and I in visualization
• A near-term outlook for big data

Greenpeace gave Google the top overall score (53 out of 100), while Cisco (49 out of 100), Ericsson (48 out of 100) and Fujitsu (48 out of 100) followed shortly behind. On the flipside of the top companies, were the stragglers, which included Oracle (10 out of 100) at the very bottom, and TCS (11 out of 100) and Telefonica (11 out of 100) at the second and third to last spots.

Greenpeace gives the most weight in its scores to companies for using their own technology to reduce the world’s green house emissions. For example, a networking company like Cisco develops smart grid technology which can save energy and reduce emissions. The second largest part of the Greenpeace score is made up by how well the company advocates publicly and politically for reducing emissions. Lastly, Greenpeace gives companies points for monitoring and managing their own company emissions footprint.

Google rose in the rankings from six overall in 2010 to number one this week. Last year Google invested close to a billion dollars into clean energy projects. Cisco lost its top spot in 2010 to Google this year.

IBM, which was 3rd in 2010, dropped to 9th overall this week. Greenpeace says that IBM fell across all three categories and obtained four penalty points in political advocacy for being a member of a trade association that is trying to block the EU’s greenhouse gas emissions reduction goals.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Smart Grid Apps: Six Trends That Will Shape Grid Evolution
• Report: An Open Source Smart Grid Primer
• Electric Cars Need Software, Not Just Hardware

Massive webscale data centers use commodity hardware.  The Open Compute Project, born of Facebook, but embraced by Rackspace, Intel, even Goldman-Sachs, specifies standard architectures for servers and eventually storage, targeting these huge, webscale data centers. Companies like EMC with a lot of proprietary IP to protect, do not fit into that vision very well.

As more workloads move into these big Amazon- or Rackspace-style public cloud platforms, they are lost to these old-line IT powers — the ranks of which also include Cisco, Hewlett-Packard and Oracle.

These legacy hardware companies are pursuing a scale-up strategy–vertically integrating more of their own compute power, storage, networking into single, pricey boxes, as opposed to the scale-out boxes underlying public clouds. In scale-out mode, huge numbers of inexpensive server are strung together to divvy up tasks. Adding more boxes, adds more scale.

Legacy vendors on wrong side of scale-out divide

The pitch from the likes of EMC or Oracle is that vendor integration holds value that customers will pay more for. Oracle even characterized Exadata at its launch a few years ago as a “cloud in a box” and Oracle CEO Larry Ellison claimed that a pair of Exadata machines could run all of Facebook. The problem is that there’s only one Facebook and it’s not running Exadatas. There is a market for those boxes, it’s just not growing as fast as the webscale world.

GigaOM Pro analyst George Gilbert summed it up: “The big disruption for EMC is that they’ve built their business around being the mainframe of the storage market. If you look at the webscale data centers, they’re all about scaling out vs. scaling up.”

EMC — like Oracle — is also hamstrung by a business model predicated on the sale of very profitable big boxes. The company fields an aggressive sales force that gets paid commission on these things. Needless to say that model won’t fly in the webscale world where Amazon or Rackspace buy thousands of servers at a time and can pretty much dictate their terms because their buying power is so huge.

Both Amazon and Rackspace wring every extra cent of cost out of their infrastructure which they then rent to high-priced software developers who build software and services for business users. Oracle and EMC, on the other hand,  sell high-priced gear to companies that then use lower-priced developers to tweak software and services. The difference to tech buyers is they are trading capital expenditure (capex) spent on Oracle and EMC for operating expenditure (opex) that flows to Amazon or Rackspace.

No one thinks EMC is going the way of the dodo bird, but until it and others in the scale-up world get the memo, their growth in these new data centers will be stunted, to say the least.

Photo courtesy of Flickr user redjar

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• The fourth quarter of 2012 in cloud
• The role of converged infrastructure in the data center

EMC’s promised Project Lightning server-based flash storage is now available — under the VFCache brand. But EMC’s not done — it plans a flash appliance dubbed Project Thunder, for later this year. The thunder-and-lightning duo takes aim at Fusion-IO, the hot-selling PCIe flash pioneer which has definitely caught EMC’s attention.

The promise of flash — or solid-state —  storage is its fast response times. That’s particularly important for performance-sensitive enterprise work loads, like SAP ERP and Oracle database applications. Putting that fast flash storage on the servers running those applications, as opposed to down the wire from them, can be a huge performance boost for them.

For Project Thunder “we’re bringing together several VFCaches in a single appliance attached to the server to create even better response times,” said Barry Ader, senior director of product marketing and management for EMC’s Flash business unit. EMC’s not-so-secret sauce is its FAST software which acts as a traffic cop, directing where different data should be stored in a tiered-storage architecture. It puts the most time-sensitive data on server-based flash storage while colder data goes to more distant disks or even tape drives.

Project Thunder, which will be demonstrated at a Monday event headlined by Pat Gelsinger, president of EMC’s Information and Infrastructure Products (pictured right), shows just how much EMC is feeling the incursions of Fusion-IO and other flash players.

Critics say EMC, the storage leader, is vulnerable to nimble flash-oriented startups, given EMC’s huge installed base of non-flash storage  and its big, expensive enterprise-focused sales force.  Ader said VFCache will be very price competitive with Fusion-IO but would not provide pricing.

EMC, never a company to shy away from confrontation, is taking the fight to its rivals — Ader said it shipped 24 petabytes of flash storage in 2011, more than anyone. He said EMC will bring more of its tools to bear as well … adding more intelligence to its hardware and software including its acquired deduplication technology with Data Domain in 2009.

EMC preaches value of mixed storage vs. all-flash-all-the time

EMC’s view is that while webscale workloads — where Fusion-IO has taken off — are important there’s also a huge market in legacy enterprise applications  that could use server-side flash for the hottest data but route less time-sensitive data to cheaper storage options companies already have deployed.

“The early adopters of PCIe technology were web scale-out early adopters. What we’re doing is bringing it to the mainstream for applications like Oracle, SQL Server and Exchange [server] — business critical applications,” Ader said.

“Many companies talk about flash but most are coming from a single [webscale] use case — that’s not what our customers are asking for. They want coordination across their systems and we’ll put the data in the  most appropriate place for them,” Ader said.

Of course, Fusion-IO CEO David Flynn has a different take:  What EMC is doing here “validates that flash in the server is unstoppable, but they represent the height of the mainframe era.”

One big obstacle for EMC  is its cost structure — it traditionally sells into the enterprise with an aggressive, highly paid sales force. As such, EMC cannot really price VFCache as aggressively as it suggests it will, Flynn maintained.  In addition, EMC sales people are used to selling to the storage people in the data center, not the server buyers. Server-side flash requires a different sale.  ”The server admins hate the storage guys,” Flynn said.

It’s clear the flash storage battle has been joined. What remains to be seen is if EMC’s mixed-storage approach will prevail in a world where more workloads are flowing to webscale data centers.

Feature photo courtesy of Flickr user Lars Kasper

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Flash memory: the continuing disruption of enterprise storage
• A near-term outlook for big data
• Infrastructure Q4: Big data gets bigger and SaaS startups shine

Now the two companies are arch-rivals and are engaged in an increasingly bitter, seemingly personal battle, the latest skirmish of which saw a California Superior Court judge throw out a fraud claim Oracle lodged against HP. He also opened up court documents that don’t show either company in a particularly good light.

How did it all go so bad?

First, Oracle bought Sun for $7.4 billion in a deal completed in January 2010. That meant Oracle, for the first time was in the hardware business and its servers would compete with HP servers. That sealed the fate of the relationship going forward.

The public bad feeling erupted in August 2010 when HP canned Hurd as CEO, then hired former Oracle president Ray Lane (pictured above right) as chairman and Leo Apotheker, former CEO of SAP, as CEO. SAP is a huge rival to Oracle in enterprise apps and Lane left Oracle after a bumping heads with Oracle chairman Larry Ellison (pictured at right.) Things have just deteriorated ever since.

Here are some highlights (low lights) of the slap fight.

In a letter to the New York Times in August 2010, Ellison took aim at HP’s firing of Hurd:

The H.P. board just made the worst personnel decision since the idiots on the Apple board fired Steve Jobs many years ago … That decision nearly destroyed Apple and would have if Steve hadn’t come back and saved them.

This is a shameless attempt to force customers to spend a lot of money to move to a platform over time that gives customers no benefits  … Oracle made this decision to slow Sun SPARC market losses.

Ray Lane calls out Hurd in his letter to The New York Times in October, 2010.

The bottom line is: Mr. Hurd violated the trust of the Board by repeatedly lying to them in the course of an investigation into his conduct. He violated numerous elements of HP’s Standards of Business Conduct and he demonstrated a serious lack of integrity and judgment

After Apotheker announced HP plans to buy Autonomy — another enterprise software company — for $11.7 billion in August, Oracle couldn’t contain itself.

In a statement on September 28, 2011, Oracle said Autonomy had shopped itself to Oracle first and Oracle turned it down. When Autonomy CEO Mike Lynch denied that, Oracle said: “Either Mr. Lynch has a very poor memory or he’s lying.”

When there was further denial, Oracle put out another statement entitled “Another whopper from Autonomy CEO Mike Lynch” and helpfully published the PowerPoint slides it said he and banker Frank Quattrone brought to the meeting.  The presentation is here and here.

According to the statement:

Ably assisting Mike Lynch’s attempt to sell Autonomy to Oracle was Silicon Valley’s most famous shopper/seller of companies, the legendary investment banker Frank Quattrone.  After the sales pitch was over, Oracle refused to make an offer because Autonomy’s current market value of $6 billion was way too high.

The next chapter in this saga may be a trial on HP’s remaining claims against Oracle which should kick off in April, but stay tuned: anything can happen and usually does.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Why the big data startup boom will likely be short-lived
• Report: How Mobile Cloud Computing Will Change Tech

The reason: These pure-play SaaS companies face heightened competition from one another and, perhaps more importantly, from legacy software companies that are selling their own Software-as-a-Service offerings. It’s a classic case of a maturing market: As more players enter, that increased competition takes its toll.

SaaS companies “are still trading at very high valuations, just not as high. Trees don’t grow to the sky,” said Marty Wolf, the president of Martin Wolf Advisors, a company that consults on merger and acquisition strategies, which compiled these numbers from Q4 2011.

In November, Wolf published research that showed valuations of SaaS companies were off the charts compared to more-traditional legacy software companies. And he predicted that growth would continue. To be fair, the growth is still there; it’s just smaller growth.

Wolf used Salesforce.com, a pure SaaS player, as a bellwether for the category, and that company’s stock price has been under pressure of late. As IBM, Oracle, SAP and Microsoft beef up their own SaaS offerings, Salesforce.com doesn’t have that market to itself anymore, Wolf said.

“In the past six months, SaaS has grown from an enterprise value that is at a 150 percent premium versus traditional software — to a 163 percent  premium today — so it’s up 10 percent versus enterprise software,” Wolf said via email.

During that same period, Salesforce.com has dropped 33 percent in its comparative enterprise value to traditional software. So Salesforce is still valued higher than enterprise software, but it has taken a hit in enterprise value, he added.

Wolf’s market-weighted-value index takes the market value of 120 companies grouped into four technology categories: IT services and BPO, IT supply chain services, software, and SaaS. The index assigned a value of 1,000 to each composite group on Dec. 31, 2008, and has tracked the category performance since then.

On the plus side, Wolf said he expects SaaS revenue to continue to grow fast, as there’s too much green-field opportunity. But as companies like Oracle, IBM, Microsoft and others sell more SaaS, pure-SaaS players like Salesforce.com will face more pressure.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• IT spending update, third quarter 2012
• Cloud computing infrastructure: 2012 and beyond
• Forecasting the future cloud computing market

This whole case is an example of two tech giants using the courts to shore up their competitive position. HP wants Oracle to keep building software for Itanium-based HP servers. Oracle wants to stop doing so. The reason for the impasse is that the two erstwhile partners are competing head on for more of their customers’ software and hardware dollars. HP is building its enterprise software business with Autonomy and other deals, while Oracle tries to make a go out of its Sun Microsystems server business.

As backstory, all of this is wrapped up in Oracle’ s hiring of deposed HP CEO Mark Hurd, a “hardware guy,” back on Sept. 6, 2010. Within three weeks, HP hired two “software guys” in former SAP CEO Leo Apotheker and former Oracle president Ray Lane as CEO and chairman, respectively.

Last night, California Superior Court Judge James P. Kleinberg dismissed Oracle’s claim that HP committed fraud by not informing Oracle of its plans to hire Apotheker and Lane.  HP sued Hurd after he left, and when Oracle hired him, the two companies worked out a secret “Hurd Agreement” whereby HP dropped the lawsuit and Hurd gave up $30 million in compensation.  Oracle’s point is that by hiring Apotheker and Lane, HP was moving into enterprise software to compete with Oracle, thus changing the dynamic of the vendors’ relationship.

In theory anyway, the current suit is all about Itanium. According to the unsealed Oracle court filing, HP’s positioning of Itanium-based Integrity servers as its go-to option for mission-critical enterprise applications at a time when Intel Xeon servers were taking off, put it in a corner.  Without Itanium, HP would be “strategically screwed,” according to an unnamed HP executive quoted in the unredacted Oracle documents.

Oracle claimed HP was funding the bulk of Intel’s Itanium work, but wanted to keep that fact to itself.  The document quotes another HP executive saying the “Itanium situation is one of our most closely guarded secrets.” That secret is now out of the bag.

The current legal tussle started last June when HP charged Oracle with breaching its promise to keep developing Oracle software for HP Itanium-based servers. Oracle had said it was de-emphasizing Itanium development because there was little demand and that Intel itself, which builds the chips, had no long-term plans to keep building them. HP, the only mainstream server company to use the chips, cried foul.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud computing infrastructure: 2012 and beyond
• A near-term outlook for big data
• Why the big data startup boom will likely be short-lived

However, I’m interested in what seems to be a growing number of startups and entrepreneurs at the event, and that seems to be a sign that real innovation and new business models are actually starting to happen when it comes to adding digital intelligence to the power grid and managing energy data.

Here are the top 10 trends that I took away from the event:

1. Managing big data. Managing large streams of energy data is fundamental to the future of the grid, as well as for helping consumers reduce their energy consumption. What methods the software companies use will also determine how well they’ll do in the industry. Startups like Opower and Tendril are essentially big-data plays, using cloud computing tools and sophisticated analytics, and both companies released interesting news at the show. Oracle, touting its software and data management, had a huge booth on the floor. I chatted with both the newly acquired Ecologic Analytics (bought by Landis+Gyr) and eMeter (acquired by Siemens), and these companies manage the massive amounts of data that flow off of meters for utilities. Cloud computing is no longer a dirty word for utilities; in fact, it’s likely the future of this business.

Cisco's connected grid router, ruggedized, inside

2. The Internet of Things. Beyond connecting meters and grid devices, the smart grid really extends to create the Internet of Things, connecting cars, home batteries, solar panels, light bulbs, thermostats, and consumer electronics like televisions and cell phones. The term the Internet of Things is oft-used in IT circles, but hasn’t really caught on in the smart grid and utility industries. But it will, and it will also provide new lines of business for the power grid vendors.

3. Prepaid electricity. Prepaid meters and electricity might not catch on in the U.S. all that quickly, but throughout the world it’s becoming a phenomenon — particularly in countries with low rates of credit card holders, and high percentages of populations that don’t have a steady income stream, like farmers. There were a couple of startups at the show selling these meters and software systems, like PayGo, but I’m not sure this is a startup play at this point. I’d expect the big meter companies to create their own solutions.

Prepaid energy and meters via PayGo

4. The year of the smart thermostat. I already did a slide show on this, but it seems like 2012 is the year utilities, consumers and device makers are starting to really pay attention to smart thermostats. As one show attendee explained to me, it’s the low-hanging fruit and the area where utilities can see the most “bang for buck.” Smart thermostats, which can be controlled by utilities, can help utilities shave off consumer power consumption when they need it (called demand response). Other players like Nest (which I didn’t see at the show) are looking to sell a coveted connected thermostat straight to consumers.

5. Energy storage batteries moving into commercial deployments. Battery makers made an appearance at the show, touting batteries that could provide energy storage and frequency regulation for the grid. Some are even selling batteries for homes, to connect with solar panels, and I could see many homes having their own batteries one day. Batteries for energy storage have long been discussed, but the problem has always been that they are one of the most expensive forms of energy storage. However, some power producers and utilities now have commercial battery farm deployments in place, so we’ll see this year how popular batteries for energy storage become.

6. Ditch the hardware; focus on software. After talking with a variety of home energy management startups, it feels like there has been an acknowledgement that focusing on the hardware piece of home energy management hasn’t worked out all that well, and that software is a much better bet for most companies. EnergyHub did a slight pivot to focus more on software to manage smart thermostats, and the company tells me it’s having its best financial year to date. Tendril also doesn’t focus at all on hardware anymore, and is only focusing on creating an open platform for connecting devices. However, Nest is the outlier in this, and is basing its business around its device.

7. The pivot. Constant reinvention is common in Silicon Valley and for startups. I feel like there’s a perception that it’s not that acceptable in the utility and power worlds. However, it’s becoming increasingly O.K. to openly discuss changing strategies and working out various business models. As the smart grid industry matures, the pivot is going to become a more common maneuver. Embrace change!

8. Cellular coming on strong. A few years ago, the cellphone companies were charging utilities way too much to use their networks for the backbone of the smart grid. But today, phone companies like Verizon have changed their pricing strategy and are actively working with utilities on strategies around a per meter basis. As I wrote earlier this week, there’s no longer a big debate about what type of wireless to use, from cellular, to mesh, to Wi-Fi to WiMAX. The network companies are offering them all to be flexible to the utility.

9. M&A still rolling. The smart grid is still one of the best places across the cleantech sector to sell a startup. Since the IPO market is pretty crappy right now, who could buy who was a topic of conversation at the show.

10. Go big or go home. This one comes courtesy of Tendril’s CEO Adrian Tuck who says he sees a lot of potential consolidation in the energy data management space this year, and he sees companies going big or going home. What do you think?

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• Key technologies for the smart city
• Smart Grid Apps: Six Trends That Will Shape Grid Evolution