GigaOM » Netezza

How data warehousing is now a cost-effective solution for businesses

nraden — Mon, 13 May 2013 06:55:34 +0000

The new economics of data warehousing provide attractive alternatives in both costs and benefits. While big data gets most of the attention, evolved data warehousing will play an important role for the foreseeable future. In order to be relevant, data-warehouse design and operation need to be simplified, taking advantage of greatly improved hardware, software, and methods.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Health care and big data in 2012

Jody Ranck — Mon, 10 Dec 2012 19:53:01 +0000

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

2012: The Hadoop infrastructure market booms

Thu, 26 Apr 2012 19:22:32 +0000

For years, technologists have been promising software that will make it easier and cheaper to analyze vast amounts of data in order to revolutionize business. More than one solution exists, but today Hadoop is fast becoming the most talked about name in enterprises. There are now more than half a dozen commercial Hadoop distributions in the market, and almost every enterprise with big data challenges is tinkering with the Apache Foundation–licensed software. This report examines the key disruptive trends shaping the Hadoop platform market, from integration with legacy systems to ensuring data security, and where companies like Cloudera, IBM, Hortonworks and others will position themselves to gain share and increase revenue.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

IBM tunes big data appliance to retailers

Derrick Harris — Tue, 24 Jan 2012 14:46:17 +0000

IBM is working the reins of its Smarter Commerce initiative by rolling out a new Netezza analytics appliance designed to help retailers churn through petabytes of consumer sales data in real time. With the IBM Netezza Customer Intelligence Appliance, the systems giant is trying to capitalize on its contention that online activity is becoming increasingly important for retailers, so they can’t afford to not understand customer behavior and improve the web experience.

According to the IBM press release, “70 percent of a customer’s first interaction with a product or service takes place online,” which underscores the need for analyzing customer activity to find out what they really want. If consumers connect with a retailer’s site and enjoy the experience of shopping there, they’re less likely to spend their money on any of the countless other sites selling the same goods.

Of course, many large retailers — especially those that can afford IBM Netezza systems — also have brick-and-mortar stores, and IBM isn’t omitting those operations. The appliance actually aggregates data from in-store and mobile shopping, as well as online shopping, to let retailers connect the dots between activities across the different channels. In theory, all this information should help stores create better promotions, target specific user types, and perhaps even optimize website or physical-store design.

The new Netezza appliance also incorporates business-intelligence software from IBM’s Cognos line and specialized retail analytics software from IBM business partner Aginity.

When IBM talks about the new appliance handling petabytes of data, however, it’s not talking (at least directly) about the type of unstructured data that has made Hadoop such a hot technology in the big data space. Essentially an enterprise data warehouse, Netezza is SQL-based and therefore ideal for structured data such as customer records, whereas Hadoop is all about unstructured or semi-structured data coming from sensors, social-media streams, server logs and other non-traditional sources of business data.

One step on the path to big data nirvana is combining structured and unstructured data to gain an even more-complete view of what customers are doing and feeling, which is something IBM is trying to do through tighter integration of Netezza with IBM’s Hadoop-based Infosphere BigInsights software. It’s not alone in this quest, as rivals Teradata, EMC Greenplum, Oracle and HP all have similar strategies in place with their respective components.

However, if revenue growth has anything to do with how IBM and other vendors continue to expand their big data capabilities — and it does — we’re only seeing the tip of the iceberg in terms of what will be possible. In IBM’s fiscal fourth quarter, business analytics revenue was up 16 percent year-over-year, Smarter Commerce-related initiatives were up 25 percent, and Netezza revenue jumped up 70 percent.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Birst, ParAccel team to put a pretty face on big data

Barb Darrow — Thu, 12 Jan 2012 08:01:40 +0000

In a meeting of analytics minds, Birst and ParAccel are teaming up to put Birst’s graphical reporting gloss atop ParAccel’s database.

This deal is just the latest evidence that while collecting big data is important, the ability to put that data into formats that make sense to business users is equally critical.

ParAccel builds a fast columnar database specifically designed to analyze lots and lots of information. In that arena, it competes with Vertica, Greenplum, Netezza and Aster Data, once-independent companies that were all acquired in the past two years by Hewlett-Packard, EMC, IBM, and Teradata respectively.

Birst CEO Brad Peters maintains that the Birst/ParAccel technology duo will compete with high-end solutions like those and SAP’s HANA analytics appliance at a fraction of the cost.

Target uses include customer service call centers or big telco billing applications, he said. ”This is for any jobs where you have a really high volume of interactions that can generate hundreds of millions or billions of data points,” that need to be sorted out, classified and visualized, he said.

The partnership means Birst will use ParAccel technology as part of its solution, and make Birst available to ParAccel customers. A ParAccel spokeswoman said the two companies have done a lot of integration work, that ParAccel is using Birst internally, she said.

ParAccel also partners with MicroStrategy, but MicroStrategy’s technology suits more complex enterprise rollouts that take time to sell and implement while Birst’s SaaS capability suits it for quicker, easier sales cycles.

Birst, which offers its business analytics both in an on-premises appliance and as software-as-a-service, competes with Pentaho, an open-source analytics provider and Jaspersoft, both of which also work with ParAccel databases.

It makes sense for database companies to partner with analytics players but given the amount of consolidation that’s already gone on in these worlds, it’s probably just a matter of time before some of these smaller analytics companies join the ranks of the acquired.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

IBM buys Emptoris for supply chain analytics smarts

Barb Darrow — Thu, 15 Dec 2011 17:30:04 +0000

IBM is buying Emptoris, a specialist in supply chain and contract management analytics, in what is the latest example of an entrenched IT provider’s buying analytics and cloud capabilities.

Emptoris bills itself as an analytics supplier, and, according to its website, it delivers its software in on-premise, hosted and Software-as-a-Service (SaaS) models. This news comes just a week after IBM announced its $440 purchase of DemandTec, a supplier of web-based analytics for retailers.

This advent of cloud computing — including SaaS — and the big data boom means that the need for analytics to help business users sort, filter and visualize all of that information is exploding. Last week SAP bought SuccessFactors, a SaaS-based human resources specialist. Part of the appeal there was SuccessFactors’ workforce analytics know-how.

This overwhelming need for better, glitzier analytics helped drive IBM’s earlier purchase of Netezza, EMC’s acquisition of Greenplum, SAP’s development of the well-regarded HANA and Oracle’s decision to build Exalytics. Netezza, Greenplum, Hana and Exalytics are all data analytics appliances.

In its statement about the Emptoris deal, IBM said:

With this acquisition, IBM extends its “buy” portfolio of solutions focusing on the needs of procurement and sourcing professionals. Procurement and sourcing professionals increasingly need better supplier management, spend analysis and contract management solutions to lower sourcing costs, improve contract compliance and help meet growing customer expectations of supply chain efficiency.

Emptoris, based in Burlington, Mass., claims 350 customers, including blue chip accounts like Delta Airlines, Keyspan, Motorola, Kohler, Heinz and UnitedHealthcare.

Terms of the deal were not disclosed. And there is some doubt whether Emptoris is as cloud-savvy as portrayed, at least partly because it’s an older company, founded in 1999, before the advent of cloud computing or SaaS.

“If you look at this compared to DemandTec, Emptoris has 725 employees versus 350 [for DemandTec] which means they have 107 percent more employees but 28 percent fewer customers,”said Marty Wolf, the president and founder of Martin Wolf M&A Advisors. That scenario smacks more of the old-school software-license-sales-and-support model as opposed to SaaS. That means IBM bought vertical expertise that it can repurpose to the cloud itself, he said.

And it got a bargain. “I expect they paid 1.5 to 2.5 times trailing revenue [for Emptoris] compared to 4.5 times trailing revenue for DemandTec,” Wolf said.

Here is an interesting aside for those of us in the Bay state: As of now IBM has bought 20 companies in Massachusetts since it snarfed up Lotus Development Corp., in 2003.

This land grab for analytics — especially in specialized areas — is bound to continue into 2012. Comments from industry leaders over the past few months show that this M&A will only continue in this arena. Pat Gelsinger, the president of EMC’s information and infrastructure products unit, said he expects more action in what he called a $70-billion-and-growing market for analytics. Many visualization, data transport, and extract, transform and load (ETL) tools still “have to re-emerge in this big data domain,” he said.

In other words: Buckle your seat belts.

Photo courtesy of Flickr user Walmart Stores

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Why the big data startup boom will likely be short-lived

Paul Miller — Fri, 09 Dec 2011 01:44:19 +0000

There has been a remarkable flowering of companies over the past year or two, all riding a wave of developer and investor enthusiasm for the loosely defined concept of “big data.” But given that the big data startup market is probably overvalued and headed for a lot of consolidation, these new companies’ days might be numbered.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Wikileaks’ Spy Files paints damning picture of tech surveillance

Stacey Higginbotham — Thu, 01 Dec 2011 20:09:08 +0000

Wikileaks released a database on Thursday of tech providers involved in government tracking around the globe and quite a few familiar names are on the list, including Alcatel Lucent, Nokia and Cisco. Called The Spy Files, the project includes 287 records gathered and curated in conjunction with several newspapers, Working with Bugged Planet and Privacy International, as well as media organizations from six countries including the Washington Post in the U.S. While the tone of the essay accompanying the release is designed to inspire fear, the accumulation of product brochures, manuals and presentations are pretty damning.

The sum of all these parts is a huge privacy violation

For example, when viewed in a single presentation, claims made by Netronome — a networking chip maker — that it can process massive flows of information in real-time in order to intercept unlawful packets might not be worrisome. However, when juxtaposed against the violation of human rights that the word “unlawful” can hide, it becomes part of a broader story of surveillance and what many in the U.S. would rightly regard as a violation of their privacy. From the intro to The Spy Files:

Intelligence agencies, military forces and police authorities are able to silently, and on mass, and secretly intercept calls and take over computers without the help or knowledge of the telecommunication providers. Users’ physical location can be tracked if they are carrying a mobile phone, even if it is only on stand by.

But the WikiLeaks Spy Files are about more than just ’good Western countries’ exporting to ’bad developing world countries’. Western companies are also selling a vast range of mass surveillance equipment to Western intelligence agencies. In traditional spy stories, intelligence agencies like MI5 bug the phone of one or two people of interest. In the last ten years systems for indiscriminate, mass surveillance have become the norm. Intelligence companies such as VASTech secretly sell equipment to permanently record the phone calls of entire nations. Others record the location of every mobile phone in a city, down to 50 meters. Systems to infect every Facebook user, or smart-phone owner of an entire population group are on the intelligence market.

WikiLeaks' leader Julian Assange

What people can do in reaction to the information

But will the shock of seeing the tools of our digital surveillance society aggregated together with tales of abuses be enough to stop governments from implementing ever more technical means of gathering, analyzing and using information? My colleague Mathew believes the transparency here will help address the problem, but I have less confidence. While the outrage currently aimed at CarrierIQ in the U.S. over its software that tracks users via their smartphones is intense, in a month it may well be forgotten. Lasting and real change is harder to come by when it comes to these issues, because it’s hard for enough people to sustain the outrage needed to go against something that benefits the government.

That the CIA or FBI can put GPS devices on cars without a warrant (since questioned by the courts) or attempt to access the data about web sites or on hard drives is a huge temptation even in the freedom-loving U.S.. Because of the ephemeral nature of our digital transactions, many citizens are unaware at how concrete and traceable their bytes really are. Just because you don’t recall a tweet you made six months ago, doesn’t mean it can’t be found and dredged up against you. Likewise, search data sticks around for nine to 18 months; your phones are a black box of information that the feds would love to crack; and the government and law enforcement don’t want to restrict their access to any of this digital gold mine. Unlike the Founding Fathers, who were protecting their rights after a revolution, our politicians are protecting the status quo and see surveillance as a tool to help maintain that.

As for sales to dictators and other countries that might use access to Internet monitoring, phone tracking, listening devices and GPS logging software as a means of repressing their citizens or activists, the practice may be morally reprehensible, but its unclear if it runs afoul of U.S. laws. Yes, there are laws that prevent U.S. firms from selling technology to certain governments, but the list of governments and technologies is by no means comprehensive.

But, other than shining a light on the practice and writing stories, what is there to be done? In the U.S., the courts are the best method of counteracting overreaching governments, which means someone has to get caught up in some zealous digital manhunt. The Wikileaks Spy Files may help someone realize this is going on, but it’s unlikely we’ll see laws designed to really protect privacy in the digital age. So then how should citizens, public policy groups and others take this data dump? More than the information that DPI inspection gear from Arbor Networks or data retention and analysis tools from Netezza or RainStor are used in some cases to target or suppress citizens or activists, these are the questions the report evokes.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

Why open source tools might need a hardware hook

Derrick Harris — Thu, 04 Aug 2011 22:00:48 +0000

The great things about open source software stacks such as OpenStack and Hadoop is that they’re free and they work. The not-so-great thing is that — like many open source projects — they can be difficult to configure and manage. Luckily, hardware vendors are stepping in to fill the void.

The hardware crew

Dell has been particularly busy trying to take the hardware guesswork out of two of the hottest open source projects around: OpenStack and, on Thursday, Hadoop (technically, Cloudera’s distribution of Hadoop). Essentially, these are hardware configurations specifically designed to optimally run their respective software targets. Accompanying both is Crowbar, Dell’s open source software for deploying the software onto the hardware and then managing the existing system.

The big difference between the two offerings is the price tag. Dell’s OpenStack solution is a reference architecture with optional support and services, while its partnership with Cloudera is a full-on product that includes hardware, software, support and services.

In the OpenStack space, Dell has competition from appliance-peddling startup Nebula, which recently launched. That company, led by former NASA CTO Chris Kemp, has developed its own appliance preloaded with OpenStack software and management tools to run the dozens of commodity servers plugged into it. However, while it’s focused on OpenStack for the time being, Kemp isn’t ruling out developing appliances designed for Hadoop, NoSQL databases and other next-generation software platforms.

Scaling out is hard

Appliances, or even tying specific hardware to specific software, can be a tough sell. But when distributed systems are involved, they might be necessary. Certainly, it’s neither an uncommon, nor an unsuccessful, practice with many proprietary software products.

Teradata , IBM Netezza and Oracle Exadata are three very successful massively parallel database offerings tied to hardware appliances. Dell has gone down this path with other cloud products, too, partnering with Joyent to sell its Dell Cloud Solution for Web Applications. EMC is selling appliances optimized and preloaded with both its Greenplum database and the Enterprise edition of its Hadoop distribution.

The consensus, it seems from the outside, is that managing distributed systems is difficult. It might be easier just to buy an appliance or a specially configured hardware stack and focus on using the application rather than installing and supporting the system.

This could be especially true for open source projects like OpenStack, Hadoop and others that don’t necessarily come with enterprise support. There’s plenty of promise, but it’s a lot of work to learn the software and start using it in production with also having to worry about what hardware architecture will provide the best price, performance, reliability, etc.

Software-plus-hardware success

For companies selling appliances, such as Nebula, or just hardware and optional support, such as Dell, it’s probably not too tough a sell getting customers to pay for something they otherwise could get for free. If they want support, they’re probably buying a commercial version of the open source software anyhow. Why not complete the prudent approach and buy specialized hardware and support, too?

Both Nebula and Dell already claim customers in the pipeline for their offerings, despite the fact that they’re brand new or, in Nebula’s case, not even generally available. I suspect new OpenStack member HP will get in on the preconfigured OpenStack system business, too. It already sells does so for its own cloud software. Perhaps Hadoop also will find its own ecosystem of systems partners outside of Dell.

Then we’ll see how this plays out. Is software enough to get mainstream businesses on board with new technologies, or would they rather just open up their checkbooks to ensure everything from the servers on up is done right? Certainly, it will vary by company, but I predict the software-plus-hardware hook will be a successful business model, if only because scale-out infrastructure isn’t easy.

Image courtesy of Flickr user JohnSeb.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

5 cloud and big data insights from IBM’s earnings call

Stacey Higginbotham — Tue, 19 Jul 2011 15:11:57 +0000

Jim Baum, IBM Netezza, at Structure Big Data 2011

IBM shares were trading up Tuesday morning after it reported strong second-quarter results yesterday. While Big Blue got a profit boost from recently released mainframe products, it also gave us some indications for the future growth of the cloud and big data. Mark Loughridge, IBM’s senior vice president and CFO for Finance and Enterprise Transformation, said IBM is still on track to double its cloud revenue in 2011 compared with the year before.

Because IBM’s definition of cloud can be pretty loose (i.e., it sells a lot of software, servers and services that conceivably could be labeled as cloud) take that with a grain of salt. However, here’s what we can learn:

The private cloud business is doing well. So much so that Loughridge said, “In private cloud, IBM’s average transaction size more than tripled from a year ago.” It also released new software to create private clouds during the quarter.

New products can keep revenue growth up. IBM has made more in the first of 2011 in cloud than it did the year before and said it had 2,000 cloud wins to date, but some of that success comes from new products such as the new private cloud software mentioned above as well as the “IBM Smart Cloud” infrastructure as a service product that launched also during the quarter.

New markets help boost growth too. IBM saw strong growth in emerging markets such as Brazil, Russia, India and China (the BRIC countries) but also growth in cloud deployments in places like Africa. Many emerging economies are turning to cloud computing as they build out their IT infrastructure, leapfrogging the legacy client server paradigm. IBM appears to be benefitting from this trend.

Big data is big business. IBM’s business analytics software grew by double digits for the seventh consecutive quarter. The company’s distributed database products experienced double-digit growth in the base business, and Netezza’s transactional volumes were up 70 percent versus a year ago,according to Loughridge’s prepared remarks.

Is it time to say sayonara to Sun gear? IBM’s Power brand of servers are winning over the competition, especially the former Sun boxes now owned by Oracle. Loughridge said that IBM had 250 competitive displacements that resulted in more than $300 million of business. About 60 percent of these wins came from Oracle’s legacy Sun-installed accounts, and 30 percent came from HP-installed accounts–a ratio holding steady from the previous quarter.

So there we have it. IBM has long been seen as a bellwether for the overall IT industry and for business spending on technology in general, and now it can help us predict the future for the cloud and big data.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.