Big data, more open
Druid, an open source database designed for real-time analysis, is moving to the Apache 2 software license in order to hopefully spur…
$3.5 billion, with a "b"
Google just finished off another record-setting quarter and year for infrastructure spending, according to the company’s earnings report released last week. The…
Microsoft is talking a lot about the scale of its cloud computing platform lately, but scale alone won’t help it steal revenue from Amazon Web Services or Google. Microsoft’s advantage is in commercial software, which brings more profi and acts as a gateway to other services.
It’s a good thing Salesforce.com has “dramatically” expanded its data center footprint in order to support its new Wave analytics service, because the product’s performance could ultimately decide how it fares against other next-generation applications that understand the visualization just as much.
The co-founders of analytics startup Interana came on the Structure Show podcast this week to talk about how to spread data analysis throughout customer accounts, the types of things you can do with event data, and the experience of starting a company with your spouse.
Chinese search engine Baidu is trying to speed the performance of its deep learning models for image search using field programmable gate arrays,…
Facebook’s James Pearce came on the Structure Show this week to talk about how Facebook handles open source projects, and how the new TODO group it’s part of is going help the rest of the world release their own code to the world.
Facebook is upping the ante in the open source world, announcing on Monday a new open source technology for routing data across server caches, and also a new open organization (along with other big-name companies) to help foster open source efforts.
A partnership between Google and Mesosphere further’s Google’s strategy to sell the world on its way of automating applications and resources. Cluster management is important — even sexy when wrapped in the lore of Google or Facebook — and now Google claims it’s easier than ever.
Google spent nearly $2.65 billion on infrastructure in the second quarter, upping its year-over-year investment by $1 billion and more than tripling the amount it spent in the second quarter of 2012.
Docker is one of the most-popular technologies in computing right now, and Solomon Hykes is one of the people responsible for it. On our latest Structure Show podcast, he talks about the past, present and future of the project and company he helped build.
On this weeks Structure Show podcast, Barb Darrow, Stacey Higginbotham and I talk about which speaker and topics have us mosted excited about this weeks Structure conference. Needless to say, theres a lot to like about it.
GoDaddy has filed for an IPO, and its S-1 report sheds some light on the company’s technological underpinnings. Here are the highlights.
Inspired by Google’s famous approach to resource management, Apache Mesos is the open source software that manages the large pools of servers and cloud instances at companies such as Twitter and Airbnb. Mesosphere, a company trying to commercialize it, has raised $12.75 million since launching.
On this week’s Structure Show, we discussed how VMware handles automation both for its customers and in its own cloud, and how a startup called Mesosphere wants to turn parts of that approach on their head by taking inspiration from Google and Twitter.
Twitter recently detailed its Manhattan database that’s supposed to be the default system for most new applications within the company. Here, the engineers behind Manhattan explain why they built it like they did.
Here are some highlights from a Reddit Ask Me Anything featuring members of Twitter’s open source engineering team. It’s a pretty informative look at Twitter’s open source culture, the statuses of its various projects and its high hiring standards.
Google continued its heavy infrastructure spending in the first quarter of 2014, to the tune of more than $2.3 billion. It’s the third consecutive quarter the company has topped $2 billion in capital expenditures.
Twitter has released some details on its Manhattan database system, which was built to power a wide variety of applications that existing technologies can no longer handle. Twitter handles thousands of tweets per second, which means speed and scale are critical.
The name pretty much says it all: WebScaleSQL. That’s the new open source project Facebook announced on Thursday, which includes early contributions from other web giants that have pushed MySQL to its limits.
Facebook doesn’t have a shortage of IT gear or smart engineers, but it does have limits on power and how many hours those engineers have in a day. On Thursday, the company explained how its Look Back videos pushed both to the limits.
On Thursday, Facebook announced via a post on its engineering page that it has revamped the Thrift framework it built in 2006…
Apache Mesos is the open source cluster-management software that automates operations at companies such as Twitter and Airbnb. Now, a startup called Mesosphere is building a business around taking it mainstream.
Say what you will about Satya Nadella, but don’t say he doesn’t understand the value of Microsoft’s technology. In an era where it’s competing against Google and Amazon to become the default digital platform company, that’s more important than ever.
Google spend a little less on infrastructure during the fourth quarter than during the third quarter, but it still spent a lot. Like $2.25 billion a lot.
Netflix has open sourced a tool called Suro that collects event data from disparate application servers before sending them to other data platforms such as Hadoop and Elasticsearch. It’s more big data innovation that hopefully finds its way into the mainstream.
Facebook has open sourced a new embedded database called RocksDB that’s meant to take advantage of all the performance flash has to offer, from right on the application server. It might be a sign of best practices to come.
Dropbox has hired Kevin Park as its new head of technical operations and IT. Park was at Facebook from 2006 until 2011,…
Airbnb open sourced a new tool called SmartStack that automates the management of the company’s various services. If they’re healthy, SmartStack facilitates communications. If they’re not, it knows and it takes them down.
Facebook has one of the largest, if not the largest, MySQL installations in the world, and has created a tool to keep…
Google spent nearly $2.3 billion on infrastructure in the third quarter — nearly 50 percent more than last quarter and nearly three times the third quarter of 2012.
Twitter’s IPO filing is full of nuggets about the company’s revenue and overall business, including our first real look into the company’s data centers. We still don’t know where they are, but we know what they cost.
At Structure: Europe 2013, New Relic Founder Lew Cirne, Kleiner Perkins General Partner Michael Abbott (former Twitter engineering VP) and North Bridge General Partner Jonathan Heiliger (former Facebook engineering VP) spoke about the business opportunities around next-gen analytics.
DataStax has raised $45 million in series D round to scale its business of selling enterprises on the Cassandra NoSQL database. It might not get all the attention, but Cassandra does have some big users.
Google and Microsoft might have disappointed investors in the last quarter, but they didn’t disappoint their equipment manufacturers and data center partners. Both companies spent boatloads on infrastructure.
Microsoft CEO Steve Ballmer’s claim that the company operates more than a million servers — second only to Google — might not be so crazy.
Oracle’s 12c database is definitely designed with software-as-a-service providers in mind, but there’s a big difference between Salesforce.com and the next generation of cloud-service startups.
Feedly has been adding servers and changing its data architecture — including a move from MySQL to HBase — as more users come aboard and ask the company to start managing many more RSS feeds.
This week, both Facebook and Yahoo detailed new efforts to manage real-time data flows within their myriad systems. Yahoo’s work is an open source implementation of Storm designed to run on the same cluster as Hadoop and even share resources.
Google Image Search just got a whole lot better, and the company’s purpose-built machine learning system infrastructure is a big reason why. No surprise, Jeff Dean helped build it.
As more companies build their businesses on cloud infrastructure, it’s important to not only understand the technical decisions behind their architecture, but also the economic ones. That’s one of the topics we’ll explore at Structure.
GigaOM contributor James Urquhart shares some of the best books, blogs and other information on the concepts of devops and complex IT systems.
Google spent $1.2 billion on property and equipment in the first quarter of 2013, nearly doubling last year’s first quarter.
Former VMware CEO and current Pivotal Initiative leader Paul Maritz shares his thoughts on how the future of enterprise IT must mirror the practices of consumer web companies.
Microsoft is no joke when it comes to building web infrastructure and developing techniques such as machine learning. The company thinks its heavy investment in these areas will pay off big-time in the years to come.
Five years ago, LinkedIn was a shell of the technology company it is today. Here’s an inside look at where it came from, what it’s become and where it’s going.
Box VP of Engineering Sam Schillace talks about building the service that became Google Docs, then fighting for the service’s life, and now rethinking collaboration for the mobile web.
Google spent more than a billion dollars on infrastructure in the fourth quarter, representing the company’s second-biggest quarterly expenditure ever. As it competes against Facebook, Apple, Yelp and Amazon, the company can’t afford to stop building data centers now.
Zynga has deployed nearly 100 nodes of MemSQL, the hot new database from two former Facebook engineers. It might not be a magic pill for Zynga’s woes, but it could help the company boost revenue and even build new types of games.
China’s big four internet companies are big — huge, in fact — but they’re not yet technological innovators like their American counterparts. However, scalability is an an issue that knows no borders, which has spurred some cross-continental cooperation. Will it also inspire a Chinese tech awakening?
At the AWS Re: Invent conference, engineers from Pinterest, Flipboard and Yelp detailed some of the strategies their companies employ in order to keep costs low as computing demand increases. The keys are keeping an eagle eye on usage and using the right types of resources.
AWS CTO Werner Vogels and I sat down at the AWS re:Invent conference yesterday to talk about whether large companies are actually using the cloud to innovate through new styles of applications. Vogels says they are, and has plenty of examples to prove his point.
Obama for America CTO Harper Reed had helped build Threadless, a site for selling hip t-shirts, but he had never done anything like this. Here’s how he and his team built a tech platform that might forever change how presidential campaigns are built.
Facebook has open sourced a new system called Corona for scheduling and managing Hadoop jobs. Corona attempts to do away with many of the problems that come along with massive-scale Hadoop operations, and soon looks to take Facebook’s Hadoop deployment beyond just MapReduce.
MongoDB startup MongoLab has built a seemingly solid business delivering the popular NoSQL database via the Platform as a Service model. The trick, CEO Will Shulman says, is knowing how to sell to developers and understanding why someone would use MongoDB in the first place.
When companies such as Google and Facebook design their own servers, switches and data centers, it’s more a business decision than it is a test of their hardware-hacking skills. Custom gear means lower power bills, better performance and the flexibility to adapt to unforeseen situations.
Facebook knows something about big data — it collects more data and has built more tools than almost anybody else. Here, Facebook’s Jay Parikh and Accel Partners’ Ping Li talk about what lessons big data startups can take from Facebook to build businesses that can succeed.
It’s not for everyone, but if you’re storing petabytes of data Hadoop, Quantcast thinks it has the cure to your woes. Its newly open sourced Quantcast File System promises smaller clusters and better performance, and it has proven itself over exabytes of data inside Quantcast.
Orbitz has transitioned a major system off of Oracle’s Coherence database and onto the NoSQL Couchbase Server, but the database giant still has a significant footprint in Orbitz’s data centers. It’s all part of being a big company trying to roll with the IT punches.
Hosting giant GoDaddy has completed its investigation of Monday’s outage and deemed it was not the result of a DDoS attack as originally rumored, but rather the result of network failures within GoDaddy’s system. The outage crippled hundreds of thousands of web sites.
Etsy shared the details of its hardware architecture on Friday, showing the world a whole lot of Supermicro servers running everything from web servers to Hadoop. At this point, software is the name of the game at webscale, so hardware openness is just welcome community service.
Netflix has open sourced Chaos Monkey, a service designed to terminate cloud computing instances in a controlled manner so companies can ensure their applications keep running when a virtual server dies unexpectedly. In the past year, Chaos Monkey has terminated more than 65,000 of Netflix’s instances.
Companies doing business on the web and in the cloud must spend a lot on infrastructure to stay running and to support new services. Here’s what Apple, Facebook, Google, Microsoft, Yahoo and Zynga spent in the recent quarter, and what it might mean.
If Twitter wants to remain opaque about its practices, that’s fine — but it shouldn’t expect any slack from upset users or investors. Blaming a two-hour outage on an “infrastructural double-whammy” after remaining mum on even where its data centers are located doesn’t exactly inspire confidence.
In cloud and big data, the second quarter of 2012 featured several high-profile deals and product launches that could reshape the marketplace for…
For better or worse, Hadoop has become synonymous with big data. In just a few years it has gone from a fringe technology to the de facto standard. But is the enterprise buying into a technology whose best day has already passed?
Last week’s AWS outage outage might have outsmarted Netflix’s Chaos Monkeys, but the content-distribution giant isn’t about to turn its back on cloud computing. It was a relatively small blip in what has been better availability since the company made the move entirely to the cloud.
AOL is taking its flexible infrastructure strategy to a whole new level of flexibility by building data centers that are about the size of French door refrigerators. Now, AOL will be able to deploy infrastructure where needed with little more than an electrical outlet required.
Lest storage vendors thought they were immune to disruption that open source hardware is having on the server industry, Netflix’s new Open Connect content-delivery network might make them think again. It’s inspired by open source storage designs first released by Backblaze almost three years ago.
10gen, the creator and commercial entity behind the popular MongoDB database has raised another $42 million and wants to take the technology to an application near you. The money will help 10gen double down on research and development to make MongoDB live up to its hype.
Facebook’s hyperinflated valuation heading into its IPO has everything to do with its promise, and very little to do with its actual profits. Here are some numbers we know about Facebook’s infrastructure that speak to its promise perhaps as much as its 900 million users.
As the world once again starts analyzing Yahoo’s myriad woes after Sunday morning’s ouster of embattled CEO Scott Thompson, I’m left wondering if its investment in Hadoop didn’t aid in the company’s demise, even if it’s a way down the long list of Yahoo’s mistakes.
If Facebook really is overvalued leading up to its IPO, privacy might be the underlying cause of the company’s missed expectations. As it turns out, pleasing both investors and users isn’t easy for a company that relies heavily on advertising and personal data.
If you think about it, Netflix’s metamorphosis into a company that runs its infrastructure completely atop cloud-based resources is truly remarkable. For many companies, such as site-optimization and CDN provider Yottaa, the bigger they get, the harder it is to justify the cloud’s cost and performance.
Updated: Twitter has shared numerous aspects of its infrastructure over the past few years, and its decision to open source its work on…
Instagram is pretty proud of the infrastructure it built atop the Amazon Web Services cloud, but does an acquisition by Facebook mean goodbye to Amazon? If I were a betting man, I’d say there are already some engineers working really hard to make that happen.
In a webscale data center, peak efficiency feels like a blast furnace. I stepped into the hot aisle of Dell Modular Data Center and 1,920 servers blasted 115-degree air right in my face. If eBay’s Dean Nelson has his way, that was just the beginning.
OMGPOP can thank the cloud for its acquisition by Zynga on Wednesday. The gaming startup, whose Draw Something iPhone used cloud computing and a NoSQL database to scale from zero (relatively speaking) to more than 35 million downloads in three weeks and never miss a beat.
It all seems so easy: You log into Facebook, update your status, tell everyone where you are and — voila! — your Timeline is geospatial. Only, while it’s just one extra step for you to add location, building that capability was a tad more complicated for Facebook.
Twitter has been on a tear lately when it comes to open sourcing big-data tools. The latest two are Cassie, a client for managing Cassandra clusters, and Scalding, a MapReduce framework for simplifying the creation of Hadoop jobs. Big data won’t be black magic forever.
Zynga is in a tricky position technologically thanks to its new frenemy status with Facebook. Zynga claims 240 million active users, and it wants a lot more, but scaling to those heights might require one heck of a computing infrastructure.
With more than 45 million users already connected to his company’s cloud storage service, Dropbox CEO Drew Houston knows he has an infrastructure challenge ahead of him. That, Houston says, is a big reason Dropbox bought Cove, which brings with it Facebook engineering cred.
Facebook’s S-1 filing shows the company is all about infrastructure. The ad revenue and user experience it relies on to exist mean Facebook can’t afford to take it easy on IT, which means shareholders and users will both find plenty of reasons to get upset.
Somewhat lost in the greater story of Amazon Web Services’ new DynamoDB NoSQL database is that the new service runs atop a solid-state storage system. By abstracting those SSDs behind a NoSQL service, AWS is trying to prove that hardware presents greater opportunities than Infrastructure-as-a-Service alone.
At last week’s MongoSV conference in Santa Clara, Calif., a number of users shared their experiences with the MongoDB NoSQL database. One common theme: NoSQL is necessary for a lot of use cases, but it’s not for companies afraid of hard work.
Never content with good enough when it comes to speed, Facebook has taken its open-source PHP-boosting HipHop technology to the next level for programmers. With the new HipHop Virtual Machine, Facebook claims it has improved upon HipHop interpreter performance by 60 percent.
Facebook held a Tech Talk on Monday night explaining how it built a MySQL environment capable of handling everything the company needs in terms of scale, performance and availability. Based on what I heard, it looks like critics of Facebook’s MySQL environment might be wrong.
E-commerce site Etsy has grown to 25 million unique visitors and 1.1 billion page views per month, and it’s generating the data volumes to match. Using tools such as Hadoop and Splunk, Etsy is turning terabytes of data per day into a better product.
Over the past couple years, Facebok has released details on a number of its internal efforts to automate and simplify the management of its massive infrastructure. As reliance on web applications and cloud services become more common, Facebook’s tools and technologies could be a cash cow.
Last quarter we highlighted the fast maturation of the Platform-as-a-Service and big data spaces. Those two trends only picked up speed during the third quarter of 2011. Joining them on the cusp of IT greatness, though, are the OpenStack project and flash storage.
Traditionally, scientists and researchers develop the latest and greatest techniques in computing, which trickle down corporate data centers where they’re relevant. But with big data — the process of analyzing voluminous quantities of data in new, unique ways — it’s industry that’s driving the innovation ship.
Earlier this year, rumors swirled about whether Twitter had actually moved into a new Utah data center, or if it was forced to move its operations to a different facility. Now there are reports that Twitter is leasing more data center space, this time in Atlanta.
When you’re running a large web infrastructure, automation is critical to ensure that administrators aren’t spending their every waking seconds putting dealing with downed servers. Google, Yahoo and other pioneers had to figure out how to automate failover in their data centers. Now it’s Facebook’s turn.
MongoDB-based startup 10gen has raised $20 million in a Series D funding round. The latest round speaks to the popularity of the MongoDB document database among large companies, even though the hype around NoSQL has lessened considerably over the past year.
Cotendo is leveraging Equinix’s global data center footprint to give itself 30 points of presence, letting Cotendo focus on differentiating elsewhere. The companies released details of their partnership Thursday morning, including a quadrupling of Cotendo’s customer base to 400 from 100 in the past two years.
Google, which serves about 7 percent of the world’s overall web traffic, isn’t any ordinary company. Google Research Director Peter Norvig recently shared some of the considerations that Google takes into account when designing its infrastructure and systems to operate at Internet scale.
For anyone who didn’t know, Facebook is a huge Hadoop user, and it does some very cool things to stretch the open source big data platform to meet Facebook’s unique needs. Today, it detailed how it migrated its 30-petabyte cluster from one data center to another.
Two weeks ago, a post quoting Michael Stonebraker, who questioned the relevance of MySQL and Facebook’s use of it, sparked an overwhelmingly negative response. The true state of the database market appears to be that while SQL has its place, ideal uses are fading fast.
Nginx creator Igor Sysoev is planning a company based around the wildly popular open-source web server. Sysoev announced the decision on the Ngnix blog Monday morning, writing that the commercial entity’s primary goals will be better support and more consistent feature releases.
Google spent $917 million on infrastructure during the second quarter, continuing an upward trend that helps ensure new services like Google+ keep running. It’s the eight consecutive quarter of increased capital expenditures for Google, which is now spending at near-record levels.
Big processors or little processors, scale-up or scale-out, on-premise or in the cloud: the answers might not be as easy as one would think. Web-style, scale-out architectures, low-power server processors and cloud computing are getting more attention by the day, but they have their limits.
According to database pioneer Michael Stonebraker, Facebook is operating a huge, complex MySQL implementation equivalent to “a fate worse than death.” It’s actually a predicament all too common among web startups, for which the solution might be a class of databases referred to as NewSQL.
Sometimes, the geeks come out of the shadows and hit the mainstream consciousness. Remember the early ’00s and the rush of publicity for Ruby and Ajax as they became the calling cards of Web 2.0? Node.js looks like the next candidate for such mainstream elevation.
Among the biggest problems with developing applications for the cloud is scaling the database layer. GenieDB, a competitor in our recent Structure 2011 LaunchPad competition, wants to give its customers the benefits of both SQL and NoSQL to scale across data centers.
Hadoop is a very valuable tool, but it’s far from perfect. While Apache, Cloudera, EMC, MapR and Yahoo focus on core architectural issues, there is a group of vendors trying to make Hadoop a more-fulfilling experience by focusing on business-level concerns such as applications and utilization.
At Google’s I/O event last month, the company announced new features and a new pricing model for its App Engine PaaS offering, and now the web giant thinks it’s prepared to compete with companies like Red Hat and Salesforce.com in bringing enterprise users to its platform.
When Steve Jobs flashed inside images of Apple’s new cloud data center during his WWDC keynote on Monday, he ignited a mini firestorm of speculation about just kind hardware is filling its immense surface area. Everyone seems to agree that HP and Teradata were big winners.
Apple officially launched its much-hyped iCloud suite of services at its Worldwide Developer Conference today, and although the capabilities are sure to be the talk of the town, it’s Apple’s cloud infrastructure that makes it all work. Steve Jobs said as much during his WWDC keynote.
Facebook today published an interesting visualization of just how complex its codebase is. Actually, the visualization is part of an application within the company, but it gets the point across: Making code changes is no small feat when every module is dependent on so many others.
According to a post today on the Facebook Engineering blog, the social networking leader undertook an effort called “Project Triforce,” which involved provisioning a replica production region from an existing cluster, to ensure the site could run smoothly across three regions without falling on its face.
The most interesting part about yesterday’s announcement that Groupon is using the Cloudera Distribution of Hadoop wasn’t the actual use but, rather, the insight that Groupon is “building a world-class infrastructure” of which Hadoop will be a key part. But recruiting big-data-savvy talent is getting rather pricey.
Two things stand out above all else when looking back on the first quarter of 2011: Infrastructure as a service (IaaS), the epitome of cloud computing, came down to Earth, while big data took flight.
Nutanix startup that sells an appliance combining computing and storage on the same nodes, has raised $13.2 million. The company is developing an appliance combining computing and storage on the same server nodes, a story that should resonate with customers concerned with scalability and performance.
NoSQL startup DataStax officially entered the pantheon of Hadoop providers today, introducing its own distribution called “Brisk.” Brisk utilizes the open source NoSQL database Cassandra as a replacement for Apache’s Hadoop Distributed File System, as well as Cassandra’s built-in MapReduce engine and Hive.
Couchbase, the company formed by the merger of Membase and CouchOne last month, has released the first version of its Couchbase Server database, as well as a board of advisors that reads like a who’s who of web infrastructure and big data.
Is it possible that the ever-increasing demand for data center space could be the cure for vacant commercial real estate plaguing cities? In some areas, data center operators are buying up vacant real estate to house new operations and helping revitalize those areas as a result.
Lost in the wake of Membase and CouchOne merging to form Couchbase, and far away from Silicon Valley, Boston-based NoSQL startup Basho has raised $7.5 million for its efforts to commercialize the Riak NoSQL database, according to a report in Mass High Tech.
NoSQL database startups Membase and CouchOne have merged to create Couchbase, a company that will combine Membase’s memcached-based Membase Server and CouchOne’s CouchDB-based products into a family of NoSQL products. Other NoSQL vendors need to broaden their scope if they want to compete against Couchbase.
Myspace’s gradual decline and a recent blog post have me wondering what the flip side is of rapid scaling. I wonder what social-media sites do with their expansive infrastructures once they no longer need them to meet high demand. They can’t just scale them back, right?
Google blogged this morning about a new no-planned-downtime for Google Apps, a promise it’s able to make because of its globally distributed infrastructure estimated at more than 1 million servers. Google’s expansive infrastructure gives it multiple options for migrating workloads during planned downtime.
In the case of the following companies (and one open-source project) — ranging from Cisco to Twitter — I think that although they made lots of headlines in the past year, the true effects of their actions won’t be realized until later this year.
Depending on how Facebook intends to evolve, both performance considerations and data privacy laws might make additional infrastructure investment a good idea. Regardless of its rationales, however, the time to do so is now — before the company goes public and must answer for every dollar spent.
From ARM Holdings to Facebook to VMware, and whether via acquisitions, innovation or challenging the status quo, many vendors were able to effect paradigmatic shifts in computing or otherwise leave indelible marks on enterprise IT by what they did in 2010.
Today’s links focus on the importance of infrastructure in building reliable services. We have Tumblr investing in a new data center, KT building a cost-efficient cloud and Citrix’s Simon Crosby telling why private clouds could have helped prevent the Wikileaks debacle on all fronts.
In pretty much every discussion about Hadoop, the question arises of who’s using it outside of web companies. But to see what’s really going on with Hadoop, we should continue to look to the web and, increasingly, to Facebook in particular.(GigaOM Pro subscription required)
Scalable SQL startup Clustrix has closed a $12 million Series B round of funding, bringing its total to $30 million. The new money came from existing investors U.S. Venture Partners, Sequoia Capital and ATA Ventures. Considering Clustrix’s steady momentum, this funding shouldn’t take anybody by surprise.
Today’s news underscores my feeling that 2011 will be a huge year for cloud computing. Aside from CloudBees’ funding, we have Mellanox buying Voltaire, rumors of Oracle buying Salesforce.com, Enomaly continuing to push cloud brokerages, and questions about whether Intel’s Open Data Center Alliance can succeed.
Today’s links demonstrate that there’s a long way to go before we have issues like cloud computing and web infrastructure figured out, but also that we’re making progress: Twitter teaches lessons on scaling, Google runs test queries, and IBM Research is tackling cloud privacy.
Yahoo has open-sourced its S4 project for developing real-time MapReduce applications. As we’ve seen with Google’s new Caffeine infrastructure for its Instant Search features, there is a growing trend of unchaining large-scale data analysis from its batch-processing roots.
If the third quarter told us anything, it was that IT M&A activity is alive and well, particularly in the big data space. When a company can store or analyze large amounts of data with any degree of innovation, a larger vendor is likely to eye them up for an acquisition — even if it means paying premium. 3PAR, Netezza, Greenplum, Storwize, Ocarina Networks and ParaScale all found new homes during the past few months.
Om’s post about Google’s spending got me thinking about the hypocrisy in the way we assess web companies’ decisions to splurge on infrastructure. Startups are praised for spending on more infrastructure, while public companies feel the wrath of financial analysts when they do the same.
The web is becoming more dynamic, context-aware and personalized by the day, and the amount of information consumed by each person is increasing exponentially. But software infrastructure is not keeping pace. We need to develop data processing architectures that go beyond technologies like memcache, MapReduce, NoSQL.
Join us for breakfast on Dec. 3 as we bring together experts from the world of web infrastructure and ask them to outline best options for startups. We’ll discuss future trends of platforms and new options for startups as the concepts behind hosting evolve.
Supporters and opponents of the Federal Communications Commission’s proposed net neutrality rules achieved a rare moment of agreement Thursday. Speaking at a panel discussion organized by the Washington, D.C.-based think tank Arts + Labs, Public Knowledge Legal Director Harold Feld, a strong proponent of the regulation, acknowledged that complete neutrality toward all bits on a network can never be achieved and should not be policymakers’ goal. Where Feld parted company with opponents of regulation was on the question of how to decide which bits are more equal than others. That question has quickly emerged as the crux of the debate since the FCC made clear that its definition of net neutrality includes allowances for “reasonable network management.”
[qi:gigaom_icon_cloud-computing] Love it or fear it, there is no denying the impact cloud computing is having on IT practices. Despite a summer…
Gear6 today released Web Cache in an effort to commercialize the Internet’s predominant (de facto, for Linux) distributed caching protocol, memcached. Every…
What goes hand-in-hand with gaudy lights, fake snow, and endless holiday music? Shopping lists, of course. Here are a few items I’m…
It’s time to exhale: Our first Structure 08 conference, slated to be held on June 25th in San Francisco, is almost sold…