Vipul Sharma, director of data engineering for Eventbrite, spoke at GigaOM’s Structure:Data conference in New York on Thursday about the data challenges facing his company, which fields customers and data from across the globe. By June 2012 the company had crossed $1 billion in ticket sales, and Sharma said Eventbrite has hosted more than a million events so far.

Sharma,who has worked as an engineer since 2010 after leaving Digg, where he worked on data mining and machine learning, said one of the biggest data challenges Eventbrite deals with is the speed of the flow of information:

“So basically what I’m trying to tell you is that the challenge in solving realtime is two-fold. One is the data flow, or how do I transfer the data I have in realtime to my downstream applications,” he said. “And the second part is processing, or how do I process this significant amount of data in realtime. To process in realtime you really need a distributed processing system.”

Check out the rest of our Structure:Data 2013 live coverage here, and a video embed of the session follows below:

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Big data budgets on the rise
• Why the next front in big data might be psychological
• What’s driving the next phase of the e-commerce evolution

(c) 2012 Pinar Ozger. pinar@pinarozger.com

Open source solutions like the Hadoop distributed file system and HBase NoSQL database are justifiably the hot platforms in data warehousing, but MySQL and other more traditional enterprise data warehousing may still be optimal if a company has a good understanding of where it needs to scale and has big focus on security and reporting.

Hadoop’s open source community still hasn’t worked out the security kinks, and building the dashboards around reporting tools in Hadoop and Hbase is quite difficult, Sharma said. Security and reporting in enterprise data warehouse platforms are much better developed, he said.

Sharma also cautioned companies that they must be willing to commit manpower to support their Hadoop clusters, otherwise they’re better off using a traditional warehouse solution.

“Since [Hadoop and Hbase] are very new, the entire code base has to mature a lot,” Sharma said. “So you end up having to do a lot of debugging yourself. You end up getting into the code base to understand what’s going on. … If you’re not willing to invest in a team around it, then probably Hadoop is not the best for you because the ecosystem is still maturing.”

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud computing infrastructure: 2012 and beyond
• A near-term outlook for big data
• Infrastructure Q4: Big data gets bigger and SaaS startups shine