Hadoop startup Hortonworks has added support for Apache Kafka, in technical preview mode, to its Hortonworks Data Platform product. Kafka is a real-time messaging system originally developed by LinkedIn, but used by many web companies in order to move message quickly from web applications into the appropriate data services.
Viewed in light of Hortonworks’ push to integrate the Apache Storm stream-processing system with Hadoop, it seems pretty clear the company is doing what it can to cater to a certain class of engineering-savvy companies. That includes the ones that have already built advanced data architectures, but also the next generation of web startups, and even traditional enterprises, looking to design their data infrastructures after the big boys’ systems.
Adding support for Kafka is likely a small deal in the grand scheme of things, but it’s part of a bigger effort to further distinguish Hortonworks from blood rivals Cloudera and MapR. Those companies certainly package a lot of open source tooling as well, but their revenue also relies much more heavily on licenses for their proprietary software. Hortonworks seems happy to attract users that might want Storm, Kafka, Spark, Hadoop, etc., as part of a single open source package, hoping they’ll ultimately pay for support when their deployments grow too large (or too important) to handle alone.