Data streaming may sound dull and technical, like plumbing for the digital age. But as companies need to ingest and process more and more data from more and more sources to compete, the infrastructure necessary to send those firehoses to the right places is ever more crucial, and traditional solutions often became bottlenecks.In the late 2000s and early 2010s, LinkedIn was one of the first companies to start bumping up against this problem. Three of LinkedIn’s engineers—Jay Kreps (center), Neha Narkhede (right), and Jun Rao (left), who had worked on a team that among other things created the Who’s Viewed My Profile Feature—worked on a messaging system project that eventually became Kafka, named by Kreps after one of his favorite writers.Released as open source, Apache Kafka has become an important part of the plumbing for a vast array of companies and projects. Kreps, Narkhede, and Rao, meanwhile, moved on to found Confluent, a company that offers commercial and cloud-based editions of Kafka.
Source link
