Hadoop and big data platforms were originally known for scale, not speed. But the arrival of high performance compute engines like Spark and streaming engines have cleared the way for bringing batch ...

A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...

Imagine it’s 3 a.m. and your pager goes off. A downstream service is failing, and after an hour of debugging you trace the issue to a tiny, undocumented schema change made by an upstream team. The fix ...

IBM's $11B Confluent acquisition completes its hybrid cloud stack, with Kafka streaming joining Red Hat and HashiCorp for ...

What are the challenges you faced in building a fully managed cloud-native Kafka services? How does Confluent engineering solve these things? Will LaForest: When we began Confluent’s journey to create ...

The culprit is misconfigured Kafdrop interfaces, used for centralized management of the open-source platform. Some of the world’s largest companies have exposed reams of sensitive information from the ...

Streaming is hot. The demand for real-time data processing is rising, and streaming vendors are proliferating and competing. Apache Kafka is a key component in many data pipeline architectures, mostly ...

Event-driven architectures are wonderful. But Kafka was never intended to be a database, and using it as a database won’t solve your problem. It’s a tale as old as time. An enterprise is struggling ...

Founded out of Helsinki in 2016, Aiven essentially solves many of the headaches involved in running open source software, which permeates just about every modern technology stack. From scripts that ...

When Confluent launched a cloud service in 2017, it was trying to reduce some of the complexity related to running a Kafka streaming data application. Today, it introduced a free tier to that cloud ...