Łukasz Chrząszcz

Digging deep in tech so you don't have to

How to assign partitions to your consumers?

Introduction to Kafka assignors

Integrating with Kafka usually starts with subscribing your consumers to Kafka with default configuration and it just works. However, after working a while with Kafka you will encounter various "weird" behaviors originating from rebalancing settings such as chosen assignor implementation. Let's see how different assignors work and what are their pros and cons.

How to test the application's integration with Kafka?

Example of testing application's integration with Kafka for both sunny-day scenarios and failure cases.

After introducing Kafka to your application you want to test it properly. For sure you want to write some integration tests with real Kafka working underneath. What is more, you probably want to test not only sunny-day scenarios but failure cases as well. How to achieve that? Let us discover how Testcontainers and Toxiproxy fit in with Kafka in your application's integration tests!

How do transactions work in Apache Kafka?

Happy path of an application performing "consume, transform, produce" in a single transaction.

Do you need an exactly-once guarantee in Apache Kafka? Transactions enable you to achieve it, but how does it work under-the-hood? Let's skim through the happy path in the code and find out.

How to count events in Kafka Streams?

Stores and aggregation in Kafka Streams

Kafka Streams enables you to use stateful processing via Stores. Let's go through a simple example of order-counting app and discover how Stores fit in the stream processing world

How to use Kafka Streams to process events from one topic to another?

Introduction to Kafka Streams

Kafka Streams is a high-level client library for interacting with Kafka. How does it help to process events from topic to topic? What are the capabilities of this library? Join me and let's find out

What happens when a new consumer joins the group in Kafka?

Rebalancing in Apache Kafka

Ever wondered how Kafka manages rebalancing? How are partitions assigned? Let's find out!

What does the heartbeat thread do in Kafka Consumer?

Behavior of Kafka consumer's background thread

We will dig into the heartbeat thread code to discover what it does and how it communicates with the rest of the consumer.

What happens when you call poll on Kafka Consumer?

Internals of Kafka consumer initialization and first fetch

We will create a new consumer, subscribe to a topic and execute first poll, describing what happens step by step.

Kafka 101

Introduction to Apache Kafka

Let's explore Kafka basics. We will discover what it does, what problem does it solve, and we will explore simple message passing from producer to consumer.