Question

Define consumer lag and ways to monitor it in Kafka

Answer 1

 Kafka follows a publisher-subscriber model in which the producer writes to a particular topic, and one or more consumers read from the concerned topic. On the other hand, the reads in Kafka tend to lag behind the writes due to delay between the moment of writing a message and the moment of consuming a message.

Therefore, Consumer Lag is the delay between the Latest Offset and the Consumer Offset. One of the unique tools for monitoring consumer lag is LinkedIn Burrow. In addition, the Confluent Kafka provides innovative tools for measuring Consumer Lag.