This tutorial will provide you with instructions to configure and enforce quotas for produce and fetch requests in Apache Kafka.

Abstract

Apache Kafka is a distributed and highly scalable messaging system capable of handing millions of messages per second.

However, it is still possible for some producers and consumers to produce/consume huge volume of data and hence monopolize broker resources and cause network saturation. These producers/consumers can result into DOS (Denial of Service) for other producers/consumers and thus impacting other applications. This problem gets more severe in case of large multi-tenant clusters where a small set of bad clients can degrade user experience for the well behaved ones.

This problem can be addressed by having quotas to limit broker resources and network bandwidth to producers and consumers. Infact, when running Kafka as a service, Quotas become even more useful to enforce API limits according to an agreed upon contract.

How Do Quotas work?

Quotas in Apache Kafka were introduced in 0.9 version. Quotas are enforced at the Kafka broker level where each unique client id receives a fixed quota in bytes/second. This means each unique client can publish/fetch a maximum of X bytes/second per broker before it gets throttled.

Client byte rate is measured over multiple small windows (e.g. 30 windows of 1 second each) in order to detect and correct quota violations quickly.

It is important to note that Kafka brokers do not throw any error when it detects a quota violation. Instead it attempts to slow down a client exceeding quota by delaying the response. Amount of delay is computed by brokers in a way that client does not violate its quota.

Since, Kafka brokers do not throw any error on quota violation, there is no need of retry mechanism at client side. However, when looking for performance issues, quotas should be kept in mind and changed to get required throughput. 

Configuring Parameters for Enforcing Quotas

Quotas are enforced at broker level and by default, there is a fixed quota assigned to each unique client. These default quotas can be changed in Kafka configuration file and require Kafka broker restart. For example, below configuration parameters can be used to set producer/consumer quota to 50 MB/sec -

# Sets producer quota to 50 MB
quota.producer.default=52428800

# Sets consumer quota to 50 MB
quota.consumer.default=52428800

However, we may need to configure different quota for different clients at run time without any restart of Kafka brokers. Fortunately, Kafka provides us with a mechanism to override quotas at client level without any restart. For example, below commands can be executed from Kafka broker home directory to configure client with id "test-client" with producer quota as 10 MB and consumer quota as 20 MB -

# Adds configuration for client with id test-client
./bin/kafka-configs.sh  --zookeeper localhost:2181 --alter --add-config 'producer_byte_rate=10485760,consumer_byte_rate=20971520' --entity-name test-client --entity-type clients

Once updated, you can also verify your configurations using below describe command -

# Describe configurations
./bin/kafka-configs.sh  --zookeeper localhost:2181 --describe --entity-name test-client --entity-type clients

#Output of above command will be as below
Configs for clients:test-client are producer_byte_rate=10485760,consumer_byte_rate=20971520

Thank you for reading through the tutorial. In case of any feedback/questions/concerns, you can communicate same to us through your comments and we shall get back to you as soon as possible.