The Kafka service is supported in E-MapReduce (EMR) V3.4.0 and later.

Create a Kafka cluster

In the EMR console, create a Kafka cluster. For more information, see Create a cluster.

Kafka clusters with local disks

For information about instances that use local disks, see Supported instances with local disks.

When you deploy the Kafka service on instances that use local disks, you must configure the parameters described in the following table on the Configure tab of the Kafka service in the EMR console.
Parameter Description
default.replication.factor The value is fixed to 3, which indicates that each topic has three replicas.
min.insync.replicas The value is fixed to 2, which indicates that the number of replicas is greater than or equal to 2.

A write is successful only if the producer sets the request.required.acks parameter to all or -1 and the number of replicas that acknowledge the write is greater than or equal to 2.

Parameters

You can view the configurations of the Kafka service on the Configure tab of the Kafka service in the EMR console.
Parameter Description
zookeeper.connect Specifies the hostname and port of the ZooKeeper server that is connected to the Kafka cluster.
kafka.heap.opts The heap memory size of the Kafka broker.
num.io.threads The number of I/O threads of the Kafka broker. The default value is twice the number of CPU cores of the master node.
num.network.threads The number of network threads of the Kafka broker. The default value is the number of CPU cores of the master node.