The delay occurs because PHP reinitializes a KafkaProducer object on every send. Each initialization requires connecting to each broker and fetching cluster metadata, which adds over 100 ms in a Virtual Private Cloud (VPC) and over 500 ms over the Internet.
Why this happens
Kafka producers must identify the broker that leads each partition before sending a message. This requires a metadata request that maps topics to partitions and partitions to broker addresses. The producer then establishes connections to the relevant brokers.
In most languages, a KafkaProducer is created once and reused across requests. A Java client, for example, initializes the producer at startup, keeps the broker connections open, and caches metadata. Subsequent sends add minimal latency.
PHP typically runs in a share-nothing, request-per-process model. Unless you explicitly manage the producer lifecycle, each request creates a new KafkaProducer and repeats the full initialization sequence: metadata fetch, broker connections, and partition leader discovery. This repeated initialization is the primary source of the delay.
Expected latency
| Network | Per-initialization overhead |
|---|---|
| VPC | > 100 ms |
| Internet | > 500 ms |
Once the producer is initialized, subsequent sends add minimal latency. Reusing the producer eliminates the per-message initialization overhead.