How to Diagnose Unexpected Errors When Adding a New Kafka Consumer Group

Operation Instructions

When starting a consumer group, an exception occurs.

Exception Log

Unexpected error from SyncGroup: The server experienced an unexpected error when processing the request.

Analysis Process

The error originates in the SyncGroup request, which is triggered when a new consumer joins a group and synchronizes. The message indicates an unexpected error.

The stack trace points to SyncGroupResponseHandler#handle and the SyncGroupRequest path.

Checking the Kafka broker logs reveals several entries such as:

[2022-10-19 15:14:01,224] ERROR [GroupMetadataManager brokerId=1] Appending metadata message for group group_consumer_4 generation 6 failed due to org.apache.kafka.common.errors.RecordTooLargeException, returning UNKNOWN error code to the client (kafka.coordinator.group.GroupMetadataManager)
[2022-10-19 15:14:01,224] WARN [GroupCoordinator 1]: Failed to write empty metadata for group group_consumer_4: The server experienced an unexpected error when processing the request. (kafka.coordinator.group.GroupCoordinator)

The logs show that during the SyncGroup request the broker tries to write group metadata into the __consumer_offsets topic. The write fails because the record size exceeds the broker limit, raising RecordTooLargeException.

The relevant configuration is message.max.bytes. In the author's test it was set to 20, far smaller than a typical message size.

Solution

Increase message.max.bytes to a value large enough to accommodate the metadata record.

The article presents this as one possible troubleshooting path; other exceptions that surface with the same message can be investigated using the same steps.