Support Kafka 2.0

[twm]

This issue tracks changes to Afkak required to support Kafka 2.0.0. See the upgrade guide:

KIP-283 Message Down-Conversion requires client changes

KIP-283 improves message down-conversion handling on Kafka broker, which has typically been a memory-intensive operation. The KIP adds a mechanism by which the operation becomes less memory intensive by down-converting chunks of partition data at a time which helps put an upper bound on memory consumption. With this improvement, there is a change in FetchResponse protocol behavior where the broker could send an oversized message batch towards the end of the response with an invalid offset. Such oversized messages must be ignored by consumer clients, as is done by KafkaConsumer.

KIP-283 also adds new topic and broker configurations message.downconversion.enable and log.message.downconversion.enable respectively to control whether down-conversion is enabled. When disabled, broker does not perform any down-conversion and instead sends an UNSUPPORTED_VERSION error to the client.

New Protocol Versions

TODO evaluate the following changes:

• KIP-279: OffsetsForLeaderEpochResponse v1 introduces a partition-level leader_epoch field.
• KIP-219: Bump up the protocol versions of non-cluster action requests and responses that are throttled on quota violation.
• KIP-290: Bump up the protocol versions ACL create, describe and delete requests and responses.