Kafka Configuration

Kafka Configuration Properties describes the most relevant Kafka-related configuration properties available for the ASN.1 sFTP Sink application.

Table 1. Kafka Configuration Properties
Property	Description
kafka.configuration.bootstrap.servers	The addresses of the Kafka brokers. The default value is `kafka:9092`.
kafka.configuration.group.id	A unique identifier for consumer group. The default value is `sinkgroup1`.
kafka.configuration.enable.auto.commit	If set to `true`, the application acknowledges receipt of records before writing to disk instead of after. The default value is `false`. By default records are only acknowledged after having been written to disk.
kafka.configuration.auto.offset.reset	Specifies whether the application reads from the earliest available records or the latest after connecting to Kafka for the first time. Valid values are `earliest` and `latest`. The default value is `earliest`. The default behavior is to read all available records. Previously, it was left as the Apache default of `latest` which made it possible that records might not be processed.
kafka.configuration.isolation.level	Controls how to read messages written transactionally. The default value is `read_committed`.
kafka.configuration.max.poll.interval.ms	The maximum delay between invocations of `poll()` when using consumer group management. The default value is `300000`.
kafka.configuration.fetch.max.bytes	The maximum number of bytes returned for a fetch request. The default value is `52428800`.
kafka.configuration.fetch.max.wait.ms	The maximum amount of time the server blocks before answering the fetch request. The default value is 500.
kafka.configuration.fetch.min.bytes	The minimum amount of data the server returns for a fetch request. The default value is `1`.
kafka.configuration.max.partition.fetch.bytes	The maximum amount of data per-partition the server returns. The default value is `10485760`.
kafka.configuration.max.poll.records	The maximum number of records returned in a single call to `poll()`. The default value is `5000`.
kafka.configuration.receive.buffer.bytes	The size of the TCP receive buffer (SO_RCVBUF) to use when reading data. The default value is `1048576`.
kafka.configuration.reconnect.backoff.max.ms	The maximum amount of time in milliseconds to wait when reconnecting to a broker that has repeatedly failed to connect. The default value is 1000.
kafka.configuration.reconnect.backoff.ms	The amount of time to wait before attempting to reconnect to a given host. The default value is `50`.
kafka.configuration.request.timeout.ms	The configuration controls the maximum amount of time the client will wait for the response of a request. The default value is `30000`.
kafka.configuration.retry.backoff.ms	The amount of time to wait before attempting to retry a failed request to a given topic partition. The default value is `100`.
kafka.configuration.send.buffer.bytes	The size of the TCP send buffer (SO_SNDBUF) to use when sending data. The default value is `1048576`.
kafka.configuration.session.timeout.ms	The timeout used to detect client failures when using Kafka's group management facility. The default value is `45000`.

The application uses a default consumer group of sinkgroup1. This identifies it to the Kafka brokers which keep track of which records have already been seen. If the consumer group identifier changes, then all available records are received again by the application. When upgrading, it is important to either keep the default consumer group ID or update it to use the ID used by earlier versions.