11.2.2 Kafka Producer/Consumer Configurations
This topic provides information on Kafka Producer/Consumer Configurations.
OBDX offers default settings for Kafka, including Broker configurations,
partitioning, replication, and consumer group size, which the system stores in the
DIGX_CFG_CONFIG_ALL_B table. These default
configurations apply to all producers and consumers.
1. Generic Configurations
Below mentioned are the generic configurations (PROP_ID) and
their default values (PROP_VALUE). These can be overridden if required, by
updating them in the table.
For more information on fields, refer to the field description table.
Table 11-3 (PROP_ID) and their default values
| PROP_ID | Default PROP_VALUE | Description |
|---|---|---|
| KAFKA_CONFIG.bootstrap.servers | localhost:8080 | Specifies the Kafka broker(s) that consumers and producers should connect to. |
| KAFKA_CONFIG.enable.auto.commit | true | Determines whether the consumer's offset is automatically committed. |
| KAFKA_CONFIG.auto.commit.interval.ms | 5000 | The frequency (in milliseconds) at which the consumer commits offsets when auto-commit is enabled. |
| KAFKA_CONFIG.auto.offset.reset | latest |
Defines the behavior when a consumer starts reading from a topic. Options: earliest: Read from the beginning of the log. latest: Read only new messages. |
| KAFKA_CONFIG.CONSUMER_POLL_TIMEOUT_MS | 2000 | The maximum time (in milliseconds) a consumer waits for records when polling from Kafka. |
Apart from the above mentioned properties, any other producer and consumer
configuration provided by Kafka can also be overridden by adding the respective entry in
the table DIGX_CFG_CONFIG_ALL_B.
2. Changing Topic Level Configurations
Bank can also override any producer and consumer configuration for a
particular topic with their custom values instead of the default ones, by adding an
entry in the table DIGX_CFG_CONFIG_ALL_B in the
column PROP_ID with the pattern
KAFKA_CONFIG.TOPIC_NAME@CONFIGURATION.
For example,
INSERT INTO DIGX_CFG_CONFIG_ALL_B (PROP_ID, PROFILE, PROP_VALUE, ENTITY_SPECIFIC, EDITABLE, MANDATORY_OVERRIDE, PROPERTY_GROUP, CREATED_BY, CREATION_DATE, LAST_UPDATED_BY, LAST_UPDATED_DATE, OBJECT_VERSION_NUMBER, MODULE, SEQUENCE, VALIDATION, IS_ENUMERATED) VALUES ('KAFKA_CONFIG.vam.virtualaccountnotification@enable.auto.commit', 'DEV', 'false', 'N', 'Y', 'N', 'Kafka', 'SYSTEM', sysdate, 'SYS', sysdate, 1, 'infra', '-1', '.*', 'N');Parent topic: Kafka