This topic provides the systematic instructions to setup the Zookeeper.

1. Extract the kafka installation files in /tools/kafka on both the servers.
2. Navigate to config folder in /tools/kafka/conf.
3. Duplicate the zookeeper.properties and rename it to zookeeper1.properties.
4. Open zookeeper1.properties and modify the following properties.

   DataDir= <kafka home directory>/data
   tickTime=2000
   clientPort= Zookeeper client Port value (2181)
   initLimit=10
   syncLimit=5
   
   server.1=<hostname>:<peer port>:<leader port>
   #1 is the id that we put in myid file.
   
   server.2=<hostname>:<peer port>:<leader port>
   #2 is the id that we will put in myid file of second node.
   
   server.3=<hostname>:<peer port>:<leader port>
   #3 is the id that we will put in myid file of third.

   Example:

   tickTime=2000

   initLimit=5

   syncLimit=2

   clientPort=2181

   dataDir=/tmp/zookeeper-oblm/zookeeper-node1

   server.1=server1-IP:2666:3666

   server.2=server2-IP:2667:3667

   server.3=server3-IP:2668:3668

   Note:

   Update the IP value with the respective server IP.
5. Duplicate the zookeeper.properties file and rename it as zookeeper2.properties and zookeeper3.properties in the same directory on Server 2 and Server 3 respectively, (other names can also be used). These configuration files are used for each of the zookeeper nodes.
6. Open zookeeper2.properties and modify the following properties.

   clientPort=2182

   dataDir=/tmp/zookeeper-oblm/zookeeper-node2

   server.1=server1-IP:2666:3666

   server.2=server2-IP:2667:3667

   server.3=server3-IP:2668:3668

   Note:

   Update the IP value with the respective server IP.
7. Open zookeeper3.properties and modify the following properties.

   clientPort=2183

   dataDir=/tmp/zookeeper-oblm/zookeeper-node3

   server.1=server1-IP:2666:3666

   server.2=server2-IP:2667:3667

   server.3=server3-IP:2668:3668

   Note:

   Update the IP value with the respective server IP.
8. Copy the zookeeper1.properties, zookeeper2.properties and zookeeper3.properties and paste it in the local.
9. Open the directory /tmp/zookeeper-oblm/zookeeper-node1 on server 1 and create a file named myid, open with text editor and write 1, save and close.
10. Open the directory /tmp/zookeeper-oblm/zookeeper-node2 on server 2 and create a file named myid, open with text editor and write 2, save and close.
11. Open the directory /tmp/zookeeper-oblm/zookeeper-node3 on server 3 and create a file named myid, open with text editor and write 3, save and close.
12. Run the command to start the zookeeper nodes.
    On Server 1:

    ./bin/zookeeper-server-start.sh ./config/zookeeper1.properties

    On Server 2:

    ./bin/zookeeper-server-start.sh ./config/zookeeper2.properties

    On Server 3:

    ./bin/zookeeper-server-start.sh ./config/zookeeper3.properties

This topic provides the systematic instruction to setup kafka.

1. Extract the kafka installation file in /tools/kafka on both the servers.
2. Navigate to config folder in Apache Kafka (/tools/kafka/config).
3. Duplicate the server.properties from config folder and rename it to server1.properties.
4. Open server1.properties and modify the following properties.

   broker.id= (Unique Integer which identifies the kafka broker in the cluster.
   listeners=PLAINTEXT://<hostname>:<Kafka broker listen port(9092)>
   log.dirs=<Kafka home directory>/logs
   log.retention.hours= <The number of hours to keep a log file 
   before deleting it (in hours), tertiary to log.retention.ms property>
   log.retention.bytes= <The maximum size of the log before deleting it>
   log.segement.bytes= <The maximum size of a single log file>
   log.retention.check.interval.ms= <The frequency in milliseconds that
   the log cleaner checks whether any log is eligible for deletion>
   zookeeper.connect=<zookeeper_hostname_1>:<zookeeper_client_port>,
   <zookeeper_hostname_2>:<zookeeper_client_port>,<zookeeper_hostname_3>:<zookeeper_client_port>

   Example:

   broker.id=0

   port=9092

   log.dirs=/tmp/kafka-oblm/logs-node1

   zookeeper.connect=server1-IP:2181,server2-IP:2182,
server3-IP:2183

   num.partitions=1

   min.insync.replicas=2

   default.replication.factor=3

   offsets.topic.min.insync.replicas=2

   offsets.topic.replication.factor=3

   transaction.state.log.replication.factor=3

   transaction.state.log.min.isr=2

   Note:

   If the Apache Zookeeper is on different server, then change the zookeeper.connect property. i.e., update the highlighted value for the respective server IPs. min.insync.replicas: A typical configuration is replication-factor minus 1.
5. Duplicate the server.properties into the same directory and rename it to server2.properties on server 2 and server3.properties on server 3.
6. Open server2.properties and modify the following properties.

   broker.id=1

   log.dirs=/tmp/kafka-oblm/logs-node2

   listeners=PLAINTEXT://<hostname>:<Kafka broker listen port(9093)>

   Note:

   By default, Apache Kafka will run on port 9092 and Apache Zookeeper will run on port 2181.
7. Open server3.properties and modify the following properties.

   broker.id=2

   log.dirs=/tmp/kafka-oblm/logs-node3

   listeners=PLAINTEXT://<hostname>:<Kafka broker listen port(9094)>

   Note:

   By default, Apache Kafka will run on port 9092 and Apache Zookeeper will run on port 2181.
8. Copy the server1.properties, server2.properties, server3.properties and paste it in local.
9. To run Kafka brokers, change path to /tools/kafka directory and run the following command in separate terminals.
   On Server 1:

   nohup ./bin/kafka-server-start.sh config/server1.properties

   On Server 2:

   nohup ./bin/kafka-server-start.sh config/server2.properties

   On Server 3:

   nohup ./bin/kafka-server-start.sh config/server3.properties

10. The values set for Logs is under the segment: “Log Retention Policy” in server*.properties file attached in the document. The values set under this segment are defaults from Apache.
11. At present, kafka takes the default value for message size as: message.max.bytes=1000012.
12. Add and update this field in server*.properties for increasing based on requirement.
13. To add compression type for all data generated by the producer, add the following property in server*.properties file.
    compression.type=none

    Note:

    The default is none (i.e. no compression). Valid values are none, gzip, snappy, lz4, or zstd.