GUI for Apache Kafka
Overview
Nowadays Apache Kafka becoming more and more popular in the Big Data world. More than 30% of Fortune 500 companies are already using Kafka. It’s a fast, durable and scalable publish-subscribe messaging system.
In this article I want to take a look at Kafka GUI, that can make a life of Big Data engineer easier.
“Kafka Tool” overview and installation.
First tool of my choice is “Kafka Tool”, sorry for tautology. It’s free for personal use and you can download it from http://www.kafkatool.com/
Also if you use Apache Avro message type it’s very useful to download Avro plugin, that will help to see actual contents of the Avro objects in a JSON format. https://github.com/laymain/kafka-tool-avro
Once you are done, open the “Kafka Tool” installation folder and put the downloaded .jar file to the {path-to-app}/java/app/plugins/.
Connecting to the Kafka cluster in “Kafka tool”.
Open “Kafka Tool” append and on the left side you will see folder “Clusters”. Right Click on it and choose “Add New Connection” from the drop-down list. It should open this window with cluster’s settings
You can name your cluster with any name and add a Bootstrap Server endpoint. In my case, I have Kafka Docker image running on my PC on port 9092. Click “Add” and now you are connected to your Kafka cluster.
Then specify your schema registry endpoint. By default it’s http://localhost:8081
Now in the “Clusters” folder newly created cluster should appear. And you can see a folder “Topics” inside. If you’ll click on it, you’ll see all the topics that exist on your cluster.
Consuming from the topics in “Kafka tool”.
Click on the topic you’re interested in and then click on the play button on the left.
In the tab “Properties” you can choose Key and message content type. If you added Avro plugin, as was described above you’ll see Avro type as well.
Choose the type used for your messages and go to the “Data” tab.
Now you can see all messages that are in the selected topic.
Creating a new topic in “Kafka tool”.
You can create a new topic as well. Just click on the folder “Topics” and then on the green positive sign.
Choose the name, partition count and replica count for your topic. If you need additional configurations — click on the left “Add” button and enter your configuration name and value.
Once you are done click on the right “Add” button and the topic will be created and will appear in your “Topics” folder.
“Conduktor” overview and installation.
“Conduktor” is the second Kafka GUI on my list. You can download it here https://www.conduktor.io
It’s free for now and you don’t need to install anything for schema registry since it’s already built-in.
Let’s open it.
Adding of new Kafka cluster in “Conduktor”.
Click on “Add new cluster”.
Type any cluster name and you Bootstrap Servers endpoints. Then go to “Schema Registry” tab and put in Schema Registry’s URL. It always should start from “http” or “https”
Save your configuration and the newly added cluster should appear on the left. Click on it and you should the window with all the information about your Kafka cluster: brokers, topics, consumers, etc.
Consuming from the topics in “Conduktor”
Click on the “Topics tab” and you will see all topics that exist on your cluster.
If you want to see the content of the specific topic click on it. In a new window, you can choose your topic’s key and value format and start consuming from it.
Now you can see all your records and their number. If you want to see a specific message just click on it.
In the opened window you will see all the needed information about your message: name of the topic, partition, offset, key and value of the message, timestamp, and headers if they are present.
Creating a new topic in “Conduktor”.
For creating a new topic in the “Conduktor”. Press the “+CREATE” button on the right.
Type in the name of the topic, number of partitions, replication factor. Also, you can override other values if needed. Tap on “Create” and newly added topic should appear in your “Topics” list.
Conclusion.
Both of reviewed tools are free for personal use and runs on all OS. You can connect to your Kafka cluster, see your existing topic. Also, you can see a number of records in each topic, record’s offset, timestamp key, value, and headers. Also, you can create and configure a new topic.
From Kafka tool advantages, you can write your own plugin for it, which can be pretty useful in some cases.
From “Conduktor” advantages, just in my personal opinion, it is more user-friendly and its design is nicer.