Kafka is publish-subscribe messaging rethought as a distributed commit log, designed to allow one cluster to serve as unique data backbone.
Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.
Installs Kafka v0.9.0.1 on Oracle Java8 together with Zookeeper V 3.4.6.
The Kafka service is automatically started following install. You can use the zookeeper client to test the connection, or telnet to port 2181.
To start and stop the service
- sudo service kafka start - sudo service kafka stop
- The data directory is set to - /var/kafka - The log files can be found here - /var/log/kafka - Conf location - /etc/kafka
The following are a couple of links we found useful.