All steps and links are for version 3.0 of confluent and macOS Sierra.
First thing we need is having confluent kafka services up and running. Download tar file under Zip and Tar archives section from here. Uncompress downloaded file and start/test services. All the steps to do so can be found in confluent quickstart guide. It is quite straightforward and no problems should rise here.
After trying to install confluent-kafka python libraries with pip install confluent-kafka I’ve seen it crashes in macOS with below error
fatal error: ‘librdkafka/rdkafka.h’ file not found
It seems is not finding librdkafka headers and the way I found to solve it is building and installing everything from scratch.
We will need librdkafka to build and install confluent-kafka python library so lets do that.
git clone https://github.com/edenhill/librdkafka
sudo make install
By default headers of librdkafka will be in /usr/local/include/librdkafka and libraries in /usr/local/lib. We will need these paths to install python libraries so make sure everything is there.
Next step is install confluent-kafka python library, lets get it from git and build it.
git clone https://github.com/confluentinc/confluent-kafka-python
We will need to set C_INCLUDE_PATH and LIBRARY_PATH environment variables to the path where librdkafka headers and libs are before building it.
Lets install it
python setup.py build
sudo python setup.py install
We are done, now confluent python library should be ready to be imported from our project.