undefined symbol: rd_kafka_msg_partitioner_consistent

[rendybjunior]

hi all, in my machine I tried to use conlfuent kafka python like this

first, I installed confluent kafka python
sudo pip install confluent-kafka

then I tried to use it. somehow I always get this error message
>>> from confluent_kafka import Producer Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/confluent_kafka/__init__.py", line 2, in <module> from .cimpl import * ImportError: /usr/local/lib/python2.7/dist-packages/confluent_kafka/cimpl.so: undefined symbol: rd_kafka_msg_partitioner_consistent

I guess it is because of librdkafka is not there, then I do git clone and make & install. but the error still appears

I also tried to git clone confluent-kafka-python instead of installing from apt-get, but it is still error

how to solve this problem?

[edenhill]

Hi,

I think your version of librdkafka is too old.
You will need librdkafka version 0.9.1 or later.

[rendybjunior]

hi @edenhill I have done git clone and make & install
is it somehow not detected by kafka-confluent?

[edenhill]

If you have an earlier version installed in /usr the one you (most likely) installed now resides in /usr/local and will not be picked up automatically.
Verify with: ldconfig -p | grep librdkafka, it should list the newly installed library first.

[edenhill]

To remove the one from deb package, do: sudo apt-get purge librdkafka1 librdkafka-dev

[rendybjunior]

wow great, so its taking the wrong package ya. it works now. thanks a lot!