Skip to content

Consumer error when connecting with SASL_PLAINTEXT #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
4 of 6 tasks
aravinda92 opened this issue Nov 19, 2020 · 2 comments
Closed
4 of 6 tasks

Consumer error when connecting with SASL_PLAINTEXT #991

aravinda92 opened this issue Nov 19, 2020 · 2 comments

Comments

@aravinda92
Copy link

Description

Hi,
I'm using a dockerized kafka consumer to connect to KAFKA Broker. Can you check below log and let me know whats the problem ?

test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.195|SASLREFRESH|rdkafka#consumer-1| [thrd:app]: Kerberos ticket refreshed in 129ms
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.195|BROKER|rdkafka#consumer-1| [thrd:app]: GroupCoordinator: Added new broker with NodeId -1
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BROKER|rdkafka#consumer-1| [thrd:app]: sasl_plaintext://10.48.148.44:9092/bootstrap: Added new broker with NodeId -1
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd:GroupCoordinator]: GroupCoordinator: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd::0/internal]: :0/internal: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|BRKMAIN|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Enter main broker thread
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.196|INIT|rdkafka#consumer-1| [thrd:app]: librdkafka v1.4.0 (0x10400ff) rdkafka#consumer-1 initialized (builtin.features snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer, GCC GXX INSTALL GNULD LDS C11THREADS LIBDL PLUGINS SSL SASL_CYRUS HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER CRC32C_HW, debug 0x202)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.197|CONNECT|rdkafka#consumer-1| [thrd:main]: sasl_plaintext://10.48.148.44:9092/bootstrap: Selected for cluster connection: coordinator query (broker has 0 connection attempt(s))
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Received CONNECT op

test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state INIT -> TRY_CONNECT
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: broker in state TRY_CONNECT connecting
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.200|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state TRY_CONNECT -> CONNECT
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.201|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connecting to ipv4#10.48.148.44:9092 (sasl_plaintext) with socket 10
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|CONNECT|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connected to ipv4#10.48.148.44:9092
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|CONNECTED|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Connected (#1)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|FEATURE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Updated enabled protocol features +ApiVersion to ApiVersion
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.212|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state CONNECT -> APIVERSION_QUERY
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.224|PROTOERR|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Protocol parse failure for ApiVersion v3 at 3/6 (rd_kafka_handle_ApiVersion:1911) (incorrect broker.version.fallback?)
test_kafka-consumer.1.ow47pujnt7r
u@testdev1 | %7|1605786768.224|PROTOERR|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: ApiArrayCnt -1 out of range
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.224|APIVERSION|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: ApiVersionRequest v3 failed due to UNSUPPORTED_VERSION: retrying with v0
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|FEATURE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Updated enabled protocol features to MsgVer1,ApiVersion,BrokerBalancedConsumer,ThrottleTime,Sasl,SaslHandshake,BrokerGroupCoordinator,LZ4,OffsetTime,MsgVer2,IdempotentProducer
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|AUTH|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Auth in state APIVERSION_QUERY (handshake supported)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.231|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state APIVERSION_QUERY -> AUTH_HANDSHAKE
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|SASLMECHS|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker supported SASL mechanisms: GSSAPI
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|AUTH|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Auth in state AUTH_HANDSHAKE (handshake supported)
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|STATE|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Broker changed state AUTH_HANDSHAKE -> AUTH_LEGACY
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.232|SASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: Initializing SASL client: service name kafka, hostname 10.48.148.44, mechanisms GSSAPI, provider Cyrus
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %7|1605786768.234|SASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: My supported SASL mechanisms: DIGEST-MD5 EXTERNAL NTLM CRAM-MD5 LOGIN PLAIN ANONYMOUS
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %2|1605786768.234|LIBSASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: No worthy mechs found

How to reproduce

Dockerfile

FROM python:3.6-slim
ENV DEBIAN_FRONTEND=noninteractive
RUN  apt-get update && apt-get install -y apt-utils  && apt-get -yqq install  libpq-dev gcc ca-certificates git build-essential libssl-dev libsasl2-dev libsasl2-modules krb5-user

WORKDIR /test
RUN git clone https://github.com/edenhill/librdkafka.git
WORKDIR librdkafka
RUN  git checkout tags/v1.4.0 && ./configure && make && make install && ldconfig

WORKDIR /test
RUN git clone https://github.com/confluentinc/confluent-kafka-python.git
WORKDIR /test/confluent-kafka-python/
RUN  git checkout tags/v1.4.0
RUN python setup.py build
RUN python setup.py install

Checklist

Please provide the following information:

  • confluent-kafka-python and librdkafka version (confluent_kafka.version() and confluent_kafka.libversion()): v1.4.0 (tried with v1.5.2). you can check the version in Dockerfile too
  • Apache Kafka broker version:
  • Client configuration:
    { 'bootstrap.servers': "broker_ip_port", 'group.id': "group_id", 'debug': 'broker,security', 'api.version.request': False, 'sasl.mechanism': 'GSSAPI', 'sasl.kerberos.service.name': 'kafka', 'security.protocol': "SASL_PLAINTEXT, 'sasl.kerberos.keytab': 'path/to/keytab', 'sasl.kerberos.principal': 'ketab_user@realm' }
  • Operating system: Linux (Using docker image)
  • Provide client logs (with 'debug': '..' as necessary)
  • [] Provide broker log excerpts
  • Critical issue
@edenhill
Copy link
Contributor 

[thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: My supported SASL mechanisms: DIGEST-MD5 EXTERNAL NTLM CRAM-MD5 LOGIN PLAIN ANONYMOUS
test_kafka-consumer.1.ow47pujnt7ru@testdev1 | %2|1605786768.234|LIBSASL|rdkafka#consumer-1| [thrd:sasl_plaintext://10.48.148.44:9092/bootstrap]: sasl_plaintext://10.48.148.44:9092/bootstrap: No worthy mechs found

The first log line is what cyrus/libsasl2 modules are installed on your client system, the second line is saying that it couldnt find a match for GSSAPI (your security.protocol) in that list.
You need to install the libsasl2-modules-gssapi-mit debian package.

@aravinda92
Copy link
Author

Thank you. It worked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@edenhill @aravinda92