Jdennison/expose stats callback

[johnistan]

I am actively investigating this client and needed topic lag for consumers. I exposed the underlying on_stats callback in librdkafka.

If you would actually like to use this it needs doc strings and likely the addition of the same callback in the producer.

Also, I wanted to expose a dict to the callback. I have no idea if just calling the builtin python json module is a good idea. My C/python C-api skills are limited.

Also, this api is good for my consumers. However, I also have use cases for synchronous querying of the offset lag and current offsets in a topic within a monitoring client(i.e. no active consumption is occurring). Does librdkafka support something like this, or is it only a callback.

[ghost]

Hey @jofusa,
thank you for your Pull Request.

@confluentinc It looks like this brave person signed our Contributor License Agreement. 👍

Always at your service,

clabot

[edenhill]

Thanks for your PR.

I do agree that the stats callback is indeed useful and it should be exposed.
But I'm not certain that auto-translating to a dict is the right choice, it increases the C complexity unproportionally to what the corresponding Python code would be (about two lines of code, including import). Let's see if it makes sense to move the JSON conversion to the Python shim.

[edenhill]

Thanks for your contribution, this was added as part of #55