Skip to content

Improve metrics in Celery instrumentation #3458

Open
@rbagd

Description

@rbagd

What problem do you want to solve?

Our instrumented Celery applications often are deployed along with Celery Flower to monitor a few metrics on tasks and workers. But those metrics can also be made available through auto-instrumentation. I am sure for those whose usage of Flower is limited having it all in a single instrumentation would be beneficial.

Describe the solution you'd like

  • Add metrics from Celery Flower, i.e. counts for executing and prefetched tasks as well as prefetch and execution duration of a task.
  • Rename those metrics according to OTel semantic conventions
  • Fix a memory leak in current metrics generation (this entry never gets removed)

Describe alternatives you've considered

No response

Additional Context

I got some parts working in my local environment but testing has been a challenge. I don't think I ever had all unit tests passing even in clean setup. I have no idea if it's something in my own environment as Celery tests appear to be generally green in PRs.

Would you like to implement a fix?

Yes

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions