Skip to content

Support protobuf 7.x in opentelemetry-protoΒ #5205

@theramved

Description

@theramved

Is your feature request related to a problem?

I would like to request support for protobuf >=7.0,<8.0 in opentelemetry-proto. Version Pin here

Currently, opentelemetry-proto appears to constrain protobuf to <7, which prevents downstream projects from using newer generated protobuf packages that require protobuf 7.x at runtime.

Use Case

We maintain a Python service that consumes generated gRPC/protobuf packages. The generated code was built with protobuf gencode 7.34.1, so the service requires:

protobuf >=7.34.1,<8.0

Problem

Our service depends on MLflow 3.x. MLflow pulls in OpenTelemetry proto transitively through mlflow-skinny / mlflow-tracing.

In our environment, poetry show opentelemetry-proto reports:

name : opentelemetry-proto
version : 1.11.1
description : OpenTelemetry Python Proto

dependencies

  • protobuf >=3.13.0

required by

  • mlflow-skinny requires >=1.9.0,<3
  • mlflow-tracing requires >=1.9.0,<3

When MLflow imports OpenTelemetry proto with protobuf 7.x installed, the application fails with:

TypeError: Descriptors cannot be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:

  1. Downgrade the protobuf package to 3.20.x or lower.
  2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

The import path is:

mlflow
-> mlflow.protos.service_pb2
-> opentelemetry.proto.trace.v1.trace_pb2
-> opentelemetry.proto.common.v1.common_pb2

I tried explicitly upgrading opentelemetry-proto:

poetry add "opentelemetry-proto>=1.41.0,<2.0"

but dependency resolution fails because newer opentelemetry-proto versions still require protobuf <7:

opentelemetry-proto 1.41.0 depends on protobuf >=5.0,<7.0
opentelemetry-proto 1.41.1 depends on protobuf >=5.0,<7.0

So, because our project depends on protobuf >=7.34.1,<8.0
and opentelemetry-proto >=1.41.0,<2.0,
version solving failed.

Describe the solution you'd like

Could OpenTelemetry Python add support for protobuf 7.x by:

widening the protobuf upper bound from <7.0 to <8.0, if compatible, and
regenerating the generated proto files with a protobuf/protoc version compatible with protobuf 7.x?

This would unblock downstream projects that need protobuf 7.x due to generated gRPC/protobuf packages, while also depending on MLflow or other libraries that transitively depend on OpenTelemetry proto.

Describe alternatives you've considered

The only workaround appears to be pure-Python protobuf parsing by setting PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python because it forces pure-Python parsing and may cause performance overhead.

Additional Context

Additional dependency context from our Poetry environment:

protobuf 7.34.1
mlflow 3.12.0
mlflow-skinny 3.12.0
mlflow-tracing 3.12.0
opentelemetry-proto 1.11.1
grpcio 1.80.0

Would you like to implement a fix?

No

Tip

React with πŸ‘ to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions