Skip to content

POtel implementation base branch #3152

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 318 commits into
base: master
Choose a base branch
from
Draft

POtel implementation base branch #3152

wants to merge 318 commits into from

Conversation

sl0thentr0py
Copy link
Member

@sl0thentr0py sl0thentr0py commented Jun 10, 2024

Full state of CI: #3744

Contains:

Simple test

import sentry_sdk
from time import sleep

sentry_sdk.init(
    debug=True,
    traces_sample_rate=1.0,
    _experiments={"otel_powered_performance": True},
)

with sentry_sdk.start_span(description="sentry request"):
    sleep(0.1)
    with sentry_sdk.start_span(description="sentry db"):
        sleep(0.5)
        with sentry_sdk.start_span(description="sentry redis"):
            sleep(0.2)
    with sentry_sdk.start_span(description="sentry http"):
        sleep(1)

References

Misc

In OTel, this:

with tracer.start_as_current_span("parent") as parent:
    with tracer.start_span("child1"):
        pass
    with tracer.start_span("child2"):
        pass

is equivalent to

from opentelemetry import trace, context

parent = tracer.start_span("parent")

# Creates a Context object with parent set as current span
ctx = trace.set_span_in_context(parent)

# Set as the implicit current context
token = context.attach(ctx)

# Child will automatically be a child of parent
child1 = tracer.start_span("child1")
child1.end()

# Child will automatically be a child of parent
child2 = tracer.start_span("child2")
child2.end()

# Don't forget to detach or parent will remain the parent above this call stack
context.detach(token)
parent.end()

@sl0thentr0py sl0thentr0py requested a review from sentrivana June 10, 2024 19:00
@sl0thentr0py sl0thentr0py force-pushed the potel-base branch 2 times, most recently from f7f153c to 28effd6 Compare June 11, 2024 11:43
@sl0thentr0py sl0thentr0py force-pushed the potel-base branch 2 times, most recently from 16f9341 to 951477f Compare June 25, 2024 15:16
Copy link

codecov bot commented Jun 26, 2024

Codecov Report

Attention: Patch coverage is 84.13340% with 314 lines in your changes missing coverage. Please review.

Project coverage is 84.36%. Comparing base (815de9f) to head (6d2e988).
Report is 11 commits behind head on master.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
sentry_sdk/tracing.py 79.25% 39 Missing and 11 partials ⚠️
sentry_sdk/integrations/aws_lambda.py 11.11% 48 Missing ⚠️
sentry_sdk/opentelemetry/utils.py 82.15% 25 Missing and 18 partials ⚠️
sentry_sdk/opentelemetry/span_processor.py 81.70% 13 Missing and 17 partials ⚠️
sentry_sdk/integrations/gcp.py 0.00% 27 Missing ⚠️
sentry_sdk/tracing_utils.py 75.80% 10 Missing and 5 partials ⚠️
sentry_sdk/integrations/ray.py 7.14% 13 Missing ⚠️
sentry_sdk/opentelemetry/sampler.py 91.40% 7 Missing and 4 partials ⚠️
sentry_sdk/integrations/tornado.py 75.86% 4 Missing and 3 partials ⚠️
sentry_sdk/scope.py 86.79% 6 Missing and 1 partial ⚠️
... and 23 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3152      +/-   ##
==========================================
+ Coverage   80.28%   84.36%   +4.08%     
==========================================
  Files         142      144       +2     
  Lines       15919    14636    -1283     
  Branches     2722     2326     -396     
==========================================
- Hits        12780    12348     -432     
+ Misses       2263     1561     -702     
+ Partials      876      727     -149     
Files with missing lines Coverage Δ
sentry_sdk/__init__.py 100.00% <100.00%> (ø)
sentry_sdk/_compat.py 84.21% <ø> (-1.84%) ⬇️
sentry_sdk/_init_implementation.py 100.00% <100.00%> (+23.33%) ⬆️
sentry_sdk/ai/monitoring.py 88.57% <100.00%> (+2.46%) ⬆️
sentry_sdk/ai/utils.py 85.00% <100.00%> (+7.72%) ⬆️
sentry_sdk/api.py 95.40% <100.00%> (+15.54%) ⬆️
sentry_sdk/client.py 84.58% <100.00%> (+5.32%) ⬆️
sentry_sdk/consts.py 99.54% <100.00%> (+6.69%) ⬆️
sentry_sdk/debug.py 95.00% <ø> (+3.69%) ⬆️
sentry_sdk/envelope.py 85.71% <100.00%> (+3.19%) ⬆️
... and 83 more

... and 43 files with indirect coverage changes

@sl0thentr0py sl0thentr0py changed the title Skeletons for new POTEL components New POTEL base branch Jul 9, 2024
@sl0thentr0py sl0thentr0py changed the title New POTEL base branch potel implementation base branch Jul 9, 2024
@antonpirker antonpirker changed the title potel implementation base branch POtel implementation base branch Aug 5, 2024
@sentrivana sentrivana removed their request for review August 28, 2024 09:12
sentrivana and others added 18 commits October 4, 2024 09:36
I messed up the merge. If an error happens, we should still report it, whatever the HTTP method. Fixing here.
* Populates a DSC with correct values when we don't have an incoming trace.
* We rely on `trace_state.add` only adding new keys to the tracestate so these values will be populated in the first sampling decision on the root and just be propagated further.

Note that transaction name is missing here for now and will be dealt with separately as part of the transaction name PRs.


closes #3479
Set the correct span `op` for database spans, instead of the generic `"db"`.
…ts (#3631)

Otel `span.set_attribute()` does not allow `dict` values, thus we change the data to be compatible
`trace_state.add` will never overwrite existing entries -- this is good. However, everytime it bails, it logs a warning, which then shows up in breadcrumbs in tests.

With this commit we explicitly check before adding.
* while sending the event in the final payload, the transaction name
  will be picked up from the attributes `sentry.name` or fallback to the
  `description` when we only have otel instrumentation
* for populating DSC in the head case, we are doing a best attempt
  solution and fetching the transaction name/source from the current and
  isolation scopes
    * NOTE that there are cases where this will be
      inaccurate if we never set the transaction name on the scope in some
      integration, so we will go through and fix those cases separately.
sentrivana and others added 30 commits April 9, 2025 09:46
- trytond<5, falcon<3, django<2 are all more than 5 years old

Closes #4049
Make the `Span` constructor actually fail if it gets unsupported
arguments, as opposed to silently ignoring them, so that folks get
notified early.

Deprecating some of these in
#4244

Closes #4200
Changing the default of `traces_sample_rate` to `0`. This means incoming
traces will be continued, but we will not start traces on our own. (It
used to be set to: never start or continue traces by default)

Refs #4102
BREAKING CHANGE: Remove `Span.containing_transaction`. Use
`Span.root_span` instead.

Closes #4253

<!-- Describe your PR here -->

---

Thank you for contributing to `sentry-python`! Please add tests to
validate your changes, and lint your code using `tox -e linters`.

Running the test suite on your PR might require maintainer approval.
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/sampler.py` ->
`sentry_sdk/opentelemetry/sampler.py`

Ref #3853
This reverts commit 955108e (#3791) and
simply enables `cache_spans` by default.

Co-authored-by: Anton Pirker <[email protected]>
Co-authored-by: Ivana Kellyer <[email protected]>
Closes #4235

---------

Co-authored-by: Ivana Kellyer <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/span_processor.py` ->
`sentry_sdk/opentelemetry/span_processor.py`

Ref #3853

---------

Co-authored-by: Daniel Szoke <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/propagator.py` ->
`sentry_sdk/opentelemetry/propagator.py`

Ref #3853

---------

Co-authored-by: Daniel Szoke <[email protected]>
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves
`sentry_sdk/integrations/opentelemetry/contextvars_context.py` ->
`sentry_sdk/opentelemetry/contextvars_context.py`

Ref #3853
Port of #4278 for POTel.

<!-- Describe your PR here -->

---

Thank you for contributing to `sentry-python`! Please add tests to
validate your changes, and lint your code using `tox -e linters`.

Running the test suite on your PR might require maintainer approval.
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK. This moves
`sentry_sdk/integrations/opentelemetry/{utils,consts}.py` ->
`sentry_sdk/opentelemetry/{utils,consts}.py`.

There's quite some stuff moving around in this PR in order to remove
circular dependencies that have to do with `tracing.py`:
- `get_span_status_from_http_code` from `tracing.py` is now in
`tracing_utils.py`
- various constants from `tracing.py` are now in `consts.py`

Additionally, Sphinx was unhappy, so tweaked the way we type some things
in `api.py` and `_init_implementation.py`.

I'll possibly follow this up with creating a nice structure for the
tracing files, maybe a common `tracing` directory with `tracing.py`,
`tracing_utils.py` -> `utils.py`, and dedicated `consts.py`.

Ref #3853
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

This moves `sentry_sdk/integrations/opentelemetry/scope.py` ->
`sentry_sdk/opentelemetry/scope.py`

Ref #3853
Moving stuff out of `integrations/opentelemetry/` step by step since
there is no OpenTelemetry integration anymore -- it's part of the core
SDK.

- Moved `sentry_sdk/integrations/opentelemetry/integration.py` ->
`sentry_sdk/opentelemetry/tracing.py`.
- Removed all the experimental autoinstrumentation stuff from
`integration.py`.
- Removed `integrations/opentelemetry/` altogether (there was nothing
left but `__init__.py`, which is now also gone).
- Moved all tests from `tests/integrations/opentelemetry` to
`tests/opentelemetry`.
- Removed the potel/opentelemetry integration test targets from
`tox.ini`. These will now be run as part of the Common test suite.

Ref #3853
Porting stuff from master and other fixes
* add correct `event_level` to new logging tests (on `potel-base`, we
don't capture logging errors by default so this has to be set
explicitly)
* add compat for `start_transaction`
* re-enable an old test

**Note:** This still leaves one failing threading test, will address
that separately
For testing purposes I set the next major version of the SDK.
Currently, this property has type `Any`, but it can now be changed to
`Optional[Span]`

Depends on:
  - #4263
`ThreadingIntegration` can optionally **NOT** propagate scope data to
threads (`propagate_scope=False`). In that case, in POTel we were
wrapping the thread's task in an `isolation_scope()`:

```python
with sentry_sdk.isolation_scope() as scope:
    return _run_old_run_func()
```

But as this forks the currently active isolation scope, the thread
effectively gets all scope data from the parent isolation scope -- so
the scope is actually propagated to the thread, even though it shouldn't
be since `propagate_scope=False`.

~We effectively need some way to give the thread a clear isolation scope
instead. In this PR, I'm just clearing the forked iso scope, but I'm not
sure if this is good enough and if something doesn't need to be done on
the OTel side too.~

~Another option would be to set the iso/current scopes to the initial,
empty iso/current scopes instead, before running the thread's target
function.~

UPDATE: we're just instantiating new scopes now


Another change is that in OTel, the spans in the threads, now without a
parent, automatically get promoted to transactions. (On master they'd
just be orphaned spans, so they wouldn't be taken into account at all.)
We probably need to instruct folks to add `only_if_parent` if they don't
want this to happen.

---------

Co-authored-by: Neel Shah <[email protected]>
With the switch to OTel, the Common test suite is now dependent on an
otel package, so it technically fits the toxgen usecase. By letting
toxgen take care of it, we're making sure we're always testing a good
range of otel versions, including the oldest one (to catch regressions)
and the newest one (to catch incompatibilities early).

Couple things surfaced in terms of incompatibility with older versions:
- Some semantic attributes we're using weren't there from the get go
open-telemetry/opentelemetry-python@495d705.
Changed the code that uses them to handle failure.
- The signature of `span.set_status()` changed at some point
open-telemetry/opentelemetry-python@6e282d2.
Added a compat version of `set_status()` for older otel.

Also included:
- removing the `opentelemetry-experimental` extra (not used anymore)
- ❗ switching to using `opentelemetry-sdk` instead of
`opentelemetry-distro` -- the `distro` only seems to [be setting up some
defaults](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/8390db35ae2062c09d4d74a08d310c7bde1912c4/opentelemetry-distro/src/opentelemetry/distro/__init__.py)
that we're not using


Closes #3241
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants