chore(tracing): limit strings to 25000 chars #13120

mabdinur · 2025-04-08T20:13:29Z

Checklist

PR author has checked that all the criteria below are met
The PR description includes an overview of the change
The PR description articulates the motivation for the change
The change includes tests OR the PR description describes a testing strategy
The PR description notes risks associated with the change, if any
Newly-added code is easy to change
The change follows the library release note guidelines
The change includes or references documentation updates if necessary
Backport labels are set (if applicable)

Reviewer Checklist

Reviewer has checked that all the criteria below are met
Title is accurate
All changes are related to the pull request's stated goal
Avoids breaking API changes
Testing strategy adequately addresses listed risks
Newly-added code is easy to change
Release note makes sense to a user of the library
If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
Backport labels are set in a manner that is consistent with the release branch maintenance policy

github-actions · 2025-04-08T20:14:02Z

CODEOWNERS have been resolved as:

ddtrace/internal/_encoding.pyx                                          @DataDog/apm-core-python
tests/integration/test_integration.py                                   @DataDog/apm-core-python
tests/tracer/test_encoders.py                                           @DataDog/apm-sdk-api-python

ddtrace/internal/_encoding.pyx

tests/tracer/test_encoders.py

github-actions · 2025-04-10T21:34:54Z

Bootstrap import analysis

Comparison of import times between this PR and base.

Summary

The average import time from this PR is: 238 ± 4 ms.

The average import time from base is: 246 ± 5 ms.

The import time difference between this PR and base is: -8.6 ± 0.2 ms.

Import time breakdown

The following import paths have appeared:

ddtrace.auto 0.323 ms (0.14%)

ddtrace 0.323 ms (0.14%)

ddtrace._logger 0.323 ms (0.14%)

ddtrace.internal.telemetry 0.323 ms (0.14%)

ddtrace.internal.telemetry.writer 0.323 ms (0.14%)

ddtrace.internal.encoding 0.323 ms (0.14%)

ddtrace.internal._encoding 0.323 ms (0.14%)

ddtrace._trace._limits 0.323 ms (0.14%)

ddtrace._trace 0.131 ms (0.06%)

The following import paths have disappeared:

ddtrace.auto 0.495 ms (0.21%)

ddtrace 0.495 ms (0.21%)

ddtrace.trace 0.495 ms (0.21%)

ddtrace._trace.context 0.254 ms (0.11%)

ddtrace._trace 0.254 ms (0.11%)

ddtrace._trace.filters 0.241 ms (0.10%)

ddtrace._trace.processor 0.241 ms (0.10%)

ddtrace._trace.sampler 0.241 ms (0.10%)

ddtrace._trace.span 0.241 ms (0.10%)

ddtrace._trace._limits 0.241 ms (0.10%)

The following import paths have grown:

ddtrace.auto 1.918 ms (0.81%)

ddtrace.bootstrap.sitecustomize 0.959 ms (0.40%)

ddtrace.bootstrap.preload 0.959 ms (0.40%)

ddtrace.internal.products 0.959 ms (0.40%)

importlib.metadata 0.903 ms (0.38%)

importlib.abc 0.782 ms (0.33%)

importlib.resources 0.782 ms (0.33%)

importlib.resources._legacy 0.782 ms (0.33%)

zipfile 0.064 ms (0.03%)

zipfile._path 0.064 ms (0.03%)

importlib.metadata._collections 0.057 ms (0.02%)

ddtrace.internal.remoteconfig._connectors 0.056 ms (0.02%)

ctypes 0.056 ms (0.02%)

ddtrace 0.959 ms (0.40%)

ddtrace._logger 0.733 ms (0.31%)

ddtrace.internal.telemetry 0.733 ms (0.31%)

ddtrace.internal.telemetry.writer 0.733 ms (0.31%)

ddtrace.internal.encoding 0.733 ms (0.31%)

ddtrace.internal.compat 0.733 ms (0.31%)

ddtrace.trace 0.225 ms (0.09%)

ddtrace._trace.filters 0.225 ms (0.09%)

ddtrace._trace.processor 0.225 ms (0.09%)

ddtrace.internal.dogstatsd 0.094 ms (0.04%)

ddtrace.vendor.dogstatsd 0.094 ms (0.04%)

ddtrace.vendor.dogstatsd.base 0.094 ms (0.04%)

queue 0.094 ms (0.04%)

heapq 0.094 ms (0.04%)

_heapq 0.094 ms (0.04%)

ddtrace._trace.sampler 0.076 ms (0.03%)

ddtrace._trace.span 0.076 ms (0.03%)

ddtrace.internal.sampling 0.076 ms (0.03%)

ddtrace._trace.sampling_rule 0.076 ms (0.03%)

ddtrace.internal.glob_matching 0.076 ms (0.03%)

ddtrace.internal.writer 0.056 ms (0.02%)

ddtrace.internal.writer.writer 0.056 ms (0.02%)

The following import paths have shrunk:

ddtrace.auto 5.730 ms (2.41%)

ddtrace.bootstrap.sitecustomize 3.183 ms (1.34%)

ddtrace.bootstrap.preload 3.156 ms (1.33%)

ddtrace.internal.products 3.038 ms (1.28%)

importlib.metadata 1.116 ms (0.47%)

importlib.abc 0.932 ms (0.39%)

importlib.resources 0.932 ms (0.39%)

importlib.resources._common 0.932 ms (0.39%)

zipfile 0.103 ms (0.04%)

importlib.metadata._meta 0.081 ms (0.03%)

ddtrace.internal.remoteconfig.client 0.787 ms (0.33%)

ddtrace.internal.remoteconfig._connectors 0.101 ms (0.04%)

ctypes 0.101 ms (0.04%)

_ctypes 0.101 ms (0.04%)

ddtrace.internal.symbol_db.remoteconfig 0.098 ms (0.04%)

ddtrace.internal.symbol_db.symbols 0.046 ms (0.02%)

gzip 0.046 ms (0.02%)

ddtrace._trace.trace_handlers 0.028 ms (0.01%)

ddtrace._trace._inferred_proxy 0.028 ms (0.01%)

ddtrace.propagation.http 0.028 ms (0.01%)

ddtrace.internal._tagset 0.028 ms (0.01%)

ddtrace 2.547 ms (1.07%)

ddtrace._logger 1.419 ms (0.60%)

ddtrace.internal.telemetry 1.259 ms (0.53%)

ddtrace.internal.telemetry.writer 1.259 ms (0.53%)

ddtrace.internal.encoding 0.877 ms (0.37%)

http.client 0.220 ms (0.09%)

email.parser 0.087 ms (0.04%)

email.feedparser 0.087 ms (0.04%)

email._policybase 0.087 ms (0.04%)

email.utils 0.066 ms (0.03%)

email._parseaddr 0.035 ms (0.01%)

calendar 0.035 ms (0.01%)

locale 0.035 ms (0.01%)

socket 0.031 ms (0.01%)

_socket 0.031 ms (0.01%)

email.header 0.022 ms (0.01%)

binascii 0.022 ms (0.01%)

ddtrace.internal.telemetry.data 0.075 ms (0.03%)

ddtrace.internal.packages 0.041 ms (0.02%)

_sysconfigdata__linux_x86_64-linux-gnu 0.041 ms (0.02%)

sysconfig 0.034 ms (0.01%)

ddtrace.internal.utils.http 0.047 ms (0.02%)

dataclasses 0.047 ms (0.02%)

inspect 0.047 ms (0.02%)

ast 0.047 ms (0.02%)

ddtrace.internal.telemetry.metrics_namespaces 0.041 ms (0.02%)

logging 0.160 ms (0.07%)

traceback 0.160 ms (0.07%)

contextlib 0.160 ms (0.07%)

ddtrace.trace 0.396 ms (0.17%)

ddtrace._trace.filters 0.295 ms (0.12%)

ddtrace._trace.processor 0.295 ms (0.12%)

ddtrace.internal.dogstatsd 0.114 ms (0.05%)

ddtrace.vendor.dogstatsd 0.114 ms (0.05%)

ddtrace.vendor.dogstatsd.base 0.114 ms (0.05%)

queue 0.114 ms (0.05%)

heapq 0.114 ms (0.05%)

ddtrace.internal.writer 0.093 ms (0.04%)

ddtrace.internal.writer.writer 0.093 ms (0.04%)

ddtrace.internal.writer.writer_client 0.093 ms (0.04%)

ddtrace._trace.sampler 0.088 ms (0.04%)

ddtrace._trace.span 0.088 ms (0.04%)

ddtrace.internal.sampling 0.088 ms (0.04%)

ddtrace._trace.sampling_rule 0.088 ms (0.04%)

ddtrace._trace.context 0.101 ms (0.04%)

ddtrace._monkey 0.036 ms (0.02%)

ddtrace.appsec._listeners 0.036 ms (0.02%)

ddtrace.internal.core 0.036 ms (0.02%)

pr-commenter · 2025-04-10T21:53:22Z

Benchmarks

Benchmark execution time: 2025-04-23 19:37:23

Comparing candidate commit d5da517 in PR branch munir/use-tag-limits with baseline commit a28cca3 in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 486 metrics, 4 unstable metrics.

scenario:ddtracerun-auto_profiling

🟥 execution_time [+171.963ms; +175.413ms] or [+26.293%; +26.821%]

scenario:iast_aspects-ljust_aspect

🟥 execution_time [+1.115µs; +1.292µs] or [+8.587%; +9.952%]

…p if it is more than the buffer for resource names.

ddtrace/internal/_encoding.pyx

quinna-h · 2025-04-23T17:57:07Z

tests/tracer/test_encoders.py

+    span.resource = "resource" * repetition
+
+    # test encoding for MsgPack format using the 8388608 default buffer size
+    encoder = MSGPACK_ENCODERS["v0.5"](1 << 23, 1 << 23)


change this to run in the subprocess with environment variables

tests/integration/test_integration.py

mabdinur · 2025-04-28T16:35:28Z

Closing in favor of: #13270

chore(tracing): limit strings to 25000 chars

d8cd4df

mabdinur force-pushed the munir/use-tag-limits branch from 5577a83 to d8cd4df Compare April 8, 2025 20:14

wantsui reviewed Apr 10, 2025

View reviewed changes

ddtrace/internal/_encoding.pyx Outdated Show resolved Hide resolved

Add a test for encoding a large resource name.

22dfcd0

wantsui reviewed Apr 10, 2025

View reviewed changes

tests/tracer/test_encoders.py Outdated Show resolved Hide resolved

wantsui added 3 commits April 17, 2025 14:07

Truncate insert payload if it is larger than the max meta len but dro…

f77ceab

…p if it is more than the buffer for resource names.

Change resource name too large test.

f72f195

Merge branch 'main' into munir/use-tag-limits

120c985

wantsui mentioned this pull request Apr 17, 2025

[BUG]: RuntimeError: failed to pack span for large SQL queries #13221

Closed

quinna-h added 2 commits April 22, 2025 13:10

update test

739d3af

update integration test

e060da4

quinna-h reviewed Apr 23, 2025

View reviewed changes

ddtrace/internal/_encoding.pyx Outdated Show resolved Hide resolved

quinna-h reviewed Apr 23, 2025

View reviewed changes

ddtrace/internal/_encoding.pyx Outdated Show resolved Hide resolved

quinna-h reviewed Apr 23, 2025

View reviewed changes

tests/integration/test_integration.py Outdated Show resolved Hide resolved

refactor for nits

d5da517

mabdinur closed this Apr 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(tracing): limit strings to 25000 chars #13120

chore(tracing): limit strings to 25000 chars #13120

mabdinur commented Apr 8, 2025 •

edited

Loading

github-actions bot commented Apr 8, 2025 •

edited

Loading

github-actions bot commented Apr 10, 2025 •

edited

Loading

pr-commenter bot commented Apr 10, 2025 •

edited

Loading

quinna-h Apr 23, 2025

mabdinur commented Apr 28, 2025

chore(tracing): limit strings to 25000 chars #13120

chore(tracing): limit strings to 25000 chars #13120

Conversation

mabdinur commented Apr 8, 2025 • edited Loading

Checklist

Reviewer Checklist

github-actions bot commented Apr 8, 2025 • edited Loading

github-actions bot commented Apr 10, 2025 • edited Loading

Bootstrap import analysis

Summary

Import time breakdown

pr-commenter bot commented Apr 10, 2025 • edited Loading

Benchmarks

scenario:ddtracerun-auto_profiling

scenario:iast_aspects-ljust_aspect

quinna-h Apr 23, 2025

Choose a reason for hiding this comment

mabdinur commented Apr 28, 2025

mabdinur commented Apr 8, 2025 •

edited

Loading

github-actions bot commented Apr 8, 2025 •

edited

Loading

github-actions bot commented Apr 10, 2025 •

edited

Loading

pr-commenter bot commented Apr 10, 2025 •

edited

Loading