Skip to content

Use OTLP/UDP Span Exporter to export spans to XRay via Lambda's UDP endpoint #1685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jj22ee opened this issue Jan 31, 2025 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@jj22ee
Copy link

jj22ee commented Jan 31, 2025

Is your feature request related to a problem? Please describe.

X-Ray users who want to use OTel in AWS Lambda are recommended to use the OTel Lambda Layers that packages both the OTel Collector and OTel SDK components to instrument a Lambda Function. However, using OTel on Lambda may incur performance impact, partially due to using OTel Collector.

Describe the solution you'd like

The approach to reduce the performance impact in Lambda by OTel involves avoid running the Collector in Lambda. Lambda supports their own endpoint to receive OTLP/UDP spans and send them to AWS X-Ray. It is re-using the existing UDP endpoint from Lambda's AWS_XRAY_DAEMON_ADDRESS env var that is currently used by the X-Ray tracing SDKs. It exists as an alternative to the Collector's OTLP Receiver.

Today, AWS Observability's auto-instrumentation is leveraging a UDP Exporter to export traces to Lambda's UDP endpoint to send traces to AWS X-Ray. I want to add this UDP Exporter (Java, Python, JS, .NET) to the respective OTel repositories (likely the contrib one unless this UDP exporter could be part of the otel spec).

Additional context

  • There is currently no OTel specification for this OTLP/UDP Exporter
  • This UDP endpoint (provided by Lambda's AWS_XRAY_DAEMON_ADDRESS env var) is specific to vending to X-Ray. This means this performance impact mitigation solution is not compatible with the vendor-agnostic principal; the existing collector-in-Lambda solution is still needed to vend to non-XRay. I think AWS Lambda still needs a new solution from their end to mitigate performance in this scenario.
@jj22ee
Copy link
Author

jj22ee commented Feb 6, 2025

To clarify, I'm currently working on adding this UDP Exporter to the language-contrib repos, but I wanted to know if opentelemetry-lambda is open to allow this UDP Exporter to configurable in the Lambda Layer (possibly through environment variable to use UDP Exporter here in Nodejs layer's wrapper.ts for example).

@tylerbenson
Copy link
Member

If it's listed as a standard exporter in the specification with a well defined config setting and has use beyond sending to just x-ray, I'd be open to considering it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants