Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter/elasticsearch] Sort and deduplicate attributes for metrics grouping #38083

Open
carsonip opened this issue Feb 20, 2025 · 2 comments · May be fixed by #39309
Open

[exporter/elasticsearch] Sort and deduplicate attributes for metrics grouping #38083

carsonip opened this issue Feb 20, 2025 · 2 comments · May be fixed by #39309
Labels
bug Something isn't working exporter/elasticsearch

Comments

@carsonip
Copy link
Contributor

carsonip commented Feb 20, 2025

Component(s)

exporter/elasticsearch

What happened?

Description

ECS and OTel mapping modes implement metrics grouping. They hash the attributes in the order as they are in the attribute maps. However, in the edge case where 2 set of attributes have the same key value pairs but in different orders, they will generate different hash, which will in turn create 2 documents for the same set of dimensions, resulting in version_conflict_engine_exception from TSDB.

OTel mode attributes need to be sorted. ECS mode attributes need to be sorted and deduplicated as attributes at different level are all propagated up to the top level.

Steps to Reproduce

Expected Result

Actual Result

Collector version

v0.120.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

OpenTelemetry Collector configuration

Log output

Additional context

No response

@carsonip carsonip added bug Something isn't working needs triage New item requiring triage labels Feb 20, 2025
@carsonip carsonip changed the title [exporter/elasticsearch] Sort and deduplicate as part of metrics grouping [exporter/elasticsearch] Sort and deduplicate attributes for metrics grouping Feb 20, 2025
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@carsonip
Copy link
Contributor Author

/label -needs-triage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working exporter/elasticsearch
Projects
None yet
1 participant