Skip to content

Commit 66cc26e

Browse files
committed
processor_opentelemetry_envelope: add new processor docs
Signed-off-by: Eduardo Silva <[email protected]>
1 parent 7de88a4 commit 66cc26e

File tree

2 files changed

+167
-1
lines changed

2 files changed

+167
-1
lines changed

SUMMARY.md

+2-1
Original file line numberDiff line numberDiff line change
@@ -130,9 +130,10 @@
130130
* [Logfmt](pipeline/parsers/logfmt.md)
131131
* [Decoders](pipeline/parsers/decoders.md)
132132
* [Processors](pipeline/processors/README.md)
133-
* [Labels](pipeline/processors/labels.md)
134133
* [Content Modifier](pipeline/processors/content-modifier.md)
134+
* [Labels](pipeline/processors/labels.md)
135135
* [Metrics Selector](pipeline/processors/metrics-selector.md)
136+
* [OpenTelemetry Envelope](pipeline/processors/opentelemetry-envelope.md)
136137
* [SQL](pipeline/processors/sql.md)
137138
* [Filters](pipeline/filters/README.md)
138139
* [AWS Metadata](pipeline/filters/aws-metadata.md)
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# OpenTelemetry Envelope
2+
3+
The _OpenTelemetry Envelope` processor is used to transform your data to be compatible with the OpenTelemetry Log schema. If your data was __not__ generated by [OpenTelemetry input](../inputs/opentelemetry.md) and your backend or destination for your logs expects to be in an OpenTelemetry schema.
4+
5+
![](/imgs/processor_opentelemetry_envelope.png)
6+
7+
## Configuration Parameters
8+
9+
The processor does not provide any extra configuration parameter, it can be used directly in your _processors_ Yaml directive.
10+
11+
## Usage Example
12+
13+
In this example, we will use the Dummy input plugin to generate a sample message per second, right after is created the processor `opentelemetry_envelope` is used to transform the data to be compatible with the OpenTelemetry Log schema. The output is sent to the standard output and also to an OpenTelemetry collector which is receiving data in port 4318.
14+
15+
16+
__fluent-bit.yaml__
17+
18+
```yaml
19+
service:
20+
flush: 1
21+
log_level: info
22+
23+
pipeline:
24+
inputs:
25+
- name: dummy
26+
dummy: '{"message": "Hello World"}'
27+
28+
processors:
29+
logs:
30+
- name: opentelemetry_envelope
31+
32+
outputs:
33+
- name : stdout
34+
match: '*'
35+
36+
- name: opentelemetry
37+
match: '*'
38+
host: 127.0.0.1
39+
port: 4318
40+
```
41+
42+
__otel-collector.yaml__
43+
44+
```yaml
45+
receivers:
46+
otlp:
47+
protocols:
48+
http:
49+
endpoint: 127.0.0.1:4318
50+
51+
exporters:
52+
file:
53+
path: out.json
54+
logging:
55+
loglevel: info
56+
57+
service:
58+
telemetry:
59+
logs:
60+
level: debug
61+
pipelines:
62+
logs:
63+
receivers: [otlp]
64+
exporters: [file, logging]
65+
```
66+
67+
You will notice in the standard output of FLuent Bit will print the raw representation of the schema, however, the OpenTelemetry collector will receive the data in the OpenTelemetry Log schema.
68+
69+
Inspecting the output file `out.json` you will see the data in the OpenTelemetry Log schema:
70+
71+
72+
```json
73+
{
74+
"resourceLogs": [
75+
{
76+
"resource": {},
77+
"scopeLogs": [
78+
{
79+
"scope": {},
80+
"logRecords": [
81+
{
82+
"timeUnixNano": "1722904188085758000",
83+
"body": {
84+
"stringValue": "dummy"
85+
},
86+
"traceId": "",
87+
"spanId": ""
88+
}
89+
]
90+
}
91+
]
92+
}
93+
]
94+
}
95+
```
96+
97+
While OpenTelemetry Envelope enrich your logs with the Schema, you might be interested into take a step further and use the [Content Modifier](../processors/content-modifier.md) processor to modify the content of your logs. Here is a quick example that will allow you to add some resource and scope attributes to your logs:
98+
99+
```yaml
100+
service:
101+
flush: 1
102+
log_level: info
103+
104+
pipeline:
105+
inputs:
106+
- name: dummy
107+
dummy: '{"message": "Hello World"}'
108+
109+
processors:
110+
logs:
111+
- name: opentelemetry_envelope
112+
113+
- name: content_modifier
114+
context: otel_resource_attributes
115+
action: upsert
116+
key: service.name
117+
value: my-service
118+
119+
outputs:
120+
- name : stdout
121+
match: '*'
122+
123+
- name: opentelemetry
124+
match: '*'
125+
host: 127.0.0.1
126+
port: 4318
127+
```
128+
129+
The collector JSON output will look like this:
130+
131+
```json
132+
{
133+
"resourceLogs": [
134+
{
135+
"resource": {
136+
"attributes": [
137+
{
138+
"key": "service.name",
139+
"value": {
140+
"stringValue": "my-service"
141+
}
142+
}
143+
]
144+
},
145+
"scopeLogs": [
146+
{
147+
"scope": {},
148+
"logRecords": [
149+
{
150+
"timeUnixNano": "1722904465173450000",
151+
"body": {
152+
"stringValue": "Hello World"
153+
},
154+
"traceId": "",
155+
"spanId": ""
156+
}
157+
]
158+
}
159+
]
160+
}
161+
]
162+
}
163+
```
164+
165+
For more details about further processing, read the [Content Modifier](../processors/content-modifier.md) processor documentation.

0 commit comments

Comments
 (0)