Skip to content

Pipeline: inputs: otlp: style #1772

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 23, 2025
+33 −29
Merged

Pipeline: inputs: otlp: style #1772

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2886bd5
Pipeline: inputs: otlp: style
esmerel Jun 23, 2025
a503993
Apply suggestions from code review
esmerel Jun 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 33 additions & 29 deletions pipeline/inputs/opentelemetry.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,44 @@
---
description: An input plugin to ingest OTLP Logs, Metrics, and Traces
description: An input plugin to ingest OpenTelemetry logs, metrics, and traces
---

# OpenTelemetry

The OpenTelemetry input plugin allows you to receive data as per the OTLP specification, from various OpenTelemetry exporters, the OpenTelemetry Collector, or Fluent Bit's OpenTelemetry output plugin.
The OpenTelemetry input plugin lets you receive data based on the OpenTelemetry specification from various OpenTelemetry exporters, the OpenTelemetry Collector, or the Fluent Bit OpenTelemetry output plugin.

Our compliant implementation fully supports OTLP/HTTP and OTLP/GRPC. Note that the single `port` configured which defaults to 4318 supports both transports.
Fluent Bit has a compliant implementation which fully supports `OTLP/HTTP` and `OTLP/GRPC`. The single `port` configured defaults to `4318` and supports both transport methods.

## Configuration <a href="#configuration" id="configuration"></a>
## Configuration

| Key | Description | default |
| ----------------- | -----------------------------------------------------------------------------------| ------- |
| listen | The network address to listen. | 0.0.0.0 |
| port | The port for Fluent Bit to listen for incoming connections. Note that as of Fluent Bit v3.0.2 this port is used for both transport OTLP/HTTP and OTLP/GRPC. | 4318 |
| tag | Tag for all the data ingested by this plugin. Important: This will only be used if `tag_from_uri` is set to false. Otherwise, the tag will be created from URI. | |
| tag_key | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key | |
| raw_traces | Route trace data as a log | `false` |
| buffer_max_size | Specify the maximum buffer size in KB/MB/GB to the HTTP payload. | 4M |
| buffer_chunk_size | Initial size and allocation strategy to store the payload (advanced users only) | 512K |
|successful_response_code | It allows to set successful response code. `200`, `201` and `204` are supported.| 201 |
| tag_from_uri | By default, tag will be created from uri. e.g. v1_metrics from /v1/metrics. This must be set to false if using `tag` | `true` |
| threaded | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |
| Key | Description | Default |
| -------- | ------------| ------- |
| `listen` | The network address to listen on. | `0.0.0.0` |
| `port` | The port for Fluent Bit to listen for incoming connections. In Fluent Bit 3.0.2 or later, this port is used for both transport `OTLP/HTTP` and `OTLP/GRPC`. | `4318` |
| `tag` | Tag for all data ingested by this plugin. This will only be used if `tag_from_uri` is set to `false`. Otherwise, the tag will be created from the URI. | _none_ |
| `tag_key` | Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key. | _none_ |
| `raw_traces` | Route trace data as a log. | `false` |
| `buffer_max_size` | Specify the maximum buffer size in `KB`, `MB`, or `GB` to the HTTP payload. | `4M` |
| `buffer_chunk_size` | Initial size and allocation strategy to store the payload (advanced users only)` | `512K` |
| `successful_response_code` | Allows for setting a successful response code. Supported values: `200`, `201`, or `204`. | `201` |
| `tag_from_uri` | By default, the tag will be created from the URI. For example, `v1_metrics` from `/v1/metrics`. This must be set to false if using `tag`. | `true` |
| `threaded` | Indicates whether to run this input in its own [thread](../../administration/multithreading.md#inputs). | `false` |

Important note: Raw traces means that any data forwarded to the traces endpoint (`/v1/traces`) will be packed and forwarded as a log message, and will NOT be processed by Fluent Bit. The traces endpoint by default expects a valid protobuf encoded payload, but you can set the `raw_traces` option in case you want to get trace telemetry data to any of Fluent Bit's supported outputs.
Raw traces means that any data forwarded to the traces endpoint (`/v1/traces`) will be packed and forwarded as a log message, and won' be processed by Fluent Bit. The traces endpoint by default expects a valid `protobuf` encoded payload, but you can set the `raw_traces` option in case you want to get trace telemetry data to any of the Fluent Bit supported outputs.

### OTLP Transport Protocol Endpoints
### OpenTelemetry transport protocol endpoints

Fluent Bit based on the OTLP desired protocol exposes the following endpoints for data ingestion:
Fluent Bit exposes the following endpoints for data ingestion based on the OpenTelemetry protocol:

For `OTLP/HTTP`:

__OTLP/HTTP__
- Logs
- `/v1/logs`
- Metrics
- `/v1/metrics`
- Traces
- `/v1/traces`

__OTLP/GRPC__
For `OTLP/GRPC`:

- Logs
- `/opentelemetry.proto.collector.log.v1.LogService/Export`
Expand All @@ -49,8 +50,7 @@ __OTLP/GRPC__
- `/opentelemetry.proto.collector.trace.v1.TraceService/Export`
- `/opentelemetry.proto.collector.traces.v1.TracesService/Export`


## Getting started
## Get started

The OpenTelemetry input plugin supports the following telemetry data types:

Expand All @@ -60,12 +60,12 @@ The OpenTelemetry input plugin supports the following telemetry data types:
| Metrics | Unimplemented | Stable | Stable |
| Traces | Unimplemented | Stable | Stable |

A sample config file to get started will look something like the following:

A sample configuration file to get started will look something like the following:

{% tabs %}

{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
inputs:
Expand All @@ -76,10 +76,12 @@ pipeline:
- name: stdout
match: '*'
```

{% endtab %}

{% tab title="fluent-bit.conf" %}
```

```python
[INPUT]
name opentelemetry
listen 127.0.0.1
Expand All @@ -89,13 +91,15 @@ pipeline:
name stdout
match *
```

{% endtab %}

{% endtabs %}

With the above configuration, Fluent Bit will listen on port `4318` for data. You can now send telemetry data to the endpoints `/v1/metrics`, `/v1/traces`, and `/v1/logs` for metrics, traces, and logs respectively.
With this configuration, Fluent Bit listens on port `4318` for data. You can now send telemetry data to the endpoints `/v1/metrics` for metrics, `/v1/traces` for traces, and `/v1/logs` for logs.

A sample curl request to POST json encoded log data would be:
```
A sample curl request to POST JSON encoded log data would be:

```bash
curl --header "Content-Type: application/json" --request POST --data '{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"timeUnixNano":"1660296023390371588","body":{"stringValue":"{\"message\":\"dummy\"}"},"traceId":"","spanId":""}]}]}]}' http://0.0.0.0:4318/v1/logs
```