Skip to content

Commit 7049e9e

Browse files
edsiperedsiper
committed
Merge branch 'master' of github.com:fluent/fluent-bit-docs
2 parents 2c2ca5f + 67ac395 commit 7049e9e

18 files changed

+202
-29
lines changed

‎SUMMARY.md

+3
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@
4848
* [Collectd](input/collectd.md)
4949
* [CPU Usage](input/cpu.md)
5050
* [Disk Usage](input/disk.md)
51+
* [Docker](input/docker.md)
5152
* [Dummy](input/dummy.md)
5253
* [Exec](input/exec.md)
5354
* [Forward](input/forward.md)
@@ -61,6 +62,7 @@
6162
* [Random](input/random.md)
6263
* [Serial Interface](input/serial.md)
6364
* [Standard Input](input/stdin.md)
65+
* [StatsD Input](input/statsd.md)
6466
* [Syslog](input/syslog.md)
6567
* [Systemd](input/systemd.md)
6668
* [Tail](input/tail.md)
@@ -74,6 +76,7 @@
7476
* [Logfmt Parser](parser/logfmt.md)
7577
* [Decoders](parser/decoder.md)
7678
* [Filter Plugins](filter/README.md)
79+
* [AWS](filter/aws.md)
7780
* [Grep](filter/grep.md)
7881
* [Kubernetes](filter/kubernetes.md)
7982
* [Lua](filter/lua.md)

‎configuration/variables.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
11
# Configuration Variables
22

3-
Fluent Bit support the usage of environment variables in any value associated to a key when using a configuration file.
3+
Fluent Bit supports the usage of environment variables in any value associated to a key when using a configuration file.
44

55
The variables are case sensitive and can be used in the following format:
66

77
```text
88
${MY_VARIABLE}
99
```
1010

11-
When Fluent Bit starts, the configuration reader will detect any request for ${MY\_VARIABLE} and will try to resolve it value.
11+
When Fluent Bit starts, the configuration reader will detect any request for `${MY_VARIABLE}` and will try to resolve its value.
1212

1313
## Example
1414

15-
Create the following configuration file \(_fluent-bit.conf_\):
15+
Create the following configuration file (`fluent-bit.conf`):
1616

1717
```text
1818
[SERVICE]
@@ -35,7 +35,7 @@ Open a terminal and set the environment variable:
3535
$ export MY_OUTPUT=stdout
3636
```
3737

38-
> The above command set the 'stdout' value to the variable MY\_OUTPUT.
38+
> The above command set the 'stdout' value to the variable `MY_OUTPUT`.
3939
4040
Run Fluent Bit with the recently created configuration file:
4141

‎filter/README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ The _filter plugins_ allows to **alter** the incoming data generated by the _inp
44

55
| name | title | description |
66
| :--- | :--- | :--- |
7+
| [aws](aws.md) | AWS | Enrich logs with AWS Metadata. |
78
| [grep](grep.md) | Grep | Match or exclude specific records by patterns. |
89
| [kubernetes](kubernetes.md) | Kubernetes | Enrich logs with Kubernetes Metadata. |
910
| [lua](lua.md) | Lua | Filter records using Lua Scripts. |
@@ -15,4 +16,3 @@ The _filter plugins_ allows to **alter** the incoming data generated by the _inp
1516
| [modify](modify.md) | Modify | Modifications to record. |
1617

1718
In order to let a Filter be applied over some data, the **Match** rule must exists and it must match the Tag for the incoming data.
18-

‎filter/aws.md

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# AWS
2+
3+
The _AWS Filter_ Enriches logs with AWS Metadata. Currently the plugin adds the EC2 instance ID and availability zone to log records. To use this plugin, you must be running in EC2 and have the [instance metadata service enabled](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/configuring-instance-metadata-service.html).
4+
5+
## Configuration Parameters
6+
7+
The plugin supports the following configuration parameters:
8+
9+
| Key | Value Format | Description |
10+
| :--- | :--- | :--- |
11+
| imds_version | VERSION | Specify which version of the instance metadata service to use. Valid values are 'v1' or 'v2'; 'v2' is the default. |
12+
13+
Note: *If you run Fluent Bit in a container, you may have to use instance metadata v1.* The plugin behaves the same regardless of which version is used.
14+
15+
## Usage
16+
17+
### Command Line
18+
19+
```
20+
$ bin/fluent-bit -i dummy -F aws -m '*' -o stdout
21+
22+
[2020/01/17 07:57:17] [ info] [engine] started (pid=32744)
23+
[0] dummy.0: [1579247838.000171227, {"message"=>"dummy", "az"=>"us-west-2b", "ec2_instance_id"=>"i-06bc83dbc2ac2fdf8"}]
24+
[1] dummy.0: [1579247839.000125097, {"message"=>"dummy", "az"=>"us-west-2b", "ec2_instance_id"=>"i-06bc87dbc2ac3fdf8"}]
25+
```
26+
27+
### Configuration File
28+
29+
```
30+
[INPUT]
31+
Name dummy
32+
Tag dummy
33+
34+
[FILTER]
35+
Name aws
36+
Match *
37+
imds_version v1
38+
39+
[OUTPUT]
40+
Name stdout
41+
Match *
42+
```

‎input/README.md

+2
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ The _input plugins_ defines the source from where [Fluent Bit](http://fluentbit.
77
| [collectd](collectd.md) | Collectd | Listen for UDP packets from Collectd. |
88
| [cpu](cpu.md) | CPU Usage | measure total CPU usage of the system. |
99
| [disk](disk.md) | Disk Usage | measure Disk I/Os. |
10+
| [docker](docker.md) | Docker | collect Docker container metrics. |
1011
| [dummy](dummy.md) | Dummy | generate dummy event. |
1112
| [exec](exec.md) | Exec | executes external program and collects event logs. |
1213
| [forward](forward.md) | Forward | Fluentd forward protocol. |
@@ -20,6 +21,7 @@ The _input plugins_ defines the source from where [Fluent Bit](http://fluentbit.
2021
| [random](random.md) | Random | Generate Random samples. |
2122
| [serial](serial.md) | Serial Interface | read data information from the serial interface. |
2223
| [stdin](stdin.md) | Standard Input | read data from the standard input. |
24+
| [statsd](statsd.md) | StatsD | receive metrics in StatsD protocol. |
2325
| [syslog](syslog.md) | Syslog | read syslog messages from a Unix socket. |
2426
| [systemd](systemd.md) | Systemd | read logs from Systemd/Journald. |
2527
| [tail](tail.md) | Tail | Tail log files |

‎input/collectd.md

+3-3
Original file line numberDiff line numberDiff line change
@@ -5,15 +5,15 @@ The **collectd** input plugin allows you to receive datagrams from collectd.
55
Content:
66

77
* [Configuration Parameters](collectd.md#config)
8-
* [Configuration Parameters](collectd.md#config_example)
8+
* [Configuration Examples](collectd.md#config_example)
99

1010
## Configuration Parameters {#config}
1111

1212
The plugin supports the following configuration parameters:
1313

1414
| Key | Description | Default |
1515
| :------- | :------------------------------ | :--------------------------- |
16-
| Address | Set the address to listen to | 0.0.0.0 |
16+
| Listen | Set the address to listen to | 0.0.0.0 |
1717
| Port | Set the port to listen to | 25826 |
1818
| TypesDB | Set the data specification file | /usr/share/collectd/types.db |
1919

@@ -24,7 +24,7 @@ Here is a basic configuration example.
2424
```python
2525
[INPUT]
2626
Name collectd
27-
Address 0.0.0.0
27+
Listen 0.0.0.0
2828
Port 25826
2929
TypesDB /usr/share/collectd/types.db,/etc/collectd/custom.db
3030

‎input/docker.md

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# Docker
2+
3+
The **docker** input plugin allows you to collect Docker container metrics
4+
like memory usage and CPU consumption.
5+
6+
Content:
7+
8+
* [Configuration Parameters](docker.md#config)
9+
* [Configuration Parameters](docker.md#config_example)
10+
11+
## Configuration Parameters {#config}
12+
13+
The plugin supports the following configuration parameters:
14+
15+
| Key | Description | Default |
16+
| :------------ | :---------------------------------------------- | :--------|
17+
| Interval\_Sec | Polling interval in seconds | 1 |
18+
| Include | A space-separated list of containers to include | |
19+
| Exclude | A space-separated list of containers to exclude | |
20+
21+
If you set neither `Include` nor `Exclude`, the plugin will try to get
22+
metrics from _all_ the running containers.
23+
24+
## Configuration Examples {#config_example}
25+
26+
Here is an example configuration that collects metrics from two docker
27+
instances (`6bab19c3a0f9` and `14159be4ca2c`).
28+
29+
```python
30+
[INPUT]
31+
Name docker
32+
Include 6bab19c3a0f9 14159be4ca2c
33+
34+
[OUTPUT]
35+
Name stdout
36+
Match *
37+
```
38+
39+
This configuration will produce records like below.
40+
41+
```
42+
[1] docker.0: [1571994772.00555745, {"id"=>"6bab19c3a0f9", "name"=>"postgresql", "cpu_used"=>172102435, "mem_used"=>5693400, "mem_limit"=>4294963200}]
43+
```

‎input/exec.md

+2
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ The plugin supports the following configuration parameters:
1212
| Parser | Specify the name of a parser to interpret the entry as a structured message. |
1313
| Interval\_Sec | Polling interval \(seconds\). |
1414
| Interval\_NSec | Polling interval \(nanosecond\). |
15+
| Buf\_Size | Size of the buffer (check [unit sizes](https://docs.fluentbit.io/manual/configuration/unit_sizes) for allowed values) |
1516

1617
## Getting Started
1718

@@ -47,6 +48,7 @@ In your main configuration file append the following _Input_ & _Output_ sections
4748
Command ls /var/log
4849
Interval_Sec 1
4950
Interval_NSec 0
51+
Buf_Size 8mb
5052

5153
[OUTPUT]
5254
Name stdout

‎input/statsd.md

+42
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# StatsD
2+
3+
The **statsd** input plugin allows you to receive metrics via StatsD protocol.
4+
5+
Content:
6+
7+
* [Configuration Parameters](statsd.md#config)
8+
* [Configuration Examples](statsd.md#config_example)
9+
10+
## Configuration Parameters {#config}
11+
12+
The plugin supports the following configuration parameters:
13+
14+
| Key | Description | Default |
15+
| :------- | :--------------------------------------- | :------ |
16+
| Listen | Listener network interface. | 0.0.0.0 |
17+
| Port | UDP port where listening for connections | 8125 |
18+
19+
## Configuration Examples {#config_example}
20+
21+
Here is a configuration example.
22+
23+
```python
24+
[INPUT]
25+
Name statsd
26+
Listen 0.0.0.0
27+
Port 8125
28+
29+
[OUTPUT]
30+
Name stdout
31+
Match *
32+
```
33+
34+
Now you can input metrics through the UDP port as follows.
35+
36+
$ echo "click:10|c|@0.1" | nc -q0 -u 127.0.0.1 8125
37+
$ echo "active:99|g" | nc -q0 -u 127.0.0.1 8125
38+
39+
Fluent Bit will produce the following records:
40+
41+
[0] statsd.0: [1574905088.971380537, {"type"=>"counter", "bucket"=>"click", "value"=>10.000000, "sample_rate"=>0.100000}]
42+
[0] statsd.0: [1574905141.863344517, {"type"=>"gauge", "bucket"=>"active", "value"=>99.000000, "incremental"=>0}]

‎input/syslog.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,8 @@ The plugin supports the following configuration parameters:
1414
| Path | If _Mode_ is set to _unix\_tcp_ or _unix\_udp_, set the absolute path to the Unix socket file. | |
1515
| Unix_Perm | If _Mode_ is set to _unix\_tcp_ or _unix\_udp_, set the permission of the Unix socket file. | 0644 |
1616
| Parser | Specify an alternative parser for the message. By default, the plugin uses the parser _syslog-rfc3164_. If your syslog messages have fractional seconds set this Parser value to _syslog-rfc5424_ instead. | |
17-
| Buffer\_Chunk\_Size | By default the buffer to store the incoming Syslog messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Buffer\_Chunk\_Size_ in KB. If not set, _Buffer\_Chunk\_Size_ is equal to 32 \(32KB\). Read considerations below when using _udp_ or _unix\_udp_ mode. | |
18-
| Buffer\_Max_Size | Specify the maximum buffer size in KB to receive a Syslog message. If not set, the default size will be the value of _Buffer\_Chunk\_Size_. | |
17+
| Buffer\_Chunk\_Size | By default the buffer to store the incoming Syslog messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by _Buffer\_Chunk\_Size_. If not set, _Buffer\_Chunk\_Size_ is equal to 32000 bytes \(32KB\). Read considerations below when using _udp_ or _unix\_udp_ mode. | |
18+
| Buffer\_Max_Size | Specify the maximum buffer size to receive a Syslog message. If not set, the default size will be the value of _Buffer\_Chunk\_Size_. | |
1919

2020
### Considerations
2121

@@ -50,8 +50,8 @@ In your main configuration file append the following _Input_ & _Output_ sections
5050
[INPUT]
5151
Name syslog
5252
Path /tmp/in_syslog
53-
Buffer_Chunk_Size 32
54-
Buffer_Max_Size 64
53+
Buffer_Chunk_Size 32000
54+
Buffer_Max_Size 64000
5555

5656
[OUTPUT]
5757
Name stdout

‎input/tail.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ The plugin supports the following configuration parameters:
3232
| Mem\_Buf\_Limit | Set a limit of memory that Tail plugin can use when appending data to the Engine. If the limit is reach, it will be paused; when the data is flushed it resumes. | |
3333
| Parser | Specify the name of a parser to interpret the entry as a structured message. | |
3434
| Key | When a message is unstructured \(no parser applied\), it's appended as a string under the key name _log_. This option allows to define an alternative name for that key. | log |
35-
| Tag | Set a tag (with regex-extract fields) that will be placed on lines read. E.g. `kube.<namespace_name>.<pod_name>.<container_name>` | |
35+
| Tag | Set a tag (with regex-extract fields) that will be placed on lines read. E.g. `kube.<namespace_name>.<pod_name>.<container_name>`. Note that "tag expansion" is supported: if the tag includes an asterisk (*), that asterisk will be replaced with the absolute path of the monitored file (also see [Workflow of Tail + Kubernetes Filter](../filter/kubernetes.md)). | |
3636
| Tag_Regex | Set a regex to exctract fields from the file. E.g. `(?<pod_name>[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?<namespace_name>[^_]+)_(?<container_name>.+)-` | |
3737

3838
Note that if the database parameter _db_ is **not** specified, by default the plugin will start reading each target file from the beginning.

‎installation/build_install.md

+16-9
Original file line numberDiff line numberDiff line change
@@ -65,33 +65,39 @@ it's likely you may need root privileges so you can try to prefixing the command
6565

6666
## Build Options
6767

68-
Fluent Bit provides certain options to CMake that can be enabled or disabled when configuring, please refer to the following tables under the _General Options_, _Input Plugins_ and _Output Plugins_ sections.
68+
Fluent Bit provides certain options to CMake that can be enabled or disabled when configuring, please refer to the following tables under the _General Options_, _Development Options_, Input Plugins_ and _Output Plugins_ sections.
6969

7070
### General Options
7171

7272
| option | description | default |
7373
| :--- | :--- | :--- |
7474
| FLB\_ALL | Enable all features available | No |
75-
| FLB\_DEBUG | Build binaries with debug symbols | No |
7675
| FLB\_JEMALLOC | Use Jemalloc as default memory allocator | No |
77-
| FLB\_TLS | Buils with SSL/TLS support | No |
76+
| FLB\_TLS | Build with SSL/TLS support | No |
7877
| FLB\_BINARY | Build executable | Yes |
7978
| FLB\_EXAMPLES | Build examples | Yes |
8079
| FLB\_SHARED\_LIB | Build shared library | Yes |
81-
| FLB\_VALGRIND | Enable Valgrind support | No |
82-
| FLB\_TRACE | Enable trace mode | No |
83-
| FLB_TESTS_RUNTIME | Enable runtime tests | No |
84-
| FLB_TESTS_INTERNAL | Enable internal tests | No |
85-
| FLB\_TESTS | Enable tests | No |
8680
| FLB\_MTRACE | Enable mtrace support | No |
8781
| FLB_INOTIFY | Enable Inotify support | Yes |
8882
| FLB\_POSIX\_TLS | Force POSIX thread storage | No |
8983
| FLB_SQLDB | Enable SQL embedded database support | No |
9084
| FLB_HTTP_SERVER | Enable HTTP Server | No |
91-
| FLB_BACKTRACE | Enable backtrace/stacktrace support | Yes |
9285
| FLB_LUAJIT | Enable Lua scripting support | Yes |
9386
| FLB_STATIC_CONF | Build binary using static configuration files. The value of this option must be a directory containing configuration files. | |
9487

88+
### Development Options
89+
90+
| option | description | default |
91+
| :--- | :--- | :--- |
92+
| FLB\_DEBUG | Build binaries with debug symbols | No |
93+
| FLB\_VALGRIND | Enable Valgrind support | No |
94+
| FLB\_TRACE | Enable trace mode | No |
95+
| FLB\_SMALL | Minimise binary size | No |
96+
| FLB_TESTS_RUNTIME | Enable runtime tests | No |
97+
| FLB_TESTS_INTERNAL | Enable internal tests | No |
98+
| FLB\_TESTS | Enable tests | No |
99+
| FLB_BACKTRACE | Enable backtrace/stacktrace support | Yes |
100+
95101
### Input Plugins
96102

97103
The _input plugins_ provides certain features to gather information from a specific source type which can be a network interface, some built-in metric or through a specific input device, the following input plugins are available:
@@ -108,6 +114,7 @@ The _input plugins_ provides certain features to gather information from a speci
108114
| [FLB\_IN\_SERIAL](../input/serial.md) | Enable Serial input plugin | On |
109115
| [FLB\_IN\_STDIN](../input/stdin.md) | Enable Standard input plugin | On |
110116
| FLB\_IN\_TCP | Enable TCP input plugin | On |
117+
| [FLB\_IN\_THERMAL](../input/thermal.md) | Enable system temperature(s) input plugin | On |
111118
| [FLB\_IN\_MQTT](../input/mqtt.md) | Enable MQTT input plugin | On |
112119
| [FLB\_IN\_XBEE](https://github.com/fluent/fluent-bit-docs/tree/ad9d80e5490bd5d79c86955c5689db1cb4cf89db/input/xbee.md) | Enable Xbee input plugin | Off |
113120

‎installation/redhat_centos.md

+5-3
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,15 @@ We provide **td-agent-bit** through a Yum repository. In order to add the reposi
1111
```text
1212
[td-agent-bit]
1313
name = TD Agent Bit
14-
baseurl = http://packages.fluentbit.io/centos/7
14+
baseurl = https://packages.fluentbit.io/centos/7
1515
gpgcheck=1
16-
gpgkey=http://packages.fluentbit.io/fluentbit.key
16+
gpgkey=https://packages.fluentbit.io/fluentbit.key
1717
enabled=1
1818
```
1919

20-
note: we encourage you always enable the _gpgcheck_ for security reasons. All our packages are signed.
20+
note: we encourage you always enable the _gpgcheck_ for security reasons. All our packages are signed.
21+
22+
The GPG Key fingerprint is `F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A`
2123

2224
### Install
2325

‎installation/td-agent-bit.md

+1
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ The following distributions are supported:
1111
| [Debian](debian.md) | 10 | Buster |
1212
| [Debian](debian.md) | 9 | Stretch |
1313
| [Debian](debian.md) | 8 | Jessie |
14+
| [Raspbian](raspberry_pi.md) | 9 | Stretch |
1415
| [Raspbian](raspberry_pi.md) | 8 | Jessie |
1516
| [CentOS](redhat_centos.md) | 7 |   |
1617

‎installation/upgrade_notes.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ If you are migrating from Fluent Bit v1.2 to v1.3, there are not breaking change
1212

1313
### Docker, JSON, Parsers and Decoders
1414

15-
On Fluent Bit v1.2 we have fixed many issues associated with JSON encoding and decoding, for hence when parsing Docker logs __is not longer necessary__ to use decoders. The new Docker parser looks like this:
15+
On Fluent Bit v1.2 we have fixed many issues associated with JSON encoding and decoding, for hence when parsing Docker logs __is no longer necessary__ to use decoders. The new Docker parser looks like this:
1616

1717
```
1818
[PARSER]
@@ -68,7 +68,7 @@ If you are upgrading from **Fluent Bit <= 1.0.x** you should take in considerati
6868

6969
We introduced a new configuration property called _Kube\_Tag\_Prefix_ to help Tag prefix resolution and address an unexpected behavior that landed in previous versions.
7070

71-
Duing 1.0.x release cycle, a commit in Tail input plugin changed the default behavior on how the Tag was composed when using the wildcard for expansion generating breaking compatibility with other services. Consider the following configuration example:
71+
During 1.0.x release cycle, a commit in Tail input plugin changed the default behavior on how the Tag was composed when using the wildcard for expansion generating breaking compatibility with other services. Consider the following configuration example:
7272

7373
```
7474
[INPUT]

0 commit comments

Comments
 (0)