1.4: migrating to new structure (wip)

Signed-off-by: Eduardo Silva <[email protected]>

Author: edsiper

.gitignore

@@ -1,2 +1,5 @@
 _book/*
 node_modules/*
+.gitignore
+old/*
+

README.md

@@ -1,7 +1,41 @@
-# Introduction
+---
+description: Next generation Log and Stream processor
+---
+
+# Fluent Bit v1.4 Documentation
+
+![](.gitbook/assets/logo_documentation_1.4.png)
+
+[Fluent Bit](http://fluentbit.io) is a Fast and Lightweight Log Processor, Stream Processor and Forwarder for Linux, OSX, Windows and BSD family operating systems. It has been made with a strong focus on performance to allow the collection of events from different sources without complexity.
+
+### Features
+
+* High Performance
+* Data Parsing
+  * Convert your unstructured messages using our parsers: [JSON](pipeline/parsers/json.md), [Regex](pipeline/parsers/regular-expression.md), [LTSV](pipeline/parsers/ltsv.md) and [Logfmt](pipeline/parsers/logfmt.md)
+* Reliability and Data Integrity
+  * [Backpressure](administration/backpressure.md) Handling
+  * [Data Buffering](administration/buffering-and-storage.md) in memory and file system
+* Networking
+  * Security: built-in TLS/SSL support
+  * Asynchronous I/O
+* Pluggable Architecture and [Extensibility](development/library_api.md): Inputs, Filters and Outputs
+  * More than 50 built-in plugins available
+  * Extensibility
+    * Write any input, filter or output plugin in C language
+    * Bonus: write [Filters in Lua](pipeline/filters/lua.md) or [Output plugins in Golang](development/golang-output-plugins.md)
+* [Monitoring](administration/monitoring.md): expose internal metrics over HTTP in JSON and [Prometheus](https://prometheus.io/) format
+* [Stream Processing](stream-processing/untitled.md): Perform data selection and transformation using simple SQL queries
+  * Create new streams of data using query results
+  * Aggregation Windows
+  * Data analysis and prediction: Timeseries forecasting
+* Portable: runs on Linux, MacOS, Windows and BSD systems
+
+### Fluent Bit, Fluentd and CNCF
+
+[Fluent Bit](http://fluentbit.io) is a sub-component of the [Fluentd](http://fluentd.org) project ecosystem, it's licensed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0). This project was created by [Treasure Data](https://www.treasuredata.com) and is its current primary sponsor.
+
+Nowadays Fluent Bit get contributions from several companies and individuals and same as [Fluentd](https://www.fluentd.org), it's hosted as a [CNCF](https://cncf.io) subproject.
 
-![](imgs/logo_documentation_1.2.png)
 
-[Fluent Bit](http://fluentbit.io) is a Fast and Lightweight Log Processor and Forwarder for Linux, OSX and BSD family operating systems. It has been made with a strong focus on performance to allow the collection of events from different sources without complexity.
 
-[Fluent Bit](http://fluentbit.io) is part of the [Fluentd](http://fluentd.org) project ecosystem, it's licensed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0). This project is made and sponsored by [Treasure Data](https://www.treasuredata.com).

SUMMARY.md

@@ -1,113 +1,133 @@
 # Table of contents
 
-* [Introduction](README.md)
-* [About](about/README.md)
-  * [Why ?](about/why.md)
-  * [Fluentd & Fluent Bit](about/fluentd_and_fluentbit.md)
-  * [License](about/license.md)
-* [Installation](installation/README.md)
-  * [Supported Platforms](installation/supported_platforms.md)
-  * [Requirements](installation/requirements.md)
-  * [Download Sources](installation/download_sources.md)
-  * [Upgrade Notes](installation/upgrade_notes.md)
-  * [Build and Install](installation/build_install.md)
-  * [Build with Static Configuration](installation/build_static_configuration.md)
-  * [Docker Images](installation/docker.md)
-  * [Kubernetes](installation/kubernetes.md)
-  * [TD Agent Bit](installation/td-agent-bit.md)
-  * [Debian Packages](installation/debian.md)
-  * [Ubuntu Packages](installation/ubuntu.md)
-  * [CentOS Packages](installation/redhat_centos.md)
-  * [Raspberry Pi](installation/raspberry_pi.md)
-  * [Yocto Project](installation/yocto.md)
-  * [Windows](installation/windows.md)
-* [Getting Started](getting_started/README.md)
-  * [Service](getting_started/service.md)
-  * [Input](getting_started/input.md)
-  * [Parser](getting_started/parser.md)
-  * [Filter](getting_started/filter.md)
-  * [Buffer](getting_started/buffer.md)
-  * [Routing](getting_started/routing.md)
-  * [Output](getting_started/output.md)
-* [Configuration](configuration/README.md)
-  * [Configuration Schema](configuration/schema.md)
-  * [Configuration File](configuration/file.md)
-  * [Configuration Variables](configuration/variables.md)
-  * [Configuration Commands](configuration/commands.md)
-  * [Buffering / Storage](configuration/buffering.md)
-  * [Monitoring](configuration/monitoring.md)
-  * [Unit Sizes](configuration/unit_sizes.md)
-  * [TLS / SSL](configuration/tls_ssl.md)
-  * [Backpressure](configuration/backpressure.md)
-  * [Memory Usage](configuration/memory_usage.md)
-  * [Upstream Servers](configuration/upstream_servers.md)
-  * [Scheduler](configuration/scheduler.md)
-  * [Stream Processor](configuration/stream_processor.md)
-* [Service](service/README.md)
-* [Input Plugins](input/README.md)
-  * [Collectd](input/collectd.md)
-  * [CPU Usage](input/cpu.md)
-  * [Disk Usage](input/disk.md)
-  * [Docker](input/docker.md)
-  * [Dummy](input/dummy.md)
-  * [Exec](input/exec.md)
-  * [Forward](input/forward.md)
-  * [Head](input/head.md)
-  * [Health](input/health.md)
-  * [Kernel Log Buffer](input/kmsg.md)
-  * [Memory Usage](input/mem.md)
-  * [MQTT](input/mqtt.md)
-  * [Network Traffic](input/netif.md)
-  * [Process](input/proc.md)
-  * [Random](input/random.md)
-  * [Serial Interface](input/serial.md)
-  * [Standard Input](input/stdin.md)
-  * [StatsD Input](input/statsd.md)
-  * [Syslog](input/syslog.md)
-  * [Systemd](input/systemd.md)
-  * [Tail](input/tail.md)
-  * [TCP](input/tcp.md)
-  * [Thermal](input/thermal.md)
-  * [Windows Event Log](input/winlog.md)
-* [Parsers](parser/README.md)
-  * [JSON Parser](parser/json.md)
-  * [Regular Expression Parser](parser/regular_expression.md)
-  * [LTSV Parser](parser/ltsv.md)
-  * [Logfmt Parser](parser/logfmt.md)
-  * [Decoders](parser/decoder.md)
-* [Filter Plugins](filter/README.md)
-  * [AWS](filter/aws.md)
-  * [Grep](filter/grep.md)
-  * [Kubernetes](filter/kubernetes.md)
-  * [Lua](filter/lua.md)
-  * [Parser](filter/parser.md)
-  * [Record Modifier](filter/record_modifier.md)
-  * [Standard Output](filter/stdout.md)
-  * [Throttle](filter/throttle.md)
-  * [Nest](filter/nest.md)
-  * [Modify](filter/modify.md)
-* [Output Plugins](output/README.md)
-  * [Azure](output/azure.md)
-  * [BigQuery](output/bigquery.md)
-  * [Counter](output/counter.md)
-  * [Datadog](output/datadog.md)
-  * [Elasticsearch](output/elasticsearch.md)
-  * [File](output/file.md)
-  * [FlowCounter](output/flowcounter.md)
-  * [Forward](output/forward.md)
-  * [GELF](output/gelf.md)
-  * [HTTP](output/http.md)
-  * [InfluxDB](output/influxdb.md)
-  * [Kafka](output/kafka.md)
-  * [Kafka REST Proxy](output/kafka-rest-proxy.md)
-  * [NATS](output/nats.md)
-  * [Null](output/null.md)
-  * [Stackdriver](output/stackdriver.md)
-  * [Standard Output](output/stdout.md)
-  * [Splunk](output/splunk.md)
-  * [TCP & TLS](output/tcp.md)
-  * [Treasure Data](output/td.md)
-* [Fluent Bit for Developers](development/README.md)
-  * [Library API](development/library_api.md)
-  * [Ingest Records Manually](development/ingest_records_manually.md)
-  * [Fluent Bit and Golang Plugins](development/golang_plugins.md)
+* [Fluent Bit v1.4 Documentation](README.md)
+
+## About
+
+* [What is Fluent Bit ?](about/what-is-fluent-bit.md)
+* [A Brief History of Fluent Bit](about/history.md)
+* [Fluentd & Fluent Bit](about/fluentd-and-fluent-bit.md)
+* [License](about/license.md)
+
+## Concepts
+
+* [Key Concepts](concepts/key-concepts.md)
+* [Buffering](concepts/buffering.md)
+* [Data Pipeline](concepts/data-pipeline/README.md)
+  * [Input](concepts/data-pipeline/input.md)
+  * [Parser](concepts/data-pipeline/parser.md)
+  * [Filter](concepts/data-pipeline/filter.md)
+  * [Buffer](concepts/data-pipeline/buffer.md)
+  * [Router](concepts/data-pipeline/router.md)
+  * [Output](concepts/data-pipeline/output.md)
+
+## Installation
+
+* [Upgrade Notes](installation/upgrade-notes.md)
+* [Supported Platforms](installation/untitled.md)
+* [Requirements](installation/requirements.md)
+* [Sources](installation/sources/README.md)
+  * [Download Source Code](installation/sources/download-source-code.md)
+  * [Build and Install](installation/sources/build-and-install.md)
+  * [Build with Static Configuration](installation/sources/build-with-static-configuration.md)
+* [Linux Packages](installation/linux/README.md)
+  * [Amazon Linux](installation/linux/amazon-linux.md)
+  * [CentOS](installation/linux/centos.md)
+  * [Debian](installation/linux/debian.md)
+  * [Ubuntu](installation/linux/ubuntu.md)
+  * [Raspbian / Raspberry Pi](installation/linux/raspbian-raspberry-pi.md)
+* [Docker](installation/docker.md)
+* [Kubernetes](installation/kubernetes.md)
+* [Yocto / Embedded Linux](installation/yocto-embedded-linux.md)
+* [Windows](installation/windows.md)
+
+## Administration
+
+* [Configuring Fluent Bit](administration/configuring-fluent-bit/README.md)
+  * [Files Schema / Structure](administration/configuring-fluent-bit/files-schema-structure/README.md)
+    * [Variables](administration/configuring-fluent-bit/files-schema-structure/variables.md)
+    * [Commands](administration/configuring-fluent-bit/files-schema-structure/commands.md)
+  * [Configuration File](administration/configuring-fluent-bit/main-configuration-file.md)
+  * [Upstream Servers](administration/configuring-fluent-bit/upstream-servers.md)
+  * [Unit Sizes](administration/configuring-fluent-bit/unit-sizes.md)
+* [Security](administration/security.md)
+* [Buffering & Storage](administration/buffering-and-storage.md)
+* [Backpressure](administration/backpressure.md)
+* [Memory Management](administration/memory-management.md)
+* [Monitoring](administration/monitoring.md)
+* [Scheduling and Retries](administration/scheduling-and-retries.md)
+
+## Data Pipeline <a id="pipeline"></a>
+
+* [Inputs](pipeline/inputs/README.md)
+  * [Collectd](pipeline/inputs/collectd.md)
+  * [CPU Metrics](pipeline/inputs/cpu-metrics.md)
+  * [Disk I/O Metrics](pipeline/inputs/disk-i-o-metrics.md)
+  * [Dummy](pipeline/inputs/dummy.md)
+  * [Exec](pipeline/inputs/exec.md)
+  * [Forward](pipeline/inputs/forward.md)
+  * [Head](pipeline/inputs/head.md)
+  * [Health](pipeline/inputs/health.md)
+  * [Kernel Logs](pipeline/inputs/kernel-logs.md)
+  * [Memory Metrics](pipeline/inputs/memory-metrics.md)
+  * [MQTT](pipeline/inputs/mqtt.md)
+  * [Network I/O Metrics](pipeline/inputs/network-i-o-metrics.md)
+  * [Process](pipeline/inputs/process.md)
+  * [Random](pipeline/inputs/random.md)
+  * [Serial Interface](pipeline/inputs/serial-interface.md)
+  * [Standard Input](pipeline/inputs/standard-input.md)
+  * [Syslog](pipeline/inputs/syslog.md)
+  * [Systemd](pipeline/inputs/systemd.md)
+  * [Tail](pipeline/inputs/tail.md)
+  * [TCP](pipeline/inputs/tcp.md)
+  * [Thermal](pipeline/inputs/thermal.md)
+  * [Windows Event Log](pipeline/inputs/windows-event-log.md)
+* [Parsers](pipeline/parsers/README.md)
+  * [JSON](pipeline/parsers/json.md)
+  * [Regular Expression](pipeline/parsers/regular-expression.md)
+  * [LTSV](pipeline/parsers/ltsv.md)
+  * [Logfmt](pipeline/parsers/logfmt.md)
+  * [Decoders](pipeline/parsers/decoders.md)
+* [Filters](pipeline/filters/README.md)
+  * [AWS Metadata](pipeline/filters/aws-metadata.md)
+  * [Grep](pipeline/filters/grep.md)
+  * [Kubernetes](pipeline/filters/kubernetes.md)
+  * [Lua](pipeline/filters/lua.md)
+  * [Parser](pipeline/filters/parser.md)
+  * [Record Modifier](pipeline/filters/record-modifier.md)
+  * [Standard Output](pipeline/filters/standard-output.md)
+  * [Throttle](pipeline/filters/throttle.md)
+  * [Nest](pipeline/filters/nest.md)
+  * [Modify](pipeline/filters/modify.md)
+* [Outputs](pipeline/outputs/README.md)
+  * [Azure](pipeline/outputs/azure.md)
+  * [BigQuery](pipeline/outputs/bigquery.md)
+  * [Counter](pipeline/outputs/counter.md)
+  * [Datadog](pipeline/outputs/datadog.md)
+  * [Elasticsearch](pipeline/outputs/elasticsearch.md)
+  * [File](pipeline/outputs/file.md)
+  * [FlowCounter](pipeline/outputs/flowcounter.md)
+  * [Forward](pipeline/outputs/forward.md)
+  * [GELF](pipeline/outputs/gelf.md)
+  * [HTTP](pipeline/outputs/http.md)
+  * [InfluxDB](pipeline/outputs/influxdb.md)
+  * [Kafka](pipeline/outputs/kafka.md)
+  * [Kafka REST Proxy](pipeline/outputs/kafka-rest-proxy.md)
+  * [NATS](pipeline/outputs/nats.md)
+  * [NULL](pipeline/outputs/null.md)
+  * [Stackdriver](pipeline/outputs/stackdriver.md)
+  * [Standard Output](pipeline/outputs/standard-output.md)
+  * [Splunk](pipeline/outputs/splunk.md)
+  * [TCP & TLS](pipeline/outputs/tcp-and-tls.md)
+  * [Treasure Data](pipeline/outputs/treasure-data.md)
+
+## Stream Processing
+
+* [FIXME](stream-processing/untitled.md)
+
+## Fluent Bit for Developers <a id="development"></a>
+
+* [C Library API](development/library_api.md)
+* [Ingest Records Manually](development/ingest-records-manually.md)
+* [Golang Output Plugins](development/golang-output-plugins.md)
+

about/README.md

@@ -0,0 +1,2 @@
+# Fluentd & Fluent Bit
+

about/fluentd-and-fluent-bit.md

@@ -0,0 +1,10 @@
+---
+description: Every project has a story
+---
+
+# A Brief History of Fluent Bit
+
+On 2014,  the [Fluentd](https://fluentd.org) team at [Treasure Data](https://www.treasuredata.com) forecasted the need of a lightweight log processor for constraint environments like Embedded Linux and Gateways, the project aimed to be part of the Fluentd Ecosystem and we called it [Fluent Bit](https://fluentbit.io), fully open source and available under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
+
+After the project was around for some time, it got some traction in the Embedded market but we also started getting requests for several features from the Cloud community like more inputs, filters, and outputs. Not so long after that, Fluent Bit becomes one of the preferred solutions to solve the logging challenges in Cloud environments.
+