Fluentd, the Open Source Data Collection tool

Fluentd is an enterprise open source tool to collect and unify logs in a structured way. It allows to collect data from different sources such as the file system, syslog, TCP and HTTP within many others. Once the data is collected, it provides several ways to modify and route the data in a structured way to backends or cloud services in a reliable manner, common use cases are aggregating logs into NoSQL databases.

Fluentd is used widely by companies and individuals who needs to solve their logging layer scalability problems, it design provides a flexible and pluggable architecture where is possible to extend it easily for any custom case.

On this presentation, I will cover the internals of Fluentd and it different internals interfaces such as: input, parser, filter, buffer, output and formatter. Also I will introduce our native Fluentd support landed in Docker v1.8 and provide several use cases that will benefit system administrators and application developers who needs to unify their data with fault-tolerance and zero data loss capabilities.