Configuring Parser
Parsers are an important component of Calyptia Calyptia Fluent Bit, with them you can take any unstructured log entry and give them a structure that makes easier it processing and further filtering.
The parser engine is fully configurable and can process log entries based in two types of format:
Regular Expressions (named capture)
By default, Calyptia Fluent Bit provides a set of pre-configured parsers that can be used for different use cases such as logs from:
Apache
Nginx
Docker
Syslog rfc5424
Syslog rfc3164
Parsers are defined in one or multiple configuration files that are loaded at start time, either from the command line or through the main Calyptia Fluent Bit configuration file.
Note: If you are using Regular Expressions note that Calyptia Fluent Bit uses Ruby based regular expressions and we encourage to use Rubular web site as an online editor to test them.
Configuration Parameters
Multiple parsers can be defined and each section has it own properties. The following table describes the available options for each parser definition:
Parsers Configuration File
All parsers must be defined in a parsers.conf file, not in the Calyptia Fluent Bit global configuration file. The parsers file expose all parsers available that can be used by the Input plugins that are aware of this feature. A parsers file can have multiple entries like this:
For more information about the parsers available, please refer to the default parsers file distributed with Calyptia Fluent Bit source code:
https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf
Time Resolution and Fractional Seconds
Time resolution and its format supported are handled by using the strftime(3) libc system function.
In addition, we extended our time resolution to support fractional seconds like 2017-05-17T15:44:31**.187512963**Z. Since Calyptia Fluent Bit v0.12 we have full support for nanoseconds resolution, the %L format option for Time_Format is provided as a way to indicate that content must be interpreted as fractional seconds.
Note: The option %L is only valid when used after seconds (
%S
) or seconds since the Epoch (%s
), e.g:%S.%L
or%s.%L
Last updated