Parsers

When reading data from multiple formats you may need to parse that data to either filter out based on certain characteristics or make the data more useful for your particular backend observability system. Calyptia Core makes use of parsers that let you add schema to any type of data.

Default Pipeline Configuration Parsers

Calyptia Fluent Bit Enterprise enables a set of default parsers to be used across the different pipeline configuration sections.

Each pipeline will create a file name parsers that defaults to the current fluent-bit default set of parsers as specified in https://github.com/fluent/fluent-bit/blob/master/conf/parsers.conf

Update the default parsers file

In case you need to add a custom parser, you can also update the default parser used by new pipelines. To update you can fetch the current contents of the parsers file with the following command.

calyptia get pipeline_file --pipeline PIPELINE --name parsers --only-contents > parsers.conf

From here, edit the file to add the new parsers and update the parsers file.

calyptia update pipeline_file --pipeline PIPELINE --file ./parsers.conf

Test out parsers

If you need to validate whether your parser will function as needed you can use the Calyptia Cloud Regex page to make sure that the parsers are performing as needed.

\