Creating and Deploying

Learn how to create and deploy a pipeline
To create a pipeline you need to use the Calyptia Cloud CLI. First create a configuration, in the following example pipeline-es.conf we will read data from a TCP forward port and output that data to an Elasticsearch endpoint
Name forward
Port 24284
Name es
Match *
HTTP_User es
HTTP_Passwd 123123
Port 443
TLS on

Deploying a Pipeline

After a pipeline configuration is created we can run the following to deploy
calyptia create pipeline --aggregator <AGGREGATOR ID> --config-file pipeline-es.conf
The pipeline auto-creates and we can retrieve the auto-generated name for the pipeline
honest-sunfire-cc22 125 milliseconds

Validating a Pipeline

As part of a pipeline deployment, a pipeline is automatically validated.

Advanced options

Similar to Fluent Bit configuration there are many options you can also specify to customize the behavior and meet your requirements for your observability pipelines
These include the following
  • Resource Profiles
  • Secrets
  • Parsers
  • Referencing External Files

Updating a Pipeline

Pipelines can be updated at any time by running the following command
calyptia update pipeline <PIPELINE NAME>
calyptia update pipeline PIPELINE [flags]
--auto-create-ports Automatically create pipeline ports from config if updated (default true)
--config-file string New Fluent Bit config file used by pipeline
-h, --help help for pipeline
--new-name string New pipeline name
--output-format string Output format. Allowed: table, json (default "table")
--replicas uint New pipeline replica size

Deleting a Pipeline

You can run the following to delete a Pipeline
calyptia delete pipeline <PIPELINE NAME OR ID>