Creating and Deploying
Learn how to create and deploy a pipeline
To create a pipeline you need to use the Calyptia Cloud CLI. First create a configuration, in the following example
pipeline-es.conf we will read data from a TCP forward port and output that data to an Elasticsearch endpointpipeline-es.conf
[INPUT]
Name forward
Host 0.0.0.0
Port 24284
[OUTPUT]
Name es
Match *
Host hostname.us-east-2.es.amazonaws.com
HTTP_User es
HTTP_Passwd 123123
Port 443
TLS on
After a pipeline configuration is created we can run the following to deploy
calyptia create pipeline --aggregator <AGGREGATOR ID> --config-file pipeline-es.conf
The pipeline auto-creates and we can retrieve the auto-generated name for the pipeline
NAME AGE
honest-sunfire-cc22 125 milliseconds
As part of a pipeline deployment, a pipeline is automatically validated.
Similar to Fluent Bit configuration there are many options you can also specify to customize the behavior and meet your requirements for your observability pipelines
These include the following
- Resource Profiles
- Secrets
- Parsers
- Referencing External Files
Pipelines can be updated at any time by running the following command
calyptia update pipeline <PIPELINE NAME>
Usage:
calyptia update pipeline PIPELINE [flags]
Flags:
--auto-create-ports Automatically create pipeline ports from config if updated (default true)
--config-file string New Fluent Bit config file used by pipeline
-h, --help help for pipeline
--new-name string New pipeline name
--output-format string Output format. Allowed: table, json (default "table")
--replicas uint New pipeline replica size
You can run the following to delete a Pipeline
calyptia delete pipeline <PIPELINE NAME OR ID>