When ingesting data with network sources or agents, you may want to scale the pipeline to process additional records. With Calyptia Core, all ingested data is automatically load balanced within a pipeline replica.
Within Advanced Settings, increase the number of replicas to the desired amount.
For replica count recommendations over 100 TB a day, contact the Calyptia team prior to scaling up to ensure proper stability.
Scaling up an existing Calyptia Core pipeline can be performed using the update command:
calyptia update pipeline <PIPELINE-NAME-OR-ID> --replicas <NUM OF REPLICAS>
To examine the number of replicas for a given pipeline, run:
calyptia get pipelines --core-instance <CORE INSTANCE NAME>