Processing rules
Add powerful transformations to your pipelines
Calyptia Core provides a set of processing rules that are built-in and ready to use. These enable you to transform your data in real time as it passes through your Telemetry pipeline.
Examples can include:
Adding a new field to every record
Blocking extraneous fields
Encoding or decoding CSV or other formats
Redacting or removing sensitive information
Processing rules use a set of actions to perform transformations on the data. These actions can work with any data type, allowing you to transform various data formats.
When you use the processing rules, you specify one or more actions that will be applied to your data as it flows through your pipeline. Each action represents a specific data transformation unit, allowing you to perform complex transformations efficiently. The collection of actions you choose for your pipeline will inform the processing rule for your pipeline.
This means that you can customize your pipeline to match the specific needs of your data and application, ensuring that you're getting the insights you need to make informed decisions.
You can specify processing rules for your pipeline using the drag-and-drop interface.
Add processing rules to your pipeline
You can add actions to your pipeline by using the user interface.
To add a processing rule to your pipeline with the user interface, select the pipeline to which you want to add a processing rule. From the pipeline overview page, click Edit in the top right corner.
Click the bare tile in the middle of your pipeline configuration from your Pipeline Configuration page.
You can start configuring your pipeline's processing rule by adding an action. To add an action, click the Add New tile at the right corner of your processing rule pop-up.
You can view a list of actions to include in your processing rule from the actions dropdown. Choose the actions you want to include and click Apply.
You can add multiple actions to your pipeline process rules, and after you're satisfied with your configuration, click Apply. Your pipeline will be updated immediately.
Use the toggle bar besides an Action to turn it on or off in your processing rule. This can be helpful when testing transformation logic.
Before applying your processing rules to your pipeline, you can test your actions by adding sample data to the Input Test box, then clicking Run. View the result of your data transformation in the Output box.
When using the input test, convert any text into JSON, or plain text using the toggle button beneath the input box. If you already have JSON, click the toggle and paste in the data as is.
Last updated
