Calyptia Core release notes

05-01-2024

Deprecations

The k8s, GCP and AWS providers have been deprecated from the Calyptia CLI. The update, delete operations within the three deprecated providers should now use the operator sub-command.

Now, to create a new core instance with Helm or Calyptia CLI, you will need to run the install operator command first as documented in the Getting started with operator.

13-10-2023

What's New?

Calyptia Core brings new exciting features with an even more scalable architecture that takes native Kubernetes experience to the next level. If you need centralized or edge data pipelines, Core is here for you. Follow the upgrade instructions to start using the latest version.

Calyptia Core deploys in hybrid mode, where the management plane lives in our cloud, but data processing happens in your environment. For more enterprise needs like financial or government needs, we are announcing a full on-prem version that you can enjoy in your favorite Virtual Private Cloud (VPC)

You can now install multiple core instances in different namespaces using only one core operator. This simplifies your deployments and reduces maintenance core on your engineers. A little engineering camaraderie from our engineers to yours!

Features

Calyptia Core powers any type of data pipeline, where it can be deployed inside your Kubernetes Cluster for aggregation and high power processing needs, or also deploy remote pipelines. With our new Core Agent built on top of Fluent Bit, create edge pipeline in your own VMs, bare metal or remote endpoint and manage them from a single interface.

Collecting logs or events from custom HTTP endpoints is becoming a challenge, and to facilitate our customers and users, we have launched a new flexible HTTP Collector that allows the use of custom URI, and dynamic compose logic based on a templating request and responses. You no longer need to write a custom collector by hand, just leverage this new functionality.

If your event-driven system is decoupled using AWS SQS as an event source, use the new Input S3 from SQS plugin to collect and process events from SQS and send them to an S3 Bucket for storage.

If you have a group of processing rules you and your team use often, create Processing Rules Templates

Enhancements and Fixes

View more pipelines per page? Yes, you can! We increased the number of pipelines you can view per page to make navigating your resources easier.

Spend less time worrying about syntax errors and more time on your configurations, as the new Core ignores whitespace in variables.

Your Core pods are now more resilient during configuration upgrades, thanks to our engineers.

We have now randomised the way we name our pipelines so improve the Kubernetes multi-user and multi-pipeline experience. Feel free creating multiple core instances and multiple pipelines without worrying about sync failures.

Deprecations

Some of our commands have been deprecated. Now, to create a new core instance with Helm or Calyptia CLI, you will need to run the install operator command first, then your create core-instance command;

calyptia install operator --wait &&
calyptia create core_instance operator --name {CORE_INSTANCE_NAME} --wait

The same goes for Helm CLI

helm install core-operator calyptia/core-operator --namespace calyptia --create-namespace &&
helm install {CORE_INSTANCE_NAME} calyptia/core-instance --namespace calyptia --create-namespace --set coreInstance={CORE_INSTANCE_NAME} --set

20-10-2023

What's New?

Look out for the new Test Connection button when creating your pipeline. The test connection button allows you check if your configured plugin can connect to your core instance.

New with the Cloud v.1.5.6, is the skipConfigValidation parameter which, when set to true, allows you to skip your Fluent Bit config validation when creating and updating your configuration.

Enhancements and Fixes

Our new Calyptia CLI v1.8.1 uninstall commands now makes sure to remove all resources following an uninstall.

27-10-2023

What's New?

Our hot new Signal Sciences Next-Gen WAF collector lets you route and aggregate your security data with your core instances. Backed by our http loader plugin, the Signal Sciences Next-Gen WAF is the first of many SAAS collectors we will be introducing in the coming releases.

You can now manage your fleets from the UI with our Fleet Manager. Manage your fleets; including creating, updating and deleting them right from the UI.

Enhancements and Fixes

Our pipelines now offers you more flexibility. Choose your preferred port type for your pipelines from LoadBalancer , ClusterIP or NodePort.

More on flexibility, while creating your core instance, you can also configure health checks for your pipelines. Set the enable-health-check-pipeline parameter to true to enable the your health check, health-check-pipeline-service-type sets the service type to LoadBalancer , ClusterIP or NodePort, and add a port number using health-check-pipeline-port parameters.

If decoupled and loosely dependent services excites you, this is for you. Our pipelines now have their secrets scoped to them. When creating a new pipeline, use the --secrets-file parameter to add your pipeline sensitive data as a secret.

Last updated