Kafka
Kafka source plugin
Apache Kafka is an open source distributed streaming platform that is used to collect, store, and process large volumes of real-time data streams. It's particularly well-suited for collecting telemetry data, which is the data produced by sensors and devices that monitor the performance and health of systems, machines, and applications.
Kafka is designed to handle high volumes of data in real time, making it a popular choice for applications that require low latency and high throughput. It uses a publish-subscribe model to collect and distribute data among multiple data producers and consumers. Data producers, such as sensors or applications, publish data to Kafka topics, while data consumers subscribe to these topics to receive the data.
You can use the Kafka source plugin to configure your Calyptia Core pipeline to collect data from your OpenTelemetry instances.
Configuration parameters
The Kafka source plugin provides these configuration parameters:
| Key | Description |
|---|---|
Brokers | Single of multiple lists of Kafka Brokers. For example, |
Topics | Single entry or list of topics separated by comma (,) that Calyptia Core uses to receive messages from Kafka. |
Minimum Queued Messages | Minimum number of messages per topic+partition Calyptia Core tries to maintain in the local consumer queue. |
Last updated