OpenSearch destination plugin

OpenSearch is a distributed, scalable, and highly available search and analytics engine that is built on Apache Lucene and Elasticsearch. It provides a powerful platform for storing, indexing, and querying large amounts of data, including logs, metrics, traces, and other types of telemetry data.

With the OpenSearch destination plugin, you can configure your Calyptia Core pipeline to send your data to OpenSearch.

Configuration parameters

The OpenSearch destination plugin provides these configuration parameters.




IP address or hostname of the target OpenSearch instance.


TCP port of the target OpenSearch instance.


Index name.

Logstash format

Enable Logstash format compatibility.

Security and TLS



Enable or disable TLS/SSL support.

TLS Certificate Validation

Turn TLS/SSL certificate validation on or off. TLS must be on for this setting to be enabled.

TLS Debug Level

Set TLS debug verbosity level. Accepts these values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational), 4 (Verbose).

CA Certificate File Path

Absolute path to CA certificate file.

Certificate File Path

Absolute path to certificate file.

Private key File Path

Absolute path to private key file.

Private Key Path Password

Optional password for tls.key_file file.

TLS SNI Hostname Extension

Hostname to be used for TLS SNI extension.



Replace Dots

When enabled, replace field name dots with underscore, required by versions of OpenSearch.


Type Name.

Suppress Type Name

If true, mapping types are removed. for v7.0.0 or later.

Buffer Size

Specify the buffer size used to read the response from the OpenSearch HTTP service. This option is useful for debugging purposes where is required to read full responses. Response size grows depending of the number of records inserted. To set an unlimited amount of memory set this value to 'false', otherwise the value must be according to the Unit Size specification


OpenSearch accepts new data on HTTP query path /_bulk. But it is also possible to serve Elasticsearch behind a reverse proxy on a sub path. This option defines such path on the fluent-bit side. It adds only a path prefix in the indexing HTTP POST URI.


Newer versions of OpenSearch allows to setup filters called pipelines. This option allows to define which pipeline the database should use. For performance reasons is strongly suggested to do parsing and filtering on Fluent Bit side, avoid pipelines.

Generate ID

When enabled, generate _id for outgoing records. This prevents duplicate records when retrying ES.

Write Operation

Operation to use to write in bulk requests.

ID Key

If set, _id will be the value of the key from incoming record.

Replace Dots

Use current time for index generation instead of message record.

Logstash Prefix

When Logstash_Format is enabled, the Index name is composed using a prefix and the date. For example, if Logstash_Prefix is equals to mydata your index will become mydata-YYYY.MM.DD. The last string appended belongs to the date when the data is being generated'.

Logstash Prefix Key

When included: the value in the record that belongs to the key will be looked up and over-write the Logstash_Prefix for index generation. If the key/value is not found in the record then the Logstash_Prefix option will act as a fallback. Nested keys are supported through record accessor pattern.

Logstash Date Format

Time format (based on strftime) to generate the second part of the Index name.

Time Key

When Logstash_Format is enabled, each record will get a new timestamp field. The Time_Key property defines the name of that field.

Time Key Format

When Logstash_Format is enabled, this property defines the format of the timestamp.

Time Key Nanos

When Logstash_Format is enabled, enabling this property sends nanosecond precision timestamps.

Include Tag Key

When enabled, it append the Tag name to the record.

Tag Key

When Include_Tag_key is enabled, this property defines the key name of the tag in the message.

Basic authentication


HTTP Username

Basic Auth Username.

HTTP Password

Basic Auth Password. Requires HTTP_User to be set.

AWS authentication


Enable AWS Authentication

Enable AWS Sigv4 Authentication.

AWS Region

AWS Region of your service.

AWS STS Endpoint

Custom endpoint for the AWS STS API, used with the AWS_Role_ARN option.

AWS IAM Assume Role

ARN of an IAM role to assume (ex. for cross-account access).

AWS External ID

Specify an external ID for the STS API, can be used with the aws_role_arn parameter if your role requires an external ID.

Advanced networking


DNS Mode

Select the primary DNS connection type (TCP or UDP).

DNS Resolver

Select the primary DNS connection type (TCP or UDP).

Prefer IPv4

Prioritize IPv4 DNS results when trying to establish a connection.


Enable or disable Keepalive support.

Keepalive Idle Timeout

Set maximum time allowed for an idle Keepalive connection.

Max Connect Timeout

Set maximum time allowed to establish a connection, this time includes the TLS handshake.

Max Connect Timeout Log Error

On connection timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message.

Max Keepalive Recycle

Set maximum number of times a keepalive connection can be used before it is retired.

Source Address

Specify network address to bind for data traffic.



Trace Output

When enabled, print the Elasticsearch API calls to stdout (for diag only).

Trace Error

When enabled, print the Elasticsearch exception to stderr (for diag only).

Last updated