Links

Elasticsearch HTTP

Receive data from Elastic Beats, Agent, Logstash and other ES HTTP compatible agents
The Elasticsearch HTTP plugin allows you to recieve data from Beats, Elastic Agent, Logstash and other agents that can send data via an Elasticsearch HTTP bulk request. Using this plugin allows you to recieve the data in Calyptia Core to perform the following:
  1. 1.
    Reduce noisy or non-essential data
  2. 2.
    Modify, Enrich, or Redact incoming data
  3. 3.
    Route Elastic Agent, Beats, Logstash generated data to alternative backends (E.g. Azure Blob)

Configuration Parameters

The following are configuration Parameters for Elasticsearch HTTP Source Plugin.
Key
Description
Port
The TCP Port the pipeline should listen on.
The following are Advanced configuration Parameters for Elasticsearch HTTP Source Plugin.
Key
Description
Buffer Chunk Size
Set the chunk size for incoming messages
Buffer Max Size
This sets the chunk size for incoming incoming JSON messages. These chunks are then stored/managed in the space available by buffer_size.
The following are Security and TLS configuration Parameters for Elasticsearch HTTP Source Plugin.
Key
Description
TLS
Enable or Disable TLS/SSL support
TLS Certificate Validation
Turn TLS/SSL certificate validation on / off, TLS must be on for this setting to be enabled.
TLS Debug Level
Set TLS debug verbosity level. It accepts the following values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational), and 4 Verbose
CA Certificate File Path
Absolute path to CA certificate file
Certificate File Path
Absolute path to Certificate file
Private key File Path
Absolute path to private Key file
Private Key Path Password
Optional password for tls.key_file file
TLS SNI Hostname Extension
Hostname to be used for TLS SNI Extension

Elastic Agent Configuration

in the Elastic Agent configuratoin within the output for Elasticsearch two settings are essential to receive data. allow_older_versions and ilm . A sample configuration of Elastic Agent can be seen below
######################################
# Fleet configuration
######################################
outputs:
default:
type: elasticsearch
hosts: [127.0.0.1:9200]
allow_older_versions: true
ilm: false
Last modified 2mo ago