EKS Add On

Using Calyptia Core Agent directly through AWS marketplace

Flexible configuration

The Calyptia Core Agent builds on top of the vendor-neutral CNCF OSS Fluent Bit offering to support various inputs and outputs with inline processing too.

The default configuration of the add on provides basic output but the intention is this is customized to support whatever use case required.

Amazon Marketplace Integration

Calyptia Core Agent is provided as an AWS marketplace offering either by directly running the container images or using the dedicated EKS Add On.

Prerequisites

  1. Ensure you have subscribed to the Calyptia Core Agent via AWS marketplace.

  2. Install kubectl, AWS CLI and eksctl.

  3. Ensure you have access to an Amazon EKS cluster (or the ability to create one).

You need to replace $YOUR_CLUSTER_NAME and $AWS_REGION accordingly with your actual Amazon EKS cluster name and AWS region.

Provision OIDC and IRSA

The container requires ongoing access to the AWS Marketplace metering so to support this ensure the cluster has OIDC enabled and an appropriate IAM role attached to the service account (IRSA).

By default, the namespace and service account are set to calyptia-fluentbit. These can be changed for the Helm chart so update if appropriate but when using the EKS add on this will be the default.

Step 1: Create IAM policy

This service account needs to be assigned the AWS managed policy arn:aws:iam::aws:policy/AWSMarketplaceMeteringRegisterUsage (or an equivent one).

The AWS Create and attach your first customer managed policy tutorial provides more details.

Here is an example IAM policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "aws-marketplace:RegisterUsage"
                ],
                "Effect": "Allow",
                "Resource": "*"
        }
    ]
}

Step 2: Ensure OIDC provider is set up

The Amazon EKS cluster needs to have the IAM OIDC provider enabled.

The AWS Creating an IAM OIDC provider for your cluster documentation provides more details.

eksctl utils associate-iam-oidc-provider \
    --cluster "$YOUR_CLUSTER_NAME" --region "$AWS_REGION" --approve

Step 3: Create an IAM role for service account (IRSA)

The steps below help to automate these manual steps:

  1. Create an IAM role with the AWS-managed IAM policy above.

  2. Set up a trust relationship between the created IAM role with the service account automatically created by the add-on called calyptia-fluentbit.

eksctl create iamserviceaccount \
    --name calyptia-fluentbit \
    --namespace calyptia-fluentbit \
    --cluster $YOUR_CLUSTER_NAME --region $AWS_REGION \
    --attach-policy-arn arn:aws:iam::aws:policy/AWSMarketplaceMeteringRegisterUsage \
    --approve \
    --override-existing-serviceaccounts \
    --role-only \
    --role-name calyptia-fluentbit-role

AWS provides more details on how to set up these kinds of relationships here.

Note we do not create a service account above, only the role - the service account will be created automatically by the add-on creation process (or by Helm).

Enable Calyptia Core Agent add-on from AWS console

After subscribing to the Calyptia Core Agent on AWS Marketplace and following the on-screen instructions successfully, you will be redirected to the Amazon EKS console.

To get started in the Amazon EKS console, go to your EKS clusters, and in the Add-ons tab, select Get more add-ons to find the Calyptia Core Agent EKS add-on in the cluster setting of your existing EKS clusters.

You can also use the search bar to find "Calyptia Core Agent" and follow the on-screen instructions to enable the Calyptia Core Agent add-on for your Amazon EKS clusters.

You can learn more about direct deployment to Amazon EKS clusters from this AWS blog post.

AWS CLI Deployment

Enable add-on

To enable the Calyptia Core Agent add-on for your Amazon EKS cluster run the following commands:

Please replace$ACOUNT_ID with your account ID to find the appropriate calyptia-fluent-bit role created earlier.

aws eks create-addon --addon-name calyptia_fluent-bit --cluster-name $YOUR_CLUSTER_NAME --region $AWS_REGION --service-account-role-arn arn:aws:iam::$ACCOUNT_ID:role/calyptia-fluentbit-role

Refer to the AWS documentation for the details on using the create-addon command and here for an overview of role management. The service account create by the add-on has to be associated with the role created earlier so confirm the information is correct and matches your role identifier.

To monitor the installation status, you can run the following command:

aws eks describe-addon --addon-name calyptia_fluent-bit --cluster-name $YOUR_CLUSTER_NAME --region $AWS_REGION

Disable add-on

To disable the add-on, you can run the following command:

aws eks delete-addon --addon-name calyptia_fluent-bit --cluster-name $YOUR_CLUSTER_NAME --region $AWS_REGION

Helm Chart Deployment

A Helm chart is provided to deploy the EKS Add On as well as via the AWS CLI and UI.

The eksctl and helm tools are used below to simplify deployment.

Create a service account associated with the appropriate policy.

eksctl create iamserviceaccount \
    --name calyptia-fluentbit \
    --namespace calyptia-fluentbit \
    --cluster $YOUR_CLUSTER_NAME --region $AWS_REGION \
    --attach-policy-arn arn:aws:iam::aws:policy/AWSMarketplaceMeteringRegisterUsage \
    --approve \
    --override-existing-serviceaccounts

Deploy the helm chart using the pre-created service account in the right region.

helm repo add calyptia https://helm.calyptia.com --force-update
helm repo update
helm upgrade --install calyptia-fluentbit calyptia/fluent-bit-eks-addon \
    --set serviceAccount.create=false,region="$AWS_REGION" \
    --wait

Calyptia Core integration

The Helm chart also supports direct integration with Calyptia Core by providing the token to use.

Remember to replace PROJECT_TOKEN with your actual Calyptia Core token.

helm upgrade --install calyptia/fluent-bit-eks-addon \
    --set serviceAccount.create=false,region="$AWS_REGION" \
    -set project_token="$PROJECT_TOKEN" \
    --wait

Modify version deployed

To deploy with different versions of the Calyptia Fluent Bit AWS marketplace offering, just configure the image.tag key:

helm upgrade --install calyptia-fluentbit calyptia/fluent-bit-eks-addon \
    --set serviceAccount.create=false,region="$AWS_REGION" \
    --set image.tag="23.4.5" \
    --wait

Other options

A separate helm chart is provided to deploy the AWS marketplace offering of Calyptia Fluent Bit directly. Refer to the installation instructions for the helm chart: https://github.com/calyptia/charts/tree/master/charts/aws-fluent-bit.

The container image can also be directly consumed in ECS or EKS pods, ensuring the IAM role is set up to allow it.

PreviousWindowsNextConfiguring Calyptia Fluent Bit

Last updated