Thundra

Thundra: Serverless Observability for AWS Lambda

The black box nature of AWS Lambda and other serverless environments means that identifying and fixing performance issues is difficult and time-consuming. Built for straightforward debugging, monitoring, and observability, Thundra provides deep insight into your entire serverless environment. Thundra collects and correlates all your metrics, logs, and traces, allowing you to quickly identify problematic invocations and also analyzes external services associated with that function. With Thundra’s zero overhead and automated instrumentation capabilities, your developers are free to write code without worrying about bulking up their Lambdas or wasting time on chasing black box problems.

Get Started    Discussions

Installation Guide

In order to see Thundra data in Elasticsearch there are three steps you will need to complete.

Step 1: Get a license key from us

In order to run Thundra Integrator for Elasticsearch AMI, you need to have Thundra Licence Key. Please send an email to integrations@thundra.io and we'll turn back to you with your license key which is free for 30-days.

Step 2: Install the Thundra Integrator for Elasticsearch AMI on your AWS EC2 Instance

Thundra provides an AWS CloudFormation template to easily set up your integration.
Below is the deployment topology created from the CloudFormation template. It includes:

  • The Thundra Receiver App that collects, transforms and aggregates data from the Thundra Lambda agent.
  • Filebeat that sends that data to your Elasticsearch.
  • A data volume that temporarily stores the files and data collected from the Thundra integrator. This data volume cleaned by the Thundra integrator hourly.

To deploy the Thundra Integration for Elasticsearch AMI:

  • Access AWS CloudFormation console
  • Click Create Stack
  • Select Specify an Amazon S3 template URL and enter https://s3-us-west-2.amazonaws.com/thundra-elasticsearch-staging-dist/thundra-elasticsearch-cf-template.json on which we host the CloudFormation template.
  • Click Next
  • Fill out the template:
    1. Enter a Stack Name (Example: thundra-elasticsearch)
    2. Optional: Enter an EBS Volume Size (Default: 50GBs. Thundra cleans out the data written to this volume hourly. In most cases, 50GBs is enough capacity. However, please feel free to adjust this size for your expected hourly data volumes.
    3. Select Enable HTTPS to enable or disable SSL with a self-signed certificate

Enable Certificate Validation Using HTTPS

If EnableHTTPS set to ‘True’ please add thundra_lambda_publish_rest_trustAllCertificates to the environment variables of your AWS lambda function and set to True.

  1. Enter ElasticSearchUrl
  2. Enter ElasticSearchPort
  3. Enter ElasticSearchUsername (optional)
  4. Enter ElasticSearchPassword ((optional)
  5. Choose ElasticSearchEnableHttps
  6. Enter ThundraElasticsearchReceiverElasticIp, this IP can be used to allow Thundra receiver to reach your Elasticsearch instance while sending data.
  7. Select the appropriate EC2 KeyPair name to enable SSH access to your EC2 instance
  8. Enter the IP address range that your EC2 instance will have SSH access to
  9. Enter optional API Key Prefix and/or API Key values. These API keys are used to update the Thundra agent to send data to the Thundra Integrator for Elasticsearch instead of the Thundra Web Console.

If you do not provide an API key or API key prefix the Thundra Integrator for Elasticsearch will accept data without any authentication. We highly discourage this approach as it is not considered best practice.

  1. Enter API Key Prefix. Any Thundra agents that use this API key prefix will be allowed to send data to the Thundra Integrator for Elasticsearch.

  2. Enter allowed API Keys. Any Thundra agent that uses this explicitly set API key will be able to send data to the Thundra Integrator for Elasticsearch. Enter API Key(s) as a comma-separated list.

** IMPORTANT**: Record your API key information!

Write down or record your API key prefix or API keys!!! You will need to enter them into the Thundra agent YML file in Step 3 of the setup.

  1. Enter the Thundra license key created from the Thundra Web Console
  2. Enter the port number through which Thundra Integrator for Elasticsearch will receive the Thundra agent data
  3. Select your EC2 instance type

Below is a list of the parameters and value requirements needed to fill out the CloudFormation template:

Parameter Description Values Defaults
EBSVolumeSize Size of the attached EBS volume in GBs Numeric value 50
EnableHTTPs To run Thundra receiver over HTTPS with a self-signed certificate, set this parameter to true true/false false
KeyName Keypair name for SSH into EC2 server AWS EC2 KeyPair values in the region Empty
SSHLocation The IP address range that is allowed to SSH to the EC2 instances Valid IP range in x.x.x.x/x notation. Use 0.0.0.0/0 for no restrictions. 0.0.0.0/0
ThundraAllowedApiKeyPrefix Prefix of API keys from which the Thundra-Elasticsearch Integrator will receive data. For example, if you set the API key prefix to elasticsearch-agent-key, the Thundra agents that can send data using this prefix include elasticsearch-agent-key-node, elasticsearch-agent-key-java, elasticsearch-agent-key-go, and elasticsearch-agent-key-python ` Alphanumeric value Empty
ThundraAllowedApiKeys Comma (',') separated list of API keys from which the Thundra Integrator for Elasticsearch will receive data. For example, elasticsearch-agent-key-java,elasticsearch-agent-key-node,elasticsearch-agent-key-go,elasticsearch-agent-key-python Alphanumeric value Empty
ThundraLicenseKey License key for the Thundra Integrator for Elasticsearch can be generated from Thundra Web Console. Alphanumeric value Empty
ThundraElasticsearchReceiverElasticIp Elasticip of your receiver Numeric value 110.110.110.110
ThundraReceiverPort Port number for Thundra Integrator for Elasticsearch to receive data over HTTP/HTTPS Numeric value between 1024 and 65536 8080
InstanceType EC2 instance type Supported AWS EC2 instance types d2.xlarge
ElasticSearchUrl Elasticsearch url Alphanumeric value Empty
ElasticSearchPort Elasticsearch port Alphanumeric value Empty
ElasticSearchUsername Elasticsearch username Alphanumeric value Empty
ElasticSearchPassword Elasticsearch password Alphanumeric value Empty
ElasticSearchEnableHttps If you want to access Elasticsearch with https, set this value to true true/false false
  • When you are finished filling out the template, click Next

While it is best practice to set tags for your EC2 instance, you are not required to do so to set up this Thundra Integrator for Elasticsearch. These options are specific to CloudFormation. For a detailed explanation see the AWS CloudFormation docs: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-console-add-tags.html

  • Click Next to move into the Review screen.
  • On the Review screen, review your stack details. When ready, click Create on the lower right corner of the screen to deploy your stack.

Step 3: Configure Thundra Lambda Agents to Send Data to the Thundra Integrator for Elasticsearch

Now that the stack creation is complete, you need to send data to the Thundra Integrator for Elasticsearch by updating your publish base URL and API key.

  • If this is your first time using Thundra, you will also need to set up the Thundra agents. Otherwise, if you are already have been using Thundra with the web console, these steps will re-direct data to the Thundra Integrator for Elasticsearch instead of the Thundra Web Console.

Set up your Thundra agents if this is your first time using Thundra

If you haven't already set up Thundra agents for your Lambda functions, you will need to go through the installation instructions for Java, Python, Node.js, or Golang.

Find the endpoint URL for your Thundra Integrator for Elasticsearch in the Output tab of your CloudFormation stack.

The format should be: <URLofYourThundraElasticsearchIntegratorInstance>/v1.

Now, navigate to your Thundra Lambda environment variable configuration and change the API key and publish base URL.

Update thundra_apiKey values that you entered during the CloudFormations template set up and thundra_agent_lambda_report_rest_baseUrl that you obtained from the Output tab in the CloudFormation stack.

Then click Save in the upper right corner of your Thundra Lambda environment variable configuration.

You are now set up for Thundra Integrator for Elasticsearch!

After redeploying your Thundra Lambda with these modifications, you will be able to see the Lambda data on your Kibana UI within approximately 5 minutes.

How to Manually Start and Stop the Thundra Integrator for Elasticsearch

After successfully deploying theThundra Integrator for Elasticsearch CloudFormation stack, an EC2 instance will be launched with name Thundra Elasticsearch Receiver.

To manually start and stop your Thundra Integrator for Elasticsearch instance :

  1. SSH into your EC2 instance with the KeyPair you entered into the CloudFormation template
  2. Go to /opt folder
  3. To start your instance: Run start.sh as root
  4. To stop your instance: Run stop.sh as root

Installation Guide


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.