Thundra

Thundra: Serverless Observability for AWS Lambda

The black box nature of AWS Lambda and other serverless environments means that identifying and fixing performance issues is difficult and time-consuming. Built for straightforward debugging, monitoring, and observability, Thundra provides deep insight into your entire serverless environment. Thundra collects and correlates all your metrics, logs, and traces, allowing you to quickly identify problematic invocations and also analyzes external services associated with that function. With Thundra’s zero overhead and automated instrumentation capabilities, your developers are free to write code without worrying about bulking up their Lambdas or wasting time on chasing black box problems.

Get Started    Discussions

Traces Page

Serverless architectures enable developers to build resilient distributed applications in which messages are exchanged between different functions and creates a business transaction to achieve a certain business goal. Out-of-the-box solutions provide visibility through functions but it is hard to track a business transaction happening in a serverless architecture. With Thundra's Traces page, developers can track the asynchronous serverless transactions which are basically a chain of several function invocations.

Click on the Traces icon on the left bar in order to reach out Traces page.

Click on the Traces icon on the left bar in order to reach out Traces page.

When we go to “Traces” page, we can see all the individual traces with some information like

  • The trigger of the origin (AWS API Gateway, AWS SQS, AWS SNS, …)
  • The origin (aka entry point) Lambda function which is the first executed Lambda function in the flow
  • The start time of the trace and end-to-end duration of the entire trace
  • All of the interacted resources (AWS DynamoDB, AWS S3, Redis, …) from any Lambda function in the entire trace
  • Types of the thrown any error from any Lambda function in the entire trace
  • The duration breakdown of all the executed Lambda functions in the entire trace with respect to their duration in the overall transaction.

Query Bar

The query bar on the Traces page allows you to write custom queries to filter your traces. You can save your queries to easily apply custom filters to the traces. Another option is to save your trace queries as Alert Policies. You may want to check alert policies page for detailed information.

Query Save Details

If your role in the organization is Admin or Account Owner, you can save queries as public so that everyone in the organization can see the saved query. You can still save some queries as private so that only you can see the saved query. If you are User or Developer, you can only save the query for yourself and your query won't be visible to the whole organization.

Saving Queries as Alert Policies

If your role in the organization is Admin or Account Owner or Developer, you can save queries as alert policies. The dropdown menu is not visible to the User role.

You can use trigger types, resource properties, and many other criteria to perform a fine-grained search in your traces. The below image illustrates an example of a trace query.

Example trace query

Example trace query

Here the full query is trigger.type=AWS-APIGateway AND resource.AWS-DynamoDB.duration > 150 ORDER BY StartTime DESC. We have two different filters AND'ed together to form a compound query. The first one trigger.type=AWS-APIGateway selects the traces triggered by API Gateway. The second filter resource.AWS-DynamoDB.duration > 150 states that we want traces with total DynamoDB operations more than 150 ms. Lastly, we are sorting the resulting traces according to their start time.

Setting a time range for the trace queries

Note that the global time component on the upper right corner of the console is acting as an implicit filter for all the queries you write.

Trace details

When you click on a row on this list, the Trace Map of the clicked transaction is opened. It provides a flow of a transaction with a flow-chart like representation. This enables you to understand the transaction in a visual way.

In the Trace Map, you can click on a Lambda function node, in order to look inside of a Lambda invocation. In this way, you can combine invocation internal information with transaction information on the map. You can see from the below image.

Clicking on a non-Lambda node shows the messages exchanged on this service. For example; clicking on an SNS node will open the right bar with "Inbound" and "Outbound" messages on this node. You can see in the below image.

Clicking on an edge which will show the message which is flowing on the edge on the right side of the screen.

Required Thundra Library Versions

In order to have the trace map with your functions, you need to update your agent versions as follows:

  • For Java, the agent library version is 2.2.0 or higher. The layer version needs to be 10 or higher.
  • For Node.js, the agent library version is 2.3.0 or higher. The layer version needs to be 11 or higher.
  • For Python, the agent library version is 2.3.0 or higher. The layer version needs to be 7 or higher.
  • For Go, the agent library version is 2.1.0 or higher.

Agent Configurations for Traces

By default, SQS, SNS, DynamoDB, Lambda, HTTP / API Gateway messages are shown when clicked on their nodes.

SQS:

  • thundra_agent_lambda_trace_integrations_aws_sqs_message_mask: Masks sent SQS message at client side which calls AWS SDK if it is true.

SNS:

  • thundra_agent_lambda_trace_integrations_aws_sns_message_mask: Masks sent SNS message at client side which calls AWS SDK if it is true.

DynamoDB:

  • thundra_agent_lambda_trace_integrations_aws_dynamodb_statement_mask: Masks sent DynamoDB statements (query, scan, get, put, modify, delete, etc ... requests) at client side which calls AWS SDK if it is true.

Lambda:

  • thundra_agent_lambda_trace_integrations_aws_lambda_payload_mask: Masks sent Lambda invocation payload at client side which calls AWS SDK if it is true.

HTTP / API Gateway:

  • thundra_agent_lambda_trace_integrations_http_body_mask: Masks sent HTTP request body at caller side if it is true.

However, this behaviour is disabled for Kinesis, Firehose and CloudWatch logs by default. You can change this by adjusting the following variables from the environment variables:

Kinesis:

  • thundra_agent_lambda_trace_integrations_aws_kinesis_record_unmask: Traces sent Kinesis record at client side which calls AWS SDK if it is true.
  • thundra_agent_lambda_trace_kinesis_request_enable: Traces incoming Kinesis record at triggered Lambda side if it is true.

Firehose:

  • thundra_agent_lambda_trace_integrations_aws_firehose_record_unmask: Traces sent Firehose record at client side which calls AWS SDK if it is true.
  • thundra_agent_lambda_trace_kinesis_request_enable: Traces incoming Firehose record at triggered Lambda side if it is true.

CloudWatch Log:

  • thundra_agent_lambda_trace_cloudwatchlog_request_enable: Traces incoming CloudWatch log message at triggered Lambda side if it is true.

Traces Page


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.