Traces Page

Traces Page Breakdown

Serverless architectures enable developers to build resilient distributed applications in which messages are exchanged between different functions and creates a business transaction to achieve a certain business goal. Out-of-the-box solutions provide visibility through functions but it is hard to track a business transaction happening in a serverless architecture.

Thundra provides developers to track changes the asynchronous serverless transactions which are basically a chain of several function invocations via Traces page.

You can navigate to traces page by clicking Traces icon button on left side bar.

Traces page includes 3 different sections:

Traces List

In traces page, traces are listed in terms of:

  • Trace ID - ID of trace

  • Origin Trigger - The trigger of the origin (AWS API Gateway, AWS SQS, AWS SNS, …)

  • Origin Lambda - The origin (aka entry point) Lambda function which is the first executed Lambda function in the flow

  • Start Time - The start time of the trace

  • Duration - End-to-end duration of the entire trace

  • Resources - All of the interacted resources (AWS DynamoDB, AWS S3, Redis, …) from any Lambda function in the entire trace

  • Errors - Types of the thrown any error from any Lambda function in the entire trace

Query Bar

The query bar on the Traces page allows you to write custom queries to filter your traces. You can save your queries to easily apply custom filters to the traces. Another option is to save your trace queries as Alert Policies. You may want to check alert policies page for detailed information. You can use Query Helper to write your own queries by selecting the parameters with which you would like to filter your data.

Query Save Details

If your role in the organization is Admin or Account Owner, you can save queries as public so that everyone in the organization can see the saved query. You can still save some queries as private so that only you can see the saved query. If you are User or Developer, you can only save the query for yourself and your query won't be visible to the whole organization.

Saving Queries as Alert Policies

If your role in the organization is Admin or Account Owner or Developer, you can save queries as alert policies. The dropdown menu is not visible to the User role.

Queries

In queries section you can display list of predefined queried which are created for you by Thundra. These predefined queries helps you to list and filter your traces for different purposes.

Also you can display your saved queries for your traces. You can save your queries from save button next to query bar. You can set any query as your default query which will be run when you open Traces page. Also you can delete your saved queries.

Trace Details

Trace Map

When you click on a trace in traces list, you can display trace details as trace map. It provides a flow of a transaction with a flow-chart like representation. It helps you to understand specific trace in visual way.

If there is a configuration for a lambda and any violation occurs to access any resources, block or violated icons displayed on resource icons.

Blocked Resource
  • Red block icon - Access to that resource is blocked by Thundra due to security configuration.

  • Yellow block icon - Access to that resources is allowed however it violates a security configuration.

Allowed Resource

Filters

You can customize your architecture view using filters. When you hover on Filters button in architecture page, options will be displayed.

  • Show Labels is selected by default and it shows labels on vertices. Specifically, for lambda vertices, you can see the function name and the average duration of invocations.

Show labels
  • Show Metrics setting show more information about the interaction between Lambda function and other resources. Without clicking on edges, you can see the count and duration between your function and resources at a glance.

Show metrics
  • Old AWS Icons setting comes as selected by default to show your serverless architecture with old icons. AWS announced new icons very recently. Uncheck this setting in order to see the new icons in your architecture.

New icons
  • Export PNG exports your architecture image in png format.

Lambda Trace Details

When you click on a Lambda function in trace map to take a closer look to inside of Lambda invocation. You can display details of invocation which shown below.

  • Summary - Overall information about the span, including request and response data.

  • Tags - Refers to specific information passed with the span and can even include custom tags if configured with custom spans.

  • Logs - Displays all the logs that occur with the specific part of the Lambda function represented in the span.

SQS Messages

If you click on non-lambda you will see messaged exchanged on this service. For example, If SQS node is clicked, INBOUND and OUTBOUND messages will be displayed.

Clicking on an edge which will show the message which is flowing on the edge on the right side of the screen.

Required Thundra Library Versions

In order to have the trace map with your functions, you need to update your agent versions as follows:

  • For Java, the agent library version is 2.2.0 or higher. The layer version needs to be 10 or higher.

  • For Node.js, the agent library version is 2.3.0 or higher. The layer version needs to be 11 or higher.

  • For Python, the agent library version is 2.3.0 or higher. The layer version needs to be 7 or higher.

  • For Go, the agent library version is 2.1.0 or higher.

Agent Configurations for Traces

By default, SQS, SNS, DynamoDB, Lambda, HTTP / API Gateway messages are shown when clicked on their nodes.

SQS:

  • thundra_agent_lambda_trace_integrations_aws_sqs_message_mask: Masks sent SQS message at client side which calls AWS SDK if it is true.

SNS:

  • thundra_agent_lambda_trace_integrations_aws_sns_message_mask: Masks sent SNS message at client side which calls AWS SDK if it is true.

DynamoDB:

  • thundra_agent_lambda_trace_integrations_aws_dynamodb_statement_mask: Masks sent DynamoDB statements (query, scan, get, put, modify, delete, etc ... requests) at client side which calls AWS SDK if it is true.

Lambda:

  • thundra_agent_lambda_trace_integrations_aws_lambda_payload_mask: Masks sent Lambda invocation payload at client side which calls AWS SDK if it is true.

HTTP / API Gateway:

  • thundra_agent_lambda_trace_integrations_http_body_mask: Masks sent HTTP request body at caller side if it is true.

However, this behaviour is disabled for Kinesis, Firehose and CloudWatch logs by default. You can change this by adjusting the following variables from the environment variables:

Kinesis:

  • thundra_agent_lambda_trace_integrations_aws_kinesis_record_unmask: Traces sent Kinesis record at client side which calls AWS SDK if it is true.

  • thundra_agent_lambda_trace_kinesis_request_enable: Traces incoming Kinesis record at triggered Lambda side if it is true.

Firehose:

  • thundra_agent_lambda_trace_integrations_aws_firehose_record_unmask: Traces sent Firehose record at client side which calls AWS SDK if it is true.

  • thundra_agent_lambda_trace_kinesis_request_enable: Traces incoming Firehose record at triggered Lambda side if it is true.

CloudWatch Log:

  • thundra_agent_lambda_trace_cloudwatchlog_request_enable: Traces incoming CloudWatch log message at triggered Lambda side if it is true.