Dealing with Cold Starts

Since the beginning of serverless adoption, cold starts have been the biggest headache for practitioners. As a serverless practitioning team, we developed our own way of battling with cold starts. Our warm-up plugin has helped thousands of serverless practitioners to get rid of the potential effect of cold starts.

Installing Thundra's warm-up plugin

We provide two ways of installing Thundra's warm-up plugin, you can either manually set up it from our CloudFormation stack or you can use Serverless Framework to add it into your functions.

Manual setup with CloudFormation

Thundra provides an AWS CloudFormation template to setup thundra-lambda-warmup Lambda function with required configurations, permissions and roles on user end.

  • Access AWS CloudFormation console

  • Click Create Stack

  • Select I acknowledge that AWS CloudFormation might create IAM resources with custom names. under Capabilities section and click Create.

  • When the CloudFormation stack completes its installation you'll find a Java function among your Lambda functions with name thundra_lambda_warmup . This function makes dummy requests to your functions in order to keep their container warm.

Setup with Serverless Framework

At first, you need to have thundra-lambda-warmup artifact with one of the following ways:

After getting the artifact,

  • If you have already clone the repository, just go to the root directory where the project has been cloned.

  • Or if you just download the artifact as mentioned above,

Then, specify the following mandatory configurations under custom section in the serverless.yml:

  • configure region configuration with the region where the thundra-lambda-warmup function will be deployed

  • configure deploymentBucket configuration with the S3 bucket where the thundra-lambda-warmup artifact will be deployed

  • configure other configurations if you want (this is optional)

And that's all. Finally deploy the thundra-lambda-warmup artifact as Lambda function by sls deploy.

Configuring Thundra's warm-up plugin

If your Lambda functions are wrapped by Thundra agent, there is no additional effort required.

You just need to specify Lambda functions to be monitored. There are two ways for this:

  • Active way: Lambda functions to be warmed-up are specified by thundra_lambda_warmup_function environment variable as comma separated. This environment variable is put onto thundra-lambda-warmup Lambda function. For example environment variable key (name) is thundra_lambda_warmup_function and value is my-func-1,my-func-2,my-func-3

  • Passive way: Lambda functions to be warmed-up declare themselves by thundra_lambda_warmup_warmupAware environment variable enabled (environment variable key (name) is thundra_lambda_warmup_warmupAware and the value is true). Note that this environment variable is put onto Lambda function to be warmed-up (NOT onto thundra-lambda-warmup Lambda function).

By default Lambda functions are warmed-up with concurrency factor 8 which is the number of warmed-up container count to be kept up . This factor can be configured by thundra_lambda_warmup_invocationCount environment variable. In here there are a few points you should know:

  • This approach is just best effort way for keeping multiple containers up. So this means that even though concurrency factor is 8, there is no guarantee that there are exactly 8 containers up. There are might be more or less containers up. But in practice, mostly there are 8 containers up as we tested since long time.

  • In addition, even though there are active containers and they are not busy with the invocation, AWS Lambda might dispatch the request to a new container. So this leads to cold start unfortunately.

As a result, thundra-lambda-warmup Lambda function can dramatically reduce your cold starts (it is battle-tested on production). However keep in mind that it is not %100 cold start free and cannot guarantee that there won't be any cold start due to reasons explained above.

NOTE: Currently all the agents (Java, Node.js, Python, Go and .NET agents) have warmup support.

See here for more details.

.Net Agent Specific Configurations

When enabling warm up functionality for your .Net Lambda functions using Thundra, make sure to implement the 'IsEmpty' method that is implemented when the 'ISelfAware' class is extended to your Request object class.

For example, if your handler class extends the Thundra's Lambda Request Handler as below: public class HandlerClass : LambdaRequestHandler<GetAlbumRequest, Album> then you must extend 'ISelfAware' in the GetAlbumRequest class.

using Thundra.Agent.Lambda.Core;
namespace dotnet
{
public class GetAlbumRequest : ISelfAware
{
public int id { get; set; }
public GetAlbumRequest()
{
}
public bool IsEmpty()
{
return id.Equals(0);
}
}
}

Non-Object Request

If you are not passing an object as a request, and instead, a string for example, then you do not have to implement the IsEmpty method. The same goes for dealing with Stream handlers.

Fast Start-up for Java Lambda functions

As a serverless team originated from JVM world, we provide a custom runtime to our users with Java. When you use this custom runtime which can be found in our AWS Lambda Layer, we provide you an option to fast start-up your Lambda functions with Java runtime. In this way, our customers achieved to improve the start-up performance of their Java functions 3 times better than normal runtime. In order to enable fast-startup you have two options:

  • Configuring Thundra libraries: After adding Thundra Layer to your function and setting the runtime to the custom runtime that we provided. All you need to is setting the environment variable with name

    thundra_agent_lambda_jvm_optimizeForFastStartup

    to true. More detailed information can be seen here.

  • Instrumenting from Thundra Web Console: If you already installed our AWS integration, you can instrument your Java functions from Thundra console with fast start-up mode with only one click. In this way, we will make all the necessary arrangement on behalf of you.