Thundra's search mechanism on functions, invocations or traces allows you to filter them with detailed queries. You can write specific queries to filter your functions, invocations or traces to find problematic ones or detect check if something goes wrong such as:
AVG(Duration)>200 AND COUNT(ColdStart)>0
This query lists functions that average invocation time is more than 200 milliseconds and contains cold start. So, if something goes wrong with your functions you can easily see in list. So, searching for issues is a way to track your serverless system. What if you can check for these issues using your detailed queries periodically? Thundra provides you to troubleshoot problems in your serverless system and handle them using Alerts.
In Thundra, you can save queries that are written for function, invocations and traces as alert policy. Thanks to alerts in Thundra you will be notified about your system metrics and bottlenecks.
For example, you can say that when there are functions whose health is under 95 in 1 hour, I'd like to be notified. All you have to do is writing a query on Functions Page and save that as alert. While saving alert policy, you can decide in which periods this condition will be checked. In Thundra, you can select a period in a wide range such as "Every 1 minute" to "Every month".
You can save bunch of alert poIicy. So, if you want to easily recognize your alert later, you need give a name that you can remember. Also, every alert doesn't have same importance. You can define your alerts with different severities. INFO can be used to make every one heard about it, WARNING can be used to express there is something goes wrong and you need to fix it. Lastly, you can use CRITICAL to emphasize it is an urgent situation.
You defined your condition, checking periods and type of your alert policy. What about how do you know your alert condition violated? At this point, Thundra provides you different notification types such as email, Slack message. Also you can integrate Thundra alerts with the alerting tools such as OpsGenie, PagerDuty, and VictorOps.
We all know that getting alerts over and over again because of the same reason could be annoying sometimes. For that reason, Thundra provides you two solutions:
Throttle - After you get the first alert, you may want to throttle your alert. This solution is useful if you know that your system will violate this alert policy after the first time. Alerts can be throttled for x minutes, hours or days.
Disable - If an alert starts to disturb you, you can disable it from Alerts page.
Now it's all done! You will be notified through notification channel if you preferred one of them when your alert policy condition is met. After this point, you can jump to handle your alert issue as detailed here.
Alerts for invocations have a little bit different part from other alert policies. As the other policies, you can write a detailed query and save it as an alert. While saving an alert for any invocation, you can select a target. You can prefer to target not only the function you are working on but also the other functions in the account or a specific project. For example; you may want to check the same condition at every function dealing with users. With selecting target type, your alert can check the condition for a function, for all functions in a specific project or all functions in Thundra.