Skip to main content

Alert Types

This section describes the types of alert sources.

Rule-based Alerts

Container Observability detects an anomaly when a predefined alert rule is violated and generates incident tickets where the user can analyze the alert details from the alert panels.

Note

Note:

  • Container Observability enables you to analyze RCA for rule-based alerts triggered for pods and deployments when anomalies are detected.

  • In Alerts, if the Stage column for an alert ticket lists “R”, then you can view RCA results for the alert ticket to determine the cause of the issue.

Container Observability triggers an alert based on the following:

  1. Rule-based:

    1. Metric

    2. Log

    3. Trace

  2. SLO-based

Note

Note: Container Observability also allows you to define new custom Alert Rules for the Alert notifications.

Metric-based alert

Container Observability triggers an alert if metric health is not as per the pre-defined metric rule conditions. It also allows users to define a new custom Metric-Rule for an entity within the Kubernetes cluster as per users' requirements.

Note

Note: You can define new metrics conditions for an entity from the cluster as per your requirement.

  • Container Observability triggers a metric-based alert based on pre-defined metric conditions.

  • Helps you to define the new metric condition for an entity like SLO target for a metric on Containers, Pods, and Nodes.

  • If a metric is not behaving as per the defined metric-rule condition, Container Observability detects and pushes an alert to the user notification channels immediately.

  • You view the details of the metric-based alerts triggered by Container Observability in Alert panels and determine the cause of the anomaly in detail.

Log-based alert

Logs-based alerts are triggered if an unexpected error is found in the logs list. Container Observability identifies such errors, and you can create new Log alert rule conditions that will take care of such system-based log errors in the future. If a similar log error appears, Container Observability creates a log-based alert notification to users' notification channels.

From log-based alert, you can:

  • Analyze the cause of log errors through various alert panels from Alerts.

  • Compare the current log error to previous logs.

Auto SLO based alert

2690388265.png

Container Observability monitors services of each metric health and automatically configures and suggests the Threshold SLO value for the metric based on the data captured during the last 24 hours. It automatically compares the current behavior of metric SLO with the SLO value captured for the last 24 hours.

For any deviation in the metric, Container Observability automatically detects and triggers an SLO-based Alert. For example, an increase in latency, an increase in SLO value, or an increase in response time.

The SLO-based Alerts allow you to perform the root cause analysis. To analyze the RCA for SLO-based Alerts, see SLO-Breach.

Note

Note: As Container Observability automatically triggers SLO-based Alerts, it also enables users to configure SLO targets manually based on the 24 hours of suggested data.

Event-based alert

Event alert is a type of rule identified by Container Observability where the conditions are pre-defined concerning a state change that occurs in the Kubernetes cluster.

If the cluster exhibits an unexpected change or a new error that caused the creation of events, the user can identify such new errors. You can define a new rule for event-based alerts so that Container Observability detects and triggers an immediate alert for the errors related to repeated events occurring in the cluster.

  • Helps you to debug and resolve the tickets generated based on the event and track them as required if the same error occurs.

  • You can identify and analyze the cause of the occurrence through various alert panels to ensure whether it is still ongoing or resolved. For more information, see Alert Panels.

Tracepath-based alert

As with containers, alerts are created when traces show anomalies. Typically, these are driven by breaches on SLOs on response time on the trace or when errors are detected.

Since detecting alerts on individual traces is challenging given the large number of traces an application can generate per monitoring period or scrape interval, Container Observability detects anomalies in traces by detecting them on Tracepaths which are the common business flows that group traces sharing the same flow.

With Container Observability’s Tracepath-based alerting, users can

  • Automatically detect anomalous Tracepaths that contain the traces that show response time anomalies or errors without searching for them.

  • Drill down to the individual traces from the anomalous Tracepaths.

Machine Learning (ML) alerts

Container Observability creates an alert whenever there is a deviation of the container from the normal behavior and ML learns such deviations. This behavior model is built on a curated set of container metrics. Container Observability can trigger ML-based alerts for containers, nodes, and App Objects.

2690388395.png

Note

Note: Currently, you cannot define a new set of conditions to create a new ML model as Container Observability does not support it in the system.

ML-based Alerts help you:

  • Compare the current metric value with what the ML model expects and trigger an alert if the behavior is not as expected by the model.

  • Track the occurrence time and updated time of the anomaly.

  • Allows you to view the RCA of the anomaly that occurred to determine the possible cause and its dependencies that can likely cause an error.

  • Enables predictive analysis to isolate the cause of the problem.

In this section: