Skip to main content

​
Creating and Managing Alerts in Metoro

Metoro provides two methods for creating and managing alerts:
  1. Using the Metoro UI
  2. Using Kubernetes ConfigMaps

​
Managing Alerts Using the UI

To create a new alert using the Metoro UI:
  1. Navigate to the Alerts page
  2. Click on the Create Alert button
  3. Configure your alert conditions. For more information about alert configuration, please check the API documentation.
Create Alert You can update existing alerts by clicking on the alert name in the list. This will take you to the alert details page. You can click Edit to modify alert configuration.
You need ‘update’ permissions on alerts to edit an alert. If you don’t have these permissions, the Edit button will be disabled.
Edit Alert

​
Managing Alerts Using Kubernetes ConfigMaps

For organizations that prefer Infrastructure as Code (IaC) practices, Metoro supports creating and managing alerts using Kubernetes ConfigMaps.

​
ConfigMap Requirements

  1. ConfigMaps must include the label metoro.io/alert: "true" to be recognized by Metoro as an alert definition.
    • ConfigMaps without this label will be ignored by Metoro.
  2. The ConfigMap data must include a key named alert.yaml with alerts and the toplevel key containing the alert definition(s).
  3. The alert definition must follow the format specified in the API documentation.

​
Alert Synchronization

  • Alerts defined in ConfigMaps are automatically synced with Metoro once every hour.
  • Alerts managed via ConfigMaps cannot be edited through the UI to prevent conflicts.
  • Any changes made via the API would be overwritten during the next sync cycle.
  • If you delete a ConfigMap, the corresponding alert will not be deleted in Metoro. This is to prevent cluster outages deleting alerts etc. After removing the configmap you should subsequently delete the alert in the UI. Subsequent syncs will not recreate the alert.

​
Manual Synchronization

If you don’t want to wait for the hourly sync, you can:
  • Use the “Sync Alerts” button on the Alerts page in the Metoro UI
  • Call the endpoint GET "/api/v1/alerts/sync" with your auth token. For more information, please check the API documentation.
Sync Alerts Button

​
Example ConfigMap

Please check the API documentation for the full alert definition format. Below is a simple example of a ConfigMap that defines an alert:
You have to provide an unique id for the alert to ensure that the alert can be referenced in the future.
kind: ConfigMap
apiVersion: v1
metadata:
  name: alert-config
  labels:
    metoro.io/alert: "true"
data:
  alert.yaml: |
   alerts:
   - metadata:
       id: "trace-count-alert-001"
       name: "Trace Count Upper Bound"
       description: "Alert when there are too many traces"
     type: timeseries
     timeseries:
       expression:
         metoroQLTimeseries:
           query: "count(traces)"
           bucketSize: 60
       evaluationRules:
         - name: critical
           type: static
           static:
             operators:
               - operator: greaterThan
                 threshold: 15
             persistenceSettings:
               datapointsToAlarm: 3
               datapointsInEvaluationWindow: 5
               missingDatapointBehavior: notBreaching
For more examples, please check the Example Alerts section.

​
Troubleshooting ConfigMap Alert Synchronization

If you don’t see your alerts being created or updated from ConfigMaps, the issue is most likely due to one of the following reasons:
  1. Ingestion Delay: Allow at least one full minute for your new ConfigMap definition to be ingested by Metoro. After this period, try to sync alerts manually either via the UI or the API.
  2. Missing or Incorrect Labels: Double-check that your ConfigMap has the necessary label metoro.io/alert: "true" as described in the ConfigMap Requirements section.
Invalid Alert Definition: If your alert is still not appearing, the most likely cause is an invalid alert definition format. In this case, Metoro will log an error in your account. You can find these error logs by filtering for:
  • Environment: metoro-internal
  • Service name: configmap-to-alert
These logs will contain the specific reason why your alert definition could not be processed.Sync Error Log
⌘I