Oracle Cloud Infrastructure Documentation

Set Up Management Agents to Collect Prometheus Metrics

Oracle Cloud Infrastructure Management Agents service (Management Agents) supports collecting time series performance metric data from any resource that exposes metric data in the Prometheus format. To learn more about Prometheus, see the official Prometheus Getting Started documentation.

Prometheus and Management Agents

The following steps describe how to collect Prometheus metrics with Management Agents:

  • Step 1. Install Management Agent with access to the Prometheus metrics endpoint.
  • Step 2. Enable Prometheus metrics
  • Step 3. Configure a Prometheus data source
  • Step 4. Create a Prometheus Dashboard

    Step 1: Install Management Agent

    First, you need to install a Management Agent to register agents with Oracle Cloud Infrastructure (OCI).
    Depending on your environment, you can select one of the following ways to install Management Agent:
    • Install a Management Agent on an OCI instance.

      If you have an OCI compute instance, then you must follow the instructions to enable the Management Agent on an OCI compute instances. For information about the prerequisites and the installation process, see Deploy Management Agents on Compute Instances.

    • Install a Management Agent in a non-OCI instance.

      If you have a standalone VM, you need to follow the instructions to install a Management Agent on a VM. You must install Management Agent on the same VM or on a VM with access to the Prometheus endpoint. For information about the prerequisites and the installation process, see Install Management Agents.

      Create a dynamic group and attach a policy

      Next, create a Management Agent dynamic group with the following policy.
      1. Create a dynamic group for the Management Agents if it doesn't already exist. For example: Management-Agent-Dynamic-Group. Replace the <AGENT_COMPARTMENT_OCID> with the compartment OCID for the Management Agent. 
        ALL {resource.type='managementagent' resource.compartment.id='<AGENT_COMPARTMENT_OCID>'}
      2. Attach the following policy to allow the agent to post the metrics to the OCI Monitoring namespace. 
        allow dynamic-group Management-Agent-Dynamic-Group to use metrics in compartment <METRIC_COMPARTMENT_OCID> where target.metrics.namespace = '<OCI_MONITORING_NAMESPACE>'
      Note

      If you want to see the metrics immediately then you need to restart the Management Agent.
      For more information about dynamic groups, see Managing Dynamic Groups and Managing Policies.

      Step 2. Enable Prometheus metrics

      To enable Prometheus metrics for the software or hardware go to the official instructions for what you want to monitor to determine which method to use.

      For example, this blog post explains How to Configure Docker Containers in Oracle Cloud to use Prometheus metrics. In this example, the Docker instructions on the official website contain the latest instructions on how to enable Prometheus on a Docker container.

      Step 3. Configure a Prometheus data source

      Next, in Oracle Cloud you can configure the Management Agent to collect Prometheus metrics by adding and configuring a Prometheus data source in the Oracle Cloud Console.

      This video demonstrates how to configure a Prometheus data source. In this video, a Prometheus data source is configured for a Kafka service. The process to add, edit, and configure a data source in the Oracle Cloud console is the same for any data source that exposes Prometheus metrics over a URL.

      To add and configure a Prometheus data source:

      1. In the Oracle Cloud Console, go to the Navigation menu, select Observability and Management, go to Management Agents, and then select Agents.
      2. Select Agents and Gateways, select required Compartment and then select the Agent.
      3. Select Manage Data Sources, and then select Add Data Source.
      4. Select Prometheus from the Data Source Type drop-down list, add the Metric namespace, the URL, select the Metric compartment. If required, you can add Custom metric dimensions, or set any optional properties. For a list of options, see Required Data Source Properties and Optional Data Source Properties.
        Note

        The Metric namespace and the Metric compartment values must be same as the values in the policy.

      5. Click Add.

        Now if configured correctly, after a few minutes the metrics will display in the OCI Monitoring Service.

        Note

        If the required policies are missing, then you will receive HTTP 404 NotAuthorized error messages in the Management Agent logs. For more details, see Create a dynamic group and attach a policy.

      To view or edit a Prometheus data source:

      After you add a Data Source, you can go to Manage Data Sources and then select the vertical ellipse next to the Data Source you want to view or edit.

Required Data Source Properties

The following properties are required when you create a Prometheus data source.

Name Units/Format Description
url String The url where the Prometheus Exporter publishes it's metrics. Supports HTTP requests only, for example:

http://localhost:9100/metrics
namespace String The OCI monitoring namespace where the scraped metrics are uploaded.
compartmentId String The OCID of the compartment where the scraped metrics are uploaded.

Optional Data Source Properties

The following properties are optional when you create a Prometheus data source. To collect a more specific range of Prometheus metrics, you can add any of these optional properties when you create the data source.

Name Units/Format Description
allowMetrics String - comma separated metric name list The complete set of desired scraped metrics. Use this property to limit the set of metrics uploaded if required.
connection-timeout Number in milliseconds The timeout for connecting to the Prometheus Exporter's endpoint.
metricDimensions String - comma separated property name list The names of other user-supplied properties expressed as fixed values to be used as dimensions for every uploaded data point.
proxy-url String - URL The URL of the network proxy that provides access to the Prometheus Exporter's endpoint, URL required property.
read-data-limit Number in kilobytes The limit on the data being sent. For example, if 400 KB is entered then the will not exceed the management agent's fixed limit, 400 KB of disk space.
read-timeout Number in milliseconds The timeout for reading the response from the Prometheus Exporter's endpoint.
resourceGroup String. It includes only alphanumeric characters, periods (.), underscores (_), hyphens (-), and dollar signs ($). The OCI Monitoring Service's resource group.

If it's specified, the resource group will be assigned to each of the emitted metrics.
scheduleMins Scraping interval in minutes If this value is specified, the data collection or scraping occurs at this specific time interval.

The minimum value is 1 minute.

If this value is not specified, the default value is 5 minutes.

Note: Lowering the collection interval can have a positive impact on the CPU performance.

Example: scheduleMins=1

Create a Prometheus Dashboard

After the steps above to set up Management Agent to export Prometheus metrics are complete, you can use the OCI Management Dashboards service to create a dashboard and visualize the Prometheus metrics. You can configure one metric data explorer widget for each metric you want to visualize. For information, see Create a Metric Data Explorer Widget.