By default, Pushgateway exposes BRM metrics at http://localhost:9091/metrics, and Prometheus scrapes the brm_status_tracker and brm_memory_usage metrics from Pushgateway every 5 seconds. However, you can customize where Pushgateway exposes the BRM metrics and how Prometheus scrapes them.

To customize Pushgateway for BRM:

1. Open the BRM_home/bin/configurations.yaml file in a text editor.

2. To customize where Pushgateway exposes BRM metrics, edit the following keys:

   • pushgateway.host: The hostname of the machine where you deploy Pushgateway. The default is localhost.

   • pushgateway.port: The port number for Pushgateway. The default is 9091.

   • pushgateway.protocol: The protocol type, such as http or https. The default is http.

3. To customize the metric names and scrape intervals for Prometheus, modify these keys:

   • prometheus.service_monitoring.brm_status_metric_name: The name of the metric for monitoring the current status of BRM services.

   • prometheus.service_monitoring.scrape_interval: The frequency, in seconds, at which Prometheus scrapes the BRM service status metric.

   • prometheus.memory_monitoring.brm_status_metric_name: The name of the metric for monitoring CPU memory used by BRM services.

   • prometheus.memory_monitoring.scrape_interval: The frequency, in seconds, at which Prometheus scrapes the CPU memory status metric.

4. Save and close the file.

pushgateway:
   host: localhost
   port: 9091
   protocol: "https"

prometheus:
   service_monitoring: 
      brm_status_metric_name: "brm_status_tracker"
      scrape_interval: 1
   memory_monitoring:
      brm_status_metric_name: "brm_memory_usage"
      scrape_interval: 1

To enable monitoring of your CM:

1. In your CM configuration file (BRM_home/sys/cm/pin.conf), edit the following Perflib-related entries:

   • In the fm_perflib_config entry, enter the full path to your fm_perflib.so file:

     - cm fm_module Path/fm_perflib.so fm_perflib_config - pin

   • In the perflib_monitor_file entry, enter the full path to the CM Perflib data file to create:

     - perflib_monitor_file perflib_data Path/perflib_data.dat

   • In the perflib_pin_shlib entry, enter the full path to the shared library in which core BRM functionality for the CMs and applications are defined:

     - perflib perflib_pin_shlib Path/libcmpin.so

2. In the BRM_home/apps/perflib/perf.env file, set the entries listed in Table 42-1.

   Table 42-1 Perflib Timing Entries

   Entry Name	Description
   PERFLIB_VAR_TIME	Specify whether to enable tracing of real-time data, batch data, or both: 0: Tracing is disabled. This is the default. 1: Perflib traces only real-time data. 2: Perflib traces only batch data. 3: Perflib traces both batch data and real-time data.
   PERFLIB_VAR_FLIST	Specify whether to enable Perflib flist tracing: 0: Flist tracing is disabled. This is the default. 1: Perflib starts logging summary flist data. 2: Perflib starts logging full flist data.
   PERFLIB_VAR_ALARM	Specify whether Perflib alarm functionality is enabled: 0: Disables the generation of alarms. This is the default. 1: Enables the generation of alarms.
   PERFLIB_RT_FLAGS	Specify the additional details to include in the real-time tracing data: 0: Do not include additional details in real-time trace files. This is the default. 1: Include source references (the source code file name and line number) in real-time trace files. 2: Include opcode flags in real-time trace files. 3: Include both source references and opcode flags in real-time trace files.

3. Start the CM process with Perflib enabled by running one of the following:

   • Run the start_cm script:

     ./start_cm perflib

     Ignore the LD_PRELOAD errors.

   • Run the pin_ctl utility:

     /pin_ctl start cm_perflib

4. Use testnap to run a few opcodes. For example:

   xop PCM_OP_SEARCH 0x10 1

   For more information about testnap, see "Using the testnap Utility to Test BRM" in BRM Developer's Guide.

5. If you enabled real-time metrics tracing, run the pstatus utility to verify that opcode metrics are getting captured in the Perflib data file (perflib.dat):

   pstatus -s10 -csponceNtaARP perflib_data.dat

   For information about the utility's syntax and parameters, see "pstatus".

   If successful, the utility generates a counter report that looks similar to this:

   Timestamp            Program     Opcode  Opcode Name            Calls    Errors    Records     Elapsed       OpAvg      SysAvg   SysRate/s     PID
   ---------            -------     ------  -----------            -----    ------    -------     -------       -----      ------   ---------     ---
   14/07/2025 04:07:13  testnap        155  PCM_OP_ACT_LOGIN           2         0          0    0.288688    0.144344         inf        0.00    9250
   14/07/2025 04:07:23  testnap          3  PCM_OP_READ_OBJ            2         0          2    0.052501    0.026251         inf        0.00    9250
   14/07/2025 04:07:33  testnap        156  PCM_OP_ACT_LOGOUT          2         0          0    0.043381    0.021691         inf        0.00    9250

6. To expose Prometheus metrics for the CM, run the start_monitor script:

   ./start_monitor -P 9090 perflib_data.dat,cm

   For information about the start_monitor utility's syntax and parameters, see "start_monitor".

7. Run the following cURL command to view the exposed metrics:

   curl http://localhost:9090/metrics

   If successful, you should see something similar to this:

   brm_opcode_calls_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 6
   brm_opcode_errors_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 0
   brm_opcode_records_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 0
   brm_opcode_exec_time_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 1.371886758
   brm_opcode_user_cpu_time_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 0.019318000
   brm_opcode_system_cpu_time_total{application="cm", opcode="20", opflags="0", program_name="cm", object_type="", opcode_name="PCM_OP_GET_DD"} 0.006629000

By default, Oracle DM monitoring is disabled, but you can configure it to generate one or more of the following:

• Metrics: When enabled, metrics include real-time data such as the application name, opcode name and number, database number, program name, object type, and elapsed time. You can also configure Perflib to include the following additional data:

  • Opcode flags.

  • Whether the data is for a front-end operation (FE) or a back-end operation (BE), and the specific operation's ID.

• Flist trace files: When enabled, Perflib generates flist log files (flog.txt) that track all Oracle DM operations. You can also configure Perflib to generate separate flist log files for:

  • Front-end operations (files prefixed with FE_).

  • Back-end operations (files prefixed with BE_).

  • Both front-end and back-end operations (front-end files prefixed with FE_, back-end files prefixed with BE_). In this case, BRM also traces additional operations such as the time taken for front-end operations and back-end operations to communicate, the time taken to start a database operation, and the tables that are queried.

• Performance trace files: When enabled, Perflib generates performance log files (plog.txt) that track all Oracle DM operations. You can also configure Perflib to generate separate performance log files for:

  • Front-end operations (files prefixed with FE_).

  • Back-end operations (files prefixed with BE_).

  • Both front-end and back-end operations (front-end files prefixed with FE_, back-end files prefixed with BE_). In this case, BRM also traces additional operations such as the time taken for front-end operations and back-end operations to communicate, the time taken to start a database operation, and the tables that are queried.

• Alarms: When enabled, Perflib generates alarm log files (alarm.txt) that track all Oracle DM operations.

The following shows sample contents from a front-end flist trace file:

2025/06/24 21:45:00.033348  0.000000000    0  PCM_OP_SEARCH(DM_FE[0] 0.0.0.1 /search[500]/business_params 500)[3:cm:cm:1:45907136:0:1750078094974222:0::::opcode_timeout=-1]
2025/06/24 21:45:00.046093  0.012745442       <opcode_end PCM_OP_SEARCH>
2025/06/24 21:45:00.048059  0.000000000    0  PCM_OP_SEARCH(DM_FE[0] 0.0.0.1 /config/notification_spec -1)[3:cm:cm:1:45907136:0:1750078094974222:0::::opcode_timeout=-1]
2025/06/24 21:45:00.049795  0.001736104       <opcode_end PCM_OP_SEARCH>
2025/06/24 21:45:00.050172  0.000000000    0  PCM_OP_SEARCH(DM_FE[0] 0.0.0.1 /config -1)[3:cm:cm:1:45907136:0:1750078094974222:0::::opcode_timeout=-1]
2025/06/24 21:45:00.051987  0.001815385       <opcode_end PCM_OP_SEARCH>

The following shows sample contents from a back-end performance trace file:

S,1,0,7,0,PCM_OP_SEARCH,0,DM_BE,0,339,139981503321920,cm,0,/config/notification_spec,1
S,2,0,0,0,TRANS_START,0,DM_BE,1,339,139981503321920,cm,0,,0
F,0.000524287,1,1,0.000524287,0.000524287,0,0,0,0,0,0,0
S,2,0,0,0,SELECT,0,DM_BE,1,339,139981503321920,cm,0,config_t,0
S,3,0,0,0,EXECUTE,0,DM_BE,1,339,139981503321920,cm,0,,0
F,0.000479994,1,1,0.000479994,0.000479994,0,0,0,0,0,0,0
S,3,0,0,0,PREPARE,0,DM_BE,1,339,139981503321920,cm,0,,0
F,0.000007055,1,1,0.000007055,0.000007055,0,0,1,0.000007055,0,0,0
F,0.000513067,1,1,0.000513067,0.000513067,0,0,0,0,0,0,0
S,2,0,0,0,TRANS_COMMIT,0,DM_BE,1,339,139981503321920,cm,0,,0
F,0.000346088,1,1,0.000346088,0.000346088,0,0,0,0,0,0,0F,0.001673534,1,1,0.001673534,0.001673534,0,0,0,0,0,0,0.000486000

To enable monitoring of your Oracle DM:

1. In the BRM_home/apps/perflib/perf_dm.env file, set the entries in Table 42-2.

   Table 42-2 Oracle DM Perflib Entries

   Entry Name	Description
   PERFLIB_DATA_FILE	Specify the path and name of the Oracle DM Perflib data file to generate. The default is $PERFLIB_HOME/perflib_dm_oracle_data.dat. Optionally, you can add these tokens to the file name: %p: Includes a PID. %t: Includes a timestamp in UNIX format. %T: Includes the timestamp as a YYYYMMDDhhmiss time string.
   PERFLIB_VAR_TIME	Specify whether to generate real-time metrics, performance trace files, or both: 0: Disables tracing of timing data. This is the default. 1: Generates real-time metrics. 2: Generates performance trace files. 3: Generates both metrics and performance trace files.
   PERFLIB_VAR_FLIST	Specify the level of data to log in flist trace files: 0: Disables flist logging. This is the default. 1: Logs summary flist data in flist trace files. 2: Logs full flist data in flist trace files.
   PERFLIB_VAR_ALARM	Specify whether Perflib alarm functionality is activated immediately: 0: Disables the generation of alarms. This is the default. 1: Enables the generation of alarms.
   DM_PERFLIB_ENABLED	Specify whether to enable front-end (FE) and back-end (BE) tracing for the Oracle DM: 0: Disable front-end and back-end tracing. This is the default. 1: Enable tracing for the Oracle DM FE operations. In this case, Perflib generates FE performance and flist tracing files, prefixed with FE_. 2: Enable tracing for the Oracle DM BE operations. In this case, Perflib generates BE performance and flist tracing files, prefixed with BE_. 3: Enable tracing for both the Oracle DM FE and BE operations. In this case, Perflib generates separate FE and BE tracing files.
   PERFLIB_RT_FLAGS	Specify the additional details to include in the real-time metrics: 0: Do not log additional details in the real-time metrics. This is the default. 1: Include source references (whether the source data is from an FE or BE operation as well as the associated process ID) in the real-time metrics. 2: Include opcode flags in the real-time metrics. 3: Include both source references and opcode flags in the real-time metrics.

2. To configure real-time metrics to include details about each individual front-end or back-end process, do the following:

   • For on-premises systems, add the ENABLE_FILENAME_LABELS environment variable to the start_monitor script:

     ENABLE_FILENAME_LABELS=true

   • For cloud native systems, add the ENABLE_FILENAME_LABELS environment variable to the DM Oracle Perflib ConfigMap (configmap_env_perf_dm.yaml):

     data:
        ENABLE_FILENAME_LABELS: "true"

3. Start the Oracle DM process by running the following command. Ignore any LD_PRELOAD errors.

   ./start_it start_dm_oracle

   For the utility's syntax and parameter details, see "start_it".

4. Start testnap and run a few opcodes. For example:

   xop PCM_OP_READ_OBJ 0x0040 1

   For more information about testnap, see "Using the testnap Utility to Test BRM" in BRM Developer's Guide.

5. Run pstatus to verify that opcode metrics are getting captured in the Perflib data file:

   pstatus -s10 -csponceNtaARP perflib_dm_oracle.dat

   For the utility's syntax and parameter details, see "pstatus".

   If successful, you see a counter report that looks similar to the following:

   Timestamp            Program     Opcode  Opcode Name            Calls    Errors    Records     Elapsed       OpAvg      SysAvg   SysRate/s     PID
   ---------            -------     ------  -----------            -----    ------    -------     -------       -----      ------   ---------     ---
   14/07/2025 04:07:34  testnap        155  PCM_OP_ACT_LOGIN           2         0          0    0.288688    0.144344         inf        0.00    9250
   14/07/2025 04:07:34  testnap          3  PCM_OP_READ_OBJ            2         0          2    0.052501    0.026251         inf        0.00    9250
   14/07/2025 04:07:34  testnap        156  PCM_OP_ACT_LOGOUT          2         0          0    0.043381    0.021691         inf        0.00    9250

After you start the monitoring process, the Oracle DM metrics are exposed at the following endpoint:

http://localhost:9091/metrics

If successful and both Oracle DM processes are enabled, you should see metrics similar to this:

brm_opcode_user_cpu_time_total{application="dmo", opcode="3", opflags="0", dbNumber="0.0.0.1", program_name="testnap", filename="DM_BE", lineno="3", object_type="/event", opcode_name="PCM_OP_READ_OBJ"} 0.002289000
brm_opcode_system_cpu_time_total{application="dmo", opcode="3", opflags="0", dbNumber="0.0.0.1", program_name="testnap", filename="DM_BE", lineno="3", object_type="/event", opcode_name="PCM_OP_READ_OBJ"} 0.000779000
brm_opcode_calls_total{application="dmo", opcode="3", opflags="0", dbNumber="0.0.0.1", program_name="testnap", filename="DM_FE", lineno="2", object_type="/event", opcode_name="PCM_OP_READ_OBJ"} 1
brm_opcode_errors_total{application="dmo", opcode="3", opflags="0", dbNumber="0.0.0.1", program_name="testnap", filename="DM_FE", lineno="2", object_type="/event", opcode_name="PCM_OP_READ_OBJ"} 1
brm_opcode_records_total{application="dmo", opcode="3", opflags="0", dbNumber="0.0.0.1", program_name="testnap", filename="DM_FE", lineno="2", object_type="/event", opcode_name="PCM_OP_READ_OBJ"} 0

You use the JMX framework to expose metrics for BRM Java applications, such as Batch Controller, REL daemon, and EAI Java Server (JS), in Prometheus format.

To enable the monitoring of BRM Java applications:

1. Download the latest JMX Exporter .jar file from https://github.com/prometheus/jmx_exporter.

   For a list of compatible versions, see "Additional BRM Software Requirements" in BRM Compatibility Matrix.

2. Create a configuration file named config.yaml in your BRM_home/bin directory.

3. Configure the JMX Exporter by adding the parameters defined in https://github.com/prometheus/jmx_exporter#configuration.

4. To configure BRM Java applications to expose JMX metrics in Prometheus format, edit the component's start script to include the JMX_EXPORTER_OPTS environment variable.

   For example, you would expose JMX metrics for EAI JS by adding the following lines to the start_eai_js script:

   JMX_EXPORTER_OPTS="-javaagent:/scratch/jmx_prometheus_javaagent-version.jar=12347:/scratch/config.yaml" 
   $JAVA -Deai_js=1 ${JMX_EXPORTER_OPTS} -mx256m -ms64m -ss1m com.portal.js.JSMain >>${JSLOG} 2>&1 &

   where version is the JMX Exporter version number.

   The JMX metrics in Prometheus format will be available at the /metrics endpoint using the port configured in the component's start script. In this example, the endpoint for EAI JS metrics would be http://localhost:12347/metrics.

If you re using Web Services Manager in a standalone configuration (not deployed into Oracle WebLogic Server, for example), metrics information can be found in "Working with Metrics" in BRM Web Services Manager.

If you have deployed Web Services Manager into a WebLogic Server, you use the WebLogic Monitoring Exporter to expose metrics for Web Services Manager in Prometheus format.

To enable monitoring of Web Services Manager:

1. Download the latest supported version of WebLogic Monitoring Exporter (getN.N.sh) from https://github.com/oracle/weblogic-monitoring-exporter/releases, where N.N is the version number.

   For a list of compatible versions, see "Additional BRM Software Requirements" in BRM Compatibility Matrix.

2. Edit the wls-exporter-config.yaml file to include the metrics to scrape from BRM. For the list of supported metrics, see "WebLogic-Based Application Metrics".

   For example:

   metricsNameSnakeCase: true
   domainQualifier: true
   #restPort: 7001
   queries:
   - key: name
     keyName: server
     applicationRuntimes:
       key: name
       keyName: app
       componentRuntimes:
         type: WebAppComponentRuntime
         prefix: webapp_config_
         key: name
         values: [deploymentState, contextRoot, sourceInfo, openSessionsHighCount, openSessionsCurrentCount, sessionsOpenedTotalCount, sessionCookieMaxAgeSecs, sessionInvalidationIntervalSecs, sessionTimeoutSecs, singleThreadedServletPoolSize, sessionIDLength, servletReloadCheckSecs, jSPPageCheckSecs]
         servlets:
           prefix: wls_servlet_
           key: servletName
           values: invocationTotalCount
    
   - JVMRuntime:
       prefix: wls_jvm_
       key: name
    
   - executeQueueRuntimes:
       prefix: wls_socketmuxer_
       key: name
       values: [pendingRequestCurrentCount]
    
   - workManagerRuntimes:
       prefix: wls_workmanager_
       key: name
       values: [stuckThreadCount, pendingRequests, completedRequests]
    
   - threadPoolRuntime:
       prefix: wls_threadpool_
       key: name
       values: [executeThreadTotalCount, queueLength, stuckThreadCount, hoggingThreadCount]

3. Run the following command:

   bash getN.N.sh wls-exporter-config.yaml

   The wls-exporter.war is downloaded to your current directory.

4. Deploy the wls-exporter.war in the same WebLogic domain as BrmWebServices.war or infranetwebsvc.war.

WebLogic Monitoring Exporter exposes Web Services Manager metrics in Prometheus format to the http://localhost:8080/wls-exporter/metrics endpoint on the WebLogic Server.

To configure Prometheus to scrape metric data from your Pushgateway endpoint:

1. Edit your prometheus.yaml file to include:
   • The targets to scrape for your components that use Perflib-based monitoring (CM and Oracle DM), REL daemon, Batch Controller, and BRM REST Services Manager, and optionally, node exporter.
   • Prometheus Alertmanager configuration.

   For example:

   global:  
     scrape_interval: 1s  
     evaluation_interval: 15s
   
   alerting:  
     alertmanagers:  
       - static_configs:    
         - targets:       
           - alertmanager_host:9093 
   
   rule_files:  
     - "first_rules.yaml"  
     - "second_rules.yaml" 
   
   scrape_configs:    
     - job_name: 'prometheus'     
        
       static_configs:    
       - targets: ['localhost:9090','localhost:9091','localhost:3000']   
   
     - job_name: 'perflib'   
   
       static_configs:    
       - targets: ['localhost:12345']   
   
     - job_name: 'rel_daemon'     
       
       static_configs:    
       - targets: ['localhost:12346']   
   
     - job_name: 'batch_controller'     
       
       static_configs:    
       - targets: ['localhost:12347']
   
     - job_name: 'brm-rest-services-manager'     
       
       static_configs:    
       - targets: ['localhost:12348']
   
     - job_name: 'node-exporter'     
       
       static_configs:    
       - targets: ['localhost:12349']

   For information about editing this file, see "Prometheus Configuration" in the Prometheus documentation.

2. Configure the alert rules in Prometheus.

   To do so, add alert rules for your components that are similar to the ones shown below to the rules files referenced in prometheus.yaml.

   groups:  
     - name: brm-rsm-alert-rules    
        rules:
         - alert: CPU_UsageWarning
           annotations:
             message: CPU has reached 80% utilization
           expr: avg without(cpu) (rate(node_cpu_seconds_total{job="node-exporter", instance="node_exporter_host:node_exporter_port", mode!="idle"}[5m])) > 0.8
           for: 5m
           labels:
             severity: critical
         - alert: Memory_UsageWarning
           annotations:
             message: Memory has reached 80% utilization
           expr: node_memory_MemTotal_bytes{job="node-exporter", instance="node_exporter_host:node_exporter_port"} 
                 - node_memory_MemFree_bytes{job="node-exporter", instance="node_exporter_host:node_exporter_port"} 
                 - node_memory_Cached_bytes{job="node-exporter",instance="node_exporter_host:node_exporter_port"} 
                 - node_memory_Buffers_bytes{job="node-exporter", instance="1"} > 22322927872
           for: 5m
           labels:
             severity: critical

   For more information about defining alert rules, see "Alerting Rules" in the Prometheus documentation.

   Note:

   You can also configure alert rules and add or remove email recipients in the Grafana user interface. See "Legacy alerting" in the Grafana documentation for more information.
3. Restart Prometheus by running this command:

   ./prometheus --config.file=prometheus.yaml

4. Start monitoring BRM services for their current status by running this command:

   start_service_monitor

5. Start monitoring the CPU and memory used by BRM services by running this command:

   start_memory_monitor

To ensure that you configured Prometheus correctly and that it has started scraping BRM metric data, go to the following URL: http://localhost:9090/graph.

stop_service_monitor
stop_memory_monitor

You can create a dashboard in Grafana for displaying the metric data for your BRM components.

Alternatively, you can use the sample dashboards that are included with the BRM SDK package. To use the sample dashboards, import the JSON files from the BRM_home/PortalDevKit/source/samples/dashboards directory into Grafana. Table 42-3 describes each sample dashboard.

For information about importing dashboards into Grafana, see "Export and Import" in the Grafana Dashboards documentation.