4 Using the Enterprise Manager Plug-In for Oracle GoldenGate

The Target page displays the following tabs: Metrics, Logs, and Configurations, which enable you to monitor metrics and to alert users about specific metric results, search and interpret audit logs, and to configure parameter files respectively.

Enabling Audit Logging

Messages are automatically logged to the server log file for all Oracle GoldenGate actions, such as start and stop as well as for file access, such as parameter, report, and discard.

This topic discusses how to enable these logs for auditing. To enable or disable an audit for a specific action, run the following commands from the oms/bin directory. Enter the values you want to use for each setting:

emcli update_audit_settings
  -audit_switch="ENABLE|DISABLE"
  -operations_to_enable="name_of_operations_to_enable"
  -operations_to_disable="name_of_operations_to_disable"
  -externalization_switch="ENABLE|DISABLE"
  -directory="directory_name"
  -file_prefix="file_prefix"
  -file_size="file_size"
  -data_retention_period="data_retention_period"

You can enable or disable one or more operations using the -operations_to_enable flag. Here is a list of the Oracle GoldenGate operations and the values to use.

Operation Value

Start Oracle GoldenGate process

OGG_START_TARGET

Stop Oracle GoldenGate process

OGG_STOP_TARGET

Kill Oracle GoldenGate process

OGG_KILL_TARGET

View report file

OGG_VIEW_REPORT

View discard file

OGG_VIEW_DISCARD

View ggserr.log contents

OGG_VIEW_GGSERRLOG

Edit parameter file

OGG_EDIT_PARAM

Operations can be combined and separated by a semicolon (;). The following is the command to enable all audit logging for the Enterprise Manager Plug-In for Oracle GoldenGate.

emcli update_audit_settings -operations_to_enable="OGG_START_TARGET;OGG_STOP_TARGET;OGG_KILL_TARGET;OGG_VIEW_REPORT;OGG_VIEW_DISCARD;OGG_VIEW_GGSERRLOG;OGG_EDIT_PARAM"

Viewing the Audit Logs

A Cloud Control user with Super Administrator privileges has the access to search for and view audit logs. This topic discusses how to search for and view a specific audit log using Cloud Control.

To view a specific audit log:

  1. Select Setup, Security, Audit Data to open the Audit Data page.
  2. Select your search criteria, such as date range, operations, or status.

    You can select specific operations from the Operations drop-down menu. For example, you can select all the operations that begin with OGG.

  3. Click Search to display the search results in a grid format.
  4. To view the audit log, select an audit log from the search results list.
  5. Once selected, you can view audit log information in the Audit Record Details region, as shown. The Audit Record Details are updated automatically for each audit log you select. Click the General, Client Information, CMS Information, and Operation Specific Information tabs for specific information.

For additional information about the auditing feature in Enterprise Manager, see Configuring Auditing Frameworkin Enterprise Manager Cloud Control Getting Started Guide and Configuring the Audit Data Export Servicein Enterprise Manager Cloud Control Security Guide.

Displaying Discard Files

The Discard files are displayed in the Discards sub tab under the Logs tab.

To display Discard files:
  1. In the Oracle GoldenGate Home Page, click Oracle GoldenGate Replicat or Oracle GoldenGate Extract to display the Detail Metrics, Logs, and Configuration tabs.
  2. Click the Logs tab and then the Discards tab to display the discard file contents.
    If there are any discard files specified in the parameter files and the file exists in Oracle GoldenGate Core, then these files are also displayed in the Discards tab as a list of Discard Files. You can specify the names of the folder, files, or file extensions of your choice. The default discard files are read from the dirrpt folder, for example, dirrpt/processName*.dsc. Note that the file name is an absolute path of the discard file or path related to the OGGCORE location and file extension can be any of the following: .txt, .discard, or .dsc. You can specify multiple discard files as follows:
    DISCARDFILE dirrpt/File1.txt, APPEND, MEGABYTES 
    DISCARDFILE dirdat/File2.txt, APPEND, MEGABYTES

Note:

  • If you want to view a discard file listed in a respective parameter file, then restart the corresponding Extract or Replicat process, select either of the following Oracle GoldenGate Replicat or Oracle GoldenGate Extract. Then click the Logs tab and then the Discards tab.

  • If you want to view all the discard files in a respective parameter file, then set the discard file location more than once, then restart the corresponding Extract or Replicat process, select either of the following Oracle GoldenGate Replicat or Oracle GoldenGate Extract. Then click the Logs tab and then the Discards tab.

Note:

The Logs tab contains the following 3 sub tabs: GGSERR log, Discards, and Reports. The GGSERR log tab shows the file contents of the ggserr.log file, the Report tab show report file contents and the Discard tab show discard file contents.

Editing Files on the Configuration Tab

The Configuration tab displays the entire parameter file in view mode. At runtime, new tabs get added on the Configuration tab for the Oracle GoldenGate properties file. There can be multiple such tabs for these files. You can modify the content of the property and parameter files.

To modify the files on the Configuration tab:

  1. In the Configuration tab, click Edit to reopen the parameter file in an edit mode.

  2. Click the filename (hyperlink) in the parameter file to create a new tab next to the parameter tab. The tab title is displayed as the include/obey file name.

    Note:

    The absolute path to the file is displayed at the bottom of the tab. The content of the existing include/obey file is displayed in new tab. If the file doesn't exist (for example, user-typed new file name in editing mode) the empty tab is displayed with a warning message above the text area.
  3. Click Save. If you haven't modified any content, then no action is taken. If you modified the content was modified and then clicked Save , then the new content is saved.

If you want to revert the changes to the parameter configuration files, then click Reload. Changes made to the parameters file in the text area is discarded.

If you want to verify whether the property (or parameter) file is edited, then:

  1. Edit the properties file from the Oracle GoldenGate Enterprise Manager Plug-In user interface and save it.

  2. Go to the Oracle GoldenGate Core and check for these changes.

  3. Add or remove content from the Oracle GoldenGate side and click Refresh on the Oracle GoldenGate Enterprise Manager Plug-In side.

    Existing properties files are displayed in the Oracle GoldenGate Enterprise Manager Plug-In UI.

Monitoring Processes

This topic discusses the metrics used to monitor the Oracle GoldenGate processes.

The following processes are monitored by Oracle GoldenGate Enterprise Manager Plug-In:

  • Extract - An Extract process picks up changes from transaction logs and writes them to a trail. That trail is picked up by a Replicat process and changes are written to the target database. If the Replicat is across the network, then the trail is across the network. If the network is down the changes are lost.

    Best practice is to always write changes to a trail that is local to the Extract. Another Extract is set up as a data pump in the same location and reads data from the local trail and passes it across the network. In this way, changes are not lost if the network goes down.

  • Replicat - The Replicat process runs on the target system, reads the trail on that system, and applies the operations to the target database.

  • Manager - The Manager process is the administrative process of an Oracle GoldenGate classic instance. It controls all of the other Oracle GoldenGate processes in the instance. Part of its role is to generate information about critical monitoring events, which it passes to the agent.

    • For target types Replicat, Extract, and Manager, you can control the process though start, stop, kill, and resume actions.

    • For target types Extract and Replicat, you can view and edit the associated configuration files.

    • For target types Extract and Replicat, you can view all the associated report and discard files.

  • You can also monitor the status of Service manager, Administration Server, Performance Metrics Server, Distribution Server, and the Receiver Server.

Here is a list of the metrics used for the Extract and Replicat processes.

Metric Description

Checkpoint Position

Valid for Extract and Replicat

Shows a composite representation of the checkpoints that were persisted to disk most recently by Extract or Replicat. The value is captured by the monitoring agent when the attribute is published, right after the checkpoint gets persisted.

Extract creates read and write checkpoints, and Replicat creates only read checkpoints. Each individual checkpoint within the composite Checkpoint Position consists of the RBA (relative bye address) of a record in the transaction log or trail (depending on the process and whether it is a read or write checkpoint) and the sequence number of the log or trail file that contains the record. There can be a series of read checkpoints in multiple data source log files (such as Extract from Oracle Real Application Cluster), and/or multiple write checkpoints such as in Extract configurations with multiple trail files.

Valid values: Different databases use different representations of the position of a record in the log. Therefore, instead of numeric values, Checkpoint Position is published as a string of text characters encoded in UTF8. For each individual checkpoint within Checkpoint Position, the following are shown the way that they are returned by the GGSCI SEND group-name STATUS command:

  • The values of the RBA (relative byte address)

  • The file sequence number

  • The time stamp

Delta Deletes

Valid for Extract and Replicat

Shows the number, since the metric was last reported, of DELETE operations that were processed by the selected Oracle GoldenGate process in its current run session.

Valid values: A positive integer

Delta Discards

Valid for Extract and Replicat

Shows the number, since the metric was last reported, of DISCARD operations that were processed by the selected Oracle GoldenGate process in its current run session. The records are written to the discard file that is associated with the process.

Valid values: Positive integer.

Delta Executed DDLs

Valid for Extract and Replicat

Shows the count of executed Data Definition Language (DDL) operations that were processed by the selected Oracle GoldenGate process since the last sample time.

Valid values: Positive integer

Delta Ignores

Valid for Extract

Shows the number of data manipulation language (DML) operations that through an error were configured to be ignored since the last sample time.

Valid values: Positive integer

Delta Inserts

Valid for Extract and Replicat

Shows the number of data manipulation language (DML) INSERT operations that were processed by the selected Oracle GoldenGate process since the last sample.

Valid values: A positive integer

Delta Operation Per Second

Valid for Extract and Replicat

Shows the number of operations (per second) that were processed by the selected Oracle GoldenGate process since the last sample.

Valid values: A positive integer

Delta Operations

Valid for Extract and Replicat

Shows the total number of Data Definition Language (DDL) and Data Manipulation Language (DML) INSERT, UPDATE, DELETE, AND TRUNCATE operations that were processed by the selected Oracle GoldenGate process since the last sample.

Valid values: A positive integer

Delta Row Fetch Attempts

Valid for Extract

Shows the number of row fetch attempts that were processed by the selected Oracle GoldenGate process since the last sample. A fetch must be done occasionally to obtain row values when the information is incomplete or absent in the transaction log.

Valid values: Positive integer

Delta Row Fetch Failures

Valid for Extract

Shows the number of row fetch failures that were processed by the selected Oracle GoldenGate process since the last sample. A fetch must be done occasionally to obtain row values when the information is incomplete or absent in the transaction log

Valid values: Positive integer

Delta Truncates

Valid for Extract and Replicat

Shows the number of TRUNCATE operations that were processed by the selected Oracle GoldenGate process in its current run session since the last sample.

Valid values: A positive integer

Delta Updates

Valid for Extract and Replicat

Shows the number of UPDATE (including primary key updates) operations that were processed by the selected Oracle GoldenGateprocess in its current run session since the last sample.

Valid values: A positive integer

End of File

Valid for Extract and Replicat

Shows whether or not the selected process has reached the end of the input from its data source (transaction log or trail file).

Valid values: TRUE (at end of file) or FALSE.

Note:

End of File metrics value 0 means FALSE. For the alert template, ensure to use the stored metric values 0 and 1, where 0 means FALSE and 1 means TRUE.

Lag (sec)

Valid for Extract and Replicat

Shows the time difference between the Last Operation Timestamp and the Last Processed Timestamp. This attribute represents the true lag between theOracle GoldenGate process and its data source. This lag value should match the value that is returned from the GGSCI command SEND groupGETLAG .

Valid values: The lag time, in seconds

Last Checkpoint Timestamp

Valid for Extract and Replicat

Shows the time when the last checkpoint was written by the process.

Valid values: Datetime value in the format of MM/DD/YYYY HH:MM:SS {AM | PM}, for example: 01/14/2011 09:36:32 AM.

Last Operation Timestamp

Valid for Extract and Replicat

Shows the time when an operation (INSERT, UPDATE, DELETE) was committed in the data source, as recorded in the transaction log.

Valid values: Datetime value in the format of MM/DD/YYYY HH:MM:SS {AM | PM}, for example:01/14/2011 09:36:32 AM

Last Processed Timestamp

Valid for Extract and Replicat

Shows the time when a valid record was returned to the selected process. For Extract, this time value is assigned when the record is processed after the container transaction commits (not the time when the record is read from the transaction log). For a Data Pump or Replicat, this time value is returned immediately, because all transactions in the trail are known to be committed.

Valid values: Date time value in the format of MM/DD/YYYY HH:MM:SS {AM | PM}, for example: 01/14/2011 09:36:32 AM

Message

Valid for Extract and Replicat

The message includes the following information:

  • Message code number of an event message from the Oracle GoldenGate error log.

    Valid values: The numerical code of an Oracle GoldenGate event message in the event log, for example, OGG-00651.

  • Message Date: Timestamp of an event message from the Oracle GoldenGate log.

    Valid values: A datetime value in the form of YYYY-MM-DD HH:MM:SS (in 24-hour clock format)

  • Message Text: Text of an event message from the Oracle GoldenGate error log.

    Valid values: A text string from the message.

Name

Valid for Extract and Replicat

Name of the selected object.

Valid values: Name of the object as displayed in the Oracle GoldenGate Monitor interface.

Seconds Since Last OGG Checkpoint

Valid for Extract and Replicat

Time (in seconds) since the last OGG checkpoint.

Start Time

Valid for Extract and Replicat

Shows the time that an Oracle GoldenGate component received its startup information after it has been created.

Valid values: 64-bit Julian GMT time stamp in microseconds

Status

Valid for Extract and Replicat

Shows the run status of the selected process.

Valid values: Starting, Running, Stopped, Abended, or Aborted.

Total Deletes

Valid for Extract and Replicat

Shows the total number of DELETE operations that were processed by the selected Oracle GoldenGate process in its current run session.

Valid values: A positive integer

Total Discards

Valid for Extract and Replicat

Shows the total number of operations that were discarded by the selected Oracle GoldenGate process in its current run session. The records are written to the discard file that is associated with the process.

Valid values: Positive integer.

Total Executed DDLs

Valid for Extract and Replicat

Shows the total number of Data Definition Language (DDL) operations that were processed by the selected Oracle GoldenGate process in its current run session.

Valid values: Positive integer

Total Ignores

Valid for Extract

Shows the total number of Data Manipulation Language (DML) operations that were ignored by the process in its current run session. Errors are included in the Total Ignores metric.

Valid values: Positive integer

Total Inserts

Valid for Extract and Replicat

Shows the total number of Data Manipulation Language (DML) INSERT operations that were processed by the selected Oracle GoldenGate process in its current run session. The statistic reflects the total operations performed on all of the tables that are specified in the parameter file for that process. Note: If any tables are mapped to targets in the Extract configuration, the statistics will reflect the total operations for all of the targets.

Valid values: A positive integer

Total Operations

Valid for Extract and Replicat

Shows the total number of Data Definition Language (DDL) and Data Manipulation Language (DML) INSERT, UPDATE, DELETE, and TRUNCATE operations that were processed by the selected Oracle GoldenGate process in this current run session.

Valid values: A positive integer

Total Row Fetch Attempts

Valid for Extract

Shows the total number of row fetches that the selected process performed in its current run session. A fetch must be done sometimes to obtain row values when the information is incomplete or absent in the transaction log.

Valid values: Positive integer

Total Row Fetch Failures

Valid for Extract

Shows the total number of row fetches that the selected process was unable to perform in its current run session.

Valid values: Positive integer

Total Truncates

Valid for Extract and Replicat

Shows the total number of TRUNCATE operations that were processed by the selected Oracle GoldenGate process in its current run session. The statistic reflects the total operations performed on all of the tables that are specified in the parameter file for that process. Note: if any tables are mapped to targets in the Extract configuration, the statistics will reflect the total operations for all of the targets.

Valid values: A positive integer

Total Updates

Valid for Extract and Replicat

Shows the total number of UPDATE (including primary key updates) operations that were processed by the selected Oracle GoldenGate process in its current run session. The statistic reflects the total operations performed on all of the tables that are specified in the parameter file for that process. Note: If any tables are mapped to targets in the Extract configuration, the statistics will reflect the total operations for all of the targets.

Valid values: A positive integer

On the Oracle GoldenGate Home page, the status is displayed in String, and internally, the values are in numeric. Numeric values are shown in the Metric History page and alerts are created based on these numeric values.

The following table provides list of Metric status values of the Extract and Replicat processes that are stored in the Enterprise Manager Plugin repository:

ProcessStatus Mapped Value Integer Values for Status
ProcessStatus.REGISTERED_STATE 2
ProcessStatus.INITIALIZING_STATE 3
ProcessStatus.RUNNING_STATE 7
ProcessStatus.STOPPING_STATE 8
ProcessStatus.FORCESTOPPING_STATE 9
ProcessStatus.STOPPED_STATE 10
ProcessStatus.FORCESTOP_STATE 11
ProcessStatus.ABEND_STATE 12
ProcessStatus.KILLED_STATE 13
ProcessStatus.UNRESPONSIVE_STATE 16
ProcessStatus.UNKNOWN_STATE 0

Here is a list of the metrics used for the Manager process.

Metric Description

Host Name

Shows the name of the host system.

Valid values: The fully qualified DNS name of the host, or its IP address

Manager Port

Shows the port on which the Manager process of the Instance is running on its local system. The default port number is 7809, but a different port could be specified for this Manager and can be identified by viewing the Manager parameter file or by issuing the INFO MANAGER command in GGSCI (if Manager is running).

Valid values: The port number for the Manager process, as specified in the Manager parameter file

Start Time

Shows the time that an Oracle GoldenGate component received its startup information after it has been created.

Valid values: 64-bit Julian GMT time stamp in microseconds

Version

Indicates the version of Oracle GoldenGate that the selected Oracle GoldenGate Instance represents.

Valid values: X.x.x (major, minor, and maintenance version levels), for example 11.1.1

Working Directory

Shows the directory that contains the Manager executable file for the selected Oracle GoldenGate Instance. This is the home directory of the Oracle GoldenGateinstallation.

Valid values: The full path name of the directory

Starting, Stopping, or Killing Extract and Replicat Processes

You can use the start, stop, or kill the Extract and Replicat processes from the OGG Home page itself.

To start, stop, or kill Extract and Replicat processes:
  1. Go to the OGG Home page.
    (Optional) Enter the result of the step here.
  2. Select either the Extract or Replicat Process.
  3. Click one of the following icons: Start, Stop, or Kill.

Monitoring Current Oracle GoldenGate Metrics and Historical Trends

If you are interested in viewing the data pattern of the targets and analyzing the historical trends, then you can view the details under Metric Data. You can also modify the thresholds to generate alerts.

To monitor the Oracle GoldenGate metrics and historical trends:
  1. Click the OGG Home tab in the Oracle GoldenGate Home page.
  2. Click a target in the Target Name column to display the corresponding process page (either a REPLICAT or an EXTRACT page).
  3. Click a metric from the Metrics tab to display the All Metrics page and select a Metric, such as Total Deletes to display the details, such as Statistics, Thresholds, Metric Value History chart, and Metric Alert History table.
    You can also view the Metric Value History in a tabular format, apart from the chart that is displayed by default.
  4. Click Table View to display the Metric Value History table.
  5. Click Modify Thresholds... to display the Modify Thresholds page.
  6. Set the following values:
    Threshold
    Sample Values
    Warning Threshold
    1. If the lead value is 1, then a warning alert is generated.
    Critical Threshold
    4. If the total lead value is greater than or equal to 4, then an alert is generated.
    The generated alert is displayed in the Metric Alert History table.
  7. Click the Alerts icon in the top-right corner of the All Metrics page to display the Metric Events details.
    All the metrics of the corresponding process along with the alerts are displayed in the Metric Events page.
  8. In the Metric Alert History table, click the Details icon to display the Event Details and Metric Details.

Generating Automatic Alerts and Incidents When Thresholds are Breached

The notification system allows you to notify Enterprise Manager administrators when specific incidents, events, or problems arise. In addition to notifying administrators, the notification system can perform actions such as executing operating system commands (including scripts) and PL/SQL procedures when an alert is triggered. This capability allows you to implement automatically specific IT practices under particular alert conditions.

To generate automatic alerts and incidents:
  1. Click the OGG Home tab in the Oracle GoldenGate Home page.
  2. Click a target in the Target Name column to display the corresponding process page (either a REPLICAT or an EXTRACT page).
  3. Select the process drop-down list and click Monitoring and then click Metric and Collection String to display the Metrics and Collection Settings page.
    There are two tabs on this page: Metrics and Other Collected Items. The Metrics tab displays all the configured metrics for the selected process. From the View drop-down list on this page, you can select the following to view the corresponding details: All metrics, Metrics with Thresholds, Metrics with Adaptive Thresholds, and Metrics with Time based Static thresholds.
  4. Select All metrics and specify the Warning Threshold and Critical Threshold in the Metric table, and then click OK to save the changes.
    A threshold gets created for the respective metrics.
Once you have created a threshold, you need to set the incident rule.

Creating an Incident Rule

If you want to create incidents to be reported for a particular threshold, you need to create an incident rule. An incident rule instructs Enterprise Manager to take specific actions when incidents, events, or problems occur, such as performing notifications. An incident rule set is a collection of rules that apply to a common set of objects such as targets (hosts, databases, groups), jobs, metric extensions, or self updates, and take appropriate actions when there are events and incidents. An event is a significant occurrence of interest on a target that has been detected by Enterprise Manager. An incident is a set of significant events or combination of related events that pertain to the same issue. Create your incident rule sets and subscribe to them so that you are notified every time there is an event or incident.

To create an incident Rule:
  1. In the Home page, click Enterprise, select Monitoring, and then click Incident Manager to display the Incident Manager page. All the open incidents are listed in this page.
    Alternatively, from the Home page, click Targets, and in the respective process page, click any Incident link in the Incident table to display the Incident Manager page that lists all the incidents for the selected target.
  2. To create an Incident Rule, in the Home page, click Setup, select Incidents, and then click Incident Rule.
  3. In the Incident Rule page, click Create Rule Set... to display the Create Rule Set page.
  4. Enter the following details to create a Rule Set:
    • Name: Enter a name for the Rule set.

    • Applies To: Select an option, such as Targets from the Applies To drop-down list.

    • In the Targets area, select a Targets, such as Oracle GoldenGate Extract from the All targets of types drop-down list to create incident rule sets for all Oracle GoldenGate Extract targets.

  5. In the Rules area, click Create... to display the Select Type of Rule to Create dialog box.
  6. Select Incoming events and updates to events and click Continue.
  7. In the Create New Rule: Select Events page, select Metric Alert from the Type drop-down list to define incident rule for a metric alert.
  8. Click Add to display the Select Specific Metric Alert page.
  9. Select a target type, such as Oracle GoldenGate Extract from the Target Type drop-down list, and then click Search to display the search results in a table.
  10. Select a metric, such as Total Insert from the table
  11. From the Select Severity and Corrective Action Status area, select a value, such as All from the Severity drop-down list, and then click OK.
    When a threshold is reached for the Total Inserts metrics, an alert is displayed.
  12. In the Create New Rule: Add Actions page, click Next to display the Create New Rule: Add Actions page to add an action for the incident rule.
  13. Click Add to display Add Conditional Actions page to add any action, such as sending an email notification.
  14. In the Send Notifications area, enter an email address in the E-mail To field, and then click Continue.
  15. In the Create New Rule: Add Actions page, click Next to specify a name and description for the incident rule in the Create New Rule: Specify Name and Description page, and then click Next to display the Create New Rule: Review page.
  16. Click Continue to go back to the Incident Rules page.
    You can view the new rule in the Incident Rules table.
  17. In the Home page, click Target, select Golden Gate to display the OGG Home and Manage Agents tabs.
  18. In the OGG Home tab, click the respective Target Name to display the corresponding process page, with the Metrics, Logs, and Configuration tabs.
  19. Click the respective metrics, such as Total Inserts, in the Metrics tab to view the notification that is set. An email is also sent to the recipient’s email address. Ensure that you configure the notifications as described in the next steps:
  20. In the Home page, click Setup, and select Notifications, and then click Mail Servers to display the Mail Servers page.
  21. In the Outgoing Mail (SMTP) Servers page, click Edit to display the Outgoing Mail (SMTP) Server dialog box.
  22. Enter the Host, Port, User Name, and Password.
  23. Select SSL/TLS as a Use Secure Connection option.
  24. Click OK to display the Mail Servers page.
  25. In the Outgoing Mail (SMTP) Servers page, click Test Mail Servers to display an Information in the top of the Mail Servers page.
    An Information is displayed in the

Sending Email Alerts

You can configure Enterprise Manager to send email to administrators when a metric alert threshold is reached.

See Sending Email for Metric Alerts in Enterprise Manager Cloud Control Administrator’s Guide for details about how to set up an email alert.