6 Fault Manager

Fault Management system allows you to manage all events, notifications, and alerts generated by either Networks Function (NF) or Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) components. For the Session Border Controller (SBC), Enterprise Session Border Controller (E-SBC), and Oracle Communications Session Monitor (OCSM), the Events and Alarm information is based on the Oracle® standard and proprietary Management Information Bases (MIBs). All SNMP traps generated from these NFs are managed by Oracle SDM Cloud. The NFs send their traps to the Management Cloud Engine (MCE) located on the customer premises which acts as the trap receiver. The MCE then converts the SNMP trap into a REST payload which it sends to the cloud Oracle SDM Cloud SaaS offering. For more information on configuring traps, refer to the appropriate configuration user guide for that product.
The Fault Manager provides views for events and alarms.
  • Events view—Provides a historical view of all events generated by either managed NFs or by Oracle SDM Cloud components. This allows you to track the time and state of when NF traps of Oracle SDM Cloud alerts entered the system and how, for a specific failed resource, the associated events transitions to different states on a row per row basis.
  • Alarm view—Provides a summary view of the latest state of an alarm for a specific failed resource. This table provides only one row for each unique failed resource and updates this row with the latest information as new events for the same failed resource are identified.
For example, consider that "SBC-1" sends a apSysMgmtFanTrap trap which crosses the following states:
  • Fan speed Trap Minor alert: fan speed is more than minor alarm threshold, but less than major alarm threshold.
  • Fan speed Trap Major alert: fan speed is more than major alarm threshold, but less than critical alarm threshold.
  • Fan speed Trap Critical alert: the environment is very bad, such as Fan speed is more than critical threshold.
The Events view displays 3 events in the table for the failed resource Fan for device "SBC-1".
Event 1 Fan speed Minor alert event at Time.0.
Event 2 Fan speed Major alert event at Time.1.  
Event 3 Fan speed Critical alert event at Time.2.
The Alarms view displays only 1 row for the failed resource, displaying only the most current state.
Event 1 Fan speed Critical alarm at Time.2.
The following pre-requisites are required for receiving fault notifications:

  • You must use the sudo password (the password of the NNCentral user account on the server operating system) for the port on which TrapRelay listens. This port is configured during Media Cloud Engine (MCE) installation. For more information, see the Getting Started guide.

    Note:

    If you use port 1024 for the TrapRelay function, root permission is not required.

  • Ensure that SNMP communities and the MIB administrator contact name is configured on your southbound system(s).

  • A trap receiver for each MCE node in a cluster must be configured on each southbound device. Also, the SNMP community defined in the trap receiver must be the same for all MCE cluster nodes.

Alarm and Event Configuration Tasks

The following sections describe the Events table and Alarms table, with their accompanying features. The Events table shows a one to one correspondence with all device traps and generated server events. The Events table maintains the precise history of all events created and recorded. The Alarms table summarizes the Events table by showing the most recent update for the specific categories, failed resources, state and devices in each row.

Note:

Users can view only the alarms and events for the devices to which they have access, however, events and alarms generated by Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) itself are accessible to all users.

Manage How Events are Displayed

  1. Expand the Fault Manager slider and select Events.
  2. Click the More Actions icon and select Set Columns to choose which columns are displayed in the Events table. The following table describes all of the columns available to view.
  3. In the events pane, select an event that you want to view, click the More Actions icon and select View.
  4. In the Event detail dialog box, view the following fields for this specific event:
    • Time Created
    • Description
    • Severity
    • Default Severity
    • Source
    • Source IP
    • Failed Resource
    • Category
    • Trap Name
    • System Up Time
    • Type

Manage How Alarms are Displayed

  1. Expand the Fault Manager slider and select Alarms.
  2. Click the More Actions icon and select Set Columns to choose which columns are displayed in the Alarms table. The following table describes all of the columns available to view.
  3. In the alarms pane, select an alarm that you want to view, click the More Actions icon and select View.
  4. In the Alarm detail dialog box, view the following fields for this specific alarm:
    • Annotation
    • Acknowledged by
    • Time
    • Description
    • Severity
    • Source
    • Source IP
    • Failed Resource
    • Category
    • System Up Time
    • Trap Name
    • Type

Manage the Page View for Events

  1. Expand the Fault Manager slider and select Events.
  2. In the Events pane, you can select from the following actions:

Oracle SDM Cloud Alarm Auto Refresh

  1. Expand the Fault Manager slider and select Alarms.
  2. In the alarms pane, you can select from the following actions:

Search for Alarms or Events by Specifying a Criteria

You can search for events and alarms by specifying one, some, or all of the search selection criteria. For example, you can select alarms for a specific IP address during a specified date-time range.

  1. Expand the Fault Manager slider and select from the following options:
    • Events
    • Alarms
  2. In the alarms or events pane, click Search.
  3. In the Search dialog box, complete the following fields:

Save Alarms or Event Data to a File

You can save event or alarm data in the content area to a comma-separated values (CSV) file that stores table data (numbers and text) in plain-text form.

  1. Expand the Fault Manager slider and select from the following options:
    • Events
    • Alarms
  2. Click Save to file.

    Note:

    The files are saved to your browser's default download location. Only the first 1000 entries can be saved to file.

Delete Alarms or Events

The appropriate administrator privileges must be assigned to delete alarms or events.

Note:

Deleting an alarm in Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) has no affect on the node because the node is unaware that Oracle SDM Cloud displayed the alarm or deleted it from the alarms table.
  1. Expand the Fault Manager slider and select from the following options:
    • Events
    • Alarms
  2. In the alarms or events table, click the alarm or event that you want to remove, click the More Actions icon, and select Delete.
  3. In the Delete dialog box, click Yes to confirm the deletion of the alarm or event.

Specify a Criteria to Delete Alarms and Events

The appropriate administrator privileges must be assigned to delete alarms or events.

Use this task to specify one or more criterion for deleting alarms or events from Oracle® Session Delivery Management Cloud (Oracle SDM Cloud).
  1. Expand the Fault Manager slider and select from the following options:
    • Events
    • Alarms
  2. In the events or alarms pane, click the More Actions icon, and select Delete by criteria.
  3. In the Search dialog box, complete the following fields:

    Note:

    When there is a high number of faults that are being sent from devices, a purge interval of 2 days for events and 7 days for alarms is suggested.
  4. Click OK.

Alarm Specific Configuration Tasks

Alarms play a significant role in determining the overall health of the system. An alarm is triggered when a condition or event happens within the hardware or software of a system (node). Alarms contain an alarm code, a severity level, a textual description of the event, and the time the event occurred. The following sections describe how to configure the way alarms display in Oracle® Session Delivery Management Cloud.

Add an Annotation to an Alarm

  1. Expand the Fault Manager slider and select Alarms.
  2. In the alarms table, click the alarm to which you want to add explanatory note, click the More Actions icon, and select Edit.
  3. In the Edit annotation dialog box, add your explanatory note about this alarm in the Annotation field.
  4. Click OK.

Enable Alarm Acknowledgment

The appropriate administrator privileges must be assigned to acknowledge alarms.

  1. Expand the Fault Manager slider and select Alarms.
  2. In the alarms table, select the alarm that you want to acknowledge and click Acknowledge.
  3. In the Acknowledge dialog box, click Yes.
  4. In the Info dialog box, click OK.
  5. Click the alarm to view an updated Alarm detail dialog box with the Acknowledged by and Last modified fields updated.
  6. Click OK.

Disable Alarm Acknowledgment

The appropriate administrator privileges must be assigned to unacknowledge alarms.

  1. Expand the Fault Manager slider and select Alarms.
  2. In the alarms table, select the alarm that you want to unacknowledge and click Unacknowledge. The Acknowledge dialog box appears.
  3. In the Unacknowledge dialog box, click Yes.
  4. In the Info dialog box, click OK.

Clear an Alarm

The appropriate administrator privileges must be assigned to clear alarms.

Note:

Clearing an alarm in Oracle® Session Delivery Management Cloud (Oracle SDM Cloud)has no affect on the node because the node is unaware that Oracle SDM Cloud displayed the alarm or changed its severity to clear.
  1. Expand the Fault Manager slider and select Alarms.
  2. In the alarms table, select the alarm that you want to clear, click the More Actions icon, and select Clear.
  3. In the Clear dialog box, click Yes.
  4. In the Info dialog box, click OK.

Customize Trap Severity Levels

  1. Expand the Fault Manager slider and select Trap event setting.
  2. In the Trap Event Setting page, select the alarm trap groups you are customizing from the Trap Groups table:

    Note:

    The Oracle SDM Cloud determines the trap groups that you can access.
  3. Select a trap from the Trap OIDs table.
  4. In the Severity Mapping table, select a severity cell from the Current severity column for a trap condition row that you want to modify.
  5. In the drop-down list of severity levels that appears, click the severity level that you want to apply.

    Note:

    The Default severity column serves as a reference point and continues to show the default severity setting for the trap condition.
    The new level appears in the Current Severity column for the trap condition.
  6. Click Apply.
  7. In the success dialog box, click OK.

Customize Product Plugin Event Traps

The trap event setting allows you to override the default severities and customize them. Traps groups are provided for each product plugin that is installed in Oracle SDM Cloud. When you select a trap group the product plugin, SNMP trap (OID) list is provided. For more information on product-specific traps, refer to the appropriate MIB Reference Guide.

See your element manager product plugin documentation for the list of SNMP event traps and their definitions.

  1. Expand the Fault Manager slider and select Trap event setting.
  2. Select a trap group row from the Trap groups table and click OK.

Customize Session Delivery Manager Event Traps

The trap event setting allows you to override the core Oracle SDM Cloud default event trap severities and customize them.

  1. Expand the Fault Manager slider and select Trap event setting.
  2. In the Select dialog box, select the SDM trap group row from the Trap groups table and click OK.
  3. The following table describes the Oracle SDM Cloud product event types and a description that references its respective trap.
    Trap Description
    apUmsNodeUnreachable The trap is generated when the status of a node changes from reachable to unreachable. The trap contains the node ID of the device and the time of the event.
    apUmsNodeUnreachableClear The trap is generated when the status of a node changes from unreachable to reachable. The trap contains the node ID of the device and the time of the event.
    apUmsRegistration The trap is generated when, upon startup, the MCE successfully registers with Oracle SDM Cloud automatically.
    apUmsMCERegistrationClear The trap is generated when, after an unsuccessful registration attempt, the MCE is able to register with Oracle SDM Cloud successfully.
    apUmsServiceStarted When the Oracle SDM Cloud starts and initializes each of its sub-services, this trap displays the status of each sub-service.
    apUmsONSThrottling When email notifications are temporarily suspended, as the OCI Notification Service is unable to deliver messages.
    apUmsONSThrottlingClear The trap generated when the OCI Notification Service is able to deliver messages again.
    apUmsDBUsage When the DB usage crosses the threshold value:
    • WARNING - Total DB utilization (default + optional) across > 80% and < 85%
    • MAJOR - Total DB utilization (default + optional) across >= 85% and < 90%
    • CRITICAL - Total DB utilization (default + optional) across >= 90%
    apUmsDBUsageClear When DB usage returns to < 80%.
    apUmsInvalidMCERegistration When an MCE tries to connect to a site using invalid registration ID or when an MCE tries to connect to a site which already has a MCE connection.
    apUmsTestTrap Checks the connectivity to a specific trap receiver.

Search on Trap OIDs

The Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) supports searching on Trap OIDs to find specific traps.

To search for traps using Trap OIDs:
  1. Expand the Fault Manager slider and click Trap Event Settings.

    The Trap Event Settings page displays.

  2. Trap Groups—Select the Trap Group in which you want to search.
  3. Trap OIDs—Enter the Trap OID you want to search on. As you start to type, the Trap Descriptor list shows all matches which contain the typed string.
  4. Trap Descriptor—Select the Trap you are searching on.

    The trap displays in the Severity Mapping table.

Manage Trap Receivers

The Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) supports trap forwarding to northbound trap receivers using the Management Cloud Engine (MCE) for local trap forwarding of traps generated by NFs managed by Oracle SDM Cloud and the Oracle SDM Cloud itself in the ITUX.733 format and Pass through (the same format as generated by the NF itself).

To enable this functionality, users must configure trap receivers and trap filters, then create trap forwarding maps.

The Trap Receivers page displays a table with a list of all trap receivers configured on the Oracle SDM Cloud. Users can add, edit, delete, refresh the list, and synchronize traps.

The Trap Receivers table displays the following information: The Trap Receivers page contains the following buttons:

Add a Trap Receiver

To add a trap receiver:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Receivers.

    The Trap Receivers page displays.

  2. Click +Add.

    The Add Trap Receiver page displays.

  3. Complete the following fields:
  4. Click Apply to create the trap receiver or Cancel to discard your inputs and return to the Trap Receiver screen.

    The newly created trap receiver displays in the Trap Receiver table.

Edit a Trap Receiver

To edit a trap receiver:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Receivers.

    The Trap Receivers page displays.

  2. Select the trap receiver you want to edit by checking its checkbox and click Edit.
  3. Update the trap receiver as necessary.
  4. Click Apply to save your changes or Cancel to discard the changes and return to the Trap Receiver screen.

Delete a Trap Receiver

To delete a trap receiver:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Receivers.

    The Trap Receivers page displays.

  2. Select the trap receiver(s) you want to delete by checking its checkbox and click Delete.

    A confirmation pop-up displays.

  3. Click Yes to delete the trap receiver or No to cancel.

Synchronize Trap Receivers

The Synchronization feature allows users to synchronize the traps between Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) and trap receivers.

Only one trap receiver can be synchronized at a time and the Synchronize button is disabled when either no trap receiver is selected or more than one trap receiver is selected.

To synchronize a trap receiver:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Receivers.

    The Trap Receivers page displays.

  2. Select the trap receiver you want to synchronize by checking its checkbox and click Synchronize.

    The Trap Synchronization dialog box displays.

  3. Enter values for the following fields:
  4. Click Apply to synchronize the traps or Cancel to quit the action.

    When a user clicks Apply, either the traps from events or alarms in the Oracle SDM Cloud, filtered based on the criteria specified, are sent to the Management Cloud Engine (MCE). The MCE then forwards these traps to the trap receiver. If the selected trap receiver is associated with multiple MCEs, the filtered traps are sent only to one MCE.

    The trap filter Preview displays the details of the trap filter selected, as follows:
    • If the selected trap filter has a trap name, the preview displays:
      • Filter Name
      • Trap Name
      • Forward Clear Trap association
    • If the selected trap filter has trap severity, the preview displays:
      • Filter Name
      • Trap Severity
      • Forward Clear Trap association
    • If the selected trap filter has trap source, the preview displays:
      • Filter Name
      • Trap Source
      • Suppress Forwarding Option (true/false)
      • Forward Clear Trap association

Manage Trap Filters

The Trap Filters page displays a table with a list of all trap filters configured on the Oracle® Session Delivery Management Cloud (Oracle SDM Cloud). Users can add, edit, delete, and refresh trap filters.

The Trap Filters table displays the following information:
The Trap Filters page contains the following buttons:

Add a Trap Filter

Users can create up to 100 trap filters.

To add a trap filter:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Filters.

    The Trap Filters page displays.

  2. Click +Add.

    The Add Trap Filters page displays.

  3. Complete the following fields:

    Note:

    You must choose to use either Trap Name, Trap Severity, or Trap Source to add a trap filter. Once a value is added to one of those parameters, the other two are disabled.
  4. Click Apply to create the trap filter or Cancel to discard your inputs and return to the Trap Filter screen.

    The newly created trap filter displays in the Trap Filter table.

Edit a Trap Filter

To edit a trap filter:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Filters.

    The Trap Filters page displays.

  2. Select the trap filter you want to edit by checking its checkbox and click Edit.
  3. Update the trap filter as necessary.
  4. Click Apply to save your changes or Cancel to discard the changes and return to the Trap Filter screen.

Delete a Trap Filter

To delete a trap filter:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Filters.

    The Trap Filters page displays.

  2. Select the trap filter(s) you want to delete by checking its checkbox and click Delete.

    A confirmation pop-up displays.

  3. Click Yes to delete the trap filter or No to cancel.

Manage Trap Forwarding Maps

The Trap Forwarding Map page displays the association of Management Cloud Engines (MCE) with the trap receivers and trap filters, allowing users to view, create, and enable new trap receiver and trap filter mappings with MCEs.

The Trap Forwarding Map page contains two tables, the first showing the Association of MCE with trap receivers and the second showing the Association of MCE with trap filters. Both tables contain the following columns:
This page contains the following fields and buttons:

Enable or Disable Trap Forwarding

To enable or disable trap forwarding:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Forwarding Map.

    The Trap Forwarding Map page displays.

  2. Browse to the map(s) you want to enable or disable and check or uncheck the checkbox.
  3. Click Save to save your updates or Cancel to undo any unsaved changes.

Generate a Test Trap

The Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) provides users a way to generate a test trap, ensuring a trap is configured properly.

To generate a test trap:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Forwarding Map.

    The Trap Forwarding Map page displays.

  2. Browse to the trap receiver you want to test and click the Generate Test Trap icon next to the IP address/FQDN (the icon cirlced in red below).

    This screenshot shows the generate test trap icon.

    A Confirmation dialog box displays.

  3. Click Yes to continue to generate a test trap or No to cancel the action.

    The Oracle SDM Cloud provides either a Success or Error message.

  4. Browse to Fault Manager, Events to see details about the test trap.

Associate Trap Receivers and Trap Filters

The Oracle® Session Delivery Management Cloud (Oracle SDM Cloud) provides a two step process for users to associate trap receivers and trap filters, first selecting the Management Cloud Engine(s) (MCE), then selecting the trap receivers and trap filters with which to associate.

To associate trap receivers and trap filters:
  1. Expand the Fault Manager slider and select Trap Forwarding, Trap Forwarding Map.

    The Trap Forwarding Map page displays.

  2. Click Associate.

    The Select MCEs page displays, listing all of the MCEs available.

  3. Select the MCE(s) to use for the association.
  4. Click Next.

    A confirmation dialog displays explaining that if you continue, any existing associations for the selected MCEs will be removed and the new associations will be applied to all MCEs in cross-site with them.

  5. Click Yes to continue or No to cancel the action.

    If you continue, the Trap Receivers & Filters page displays, showing all available trap receivers and trap filters.

  6. Select the trap receivers to associate with the MCE(s).

    Note:

    A maximum of 3 trap receivers can be selected.
  7. Select the trap filters to associate with the MCE(s).

    Note:

    A maximum of 50 trap filters can be selected.
  8. Click Finish to save the mappings.

    The Trap Forwarding Map displays showing the updated mappings. By default all of the trap receivers and trap filters associated with the MCE are enabled.

    When users assign a site with an MCE to a site with another MCE, the trap receiver and trap filter mappings are applied for all MCEs associated with the site.