Fault

Introduction

Learn how to use the following applications provided by the Oracle Communications Unified Assurance COM SDK:

• MIB2FCOM: Generates FCOM definition files from Management Information Base (MIB) files.

• FCOM2RULES: Generates foundation rules files from the curated FCOM files.

• FCOM2TEST: Generates synthetic test traps to test the workflow of the curated FCOM file.

See Foundation Rules and Device Support Overview for more information.

Generating the FCOM Definition Files, Rules Files, and Tests

To generate the FCOM definition files, rules files, and tests:

1. On Oracle Linux 8 systems, add the Net-SNMP libraries to the $LD_LIBRARY_PATH environment variable by running the following command:

   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$A1BASEDIR/vendor/net-snmp/lib
   

2. Convert MIB files to FCOM files by running one of the following commands:

   • To convert a single MIB file:

     $A1BASEDIR/bin/sdk/MIB2FCOM --in=<MIB_name>.mib --out=<MIB_name>-FCOM.json --use_parent_mibs
     

     where <MIB_name> is the name of the MIB you are converting, such as ZXDUMIB.

   • To convert all MIBs that contain traps in a single directory , put your temporary working files into a subdirectory of $A1BASEDIR/distrib/mibs, and run the following command from that subdirectory:

     $A1BASEDIR/bin/sdk/MIB2FCOM --in=. --out=<MIB_type_name>-FCOM.json --use_parent_mibs
     

     where <MIB_type_name> is the name of the type of MIBs you are converting, such as NET-SNMP.

     This option is useful when you have many dependencies and you want to keep your working directories clean.

3. Open the FCOM JSON file in a text editor and curate it. See FCOM Curation for details about which fields can be curated.

   Note:

   Oracle recommends working with the MIB files open in a MIB browser during curation to use as a contextual reference.

4. Convert the curated FCOM file to a foundation rules file by running the following command:

   $A1BASEDIR/bin/sdk/FCOM2Rules <MIB_name>.json >> <MIB_name>.foundationrules
   

   The application prints to the standard output, which lets you verify the code before writing it to the file.

5. Generate the test file by running the following command:

   $A1BASEDIR/bin/sdk/FCOM2Test ZXDUMIB-FCOM.json >> ZXDUMIB.test.sh
   

   The application prints to the standard output, which lets you verify the tests before writing to a file. The application generates tests based on the test/tests fields from the FCOM.

6. Upload the Foundation rules file to Unified Assurance:

   1. In a browser, log in to the Unified Assurance UI.

   2. From the Configuration menu, select Rules.

   3. Expand the following folders: path and select the trap folder:

      Core Rules (core)/Default read-write branch (default)/collection/event

   4. Select the trap folder.

   5. If a custom folder does not already appear under the trap folder, add one.

   6. Select the custom folder.

   7. On the menu bar, click Upload.

   8. Enter a name and browse to select the <MIB_name>.foundationrules file.

   9. Click Submit.

   10. Restart the Event Trap Aggregator service.

7. In the command line, set the environment variables for the tests by running the following commands:

   export BASEDIR=<Devices_or_MIBs_directory>
   export HOSTFQDN=<Testing_host_FQDN>
   

   where <Devices_or_MIBs_Repository> is the devices repository or directory containing MIBs and <Testing_host_FQDN> is the FQDN or IP of the Unified Assurance server you will use for testing.

8. Run the <MIB_name>.test.sh file.

9. Review the events that are appearing on the Event List in the UI. If necessary, repeat the FCOM curation process to correct any issues until no more appear.

FCOM Curation

Fields that you can modify are marked with an asterisk ( * ). Other fields that are not marked should not be modified during the curation process.

• "@objectName": "JUNIPER-VPN-MIB::jnxVpnIfUp" - This field contains the name of the trap, in the format MIB-FILE::trapName.

• "certification": "STANDARD" - This field is used by the product to identify the certification type of the trap.

• "description": [] - This field contains the contextual description of the trap extracted from the MIB file. Pay attention to this field as the description can give helpful information about the nature of the trap, what the notification does, and potentially how it should be handled. This information should be used during the curation process to craft an understandable human description of the trap summary.

• "domain": "FAULT" - This field is used by the product to identify telemetry domain.

• "event": {} - Event object is a container for modifiable fields that will be populated upon event creation.

  • * "EventCategory": 1 - This field indicates the EventCategory for the event in Unified Assurance. Possible values are:

    • 3 which means "Discrete" - Use this value if the coming event is neither a problem nor resolution, and should be used for any informational events.

    • 2 which means "Problem" - Use this value if the coming event indicates a problem that can be "resolved" by another trap coming with different values. Use this field along with setting Severity to a value greater than 0. See the Severity description below.

    • 1 which means "Resolution" - Use this value if the coming event indicates a resolution of a previous problem that was raised by any another trap coming with different values. Use this field along with setting Severity to a value of 0. See the Severity description below.

      For this field, it is very common to use an eval statement along with ternary operators to set a different EventCategory based on the incoming trap values. For example, if $v1 field has three possible enumerations of 1: Major, 2: Clear, 3: Info, you can use an eval statement like this:

      $v1==1 ? 2 : $v1==2 ? 1 : 3
      

      In that example:

      • If $v1 has a value of 1, we set EventCategory to 2 (Problem).

      • If $v1 has a value of 2, we set EventCategory to 1 (Resolution).

      • In any other case, we set EventCategory to 3 (Discrete).

  • * "EventType": "jnxVpnIf" - Original: "jnxVpnIfUp" This field is automatically populated by the trap object name itself, and upon initial generation it is unique among all traps. EventType is part of the Correlation Set which consists of:

    [Device]+[SubNode]+[EventType]+[EventCategory]
    

    For the correlation to work, the first three fields must be identical throughout the correlated traps. In terms of curation, we are interested only in SubNode (see the description below) and EventType. Sometimes the correlation does not happen within the same trap (one varbind of few possible values like clear or major, see EventCategory notes), but along few different traps. By reading the full contextual descriptions of traps in the MIBs, the user should be able to identify separate traps that are intended to correlate between themselves.

    Let's consider an example of two separate traps with the type: jnxVpnIfUp and jnxVpnIfDown. By reading the MIB descriptions of the traps, we know that the first is sent if the VPN interface is Up and the other one when the same interface is Down. To make the two traps correlate, we would need to make sure that the SubNode (see further notes) and EventType for those two traps are identical. In this example, the user would remove the suffix "Up" and "Down" from the EventTypes in both traps leaving only: jnxVpnIf. In this case, when one trap comes with severity > 0, it would raise the event and then correlate it when the second trap comes that clears this event.

  • * "ExpireTime": 3600 - This field indicates default expiry time for the event if no action is taken by a user. It is common to set it to 1 hour (3600 seconds) for any Discrete/Informational event, and 24 hours (86,400 seconds) for any Problem/Resolution pair events that correlate.

  • * "Severity": 0 - This field determines the severity of the event inserted into Unified Assurance. Refer to the trap description and possible enums to determine the actual problem severity. Possible values are:

    • 5 which means "Critical" - Use this severity for any critical system breaking problems. Generally, EventCategory should be set to 2.

    • 4 which means "Major" - Use this severity for any major problems that can affect production systems. Generally, EventCategory should be set to 2.

    • 3 which means "Minor" - Use this severity for any minor and non-critical problems. Generally, EventCategory should be set to 2.

    • 2 which means "Info" - Use this severity for any informational events that do not affect system operation in any significant way, for example, "User logged in". Generally, EventCategory should be set to 3.

    • 1 which means "Unknown" - This severity should not be used during trap curation unless absolutely necessary, unless indicated by the trap definition itself, or if the nature of trap needs additional custom work from the customer operations, for example, the incoming event is a table of custom metric pre-set on EMS system by customer. In that case, the customer would have to create their own flow to process this very custom trap. Generally, EventCategory should be set to 3.

    • 0 which means "Normal" - Use this severity for any clear/OK events. Generally, EventCategory should be set to 1.

  • * "SubNode": "$v2-$v3"- This field is used as a unique identifier of the SubNode for the trap. For example, in many cases we are dealing with shelved network equipment. This equipment will be automatically identified by the Device field, but to identify any interfaces that are present on that shelf, you might need to use certain varbinds for the trap, if present. If $v2 would be a shelf number and $v3 an interface number, they would be ideal candidates for subnode identification. It is needed to distinguish between various interfaces on the same device for the correlation to work.

    WARNING:

    In this field, dynamic variables like Date/Time should be avoided, and variables that indicate unique interface naming should be used. Try to avoid spaces and special characters, keeping the string as simple as possible. If the specified device does not have any unique interfaces, it is ok to leave the default "device" string in that field.

  • * "Summary": "VPN $v2, Type: $v1 is up. [IfIndex: $v3]" - This field represents the Summary text that will appear in Unified Assurance Events. It is important to use common sense when creating that string, making it as indicative and human-readable as possible, and avoid any unnecessary information. (All the varbind values are populated in the events Details field for reference.) A good indication of event summary would be the textual description of the trap. If it is required to have different Summary texts for correlated events (like problem and resolution), you can use "eval" statements to branch out based on varbind values.

• "metaData": {} - This object contains metadata information for the FCOM files.

  • "certified": false - This field is used by Oracle Communications to certify Q/A'ed FCOM for production use.

• "method": "trap" - This field is automatically populated. Internal system use.

• * "test": "$SNMPTRAPCMD JUNIPER-VPN-MIB::jnxVpnIfUp JUNIPER-VPN-MIB::jnxVpnIfVpnType i 1 JUNIPER-VPN-MIB::jnxVpnIfVpnName s EXAMPLE JUNIPER-VPN-MIB::jnxVpnIfIndex u 9999" - This field is automatically populated during FCOM generation and is used to create a "test trap" that can be sent in via the command line. Please bear in mind that it is purely synthetic and cannot substitute the real-life data.

• * "tests" (optional)- Instead of single test trap, you can use "tests" as an array of strings. It allows the operator to include more correlation tests within the set. During test generation additional tests will be automatically grouped and commented into sets within .sh file.

• "trap" : {} - This object contains detailed information about trap structure and its varbinds.

  • "name": "JUNIPER-VPN-MIB::jnxVpnIfUp" - Name of the main trap, format MIB::trapName

  • "oid": "1.3.6.1.4.1.2636.3.26.0.1" - OID of the main trap.

  • "variables": [] - This field contains list of all the trap varbinds.

    • "description": [] - This field contains textual description of the varbind.

    • "enums": {} - If present in the MIB, this field will contain any possible enumeration of the varbind values that will be automatically used to resolve the values in the FCOM process.

    • "name": "JUNIPER-VPN-MIB::jnxVpnIfVpnType" - Name of the trap, format: MIB-FILE::varbindName

    • "oid": "1.3.6.1.4.1.2636.3.26.1.3.1.1" - OID of the trap.

    • "valueType": "INTEGER" - The type of the value the varbind carries.

Preprocessors

• "preprocessors" : [] - This object stores any preprocessors used in the FCOM runtime. As the name indicates, all the preprocessors would be run first during runtime to "pre-process" any required variables. Currently available and implemented preprocessors are:

  • "regex" : { - This preprocessor is used to run Regex match pattern against any available variable and store it in capture group named variable for runtime use. Possible values are:

    • "value" : This field indicates the variable you would like to run regex against, for example $oid1.

    • "pattern" : This field is used to store pure regex pattern you want to run against the value field. To store a capture group for further use in any variable, use the following syntax:

      (?P<myCaptureVariable>)
      

      If the regex match was successful for the variable specified within brackets, in our example myCaptureVariable, the variable will be available for the FCOM during runtime and can be accessed with Perl variable syntax: $myCaptureVariable. If the match was unsuccessful, the variable will be undefined.

    • "flags" : - This is an optional field, where you can specify regex flags you want to use during match. Possible values are g,I,m,s,u. If the field is not specified, regex will default to global g flag.

    }

  • "lookup" : { - This preprocessor is used to lookup up any values against a custom hash lookup. Bear in mind that for the moment lookup table (Perl hash) has to be manually added by the user to the FCOM rules files. Oracle recommends putting a *.lookup file in the custom directory, and loading it in base.load.

    • "source" : - This variable indicates the name of the hash lookup table to check the key against.

    • "key" : - This variable indicates the variable we want to lookup against in the table.

    • "target" : - This field indicates the variable name that should store the value of successful lookup. The variable will be available for use during runtime.

    }

  • "conversion" : { - This preprocessor is used to convert between different value types (integer, strings, etc.).

    • "source" : - This field indicates the source variable you want to convert.

    • "target" : - This field indicates the target variable in which you want to store conversion result.

    • "type" : - Type of conversion we want to run, possible values are:

      • "StringToInt"

        • This conversion type will convert any String into a sum of its integer ASCII values.

      • "CharToInt"

        • This conversion type will convert a character into its ASCII value.
        • If the source is a string, only the first character will be converted.

      • "OctetStrToIPv4"

        • Convert a four-character OCTET STRING into an IPv4 address.
        • For example, a variable containing a string of octets represented by the hex number 7f000001 would be converted to the string "127.0.0.1".

      • "OctetStrToIPv6"

        • Convert a sixteen-character OCTET STRING into an IPv6 address.
        • For example, a variable containing a string of octets represented by the hex number 0A4041C20A4041C20A4041C20A4041C2 would be converted to the string "400A:C241:400A:C241:400A:C241:400A:C241".

      • "OctetStrToIPv4z"

        • Convert an eight-character OCTET STRING into an IPv4 address with zone index.
        • For example, a variable containing a string of octets represented by the hex number 7f00000165746830 would be converted to the string "127.0.0.1#eth0".

      • "OctetStrToIPv6z"

        • Convert a twenty-character OCTET STRING into an IPv6 address with zone index.
        • For example, a variable containing a string of octets represented by the hex number 0A4041C20A4041C20A4041C20A4041C265746830 would be converted to the string "400A:C241:400A:C241:400A:C241:400A:C241#eth0".

      • "OctetStrToDateAndTime"

        • Convert an eight or eleven character OCTET STRING into a DateAndTime format string, as defined in the standard SNMPv2 MIB SNMPv2-TC.
        • Eight-character example: a variable containing a string of octets represented by the hex number 07cc09010F1E0000 would be converted to the string "1996-9-1,15:30:0.0".
        • Eleven-character example: a variable containing a string of octets represented by the hex number 07cc09010F1E00002b011E would be converted to the string "1996-9-1,15:30:0.0,+1:30".

      • "OctetStrToMacAddress"

        • Convert a six-character OCTET STRING into a MAC address.
        • For example, a variable containing a string of octets represented by the hex number 000eaa12ddff would be converted to the string "00:0E:AA:12:DD:FF".

      • "OctetStrToHex"

        • Convert an OCTET STRING of any length into a string containing two-digit hex values of each character.
        • For example, a variable containing "Example" would be converted to the string "4578616D706C65".

    }

FCOM Runtime Variables

The following variables are exposed by Trapd for FCOM during runtime, and are available for the user to use in the curation process:

• $v1 .. $v20 - These variables store raw varbind values for the trap. Varbind 1 would be $v1 and so on.

• $oid1 .. $oid20 - These variables contain full OID of the specific varbind for the trap. Varbind 1 OID would be $oid1 and so on.

• $ip - This variable stores the source IP of the trap.

• $trapoid - This variable contains the full OID of the trap.

• $node - This variable stores source node name of the incoming trap.