System Fault Statistics
This section contains information about system faults. System faults include problems related to CPU usage, memory usage, and license capacity. System faults also include the functionality of the Address Resolution Protocol (ARP) on the system.
System State
You can use the following commands to view system uptime and state information:
- show uptime
- show system-state
System Resources
You can use the following command to view the system resource statistics:
- show processes cpu
Viewing CPU Usage
Display CPU usage information, categorized on a per task/process basis, for your Oracle Communications Session Border Controller by using the show processes cpu command.
ORACLE> show processes cpu
Task Name Task Id Pri Status Total CPU Avg Now Load Processor
----------- ------- ---- -------- ------------ ----- ----- ----- ---------
tCli 1799 -2 RUNNING 0:00:00.910 0.00 2.37 4.74 1
tAsctpd 1770 -70 SLEEPING 0:01:46.440 0.03 0.47 0.95 1
kworker/1:1 423 20 RUNNING 0:28:22.590 0.40 0.47 0.95 1
kworker/u:1 2478 20 SLEEPING 0:00:01.430 0.00 0.00 0.00 0
kworker/u:2 2477 20 SLEEPING 0:00:02.450 0.00 0.00 0.00 1
kworker/u:0 2476 20 SLEEPING 0:00:06.150 0.00 0.00 0.00 0
telnetD 1805 -100 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliTnet5 1804 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliTnet4 1803 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliTnet3 1802 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliTnet2 1801 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliTnet1 1800 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tSSH 1798 -100 SLEEPING 0:00:04.590 0.00 0.00 0.00 0
tCliSSH4 1797 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliSSH3 1796 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliSSH2 1795 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliSSH1 1794 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tCliSSH0 1793 -2 SLEEPING 0:00:00.000 0.00 0.00 0.00 1
tLogCleaner 1792 -100 SLEEPING 0:00:00.100 0.00 0.00 0.00 0
tAlarm 1791 -76 SLEEPING 0:00:29.130 0.01 0.00 0.00 1
tifXCheck 1787 -100 SLEEPING 0:00:00.130 0.00 0.00 0.00 0
tSnmpd 1786 -62 SLEEPING 0:00:00.070 0.00 0.00 0.00 0
The output of the show processes cpu command includes the following information:
- Task Name—Name of the system task or process
- Task Id—Identification number for the task or process
- Pri—Priority for the CPU usage
- Status—Status of the CPU usage
- Total CPU—Total CPU usage since last reboot in hours, minutes, and seconds
- Avg—Displays percentage of CPU usage since the system was last rebooted
- Now—CPU usage in the last second
- Load—The CPU load
- Processor—The processor number where this task runs
CPU Utilization Alarm
The following table lists the CPU utilization alarm.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| CPU UTILIZATION | 131099 | MINOR | CPU usage reached 90% or greater of its capacity. | CPU usage X% over threshold X% | apSysMgmtGroupTrap trap generated
minor dry contact syslog |
System Task Suspended Alarm
The following table describes the system task suspended alarm information.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| SYSTEM TASK SUSPENDED | 131108 | CRITICAL | A system task (process) suspends or fails. | Task X suspended, which decremented health by 75! (where X is the task/process name) | apSyslogMessageGenerated trap generated
major dry contact syslog reboot (if the system is configured to do so) |
Memory Usage
You can use the following commands to view memory statistics:
- show memory usage
- show buffers
Viewing Memory Usage Statistics
Display memory usage statistics by using the show memory usage command. For example:
ORACLE# show memory usage
Mem Total : 3698 MB
Mem App : 505 MB
Mem OS : 179 MB
Viewing Memory Buffer Statistics
Display memory buffer statistics using the show buffers command. Use this command only for debugging purposes under the direction of Oracle support.
Components displayed vary based on platform and configuration, and commonly include:
- L2 Resolver
- Service Pipe
- Memory Buffer Process
- Memory Buffer Redundancy
- Memory Buffer Transport
- Network Buffer
- Network Buffer Control
- NP Application Fragments
- NP Application GARP
- NP DMA
Component statistics include:
- Pool Instances (number of pool instances)
- Memory Footprint (allocation in MB across all instances)
- Pool Size (aggregate pool size across all instances)
- Buffer Size (Bytes) (fixed buffer size)
- Allocated Buffers (aggregate allocated buffers across all instances)
- Used Buffers (aggregate in-use buffers across all instances)
- Errors (aggregate errors across all instances)
For example:
ORACLE# show buffers component refs MB total size alloc usage error ------------- ---- ------ ----- ----- ----- ----- ----- L2Resolver 1 1.74 4358 418 4358 0 0 MemBufProc 62 15.88 12400 65536 254 174 0 MemBufRed 1 0.00 200 10240 0 0 0 MemBufTrans 1 0.06 200 65536 1 0 0 NetBuf 1 317.87 32000 10416 32000 0 0 NetBufCtrl 1 13.67 32000 448 32000 0 0 NpAppGarp 1 0.01 20 352 20 0 0 SvcPipe 1 130.49 16000 8552 16000 180 0
Subsequent, optional arguments include:
- histogram - Show the histogram of requested buffer sizes by the Memory Buffer Transport, Network Buffer, and Service Pipe components for use in future buffer pool optimizations.
- usage - Requires buffer tracking to be enabled by Oracle Support.
Memory Utilization Alarm
The following table describes the memory utilization alarm.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| MEMORY UTILIZATION | 131100 | MAJOR | Memory usage reached 90% or greater of its capacity. | Memory usage X% over threshold X% | apSysMgmtGroupTrap trap generated
minor dry contact syslog |
License Capacity
If the total number of active sessions on the system (across all protocols) is within 98-100% of the system’s licensed capacity, an alarm and trap will be generated. The severity of this application alarm is MAJOR, but is not HA health-affecting.
The total number of active sessions is checked at an interval of 5 seconds (just as the system temperature and fans speed are). Once an approaching capacity alarm is triggered, another one will not be triggered until after the current alarm is cleared. This alarm will be cleared (and the trap sent, apLicenseNotApproachingCapacityNotification) after the total number of active sessions has gone to or below 90% of capacity, but no sooner than 15 seconds after the original alarm was triggered.
The following table describes the license capacity alarm
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| LICENSE ALARM APPROACHING SESSION CAPACITY | 327684 | MAJOR | Total number of active sessions on the system (across all protocols) is within 98 to 100% of the system’s licensed capacity. | Total number of sessions (<#>) is approaching licensed capacity (<#>) | apLicenseApproachingCapacityNotification |
Configuration Statistics
You can use the following commands to display configuration information:
- show configuration
- show running-config
- realm-specifics <realm ID>
- show virtual-interfaces
Specifying a Configuration Element
Both the show configuration and the show running-config commands let you include a configuration element name as an argument to view only instances for that configuration element. The list of valid configuration elements you can use as an argument include the following:
- account-config—Show account-config object
- h323-config—Show h323-config object
- h323-stack—Show all h323-stack objects
- iwf-stack—Show iwf-stack object
- host-route—Show all host-route objects
- local-policy—Show all local-policy objects
- media-profile—Show all media-profile objects
- media-manager—Show media-manager object
- dns-config—Show all dns-config objects
- network-interface—Show all network-interface objects
- ntp-config—Show ntp-config object
- phys-interface—Show all phys-interface objects
- realm—Show all realm objects
- MediaPolicy—Show all MediaPolicy objects
- ClassPolicy—Show all ClassPolicy objects
- redundancy-config—Show redundancy-config object
- ResponseMap—Show all ResponseMap objects
- session-agent—Show all session-agent objects
- session-group—Show all session-group objects
- session-translation—Show all session-translation objects
- translation-rules—Show all translation-rules objects
- session-router—Show session-router object
- sip-config—Show all sip-config objects
- sip-feature—Show all sip-feature objects
- sip-interface—Show all sip-interface objects
- sip-nat—Show all sip-nat objects
- snmp-community—Show all snmp-community objects
- static-flow—Show all static-flow objects
- steering-pool—Show all steering-pool objectssystem-config—show system-config object
- TrapReceiver—Show all TrapReceiver objects
- call-recording-server—Show call-recording-server configurations
- capture-receiver—Show capture-receiver configurations
- rph-profile—Show rph-profile configurations
- rph-policy—Show rph-policy configurations
- password-policy—Show password-policy configuration
- enforcement-profile—Show enforcement-profile configurations
- realm-group—Show realm-group configurations
- inventory—Displays an inventory of all configured elements on the Oracle Communications Session Border Controller
Viewing Current Configuration
Display information about the current configuration (used once the activate-config command is executed) by using the show configuration command. You can include the name of a configuration element with the show configuration command to display only instances for that configuration element.
For example:
ORACLE# show configuration media-manager media-manager state enabled latching enabled flow-time-limit 86400 initial-guard-timer 300 subsq-guard-timer 300 tcp-flow-time-limit 86400 tcp-initial-guard-timer 300 tcp-subsq-guard-timer 300 tcp-number-of-ports-per-flow 2 hnt-rtcp disabled mbcd-log-level NOTICE max-signaling-bandwidth 10000000 max-untrusted-signaling 100 min-untrusted-signaling 30 app-signaling-bandwidth 0 tolerance-window 30 rtcp-rate-limit 0 min-media-allocation 32000 min-trusted-allocation 1000 deny-allocation 1000 anonymous-sdp disabled arp-msg-bandwidth 32000 last-modified-date 2007-04-05 09:27:20 task done
Viewing Running Configuration
Display the running configuration information currently in use on the system by using the show running-config command. You can include the name of a configuration element with the show configuration command to display only the instances for that configuration element.
For example:
ORACLE# show running-config realm realm-config identifier testrealm addr-prefix 0.0.0.0 network-interfaces mm-in-realm disabled mm-in-network enabled mm-same-ip enabled mm-in-system disabled bw-cac-non-mm disabled msm-release disabled qos-enable disabled max-bandwidth 0 ext-policy-svr boffo.com max-latency 0 max-jitter 0 max-packet-loss 0 observ-window-size 0 parent-realm dns-realm media-policy in-translationid out-translationid in-manipulationid out-manipulationid class-profile average-rate-limit 0 access-control-trust-level low invalid-signal-threshold 0 maximum-signal-threshold 0 untrusted-signal-threshold 758 deny-period 30 symmetric-latching disabled pai-strip disabled trunk-context early-media-allow reverse additional-prefixes 10.0.0.0/24 172.16.0.0 restricted-latching peer-ip restriction-mask 17 accounting-enable enabled user-cac-mode none user-cac-bandwidth 0 user-cac-sessions 0 net-management-control disabled delay-media-update disabled codec-policy codec-manip-in-realm disabled last-modified-date 2006-07-06 12:43:39
Viewing Realm-Specific Configuration
Display realm-specific configuration based on the input realm ID by using the realm-specifics <realm ID> command. The information displayed includes the following:
- realm-config
- steering-pool
- session-agent
- session-translation
- class-policy
- local-policy (if the source realm or destination realm are defined)
For example:
ORACLE# realm-specifics testrealm realm-config identifier testrealm addr-prefix 0.0.0.0 network-interfaces mm-in-realm disabled mm-in-network enabled mm-same-ip enabled mm-in-system disabled bw-cac-non-mm disabled msm-release disabled qos-enable disabled max-bandwidth 0 ext-policy-svr boffo.com max-latency 0 max-jitter 0 max-packet-loss 0 observ-window-size 0 parent-realm dns-realm media-policy in-translationid out-translationid in-manipulationid out-manipulationid class-profile average-rate-limit 0 access-control-trust-level low invalid-signal-threshold 0 maximum-signal-threshold 0 untrusted-signal-threshold 758 deny-period 30 symmetric-latching disabled pai-strip disabled trunk-context early-media-allow reverse additional-prefixes 10.0.0.0/24 172.16.0.0 restricted-latching peer-ip restriction-mask 17 accounting-enable enabled user-cac-mode none user-cac-bandwidth 0 user-cac-sessions 0 net-management-control disabled delay-media-update disabled codec-policy codec-manip-in-realm disabled last-modified-date 2006-07-06 12:43:39 sip-interface state enabled realm-id testrealm sip-port address 192.168.10.12 port 5060 transport-protocol UDP tls-profile allow-anonymous register-prefix carriers trans-expire 0 invite-expire 0 max-redirect-contacts 0 proxy-mode redirect-action contact-mode maddr nat-traversal none nat-interval 30 tcp-nat-interval 30 registration-caching disabled min-reg-expire 300 registration-interval 3600 route-to-registrar disabled secured-network disabled teluri-scheme disabled uri-fqdn-domain options disable-privacy trust-mode all max-nat-interval 3600 nat-int-increment 10 nat-test-increment 30 sip-dynamic-hnt disabled stop-recurse 401,407 port-map-start 0 port-map-end 0 in-manipulationid out-manipulationid sip-ims-feature disabled operator-identifier anonymous-priority none max-incoming-conns 0 per-src-ip-max-incoming-conns 0 inactive-conn-timeout 0 untrusted-conn-timeout 0 network-id ext-policy-server default-location-string charging-vector-mode pass charging-function-address-mode pass ccf-address ecf-address term-tgrp-mode none implicit-service-route disabled rfc2833-payload 101 rfc2833-mode transparent constraint-name response-map local-response-map last-modified-date 2006-06-12 12:08:34
Configuration Save Failed Alarm
The following table lists the CFG ALARM SAVE FAILED alarm.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| CFG ALARM SAVE FAILED | 393217 | MAJOR | The save-config command execution failed on a standby Oracle Communications Session Border Controller peer operating as part of an HA pair. | save-config failed on targetName!/code full,
config sync stopped!
or save-config failed on targetName!/code full, config sync stopped! (where the targetName is the target name (tn) configured in the boot parameters) |
apSyslogMessageGenerated trap generated
syslog |
HA Functionality
You can monitor HA Oracle Communications Session Border Controller functionality using the following ACLI commands:
- show health to view information about the HA architecture and associated HA Oracle Communications Session Border Controller peers.
- show redundancy to view information about the synchronization of media flows and signaling for the members of an HA Oracle Communications Session Border Controller pair.
You can also view state displays on the graphical window display of the chassis.
Viewing Health Information
Display the following information for HA architectures by using the show health command:
Note:
The spaces are intentionally used in the following examples because they appear on the screen.- Health score
- Whether the current HA Oracle Communications Session Border Controller is active, standby, or out of service
- Whether the media flow information is synchronized for all supported protocols: SIP and H.323 (true/false)
- If media flow information is not available, Media Synchronized disabled will be displayed in the show health output.
- Whether SIP signaling information is synchronized (true/false)
- If SIP signaling is not available, SIP Synchronized disabled will be displayed in the show health output.
- Whether configuration information is synchronized (true/false)
- If configuration checkpointing is not available, Config Synchronized disabled will be displayed in the show health output.
- The IPv4 or IPv6 address of the current HA Oracle Communications Session Border Controller’s active peer (an HA Oracle Communications Session Border Controller that is currently active does not have an active Oracle Communications Session Border Controller peer and shows 0.0.0.0)
- The last message received from the HA Oracle Communications Session Border Controller peer
- A switchover log containing the last 20 switchover events (whether becoming active or relinquishing the active role)
The following example shows a currently active Oracle Communications Session Border Controller.
ORACLE# show health Media Synchronized enabled SIP Synchronized enabled Config Synchronized enabled Collect Synchronized enabled Radius CDR Synchronized enabled Rotated CDRs Synchronized enabled Active Peer Address 163.4.12.2 Redundancy Protocol Process (v2): State Active Health 100 Lowest Local Address 11.0.0.1:9090 1 peer(s) on 1 socket(s): systest3B: v2, Standby, health=100, max silence=1050 last received from 11.0.0.2 on wancom1:0 Switchover log: Jul 11 14:18:21.442: Active to RelinquishingActive Jul 11 14:24:00.872: Standby to BecomingActive, active peer systest3B has timed out.
Viewing Redundancy Information
Display the following information about HA architecture by using the show redundancy command:
- General HA statistics
- Statistics related to HA transactions that have been processed
- Timestamp showing when the current period began
- The numerical identifier for the last redundant transaction processed (each transaction is numbered)
In an HA architecture that is functioning properly, the number for the last redundant transaction processed on a standby Oracle Communications Session Border Controller peer should not be far behind (if not exactly the same as) the one shown for the active Oracle Communications Session Border Controller peer.
Several subcommands appear under the show redundancy command. Within this set of subcommands, system administrators can view information related to HA transactions, including specific transaction information.
The following example shows the subcommands available for the show redundancy command.
ORACLE# show redundancy ? algd MGCP Redundancy Statistics collect Collect Redundancy Statistics config Configuration Redundancy Statistics iked Iked Redundancy Statistics manuald Manuald Redundancy Statistics mbcd MBC Redundancy Statistics radius-cdr Radius CDR Redundancy Statistics rec SIPREC Redundancy Statistics rotated-cdr Rotated Radius CDR Redundancy Statistics sipd SIP Redundancy Statistics
HA Alarms
There are currently five alarms directly associated with the HA feature. A system alarm is triggered when any of the following HA conditions occurs:
- When the health score falls below 60. This is a hard-coded threshold value. It is not configurable.
- By the Active-BecomingStandby peer upon switchover.
- By the Standby-BecomingActive peer upon switchover.
- When the HA Oracle Communications Session Border Controller peer times out.
- When the standby system is unable to synchronize with its active Oracle Communications Session Border Controller peer within the amount of time set for the becoming standby time field of the redundancy element.
When certain alarms associated with the HA feature are triggered, traps are sent via the appropriate MIB (for example, syslog or system management). Traps for switchover alarms indicate that a switchover has occurred and identify the state transition of the HA Oracle Communications Session Border Controller reporting the switchover. For example:
- Standby to BecomingActive
- BecomingStandby to BecomingActive
- Active to RelinquishingActive and so on
In the case of an alarm from the Standby to BecomingActive peer, the associated trap also indicates the reason for switchover (as far as high availability is concerned). These reasons might include reporting the degraded health of the HA Oracle Communications Session Border Controller peer or indicating that the HA Oracle Communications Session Border Controller peer has timed out or that a switchover was forced by command.
The following table provides a list, by name, of the Oracle Communications Session Border Controller’s HA-related alarms, including their alarm IDs, severities, causes, associated log messages, and actions.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| HEALTH SCORE | 131101 | MAJOR | System’s health score fell below 60. | Health score X is under threshold (where X is the health score) | apSysMgmtGroupTrap |
| NAT TABLE UTILIZATION | 131102 | MINOR | NAT table usage reached 90% or greater of its capacity. | NAT table usage X% over threshold X% | apSysMgmtGroupTrap |
| ARP TABLE UTILIZATION | 131103 | MINOR | ARP table usage reached 90% or greater of its capacity. | ARP table X% over threshold X% | apSysMgmtGroupTrap |
| REDUNDANT SWITCH-TO-ACTIVE | 131104 | CRITICAL | A state transition occurred from Standby/BecomingStandby to BecomingActive. | Switchover, <state to state>, active peer <name of HA peer> has timed out or Switchover, <state to state>, active peer <name of HA peer> has unacceptable health (x) (where x is the health score) or Switchover, <state to state>, forced by command | apSyslogMessageGenerated
apSysMgmtRedundancyTrap |
| REDUNDANT SWITCH-TO-STANDBY | 131105 | CRITICAL | A state transition occurred from Active/BecomingActive to BecomingStandby/RelinquishingActive. | Switchover, <state to
state>, peer <name of HA peer> is healthier (x) than us (x) (where x
is the health score)
or Switchover, <state to state>, forced by command |
apSyslogMessageGenerated
apSysMgmtRedundancyTrap |
| REDUNDANT TIMEOUT | 131106 | MAJOR | A HA system peer was not heard from within a time period. | Peer <name of HA peer> timed out in state x, my state is x (where x is the state (for example, BecomingStandby)) | apSyslogMessageGenerated
apSysMgmtRedundancyTrap |
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
| REDUNDANT OUT OF SERVICE | 131107 | CRITICAL | Unable to synchronize with Active HA system peer within BecomingStandby timeout. | Unable to synchronize with Active redundant peer within BecomingStandby timeout, going OutOfService | apSyslogMessageGenerated
apSysMgmtRedundancyTrap |
| CFG ALARM SAVE FAILED | 393217 | MAJOR | The save-config command execution failed on a standby Oracle Communications Session Border Controller peer operating as part of an HA pair. | save-config failed on
targetName!/code full, config sync stopped!
or save-config failed on targetName!/code full, config sync stopped! (where the targetName is the target name (tn) configured in the boot parameters) |
apSyslogMessageGenerated trap
generated
syslog |
Base Display Level
The base display level of the graphic display window on the front panel of the chassis shows the state of an HA Oracle Communications Session Border Controller. The base display appears when the Oracle Communications Session Border Controller first starts up and when the graphic display times out at any menu level.
System administrators can distinguish between an active SBC and a standby Oracle Communications Session Border Controller in an HA architecture by looking at the front of the chassis. The chassis operating as the standby Oracle Communications Session Border Controller in an HA architecture displays an (S) in the graphic display window to indicate its status as the standby system. The chassis operating as the active Oracle Communications Session Border Controller in an HA architecture does not display anything in parentheses in the graphic display window.
HA State Display Stats
The chassis’s graphic display window shows the current state of the HA Oracle Communications Session Border Controller using an abbreviation that follows the Oracle Communications Session Border Controller name. The states are defined in the following table.
| State Abbreviation | Description |
|---|---|
| (I) | Initial (the Oracle Communications Session Border Controller is in this state when it is booting) |
| (O/S) | Out of service |
| (B/S) | Becoming standby |
| (S) | Standby |
| (nothing displayed after the Oracle Communications Session Border Controller name) | Active |
Refer to the following sections for examples of the graphic display window output.
Initial State Displays
The following example shows the output in the graphic display window of a Oracle Communications Session Border Controller in the initial state:
NET - NET SESSION DIRECTOR (I)
Out Of Service State Displays
The following examples show the output in the graphic display window of an out-of-service Oracle Communications Session Border Controller:
NET - NET SESSION DIRECTOR (O/S)
Becoming Standby State Displays
The following example shows the output in the graphic display window of a Oracle Communications Session Border Controller becoming standby:
NET - NET SESSION DIRECTOR (B/S)
Standby State Displays
The following example shows the output in the graphic display window of a standby Oracle Communications Session Border Controller:
NET - NET SESSION DIRECTOR (S)
Active State Displays
HA Oracle Communications Session Border Controllers in the active state use the default graphic display. The following example show the display of an active Oracle Communications Session Border Controller.
ACME PACKET SESSION DIRECTOR
For further information about the Oracle Communications Session Border Controller chassis and graphic display window, refer to the Oracle Communications Session Border Controller Installation Guide.
ARP Functionality
You can use the following command to view ARP functionality information:
- arp-check
- show arp
Testing Address Resolution
Test a specific address resolution by using the arp-check command; which causes a a test message to be sent. The test is successful when an OK is returned. Note that the command does not send an ARP request if the specified address is already in the ARP table or is in a different subnet.
To run this test, you must enter the following information after typing arp-check and a Space:
- media interface slot (either of two values: 1 is for the left, and 2 is for the right)
- VLAN identifier
Note:
If there is no VLAN identifier to be entered, enter a value of 0. - IPv4 address (in dotted notation).
For example:
ORACLE# arp-check 1 6 192.168.100.1 ARP: Sending ARP REQ port=0, vlan=6, source_ipa=192.168.200.10, target_ipa=192.168.100.1 ORACLE#
Viewing Current Address Mappings
Display the current Internet-to-Ethernet address mappings in the ARP table by using the show arp command. The first section of this display shows the following information: destination, gateway, flags, reference count, use, and interface. The second section shows the interface, VLAN, IP address, MAC address, timestamp, and type.
The intf (interface) column in the ARP includes both slot and port information. If a value of 0/1 appears, 0 refers to the slot and 1 refers to the port. For example:
ORACLE# show arp LINK LEVEL ARP TABLE destination gateway flags Refcnt Use Interface ------------------------------------------------------------------------- 172.30.0.1 00:0f:23:4a:d8:80 405 1 0 wancom0 ------------------------------------------------------------------------- Total ARP Entries = 3 ----------------------- Intf VLAN IP-Address MAC time-stamp type 0/0 0 010.000.045.001 00:00:00:00:00:00 1108462861 invalid Special Entries: 0/0 0 000.000.000.000 00:00:00:00:00:00 1108462861 gateway 0/0 0 010.000.045.000 00:00:00:00:00:00 1108462861 network Gateway Status: Intf VLAN IP-Address MAC time-stamp hb status 0/0 0 010.000.045.001 00:00:00:00:00:00 1108462861 unreachable -- ARP table info -- Maximum number of entries : 512 Number of used entries : 3 Length of search key : 1 (x 64 bits) First search entry address : 0x3cb0 length of data entry : 2 (x 64 bits) First data entry address : 0x7960 Enable aging : 0 Enable policing : 0
ARP Table Utilization Alarm
The following table describes the ARP table utilization alarm.
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| ARP TABLE UTILIZATION | 131103 | MINOR | ARP table usage reached 90% or greater of its capacity. | ARP table X% over threshold X% | apSysMgmtGroupTrap trap generated
syslog |
Local Policy
Use the following commands to view local policy statistics and information:
- show running-config local-policy
- show configuration local-policy
Viewing Running Configuration Local Policy
Display information about the local policy in the running configuration information in use on the system by using the show running-config local-policy command. For example:
ORACLE# show running-config local-policy local-policy from-address 192.168.0.50 to-address 10.10.10.10 source-realm * activate-time N/A deactivate-time N/A state enabled policy-priority urgent last-modified-date 2006-06-12 08:48:57 policy-attribute next-hop 172.168.0.10 realm action none terminate-recursion enabled carrier start-time 0000 end-time 2400 days-of-week U-S cost 0 app-protocol state enabled media-profiles task done
Viewing Current Configuration Local Policy
Display information about the local policy in the current configuration that will be used once the activate-config command is executed by using the show configuration command. For example:
ORACLE# show configuration local-policy ORACLE# show running-config local-policy local-policy from-address 192.168.0.50 to-address 10.10.10.10 source-realm * activate-time N/A deactivate-time N/A state enabled policy-priority urgent last-modified-date 2006-06-12 08:48:57 policy-attribute next-hop 172.168.0.10 realm action none terminate-recursion enabled carrier start-time 0000 end-time 2400 days-of-week U-S cost 0 app-protocol state enabled media-profiles task done
Session and Protocol Statistics
You can use the following commands to access protocol tracing statistics:
- notify
- monitor sessions
Viewing Runtime Protocol Tracing
Display information about runtime protocol tracing for UDP/ TCP sockets by using the notify command. This command provides information for all protocol messages for ServiceSocket sockets to be written in a log file or sent out of the system to a UDP port.
This mechanism allows for tracing to be enabled for any socket, provided that the class has a logit method for displaying and formatting the protocol message. All ACP classes and SIP supports this. Tracing can be enabled for all processes, specific sockets, all sockets, or specific processes. Tracing for specific sockets is specified by the local IPv4 or IPv6 address and port on which the socket is connected.
notify all|<process-name> trace all|<socket-address><file-name> [<out-udp-port>] notify all|<process-name> notrace all|<socket-address>
The <socket-address> is the IPv4 or IPv6 address and the port on which the socket is connected. The <out-udp-port> is the UDP IPv4 or IPv6 address and port to which the log messages are sent. If the <out-udp-port> is not specified, the logs are written to the <filename>.
Viewing Real-Time SIP Session Statistics
If you have Superuser access, you can display real-time SIP session statistics by using the monitor sessions command. For example:
ORACLE# monitor sessions 09:10:26-172 SIP Status -- Period -- -------- Lifetime -------- Active High Total Total PerMax High Sessions 0 0 0 0 0 0 Subscriptions 0 0 0 0 0 0 Dialogs 0 0 0 0 0 0 CallID Map 0 0 0 0 0 0 Rejections - - 0 0 0 ReINVITEs - - 0 0 0 Media Sessions 0 0 0 0 0 0 Media Pending 0 0 0 0 0 0 Client Trans 0 0 0 0 0 0 Server Trans 0 0 0 0 0 0 Resp Contexts 0 0 0 0 0 0 Saved Contexts 0 0 0 0 0 0 Sockets 0 0 0 0 0 0 Req Dropped - - 0 0 0 DNS Trans 0 0 0 0 0 0 DNS Sockets 0 0 0 0 0 0 DNS Results 0 0 0 0 0 0
Real-time statistics for the following categories appear on the screen:
- Dialogs
- Sessions
- CallID Map
- Rejections
- ReINVITES
- Media Sessions
- Media Pending
- Client Trans
- Server Trans
- Resp Contexts
- Sockets
- Reqs Dropped
- DNS Trans
- DNS Sockets
- DNS Results
By default, the statistics refresh every second. Press any numerical digit (0-9) to change the refresh rate. For example, while viewing the statistics, you can press <6> to cause the system statistics to refresh every 6 seconds.
Pressing <q> or <Q> allows you to exit the statistics display and returns you to the ACLI system prompt.
Media and Bandwidth Statistics
You can use the following commands to display media and bandwidth statistics:
- show mbcd errors
- show mbcd realms
- monitor media
Viewing MBCD Task Errors
Display Middle Box Control Daemon (MBCD) task error statistics by using the show mbcd errors command. There are two categories of MBCD error statistics: Client and Server.
For example:
ORACLE# show mbcd errors 16:19:18-139 MBC Errors ---- Lifetime ---- Recent Total PerMax Client Errors 0 0 0 Client IPC Errors 0 0 0 Open Streams Failed 0 0 0 Drop Streams Failed 0 0 0 Exp Flow Events 0 0 0 Exp Flow Not Found 0 0 0 Transaction Timeouts 0 0 0 Server Errors 0 0 0 Server IPC Errors 0 0 0 Flow Add Failed 0 0 0 Flow Delete Failed 0 0 0 Flow Update Failed 0 0 0 Flow Latch Failed 0 0 0 Pending Flow Expired 0 0 0 ARP Wait Errors 0 0 0 Exp CAM Not Found 0 2 2 Drop Unknown Exp Flow 0 0 0 Drop/Exp Flow Missing 0 0 0 Exp Notify Failed 0 0 0 Unacknowledged Notify 0 0 0 Invalid Realm 0 5 5 No Ports Available 0 0 0 Insufficient Bandwidth 0 0 0 Stale Ports Reclaimed 0 0 0 Stale Flows Replaced 0 0 0 Pipe Alloc Errors 0 0 0 Pipe Write Errors 0 0 0
Client statistics count errors and events encountered by applications that use the MBCD to set up and tear down media sessions:
- Client Errors—Number of errors in the client application related to MBC transactions that are otherwise uncategorized
- Open Streams Failed—Number of errors related to sending Add or Modify requests to MBCD
- Drop Streams Failed—Number of errors related to sending Subtract requests to MBCD
- Exp Flow Events—Number of flow timer expiration notifications received from the MBCD by all applications
- Exp Flow Not Found—Number of flow timer expiration notifications received from the MBCD by all applications for which no media session or flow information was present in the application
- Transaction Timeouts—Number of MBC transaction timeouts
- Server statistics count errors and events encountered by MBCD
- Server Errors—Number of uncategorized errors in the MBC server
- Flow Add Failed—Number of errors encountered when attempting to add an entry to the NAT table
- Flow Delete Failed—Number of errors encountered when attempting to remove an entry from the NAT table
- Flow Update Failed—Number of errors encountered when attempting to update an entry in the NAT table upon receipt of the first packet for a media flow
- Flow Latch Failed—Number of errors when attempting to locate an entry in the NAT table upon receipt of the first packet for a media flow
- Pending Flow Expired—Number of flow timer expirations for pending flows that have not been added to the NAT table
- ARP Wait Errors—Number of errors and timeouts related to obtaining the Layer 2 addressing information necessary for sending media
- Exp CAM Not Found—This statistic shows the number that the NAT table entry for an expired flow could not find in the NAT table. This usually occurs due to a race condition between the removal of the NAT entry and the flow timer expiration notification being sent to MBCD from the NP
- Drop Unknown Exp Flow—Number of flows deleted by the MBCD because of a negative response from the application to a flow timer expiration notification
- Drop/Exp Flow Missing—Number of negative responses from the application to a flow timer expiration notification for which the designated flow could not be found in MBCD's tables. Also includes when a flow for a Subtract request to MBCD cannot be found
- Exp Notify Failed—Number of errors encountered when the MBCD attempted to send a flow timer expiration notification to the application.
- Unacknowledged Notify—Number of flow expiration notification messages sent from MBCD to the application for which MBCD did not receive a response in a timely manner.
- No Ports Available—Number of steering port allocation requests not be satisfied due to a lack of free steering ports in the realm
- Invalid Realm—Number of flow setup failures due to an unknown realm in the request from the application
- Insufficient Bandwidth—Number of flow setup failures due to insufficient bandwidth in the ingress or egress realm
Viewing Steering Port and Bandwidth Usage
Display steering ports and bandwidth usage for home, public, and private realms by using the show mbcd realms command.
For example:
acmepacket# show mbcd realms
18:26:39-1629
--- Steering Ports --- ----------- Bandwidth Usage ----------
Realm Used Free No Ports Flows Ingrss Egress Total Insuf BW
acme 0 0 0 0 0K 0K 0K 0
h323172 2 29999 0 0 0K 0K 0K 0
sip172 2 29999 0 0 0K 0K 0K 0
sip192 0 30001 0 0 0K 0K 0K 0
Information in the following categories is displayed:
- Used—Number of steering ports used
- Free—Number of free steering ports
- No Ports—Number of times that a steering port could not be allocated
- Flows—Number of established media flows
- Ingrss—Amount of bandwidth being used for inbound flows
- Egress—Amount of bandwidth being used for outbound flows
- Total—Maximum bandwidth set for this realm
- Insuf BW—Number of times that a session was rejected due to insufficient bandwidth
Viewing Real-Time Media Monitoring Statistics
If you have Superuser access, you can display real-time media monitoring statistics by using the monitor media command. For example:
acmepacket# monitor media
17:31:00-160
MBCD Status -- Period -- -------- Lifetime --------
Active High Total Total PerMax High
Client Sessions 143 182 1930 1218332 4225 683
Client Trans 0 18 5744 2500196 8439 625
Contexts 144 182 1930 834745 2783 2001
Flows 296 372 3860 1669498 5566 3689
Flow-Port 286 362 3860 1669488 5566 3679
Flow-NAT 294 365 3788 1658668 5563 2051
Flow-RTCP 0 0 0 0 0 0
Flow-Hairpin 0 0 0 0 0 0
Flow-Released 0 0 0 0 0 0
MSM-Release 0 0 0 0 0 0
Rel-Port 0 0 0 0 0 0
Rel-Hairpin 0 0 0 0 0 0
NAT Entries 295 365 3791 1658671 5563 2051
Free Ports 7430 7518 7828 3346410 11604 8002
Used Ports 572 724 7724 3338980 11132 8000
Port Sorts - - 0 14796 4156
MBC Trans 1141 1234 5748 2503147 8440 2974
MBC Ignored - - 0 0 0
ARP Trans 0 0 0 8 8 1
Real-time statistics for the following categories appear on the screen:
- Client Sessions
- Client Trans
- Contexts
- Flows
- Flow-Port
- Flow-NAT
- Flow-RTCP
- Flow-Hairpin
- Flow-Release
- MSM-Release
- NAT Entries
- Free Ports
- Used Ports
- Port Sorts
- MBC Trans
- MBC Ignored
- ARP Trans
By default, the statistics refresh every second. Press any numerical digit (0-9) to change the refresh rate. For example, while viewing the statistics, you can press <6> to cause the system statistics to refresh every 6 seconds.
Pressing <q> or <Q> allows you to exit the statistics display and returns you to the ACLI system prompt.
Media Alarms
The following table describes the Media alarms:
| Alarm Name | Alarm ID | Alarm Severity | Cause(s) | Example Log Message | Actions |
|---|---|---|---|---|---|
| MBCD ALARM OUT OF MEMORY | 262145 | CRITICAL: for flow
MAJOR: for media (if server cannot allocate a new context) |
No further memory can be allocated for MBCD. | Flow: Cannot create free port list for realm.
Media Server: Failed to allocate new context. |
apSyslogMessageGenerated(ap-slog.mib)
apSysMgmtMediaOutofMemory |
| MBCD ALARM UNKNOWN REALM | 262147 | MAJOR: if media server is adding a new flow | Media server is unable to find realm interface. | Realm type (ingress, egress, hairpin) X, not found | apSyslogMessageGenerated(ap-slog.mib)
apSysMgmtUnknownRealm |
| MBCD ALARM OUT OF BANDWIDTH | 262149 | CRITICAL: failure rate = 100%
MAJOR: failure rate > or = 50% |
The realm is out of bandwidth. | Out of bandwidth | apSyslogMessageGenerated(ap-slog.mib)
apSysMgmtMediaBandwidthTrap |
| MBCD ALARM OUT OF PORTS | 262150 | CRITICAL: failure rate = 100%
MAJOR: failure rate > or = 50% |
The realm is out of steering ports. | Out of steering ports | apSyslogMessageGenerated(ap-slog.mib)
apSysMgmtMediaPortsTrap |
System Problem Statistics
Packet Tracing
When you enable packet tracing (using the packet-capture configuration and related ACLI commands), the Oracle Communications Session Border Controller can mirror any communication between two endpoints, or between itself and a specific endpoint. To accomplish this, the Oracle Communications Session Border Controller replicates the packets sent and received, and can then send them to a trace server that you designate. Using the trace server, you can display the packets on software protocol analyzer. Currently, the Oracle Communications Session Border Controller supports:
- One configurable trace server (on which you have installed your software protocol analyzer)
- Sixteen concurrent endpoint traces
For more information about how to set up packet tracing, refer to the Oracle Communications Session Border Controller ACLI Configuration Guide.
You can see statistics for packet traces initiated on the Oracle Communications Session Border Controller by using the show packet-trace command. The display shows you a summary of the active packet traces on the Oracle Communications Session Border Controller. Displayed information includes: the IP address, local and remote port (which displays as 0 if no ports have been designated), slot, port, and VLAN.
ORACLE# show packet-trace IP Address Local-Port Remote-Port Slot Port VLAN --------------------------------------------------------- 192.168.10.1 0 0 0 1 0 192.168.10.99 5060 5060 0 1 0 10.0.0.1 23 0 1 0 0
Capturing and Viewing Packets
You can capture and view packets for debugging purposes by using the packet-capture command. For example, if you detect an issue with the system flows, you can capture certain packets so that you can resolve the problem. Using this command, you can examine the packets in question and then perform any debugging that might be necessary.
When you use packet-capture, you work with the following subcommands:
- packet-capture enable
- packet-capture show
- packet-capture detail
Use the packet-capture enable command to enable packet-capture before using it. Because enabling this function uses system resources that should otherwise be kept free, you should enable it only when you need it and then disable it when you finish debugging.
Use the packet-capture show command to view a summary of the most recently captured packets, including the following:
- ingress interface
- frame format
- type/length
- VLAN identifier
- source IPv4 or IPv6 address
- destination IPv4 or IPv6 address
- protocol
- source port
- destination port
For example:
acmepacket# packet-capture show
Entry Ingress Format Length VLAN-ID Src-IP Dest-IP Prot Src-Port Dest-Port
1 1/0 unknown 0x0026 - - - - - -
2 1/0 unknown 0x0026 - - - - - -
3 1/0 unknown 0x0026 - - - - - -
4 1/0 unknown 0x0026 - - - - - -
5 1/0 unknown 0x0026 - - - - - -
6 1/0 unknown 0x0026 - - - - - -
7 1/0 unknown 0x0026 - - - - - -
8 1/0 unknown 0x0026 - - - - - -
9 1/0 unknown 0x0026 - - - - - -
10 1/0 unknown 0x0026 - - - - - -
11 1/0 unknown 0x0026 - - - - - -
12 1/0 unknown 0x0026 - - - - - -
13 1/0 unknown 0x0026 - - - - - -
14 1/0 unknown 0x0026 - - - - - -
15 1/0 unknown 0x0026 - - - - - -
16 1/0 unknown 0x0026 - - - - - -
17 1/0 unknown 0x0026 - - - - - -
Use the packet-capture detail command to view the details of a particular packet, including: the ingress interface, MAC source address, MAC destination address, VLAN identifier, and the length/type. For example:
acmepacket# packet-capture detail 30
Ingress Slot/Port: 1/0
FF FF FF FF FF FF 00 0D 28 74 A2 01 08 00
45 00 00 4C 08 E9 00 00 40 11 61 18 AC 10 64 90 FF FF FF FF
00 7B 00 7B 00 38 00 00
1B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
DIX header ---
MAC Src Addr : 0x FF FF FF FF FF FF
MAC Dest Addr : 0x 00 0D 28 74 A2 01
VLAN ID : 0x XX
Length/Type : 0x 0800
IP Header ---
IP Version : 4
IP Header Length : 5
Type-of-Service : 0
Total Length : 76
Identificaton : 2281
Flags : 0
Fragment Offset : 0
Time-to-Live : 64
protocol : 17
Header Checksum : 0x6118
Source IP Addr : 172.16.100.144
Destination IP Addr : 255.255.255.255
UDP Header ---
Source Port : 123
Destination Port : 123
Length : 56
Checksum : 0x0000
System ACLs
This section provide information about system ACL removal, and about viewing system ACL statistics and configurations.
Notes on Deleting System ACLs
If you delete a system ACL from your configuration, the Oracle Communications Session Border Controller checks whether or not there are any active SFTP or SSH client was granted access when the entry was being removed. If such a client were active during ACL removal, the Oracle Communications Session Border Controller would warn you about the condition and ask you to confirm the deletion. If you confirm the deletion, then the Oracle Communications Session Border Controller’s session with the active client is suspended.
The following example shows you how the warning message and confirmation appear. For this example, and ACLI has been deleted, and the user is activating the configuration that reflects the change.
ORACLE # activate-config Object deleted will cause service disruption: system-access-list: identifier=172.30.0.24 ** WARNING: Removal of this system-ACL entry will result in the lockout of a current SFTP client Changes could affect service, continue (y/n) y Activate-Config received, processing.
Viewing System ACL Configurations
The system-access-list configuration has been added to the list of configurations you can display using the show configuration and show running-config ACLI commands. It will display each system ACL entry.
ORACLE# show running-config system-access-list system-access-list dest-address 165.31.24.2 netmask 225.225.0.0 last-modified-date 2007-04-30 13:00:02 system-access-list dest-address 175.12.4.2 netmask 225.225.225.0 last-modified-date 2007-04-30 13:00:21 task done
Viewing System ACL Statistics
You can display statistics for system ACLs using the show ip stats ACLI command. Two new entries have been added to let you see the total number of ACL denials and the last ACL denial the Oracle Communications Session Border Controller made.
ORACLE# show ip stats total 3170 badsum 0 tooshort 0 toosmall 0 badhlen 0 badlen 0 infragments 0 fragdropped 0 fragtimeout 0 forward 0 fastforward 0 cantforward 14 redirectsent 0 unknownprotocol 0 delivered 1923 localout 855 nobuffers 0 reassembled 0 fragmented 0 outfragments 0 cantfrag 0 badoptions 0 noroute 0 badvers 0 rawout 0 toolong 0 notmember 0 nogif 0 badaddr 0 acl-denials 1233 last-srcip-denied 174.35.60.72 ORACLE#
Wancom Port Speed and Duplex Mode Display
You can display the negotiated duplex mode and speed for all system control ports by using the ACLI show wancom command. This command allows you to diagnose network issues more efficiently.
When you use this command, the systems shows information for all three control ports with the numbers starting at 0. It will then tell you the negotiated duplex mode and speed, or that the link is down.
To display negotiated duplex mode and speed for control interfaces:
