Viewing Active Processes
This section explains how to display statistics for active processes by displaying the task information for the system. By using the show processes command, you can view the system tasks in a table.
The information in this table is useful not only for viewing the process running on the system, but also for obtaining task names and identification numbers (TIDs in this table) for carrying out notify and stop-task commands.
This table contains the following information: names of tasks, entries, task identification codes, priority of a task, status, program counter, error numbers, and protector domain identification.
ORACLE# show processes
TaskName Id PPID Pri Status EIP Up Time RSS(kb) vMEM(kb) Proc
-------- ---- ---- --- -------- ------------ ----------- ------- ------- ----
uscc 2115 1557 20 SLEEPING 7fe09fa86974 0:16:37.660 1366524 5932844 0
intt 2110 1 20 SLEEPING 7fafeee76350 0:16:37.670 412 19100 3
intt 2109 1 20 SLEEPING 7fafeee76350 0:16:37.670 412 19100 2
dropberr 1984 1 20 SLEEPING 7fdaa36169c3 0:16:38.710 668 22668 1
getty 1849 1 20 SLEEPING 7f75dae9c350 0:16:39.010 1016 19104 5
syslod) 1840 1 20 SLEEPING 7f5def91b350 0:16:39.030 660 19100 0
klogd 1838 1 20 SLEEPING 7ff68d78a267 0:16:39.030 660 19100 0
Accessing Process Subcommands
Display the help text for the show processes command to access the following subcommands:
ORACLE# show processes ?
The following table lists and defines some of the subcommands and additional capabilities of the show processes command.
show processes Subcommand | Description |
---|---|
sysmand | Statistics for the sysmand process, which is related to the system startup tasks. sysmand starts and keeps track of many of the system tasks. All application tasks send their system log messages to the sysmand task and all notify requests go through sysmand. |
lemd | Statistics for the local element management (lemd) process, which is responsible for maintaining and providing local and remote access to data (including configuration and policy data) stored in the system. |
brokerd | Statistics for the brokerd process, which is a log concentrator and sequencer used for forwarding path and hardware monitor tasks. |
mbcd | Statistics for the mbcd process, which is the process for the middlebox control daemon. It provides signalling applications with the ability to dynamically manage (crete, modify, delete, and receive flow event notifications) NAT entries (pinholes) for media flows via the MIBOCO protocol. |
sipd | Statistics for sipd process statistic, which acts as a SIP server that receives and forwards them on the behalf of the requestor. sipd is responsible for processing SIP (RFC 3261) messages. It NATs the Layer 5 signaling content (for example, SIP message headers) and manages the associated media flows via tMBCD. |
current | Current statistics for all processes. |
total | Total statistics for all processes. |
all | All statistics for all processes. |
cpu | Percentage of CPU utilization by all processes. |
Viewing Totals for all Processes
Display total statistics for all processes by using the show processes total command.
ORACLE# show processes total
12:32:34-94
Process Svcs Rcvd Sent Events Alarm Slog Plog CPU Max
sysmand 29 35961 45 5340 0 11 58 0.0 0
acliSSH0 4 0 3 0 0 6 6 0.0 0
brokerd 2 20 4 0 3 4 4 0.0 0
cliWorke 2 0 2 0 0 5 6 0.0 0
lemd 5 5 28 3 0 26 36 0.0 0
collect 3 1 6 0 0 8 8 0.0 0
atcpd 5 1 8 1062468 0 10 12 0.0 0
atcpApp 4 1 5 0 0 7 8 0.0 0
mbcd 9 1 30 23112 0 32 38 0.0 0
lid 3 1 6 0 0 8 8 0.0 0
algd 6 1 9 5334 0 11 13 0.0 0
radd 3 1 9 5333 0 11 11 0.0 0
pusher 3 1 6 0 0 8 8 0.0 0
ebmd 5 1 9 10668 0 11 11 0.0 0
sipd 5 3 17796 58671 0 17796 17799 0.0 0
lrtd 4 1 5 0 0 7 10 0.0 0
h323d 6 1 17835 80005 0 17837 17843 0.0 0
h248d 2 0 24 5334 0 27 27 0.0 0
secured 5 1 6 0 0 8 10 0.0 0
snmpd 4 1 7 0 0 9 9 0.0 0
acliSSH1 4 0 3 0 0 6 6 0.0 0
acliSSH2 4 0 3 0 0 6 6 0.0 0
acliSSH3 4 0 3 0 0 6 6 0.0 0
acliSSH4 4 0 3 0 0 6 6 0.0 0
acliCons 3 1 16 0 0 18 18 0.0 0
acliTeln 4 22 92 3 0 73 73 0.0 0
acliTeln 4 6 20 0 0 16 16 0.0 0
acliTeln 4 0 3 0 0 6 6 0.0 0
acliTeln 4 0 3 0 0 6 6 0.0 0
acliTeln 4 0 3 0 0 6 6 0.0 0
tTaskChe 0 0 0 0 0 0 0 0.0 0
Viewing Current Statistics
Display the current statistics for all processes by using the show processes current command.
ORACLE# show processes current
12:35:12-52
Process Svcs TOQ Ops Rcvd Sent Events Alrm Slog Plog CPU Now
sysmand 29 2 15 11 0 1 0 0 0 0.0 0
acliSSH0 4 1 1 0 3 0 0 6 6 0.0 0
brokerd 2 0 8 3 0 0 0 0 0 0.0 0
cliWorke 2 0 2 0 0 0 0 0 0 0.0 0
lemd 5 0 3 0 0 0 0 0 0 0.0 0
collect 3 0 34 0 0 0 0 0 0 0.0 0
atcpd 5 1 307 0 0 304 0 0 0 0.0 0
atcpApp 4 0 3 0 0 0 0 0 0 0.0 0
mbcd 9 2 7 0 0 6 0 0 0 0.0 0
lid 3 0 3 0 0 0 0 0 0 0.0 0
algd 6 1 4 0 0 1 0 0 0 0.0 0
radd 3 1 5 0 0 2 0 0 0 0.0 0
pusher 3 0 3 0 0 0 0 0 0 0.0 0
ebmd 5 2 4 0 0 2 0 0 0 0.0 0
sipd 5 2 16 0 5 16 0 5 5 0.0 0
lrtd 4 0 3 0 0 0 0 0 0 0.0 0
h323d 6 3 16 0 5 22 0 5 5 0.0 0
h248d 2 1 4 0 0 1 0 0 0 0.0 0
secured 5 0 3 0 0 0 0 0 0 0.0 0
snmpd 4 0 3 0 0 0 0 0 0 0.0 0
acliSSH1 4 1 1 0 3 0 0 6 6 0.0 0
acliSSH2 4 1 1 0 3 0 0 6 6 0.0 0
acliSSH3 4 1 1 0 3 0 0 6 6 0.0 0
acliSSH4 4 1 1 0 3 0 0 6 6 0.0 0
acliCons 3 0 3 0 0 0 0 0 0 0.0 0
acliTeln 4 0 48 0 0 0 0 0 0 0.0 0
acliTeln 4 1 4 0 4 0 0 3 3 0.0 0
acliTeln 4 1 1 0 3 0 0 6 6 0.0 0
acliTeln 4 1 1 0 3 0 0 6 6 0.0 0
acliTeln 4 1 1 0 3 0 0 6 6 0.0 0
tTaskChe 0 0 1 0 0 0 0 0 0 0.0 0
Realtime CPU usage
The show processes top command displays realtime updates of per-process CPU utilization. To quit, press "q".
ORACLE> show processes top
top - 13:57:22 up 23:48, 0 users, load average: 0.00, 0.01, 0.05
Tasks: 117 total, 2 running, 115 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 2966240k total, 398016k used, 2568224k free, 66536k buffers
Swap: 0k total, 0k used, 0k free, 116032k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 3724 1100 936 S 0 0.0 0:30.13 init
2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0 0.0 0:04.49 ksoftirqd/0
6 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/0
7 root RT 0 0 0 0 S 0 0.0 0:00.33 watchdog/0
8 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/1
9 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/1:0
10 root 20 0 0 0 0 S 0 0.0 0:00.08 ksoftirqd/1
11 root 20 0 0 0 0 R 0 0.0 0:02.16 kworker/0:1
12 root RT 0 0 0 0 S 0 0.0 0:00.17 watchdog/1
13 root 0 -20 0 0 0 S 0 0.0 0:00.00 khelper
418 root 20 0 0 0 0 S 0 0.0 0:00.13 sync_supers
420 root 20 0 0 0 0 S 0 0.0 0:00.00 bdi-default
421 root 0 -20 0 0 0 S 0 0.0 0:00.00 kblockd
423 root 20 0 0 0 0 S 0 0.0 11:31.98 kworker/1:1
559 root 0 -20 0 0 0 S 0 0.0 0:00.00 ata_sff
570 root 20 0 0 0 0 S 0 0.0 0:00.00 khubd
Checking Remaining Space
Check the amount of storage space is available on the flash file system on the following devices by using the check-space-remaining command:
- /boot
- /code
- /crash
- /opt
- the data-disk, system-disks or a combination
For example:
ORACLE# check-space-remaining boot
boot: 20127744/29760512 bytes (67%) remaining
SMP-Aware Task Load Limiting
The ability to manage CPU load is critical for highly available devices. SMP architectures require unique load limiting because a task’s threads may be spread out over several cores. The SBC employs a method of determining aggregate load in its SMP environment so that resources may be evenly spread across all CPUs and applications can decrease their load when necessary. In turn, traffic may be dropped or rejected depending on the application to reduce the CPU load to an acceptable value.
Load limiting can be performed on three fundamental processing areas of the SBC:
- Transport Limiting—When the system’s CPU load rises above the transport limiting value, traffic received from endpoints is dropped. The system scales back the load of the processes where network packets (TCP, SCTP, and UDP) are disassembled.
- Media Limiting—When the system’s CPU load rises above the Media limiting value, the process that creates end-to-end media interface connections begins to drop requests (changes to existing transport sessions can still occur) thereby reducing CPU load. The system replies to SIP requests that initiate transport sessions with 503 Service Unavailable (this would typically be an INVITE with SDP).
- SIP Limiting—The system can manage percent CPU
utilization by limiting the number of SIP messages it processes on a per thread
basis, as follows:
- Begins rejecting SIP requests when the CPU reaches its throttling threshold, and Rejects all SIP requests when the CPU reaches its maximum utilization.
- Overall CPU Usage—The system can manage overall percent CPU utilization and reject
calls regardless of the thread utilization managed by your SIP Limiting
configuration. This establishes a second level of limiting, protecting against
issues with CPU usage beyond usage of a given thread. Call rejection follows the
same logic as SIP Limiting:
- Begins rejecting SIP requests when the CPU reaches its throttling threshold, and
- Rejects all SIP requests when the CPU reaches its maximum utilization.
Calculating CPU Load Limits
This section explains how and when the Oracle Communications Session Border Controller performs SIP message throttling to limit CPU utilization. This configuration also affects IDS management information.
The Oracle Communications Session Border Controller limits CPU utilization by SIP requests using two utilization percentage points. When CPU utilization reaches the first point, the system begins to reject SIP requests. When it reaches the second point, the system rejects all SIP requests. By default, these values are 90% and 99%. The user can change these values using a sip-config option called load-limit.
The load-limit option accepts two parameters, from which it determines these levels, including:
- Minimum CPU Utilization - The CPU utilization percentage at which the system begins to throttle back on the SIP requests it processes.
- CPU Limit for Headroom Calculation - A variable the system uses to compute maximum CPU utilization, at which it stops accepting SIP requests.
Syntax for this option is:
load-limit= < Minimum CPU Utilization > [-< CPU Limit for Headroom Calculation >]
User settings include:
- Minimum CPU Utilization - The range is 15% to 100%, and the default is 90%.
- (Optional) CPU Limit for Headroom Calculation - The range is 15 to 100, and the default is 100.
The calculation the system uses to determine the maximum CPU percent utilization is shown below, with X representing the Minimum CPU Utilization, and Y representing the CPU Limit for Headroom Calculation.
X + ((Y - X) * X/Y)%
The example setting below sets the throttling threshold to 60% and the maximum utilization to 75%.
load-limit= 60-80
The calculation the system uses is 60 + (80 - 60) * 60/80 = 75
The system progressively rejects requests as CPU utilization increases. When CPU utilization is between X% and the maximum, the system accepts some new SIP requests, depending on CPU utilization and utilization configuration. The system calculates this acceptance rate using the formula below.
100 - ((Current CPU Util - X) * 100 / ((Y - X) * X/Y))
When CPU utilization reaches its maximum, it drops all new SIP requests. The system resumes accepting requests when CPU utilization falls below its maximum, and stops throttling when it falls below the Minimum CPU Utilization.
While rejecting a SIP request, the system returns a 503 service unavailable message, along with a Retry-After header. The user can configure the reject-interval using the sip-config's reject-interval=X option. The default value is 1 second.
The actual value of reject interval header embedded in the 503 message is:
RetryAfter = (100 - Acceptance Rate) /10 * rejectInterval
If this value is smaller than the configured reject-interval, the system overwrites it with the configured reject-interval value.
Note the table below, which displays some valid and invalid configuration entries.
User Configuration | Headroom | Max CPU Limit | Configuration Valid? |
---|---|---|---|
load-limit=60 | (100 - 60) * 60 /100 = 24 | 60+24 = 84 |
Yes |
load-limit=60-80 | (80 - 60) * 60/80 = 15 | 60+15 = 75 |
Yes |
load-limit=80-60 | (100 - 80) * 80 / 100 = 16 | 80+16 = 96 |
No - (Upper limit < Lower limit) |
load-limit=80-101 | (100 - 80) * 80 / 100 = 16 | 80+16 = 96 |
No - (Upper limit > 100) |
load-limit=-70 | (100 - 90) * 90 / 100 = 9 | 90 + 9 = 99 |
No - default of 90% is used for lower limit, 100% for upper limit |
load-limit=70- | (100 - 70) * 70 / 100 = 21 | 70 + 21 = 91 |
No - upper limit default - 100% is used |
load-limit=- | (100 - 90) * 90 / 100 = 9 | 90 + 9 = 99 |
No - default - 90% is used. |
load-limit=80-ABC | (100 - 80) * 80 / 100 = 16 | 80 + 16 = 96 |
No - the upper limit default of 100% is used |
load-limit=80 90 | (100 - 80) * 80 / 100 = 16 | 80 + 16 = 96 |
No - the upper limit default of 100% is used |
Managing Overall CPU Utilization
You can configure the SBC to use overall system CPU thresholds in addition SIP thread specific CPU thresholds as criteria for managing CPU utilization. This feature is supported on both SWDP and HWDP platforms. For SWDP platforms, the SBC calculates overall CPU as the cumulative utilization of all cores. For HWDP platforms, the SBC calculates overall CPU as the cumulative utilization of all signaling cores. This feature is applicable only for dialog creating SIP requests, including INVITE and SUBSCRIBE, and is not applicable for responses. The action the system takes for this feature is to drop random, then all new SIP calls. This feature, however, never causes the system to drop priority calls.
Without this feature, the SBC can admit/reject new SIP calls on a per-sipd thread basis rather than considering overall CPU utilization. You configure the load-limit option in the sip-config to perform this function. But it is possible for the overall system CPU to be overloaded even though the sipd thread CPU is below its threshold limit. With this new feature, you can extend your CPU monitoring configuration by setting the syscpu-load-limit option in the system-config. This evaluates CPU resources available system wide, rather than those consumed by a specific thread.
Details on load-limit configuration, operational calculations and behaviors are the same with the syscpu-load-limit. The former measures and acts on CPU utilization by individual SIPD threads; the latter is based on overall CPU utilization. Both the load-limit and syscpu-load-limit options begin to reject random, new calls (INVITE/SUBSCRIBE) when they reach their minimum thresholds, and reject all new calls when they reach their maximum thresholds. When a syscpu-load-limit threshold is exceeded, the system stabilizes by rejecting calls, regardless of the sipd thread status.
These rejections are the only actions taken to reduce CPU load. The system rejects new calls (INVITE/SUBSCRIBE) from SIPD with a 503, thereby preventing retransmissions from the UAS.
This enhancement is disabled by default. Option syntax is shown below.
ORACLE(system-config)# options +syscpu-load-limit=[X]-[Y].
- X specifies the percent CPU utilization at which the system starts rejecting some SIP requests. The default is 90% and the valid range is 15 to 100.
- Y specifies the calculation variable the system uses to determine
maximum percent CPU utilization, which is based on current system CPU headroom, and
is optional. This setting determines the difference between system MinCPU
utilization and MaxCPU utilization. The default is 100 and the valid range is 15 to
100.
The SBC calculates system CPU headroom using the formula ((Y-X)*X/Y)%. This value becomes the percentage CPU above MinCPU, at which the system proceeds from rejecting random new calls to rejecting all new calls (INVITE/out of dialog SUBSCRIBE).
- If X is greater than the Y, the system sets Y to 100 and uses X as the operational value.
System behaviors when configured with the syscpu-load-limit option include:
- If current system CPU is less than MinCPU, it accepts all new requests.
- If the current SIPD threads reaches the minimum CPU threshold, the system begins to reject some random calls (throttling).
- While rejecting a SIP request, the system returns a 503 service unavailable message, along with a Retry-After header.
- If overall CPU utilization reaches the maximum CPU threshold (throttling threshold + CPU headroom), the system drops all new SIP requests.
- The system resumes accepting requests when CPU utilization falls below its maximum, and stops throttling when it falls below the Minimum CPU Utilization.
You can also configure the value the system sends for the Retry Header. When configuring syscpu-load-limit, you can configure the syscpu-reject-interval in the system-config. This value specifies the time in seconds the system inserts as the value of the Retry-After header.
Note:
The reject-interval option performs the same function to support the load-limit option.The default value for the syscpu-reject-interval option is 1 second. If the calculated reject-interval value is smaller than the configured syscpu-reject-interval, the system overwrites it with the configured reject-interval value.
ORACLE(system-config)# options +syscpu-reject-interval=5
Traffic that Applies to CPU Management
The SBC evaluates all system requests before rejecting any traffic based on CPU utilization. When system CPU reaches its rejection thresholds, the system rejects out-of-dialog requests, which are requests received without a to-tag. The SBC considers these requests to be new, as opposed to existing CPU loads. In contrast, the SBC does not reject In-dialog requests, allowing it to maintain existing sessions. In addition, the system does not reject any emergency calls based on CPU utilization because those consist of the highest priority traffic.
Applicable out-of-dialog requests, which the system rejects based high CPU utilization, include:
- INVITE—Establish a session
- SUBSCRIBE—Subscribe for notifications from the notifier
- MESSAGE—Transport Instant Messages
- OPTIONS—Communicate information about the capabilities of the calling and receiving SIP phones
Applicable in-dialog requests, which the system does not reject, include:
- UDPATE—Modify the state of a session
- REFER—Ask the recipient to issue call transfer
- INFO—Send mid-session information
- PUBLISH—Publish an event to a Server
- NOTIFY—Notify the subscriber of a new event
- PRACK—Provisional acknowledgments
- CANCEL—Cancel the establishment of a session
- BYE—End a session
- ACK—Confirm an INVITE request
- REGISTER—Communicate user location
This feature does not apply to SIP responses.
Reporting
You can see statistics related to this feature's behavior within the show sipd status, show platform cpu-load and show processes overload commands.
ORACLE# show platform cpu-load
Timestamp: 14:28:00 Tue 2024-05-28
Total load : 1%
CPU 00 load : 2%
CPU 01 load : 1%
CPU 02 load : 2%
CPU 03 load : 1%
ORACLE# show processes overload
ATCPD:
Thread Name Active CPU Usage CPU Overloaded
atcpd01 No 0.0% No
MBCD:
Thread Name Active CPU Usage CPU Overloaded
mbcd01 No 0.0% No
SIPD:
Thread Name Active CPU Usage CPU Overloaded
sipd01 No 0.0% No
ORACLE#
In addition, you can run the show system-overload command to display the configured syscpu-load-limit values as well as current CPU utilization.
ORACLE# show system-overload
10:51:47-80
Minimum CPU Load Limit (Configured) : 20
Maximum CPU Load Limit (Configured) : 30
Maximum CPU Load Limit (Calculated) : 26
System CPU Headroom (Calculated) : 6
System:
Active CPU Usage CPU Overloaded
Yes 1% No
Alarm
The system raises an alarm when the system CPU reaches its critical threshold and above. Severity levels for this alarm include Minor, Major and Critical. This alarm is dedicated to this feature.
Alarm ID 131342
System CPU is at 92 percent, over minor/major/critical threshold of 85 percent
The name of this alarm is SYSTEM_CPU_UTIL_OVER_THRESHOLD_BASE. You can see this alarm on the ACLI using the display-alarms command. You can clear this alarm manually or by rebooting the system.