The pre-tuned configuration files are generated by a script which is run as part of the installation process for new installs only. The files are placed in ASAP_Home/samples/sample_configs, where ASAP_Home is the directory in which ASAP is installed.

Table 4-1 lists and described the pre-tuned configuration files.

After the installation of ASAP is complete, back up the following files:

• ASAP_Home/config/ASAP.cfg

• ASAP_Home/ASAP.properties

• ASAP_Home/Environment_Profile

Replace these files with the appropriate three files corresponding to the desired pre-tuned configuration from ASAP_Home/samples/sample_configs.

For upgrade installations, the pre-tuned configuration files are not generated. However, the generation script can be run manually:

$ASAP_BASE/scripts/generate_sample_configs.ksh $ASAP_BASE

This will generate the pre-tuned configuration files to ASAP_home/samples/sample_configs.

If you have already made changes to the ASAP.cfg, ASAP.properties, or Environment_Profile files, you will have to manage the differences between your altered files and the pre-tuned files. For example, simply copying over the pre-tuned files will overwrite any changes you have made. To merge the pre-tuned file settings with your existing settings, compare the differences between your existing files and the pre-tuned files and manually add in the changes from the pre-tuned files.

Table 4-2, Table 4-3, and Table 4-4 provide example performance results on hardware. Your actual results will vary.

The following Sysmon sample output illustrates the cause of a slowdown in production. When the number in the “all connections in use (count)" row grows continually, the system is waiting for a free connection and is not being used to its maximum capacity.

-------------------------
Tuning - Connection Pool
-------------------------
Description                     Count  Total  Min   Max   Average  Mean  Deviation
-----------                     -----  -----  ---   ---   -------  ----- ---------
Application Pool 
  add connections                2      4.0    2.0   2.0    2.0     2.0     0.0
  all connections in use (count) 250  
  allocate wait-time             34454  1276   0.1   26.2   37.1    4386.3  4355.4
  connections in use             34454  1817.0 1.0   9.0    5.3     5.9     1.3
  pool size                      6      42.0   5.0   9.0    7.0     7.3     0.7
  Remove connection (count)      4
  Returned (count)               34453 

To correct this situation, do one of the following:

Increase the value of the configuration variable, APPL_POOL_SIZE to make more connections available.

Enable the auto-tuning option of ASAP using the configuration variable, CPM_OPTIMIZE_POOLS in the ASAP configuration file (ASAP.cfg), so that connections can be added automatically when necessary.

If a stored procedure fails, the thread running the procedure goes to sleep for the time determined by the RPC_RETRY_SLEEP configuration parameter. ASAP then tries to run the procedure with the number of times determined by the RPC_RETRY_COUNT configuration parameter. All of these attempts may fail. Since the thread cannot be released for the entire duration of this error retry process, poor performance is reported, increasing the number of connections in the use and long waiting times.

Table 4-5 lists and describes terms used in the Sysmon connection pool output.

The following Sysmon output section demonstrates the use of the Sysmon tool, and illustrates a condition which can degrade the responsiveness of ASAP.

The Group Manager Msg Q size has maximum, average, and mean sizes that are greater than required. For tunable message queues in ASAP, a non-zero size less than 5 is optimal, but not always achievable. In this case, messages are building up as they wait for processing by the group manager threads.

----------------------
Tuning - Message Queue
----------------------
Description             Count   Total   Min   Max   Average  Mean   Deviation
------------            ------  ------  ----  ----- -------  -----  ----------
ASDL Provision Queue 
  message read wait time 1056  5995.5   0.9   62.2   5.68    14.30   10.22
  messages sent (count)  1056
  queue idle-time (ms)   1056  59705.0  0.0   114.2  56.54   56.73   19.03
  queue size (count)     1056  0.0      0.0   0.0    0.0     0.0     0.0
  ... 

Group Manager Msg Q
  message read wait time 2469  38896.8  2.5   290.5  15.75   59.33    48.0
  messages sent (count)  2469   
  queue idle-time (ms)   2469  1870.6   0.0   718.3  0.758   120.22   119.72
  queue size (count)     2469  27184.0  0.0   27.1   11.01   11.86    4.5
  ...

To decrease the length of the queue and enhance the responsiveness of the system, increase the number of group manager threads in the system.

This is a simplified example of how to use the Sysmon tool to tune a message queue. Refer to the sections on the SRP, NEP, and SARM tuning for further details.

Table 4-6 lists and describes terms used in Sysmon output.

The purpose of tuning the SRP is to provide WOs at a rate that creates an even flow to the downstream SARM process.

Figure 4-1 illustrates the schematic flow of the SRP.

Table 4-8 lists and describes the WO manager queue.

Table 4-9 lists the SARM driver message queues.

To tune the SRP, use the following:

• As an initial estimate, set the number of WO Manager Threads (MAX_WO_MGRS) in the SRPs equal to the number of configured MAX_SRP_DRIVERS threads in the SARM. The initial setting can be modified depending upon the complexity of the WO (number of CSDLs and parameters), and the number of events defined to return to the SRP. Monitor the WO Manager queue(s) in the SRP(s) and the SRP driver queue in the SARM to verify these configuration variables.

• The sum of all MAX_SARM_DRIVER threads in the operating SRPs must be equal to the number of configured MAX_WO_HANDLERS threads in the SARM.

• The number or existence of translation threads is entirely dependent upon the customized implementation of the SRP at your site. If multiple translation threads are available and the SARM Driver queue is consistently empty, consider increasing the number of translation threads. However, increasing these threads may have a negative affect on the downstream process. You must examine the system for downstream bottlenecks.

• You must disable all unnecessary notifications being sent to the SRP by updating tbl_asap_srp in the SARM database.

• During production, you can use sanity level diagnostics.

• If the save and dump WO features, which are enabled by using configuration variables SAVE_SARM_DATA and DUMP_WO_FLAG, are not needed, disable them.

The purpose of tuning the SARM is to:

• Provide the Atomic Service Description Layer (ASDL) commands to the NEPs.

• Send event notices back to the SRPs at an even rate to both the upstream and downstream processes.

Since only one SARM process exists in an ASAP system, the performance of the SARM cannot be enhanced by spreading the load across multiple Central Processing Units (CPUs) or systems. Therefore, the SARM must be well tuned to get high performance from ASAP.

Figure 4-2 illustrates the flow of messages through the SARM.

Table 4-10 lists and describes the group Mgr message queue.

Table 4-11 lists and describes the WO Mgr message queues.

Table 4-12 lists and describes the WO provision queue.

Table 4-13 lists and describes the ASDL Provision Message Queues.

Table 4-14 lists and describes the NEP driver message queues.

There is one NEP Driver Queue for each NEP in the system.

Table 4-15 lists and describes the SRP driver message queues.

There is one SRP Drive Queue for each SRP in the system.

To tune the SARM, use the following:

• The number of WO threads (MAX_WO_HANDLERS) in the SARM must be equal to the sum of all SARM Driver Threads (MAX_SARM_DRIVER) in all of the SRPs.

• The total number of configured handler threads (MAX_WO_MGRS) in all SRPs must be equal to the driver threads (MAX_SRP_DRIVERS) in the SARM.

• All event notifications not used by any customized SRP implementation must be turned off.

• Sanity level diagnostics during production should be used.

• The configured number of provision handlers (ASDL and WO) must be no less than the sum of the number of handler threads (MAX_WO_HANDLERS) and the number of operating NEP servers. Start tuning with a ratio of 1:3 of WO Manager Threads to provision handlers (ASDL and WO).

The purpose of tuning a NEP is to provide:

• ASDLs to the NEs

• Remote Procedure Call (RPC) responses back to the SARM at a rate that does not cause excessive buildup in any of the SARM Notify, Session Manager, or NE Command queues.

Figure 4-3 illustrates the schematic flow of the NEP.

Table 4-16 lists and describes the SARM notify message queue.

You can manage this queue indirectly by decreasing the number of NEs supported by the NEP (usually by increasing the absolute number of NEPs, if machine resources permit) or by balancing the load between NEPs (by moving a busy NE from a busy NEP to a less busy NEP).

Table 4-17 lists and describes the NE command queue.

You can manage this queue indirectly by increasing or decreasing NE response times (faster response times decrease the length of the queue).

Table 4-18 lists and describes the session manager queue for the NE.

You can manage this queue indirectly using the NE response times (fast responses increase the length of the queue). However, depending on communication and switch technology, this may not be configurable.

To tune the NEP, use the following:

• Unless machine resources are limited, target for a ratio of 50 NEs per NEP.

• If possible, group similar NE technologies and switch software loads on a single NEP. This reduces the number of ASDL programs held in memory because each NEP caches its own copy of every ASDL command that it has been asked to perform.

• Balance the load on each NEP by distributing your busy NEs across several NEPs. You must determine the expected load based on both the number and complexity of the ASDL commands being serviced.

ASAP diagnostic files on NFS-mounted file systems increase network traffic and slow down disk I/O. For production systems, the log directories should be local, not NFS mounted.

Low-level server diagnostic levels increase disk I/O. Oracle recommends that you set the diagnostic level for the production system at either PROGRAM_LEVEL or SANITY_LEVEL.

In the Control server database table tbl_appl_proc, set the diagnostics level (column diag_level) to SANE.

Diagnostic levels set in the tbl_appl_proc are persistent through reboots. For more information on tbl_appl_proc, see ASAP Developer's Guide.

To set a diagnostic level temporarily, use asap_utils parameter 107. See the ASAP Server Configuration Guide for more information.

Diagnostic levels are defined as follows in the common.h file. Both diagnostics and C/C++ code values are provided.

The most commonly used diagnostic levels are LOW_LEVEL, SANITY_LEVEL, and PROGRAM_LEVEL.

Output of diagnostic messages can be written to disk line-by-line or buffered by the UNIX I/O subsystem. Buffered output results in diagnostics being written to disk in pages, which results in optimal performance.

To disable diagnostic line flushing, set the configuration parameter DIAG_LINE_FLUSH (in the common application programming interface (API) section of the ASAP.cfg file) to “0." The default value is 1.

Oracle RDBMSs use a cost-based query optimizer to determine query access paths. The optimizer is primarily influenced by table and index statistics (number or rows, data distribution, etc.) available to it in the system catalogue tables. These statistics can be updated manually (usually when the amount and distribution of data in a table changes significantly) using utilities provided by the RDBMS. Keeping these statistics current is extremely important since the optimizer will make default assumptions in the absence of these statistics. The quality of the database optimizer decisions depends on the accuracy of these statistics, and hence, directly affects the performance of the ASAP application.

Updating the statistics on large tables (over 1 million rows) can take a long time. This can affect your online response time or induce rollover to small error messages.