1. Administering Oracle GoldenGate for HP NonStop (Guardian Data Sources)
  2. Configuring the Manager and Collector

5 Configuring the Manager and Collector

The Manager and the Collector are Oracle GoldenGate components that facilitate day-to-day data management. See how to configure, start, and run these components.

This topic includes the following sections:

Introducing Manager

Manager runs as a NonStop process, ensuring that Oracle GoldenGate for NonStop components run and restart if a CPU failure or operator error occurs. Manager spawns a copy of itself so that tasks that take longer, such as duplicating a TMF audit trail, do not interfere with real-time tasks. Tasks are divided between the Manager and its child process accordingly. Manager tasks on NonStop include:

  • Starting Logger, Extract, Replicat and Syncfile

  • Monitoring and reporting status of Oracle GoldenGate processing

  • Starting the dynamic Collector process on the target

  • Automatically restarting critical processes

  • Threshold reporting, such as when Extract falls behind the TMF-audit trail

  • Managing resources for the TMF audit trail, such as maintaining copies of audit trails on backup volumes

  • Purging trails when Extract and Replicat has finished with them

  • Pre-allocating log trail space for Logger processing

Configuring and Starting Manager

Now that you have your GGSCI prompt, you are ready to configure and start Manager. To configure Manager, create an appropriate parameter file. To control an Oracle GoldenGate Manager process, use the following commands.

Command Description 
START MANAGER

Starts Manager.

 
STOP MANAGER

Stops Manager. You can stop Manager gracefully, or forcefully with the ! option.

Creating and Configuring the Manager Parameter File

Enter Manager parameters in the MGRPARM file. If no MGRPARM file exists, default management parameters are taken. To add parameters, edit this file using the EDIT PARAMS MGRPARM command.

Manager retrieves parameters as established by GGSCI ATCONFIG commands. These parameters affect audit trail resource management.

See Oracle GoldenGate Parameters for more information about Manager parameters.

A Sample Manager Parameter File

A Manager parameter file would be similar to this sample.

Example 5-1 Sample Manager Parameter File

TCPIPPROCESSNAME $ZTC2
PORT 7844
DYNAMICPORTLIST 7850 - 7880, 7895
CHECKMINUTES 30
PURGEOLDEXTRACTS $DATA1.GGSDAT.*, USECHECKPOINTS, MINKEEPDAYS 2
THRESHOLD 30
LAGREPORTMINUTES 60
LAGINFOMINUTES 10
LAGCRITICALMINUTES 10
LOGFILESBEHIND 2
LOGFILESBEHINDINFO 10
DOWNCRITICAL
DOWNREPORTHOURS 1

In this Sample Manager Parameter File

  • TCPIPPROCESSNAME specifies the TCP/IP process. The default process is $ZTC0. Use the TCPIPPROCESSNAME parameter to specify a process other than the default.

  • Specify the PORT parameter so Manager can create a dynamic Collector process.

  • DYNAMICPORTLIST specifies up to 256 entries for ports to be dynamically assigned to processes started by Manager. If no dynamic ports are specified, Manager starts with port 7819 and increments until it finds an available port.

  • CHECKMINUTES 30 directs Manager to perform maintenance activities every 30 minutes. The default is 10 minutes.

  • Use the PURGEOLDEXTRACTS parameter when multiple Replicat processes are reading a set of trails. For this sample, PURGEOLDEXTRACTS directs manager to purge old files from the trail $DATA1.GGSDAT.*. The options:

    • USECHECKPOINTS specifies that Replicat checkpoints are to be used to determine when Replicat has finished processing.

    • MINKEEPDAYS 2 purges files only after they have been closed for 2 days.

  • THRESHOLD 30 directs Manager to generate an event message when the number of audit files remaining to be processed falls below 30%.

  • LAGREPORTMINUTES 60 specifies that Manager check lag every 60 minutes.

  • LAGINFOMINUTES 10 specifies that Manager report lag information to the event log every 10 minutes.

  • LAGCRITICALMINUTES 10 specifies that Manager write a critical message to the event log when there is 10 minute lag.

  • LOGFILESBEHIND 2 sends a critical message whenever a process lags a specified number of files behind the current log trail file.

  • LOGFILESBEHINDINFO 10 sends an informational message whenever the process falls the specified number of files behind.

  • DOWNCRITICAL sends a critical message whenever Extract or Replicat abends.

  • DOWNREPORTHOURS sends reports of Extract and Replicat abending.

Starting and Stopping Manager

You must start Manager before you can configure and run other Oracle GoldenGate components. The following example starts Manager in CPU 3. Manager selects a CPU in which to run a backup process for fault-tolerance. 

GGSCI> START Manager, CPU 3, PRI 170

If Manager encounters a TCP/IP error, for example if it attempts to bind to a port that is in use, it retries the error every 60 seconds and does not abend after a set number of attempts. Unlike other Oracle GoldenGate processes, it does not use the TCPERRS file to set the delay and the number of retries.

Manager runs indefinitely, or until you enter the GGSCI STOP MANAGER command. You might stop Manager if you need to stop the Extract and Replicat groups it manages or if you to want to activate a change to a Manager parameter.

See STOP MANAGER for more information about GGSCI commands for Manager.

Configuring and Running the Collector

The Collector collects data from Extract and writes data to files on the target system. Extract requests Manager to start a collector process when it sees data must transmit over TCP/IP to a remote trail. Once started, the Collector waits for and performs requests to write, open, and close files in the Oracle GoldenGate trail during Extract processing.

Note:

You do not need to run collector if data transmits over an Expand connection.

Maintaining Ports for Remote Connections through Firewalls

If a firewall is being used at an Oracle GoldenGate target location, additional ports are required on the target system to receive dynamic TCP/IP communications from remote Oracle GoldenGate processes. These ports are:

  • One port for each Collector process that is started by the local Manager to receive propagated transaction data from remote online Extract processes. When an Extract process sends data to a target, the Manager on the target starts a dedicated Collector process.

  • One port for each Replicat process that is started by the local Manager as part of a remote task. A remote task is used for initial loads and is specified with the RMTTASK parameter. This port is used to receive incoming requests from the remote Extract process. For more information see, RMTTASK

  • Some extra ports in case they are needed for expansion of the local Oracle GoldenGate configuration.

  • Ports for the other Oracle GoldenGate products if they interact with the local Oracle GoldenGate instance, as stated in the documentation of those products.

To specify these ports, use the DYNAMICPORTLIST parameter in the Manager parameter file. Follow these guidelines:

  • You can specify up to 300 ports in any combination of the following formats:

    7830, 7833, 7835
7830-7835
7830-7835, 7839

  • The ports must be unreserved and unrestricted.

  • Each Manager instance on a system must use a different port list.

Although not a required parameter, DYNAMICPORTLIST is strongly recommended for best performance. The Collector process is responsible for finding and binding to an available port, and having a known list of qualified ports speeds this process. In the absence of DYNAMICPORTLIST (or if not enough ports are specified with it), Collector tries to use port 7840 for remote requests. If 7840 is not available, Collector increments by one until it finds an available port. This can delay the acceptance of the remote request. If Collector runs out of ports in the DYNAMICPORTLIST list, the following occurs:

  • Manager reports an error in its process report and in the Oracle GoldenGate LOGGGS.

  • Collector retries based on the rules in the Oracle GoldenGate tcperrs file. For more information about the tcperrs file, see section"TCP/IP Error Handling".

For more information see, DYNAMICPORTLIST.

Configuring Collector

To configure a Collector, you must know the port the Collector will use, the host name or IP address where the remote trail resides, and edit your Manager and Extract parameter files. You may also specify a variety of operating options, described in the "Configuration Examples".

To configure and start Collector:

  1. In the Manager parameter file, specify the port parameter, such as: PORT 7809.

  2. In the Extract parameter file, specify the RMTHOST parameter as follows: 

    RMTHOST host, [MGRPORT port_number] [, option] [, . . .]
    Argument Description 
    host

    Either a remote system name or an IP address, such as: RMTHOST eastnode or RMTHOST 192.0.2.20. 

    MGRPORT port_number

    Specify the port that is defined in the Manager parameter file.

    		 
    options

    You can specify a variety of options, including Collector parameters. See "Creating and Configuring the Manager Parameter File" for information on these options. See Oracle GoldenGate Parameters for information about other RMTHOST options.

  3. The remote system must be the same system on which Collector was started, and the port number must match the port number in the Collector startup command. See "Configuration Examples" for more information.

  4. If you specify a remote system name in the RMTHOST parameter, you must also enter the remote system name in the TCP/IP hosts file on the target system, or on the names server for your network. For example, if you specify: RMTHOST eastnode, you must make an entry similar to: 192.0.2.20 eastnode in the HOSTS file.

    If you specify the remote system IP address in the RMTHOST parameter, there is no need to make a corresponding HOSTS file entry.

Configuration Examples

Following are the examples for configuration:

To configure Collector for port 5432 on remote system named eastnode:

  1. In the Manager parameter file, specify: PORT 5432

  2. In the Extract parameter file, specify: RMTHOST eastnode, MGRPORT 5432, and options if desired.

  3. In the TCP/IP HOSTS file, enter: 192.0.2.20 eastnode

To configure Collector for the default port on remote system address 192.0.2.20:

  1. In the Manager parameter file, specify: PORT 7809

  2. In the Extract parameter file, specify: RMTHOST 192.0.2.20, and options if desired.

  3. No TCP/IP hosts file entry is required.

The TCP/IP Port

There are two ways to use Collector and your TCP/IP port: dynamically and explicitly. The dynamic method lets Extract request Manager to start Collector as needed. However, a user can explicitly start the Collector and let it run in the background, ready to transmit data as needed. This method is called the explicit method.

Dynamic Method

Dynamic method is the default way to use Collector. The examples above illustrate how this is configured: a port is specified in the Manager parameter file, a remote trail is specified in the Extract parameter file, and, if required, the IP address is added to your hosts file.

Explicit Method

When capturing data over TCP/IP to remote systems that do not support dynamic Collectors, you must explicitly start a Collector on the target system before starting Extract. Each Extract must explicitly name the port to which it is writing, using the RMTHOST parameter.

To explicitly configure your Collector, start GGSCI and enter the following:

TACL > ASSIGN STDERR, event_message_collector
TACL > RUN SERVER /NOWAIT/ [-P port_number]

Note:

Your event_message_collector may be the standard system log, $0, or a virtual process, such as $VHS.

In the above example, the Collector listens on port 12345. When you start the Collector, it references a default TCP/IP process ($ZTC0). You can change this to the process of your choice by running a DEFINE statement before you start your collector. 

TACL > ASSIGN STDERR, $0
TACL> DEFINE =TCPIP^PROCESS^NAME, FILE $ZTC8
TACL > RUN SERVER /NOWAIT, NAME $COLL/ -P 12345

See Collector Parameters for more information about the Collector parameters.

Example 5-2 Explicit Method

TACL > ASSIGN STDERR, $0
TACL > RUN SERVER /NOWAIT, NAME $COLL/ -P 12345

Monitoring Collector

Collector event messages are output to the ggserr.log file. You can view this file using the GGSCI VIEW GGSEVT command.

Security Considerations

The user ID under which the Collector is started determines whether target files can be written and purged. Ensure that the ID has the proper system access to the files and locations written by the Collector.

Collecting Between Open Systems and NonStop

Event messages created by the Collector and Replicat on Windows and UNIX systems can be extracted and sent back to EMS on NonStop systems. This feature enables centralized viewing of Oracle GoldenGate messages across platforms.

To collect events from other systems:

  1. Run Collector on NonStop to collect and distribute EMS messages. For each EMSCLNT process, run one Collector process.

    The following example runs Collector and outputs its messages to $0.

    TACL> ASSIGN STDERR, $0
TACL> RUN SERVER /NOWAIT/ –p 7880

  2. Run the EMSCLNT utility on the remote target. EMSCLNT reads a designated error log and runs indefinitely, waiting for more messages to send. When EMSCLNT receives a message, it sends the message to a collector process on NonStop.

    See the examples for running EMSCLNT on open systems for syntax information.

    This UNIX example reads the file ggslog.err for error messages. Error messages are sent to the collector to the NonStop at IP address 192.0.2.2 listening on port 7850. The Collector on NonStop writes formatted messages to EMS Collector $0.

The Windows example below (from the DOS prompt) reads the file d:\ggserrs\log.txt for error messages. Error messages are sent to the collector on host ggs2 listening on port 9876. The Collector on NonStop writes formatted messages to EMS Collector $P0. 

> emsclnt –h ggs2 –p 9876 –f d:\ggserrs\log.txt –c $P0
Argument Description 
–h ggs2

The node on which the collector is being run. Can be a name or IP address. This is a required parameter.

 
–p 9876

The port where the collector is listening for messages. This is a required parameter.

 
–f d:\ggserrs\log.txt

The error file where EMSCLNT retrieves error messages. This is a required parameter.

 
–c $P0

The collector where EMS messages should be written on the NonStop (default is $0).

Example 5-3 Running EMSCLNT on Open Systems

> $emsclnt –h 192.0.2.2 –p 7850 –f ggserr.log –c $0