23 Monitoring Integration Processes

Cleaning Stale Sessions

Stale sessions are sessions that are incorrectly left in a running state after an agent or repository crash.

The Agent that started a session automatically detects when this session becomes stale and changes it to Error status. You can manually request specific Agents to clean stale sessions in Operator Navigator or Topology Navigator.

To clean stale sessions manually:

1. Do one of the following:

   • From the Operator Navigator toolbar menu, select Clean Stale Sessions.

   • In Topology Navigator, from the Physical Architecture accordion, select an Agent, right-click and select Clean Stale Sessions.

   The Clean Stale Sessions Dialog opens.

2. In the Clean Stale Sessions Dialog specify the criteria for cleaning stale sessions:

   • From the list, select the Agents that will clean their stale sessions.

     Select Clean all Agents if you want all Agents to clean their stale sessions.

   • From the list, select the Work Repositories you want to clean.

     Select Clean all Work Repositories if you want to clean stale sessions in all Work Repositories.

3. Click OK to start the cleaning process. A progress bar indicates the progress of the cleaning process.

Filtering Sessions

Filtering log sessions allows you to display only certain sessions in Operator Navigator, by filtering on parameters such as the user, status or duration of sessions. Sessions that do not meet the current filter are hidden from view, but they are not removed from the log.

To filter out sessions:

1. In the Operator Navigator toolbar menu, click Filter. The Define Filter editor opens.

2. In the Define Filter Editor, set the filter criteria according to your needs. Note that the default settings select all sessions.

   • Session Number: Use blank to show all sessions.

   • Session Name: Use % as a wildcard. For example DWH% matches any session whose name begins with DWH.

   • Session's execution Context

   • Agent used to execute the session

   • User who launched the session

   • Status: Running, Waiting etc.

   • Date of execution: Specify either a date From or a date To, or both.

   • Duration greater than a specified number of seconds

3. Click Apply for a preview of the current filter.

4. Click OK.

Sessions that do not match these criteria are hidden in the Session List accordion. The Filter button on the toolbar is activated.

To deactivate the filter click Filter in the Operator toolbar menu. The current filter is deactivated, and all sessions appear in the list.

Purging the Log

Purging the log allows you to remove past sessions and Load Plan runs from the log. This procedure is used to keeping a reasonable volume of sessions and Load Plans archived in the work repository. It is advised to perform a purge regularly. This purge can be automated using the OdiPurgeLog tool (see: "OdiPurgeLog") in a scenario.

To purge the log:

1. From the Operator Navigator toolbar menu select Connect Navigator > Purge Log... The Purge Log editor opens.

2. In the Purge Log editor, set the criteria listed in Table 23-4 for the sessions or Load Plan runs you want to delete.

   Table 23-4 Purge Log Parameters

   Parameter	Description
   Purge Type	Select the objects to purge.
   From ... To	Sessions and/or Load Plan runs in this time range will be deleted. When you choose to purge session logs only, then the sessions launched as part of the Load Plan runs are not purged even if they match the filter criteria.When you purge Load Plan runs, the Load Plan run which matched the filter criteria and the sessions launched directly as part of the Load Plan run and its child/grand sessions will be deleted.
   Context	Sessions and/or Load Plan runs executed in this context will be deleted.
   Agent	Sessions and/or Load Plan runs executed by this agent will be deleted.
   Status	Session and/or Load Plan runs in this status will be deleted.
   User	Sessions and/or Load Plan runs executed by this user will be deleted.
   Name	Sessions and/or Load Plan runs matching this session name will be deleted. Note that you can specify session name masks using % as a wildcard.
   Purge scenario reports	If you select Purge scenario reports, the scenario reports (appearing under the execution node of each scenario) will also be purged.

   
   

   Only the sessions and/or Load Plan runs matching the specified filters will be removed:

   • When you choose to purge session logs only, then the sessions launched as part of the Load Plan runs are not purged even if they match the filter criteria.

   • When you purge Load Plan runs, the Load Plan run which matched the filter criteria and the sessions launched directly as part of Load Plan run and its child/grand sessions will be deleted.

   • When a Load Plan run matches the filter, all its attached sessions are also purged irrespective of whether they match the filter criteria or not.

3. Click OK.

Oracle Data Integrator removes the sessions and/or Load Plan runs from the log.

Organizing the Log with Session Folders

You can use session folders to organize the log. Session folders automatically group sessions and Load Plan Runs that were launched with certain keywords. Session folders are created under the Keywords node on the Session List or Load Plan Executions accordions.

Each session folder has one or more keywords associated with it. Any session launched with all the keywords of a session folder is automatically categorized beneath it.

To create a new session folder:

1. In Operator Navigator, go to the Session List or Load Plan Executions accordion.

2. Right-click the Keywords node and select New Session Folder.

3. Specify a Folder Name.

4. Click Add to add a keyword to the list. Repeat this step for every keyword you wish to add.

Table 23-5 lists examples of how session folder keywords are matched.

To launch a session with keywords, you can for example start a scenario from a command line with the -KEYWORDS parameter. Refer to Chapter 21, "Running Integration Processes," for more information.

Exporting and Importing Log Data

Export and import log data for archiving purposes.

Exporting Log Data

Exporting log data allows you to export log files for archiving purposes.

To export the log:

1. Select Export... from the Designer, Topology, Security or Operator Navigator toolbar menu.

2. In the Export Selection dialog, select Export the Log.

3. Click OK.

4. In the Export the log dialog, set the log export parameters as described in Table 23-6.

   Table 23-6 Log Export Parameters

   Properties	Description
   Export to directory	Directory in which the export file will be created.
   Export to zip file	If this option is selected, a unique compressed file containing all log export files will be created. Otherwise, a set of log export files is created.
   Zip File Name	Name given to the compressed export file.
   Filters	This set of options allow to filter the log files to export according to the specified parameters.
   Log Type	From the list, select for which objects you want to retrieve the log. Possible values are: All|Load Plan runs and attached sessions|Sessions
   From / To	Date of execution: specify either a date From or a date To, or both.
   Agent	Agent used to execute the session. Leave the default All Agents value, if you do not want to filter based on a given agent.
   Context	Session's execution Context. Leave the default All Contexts value, if you do not want to filter based on a context.
   Status	The possible states are Done, Error, Queued, Running, Waiting, Warning and All States. Leave the default All States value, if you do not want to filter based on a given session state.
   User	User who launched the session. Leave the default All Users value, if you do not want to filter based on a given user.
   Session Name	Use % as a wildcard. For example DWH% matches any session whose name begins with DWH.
   Advanced options	This set of options allow to parameterize the output file format.
   Character Set	Encoding specified in the export file. Parameter encoding in the XML file header. <?xml version="1.0" encoding="ISO-8859-1"?>
   Java Character Set	Java character set used to generate the file.

   
   

5. Click OK.

The log data is exported into the specified location.

Note that you can also automate the log data export using the OdiExportLog tool (see "OdiExportLog").

Importing Log Data

Importing log data allows you to import into your work repository log files that have been exported for archiving purposes.

To import the log:

1. Select Import... from the Designer, Topology, Security or Operator Navigator toolbar menu.

2. In the Import Selection dialog, select Import the Log.

3. Click OK.

4. In the Import of the log dialog:

5. 1. Select the Import Mode. Note that sessions can only be imported in Synonym Mode INSERT mode. Refer to "Import Types" for more information.

   2. Select whether you want to import the files From a Folder or From a ZIP file.

   3. Enter the file import folder or zip file.

   4. Click OK.

The specified folder or ZIP file is imported into the work repository.

Runtime Logging for ODI components

You can set up runtime logging to trace ODI components or set a verbose level to investigate different issues or to monitor the system. The following ODI components can be traced: ODI Studio, ODI Java EE Agents, and ODI Standalone Agents.

Log Level

The log level can be set against a log_handler and/or logger elements. Note the following when setting the log level:

• If it is set against a log_handler, then it applies to all usages of the log_handler.

• If it is set against a logger, then it applies to all of its handlers and any descendent loggers that do not have an explicit level setting.

• A message is logged if its log-level is:

  • Greater or equal than (>=) the level of its logger AND

  • Greater or equal than (>=) the level of its log_handler

Table 23-7 shows the mapping between the Java log levels and the Oracle Java Debugging level (OJDL).

Setting Up Runtime Logging

To set up runtime logging you have to enable different ODI loggers and log handlers and set the related log level in the ODI logging system configuration file.

1. Open the ODI logging system configuration file of the ODI component.

   Each component has its own configuration file:

   • ODI Studio:

     $ODI_HOME/odi/studio/bin/ODI-logging-config.xml
     

   • ODI Java EE Agent:

     <DOMAIN_HOME>/config/fmwconfig/logging/oraclediagent-logging.xml
     

   • ODI Standalone Agent:

     <DOMAIN_HOME>/config/fmwconfig/components/ODI/<INSTANCE_NAME>/ODI-logging-config.xml
     

2. Make sure that the path to your log files is a valid and existing path. For example:

   <log_handler name="ODI-file-handler" class="oracle.core.ojdl.logging.ODLHandlerFactory"
   level="ALL">
     <property name="format" value="ODL-Text"/>
     <property name="path" value="/u01/oracle/odi11g/oracledi/agent/log/${LOG_FILE}"/>
     <property name="maxFileSize" value="1000000"/> <!-- in bytes -->
     <property name="maxLogSize" value="50000000"/> <!-- in bytes -->
     <property name="encoding" value="UTF-8"/>
   </log_handler>
   

   Note the following concerning the log files path:

   • For ODI Standalone Agents the default value points to a non-existing folder. Create a log folder or set another path.

   • If you are on Windows, the path could be for example:

     %ODI_HOME%\oracledi\agent\log\${LOG_FILE}
     

   • You can use a relative path on Windows and Unix.

3. Enable the logger and set the log level. For example, for logger oracle.odi.agent you can enable the most verbose logging setting:

   <logger name="oracle.odi.agent" level="TRACE:32" useParentHandlers="false">
     <handler name="ODI-file-handler"/>
     <handler name="ODI-console-handler"/>
   </logger>
   

4. Save the configuration file to take the changes into account.

Example INFO (NOTIFICATION:1) Message

The INFO (NORTIFICATION:1) Message is logged if:

• The logger level (possibly inherited) is <= NOTIFICATION:1

• The log_handler level is <= NOTIFICATION:1 (for example: TRACE:1)

The INFO (NORTIFICATION:1) Message is not logged if:

• The logger level (possibly inherited) is > NOTIFICATION:1

• The log_handler level is > NOTIFICATION:1 (for example: WARNING:1)

The Runtime logger is called oracle.odi.agent. Its message levels cover the following content:

NOTIFICATION:1 (or INFO) Agent Level
NOTIFICATION:16 (or CONFIG) the above + Session Level
TRACE:1 (or FINE) the above + Step Level
TRACE:16 (or FINER) the above + Task + SQL
TRACE:32 (or FINEST) the above + more detail

It is recommended that the console level for oracle.odi.agent is set so that Agent startup messages are displayed (NOTIFICATION:1).

Load Plan and Scenario Folders

In Operator Navigator, scenarios and Load Plans can be grouped into Load Plan and Scenario folders to facilitate organization. Load Plan and Scenario folders can contain other Load Plan and Scenario folders.

To create a Load Plan and Scenario folder:

1. In Operator Navigator go to the Load Plans and Scenarios accordion.

2. From the Load Plans and Scenarios toolbar menu, select New Load Plan and Scenario Folder.

3. On the Definition tab of the Load Plan and Scenario Folder editor enter a name for your folder.

4. From the File menu, select Save.

You can now reorganize your scenarios and Load Plans. Drag and drop them into the Load Plan and Scenario folder.

Importing Load Plans, Scenarios, and Solutions in Production

A Load Plan or a scenario generated from Designer can be exported and then imported into a development or execution repository. This operation is used to deploy Load Plans and scenarios in a different repository, possibly in a different environment or site.

Importing a Load Plan or scenario in a development repository is performed via Designer or Operator Navigator. With a execution repository, only Operator Navigator is available for this purpose.

See "Importing Scenarios in Production" for more information on how to import a scenario in production, and "Importing Load Plans" for more information on the Load Plan import.

Similarly, a solution containing several scenarios can be imported to easily transfer and restore a group of scenarios at once. See Chapter 19, "Using Version Control," for more information. Note that when connected to an execution repository, only scenarios may be restored from solutions.