Cataloger

The Oracle Tuxedo Application Rehosting Workbench (Tuxedo ART Workbench) Cataloger analyzes all the components extracted from the source environment separately and together in order to determine whether the asset is consistent and can be migrated. The Cataloger also produces an internal form to be used by other tools.

This chapter contains the following topics:

•	Overview of the Cataloger

•	Description of the Input Components

•	Description of the Configuration Files

•	Description of the Output Files

•	Detailed Processing

•	Component Search Operation

•	Repetitive and Incremental Operation

Overview of the Cataloger

The Cataloger is one of the migration tools composing Tuxedo ART Workbench. Its purpose is twofold:

•	Analyze separately and together all the components in the source software system to determine whether this system is consistent and can be migrated. The anomalies detected during this analysis are reported in the cataloging reports, along with other inventory information.

•	Produce an internal form of these components and of their mutual relationships, to be used by the other tools in Tuxedo ART Workbench.

Inputs to the Cataloger Process

•	The source files for the components in the source asset, after they have been transferred from the source platform to the migration platform and converted to be better viewed and processed;

•	One or more configuration files:

•	The System Description File (mandatory), which describes how the input source files are organized on the migration platform file system, and also gives additional parameters necessary for their parsing;

•	The Cataloger option file (optional), which gives parameters for the analysis phase of the Cataloger (see Detailed Processing).

•	The JCL-Launcher Specification Files are used to describe the launchers used in a given asset, so that the cataloguer and the JCL translator can extract relevant information such as the name of the real program to launch.

•	Hint files (optional), which give information that the Cataloger cannot find out by itself, for instance on dynamic program calls.

Outputs from the Cataloger Process

•	A set of cataloging reports, in CSV form, describing the components and their status in the asset (correct, unused, missing);

•	For each (parsable) component, a binary pob-file containing the internal form of the component, suitable for further processing such as conversion or translation;

•	Other system-wide pob files, such as the Symtab (see The Cataloger Symtab and Other Miscellaneous Files), for use by the Cataloger and other Tuxedo ART Workbench tools;

•	When necessary, some other non-binary files for use by other Tuxedo ART Workbench tools.

The Cataloger Process

The Cataloger process is divided into four logical phases:

1.	Parsing: each component in turn is read, parsed (syntactic analysis) and linked (semantic analysis), and the corresponding pob-file is generated.

2.	Analysis: for each component, the pob-file is re-read and the most significant constructs in the component are translated into a smaller summary information stored in the Cataloger symbol table (symtab).

3.	Post-analysis: working with just the symtab and the summary information, the Cataloger computes some cross-reference links allowing to label each component as correct, unused or missing.

4.	Report generation: the symtab decorated with cross-reference links is traversed and information is printed out for each component.

Depending on the needs of the project and the migration-platform configuration, these phases can be executed sequentially or concurrently, in a single run or incrementally. See Repetitive and Incremental Operation and the Oracle Tuxedo Application Runtime Process Guide for further information.

Description of the Input Components

The Cataloger accepts as input the source files of a complete, working software application running on a z/OS platform. It should be composed entirely of the following types of files which are described in greater detail in the following sections:

•	COBOL programs (possibly containing EXEC SQL and/or EXEC CICS statements)

•	JCL scripts

•	SQL DDL scripts defining the database schema

•	CICS RDO files defining the CICS transactions

•	Sub-files of the above types, when relevant

COBOL

References

The Tuxedo ART Workbench COBOL parser accepts the COBOL language as specified in the IBM Enterprise COBOL for z/OS Language Reference Version 3 Release 4 (document number SC27-1408-04).

Restrictions

The following constructs or features are not accepted:

•	Multiple programs per compilation unit, especially nested programs.

•	All object-oriented features.

•	DBCS (USAGE DISPLAY-1) and Unicode (USAGE NATIONAL) constructs.

•	MBCS does not support variable names.

•	Millennium Language Extensions (date fields).

Sub-Programs

The parser processes COBOL programs and their sub-programs. If a sub-program does not exist in a Workbench project and it is called inside a COBOL program, Workbench reports the sub-program is missing. If this sub-program is provided in a runtime system, and its name is listed in the file specified by the optional supplied-batch-module and supplied-cics-module in the Cataloger option file, Workbench will not report "missing" for these programs.

For example:

The Cataloger option file configuration is shown in Listing 3‑1.

Listing 3‑1 Cataloger Option File

%% Options for cataloguing the system

call-hints="../param/call-hints.desc".

supplied-batch-module="../param/supplied-batch-module".

supplied-cics-module="../param/supplied-cics-module".

job-card-optional.

 

In file "../param/supplied-batch-module", the configuration is:

#Comment line

ILBOABN0

CEE3ABD

In file "param/supplied-cics-module", the configuration is:

#Comment line

CICS001

CICS002

ILBOABN0 and CEE3ABD are provided in Batch runtime systems. CICS001 and CICS002 are provided in CICS runtime systems. Workbench will not report "missing" for these programs.

Embedded CICS

References

The COBOL parser uses a sub-parser to parse embedded EXEC CICS statements (commands). The parser accepts the language defined in IBM CICS Application Programming Reference Version 3 Release 1 (document number SC34-6434-05).

SQL

References

The same parser is used in standalone mode to parse SQL DDL files and as a sub-parser to parse EXEC SQL code embedded in COBOL programs. It is based on the language specifications in IBM DB2 Version 9.1 for z/OS Application Programming and SQL Guide (document number SC18-9841-00) and IBM DB2 Version 9.1 for z/OS SQL Reference (document number SC18-9854-00).

JCL

References

The JCL parser is based on the language specification in IBM z/OS MVS JCL Reference (document number SA22-7597-09).

General Information

Sub-Files

The parser processes various forms of sub-files and file inclusion directives (EXEC [PROC], INCLUDE, SYSIN, SYSTSIN…) and searches the asset for sub-files as directed in the System Description File. Since the cataloger and other tools such as the JCL translator need to have a complete understanding of all of the steps in all of the JCL scripts that they handle, it is very important that all the referenced sub-files be present in the asset and that file types and search paths be set so that the correct sub-file is found for every reference. The cataloger will report missing sub-files as severe anomalies, since they prevent the correct analysis and translation of the whole affected JCL(s). Translation should not be attempted until all such anomalies have disappeared.

There are two types of SYSIN/SYSTSIN files:

•

SYSINs (or SYSTSINs, or other "command files") for utility programs: in many cases, these files contain information which is of interest for the cataloger (e.g. the name of the program launched by the DB2 launcher IEKJFT01) or for the translator (e.g. operations performed by IDCAMS, or the sort script for DFSORT). In consequence, they must be present in the asset before cataloging (with type JCL-Sysin, see Type clause).

•	SYSINs (or SYSTSINs) for applicative (COBOL) programs: these files will be handled like standard data files and do not need to be present in the asset processed by Tuxedo ART Workbench.

All referenced PROC and INCLUDE sub-files need to be present in the asset.

Note that the parser also handles in-stream PROCs (delimited by PROC and PEND cards) and SYSINs/SYSTSINs (DD *), but of course these are never missing.

JCL Syntax

A JCL job is a sequence of steps, with or without execution conditions. It must begin with a JOB card, except if the job-card-optional option is given in the cataloger option file.

All JCL, JES2 and JES3 statements are parsed. The JCL must be directly executable by JES2. The parser performs JES variable substitution. Variables which are not defined locally in the JCL may be set using the JCL-globals option of the system description file, see Special Options.

Comment cards (starting with "//*") are recognized as such and retained for translation.

The parser recognizes all kinds of JCL cards. It handles overrides and refbacks in PROCs, but only for DD cards. It also handles continuation cards.

Restrictions

Using a Sub-file as Both a PROC and an INCLUDE File

In certain conditions, the same sub-file can be used both as a PROC file and as an INCLUDE file. However, the translation to target files is different in each case, so it is necessary to duplicate the file(s) in question, so that one copy is used and translated as a PROC and the other as an INCLUDE file. To achieve this, the two copies must, of course, be placed in separate directories, and the search paths must be set up so that the JCL which use these files as PROCs find the PROC version first and those which use them as INCLUDEs find the INCLUDE version first (it is not possible that the same JCL uses the same file as both a PROC and an INCLUDE).

Using One JOB Card Per JCL

Only one JOB card per JCL is allowed. If you have files with more than one JOB card, you must split them before running the cataloger. Make sure that the job name in the JOB card and the (simple) file name match.

Standard Utility Commands Parsed

The JCL parser fetches (when not in-stream) and parses the contents of command files (SYSIN, SYSTSIN, etc.) for various standard utilities. The current list of such utilities is:

•

IEBUPDTE

•

IEBGENER

•

IEBCOMPR

•

IEBCOPY

•

IEHPROGM

•

IEBDG

•

IEHLIST

•

IEBMOVE

•

IEBPTPCH

•

IDCAMS

•

DFHSORT

•

ICETOOL

•

IEH-MOVE

•

IEKJFT01

•

DSNUTILB

•

DSNTIAUL.

Note:

The support for some of these utilities, i.e., the ability of the parser to handle their command language, may be partial.

BMS screen definition

The BMS parser handles the BMS language as defined in the following documents:

•	Chapter BMS macros in appendix Detailed reference information for the CICS API commands of the IBM book CICS Application Programming Reference (document number SC34-6434-05 for CICS V3R1);

•	Chapter Creating the map in Part 6, Basic Mapping Support (BMS) of the IBM book CICS Application Programming Guide (document number SC34-6433-04 for CICS V3R1).

The parser will accept all correct BMS definitions. It will also report the most obvious syntax errors, but it is not meant to recognize all such errors.

CICS Configuration

The CICS configuration parser handles the CICS resource definition language as read by batch utility DFHCSDUP (see IBM CICS Transaction Server for z/OS Resource Definition Guide Version 3 Release 2, document number SC34-6815-00). The following commands are recognized: DELETE ALL, ADD, REMOVE and DEFINE. All resource types and attributes are recognized but only a few are really exploited in the parser and extractor.

Description of the Configuration Files

System Description File

The system description file describes the location, type and possible dependencies of all the source files in the asset to process. As such, it is the key by which not only the Cataloger, but also all of Tuxedo ART Workbench tools, can access the source files and the corresponding components. The system description file also specifies a number of parameters which influence parsing.

General Structure

Listing 3‑2 System Description File Structure

Sys-desc-file ::= “system” system-name “root” system-root-path

global-options special-options

directories

 

Notes:

The format of the file is basically free, lines can be as long as desired. Comments start with the percent character and end at the end of the line.

Notes:

The format of symbols (names) can include the following characters [A-Z][a-z][0-9][*-_]. Symbols may start with digits but at least one letter is required. Keywords and symbols (names) are case-insensitive; strings are of course case-sensitive.

system-name

The first element in the system description file is a symbol giving the name of the asset. This name can be freely chosen, since it is used only by Tuxedo ART Workbench tools for reference. The names of some files and directories produced by Tuxedo ART Workbench tools also contain this name.

system-root-path

The second element is a string giving the path of the directory which contains all component source files on the Linux migration platform. This directory can be located anywhere convenient on the file system. The path can be given either in absolute form (starting with the slash character) or in relative form. In the latter case, the path is relative to the directory containing the system description file itself (usually located in some “param” directory besides the “source” directory containing the source files, but Tuxedo ART Workbench tools accept any configuration described here).

Global Options

The elements in this clause specify various settings influencing the parsing, cataloging and, generally speaking, handling of component source files. The generic syntax for this clause is:

Listing 3‑3 System Description File Global Options

( “options” | “global-options” ) opt-name-1 “=” opt-value-1 “,”
opt-name-2 “=” opt-value-2 “,” ... “.”

 

The value of each option can be an integer number, a symbol, a string or a Boolean indicator. (The following are accepted as Boolean indicators:

•	Nothing (meaning true).

•	The string “true” or “TRUE”.

•	The symbol true (case-insensitive).

•	The string “false” or “FALSE”.

•	The symbol false.

Option names and option values are case-insensitive, except for strings. In general, these settings can apply globally on the whole asset and/or be overridden locally for a specific directory (see below).

The various possible options accepted here are listed in the following table:

 

Table 3‑1 Global Options

Option Name	Type	Local	Description
Catalog, catalog-option	String	No	Path of the Cataloger options file (see below). This path can be given in absolute form or in relative form; in the latter case, the path is relative to the directory containing the system description file itself. Note that this clause is optional: if it is not given, then the Cataloger will not attempt to read an option file and will use default values for all options.
Cobol-Right-Margin	integer (> 60)	Yes	Column for start of Area C in COBOL programs. The default value is 66, suitable for fixed-format programs with columns 1-6 and 72-80 removed.
Cobol-Left-Margin	integer (< 12)	Yes	Comment column in COBOL programs. The default value is 1, suitable for fixed-format programs with columns 1-6 and 72-80 removed.
Remove-Cobol-Reserved-Word	String	Yes	This option is used to tell the COBOL parser that some keyword it considers as reserved (in the IBM COBOL LE dialect) is in fact not reserved in the actual dialect used for this asset or directory. This option may be given several times for the same system or directory. For each program, the list of such keywords to consider as non-reserved is formed by accumulating all values given in all local (directory) or global (system) options of this name. Default value is of course the empty list.
No-END-Xxx-Warnings, No-END-Xxx	Boolean	Yes	If true, don't complain loudly when some statement-containing construct is not closed otherwise than with the appropriate END-xxx keyword. Default value is false, i.e. complain. Note that you can associate a “false” value with this option, which would reverse its meaning.
Yes-End-Xxx-Warnings, Yes-End-Xxx, END-Xxx-Warnings	Boolean	Yes	The reverse of previous option, No-END-Xxx-Warnings. You may not use both options on the same directory or on the global level, but you may have an option at the global level and a different option on some directory(ies). Note that you can associate a “false” value with this option, which would reverse its meaning.
SQL-Schema, Default-SQL-Schema	string or symbol	Yes	Name of the schema to use in SQL code when not explicitly given. This applies to standalone SQL code (DDL, files of type SQL-Script) and to SQL code embedded in COBOL programs.
SQL-Use-Reversed-Delimiters, SQL-Reversed-Delimiters	Boolean	Yes	if true, SQL code in this directory or system uses double quotes for strings and single quotes for delimited identifiers. Default behavior (for value false) is the reverse, single quotes for strings and double quotes for delimited identifiers.
SQL-No-Keyword-Is-Reserved, SQL-Keywords-Not-Reserved	Boolean	Yes	If true, no keyword in the SQL code in this directory or system will be considered as reserved (use for DB2 version 8 or earlier). Default behavior (for value false) is to consider all keywords as reserved, and hence complain if they are used for other purposes (like DB2 version 9).
SQL-Right-Margin	Integer (> 0)	Yes	The “end-of-line” column for SQL-Script files (only—this does not apply to SQL code embedded in COBOL programs). Set it to 66 for fixed-format files with columns 1-6 and 72-80 removed (COBOL). Default value is “infinite”, which is suitable for free-format files. Note that left margin is always 1, so you must physically remove columns 1-6 of fixed format files if they are to be ignored.
jclz-launcher-spec-file, jclz-launcher-specs-file	string	Yes	Path of the JCL-launcher specification file to use for this system or directory; see JCL-Launcher Specification Files for more information on the contents and use of such files. This path can be given in absolute form or in relative form; in the latter case, the path is relative to the directory containing the system description file itself.

Special Options

Special options are clauses which cannot be integrated in the previous global options mechanism, mostly for syntactic reasons (values are lists). They can appear before or after global options, but not in-between. They are all of the following syntactic form:

Listing 3‑4 System Description File Special Options

opt-name “=” opt-value “.”

 

The equal sign and trailing period are mandatory. When a value is a list, the items in the list must be separated by commas. The special options are described in the following table:

 

Table 3‑2 Special Options

Option Name	Type	Description
minimum-free-ram-percent	Integer (> 0 and < 100)	The fraction of physical memory which should remain free and available to other processes during execution of the Cataloger and all of the various Tuxedo ART Workbench tools. In general, these tools consume more and more memory, depending on the number of components they process. When this limit is reached, the tool stops and restarts execution; incremental execution ensures that the components already processed are not re-processed, so that eventually, all the required work is achieved.
JCL-globals	List of pairs var-name = var-value, separated by commas	Var-name is a symbol (or string interpreted as a symbol) and var-value is a string. When parsing a JCL script, the parser simulates the JCL-variable substitution process performed by JES2. The name-value pairs given here are used to substitute global variables (as opposed to parameters, etc.). The parser reports an error when it cannot find a suitable value for some variable.
strict-jcl-libraries	None (Boolean flag)	The presence of this option influences how SYSIN/SYSTSIN files referenced by a JCL are searched in the whole system. See chapter Sub-file search operation below.
Dbms-version	String or symbol	Version of the DB2 relational DBMS used on the source platform. This is used by the RDBMS tool (q.v.).
use-file-catalog	None (Boolean flag)	The presence of this option introduces the "catalog" and "volume" information for the files or DD. Without it, the "catalog" and "volume" are not introduced. For detail descriptions please refer to JCL translator and File Convertor. It also affects FILE-TO-FILE, FILE-TO-ORACLE and FILE-TO Db2/luw (udb) convertors.
enable-buffer-converter	None (Boolean flag)	The presence of this option enable the "buffer converter" feature: generate two COBOL programs for converting the data between z/OS format and UNIX/Linux format; both the source data and target data are stored in a buffer. Without the setting of this option, no related COBOL program is generated. Please refer to the FILE-TO-FILE for details.
enable-reverse-converter	None (Boolean flag)	The presence of this option enable the "reverse converter" feature: generate one COBOL programs for converting the file from UNIX/Linux record sequential file to z/OS sequential file, and also generate one script file to call the COBOL program. Without the setting of this option, no related COBOL program and script file is generated. Please refer to the FILE-TO-FILE for details.
default-sequential-mode	String or symbol	This option indicate the default behavior of migrating sequential file. default-sequential-mode can be set to record or line. • "line" indicates migrating to line sequential file. It is the default value. • "record" indicates keeping record sequential file.
dont-parse-exec-sql	None (Boolean flag)	This option disables the EXEC SQL statement parsing (but cannot disable EXEC SQL INCLUDE statement parsing).
microfocus-cobol (or mf-cobol)	None (Boolean flag)	This option enables the Micro Focus COBOL extension, for example support of COMP-6, support of statement STOP RUN RETURNING 1
use-fileschema-as-dbschema	None (Boolean flag)	This option applies the configured "file schema" as "database schema" to the generated table and other database objects. If not set, no database schema is specified for the generated database objects; this is the default behavior. This is only valid for File-to-Oracle converter.

Directories

The main component of the system description file is the list of directory clauses, which specifies the location of the various source files for the given asset, their type and their relation with each other. Each such clause has the following syntax:

Listing 3‑5 System Description File Directories

“directory” directory-path

type-clause file-clause logical-name-clause options-clause

libraries-clause sql-libraries-clause

subdirectories “.”

 

The (mandatory) directory path must come first. The optional subdirectories, if any, must come last. The other clauses may come in any order. Of these clauses, only the type clause and the file clause are mandatory, the others are optional.

Directory-path

This is a string giving the path (location) of the directory relative to the root directory of the system (see system-root-path). Although it is not an absolute requirement, it is strongly advised that all the directories of the same system are physical descendants of the system root directory. (A simple and readable way to achieve this is that no directory path contains the “../” upward-going name). Different directories must have different paths.

Type clause

“type” directory-type

The type clause specifies the type of the directory, that is the type of the source files (components) it contains. The type is given as a case-insensitive symbol. Only the following types are accepted

 

Table 3‑3 Valid Directory Type Clauses

Type	Description
Cobol-Batch	Main COBOL programs used in batch operation (referenced by JCLs), possibly containing EXEC SQL code (DML, data manipulation language).
Cobol-TPR	Main COBOL programs used in TP operation (referenced by transactions and CICS XCTL commands), possibly containing EXEC SQL and EXEC CICS code.
COBOL-Library, COBOL-Copy	COBOL copy files (copy books), to be included in main program files.
SQL-Script	Standalone SQL code containing essentially DDL (data definition language) statements. The set of SQL-Script files collectively defines the database schema(s) used in the system.
JCL	Main JCL files, defining one or more JCL jobs.
JCL-Lib	JCL sub-files, either defining procedures invoked by EXEC or containing statements invoked by INCLUDE
JCL-Sysin	SYSIN/SYSTSIN files used by utility programs or program launchers in JCL scripts. Not all SYSIN files are required by the parser/Cataloger, see more details in Tuxedo ART Workbench JCL Translator Reference Manual (section Description of Input Components).
BMS	BMS screen definitions (in source form)
RDO	CICS system definition files (CSD) as used by RDO to configure CICS, see IBM’s CICS Resource Definition Guide.

files Clause

“files” file-specs “,” ...

The file-specs are strings designating one or more files in a directory. The string identifies the inclusive members of the asset and excludes the others. The simplest form of file-spec is a complete file name such as toto.cbl. No indication of directory should be given, the designated files must be located directly in the directory in question. To avoid the task of explicitly listing all components in the directory, you can also use shell-like regular expressions such as *.cbl or [A-F][D-Z]*.jcl.

Note:

It is important that all files designated by the system description file, that is all source files in the asset, have a file extension rather than just a bare file name. The extension can be chosen freely — although we advise the use of “standard” extensions such as cbl for COBOL programs, cpy for COBOL copy files and jcl for main JCL files—but must be present.

logical-name clause

logical-name lname

This clause is to be used on directories of type JCL-Sysin, together with the special option strict-jcl-libraries, see above. Together, they enable the Strict JCL-Sysin Search mode. lname is a string of the form “A.B.C”, naming a library (PDS) of JCL SYSIN/SYSTSIN files on the source platfom. It is assumed that all the files in the directory bearing this clause belong to this library.

If the special option strict-jcl-libraries is not set, the logical-name clause is ignored.

options-clause

Listing 3‑6 options-clause

“options” opt-name-1 “=” opt-value-1 “,”
opt-name-2 “=” opt-value-2 “,” ...

 

Syntactically, the directory-specific options clause is similar to the system-wide Global Options clause, except for the trailing period. Semantically, the listed options and values have the same effect as the global options, but only locally on the files contained in the directory (they override global options with the same name). The same options as the ones marked yes in the local? column of the global-option table apply to directories, provided that they are relevant for the type of source files in the directory. For instance, the option cobol-right-margin is relevant for directories of type COBOL-Batch and COBOL-TPR, but not for type JCL or SQL-Script.

In addition, there exists one directory-specific option: “Right-margin” for directories of type JCL. The value is an integer number which specifies the “end-of-line” column for JCL files. The default value is 72, which is appropriate for most cases of IBM JCL source files.

libraries-clause

“libraries” directory-path “,” ...

The libraries clause specifies an ordered search path of (other) directories in the asset. Whenever the Cataloger finds in a source file a reference to another component it searches, from first to last, the list of directories given in this clause, until it finds a component (source file) whose name and type matches those of the reference (see more details in section Sub-file search operation below). It is used both for compile and parse-time references, such as a COBOL program referencing a copy file or a JCL file referencing a PROC, and for run-time references, such as a COBOL program calling a COBOL subprogram or a JCL job invoking a COBOL program. This way, it is possible to simulate the effects of various source-platform library-search operations, such as SYSLIB or COPYLIB for COBOL compilation, JOBLIB and STEPLIB for JCL preparation and execution, etc.

Note:

The directory paths are strings which must match those given in the definition of the referenced directories. However, the definition of a directory may be placed before or after any of its references.

sql-libraries-clause

“sql-libraries” directory-path “,” ...

The SQL-libraries clause plays the same role as the libraries clause for resolving EXEC SQL INCLUDE directives in COBOL programs. When it is omitted, the resolution of such references uses the same search path as the normal libraries clause, but sometimes it is necessary to use a different order for normal COPY directives and SQL INCLUDE directives.

Example of System Description File

Listing 3‑7 Example of System Description File

system BNL root "../source"

 

options catalog = "./options-catalog.desc",

no-end-xxx-warnings,

cobol-left-margin = 7,

cobol-right-margin = 72,

SQL-Schema = DB2A1,

SQL-Server = BNL.

minimum-free-ram-percent = 20.

 

%

% Copies

%

directory "COPY" type Cobol-Library files "*.cpy".

directory "INCL" type Cobol-Library files "*.cpy".

directory "IBMCPY" type Cobol-Library files "*.cpy".

 

%

% Sysin

%

directory "SYSIN" type JCL-SYSIN files "*.sysin".

directory "SYSINCDB" type JCL-SYSIN files "*.sysin".

 

%

% DDL

%

directory "DDL" type SQL-SCRIPT

files "*.sql"

options SQL-Schema = "DB2A0".

 

%

% Batch

%

directory "Batch" type COBOL-Batch files "*.batch"

libraries "COPY", "INCL", "IBMCPY"

options cobol-right-margin=73.

 

%

% TPR

%

directory "TPR" type COBOL-TPR files "*.tpr"

libraries "COPY", "INCL", "IBMCPY".

 

%

% JCL

%

directory "JCL" type JCL files "*.jcl"

libraries "SYSINCDB", "SYSIN".

 

%

% CICS

%

directory "MAPS" type BMS files "*.bms".

directory "CICS" type RDO files "*.rdo".

 

This system-description file is for an asset named BNL, for example the name of a customer or a standalone application in a larger system. The location and name of this file are not constrained, but conventionally, the complete path should be something like: /.../BNL/param/system.desc. Given this assumption, and since the path for the system root directory given in this file is relative (../source), the absolute path for the root directory is /.../BNL/source. Similarly, the path for the Cataloger options file is given as ./options-catalog, so its absolute path is /.../BNL/param/options-catalog. The global options call for the following comments:

•	The no-end-xxx-warnings option enables the lenient mode of parsing implicitly-closed COBOL constructs.

•

The cobol-left-margin and cobol-right-margin values are set for untransformed, IBM-like fixed-format programs with left-side numbering column and right-side comment column (area C). Note that, while this format causes no trouble for the COBOL parser, the correct operation of the COBOL converter cannot be guaranteed.

The naming and organization of the various directories is quite standard, with source files in the asset being identified only with their file extensions. The only unusual feature here is the special cobol-right-margin value for directory “Batch”.

JCL-Launcher Specification Files

Purpose

Most IBM source assets contain JCL steps invoking program launchers, i.e. utility programs that launch applicative programs. Many of these launchers, such as the DB2 launcher IEKJFT01, are recognized directly by the JCL parser and analyzer in Tuxedo ART Workbench cataloger. However, in many cases, some of these launchers are installation-specific and require specific handling. Fortunately, most of them use the generic JCL-invocation mechanism and syntax (EXEC PGM card) and the relevant launch information is contained in the PARM value.

The purpose of the JCL-launcher specification file are to describe the launchers used in a given asset, so that the cataloger and the JCL translator can extract relevant information such as the name of the real program to launch. The specification is based on the fact that, in most cases, the PARM value is split into individual parameters by some separator character (not always the standard JCL separator, the comma), and that the parameters which give the program name, the PSB name, the PLAN name, etc., have a well-defined position in the sequence.

Syntax

A JCL-launcher specification file is a free-format text file with the following syntax, where all the keywords and symbols are case-insensitive:

Listing 3‑8 JCL-launcher Syntax

LAUNCHER <Launcher name>

[<option-name> = <option-value> [,

... ]

]

END

...

 

Option List

For the last three options, there is no default value: if the option is absent, then the corresponding information is simply not available.

•	Separator: Character used as a field separator, must be a one-character string. Default is the comma character ",".

•	IndexProg: index of program name, must be an integer. This is the only mandatory option for a given launcher.

•	IndexPSB: index of PSB name, must be an integer.

•	IndexPlan: index of plan name, must be an integer.

•	IndexParm: index of parameter name, must be an integer.

Usage and Default Value

The local jclz-launcher-spec-file option attached to a directory, when present, overrides the global one, as usual. When no launcher specification file is specified either for a given directory or the whole system, then the default value is as if we used the following file:

Listing 3‑9 Default Launcher Value

LAUNCHER DFSRRC00

IndexProg : 2,

IndexPSB : 3

END

LAUNCHER DB2BATCH

IndexProg : 2,

IndexPSB : 3

END

 

Description of the Output Files

Catalog Reports

Format and Location

All these reports are produced in CSV format, with fields delimited by a single semi-colon.

They are generated in the $SYSROOT/Reports-${SYSNAME} directory, where $SYSROOT is the root directory for the current asset and $SYSNAME is the asset name, both as defined in the System Description File.

The name of each report also contains $SYSNAME, to avoid any confusion.

Field Definitions

The following field definitions are used in several reports:

Path (string)

The identification of the (main) source file defining the entity in question, as a path relative to the root of the "system" given in the system description file.

Status (enumeration: CORRECT, UNUSED or MISSING)

CORRECT

The component is present in the asset and at least one reference to it has been found in one or more other components, i.e. the component is used.

UNUSED

The component is present in the asset but no reference to it could be found in any other component;

MISSING

The component is not present in the asset and at least one reference to it has been found.

Note:

UNUSED and MISSING are to be considered as (inter-component) anomalies.

Anomaly level (enumeration: FATAL, ERROR, WARNING, NOTICE, OK)

This is the maximum level of anomalies detected on the component in question during internal analysis.

FATAL

Irrecoverable errors such as syntax errors found. The results of the analysis are incomplete and the component or asset is unsuitable for conversion.

ERROR

Recoverable errors such as undeclared variables found. The results of the analysis may be inaccurate and the component or asset is unsuitable for conversion.

WARNING

Situations which can cause problems (inaccuracies) during analysis or after conversion have been found, but the component is suitable for conversion.

NOTICE

A remarkable situation was detected, but it causes no harm.

OK

No anomaly found.

Note:

When the component is MISSING, this field is replaced by indications describing the cause for the component to be absent from the asset (SYSTEM, CORRECT or PROBLEM), depending on information supplied entirely by the user.

MISSING

When a component is MISSING, this field is replaced by indicators describing the cause for the component to be absent from the asset (SYSTEM, CORRECT or PROBLEM), depending on information supplied entirely by the user.

report-${SYSNAME}-COBOL-Programs

This report lists all the (COBOL) programs defined or referenced in the asset. It accounts for the -Cobol-Batch and -Cobol-TPR reports.

The following fields are contained in the report:

 

Table 3‑4 COBOL Report Fields

Field	Type	Description
Name	symbol	Name of the program as defined by the (envelope) file name.
Path	string	See Field Definitions. This is the path of the (main) source file defining the program. Note: Tuxedo ART Workbench does not currently handle multiple programs in the same file.
Type	enum: BATCH, TP or SUB	Type of program as defined by its classification in the system description file (Cobol-Batch, Cobol-TPR).
Loc	integer	Total number of lines in the program after copy expansion.
NPar	integer	Number of paragraphs in the Procedure Division.
Status	enum	See Field Definitions.
Anomaly level	enum	See Field Definitions.

The following fields are empty (undefined) when the component is MISSING:

•

Path

•

LOC

•

Npar

In addition, for components of type SUB, the name may be that of an entry point in a subprogram, rather than the name of the subprogram itself It is the name as referenced in a CALL and the cataloger can’t determine whether it designates an entry point or a complete subprogram.

report-${SYSNAME}-COBOL-Copy

This report lists all the COBOL copy files (copybooks) contained or referenced in the asset. The following fields are contained in the report:

 

Table 3‑5 COBOL Copy Report Fields

Field	Type	Description
Name	string	Logical name of the copy file, as defined by the (envelope) file name and possibly by the logical-name clause of the containing directory in the system description file.
Path	string	See Field Definitions.
Loc	integer	Number of lines in the copy file.
Status	enum	See Field Definitions.

The following fields are empty (undefined) when the component is MISSING:

•

Path

•

LOC

report-${SYSNAME}-JCL-Files

For JCLs, we separate between reports on (main) source files and reports on jobs, because we handle multiple jobs per file. For (main) source files, the following fields are contained in the report:

 

Table 3‑6 JCL Source File Report Fields

Field	Type	Description
Path	string	See Field Definitions.
Loc	integer	Total number of lines in the JCL file after expansion of PROCs, INCLUDEs and SYSINs.
NJob	enum	Number of jobs defined in this file.
Anomaly level	enum	See Field Definitions. It is at least as high as the maximum anomaly level in all the contained jobs, and may be higher in case of syntax errors.

A JCL source file is never MISSING, only JCL jobs can be missing.

report-${SYSNAME}-JCL-Sub-Files

This report describes JCL sub-files required for the analysis of main files: PROCs, INCLUDEs and some SYSIN files. The following fields are contained in the report:

 

Table 3‑7 JCL sub-file Report Fields

Field	Type	Description
Name	string	Name of the sub-file, as referenced from main files.
Path	string	See Field Definitions.
Type	enum: PROC, INCLUDE or SYSIN	Type of sub-file, as defined by its classification in the system description file and the construct by which it is referenced in the main files.
Loc	integer	Number of lines in the sub-file.
Status	enum	See Field Definitions.

The following fields are empty (undefined) when the component is MISSING:

•

Path

•

LOC

report-${SYSNAME}-JCL-Jobs

This report lists all JCL jobs defined or referenced in the asset. The following fields are contained in the report:

 

Table 3‑8 JCL Jobs Report Fields

Field	Type	Description
Name	string	Name of the job, as defined in the JOB card.
Path	string	See Field Definitions. This is the path of the (main) file defining the job.
Loc	integer	Number of lines in the job itself (from the JOB card to the ENDJOB card) after expansion of sub-files.
NStep	integer	Number of steps defined in this job
Status	enum	See Field Definitions.
Anomaly level	enum	See Field Definitions. This is the anomaly level of the job itself, and generally does not take into account syntax errors (because the latter prevent the analysis of the job).

report-${SYSNAME}-Screens

This report lists all BMS screens defined or referenced in the asset. The following fields are contained in the report:

 

Table 3‑9 Screens Report Fields

Field	Type	Description
Name	string	Name of the screen, in the form mapset-name.map-name.
Path	string	See Field Definitions. This is the path of the file defining the screen.
Line	integer	The number of the line in the source file at which the screen definition begins (the line containing the DFHMDI macro).
NField	integer	Number of fields defined in this screen
Status	enum	See Field Definitions.
Anomaly level	enum	See Field Definitions. In fact, this is the anomaly level of the complete source file; see the anomaly report to see whether the anomalies really apply to this screen definition.

When the screen is MISSING, the following fields are empty:

•

Path

•

Line

•

NField

•	Anomaly level

report-${SYSNAME}-SQL-Tables

This report lists all SQL tables defined or referenced in the asset. The following fields are contained in the report:

 

Table 3‑10 SQL Table Report Fields

Field	Type	Description
Name	symbol	Name of the SQL table, as defined in the CREATE TABLE statement.
Schema	symbol	Name of the schema containing the table definition, or of its owner.
Path	string	See Field Definitions. This is the path of the SQL-script file defining the table.
Line	integer	Number of the line in this source file at which the definition of the table begins.
NCol	integer	Number of columns defined in the table.
Status	enum	See Field Definitions.
Comment	string	Comment, if any, associated with the table.

When the table is MISSING, the following fields are empty:

•

Path

•

Line

•

Ncol

•

Comment.

report-${SYSNAME}-SQL-Views

This report lists all SQL views defined in the asset. The following fields are contained in the report:

 

Table 3‑11 SQL Views Report Fields

Field	Type	Description
Name	symbol	Name of the SQL view, as defined in the CREATE VIEW statement.
Schema	symbol	Name of the schema containing the view definition, or of its owner.
Path	string	See Field Definitions. This is the path of the file defining the view.
Line	integer	Number of the line in this source file at which the definition of the view begins.
NCol	integer	Number of columns defined in the view.
Status	enum	See Field Definitions. Note: A view is never MISSING, because references to views are indistinguishable from references to tables. So, when a reference to a table-or-view links to no definition of any kind, the Cataloger creates a missing table, not a missing view.
Comment	string	Comment, if any, associated with the table.

report-${SYSNAME}-Transactions

This report lists all CICS transactions defined in RDO files in the asset. The following fields are contained in the report.

 

Table 3‑12 CICS Transaction Report Fields

Field	Type	Description
Name	symbol	Name of the transaction, as defined in the DEFINE TRANSACTION statement.
Group	symbol	Name of the group containing the transaction definition.
Path	string	See Field Definitions. This is the path of the file defining the transaction.
Line	integer	Number of the line in this source file at which the definition of the transaction begins.

When a transaction is referenced in the asset (e.g. in a RETURN TRANSID statement) and it is not defined in an RDO file, it is listed in this report with empty Path and Line fields.

report-${SYSNAME}-Anomalies

This report lists all anomalies found in all components of the asset. The following fields are contained in the report:

 

Table 3‑13 Anomaly Report Fields

Field	Type	Description
Path	string	See Field Definitions. This is the path of the main file defining the component in which the anomaly occurs.
Sub-Path	string	See Field Definitions. If the real location of the error (statement or other construct) is inside some sub-file (COBOL copy file, JCL PROC file, etc.), this is the path of this sub-file, otherwise this field is empty.
Line	integer	Number of the line in the real source file at which the anomaly occurs: the sub-file if previous field is not empty, otherwise the main file.
Sub-Line	integer	If the anomaly occurs in some sub-file, this field contains the number of the line in the main source file at which the sub-file is included, otherwise it is empty.
Severity	enum: FATAL, ERROR, WARNING, NOTICE	The severity of the anomaly, from FATAL as the highest severity to NOTICE with the lowest severity.
Category	enum: SYNTAX, LINKAGE, ANALYSIS, MISC	Defines the category to which the anomaly belongs: • SYNTAX is for parse errors and all errors related to syntax; • LINKAGE is for errors related to links between a reference to some construct and the corresponding definition, such as undeclared variables; • ANALYSIS is for anomalies related to constructs which do not allow the Cataloger to perform an accurate analysis of the component, such as dynamic calls; • MISC is for all others.
Tag	Symbol	A synthetic but significant name identifying the precise kind of anomaly. In fact, the possible values are a finite enumeration, but too numerous to be all listed here.
Description	string	A precise and specific description of the anomaly, possibly including references to the offending source construct, for instance: "SQL-TABLE variable is not defined: LVDSYS00".

Description of Other Output Files

The visible result of the Cataloger is the set of cataloging reports described above. These reports are far from the only or even the most important output. This section briefly describes the other result files; these are binary files in a proprietary format, called Persistent Object Base (POB). These files are not suitable for human processing or processing by traditional text-based tools; they are intended for use by the Cataloger itself or with other tools in Tuxedo ART Workbench.

POB Files for ASTs

During the parsing phase (see The Cataloger Process), for each parsable-component source file A/B/C/file.ext in the system, the Cataloger produces a POB file named A/B/C/pob/file.ext.pob (the pob directory is created on demand by the Cataloger). This file contains the result of the parsing, namely the Abstract Syntax Tree (AST) of the component. It is re-read by the analysis phase of the Cataloger and by other Tuxedo ART Workbench tools such as the COBOL converter or JCL translator.

CDM Files for COBOL Programs and Copy Files

CDM (Common Data Model) files contain additional information about COBOL variables (so-called data description entries). For each COBOL program A/B/C/prog.cbl in the system, the Cataloger produces a CDM file named A/B/C/pob/prog.cbl.cdm to store information about variables defined in the main source file. In addition, for each COBOL copy file D/E/file.cpy which defines variables (as opposed to copy files containing Procedure Division code, for instance), the Cataloger produces a CDM file named D/E/pob/file.cpy.cdm to store information about those variables; this CDM file is shared by all programs which include this copy file.

In some circumstances, the information about a variable apparently defined in a copy file cannot actually be shared by all programs which include this copy file; for instance, this is the case for copy files included with REPLACING directives, or files defining only parts of a complete structure (01-level record). In these cases, the CDM information is stored in the programs CDM files rather than that of the copy file itself. When no shared CDM information at all can be associated with the copy file, the CDM file is not produced.

The Cataloger Symtab and Other Miscellaneous Files

•	$SYSROOT/symtab-$SYSNAME.pob: this file houses the symbol table created by the Cataloger (during the analysis phase) and contains summary information for all the various components in the asset. This information is used to compute cross-reference information between these components.

•	$SYSROOT/Cobol-dump-map.pob: contains information (so-called dump descriptors) necessary to read and write Abstract Syntax Tree (AST) pobs for COBOL programs. Do not delete this file or you will not be able to re-read your existing COBOL pobs.

•	$SYSROOT/sql-system-$SYSNAME.pob, $SYSROOT/sql-system-$SYSNAME-State-ments.pob: contains various internal forms of the complete SQL schema of the asset, derived from the union of all DDL files (SQL-Script files). These files are required for parsing (and linking) COBOL programs.

Default File-Convertor Configuration Files

During cataloging process, Tuxedo ART Workbench parses both JCL files and Batch COBOL programs to collect dataset related metadata, puts them into param/dynamic-config/dataset directory, and generates file convertor configuration files (for example, Datamap-<schema>.re and mapper-<schema>.re files under param/file directory).

After that, you can use the generated configuration files to do file conversion or make some modifications when necessary to do file conversion.

Detailed Processing

Processing Phases

As described in The Cataloger Process, the operation is logically divided into four phases: parsing, analysis, post-analysis and report generation (see below for more details). Depending on the needs of the project and the migration-platform configuration, these phases can be executed sequentially or concurrently (parsing phase only), in a single run or incrementally.

Depending on the needs of the project and the migration-platform configuration, these phases can be executed sequentially or concurrently (parsing phase only), in a single run or incrementally. There are three basic Tuxedo ART Workbench commands invoking the Cataloger:

•	preparse and its variant preparse-files: runs the parsing phase only.

This is the only phase which can be run concurrently, at least after the SQL-System files have been generated. This is also the only phase for which you can request the processing of one or more specific components; otherwise, the Cataloger determines itself which components it must process (see Changes in the Asset: Incremental Operation). In this phase, the Cataloger reads the component source files, any included sub-files and the SQL-System files, and produces (only) the POB-files for the processed components.

•	analyze: runs the analysis phase: for each component, the pob-file is re-read and the most significant constructs in the component are translated into a smaller summary information stored in the cataloger symbol table (symtab).

This phase cannot run in concurrent mode because the Symtab does not support concurrent accesses. In this phase, the Cataloger reads the component POB files (parsing them on demand if necessary) and updates (reads and writes) the Symtab file.

•	fast-final: runs both the post-analysis and report generation phases.

•	Post-analysis: working with just the symtab and the summary information, the cataloger computes some cross-reference links allowing to label each component as correct, unused or missing.

•	Report generation: the symtab decorated with cross-reference links is traversed and information is printed out for each component.

There is no need to run this phase concurrently, especially since it performs a system-wide operation. In this phase, the Cataloger reads the Symtab file (without trying to update it) and writes the cataloging reports.

There is also a combined command:

•	catalog: runs in sequence the analysis phase (and hence the parsing phase, on demand), the post-analysis phase and the report generation phase.

Note:

For all these commands, the whole configuration information comes from the system description file and the Cataloger option file. Except for preparse-files, the only command-line arguments are the path to the system description file and standard Tuxedo ART Workbench tool arguments.

Command-line Syntax

The Tuxedo ART Workbench Launcher

The Cataloger is designed to be run through the refine command. The refine command is the generic Tuxedo ART Workbench launcher that is used to launch the major Tuxedo ART Workbench tools. The launcher handles various aspects of the operation of these tools, such as execution log management and the incremental and repetitive operations described below (Repetitive and Incremental Operation). The Tuxedo ART Workbench launcher also handles a couple of generic command-line options.

Synopsis

The general form used to invoke an Tuxedo ART Workbench tool using the command line is:

$REFINEDIR/refine command [ launcher-options… ] \

( -s | -system-desc-file ) system-desc-path \

[ command-specific-options-and-arguments… ]

Options

The following options relate to Tuxedo ART Workbench command.

-h, -help, --help

Print out a short description (usage) of the command, and then exits.

–whoami

Prints out the version number and build history of the command, and then exits.

-archi64 / -archi32

Use the executable tool built for the specified architecture. The default is to use the tool for the native architecture of the host machine.

-quiet

Do not print anything in the log except errors (this is currently not obeyed by all tools).

-time

Display timing information at the end of the command execution.

-nolog

Disable log redirection so that the log appears on the terminal and is not captured into a permanent file.

-n, -N, -verbose, -VERBOSE

Prints out a description of which work (phase) needs to be performed on which components but do not actually undertake the work see Changes in the Asset: Incremental Operation.

The following option is technically not a launcher option, but it is accepted (and in fact mandatory) in all of Tuxedo ART Workbench tools:

(-s | -system-desc-file) system-desc-path

Specifies the location of the System Description File. As usual for Unix/Linux commands, the given path can be absolute or relative to the current working directory.

Note:

Many other paths used by many of Tuxedo ART Workbench tools are then derived from the location of this file; this makes it easy to run the same command from different working directories.

In addition, the following option is reserved for future use (presently, it is accepted but otherwise ignored):

(-v | -V | -version) version-string

Generic launcher options

Lastly, the launcher can be invoked without a command, using generic options:

$REFINEDIR/refine (-h | -help)

Prints out a short generic description (usage) of the launcher itself.

$REFINEDIR/refine -print-info-version

Prints out version information about the launcher itself, and more generally about the whole of Tuxedo ART Workbench.

System-Wide Commands

As explained in Processing Phases, system-wide commands are preparse, analyze, fast-final and catalog. They operate globally on the whole asset. The generic command-line syntax for all these commands is:

$REFINEDIR/refine command [ launcher-options… ] \

-s | -system-desc-file ) system-desc-path

There is no specific option for these commands: all configuration information is located in the system description file, the Cataloger option file and possibly the hint files.

The preparse-files Command

Description

Unlike the system-wide cataloging commands described above, which operate globally on the whole asset and decide by themselves which components to process, the preparse-files command allows you to specify yourself which component or components to parse.

The fact that you can specify which components to process makes the preparse-files command suitable for use in a makefile. In addition, it is amenable to concurrent execution, especially if you partition the set of source files into several lists and give each list to a separate process.

Note:

Before parsing any COBOL programs, the Cataloger ensures that the SQL-system POB file is present and up-to-date with respect to all of the SQL DDL files. This may entail building or rebuilding the POB file, which may take some time.

Synopsis

The command line for preparse-files is as follows:

$REFINEDIR/refine preparse-files [ launcher-options… ] \

( -s | -system-desc-file ) system-desc-path \

( source-file-path | ( -f | -file | -file-list-file ) file-of-files )…

Options

The extra options indicate which component source files to process:

source-file-path

Adds to the work-list the component source file designated by this path. The path must be given as relative to the root directory of the system, $SYSROOT, even if the current working directory is different.

(-f | -file | -file-list-file) file-of-files

Adds to the work-list the component source files listed in the file designated by this path. The file-of-files itself may be located anywhere, and its path is either absolute or relative to the current working directory. The component source files listed in this file, must however be given relative to the root directory of the system.

You can provide as many individual components and or files-of-files as you wish. The work-list is built when the command line is analyzed by the Cataloger, and each of its elements is examined in turn:

•	If the given path does not match any actual component source file in the system, an error message is printed out and the Cataloger skips to the next element.

•	If the component identified by the given path is already parsed and its POB file is up-to-date with the source file, no action takes place, and the Cataloger silently skips to the next element.

•	Otherwise, the component is parsed normally (and verbosely, unless the -quiet option is given in the launcher-options) and its POB file is produced.

Component Search Operation

This section describes how the Cataloger uses the libraries and sql-libraries clauses in the system description file to locate the components referenced in a specific construct of the currently processed component. The operation is slightly different depending on whether the reference is:

•	As seen from the source platform

•	A compile-time reference

•	A run-time reference.

Compile-Time References

The compile-time case applies to references to sub-files which are an integral part of the current component, so that, if the sub-file is not found, the component cannot be analyzed and "understood" correctly. For example a COBOL copybook referenced from a COBOL main (program) source file.

JCL sub-files referenced from a JCL job, such as PROC files, INCLUDE files and some SYSIN files are also included because whereas on the MVS platform these sub-files are searched when the JCL job is run, hence it is a run-time reference; on the migration platform, the Cataloger has to resolve these references at parse-time, to make them available to the JCL translator, and hence they qualify as compile-time references. Even SYSIN files are in this case, since they contain information which is needed at parse time, such as the program invoked by some DB2 launcher. In the Cataloger, "compile-time" is equivalent to parsing, and "run-time" is equivalent to post-analysis.

The search starts with a component identified as SRCFIL of a certain type SRCTYP, that is located in some directory SRCDIR and which references a component named TGTFIL of a certain type TGTTYP. The following table describes the various possible combinations:

 

Table 3‑14 Compile-Time References

Source Type (SRCTYP)	Construct	Target Type (TGTTYP)	Search Path
COBOL program (Cobol-Batch, Cobol-TPR)	COPY TGTFIL COPY TGTFIL REPLACING …	COBOL copybook (COBOL-Copy, COBOL-Library)	libraries
COBOL program	EXEC SQL INCLUDE TGTFIL END-EXEC.	COBOL copybook	sql-libraries, or libraries if not specified
JCL job	EXEC TGTFIL, …	JCL PROC (JCL-Select)	libraries
JCL job	INCLUDE TGTFIL…	JCL INCLUDE (JCL-Select)	libraries
JCL job	SYSIN DD A.B.TGTFIL… SYSIN DD A.B.C(TGTFIL)…	JCL SYSIN (JCL-Sysin)	libraries

Normal Sub-File Search

The search algorithm process is as follows:

1.	For each directory SUBDIR listed in the libraries (or sql-libraries, if applicable) clause associated with the definition of SRCDIR in the system description file, in order, locate the definition of SUBDIR in the same file, then:

•	If there is no such definition, complain (this is done once and for all when the Cataloger starts and reads the system description file) and skip to the next element of the list.

•	If the type of SUBDIR is not the TGTTYP appropriate to the reference at hand, skip to the next element of the list.

•	Otherwise (type matches), search the list of component files in the directory (those which match the files clause):

•	If a component file with the base name TGTFIL (and some appropriate extension) exists, return it.

•	Otherwise, skip to the next element of the list.

2.	When all library directories have been examined without finding an appropriate TGTFIL, return "not found".

Strict JCL-Sysin Search

This algorithm is modified for searching JCL-Sysin files in presence of the strict-jcl-libraries special option. When this option is set, the search for SYSIN file A.B.TGTFIL or A.B(TGTFIL) proceeds as follows:

•	if there exists a directory of type JCL-Sysin with logical name “A.B”, then TGTFIL is searched exactly in this directory (base name search, according to the files clause and the physical extension); note that it is an error if two or more directories have the same logical name;

•	otherwise, return “not found”, even if a file with base name TGTFIL exists in another JCL-Sysin directory.

This behavior implies that the libraries clause is ignored on directories of type JCL, at least when it comes to searching JCL-Sysin files (it is still valid to search JCL-Select files). On the other hand, as described above, if the strict-jcl-libraries special option is not set, the logical-name clauses on JCL=Sysin directories are ignored.

It is suggested to use strict search rather than path-based search when there exist many cases of duplicate names, i.e. many files with the same name in different libraries (PDS). In this case, indeed, it is easier to transfer the whole contents of each SYSIN library in a separate directory, and give the name of the library as the logical name of the directory, rather than try to order the various JCL-Sysin directory names in the libraries clauses of the JCL directories to ensure that the appropriate file is found at each reference.

Full Name JCL-Sysin Search

This algorithm is modified for searching JCL-Sysin files in presense of the Full-Name-Sysin-Search special option. When this option is set, the search for SYSIN/SYSTSIN file

A.B.C or A.B(C) proceeds as follows:

File name "A.B.C" or "A.B/C" is used for searching the file under the JCL-Sysin directory.

Note:

If both "strict-jcl-libraries" and "full-name-sysin-search" are set, "strict-jcl-libraries" precedes.

Run-Time Reference

The run-time case applies to references to external components that are not really part of the referencing component. The referencing component can be analyzed or translated even in the absence of the referenced component – even though this absence will cause improper execution. For example, a COBOL program calling a subprogram or a JCL job EXECuting a program. In the Cataloger, such references are handled during the post-analysis phase.

The search starts with a component SRCFIL of a type SRCTYP located in a directory SRCDIR referencing a name TGTFIL of a type TGTTYP. There are two cases to consider.

Unrestricted Search

This case applies when the libraries clause associated with directory SRCDIR does not contain any element (directory) of the type TGTTYP. This can be considered as the "default case". The search algorithm is then:

1.	Gather the (unordered) set of components of type TGTTYP, by searching the directories of this type and analyzing the components they contain.

2.	Record their (base) name.

3.	Search this set for components of name TGTFIL:

•	If there exists exactly one of them, link it with the reference.

•	If there exists more than one of them, link all of them with the reference, but complain about ambiguous references.

•	If there are none, complain about the missing component.

Directed Search

Directed Search is similar to Normal Sub-File Search for compile-time references. It applies when the libraries clause associated with directory SRCDIR contains one or more elements (directories) of the type TGTTYP. The search algorithm is then.

1.	For each directory SUBDIR listed in the libraries clause associated with the definition of SRCDIR in the system description file, in order, locate the definition of SUBDIR in the same file, then:

•	If there is no such definition, complain (this is done once and for all when the Cataloger starts and reads the system description file) and skip to the next element of the list.

•	If the type of SUBDIR is not the TGTTYP appropriate to the reference at hand, skip to the next element of the list.

•	Otherwise (type matches), search the list of component files in the directory (those which match the files clause):

•	If a component file with the base name TGTFIL (and some appropriate extension) exists, return it.

•	Otherwise, skip to the next element of the list.

2.	When all library directories have been examined without finding an appropriate TGTFIL, return "not found".

It is clear that, with this algorithm, no “ambiguous reference” anomaly can occur, since there is at most one file with a given base name in a given directory. This algorithm is hence well suited to analyze systems which contain more than one component with the same name in different directories (or libraries). However, it requires additional effort to set up, since care must be taken to define the appropriate TGTTYP elements in the libraries clause in the appropriate order, for each directory of the SRCTYP at hand.

For a given TGTTYP of components to search, and for each appropriate SRCTYP, it is possible to use directed search for some SRCTYP directory, and unrestricted search for another directory of the same type. Indeed, duplicate component names cause trouble (anomalies) only when they are actually referenced. However, we advise against such practices, which only makes things confusing. For a given SRCTYP/TGTTYP combination, either use unrestricted search on all source directories, or use directed search on all of them.

Note:

Directed Search is not yet available in the current version of the cataloger; if you use the libraries clause to point to components involved in run-time references, these elements will be simply ignored. Directed Search will be added progressively for selected SRCTYP/TGTTYP combinations in the forthcoming versions. Check the release notes.

Repetitive and Incremental Operation

Even with the powerful computing platforms easily available nowadays, processing a complete asset using Tuxedo ART Workbench remains a computing-intensive, long-running, memory-consuming task.

Tuxedo ART Workbench tools are therefore designed to be easily stopped and restarted. The tools use a make-like mechanism to avoid repeating any work which has already been done. This allows efficient operation in all phases of a migration project.

Initial Processing: Repetitive Operation

In the initial phase, when starting with a completely fresh asset and up to the end of the first conversion-translation-generation cycle of a stable asset, the make-like mechanism is used to allow repetitive operation, as follows:

1.	When a tool such as the Cataloger starts, it begins with studying the current state of the asset (source files and target files such as the POB files or the Symtab) and determining what work remains to do to reach a complete and consistent set of results.

2.	The tool then undertakes this work, producing more and more result files (or updating the Symtab with new results).

As the volume of processed files grows, the Refine process consumes more and more memory.

3.	Regularly, the tool checks whether the available physical memory drops below the threshold set by the minimum-free-ram-percent option in the system description file.

•	If the work to be performed is complete before running our of memory, the process definitely stops.

•	Otherwise, the process stops but restarts immediately, after memory is freed. Going back to step 1 above, there is less work to do, so that the process eventually terminates.

This mode is particularly well suited for tools or commands which operate globally on the whole asset, such as the analyze or catalog commands of the Cataloger. This is the normal mode of operation for Tuxedo ART Workbench tools and there is nothing specific to choose it.

Changes in the Asset: Incremental Operation

The Cataloger knows the dependencies between the various components and associated result files. For instance, it records which copy files are used in which COBOL programs. Using this information, it is able to react incrementally when some change occurs in the asset. For example, when a component source file is added, modified or removed: the Cataloger determines which result files are affected by this change and re-computes only those files. Again, this is the normal mode of operation for Tuxedo ART Workbench tools and there is nothing specific to choose it.

Note:

Important: Incremental operation is enabled only after the initial processing of the asset is complete. If you perform changes in the asset before the end of the initial cycle, some dependencies may not yet be recorded, in which case the evaluation of the work to re-do will be incorrect and the final results will be inconsistent. It is therefore very important that you let the Cataloger run to completion on the initial asset before you make any change in the asset.