7 Configuring Custom Operations
You can use the custom operations to manipulate Oracle GoldenGate to your specific organization requirements.
You can write C or COBOL routines and call them with Oracle GoldenGate user exits. You can also save frequently used Oracle GoldenGate routines as macros then call the macros from within Extract or Replicat parameter files. You can use OBEY
files to access frequently used Oracle GoldenGate parameters.
This topic includes the following:
User Exits
User exits allow you to extend and customize the functionality of Extract and Replicat. At different points during Extract and Replicat processing, you can call COBOL, C or TAL routines to perform an unlimited number of functions. You can also easily add functions to the application and respond to database events almost as soon as they occur without altering production programs. For example, user exits can:
-
Perform arithmetic operations, special date conversions or table lookups while mapping from one file format to another.
-
Implement record archival functions off-line.
-
Respond to unusual database events in custom ways, for example, by sending a formatted e-mail message or paging a supervisor based on some field value.
-
Accumulate totals and gather statistics.
-
Clean up invalid data.
-
Determine the net difference in a record before and after an update.
-
Accept or reject records based on complex criteria.
-
Normalize a database during conversion.
-
Eliminate indexes that exist to identify recently changed records.
Record Formats for User Exits
User exits expect records to have a specific format. For example, user exits expect:
-
Deletes, inserts, and updates to appear in the buffer as full record images
-
Non-compressed data to have no offset or length preceding data
-
Compressed Enscribe and SQL updates to both have the following format:
(offset)(length)(value)(offset)(length)(value)(. . .)
where
-
(offset) is the offset into the Enscribe record of the data fragment that changed.
-
(length) is the length of the fragment.
-
(value) is the data. Fragments can span field boundaries, so full fields are not always retrieved (unless compression is off or
FETCHCOMPS
is used). -
Enscribe has an I/O type of
11
; SQL has an I/O type of15
. All other I/O types for deletes, inserts, and updates are in non-compressed format.Note:
The above record formats only apply to data sourced from an HP NonStop system.
Creating User Exits
Create user by performing the following process.
To implement user exits:
-
Create a user exit shell routine in C, TAL or COBOL. The user shell routine is the communication point between Extract or Replicat and your routines.
-
C shell routines. Shell routines written in C must be named
CUSEREXIT
and must accept theEXIT-CALL-TYPE
,EXIT-CALL-RESULT
,EXIT-PARAMS
, andEXIT-REC-BUF
parameters. These parameters are supplied by Oracle GoldenGate in theXLIBC
include file. -
COBOL shell routines. Shell routines written in COBOL must specify the
ENV COMMON
directive, and thePROGRAM-ID
of one of the modules must be namedCOBOLUSEREXIT
. TheCOBOLUSEREXIT
program must have a linkage section that containsEXIT-CALL-TYPE
,EXIT-CALL-RESULT
,EXIT-PARAMS
, andEXIT-REC-BUF
parameters. These parameters are supplied by Oracle GoldenGate in theXLIBCOB
copy library. -
TAL shell routines. Shell routines written in TAL must be named
TALUSEREXIT
and must accept theEXIT-CALL-TYPE
,EXIT-CALL-RESULT
,EXIT-PARAMS
, andEXIT-REC-BUF
parameters. These parameters are supplied by Oracle GoldenGate in theXLIBTAL
include file.
See COBOLUSEREXIT for details about COBOL and C programming language routines.
-
-
Include Calling Environment Functions to retrieve information such as record buffers and transaction contexts, if necessary. If the user exit is written in C, you must include the
USRDECS
file. If the exit is written in COBOL, you must furnish aCONSULT
directive to either Extract or Replicat. If the exit is written in TAL, you must source theUSRDECT
file. -
In any language, create routines to respond to each type of event generated by Extract and Replicat.
-
Compile and bind the shell routine and the routines that respond to individual events, creating the user exit module.
-
Bind the user exit module with Extract or Replicat by running the BINDEXIT macro and creating a custom Extract or Replicat module with a different name. For further information, see "Binding the User Exit".
-
Include the
CUSEREXIT, COBOLUSEREXIT
orTALUSEREXIT
parameter in your Extract or Replicat parameter file. -
Run the custom Extract or Replicat module.
Binding the User Exit
BINDEXIT
is an interactive macro that creates a new object file to combine Extract or Replicat with user exit routines. BINDEXIT
syntax is similar to:
TACL> RUN $vol.subvol.BINDEXIT [options] [object_type]
The following information can be entered as options or BINDEXIT
will prompt you for it.
Argument | Description |
---|---|
object_type |
The type of file to create, either Extract or Replicat. |
BINDEXIT
binds your code with the Extract or Replicat code, creating the new object file. BINDEXIT
ensures that you included either a CUSEREXIT
or COBOLUSEREXIT
routine, and that no conflicts exist between your code and the Extract or Replicat module (such as having the same names for different functions). Once the new object file is created, run that file rather than Extract or Replicat.
Example 7-1 BINDEXIT Help
TACL> RUN BINDEXIT Help
Usage: RUN BINDEXIT [options ...] [object_type]
options
are
USEROBJ NEWOBJ GGSUBVOL AXCEL CATALOG SHOWCMD HELP
object_type [EXTRACT | REPLICAT]
Binding User Exits in Native Mode
If you are running your NonStop environment in native mode, you must bind your native exits using NLDEXIT
instead of BINDEXIT
. NLDEXIT
runs just as BINDEXIT
does, and prompts you for the same and some additional information.
TACL >RUN $vol.subvol.NLDEXIT [options] [object_type]
Example 7-2 Some Additional Information Displayed with NLDEXIT and SHOWCMD
-o $DATA1.GGSSRC.TESTREP $DATA2.TSPAK.XSKLCON $DATA2.TEST.REPR $DATA2.TSSOBJ.USRESQL -nostdfiles -allow_duplicate_procs -set runnamed on -set highpin on -set highrequesters on -set saveabend on -set libname $DATA1.GGSSRC.PRIVLIB $system.system.crtlmain -obey $system.system.libcobey NLD - NATIVE MODE LINKER - T6017D45. . . (C)1993 Tandem (C)2004 Hewlett-Packard Development Company, L.P. NLD's command line was: \LA.$system.system.nld -stdin **** INFORMATIONAL MESSAGE **** [20022]: The SRL name or archive name specified as 'zcresrl' in a -l, -lib, or -import flag was resolved to the SRL named '\LA.$SYSTEM.SYS04.zcresrl'. . . . (11 informational messages omitted from this sample) NLD reported 0 errors. NLD reported 0 warnings. NLD reported 12 informational messages. NLD created the following type of object file: \TRILL.$DATA1.GGSSRC.TESTREP (ELF, executable) NLD Timestamp: 15DEC2010 15:07:30 Elapsed Time: 00:00:06
The following example creates a new native user exit in Extract
Example 7-3 Creating a New Native User Exit
TACL> RUN $vol.subvol.NLDEXIT Creates a new Native Extract or Replicat object file linked with a USEREXIT module. Enter X at any prompt to quit. Enter type of GGS object to create Extract or Replicat: GGS Object type: extract_name Enter $Vol.Subvol for Extract relinkable installation_location Enter location of userexit object: your_native_compiled_C_object Enter name for new object file: new_native_extract Does your C User Exit contain C++ modules (Y/N): Y What version compiler was used for C++ (2/3): number Does your C User Exit contain Cobol modules (Y/N)? Y New Extract file $vol.subvol.extractname.filename created with user exits. SQL Catalog for SQLCOMP (or N to avoid SQL compile): SQL_catalog_subvol
Debugging Replicat User Exits
Once you have bound your user exit into Replicat to create a new object, you will want to debug your new code. If your Replicat is in TNS mode, use the following command:
TACL> RUN replicat_name/in $vol.subvol.parameter_name, name $xxxx, lib/
This decouples your Replicat from our licensed PRIVLIB
and prevents errors.
If you are running the native form of Replicat, you may debug as usual.
Sample User Exits
Two sample user exits are supplied with Oracle GoldenGate: DEMOXCOB
(written in COBOL) and DEMOXC
(written in C programming language). You can use these exits as skeletons for your own routines.
DEMOXCOB
illustrates several applications of user exits. DEMOXCOB
responds to Extract events and performs several tasks, including:
-
Mapping data from Enscribe to SQL formats
-
Writing a record to an attention log file under certain conditions
-
Rejecting records with invalid codes
-
Accumulating and outputting order totals
-
Writing archive records when delete records are encountered
DEMOXC
provides an example of how to write a user exit that responds to Replicat events. DEMOXC
maps records from a source to a target layout and creates a summary transaction record for each delivered transaction.
Using Oracle GoldenGate Macros
By using Oracle GoldenGate macros in parameter files you can easily configure and reuse parameters, commands, and functions. You can use macros for a variety of operations, including:
-
Enabling easier and more efficient building of parameters
-
Writing once and using many times
-
Consolidating multiple statements
-
Eliminating redundant column specifications
-
Calling other macros
-
Creating Macro libraries to share across parameter files.
Oracle GoldenGate macros work with Extract and Replicat parameter files.
Creating a Macro
Create an Oracle GoldenGate macro with the MACRO
statement.
MACRO #macro_name PARAMS ([param1] [, param2] [...]) BEGIN macro_body END;
Argument | Description |
---|---|
MACRO #macro_name |
Defines an Oracle GoldenGate macro. macro_name must begin with the # character, as in #macro1. If the # macro character is used elsewhere in the parameter file, such as in a table name, you can change it to something else with the |
PARAMS ([param1] [, param2] [...]) |
Optional. Used to describe parameters to the macro. Each parameter used in the macro must be declared in the |
BEGIN |
Indicates the beginning of the body of the macro. Must be specified before the macro body. |
macro_body |
Represents one or more statements to be used as parameter file input. macro_body can include simple parameter statements, such as COL1 = COL2 or more complex statements that include parameters, such as COL1 = #val2 In addition, macro_body may include invocations of other macros. For example: #colmap(COL1, #sourcecol) |
END; |
Ends the macro definition. |
Creating Macro Parameters
When you specify the optional PARAMS
statement in a macro, the macro processor reads through the macro body looking for instances of the parameter names you defined in the PARAMS
statement. For each occurrence of a parameter name, you must specify a corresponding value, which is substituted for the parameter name during invocation.
For example, to convert a proprietary date format, the following macro defines the #year
, #month
, and #day
parameters.
MACRO #make_date PARAMS (#year, #month, #day) BEGIN @DATE("YYYY-MM-DD", "CC", @IF(#year < 50, 20, 19), "YY", #year, "MM", #month, "DD", #day) END;
Parameter values are substituted within the macro body according to the following rules.
Changing the Macro Character
Anything in the parameter file that begins with the # macro character is assumed to be either a macro or macro parameter. This rule does not apply to text within quotation marks; quoted text is ignored.
If the macro character conflicts with a specification in the parameter file, such as table names that include the # character, you specify a different macro character with the MACROCHAR
parameter. In the following example, $ is defined as the macro character, rather than #.
MACROCHAR $ MACRO $mymac PARAMS ($p1) BEGIN col = $p1 END;
The MACROCHAR
can only be specified once, and must be specified before any macros are defined.
Running the Macro
To run a macro, place the run statement in the parameter file at every place you want the process to occur.
[target =] #macro_name ([value1] [, value2] [, . . .])
Argument | Description |
---|---|
target = |
An optional target to which the results of the macro processing are assigned, such as: DATECOL1 = #make_date(YR1, MO1, DAY1) |
#macro_name |
The name of the macro, such as |
([value1] [, value2] [, . . .]) |
The parameter values to be substituted inside the macro, such as Valid parameter values include plain text, quoted text, and invocations of other macros. Some examples of valid parameter values are: my_col_1 "your text here" #mycalc (col2, 100) #custdate (#year, #month, #day) #custdate (#getyyyy (#yy), #month, #day) |
Invoking a Macro Without Parameters
If the macro does not specify parameters, the parameter value list is empty, but the parentheses are still required. For example:
#no_params_macro ()
Sample Macros
This section shows you sample macros for implementing multiple uses of a statement and invoking another macro.
Implementing Multiple Uses of a Statement
You can use macros to implement multiple uses of a statement, and eliminate the need for entering one statement several times.
The following example illustrates how mapping can be improved with a macro. In this example, a proprietary date format must be converted and the process is used several times. For such a scenario, you could implement a date format conversion in a macro similar to the following:
MACRO #make_date PARAMS (#year, #month, #day) BEGIN @DATE("YYYY-MM-DD", "CC", @IF(#year < 50, 20, 19), "YY", #year, "MM", #month, "DD", #day) END;
To run the macro
-
Place the run statements at the appropriate location, similar to:
MAP $DATA.PROD.ACCOUNT, TARGET $DATA.BACK.ACCOUNT, COLMAP ( TARGCOL1 = SOURCECOL1, DATECOL1 = #make_date(YR1,MO1,DAY1), DATECOL2 = #make_date(YR2,MO2,DAY2) );
-
Upon invocation, the macro expands to:
MAP $DATA.PROD.ACCOUNT, TARGET $DATA.BACK.ACCOUNT, COLMAP( TARGCOL1 = SOURCECOL1, DATECOL1 = @DATE("YYYY-MM-DD", "CC", @IF(YR1 < 50, 20, 19), "YY", YR1, "MM", MO1, "DD", DAY1) DATECOL2 = @DATE("YYYY-MM-DD", "CC", @IF(YR2 < 50, 20, 19), "YY", YR2, "MM", MO2, "DD", DAY2) );
Consolidating Multiple Commands
In addition, frequently used sets of commands can be specified in a macro, as in this example of the macro #option_defaults
.
MACRO #option_defaults BEGIN GETINSERTS GETUPDATES GETDELETES INSERTDELETES END;
Invoking the macro:
#option_defaults () IGNOREUPDATES MAP $DATA.PROD.TCUSTMER, TARGET $DATA.BACK.TCUSTMER;
expands to:
GETINSERTS GETUPDATES GETDELETES INSERTDELETES IGNOREUPDATES MAP $DATA.PROD.TCUSTMER, TARGET $DATA.BACK.TCUSTMER;
Invoking the macro:
#option_defaults () MAP $DATA.PROD.TCUSTORD, TARGET $DATA.BACK.TCUSTORD
expands to:
GETINSERTS GETUPDATES GETDELETES INSERTDELETES MAP $DATA.PROD.TCUSTORD, TARGET $DATA.BACK.TCUSTORD;
Macro Libraries
You can create libraries of macros to be included in different parameter files.
To create a macro library:
-
Create the macros using a text editor, saving them to a file name with the format
$DATA.GGSMACR.
filename, where filename is the name of the file.Note:
A macro library file can contain multiple macros.
-
Store your macro library files in
$DATA.GGSMACR
. -
Specify the
INCLUDE
parameter in your parameter file to include the macro library.
Sample Macro Libraries
These samples show:
Example 7-4 The $DATA.GGSMACR.DATELIB Macro Library
-- -- Date macro library -- MACRO #make_date PARAMS (#year, #month, #day) BEGIN @DATE("YYYY-MM-DD", "CC", @IF(#year < 50, 20, 19), "YY", #year, "MM", #month, "DD", #day) END; MACRO #assign_date PARAMS (#target_col, #year, #month, #day) BEGIN #target_col = #make_date (#year, #month, #day) END;
Example 7-5 The $DATA.GGSMACR.MAINLIB Macro Library
-- -- Main macro library -- INCLUDE $DATA.GGSMACR.DATELIB MACRO #option_defaults BEGIN GETINSERTS GETUPDATES GETDELETES INSERTDELETES END;
Example 7-6 Sample Extract Parameter File
-- Parameter file for EXTRACT EXT1 -- INCLUDE $DATA.GGSMACR.DATELIB EXTRACT EXT1 ... MAP $DATA.PROD.ACCOUNT, TARGET $DATA.BACK.ACCOUNT, COLMAP ( TARGCOL1 = SOURCECOL1, #assign_date(DATECOL1,YR1,MO1,DAY1), #assign_date(DATECOL2,YR1,MO1,DAY1) ); ...
The parameter file processes the macro as follows:
-
The
INCLUDE
statement pointing toDATELIB
is specified at the beginning of the parameter file. -
The
#assign_date
macro is called when needed.
Suppressing Report File Listing
When including long, standard macro libraries, you may want to suppress listing each macro in the report file. Listing can be turned off and on by placing the LIST
and NOLIST
commands anywhere within the parameter file or within the included library.
For example, in the following, NOLIST
suppresses listing each macro in HUGELIB
. Specifying LIST
after the INCLUDE
statement restores listing to the report file.
NOLIST INCLUDE $DATA.GGSMACR.HUGELIB LIST EXTRACT EXT1 . . .
Tracing Parameter Expansion
You can trace macro expansion with the CMDTRACE
parameter. When CMDTRACE
is enabled, the macro processor displays macro expansion steps in the process's report file.
The syntax is:
CMDTRACE [ON | OFF | DETAIL]
Argument | Description |
---|---|
ON |
Enables tracing. |
OFF |
Disables tracing. This is the default setting. |
Tracing is enabled before #testmac
is called, then disabled after the macro runs, as shown in the following example:
EXTRACT EXT1 MACRO #testmac BEGIN COL1 = COL2, COL3 = COL4 END; . . . CMDTRACE ON MAP $DATA.TEST.TEST1, TARGET $DATA.TEST.TEST2, COLMAP ( #testmac ); CMDTRACE OFF . . .
Using OBEY Files
With OBEY
files, you can direct Oracle GoldenGate to parameters stored in a different file, then return processing to the current parameter file. OBEY
files are useful for frequently used parameter statements, or parameters that are used by multiple parameter files.
OBEY filename
To use an OBEY file:
-
Use the NonStop editor to create a file and enter the desired parameters.
-
Edit the file where you want to place an
OBEY
parameter. -
Enter the
OBEY
parameter, specifying the name of the file asfilename
.OBEY
filename
For example:
OBEY $DATA03.GGS.FINANCE
Creating High Pin Processes
Use the PCREATE
library to intercept the C
run-time creation of new processes to create high pin processes.
Note:
The PCREATE
intercept is only available for native mode on the operating systems.
Replicat
For Replicat PCREATE
must be combined with the relinkable PRIVLIB
to build a combined library that will include intercepts to create a high pin TACL.
The following example combines the PCREATE
intercept object, PCREATEO,
with the relinkable (R
) native mode (N
) PRIVLIB
to create a new user library named PRIVLIBX
.
eld -ul -o PRIVLIBX PRIVLIBR PCREATEO -set interpose_user_library on FUP LICENSE PRIVLIBX eld -change libname $DATA.GGS1000.PRIVLIBX REPLICAT
In the last step the new PRIVLIBX
is assigned as Replicat's library. The library name must be fully qualified as shown in the example.
Extract and GGSCI
Extract and GGSCI can use a combined library, such as created in the above example, or PCREATE
can be linked into a user library, such as the PCREATEL
in the example below.
eld -ul -o PCREATEL PCREATEO -set interpose_user_library_on
TACL DEFINE
Enter the DEFINE
for TACL, =GGS_TACL_PROGRAM,
in GLOBALS
if it is to be the same for all Extract, Replicat, and GGSCI programs for that Oracle GoldenGate instance. If it is more specific, include it in the Extract or Replicat parameter file. Alternatively it can also be added to TACLLOC
or to TACLCSTM
.
The following example DEFINE
assumes you first FUP DUP $SYSTEM.SYS
nn.TACL
to $SYSTEM.SYS
nn.TACLHP
, turn HighPin ON
, then add the DEFINE
.
ADD DEFINE =GGS_TACL_PROGRAM, CLASS MAP, FILE $SYSTEM.SYSnn.TACLHP