Veridata Export and Import Utilities

6.3 Veridata Export and Import Utilities

You can use the Export and Import utilities, provided with the Oracle GoldenGate Veridata installation, to define portions of your configuration. Starting Oracle GoldenGate Veridata 23c, you can access these Utilities from the Web User Interface.

Parent topic: Administer

6.3.1 Introduction to the Export and Import Utilities

Using the Export and Import utilities, you can create XML documents that are used to configure Oracle GoldenGate Veridata. The DTD (Document Type Definition) that governs these XML documents is stored in the VERIDATA_23C_HOME/cli/vdt-export-client.jar file.

The Export utility helps you to either selectively or completely export the compare configuration data to an XML file. Additionally, you can use it to export configurations between different Veridata repository types using the import functionality. For example, from a SQL Server configuration to an Oracle configuration.

The Import utility allows you to configure database connections, comparison groups including compare pairs, comparison jobs, and profiles. It takes an XML document as input then creates comparison objects in Oracle GoldenGate Veridata. Typically, the XML document matches the inputs on the configuration pages in the user interface.

You should have an understanding of basic XML and its rules.

These utilities provide the following advantages:

It can reduce the time required to define repetitive tasks
It allows you to create reusable configurations
It can ensure that your test configuration mirrors the one you use for production

Supported Configurations

Parent topic: Veridata Export and Import Utilities

6.3.1.1 Supported Configurations

Oracle GoldenGate Veridata import and export utilities support configuring:

Database connections
Comparison groups (jobs, groups, and compare pairs)
Profiles

Parent topic: Introduction to the Export and Import Utilities

6.3.2 Running the Export and Import Utilities

The Export and Import utilities run from the VERIDATA_23C_HOME/cli/bin directory of the Oracle GoldenGate Veridata installation location. The UNIX and Linux scripts for Import and Export are vdt-import.sh and vdt-export.sh respectively.

Parent topic: Veridata Export and Import Utilities

6.3.2.1 Using the Export Utility

The syntax for running the export utility is:

vdt-export [-host <host> ] [-port <port> ] -user <user Name> -export <fileName>

Required Parameters

-host <host>: The host for Veridata server. The default is localhost.
-port <port>: The port number for the Veridata Web server. The default value is 8089.
-user <user Name>: The Oracle GoldenGate Veridata user. For example, veridata.
-export <fileName>: The file where the Veridata config file gets exported to.

Optional Parameters

One of these optional operations can be requested at run time:
- -help: Prints the usage of all the Export Config file flags.
- -version v: Prints version.
- -jobs [<job1> <job2> ... <jobN>]: Job Names space separated for exporting Configuration.
- groups [<group1> <group2> ... <groupN>]: Export all groups in the repository or add group names separated by a space, such as group1, group2, or group3.
- -connections [<conn1> <conn2> ... <connN>]:Export all connections in the repository or add connections separated by a space, such as conn1, conn2, or conn3.
- -profiles [<profile1> <profile2> ... <profileN>]: Export all profiles in the repository or add profiles separated by a space, such as profile1, profile2, or profile3..
- -all: All the repository configuration will be exported, takes precedence over above options. Also this is the default behavior when none of the optional flag values are entered.
- -exportPassword: Passwords for connections that are exported. By default, the Connection passwords are exported.
- -ssl: To communicate with Oracle GoldenGate Veridata 23c using the https protocol.

Parent topic: Running the Export and Import Utilities

6.3.2.2 Using the Import Utility

The syntax for running the Import utility is:

./vdt-import.sh < -create| -delete | -update | -replace > <fileName> -user <username> -host
<hostname> -port <port> [more_flags]
                         | -p
                         | -extract
                         | -ssl
                         | -help | { -version | -v}

Required Parameters

You need to specify one of the following required parameters while running the Import utility:

-create <fileName>: Create Oracle GoldenGate Veridata config or parameter from file <fileName>.
-delete <fileName>: Deletes Oracle GoldenGate Veridata config from file <fileName>. All named items that exist for the configuration are removed from the repository.
-update <fileName>: Updates Veridata config from file <fileName>. New items are added to the repository and existing items are modified. Items existing in the repository and not listed in the configuration are deleted.
-replace <fileName>: Replaces Veridata config from file <fileName>. All items listed to be replaced in the configuration are replaced as specified.
-user <username>: Oracle GoldenGate Veridata user.
-host <hostname>: Server host where Oracle GoldenGate Veridata is running.
-port <port>: Server port where Oracle GoldenGate Veridata is running.
<fileName>: Veridata config.xml or GoldenGate parameter file path to perform create, delete, update, or replace operation .

Optional Parameters

One of these optional operations can be requested at run time:
- -p: Properties file path.
- -extract: Extract file path and name.
- -ssl: To communicate with the veridata 23c using https, which is the tls protocol.
- -help: Prints usage.
- -version v: Prints version.

Parent topic: Running the Export and Import Utilities

6.3.2.3 Processing the Configuration

The import utility first parses the configuration.xml file attempting to complete the entire file before aborting due to the errors. Any errors it finds are logged in the following location:

DOMAIN_HOME/<VERIDATA_23C_INSTALL_LOCATION>/cli/veridata/logs/vdt_import.log

If it does not abort because of errors, it makes a second parsing pass, this time processing the configuration.

Matching Object Names

Database object names, such as catalogs, schema, tables, indexes, and columns will be matched according to these rules:

The matching is case insensitive
The hyphen (-) is considered a match to the underscore (_) to support matching Enscribe DDL and SQL columns
Wildcard expressions for table names and source column names match against the exact name and against the uppercase version of the name.
Quoted names for schemas and wildcards match everything within the quotations must be matched exactly. A wildcard character within quotes is treated as an ordinary character. An example of a quoted name as it would appear in the XML is:
```
source-table="&quot;CHAR_TYPES&quot;*"
```
This would match CHAR_TYPES, CHAR_TYPES2, and CHAR_TYPES_NOTNULL.
Filters can either include or exclude schemas and tables. If include filters are used, at least one filter must be matched before a table can be included in a compare pair. If exclude filters are used, a table is excluded if it matches any exclude filter. Include filters can include a COLFILTER element that contains a list of columns to include or exclude. When a table matches a include filter, the include filter's COLFILTER is used to specify the columns for the generated compare pair. The schema and table name can use wildcards.
Filters can be used to exclude tables with specific names. For example, you can use the following pattern to exclude tables containing FIRST in their names:
```
<filter type="exclude" table="*FIRST"/>
```
Filters can be used to exclude views by specifying as follows in the XML:
```
<filter exclude-views="true"/>
```
.
For NonStop Enscribe files, file pattern filters are used. The file pattern is any valid NonStop file name pattern.
A compare pair may have a column specification with the Boolean attribute "optional". When this attribute is true, the column is only included in the compare pair if the source table includes the specified source column.

Determining Key Columns

The key columns are selected in the following order:

Explicit key column definitions if they are available. In this case if source-pkey and target-pkey compare-pair element attributes are set it will generate an error.
Columns in the index specified by source-pkey and target-pkey attributes of the compare-pair element. The number of columns and all data types must match and the data types must be compatible.
Columns in the system-selected primary key.

Generating Compare Pairs

Compare pair generation has the following characteristics:

Generating from wild cards works the same as the user interface generation except that regular expressions can be used.
Compare pairs are processed in the order specified in the configuration.xml file
The compare pairs generated by a single compare pair element are generated in alphabetical order of the source table name.
When compare pairs are generated by more than one compare pair element, the first one will be used.

As a general rule, the order of the compare pair elements should be:

Compare pairs with specialized configuration requirements, such as user-defined keys.
Compare pairs that match general patterns.
Exclusions of compare pairs that would otherwise match general patterns.

Parent topic: Running the Export and Import Utilities

6.3.2.4 SSL Configuration for Export and Import Utilities

The Export and Import utilities are now supported to communicate with Oracle GoldenGate Veridata server using SSL protocol.

To configure SSL for Export and Import Utilities:

Check whether the SSL ceritificate is self-signed or not in the Oracle GoldenGate Veridata 23c Server. If its a self-signed certificate, then the client certificate is shipped as part of the installer under <install_location>/config/vdtWebKeystore.p12.
Copy the <install location>/config/vdtWebKeystore.p12 to <install location>/cli/config/veridata-23c.p12.

Note:
To enable SSL, there is no modification required for the self-signed certificate. The Oracle GoldenGate Veridata 23c server should start with SSL.
Access the CLI utility through the SSL mode as follows:
```
./vericom.sh -j <job> -user <user> -ssl
```
Here -ssl is the option for ssl communication between the cli and Server.
If you have your own ssl certificate, replace the <install_location>/cli/config/veridata-23c.p12 to the client certificate and rename it to veridata-23c.p12.

Parent topic: Running the Export and Import Utilities

6.3.3 Configuration File Element Reference

The configuration is defined by the top level configuration element and several nested elements. Most of these elements have attributes that define their characteristics, such as the operation attribute for the configuration element or the port attribute for the connection element.

The following is the high-level element hierarchy in the configuration XML file. For more information about an element and its attributes, click the element name in the hierarchy.

configuration
    connection
       conn-properties
    group
       description
       filter
       sql-partition
       enscribe-key
       compare-pair
          enscribe-info
          enscribe-key
          sql-partition
          column
          excluded-column
          delta-config
    job
    profile

configuration
column
colfilter
colfiltercol
compare-pair
connection
conn-properties
delta-config
description
enscribe-info
enscribe-key
excluded-column
expandddl
filter
group
job
profile
key-column
profile-general
sorting-method
initial-compare
confirm-out-of-sync
param
repair
sql-partition
table partition
The table-partition element helps in specifiying the partition name details in the Oracle GoldenGate Export and Import tools.

Parent topic: Veridata Export and Import Utilities

6.3.3.1 configuration

The root element is configuration.

The following elements can be nested within the configuration element:

Table 6-1 configuration Elements

Elements	Description
connection	One or more Veridata database connection definitions.
group	One or more Veridata comparison group definitions.
job	One or more comparison job definitions.
`profile`	One or more profile definitions.

The following attributes describe the configuration element:

Table 6-2 configuration Attributes

Attribute Description

Attribute	Description
validation	Specifies the type of validation that is used for the configuration. The options are: "`required`" - All compare pairs must be successfully validated before any pairs are added to the repository. This is the default value. "`omit-failures`" - Successfully validated compare pairs are added to the repository and compare pairs that cannot be validated are ignored. "`none`" - Compare pairs are added to the repository without any validation. If this option is selected, the Oracle GoldenGate Veridata Web User Interface should be used to review and fix validation problems.
operation	Specifies how data is applied to the repository. The options are: "`create`" - All items listed in the configuration are new. If any item in the list exists in the repository, nothing is added. This can be used to prevent unintended modification to existing repository items. This is the default value. "`update`" - New items are added to the repository and existing items modified. Items existing in the repository and not listed in the configuration are deleted. "`delete`" - All named items in the configuration are removed from the repository. You can use a command line flag to override the value entered for this attribute.
wildcard	Specifies the pattern matching method that is used. The options are: "`ggs`" - Use the typical Oracle GoldenGate pattern using an asterisk (*). See the Oracle GoldenGate Veridata Web User Interface help for details on this type of matching. This is the default value. "`regex`" - Use regular expressions for matching.

validation

Specifies the type of validation that is used for the configuration. The options are:

"required" - All compare pairs must be successfully validated before any pairs are added to the repository. This is the default value.

"omit-failures" - Successfully validated compare pairs are added to the repository and compare pairs that cannot be validated are ignored.

"none" - Compare pairs are added to the repository without any validation. If this option is selected, the Oracle GoldenGate Veridata Web User Interface should be used to review and fix validation problems.

operation

Specifies how data is applied to the repository. The options are:

"create" - All items listed in the configuration are new. If any item in the list exists in the repository, nothing is added. This can be used to prevent unintended modification to existing repository items. This is the default value.

"update" - New items are added to the repository and existing items modified. Items existing in the repository and not listed in the configuration are deleted.

"delete" - All named items in the configuration are removed from the repository.

You can use a command line flag to override the value entered for this attribute.

wildcard

Specifies the pattern matching method that is used. The options are:

"ggs" - Use the typical Oracle GoldenGate pattern using an asterisk (*). See the Oracle GoldenGate Veridata Web User Interface help for details on this type of matching. This is the default value.

"regex" - Use regular expressions for matching.

Example

The following example adds compare pairs that can be validated and ignores those that cannot; uses regular expressions for wild carding; and uses the "create" default to adds all items as new items, adding nothing if any item already exists.

<configuration validation="omit-failures" wildcard="regex">
.
.
.
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.2 column

The column element defines a set of columns to be included or excluded from the compare pair. The column element has no nested elements or text data.

The following attributes describe the column element:

Table 6-3 column attributes

Attribute	Description
source-name	A regular expression that defines a set of source column names. This value is required.
target-name	A regular expression that defines a set of target column names. It can include references to groups captured by the `source-name` expression.
exclude	Indicates whether or not the matched columns should be excluded from the compare pair. The options are: `"true"` - The matched columns should be excluded. `"false"` - The matched columns should be included. This is the default.
type	Indicates the type of the column. The options are: `"key"` - The column is used as a key. `"hash"` - The column is compared using a hash value. This is the default value. `"literal"`- The column is a literal value.
format	Specifies a format to override the comparison format that would normally be used.
scale	Specifies a scale to override the default scale for the comparison.
precision	Specifies a precision to override the default precision used for the comparison.
timezone	Specifies a time zone to override the default time zone of the comparison.
`optional`	Indicates whether the column mapping is optional. For example, mapping will not fail if the base tables do not have the column patterns specified. Default is `"false"`.

Parent topic: Configuration File Element Reference

6.3.3.3 colfilter

The colfilter element defines a set of columns to be included or excluded. It is used to specify the names of the columns to use as filtering criteria.

The following element describes the colfilter element:

Table 6-4 colfilter Element

Attribute	Description
colfiltercol	Specifies a set of columns to be included or excluded.

The following attribute describes the colfilter element:

Table 6-5 colfilter Attribute

Attribute	Description
type	Specifies whether to include the columns or exclude them. The options are `include` or `exclude`; the default is `include`. This is a required attribute.

Example

This example excludes COL3 and COL5 for the table TABLE_NAME from the generated compare pair.

<filter type="include" table="TABLE_NAME">
    <colfilter type ="exclude">
        <colfiltercol name="COL3" />
        <colfiltercol name="COL5" />
    </colfilter>
</filter>

Parent topic: Configuration File Element Reference

6.3.3.4 colfiltercol

The colfiltercol element defines a set of columns to be included or excluded. It is used to specify the names of the columns to use as filtering criteria.

The following attribute describes the colfiltercol element:

Table 6-6 colfiltercol Attribute

Attribute	Description
name	A regular expression that defines a set of source column names. This is a required attribute.

Parent topic: Configuration File Element Reference

6.3.3.5 compare-pair

The compare-pair element specifies a set of compare pair items. As in the Oracle GoldenGate Veridata Web User Interface, the compare pairs default to system mapped keys and columns.

The following elements can be nested within the compare-pair element:

Table 6-7 compare-pair Elements

Element	Description
enscribe-info	One or more sets of information used when comparing NonStop Enscribe files.
sql-partition	One or more specifications of a subset of rows within the table.
table-partition	Specifies the database table partitions.
enscribe-key	One or more specifications of a subset of records within an Enscribe file.
key-column	A set of columns to be used as the user-defined key for the comparison.
column	One or more definitions of a set of columns to be included.
excluded-column	Defines a set of columns to excluded from the a compare pair when the compare pair uses system mapped columns.
`delta-config`	Defines the delta processing configuration for the compare pair. The maximum is to add it once per compare pair.

The following attributes describe the compare-pair element:

Table 6-8 compare-pair Attributes

Attribute	Element
name	An expression defining the name of the compare pair. This expression can include groups captured with `source-table` expressions and target table group `$0`.
source-table	A regular expression that defines the table or tables to be compared. See "Regular Expression Grouping" later in this section for more detail. The default is to match all tables.
target-table	A regular expression that defines the target tables for the comparison. This may contain references to groups captured by the source table expression. The default is $0 for the full source table name.
source-schema	The name of the default schema for the source tables referenced for the compare pair. The default is the value specified for the `group`. For SQL/MP, this is the subvolume of the SQL catalog. This is not used with Enscribe files.
target-schema	The name of the default schema for the target tables referenced for the compare pair. The default is the value specified for the `group`. For SQL/MP, this is the subvolume of the SQL catalog. This is not used with Enscribe files.
source-catalog	The default catalog for the source tables referenced in this compare pair. For SQL/MP, this is the volume of the SQL catalog. This is not used for the following databases: Oracle (Non-PDB), DB2, Enscribe, or Teradata. For Oracle PDB, this holds the PDB name.
target-catalog	The default catalog for the target tables referenced in this compare pair. For SQL/MP, this is the volume of the SQL catalog. This is not used for the following databases: Oracle (Non PDB), DB2, Enscribe, or Teradata. For Oracle PDB, this holds the PDB name.
exclude	Indicates whether or not the compare pair should be included in the `group` element. This can be used to remove a compare pair generated by an earlier compare pair element. The options are: `"true"` - Exclude the compare pair. `"false"` - Include the compare pair. This is the default.
source-file-pattern	The default file pattern for the source if the data source is Enscribe or SQL/MP.
target-file-pattern	The default file pattern for the target if the data target is Enscribe or SQL/MP.
source-pkey	The name of the unique index to use as the source portion of the user-specified primary key. The default is no user-specified index name.
target-pkey	The name of the unique index to use as the target portion of the user-specified primary key. The default is the value of the `source-pkey`.
delta-processing	Indicates whether or not delta processing is enabled for this compare pair. The options are: `"true"` - delta processing is enabled. `"false"` - delta processing is not enabled. This is the default.
profile-name	The name of the profile to use when running the compare-pair comparison.
system-key	If the compare pair has no column elements and no specified source-pkey, Oracle GoldenGate Veridata will select the most appropriate primary key or unique index to use. The options are: "`true"` - Oracle GoldenGate Veridata selects the key if it is not defined. This is the default. `"false"` - Oracle GoldenGate Veridata does not select the key.
system-columns	Indicates that the compare pair contains column elements with the type attribute set to `key`, so the generated compare pair will have user-defined columns for the key. The options are: `"true"` - Compare pair has `key` column elements. This is the default. `"false"` - Compare pair does not have `key` column elements.
wildcard	Specifies the pattern matching method that is used. The options are: "`ggs`" - Use the typical Oracle GoldenGate pattern that matches an asterisk (*) to any number of characters. For Oracle GoldenGate HP Nonstop, the following is supported: `source-table=""target-table=""`, and the following is not supported: `source-table="TBL" target-table=""`. "`regex`" - Use regular expressions for matching. `"default"` - Use the setting for the `configuration`. This is the default.
is-auto	Specifies whether or not the Automatic Partitioned compare pairs need to be created in Oracle GoldenGate Veridata. `"true"` - Oracle Goldengate Veridata creates the auto partitioned compare pairs. The number of compare-pairs to be created is based on the value specified in `no-of-auto-partitions`. `"false"` - Oracle GoldenGate Veridata does not create auto-partitioned compare pairs. This is the default.
no-of-auto-partitions	Oracle GoldenGate Veridata creates the Auto Partitioned compare pairs. No of compare-pairs to be created is based on the value specified in `no-of-auto-partitions`.
use-source-keys	Defines key column as Source Key Columns OR Target key columns as key columns, when there are no keys selected for column mapping.
use-target-keys	Defines column values from target key to source columns, when there are no keys selected for column mapping.
use-all-columns	Defines key column values from All Columns, when there are no keys selected for column mapping.

Regular Expression Grouping

Regular expression grouping can be used to capture the parts of the source table names to be used for matching the target table name. You can do this by changing the wildcard attribute should be changed to regex. Groups to be matched are referenced as $1, $2, $3 and so on. Group $0 matches the entire source table name.

Examples of matching groups include:

P(.*) - Matches table names that begin with P. It captures the variable portion in $1. This matches table PROSPECTS.
[^PV].* - Matches table names that do not begin with P or V. This does not match the table PROSPECTS, but does match the table REGIONS.
([P-R])(.*) - Matches table names starting with P, Q, or R and captures the initial letter in group $1 and the rest of the name in group $2. Groups are defined by parenthesis pairs. Group numbers are defined by the count of left parenthesis. Group $1 starts at the first left parenthesis and group $2 starts at the second parenthesis.

Captured groups ($n) are then used in expressions for selecting the target tables.

Example

The following example describes the key-only compare-pair. It's source tables are defined in the "test" schema and target tables in the "other" schema. It creates a compare pair in which the source table name begins with S and target table name begins with T. For example, S_TABLE and T_TABLE, where S_TABLE is a table in schema "test" and T_TABLE is table in schema "other". It also excludes all non-key columns in the generated compare pairs.

<configuration> 
    <connection name="source" host="somehost"
               .... use-ssl="true">
        <description>
             <![CDATA[
                Group SQL Scripting Source Connection
              ]]>
          </description>
     </connection>
...
...
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.6 connection

The connection element defines a connection to a source or target comparison database through an Oracle GoldenGate Veridata agent.

The following elements can be nested within the connection element:

Table 6-9 connection Elements

Element	Description
description	Provides a description of the connection.
`conn-properties`	Defines the connection properties for a connection.

The following attributes describe the connection element:

Table 6-10 connection Attributes

Attribute	Description
name	A name that identifies the connection. This is a required attribute.
host	The name of the system on which the Oracle GoldenGate Veridata agent is running.
port	The port number of the system on which the agent is running.
user	The user name the agent uses to connect to the database.
password	The password the agent uses to connect to the database.
repairUser	The database user with privileges to perform repair operations. See Database Privileges for the Agent Component.
repairPassword	The password for the repairUser.
agent-timeout	The amount of time Oracle GoldenGate Veridata will wait before timing out when sending requests to the agent.
truncate-spaces	Either `"true"` or `"false"` to indicate whether or not spaces will be removed from the end of character columns. The default is `"true"` to truncate spaces.
fetch-size	(Oracle only) The number of rows fetched in each batch.
`use-ssl`	Defines using SSL communication between the Veridata Agent and the Server. The default is `"true"`.
use-source-keys	Defines key column as Source Key Columns OR Target key columns as key columns, when there are no keys selected for column mapping.
use-all-columns	Defines key column values from All Columns, when there are no keys selected for column mapping.

Example

The following example identifies the connection named source.

<configuration>
   <connection name="source" host="somehost"
    port="7850" user="somename" password="somepw"repairUser="veridata1" repairPassword="veridata1" agent-timeout="4000" truncate-spaces="false" fetch-size="3" use-ssl="true">
      <description>
             <![CDATA[
                Group SQL Scripting Source Connection
            ]]>
...
...
      </description>
   </connection>
.
.
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.7 conn-properties

The conn-properties element provides additional connection to a source or target comparison database elements.

The following attributes can be nested within the conn-properties element:

Table 6-11 conn-properties

Element	Description
`datatype-name`	Specifies the data type for which properties have changed.
`format`	Specifies the Veridata comparison format to be used for comparison.
`precision`	Specifies the precision to be applied to the comparison.
`scale`	Specifies the scale to be applied to the comparison.
`timezone`	Timezone name is same as in the Veridata GUI.

Parent topic: Configuration File Element Reference

6.3.3.8 delta-config

The delta-config element defines the delta processing configuration for the specified compare pair. It can be used once per compare pair. This element can appear once or not at all depending on the type of configuration you want. When the source or target configuration specified, the corresponding column-name attribute and query element are mandatory.

The following elements describe the delta-config:

Table 6-12 delta-config Elements

Attribute	Description
`source-config`	Provides source side configuration for delta processing.
`target-config`	Provides target side configuration for delta processing.
`query`	Specifies the query for delta processing.

Example

This example creates a compare pair with delta processing enabled. Delta processing is enabled on COL1 of SYSMAPPING1 table for both source and target side. The SQL query is defined within the "query" tag.

<configuration validation="required">
   .
   .
   <group name="testGroup" source-conn="sourceConn" target-conn="targetConn" source-schema="sourceSchema" target-schema="targetSchema">
        <compare-pair source-table="SYSMAPPING1" target-table="SYSMAPPING1" name="sameTables" delta-processing="true" >
                  <delta-config>
                        <source-config column-name="COL1">
                              <query><![CDATA[ SELECT MAX(COL1) from SYSMAPPING1 ]]></query>
                        </source-config>
                        <target-config column-name="COL1">
                              <query><![CDATA[ SELECT MAX(COL1) from SYSMAPPING1 ]]></query>
                        </target-config>
                  </delta-config>
            </compare-pair> 
    </group>
      .
      .
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.9 description

The description element is free-form text that can be used to attach a description to the containing element. It has no associated attributes.

Example

The following example provides a description for the connection named source.

<configuration>
   <connection name="source" host="somehost"
    port="7850" user="somename" password="somepw"
      <description>
         <![CDATA[
           This connection is used when the Veridata agent connects
           to the source.
         ]]>
      </description>
   </connection>
.
.
.
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.10 enscribe-info

The enscribe-info element provides additional information used to compare NonStop Enscribe records at the field level.

The following elements can be nested within the enscribe-info element:

Table 6-13 enscribe-info Elements

Element	Description
expandddl	Describes the rules that are used when applying the DDL.

The following attributes describe the enscribe-info element:

Table 6-14 enscribe-info Attributes

Attribute Description

Attribute	Description
`side`	Indicates whether the information applies to the source or the target table. The options are: `"source"` to specify the source table. This is the default. `"target"` to specify the target table.
dictionary	The volume and subvolume containing the data dictionary.
record	The name of the record in the data dictionary.

side

Indicates whether the information applies to the source or the target table. The options are:

"source" to specify the source table. This is the default.

"target" to specify the target table.

dictionary

The volume and subvolume containing the data dictionary.

record

The name of the record in the data dictionary.

Parent topic: Configuration File Element Reference

6.3.3.11 enscribe-key

The enscribe-key element defines the key that is to be used for Enscribe files. The enscribe-key element defines a delta processing that can used in a where clause on the initial comparison query.

The following attributes describe the enscribe-key:

Table 6-15 enscribe-key Attributes

Attribute	Description
name	A name that identifies the key. This is a required attribute.
start-key	The key that is to be used to begin reading the Enscribe file. This is a required entry.
end-key	The key of the last Enscribe record that should be read. This is a required entry.
format	Specifies the format of the Enscribe key. The options are: `"ascii"` - The format of the key is ASCII. This is the default. `"hexadecimal"` - The format of the key is hexadecimal.
side	Indicates whether the partition should be applied at the source database, the target database, or both databases.
default	Indicates whether this is the default partition. This is equivalent to the "use at run time" indicator on the UI. The default is `both`.

Examples

<enscribe-key name = "Part1" end-key ="1000" format ="hexadecimal" default ="false" side="source"/>
<enscribe-key name = "Part1" start-key ="001" format ="hexadecimal" default ="false" side="target"/>
<enscribe-key name = "Both" start-key ="001" end-key ="1000" default ="true"/>

Parent topic: Configuration File Element Reference

6.3.3.12 excluded-column

The excluded-column element defines a set of columns to be excluded from a compare pair when the compare pair uses system mapped columns.

The following attribute describes the excluded-column element:

Table 6-16 excluded-column Attributes

Attribute	Description
name	A regular expression that defines a set of source column names. This is a required attribute.

Parent topic: Configuration File Element Reference

6.3.3.13 expandddl

The expandddl element describes the rules used when applying the DDL.

The following attributes describe the expandddl element:

Table 6-17 expandddl Attributes

Attribute	Description
expandGroupArrays	Whether or not to expand group arrays. The options are: `"true"` to expand the array. This is the default. `"false"` not to expand the array.
redefined-columns	Whether or not to include redefined columns. The options are: `"include"` - Includes redefined columns `"omit"` - Leaves out redefined columns. This is the default.
resolvedups	Specifies how to resolve duplicates that result when the array is expanded. The options are: `"appendIndex"` - Adds a unique numeric index to the end of the duplicate. This is the default. `"appendAlphaIndex"` - Adds an alpha character index to the end of the duplicate. `"prependGroup"` - Prefixes the name of the array group to the duplicate.
ddl-separator	The character separator for defining array output into columns. An example is the dash used in FIELDX-3, which is the third occurrence of `FIELDX` in the array. The options are: `"none"` - There is no separator. This is the default. `"dash"` - Use a dash (-) as the separator. `"bracket"` - Use brackets [] as the separator. `"underscore"` - Use underscore (_) as the separator. `"double-underscore"` - Use double underscore (__) as the separator.
zero-fill-length	Prepends zeros to adjust the number of the occurrence. The value is the number of digits enclosed in quotation marks. "0" is the default.
fix-long-names	Whether to fix the names that result from resolving duplicates if they exceed the `max-col-name-length`. The options are: `"true"` - Fix the names that exceed the maximum. This is the default. `"false"` - Do not change the names that exceed the maximum.
max-col-name-length	The maximum length allowed for a column name. The entry is a number within quotation marks. The default is `"120"`.

Parent topic: Configuration File Element Reference

6.3.3.14 filter

The filter element defines a set of schemas and tables to either be included or excluded.

When using include filters, at least one filter must be matched before a table can be included in a compare pair. When a table matches a include filter, the include filter's colfilter is used to specify the columns for the generated compare pair.

When using exclude filters, a table is excluded if it matches any exclude filter. Include filters can include a colfilter element, which contains a list of columns to include or exclude.

Instead of schema and table filters, NonStop platforms use file pattern filters. The file pattern is any valid NonStop platform file name pattern.

The schema and table name can use wildcards.

The following attribute describes the filter element:

Table 6-18 filter Attributes

Attribute	Description
`type`	Specifies either to include or exclude schemas and tables. Valid values are `include` or `exclude`.
`catalog`	Specifies the default catalog name.
`exclude-views`	Excludes all the views while generating compare pairs. Valid values are `true` or `false`. Default value is `false`.
`schema`	Specifies the schema name.
`table`	Specifies the table name.
`file-pattern`	For NonStop platforms only, specifies the file patter filter.

Example

When the source and target schemas have CHAR_TYPES3, INT_TYPE1, and INT_TYPE2 tables, then the following filters only create compare pairs for tables CHAR_TYPES1 and CHAR_TYPES3. The CHAR_TYPES2 table is excluded because of exclude filter and INT_TYPE1 and INT_TYPE2 are excluded because they were not part of include filter.

<group 
      ..
        <filter type="include" table="CHAR_TYPES*" />
        <filter type="exclude" table="CHAR_TYPES2" />
        <compare-pair source-table="*" target-table="*">
        </compare-pair>
      ..
</group>

Parent topic: Configuration File Element Reference

6.3.3.15 group

The group element defines a set of compare pairs that all have the same source and target database connections. These compare pairs also have other properties in common.

The following elements can be nested within the group element.:

Table 6-19 Group Elements

Element	Description
description	Provides a description of the group.
`filter`	One or more filter specifications, which allows table name filtering at the group level.
sql-partition	One or more specifications of a subset of rows within the table.
enscribe-key	One or more specifications of a subset of records within an Enscribe file.
compare-pair	Defines one or more compare pairs. The `compare-pair` elements are added to the group in the order they are specified. If the same compare pair fits the criteria of another specification in the group, the first compare pair will be used.

The following attributes describe the group element:

Table 6-20 Group Attributes

Attribute	Description
name	A name that identifies the group. This value is required.
source-conn	The name of the connection to the source database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository.
target-conn	The name of the connection to the target database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository.
source-schema	The name of the default schema for the source tables referenced in the compare pairs that make up the group.
target-schema	The name of the default schema for the target tables referenced in the compare pairs that make up the group.
`source-catalog`	The default catalog for the source tables referenced in this group.
`target-catalog`	The default catalog for the target tables referenced in this group.
`validation`	Specifies the type of validation that will be used for the configurations. The options are: "`required`" - All compare pairs must be successfully validated before any pairs are added to the repository. "`omit-failures`" - Successfully validated compare pairs are added to the repository and compare pairs that cannot be validated are ignored. "`none`" - Compare pairs are added to the repository without any validation. If this option is selected the Oracle GoldenGate Veridata Web User Interface should be used to review and fix validation problems. `"default"` - Use the type of validation specified for a higher level, such as the `configuration` element. This is the default.
`source-file-pattern`	The default file pattern for the source if the data source is Enscribe or SQL/MP.
`target-file-pattern`	The default file pattern for the target if the data target is Enscribe or SQL/MP.

Example

<group name="weekly-tables" source-conn="source" target-conn"="target">
   <description>
   .
    .
    .
   </description>
   <sql-partition>
    .
    .
   </sql-partition>
   <compare-pair>
    .
    .
    .
</compare-pair>
</group>

Parent topic: Configuration File Element Reference

6.3.3.16 job

The job element defines an Oracle GoldenGate Veridata comparison job.

The following elements can be nested within the job element:

Table 6-21 job Elements

Element	Description
description	Provides a description of the job.
group	The name of the group associated with the job. This can be a new group or a previously defined group.

The following attributes describe the job element:

Table 6-22 job Attributes

Attribute	Description
name	A name that identifies the job. This is a required attribute.
source-conn	The name of the connection to the source database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is required if it references an existing connection in the repository. The job `source-conn` is used to override the source connection specified for the groups included in the job.
target-conn	The name of the connection to the target database. This can reference an existing connection in the Oracle GoldenGate Veridata repository or a connection previously defined in this configuration. This attribute is used to override the target connection for the groups included in the job.
profile	The default profile to use when running the job.

Example

<job name="all-groups" profile="server-sort">
    <group name="all-tables"/>
    <group name="selected-tables"/>
</job>

Parent topic: Configuration File Element Reference

6.3.3.17 profile

The profile element defines the connection properties of a comparison job connection.

The following elements can be nested within the profile element:

Table 6-23 profile Elements

Element	Description
description	Provides a description of the profile.
profile-general	Defines the profile parameters that control the output options.
`sorting-method`	Defines the profile parameters that control the sorting method and memory management. The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.
`initial-compare`	Defines the profile parameters that control the parameters for the job that performs the initial compare step
`confirm-out-of-sync`	Specifies the profile parameters that control the parameters for the job that performs the confirmation step
`repair`	Specifies the profile parameters that control the parameters for the repair job.

The following attributes describe the profile element:

Table 6-24 profile Attributes

Attribute	Description
name	A name that identifies the profile. This is a required attribute.

Example

This example creates profile named "userDefinedProfile". The parameter names like "oos-format", "sort-method" are described in the table (link for table is in another pin)

<configuration validation="required">
   .
   .
<profile name="userDefinedProfile">
    <profile-general>
        <param name="oos-format" value="xml" />
        <param name="oos-xml-chunk-size" value="1000" />
    </profile-general>
    <sorting-method>
        <param name="sort-method" value="server" />
    </sorting-method>
</profile>
      .
      .
</configuration>

Parent topic: Configuration File Element Reference

6.3.3.18 key-column

The key-column element defines a set of columns to be used as the user defined key for the comparison job.

The following attributes describe the key-column element:

Table 6-25 profile Attributes

Attribute	Description
source-name	A regular expression that defines a set of source column names. This value is required.
target-name	A regular expression that defines a set of target column names. It can include references to groups captured by the `source-name` expression.
format	Specifies a format to override the comparison format that would normally be used.
scale	Specifies a scale to override the default scale for the comparison.
precision	Specifies a precision to override the default precision used for the comparison.
timezone	Specifies a time zone to override the default time zone of the comparison.

Parent topic: Configuration File Element Reference

6.3.3.19 profile-general

The profile-general element provides parameters to control the output options.

The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.

The following elements can be nested within the profile-general element:

Table 6-26 profile-general Element

Element	Description
param	Defines the parameter to change for the profile.

Parent topic: Configuration File Element Reference

6.3.3.20 sorting-method

The sorting-method element provides parameters for sorting method and memory management. The data is sorted to match keys (or a key specification) so that the correct source and target rows are compared.

The following elements can be nested within the sorting-method element:

Table 6-27 sorting-method Element

Element	Description
param	Defines the parameter to change for the profile.

Parent topic: Configuration File Element Reference

6.3.3.21 initial-compare

The initial-compare element provides parameters for the process that performs the initial compare step.

The following elements can be nested within the initial-compare element:

Table 6-28 initial-compare Element

Element	Description
param	Defines the parameter to change for the profile.

Parent topic: Configuration File Element Reference

6.3.3.22 confirm-out-of-sync

The confirm-out-of-sync element provides parameters for the process that performs the confirmation step.

The following elements can be nested within the confirm-out-of-sync element:

Table 6-29 confirm-out-of-sync Element

Element	Description
param	Defines the parameter to change for the profile.

Parent topic: Configuration File Element Reference

6.3.3.23 param

The param element defines the parameters that are used for configuring profile options.

The following attributes describe the repair element:

Table 6-30 param Attributes

Attribute	Description
name	The name of the parameter. This is a required attribute.
`value`	The value of the parameter

Parent topic: Configuration File Element Reference

6.3.3.24 repair

The repair element provides parameters for the repair process.

The following elements can be nested within the repair element:

Table 6-31 repair Element

Element	Description
param	Defines the parameters that are used to configure the profile options.

Parent topic: Configuration File Element Reference

6.3.3.25 sql-partition

The sql-partition element defines a boolean SQL expression that can be used in a where clause in the initial comparison query.

The following attributes describe the sql-partition element:

Table 6-32 sql-partition Attributes

Attribute	Description
`name`	A name that identifies the partition. This is a required attribute.
`side`	Indicates whether the partition should be applied at the source database, the target database, or both databases. The default is `"both"`.
`default`	Indicates whether this is the default partition. This is equivalent to the "use at run time" indicator on the UI. The default is `"false"`.
`type`	Distinguishes between Manual and Automatic Row Partition. When `type` is set to `sql`, it defines Manual Partition and `type` is set to `auto`, defines Automatic Row Partition. This attribute is displayed during export of compare-pairs using Oracle GoldenGate Veridata Export tool.

Example

<sql-partition name="replicate" default="true" side="source" type="sql">
  <![CDATA[ replicated='false']]>
</sql-partition>
<sql-partition name="replicate" default="true" side="source" type="sql">
  <![CDATA[ replicated='true']]>
</sql-partition>
<sql-partition name="AutoPartition0" default="true" side="source" type="auto>
<![CDATA[2,0]]>
</sql-partition>

Parent topic: Configuration File Element Reference

6.3.3.26 table partition

The table-partition element helps in specifiying the partition name details in the Oracle GoldenGate Export and Import tools.

For more information, see Mapping Database Table Partitions for Manual and Automatic Row Partitioning. To create compare-pairs using existing Database Table partitions, include the following lines in the XML when you execute the Import and Export tools:

<configuration operation="create" validation="required">
<group name="oracle_oracle_grp" source-conn="oracle_src" target-conn="oracle_tgt">
 <compare-pair name="SALES_SRC_Q1_2006=SALES_GT_SALES_Q1_2006_AutoPartition0"
  source-table="SALES_SRC" target-table="SALES_TGT" source-schema="SYSTEM" target-schema="SYSTEM"
  use-source-keys="true" use-target-keys="true" use-all-columns="true">

<table-partition name = "SALES_Q1_2006" default="true" side="target"/>
<table-partition name = "SALES_Q1_2006" default="true" side="target"/>
</compare-pair>
</group>
</configuration>

Parent topic: Configuration File Element Reference