You can perform more powerful search and replace operations of both schema data (catalog names, schema names, table names, and column names) and column value data, which are separately configured. Regular expressions (regex) are characters that customize a search string through pattern matching.

You can match a string against a pattern or extract parts of the match. Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA) uses the standard Oracle Java regular expressions package, java.util.regex, see Regular Expressions in  The Single UNIX Specification, Version 4.

• Using Schema Data Replace
  
• Using Content Data Replace
  

gg.schemareplaceregex=[$] 
gg.schemareplacestring= 

gg.schemareplaceregex=CDATA[[$]]
gg.schemareplacestring=CDATA[]

gg.schemareplaceregex=some_regex
gg.schemareplacestring=some_value
gg.schemareplaceregex1=some_regex
gg.schemareplacestring1=some_value
gg.schemareplaceregex2=some_regex
gg.schemareplacestring2=some_value

gg.contentreplaceregex=some_regex 
gg.contentreplacestring=some_value 
gg.contentreplaceregex1=some_regex 
gg.contentreplacestring1=some_value 
gg.contentreplaceregex2=some_regex 
gg.contentreplacestring2=some_value

gg.contentreplaceregex=[\n] 
gg.contentreplacestring=CDATA[ ]

this is 
me

this is me

^\\s+|\\s+$

gg.contentreplaceregex1=^\\s+|\\s+$ 
gg.contentreplacestring1=CDATA[]

Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA) supports breaking down the source trail files into either multiple Replicat processes or by using Coordinated Delivery to instantiate multiple Java Adapter instances inside a single Replicat process to improve throughput. This allows you to scale GG for DAA delivery.

There are some cases where the throughput to GG for DAA integration targets is not sufficient to meet your service level agreements even after you have tuned your handler for maximum performance. When this occurs, you can configure parallel processing and delivery to your targets using one of the following methods:

• Multiple Replicat processes can be configured to read data from the same source trail files. Each of these Replicat processes are configured to process a subset of the data in the source trail files so that all of the processes collectively process the source trail files in their entirety. There is no coordination between the separate Replicat processes using this solution.

• Oracle GoldenGate Coordinated Delivery can be used to parallelize processing the data from the source trail files within a single Replicat process. This solution involves breaking the trail files down into logical subsets for which each configured subset is processed by a different delivery thread. For more information abour Co-ordinated Replicat, see About Coordinated Replicat in the Oracle GoldenGate Microservices Architecture Documentation.

With either method, you can split the data into parallel processing for improved throughput. Oracle recommends breaking the data down in one of the following two ways:

• Splitting Source Data By Source Table –Data is divided into subsections by source table. For example, Replicat process 1 might handle source tables table1 and table 2, while Replicat process 2 might handle data for source tables table3 and table2. Data is split for source table and the individual table data is not subdivided.

• Splitting Source Table Data into Sub Streams – Data from source tables is split. For example, Replicat process 1 might handle half of the range of data from source table1, while Replicat process 2 might handler the other half of the data from source table1.

• If you are using Coordinated Replicat, please make sure that you add TARGETDB LIBFILE libggjava.so SET property=path_to_deployment_home/etc/conf/ogg/your_replicat_name.properties.

Additional limitations:

• Parallel apply is not supported.

• The BATCHSQL parameter not supported.

Feature index

• Option1 - Discreet threads per table - Initial load
• Option2 - Shared threads per table - Initial load
• Option3 - Discreet threads per table - CDC
• Option4 - Shared threads per table - CDC

*SNOW

• Option4 is supported for Snowflake but not recommended.
• MERGE statement holds a table lock until the transaction is committed. The locks times may hinder the performance of the replicat.

*BQ

• Option4 is supported for BigQuery but not recommended.
• At times, replicat can ABEND with Query error: Could not serialize access to table due to concurrent update
• BigQuery jobs are retried, by default up to three times. We have a few configuration parameters related to this. For more information, see gg.eventhandler.bq.retries and gg.eventhandler.bq.totalTimeout in BigQuery Event Handler Configuration.

*DBX

• Option4 is supported for Databricks but not recommended.
• If the INSERT or MERGE DML modifies the same partition in a table (or if the table is unpartitioned), it could lead to ConcurrentAppendException. Seehttps://docs.databricks.com/en/optimizations/isolation-level.html#concurrentappendexception

*SYN

• Option 4 is supported. For Synapse, the isolation level of the transactional support is default to READ UNCOMMITTEDSee: https://learn.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql?view=sql-server-ver16#arguments
• If you modifiy the default isolation level, then there could be a performance impact and errors when using the MERGE statement.

*RS

• Option 2 and Option 4 are supported for Redshift but provided the Redshift database's isolation level is set to SNAPSHOT.
• To enable co-ordinated apply for Redshift, ensure that the Redshift database's isolation level is set to SNAPSHOT.
• The Redshift SNAPSHOT ISOLATION option allows higher concurrency, where concurrent modifications to different rows in the same table can complete successfully. SQL:

  ALTER DATABASE <sampledb> ISOLATION LEVEL SNAPSHOT;

*ICE

• Option 2 and Option 4 are supported for most Iceberg catalogs except the Hadoop catalog.

*TC

• Not recommended if auto table creation and/or altering is enabled. Threads of execution may conflict during table creation and/or altering.

*HBASE

• Precreate namespaces and target tables in HBase.

*REDIS

• Disable automatic index creation in the configuration otherwise threads of execution may conflict with each other when creating indexes.

*FW

• Configure a file naming template which ensures that file names are unique across threads of execution. A good strategy is to use ${groupname} in the template.

Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA) doesn't have built-in high availability functionality. You need to use a standard cluster software's high availability capability to provide the high availability functionality.

You can configure a high availability scenario on a cluster so that if the leader instance of (GG for DAA) on machine fails, another GG for DAA instance could be started on another machine to resume where the failed instance left off.

If you manually configure your instances to share common GG for DAA and Oracle GoldenGate files using a shared disk architecture you can create a fail over situation. For a cluster installation, these files would need to accessible from all machines and accessible in the same location.

The configuration files that must be shared are:

• replicat.prm

• Handler properties file.

• Additional properties files required by the specific adapter. This depends on the target handler in use. For example, Kafka would be a producer properties file.

• Additional schema files you've generated. For example, Avro schema files generated in the dirdef directory.

• File Writer Handler generated files on your local file system at a configured path. Also, the File Writer Handler state file in the dirsta directory.

• Any log4j.properties or logback.properties files in use.

Checkpoint files must be shared for the ability to resume processing:

• Your Replicat checkpoint file (*.cpr).

• Your adapter checkpoint file (*.cpj).

The Oracle GoldenGate credential store manages user IDs and their encrypted passwords (together known as credentials) that are used by Oracle GoldenGate processes to interact with the local database. The credential store eliminates the need to specify user names and clear-text passwords in the Oracle GoldenGate parameter files.

An optional alias can be used in the parameter file instead of the user ID to map to a userid and password pair in the credential store. The credential store is implemented as an auto login wallet within the Oracle Credential Store Framework (CSF). The use of an LDAP directory is not supported for the Oracle GoldenGate credential store. The auto login wallet supports automated restarts of Oracle GoldenGate processes without requiring human intervention to supply the necessary passwords.

In Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA), you specify the alias and domain in the property file not the actual user ID or password. User credentials are maintained in secure wallet storage.

• Creating a Credential Store
  
• Adding Users to a Credential Store
  
• Configuring Properties to Access the Credential Store
  

ALTER CREDENTIALSTORE ADD USER scott PASSWORD tiger ALIAS scsm2 domain ggadapters

ORACLEWALLETUSERNAME[alias domain_name]

ORACLEWALLETPASSWORD[alias domain_name]

gg.handler.hdfs.hiveJdbcUsername=ORACLEWALLETUSERNAME[myalias mydomain] 
gg.handler.hdfs.hiveJdbcPassword=ORACLEWALLETPASSWORD[myalias mydomain]

gg.handler.<name>.user=ORACLEWALLETUSERNAME[JMS_USR JMS_PWD]
 
gg.handler.<name>.password=ORACLEWALLETPASSWORD[JMS_USR JMS_PWD]

Oracle GoldenGate for Distributed Applications and Analytics now includes the ability to log SQL statements for stage and merge targets into a file. This feature is designed to assist with debugging by providing insight into the execution of SQL statements processed by Replicat.

By default, SQL logging is set to the INFO level. The generated log files are stored in the OGG_DIRLOG directory. Each SQL log entry includes a timestamp to make it easier to trace the execution sequence. Note that SHOWSYNTAX is not supported for coordinated Replicat groups.

This functionality is intended specifically for debugging purposes. It is recommended to use it when you need to verify the SQL syntax generated by Replicat as part of troubleshooting or validation.

• Configuring SQL Statement Logging