Support to encrypt Kafka client credentials

Users can now create Kafka connections using SASL-SCRAM. For more information, see Encrypt Kafka Producer Properties in the Oracle GoldenGate for Distributed Applications and Analytics Documentation

MongoDB Capture Support provided for multiple input formats in Oracle GoldenGate for Distributed Applications and Analytics

MongoDB extract process allows us to position to a given LSN using the timestamp and increment formats, which show up in the timestamp info of the oplog document. For more information, see Positioning the Extract (LSN: Positions to a given LSN) in the Oracle GoldenGate for Distributed Applications and Analytics Documentation.

Support to encrypt Kafka client credentials

Users can now create Kafka connections using SASL-SCRAM. For more information, see Encrypt Kafka Producer Properties in the Oracle GoldenGate for Distributed Applications and Analytics Documentation

MongoDB Capture Support provided for multiple input formats in Oracle GoldenGate for Distributed Applications and Analytics

MongoDB extract process allows us to position to a given LSN using the timestamp and increment formats, which show up in the timestamp info of the oplog document. For more information, see Positioning the Extract (LSN: Positions to a given LSN) in the Oracle GoldenGate for Distributed Applications and Analytics Documentation.

Updates to Kinesis Event Handler to support different credential scenarios

Changes have been made to the Kinesis Event Handler to support different credential scenarios. Oracle GoldenGate for Distributed Applications and Analytics (GG for DAA) supports the use of temporary credentials, and also supports the setting of access key and secret when using STS to connect. For more information, see Kinesis Streams Handler Configuration section in the Oracle GoldenGate for Distributed Applications and Analytics Documentation.

Support for Data Transform Framework in GG for DAA

GG for DAA now supports the Data Transform Framework. For more information, see Configuring Data Transform Framework in the Oracle GoldenGate for Distributed Applications and Analytics Documentation.

Modified Source drop-down list in the Target UI

GG for DAA now provides an option to select Kafka target from the Source drop-down list. Kafka Connect option is now listed as a toggle button once Kafka target is selected.

A new parameter eventHandlerTimeoutMinutes has been introduced to handle the event handler thread timeout in minutes. For more information, see Configuring the File Writer Handler in the Oracle GoldenGate for Distributed Applications and Analytics documentation.

The Elasticsearch Handler has been modified to support Elasticsearch 8.x.

For more information, see Using the Elasticsearch Handler with Elasticsearch 8 in Oracle GoldenGate for Big Data articles.

TRACKSCHEMACHANGES support to capture table level DDL changes for Cassandra Capture.

The TRANLOGOPTIONS TRACKSCHEMACHANGES enables Cassandra extract to capture table level DDL changes from the source at runtime. For more information, see Cassandra Capture - Handling Schema Evolution in Oracle GoldenGate for Big Data articles.

BigQuery Replication to Datasets in multiple Google Cloud Platform (GCP) projects

BigQuery Stage and Merge handler now supports replication to datasets that reside in multiple GCP projects. The table catalog name can be mapped to the GCP projectId. For more information, see Google BigQuery Dataset and GCP ProjectId Mapping and Three-Part Table Names in Oracle GoldenGate for Big Data.

Support multiple sources in a single deployment

Enhancements have been made to Cassandra, MongoDB, and Kafka parameters to support capture from multiple sources (JMS/Cassandra/MongoDB/Kafka) in a single deployment without the need to restart Microservices or configuration of the GLOBALS file.

Redis Handler

A Redis Handler handler that supports pushing data to Redis has been added. The handler supports Redis hashmaps, Redis streams, and Redis JSONs. For more information see the Redis Handler - Using the Redis Handler in the Oracle GoldenGate for Big Data User Guide.

INSERTALLRECORDS support for BigQuery

The INSERTALLRECORDS Oracle GoldenGate parameter is now supported for BigQuery target.

Snowflake Initial load COPY SQL support

Snowflake Initial load uses the COPY SQL statement to insert records into the target table. This improves the initial load throughput.

Support multiple sources in a single deployment

Oracle GoldenGate for Big Data now supports capture from multiple sources (JMS/Cassandra/MongoDB/Kafka) in a single deployment without the need to restart microservices or configuration of the GLOBALS file.

Support has been added for DSE Cassandra 6.x capture

You can now configure capture from DSE Cassandra 6.x versions. For more information, see Using Change Data Capture Extract - DSE Cassandra SDK, compatible, compatible with DSE Cassandra 6.x versions in the Oracle GoldenGate for Big Data User Guide.

Updated delimited text formatter

The delimited text formatter has been modified to allow file headers to be created. A file header is a delimited row which names each field in the delimited text output. For more information, see gg.handler.name.format.metaColumnsTemplate in Delimited Text Formatter Configuration Properties (gg.handler.name.format.metaColumnsTemplate) in the Oracle GoldenGate for Big Data User Guide.

Updated JSON formatter

The JSON formatter has been modified to allow files created in JSON format to be represented as a JSON array. This allows the generated file to be represented as a single JSON document whereby the JSONs representing the operation data are elements within the JSON array. For more information, see JSON Formatter Configuration in the Oracle GoldenGate for Big Data User Guide.

Updated HBase Handler to support metacolumns.

The HBase Handler has added functionality whereby configurable metacolumn data can be propagated. For more information, see HBase Handler Configuration in the Oracle GoldenGate for Big Data User Guide.

Snowflake internal stage performance improvement settings

The following new parameters have beed added to improve Snowflake initial load throughput when using internal stage: gg.eventhandler.snowflake.putSQLThreads and gg.eventhandler.snowflake.putSQLAutoCompress. For more information, see Snowflake Event Handler Configuration in the Oracle GoldenGate for Big Data User Guide.

INSERTALLRECORDS support for Snowflake, Synapse, Autonomous Datawarehouse, and Redshift

Initial load support for stage and merge targets

OCI Object Storage Auto configuration

Auto-configuration can be used to simplify configuration for the OCI Object Storage target. The parameter gg.target can be set to oci to enable this feature. For more information, see OCI Event Handler - Automatic Configuration in the Oracle GoldenGate for Big Data User Guide.

AWS S3 Transfer Acceleration

Amazon S3 Transfer Acceleration can be enabled to transfer files quickly and securely over long distances to an S3 bucket. For more information, see gg.eventhandler.s3.enableAccelerateMode in S3 Handler Configuration in the Oracle GoldenGate for Big Data User Guide.

Support has been added for Apache Cassandra 4.0 capture

You can now configure capture from Apache Cassandra 4.0. For more information, see Apache Cassandra 4x SDK, compatible with Apache Cassandra 4.0 version in the Oracle GoldenGate for Big Data User Guide in the Oracle GoldenGate for Big Data User Guide.

Support has been added for BigQuery Metadata provider.

Support has been added for Metadata Provider for Google BigQuery targets. It can be configured to work with BigQuery Handler and BigQuery Event Handler from a source to the BigQuery target using Replicat parameter file. For more information about the datatypes supported, see Supported BigQuery Datatypes in the Oracle GoldenGate for Big Data User Guide.

Support for Snowflake public/private key pair authentication

Support for Snowflake public/private key pair authentication has been added. For more information, see Snowflake Key Pair Authentication topic in the Oracle GoldenGate for Big Data User Guide.

Support has been added to set all fields as nullable in the generated Kafka Connect handler.

Support has been added to set all fields as nullable in the generated Kafka Connect handler. For more information, see gg.handler.name.enableNonnullable in Kafka Connect Handler Configuration in the Oracle GoldenGate for Big Data User Guide.

Support has been added to map the JSON and XML source column types in the Oracle GoldenGate trail to Snowflake VARIANT.

For more information, see Snowflake Event Handler Mapping Source JSON/XML to Snowflake VARIANT in the Oracle GoldenGate for Big Data User Guide.

Support has been added for replication of Synapse tables with IDENTITY columns.

For more information, see Troubleshooting and Diagnostics - Microsoft Azure Synapse Analytics Event Handler in the Oracle GoldenGate for Big Data User Guide.

BigQuery stage and merge now works with the Google BigQuery SDK 2.x

For more information, see Google BigQuery ClassPath Configuration in the Oracle GoldenGate for Big Data User Guide.

Support has been added for MongoDB Capture to connect to MongoDB databases.

For more information, see MongoDB Cassandra Capture Client Dependencies in the Oracle GoldenGate for Big Data User Guide.

Support for the BigQuery Bignumeric data type

The BigQuery Handler has added support for the BigQuery Bignumeric data type. The BigQuery Bignumeric data type provides support for larger scale and precision than the BigQuery Numeric data type.

BigQuery replication using Stage and Merge

Support has been added to replicate data into BigQuery using the Stage and Merge framework.

The change data records are staged into Google Cloud Storage and then merged into BigQuery tables. The BigQuery Event Handler uses the BigQuery Query Job and does not use the BigQuery Streaming API. For more information, see Using the Google BigQuery Event Handler in the Oracle GoldenGate for Big Data User Guide.

MongoDB Capture

Support has been added to capture operations from MongoDB and write them to trail using the Oracle GoldenGate extract process. See Using Oracle GoldenGate Capture for MongoDB in the Oracle GoldenGate for Big Data User Guide.

Support for AJD and ATP Databases

Support has been added to use the Mongo DB Handler replicate transactional data from Oracle GoldenGate to Autonomous JSON Databases (AJD and ATP Database) also. For more information, see Using the MongoDB Handler in the Oracle GoldenGate for Big Data User Guide.

Azure Blob Storage (ABS) Event Handler

Support has been added to store objects in Azure Data Lake using Azure Blob Storage (ABS). For more information, see Using the Azure Blob Storage Event Handler in the Oracle GoldenGate for Big Data User Guide.

Snowflake Event Handler.

Support has been added to replicate data into Snowflake using the Snowflake Event Handler. Oracle no longer recommends to use Command Event Handler and merge scripts for replication to the Snowflake target. For more information, see Using the Snowflake Event Handler in the Oracle GoldenGate for Big Data User Guide.

Google Cloud Storage Event Handler.

Support has been added to store objects in the Google Cloud Platform using the Google Cloud Storage Event Handler. For more information, see Using the Google Cloud Storage Event Handler in the Oracle GoldenGate for Big Data User Guide.

Oracle GoldenGate for Big Data has been certified to run with Java 11 Long-Term-Support (LTS) runtime.

For more information, see Java Boot Options in the Oracle GoldenGate for Big Data User Guide.

Kafka Capture

Support has been added to capture messages from Kafka and write them to trail using the GoldenGate extract process.

The Oracle NoSQL Handler has been changed to use the new NoSQL SDK client.

The new NoSQL SDK client supports connectivity to both on-premise and Oracle Cloud Infrastructure instances of Oracle NoSQL.

New target: Azure Synapse Analytics.

Oracle GoldenGate for Big Data can now natively deliver to Azure Synapse Analytics. Replication to Synapse uses stage and merge data flow. The changed data is staged in Azure Data Lake Storage (ADLS) Gen 2 and eventually merged to Synapse. For more information, see Using the Microsoft Azure Synapse Analytics Event Handler in the Using Oracle GoldenGate for Big Data guide.

Support for the Confluent Protobuf Converter which integrates with the Confluent Schema Registry.

The Kafka Connect Handler has been modified to support the Confluent Protobuf Converter which integrates with the Confluent Schema Registry.

Oracle GoldenGate Microservices Architecture.

Oracle GoldenGate Microservices Architecture is now available with Oracle GoldenGate for Big Data.

Support for partitioning had been added to the File Writer Handler. You can configure partitions based on templated strings. Additionally, partitioning functionality in the HDFS Handler has been changed to also use templated strings to control partitioning.

For more information, see Template Keywords in the Using Oracle GoldenGate for Big Data guide.

Conversion of char column to binary when its charset is set to PASSTHRU is supported.

If you do not specify SOURCECHARSET PASSTHRU, and specify COLCHARSET (PASSTHRU) for some columns, then each of those columns get written as binary data. For more information about SOURCECHARSET with PASSTHRU option, see SOURCECHARSET in Oracle GoldenGate Reference for Oracle GoldenGate.

Oracle wallet feature for JMS handlers is enabled.

For more information about ORACLEWALLETUSERNAME and ORACLEWALLETPASSWORD, see Configuring Properties to Access the Credential Store in the Using Oracle GoldenGate for Big Data guide.

Utility scripts have been added to download 3rd party dependency libraries for various integrations. Obtaining the correct 3rd party dependency libraries for a given type and version of integration was challenging. The Dependency Downloader utility scripts allow you to download 3rd party dependencies for a given integration providing the integration version as an argument..

Utility scripts are located in the {OGGBD install}/DependencyDownloader directory to download client dependency jars for the various supported Oracle GoldenGate for Big Data integrations. For more information, see Dependency Downloader in the Installing and Upgrading Oracle GoldenGate for Big Data guide.

Encryption support on OCI Object Storage has been provided.

A new property gg.handler.name.SSEKey now allows you to control the encryption of data files loaded to OCI. For more information, see Configuring the Oracle Cloud Infrastructure Event Handler in Using Oracle GoldenGate for Big Data guide.