5 Connecting to Data Sources
Learn about different methods of connecting Oracle GoldenGate data sources and targets and how to add Extract and Replicats.
Configuring Data Sources for SQL Server and DB2
After you have installed the SQL Server and DB2 client drivers, the next steps are to create connection data sources to your source and target databases.
For SQL Server on Oracle GoldenGate Classic Architecture, follow the instructions in Configuring a Database Connection on Linux and for Db2 on Oracle GoldenGate Classic Architecture, see Using Oracle GoldenGate for DB2 for z/OS in Using Oracle GoldenGate for Heterogeneous Databases guide.
See the Oracle GoldenGate Microservices Documentation for configuring Oracle GoldenGate Microservices Architecture for SQL Server and Db2.
Creating Data Source Connections in Oracle GoldenGate Microservices
To create and run Extract and Replicat processes, enable supplemental logging and add heartbeat and checkpoint tables, you need to set up database connections.
- Launch the Administration Service interface and log in.
- Click DB Connections from the Application Navigation pane.
- Click the + sign next to DB Connections, and set up your new credential alias, then click Submit.
- Click the Connect icon to verify that the new alias can
correctly log in to the database.
If an error occurs, then click the Alter Credential icon to correct the credential information, and then test the log in.
You can edit existing credentials to change the user name and password. Delete a credential by clicking the trash icon.
When you successfully log into your database, you can add and manage checkpoint tables, transaction information, and heartbeat tables. All of the tables can be searched using the various search fields.
Enabling Logging, Checkpoint Tables, and Implementing Heartbeat Monitoring
After you have created Credentials for your source and target databases, you can enable transaction log for source databases, create checkpoint tables for target databases, and create heartbeat monitoring, all through the Credentials.
For more information about how to create the processes, see Before Adding Extract and Replicat in Oracle GoldenGate Microservices Documentation.
Creating the Oracle GoldenGate CDC Cleanup Job for SQL Server
After you have enabled TRANDATA
for SQL Server, the next step is to
disable the default SQL Server CDC Cleanup job and install the Oracle GoldenGate CDC Cleanup
job.
For more information about these steps, see the Purging CDC Staging Data in Using Oracle GoldenGate for Heterogeneous Databases.
For the OCI Marketplace compute node, the ogg_cdc_cleanup_setup.sh file is
located in the /u02/deployments/<deployment>/etc/conf/ogg
directory.
Connecting to Oracle Database (on-premises)
You can use Oracle GoldenGate Microservices on Marketplace to remotely capture from and apply data to on-premises Oracle database resources. This allows you to enable replication and centrally manage the replication processes.
You can use Oracle GoldenGate Microservices to replicate data between data resources in the following use cases:
- Migrations
- Data Distribution
- Real-Time Data Warehousing
- Operational Reporting
Prerequisites
Ensure that the following are set up before you begin replication:
- Oracle GoldenGate Microservices
- Source Database
- Target Database
To move data from on-premises to the cloud or from on-premises to on-premises, perform the following tasks :
Configure Oracle Database for Replication
To prepare your Database as a Service (DBaaS) instance for replication, perform the following tasks:
- Configure Logging Properties
- Enable Supplemental Logging
- Enable Oracle GoldenGate within the Oracle Database
- Update parameter for
enable_goldengate_replication
- Update parameter for
For more details, see Preparing the Database for Oracle GoldenGate in Using Oracle GoldenGate for Oracle Database.
Configure Oracle GoldenGate Microservices Compute Node
To connect the Microservices Compute Node to any database, edit the
tnsnames.ora
file and point the entry to your database
resources.
By default, the environment variable TNS_ADMIN
is pre-configured
for each deployment. But the files tnsnames.ora
or
sqlnet.ora
are not readily available on the compute node. You need
to create the files or copy them from an existing file.
Note:
- If you are using Oracle Autonomous products, see Configure Oracle GoldenGate for Replication with Autonomous Databases.
- If you want your networking directory structure to be consistent
with other Oracle products, you must append
/network/admin
to the directory structure. For this you have to change the environment variableTNS_ADMIN
. For more information on Local Naming Parameters, refer to Database Net Services Reference.
Connecting Oracle GoldenGate on Marketplace with Different Oracle GoldenGate Instances
Oracle GoldenGate on Oracle Cloud Marketplace enables you to connect and work with many existing and new Oracle GoldenGate instances.
This topic helps you with a few examples and steps required to set up and configure the architectures. The supported architectures are:
- On-premise to Oracle Cloud (Marketplace)
- Oracle Cloud to Oracle Cloud (Marketplace to Marketplace)
Securely Connecting Oracle GoldenGate Microservices On Premise to Oracle GoldenGate Microservices on Marketplace
You can connect your on-premises Oracle GoldenGate Microservices architecture to the Oracle GoldenGate Microservices architecture on Oracle Cloud Marketplace using the following methods:
- Connecting through Public IP Address
- Connecting through IPSec VPN
- Connecting through FastConnect
The following section helps you to connect through public IP address. To use the IPSec VPN or the FastConnect approach, refer to the respective VPN Connect and FastConnect documentation.
Connecting Through Public IP Address
By default, we configure Oracle GoldenGate Microservices on Oracle Cloud Marketplace behind the Nginx Reverse Proxy. This simplifies the architecture on Oracle Cloud and also makes the deployment secure. When you connect over a public IP address, on-premises Oracle GoldenGate Microservices architecture has to be secure.
When the deployment is secure and in order to connect to it securely, perform the following steps on the on premise machine and the Oracle Cloud compute node:
On Oracle Cloud Compute Node
On Premise Machine
- From the Administration Service tab on the on premise environment, create an alias that can connect to the network user created in the previous step. This alias is used by the Distribution Service to connect to the Receiver Service on Oracle Cloud.
- Log in to the Administration Service.
- Open the context menu and select Configuration.
- Click the plus (+) icon, to add a new credential.
- Add a new Credential by providing the following information:
Credential Domain: Network Credential Alias: streamnetwork User ID: streamnetwork Password: ********** Verify Password: *********
Note:
You cannot validate the new credential, as it is not logged in to the database.
- In the Distribution Service on the on-premise environment,
- Provide information needed for the Distribution Path. The basic information required are:
Path Name: OP2OCI Reverse Proxy Enabled: Toggle to on Use Basic Authentication: Toggle to on Source: Select Extract and provide source trail file info Target: Keep the WSS protocol Provide the Hostname of the OCI Compute node – IP will not work Provide remote trail file name Provide Deployment name Domain: Network Alias: securitynetwork Trail Size (MB): set to desired size
- Click Create or Create and Run.
- Provide information needed for the Distribution Path. The basic information required are:
Connecting to Oracle Autonomous Database
You can replicate data to an Oracle Autonomous Database instance and capture data from an Oracle Autonomous Database instance by using Oracle GoldenGate Microservices on Oracle Cloud Marketplace. The steps described in this section, streamline the approach for making a remote connection to Oracle Autonomous Database.
To learn about replicating data with an Autonomous Database, see Configure Autonomous Database Capture for Replication in the Oracle GoldenGate Microservices Documentation guide.
Configure the Autonomous Database for Replication
Unlock the Pre-created Oracle GoldenGate User (ggadmin)
Perform the following steps to configure the Autonomous Database for Replication:
- Unlock and change the password for the pre-created Oracle GoldenGate user (ggadmin)
within the Autonomous Data Warehouse. Use any SQL
client tool to unlock the account.
For more details, see Configure the Autonomous Database for Replication in Using Oracle GoldenGate for Oracle Database.
SQL> select * from dba_users order by username; SQL> alter user ggadmin identified by password account unlock;
- Check whether the parameter
enable_goldengate_replicaton
is set to true. If not, then modify the parameter.SQL> select * from v$parameter where name = 'enable_goldengate_replication'; SQL> alter system set enable_goldengate_replication = true scope=both;
Create Target Schema
Complete the following steps to create schema and target objects that can be used in replication. This schema and associated objects does not support DDL replication.
- Create a new application user/schema. This user/schema stores the target objects for replication.
Note:
appadmin
is an example user.SQL> create user appadmin identified by ******** SQL> grant create session, resource, create view, create table to appadmin; SQL> alter user appadmin quota unlimited on data;
- Connect to Oracle Autonomous Database as a user/schema and create your application tables.
Autonomous Database Client Credentials
Obtain the Autonomous Database Client Credentials
Note:
If you do not have administrator access to the Autonomous Database, ask your service administrator to download and provide the credential files to you. Once you have the credential files for your Autonomous Database, you should upload the zip file to the Oracle GoldenGate Compute Node.Perform the following steps to obtain the Oracle Autonomous Data Warehouse Cloud account details:
- Log in to your Oracle Autonomous Data Warehouse Cloud account.
- From the
Instance
page, click the menu option for the Autonomous Database instance and selectService Console
. - Log in to the Service Console using the admin username, and its associated password.
- In the
Service Console
, click theAdministration
tab. - Click
Download Client Credentials
. - Enter a password to secure your credentials zip file and click
Download
. - Save the credentials zip file to your local system.
Move Client Credentials to Oracle GoldenGate Compute Node
In order to establish a connection from Oracle GoldenGate to the Autonomous Data Warehouse, you need to move the client credentials to Oracle GoldenGate Compute Node. The following steps will illustrate how to move the credential zip file from your machine to Oracle GoldenGate Compute Node.
- Connect to the Oracle GoldenGate Classic Compute Node using SSH and
opc
user credentials.ssh -i <private_key> opc@<public_ip_address>
- Create a staging directory and grant the essential permissions and then exit the session.
$ mkdir stage $ exit
- Copy the credentials zip file to the Oracle GoldenGate Classic Compute
Node.
$ scp ./<credential_file>.zip opc@<public_id_address>:~/stage
- Connect to the Oracle GoldenGate Classic Compute
Node.
ssh -i <private_key> opc@<public_ip_address>
- Verify whether the credentials zip file is available in the stage location.
$ cd ~/stage $ ls -ltr
Configure Oracle Goldengate Compute Node with Autonomous Client Credentials
After moving the ADWC Client Credentials to the Oracle GoldenGate Compute Node, you have to install the necessary files and ensure you have a connection to the Autonomous Data Warehouse. The following steps will help you configure the required SQL*Net components:
- Log in to the Oracle GoldenGate Classic Compute Node using SSH and the
opc
user credentials.ssh -i <private_key> opc@<public_ip_address>
- Unzip the client credentials file into a temporary
directory.
unzip ./<credential_file>.zip -d ./client_credentials
- Copy the
sqlnet.ora
andtnsnames.ora
files to the location of your TNS_ADMIN.$ cd ~/stage/client_credentials $ cp ./sqlnet.ora /u02/deployments/<deployment>/etc $ cp ./tnsnames.ora /u02/deployments/<deployment>/etc
Note:
If you want your networking directory structure to be consistent with other Oracle products, you must append/network/admin
to the directory structure. For this you have to change the environment variableTNS_ADMIN
. For more information on Local Naming Parameters, refer to Database Net Services Reference. - Edit the
sqlnet.ora
file and replace the directory parameter with the location of the information pointing to the location where the client credentials were unzipped.$ cd /u02/deployments/<deployment>/etc $ vi ./sqlnet.ora
Change
?/network/admin
to/home/opc/stage/client_credentials
. - For testing purposes, set the
TNS_ADMIN
andORACLE_HOME
environment variables at the operating system level.Note:
The Oracle GoldenGate Deployment(s) use theORACLE_HOME
andTNS_ADMIN
environment variables that are set per deployment.$ export ORACLE_HOME=/u01/app/client/<oracle version> $ export TNS_ADMIN=/u02/deployments/<deployment>/etc
- Test the connection to Autonomous Data Warehouse by connecting to one of the entries in the
tnsnames.ora
file.$ cd $ORACLE_HOME/bin $ ./sqlplus appadmin/**********@orcladw_low
Configure Oracle GoldenGate Microservices for Replication
Perform the following steps for establishing a successful connection to the Autonomous Database with Oracle GoldenGate Microservices.
Add Oracle GoldenGate Credential to connect to Autonomous Database
To add Oracle GoldenGate Credential details, to connect to Autonomous Database:
- Log in to the
Service Manager
using the password foroggadmin
. - From the
Service Manager
main page, select the hyperlink for the port number associated with theAdministration Service
. - Open the context menu in the top left corner of the
Overview
page. - From the context menu, select
Configuration
. - From the
Database
tab, click the plus ( + ) icon, to add a new credential. - Provide the following information and click
Submit
.Credential Domain: [Defaults to OracleGoldenGate] Credential Alias: [Name of the Alias] User ID: ggadmin@<adw_tnsnames_reference> Pasword: [Password for ggadmin] Verify Password: [Password for ggadmin]
- Test the connection to the Autonomous Database by clicking the
Log in Database
icon after the credential has been added.