3 Installation and Configuration
Overview
The Oracle Global Data Services (GDS) framework consists of core, client-side, and database-side components that work together to setup, manage and orchestrate data services efficiently. At the core, the Global Service Manager (GSM) and GDS Catalog, handle intelligent routing, dynamic load balancing, service failover, and store configuration metadata. These components may use redundancy for high availability. On the client side, Oracle GDS integrates with an Oracle-enabled client and connection pool to support features like Fast Connection Failover (FCF) and intelligent connection and work request routing. On the database side, Oracle Notification Service (ONS) provides real-time event notifications to GSM, while the gsmuser schema enables local service orchestration. These components collectively ensure a scalable, resilient, and optimized global services management framework.
Some components of the framework are installed when you install Oracle Database. Other components require that you perform certain tasks using the Global Data Services control utility (GDSCTL).
3.1 Oracle GDS Capacities and Requirements
Capacity planning and sizing should be done before deployment, and periodically afterward, to ensure that there are sufficient system resources to meet application performance requirements. Capacity planning needs to accommodate growth or consolidation of databases, additional application workloads, additional processes, or anything that strains existing system resources. The table below lists GDS sizing capacities and requirements.
A Single GDS Manages | GDS Databases |
|
|
3.2 Planning GDS Deployment
Configure network connectivity for your GDS setup using the following guidelines:
- Global Service Manager (GSM) Listener and ONS Port Accessibility: All GDS pool databases must be able to communicate with every Global Service Manager’s Listener (default port: 1522) and ONS ports (default: 6123 for local ONS and 6234 for remote ONS) in both directions. These ports must also be accessible from the Application/Client tier, all GDS pool databases, the GDS catalog database, other Global Service Managers in the deployment.
- TNS Listener Port Configuration for Pool Databases: The TNS Listener port (default: 1521) for each GDS pool database must be open in both directions to enable communication with all Global Service Managers, and the GDS catalog database.
- GDSCTL Connectivity: If the GDSCTL utility is run from a separate machine, ensure that this machine has direct, bidirectional port access to the TNS Listener port (default: 1521) of GDS catalog database.
- Default ONS Port Usage: On most platforms, the default ports for Oracle Notification Service (ONS) are: 6123 for local ONS, and 6234 for remote ONS.
For detailed information about memory, physical storage, kernel versions and packages required by Global Data Services see: Database Installation Guide for Linux
3.3 GDS Software Installation
The global service manager (GSM) is the central component of the Global Data Services framework, and you must install the global service manager using separate media. No other Oracle software is required to install and run the global service manager.
You can install the global service manager on a system where you have other Oracle products installed, but you must install the global service manager in a separate Oracle home directory. You can install more than one global service manager on a single system, but each global service manager must have a separate Oracle home directory. For performance reasons, depending on the number of databases in your Global Data Services configuration, you may want to deploy the global service manager on a dedicated host.
You must install at least one global service manager for each Global Data Services region. Global service managers can be hosted on physical or virtual environments. For high availability, Oracle recommends installing multiple (typically 3) global service managers in each region running on separate hosts.
Oracle Universal Installer does not currently support installing software on multiple hosts. You must install each global service manager on its respective host.
The Global Data Services administrator installs the global service manager. The Global Data Services administrator's responsibilities include:
-
Administering global service managers
-
Administering the Global Data Services catalog
-
Administering regions and database pools
The Global Data Services administrator must have an operating system user
account on all hosts where global service managers are deployed, and you must run the
installation under that user account. The installation must not be run by a
root
user.
3.3.1 Installing a Global Service Manager
What You Need to Know About Installing a Global Service Manager
The global service manager is the central component of the Global Data Services framework, and you must install the global service manager using separate media. No other Oracle software is required to install and run the global service manager.
You can install the global service manager on a system where you have other Oracle products installed, but you must install the global service manager in a separate Oracle home directory. You can install more than one global service manager on a single system, but each global service manager must have a separate Oracle home directory. For performance reasons, depending on the number of databases in your Global Data Services configuration, you may want to deploy the global service manager on a dedicated host.
You must install at least one global service manager for each Global Data Services region. Global service managers can be hosted on physical or virtual environments. For high availability, Oracle recommends installing multiple (typically 3) global service managers in each region running on separate hosts.
The Global Data Services administrator installs the global service manager. The Global Data Services administrator's responsibilities include:
-
Administering global service managers
-
Administering the Global Data Services catalog
-
Administering regions and database pools
Note:
The Global Data Services administrator must have an operating system
user account on all hosts where global service managers are deployed, and you
must run the installation under that user account. The installation must
not be run by a root
user.
Installing a Global Service Manager:
Note:
After installing the Global Data Services software, it is recommended that the installation be updated to the latest Oracle Database release.3.4 GDS Catalog Database and GDS Catalog Setup
What You Need to Know About Creating the GDS Catalog
Every Global Data Services configuration must have a Global Data Services catalog. The Global Data Services catalog can reside on the same host as a GDS configuration database, but Oracle does not recommend this scenario for large configurations. Oracle recommends that you use Oracle high availability features such as Oracle Real Application Clusters (Oracle RAC) and Oracle Data Guard to protect the Global Data Services catalog against outages.
Global Data Services Catalog Requirements
-
The Global Data Services catalog must reside on an Oracle database that uses a server parameter file (SPFILE).
If you create the Global Data Services catalog in an Oracle RAC database, then Oracle recommends that you set up Single Client Access Name (SCAN) for that database.
-
The Global Data Services catalog must be protected for high availability and disaster recovery.
Note:
Oracle recommends that the Global Data Services administrator does not directly connect to the catalog database, despite having a user account on the catalog database. Global Data Services administrators can use the GDSCTL utility to manage Global Data Services. GDSCTL connects to the Global Data Services catalog with the credentials that the Global Data Services administrator provides when running GDSCTL commands.
For example:
GDSCTL> create gdscatalog -database serv1:1521:catdb.example.com -user gsm_admin
In the preceding example, serv1:1521:catdb.example.com
is an Easy Connect string that contains the host name and port number of the
listener that is used to connect to the database, and
catdb.example.com
is the service name for the Global Data
Services catalog database.
You designate one database as the primary repository for the Global Data Services catalog. You can use existing high availability technologies, such as Oracle RAC, Oracle Data Guard, and Oracle Clusterware, to protect the Global Data Services catalog.
If you use Oracle GoldenGate, then ensure that the Global Data Services catalog gets replicated to a secondary database.
See Also:
create gdscatalog for complete usage information
3.5 Global Data Services Configuration
Oracle Global Data Services (GDS) implements the Oracle Database service model across a set of replicated databases known as a Global Data Services configuration.
3.5.1 Oracle GDS Deployment Steps
The following are the basic steps you would take to implement Global Data Services.
- Install Oracle GDS global service manager software on global service
manager servers.
- Minimum of 1 global service manager for each region
- Recommended 3 global service managers for each region
- Pre-create the Oracle GDS catalog database.
- Set up Oracle GDS Administrator accounts and privileges.
- Configure Oracle GDS.
- Create the GDS catalog and standby databases.
- Add global service managers, regions, pools, databases, and global services.
- Set up client connectivity.
3.5.2 GDS Configuration Example
The following steps describe how to implement Global Data Services.
This example configuration of Global Data Services (GDS) uses an Administrator-managed Oracle RAC database. Administrator-managed deployment means that you configure database services to run on specific instances belonging to a particular database using a preferred and available designation.
Policy-managed deployment is based on server pools, where database services run within a server pool as singletons or uniformly across all of the servers in the server pool. Databases are deployed in one or more server pools, and the size of the server pools determines the number of database instances in the deployment.
- Create and prepare a GDS catalog database.
GDS uses a catalog database to store meta-data relating to the layout and status of the GDS configuration. For maximum availability, Oracle recommends that the GDS catalog database be deployed independently and that Oracle's high-availability features, such as Oracle Real Application Clusters (Oracle RAC) and Oracle Data Guard, be used to protect the catalog database against outages.
- Create the
GSM_ADMIN
user and assign that user theGSMADMIN_ROLE
.Note that by default, the password for both
GSM_ADMIN
,GSMUSER
, andGSMCATUSER
expires after 180 days.SQL> create user gsm_admin identified by password; User created. SQL> grant gsmadmin_role to gsm_admin; Grant succeeded. SQL> exit
- With the environment configured for the global service manager home, use
GDSCTL to create the GDS catalog database with Auto VNCR disabled (Auto VNCR can
cause problems with Oracle RAC deployments).
GDSCTL> create gdscatalog -database gdscat -user gsm_admin -autovncr OFF
- Connect to the catalog database, unlock the
GSMCATUSER
user, and set the password.SQL> alter user gsmcatuser account unlock; User altered. SQL> alter user gsmcatuser identified by password; User altered.
- With the environment configured for the global service manager home, use GDSCTL to
connect to, create, and start the global service manager listeners.
As a best practice, global service manager listeners should reside on hardware separate from that hosting the Oracle Databases in the GDS configuration. The resource requirements for hardware needed to run global service manager listeners are lightweight and can easily be accommodated using virtual machines.
GDSCTL> add gsm -gsm gsm1 -listener 1522 -catalog gdscat "gsmcatuser" password: Create credential oracle.security.client.connect_string1 GSM successfully added GDSCTL>start gsm -gsm gsm1 GSM is started successfully GDSCTL>status Alias GSM1 Version 19.17.0.3.0 Start Date 13-APR-2023 09:40:59 Trace Level off Listener Log File /u01/app/oracle/diag/gsm/hostname/gsm1/alert/log.xml Listener Trace File /u01/app/oracle/diag/gsm/hostname/gsm1/trace/ora_64863_139739749930432.trc Endpoint summary (ADDRESS=(HOST=hostname.example.com)(PORT=1522)(PROTOCOL=tcp)) GSMOCI Version 0.6.11 Mastership Y Connected to GDS catalog Y Process Id 64883 Number of reconnections 0 Pending tasks. Total 0 Tasks in process. Total 0 Regional Mastership TRUE Total messages published 0 Time Zone -04:00 Orphaned Buddy Regions: None GDS region regionora
- With the environment configured for the global service manager home, use GDSCTL to
create a default GDS pool and default
region.
GDSCTL> add gdspool -gdspool sales GDSCTL> add region -region slc GDSCTL> add region -region sca
- With Auto VNCR disabled during GDS catalog creation to avoid issues, use GDSCTL to
add hosts using the
add invitednode
command, using the host name or IP address appropriately.GDSCTL> add invitednode 192.0.2.1 GDSCTL> add invitednode host1.example.com
- Unlock the
GSMUSER
account.Before adding a database to a pool, the database administrator should unlock theGSMUSER
account and give the password to the GDS pool administrator, as shown in the following example.SQL> alter user gsmuser account unlock; User altered. SQL> alter user gsmuser identified by password; User altered.
- Add databases to the GDS pool.
To be part of a GDS pool, a database must use a server parameter file (SPFILE). An Oracle RAC database should also have SCAN set up.
To add a database, connect to the GDS catalog using the GDS pool or GDS administrator credentials. For example, without Data Guard, the following add database command can be used.
Note:
When using Oracle Active Data Guard with GDS, useadd brokerconfig
instead ofadd database
, and then usemodify database
to configure the standby database (see add brokerconfig). The syntax for these commands would be like the following:GDSCTL> add brokerconfig -connect <primary_db> -gdspool <dbpool> -region <dc> -pwd <gsmuser_pwd> GDSCTL> modify database -database <standby_db> -connect <dc> -gdspool <dbpool> -region <dc> -pwd <gsmuser_pwd>
Database instance registration with a global service manager succeeds only when the request originates from a valid node. If a host on which a database resides contains multiple network interfaces, the auto-configuration could register the wrong set of IP addresses, leading to the database registration being rejected.
- Correct any rejected registration and properly discover all database
instances.
If a firewall exists between the global service managers, and the databases and the ports are not opened, the registration fails. From the global service manageralert log, you will see entries similar to the following.
Listener(VNCR option 1) rejected Registration request from destination 192.0.2.2 Listener(VNCR option 1) rejected Registration request from destination 192.0.2.3
You will find that the database object exists in the GDS catalog, but some or all instances associated with specific hosts are missing.
GDSCTL> databases Database: "mts" Registered: Y State: Ok ONS: Y. Role: PRIMARY Instances: 1 Region: slc Registered instances: sales%1
To correct the rejected registration and properly discover all database instances, run
add invitednode
using the rejected IP address listed in the global service manager alert log. - If there is a firewall between the global service managers and the database, then
once the ports have been opened and verified using tnsping, issue the
add invitenode
command as shown here.GDSCTL> add invitednode 192.0.2.3 GDSCTL>databases Database: "mts" Registered: Y State: Ok ONS: Y. Role: PRIMARY Instances: 2 Region: slc Registered instances: sales%1 sales%2
- Create a service on the GDS pool databases.
The GDSCTL
add service
command creates a service on the GDS pool databases.GDSCTL> add service -service sales_sb -preferred_all -gdspool sales -notification TRUE
If this is an Oracle RAC database being added with multiple instances, then you must modify the service to add the database instances.
GDSCTL> modify service -gdspool sales -service sales_sb -database mts -add_instances -preferred mts1,mts2 GDSCTL> modify service -gdspool sales -service sales_sb -database stm -add_instances -preferred stm1,stm2 GDSCTL> start service -service sales_sb -gdspool sales
- Verify that the global service is running.
GDSCTL> services Service "sales_sb.sales.oradbcloud" has 2 instance(s). Affinity: ANYWHERE Instance "sales%1", name: "mts1", db: "mts", region: "slc", status: ready. Instance "sales%2", name: "mts2", db: "mts", region: "slc", status: ready.
3.5.3 GDS Configuration Best Practices
Oracle MAA recommends the following best practices for implementing Global Data Services:
-
Each client communicates using an Oracle-integrated connection pool such as UCP, OCI, or ODP.NET. The connection pools will be notified about any service failovers and load balancing advisory notifications using Fast Application Notification Events.
-
Run three global service managers in each region. Create three global service managers in each region so that if one global service manager goes down, you have two remaining global service managers to provide redundancy. Each global service manager should reside on separate hardware. Global service managers enable connection routing among replicated databases. A global service manager is a stateless, lightweight, and intelligent listener that can repopulate its metadata from the GDS catalog.
-
Protect the GDS catalog database with Oracle Data Guard.The GDS catalog is a small (less than 100 GB) repository that hosts the metadata of the GDS configuration, regions, global service managers, global services, databases, and so on. MAA recommends that you set up a local Data Guard standby database configured with Maximum Availability database protection mode, Data Guard Fast-Start failover, and a remote physical standby database. All GDS catalog standby databases should use Oracle Active Data Guard for the best data protection and reside on separate hardware and storage.