Using Wallets

Beginning with Oracle Database 21c, Oracle wallets created for a distributed database are an important part of any deployment. All primary databases and their replicas within the distributed database configuration must have a sharding-specific wallet file present to ensure proper operation.

These wallets are created during the deployment of a distributed database and enable encrypted data to be sent between the shard catalog and individual shards. The process by which the wallets are created establishes a trust relationship between the different components of a distributed database deployment and prevents unauthorized operations from occurring on a shard.

The wallets themselves are created on the shard catalog and any shard catalog replicas when the GDSCTL command CREATE SHARDCATALOG is issued, and the wallets are created on the shards when the GDSCTL command DEPLOY is issued.

After a successful deployment, the wallet files contain information needed for shard catalogs and shards to connect to one another to perform operations such as DDL processing, user context propagation, and the passing of other sensitive data. The information stored in the wallet includes sharding-specific encryption and decryption keys, connect strings, and encrypted passwords. Any command issued from GDSCTL or SQL*Plus which changes this data will automatically cause the wallet to be updated with the new information.

Compatibility and Migration from Oracle Database 19c

For existing Oracle Globally Distributed Database configurations which are being upgraded from a previous Oracle Database release, perform the steps in Post-Upgrade Steps for Oracle Globally Distributed Database 21c after the database upgrade.

Locating the Wallet

The location of the wallet files is under the directory specified by the wallet_root database initialization parameter. If wallet_root is not set before issuing CREATE SHARDCATALOG or DEPLOY, then wallet_root is set to $ORACLE_BASE/admin/db_unique_name on the shard catalog or shards, respectively.

For example, assume the following after logging into the shard catalog or into a shard.

SQL> select guid from v$pdbs where con_id = sys_context('userenv','con_id');

GUID
--------------------------------
C23E7C78D5B77D50E0537517C40ACE4A

SQL> select value from v$parameter where name='wallet_root';

VALUE
--------------------------------------------------------------------------------
your-path-to-keystore

Given these values, the sharding-specific wallet file name is your-path-to-keystore/C23E7C78D5B77D50E0537517C40ACE4A/shard/cwallet.sso.

Wallets on Shard Catalog Replicas

If a standby database is created as a replica of the shard catalog, the shard wallet for the catalog must be manually copied from the primary shard catalog wallet. Find the location of the primary wallet using the above method, and make a copy to the correct location on the standby shard catalog database.

Note:

The value of wallet_root may be different on the standby shard catalog, and may not be set. Remember to set the value of wallet_root before copying the wallet to the standby location

The wallet only exists on the primary shard catalog after the GDSCTL create shardcatalog command is run. If a standby shard catalog database is created before running create shardcatalog, then first run create shardcatalog to create the shard wallet on the primary shard catalog, then copy the wallet to the standby shard catalog.

The shard catalog database also requires a wallet for CDB$ROOT. When copying and backing up shard wallets for a shard catalog database, you should also copy the shard wallet for CDB$ROOT regardless of which PDB is being used for the shard catalog.

Wallet Life Cycle Management

Once a distributed database has been deployed, it is crucial that the shard wallet is maintained throughout the life cycle of the shard catalog, the shards, and their replicas. Specifically, the shard wallet should be included in all backup and restore operations for each database, just as if it were a database data file.

Likewise, if a PDB is cloned, relocated, or otherwise moved, then the shard wallet should accompany the PDB to its new location. Note that in the case of PDB cloning specifically, the GUID for the PDB changes during the cloning operation, and therefore the path to the wallet will change as described above.

Updating a Wallet

If the shard wallet becomes lost, out of date. or is no longer accessible, a newly populated wallet can be created using the following GDSCTL command:

gdsctl sync database -database shard_name

Attempting to perform certain operations when the wallet is not present, or its contents are out of date, results in one or more of the following errors.

ORA-03873: unable to encrypt DDL statement with error ...

ORA-03874: unable to encrypt GSMUSER password with error ...

ORA-03876: error ... when attempting to generate a temporary key to add new shards

ORA-03894: "Failed to send keys to shard %s with error ...."

ORA-03896: Unable to load the sharding wallet successfully.

ORA-00600: internal error code, arguments: [gwsec_get_latest_key]