Task 1: Configure TDE Encryption

To enable the encryption of backup sets, a user having ADMINISTER KEY MANAGEMENT or SYSKM privilege needs to perform the following actions on all container databases (CDBs) at the root level for the shards and the shard catalog:

1. Configure the keystore location and type by setting the WALLET_ROOT and TDE_CONFIGURATION initialization parameters.

   1. First check if WALLET_ROOT points to an existing file location.

      If necessary, you can use the following SQL*Plus statement to change it.

      ALTER SYSTEM SET WALLET_ROOT=wallet-root-location SCOPE=BOTH 

   2. Specify the keystore type.

      ALTER SYSTEM SET TDE_CONFIGURATION="KEYSTORE_CONFIGURATION=FILE" SCOPE=BOTH;

   3. Restart the database.

   Note the following:

   • The WALLET_ROOT parameter specifies the top directory for many different software keystores (such as TDE, Oracle Enterprise User Security (EUS), TLS). For TDE, the directory for automated discovery is WALLET_ROOT/tde.

   • In releases older than 19c, the SQLNET.ENCRYPTION_WALLET_LOCATION parameter was used to define the keystore directory location. This parameter has since been deprecated. Oracle recommends that you use the WALLET_ROOT static initialization parameter and TDE_CONFIGURATION dynamic initialization parameter instead.
   • If the value of parameter TDE_CONFIGURATION is KEYSTORE_CONFIGURATION=FILE, software keystore will be used. Two alternative keystore type values are KEYSTORE_CONFIGURATION=OKV|HSM for Oracle Key Vault or hardware security module keystores.

2. Create a password-protected software keystore.

   ADMINISTER KEY MANAGEMENT CREATE KEYSTORE 'wallet-root-location/tde'
    IDENTIFIED BY keystore_password;

   Note that wallet-root-location should be the same value as the initialization parameter WALLET_ROOT.

3. Open the software keystore.

   ADMINISTER KEY MANAGEMENT SET KEYSTORE OPEN
    IDENTIFIED BY keystore_password CONTAINER=ALL;

4. Set the TDE encryption master key.

   ADMINISTER KEY MANAGEMENT SET KEY
    IDENTIFIED BY keystore_password WITH BACKUP CONTAINER=ALL;

See also: Configuring a Software Keystore

Task 2: Enable Encryption

To enable the encryption of backup sets, specify an encryption algorithm in the GDSCTL CONFIG BACKUP command using option -encryption. For example:

GDSCTL> CONFIG BACKUP … -encryption shard-list:encryption-algorithm

You can get a list of supported algorithms from column ALGORITHM_NAME in V$RMAN_ENCRYPTION_ALGORITHMS. When an algorithm is given, the CONFIG BACKUP command invokes the following RMAN scripts after registering a backup target database in the recovery catalog:

CONFIGURE ENCRYPTION FOR DATABASE ON; 
CONFIGURE ENCRYPTION ALGORITHM TO encryption_algorithm;

Disabling Backup Set Encryption

To disable encryption, you can set the CONFIG BACKUP -encryption option to OFF.

When OFF is specified, the encryption on the shards and/or shard catalog in shard-list is disabled. For example:

GDSCTL> CONFIG BACKUP … -encryption shard-list:OFF

Then the GDSCTL invokes the following RMAN script:

CONFIGURE ENCRYPTION FOR DATABASE OFF;

Set the Backup Destination in the Distributed Database Backup Configuration

In the following example, Object Storage is set as the backup destination of shardspace dbs1.

GDSCTL> config backup
 –shard dbs1
 -rccatalog rccatalog_connect_string
 -catpwd password
 -cdb catcdb_connect_string
 -encryption dbs1:AES256 
 -destination dbs1:”CONFIGURE DEFAULT DEVICE TYPE TO SBT”:”CONFIGURE CHANNEL DEVICE TYPE SBT
 PARMS='SBT_LIBRARY=/orclhome/lib/libopc.so,SBT_PARMS=(OPC_PFILE=/orclhome/dbs/opct1.ora)’” 

As shown above, Object Storage requires the following CONFIG BACKUP parameter settings in addition to the standard options:

• Enable encryption on the shardspace with option -encryption. To support the Oracle Object Storage as a backup destination, the backup sets must be encrypted.

• Specify the following in option -destination using RMAN device configuration and channel configuration statements.

  • SBT_TAPE as the backup default device

  • SBT library file and OPC configuration file locations in the backup channel parameters, using the following template:

    CONFIGURE CHANNEL DEVICE TYPE sbt 
       PARMS='SBT_LIBRARY=location-of-SBT-library-for-backup-module, 
       SBT_PARMS=(OPC_PFILE=location-of-configuration-file)'; 

Set the Backup Destination in the Distributed Database Backup Configuration

The following example shows how to issue GDSCTL command CONFIG BACKUP to set up a backup configuration using the Recovery Appliance “ra_east” as the backup destination of a distributed database:

GDSCTL> config backup
 –shard ALL
 -cdb catcdb_connect_string
 -zdlra_catalog username@ra_east
 -zdlra_vpc CATALOG:catalog_vpc_username
 -zdlra_vpc SH1:shard_vpc_username 
 -zdlra_policy CATALOG:catalog_protection_policy_name
 -zdlra_policy SH1:shard_protection_policy_name
 -zdlra_space CATALOG:20G
 -zdlra_space SH1:2T
 -zdlra_lib_dir CATALOG:?/lib
 -zdlra_lib_dir SH1:/u01/app/oracle/product/19.0.0/dbhome_1/lib

The -zdlra_* parameters used in the example are explained below:

See Global Data Services Control Utility (GDSCTL) Command Reference in Global Data Services Concepts and Administration Guide for details about other CONFIG BACKUP parameters.

Real-Time Redo Transport Post-Configuration Requirements

If the real-time redo transport is required, perform the following actions after running CONFIG BACKUP:

• Add the location of wallets used by redo transport to sqlnet.ora and set SQLNET.WALLET_OVERRIDE to TRUE.

  WALLET_LOCATION= 
   (SOURCE=(METHOD=FILE)(METHOD_DATA= 
   (DIRECTORY= wallet_location))) 
  SQLNET.WALLET_OVERRIDE=TRUE 

• Reboot the primary databases of the shard catalog and shards, so the changes to sqlnet.ora can take effect.

• In the shard catalog and shards (primary and standby databases for all), change the initialization parameter REDO_TRANSPORT_USER to the redo transport user (which uses the same user name as the virtual private catalog (VPC) user).

  For example:

  alter system set "redo_transport_user"=" c##rman1"; 

• You can run a manual backup once, or wait until the first automatic backup runs, then the real-time redo transport is active.

  Once active, the following query on Recovery Appliance should return TRUE:

  select NZDL_ACTIVE
  from DBMS_RA. RA_DATABASE 
  where DB_UNIQUE_NAME='DB_UNIQUE_NAME-of-the-protected-database' 

Set the Backup Destination in the Distributed Database Backup Configuration

The following example shows how to issue GDSCTL command CONFIG BACKUP to set up a backup configuration using the Amazon S3 as the backup destination of shardspace DBS1, and enabling backup set encryption.

GDSCTL> config backup
 –shard dbs1
 -rccatalog rccatalog_connect_string
 -catpwd password
 -cdb catcdb_connect_string
 -encryption dbs1:AES256 
 -destination dbs1: "CONFIGURE DEFAULT DEVICE TYPE TO SBT":
"CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' 
PARMS 'SBT_LIBRARY=/u01/app/oracle/product/19.0.0/dbhome_1/lib/libosbws.so 
ENV=(OSB_WS_PFILE=/u01/app/oracle/product/19.0.0/dbhome_1/dbs/osbwssales.ora)'"

To use Amazon S3 as a backup destination, you must provide the following information when writing channel config statement in option -destination.

• The path to shared library libra.so
• The location of the configuration file used by RMAN

  The template for the CONFIGURE CHANNEL command is:

  CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE'
     PARMS 'SBT_LIBRARY=secure-backup-library-location 
     ENV=(OSB_WS_PFILE=configuration-file-location)'

  For more information on RMAN channel settings related to Amazon S3, See Configuring RMAN SBT Channels for Recovery Appliance in Zero Data Loss Recovery Appliance Protected Database Configuration Guide.

All backup jobs are initially disabled. They can be enabled by running the GDSCTL ENABLE BACKUP command.

GDSCTL> ENABLE BACKUP

When not specified, ENABLE BACKUP enables the backup on all shards. You can optionally list specific shards, shardspaces, or shardgroups on which to enable the backup.

GDSCTL> ENABLE BACKUP -shard dbs1

The DISABLE BACKUP command disables an enabled backup.

GDSCTL> DISABLE BACKUP -shard dbs1

Monitoring an Automated Backup Job

Because full logging is enabled for the automated backup jobs, DBMS Scheduler writes job processing details in the job log and views. The Scheduler job log and views are your basic resources and starting point for monitoring the automated backups. Note that although the DBMS Scheduler makes a list of job state change events available for email notification subscription. This capability is not used for distributed database automated backups.

You can use the GDSCTL command LIST BACKUP to view the backups and find out whether backups are created at the configured backup job repeat intervals.

Automated backups are not delayed by chunk movement in the distributed database, so the backup creation times should be close to the configured backup repeat intervals and the backup start time.

Monitoring an On-Demand Backup Job

Internally, on-demand backup jobs are also started by DBMS Scheduler jobs on the database servers. The names of the temporary jobs are prefixed with tag MANUAL_BACKUP_JOB_. On-demand backups always run in the same session that GDSCTL uses to communicate with the database server. Failures from the job are sent directly to the client.

Using DBMS Scheduler Jobs Views

The automated backup jobs only run on the primary shard catalog database and the primary shards. To check the backup job details for a specific target database, connect to the database, or its primary database if the database is in a Data Guard configuration, using SQL*PLUS, and query the DBMS Scheduler views *_SCHEDULER_JOB_LOG and *_SCHEDULER_JOB_RUN_DETAILS based on the job names.

The names of the two automated backup jobs are AUTOMATED_SDB_LEVEL0_BACKUP_JOB and AUTOMATED_SDB_LEVEL1_BACKUP_JOB.

You can also use the GDSCTL command STATUS BACKUP to retrieve the job state and run details from these views. See Viewing Backup Job Status for more information about running STATUS BACKUP.

The job views only contain high level information about the job. For job failure diagnosis, you can find more details about the job in the RDBMS trace files by grepping the job names.

If no errors are found in the job, but still no backups have been created, you can find the PIDs of the processes that the jobs have created to run RMAN for the backups in the trace files, and then look up useful information in the trace files associated with the PIDs.

Using Backup Command Output

This option is only available for on-demand backups.

When you start on-demand backups with GDSCTL RUN BACKUP, you can specify the -sync command option. This forces all backup tasks to run in the foreground, and the output from the internally launched RMAN on the database servers is displayed in the GDSCTL console.

The downside of running the backup tasks in the foreground is that the tasks will be run in sequence, therefore the whole backup will take more time to complete.

See the GDSCTL reference in Oracle Database Global Data Services Concepts and Administration Guide for detailed command syntax and options.

Because the parameters -destination and -starttime can appear more than once in CONFIG BACKUP command line for different shards and backup configuration can be done more than once, multiple items could be listed in each of the Backup destinations and Backup start times sections. The items are listed in the same order as they are specified in the CONFIG BACKUP command line and the order the command is repeatedly run.

To view an existing backup configuration, run CONFIG BACKUP, as shown here.

GDSCTL> CONFIG BACKUP;

If a distributed database backup has not been configured yet, the command output will indicate it. Otherwise the output looks like the following:

GDSCTL> config backup
Recovery catalog database user: rcadmin
Recovery catalog database connect descriptor: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=den02qxr)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=cdb6_pdb1.example.com)))
Catalog database root container user: gsm_admin
Catalog database root container connect descriptor: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=den02qxr)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=v1908.example.com)))
Backup retention policy in days: 14
Level 0 incremental backup repeat interval in minutes: 10080
Level 1 incremental backup repeat interval in minutes: 1440
Level 1 incremental backup type : DIFFERENTIAL
Backup target type: STANDBY
Backup destinations:
catalog::channel device type disk format '/tmp/rman/backups/%d_%u'
dbs1,dbs2:device type disk parallelism 2:channel 1 device type disk format '/tmp/rman/backups/1/%u';channel 2 device type disk format '/tmp/rman/backups/2/%u'
catalog::configure channel device type disk format '/tmp/rman/backups/%d_%u'
dbs1,dbs2:configure device type disk parallelism 2:configure channel 1 device type disk format '/tmp/rman/backups/1/%u';configure channel 2 device type disk format '/tmp/rman/backups/2/%u'
Backup start times:
all:00:00

CONFIG BACKUP Output for Multiple Recovery Catalogs

The output of CONFIG BACKUP issued without any parameters on a configuration with multiple recovery catalogs prints the recovery catalog users and connect identifiers for different shards and shard catalog. The last used recovery catalog will be printed as well. For example:

GDSCTL> config backup
...  
Recovery catalog database user:  
last::rcadmin_west 
shard_east::rcadmin_east 
shard_west::rcadmin_west  
Recovery catalog databaseconnect identifier:   
last::<theconnect identifier of rc_west> 
shard_east::<theconnect identifier of rc_east> 
shard_west::<theconnect identifier of rc_west> 
...

You can specify a list of shards to list backups for in the command. You can also list backups usable to restore the control files of the listed databases and list backups for standby shards.

The following example shows the use of the command to list the backups from shard cdb2_pdb1 recoverable to restore point BACKUP_BEFORE_DB_MAINTENANCE.

GDSCTL> LIST BACKUP -shard cdb2_pdb1 -restorepoint BACKUP_BEFORE_DB_MAINTENANCE

If option -controlfile is used, LIST BACKUPS will only list the backups usable to restore the control files of the specified shards. If option -summary is used, the backup will be listed in a summary format.

GDSCTL> list backup -shard shd1,shd2 -controlfile -summary 

By default, the command displays the job run details of all the runs that the automated backup jobs have had from 30 days ago in the specified shards. If the job run details for different periods are needed, options -start_time and -end_time must be used.

Run STATUS BACKUP as shown in the following examples.

The following STATUS BACKUP command example lists the job state and all job run details from the SDB catalog and the primary shard “rdbmsb_cdb2_pdb1”:

GDSCTL> status backup -catpwd -shard catalog,rdbmsb_cdb2_pdb1;
"GSMCATUSER" password:***
 
Retrieving scheduler backup job status for database "rdbms" ...
Jobs:
  Incremental Level 0 backup job is enabled
    Job schedule start time: 2020-07-27 00:00:00.000 -0400
    Job repeat interval: freq=daily;interval=1
  Incremental Level 1 backup job is enabled
    Job schedule start time: 2020-07-27 00:00:00.000 -0400
    Job repeat interval: freq=minutely;interval=60
  Global restore point create job is enabled
    Job schedule start time: 2020-07-27 23:59:55.960 -0400
    Job repeat interval: freq=hourly
 
Run Details:
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 14:00:00.177 -0400
    Job run slave process ID: 9023
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 22:00:01.305 -0400
Job run slave process ID: 59526
…
Global restore point create job status: SUCCEEDED
    Job run actual start time: 2020-07-27 15:28:37.603 -0400
    Job run slave process ID: 44227
  …
  Global restore point create job status: SUCCEEDED
    Job run actual start time: 2020-07-27 17:28:38.251 -0400
    Job run slave process ID: 57611
  
Retrieving scheduler backup job status for database "rdbmsb_cdb2_pdb1" ...
Jobs:
  Incremental Level 0 backup job is enabled
    Job schedule start time: 2020-07-28 00:00:00.000 -0400
    Job repeat interval: freq=daily;interval=1
  Incremental Level 1 backup job is enabled
    Job schedule start time: 2020-07-28 00:00:00.000 -0400
    Job repeat interval: freq=minutely;interval=60
 
Run Details:
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 14:00:00.485 -0400
    Job run slave process ID: 9056
  …
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-27 14:33:42.702 -0400
    Job run slave process ID: 9056
  Incremental Level 0 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-27 00:00:01.469 -0400
    Job run slave process ID: 75176

The following command lists the scheduler backup job state and the details of the job runs in the time frame from 2020/07/26 12:00:00 to 07/27 00:00 from the SDB catalog and the primary shard “rdbmsb_cdb2_pdb1”:

GDSCTL> status backup -start_time "2020-07-26 12:00:00" -end_time "2020-07-27 00:00:00" -catpwd -shard catalog,rdbmsb_cdb2_pdb1;
"GSMCATUSER" password:***

Retrieving scheduler backup job status for database "rdbms" ...
Jobs:
  Incremental Level 0 backup job is enabled
    Job schedule start time: 2020-07-27 00:00:00.000 -0400
    Job repeat interval: freq=daily;interval=1
  Incremental Level 1 backup job is enabled
    Job schedule start time: 2020-07-27 00:00:00.000 -0400
    Job repeat interval: freq=minutely;interval=60
  Globa1 restore point create job is enabled
    Job schedule start time: 2020-07-27 23:59:55.960 -0400
    Job repeat interval: freq=hourly
 
Run Details:
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 14:00:00.177 -0400
    Job run slave process ID: 9023
  …
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 23:50:00.293 -0400
    Job run slave process ID: 74171
  Globa1 restore point create job status: SUCCEEDED
    Job run actual start time: 2020-07-26 14:28:38.263 -0400
    Job run slave process ID: 11987
  …
  Globa1 restore point create job status: SUCCEEDED
    Job run actual start time: 2020-07-26 23:28:37.577 -0400
    Job run slave process ID: 69451
 
Retrieving scheduler backup job status for database "rdbmsb_cdb2_pdb1" ...
Jobs:
  Incremental Level 0 backup job is enabled
    Job schedule start time: 2020-07-28 00:00:00.000 -0400
    Job repeat interval: freq=daily;interval=1
  Incremental Level 1 backup job is enabled
    Job schedule start time: 2020-07-28 00:00:00.000 -0400
    Job repeat interval: freq=minutely;interval=60
 
Run Details:
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 14:00:00.485 -0400
    Job run slave process ID: 9056
  Incremental Level 1 backup job status: SUCCEEDED
    Job run actual start time: 2020-07-26 22:11:50.931 -0400
    Job run slave process ID: 9056

The following example validates the backups of the control files from the shard catalog databases recoverable to restore point BACKUP_BEFORE_DB_MAINTENANCE.

GDSCTL> VALIDATE BACKUP -shard shd1,shd2 -controlfile -restorepoint BACKUP_BEFORE_DB_MAINTENANCE

Backup validation for shards are done one shard a time sequentially.

The DELETE BACKUP command deletes the distributed database backups identified with specific tags from the recovery repository. It deletes the records in the recovery database for the backups identified with the provided tags, and, if the media where the files are located is accessible, the physical files from the backup sets from those backups. This is done for each of the target databases. You will be prompted to confirm before the actual deletion starts.

The following is an example of deleting backups with tag odb_200414205057124_0400 from shard cdb2_pdb1.

GDSCTL> DELETE BACKUP -shard cdb2_pdb1 -tag ODB_200414205057124_0400
"GSMCATUSER" password:

This will delete identified backups, would you like to continue [No]?y

Deleting backups for database "cdb2_pdb1" ...

These restore points are created at a common SCN across all of the primary databases in the distributed database. The restore points created in the primary databases are automatically replicated to the Data Guard standby databases. When the databases are restored to this common SCN, the restored distributed database is guaranteed to be in a consistent state.

To create the global restore point, run the GDSCTL command CREATE RESTOREPOINT as shown here.

GDSCTL> CREATE RESTOREPOINT 

The global restore point creation job is configured on the shard catalog database. The name of the job is AUTOMATED_SDB_RESTOREPOINT_JOB. Full logging for this job is enabled.

You can optionally enter a name for the restore point by using the -name option as shown here.

GDSCTL> CREATE RESTOREPOINT -name CUSTOM_SDB_RESTOREPOINT_JOB

The job is initially disabled, so you must use GDSCTL ENABLE BACKUP to enable the job. The job runs every hour and the schedule is not configurable.

To list all global restore points, run LIST RESTOREPOINT.

GDSCTL> LIST RESTOREPOINT

This command lists all of the available global restore points in the distributed database that were created during the specified time period with SCNs (using the -start_scn and -end_scn options) in the specified SCN interval (using the -start_time and -end_time options).

The following command lists the available restore points in the distributed database with the SCN between 2600000 and 2700000.

GDSCTL> LIST RESTOREPOINT -start_scn 2600000 -end_scn 2700000

The command below lists the available restore points in the distributed database that were created in the time frame from 2020/07/27 00:00:00 to 2020/07/28 00:00:00.

GDSCTL> LIST RESTOREPOINT -start_time "2020-07-27 00:00:00" -end_time "2020-07-28 00:00:00"

This command is used to restore a shard database to a specific global restore point. It can also be used to restore only the shard database control files.

The typical procedure for restoring a distributed database is:

1. List the available restore points.
2. Select a restore point to validate the backups.
3. Restore the databases to the selected restore point.

You should validate the backups for a shard against the selected restore point to verify that all the needed backups are available before you start to restore the shard to the restore point.

Note that you are not prohibited from restoring the shard catalog or a specific shard to an arbitrary point in time. However, doing so may put that target in an inconsistent state with the rest of the distributed database and you may need to take corrective action outside of the restore operation.

For data file restore, the shards must be in MOUNT state, but if the command is to restore the control files, the shard databases must be started in NOMOUNT state. To bring the databases to the proper states will be a manual step.

To restore the shard database control files, the database must be started in nomount mode. The control files will be restored from AUTOBACKUP. To restore the database data files, the database must be mounted. The shard catalog database must be open for this command to work.

The following example restores the control files of shard cdb2_pdb1 to restore point BACKUP_BEFORE_DB_MAINTENANCE.

GDSCTL> RESTORE BACKUP -shard cdb2_pdb1 -restorepoint BACKUP_BEFORE_DB_MAINTENANCE –controlfile

The restore operation can be done for the shards in parallel. When the restore for the shards happens in parallel, you should not close GDSCTL until the command has finished running, because interrupting the restore operation can result in database corruption or get the distributed database into an inconsistent state.

Backup validation only logically restores the database while RESTORE BACKUP will do both the physical database restore and the database recovery. Therefore, after RESTORE BACKUP is done, usually the restored the databases need to be opened with the resetlogs option.

After the database restore is completed, you should open the database and verify that the database has been restored as intended and it is in a good state.

Shard catalog restore uses GDSCTL RESTORE BACKUP with some different parameters than are used for a shard restore, as shown in the syntax here.

GDSCTL> RESTORE BACKUP -shard CATALOG
    -cdb connect-string
    -catalog_name pdb-name
    -catalog_dbid dbid
    [-scn scn]
    [-controlfile]
    [-restore_only | -recover_only]

Because the shard catalog database must not be open when it is restored, some information obtainable from the catalog that is needed for the database restore will not be available automatically, so it will need to be provided with the RESTORE BACKUP command. This includes:

• A common user with SYSDG privilege in the root container and SYSBACKUP privilege for all containers in the shard catalog CDB and its password
• A connect identifier to the shard catalog CDB root (-cdb connect-string)
• The name of the shard catalog PDB (-catalog_name pdb-name)
• The shard catalog DBID (-catalog_dbid dbid)
• If the shard catalog needs to be restored to a specific global restore point, instead of the name of the global restore point, the associated SCN must be provided for the command (-scn scn)

When a shard is removed from a distributed database configuration, the shard will no longer be included in automated backup jobs, but the artifacts created on the database host when the shard was configured for backup (for example, the backup wallets and the backup jobs) are not deleted.

To remove these artifacts from the shard before it is removed from the distributed database, run CONFIG BACKUP with the -REMOVE option, and provide a list of shards that should be removed, as shown here.

GDSCTL> CONFIG BACKUP -remove -shard shard_list

The command does the following tasks:

• Deletes the shard PDB container-level backup wallet.
• Deletes the DBMS scheduler backup jobs from the shard database.
• If the CDB is not a shared target database by some other distributed database, which happens when multiple distributed databases are placed inside the same set of CDBs, deletes the CDB root container level backup wallet.

This command should only be used if the shard is removed and is not expected to be added back to the same distributed database.

All on-demand backups are level 0 incremental backups. On-demand backups have no impact on the automated backup schedules configured for the shard catalog database and the shards.

Internally, on-demand backups are started by DBMS Scheduler jobs on the database servers. The jobs are created on-the-fly when the on-demand backup command RUN BACKUP is run.

On-demand backup jobs are temporary jobs, and they are automatically dropped after the backups have finished.

The names of the temporary jobs are prefixed with tag MANUAL_BACKUP_JOB_.

GDSCTL> RUN BACKUP -shard dbs1

The -shard option lets you specify a set of shards, shardspaces or shardgroups on which to run the backup. To take an on-demand backup on shardspace dbs1, you can run RUN BACKUP as shown in the example above.

See the GDSCTL reference in Oracle Database Global Data Services Concepts and Administration Guide for detailed command syntax and options.