A distributed database is a collection of shards.

A distributed database is a single logical Oracle Database that is horizontally partitioned across a pool of physical Oracle Databases (shards) that share no hardware or software.

Each shard in the distributed database is an independent Oracle Database instance that hosts subset of a distributed database's data. Shared storage is not required across the shards.

Shards can be hosted anywhere an Oracle database can be hosted. Oracle Globally Distributed Database supports all of the deployment choices for a shard that you would expect with a single instance or clustered Oracle Database, including on-premises, any cloud platform, Oracle Exadata Database Machine, virtual machines, and so on.

Shards can all be placed in one region or can be placed in different regions. A region in the context of Oracle Globally Distributed Database represents a data center or multiple data centers that are in close network proximity.

Shards can be replicated for high availability and disaster recovery with Oracle Globally Distributed Database native replication (Raft-based) or Oracle Data Guard.

A shard catalog is an Oracle Database that supports automated shard deployment, centralized management of a distributed database, and multi-shard queries.

A shard catalog serves following purposes

• Serves as an administrative server for entire distributed database

• Stores a gold copy of the database schema

• Manages multi-shard queries with a multi-shard query coordinator

• Stores a gold copy of duplicated table data

The shard catalog is a special-purpose Oracle Database that is a persistent store for distributed database configuration data and plays a key role in centralized management of a distributed database. All configuration changes, such as adding and removing shards and global services, are initiated on the shard catalog. All DDLs in a distributed database are processed by connecting to the shard catalog.

The shard catalog also contains the primary copy of all duplicated tables in a distributed database. The shard catalog uses materialized views to automatically replicate changes to duplicated tables in all shards. The shard catalog database also acts as a query coordinator used to process multi-shard queries and queries that do not specify a sharding key.

Multiple shard catalogs can be deployed for high availability purposes. Using Oracle Data Guard for shard catalog high availability is a recommended best practice.

At run time, unless the application uses key-based queries, the shard catalog is required to direct queries to the shards. Sharding key-based transactions continue to be routed and processed by the distributed database and are unaffected by a catalog outage.

During the brief period required to complete an automatic failover to a standby shard catalog, downtime affects the ability to perform maintenance operations, make schema changes, update duplicated tables, run multi-shard queries, or perform other operations like add shard, move chunks, and so on, which induce topology change.

Shard directors are network listeners that enable high performance connection routing based on a sharding key.

Oracle Database 12c introduced the global service manager to route connections based on database role, load, replication lag, and locality. In support of Oracle Globally Distributed Database, global service managers support routing of connections based on data location. A global service manager, in the context of Oracle Globally Distributed Database, is known as a shard director.

A shard director is a specific implementation of a global service manager that acts as a regional listener for clients that connect to a distributed database. The director maintains a current topology map of the distributed database. Based on the sharding key passed during a connection request, the director routes the connections to the appropriate shard.

For a typical distributed database, a set of shard directors are installed on dedicated low-end commodity servers in each region. To achieve high availability and scalability, deploy multiple shard directors. You can deploy up to five shard directors in a given region.

The following are the key capabilities of shard directors:

• Maintain runtime data about distributed database configuration and availability of shards

• Measure network latency between its own and other regions

• Act as a regional listener for clients to connect to a distributed database

• Manage global services

• Perform connection load balancing

A global service is a database service that is use to access data in an Oracle Globally Distributed Database.

A global service is an extension to the notion of the traditional database service. All of the properties of traditional database services are supported for global services. For distributed databases, additional properties are set for global services — for example, database role, replication lag tolerance, region affinity between clients and shards, and so on.

For a read-write transactional workload, a single global service is created to access data from any primary shard in a distributed database. For highly available shards using Oracle Data Guard, a separate read-only global service can be created.

The GDSCTL command-line utility is used to configure, deploy, monitor, and manage an Oracle Globally Distributed Database. Oracle Enterprise Manager Cloud Control can also be used for monitoring and management.

Like SQL*Plus, GDSCTL is a command-line utility with which you can control all stages of a distributed database's life cycle. You can run GDSCTL remotely from a different server or laptop to configure and deploy a distributed database topology, and then montior and manage your distributed database.

GDSCTL provides a simple declarative way of specifying the configuration of a distributed database and automating its deployment. Only a few GDSCTL commands are required to create a distributed database.

You can also use Cloud Control for distributed database monitoring and life cycle management if you prefer a graphical user interface. With Cloud Control you can monitor availability and performance, and you can make changes to a distributed database configuration, such as add and deploy shards, services, and shard directors.