Oracle Cloud Infrastructure Documentation

OCI Cache Sharded Clusters

OCI Cache supports two cluster modes, sharded clusters and non-sharded clusters.

Non-sharded clusters are configured with one primary node, and one or more replica nodes, with the data duplicated on each node. With this cluster mode, the data you can store is limited by amount of memory configured for the node, with a maximum of 500 GB of memory per node.

Sharded clusters have three or more shards, with the data split across the shards in the cluster, so that each shard holds part of the data. Each shard is like a cluster, with one primary node, and up to four replica nodes. Sharded clusters supports scenarios where you need to store more data than the 500 GB limit, because while these clusters are still limited to the 500 GB of memory per node, it's really 500 GB per shard.

Sharded clusters don't have primary or replica endpoints at the cluster level like non-sharded clusters do. Instead, each shard has a private endpoint you can use to when connecting to the cluster, see Getting an OCI Cache Sharded Cluster's Connection Details for more information.

Limitations and Considerations for Sharded Clusters

Any Valkey and Redis client libraries that you use to connect to a sharded cluster must support Redis CLUSTER MODE with hostname support.

  • The number of nodes configured per shard must be between 1 and 5, with a maximum of 100 nodes per cluster.
  • The number of shards per cluster must be an uneven number from 3 to 99, though the maximum number of shards depends on the number of nodes configured per shard.
  • The amount of memory per node must be between 2 and 500 GB.
  • Any Redis client libraries that you use to connect to a sharded cluster must support Redis CLUSTER MODE with hostname support.
  • OCI Cache clusters created as non-sharded clusters can't be converted to sharded clusters and vice versa. OCI Cache doesn't provide an automated way to move data between these cluster types.

Configuring Sharded Clusters

When configuring a sharded cluster, you need to consider the requirements to help you decide about the balance between the number of shards, number of nodes per shard, and amount of memory per node. Increasing the number of nodes per shard is a good strategy to increase a cluster's read capacity, but it doesn't help with a cluster's storage, unless you increase the amount of memory per node. It also doesn't help with a cluster's write capacity. To impact a cluster's write capacity, increase the number of shards.

For example, compare the following two scenarios:

  1. The cluster needs to store a large amount of data, but not handle a large amount of writes.
  2. The cluster needs to handle a lot of writes, but not store a large amount of data.

In scenario 1, you would configure the cluster to have a fewer number of shards, with more memory per node, compared to scenario 2. For scenario 2, you would configure the cluster to have a greater number of shards, with less memory per node.

Discovery Endpoint

The discovery endpoint capability is designed for OCI Cache sharded clusters, providing a stable, single endpoint for client applications to connect to the sharded cache cluster. This endpoint abstracts the complexity of managing several shard nodes, allowing client applications to interact with the OCI Cache without needing to be aware of the underlying shard topology.

Each cluster instance features a unique discovery endpoint, which consists of a combination of an IP address and a port number. The discovery endpoint serves as the primary entry point for client connections and is crucial for enabling seamless node discovery and managing the cluster topology.

To find your cluster's discovery endpoint, see Getting a OCI Cache Cluster's Details.
Note

OCI Cache provides the discovery endpoint only for newly created sharded clusters; it does not automatically update existing sharded clusters to include a discovery endpoint.

Using Discovery Endpoint

To connect your Redis or Valkey client (such as Lettuce or Redisson), configure it to use the Fully Qualified Domain Name (FQDN) of the discovery endpoint. This endpoint resolves to a private IP address within your Virtual Cloud Network (VCN) and routes requests to a cluster node.

After connecting, the client can obtain the cluster topology by executing standard Redis commands, such as #CLUSTER SLOTS or #CLUSTER NODES. These commands return information including node IDs, roles, FQDNs or IP addresses, and slot ranges.

After this, the client can perform the following:

  • Establish direct connections to the relevant nodes using their FQDNs or IP addresses.
  • Execute Redis or Valkey commands (for example, GET, SET).
  • Maintain up-to-date routing by periodically refreshing the topology through the discovery endpoint.

Most of the procedures for working with sharded clusters are the same as for working with non-sharded clusters, however some procedures are different. See the following for details specific to sharded clusters: