Create Additional Admin Processes
Creating additional Admin processes is crucial to ensure high availability, failover support, and fault tolerance.
You can continue to administer and monitor your data store even if some of the Storage Nodes hosting the Admin processes become unreachable. You can add additional Admin processes in the data store using the deploy-admin command.
If the original master Admin process (here, the Admin process on node 01) fails or becomes unreachable, one of the other Admin processes is automatically elected as the new master, ensuring continuous administrative availability. For this re-election, quorum is considered as briefly discussed below.
The full availability of the Admin service depends on having a quorum of the total Admin processes available at a given time.
Any operation that modifies the store's configuration (e.g., deploying a new node, changing a parameter, adding a table) is considered a "write" operation to the Admin database. This operation only succeeds if a majority (quorum) of the Admin processes acknowledge and agree to the change. In a 3-Admin setup, 2 must agree. In a 4-admin setup, 3 must agree.
One Admin process is the elected "Master Admin," which coordinates these writes. The others are replicas. The quorum process is also used to elect a new master automatically if the current master fails.
For example, with a 3-Admin setup, the following table describes how failure numbers affect availability:
| Failures | Availability |
| 0 | Full |
| 1 | Full |
| 2 | None (All administrative tasks are blocked) |
| 3 | None (All administrative tasks are blocked) |
If admin quorum is lost (for example, two out of three Admin processes fail), the store can still process data reads and writes, but no administrative commands can be executed until the quorum is restored.
It is strongly recommended that you use the zone replication factor to determine how many Admin processes should exist. This means that the Admin process has the same availability for administrative operations as the data store does for data operations.
As we know, a data store can be deployed to span both primary and secondary zones. Admin processes in the primary zone actively process administrative commands. Admin processes in secondary zones facilitate failure recovery.
If all of the primary zones are lost, the administrator can use the repair-admin-quorum and plan failovercommands to resume operations by converting the secondary zone to a primary zone. But these operations can occur only if an Admin process is available. For this reason, it is recommended to have an Admin process in the secondary zone. For more details, see Repairing a Failed Zone.