Important Points to Consider Before Configuring High Availability

Because existing data on the designated standby Audit Vault Server is purged during high availability configuration, consider the following points:

• Impact on existing Database Firewalls: All Database Firewalls that are registered with the designated standby Audit Vault Server must be registered again after high availability is configured.

• Impact on existing Audit Vault Agents: There is no impact on the Audit Vault Agents that are registered on the designated primary Audit Vault Server.

  However, all the registered Audit Vault Agents on the designated standby Audit Vault Server must be redeployed on the primary after high availability is configured. See Post High Availability Pairing Steps.

• Impact on existing audit trails and Database Firewall monitoring points: Because audit and network event data that is collected from targets by the designated standby Audit Vault Server is purged during high availability configuration, you must reconfigure these audit trails and Database Firewall monitoring points on the Audit Vault Server after high availability is configured to ensure that these targets continue to be protected.

• From Oracle AVDF 20.9 to 20.12, agentless collection was supported only on standalone, unpaired Audit Vault Servers (AVS). Starting with Oracle AVDF 20.13, agentless collection is supported on both standalone and high availability AVS.

Audit Vault Server High Availability Configuration Process

To configure Audit Vault Servers for high availability, follow this high-level process:

1. Install two standalone Audit Vault Servers to use as the primary and standby servers.

   Best Practice:

   Place the two Audit Vault Servers in two different data centers.
2. Configure the designated standby Audit Vault Server.
3. Configure the designated primary Audit Vault Server.

1. Make a note of the IP address of the designated primary Audit Vault Server.

2. Copy the server certificate of the designated primary Audit Vault Server:

   1. Log in to primary Audit Vault Server console as super administrator.
   2. Click the Settings tab. The Security tab in the left navigation menu is selected by default.
   3. Now, click the Certificate sub tab in the main page.
   4. Click the Server Certificate sub tab.
   5. Click the Copy Certificate button.

3. In the left navigation menu of the designated standby Audit Vault Server, perform these steps:

   1. Select System tab.
   2. Click High Availability link under the Configuration section on the main page.
   3. In the Configure High Availability dialog, the Current status field indicates the status of the current Audit Vault Server, which is Standalone.
   4. In the Configure this server as field, select the Standby server option.

   5. In the expanded Configure High Availability dialog, enter the following settings:

      • Primary server IP address: Enter the IP address of the designated primary Audit Vault Server.
      • Primary server certificate: Paste the certificate that you copied from the designated primary Audit Vault Server.
   6. Click Save. The designated primary Audit Vault Server's IP address and certificate is now saved on the standby Audit Vault Server, and is now ready to be paired.

1. Make a note of the IP address of the designated standby Audit Vault Server.

2. Copy the server certificate from the designated standby Audit Vault Server:

   1. Log in to the designated standby Audit Vault Server console as super administrator.
   2. Click the Settings tab. The Security tab in the left navigation menu is selected by default.
   3. Now click the Certificate sub tab in the main page.
   4. Click Server Certificate sub tab.
   5. Click the Copy Certificate button.

3. Log in to the designated primary Audit Vault Server console as super administrator and perform these steps:

   1. In the left navigation menu, select System tab.
   2. Click High Availability link under the Configuration section in the main page.
   3. In the Configure High Availability dialog, the Current status field indicates the status of the current Audit Vault Server, which is Standalone.
   4. In the Configure this server as field, select the Primary server option.

   5. In the expanded Configure High Availability window, enter the following settings:

      • Standby server IP address: Enter the IP address of the standby Audit Vault Server.
      • Standby server certificate: Paste the certificate that you copied from the standby Audit Vault Server.

   6. Click Initiate Pairing button at the bottom of the dialog, to initiate high availability pairing. To get the updated status, refresh the Audit Vault Server console periodically, as the process can take at least 15 minutes. This process can take longer depending on the amount of data in the repository. When the high availability pairing is complete, the High Availability Status field in the main page displays the current status.

      Note:

      • After high availability pairing is successfully completed, perform all the configuration tasks on the primary Audit Vault Server only. This includes tasks such as downloading the Audit Vault Agent, registering targets and hosts, adding Database Firewalls and monitoring points. To perform tasks like setting system time or changing IP address for the standby Audit Vault Server refer to section Specifying the Server Date, Time, and Keyboard Settings.
      • During high availability pairing, the NFS archive locations pertaining to the primary and standby Audit Vault Servers are mapped. The mapping of these locations is displayed in the primary Audit Vault Server console after high availability pairing is successful.

After high availability pairing is successfully completed, the standby Audit Vault Server console is not accessible. Perform all tasks on the primary Audit Vault Server console. If you attempt to access the standby Audit Vault Server console, it redirects to the primary Audit Vault Server console.

Check High Availability Status Through the Console

1. In the Audit Vault Server console, click the Settings tab.
2. In the left navigation menu, select System.
3. Under the Status section, check the High Availability Status field.

   The possible values are:

   • Standalone - This server is not configured for high availability and is a standalone instance.

   • Primary - This server is currently the primary Audit Vault Server.

   • Disconnected - This primary Audit Vault Server switches to this mode if it detects that the standby Audit Vault Server changed its role to standalone or primary. This indicates that the high availability pairing is broken. Contact Oracle Support for further assistance.

Check High Availability Status Through Commands

1. Log in to the Audit Vault Server through SSH as the support user.

   Note:

   If you're using the Oracle Cloud Infrastructure (OCI) marketplace image, connect through SSH as the OPC user.

   ssh support@<audit_vault_server_ip_address>

2. Switch to the root user.

   su - root

   Note:

   If you're using the OCI marketplace image, use the sudo su - command.

3. Switch to the oracle user.

   su - oracle

4. Run the following command

   /usr/local/dbfw/bin/setup_ha.rb --status

   The output of above command will tell the current high availability (HA) status and different properties such as Data Guard broker status, fast recovery area usage, and apply lag of HA system.

Audit Vault Agents and Host Monitor Agents deployed on the designated primary Audit Vault Server require no further action. The information of Audit Vault Agents is replicated to the standby Audit Vault Server during high availability pairing.

Audit Vault Agents and Host Monitor Agents deployed on the designated standby Audit Vault Server will be unable to communicate with the designated primary Audit Vault Server after high availability pairing. To redeploy the Agents on the specific Agent machines, follow these steps:

When automatic failover is in effect, the system periodically monitors the availability of the primary Audit Vault Server. If the primary becomes unavailable for more than 10 minutes, then the failover to the standby Audit Vault Server is automatically triggered. However, if the primary Audit Vault Server has been gracefully shut down by the user, then no failover is automatically triggered. In this case, to manually initiate the failover, carefully examine the situation as required, and run the following command as the oracle user on the standby Audit Vault Server:

/usr/local/dbfw/bin/setup_ha.rb --failover

In a failover, the standby Audit Vault Server becomes the new primary. If the previous primary comes back within 20 minutes, it is reinstated as the new standby and both systems will be in a high availability configuration.

If the previous primary does not come back within 20 minutes, then it becomes unusable. The new primary unpairs and becomes a standalone instance. Perform the following procedure to bring the system back into high availability configuration:

Archive and retrieve functionality in high availability automatically handles the necessary steps to process the datafiles on both the primary and standby Audit Vault Server instances. In order to archive, you must provide an NFS archive location. An NFS archive location in a high availability environment contains separate NFS details for primary and standby Audit Vault Servers.

In case there is no NFS archive location, then follow these steps to create a new NFS archive location:

Prerequisite

First, create the Database Firewall instances and register them in the Audit Vault Server console. Afterward, configure these instances for high availability to ensure system resilience. Later, create monitoring points, register targets, and define policies for the Database Firewall instances configured for high availability.

Starting with Oracle AVDF 20.6, Database Firewall instances can be paired with existing monitoring points in Monitoring (Host Monitor) or Monitoring (Out of Band) modes. See Configuring High Availability of Database Firewall Instances With Monitoring Points for more information.

High Availability in Monitoring (Host Monitor) Or Monitoring (Out of Band) Modes

In this configuration:

• High availability (primary and standby) is configured through Audit Vault Server.
• In case of Monitoring (Host Monitor) mode, the Host Monitor Agent is configured to capture and forward the traffic to the primary and standby Database Firewall instances.
• In case of Monitoring (Out of Band) deployment mode, the network switch is configured to mirror and forward the traffic to both the primary and standby Database Firewall instances.
• The configuration of targets, monitoring points, and policies is automatically applied to the primary and standby Database Firewall instances by Audit Vault Server.

The Audit Vault Server collects network events from the primary or standby Database Firewall instance. If the Audit Vault Server is unable to contact the primary Database Firewall for a specified period of time (default of 10 minutes), then the Audit Vault Server collects the network events from the standby Database Firewall. The Audit Vault Server deletes the network events from both instances of Database Firewall after storing the data in the Audit Vault Server repository.

High Availability in Monitoring/Blocking (Proxy) Mode

Database Firewall instances deployed in Monitoring/Blocking (Proxy) mode can be configured for high availability in the following ways:

1. Active (primary) and passive (standby)
2. Active and active

In active and passive configuration:

• Client programs are configured to connect to the primary Database Firewall instance. If the primary Database Firewall instance is not reachable or is down, then they connect to the standby.
• Audit Vault Server collects the network events from the Database Firewall instance (either active or passive) that receives the traffic.

In active and active configuration:

• Multiple Database Firewall instances can be part of this configuration.
• Client programs can connect to any of the active Database Firewall instances that are part of this configuration.
• Once a client establishes a session with an active Database Firewall instance, it communicates with the same instance throughout the session.
• Audit Vault Server collects the network events from all the active Database Firewall instances that are part of this configuration.

ADDRESS_LIST

Include addresses of all the Database Firewall instances in the ADDRESS_LIST. The client programs connect to the first Database Firewall instance. In case of a failed attempt, the client connects to the next instance in the order.

For example:

dbfw1=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.1)(PORT=1111))

                                 (ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.2)(PORT=2222)))(CONNECT_DATA=(SERVICE_NAME=dbfwdb)))

where:

dbfw1 is referred to as net_service_name.

Host = 192.0.2.1 and Host = 192.0.2.2 are the IP addresses of Database Firewall instances configured for high availability.

If you are using SQL*Plus client, then use the following command:

sqlplus <username>/<password>@<net_service_name>

The SQL*Plus client attempts to connect to the first Database Firewall instance with IP 192.0.2.1. In case the first instance is down or not reachable, then the client attempts to connect to the second Database Firewall instance with IP address 192.0.2.2.

CONNECT_TIMEOUT

Use CONNECT_TIMEOUT (seconds) parameter to quickly detect if the Database Firewall instance is down.

For example:

dbfw1=(DESCRIPTION=(CONNECT_TIMEOUT=10)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.1)(PORT=1111))(ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.2)(PORT=2222)))(CONNECT_DATA=(SERVICE_NAME=dbfwdb)))

The client attempts to connect to the first Database Firewall instance with IP 192.0.2.1. In case the first instance is down or not reachable, then the client waits for the duration (seconds) mentioned in the CONNECT_TIMEOUT parameter. In the above example it is 10 seconds. Next, the client attempts to connect to the second Database Firewall instance with IP address 192.0.2.2.

LOAD_BALANCE

Use LOAD_BALANCE parameter for client connections to connect to Database Firewall instances in a random sequence.

For example:

dbfw1=(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=on)(ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.1)(PORT=1111))(ADDRESS=(PROTOCOL=TCP)(HOST=192.0.2.2)(PORT=2222)))(CONNECT_DATA=(SERVICE_NAME=dbfwdb)))

Here, clients will connect to either 192.0.2.1 or 192.0.2.2 in a random sequence.

Prerequisites

1. Install and register Database Firewall instances.

2. For each Database Firewall instance:

   • The configuration of the monitoring points must be same. For example Database Firewall instances DBFW1 and DBFW2 should have the same number of monitoring points and the configuration of these monitoring points should also be the same.

   • Deploy the same Database Firewall policy for a specific target. For example, deploy Database Firewall policy P1 (for target T1) on instances DBFW1 and DBFW2.

3. Client programs should be able to connect to the configured DNS server.

Setup a fully qualified Domain Name in DNS

1. Create a fully qualified domain name to represent IP addresses of the Database Firewall instances.

2. Configure the selected DNS server as the name resolution server on the client hosts.

3. Clients should use the fully qualified domain name in the connection string to connect to the Database Firewall instance.

4. For example, if you are using SQL*Plus, then follow these steps:

   1. Start the SQL*Plus connection as sqlplus /nolog without the username or password.
   2. Run the command: connect <username>/<password>@<fully qualified domain name>:<port/service>

5. DNS can be configured in one of the following ways:

   1. Configure DNS to always connect to an ordered list of Database Firewall instances (for example DBFW1, DBFW2, etc). If a client is not able to connect to the first instance (DBFW1), then it attempts to connect to the second instance (DBFW2).
   2. Configure DNS to use round-robin algorithm for connecting to Database Firewall instances.