15.6.2 RDMA Network Fabric Cabling for Oracle Exadata X9M-8
This topic contains RDMA Network Fabric cabling details for Oracle Exadata X9M-8.
The RDMA Network Fabric switches are located in rack units 21 and 23 for Oracle Exadata X9M-8.
In the following tables, Un denotes the slot in the rack, where n is the number, such as U23. Pn denotes the RDMA Network Fabric port n, where n is port number, such as P1. The cables used for the RDMA Network Fabric switch are black in color.
Note:
Not all the slots shown in the tables below will be used in a single configuration. Match the rack component with the appropriate slot in the tables below to determine how to cable that particular component.The following tables are in this section:
- Table 15-12 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Database Servers
- Table 15-13 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Storage Servers
- Table 15-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-8
The following table lists the rack location, ports and cables for the RDMA Network Fabric connections for database servers in an Oracle Exadata X9M-8 rack.
Table 15-12 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Database Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U24 |
PCIe 5, P2 |
U23 |
15 |
3 meters |
|
U24 |
PCIe 5, P1 |
U21 |
15 |
3 meters |
|
U24 |
PCIe 7, P2 |
U23 |
16 |
3 meters |
|
U24 |
PCIe 7, P1 |
U21 |
16 |
3 meters |
|
U24 |
PCIe 9, P2 |
U23 |
17 |
3 meters |
|
U24 |
PCIe 9, P1 |
U21 |
17 |
3 meters |
|
U24 |
PCIe 11, P2 |
U23 |
18 |
3 meters |
|
U24 |
PCIe 11, P1 |
U21 |
18 |
3 meters |
|
U16 |
PCIe 5, P2 |
U23 |
19 |
3 meters |
|
U16 |
PCIe 5, P1 |
U21 |
19 |
3 meters |
|
U16 |
PCIe 7, P2 |
U23 |
20 |
3 meters |
|
U16 |
PCIe 7, P1 |
U21 |
20 |
3 meters |
|
U16 |
PCIe 9, P2 |
U23 |
21 |
3 meters |
|
U16 |
PCIe 9, P1 |
U21 |
21 |
3 meters |
|
U16 |
PCIe 11, P2 |
U23 |
22 |
3 meters |
|
U16 |
PCIe 11, P1 |
U21 |
22 |
3 meters |
|
U10 |
PCIe 5, P2 |
U23 |
23 |
3 meters |
|
U10 |
PCIe 5, P1 |
U21 |
23 |
3 meters |
|
U10 |
PCIe 7, P2 |
U23 |
24 |
3 meters |
|
U10 |
PCIe 7, P1 |
U21 |
24 |
3 meters |
|
U10 |
PCIe 9, P2 |
U23 |
25 |
3 meters |
|
U10 |
PCIe 9, P1 |
U21 |
25 |
3 meters |
|
U10 |
PCIe 11, P2 |
U23 |
30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target. |
3 meters |
|
U10 |
PCIe 11, P1 |
U21 |
30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target. |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for storage servers in an Oracle Exadata X9M-8 rack.
Note:
On Exadata X9M-2 High Capacity (HC) and Extreme Flash (EF) storage servers, the RDMA Network Fabric ports are located at PCIe slot 3. On Extreme (XT) storage servers, the RDMA Network Fabric ports are located at PCIe slot 10.
Table 15-13 RDMA Network Fabric Cabling for Oracle Exadata Database Machine X9M-8 Storage Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U41 |
PCIe QSFP28 Port 2 |
U23 |
08 |
3 meters |
|
U41 |
PCIe QSFP28 Port 1 |
U21 |
08 |
3 meters |
|
U39 |
PCIe QSFP28 Port 2 |
U23 |
09 |
3 meters |
|
U39 |
PCIe QSFP28 Port 1 |
U21 |
09 |
3 meters |
|
U37 |
PCIe QSFP28 Port 2 |
U23 |
10 |
3 meters |
|
U37 |
PCIe QSFP28 Port 1 |
U21 |
10 |
3 meters |
|
U35 |
PCIe QSFP28 Port 2 |
U23 |
11 |
3 meters |
|
U35 |
PCIe QSFP28 Port 1 |
U21 |
11 |
3 meters |
|
U33 |
PCIe QSFP28 Port 2 |
U23 |
12 |
3 meters |
|
U33 |
PCIe QSFP28 Port 1 |
U21 |
12 |
3 meters |
|
U31 |
PCIe QSFP28 Port 2 |
U23 |
13 |
3 meters |
|
U31 |
PCIe QSFP28 Port 1 |
U21 |
13 |
3 meters |
|
U29 |
PCIe QSFP28 Port 2 |
U23 |
14 |
3 meters |
|
U29 |
PCIe QSFP28 Port 1 |
U21 |
14 |
3 meters |
|
U14 |
PCIe QSFP28 Port 2 |
U23 |
23 |
3 meters |
|
U14 |
PCIe QSFP28 Port 1 |
U21 |
23 |
3 meters |
|
U12 |
PCIe QSFP28 Port 2 |
U23 |
24 |
3 meters |
|
U12 |
PCIe QSFP28 Port 1 |
U21 |
24 |
3 meters |
|
U10 |
PCIe QSFP28 Port 2 |
U23 |
25 |
3 meters |
|
U10 |
PCIe QSFP28 Port 1 |
U21 |
25 |
3 meters |
|
U08 |
PCIe QSFP28 Port 2 |
U23 |
26 |
3 meters |
|
U08 |
PCIe QSFP28 Port 1 |
U21 |
26 |
3 meters |
|
U06 |
PCIe QSFP28 Port 2 |
U23 |
27 |
3 meters |
|
U06 |
PCIe QSFP28 Port 1 |
U21 |
27 |
3 meters |
|
U04 |
PCIe QSFP28 Port 2 |
U23 |
28 |
3 meters |
|
U04 |
PCIe QSFP28 Port 1 |
U21 |
28 |
3 meters |
|
U02 |
PCIe QSFP28 Port 2 |
U23 |
29 |
3 meters |
|
U02 |
PCIe QSFP28 Port 1 |
U21 |
29 |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for the leaf switches in an Oracle Exadata X9M-8 rack.
Table 15-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X9M-8
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U21 |
04 |
U23 |
04 |
1 meter |
|
U21 |
05 |
U23 |
05 |
1 meter |
|
U21 |
06 |
U23 |
06 |
1 meter |
|
U21 |
07 |
U23 |
07 |
1 meter |
|
U21 |
30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used. |
U23 |
30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used. |
1 meter |
|
U21 |
31 |
U23 |
31 |
1 meter |
|
U21 |
32 |
U23 |
32 |
1 meter |
|
U21 |
33 |
U23 |
33 |
1 meter |
Parent topic: RDMA Network Fabric Network Cabling