16.6.2 RDMA Network Fabric Network Cabling Tables for X8M-8 Racks
This topic contains the tables for RDMA Network Fabric network cabling for Oracle Exadata X8M-8 racks.
The RoCE Network Fabric switches are located in rack units 21 and 23 for Oracle Exadata X8M-8.
The following tables can be used for both flexible and fixed configurations. There are three sets of tables for each Exadata hardware model. One table is for storage servers, which have a 2-slot height (2RU). Another table is for the database servers, which require 5RU. The last table for each hardware model is for the inter-switch cabling, which provides the cabling information for the leaf switches.
Note:
Not all the slots shown in the tables below will be used in a single configuration. Match the rack component with the appropriate slot in the tables below to determine how to cable that particular component.In the following tables, Un denotes the slot in the rack, where n is the number, such as U23. Pn denotes the RDMA Network Fabric port n, where n is port number, such as P1. The cables used for the RDMA Network Fabric switch are black in color. The following tables are in this section:
- Table 16-12 RDMA Network Fabric Network Cabling for Oracle Exadata X8M-8 Database Servers
- Table 16-13 RDMA Network Fabric Network Cabling for Oracle Exadata X8M-8 Storage Servers
- Table 16-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata X8M-8
The following table lists the rack location, ports and cables for the RDMA Network Fabric connections for database servers in an Oracle Exadata X8M-8 rack.
Table 16-12 RDMA Network Fabric Network Cabling for Oracle Exadata Database Machine X8M-8 Database Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U24 |
PCIe 5, P2 |
U23 |
15 |
3 meters |
|
U24 |
PCIe 5, P1 |
U21 |
15 |
3 meters |
|
U24 |
PCIe 7, P2 |
U23 |
16 |
3 meters |
|
U24 |
PCIe 7, P1 |
U21 |
16 |
3 meters |
|
U24 |
PCIe 9, P2 |
U23 |
17 |
3 meters |
|
U24 |
PCIe 9, P1 |
U21 |
17 |
3 meters |
|
U24 |
PCIe 11, P2 |
U23 |
18 |
3 meters |
|
U24 |
PCIe 11, P1 |
U21 |
18 |
3 meters |
|
U16 |
PCIe 5, P2 |
U23 |
19 |
3 meters |
|
U16 |
PCIe 5, P1 |
U21 |
19 |
3 meters |
|
U16 |
PCIe 7, P2 |
U23 |
20 |
3 meters |
|
U16 |
PCIe 7, P1 |
U21 |
20 |
3 meters |
|
U16 |
PCIe 9, P2 |
U23 |
21 |
3 meters |
|
U16 |
PCIe 9, P1 |
U21 |
21 |
3 meters |
|
U16 |
PCIe 11, P2 |
U23 |
22 |
3 meters |
|
U16 |
PCIe 11, P1 |
U21 |
22 |
3 meters |
|
U10 |
PCIe 5, P2 |
U23 |
23 |
3 meters |
|
U10 |
PCIe 5, P1 |
U21 |
23 |
3 meters |
|
U10 |
PCIe 7, P2 |
U23 |
24 |
3 meters |
|
U10 |
PCIe 7, P1 |
U21 |
24 |
3 meters |
|
U10 |
PCIe 9, P2 |
U23 |
25 |
3 meters |
|
U10 |
PCIe 9, P1 |
U21 |
25 |
3 meters |
|
U10 |
PCIe 11, P2 |
U23 |
30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target. |
3 meters |
|
U10 |
PCIe 11, P1 |
U21 |
30 or 8 Note: Port 30 is the target only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 8 is the target. |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for storage servers in an Oracle Exadata X8M-8 rack.
Table 16-13 RDMA Network Fabric Network Cabling for Oracle Exadata Database Machine X8M-8 Storage Servers
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U41 |
PCIe 1, P2 |
U23 |
08 |
3 meters |
|
U41 |
PCIe 1, P1 |
U21 |
08 |
3 meters |
|
U39 |
PCIe 1, P2 |
U23 |
09 |
3 meters |
|
U39 |
PCIe 1, P1 |
U21 |
09 |
3 meters |
|
U37 |
PCIe 1, P2 |
U23 |
10 |
3 meters |
|
U37 |
PCIe 1, P1 |
U21 |
10 |
3 meters |
|
U35 |
PCIe 1, P2 |
U23 |
11 |
3 meters |
|
U35 |
PCIe 1, P1 |
U21 |
11 |
3 meters |
|
U33 |
PCIe 1, P2 |
U23 |
12 |
3 meters |
|
U33 |
PCIe 1, P1 |
U21 |
12 |
3 meters |
|
U31 |
PCIe 1, P2 |
U23 |
13 |
3 meters |
|
U31 |
PCIe 1, P1 |
U21 |
13 |
3 meters |
|
U29 |
PCIe 1, P2 |
U23 |
14 |
3 meters |
|
U29 |
PCIe 1, P1 |
U21 |
14 |
3 meters |
|
U14 |
PCIe 1, P2 |
U23 |
23 |
3 meters |
|
U14 |
PCIe 1, P1 |
U21 |
23 |
3 meters |
|
U12 |
PCIe 1, P2 |
U23 |
24 |
3 meters |
|
U12 |
PCIe 1, P1 |
U21 |
24 |
3 meters |
|
U10 |
PCIe 1, P2 |
U23 |
25 |
3 meters |
|
U10 |
PCIe 1, P1 |
U21 |
25 |
3 meters |
|
U08 |
PCIe 1, P2 |
U23 |
26 |
3 meters |
|
U08 |
PCIe 1, P1 |
U21 |
26 |
3 meters |
|
U06 |
PCIe 1, P2 |
U23 |
27 |
3 meters |
|
U06 |
PCIe 1, P1 |
U21 |
27 |
3 meters |
|
U04 |
PCIe 1, P2 |
U23 |
28 |
3 meters |
|
U04 |
PCIe 1, P1 |
U21 |
28 |
3 meters |
|
U02 |
PCIe 1, P2 |
U23 |
29 |
3 meters |
|
U02 |
PCIe 1, P1 |
U21 |
29 |
3 meters |
The following table lists the location, ports and cables for the RDMA Network Fabric connections for the leaf switches in an Oracle Exadata X8M-8 rack.
Table 16-14 RDMA Network Fabric Inter-Switch Cabling for Oracle Exadata Database Machine X8M-8
| From Rack Unit | Port | To RDMA Network Fabric Switch | Port | Description |
|---|---|---|---|---|
|
U21 |
04 |
U23 |
04 |
1 meter |
|
U21 |
05 |
U23 |
05 |
1 meter |
|
U21 |
06 |
U23 |
06 |
1 meter |
|
U21 |
07 |
U23 |
07 |
1 meter |
|
U21 |
30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used. |
U23 |
30 or 34 Note: Port 34 is used only in a configuration with 3 database servers and 11 storage servers. Otherwise, port 30 is used. |
1 meter |
|
U21 |
31 |
U23 |
31 |
1 meter |
|
U21 |
32 |
U23 |
32 |
1 meter |
|
U21 |
33 |
U23 |
33 |
1 meter |
Parent topic: RDMA Network Fabric Network Cabling Tables