5.1.2 Cabling Multiple Oracle Exadata X8M Racks
A spine switch must exist in the Oracle Exadata rack, regardless of size, to multi-rack with other racks.
Note:
To extend Oracle Exadata racks with RoCE Network Fabric, you must order cables, transceivers if needed, and a RoCE Network Fabric switch, if they are not already available.In the following steps, the number in parentheses also indicates the number of cables required.
- 
                     Split each leaf switch uplink bundle by the number of spine switches (or racks) in such a way that the total count adds up to 8. This split is represented in parentheses for each example. Example 1: For two racks, take 8 uplinks from each leaf switch and split evenly by 2. Four uplinks from each leaf switch go to rack1-spineswitch, and four uplinks from each leaf switch go torack2-spineswitch ( 4 + 4 for each leaf switch).Example 2: For three racks, take 8 uplinks from each leaf switch and split evenly by 3. Three uplinks go to rack1-spineswitch, three uplinks go torack2-spineswitch, two uplinks go torack3-spineswitch ( 3 + 3 + 2 for each leaf switch).
- 
                     Starting from the first available port on a different spine switch, round-robin the above split scheme for each leaf switch and spine switch. For example, for three racks: - rack1-leaf1switch starts from- rack1-spineswitch for first split,- rack2-spineswitch for second split,- rack3-spineswitch for third split
- rack2-leaf1switch starts from- rack2-spineswitch for first split,- rack3-spineswitch for second split,- rack1-spinefor third split
- rack3-leaf1switch starts from- rack3-spineswitch for first split,- rack1-spineswitch for second split,- rack2-spineswitch for third split
- and so on...
 
- 
                     After walking through all leaf switch uplinks in each case, you will have used all spine switch ports between port 5 and 20 inclusive. 
The remaining topics in this section provide detailed cabling information for cabling up to 8 racks together.
Parent topic: Understanding Multi-Rack Cabling for X8M Racks