2.3.3.1 Determine the Amount of Available Space

To increase the size of the disks in a disk group you must either have unallocated disk space available, or you have to reallocate space currently used by a different disk group.

You can also use a script available in "Script to Calculate New Grid Disk and Disk Group Sizes in Exadata (My Oracle Support Doc ID 1464809.1)" to assist in determining how much free space is available to shrink a disk group.

  1. View the space currently used by the disk groups. 
    SELECT name, total_mb, free_mb, total_mb - free_mb used_mb, round(100*free_mb/total_mb,2) pct_free
FROM v$asm_diskgroup
ORDER BY 1;

NAME                             TOTAL_MB    FREE_MB    USED_MB   PCT_FREE
------------------------------ ---------- ---------- ---------- ----------
DATAC1                           68812800    9985076   58827724      14.51
RECOC1                           94980480   82594920   12385560      86.96

    The example above shows that the DATAC1 disk group has only about 15% of free space available while the RECOC1 disk group has about 87% free disk space. The PCT_FREE displayed here is raw free space, not usable free space. Additional space is needed for rebalancing operations.

  2. For the disk groups you plan to resize, view the count and status of the failure groups used by the disk groups.
    SELECT dg.name, d.failgroup, d.state, d.header_status, d.mount_mode, 
 d.mode_status, count(1) num_disks
FROM V$ASM_DISK d, V$ASM_DISKGROUP dg
WHERE d.group_number = dg.group_number
AND dg.name IN ('RECOC1', 'DATAC1')
GROUP BY dg.name, d.failgroup, d.state, d.header_status, d.mount_status,
  d.mode_status
ORDER BY 1, 2, 3;

NAME       FAILGROUP      STATE      HEADER_STATU MOUNT_S  MODE_ST  NUM_DISKS
---------- -------------  ---------- ------------ -------- -------  ---------
DATAC1     EXA01CELADM01  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM02  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM03  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM04  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM05  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM06  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM07  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM08  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM09  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM10  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM11  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM12  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM13  NORMAL     MEMBER        CACHED  ONLINE   12
DATAC1     EXA01CELADM14  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM01  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM02  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM03  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM04  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM05  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM06  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM07  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM08  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM09  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM10  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM11  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM12  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM13  NORMAL     MEMBER        CACHED  ONLINE   12
RECOC1     EXA01CELADM14  NORMAL     MEMBER        CACHED  ONLINE   12

    The above example is for a full rack, which has 14 cells and 14 failure groups for DATAC1 and RECOC1. Verify that each failure group has at least 12 disks in the NORMAL state (num_disks). If you see disks listed as MISSING, or you see an unexpected number of disks for your configuration, then do not proceed until you resolve the problem.

    Extreme Flash systems should see a disk count of 8 instead of 12 for num_disks.

  3. List the corresponding grid disks associated with each cell and each failure group, so you know which grid disks to resize.
    SELECT dg.name, d.failgroup, d.path
FROM V$ASM_DISK d, V$ASM_DISKGROUP dg
WHERE d.group_number = dg.group_number
AND dg.name IN ('RECOC1', 'DATAC1')
ORDER BY 1, 2, 3;

NAME        FAILGROUP      PATH
----------- -------------  ----------------------------------------------
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_00_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_01_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_02_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_03_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_04_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_05_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_06_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_07_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_08_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_09_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_10_exa01celadm01
DATAC1      EXA01CELADM01  o/192.168.74.43/DATAC1_CD_11_exa01celadm01
DATAC1      EXA01CELADM02  o/192.168.74.44/DATAC1_CD_00_exa01celadm01
DATAC1      EXA01CELADM02  o/192.168.74.44/DATAC1_CD_01_exa01celadm01
DATAC1      EXA01CELADM02  o/192.168.74.44/DATAC1_CD_02_exa01celadm01
...
RECOC1      EXA01CELADM13  o/192.168.74.55/RECOC1_CD_00_exa01celadm13
RECOC1      EXA01CELADM13  o/192.168.74.55/RECOC1_CD_01_exa01celadm13
RECOC1      EXA01CELADM13  o/192.168.74.55/RECOC1_CD_02_exa01celadm13
...
RECOC1      EXA01CELADM14  o/192.168.74.56/RECOC1_CD_09_exa01celadm14
RECOC1      EXA01CELADM14  o/192.168.74.56/RECOC1_CD_10_exa01celadm14
RECOC1      EXA01CELADM14  o/192.168.74.56/RECOC1_CD_11_exa01celadm14  

168 rows returned.
  4. Check the cell disks for available free space.
    Free space on the cell disks can be used to increase the size of the DATAC1 grid disks. If there is not enough available free space to expand the DATAC1 grid disks, then you must shrink the RECOC1 grid disks to provide the additional space for the desired new size of DATAC1 grid disks.
    [root@exa01adm01 tmp]# dcli -g ~/cell_group -l root "cellcli -e list celldisk \
  attributes name,freespace" 
exa01celadm01: CD_00_exa01celadm01 0 
exa01celadm01: CD_01_exa01celadm01 0 
exa01celadm01: CD_02_exa01celadm01 0 
exa01celadm01: CD_03_exa01celadm01 0 
exa01celadm01: CD_04_exa01celadm01 0 
exa01celadm01: CD_05_exa01celadm01 0 
exa01celadm01: CD_06_exa01celadm01 0 
exa01celadm01: CD_07_exa01celadm01 0 
exa01celadm01: CD_08_exa01celadm01 0 
exa01celadm01: CD_09_exa01celadm01 0 
exa01celadm01: CD_10_exa01celadm01 0 
exa01celadm01: CD_11_exa01celadm01 0 
...

    In this example, there is no free space available, so you must shrink the RECOC1 grid disks first to provide space for the DATAC1 grid disks. In your configuration there might be plenty of free space available and you can use that free space instead of shrinking the RECOC1 grid disks.

  5. Calculate the amount of space to shrink from the RECOC1 disk group and from each grid disk.

    The minimum size to safely shrink a disk group and its grid disks must take into account the following:

    • Space currently in use (USED_MB)

    • Space expected for growth (GROWTH_MB)

    • Space needed to rebalance in case of disk failure (DFC_MB), typically 15% of total disk group size

    The minimum size calculation taking the above factors into account is: 

    Minimum DG size (MB) = ( USED_MB + GROWTH_MB ) * 1.15

    • USED_MB can be derived from V$ASM_DISKGROUP by calculating TOTAL_MB - FREE_MB

    • GROWTH_MB is an estimate specific to how the disk group will be used in the future and should be based on historical patterns of growth

    For the RECOC1 disk group space usage shown in step 1, we see the minimum size it can shrink to assuming no growth estimates is:

    Minimum RECOC1 size = (TOTAL_MB - FREE_MB + GROWTH_MB) * 1.15

    = ( 94980480 - 82594920 + 0) * 1.15 = 14243394 MB = 13,910 GB

    In the example output shown in Step 1, RECOC1 has plenty of free space and DATAC1 has less than 15% free. So, you could shrink RECOC1 and give the freed disk space to DATAC1. If you decide to reduce RECOC1 to half of its current size, the new size is 94980480 / 2 = 47490240 MB. This size is significantly above the minimum size we calculated for the RECOC1 disk group above, so it is safe to shrink it down to this value.

    The query in Step 2 shows that there are 168 grid disks for RECOC1, because there are 14 cells and 12 disks per cell (14 * 12 = 168). The estimated new size of each grid disk for the RECOC1 disk group is 47490240 / 168, or 282,680 MB.

    Find the closest 16 MB boundary for the new grid disk size. If you do not perform this check, then the cell will round down the grid disk size to the nearest 16 MB boundary automatically, and you could end up with a mismatch in size between the Oracle ASM disks and the grid disks. 

    SQL> SELECT 16*TRUNC(&new_disk_size/16) new_disk_size FROM dual;
Enter value for new_disk_size: 282680

NEW_DISK_SIZE
-------------
       282672

    Based on the above result, you should choose 282672 MB as the new size for the grid disks in the RECOC1 disk group. After resizing the grid disks, the size of the RECOC1 disk group will be 47488896 MB.

  6. Calculate how much to increase the size of each grid disk in the DATAC1 disk group.

    Ensure the Oracle ASM disk size and the grid disk sizes match across the entire disk group. The following query shows the combinations of disk sizes in each disk group. Ideally, there is only one size found for all disks and the sizes of both the Oracle ASM (total_mb) disks and the grid disks (os_mb) match. 

    SELECT dg.name, d.total_mb, d.os_mb, count(1) num_disks
FROM v$asm_diskgroup dg, v$asm_disk d
WHERE dg.group_number = d.group_number
GROUP BY dg.name, d.total_mb, d.os_mb;

NAME                             TOTAL_MB      OS_MB  NUM_DISKS
------------------------------ ---------- ---------- ----------
DATAC1                             409600     409600        168
RECOC1                             565360     565360        168

    After shrinking RECOC1's grid disks, the following space is left per disk for DATAC1:

    Additional space for DATAC1 disks = RECOC1_current_size - RECOC1_new_size
                                                           = 565360 - 282672 = 282688 MB

    To calculate the new size of the grid disks for the DATAC1 disk group, use the following:

    DATAC1's disks new size  = DATAC1_ disks_current_size + new_free_space_from_RECOC1
                                              = 409600 + 282688 = 692288 MB

    Find the closest 16 MB boundary for the new grid disk size. If you do not perform this check, then the cell will round down the grid disk size to the nearest 16 MB boundary automatically, and you could end up with a mismatch in size between the Oracle ASM disks and the grid disks. 

    SQL> SELECT 16*TRUNC(&new_disk_size/16) new_disk_size FROM dual;
Enter value for new_disk_size: 692288

NEW_DISK_SIZE
-------------
       692288

    Based on the query result, you can use the calculated size of 692288 MB for the disks in the DATAC1 disk groups because the size is on a 16 MB boundary. If the result of the query is different from the value you supplied, then you must use the value returned by the query because that is the value to which the cell will round the grid disk size.

    The calculated value of the new grid disk size will result in the DATAC1 disk group having a total size of 116304384 MB (168 disks * 692288 MB).

Parent topic: Resizing Grid Disks