1. Configuring Huge Pages
  2. Configuring HugeTLB Pages
  3. File-Based Configuration Parameters for HugeTLB Pages

File-Based Configuration Parameters for HugeTLB Pages

The file-based configuration parameters provide runtime access to the configuration settings.

Note:

In addition to accessing the settings at runtime, you can also initialize the parameters early in the boot process, for example, by creating a start-up bash script or by setting the parameters up in a local rc init script.
Multiple instances of each file-based parameter can be configured on a system. For example, on a system that can handle both 2 MB and 1 GB HugeTLB page sizes, several nr_hugepages settings can exist. This parameter defines the number of pages in a pool, including the following:
  • File /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages for the number of pages in the pool of 2 MB pages.
  • File /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages for the number of pages in the pool of 1 GB pages.

The following table outlines commonly used HugeTLB configuration parameters and the multiple file instances that you might find for each parameter.

Table 2-2 Commonly Used File-Based HugeTLB Parameters

Parameter Purpose File Paths for Different Instances
nr_hugepages
  • Each instance of nr_hugepages defines the current number of huge pages in the pool associated with that instance.
  • Can be changed at runtime.

  • Example command:

    echo 20 | sudo tee /proc/sys/vm/nr_hugepages
  • Default value is 0.
The file path formats for different instances of nr_hugepages are as follows:
  • File location: /proc/sys/vm/nr_hugepages (present on all systems).
  • File location: /sys/kernel/mm/hugepages/hugepages-<SIZE>kB/nr_hugepages (present on systems that support more than one huge page size).

  • File location: /sys/devices/system/node/node{0,1,2…n}/hugepages/hugepages-<SIZE>kB/nr_hugepages (present on NUMA systems only).

    Note:

    Use the NUMA node specific path format if you need to request different quantities of pages of different sizes to be supported on specific NUMA nodes. If you use any other path format (for example, /proc/sys/vm/nr_hugepages) to request HugeTLB pages. The pages are divided equally between the NUMA nodes.

nr_overcommit_hugepages
  • Each instance of nr_overcommit_hugepages defines the additional number of huge pages that's higher then the quantity specified by nr_hugepages. It can be created by the system at runtime through overcommitting memory.
  • As these additional huge pages become unused, they're freed and returned to the kernel’s normal page pool.
  • Example command:
    echo 20 | sudo tee /proc/sys/vm/nr_overcommit_hugepages
 The file path formats for different instances of nr_overcommit_hugepages are as follows:
  • File location /proc/sys/vm/nr_overcommit_hugepages (present on all systems).
  • File location: /sys/kernel/mm/hugepages/hugepages-<SIZE>kB/nr_overcommit_hugepages (present on systems that support more than one huge page size).

Note:

The nr_overcommit_hugepages parameter isn't defined at the individual node level, so no node specific file exists for this setting.

free_hugepages
  • Read-only parameter.
  • Each instance of free_hugepages returns the number of huge pages in its associated page pool that have yet to be allocated.

The file path formats for different instances of free_hugepages are as follows:

  • File location: /sys/kernel/mm/hugepages/hugepages-<SIZE>kB/free_hugepages (present on systems that support more than one huge page size).
  • File location: /sys/devices/system/node/node{0,1,2…n}/hugepages/hugepages-<SIZE>kB/ free_hugepages (present on NUMA systems only).
surplus_hugepages
  • Read-only parameter.
  • Each instance of surplus_hugepages returns the number of huge pages that have been overcommitted from its associated page pool.

The file path formats for different instances of surplus_hugepages are as follows:

  • File location: /sys/kernel/mm/hugepages/hugepages-<SIZE>kB/surplus_hugepages (present on systems that support more than one huge page size).
  • File location: /sys/devices/system/node/node{0,1,2…n}/hugepages/hugepages-<SIZE>kB/surplus_hugepages (present on NUMA systems only).

The following sections show file branches under which different instances of the HugeTLB parameters are stored:

/proc/sys/vm

All systems that support static huge pages contain HugeTLB parameter files under /proc/sys/vm.

Note:

On many systems, including many Oracle database servers, the procfs file system is the main parameter-set used.

The sysctl parameter vm.nr_hugepages that's commonly initialized in scripts that request huge pages also writes to the procfs file /proc/sys/vm/nr_hugepages.

The following are example folders under branch /proc/sys/vm: 

    ├── ...
    ├── ...
    ├── nr_hugepages
    ├── ...
    ├── nr_overcommit_hugepages
    ├── ...
    ├── ...

/sys/kernel/mm/hugepages/

Systems that support multiple size pools contain HugeTLB parameter files in size-specific folders under /sys/kernel/mm/hugepages/.

The following are example folders under branch /sys/kernel/mm/hugepages/: 

└── hugepages-2048kB
    ├── free_hugepages
    ├── nr_hugepages
    ├── ...
    ├── nr_overcommit_hugepages
    ├── ...
    └── surplus_hugepages

└── hugepages-1048576kB
    ├── free_hugepages
    ├── nr_hugepages
    ├── ...
    ├── nr_overcommit_hugepages
    ├── ...
    └── surplus_hugepages

/sys/devices/system/node/

Only NUMA systems contain HugeTLB parameter files under /sys/devices/system/node/.

The following are example folders under branch /sys/devices/system/node: 

      ├─  ...
      ├── node0
      │   ├── ...
      │   ├──hugepages
      │          hugepages-2048kB
      │              ├── free_hugepages
      │              ├── nr_hugepages
      │              └── surplus_hugepages
      │   
      │          hugepages-1048576kB
      │              ├── free_hugepages
      │              ├── nr_hugepages
      │              └── surplus_hugepages
      ├── node1
          ├── ...
          ├──hugepages
                 hugepages-2048kB
                     ├── free_hugepages
                     ├── nr_hugepages
                     └── surplus_hugepages
          
                 hugepages-1048576kB
                     ├── free_hugepages
                     ├── nr_hugepages
                     └── surplus_hugepages