Memory Management

To be executed, software must have its instructions and data available in memory. The OS instructs the CPU on which memory belongs to which process.

Physical and virtual address space

The physical address space is the actual space accessed by the memory management unit (MMU). The CPU allocates portions of this space to the executing process. These subdivided spaces are the virtual address space.

The job of the MMU is to map physical to virtual in real time, so the CPU can figure out which physical address a virtual address corresponds to.

The OS does not know the data that a process is accessing or storing. Instead it focuses on system-level units of memory.

Pages

A page is a uniformly sized region of physical memory with exact alignment. A page can be mapped to a particular virtual address by the MMU.

Any physical page can exist in multiple sets of page mappings, so that various threads (potentially running on different cores) can access the same memory page.

TLB and huge pages

A special-purpose high-speed cache in the CPU, the translation lookaside buffer (TLB), caches page-table mappings. The standard page is 4 KB, so for contiguous memory above that limit, using the standard size will lead to frequent TLB misses. To minimize this we can use huge pages.

Huge pages are a big win when your working set fits into a small number of them: good spatial locality, large contiguous allocations. Then the few huge-page TLB entries cover everything and you almost never miss.

They become a liability when you have a large number of scattered huge pages, or a working set that jumps around between many of them. If the program maps (and actively touches) more huge pages than there are huge-page TLB slots, those few slots thrash. Every miss walks the page tables: expensive memory accesses.