Make L1/L2 memory pool sizes self-configurable depending on physical memory available

[wkozaczuk]

Currently, both L1 (vCPU local) and L2 (global shared) memory pools operate based on low/high watermark logic in terms of allocating/freeing pages to satisfy memory allocation/freeing requests. For example, the low watermark for each L1 is 1/4 of the maximum number of pages which is a fixed number. This results in each L1 pool keeping at least 128 pages (512K) per CPU. This might be overkill for very tiny app deployments that do not need much memory.

Ideally, we could change this behavior by making L1 and L2 pool sizes (in pages) to be self-configurable and depend on physical memory available.

Please see for details and more discussion here.