• I have a particular VMWare ESX 4.1 host that has a very persistent 'Host memory status' alarm. This is running on an HP ProLiant DL360 G7 server. The HP ILO and System Management agents don't know any errors. If I clear the alarm in the vSphere client, it returns within a day.
  • Similarly to physical CPUs and processor cores, physical memory is not limited for an ESXi server that has a free license. Along with unlimited CPUs and CPU cores, this feature allows you to deploy a powerful physical server that contains 4, 6 or more multicore CPUs with large amount of RAM, install ESXi, apply a free license, and deploy many.

This post is going to be one of my favorite posts this year because i have been asked by lot of my readers to write about the ESXi host memory management techniques. Fairly all VMware Administrators will be aware about the ESX memory management techniques to handle the over commitment of the memory. Over commitment means nothing but you can allocated more memory to your virtual machines than the actual available memory of the ESX host. Let’s assume,your ESXi host having total memory of 40 GB but you have 10 virtual machines and each configured with 4 GB with the total of 40 GB memory. But actually total memory available at your ESXi host is just 30 GB. This over commitment can be achieved with the help of memory management techniques and also not all the VM’s will be utilizing 100% of memory allocated at all the times. If it happens, ESXi host will actively use its memory reclamation techniques to handle this situation efficiently.

Num of memory slots = Total available memory resource of ESX or cluster minus memory used for service console & ESX system / Memory Slot size. Let’s take a Example, I have 3 host on the cluster and 6 Virtual machine is running on the cluster and Each host capacity as follows. RAM = 50 GB per Host. CPU = 8 X 2.666 GHZ per host.

Below are the Memory Management techniques available as part of ESXi host

1. Transparent Page Sharing
2. Memory ballooning
3. Memory Compression
4. Hypervisor-level memory swapping

Most of the VMware administrators will be aware about these management techniques but do you know when and at what stage which technique will be used. Let’s discuss in detail about the various ESXi host Memory states. The ESXi host never attempts to reclaim the memory using memory management techniques until it is under memory contention. Memory reclamation techniques such as ( Memory ballooning, Compression or swapping) will come into action based on the amount of ESXi free host memory. There are 4 different ESXi host states

Esxi host memory slots freeEsxi Host Memory Slots

1. High
2. Soft
3. Hard
4. Low

High -> By default Transparent Page sharing will be always running

Soft -> Memory ballooning will be activate,when ESXi enters the soft state and remains active until ESXi is back to high state

Hard & Low -> Memory compression and hypervisor-level memory swapping are used by ESXi when ESX is in the hard or low state

Low -> If the host’s memory usage is above the Low state, ESXi host will stop creating the new pages for Virtual machines and continues compressing and swapping until free up more memory.

With prior to the vSphere 5, High was set by default at 6%, Soft at 4%, Hard at 2%, and Low at 1%. If the ESXi host free memory is less than the mentioned percentage , ESXi uses the respective memory-reclamation techniques to reclaim the memory but think about the Host configured with more memory. It is not necessary to protect that much free memory. Let’s take a example, ESXi 5.0 host can run with 2 TB of memory. Using these defined values as in the pre-vsphere 5.0, Host will start to reclaim the memory even if has 100 GB of free memory. This is really not a great option. So with vSphere 5.x, This predefined values has been changed to effectively handle the host reclamation techniques for the ESXi host configured with more memory.

With vSphere 5, High state level will be adjusted according to the amount of memory in the host. Below is the calculation

High -> 900 MB for 28 GB + 1 % of all memory above 28 GB (If host is having more than 28 GB of memory)
Soft -> 2/3 of High (64% of High)
Hard -> 1/3 of High (32 % of High)
Low -> 1/6 of High (16 % of High)

Below are few of the example of Memory Reclamation Levels of ESXi host

Verify the Current ESXi Host Memory State

We have understood the ESXi Memory state and How memory reclamation techniques will be used based on the ESXi host memory States. Now, Let’s learn How to verify the current memory state of the ESXi host.

Login to your ESXi host using SSH and type the below command

esxtop and Press m

You can see the current memory state of the ESXi host as like the below screenshots.

That’s it. We will take a look at the various memory reclamation techniques in upcoming posts. I hope this is informative for you. Thanks for Reading. Be Social and share it in social media, if you feel worth sharing it.

vSphere 6 uses the well known memory reclamation techniques you may already know from previous versions:

Esxi
  • transparent page sharing (TPS)
  • memory ballooning
  • memory compression
  • memory swapping
Esxi

The memory reclamation technique that is used depends on the ESXi host memory state, which is determined by the amount of free memory of the ESXi host at a given time.

Esxi Host Memory Slots

With vSphere 6 VMware introduced a new memory state, called “clear state“.

So vSphere 6 knows five different memory states associated with one or more memory reclamation techniques to reclaim memory:

But which treshold of free memory is associated with which memory state?

Esxi Check Memory Slots

ESXi uses a value called “minFree” for the memory state calculation. minFree is a dynamic value and depends on the ESXi host memory configuration.

You can calculate minFree very easy for your ESXi host:
for the first 28 GB of physical RAM in the ESXi Host: minFree = 899 MB
+ add 1 percent of the remaining RAM to your calculation

Figure: „minFree calculation example vSphere 6:

In the example above the ESXi host has 100 GB memory:
for the first 28 GB RAM minFree = 899 MB, for the remaining 72 GB (100 GB – 28 GB) we have to add 1% to minFree: 1% of 720 GB = 720 MB -> minFree is 899 MB + 720 MB = 1619

Thresholds:

  • high state: enough free memory available
  • clear state: <100% of minFree
  • soft state: <64% of minFree
  • hard state: <32% of minFree
  • low state: <16% of minFree

If you want to know the memory state of one of your ESXi hosts you can use ESXTOP (extract from the vSphere 6 ESXTOP quick Overview for Troubleshooting” diagram):

Esxi Host Memory Slots Download

Open ESXTOP and type “m” for the memory tab. The host memory state is displayed in the first line on the right.

You want to learn more about ESXTOP?

Esxi Host Memory Slots Free

Take a look at the vSphere 6 ESXTOP quick Overview for Troubleshooting” diagram: