How to tune systems to achieve high performance in virtualization circumstances?

Most time, we need to tune system parameters to achieve better performance but what the general parameters to be tuned in Linux systems.

I think you may want to add following parameters to Kernel boot (/etc/default/grub) parameters

intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll intel_pstate=disable

At the same time, you may also want to shutdown/open Pause Loop Exiting (PLE). See [2] [3] for more details about PLE, and see [1] [4] for more details about Intel C-States and P-States.

How to Shutdown PLE:

sudo rmmod kvm_intel
sudo modprobe kvm_intel ple_gap=0 ple_window=0
cat /sys/module/kvm_intel/parameters/ple*

How to turn on PLE:

sudo rmmod kvm_intel
sudo modprobe kvm_intel 
cat /sys/module/kvm_intel/parameters/ple*

References
[1] https://stackoverflow.com/questions/12111954/context-switches-much-slower-in-new-linux-kernels
[2] https://www.google.com/patents/US9021497
[3] https://surriel.com/system/files/riel_t_1620_kvm_performance.pdf
[4] https://haypo.github.io/intel-cpus.html

---

IDLE=POLL means when vCPU is idle CPU will execute No Op instruction like spin-lock does. This will trigger PLE if PLE is turned on.

Also see this slides for more details about P-States.