x86 Instruction Set – Virtualisation (VMX and SVM)

Introduction

Intel and AMD each added hardware support for virtual machines, separately and with different instructions. Both let a hypervisor run a guest operating system directly on the processor, and take control back (a VM exit, or #VMEXIT on AMD) when the guest does something the hypervisor must handle.

VMX (Virtual Machine Extensions, marketed as VT-x) came with Intel’s Pentium 4 models 662 and 672 in 2005. Each guest is described by a VMCS (virtual-machine control structure), a region of memory (its size, up to 4 KB, is in the IA32_VMX_BASIC MSR) whose layout is the processor’s own: software reads and writes it only through vmread and vmwrite. Software turns VMX on by setting CR4.VMXE (bit 13) and running vmxon. A program checks for it with CPUID leaf 1: ECX bit 5.
SVM (Secure Virtual Machine, marketed as AMD-V) came with AMD’s Athlon 64 processors for Socket AM2 in 2006. Each guest is described by a VMCB (virtual-machine control block) in ordinary memory, whose physical address is passed in rAX. Software turns SVM on with EFER.SVME (bit 12). A program checks for it with CPUID leaf 8000_0001h: ECX bit 2.

Almost all of these instructions may run only at privilege level 0, and outside a guest. The exceptions are vmcall, vmfunc and vmmcall, which a guest uses to call the hypervisor.

The instructions that manage memory protection and secure launch are on the Security page: SMX (getsec), SEV-SNP and TDX.

The instructions

In the forms, r64 is a general register in ModRM.reg, and m64 or m128 a memory operand in ModRM.r/m. NP means that no 66, F2 or F3 prefix may be present.

Intel VMX: entering and leaving VMX operation

Instruction	Forms	Encoding	What it does	Set
`vmxon`	`m64`	`F3 0F C7 /6`	Enters VMX root operation. The operand holds the physical address of the VMXON region. CR4.VMXE must be set, and IA32_FEATURE_CONTROL must allow it.	VMX (Intel Pentium 4, 2005)
`vmxoff`	–	`0F 01 C4`	Leaves VMX operation.	VMX (Intel Pentium 4, 2005)

Intel VMX: the VMCS

Instruction	Forms	Encoding	What it does	Set
`vmptrld`	`m64`	`NP 0F C7 /6`	Makes the VMCS at the physical address in the operand the current VMCS.	VMX (Intel Pentium 4, 2005)
`vmptrst`	`m64`	`NP 0F C7 /7`	Stores the physical address of the current VMCS.	VMX (Intel Pentium 4, 2005)
`vmclear`	`m64`	`66 0F C7 /6`	Writes any cached data of the VMCS at the given physical address back to memory and marks it clear, so that it can be launched or moved to another processor.	VMX (Intel Pentium 4, 2005)
`vmread`	`r/m64, r64` `r/m32, r32`	`NP 0F 78 /r`	Reads the field of the current VMCS whose encoding is in the second operand into the first.	VMX (Intel Pentium 4, 2005)
`vmwrite`	`r64, r/m64` `r32, r/m32`	`NP 0F 79 /r`	Writes the second operand to the field of the current VMCS whose encoding is in the first.	VMX (Intel Pentium 4, 2005)

Intel VMX: running a guest

Instruction	Forms	Encoding	What it does	Set
`vmlaunch`	–	`0F 01 C2`	Enters the guest described by the current VMCS, which must be in the clear state.	VMX (Intel Pentium 4, 2005)
`vmresume`	–	`0F 01 C3`	Enters the guest again, after a VM exit from a VMCS already launched.	VMX (Intel Pentium 4, 2005)
`vmcall`	–	`0F 01 C1`	Called in a guest, causes a VM exit to the hypervisor. In VMX root operation it calls the SMM monitor, when the dual-monitor treatment of system management mode is active.	VMX (Intel Pentium 4, 2005)
`vmfunc`	–	`NP 0F 01 D4`	Called in a guest, runs the VM function numbered in EAX without a VM exit. Function 0 switches to another EPT page-table pointer from a list the hypervisor set up (index in ECX).	VMX (Intel Haswell, 2013)

Intel VMX: TLB invalidation

Instruction	Forms	Encoding	What it does	Set
`invept`	`r64, m128` `r32, m128`	`66 0F 38 80 /r` APX: `EVEX map 4, F3 F0 /r`	Invalidates the translations derived from extended page tables (EPT). The register gives the type (1 single-context, 2 all-context), the 16-byte memory descriptor the EPT pointer.	VMX (Intel Core i7 (Nehalem), 2008)
`invvpid`	`r64, m128` `r32, m128`	`66 0F 38 81 /r` APX: `EVEX map 4, F3 F1 /r`	Invalidates the translations tagged with a virtual-processor identifier (VPID). The register gives the type (0 one address, 1 single-context, 2 all-context, 3 single-context keeping global translations), the descriptor the VPID and the address.	VMX (Intel Core i7 (Nehalem), 2008)

AMD SVM

Instruction	Forms	Encoding	What it does	Set
`vmrun`	`rAX` (implicit)	`0F 01 D8`	Saves the host state, loads the guest state from the VMCB at the physical address in rAX and runs the guest until a #VMEXIT.	AMD-V (SVM) (AMD Athlon 64, 2006)
`vmmcall`	–	`0F 01 D9`	Called in a guest, causes a #VMEXIT to the hypervisor when the VMCB intercepts it; otherwise raises #UD.	AMD-V (SVM) (AMD Athlon 64, 2006)
`vmload`	`rAX` (implicit)	`0F 01 DA`	Loads the state that `vmrun` does not: FS, GS, TR and LDTR with their hidden parts, KernelGsBase, the SYSCALL and SYSENTER MSRs, from the VMCB at rAX.	AMD-V (SVM) (AMD Athlon 64, 2006)
`vmsave`	`rAX` (implicit)	`0F 01 DB`	Stores the same state to the VMCB at rAX.	AMD-V (SVM) (AMD Athlon 64, 2006)
`stgi`	–	`0F 01 DC`	Sets the global interrupt flag (GIF), so that interrupts and other events are taken again.	AMD-V (SVM) (AMD Athlon 64, 2006)
`clgi`	–	`0F 01 DD`	Clears GIF, which holds off interrupts, NMIs and SMIs while the hypervisor switches state.	AMD-V (SVM) (AMD Athlon 64, 2006)
`skinit`	`EAX` (implicit)	`0F 01 DE`	Secure initialisation: reinitialises the processor, protects the 64 KB secure loader block at the physical address in EAX from device access, sends it to the TPM to be measured, and jumps to it. It has its own CPUID bit (leaf 8000_0001h: ECX bit 12).	AMD-V (SVM) (AMD Athlon 64, 2006)
`invlpga`	`rAX, ECX` (implicit)	`0F 01 DF`	Invalidates the TLB entry for the virtual address in rAX in the address space whose ASID is in ECX.	AMD-V (SVM) (AMD Athlon 64, 2006)

Notes

VMX results. A VMX instruction that succeeds clears CF, PF, AF, ZF, SF and OF. One that fails with no current VMCS sets CF (VMfailInvalid); one that fails with a current VMCS sets ZF and puts an error number in the VMCS’s VM-instruction error field (VMfailValid).
Operand size. In 64-bit mode vmread, vmwrite, invept and invvpid always take 64-bit registers, whatever REX.W says; elsewhere they take 32-bit ones.
Prefixes. 0F C7 /7 with memory is vmptrst whatever the prefix. 0F 78 and 0F 79 with 66 or F2 and a register operand are AMD’s SSE4a extrq and insertq, on the SSE4 page.
Inside a guest. A VMX instruction run in a VMX guest causes a VM exit, so a hypervisor can emulate nested virtualisation. The exceptions are vmfunc, and vmread and vmwrite when VMCS shadowing lets the guest reach a shadow VMCS. Under SVM, the VMCB says which instructions are intercepted.