x86 Instruction Set – FMA3 and FMA4

Introduction

A fused multiply-add works out a × b + c and rounds the result once. A separate multiply and add round twice, once after each step. The single rounding makes the result more exact, and it does the work of two instructions in one. Every instruction here works on floating-point values in the vector registers: single precision (the ps and ss forms) or double precision (pd and sd), packed across the whole register or on the low element only.

There are two sets, and they are not compatible:

• FMA3 has three operands. The destination is also one of the sources, and the mnemonic’s digits (132, 213 or 231) say which. Intel brought it with Haswell, in 2013. AMD added it with Piledriver, in 2012, and every AMD processor since has it. A program checks CPUID leaf 01H, ECX bit 12 (FMA).
• FMA4 has four operands, and none of the sources is overwritten. It is AMD’s: it came with Bulldozer, in 2011, beside XOP. A program checks CPUID leaf 80000001H, ECX bit 16 (FMA4). AMD dropped FMA4 from Zen (2017) on: those processors do not report it in CPUID, and programs must not use it there.

Intel’s first AVX specification, in April 2008, described a four-operand fused multiply-add. Intel changed it to the three-operand form in December 2008. AMD moved its own plans to the four-operand form in 2009, shipped it in Bulldozer, and then added Intel’s form in Piledriver. Intel has never made a processor with FMA4.

Both sets are VEX-encoded (see Instruction encoding). They use the xmm registers for 128 bits and the ymm registers for 256 bits: 16 of each in 64-bit mode, 8 elsewhere. Rounding and exception masks come from MXCSR. Like every VEX instruction, they also need the operating system to save the AVX state: CPUID leaf 01H, ECX bit 27 (OSXSAVE) must be set, and XGETBV with ECX = 0 must show bits 1 and 2 of XCR0 set. A 128-bit form clears bits 255:128 of its destination register. The 512-bit EVEX forms of FMA3 are on the AVX-512 Foundation page.

FMA3 operand orders: 132, 213 and 231

Call the three operands a, b and c, in the order they are written. a is the destination and comes from ModRM.reg. b is a register, from VEX.vvvv. c is a register or memory, from ModRM.rm. Only c can be in memory.

The three digits name the operands by number: the first two are multiplied, and the third is added. So vfmadd132ps works out operand 1 × operand 3 + operand 2. The result always goes to operand 1, which the instruction overwrites.

The three orders give the same set of operations. A compiler picks the order by which value it may overwrite and which one it wants to read from memory: in 132 the memory operand is a multiplier, in 213 it is the addend, and in 231 it is a multiplier while the addend is the destination itself.

The low four bits of the opcode give the operation, in the same way for each order: 6 multiply with alternating subtract and add, 7 multiply with alternating add and subtract, 8 and 9 multiply-add, A and B multiply-subtract, C and D negated multiply-add, E and F negated multiply-subtract. The even opcode is the packed form, and the odd one the scalar form. VEX.W picks the precision: W0 single, W1 double.

FMA3 instructions

In each row a, b and c are the operands in order, as above. The packed forms work on every element of the register. The scalar forms work on the low element only (see Notes for the rest of the register).

FMA4 instructions

FMA4 sits in VEX map 3 (0F 3A) with the 66 prefix. The four operands are a, b, c and d, in order, and every instruction works out a = b × c ± d. a comes from ModRM.reg and b from VEX.vvvv. The other two come from ModRM.rm and from bits 7:4 of a trailing byte (written /is4), and VEX.W says which is which:

• W0: c is ModRM.rm (a register or memory) and d is the register in imm8[7:4];
• W1: c is the register in imm8[7:4] and d is ModRM.rm.

So either the multiplier or the addend can come from memory. The bits 3:0 of the trailing byte are not used. The names have no digits: the order is always the same.

Notes

• Rounding and exceptions. The product is not rounded; only the final sum is, by the rounding mode in MXCSR. The instructions raise the usual SIMD floating-point exceptions (invalid, denormal, overflow, underflow and precision), masked or not by MXCSR. They do not change the flags in EFLAGS.
• Alignment. A memory operand need not be aligned.
• The rest of a scalar destination. In FMA3 the destination is also a source, so a scalar form keeps the destination’s own bits above the low element, up to bit 127; bits 255:128 are cleared.
• VEX.L and VEX.W. The scalar forms ignore VEX.L (LIG). In FMA3, VEX.W picks single or double precision. In FMA4 the precision is in the opcode, and VEX.W picks which source may be in memory.
• The fourth register outside 64-bit mode. Only xmm0–xmm7 exist there, so bit 7 of the /is4 byte has no register to name. We ignore it, as GNU binutils does; LLVM does not.
• 16-bit code. VEX instructions run in 16-bit protected mode, so we decode them in 16-bit code too, with 16-bit addressing ([bx + si]). They are not available in real mode or virtual-8086 mode.
• Assembler names. GNU and LLVM use the same mnemonics as Intel’s and AMD’s manuals. Intel’s compilers and many libraries reach these instructions through intrinsics such as _mm256_fmadd_ps (FMA3) and _mm_macc_ps (FMA4).

Links