\HeadingLevelB{CPU Microcode}
\label{sec:microcode}

% Outcome: evaluating the cost of architectural modifications

The Intel architecture features a large instruction set. Some instructions are
used infrequently, and some instructions are very complex, which makes it
impractical for an execution core to handle all instructions in hardware.
Intel CPUs use a \textit{microcode} table to break down rare and complex
instructions into sequences of simpler instructions. Architectural extensions
that only require microcode changes are significantly cheaper to implement
and validate than extensions that require changes in the CPU's circuitry.

It follows that a good understanding of what can be done in microcode is
crucial to evaluating the cost of security features that rely on architecture
extensions. Furthermore, the limitations of microcode are sometimes the
reasoning behind seemingly arbitrary architecture design decisions.

The first sub-section below presents the relevant facts pertaining to microcode
in Intel's optimization reference \citep{intel2014optimization} and SDM. The
following subsections summarize information gleaned from Intel's patents and
other researchers' findings.


\HeadingLevelC{The Role of Microcode}
\label{sec:microcode_role}

% Legacy Decode Pipeline (Instruction Decode): Optimization S 2.2.2.1
% Instruction Decode: Optimization S 2.3.2.4
% Front End Overview: Optimization S 2.4.2
% Understanding the Sources of the Micro-Op Queue: SDM S B.3.7.2

The frequently used instructions in the Intel architecture are handled by the
core's fast path, which consists of simple decoders (\S~\ref{sec:out_of_order})
that can emit at most 4 micro-ops per instruction. Infrequently used
instructions and instructions that require more than 4 micro-ops use a slower
decoding path that relies on a sequencer to read micro-ops from a
\textit{microcode store ROM} (MSROM).

The 4 micro-ops limitation can be used to guess intelligently whether an
architectural feature is implemented in microcode. For example, it is safe to
assume that \texttt{XSAVE} (\S~\ref{sec:registers}), which was takes over 200
micro-ops on recent CPUs \citep{fog2014microops}, is most likely performed in
microcode, whereas simple arithmetic and memory accesses are handled directly
by hardware.

% Assists: Optimization S B.3.5.2

The core's execution units handle common cases in fast paths implemented in
hardware. When an input cannot be handled by the fast paths, the execution
unit issues a \textit{microcode assist}, which points the microcode sequencer
to a routine in microcode that handles the edge cases. The most common cited
example in Intel's documentation is floating point instructions, which issue
assists to handle denormalized inputs.

The \texttt{REP MOVS} family of instructions, also known as \textit{string
instructions} because of their use in \texttt{strcpy}-like functions, operate
on variable-sized arrays. These instructions can handle small arrays in
hardware, and issue microcode assists for larger arrays.

% Microcode Update Facilities: SDM S 9.11
% Responsibilities of the BIOS: SDM 9.11.8.1

Modern Intel processors implement a microcode update facility. The SDM
describes the process of applying microcode updates from the perspective of
system software. Each core can be updated independently, and the updates must
be reapplied on each boot cycle. A core can be updated multiple times. The
latest SDM at the time of this writing states that a microcode update is up to
16 KB in size.

Processor engineers prefer to build new architectural features as microcode
extensions, because microcode can be iterated on much faster than hardware,
which reduces development cost \citep{intel2008genetic, intel2012clusters}. The
update facility further increases the appeal of microcode, as some classes of
bugs can be fixed after a CPU has been released.

Intel patents \citep{intel2013patent1, intel2013patent2} describing Software
Guard Extensions (SGX) disclose that SGX is entirely implemented in microcode,
except for the memory encryption engine. A description of SGX's implementation
could provide great insights into Intel's microcode, but, unfortunately, the
SDM chapters covering SGX do not include such a description. We therefore rely
on other public information sources about the role of microcode in the
security-sensitive areas covered by previous sections, namely memory
management~(\S~\ref{sec:paging}, \S~\ref{sec:tlbs}), the handling of hardware
exceptions~(\S~\ref{sec:faults}) and interrupts~(\S~\ref{sec:interrupts}), and
platform initialization~(\S~\ref{sec:booting}).

% Precise Event Based Sampling (PEBS): SDM S 18.7.1.1
% PEBS Record Format: SDM S 18.8.4.1
% Precise Distribution of Instructions Retired (PIDR): SDM S 18.8.4.4
% At-Retirement Counting: SDM S 18.13.6
% Performance Monitoring Events for the 4th Generation Intel Core Processors:
%     SDM S 19.3

The use of microcode assists can be measured using the \textit{Precise Event
Based Sampling}~(PEBS) feature in recent Intel processors. PEBS provides
counters for the number of micro-ops coming from MSROM, including complex
instructions and assists, counters for the numbers of assists associated with
some micro-op classes (SSE and AVX stores and transitions), and a counter for
assists generated by all other micro-ops.

% Non-Architectural Performance Events of the Processor Core Supported by
%   Broadwell microarchitecture: SDM Table 19-5
% Non-Architectural Performance Events In the Processor Core of 4th Generation
%   Intel Core Processors: SDM Table 19-7
% Non-Architectural Performance Events In the Processor Core of 3rd Generation
%   Intel Core i7, i5, i3 Processors: SDM Table 19-11

The PEBS feature itself is implemented using microcode assists (this is implied
in the SDM and confirmed by \citep{intel2014pebs}) when it needs to write the
execution context into a PEBS record. Given the wide range of features
monitored by PEBS counters, we assume that all execution units in the core can
issue microcode assists, which are performed at micro-op retirement. This
finding is confirmed by an Intel patent~\citep{intel1997events}, and is
supported by the existence of a PEBS counter for the ``number of microcode
assists invoked by hardware upon micro-op writeback.''

% Conditional SIMD Packed Loads and Stores: Optimization S 11.9

Intel's optimization manual describes one more interesting assist, from a
memory system perspective. SIMD masked loads (using \texttt{VMASKMOV}) read a
series of data elements from memory into a vector register. A mask register
decides whether elements are moved or ignored. If the memory address overlaps
an invalid page (e.g., the P flag is 0, \S~\ref{sec:paging}), a microcode
assist is issued, even if the mask indicates that no element from the invalid
page should be read. The microcode checks whether the elements in the invalid
page have the corresponding mask bits set, and either performs the load or
issues a page fault.

% IA32_MCG Extended Machine Check State MSRs: SDM S 15.3.2.6

The description of machine checks in the SDM mentions page assists and page
faults in the same context. We assume that the page assists are issued in some
cases when a TLB miss occurs (\S~\ref{sec:tlbs}) and the PMH has to walk the
page table. The following section develops this assumption and provides
supporting evidence from Intel's assigned patents and published patent
applications.


\HeadingLevelC{Microcode Structure}
\label{sec:microcode_structure}

% Arch feature implementation strategy
%   US 8,447,962 - 11:39-43, 12:8-13

According to a 2013 Intel patent \citep{intel2013scattergather}, the avenues
considered for implementing new architectural features are a completely
microcode-based implementation, using existing micro-ops, a microcode
implementation with hardware support, which would use new micro-ops, and a
complete hardware implementation, using finite state machines (FSMs).

% Micro-ops table
%   US 7,451,121 - 1:23-25, 1:34-35, 2:64-65
%   US 8,099,587 - 3:1
% Microcode compression
%   US 7,451,121 - Abstract 1 and 10
%   US 8,099,587 - Abstract 1-3 and 7-10, 8:36-49, 11:10-17

The main component of the MSROM is a table of micro-ops \citep{intel2008genetic,
intel2012clusters}. According to an example in a 2012 Intel patent
\citep{intel2012clusters}, the table contains on the order of 20,000 micro-ops,
and a micro-op has about 70 bits. On embedded processors, like the Atom,
microcode may be partially compressed
\citep{intel2008genetic, intel2012clusters}.

% Event ROM
%   US 5,889,982 - 16:57-63, 16:66-17:3
% Microcode handles exceptions:
%   US 5,987,600 - 2:39-57, 4:13-27, 4:39-53, 4:65-5:6, 8:42-58, 10:54-60,
%                  11:18-42, 12:11-17, 12:54-58, 15:46-48, 15:59-62
%   US 5,889,982 - 11:40-42, 11:44-46
%   US 7,213,511 - 8:45-46, 8:49-51,
% Microcode handles traps:
%   US 5,987,600 - 15:16-18, 15:36-40
% Microcode handles interrupts:
%   US 5,987,600 - 16:2-5, 16:18-21
% Microcode handles events (exceptions and assists):
%   US 5,889,982 - 9:23-25, 9:34-42, 15:7-11, 15:27-55, 16:34-38, 16:57-17:3
%   US 5,625,788 - 1:10-12, 1:64-2:133:2-7, 6:31-38, 6:53-7:2, 8:27-47, 9:2-18,
%                  11:60-12:1, 12:4-8, 12:10-15, 12:19-12:22, 12:25-42,
%                  14:12-32

The MSROM also contains an event ROM, which is an array of pointers to event
handling code in the micro-ops table \citep{intel1999events}. Microcode events
are hardware exceptions, assists, and interrupts \citep{intel1997events,
intel1999exceptions, intel2007microstack}. The processor described in a 1999
patent \citep{intel1999events} has a 64-entry event table, where the first 16
entries point to hardware exception handlers and the other entries are used by
assists.

% Microcode implementation details:
%   US 5,987,600 - 5:39-49, 5:53-6:32, 5:35-39, 5:42-53, 11:53-60, 11:64-67,
%                  12:6-10, 12:41-45, 14:15-19
%   US 5,680,565 - 2:53-56
%   US 5,889,982 - 6:49-65, 7:8-12, 10:11-14, 13:16-20,
%   US 7,231,511 - 1:49-60, 2:1-8, 2:34-42, 3:2-5, 3:22-40, 5:26-67, 6:1-20
%   US 5,636,374 - 2:47-52, 2:63-3:10, 4:39-45

The execution units can issue an assist or signal a fault by associating an
event code with the result of a micro-op. When the micro-op is committed
(\S~\ref{sec:out_of_order}), the event code causes the out-of-order scheduler
to squash all micro-ops that are in-flight in the ROB. The event code is
forwarded to the microcode sequencer, which reads the micro-ops in the
corresponding event handler \citep{intel1997events, intel1999exceptions}.

The hardware exception handling logic (\S~\ref{sec:faults}) and interrupt
handling logic (\S~\ref{sec:interrupts}) is implemented entirely in microcode
\citep{intel1999exceptions}. Therefore, changes to this logic are relatively
inexpensive to implement on Intel processors. This is rather fortunate, as the
Intel architecture's standard hardware exception handling process requires that
the fault handler is trusted by the code that encounters the exception
(\S~\ref{sec:faults}), and this assumption cannot be satisfied by a design
where the software executing inside a secure container must be isolated from
the system software managing the computer's resources.

% Microcode has microinstruction pointer stack
%   US 7,231,511 - Abstract 1-6, 2:44-45, 2:53-55, 3:9-16, 6:1-3, 12:2-9

The execution units in modern Intel processors support microcode procedures,
via dedicated microcode call and return micro-ops \citep{intel2007microstack}.
The micro-ops manage a hardware data structure that conceptually stores a stack
of microcode instruction pointers, and is integrated with out-of-order
execution and hardware exceptions, interrupts and assists.

% Microcode uses special loads / stores
%   US 5,636,374 - 2:31-36, 2:39-46, 6:10-19, 6:22-25, 6:53-57, 6:62-67,
%                  7:59-60, 8:13-24, 10:61-62, 10:65-12:64

Asides from special micro-ops, microcode also employs special load and store
instructions, which turn into special bus cycles, to issue commands to other
functional units \citep{intel1997microspace}. The memory addresses in the
special loads and stores encode commands and input parameters. For example,
stores to a certain range of addresses flush specific TLB sets.


\HeadingLevelC{Microcode and Address Translation}
\label{sec:microcode_pmh}

% Microcode gets executed on CR3 write.
%   US 7,552,255 - 8:43-46

Address translation (\S~\ref{sec:paging}) is configured by CR3, which stores
the physical address of the top-level page table, and by various bits in CR0
and CR4, all of which are described in the SDM. Writes to these control
registers are implemented in microcode, which stores extra information in
microcode-visible registers \citep{intel2009pipeline}.

% DLB misses -> PMH, PMH uses a FSM.
%   US 13/730,563 - 0065, 0066, 0067
%   US 13/730,411 - 0064
%   US 5,564,111 - 1:26-29, 1:36-38, 3:7-21, 3:58-60, 5:36-41, 5:48-57,
%                  6:51-52, 6:55-7:7, 7:16-18, 7:23-24, 8:3-8, 8:39-40,
%                  9:66-10:4, 10:16-23
% PMH implementation (stuffed loads)
%   US 5,680,565 - 2:60-3:3, 3:25-28, 3:33-52, 3:56, 3:58-4:4, 11:17-21,
%                  11:45-48, 11:50-52, 12:30-34, 12:20-22, 12:40-43, 13:20-22,
%                  14:42-58, 15:54-57
%   US 5,636,374 - 5:59-64, 6:5-8
%   US 13/730,563 - 0072, 0073, 0074, Fig. 10

When a TLB miss~(\S~\ref{sec:tlbs}) occurs, the memory execution unit forwards
the virtual address to the \textit{Page Miss Handler}~(PMH), which performs the
page walk needed to obtain a physical address. In order to minimize the latency
of a page walk, the PMH is implemented as a \textit{Finite-State Machine}~(FSM)
\citep{hildesheim2014ptm, raikin2014tlb}. Furthermore, the PMH fetches the
page table entries from memory by issuing ``stuffed loads'', which are special
micro-ops that bypass the reorder buffer (ROB) and go straight to the memory
execution units (\S~\ref{sec:out_of_order}), thus avoiding the overhead
associated with out-of-order scheduling
\citep{intel1997pmh, intel1997microspace, hildesheim2014ptm}.

% Microcode handles memory exceptions (#PF):
%   US 5,987,600 - 14:26-49, 14:55-61, 14:66-15:3
%   US 5,680,565 - 11:29-37,
%   US 5,889,982 - 14:41-43, 15:47-51,
%   US 5,564,111 - 3:7-21, 3:58-60, 5:36-41, 5:48-57,
%                  6:51-52, 6:55-7:7, 7:16-18, 7:23-24, 8:3-8, 8:39-40,
%                  9:66-10:4, 10:16-23
% Microcode handles DTLB and PMH exceptions:
%   US 5,564,111 - Abstract 15-21, 1:46-59, 3:25-45, 7:47-53, 9:33-51,
%                  10:45-54, 10:57-63
% Microcode performs assisted PMH walk
%   US 5,680,565 - Abstract 1-2 and last 3 lines, 4:9-19, 4:22-28, 12:24-25,
%                  13:42-44, 13:48-54, 13:59-64, 14:12-21, 14:23-29, 14:61-66,
%                  15:1-12, 15:16-39

The FSM in the PMH handles the fast path of the entire address translation
process, which assumes no address translation fault~(\S~\ref{sec:faults})
occurs
\citep{intel1996dtlb, intel1997pmh, intel1999exceptions, intel1999events}, and
no page table entry needs to be modified \citep{intel1997pmh}.

When the PMH FSM detects the conditions that trigger a Page Fault or a General
Protection Fault, it communicates a microcode event code, corresponding to the
detected fault condition, to the execution unit (\S~\ref{sec:out_of_order})
responsible for memory operations~\citep{intel1996dtlb, intel1997pmh,
intel1999exceptions, intel1999events}. In turn, the execution unit triggers the
fault by associating the event code with the micro-op that caused the address
translation, as described in the previous section.

% Performance Monitoring Events for Next Generation Intel Core Processor:
%     SDM S 19.2
% Non-Architectural Performance Events of the Processor Core Supported by
%     Skylake Microarchitecture: SDM Table 19-3

The PMH FSM does not set the Accessed or Dirty
attributes~(\S~\ref{sec:page_table_attributes}) in page table entries. When
it detects that a page table entry must be modified, the FSM issues a microcode
event code for a page walk assist \citep{intel1997pmh}. The microcode handler
performs the page walk again, setting the A and D attributes on page table
entries when necessary \citep{intel1997pmh}. This finding was indirectly
confirmed by the description for a PEBS event in the most recent SDM release.

The patents at the core of our descriptions above \citep{intel1996dtlb,
intel1997events, intel1997pmh, intel1999exceptions, intel1999events} were all
issued between 1996 and 1999, which raises the concern of obsolescence. As
Intel would not be able to file new patents for the same specifications, we
cannot present newer patents with the information above. Fortunately, we were
able to find newer patents that mention the techniques described above,
proving their relevance to newer CPU models.

% Microcode can prevent PMH writes
%   US 7,552,255 - 7:52-55

Two 2014 patents~\citep{hildesheim2014ptm, raikin2014tlb} mention that the PMH
is executing a FSM which issues stuffing loads to obtain page table entries.
A 2009 patent~\citep{intel2009pipeline} mentions that microcode is invoked after
a PMH walk, and that the microcode can prevent the translation result produced
by the PMH from being written to the TLB.

% VGATHER* / VSCATTER* - SDM instruction reference
% Microcode assists used for difficult cases in gather
%   US 8,688,962 - 5:26-30
% Scatter / gather implemented in microcode and hardware
%   US 8,447,962 - 11:28-30, 12:15-17, 12:20-23, 12:25-28

A 2013 patent~\citep{intel2013scattergather} and a 2014 patent
\citep{intel2014gather} on scatter / gather instructions disclose that the newly
introduced instructions use a combination of hardware in the execution units
that perform memory operations, which include the PMH. The hardware issues
microcode assists for slow paths, such as gathering vector elements stored in
uncacheable memory~(\S~\ref{sec:memory_io}), and operations that cause Page
Faults.

% Microcode used when vAPIC memory checks fail
%   US 8,806,104 - 2:38-55, 3:12-17, 3:21-35, 3:39-43, 4:6-10, 4:29-42,
%                  4:55-57, 5:6-7, 5:10-17, 5:37-53, 5:58-60, 6:16-19,
%                  7:45-47, 8:30-36, 9:12-15

A 2014 patent on APIC (\S~\ref{sec:interrupts}) virtualization
\citep{intel2014vapic} describes a memory execution unit modification that
invokes a microcode assist for certain memory accesses, based on the contents
of some range registers. The patent also mentions that the range registers are
checked when the TLB miss occurs and the PMH is invoked, in order to decide
whether a fast hardware path can be used for APIC virtualization, or a
microcode assist must be issued.

The recent patents mentioned above allow us to conclude that the PMH in recent
processors still relies on an FSM and stuffed loads, and still uses microcode
assists to handle infrequent and complex operations. This assumption plays a
key role in estimating the implementation complexity of architectural
modifications targeting the processor's address translation mechanism.


\HeadingLevelC{Microcode and Booting}
\label{sec:microcode_sec}

The SDM states that microcode performs the Built-In Self Test (BIST,
\S~\ref{sec:uefi_sec_details}), but does not provide any details on the
rest of the CPU's hardware initialization.

% Microcode initializes the CPU
%   US 8,806,104 - 4:36-41

% ACM is signed with a key in the CPU, verified by microcode
%   US 7,752,428 - 5:1-21
% EFI SEC and PEI core run in Cache-as-RAM mode, SEC loaded by microcode
%   US 7,752,428 - 5:22-28, 5:41-49, 5:56-5:67, 6:1-3, 6:9-13 6:44-6:58,
%                  7:11-45, Fig 4
%   US 8,296,528 - 7:39-50

% Microcode in ROM reads an ACM that measures firmware
%   US 8,429,418 - 2:14-28, 3:52-61, 4:2-8, Fig 1, Fig 2
% Microcode loads the ACM in Cache-as-RAM, authenticates using microcode crypto
%   US 8,429,418 - 4:13-20, 4:22-28
% Microcode runs at CPU reset and hashes an ACM in the firmware
%   US 8,321,931 - 6:47-52, 11:39-43, 11:45-47, 11:52-57, 12:6-11
%   US 8,301,907 - 4:8-10, 5:4-11, 5:16-17

In fact, the entire SEC implementation on Intel platforms is contained in the
processor microcode \citep{datta2010trustedboot, datta2013acm, intel2014vapic}.
This implementation has desirable security properties, as it is significantly
more expensive for an attacker to tamper with the MSROM circuitry
(\S~\ref{sec:microcode_structure}) than it is to modify the contents of the
flash memory module that stores the UEFI firmware.
\S~\ref{sec:physical_chip_attacks} and \S~\ref{sec:device_attacks} describe the
broad classes of attacks that an Intel platform can be subjected to.

The microcode that implements SEC performs MP initialization
(\S~\ref{sec:uefi_sec_details}), as suggested in the SDM. The microcode then
places the BSP into Cache-as-RAM (CAR) mode, looks up the PEI
\textit{Authenticated Code Module}~(ACM) in the Firmware Interface Table (FIT),
loads the PEI ACM into the cache, and verifies its signature
(\S~\ref{sec:uefi_sec_details}) \citep{datta2010trustedboot, intel2012patching,
intel2012uefihypervisor, intel2012ltsx, datta2013acm}. Given the structure of
ACM signatures, we can conclude that Intel's microcode contains implementations
of RSA decryption and of a variant of SHA hashing.

The PEI ACM is executed from the CPU's cache, after it is loaded by the
microcode \citep{datta2010trustedboot, intel2012patching, datta2013acm}. This
removes the possibility for an attacker with physical access to the SPI flash
module to change the firmware's contents after the microcode computes its
cryptographic hash, but before it is executed.

% Microcode handles SIPI
%   US 8,301,907 - 4:31-33, Fig 2

On motherboards compatible with LaGrande Server Extensions (LT-SX, also known
as Intel TXT for servers), the firmware implementing PEI verifies that each CPU
connected to motherboard supports LT-SX, and powers off the CPU sockets that
don't hold processors that implement LT-SX \citep{intel2012ltsx}. This prevents
an attacker from tampering with a TXT-protected VM by hot-plugging a CPU in a
running computer that is inside TXT mode. When a hot-plugged CPU passes
security tests, a hypervisor is notified that a new CPU is available. The
hypervisor updates its internal state, and sends the new CPU a SIPI. The new
CPU executes a SIPI handler, inside microcode, that configures the CPU's state
to match the state expected by the TXT hypervisor \citep{intel2012ltsx}. This
implies that the AP initialization described in \S~\ref{sec:uefi_sec_details}
is implemented in microcode.


\HeadingLevelC{Microcode Updates}
\label{microcode:updates}

The SDM explains that the microcode on Intel CPUs can be updated, and describes
the process for applying an update. However, no detail about the contents of an
update is provided. Analyzing Intel's microcode updates seems like a promising
avenue towards discovering the structure of microcode system. Unfortunately, the
updates have so far proven to be inscrutable \citep{chen2014microcode}.

% Microcode encryption
%   US 8,296,528 - 5:12-19

The microcode updates cannot be easily analyzed because they are encrypted,
hashed with a cryptographic hash function like SHA-256, and signed using RSA or
elliptic curve cryptography \citep{intel2012patching}. The update facility is
implemented entirely in microcode, including the decryption and signature
verification \citep{intel2012patching}.

\citep{hawkes2012microcode} independently used fault injection and timing
analysis to conclude that each recent Intel microcode update is signed with a
2048-bit RSA key and a (possibly non-standard) 256-bit hash algorithm, which
agrees with the findings above.

% Microcode sequesters cache ways
%   US 8,296,528 - 6:28-46, 7:9-34 7:48-51, 7:61-8:10, 8:12-57

The microcode update implementation places the core's cache into No-Evict Mode
(NEM, documented by the SDM) and copies the microcode update into the cache
before verifying its signature \citep{intel2012patching}. The update facility
also sets up an MTRR entry to protect the update's contents from modifications
via DMA transfers \citep{intel2012patching} as it is verified and applied.

While Intel publishes the most recent microcode updates for each of its CPU
models, the release notes associated with the updates are not publicly
available. This is unfortunate, as the release notes could be used to confirm
guesses that certain features are implemented in microcode.

However, some information can be inferred by reading through the Errata section
in Intel's Specification Updates
\citep{intel2010errata, intel2015errata, intel2015errata2}. The phrase ``it is
possible for BIOS\footnote{Basic Input/Output System (BIOS) is the predecessor
of UEFI-based firmware. Most Intel documentation, including the SDM, still uses
the term BIOS to refer to firmware.} to contain a workaround for this erratum''
generally means that a microcode update was issued. For example, Errata AH in
\citep{intel2010errata} implies that string instructions (\texttt{REP MOV}) are
implemented in microcode, which was confirmed by Intel
\citep{abraham2006repmov}.

% Microcode used for VMX instructions
%   US 8,806,104 - 2:61-66

Errata AH43 and AH91 in \citep{intel2010errata}, and AAK73 in
\citep{intel2015errata} imply that address translation (\S~\ref{sec:paging}) is
at least partially implemented in microcode. Errata AAK53, AAK63, and AAK70,
AAK178 in \citep{intel2015errata}, and BT138, BT210,  in \citep{intel2015errata2}
imply that VM entries and exits (\S~\ref{sec:faults}) are implemented in
microcode, which is confirmed by the APIC virtualization patent
\citep{intel2014vapic}.
