\HeadingLevelC{Page Walker Memory Accesses}
\label{sec:pmh_walker}

In modern high-speed CPUs, address translation is performed by a hardware
\textbf{page walker} that traverses the page tables when a TLB miss occurs.
The page walker's latency greatly impacts the CPU's performance, so it is
implemented as a finite-state machine (FSM) that reads page table entries by
issuing DRAM read requests using physical addresses, over a dedicated bus to
the L1 cache.

Unsurprisingly, page walker modifications require a lot of engineering effort.
At the same time, Sanctum's security model demands that the page walker only
references enclave memory when traversing the enclave page tables, and only
references OS memory when translating the OS page tables. Fortunately,
these requirements can be satisfied without modifying the FSM. Instead, the security
monitor configures the circuit in Figure~\ref{fig:page_walker_filter} to ensure
that the page tables only point into allowable memory.

Sanctum's security monitor must guarantee that \texttt{ptbr} points into an OS
DRAM region, and \texttt{eptbr} points into a DRAM region owned by the enclave.
This secures the page walker's initial DRAM read. The circuit in
Figure~\ref{fig:page_walker_filter} receives each page table entry fetched by
the FSM, and sanitizes it before it reaches the page walker FSM.

The security monitor configures the set of DRAM regions that page tables may
reference by writing to a DRAM region bitmap (\texttt{drbmap}) register. The
sanitization circuitry extracts the DRAM region index from the address in the
page table entry, and looks it up in the DRAM region bitmap. If the address
does to belong to an allowable DRAM region, the sanitization logic forces the
page table entry's valid bit to zero, which will cause the page walker FSM to
abort the address translation and signal a page fault.

\begin{figure}[htb]
  \centering
  \includegraphics[width=80mm]{figures/page_walker_filter.pdf}
  \caption{
    Hardware support for per-enclave page tables: check page table
    entries fetched by the page walker.
  }
  \label{fig:page_walker_filter}
%\EndFigure
\end{figure}

Sanctum's security monitor and its attestation key are stored in DRAM regions
allocated to the OS. For security reasons, the OS must not be able to modify
the monitor's code, or to read the attestation key. Sanctum extends the page
table entry transformation described above to implement a Protected Address
Range~(PAR) for each set of page tables.

Each PAR is specified using a base register (\texttt{parbase}) register and a
mask register (\texttt{parmask}) with the same semantics as the variable Memory
Type Range registers (MTRRs) in the x86 architecture. The page table entry
sanitization logic in Sanctum's hardware extensions checks if each page table
entry points into the PAR by ANDing the entry's address with the PAR mask and
comparing the result with the PAR base. If a page table entry is seen with a
protected address, its valid bit is cleared, forcing a page fault.

The above transformation allows the security monitor to set up a memory range
that cannot be accessed by other software, and which can be used to securely store
the monitor's code and data. Entry invalidation ensures no page table entries
are fetched from the protected range, which prevents the page walker FSM from
modifying the protected region by setting accessed and dirty bits.

All registers above are replicated, as Sanctum maintains separate OS
and enclave page tables. The security monitor sets up a protected range in the
OS page tables to isolate its own code and data structures (most importantly
its private attestation key) from a malicious OS.

Figure~\ref{fig:core_changes} shows Sanctum's logic inserted between the page
walker and the cache unit that fetches page table entries.

% NOTE: these are mentioned in the eval section
Assuming a 64-bit RISC-V and the example cache above, the logic requires a
64-bit MUX, 54 AND gates, a 51-bit wide equality comparator (51 XNOR gates
and 50 AND gates), a 1-bit NOT gate, and a copy of the DRAM region index
extraction logic in \S~\ref{sec:dram_regions}, which could be just wire
re-routing if the DRAM configuration is known a priori.

\begin{figure}[htb]
  \centering
  \includegraphics[width=80mm]{figures/core_changes.pdf}
  \caption{
    Sanctum's page entry transformation logic in the context of a Rocket
    core.
  }
  \label{fig:core_changes}
%\EndFigure
\end{figure}
