\reviewtitle{Does Systems Research Measure Up?}
\reviewlabel{small97measure}
\reviewauthor{Small, C., Ghosh, N., Saleeb, H., Seltzer, M., Smith,
K.}

Results in systems papers much contain three crucial qualities:
\begin{description}
\item[reproducibility]
The paper must include sufficient information for independent verification.
\item[comparability]
The experiments run should allow the behavior of the system being
measured to be compared with that of other systems.
\item[statistical rigor]
The results must be statistically valid.
\end{description}

They analyze measurement sections of conferences from the early 1990s
and find that they often do not contain these qualities.

``Trace-based simulation is often the simplest way to guarantee that an
experiment is reproducible....When traces cannot be made available,
stocastic simulation with a parameterized workload provides a
convenient alternative.''

Descriptions of equip