Commit 6b718187 authored by Robert Ricci's avatar Robert Ricci

Update eval form for 2015

parent 33f43365
...@@ -30,27 +30,35 @@ ...@@ -30,27 +30,35 @@
\section{Questions to Answer} \section{Questions to Answer}
\question{List the questions you think the paper needs to answer in order \question{List the questions you think the paper needs to answer in order
to persuasively argue that it has accomplished its goals. Which of to persuasively argue that it has accomplished its goals. Please number
these questions can be answered by evaluation? For those that can, what the questions in order to refer to them later. For each question, state:
type of evaluation would you find most convincing? What factors would \begin{itemize}
you expect to see investigated, and over what ranges (order of magnitude)? \item Can the question be answered with an empirical evaluation?\\
What metrics would you expect to see used? For questions that cannot If so:
be answered with a typical evaluation, what kinds of arguments would you \begin{itemize}
find persuasive?} \item What type of evaluation (simulation, analytic,
measurement) would you find convincing?
\item What factors would you expect to see investigated, and
over what ranges (order of magnitude)?
\item What metrics would you expect to see used?
\end{itemize}
If not:
\begin{itemize}
\item What kinds of arguments would you find persuasive?
\end{itemize}
\end{itemize}
}
\section{System Under Test} \section{System Under Test}
\question{What did the authors chose as the boundaries for the System Under \question{
Test? Did they define them clearly? If not, try defining them yourself, \begin{itemize}
either based on clues in the paper or based on what you think the \item List the inputs and outputs to the System Under Test
boundaries should be.} \item What components (software, hardware, network, etc.) are
necessary to complete the process of turning the inputs into
\section{Parameters and Factors} outputs?
\end{itemize}
\question{What (system and/or workload) parameters do the authors list? Did }
they include any that were not necessary, or leave out any you think
to be important? Which of the parameters did they vary as factors,
and do you think there are any unnecessary or left-out factors?}
\section{Major Evaluations} \section{Major Evaluations}
...@@ -63,13 +71,14 @@ ...@@ -63,13 +71,14 @@
\item Factor(s) varied \item Factor(s) varied
\item Workload used \item Workload used
\item Conclusion / outcome of the evaluation \item Conclusion / outcome of the evaluation
\item Which, if any, of your questions from Part 3 did it address?
\end{itemize} \end{itemize}
} }
\section{Questions Revisited} \section{Questions Revisited}
\question{Looking back at the list of questions, which were answered to your \question{Looking back at the list of questions, which were answered to your
satisfaction, and which were not?} satisfaction, and which were not? (Refer to them by number) }
\section{Goals Revisited} \section{Goals Revisited}
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment