Commit b91ccc0a authored by Robert Ricci's avatar Robert Ricci

Update paper eval forms

parent bcde5488
......@@ -30,27 +30,35 @@
\section{Questions to Answer}
\question{List the questions you think the paper needs to answer in order
to persuasively argue that it has accomplished its goals. Which of
these questions can be answered by evaluation? For those that can, what
type of evaluation would you find most convincing? What factors would
you expect to see investigated, and over what ranges (order of magnitude)?
What metrics would you expect to see used? For questions that cannot
be answered with a typical evaluation, what kinds of arguments would you
find persuasive?}
to persuasively argue that it has accomplished its goals. Please number
the questions in order to refer to them later. For each question, state:
\begin{itemize}
\item Can the question be answered with an empirical evaluation?\\
If so:
\begin{itemize}
\item What type of evaluation (simulation, analytic,
measurement) would you find convincing?
\item What factors would you expect to see investigated, and
over what ranges (order of magnitude)?
\item What metrics would you expect to see used?
\end{itemize}
If not:
\begin{itemize}
\item What kinds of arguments would you find persuasive?
\end{itemize}
\end{itemize}
}
\section{System Under Test}
\question{What did the authors chose as the boundaries for the System Under
Test? Did they define them clearly? If not, try defining them yourself,
either based on clues in the paper or based on what you think the
boundaries should be.}
\section{Parameters and Factors}
\question{What (system and/or workload) parameters do the authors list? Did
they include any that were not necessary, or leave out any you think
to be important? Which of the parameters did they vary as factors,
and do you think there are any unnecessary or left-out factors?}
\question{
\begin{itemize}
\item List the inputs and outputs to the System Under Test
\item What components (software, hardware, network, etc.) are
necessary to complete the process of turning the inputs into
outputs?
\end{itemize}
}
\section{Major Evaluations}
......@@ -63,13 +71,14 @@
\item Factor(s) varied
\item Workload used
\item Conclusion / outcome of the evaluation
\item Which, if any, of your questions from Part 3 did it address?
\end{itemize}
}
\section{Questions Revisited}
\question{Looking back at the list of questions, which were answered to your
satisfaction, and which were not?}
satisfaction, and which were not? (Refer to them by number) }
\section{Goals Revisited}
......
......@@ -24,35 +24,34 @@
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~2:00 PM\\[3pt]
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~3:40 PM\\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Paper Analysis \#4}
\newcommand{\assigned}{April 01}
\newcommand{\due}{April 10}
\newcommand{\turnin}{branch: \texttt{papers4-submit} \newline files: \texttt{evals/\{keller-mobisys12,varadarajan-ccs12\}.pdf}}
\newcommand{\assigned}{April 5}
\newcommand{\due}{April 21}
\newcommand{\turnin}{branch: \texttt{papers4-submit} \newline files: \texttt{evals/\{titchkosky-sigmetrics03,jeong-nsdi14\}.pdf}}
\begin{document}
\mytitle
Answer the set of questions in \texttt{evals/paper-eval.pdf} for each of the
following three papers:
following two papers:
\begin{itemize}
\item ``MicroCast: cooperative video streaming on smartphones'' by
Keller et~al., MobiSys~2012
\item ``Resource-freeing attacks: improve your cloud performance (at
your neighbor's expense)'' by Varadarajan et~al., CCS~12
\item ``A Performance Comparison of Dynamic Web Technologies'' by
Titchkosky, Arlitt, and Williamson, in ACM SIGMETRICS 2003.
\item ``mTCP: A Highly Scalable User-level TCP Stack for Multicore Systems'' by Jeong et al., in NSDI 2014
\end{itemize}
For your convenience, the papers can be found in the repository in the
\texttt{papers/} directory.
Turn in three separate PDFs in the \texttt{evals/} directory, with each filename
Turn in two separate PDFs in the \texttt{evals/} directory, with each filename
matching the paper you are analyzing.
For the questions about your expectations for the paper (eg. \#2 and \#3), it's
best if you answer them as you read through the paper; eg. write about your
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment