Commit a4859d19 authored by Robert Ricci's avatar Robert Ricci

Merge branch 's15'

parents 9aec555a 92f7575a
SUBDIRS=syllabus schedule assignments lectures
SUBDIRS=syllabus schedule assignments lectures handouts
include Makerules
......
DOCUMENTS=final-report-format
include ../../Makerules
\documentclass{article}[10pt]
\input{../../texstuff/fonts.sty}
\usepackage[margin=1.25in]{geometry}
\usepackage{url}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
\title
\end{center}
\vspace{0.75in}
}
\renewcommand{\title}{CS 6963 Final Project---Final Report Format}
\begin{document}
\mytitle
Your report should be thorough, and reflect the level of effort you put into
the project.
While length is not directly a criterion, it will likely require
several pages will be required to adequately cover everything.
Reports should be carefully edited and nicely formatted.
Produce a document you can be proud of.
Your report must contain the following elements:
\begin{enumerate}
\item \textbf{Paper:} State the title of the paper, authors, where and
when it appeared. If possible, provide a link to the conference
proceedings, journal or digital library (ACM DL, IEEE Xplore, ArXiv,
etc.) where the paper appeared.
\item \textbf{Introduction:} Provide a 1--2 paragraph summary, in your
own words, of the paper.
\item \textbf{Goals:} Concretely, what are the major problems that the
original paper is trying to solve? What are its major claims?
\item \textbf{Results achieved by original authors:} What are the key
results that the original authors achieved in service of their goals
and support of their claims?
\item \textbf{Reproduction goal:} Which figure(s) or other results from the
original paper did you attempt to reproduce? Include the figure,
table, etc.
\item \textbf{Reproduction motivation:} Why did you choose to target
these specific results to reproduce?
\item \textbf{Reproduction results:} Present the results of your
reproduction study, including your versions of any charts, graphs, etc.
Do your results match those of the original paper? If not, is there an
explanation for the difference, and do your results still support
the original paper's claims?
\item \textbf{Extensions to the original results:} What improvements
did you make to the original experiments, or what additional
experiments did you run? What was the scientific goal of these
extensions? Do they help to support or refute the claims of the
original paper, or do they help shed light on an issue that was
not originally identified as a goal?
\item \textbf{Challenges:} What challenges did you encounter in attempting
to reproduce the results of the original paper, and what did you have
to do in order to overcome them?
\item \textbf{Critique:} Provide a high-level analysis of the main thesis
that you explored-–-does it hold up, does it depend on assumptions not
stated in the original paper?
\item \textbf{CloudLab profile:} Provide a link to the profile you made on
CloudLab to help others run the same experiments.
\end{enumerate}
You can find good examples of reports here: \url{https://reproducingnetworkresearch.wordpress.com/}
\end{document}
Final Project Score:
XX / 10 : Paper summary - summary of the paper, results acheived by the original
authors, etc.
XX / 15 : Reproduction - description of motivation, goals, results obtained,
extension to the original results
XX / 10 : Challenges and critique - critical discussion of challenges faced,
and analysis of whether the orginal result holds up
XX / 15 : Reproducability - is it possible, given the profile, instructions,
and other materials checked in, to reproduce your experiments
======
Total score: XX / YY
......@@ -26,8 +26,8 @@
}
\renewcommand{\title}{CS 6963 Homework \#1}
\newcommand{\assigned}{January 07}
\newcommand{\due}{January 09 2:00 PM}
\newcommand{\assigned}{January 13}
\newcommand{\due}{January 15 midnight}
\newcommand{\turnin}{branch \texttt{hw1-submit} -- file \texttt{submit.pdf}}
\begin{document}
......@@ -55,7 +55,7 @@ git config --global user.email "YOUR EMAIL"
\item Make a repository for this assignment
\begin{enumerate}
\item Fork the assignment repository: \url{http://gitlab.flux.utah.edu/cs6963-s14/cs6963-hw1}
\item Fork the assignment repository: \url{http://gitlab.flux.utah.edu/cs6963-s15/cs6963-hw1}
\item On your local machine, clone your fork with \texttt{`git clone \ldots'}
\end{enumerate}
......@@ -75,7 +75,7 @@ git push
\item Add your grading repository as a remote (replacing
\texttt{USERNAME} with your GitLab username):
\begin{verbatim}
git remote add grading git@gitlab.flux.utah.edu:cs6963-s14-grading/USERNAME.git
git remote add grading git@gitlab.flux.utah.edu:cs6963-s15-grading/USERNAME.git
\end{verbatim}
\item Push your master branch to the grading repository with
the name \texttt{hw1-submit} to complete the submission:
......
\documentclass{article}[10pt]
\usepackage[no-math]{fontspec}
\input{../../texstuff/fonts.sty}
\usepackage{sectsty}
\usepackage{paralist}
\usepackage{ragged2e}
......@@ -11,11 +11,6 @@
\newcommand*{\git}{\texttt{git}}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
\newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A}
\allsectionsfont{\titlefont}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
......@@ -24,15 +19,15 @@
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~2:00 PM \\[3pt]
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~5:00 PM \\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Homework \#2}
\newcommand{\assigned}{January 09}
\newcommand{\due}{January 16}
\newcommand{\assigned}{January 15}
\newcommand{\due}{January 23}
\newcommand{\turnin}{branch \texttt{hw2-submit} -- file \texttt{submit.pdf}}
\begin{document}
......
......@@ -3,14 +3,14 @@ HW #2 Score:
XX / 3 : Question 1
One point for each table
One point for explaning the "trick"
One point for explaining the "trick"
XX / 7 : Question 2
One point for project description, including an evaluatable goal
One point for comprehesnive list of services, including outcomes
One point for comprehensive list of services, including outcomes
One point for comprehensive list of metrics; must be measurable
One point list of system paramters and workload parameters
One point list of system parameters and workload parameters
One point for comprehensive list of factors and their ranges
One point for choice of evaluation technique
One point for description of workload
......
\documentclass{article}[10pt]
\usepackage[no-math]{fontspec}
\input{../../texstuff/fonts.sty}
\usepackage{sectsty}
\usepackage{paralist}
\usepackage{ragged2e}
......@@ -9,13 +9,6 @@
\usepackage{url}
\usepackage[margin=1.25in]{geometry}
\newcommand*{\git}{\texttt{git}}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
\newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A}
\allsectionsfont{\titlefont}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
......@@ -24,15 +17,15 @@
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~2:00 PM\\[3pt]
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~5:00 PM\\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Homework \#3}
\newcommand{\assigned}{January 16}
\newcommand{\due}{January 23}
\newcommand{\assigned}{January 22}
\newcommand{\due}{January 30}
\newcommand{\turnin}{branch \texttt{hw3-submit} -- file \texttt{submit.pdf}}
\begin{document}
......@@ -44,25 +37,33 @@
\item What metrics might you use to compare the following systems? Don't limit
yourself to the metrics discussed in the textbook, and you may consult any
sources you wish to familiarize yourself with these systems. Your response
should be clear about the types of higher-level questions the metrics would
help you to answer.
must state the types of higher-level questions the metrics would help you
to answer.
\begin{enumerate}
\item Two routing protocols
\item Two wireless MAC layers (eg. 802.11, E-UTRAN, etc.)
\item Two block-level network storage systems (SANs)
\item Two network intrusion detection systems (NIDS)
\item Two key-value storage systems
\item Two database systems
\item Two application-layer protocols for a game
\item Two intrusion detection systems (IDSes)
\end{enumerate}
\item You are tasked with creating a synthetic workload to evaluate a proposed
improvement to a mobile wireless network (such as LTE/EPC). Your goal is
to answer the question “how many additional users could the network support
with \textit{change~X}, assuming that all other properties of the network
remain the same?” Pretend that you are allowed to ask the operators of the
network for any data you wish about the network's current workload
(including both aggregate statistics and full traces). What data would you
ask for, and what properties of the current workload would you study
in order to begin designing your synthetic workload? State explicitly any
assumptions you make about the nature of \emph{change~X}.
improvement to a network. Your goal is to answer the question “how many
additional users could the network support with \textit{change~X}, assuming
that all other properties of the network remain the same?” Pretend that you
are allowed to ask the operators of the network for any data you wish about
the network's current workload (including both aggregate statistics and
full traces). Answer the following questions (you do not need to actually
“design” the workload):
\begin{enumerate}
\item What data would you ask for?
\item What properties of the current workload would you study to begin
designing your synthetic workload?
\end{enumerate}
State explicitly any assumptions you make about the nature of the network
or of change \emph{change~X} (such as the type of network, what layer of
the network the change is at, etc.)
\end{enumerate}
......
HW #3 Score:
XX / 4 : Question 1
One point for part (a); examples include messages sent, convergence
time, stretch, percentage of packets that end up in routing loops
One point for part (b); examples include throughput, latency, loss rate,
overhead due to headers or signaling, energy use, fairness
One point for part (c); examples include througput, IOPS, size, latency,
fairness, durability
One point for part (d); examples include throughput, false positive rate,
false negative rate
One point for part (a); examples include read and write throughput,
read and write latency, overhead spent on index structures, etc.
One point for part (b); examples include query throughput (for reads,
inserts, and updates), query latency, and resource consumption
(eg. CPU and RAM)
One point for part (c); examples include bandwidth consumed, latency (how
long do updates take to reach other clients), and accuracy (how
inconsistent is state between clients)
One point for part (d); examples include true and false positive rate,
true and false negative rate, throughput
One half point may be deducted per part for failing to explain the types of
questions the metric would help answer
XX / 6 : Question 2
Two points for discussion of data to request; good examples include
traces of connection/disconnection events, traces of calls, traces
of data sessions, traces of geographical locations, or aggregate
statistics covering any of the properties listed below
Two points for discussion of data to request; examples include traces
of packets or session, number of users over time, etc., and/or
aggregate statistics covering the above
Three points for discussion of properties to study; good examples include
peak number of users, variations at different timescales (seconds,
minutes, hours, days, weeks), distribution of users across access
points, data behavior including number of connections, rate of
connections, amount of data transfered per connection
average and peak number of users, variations at different timescales
(seconds, minutes, hours, days, weeks), resources consumed per user,
etc.
One point for stating assumptions about the change, or being
sufficiently general that no assumptions seem to be needed
......
\documentclass{article}[10pt]
\usepackage[no-math]{fontspec}
\input{../../texstuff/fonts.sty}
\usepackage{sectsty}
\usepackage{paralist}
\usepackage{ragged2e}
......@@ -9,13 +9,6 @@
\usepackage{url}
\usepackage[margin=1.25in]{geometry}
\newcommand*{\git}{\texttt{git}}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
\newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A}
\allsectionsfont{\titlefont}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
......@@ -24,15 +17,15 @@
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~2:00 PM\\[3pt]
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~5:00 PM\\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Homework \#4}
\newcommand{\assigned}{January 23}
\newcommand{\due}{January 30}
\newcommand{\assigned}{January 29}
\newcommand{\due}{February 6}
\newcommand{\turnin}{branch \texttt{hw4-submit} -- file \texttt{submit.pdf}}
\begin{document}
......
\documentclass{article}[10pt]
\usepackage[no-math]{fontspec}
\usepackage{sectsty}
\input{../../texstuff/fonts.sty}
\usepackage{paralist}
\usepackage{ragged2e}
\usepackage{MnSymbol}
......@@ -9,13 +8,6 @@
\usepackage{url}
\usepackage[margin=1.25in]{geometry}
\newcommand*{\git}{\texttt{git}}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
\newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A}
\allsectionsfont{\titlefont}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
......@@ -31,8 +23,8 @@
}
\renewcommand{\title}{CS 6963 Homework \#5}
\newcommand{\assigned}{January 30}
\newcommand{\due}{February 11}
\newcommand{\assigned}{February 12}
\newcommand{\due}{February 20}
\newcommand{\turnin}{branch \texttt{hw5-submit} -- file \texttt{submit.pdf}}
\begin{document}
......@@ -45,14 +37,11 @@
\item Do Exercise 12.7 from the textbook
\item Do Exercise 12.10 from the textbook
\item Do Exercise 12.11 from the textbook
\item Do Exercise 12.13 from the textbook
\item Do Exercise 12.14 from the textbook
\item Do Exercise~13.2 from the textbook
\item Do Exercise~13.3 from the textbook (note: Table~11.2 can be found on
page~167)
\end{enumerate}
\end{document}
HW #5 Score:
XX / 3 : Exercise 12.1
One half point for correct answer to each sub-question
XX / 2 : Exercise 12.1
One third point for correct answer to each sub-question
XX / 2 : Exercise 12.7
One half point for correct answer to each sub-question
XX / 1 : Exercise 12.10
One point for picking correct index of central tendency; justification
required to get the point
XX / 1 : Exercise 12.11
One point for picking correct index of central tendency; justification
required to get the point
......@@ -18,9 +14,13 @@ XX / 1 : Exercise 12.13
One point for picking correct index of dispersion; justification required
to get the point
XX / 2 : Exercise 12.14
One point for correct index of dispersion and justification, one point for
computing range, variance/stddev, 90 percentiles, SIQR (MAD not required)
XX / 2 : Exercise 13.2
One third point for the correct answer to each of the first three
sub-questions, one half point for the correct answer to each of the
second two
XX / 2 : Exercise 13.3
One point for the correct answer to each sub-question
======
Total score: XX / YY
\documentclass{article}[10pt]
\usepackage[no-math]{fontspec}
\usepackage{sectsty}
\input{../../texstuff/fonts.sty}
\usepackage{paralist}
\usepackage{ragged2e}
\usepackage{MnSymbol}
......@@ -9,13 +8,6 @@
\usepackage{url}
\usepackage[margin=1.25in]{geometry}
\newcommand*{\git}{\texttt{git}}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
\newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A}
\allsectionsfont{\titlefont}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
......@@ -24,37 +16,44 @@
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~midnight\\[3pt]
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~by~midnight\\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Homework \#6}
\newcommand{\assigned}{February 11}
\newcommand{\due}{February 18}
\newcommand{\assigned}{February 25}
\newcommand{\due}{March 1}
\newcommand{\turnin}{branch \texttt{hw6-submit} -- file \texttt{submit.pdf}}
\begin{document}
\mytitle
\textbf{Important notes:}
\begin{itemize}
\item You must turn in evidence of work: code, spreadsheets, pictures of
handwritten notes, etc. This can be in files other than
\texttt{submit.pdf}, but please mention them in \texttt{submit.pdf}
so I don't have to do a lot of digging to find them.
\item All answers that are real numbers (not ints) must have at least 6
significant figures, so that they contain data not in the solution
manual in the book.
\item You may wish to refer to this file: \url{https://gitlab.flux.utah.edu/ricci/ens/blob/s15/lectures/lecture12/rcommands.txt} \textbf{NOTE: Most of the numbers in this file are \textit{not} directly from the book!}
\end{itemize}
\begin{enumerate}
\item \textbf{NOTE: This part must be done before class on the 18th; if not,
no points will be awarded.}
Follow the instructions here:
\url{http://groups.geni.net/geni/wiki/SignMeUp} to sign up an account
on GENI. In Step~1, you should select the University of Utah as your
identity provider. In Step~2, you should follow part 2a and
join the \texttt{CS6963-S14} project; you may ignore part 2b.
\textit{To ``answer'' this question in your submit.pdf file, simply
indicate whether you did this on time or not.}
\item Do Exercise 12.1 from the textbook
\item Do Exercise 12.7 from the textbook
\item Do Exercise 12.11 from the textbook
\item Do Exercise 12.13 from the textbook
\item Do Exercise~13.2 from the textbook
\item Do Exercise~13.3 from the textbook (note: Table~11.2 can be found on
page~167)
\end{enumerate}
\end{document}
HW #6 Score:
XX / 1 : Question 1
One point for signing up for an account on time
XX / 2 : Exercise 12.1
One third point for correct answer to each sub-question
XX / 5 : Question 2
One point each for correct answers on parts a through e
XX / 2 : Exercise 12.7
One half point for correct answer to each sub-question
XX / 4 : Question 3
Two points each for correct answers on parts and b
XX / 1 : Exercise 12.11
One point for picking correct index of central tendency; justification
required to get the point
XX / 1 : Exercise 12.13
One point for picking correct index of dispersion; justification required
to get the point
XX / 2 : Exercise 13.2
One third point for the correct answer to each of the first three
sub-questions, one half point for the correct answer to each of the
second two
XX / 2 : Exercise 13.3
One point for the correct answer to each sub-question
======
Total score: XX / YY
DOCUMENTS=handout
include ../../Makerules
\documentclass{article}[10pt]
\input{../../texstuff/fonts.sty}
\usepackage{paralist}
\usepackage{ragged2e}
\usepackage{MnSymbol}
\usepackage{fontawesome}
\usepackage{url}
\usepackage[margin=1.25in]{geometry}
\newcommand*{\git}{\texttt{git}}
\newcommand{\mytitle}{
\begin{center}%
\LARGE\titlefont%
\title
\end{center}
\vspace{5pt}
\begin{center}%
\large
{\titlefont Assigned:} \assigned \hspace{2em} {\titlefont Due:} \due~midnight\\[3pt]
{\titlefont Submission:} \turnin
\end{center}
\vspace{3pt}
}
\renewcommand{\title}{CS 6963 Lab \#1}
\newcommand{\assigned}{March 03}
\newcommand{\due}{March 06}
\newcommand{\turnin}{branch \texttt{hw1-submit} -- file \texttt{notes,submit.pdf,results/}}
\begin{document}
\mytitle
For this assignment, we will be reproducing results from the paper
``TCP ex Machina: Computer-Generated Congestion Control'' by Winstein and
Balakrishnan that appeared in SIGCOMM '13.
You can find the paper and instructions for repeating one of its key graphs
at \url{http://web.mit.edu/remy/}
\textbf{IMPORTANT:} As you go through this process, I want you to document
everything you do, to the level that someone else could repeat your steps!
For an example, see:\\
\indent\url{https://gitlab.flux.utah.edu/ricci/remyrepro/blob/master/notes.md}
Follow the steps in \url{reproduce-steps.html} in this repository to reproduce
Figure~4 from the Remy paper.
Turn in the following:
\begin{itemize}
\item Your notes (\texttt{notes.*})
\item A \texttt{results/} directory, containing the contents of the
\texttt{graphdir} mentioned in the \texttt{reproduce-steps} document.
\item Brief reflections (approx. 1 paragraph each) on the following (\texttt{submit.pdf}):
\begin{enumerate}
\item Do you trust the results of the paper more now? Why or why not?
\item Do you feel like you have a better sense of how the system
and/or code works?
\item What did the authors do right in terms of making their results
reproducible?
\item What could the authors have done to make the process of
repeating their results easier?
\end{enumerate}
\end{itemize}
\end{document}
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
This diff is collapsed.
set xrange [0:4.81667320313] reverse
set yrange [0:1.81830054762]
#set logscale x 2
#set logscale y 2
set xtics ("1" log(1)/log(2), "2" log(2)/log(2), "4" log(4)/log(2), "8" log(8)/log(2), "16" log(16)/log(2), "32" log(32)/log(2), "64" log(64)/log(2), "128" log(128)/log(2), "256" log(256)/log(2), "512" log(512)/log(2), "1024" log(1024)/log(2), "2048" log(2048)/log(2), "4096" log(4096)/log(2), "8192" log(8192)/log(2), "16384" log(16384)/log(2), "32768" log(32768)/log(2))
set xlabel "Queueing delay (ms)"
set ylabel "Throughput (Mbps)"
set grid
#set title "15 Mbps dumbbell, Empirical distribution of flow lengths, nsrc 8"
unset key
set terminal svg fsize 14
set output "graph-8.svg"
set label "Vegas" at 1.4329594072761,0.01 point textcolor lt 1
set label "RemyCC-10" at 0.378511623253742,1.11 point textcolor lt 1
set label "Compound" at 2.92599941855622,0.81 point textcolor lt 1
set label "Cubic" at 3.26303440583379,1.06 point textcolor lt 1
set label "Cubic/sfqCoDel" at 4.45285896471381,1.26 point textcolor lt 1