Commit cfda6060 authored by Robert Ricci's avatar Robert Ricci

Lecture notes for second lecture

parent 864a9ac9
......@@ -4,6 +4,7 @@
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
......@@ -12,7 +13,7 @@
\title{CS6963 Lecture \#1}
\author{Robert Ricci}
\date{January 7, 2014}
\date{January 9, 2014}
......@@ -20,11 +21,61 @@
\1 Big thing one
\2 Little thing A
\2 Little thing B
\1 Leftover from last time
\2 Anyone didn't figure out homework?
\2 Pull changes from my graded version back into your repo
\1 Big thing two
\1 Evaluate as you go along: it's part of design
\2 We should always be solving some problem
\2 Very often that means doing early evaluation to precisely identify one
\2 Start with two questions:
\3 What is the problem I'm solving?
\3 How will I know if I solved it?
\3 \emph{Students: answer this question for your own work}
\1 \emph{First things to select?}
\2 Right measures of performance
\2 Right measurement environments
\2 Right techniques
\1 \emph{What kinds of problems are there to solve?}
\2 Performance tuning (parameter optimization)
\2 Bottleneck identification
\2 Workload characterization
\2 \sout{Capacity planning}
\2 \sout{Forecasting}
\1 Goals of the book (subset of course goals) - select appropriate:
\2 Evaluation techniques
\3 \emph{Measurement} (real system, emulation)
\3 \emph{Simulation} (eg. NS2, NS3, purpose-built simulators)
\3 \emph{Analytic modeling}
\2 Performance metrics
\3 \emph{Examples} (tput, latency (response time), simultaneous connections)
\3 \emph{Examples of non-performance metrics} (security, simplicity, ease of use)
\2 Workloads
\2 I would add: criteria for ``success''---too often, not defined well in papers
\2 Proper statistic techniques
\3 Not just eyeballing graphs
\3 \emph{How do you know when you've run enough experiments?}
\2 Design experiments to give the most information with the least effort
\1 Go through example 1.7 from book
\2 \emph{How do you compare them}
\2 \emph{Convince me each system is better}
\1 + Methodical approach to evaluation
\2 Always know how you got to where you are
\2 Keep everything
\2 Build robust tools on top of flaky/fragile environments
\2 If someone else can't get similar numbers, they should be considered suspect
\2 Don't be afraid of intermediate states not being ``correct''
\2 Don't be afraid to try things that might not work: branches
\1 \textbf{For next time}
\2 Read Chapter 1 and Chapter 2 to the end of 2.1
\2 HW\#2 now up, you have one week
\2 Remember, schedule in Canvas and GitLab
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment