Commit 5d3674b6 authored by Robert Ricci's avatar Robert Ricci

Star ones I want to make sure we get to

parent b8193207
......@@ -4,6 +4,7 @@
\usepackage{sectsty}
\usepackage[margin=1.25in]{geometry}
\usepackage{outlines}
\usepackage{latexsym}
\setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A}
\setmonofont{Inconsolata}
......@@ -27,10 +28,10 @@
\1 Today is all about mistakes
\1 Mistakes from the book:
\2 No goals: identify problem (metrics, workloads, and methodology)
\2 Biased goals: prove my system better than theirs
\2 $\star$ No goals: identify problem (metrics, workloads, and methodology)
\2 $\star$ Biased goals: prove my system better than theirs
\2 Unsystematic approach: If you miss things, your conclusion may be wrong
\2 Analysis without understanding the problem: ``A problem well stated is
\2 $\star$ Analysis without understanding the problem: ``A problem well stated is
half solved'. Especially a problem when applying solutions to areas
outside expertise
\2 Incorrect performance metrics: Metrics that solve the problem, comparing
......@@ -40,33 +41,35 @@
\2 Wrong technique: Using the hammer that you have. Are you capturing all
important parts of the environment?
\2 Overlooking important parameters: Choice of parameters matters a lot
\2 Ignoring significant factors: factors are the parameters you vary.
Scientific method; control vs. variables. Sensitivity analysis:
\2 $\star$ Ignoring significant factors: factors are the parameters you
vary. Scientific method; control vs. variables. Sensitivity analysis:
something we rarely do!
\2 Inappropriate experiment design: how many experiments, how many
trials, parameter choice
\2 Inappropriate levels of detail: ``Forest for the trees''
\2 No analysis: have to tell the reader what they see
\2 $\star$ No analysis: have to tell the reader what they see
\2 Erroneous analysis: base analysis on bogus techniques such as
average of ratios or bad experiments
\2 No sensitivity analysis: Analysis gives you evidence, not facts.
\2 $\star$ No sensitivity analysis: Analysis gives you evidence, not facts.
What would change if setting changes?
\2 Ignoring errors in input: errors may be systematically biased
(eg. queues too long, drop work)
\2 Improper treatment of outliers: Is it an artifact of the evaluation
environment, or can it really happen?
\2 Assuming no change in the future: Big one for research, which is
supposed to be forward looking---assumptions and sensitivity analysis
\2 Ignoring variability: Don't just present mean! - eg. bimodal high/low
\2 $\star$ Assuming no change in the future: Big one for research, which is
supposed to be forward looking---assumptions and sensitivity analysis.
``Don't estimate the capacity needed for a bridge based on how many
people swim across today.''
\2 $\star$ Ignoring variability: Don't just present mean! - eg. bimodal high/low
distribution
\2 Too complex analysis: Don't make it complex just for the sake of seeming
hard
\2 Improper presentation of results: Remember, about persuasion
\2 Ignoring social aspects: Again, about persuasion. Remember: you know
\2 $\star$ Ignoring social aspects: Again, about persuasion. Remember: you know
a lot that the audience doesn't. Audience is rarely as expert as you in
the details of the thing you're writing about. Beliefs, values,
language, jargon.
\2 Omitting assumptions and limitations: Important in research---under what
\2 $\star$ Omitting assumptions and limitations: Important in research---under what
conditions do you expect your conclusions to be true?
\1 For next time
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment