lecturenotes.tex 3.27 KB
 Robert Ricci committed Jan 12, 2014 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 \documentclass{article}[12pt] \usepackage[no-math]{fontspec} \usepackage{sectsty} \usepackage[margin=1.25in]{geometry} \usepackage{outlines} \setmainfont[Numbers=OldStyle,Ligatures=TeX]{Equity Text A} \setmonofont{Inconsolata} \newfontfamily\titlefont[Numbers=OldStyle,Ligatures=TeX]{Equity Caps A} \allsectionsfont{\titlefont} \title{CS6963 Lecture \#3} \author{Robert Ricci} \date{January 14, 2014} \begin{document} \maketitle \begin{outline}  Robert Ricci committed Jan 12, 2014 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 \1 Left over from last time \2 Who still doesn't have the textbook? \2 To post assignments on canvas or not? \1 Today is all about mistakes \1 Mistakes from the book: \2 No goals: identify problem (metrics, workloads, and methodology) \2 Biased goals: prove my system better than theirs \2 Unsystematic approach: If you miss things, your conclusion may be wrong \2 Analysis without understanding the problem: A problem well stated is half solved'. Especially a problem when applying solutions to areas outside expertise \2 Incorrect performance metrics: Metrics that solve the problem, comparing apples to apples (eg. MIPS for CISC vs. RISC) Often chose easy rather than relevant \2 Unrepresentative workload: Representative of actual usage \2 Wrong technique: Using the hammer that you have. Are you capturing all important parts of the environment? \2 Overlooking important parameters: Choice of parameters matters a lot \2 Ignoring significant factors: factors are the parameters you vary. Scientific method; control vs. variables. Sensitivity analysis: something we rarely do! \2 Inappropriate experiment design: how many experiments, how many trials, parameter choice \2 Inappropriate levels of detail: Forest for the trees'' \2 No analysis: have to tell the reader what they see \2 Erroneous analysis: base analysis on bogus techniques such as average of ratios or bad experiments \2 No sensitivity analysis: Analysis gives you evidence, not facts. What would change if setting changes?  Robert Ricci committed Jan 12, 2014 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70  \2 Ignoring errors in input: errors may be systematically biased (eg. queues too long, drop work) \2 Improper treatment of outliers: Is it an artifact of the evaluation environment, or can it really happen? \2 Assuming no change in the future: Big one for research, which is supposed to be forward looking---assumptions and sensitivity analysis \2 Ignoring variability: Don't just present mean! - eg. bimodal high/low distribution \2 Too complex analysis: Don't make it complex just for the sake of seeming hard \2 Improper presentation of results: Remember, about persuasion \2 Ignoring social aspects: Again, about persuasion. Remember: you know a lot that the audience doesn't. Audience is rarely as expert as you in the details of the thing you're writing about. Beliefs, values, language, jargon. \2 Omitting assumptions and limitations: Important in research---under what conditions do you expect your conclusions to be true?  Robert Ricci committed Jan 12, 2014 71 72 73 74 75  \1 For next time \2 Read 2.2 \2 HW \#2 due before class starts \2 HW \#3 will be handed out by class time  Robert Ricci committed Jan 12, 2014 76  \3 Need three volunteers to go first for paper anaylsis  Robert Ricci committed Jan 12, 2014 77 78 79 80  \end{outline} \end{document}