Work on an optimization to the perl code. Maybe you have noticed, but
starting any one of our scripts can take a second or two. That time is spent including and compiling 10000s of thousands of lines of perl code, both from our libraries and from the perl libraries. Mostly this is just a maintenance thing; we just never thought about it much and we have a lot more code these days. So I have done two things. 1) I have used SelfLoader() on some of our biggest perl modules. SelfLoader delays compilation until code is used. This is not as good as AutoLoader() though, and so I did it with just a few modules (the biggest ones). 2) Mostly I reorganized things: a) Split libdb into an EmulabConstants module and all the rest of the code, which is slowly getting phased out. b) Move little things around to avoid including libdb or Experiment (the biggest files). c) Change "use foo" in many places to a "require foo" in the function that actually uses that module. This was really a big win cause we have dozens of cases where we would include a module, but use it in only one place and typically not all. Most things are now starting up in 1/3 the time. I am hoping this will help to reduce the load spiking we see on boss, and also help with the upcoming Geni tutorial (which kill boss last time).