1. 25 May, 2004 1 commit
  2. 24 May, 2004 2 commits
    • Leigh B. Stoller's avatar
      A set of small performance related changes that I made after putting · 38fa67c4
      Leigh B. Stoller authored
      in the timestamps, and seeing what was taking so much time.
      * Increase the overlap from two to four seconds (have vnodesetup exit
        earlier) so that the next jail can start sooner. Using getrusage it
        seemed pretty clear that there were plenty of cycles available to
        start the next jail going a little earlier. Instead of looking for
        the watchdog pid file to appear, I touch (in mkjail) a specific file
        in the jail directory , and have vnodesetup look for that file
        instead of the pid.
      * Cut back the number of inodes in the filesystem! Sheesh, we ere
        creating 137,000 inodes for a 50MB vnode filesystem. Takes 1
        second to do that. Reduced to 30,000 and the time when to .1
        seconds. Could probably go even lower.
      * Add -p option to fsck when restoring a jail from an existing
        filesystem. No need to force check the filesystem if its clean.
      * Instead of running MAKDEV inside each jail filesystem (which was
        also taking .8 seconds to do), do it just once and stash a copy in
        /var/emulab/jails. Each jail does a "hier cp" of that master copy
        into the jail filesystem. Reduced the time to .05 seconds.
      So, for the results:
      Original time to boot a jail: 8.2 seconds, with the last two seconds
      overlapped with the next one.
      Original time to reinstate a jail after halting it (filesystem left
      behind): 6.5 seconds, with the last 1.5 seconds overlapped with the next
      Original time to boot 16 jails on a pc850: 2 minutes and 12 seconds.
      New time to boot a jail: 6.2 seconds, with the last 4 seconds overlapped
      with the next one.
      New time to reinstate a jail after halting it: 6 seconds, with the last 4
      seconds overlapped with the next onw.
      New time to boot 16 jails on a pc850: 88 seconds. Does this number ring any
      bells? :-)
    • Leigh B. Stoller's avatar
      Add timestamp calls. · 8a2dd59d
      Leigh B. Stoller authored
  3. 23 Apr, 2004 2 commits
  4. 17 Feb, 2004 1 commit
    • Leigh B. Stoller's avatar
      Medium size cleanup of the client side code. The main impetus behind · 069dc7d3
      Leigh B. Stoller authored
      this was to add soft reconfig support so that nodes could be
      reconfigured without having to reboot them. This appears to work, and
      has been tested with jails getting moved around. I've also tested the
      new code on the MFS, but still no testing has been done on PLAB nodes.
      The main change is that most of the code moved out of libsetup.pm, and
      was split into constituent rc scripts, each of which does its own
      thing, including cleaning up and preparing for making an image. Most
      of that central knowledge has been moved out into the scripts. Still more
      to do but this was a good start.
  5. 30 Jan, 2004 1 commit
    • Mike Hibler's avatar
      Setup routes inside jail in the "correct" order to avoid ARP whining. · c650c97c
      Mike Hibler authored
      From the code:
              # N.B. privnet is setup first even before the default route.
              # This is because the gateway for the default route is on the
              # privnet, but our control net interface is not (because it has
              # a netmask by virtue of being an alias on the
              # real control interface).  Thus there is no way to use the
              # default route until the privnet route is up (ARP complains
              # about "host is not on local network").
  6. 01 Dec, 2003 2 commits
  7. 12 Nov, 2003 1 commit
  8. 16 Oct, 2003 1 commit
    • Leigh B. Stoller's avatar
      Brave new world of tmcc client side caching. The goal is to reduce the · 2b72f2c9
      Leigh B. Stoller authored
      number of connections to tmcd, and the resulting number of DB queries.
      Currently thats about 24 per node when it boots. Each vnode adds
      another 24 or so. The new approach is to use the "fullconfig" command,
      which dumps the entire config in one shot, saving about 20 of those
      connections. We still need to do the status/state commands for real of
      course. When a node boots, it requests the fullconfig; the client side
      takes this fullconfig, and dumps the individual sections to
      /var/emulab/boot/tmcc/section_name. Subsequent requests first look for
      it locally in the above named files, falling back to real tmcc if none
      exists. The update command also refreshes the cache.
      Tested for jails and plab node vservers as well.
  9. 15 Oct, 2003 1 commit
  10. 02 Oct, 2003 1 commit
    • Mike Hibler's avatar
      Add yet another route to jails: an interface route for the real control net. · 9fd95b1a
      Mike Hibler authored
      Previously, if you accessed a phys node by its real IP address (155.101.132.N)
      from inside a jail, it used the default route going to the router and back.
      Now it will put it out directly on the control net.
      The main reason for this is so that minibed vnodes can reach their boss and
      ops nodes.  Vnodes in mini don't have a default route that works, so we needed
      a way to get to boss/ops.
      Note that gated will remove this route when it starts up.  Thus, vnodes will
      revert to using the default route to get to the real control net.  On mainbed,
      this will still work (hop through the router as before).  On mini, it won't
      work at all.
      Moral: don't use session routing and vnodes on mini.
  11. 30 Sep, 2003 1 commit
  12. 03 Sep, 2003 1 commit
  13. 29 Aug, 2003 1 commit
    • Mike Hibler's avatar
      (Most of) the rest of the gated-in-jail changes. · 7ad42ea2
      Mike Hibler authored
      Add a -V option to mkjail.pl telling it to use the virtual control net
      IP ( for the default route instead of the real control net
      IP (  In this case, it also assigns a virtual control net
      IP alias (172.17.<pnode>.0) to the physical interface so that the
      aforementioned routes can actually be installed.
      Add the same -V option (only valid with -j) in vnodesetup so that we
      can get it through from bootvnodes.  (Also cleaned up the usage message
      The virtual control net address/mask are currently constants in mkjail.pl.
      At some point they can/should be changed to config time options so that we
      can easily use this on minibed too.
      In theory, this (-V) shouldn't be an option and we should just use
      virtual control net routing all the time.  But I want to get some more
      testing and we have to resolve the minibed conflicts first.  So for now,
      this is an option and it is off by default.
      Haven't tied this option in with gated yet, so creating a vnode experiment
      with Session routing still won't work.  If you want to do that, create the
      experiment, watch gated blow up right and left, login to all the physical
      nodes and change bootvnodes to add -V to the invocation of vnodesetup,
      and then reboot everything.  Now it should be working!
  14. 27 Aug, 2003 5 commits
  15. 04 Aug, 2003 1 commit
  16. 31 Jul, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add Mike's NFS/NULL mount changes to mkjail.pl · 5b16105a
      Leigh B. Stoller authored
      Also a couple perf hacks:
      * Local vnodes can start with the password/group file from the
        physnode, since locally they will be the same anyway. This avoids a
        blizzard of accounts requests at startup, which is by far the
        biggest chunk of data returned (well, except for host tables).
      * To help serialize boot startup, vnodesetup now waits for the jail to
        finish starting up before it exits. It does this via a "goofy"
        mechanism I will not bother to describe.
  17. 24 Jun, 2003 1 commit
  18. 06 Jun, 2003 1 commit
  19. 18 Apr, 2003 1 commit
    • Leigh B. Stoller's avatar
      A small set of changes to allow setting the hostname of the jail just · a8d81694
      Leigh B. Stoller authored
      like we set hostnames of local nodes (v0.wa-jail-link.testbed.emulab.net).
      Also build a proper /etc/hosts file just like local nodes inside the
      jail. Minor change to dohostnames(); do not munge /etc/hosts in place!
      Generate a new copy and then atomically move into place. Phew, this
      was a close one! Mike has me really worried about race conditions now
      that I screwed the pooch in the frisbee client.
  20. 17 Apr, 2003 1 commit
  21. 28 Mar, 2003 1 commit
  22. 15 Jan, 2003 1 commit
    • Leigh B. Stoller's avatar
      Add vnconfig fixes that Mike told me about a long time ago, and I · eb2f42a3
      Leigh B. Stoller authored
      totally forgot about (vnconfig does not return error status when a vn
      device is busy). This was casuing the crashes.
      Add a local only kludge to NFS mount /user and /proj into the jails. I
      hate this. Should be using SFS, but without an agent thats not really
      Add sshdport config setup from jailconfig. Look for that in the jail
      config file and pass to jail.
  23. 07 Jan, 2003 1 commit
  24. 06 Jan, 2003 1 commit
    • Leigh B. Stoller's avatar
      tunefs the newly created vnode FS and dial the minfree down to 2 · 9b6932b7
      Leigh B. Stoller authored
      percent, and optimize for space. Prelude to creating smaller jails
      on local nodes, as soon as I can get SFS running inside a jail the way
      I want it (in which case users will have access to their project and
      home dirs on the file server).
      Add Mike's IPADDR change, with slight modification. tmcd will specify
      a list of ip addresses as a comma separated list, which are converted
      to -i options to pass to jail. Kernel will restrict bind to these IPs.
  25. 20 Dec, 2002 1 commit
  26. 18 Dec, 2002 1 commit
  27. 27 Sep, 2002 1 commit
  28. 26 Sep, 2002 1 commit