1. 18 Dec, 2002 8 commits
  2. 17 Dec, 2002 5 commits
  3. 16 Dec, 2002 3 commits
    • Mac Newbold's avatar
    • Mac Newbold's avatar
      Decrease the sleep between loops from 2 to 1, and fix a typo. This should · 6bdba92c
      Mac Newbold authored
      help nodes in reload_pending get sucked into reloading faster. If it
      doesn't do enough, we'll need to do more batching of stuff, so we get some
      parallelism in os_load instead of forcing it to serialize by calling
      os_load one node at a time.
      I was tempted to nuke all the stuff that was in there from the netdisk
      reload type, but decided not to. It won't be too long (relatively
      speaking) before we have freed, the new "free node manager" that will
      replace/supersede our current reload_daemon anyway.
    • Mac Newbold's avatar
      Fix the 1-event-per-second limitations. Poll until I don't get more · a77a1559
      Mac Newbold authored
      events. This may delay handling of other stuff that happens in my main
      loop, but not by too much. To prevent skew, everything (including reload
      frequency) is done strictly by seconds elapsed, not by iterations or
      I found that even polling for multiple events without sleeping, I could
      only handle a little over 1 per second when I was calling inuse/statetime
      for additional info on every event. Even though this only happens in the
      worst case (every event is wrong), it won't do. So I took that out. I'll
      probably end up adding a faster lookup of the info I need (mostly
      reservation, and what osid it thinks it is running). That change took it
      up to at least 4 per second (as fast as I could send them manually), more
      than 4x our previous performance. So we should be able to keep up now.
      Also, add the support for "announcements" to testbed ops when I die and
      such. (Been in a few days, but this is the first commit of it)
  4. 13 Dec, 2002 1 commit
  5. 12 Dec, 2002 6 commits
  6. 11 Dec, 2002 8 commits
    • Leigh B. Stoller's avatar
      Fix "wonkyness" as reported by Eric ... · a6cc917b
      Leigh B. Stoller authored
    • Leigh B. Stoller's avatar
    • Mike Hibler's avatar
      all: · a6a648e9
      Mike Hibler authored
      	add -Wall to CFLAGS and clean up lint
      	update the TODO file
      	explicitly size the header fields (e.g., int32_t not int)
      	Version 2.
      	Adds two ints to the header to help track free space.  Each chunk
      	now has a first and last sector number which can describe any free
      	block before or after the data contained in the chunk.  This is
      	needed in order to properly zero all free space when laying down
      	an image.  In practice: the first chunk describes any free space
      	before the first allocated range and any free space after its
      	contained ranges and before the first allocated range in the second
      	chunk.  Every other chunk then describes just free space following
      	itself (since the previous chunk has already described the space
      	before this chunk).  The point being, we only describe each free
      	range once.
      	Added "relocation" information.  Relocation entries go in the chunk
      	header along with region descriptors.  This allows us to identify
      	chunks of data which need to be absolute disk blocks instead of
      	offsets from the containing partition.  This is now used for BSD-slice
      	partition tables which contain absolute disk blocks.  We can now
      	create an image in one slice and reload it into another slice.
      	Allow zlib compression level 0 (no compression).  This might be
      	useful on machines that have slow CPUs: do just FS-compression and
      	transfer the image elsewhere faster where it could be re-zipped
      	with regular compression.
      	Fix goof.  Previously we were not saving any DOS partition with
      	an unrecognized type.  We should be naively compressing it instead.
      	This is what we now do.  We continue to skip partitions of type 0
      	mikeism: add handler for SIGINFO (^T) to report progress of
      	a zip-age.
      	Added everybody's favorite "dots" mode for reporting progress.
      	Eliminate some excess copies left over from the conversion from
      	write-every-little-piece to buffer-up-a-full-chunk-and-then-write.
      	Eliminaged the special case handling of no skips (ranges) in
      	compress_image by creating a single allocated range describing
      	the whole disk/partition in this case.
      	For NTFS, make the behavior of calling missing unicode routines
      	be to return an error rather than exit.  These calls happen,
      	but their failing doesn't seem to be fatal.
      	Lots of typical mike-pissing on everything else.
      	Modify to handle both V1 and V2 images.
      	In slice mode, make sure we don't write past the bounds of
      	the slice.  ES&D if we try.
      	Make output to unseekable devices work again (broken when
      	pwrite was added)
      	Add debug -F (Frisbee) option to randomize the presentation of
      	chunks to the unzip/write threads.  Used to simulate frisbee.
      	Add "-T DOS-type" option to tell imageunzip, when in slice mode,
      	to set the type of the slice in the DOS partition table.
      	This is useful if you are dropping say a BSD filesystem into
      	an unused slice, you don't have to go back later and set this
      	with fdisk.  Considered making this info part of the image
      	itself (recorded by imagezip when creating a slice image),
      	but decided against it.
      	writezero takes an off_t for the size, we can be asked to write
      	many gigabytes of zero at the end of a disk.
      	Turn off dots mode by default.  Ya wanna see spots?  Ya gotta
      	turn it on!
      	Lots of typical mike-pissing on everything else.
      	New tool for checking/dumping the structure of an image and
      	reporting stats about it.
    • Mike Hibler's avatar
      Server: back to using a condvar since they seem to be fixed. · 2e77122f
      Mike Hibler authored
      Server: make file readsize independent of burstsize (previously
      readsize had to be a divisor of burstsize).  A subtle side-effect
      is that the dynamic burst rate is recalcluated at the conslusion
      of every burst instead of after every readsize count of blocks has
      been sent (less than a burst)  This just seems to be more logical.
      Client: add "-T DOS-type" option to tell frisbee, when in slice
      mode, to set the type of the slice in the DOS partition table.
      This is useful if you are dropping say a BSD filesystem into
      an unused slice, you don't have to go back later and set this
      with fdisk.  Considered making this info part of the image
      itself (recorded by imagezip when creating a slice image),
      but decided against it.
    • Mike Hibler's avatar
      botched the path · a2ccf66a
      Mike Hibler authored
    • Mike Hibler's avatar
      A better strategy for dealing with how to load network device drivers on · 78f2dd14
      Mike Hibler authored
      the different node types
    • Mike Hibler's avatar
      Retro shell-script version of setipod. · cd839813
      Mike Hibler authored
      Needed for the frisbee environment, so might as well use it everywhere.
    • Leigh B. Stoller's avatar
      Minor change to n.class clause. Look for class "pc" and "pct". pct is · 6fd445e7
      Leigh B. Stoller authored
      a temporary class for testing new images.
  7. 10 Dec, 2002 3 commits
    • Kirk Webb's avatar
      · fc985a64
      Kirk Webb authored
      Modified the timeout logic in create_image to track the image creation progress
      (size) rather than simply waiting a certain amount of time.  Also changed the
      code to report progress at regularly spaced intervals (adjustable), and
      to indicate when the timeout timer has been activated, or halted due to
      progress.  The changes also include an NFS cache slack factor, which makes
      the effective non-progress timeout equal to the sum of the slack time, plus
      the non-progress time (currently 3 + 5 = 8 minutes).
      Some changes were made to the error and cleanup logic to help revert the
      state of the DB and node as much as possible (node is not rebooted if the
      DB state cannot first be reverted) prior to exit.
    • Leigh B. Stoller's avatar
      Minor change to addpubkeys call. · 1ea32e42
      Leigh B. Stoller authored
    • Leigh B. Stoller's avatar
      Fix up ssh/openssh links. · ab2b24ca
      Leigh B. Stoller authored
  8. 09 Dec, 2002 6 commits
    • Mac Newbold's avatar
    • Leigh B. Stoller's avatar
      Tighten up the permission stuff; do not let a webonly user access any · d0f46edb
      Leigh B. Stoller authored
      protected page except those that are explicitly deemed okay for a
      webonly user. This makes me feel better and safer!
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      Minor fix to last revision. · e0398fd9
      Leigh B. Stoller authored
    • Leigh B. Stoller's avatar
      Minor fix. · 54eb8c17
      Leigh B. Stoller authored
    • Leigh B. Stoller's avatar
      New set of pages for dealing with users requesting widearea accounts. · ad90fb6f
      Leigh B. Stoller authored
      A user can request a local account on the machine he/she has dedicated
      to netbed. In fact, multiple people can request local accounts. They
      just need to fill in the form, supplying the usual personal data we
      require, and then some info about the node. This includes the IP and
      the CDKey as verification (we now save the original CDKey in the DB
      forever), as well as info about the node (location, processor type,
      connection type). They must fill out the node info for the first
      account request. Otherwise, it can be ignored (that is, if an entry is
      already in widearea_nodeinfo, we do not require those fields).
      Once submitted, the user has to go through the usual verification
      To approve the user, admin people get a new link on the menu to
      approve widearea accounts. That page looks a lot like the normal
      join project approval page, only is tailored slightly for widearea
      accounts instead of projects.
      Once approved, widearea users get a webonly account. Note that it can
      be a normal account, say if the user is also in a normal project, or
      if we just want to give out an account on ops/boss to this person.
      Just need to clear the webonly flag in the DB, and the account will be
      built as normal, except they are put in the "guest" group on boss/ops
      if not a member of any projects.
      There are two new tables. widearea_accounts and widearea_nodeinfo. The
      accounts table maps uid's to specific nodes they get an account on
      (see changes in tmcd). The mapping also includes a trust value (user
      or root, although it should be rare to give out root access) for the