1. 06 Oct, 2016 1 commit
  2. 02 May, 2015 1 commit
  3. 04 Nov, 2014 1 commit
  4. 20 Oct, 2014 1 commit
  5. 10 Jul, 2014 1 commit
  6. 01 Jul, 2014 1 commit
  7. 13 May, 2013 1 commit
  8. 17 Nov, 2012 1 commit
    • Mike Hibler's avatar
      More PRObE inspired improvements to the swapin path. · 5a2810f2
      Mike Hibler authored
      Replace an exec of the perl os_select script with a call to the OSSelect()
      node method. This cut in half the time spent in the DB setup for each node.
      Note that this change had already been made to libosload_new.
      
      Reworked the code that setup the partitions table entries. We were potentially
      updating each DB row for each image loaded. Now we just work out all the changes
      in a perl data struct and make one set of DB changes at the end. The code is
      more comprehensible now as well (I hope!)
      
      Finally, disable the "swapinfo" stuff which was the first step in doing stateful
      swapout of disk state. That code never got finished.
      5a2810f2
  9. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  10. 20 Dec, 2010 1 commit
    • David Johnson's avatar
      Allow passing arbitrary reload flags to osload() from os_load. · 844f88b4
      David Johnson authored
      Now that we're entering the type-specific osload module era, it's
      conceivable that modules might expose different parameters that users
      could set (i.e., choose to reload via xmodem or tftp).  So, we allow users
      to invoke os_load with a list of key/value pairs that get placed into the
      reload_args hash argument to osload().
      
      This will probably change later; it just preserves API compat for now.
      844f88b4
  11. 12 Nov, 2010 1 commit
    • David Johnson's avatar
      Bugfix, and protect switch flashes via feature. · 1ca84a0d
      David Johnson authored
      We now protect switch reloads via the SwitchFlash feature.  We
      don't really give people a lot of advance warning about this,
      and we wait long enough that it will irritatingly fail your
      experiment swapin.  Ideally we should check before anything
      starts happening, but we don't know if we *have* to reload
      until we see what's on the nswitch node that gets allocated
      to us... and by the time nodes are allocated, I think they
      end up having to go through reloading if *any* of them fails.
      Not sure though.
      
      Also fixup handling of multiple libossetup_switch ops; if one
      fails, abort the list instead of continuing on.
      1ca84a0d
  12. 11 Nov, 2010 1 commit
  13. 04 Nov, 2010 1 commit
    • David Johnson's avatar
      libosload rewrite and HP Procurve switch support. · 7cfa7aa6
      David Johnson authored
      libosload_new contains the new libosload.  It provides the same
      API as the original libosload, but the functionality has been
      split out into well-defined methods that can be overridden by
      per-node type subclasses in small chunks.  This is all protected
      by the NewOsload feature.
      
      libosload_switch.pm.in and libossetup_switch.pm.in contain the HP
      switch support, which requires the new libosload and new
      libossetup.  libossetup_switch doesn't do anything too radical, but
      it does support running a node it is "lighting up" through multiple
      operations -- i.e., first a RELOAD, then a RECONFIG.  This was
      necessary because switches require a config to be pushed to them
      (well, they do in the way I elected to support them), and they
      cannot fetch configuration in the normal Emulab way via tmcd.
      
      Eventually, that multi-op support will be factored out into
      libossetup more generically so that other node types which require
      push-based configuration can leverage it.
      
      libosload_switch contains the HP switch loading code.  It's built
      around expect-like interaction on the serial console.  It supports
      xmodem and tftp uploads, but tries to only flash the switch lazily.
      Thus, if the switch is already loaded with an image and a user
      (or the reload daemon, or anybody) reloads it with the same image,
      it won't actually get reloaded.  You can now force this reload
      with os_load.
      
      libosload_switch also contains reconfig code (i.e., to wipe
      switch configs and recreate based on which experiment it is in).
      libossetup_switch directly invokes this instead of going through
      libosload, since configuring is not really part of loading.  It
      is just convenient since configuration also happens over the
      serial console, and all the osload code for the switch can be
      reused.
      
      Also, of course, there are miscellaneous changes to support the
      new libosload in os_load itself.
      7cfa7aa6
  14. 29 Sep, 2010 1 commit
  15. 25 Mar, 2010 1 commit
  16. 12 Oct, 2009 1 commit
    • David Johnson's avatar
      Add the ability to load images on virtnodes. For now, we just overload · c6c57bc9
      David Johnson authored
      the tb-set-node-os command with a second optional argument; if that is
      present, the first arg is the child OS and the second is the parent OS.
      We add some new features in ptopgen (OS-parentOSname-childOSname) based
      off a new table that maps which child OSes can run on which parents, and
      the right desires get added to match.  We setup the reloads in os_setup
      along with the parents.  Also needed a new opmode, RELOAD-PCVM, to handle
      all this.
      
      For now, users only have to specify that their images can run on pcvms, a
      special hack for which type the images can run on.  This makes sense in
      general since there is no point conditionalizing childOS loading on
      hardware type at the moment, but rather on parentOS.  Hopefully this stuff
      wiill mostly work on shared nodes too, although we'll have to be more
      aggressive on the client side garbage collecting old frisbee'd images for
      long-lived shared hosts.
      
      I only made these changes in libvtop, so assign_wrapper folks are left in
      the dark.
      
      Currently, the client side supports frisbee.  Only in openvz for now, and
      this probably breaks libvnode_xen.pm.  Also in here are some openvz
      improvements, like ability to sniff out which network is the public
      control net, and which is the fake virtual control net.
      c6c57bc9
  17. 04 Aug, 2009 1 commit
    • Kevin Atkinson's avatar
      Implement frontend and middleend support for loading multiple images · e7871305
      Kevin Atkinson authored
      at once with Frisbee (excludes the actual MFS changes).
      
      Os_load now takes take a list of comma serrated image names for the
      "-i" and "-m" options.  The default OS is the OS for the last image
      specified in the list.  I also changed the "-p" option of osload to
      search both the project specified and emulab-ops for the image rather
      than just the project specified in order to simplify specifying
      multiple images (and because I personally found that behavior annoying
      when using osload).
      
      I modified the current_reloads table to be able to specify more than one
      image for a node by adding an "idx" column which controls the order of
      the reloads.  I also added a "prepare" column to the table (explained
      below)
      
      I modified tmcd to basically loop over the entries in the table and
      create a multiline LOADINFO responsive, and modified rc.frisbee to
      handle the multiline response and load each image in turn.
      
      I modified os_load to take a new option "-P" which will tell rc.frisbee
      to zap the superblocks even if a whole disk image is not specified.
      To do this I set the prepare entry for the first image in the
      current_reloads table to true.  Tmcd than passes this into to
      rc.frisbee in the LOADINFO line.  When rc.frisbee sees this it will
      make sure to zap the superblock before loading that image.
      
      To support having multiple images as the default, "default_imageid"
      can now be a comma separated list.  I implemented a hack to be able to
      set multiple imageids via editnodetype.php3.  Basically the form
      splits default_imageid into default_imageid_0, default_imageid_1, etc
      and than adds an empty default_imageid_# slot to allow adding an
      imageid.  Multiple images can be added by adding one image, than
      submitting the form, and than adding another into the empty slot.  Not
      the best, but I don't thing this will be a very common operation.
      When the form is submitted it will than combine all default_imageid_#
      into a comma separated list ignoring any that are deleted or set to
      "No ImageID" (ie 0).
      
      Everything will work fine with old MFSs as long as only one image is
      loaded.  If multiple images are loaded with an old MFS, an email will
      be sent to testbed-ops.  This works by having tmcd detect old MFS's by
      using the version number and setting the state to RELOADOLDMFS.  Stated
      will pick up on the and send the email to testbed-ops via a trigger.
      e7871305
  18. 08 Jun, 2009 1 commit
  19. 23 Feb, 2006 1 commit
    • Kevin Atkinson's avatar
      · 10fd4a08
      Kevin Atkinson authored
      Fix tied STDIN and STDOUT with perl 5.8 in libtblog.
      
      Fix a phototype warning in  os_load.in
      10fd4a08
  20. 06 Dec, 2005 2 commits
    • Mike Hibler's avatar
      Phase II in disk state saving for swapout. · ed0d25b4
      Mike Hibler authored
      Exec summary: after this checkin, the infrastructure exists (once enabled)
      to create swapout-time "delta" images for all machines in experiments.
      There is only a single, cumulative swap image per node (i.e., all diffs
      are from the base image, not from the previous swap).
      
      What doesn't yet exist, is the mechanism for reloading the delta at
      swapin time.  That is Phase III.
      
      The nitty-gritty:
      
      1. Keep disk image signature files for all nodes in an experiment.
      
         New fields in the DB to track, for each disk partition, what image the
         partition was loaded from.  This enables us at swapin or os_load time to
         create signature files in /proj/<pid>/exp/<eid>/swapinfo for the current
         contents of a node disk/partition.  All nodes with the same image loaded
         will share (via symlink) the same signature file.  TODO: no longer
         referenced signature files should be removed.
      
         Signature info is only collected in the swapinfo directory if the
         experiment is set to have disk state saving enabled (see #5 below).
         Info consists of the <vname>.sig file, which is the file created
         by imagehash, and <vname>.part which says what the root disk is
         for the node and whether to look at the whole disk or just a single
         partition when crafting the delta image.
      
      2. Swapout-time hook for creating swapout image.
      
         If the experiment is marked as allowing disk state saving, tbswap
         will arrange to run and then monitor the create-swapimage command
         on each node.  This script will run the modified version of imagezip
         which uses the signature file to create a delta image.
      
         The command to run and maximum timeout are specified via sitevars
         (previously checked in).  Note that the tbswap script currently has
         special knowledge of /usr/local/bin/create-swapimage as a swapout
         time script.  If the swap/swapout_command sitevar is set to that,
         Magic Stuff shall occur (i.e. it will monitor the command and make
         periodic reports of progress).  The sitevars are a total hack and
         will disappear at some point.
      
      3. Client-side script for creating swapout image.
      
         os/create-swapimage, very similar to create-image.  Uses the info
         stashed in /proj/..blahblah../swapinfo to create a delta image.
      
         XXX fer now hack: the script first looks in /proj/<pid>/bin for an
         imagezip binary to use.  Failing that, it uses the one in the MFS.
         This allows for easier development of the imagezip changes (i.e.,
         don't have to update the MFS every time.
      
      4. Auto creation of signature files for new images.
      
         The create_image script (the one that runs on boss when creating images
         for users) has been modified to automatically create a signature via
         imagehash.  The .sig file winds up in /usr/testbed/images/sigs or
         in /proj/<pid>/images/sigs.  From there it will be copied at swapin/os_load
         time to the per-expt swapinfo directory for any node that uses the images.
      
         The process for creating standard system images (aka, "Mike") has not
         yet been modified.  When the image creation/installation procedure
         is formalized into a script, this will be done.
      
      5. Web changes to set/clear saving of disk state at swapout time.
      
         Add a checkbox to the experiment create page to allow setting "save
         swap state".  Also added to the experiment modify page, but currently
         "if (0)"ed out as it will need some additional support.  The showstuff
         page will show it.
      
         Taking a page from Leigh's hack book, if EXPOSESTATESAVE in defs.php3
         is set to zero (as it is now), then the checkbox doesn't appear in the
         create experiment page except for STUDLY users.
      ed0d25b4
    • Mike Hibler's avatar
      Wee little hack: · abfbb5d7
      Mike Hibler authored
       Common mistake: forget the -i before the imagename, e.g.,
       "os_load FBSD54-STD pcNN", which results in pcNN getting loaded
       with the default image.  So if the first arg fails as a node, but
       is an image ID, assume they have made this mistake and stop.
      abfbb5d7
  21. 14 Apr, 2005 1 commit
    • Mike Hibler's avatar
      Changes to respect the "imageable" field of the node_types table: · 65a39cd5
      Mike Hibler authored
       - os_load will not attempt to load a non-imagable node, it will
         be skipped without error
       - nfree will respect imageable even with an entry in scheduled_reloads
       - reload_daemon will free any non-imageable nodes that happen to make
         it into reloadpending/reloading
      65a39cd5
  22. 17 Mar, 2005 1 commit
    • Mike Hibler's avatar
      Partial support for disk-zeroing on experiment termination. · 60e7adb8
      Mike Hibler authored
      I did the "back half" support.  If the 'mustwipe' field is non-zero
      in the reserved table entry for a node then its disk must be zeroed.
      How the zeroing is done, depends on the value of the mustwipe field.
      Right now, '1' means pass the '-z' option to frisbee to have it zero
      all non-allocated blocks.  The value '2' is reserved for enabling a
      "full wipe" pass of the disk before running frisbee, which Keith Sklower
      (DETER) wanted to be able to do.  Note that 1 and 2 are effectively the
      same, if we are loading a full-disk image; i.e. all non-allocated blocks
      from the new image are zeroed.  But if the disk were being loaded with
      a single-partition image, then "frisbee -z" would only wipe unused
      blocks in that partition.
      
      The reload_daemon has been modified to extract the mustwipe info and
      invoke os_load accordingly.   os_load now takes a "-z <type>" option
      to enable the zeroing by setting a value in the current_reloads table.
      tmcd will read and return that info to its caller in the "loadinfo" command.
      Finally, the rc.frisbee script that runs in the frisbee MFS extracts the
      loadinfo info and crafts the frisbee startup command.
      
      What still needs to be done is the "front end," how the user specifies
      the value and how it winds up in the DB reserved table.  This will probably
      involve addition of state to the experiments table as this will likely be
      a per-experiment setting.
      60e7adb8
  23. 07 Mar, 2005 1 commit
    • Timothy Stack's avatar
      · 898cf9a2
      Timothy Stack authored
      Checkin some changes related to experiment automation and vnode feedback:
      
      	* configure, configure.in: Add sensors/canaryd/feedbacklogs
      	template.
      
      	* db/libdb.pm.in, db/xmlconvert.in: Add "virt_user_environment"
      	table that holds environment variable names and values.
      
      	* event/lib/event.c: Allocate memory of the right size for
      	event_notifications.
      
      	* event/program-agent/GNUmakefile.in: Add version.c file and
      	add install targets for the man page.
      
      	* event/program-agent/program-agent.8: Man page describing the
      	program-agent daemon.
      
      	* event/program-agent/program-agent.c: Add a bunch of convenience
      	features: let the user specify the working directory for commands;
      	save output to separate files on every invocation of an agent; let
      	the user specify a timeout for a command; make the set of
      	environment variables sane and add vars given in the NS file in
      	the opt array; a "status" file containing process information is
      	written out when children are collected.  Internal changes: child
      	processes are collected immediately, instead of waiting for the
      	next START event, so we can send back COMPLETE events; the daemon
      	now runs with a real-time priority, to increase the chances of
      	receiving events.
      
      	* event/proxy/evproxy.c: Made it bidirectional so the
      	program-agent's COMPLETE events make it back to the scheduler.
      
      	* event/sched/error-record.c: Change the default log directory.
      
      	* event/sched/event-sched.h, event/sched/event-sched.c: Setup an
      	environment similar to a program-agent to run the user's log
      	digester.
      
      	* event/sched/node-agent.cc: Add a handler for the SNAPSHOT event
      	that runs create_image for the node.
      
      	* event/sched/simulator-agent.h, event/sched/simulator-agent.cc:
      	Let the user specify a "DIGESTER" script that digests the log
      	files into a summary of the results.  Add event handler for
      	remapping a vnode experiment.
      
      	* event/sched/timeline-agent.c: Accept the RUN event as well as
      	the START event.
      
      	* os/GNUmakefile.in: Install the install-tarfile.1 man page.
      
      	* os/install-tarfile: Automatically chown/chgrp any files that do
      	not have valid user or group IDs, the new owner will be the user
      	that swapped in the experiment.  Include the install directory in
      	the DB file.  Add a "list" mode that just dumps what files have
      	been installed and where.  Add a "force" option so the user can
      	forcefully install the file, even though the DB says its already
      	there.
      
      	* os/install-tarfile.1: Man page describing the install-tarfile
      	tool.
      
      	* os/syncd/GNUmakefile.in: Install man pages on ops.
      
      	* sensors/canaryd/GNUmakefile.in: Link canaryd statically and
      	install "feedbacklogs" tool.
      
      	* sensors/canaryd/canaryd.c: Dump dummynet pipe data.
      
      	* sensors/canaryd/canarydEvents.c: Log errors.
      
      	* sensors/canaryd/feedbacklogs.in: Tool used to generate feedback
      	data from canaryd log files.
      
      	* sensors/slothd/GNUmakefile.in: Install digest-slothd on ops.
      
      	* sensors/slothd/digest-slothd: Fix some bugs and write out an
      	"alert" file with all the nodes/links that were overloaded.
      
      	* tbsetup/os_load.in, tbsetup/libosload.pm.in: Add "waitmode"
      	argument that lets you specify that you want to wait for the disk
      	to finish loading and/or wait for the node to come back up in the
      	new OS.
      
      	* tbsetup/power.in: Remove debugging printf.
      
      	* tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/program.tcl,
      	tbsetup/ns2ir/sequence.tcl, tbsetup/ns2ir/sim.tcl.in: Fix some
      	quoting problems with event-sequences.  Add -expected-exit-code
      	and -tag options to the "$program run" event.  Add -digester to
      	the "$ns report" event that lets the user specify a program to run
      	to digest the log files.
      
      	* tbsetup/ns2ir/tb_compat.tcl.in: Change the initial scaling
      	factor for feedback nodes to 1%, instead of 100%.
      
      	* tmcd/tmcd.c, tmcd/common/libtmcc.pm: Add "userenv" command that
      	returns the values in "virt_user_environment".  Return new program
      	agent fields: dir, timeout, and expected_exit_code.
      
      	* tmcd/common/GNUmakefile.in: Install rc.canaryd.
      
      	* tmcd/common/bootvnodes: Add hack to boost the program-agents to
      	a real-time priority, they can't do it from inside the jail.
      
      	* tmcd/common/rc.canaryd: Rc script for canaryd.
      
      	* tmcd/common/watchdog: Don't fail outright if there is a bad line
      	in the battery.log
      
      	* tmcd/common/rc.progagent: Append "userenv" data to the
      	program-agent config file.
      
      	* utils/GNUmakefile.in: Install loghole and its man page on ops.
      
      	* utils/loghole.1: Document "clean" command and the change in
      	loghole directories.
      
      	* utils/loghole.in: Add "clean" command and parallelization.
      
      	* xmlrpc/emulabserver.py.in: Add "virt_user_environment" table.
      	Order the eventlist by "idx" and time, needed for sequences.  And
      	removed unnecessary nologin checks.
      898cf9a2
  24. 18 Mar, 2004 2 commits
    • Leigh Stoller's avatar
      Remove debugging code. · 5aac688c
      Leigh Stoller authored
      5aac688c
    • Leigh Stoller's avatar
      Possible rationalization of imageid search: · bb208441
      Leigh Stoller authored
      * If -p option given, use that project to find the image. Fatal error
        if no such image in that project.
      
      * If no -p option:
      
           * If in -e (pid,eid) mode, then first look in the same project as
             the experiment (pid), and then look in emulab-ops.
      
           * If not in -e mode, then look only in the emulab-ops project.
      
           * If not obvious, -p option always overrides the search.
      bb208441
  25. 12 Mar, 2004 1 commit
  26. 08 Mar, 2004 1 commit
    • Leigh Stoller's avatar
      Converted os_load and node_reboot into libraries. Basically that meant · 9bfe3d61
      Leigh Stoller authored
      splitting the existing code between a frontend script that parses arguments
      and does taint checking, and a backend library where all the work is done
      (including permission checks). The interface to the libraries is simple
      right now (didn't want to spend a lot of time on designing interface
      without knowing if the approach would work long term).
      
      	use libreboot;
      	use libosload;
      
              nodereboot(\%reboot_args, \%reboot_results);
              osload(\%reload_args, \%reload_results);
      
      Arguments are passed to the libraries in the form of a hash. For example,
      in os_setup:
      
      	$reload_args{'debug'}     = $dbg;
      	$reload_args{'asyncmode'} = 1;
      	$reload_args{'imageid'}   = $imageid;
      	$reload_args{'nodelist'}  = [ @nodelist ];
      
      Results are passed back both as a return code (-1 means total failure right
      away, while a positive argument indicates the number of nodes that failed),
      and in the results hash which gives the status for each individual node. At
      the moment it is just success or failure (0 or 1), but in the future might
      be something more meaningful.
      
      os_setup can now find out about individual failures, both in reboot and
      reload, and alter how it operates afterwards. The main thing is to not wait
      for nodes that fail to reboot/reload, and to terminate with no retry when
      this happens, since at the moment it indicates an unusual failure, and it
      is better to terminate early. In the past an os_load failure would result
      in a tbswap retry, and another failure (multiple times). I have already
      tested this by trying to load images that have no file on disk; it is nice
      to see those failures caught early and the experiment failure to happen
      much quicker!
      
      A note about "asyncmode" above. In order to promote parallelism in
      os_setup, asyncmode tells the library to fork off a child and return
      immediately. Later, os_setup can block and wait for status by calling
      back into the library:
      
      	my $foo = nodereboot(\%reboot_args, \%reboot_results);
      	nodereboot_wait($foo);
      
      If you are wondering how the child reports individual node status back to
      the parent (so it can fill in the results hash), Perl really is a kitchen
      sink. I create a pipe with Perl's pipe function and then fork a child to so
      the work; the child writes the results to the pipe (status for each node),
      and the parent reads that back later when nodereboot_wait() is called,
      moving the results into the %reboot_results array. The parent meanwhile can
      go on and in the case of os_setup, make more calls to reboot/reload other
      nodes, later calling the wait() routines once all have been initiated.
      Also worth noting that in order to make the libraries "reentrant" I had to
      do some cleaning up and reorganizing of the code. Nothing too major though,
      just removal of lots of global variables. I also did some mild unrelated
      cleanup of code that had been run over once too many times with a tank.
      
      So how did this work out. Well, for os_setup/os_load it works rather
      nicely!
      
      node_reboot is another story. I probably should have left it alone, but
      since I had already climbed the curve on osload, I decided to go ahead and
      do reboot. The problem is that node_reboot needs to run as root (its a
      setuid script), which means it can only be used as a library from something
      that is already setuid. os_setup and os_load runs as the user. However,
      having a consistent library interface and the ability to cleanly figure out
      which individual nodes failed, is a very nice thing.
      
      So I came up with a suitable approach that is hidden in the library. When the
      library is entered without proper privs, it silently execs an instance of
      node_reboot (the setuid script), and then uses the same trick mentioned
      above to read back individual node status. I create the pipe in the parent
      before the exec, and set the no-close-on-exec flag. I pass the fileno along
      in an environment variable, and the library uses that to the write the
      results to, just like above. The result is that os_setup sees the same
      interface for both os_load and node_reboot, without having to worry that
      one or the other needs to be run setuid.
      9bfe3d61
  27. 12 Jan, 2004 1 commit
    • Leigh Stoller's avatar
      Death to proxydhcp; one less specialized daemon. DHCP will return the · 2b2b8ca1
      Leigh Stoller authored
      filename to boot, and all local nodes will boot the same pxeboot kernel,
      which has been extended to allow for jumping directly into a specific MFS
      (in addition to the usual testbed boot into a partition or multiboot
      kernel).
      
      Bootinfo and the bootwhat protocol extended to tell the client node what
      MFS to jump into directly, without a reboot. pxe_boot_path and
      next_pxe_boot_path are now deprecated, with bootinfo used to control which
      MFS to boot. Nodes now boot a single pxeboot kernel, and bootinfo tells
      them what to do next.
      
      Bootinfo greatly simplifed. temp_boot_osid has been added to allow for
      temporary booting of different kernels (such as with ndoe_admin or
      create_image). Unlike next_boot_osid which is a one-shot boot,
      temp_boot_osid causes the node to boot that OS until told not too.
      
      next_boot_path and def_boot_path in the nodes table are now ignored.
      Bootinfo gets path info strictly from the os_info table entry for the osid
      given in one of def_boot_osid, temp_boot_osid, or next_boot_osid.  This
      makes the selection of what to do in bootinfo a lot simpler (and for
      TBBootWhat in libdb). The os_info table also modified to include an MFS
      flag so that bootinfo knows to tell the client that the path refers to an
      MFS and not a multiboot kernel.
      
      Change to boot sequence; free nodes no longer boot into the default OSID.
      Instead, they are told to wait in pxeboot until told what to do, which
      will typically be when the node is allocated and a specific OSID
      picked. If the node needs to be reloaded, then the node is told to jump
      directly into the Frisbee MFS, which saves one complete reboot cycle
      whether the node has the requested OS installed, or not.  New program
      added called "bootinfosend" that is used by node_reboot to "wake up" up
      nodes sitting in pxewait mode, so that they query bootinfo again and boot.
      
      node_reboot changed to look at the event state of a node, and use
      bootinfosend to wake up nodes, rather then power cycle, since pxeboot does
      not repsond to pings. Retry (if the UDP packet is lost) is handled by
      stated.
      
      Event support added to bootinfo, to replace the event generation that was
      in proxydhcp. I have not included the caching that Mac had in proxydhcp
      since it does not appear that bootinfo packets are lost very
      often. Cleaned up all of the event and DB queury code to use lib/libtb for
      DB access, and moved all of the event code into a separate file.  The
      event sequence when a node boots now looks like this:
      
      	'SHUTDOWN'    --> 'PXEBOOTING'  (BootInfo)
      	'PXEBOOTING', --> 'PXEBOOTING'  (BootInfo Retry)
      	'PXEBOOTING', --> 'BOOTING'     (Node Not Free)
      	'PXEBOOTING', --> 'PXEWAIT'     (Node is Free)
      	'PXEWAIT',    --> 'PXEWAKEUP'   (Node Allocated)
      	'PXEWAKEUP',  --> 'PXEWAKEUP'   (Bootinfo Retry)
      	'PXEWAKEUP',  --> 'PXEBOOTING'  (Node Woke Up)
      
      Change stated to support resending PXEWAKEUP events when node times out.
      After 3 tries, node is power cycled. Other minor cleanup in stated.
      
      Clean up and simplify os_select, while adding support for temp_next_boot
      and removing all trace of def_boot_path and next_boot_path processing.
      Remove all pxe_boot_path and next_pxe_boot_path processing.  Changed
      command line interface to support "clearing" fields. For example,
      node_admin changed to call os_select like this to have the node
      temporarily boot the FreeBSD MFS:
      
      	os_select -t FREEBSD-MFS pcXXX
      
      which sets temp_boot_osid. To turn admin mode off:
      
      	os_select -c -t pcXXX
      
      which says to clear temp_boot_osid.
      
      sql/database-fill-supplemental.sql modifed to add os_info table
      entries for the FreeBSD, Frisbee, and newnode MFS's.
      
      Be sure to change dhcpd config, restart dhcp, kill proxydhcp, restart
      bootinfo,
      2b2b8ca1
  28. 20 Oct, 2003 1 commit
  29. 13 Oct, 2003 1 commit
  30. 10 Oct, 2003 1 commit
    • Mac Newbold's avatar
      New StateWait changes - the main point of all this is to move to our new · 2b2a306d
      Mac Newbold authored
      model of waiting for state changes. Before we were watching the database
      (which means we can only watch for terminal/stable/long-lived states, and
      have to poll the db). Now things that are waiting for states to change
      become event listeners, and watch the stream of events flow by, and don't
      have to do any polling. They can now watch for any state, and even
      sequences of states (ie a Shutdown followed by an Isup).
      
      To do this, there is now a cool StateWait.pm library that encapsulates the
      functionality needed. To use it, you call initStateWait before you start
      the chain of events (ie before you call node reboot). Then do your stuff,
      and call waitForState() when you're ready to wait. It can be told to
      return periodically with the results so far, and you can cancel waiting
      for things. An example program called waitForState is in
      testbed/event/stated/ , and can also be used nicely as a command line tool
      that wraps up the library functionality.
      
      This also required the introduction of a TBFAILED event that can be sent
      when a node isn't going to make it to the state that someone may be
      waiting for. Ie if it gets wedged coming up, and stated retries, but
      eventually gives up on it, it sends this to let things know that the node
      is hozed and won't ever come up.
      
      Another thing that is part of this is that node_reboot moves (back) to the
      fully-event-driven model, where users call node reboot, and it does some
      checks and sends some events. Then stated calls node_reboot in "real mode"
      to actually do the work, and handles doing the appropriate retries until
      the node either comes up or is deemed "failed" and stated gives up on it.
      This means stated is also the gatekeeper of when you can and cannot reboot
      a node. (See mail archives for extensive discussions of the details.)
      
      A big part of the motivation for this was to get uninformed timeouts and
      retries out of os_load/os_setup and put them in stated where we can make a
      wiser choice. So os_load and os_setup now use this new stuff and don't
      have to worry about timing out on nodes and rebooting. Stated makes sure
      that they either come up, get retried, or fail to boot. tbrestart also
      underwent a similar change.
      2b2a306d
  31. 25 Sep, 2003 1 commit
  32. 17 Sep, 2003 1 commit
  33. 23 Jul, 2003 1 commit
  34. 14 May, 2003 1 commit
  35. 26 Mar, 2003 1 commit
    • Leigh Stoller's avatar
      Add "gid" slot to the images table for changing permission scheme from · 4c56daf6
      Leigh Stoller authored
      only pid, to pid/gid like most other things in the testbed. Also add a
      "global" slot to denote images that are globally available to all
      projects (system images). The older "shared" attribute is now used to
      denote images that are shared within a project (available to all
      subgroups in the project). The migration path for existing DBs is
      given in the migrate file. Be sure to run those commands on an
      existing testbed or things will break!
      
      www/newimageid, www/newimageid_ez: A bunch of changes for
      shared/global attributes. Added a group menu to the form so users can
      create images in subgroups. Beefed up the Java code that constructs
      the path name to use the gid, shared, and global attributes of the
      form to give the user the best possible path that we can. Improved the
      pathname checking code so that we do not allow just any old path in
      case the user elects to disregard the path we carefully constructed
      for them. Also check the proj/group membership, and setup defaults for
      users that have permission in just one pid/gid to create images.
      
      libdb.in: Changed permission check in TBImageIDAccessCheck() to
      reflect shared/global attribute changes.
      
      os_load: Get rid of test that checked path of the image. The path
      checking is done in the web interface anyway, so why duplicate in 4
      places. Other minor changes reflecting shared->global name change.
      Also note that images can come from the group directory now.
      
      create_image: Get rid of test that checked path of the image. The path
      checking is done in the web interface anyway, so why duplicate in 4
      places. Also note that images can come from the group directory now.
      
      www/dbdefs: Changed permission check in TBImageIDAccessCheck() to
      reflect shared/global attribute changes.
      
      www/showimageid_list, www/showstuff: Minor global/shared attribute
      changes.
      
      www/menu: Change osids/imageids pointer to point to the image list,
      not the osid list. This is more reasonable for mere users who have
      access to the EZ form, and thus never really need to concern
      themselves with osids.
      
      www/editimageid: Add proper pathname checking. There were no checks at
      all before!
      4c56daf6
  36. 25 Mar, 2003 1 commit
  37. 19 Feb, 2003 1 commit
  38. 29 Jan, 2003 1 commit