GitLab

instead of outside since the BEGIN block is run first.

these scripts!

1. Make imagezip (and imageunzip) understand LILO bootblocks.
   Imagezip will create relocation information for all the absolute
   blocks that LILO embeds in the partition.  Imageunzip will correctly
   relocate those locations when laying down the new image.  Thus
   Linux slice images can be layed down anywhere on the disk (as can
   BSD images).

2. Because #1 is incompatible with the old imageunzip, I bumped the
   imagezip format version number (to 3) on images that contain relocations
   (i.e., slice images).  Make sure you have a new imageunzip to unzip
   new slice images.  Whole disk images are still created as version 2
   since they contain no relocs and thus are no different than before.

3. Added -N option to imagezip to create slice images without any
   relocations.  [ We could create our Emulab images this way, since we
   don't have any mechanism for, or reason to, let the user choose which
   partition to load an image in.  Hence our slice images always wind up
   at the same offset on the disk and relocations are unnecessary. ]
   Note that there is no significant expense in either time or space for
   relocations, this option was more of a debugging thing.

moved to the heap on a per-flow basis. This had caused the size of
the nse binary to go up by > 1MB. Removed that in this patch

- Fix to deal with FreeBSD 4.7 (4.x) problems with select() timeout and using
  BPF ioctl() BIOCSSEESENT. Several packets were being lost and depending on
  when exactly it happens, flows weren't making progress. In the particular
  case of 1 nse sending packets over 2 veths on the same pnode, this problem
  was the most severe. The kernel issues are discussed in the following
  freebsd arch mailing list. Also more discussions are on testbed-ops in
  the thread "bizarre veth/bpf/tcpdump behaviour"

http://lists.freebsd.org/pipermail/freebsd-arch/2003-November/001446.html
http://lists.freebsd.org/pipermail/freebsd-arch/2003-November/001439.html
http://lists.freebsd.org/pipermail/freebsd-arch/2003-November/001444.html
http://lists.freebsd.org/pipermail/freebsd-arch/2003-November/001447.html

- The port classifier stores packet timestamps and sizes and can be made to
  dump them from the tcl side. The default is 100000 packets which is compiled
  in. This is on a per destination port basis. For a large number of simulator
  flows, it can eat up memory quickly. Will be removed out of the standard
  build after my thesis. The format of the output is
  <packet_timestamp> <size> <simulator_clock>, one per line.
- In packets received from the wire, bpf timestamps (relative to start of nse)
  are stored in the simulator packet that is injected into the simulator. Again
  useful for traces. If the simulator packet is scheduled again, this timestamp
  will be overwritten to whenever the packet/event needs to be dispatched.
- IPTap supports sendcount and recvcount methods (callable from Tcl) to print
  debugging info

now a 3.0 branch, but it seems substantially different.)

This doesn't solve the problem I was hoping it would (being unable to
grab a tarball of the top-level directory), but I've already done the
work to update...

anymore.

correct.

Add some more sanity checks when adding nodes:
* Make sure nodes have control-net IP addresses
* Make sure the IP address given to nodes does not conflict
* Don't let them add nodes that have experimental interfaces for which
  we don't know the port

Copy over the cable number and length to the wires table, if the user
gave them.

programs before I start whacking on them.

that point, I switched the meaning of the last parameter to fd_score,
but forgot to change one place it gets called.

The result was that if there were desires that caused violations,
they would not be removed properly by remove_node() .

that gave us the error.

node id (which could be virtual).

to print out MAC addresses for the interfaces.

* Add proper check_slot() calls to all of the user input that is going into
  the DB (already had taint checking), since batchexp is now available for
  interactive use from ops.

* Remove separate DB insertions of noswap/noidleswap reasons from web
  script, and pass on the command line from web to batchexp. Now inserted
  in the backend script so that they can be provided on the command line
  when batchexp is used interactively.

* Change defaults in backend script; experiments now default to swappable
  and idleswap; previously defaulted to not swappable and no idleswap.

* Remove [-s] (swappable) and add [-S <reason>] option. -S sets experiment to
  not swappable, with supplied reason (text string).

* Add [-L <reason>] option. -L sets experiment to no idleswap, with
  supplied reason (text string).

* Add several missing table_regex entries for experiments table.

longer referenced so lets not confuse us.

to accept new solutions, so that we don't give violtions any special
treatment.

This can greatly reduce 'thrasing' at low scores, allowing assign to
converge on a solution much faster. For example, in the 1000-node
topology used for the virtualization paper, assign's runtime dropped
from nearly two hours to just over half an hour, with no degredation
in scores found.

You can get the old accept behavior by enabling the
SPECIAL_VIOLATION_TREATMENT #define, which is on right now (hence, we
are not using this change yet.)

Here are the pertinent comments from the code:

#ifdef SPECIAL_VIOLATION_TREATMENT
         /*
          * In this ifdef, we always accept new solutions that have fewer
          * violations than the old solution, and when we're trying to
          * determine whether or not to accept a new solution with a higher
          * score, we don't take violations into the account.
          *
          * The problem with this shows up at low temperatures. What can often
          * happen is that we accept a solution with worse violations but a
          * better (or similar) score. Then, if we were to try, say the first
          * solution (or a score-equivalent one) again, we'd accept it again.
          *
          * What this leads to is 'thrashing', where we have a whole lot of
          * variation of scores over time, but are not making any real
          * progress. This prevents the cooling schedule from converging for
          * much, much longer than it should really take.
          */
#else // no SPECIAL_VIOLATION_TREATMENT
         /*
          * In this branch of the ifdef, we give violations no special
          * treatment when it comes to accepting new solution - we just add
          * them into the score. This makes assign behave in a more 'classic'
          * simulated annealing manner.
          *
          * One consequence, though, is that we have to be more careful with
          * scores. We do not want to be able to get into a situation where
          * adding a violation results in a _lower_ score than a solution with
          * fewer violations.
          */

fix up standalone makefiles
have imagezip -v spit out the FSes it supports

parameter the virtual node it is being called for, and considers only
bandwidths that can be supported by physical nodes that virtual node
can be mapped to.

This fixes some cases where we were mistakenly setting interfaces
on slow links to 10Mbps, which does not work reliably.

Not surpisingly, tracking down problems with vtypes was half the
trouble with making this change.

splitting the existing code between a frontend script that parses arguments
and does taint checking, and a backend library where all the work is done
(including permission checks). The interface to the libraries is simple
right now (didn't want to spend a lot of time on designing interface
without knowing if the approach would work long term).

	use libreboot;
	use libosload;

        nodereboot(\%reboot_args, \%reboot_results);
        osload(\%reload_args, \%reload_results);

Arguments are passed to the libraries in the form of a hash. For example,
in os_setup:

	$reload_args{'debug'}     = $dbg;
	$reload_args{'asyncmode'} = 1;
	$reload_args{'imageid'}   = $imageid;
	$reload_args{'nodelist'}  = [ @nodelist ];

Results are passed back both as a return code (-1 means total failure right
away, while a positive argument indicates the number of nodes that failed),
and in the results hash which gives the status for each individual node. At
the moment it is just success or failure (0 or 1), but in the future might
be something more meaningful.

os_setup can now find out about individual failures, both in reboot and
reload, and alter how it operates afterwards. The main thing is to not wait
for nodes that fail to reboot/reload, and to terminate with no retry when
this happens, since at the moment it indicates an unusual failure, and it
is better to terminate early. In the past an os_load failure would result
in a tbswap retry, and another failure (multiple times). I have already
tested this by trying to load images that have no file on disk; it is nice
to see those failures caught early and the experiment failure to happen
much quicker!

A note about "asyncmode" above. In order to promote parallelism in
os_setup, asyncmode tells the library to fork off a child and return
immediately. Later, os_setup can block and wait for status by calling
back into the library:

	my $foo = nodereboot(\%reboot_args, \%reboot_results);
	nodereboot_wait($foo);

If you are wondering how the child reports individual node status back to
the parent (so it can fill in the results hash), Perl really is a kitchen
sink. I create a pipe with Perl's pipe function and then fork a child to so
the work; the child writes the results to the pipe (status for each node),
and the parent reads that back later when nodereboot_wait() is called,
moving the results into the %reboot_results array. The parent meanwhile can
go on and in the case of os_setup, make more calls to reboot/reload other
nodes, later calling the wait() routines once all have been initiated.
Also worth noting that in order to make the libraries "reentrant" I had to
do some cleaning up and reorganizing of the code. Nothing too major though,
just removal of lots of global variables. I also did some mild unrelated
cleanup of code that had been run over once too many times with a tank.

So how did this work out. Well, for os_setup/os_load it works rather
nicely!

node_reboot is another story. I probably should have left it alone, but
since I had already climbed the curve on osload, I decided to go ahead and
do reboot. The problem is that node_reboot needs to run as root (its a
setuid script), which means it can only be used as a library from something
that is already setuid. os_setup and os_load runs as the user. However,
having a consistent library interface and the ability to cleanly figure out
which individual nodes failed, is a very nice thing.

So I came up with a suitable approach that is hidden in the library. When the
library is entered without proper privs, it silently execs an instance of
node_reboot (the setuid script), and then uses the same trick mentioned
above to read back individual node status. I create the pipe in the parent
before the exec, and set the no-close-on-exec flag. I pass the fileno along
in an environment variable, and the library uses that to the write the
results to, just like above. The result is that os_setup sees the same
interface for both os_load and node_reboot, without having to worry that
one or the other needs to be run setuid.