GitLab

More work on the hierarchical configuration for subboss. When doing host-based
authentication, allow client to pass an explicit host (IP) to the mserver.
If the mserver is configured to allow it, that IP is used for authenticating
the request instead of the caller's IP. Add a default ("null") configuration
so the mserver can operate out-of-the-box with no config file. The goal of
these two changes is for an mserver instance with the default config and a
proxy option to serve the needs of a subboss node (i.e., so no explicit
configuration will be needed).

proxy, then it will perform its checks against the hostIP provided rather
than the IP of the message sender.  For the Emulab subboss case, subboss
nodes (as determined from the DB) are allowed to explicitly specify hosts
that are under their control.  The master server on real boss will run with
proxying enabled.

Also, in a fit of madness, I added a version number to the master server
protocol.  This way, if at some distant point in the future (say next week)
I realize I screwed up the protocol, I can fix it without resorting to creative
retrofitting of a version number (see imagezip) or the more Orwellian
eradication and denial of past versions (what I am doing now). Furthermore,
using an ill-thought-out insight, I made the version number be an ASCII string
in case I decide to change to an all-text protocol at some equally distant
point in the future.

The frisbee server (and client) return a special exit code if they cannot
bind to the given port.  Arrange for server startup to be retried if this
happens.

Added GetNodeByVirtualId, GetLinkByVirtualId, and ReplaceNode as utility functions for managing rspecs.

There are some hacks involved here right now since a unicast frisbee server
currently can only support a single client.  So for now, we only allow a
single unicast server for any image (i.e., because the right way to do this
is to fix the server to support multiple clients, not to start up multiple
single-client servers).

The master server can be started with a "parent" mserver that it can call
if it doesn't have an image.  The master server will return an EAGAIN type
error to any clients that contact it while it is downloading an image from
its parent.  The client now has a "-B N" option to tell it to try again
every N seconds as long as it gets back that error.  This is the "store and
forward" mode.  The mserver also has a "cut through" style where it will
return to clients the mcast info it got from its parent so they can download
directly from the parent until the local mserver has it.

Add the ability of the master server to have a "parent" from which it can
download an image if it doesn't have it or if the image is out of date.
Had to add some more goo to the GET reply, notably a hash so that we can
check for up-to-dateness.

The actual part where we upcall to the parent isn't done yet, that is why
this is "inching toward" and not "leaping and bounding toward"...

Also redid the child process management to not use SIGCHLD, no need for that.

Merge in the client keepalive changes.

Experiment: lets see if this helps revive stuck multicast sessions.
With "-K <seconds>", the client will send a IGMP-leave/IGMP-join after
<seconds> of no received packets.

should retry another time.

There are a couple of new packet types in the frisbee protocol which are
exchanged via TCP with the master server: GETREQUEST and GETREPLY.  The
client passes to the master server an opaque imageid and a couple of options
and gets back the addr/port to use to actually download the image.  The
implementation of the master server is fragile and is more of a test
framework, Grant is working on a more robust master server.  I am mostly
doing a backend that communicates with the Emulab DB to do its authentication
and making the client changes.

The client now uses the -S option to specify the IP address of the master
server and the -F option to specify an imageid.  If no error is returned,
the image is downloaded using the returned addr/port.  If -Q is used in place
of -F, then the client makes a "status only" call getting back info about
whether the named image is accessible to the client and whether a server is
currently running.

On the server side, the new master server (mserver.c) has an Emulab
configuration "backend" that supports host-based authentication.
The IP address of the caller is mapped to a node_id/pid/gid/eid combo
that is used to determine access.  On a request, the specified imageid is
treated either as a pathname (if it starts with '/') or an image identifier
of the form "<pid>/<imagename>".  If it is a pathname, we check to make
sure that pathname (after running through "realpath") is contained in one
of the directories accessible to that node in its current experiment context;
i.e., /share, /proj/<pid>, /groups/<pid>/<gid>, or /users/<swapper-uid>.
If it is an image identifier, the DB is queried to ensure that access is
allowed to that image; i.e., it must be "global" or in the appropriate
project/group.

The master server forks a frisbeed for each valid request, if one is not
already running.  The multicast address selection is still based on the
emulab_indicies.frisbee_index field, but the address/port/server info is no
longer stored in the frisbee_blobs table (frisbee_pid, load_address,
load_busy are not set).

Note that this is not yet integrated in the os_load path.  Further work is
required to replace frisbeelauncher.

For this update script we have to be able to find the frisbee_pid field
whether it is in the images table or the frisbee_blobs table.  Where it is
depends on what version of the software we are updating from!

This including redirecting Frisbee output to a logfile.

The "-u" option depends on GNU tar and will do the incorrect thing
with BSD tar.  Thus, try to find "gtar" and use it if available, if
not use bsd tar and print a warning if the rarely used "-u" option is
used.

Fixed a segfault in the new event scheduler

Cleaned up some code that dealt with converting
xmlrpc_c::value_strings to C string constants to avoid further memory
corruption issues.

Also converted AddAgent(), AddGroup(), AddUserEnv() to take 'const
char *' pointers instead of 'char *'s to silence GCC's warnings about
invalid conversions from 'const char *' to 'char *'.

This includes create a large local filesystem (using mkextrafs) if the
target directory starts with "/local".

These changes where suppose to go in with my initial commit, but I
forgot to push from my test node to my devel tree before rebasing and
hanse they didn't make it in.

No changes in functionally.

This prevents it from powering off nodes and generally being quite so
anal about "security violations" in the SECURE boot/load path.  We will
leave this on til we get all the d710 kinks worked out.

This should never have been there in the first place, and I don't want to
carry that hack forward into the post-frisbeelauncher world.