Add a "monitor" process to start/restart sliver to watch nodes.
This is very similar to what Emulab does on the swapin path for normal experiments; wait and watch the nodes to see which ones fail or otherwise timeout. Up till now, we did not do this on the PG path, and so failed nodes were never signaled, and the slice was left in a changing state forever. This also allows us to capture the node bootlogs and convert them to logfiles that we can associate with the slice on the showslice web page. Details: start/restart forks a child (WrapperFork()) and allows the parent to return to the client. The slice is unlocked so that the client can call SliverStatus(), etc. But the client cannot do anything that actually changes the sliver (update, stop, etc) until the monitor finishes (or times out on its own). The lone exception is Deleteslice(), which will asynchronously kill the monitor and then terminate the slice. Ditto the command line script "cleanupslice". We will probably need to add another way to allow the client to terminate the monitor early, but have not decided where yet.
Showing with 448 additions and 19 deletions