GitLab

In APT, the management network is not the same as was defined in
dhcpd_makeconf. The values that were hardwired there are now the
defaults.

The goal is to put all of the ilo management interfaces (except for
the control node of course) on to a private network. We determine
this by looking at the IP in the interface entry, but since dhcpd
will not allow a mix of subnets have to create a new subnet for it.

Then when we have to interact with the ilo we will have to set up an
openvpn tunnel to the control node. Why a VPN? Cause the remote
console will not work across an http proxy.

This code can be further generalized if we think it will be useful in
a non-genirack setting.

1. web_tasks: This table is to track background processes that we
   start either from the web interface or from the protogeni XMLRPC
   interface. The goal is to not have a bunch of task specific stuff
   in tables, but a more general way to start a process tied to an
   object (say, like taking an image) and record state of the process
   in the descriptor (say, the image size as it grows). A client can
   poll for info about the process; the server just looks for the task
   descriptor associated with the object and returns the (currently)
   adhoc data in JSON format to the client.

   Work in progress, still determining if this general approach is
   worth it.

2. web_sessions: Playing with real PHP sessions, using the DB to store
   the session data (instead of the default, which is data files in
   /var/run). No idea how I am going to use this one, just want to
   play with sessions.

interface_type() should actually be called type().
Inadequate quoting in a SQL query.
And assigning through a reference needs an extra '$'.

notion of "dedicated" is currently a type specific attribute, but we
also have "shared" nodes running on "dedicated" nodes, which messes
everything up. I am not inclined to fix the underlying problem since
Utah is the only site that uses this stuff, and these nodes are slowly
dying out anyway.

Ironic. After all that hoo-haw about dynamically-created MBRs in the
previous commit, we introduce a new hardcoded static MBR...

Goals:
 * Single OS partition, in the fast area of rotating disks
 * Proper alignment for 4k sector drives and SSDs (1MB alignment)
 * Still works on 40GB disk (aka, pc850s)
 * P1 16 GiB: goldilocks FS: not too small (> 10GB), but not too big (< 30GB)
 * P2  3 GiB: can hold a co-loaded MBR 1 image (e.g., FBSD410 delay node OS)
 * P3  3 GiB: at least as much swap at currently (>= 1GB)

The partition table:
  P1:     2048	33554432	FS, 16GiB
  P2: 33556480	 6291456	possible delay-node OS or additional swap, 3GiB
  P3: 39847936	 6291456	swap, 3GiB
  P4: 46139392	34278848+	extrafs, 16+GB

The larger size and partition alignment are why I pushed this through despite
my noble intentions.

Hopefully, my last schema change related to images. If relocatable is not
set then an image must be loaded at the lba_low offset. If set, then the
image can be loaded at other offsets. Currently, all FBSD images are
relocatable courtesy of the relocation mechanism in imagezip (which can
fix up otherwise absolute offsets in an image). Sadly, Linux images are
not relocatable due to absolute block numbers in the grub partition
bootblock that we require. Ryan "taught" imagezip to relocate these, but
I need to find his changes.

These are computed by imagedump for .ndz images. The plan is to
pass this info on to clients via tmcc so they can know the max disk
size required.

There will shortly be a utility to automatically update these values
when an image is created or updated. Stay tuned.

Doing this here allows jailconfig in tmcd to return it.

It's going to be used by both OSinfo and Node objects.  New OSes will
want to inherit taint states from the OS they are derived from.

This will currently work with os descriptors and nodes.

Can't do the untainting for all cases in libosload*.  The untainting
is now hooked into stated, where we catch the nodes as they send
along their "RELOADDONE" events to update their taint state according
to the final state of their partitions.

Emulab can now propagate OS taint traits on to nodes that load these OSes.
The primary reason for doing this is for loading images which
require special treatment of the node.  For example, an OS that has
proprietary software, and which will be used as an appliance (blackbox)
can be marked (tainted) as such.  Code that manages user accounts on such
OSes, along with other side channel providers (console, node admin, image
creation) can key off of these taint states to prevent or alter access.

Taint states are defined as SQL sets in the 'os_info' and 'nodes' tables,
kept in the 'taint_states' column in both.  Currently these sets are comprised
of the following entries:

* usermode: OS/node should only allow user level access (not root)
* blackbox: OS/node should allow no direct interaction via shell, console, etc.
* dangerous: OS image may contain malicious software.

Taint states are inherited by a node from OSes it loads during the OS load
process.  Similarly, they are cleared from nodes as these OSes are removed.
Any taint state applied to a node will currently enforce disk zeroing.

No other tools/subsystems consider the taint states currently, but that will
change soon.

Setting taint states for an OS has to be done via SQL presently.

We have had the mechanism implemented in the client for some time and
available at the site-level or, in special cases, at the node level.
New NS command:

    tb-set-nonfs 1

will ensure that no nodes in the experiment attempt to mount shared
filesystems from ops (aka, "fs"). In this case, a minimal homdir is
created on each node with basic dotfiles and your .ssh keys. There will
also be empty /proj, /share, etc. directories created.

One additional mechanism that we have now is that we do not export filesystems
from ops to those nodes. Previously, it was all client-side and you could
mount the shared FSes if you wanted to. By prohibiting the export of these
filesystems, the mechanism is more suitable for "security" experiments.

for stitching, via the external_networks table. For Nick.

If they specify filesystem creation, it can take 5 minutes or longer
depending on the FS type and size. We print a warning about this.

The -f <fstype> option to createdataset will now pre-initialize the dataset
with an empty filesystem. Supported types are: ufs and ext[234].

We want the lease to appear unused up until it is first mapped.
Also, fixed some typos in comments.

start of a page to create new profiles, lots of other changes
and additions.

Add -1 option to run the lease_daemon for exactly one pass and then exit.

Allow fractional values for some of the sitevars whose values are measured
in days. Mostly for debugging, in normal use, a granularity of days is fine.

Tweak the log output.

Add CLI for extending a lease (called extenddataset on ops). The length
of the extension and the number of times it can be extended are controlled
by site variables.

Walks leases through their various states, enforces expiration and idle
times and grace periods, and expires/locks leases.

As always, probably sends too much email.

Currently, these are not yet in the sitevariables table, I just hardwire
them in Lease.pm til we get the right set.