GitLab

It is back to unique per experiment, per dataset.

Any number of users/experiments can mount a given dataset (given that
they have permission) in read-only mode.  Attempts to mount RW will
fail if the dataset is currently in use.  Attempts to mount RO while
the dataset is in use RW are also prohibited.

Under the hood, iSCSI lease exports (targets) are now managed per-lease
instead of per-experiment.  The set of authorized initiators (based
on network) is manipulated as consumers come and go.  When the last
consumer goes, the export is torn down. Likewise, if there are no
current consumers, a new consumer will cause an iSCSI export to be
created for the lease.

Also included in this commit is a small tweak to implicit lease permissions.

Long ago Leigh added code in the bootlog command to account for the fact that
you might not get all the data associated with a TCP-based command in one
read from the socket. This case also showed up with the hostkeys command
with uploaded multiple pubkeys.

So I moved the code up to handle_requests so that for all TCP-based
commands, we read til we get EOF.

the public IP of ops for the mounts.

We pass through a flag in the tmcd loadinfo call to tell whether to attempt
to do a TRIM when loading the disk (or after loading the disk). If TRIM=1
then we do so.

Since it is not clear from what I have read whether repeated TRIMming is
a detriment to SSD life, we throttle it as follows:

1. We don't TRIM at all unless the sitevariable general/bootdisk_trim_interval
   is non zero. If it is set, we will wait at least that many seconds after
   the previous TRIM before we do it again.

2. We keep track of the last trim via the node_attribute "bootdisk_lasttrim"
   which is a unix timestamp of the last time that tmcd responded to a
   loadinfo request in which it returned TRIM=1.

2. We track, on a per-node basis, whether the boot disk should be TRIMmed
   or not. If the node or node-type attribute "bootdisk_trim" is non-zero,
   we will attempt a trim if the interval has passed since the last trim.

So, we never trim if the sitevariable is 0 (the default value). If it is
non-zero, we only trim the boot disk of those nodes that have the node or
node_type attribute set and only after a sufficient interval has passed.

This does not address non-boot disks, but currently frisbee won't mess
with any other disk anyway. Eventually, we will have to have per-disk or
per-disktype attributes if we want to do this better.

In my cleanup of the "doaccounts" code, it looks like I was a bit
overzealous. The logic for detecting when a node is a shared vnode
host was incomplete, and so matched for the vnodes themselves too.

Keeping them up to date throughout the node lifecycle is not a lot of fun...

Also add utility function to allow the node to get the exact details of
the image it is running ('imageinfo').

Some of the taint checks are rather heavy-handed presently.  Pretty much
any vector that could be used by the user to do something as root has
been severed right at the top of the relevant tmcd calls.

Calls affected:

manifest ('blackbox' and 'useronly' taintstates)
rpms ('blackbox' and 'useronly' taintstates)
tarballs ('blackbox' and 'useronly' taintstates)
blobs ('blackbox' and 'useronly' taintstates)
startupcmd ('blackbox' taintstate)
mounts ('blackbox' taintstate)
programs ('blackbox' taintstate)

Taint handling for the 'accounts' call was dealt with in a prior commit.

Taints applied to a node are now extracted from the database and can be
used wherever taint checking needs to happen in tmcd.  I've applied them
to the "accounts" command.  tmcd will never set the "root" flag for
non-admin users on OSes tainted with "useronly".  It won't return any
regular user accounts on nodes with the "blackbox" taint (but will
pass along admin accounts).

We do NOT return interfaces:
  - with obviously hacky and fake addresses: '000000xxxxxx'

  - when they are management (role == 'mngmnt') interfaces.
    Currently there is no OS visible HW for these, the corresponding
    MACs are just used by the management HW.

  - for Infiniband (guid != NULL) interfaces. We need support on
    the client side before we start sending those over.

No more hardwiring in bsdefs.h (which has been moved out
of the clientside dir since it is only used in tmcd.c).

experiment or node, then tell client USEWEB=1 for downloading tar
files and rpms.

shared hosts. This is required for OpenVZ but not for XEN, so this does
mean that when using ZFS, shared OpenVZ hosts are not supported. Not likely
to be a problem.

If the libevent package is installed, there will be /usr/local/include/event.h
which conflicted with our event.h in the clientside/lib/event directory.

It was returning info for blockstores of the same name in different
experiments...oops!

IMAGEPROVENANCE is on, since that means the site has also upgraded
their MFSs and XEN client side.

From the comment:
     * "BOOTPART=" confuses the old rc.frisbee argument parsing
     * which looks for "PART=" with the RE ".*PART=" which will
     * match BOOTPART= instead. Thus an old script loading a
     * whole disk image (PART=0) winds up trying to load it in
     * partition 2 (BOOTPART=2). So we can pick one of two
     * versions, the one in effect when rc.frisbee changed its
     * argument parsing (v30, circa 6/28/2010) or the version
     * in effect when BOOTPART was added (v36, circa 6/13/2013).
     * We choose the latter.

This prevents a line with a single "." from meaning EOF to sendmail.
How arcane!

I discovered this when I ran a create_image and I didn't get the complete
log mailed to me. This is because create_image did a frisbee download of an
image with a single chunk, which of course printed out:

  Using Multicast 235.252.1.187
  Joined the team after 0 sec. ID is 1586355915. File is 1 chunks (963200 bytes)
  .

Fortunately, "arcane" is my middle name, so it didn't take me long to find
this...

of gre so we can tell on the client to reduce the MTU. MTU is a pain.

New loadinfo returns:

IMAGELOW, IMAGEHIGH: range of sectors covered by the image.
    This is NOT the same as what imageinfo or imagedump will show.
    For partition images, these low and high values are adjusted
    for the MBR offset of the partition in question. So when loading
    a Linux image, expect values like 6G and 12G. The intent here
    (not yet realized) is that these values will be used to construct
    an MBR/GPT on the fly, rather than using hardcode magic MBR versions.
    You can get the uncompressed size of the image with (high - low + 1).

IMAGESSIZE: the units of the low/high values.
    Always 512 right now, may be 4096 someday.

IMAGERELOC: non-zero if the image can be placed at an offset other
    than IMAGELOW (i.e., it can be relocated). This may or may not
    prove useful for dynamic MBR construction...we will see.

Probably didn't need to bump the version here, but I am playing it safe.

I hear people are complaining about getting csh, and it seems reasonable
that most GENI folk are more likely to be surprised/confused by tcsh than
bash.

The old limit (2K) was big enough that essentially any hand-written
rspec would work fine, but also small enough that pretty much any manifest
for a Flack-generated request rspec would fail.

This allows dom0 to set the password of the guest at creation time, so
that if something goes wrong, we can get in on the console. This also
fixes an error where on a shared node, we were returning the password
hash for the physical host. Return a per-node hash instead.

Also abstract out the various places we get read from /dev/urandom.

This differs from the current firewall support, which assumes a single
firewall for an entire experiment, hosted on a dedicated physical
node. At some point, it would be better to host the dedicated firewall
inside a XEN container, but that is a project for another day (year).

Instead, I added two sets of firewall rules to the default_firewall_rules
table, one for dom0 and another for domU. These follow the current
style setup of open,basic,closed, while elabinelab is ignored since it
does not make sense for this yet.

These two rules sets are independent, the dom0 rules can be applied to
the physical host, and domU rules can be applied to specific
containers.

My goal is that all shared nodes will get the dom0 closed rules (ssh
from local boss only) to avoid the ssh attacks that all of the racks
are seeing.

DomU rules can be applied on a per-container (node) basis. As
mentioned above this is quite different, and needed minor additions to
the virt_nodes table to allow it.

We have had the mechanism implemented in the client for some time and
available at the site-level or, in special cases, at the node level.
New NS command:

    tb-set-nonfs 1

will ensure that no nodes in the experiment attempt to mount shared
filesystems from ops (aka, "fs"). In this case, a minimal homdir is
created on each node with basic dotfiles and your .ssh keys. There will
also be empty /proj, /share, etc. directories created.

One additional mechanism that we have now is that we do not export filesystems
from ops to those nodes. Previously, it was all client-side and you could
mount the shared FSes if you wanted to. By prohibiting the export of these
filesystems, the mechanism is more suitable for "security" experiments.

For persistent blockstores, is based on the value of the "readonly"
virt_blockstore_attributes attribute if it exists. The RO attribute
is set by libvtop when an attempt is made to use a lease that is in
the 'grace' state.

Based on suggestions from <ahelsing@bbn.com>.

or per vhost), we return the mac of the xen host since the host is
acting as a router for the containers on that vhost.