...
 
Commits (11)
  • David Johnson's avatar
    Add more docker clientside tools. · b0f17e8e
    David Johnson authored
    Especially libvnode_docker::analyzeImageWithBusyboxCommand, which runs
    the busybox-static binary from the root context in the container with
    the given command (and extra container run API configuration).
    b0f17e8e
  • Leigh Stoller's avatar
    43cdb619
  • Leigh Stoller's avatar
    Minor changes. · ed2dce21
    Leigh Stoller authored
    ed2dce21
  • Leigh Stoller's avatar
    bbf42391
  • Leigh Stoller's avatar
    Minor fix. · 2692b61f
    Leigh Stoller authored
    2692b61f
  • Leigh Stoller's avatar
    New script to compute reservation timelines and utilization number. · 872a5af1
    Leigh Stoller authored
    Initially intended for debugging, but now its more useful. :-)
    872a5af1
  • Leigh Stoller's avatar
    adefb6f5
  • Leigh Stoller's avatar
    Fix a bug that was introduced when we shifted to using os_setup · e59fc714
    Leigh Stoller authored
    directly (on the Cloudlab clusters); we were losing a lock out that
    allowed DeleteSliver() to run while in the middle of a CreateSliver().
    This was resulting in a lot of email about node failures since the nodes
    were getting yanked out from underneath the CreateSliver(). From the
    user perspective, this did not matter much, since they wanted the slice
    gone, but it finally bothered me enough to look more closely.
    e59fc714
  • Leigh Stoller's avatar
    c5259a31
  • David Johnson's avatar
    Docker server-side core, esp new libimageops support for Docker images. · 66366489
    David Johnson authored
    The docker VM server-side goo is mostly identical to Xen, with slightly
    different handling for parent images.  We also support loading external
    Docker images (i.e. those without a real imageid in our DB; in that
    case, user has to set a specific stub image, and some extra per-vnode
    metadata (a URI that points to a Docker registry/image repo/tag);
    the Docker clientside handles the rest.
    
    Emulab Docker images map to a Emulab imageid:version pretty seamlessly.
    For instance, the Emulab `emulab-ops/docker-foo-bar:1` image would map
    to `<local-registry-URI>/emulab-ops/emulab-ops/docker-foo-bar:1`; the
    mapping is `<local-registry-URI>/pid/gid/imagename:version`.  Docker
    repository names are lowercase-only, so we handle that for the user; but
    I would prefer that users use lowercase Emulab imagenames for all Docker
    images; that will help us.  That is not enforced in the code; it will
    appear in the documentation, and we'll see.
    
    Full Docker imaging relies on several other libraries
    (https://gitlab.flux.utah.edu/emulab/pydockerauth,
    https://gitlab.flux.utah.edu/emulab/docker-registry-py).  Each
    Emulab-based cluster must currently run its own private registry to
    support image loading/capture (note however that if capture is
    unnecessary, users can use the external images path instead).  The
    pydockerauth library is a JWT token server that runs out of boss's
    Apache and implements authn/authz for the per-Emulab Docker registry
    (probably running on ops, but could be anywhere) that stores images and
    arbitrates upload/download access.  For instance, nodes in an experiment
    securely pull images using their pid/eid eventkey; and the pydockerauth
    emulab authz module knows what images the node is allowed to pull
    (i.e. sched_reloads, the current image the node is running, etc).  Real
    users can also pull images via user/pass, or bogus user/pass + Emulab
    SSL cert.  GENI credential-based authn/z was way too much work, sadly.
    There are other auth/z paths (i.e. for admins, temp tokens for secure
    operations) as well.
    
    As far as Docker image distribution in the federation, we use the same
    model as for regular ndz images.  Remote images are pulled in to the
    local cluster's Docker registry on-demand from their source cluster via
    admin token auth (note that all clusters in the federation have
    read-only access to the entire registries of any other cluster in the
    federation, so they can pull images).  Emulab imageid handling is the
    same as the existing ndz case.  For instance, image versions are lazily
    imported, on-demand; local version numbers may not match the remote
    image source cluster's version numbers.  This will potentially be a
    bigger problem in the Docker universe; Docker users expect to be able to
    reference any image version at any time anywhere.  But that is of course
    handleable with some ex post facto synchronization flag day, at least
    for the Docker images.
    
    The big new thing supporting native Docker image usage is the guts of a
    refactor of the utils/image* scripts into a new library, libimageops;
    this is necessary to support Docker images, which are stored in their
    own registry using their own custom protocols, so not amenable to our
    file-based storage.  Note: the utils/image* scripts currently call out
    to libimageops *only if* the image format is docker; all other images
    continue on the old paths in utils/image*, which all still remain
    intact, or minorly-changed to support libimageops.
    
    libimageops->New is the factory-style mechanism to get a libimageops
    that works for your image format or node type.  Once you have a
    libimageops instance, you can invoke normal image logical operations
    (CreateImage, ImageValidate, ImageRelease, et al).  I didn't do every
    single operation (for instance, I haven't yet dealt with image_import
    beyond essentially generalizing DownLoadImage by image format).
    Finally, each libimageops is stateless; another design would have been
    some statefulness for more complicated operations.   You will see that
    CreateImage, for instance, is written in a helper-subclass style that
    blurs some statefulness; however, it was the best match for the existing
    body of code.  We can revisit that later if the current argument-passing
    convention isn't loved.
    
    There are a couple outstanding issues.  Part of the security model here
    is that some utils/image* scripts are setuid, so direct libimageops
    library calls are not possible from a non-setuid context for some
    operations.  This is non-trivial to resolve, and might not be worthwhile
    to resolve any time soon.  Also, some of the scripts write meaningful,
    traditional content to stdout/stderr, and this creates a tension for
    direct library calls that is not entirely resolved yet.  Not hard, just
    only partly resolved.
    
    Note that tbsetup/libimageops_ndz.pm.in is still incomplete; it needs
    imagevalidate support.  Thus, I have not even featurized this yet; I
    will get to that as I have cycles.
    66366489
  • Elijah Grubb's avatar
......@@ -811,7 +811,8 @@ sub ComputeNodeCounts($)
if (defined($virtualization_type) &&
($virtualization_type eq "emulab-xen" ||
$virtualization_type eq "emulab-blockstore")) {
$virtualization_type eq "emulab-blockstore" ||
$virtualization_type eq "emulab-docker")) {
$vcount++;
next;
}
......
......@@ -860,8 +860,9 @@ sub CheckFirewall($$)
# and closed.
#
my $style = "closed";
if (defined($virtualization_type) &&
$virtualization_type eq "emulab-xen" && !@routable_control_ip) {
if (defined($virtualization_type) && !@routable_control_ip &&
($virtualization_type eq "emulab-xen"
|| $virtualization_type eq "emulab-docker")) {
$style = "basic";
}
......
#!/usr/bin/perl -wT
#
# Copyright (c) 2007-2017 University of Utah and the Flux Group.
# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -831,6 +831,8 @@ sub new($$$$$)
"jacks_site" => undef,
"xen_settings" => undef,
"xen_ptype" => undef,
"docker_settings" => undef,
"docker_ptype" => undef,
"instantiate_on" => undef,
"services" => [],
"statements" => [],
......@@ -1101,6 +1103,18 @@ sub addNode($$$)
$node->{"xen_settings"} = $settings;
last SWITCH;
};
# Docker settings
/^docker$/i && do {
my $settings = GeniXML::GetDockerSettings($ref->parentNode);
fatal("Failed to get docker settings")
if (!defined($settings));
$node->{"docker_settings"} = $settings;
last SWITCH;
};
/^docker_ptype$/i && do {
$node->{"docker_ptype"} = GetTextOrFail("name", $child);
last SWITCH;
};
/^(sliver_type_shaping|firewall_config)$/i && do {
# We handled this above.
last SWITCH;
......@@ -1291,7 +1305,7 @@ sub Compare($$)
last SWITCH;
};
(/^(component_id|component_manager_id|disk_image)$/i ||
/^(hardware_type|jacks_site|xen_ptype|instantiate_on)$/i ||
/^(hardware_type|jacks_site|xen_ptype|docker_ptype|instantiate_on)$/i ||
/^(adb_target|failure_action)$/i ||
/^(isexptfirewall|firewall_style)$/i ||
/^(use_type_default_image|routable_control_ip)$/i) && do {
......@@ -1304,7 +1318,7 @@ sub Compare($$)
# Handled up above in CompareNodes.
last SWITCH;
};
/^(xen_settings|desires|pipes|blockstores|attributes)$/i && do {
/^(xen_settings|docker_settings|desires|pipes|blockstores|attributes)$/i && do {
return 1
if (APT_Rspec::CompareHashes("Node: $client_id: $key",
$val1, $val2));
......
......@@ -29,8 +29,10 @@ package APT_Utility;
use strict;
use English;
use Data::Dumper;
use Date::Parse;
use Carp;
use Exporter;
use POSIX qw(ceil);
use vars qw(@ISA @EXPORT);
@ISA = "Exporter";
......@@ -44,6 +46,7 @@ use Project;
use Group;
use GeniHRN;
use GeniUser;
use emutil;
# Configure variables
my $TB = "@prefix@";
......@@ -126,3 +129,88 @@ sub MapUserURN($)
return undef;
}
#
# Given a reservation details hash, calculate a utilization number
# from the history array.
#
sub ReservationUtilization($$)
{
my ($res, $active) = @_;
my $count = $res->{'nodes'};
my $resstart = str2time($res->{'start'});
my $resend = ($active ? time() :
(str2time(defined($res->{'deleted'}) ?
$res->{'deleted'} : $res->{'end'})));
my $reshours = (($resend - $resstart) / 3600) * $count;
my $usedhours = 0;
my $inuse = 0;
my $laststamp = $resstart;
my $type = $res->{'type'};
my $remote_uid = $res->{'remote_uid'};
my @tmp = @{$res->{'history'}};
# Init for web interface.
$res->{'reshours'} = $reshours;
$res->{'usedhours'} = undef;
$res->{'utilization'} = undef;
#
# Scan past timeline entries that are *before* the start of the
# reservation; these are experiments that were running when the
# reservation started, and provide the number of nodes allocated
# at the time the reservation starts.
#
while (@tmp) {
my $ref = $tmp[0];
my $stamp = $ref->{'t'};
my $allocated = $ref->{'allocated'};
$ref->{'tt'} = TBDateStringGMT($ref->{'t'});
last
if ($stamp >= $resstart);
# Watch for nothing allocated by the user at this time stamp
my $using = 0;
if (exists($allocated->{$remote_uid})) {
$using = $allocated->{$remote_uid}->{$type};
}
$inuse = $using;
$inuse = $count if ($inuse > $count);
shift(@tmp);
}
foreach my $ref (@tmp) {
my $stamp = $ref->{'t'};
my $reserved = $ref->{'reserved'};
my $allocated = $ref->{'allocated'};
$ref->{'tt'} = TBDateStringGMT($ref->{'t'});
# If this stamp is after the reservation, we can stop. The
# last entry will be the current number of nodes used till
# the end of the reservation. This entry is typically for the
# end of an experiment start before the end of the reservation.
last
if ($stamp > $resend);
# Watch for nothing allocated by the user at this time stamp
my $using = 0;
if (exists($allocated->{$remote_uid})) {
$using = $allocated->{$remote_uid}->{$type};
}
$usedhours += (($stamp - $laststamp) / 3600) * $inuse;
$laststamp = $stamp;
$inuse = $using;
$inuse = $count if ($inuse > $count);
}
# And then a final entry for usage until the end of the reservation.
if ($laststamp) {
$usedhours += (($resend - $laststamp) / 3600) * $inuse;
}
$res->{'reshours'} = $reshours;
$res->{'usedhours'} = $usedhours;
$res->{'utilization'} = POSIX::ceil(($usedhours/$reshours) * 100.0);
$res->{'utilization'} = 100 if ($res->{'utilization'} > 100);
return 0;
}
......@@ -2112,19 +2112,24 @@ sub DoMaxExtensionInternal($$)
#
$res->{'cluster'} = $aptagg->urn();
$res->{'cluster_id'} = $aptagg->nickname();
# Backwards compat
if (!exists($res->{'nodes'})) {
$res->{'nodes'} = $res->{'count'};
}
if ($res->{'approved'} eq "") {
# Maps to JSON NULL.
$res->{'approved'} = undef;
}
if (!exists($res->{'cancel'}) || $res->{'cancel'} eq "") {
# Maps to JSON NULL.
$res->{'cancel'} = undef;
# We need numbers.
$res->{'nodes'} = int($res->{'nodes'});
$res->{'using'} = int($res->{'using'});
#
# Hmm, undef/null is a pain with XMLRPC.
#
foreach my $key ("approved", "cancel", "deleted") {
if (!exists($res->{$key}) || $res->{$key} eq "") {
$res->{$key} = undef;
}
}
#
# Lets calculate a utilization numbers, this will update the
# res hash. Flag says not an active reservation.
#
APT_Utility::ReservationUtilization($res, 0);
}
$blob->{'reservations'}->{$aptagg->urn()} = $reslist;
}
......
......@@ -28,7 +28,7 @@ use XML::Simple;
use Data::Dumper;
use CGI;
use POSIX ":sys_wait_h";
use POSIX qw(:signal_h);
use POSIX qw(:signal_h ceil);
use Date::Parse;
#
......@@ -516,6 +516,10 @@ sub DoList()
$details->{'cluster_id'} = $aggregate->nickname();
$details->{'cluster_urn'} = $aggregate->urn();
# We need numbers.
$details->{'nodes'} = int($details->{'nodes'});
$details->{'using'} = int($details->{'using'});
# Backwards compat
if (!exists($details->{'nodes'})) {
$details->{'nodes'} = $details->{'count'};
......@@ -524,18 +528,19 @@ sub DoList()
#
# Hmm, undef/null is a pain with XMLRPC.
#
if ($details->{'approved'} eq "") {
# Maps to JSON NULL.
$details->{'approved'} = undef;
}
if ($details->{'cancel'} eq "") {
# Maps to JSON NULL.
$details->{'cancel'} = undef;
}
if (!exists($details->{'uuid'}) || $details->{'uuid'} eq "") {
$details->{'uuid'} = NewUUID();
foreach my $key ("approved", "cancel", "deleted") {
if (!exists($details->{$key}) || $details->{$key} eq "") {
$details->{$key} = undef;
}
}
#
# Lets calculate a utilization numbers, this will update the
# res hash. Flag says not an active reservation.
#
APT_Utility::ReservationUtilization($details, 1);
#
# If we have the history, then go through and map the
# experiments to local experiments so we can link to them in
......@@ -1141,6 +1146,10 @@ sub DoHistory()
$res->{'cluster_id'} = $aggregate->nickname();
$res->{'cluster_urn'} = $aggregate->urn();
# We need numbers.
$res->{'nodes'} = int($res->{'nodes'});
$res->{'using'} = int($res->{'using'});
# Backwards compat
if (!exists($res->{'nodes'})) {
$res->{'nodes'} = $res->{'count'};
......@@ -1148,22 +1157,17 @@ sub DoHistory()
#
# Hmm, undef/null is a pain with XMLRPC.
#
if (!exists($res->{'approved'}) ||
$res->{'approved'} eq "") {
$res->{'approved'} = undef;
}
if (!exists($res->{'cancel'}) ||
$res->{'cancel'} eq "") {
$res->{'cancel'} = undef;
}
if (!exists($res->{'deleted'}) ||
$res->{'deleted'} eq "") {
$res->{'deleted'} = undef;
}
if (!exists($res->{'uuid'}) ||
$res->{'uuid'} eq "") {
$res->{'uuid'} = NewUUID();
foreach my $key ("approved", "cancel", "deleted") {
if (!exists($res->{$key}) || $res->{$key} eq "") {
$res->{$key} = undef;
}
}
#
# Lets calculate a utilization numbers, this will update the
# res hash. Flag says not an active reservation.
#
APT_Utility::ReservationUtilization($res, 0);
}
done:
if (defined($webtask)) {
......
#!/usr/bin/perl -w
#
# Copyright (c) 2000-2017 University of Utah and the Flux Group.
# Copyright (c) 2000-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -201,6 +201,45 @@ sub GenerateNodeStatements($)
$node->addTagStatement("InstantiateOn('$vhost')");
}
}
elsif ($ntype eq "emulab-docker") {
$node->addStatement("$ntag = request.DockerContainer('$client_id')");
#
# This is the only time we need to spit this out, since
# the default is False.
#
if (defined($node->{'exclusive'}) && $node->{'exclusive'}) {
$node->addTagStatement("exclusive = True");
}
if (defined($node->{'docker_settings'})) {
my $settings = $node->{'docker_settings'};
foreach my $setting (sort(keys(%{$settings}))) {
my $value = $settings->{$setting};
if ($setting eq "ram") {
$node->addTagStatement("ram = $value");
}
elsif ($setting eq "cores") {
$node->addTagStatement("cores = $value");
}
elsif ($setting eq "extimage") {
$node->addTagStatement("docker_extimage = $value");
}
elsif ($setting eq "dockerfile") {
$node->addTagStatement("docker_dockerfile = $value");
}
}
}
if (defined($node->{'docker_ptype'})) {
my $ptype = $node->{'docker_ptype'};
$node->addTagStatement("docker_ptype = '$ptype'");
}
if (defined($node->{'instantiate_on'})) {
my $vhost = $node->{'instantiate_on'};
$node->addTagStatement("InstantiateOn('$vhost')");
}
}
elsif ($ntype eq "delay") {
#
# Bridges are also special, see comment above for blockstore.
......
......@@ -174,6 +174,7 @@ my %xmlfields =
"global", => ["global", $SLOT_ADMINONLY, 0],
"mbr_version", => ["mbr_version", $SLOT_OPTIONAL],
"makedefault", => ["makedefault", $SLOT_ADMINONLY, 0],
"format", => ["format", $SLOT_ADMINONLY, "ndz"],
);
#
......@@ -490,14 +491,27 @@ elsif (! $isadmin) {
UserError("Path: Invalid Path");
}
}
if ($newimageid_args{"path"} =~ /\/$/) {
if (-e $newimageid_args{"path"} && ! -d $newimageid_args{"path"}) {
UserError("Path: invalid path, it should be a directory");
}
if (defined($newimageid_args{"format"})
&& $newimageid_args{"format"} eq "docker") {
#
# We only allow a specific path for docker images, since the storage
# backend relies on an ACL of repos a user can access. It's not a
# filesystem with UNIX permissions. With a docker registry,
# permissions are tied to specific paths. Don't even let admins
# override this for now; there is no point.
#
}
elsif (-d $newimageid_args{"path"} =~ /\/$/) {
UserError("Path: invalid path, its a directory");
else {
if ($newimageid_args{"path"} =~ /\/$/) {
if (-e $newimageid_args{"path"} && ! -d $newimageid_args{"path"}) {
UserError("Path: invalid path, it should be a directory");
}
}
elsif (-d $newimageid_args{"path"} =~ /\/$/) {
UserError("Path: invalid path, its a directory");
}
}
#
# See what node types this image will work on. Must be at least one!
#
......
......@@ -210,6 +210,7 @@ my %xmlfields =
"global", => ["global", $SLOT_OPTIONAL, 0],
"noexport", => ["noexport", $SLOT_OPTIONAL, 0],
"mbr_version", => ["mbr_version", $SLOT_OPTIONAL],
"format", => ["format", $SLOT_OPTIONAL],
"metadata_url", => ["metadata_url", $SLOT_ADMINONLY],
"imagefile_url", => ["imagefile_url", $SLOT_ADMINONLY],
"origin_uuid", => ["origin_uuid", $SLOT_ADMINONLY],
......
......@@ -516,7 +516,10 @@ docker-install: dir-install
$(BINDIR)/run/rcmanifest.d
$(INSTALL) -m 755 $(SRCDIR)/docker/create-docker-image $(LBINDIR)/
$(INSTALL) -m 755 $(SRCDIR)/docker/emulabize-image $(LBINDIR)/
$(INSTALL) -m 755 $(SRCDIR)/docker/analyze-image $(LBINDIR)/
$(INSTALL) -m 755 $(SRCDIR)/docker/analyze-image-with-busybox $(LBINDIR)/
$(INSTALL) -m 755 $(SRCDIR)/vnodectl $(BINDIR)/
$(INSTALL) -m 755 $(SRCDIR)/docker/cleanup-docker-vnodes.sh $(BINDIR)/
# Note that we install this, but do not enable it. It is dynamically
# started and stopped as needed by libvnode_docker.pm .
$(INSTALL) -m 644 -o root -g $(DIRGROUP) \
......
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Std;
use English;
use POSIX;
use Data::Dumper;
#
# A simple CLI wrapper around libvnode_docker::analyzeImage.
#
sub usage()
{
print "Usage: analyze-image [-d <level>] [-f] <image>\n".
"\n".
" -d <level> Debug mode\n".
" -f Analyze the image even if it has already been done\n";
exit(1);
}
my $optlist = "hd:f";
my $debug = 0;
my $force = 0;
#
# Parse command arguments. Once we return from getopts, all that should be
# left are the required arguments.
#
my %options = ();
if (!getopts($optlist,\%options)) {
usage();
}
if (defined($options{"h"})) {
usage();
}
if (defined($options{"d"})) {
$debug = $options{"d"};
}
if (defined($options{"f"})) {
$force = 1;
}
usage()
if (@ARGV > 1);
#
# Must be root.
#
if ($UID != 0) {
die("*** $0:\n".
" Must be root to run this script!\n");
}
#
# Turn off line buffering on output
#
$| = 1;
# Drag in path stuff so we can find emulab stuff.
BEGIN { require "/etc/emulab/paths.pm"; import emulabpaths; }
use libsetup;
use libvnode_docker;
if ($debug) {
TBDebugTimeStampsOn();
libvnode_docker::setDebug($debug);
}
my %rethash = ();
my $image = $ARGV[0];
my $rc = libvnode_docker::analyzeImage($image,\%rethash,$force);
if ($rc) {
fatal("ERROR: failed to analyze $image!\n");
}
else {
print "Successfully analyzed $image:\n";
for my $k (keys(%rethash)) {
my $v = '';
if (defined($rethash{$k})) {
$v = $rethash{$k};
}
print "$k=$v\n";
}
}
exit(0);
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Std;
use English;
use POSIX;
use Data::Dumper;
use JSON::PP;
#
# A simple CLI wrapper around libvnode_docker::analyzeImageWithBusyboxCommand.
#
sub usage()
{
print "Usage: analyze-image-with-busybox [-d <level>] [-c] <image> [command... ]\n".
"\n".
" -d <level> Debug mode\n".
" -c <json> Customize the temporary container config according".
" to JSON passed to the Docker run API command.\n";
exit(1);
}
my $optlist = "hd:c:";
my $debug = 0;
my $config = {};
#
# Parse command arguments. Once we return from getopts, all that should be
# left are the required arguments.
#
my %options = ();
if (!getopts($optlist,\%options)) {
usage();
}
if (defined($options{"h"})) {
usage();
}
if (defined($options{"d"})) {
$debug = $options{"d"};
}
if (defined($options{"c"})) {
$config = decode_json($options{"c"});
}
usage()
if (@ARGV < 2);
#
# Must be root.
#
if ($UID != 0) {
die("*** $0:\n".
" Must be root to run this script!\n");
}
#
# Turn off line buffering on output
#
$| = 1;
# Drag in path stuff so we can find emulab stuff.
BEGIN { require "/etc/emulab/paths.pm"; import emulabpaths; }
use libsetup;
use libvnode_docker;
if ($debug) {
TBDebugTimeStampsOn();
libvnode_docker::setDebug($debug);
}
my $ret;
my $image = shift(@ARGV);
my $rc = libvnode_docker::analyzeImageWithBusyboxCommand(
$image,$config,\$ret,@ARGV);
if ($rc) {
fatal("ERROR: failed to analyze $image with busybox!\n");
}
else {
print "Successfully analyzed $image with busybox:\n$ret";
}
exit(0);
#!/bin/sh
#
# Cleans up *everything* related to the specified vnode, or to all
# vnodes if -a is specified instead of a vnode.
#
usage() {
if [ -n "$1" ]; then
echo "ERROR: $1"
fi
echo "USAGE: $0 [-h] [-a | vnode... ]"
echo " -a Remove all vnodes"
echo " vnode... A space-separated list of vnodes to remove"
exit 1
}
cleanupvnode() {
vnode=$1
echo "Cleaning up vnode $vnode..."
/usr/local/etc/emulab/vnodesetup -d -k -j $vnode
if [ -d /var/emulab/boot/tmcc.$vnode ]; then
echo "WARNING: /var/emulab/boot/tmcc.$vnode exists; removing!"
fi
rm -rf /var/emulab/boot/tmcc.$vnode
}
doall=0
VNODES=""
count="$#"
i=0
if [ "$#" -eq 0 ]; then
usage "no arguments specified"
fi
while [ $i -lt $count ]; do
arg="$1"
case "$arg" in
-h|--help)
usage
;;
-a)
doall=1
if [ -n "$VNODES" ]; then
usage "cannot specify both -a and specific vnodes"
fi
;;
*)
if [ $doall -eq 1 ]; then
usage "cannot specify both specific vnodes and -a ($arg)"
fi
VNODES="$VNODES $arg"
;;
esac
shift
i=`expr $i + 1`
done
if [ $doall -eq 1 ]; then
VNODES=`cat /var/emulab/boot/tmcc/vnodelist | sed -e 's/^VNODEID=\([^ ]*\).*/\1/' | sort | xargs`
fi
for vnode in $VNODES; do
cleanupvnode $vnode
done
......@@ -38,7 +38,7 @@ sub usage()
exit(1);
}
my $optlist = "hfd:e:P:u:p:D";
my $debug = 1;
my $debug = 0;
my $emulabization;
my $pullpolicy;
my $update = 0;
......
......@@ -4306,6 +4306,55 @@ sub vnodeDestroy($$$$)
## Utility and helper functions.
##
sub analyzeImageWithBusyboxCommand($$$@)
{
my ($image,$configref,$outputref,@bargv) = @_;
TBDebugTimeStamp("running static busybox (".join('',@bargv).")".
" for image $image...");
my $args = {
'HostConfig' => {
'Binds' => [ "/bin/busybox:/tmp/busybox:ro" ]
},
'Entrypoint' => '',
};
my @argv = ('/tmp/busybox',@bargv);
if (defined($configref)) {
require Hash::Merge;
$args = Hash::Merge::merge($args,$configref);
if ($debug) {
print STDERR "DEBUG: merged args = ".Dumper($args)."\n";
}
}
my $tmpname = "busybox-analyzer-".int(rand(POSIX::INT_MAX));
our $buf = '';
my ($code,$json,$resp,$retval) = getClient->container_run(
$tmpname,$image,\@argv,1,$args,sub { $buf .= $_[0]; });
if ($code) {
warn("failed to run busybox analysis container $tmpname for $image");
return $code;
}
# Santize the output. For whatever reason(s), under heavy load, it
# comes with non-printable chars occasionally, and with CRLF. Odd.
$buf =~ s/[\000-\011\014\015-\037\176-\255]//g;
TBDebugTimeStamp("busybox analyze output:\n$buf");
open(FD,">/vms/contexts/$tmpname.log");
print FD $buf;
close(FD);
if (defined($outputref)) {
if (ref($$outputref) eq 'ARRAY') {
$$outputref = split("\n",$buf);
}
else {
$$outputref = $buf;
}
}
return 0;
}
#
# Analyze an existing Docker image to extra image metadata,
# distro/version, and so on.
......
#!/usr/bin/perl -wT
#
# Copyright (c) 2007-2017 University of Utah and the Flux Group.
# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -638,7 +638,7 @@ sub Create($$$$$$$$)
}
# Pass-through optional slots, otherwise the DB default is used.
foreach my $key ("path", "shared", "global", "ezid", "mbr_version",
"metadata_url", "imagefile_url", "released",
"format", "metadata_url", "imagefile_url", "released",
"isdataset", "lba_size", "lba_low", "lba_high",
"origin_uuid", "origin_urn", "origin_name") {
if (exists($argref->{$key})) {
......@@ -1795,6 +1795,31 @@ sub GetDiskOffset($)
return $mbr{$self->mbr_version()}[$self->loadpart()];
}
#
# Return created time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
#
sub GetCreate($$)
{
my ($self,$stampp) = @_;
# Must be a real reference.
return -1
if (! ref($self));
my $imageid = $self->imageid();
my $version = $self->version();
my $result =
DBQueryWarn("select UNIX_TIMESTAMP(created) from image_versions ".
"where imageid='$imageid' and version='$version'");
if ($result && $result->numrows) {
my ($stamp) = $result->fetchrow_array();
$$stampp = $stamp;
return 0;
}
}
#
# Return updated time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
......@@ -2714,6 +2739,46 @@ sub SetImageMetadataURL($$)
return 0;
}
sub SetImageFileURL($$)
{
my ($self, $url) = @_;
$url = DBQuoteSpecial($url);
# Must be a real reference.
return -1
if (! ref($self));
my $imageid = $self->imageid();
my $version = $self->version();
DBQueryWarn("update image_versions set imagefile_url=$url ".
"where imageid='$imageid' and version='$version'")
or return -1;
$self->{'IMAGE'}->{'image_metadata_url'} = $url;
return 0;
}
sub SetFormat($$)
{
my ($self, $format) = @_;
return -1
if ($self->Update({'format' => $format}));
return 0;
}
sub SetPath($$)
{
my ($self, $path) = @_;
return -1
if ($self->Update({'path' => $path}));
return 0;
}
sub LocalURL($)
{
my ($self) = @_;
......@@ -2856,7 +2921,8 @@ sub IsDirPath($)
return 0;
}
return 1
if ($self->path() =~ /\/$/);
if ($self->path() =~ /\/$/
&& (!defined($self->format()) || $self->format() ne 'docker'));
return 0;
}
sub FullImagePath($)
......@@ -2958,7 +3024,9 @@ sub IsSystemImage($)
my ($self) = @_;
# Yuck.
return ($self->path() =~ /^\/usr\/testbed\// ? 1 : 0);
return ($self->path() =~ /^\/usr\/testbed\// ? 1 : 0
|| ($self->format() eq 'docker' && $self->global()
&& $self->path() =~ /emulab-ops\/emulab-ops/));
}
#
# When images are stored in sub directories.
......
#!/usr/bin/perl -wT
#
# Copyright (c) 2007-2017 University of Utah and the Flux Group.
# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -662,6 +662,17 @@ sub Chunks($)
return $self->image()->Chunks();
}
#
# Return updated time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
#
sub GetCreate($$)
{
my ($self,$stampp) = @_;
return $self->image()->GetCreate($stampp);
}
#
# Return updated time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
......@@ -1052,6 +1063,20 @@ sub SetImageMetadataURL($$)
return $self->image()->SetImageMetadataURL($url);
}
sub SetFormat($$)
{
my ($self, $format) = @_;
return $self->image()->SetFormat($format);
}
sub SetPath($$)
{
my ($self, $path) = @_;
return $self->image()->SetPath($path);
}
sub LocalURL($)
{
my ($self) = @_;
......
......@@ -61,6 +61,7 @@ my %FEATURES = ( "ping" => 1,
"linktest" => 1,
"linkdelays" => 0,
"xen-host" => 0,
"docker-host"=> 0,
"suboses" => 0 );
# Valid OS names. Mirrored in the web interface. The value is a user-okay flag.
......
......@@ -87,7 +87,18 @@ sub CollectReservations($$$)
sub CreateTimeline($@)
{
my ($project, @reservations) = @_;
#
# Remember all the types we care about; when computing the counts
# do not bother with nodes that are not in the set of types reserved.
#
my %typesinuse = ();
foreach my $res (@reservations) {
my $type = $res->type();
$typesinuse{$type} = $type;
}
# We want the earliest/latest reservation for getting project usage();
my $earliest = $reservations[0];
if ($debug) {
......@@ -111,31 +122,43 @@ sub CreateTimeline($@)
};
my @tmp = sort $sortfunc @reservations;
my $latest = $tmp[-1];
my $end = (defined($latest->deleted()) ?
$latest->deleted() : $latest->end());
$end = time() if ($end > time());
if ($debug) {
print "The latest reservation end is " .
POSIX::strftime("%m/%d/20%y %H:%M:%S",
localtime($latest->end())) . " \n";
POSIX::strftime("%m/%d/20%y %H:%M:%S", localtime($end)) . " \n";
}
my $end = $latest->end();
$end = time() if ($end > time());
# Get the usage since the beginning of the earliest reservation.
my $usage = $project->Usage($earliest->start() - (3600 * 24), $end);
my $usage = $project->Usage($earliest->start(), $end);
if ($debug) {
print "There are " . scalar(@$usage) . " usage records\n";
if ($debug > 1 && scalar(@$usage)) {
print Dumper(@$usage);
}
@tmp = ();
foreach my $ref (@$usage) {
foreach my $type (keys(%{$ref->{'types'}})) {
if (exists($typesinuse{$type})) {
push(@tmp, $ref);
last;
}
}
}
if (!scalar(@$usage)) {
print STDERR "There are no usage records to process\n"
if (!scalar(@tmp)) {
print STDERR "There are no usage records left to process\n"
if ($debug);
return ();
}
if ($debug > 1) {
print Dumper(@tmp);
}
# Form a timeline of changes in allocation.
my @timeline = ();
foreach my $ref (@$usage) {
foreach my $ref (@tmp) {
push(@timeline,
{"t" => $ref->{'start'}, "details" => $ref, "op" => "alloc"});
......@@ -147,12 +170,6 @@ sub CreateTimeline($@)
# And sort the new list.
@timeline = sort {$a->{'t'} <=> $b->{'t'}} @timeline;
#
# Remember all the types we care about; when computing the counts
# do not bother with nodes that are not in the set of types reserved.
#
my %typesinuse = ();
#
# Correlate the reservations with the sorted list using the start/end
# of the reservation and the timestamps in the timeline. This tells us
......@@ -171,8 +188,26 @@ sub CreateTimeline($@)
my $resStart = $res->start();
my $resEnd = $res->end();
push(@reslist, $res)
if ($stamp >= $resStart && $stamp <= $resEnd);
if ($stamp >= $resStart && $stamp <= $resEnd) {
push(@reslist, $res);
next;
}
#
# But what if this usage record is for an experiment that started
# prior to the beginning of the reservation, and one of two cases
# is true: 1) The experiment is still running. 2) The end of the
# experiment is during the reservation.
#
# In both these cases the reservation is overlaps with the
# experiment.
#
if ($ref->{'op'} eq "alloc" && $stamp <= $resStart &&
($ref->{'details'}->{'end'} eq "" ||
($ref->{'details'}->{'end'} >= $resStart &&
$ref->{'details'}->{'end'} <= $resEnd))) {
push(@reslist, $res);
print "$res is active for $stamp\n" if ($debug);
}
}
next
if (!@reslist);
......@@ -208,23 +243,6 @@ sub CreateTimeline($@)
}
$ref->{'reserved'} = $reserved;
}
#
# Kill off timeline entries that do not include the types we care
# about (types that are reserved at some point during the timeline).
# This reduces entries the number of entries that do not show an
# interesting change (ie: same as previous entry).
#
@tmp = ();
foreach my $ref (@timeline) {
foreach my $type (keys(%{$ref->{'details'}->{'types'}})) {
if (exists($typesinuse{$type})) {
push(@tmp, $ref);
last;
}
}
}
@timeline = @tmp;
# Hmm, this happens.
return ()
if (!@timeline);
......@@ -234,6 +252,9 @@ sub CreateTimeline($@)
#
my @counts = ComputeCounts($project, \%typesinuse, @timeline);
return @counts
if (1);
#
# We want to add additional entrys for the start of each reservation.
#
......
......@@ -1225,6 +1225,49 @@ sub ProjectReservations($$$;$$) {
return @answer;
}
#
# Fake up an object for a historical reservation entry.
#
sub LookupHistorical($$) {
my ($class, $uuid) = @_;
my $query_result =
DBQueryWarn("select *,UNIX_TIMESTAMP(start) AS s, " .
" UNIX_TIMESTAMP(end) AS e, ".
" UNIX_TIMESTAMP(created) AS c, " .
" UNIX_TIMESTAMP(canceled) AS d, " .
" UNIX_TIMESTAMP(deleted) AS k " .
" from reservation_history ".
"where uuid='$uuid'");
return undef
if (!defined($query_result) || !$query_result->numrows);
my $record = $query_result->fetchrow_hashref();
my $self = {};
$self->{'IDX'} = undef;
$self->{'PID'} = $record->{'pid'};
$self->{'PID_IDX'} = $record->{'pid_idx'};
$self->{'EID'} = undef;
$self->{'START'} = $record->{'s'};
$self->{'END'} = $record->{'e'};
$self->{'CREATED'} = $record->{'c'};
$self->{'DELETED'} = $record->{'k'};
$self->{'CANCEL'} = $record->{'d'};
$self->{'TYPE'} = $record->{'type'};
$self->{'NODES'} = $record->{'nodes'};
$self->{'UID'} = $record->{'uid'};
$self->{'UID_IDX'} = $record->{'uid_idx'};
$self->{'NOTES'} = $record->{'notes'};
$self->{'ADMIN_NOTES'} = $record->{'admin_notes'};
$self->{'APPROVED'} = undef;
$self->{'APPROVER'} = undef;
$self->{'UUID'} = $record->{'uuid'};
bless($self, $class);
return $self;
}
#
# Return a list of historical project reservations. Optional type.
# Type can be a single type or a list reference of types.
......@@ -1254,36 +1297,17 @@ sub HistoricalReservations($$$;$) {
push(@clauses, "pid='$pid'");
}
my $query_result =
DBQueryFatal("select *,UNIX_TIMESTAMP(start) AS s, " .
" UNIX_TIMESTAMP(end) AS e, ".
" UNIX_TIMESTAMP(created) AS c, " .
" UNIX_TIMESTAMP(canceled) AS d, " .
" UNIX_TIMESTAMP(deleted) AS k " .
" from reservation_history ".
"where " . join(" AND ", @clauses) . " " .
"order by start asc");
while (my $record = $query_result->fetchrow_hashref()) {
my $self = {};
$self->{'PID'} = $record->{'pid'};
$self->{'PID_IDX'} = $record->{'pid_idx'};
$self->{'EID'} = undef;
$self->{'START'} = $record->{'s'};
$self->{'END'} = $record->{'e'};
$self->{'CREATED'} = $record->{'c'};
$self->{'DELETED'} = $record->{'k'};
$self->{'CANCEL'} = $record->{'d'};
$self->{'TYPE'} = $record->{'type'};
$self->{'NODES'} = $record->{'nodes'};
$self->{'UID'} = $record->{'uid'};
$self->{'UID_IDX'} = $record->{'uid_idx'};
$self->{'NOTES'} = $record->{'notes'};
$self->{'ADMIN_NOTES'} = $record->{'admin_notes'};
$self->{'APPROVED'} = undef;
$self->{'APPROVER'} = undef;
$self->{'UUID'} = $record->{'uuid'};
bless($self, $class);
push(@answer, $self);
DBQueryWarn("select uuid from reservation_history ".
"where " . join(" AND ", @clauses) . " " .
"order by start asc");
return ()
if (!defined($query_result) || !$query_result->numrows);
while (my ($uuid) = $query_result->fetchrow_array()) {
my $record = Reservation->LookupHistorical($uuid);
push(@answer, $record)
if (defined($record));
}
return @answer;
}
......
......@@ -788,6 +788,8 @@ sub Purge($)
or return -1;
DBQueryWarn("delete from user_policies where uid='$uid_idx'")
or return -1;
DBQueryWarn("delete from user_token_passwords where uid_idx='$uid_idx'")
or return -1;
return 0;
}
......@@ -2317,6 +2319,187 @@ sub UpdateExports($)
return 0;
}
#
# A simple interface to the user_token_passwords table. This table
# holds non-login passwords for user access to various testbed
# subsystems, each associated with a user account for accounting
# purposes. If you need a non-user oauth-like token, that will need
# another table. When users login with these usernames/passwords, they
# should be given an oauth (or similar) token.
#
# Table fields:
# * uid_idx - users.idx
# * uid - users.uid
# * subsystem - a name that identifies the subsystem/function/token issuer
# * scope_type - the type of access being granted
# * scope_value - the value of access being granted
# * username - the login username provided to the authentication endpoint;
# need not be the username!
# * plaintext - the plaintext value of the password; only necessary if the
# user should be able to retrieve it through some secure interface
# * hash - the password hash to be checked at authentication
# * expiration - the expiration time of this username/password; either an
# int (a unix timestamp) or a string (a local mysql datetime).
# * token_lifetime - the lifetime of each token generated for this
# username/password
# * token_onetime - 1 if each token generated for this username/password
# should be a one-time-use token; 0 otherwise
# * system - 1 if this is a system-created username necessary for an
# infrastructure service (i.e. capturing a disk image); 0 otherwise.
#
# Obviously, not all subsystems/token issuers would require or be
# capable of using all these fields; but this should be a good set for
# supporting a variety of OAuth tokens.
#
# There is a UNIQUE KEY set:
# UNIQUE KEY `user_token` (`subsystem`,`username`,`plaintext`)
# We want to allow a user to have multiple, same-named passwords per
# service; but they ought to have a different plaintext value. If we
# included hash in the key, we could not the latter restriction due to
# the random bits in the salt; thus we opt to only allow one empty
# password per username. This should be a fine compromise.
#
#
# Add a username/password, associated with an existing user account, to
# be used for token generation during authn/authz (i.e., OAuth):
#
# $user->AddTokenPassword($target_user,$subsystem,$scope_type,$scope_value,
# $username,$plaintext,$hash;$expiration,$token_lifetime,$token_onetime,
# $system)
#
# where subsystem must be set; scope_type and scope_value default to ''
# if undef; username defaults to this_user or target_user if unset. One
# of plaintext or hash must be set; if both are set,
# crypt(plaintext,hash) == hash must hold. plaintext defaults to '' if
# undef. expiration should be set by the caller if necessary for that
# subsystem; it defaults to NULL (never expires); token_lifetime, if
# unset, will be set to a default value by the subsystem token issuer;
# token_onetime defaults to 0 (i.e., token will be multi-user) if not
# set; system defaults to 0 if not set.
#
# This is a library function, never called based on direct input from
# the user; therefore we do not check the password strength! We are
# zealous in escaping input ags anyway.
#
sub AddTokenPassword($$$$$$$$$;$$$$)
{
my ($class, $this_user, $target_user, $subsystem, $scope_type, $scope_value,
$username, $plaintext, $hash,
$expiration, $token_lifetime, $token_onetime, $system) = @_;
my $this_uid = $this_user->uid();
my $isadmin = $this_user->IsAdmin();
my $target_uid = $target_user->uid();
my $target_uid_idx = $target_user->uid_idx();
my $status = $target_user->status();
return 1
if (!defined($subsystem) || !$subsystem);
return 1
if ((!defined($plaintext) || !$plaintext)
&& (!defined($hash) || !$hash));
return 1
if (defined($plaintext) && $plaintext ne ""
&& defined($hash) && $hash ne ""
&& crypt($plaintext,$hash) ne $hash);
if (!defined($username) || !$username) {
$username = $target_uid;
}
$scope_type = ""
if (!defined($scope_type));
$scope_value = ""
if (!defined($scope_value));
if (!defined($hash) || $hash eq "") {
#$rs = join("",(".","/",0..9,"A".."Z","a".."z")[map { rand(64) } (0..15)]);
#$hash = crypt($plaintext,"\$5\$$rs\$");
$hash = PassWordHash($plaintext);
return 2
if (!defined($hash) || $hash eq '');
}
my $qpt = DBQuoteSpecial($plaintext);
my $qpw = DBQuoteSpecial($hash);
my $exp = "NULL";
if (defined($expiration) && $expiration ne "") {
if ($expiration =~ /^\d+$/) {
$exp = "DATE_ADD(NOW(), INTERVAL $expiration SECOND)";
}
else {
$exp = DBQuoteSpecial($expiration);
}
}
my ($lt,$os,$sys) = (3600,0,0);
$lt = $token_lifetime
if (defined($token_lifetime) && $token_lifetime ne "");
$os = 1
if (defined($token_onetime) && $token_onetime);
$sys = 1
if (defined($system) && $system);
my $q = "insert into user_token_passwords".
" (uid_idx,uid,subsystem,scope_type,scope_value,username,plaintext,".
" hash,issued,expiration,token_lifetime,token_onetime) values".
" ('$target_uid_idx','$target_uid','$subsystem','$scope_type',".
" '$scope_value','$username',$qpt,$qpw,NOW(),$exp,$lt,$os)";
return 1
if (!DBQueryWarn($q));
return 0;
}
sub DeleteTokenPasswords($$$$$;$)
{
my ($class, $this_user, $target_user, $subsystem, $username,
$plaintext) = @_;
my $this_uid = $this_user->uid();
my $isadmin = $this_user->IsAdmin();
my $target_uid = $target_user->uid();
my $target_uid_idx = $target_user->uid_idx();
my $status = $target_user->status();
return 1
if (!defined($subsystem) || !$subsystem);
if (!defined($username) || !$username) {
$username = $target_uid;
}
my $q = "delete from user_token_passwords where".
" uid_idx='$target_uid_idx' and subsystem='$subsystem'".
" and username='$username'";
if (defined($plaintext)) {
$q .= " and plaintext=" . DBQuoteSpecial($plaintext);
}
return 1
if (!DBQueryWarn($q));
return 0;
}
sub DeleteTokenPasswordByIdx($$$$)
{
my ($class, $this_user, $target_user, $idx) = @_;
my $this_uid = $this_user->uid();
my $isadmin = $this_user->IsAdmin();
my $target_uid = $target_user->uid();
my $target_uid_idx = $target_user->uid_idx();
return 1
if (!defined($idx) || !$idx);
my $q = "delete from user_token_passwords where".
" uid_idx='$target_uid_idx' and idx=".DBQuoteSpecial($idx);
return 1
if (!DBQueryWarn($q));
return 0;
}
# _Always_ make sure that this 1 is at the end of the file...
1;
......@@ -1296,19 +1296,26 @@ sub ActionStart($$;$)
}
$sliver = undef;
#
# os_setup requires the expstate to be set appropriately, which we
# generally do not do on the geni path.
#
$experiment->SetState(EXPTSTATE_ACTIVATING());
#
# So we are going to fork and let os_setup proceed.
#
my $childpid = main::WrapperFork();
if ($childpid) {
#
# This indicates that while not locked, we are still busy.
# KillMonitor operates using the cancel flag, so we should
# be fine.
#
$slice->SetMonitorPid($childpid);
print STDERR "Monitor PID $childpid\n";
return 0;
}
#
# os_setup requires the expstate to be set appropriately, which we
# generally do not do on the geni path.
#
$experiment->SetState(EXPTSTATE_ACTIVATING());
#
# We want to let snmpit run in parallel with os_setup, like the
# classic path does.
......@@ -1379,7 +1386,6 @@ sub ActionStart($$;$)
$rval = system("$OSSETUP $pid $eid @nodes");
print STDERR "os_setup exited with status $rval\n";
}
$experiment->SetState($expstate);
#
# See what nodes succeeded or failed. We want to hold off setting
......@@ -1544,6 +1550,8 @@ sub ActionStart($$;$)
# ready or failed state until we are fully done here.
$self->SetStatus("mixed");
$self->ComputeState();
$experiment->SetState($expstate);
$slice->ClearMonitorPid();
return 0;
bad:
......@@ -1562,6 +1570,8 @@ sub ActionStart($$;$)
$self->SetStatus("mixed");
$self->ComputeState();
}
$experiment->SetState($expstate);
$slice->ClearMonitorPid();
return -1;
}
......
......@@ -1175,6 +1175,7 @@ sub GetTicketAuxAux($)
my $isbridge = 0;
my $isfirewall = 0;
my $xensettings;
my $dockersettings;
my $fwsettings;
# Always populate iface2node mapping, even if we let the node
......@@ -1410,6 +1411,50 @@ sub GetTicketAuxAux($)
$fwsettings = GeniXML::GetFirewallSettings($ref);
}
}
elsif ($virtualization_subtype eq "emulab-docker") {
# Allow caller to set the image to use, but also
# trick to set the parent.
if (defined($osinfo)) {
if (! $osinfo->IsSubOS()) {
$parent_osname = $osname;
$osname = "DOCKER-EXT";
}
}
elsif (!defined($osname)) {
# Allow for Docker image url via node attribute.
$osname = "DOCKER-EXT";
}
# If not set, we will pick up the default_image below.
#
# Look for the knobs
#
if (GeniXML::HasDockerSettings($ref)) {
$dockersettings = GeniXML::GetDockerSettings($ref);
}
# If they want an external image (or we
# defaulted to one), but one isn't set, set a
# good default.
if ($osname eq 'DOCKER-EXT') {
if (!defined($dockersettings)) {
$dockersettings = {};
}
if (!exists($dockersettings->{'extimage'})
&& !exists($dockersettings->{'dockerfile'})) {
$dockersettings->{'extimage'} = "ubuntu:16.04";
}