From 66366489a107b758f4e7e5cf2fc4dc8b487e95cc Mon Sep 17 00:00:00 2001
From: "David M. Johnson" <johnsond@flux.utah.edu>
Date: Mon, 4 Jun 2018 17:19:15 -0600
Subject: [PATCH] Docker server-side core, esp new libimageops support for
Docker images.
The docker VM server-side goo is mostly identical to Xen, with slightly
different handling for parent images. We also support loading external
Docker images (i.e. those without a real imageid in our DB; in that
case, user has to set a specific stub image, and some extra per-vnode
metadata (a URI that points to a Docker registry/image repo/tag);
the Docker clientside handles the rest.
Emulab Docker images map to a Emulab imageid:version pretty seamlessly.
For instance, the Emulab `emulab-ops/docker-foo-bar:1` image would map
to `<local-registry-URI>/emulab-ops/emulab-ops/docker-foo-bar:1`; the
mapping is `<local-registry-URI>/pid/gid/imagename:version`. Docker
repository names are lowercase-only, so we handle that for the user; but
I would prefer that users use lowercase Emulab imagenames for all Docker
images; that will help us. That is not enforced in the code; it will
appear in the documentation, and we'll see.
Full Docker imaging relies on several other libraries
(https://gitlab.flux.utah.edu/emulab/pydockerauth,
https://gitlab.flux.utah.edu/emulab/docker-registry-py). Each
Emulab-based cluster must currently run its own private registry to
support image loading/capture (note however that if capture is
unnecessary, users can use the external images path instead). The
pydockerauth library is a JWT token server that runs out of boss's
Apache and implements authn/authz for the per-Emulab Docker registry
(probably running on ops, but could be anywhere) that stores images and
arbitrates upload/download access. For instance, nodes in an experiment
securely pull images using their pid/eid eventkey; and the pydockerauth
emulab authz module knows what images the node is allowed to pull
(i.e. sched_reloads, the current image the node is running, etc). Real
users can also pull images via user/pass, or bogus user/pass + Emulab
SSL cert. GENI credential-based authn/z was way too much work, sadly.
There are other auth/z paths (i.e. for admins, temp tokens for secure
operations) as well.
As far as Docker image distribution in the federation, we use the same
model as for regular ndz images. Remote images are pulled in to the
local cluster's Docker registry on-demand from their source cluster via
admin token auth (note that all clusters in the federation have
read-only access to the entire registries of any other cluster in the
federation, so they can pull images). Emulab imageid handling is the
same as the existing ndz case. For instance, image versions are lazily
imported, on-demand; local version numbers may not match the remote
image source cluster's version numbers. This will potentially be a
bigger problem in the Docker universe; Docker users expect to be able to
reference any image version at any time anywhere. But that is of course
handleable with some ex post facto synchronization flag day, at least
for the Docker images.
The big new thing supporting native Docker image usage is the guts of a
refactor of the utils/image* scripts into a new library, libimageops;
this is necessary to support Docker images, which are stored in their
own registry using their own custom protocols, so not amenable to our
file-based storage. Note: the utils/image* scripts currently call out
to libimageops *only if* the image format is docker; all other images
continue on the old paths in utils/image*, which all still remain
intact, or minorly-changed to support libimageops.
libimageops->New is the factory-style mechanism to get a libimageops
that works for your image format or node type. Once you have a
libimageops instance, you can invoke normal image logical operations
(CreateImage, ImageValidate, ImageRelease, et al). I didn't do every
single operation (for instance, I haven't yet dealt with image_import
beyond essentially generalizing DownLoadImage by image format).
Finally, each libimageops is stateless; another design would have been
some statefulness for more complicated operations. You will see that
CreateImage, for instance, is written in a helper-subclass style that
blurs some statefulness; however, it was the best match for the existing
body of code. We can revisit that later if the current argument-passing
convention isn't loved.
There are a couple outstanding issues. Part of the security model here
is that some utils/image* scripts are setuid, so direct libimageops
library calls are not possible from a non-setuid context for some
operations. This is non-trivial to resolve, and might not be worthwhile
to resolve any time soon. Also, some of the scripts write meaningful,
traditional content to stdout/stderr, and this creates a tension for
direct library calls that is not entirely resolved yet. Not hard, just
only partly resolved.
Note that tbsetup/libimageops_ndz.pm.in is still incomplete; it needs
imagevalidate support. Thus, I have not even featurized this yet; I
will get to that as I have cycles.
---
apt/APT_Instance.pm.in | 3 +-
apt/APT_Profile.pm.in | 5 +-
apt/APT_Rspec.pm.in | 20 +-
apt/rspec2genilib.in | 41 +-
backend/newimageid.in | 26 +-
backend/newimageid_ez.in | 1 +
db/Image.pm.in | 76 +-
db/OSImage.pm.in | 27 +-
db/OSinfo.pm.in | 1 +
db/User.pm.in | 183 +++
protogeni/lib/GeniCM.pm.in | 162 ++-
protogeni/lib/GeniXML.pm.in | 76 ++
protogeni/scripts/postimagedata.in | 7 +-
tbsetup/GNUmakefile.in | 4 +-
tbsetup/libimageops.pm.in | 1468 ++++++++++++++++++++++
tbsetup/libimageops_docker.pm.in | 1347 ++++++++++++++++++++
tbsetup/libimageops_ec2.pm.in | 126 ++
tbsetup/libimageops_ndz.pm.in | 1843 ++++++++++++++++++++++++++++
tbsetup/libosload_new.pm.in | 211 +++-
tbsetup/libvtop_stable.pm.in | 11 +-
tbsetup/ptopgen.in | 15 +
utils/clone_image.in | 3 +
utils/create_image.in | 82 +-
utils/delete_image.in | 35 +-
utils/dumpdescriptor.in | 1 +
utils/image_import.in | 59 +-
utils/imagehash.in | 68 +-
utils/imagerelease.in | 31 +-
utils/imagevalidate.in | 26 +
29 files changed, 5868 insertions(+), 90 deletions(-)
create mode 100644 tbsetup/libimageops.pm.in
create mode 100644 tbsetup/libimageops_docker.pm.in
create mode 100644 tbsetup/libimageops_ec2.pm.in
create mode 100644 tbsetup/libimageops_ndz.pm.in
diff --git a/apt/APT_Instance.pm.in b/apt/APT_Instance.pm.in
index 3780c9763c..6eff54d18c 100644
--- a/apt/APT_Instance.pm.in
+++ b/apt/APT_Instance.pm.in
@@ -811,7 +811,8 @@ sub ComputeNodeCounts($)
if (defined($virtualization_type) &&
($virtualization_type eq "emulab-xen" ||
- $virtualization_type eq "emulab-blockstore")) {
+ $virtualization_type eq "emulab-blockstore" ||
+ $virtualization_type eq "emulab-docker")) {
$vcount++;
next;
}
diff --git a/apt/APT_Profile.pm.in b/apt/APT_Profile.pm.in
index 696fcc4630..8f01274a7f 100755
--- a/apt/APT_Profile.pm.in
+++ b/apt/APT_Profile.pm.in
@@ -860,8 +860,9 @@ sub CheckFirewall($$)
# and closed.
#
my $style = "closed";
- if (defined($virtualization_type) &&
- $virtualization_type eq "emulab-xen" && !@routable_control_ip) {
+ if (defined($virtualization_type) && !@routable_control_ip &&
+ ($virtualization_type eq "emulab-xen"
+ || $virtualization_type eq "emulab-docker")) {
$style = "basic";
}
diff --git a/apt/APT_Rspec.pm.in b/apt/APT_Rspec.pm.in
index 5430cb891f..410703aa50 100644
--- a/apt/APT_Rspec.pm.in
+++ b/apt/APT_Rspec.pm.in
@@ -1,6 +1,6 @@
#!/usr/bin/perl -wT
#
-# Copyright (c) 2007-2017 University of Utah and the Flux Group.
+# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
@@ -831,6 +831,8 @@ sub new($$$$$)
"jacks_site" => undef,
"xen_settings" => undef,
"xen_ptype" => undef,
+ "docker_settings" => undef,
+ "docker_ptype" => undef,
"instantiate_on" => undef,
"services" => [],
"statements" => [],
@@ -1101,6 +1103,18 @@ sub addNode($$$)
$node->{"xen_settings"} = $settings;
last SWITCH;
};
+ # Docker settings
+ /^docker$/i && do {
+ my $settings = GeniXML::GetDockerSettings($ref->parentNode);
+ fatal("Failed to get docker settings")
+ if (!defined($settings));
+ $node->{"docker_settings"} = $settings;
+ last SWITCH;
+ };
+ /^docker_ptype$/i && do {
+ $node->{"docker_ptype"} = GetTextOrFail("name", $child);
+ last SWITCH;
+ };
/^(sliver_type_shaping|firewall_config)$/i && do {
# We handled this above.
last SWITCH;
@@ -1291,7 +1305,7 @@ sub Compare($$)
last SWITCH;
};
(/^(component_id|component_manager_id|disk_image)$/i ||
- /^(hardware_type|jacks_site|xen_ptype|instantiate_on)$/i ||
+ /^(hardware_type|jacks_site|xen_ptype|docker_ptype|instantiate_on)$/i ||
/^(adb_target|failure_action)$/i ||
/^(isexptfirewall|firewall_style)$/i ||
/^(use_type_default_image|routable_control_ip)$/i) && do {
@@ -1304,7 +1318,7 @@ sub Compare($$)
# Handled up above in CompareNodes.
last SWITCH;
};
- /^(xen_settings|desires|pipes|blockstores|attributes)$/i && do {
+ /^(xen_settings|docker_settings|desires|pipes|blockstores|attributes)$/i && do {
return 1
if (APT_Rspec::CompareHashes("Node: $client_id: $key",
$val1, $val2));
diff --git a/apt/rspec2genilib.in b/apt/rspec2genilib.in
index 4085e53ca5..156be3c832 100644
--- a/apt/rspec2genilib.in
+++ b/apt/rspec2genilib.in
@@ -1,7 +1,7 @@
#!/usr/bin/perl -w
#
-# Copyright (c) 2000-2017 University of Utah and the Flux Group.
+# Copyright (c) 2000-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
@@ -201,6 +201,45 @@ sub GenerateNodeStatements($)
$node->addTagStatement("InstantiateOn('$vhost')");
}
}
+ elsif ($ntype eq "emulab-docker") {
+ $node->addStatement("$ntag = request.DockerContainer('$client_id')");
+
+ #
+ # This is the only time we need to spit this out, since
+ # the default is False.
+ #
+ if (defined($node->{'exclusive'}) && $node->{'exclusive'}) {
+ $node->addTagStatement("exclusive = True");
+ }
+
+ if (defined($node->{'docker_settings'})) {
+ my $settings = $node->{'docker_settings'};
+
+ foreach my $setting (sort(keys(%{$settings}))) {
+ my $value = $settings->{$setting};
+ if ($setting eq "ram") {
+ $node->addTagStatement("ram = $value");
+ }
+ elsif ($setting eq "cores") {
+ $node->addTagStatement("cores = $value");
+ }
+ elsif ($setting eq "extimage") {
+ $node->addTagStatement("docker_extimage = $value");
+ }
+ elsif ($setting eq "dockerfile") {
+ $node->addTagStatement("docker_dockerfile = $value");
+ }
+ }
+ }
+ if (defined($node->{'docker_ptype'})) {
+ my $ptype = $node->{'docker_ptype'};
+ $node->addTagStatement("docker_ptype = '$ptype'");
+ }
+ if (defined($node->{'instantiate_on'})) {
+ my $vhost = $node->{'instantiate_on'};
+ $node->addTagStatement("InstantiateOn('$vhost')");
+ }
+ }
elsif ($ntype eq "delay") {
#
# Bridges are also special, see comment above for blockstore.
diff --git a/backend/newimageid.in b/backend/newimageid.in
index a7d79c18ef..6b70312137 100644
--- a/backend/newimageid.in
+++ b/backend/newimageid.in
@@ -174,6 +174,7 @@ my %xmlfields =
"global", => ["global", $SLOT_ADMINONLY, 0],
"mbr_version", => ["mbr_version", $SLOT_OPTIONAL],
"makedefault", => ["makedefault", $SLOT_ADMINONLY, 0],
+ "format", => ["format", $SLOT_ADMINONLY, "ndz"],
);
#
@@ -490,14 +491,27 @@ elsif (! $isadmin) {
UserError("Path: Invalid Path");
}
}
-if ($newimageid_args{"path"} =~ /\/$/) {
- if (-e $newimageid_args{"path"} && ! -d $newimageid_args{"path"}) {
- UserError("Path: invalid path, it should be a directory");
- }
+if (defined($newimageid_args{"format"})
+ && $newimageid_args{"format"} eq "docker") {
+ #
+ # We only allow a specific path for docker images, since the storage
+ # backend relies on an ACL of repos a user can access. It's not a
+ # filesystem with UNIX permissions. With a docker registry,
+ # permissions are tied to specific paths. Don't even let admins
+ # override this for now; there is no point.
+ #
}
-elsif (-d $newimageid_args{"path"} =~ /\/$/) {
- UserError("Path: invalid path, its a directory");
+else {
+ if ($newimageid_args{"path"} =~ /\/$/) {
+ if (-e $newimageid_args{"path"} && ! -d $newimageid_args{"path"}) {
+ UserError("Path: invalid path, it should be a directory");
+ }
+ }
+ elsif (-d $newimageid_args{"path"} =~ /\/$/) {
+ UserError("Path: invalid path, its a directory");
+ }
}
+
#
# See what node types this image will work on. Must be at least one!
#
diff --git a/backend/newimageid_ez.in b/backend/newimageid_ez.in
index 2e85c5421f..b37c5fee6a 100644
--- a/backend/newimageid_ez.in
+++ b/backend/newimageid_ez.in
@@ -210,6 +210,7 @@ my %xmlfields =
"global", => ["global", $SLOT_OPTIONAL, 0],
"noexport", => ["noexport", $SLOT_OPTIONAL, 0],
"mbr_version", => ["mbr_version", $SLOT_OPTIONAL],
+ "format", => ["format", $SLOT_OPTIONAL],
"metadata_url", => ["metadata_url", $SLOT_ADMINONLY],
"imagefile_url", => ["imagefile_url", $SLOT_ADMINONLY],
"origin_uuid", => ["origin_uuid", $SLOT_ADMINONLY],
diff --git a/db/Image.pm.in b/db/Image.pm.in
index f004daaa22..a9954e9fc4 100644
--- a/db/Image.pm.in
+++ b/db/Image.pm.in
@@ -1,6 +1,6 @@
#!/usr/bin/perl -wT
#
-# Copyright (c) 2007-2017 University of Utah and the Flux Group.
+# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
@@ -638,7 +638,7 @@ sub Create($$$$$$$$)
}
# Pass-through optional slots, otherwise the DB default is used.
foreach my $key ("path", "shared", "global", "ezid", "mbr_version",
- "metadata_url", "imagefile_url", "released",
+ "format", "metadata_url", "imagefile_url", "released",
"isdataset", "lba_size", "lba_low", "lba_high",
"origin_uuid", "origin_urn", "origin_name") {
if (exists($argref->{$key})) {
@@ -1795,6 +1795,31 @@ sub GetDiskOffset($)
return $mbr{$self->mbr_version()}[$self->loadpart()];
}
+#
+# Return created time for image as a UNIX timestamp via the passed ref.
+# Return 0 on success, non-zero otherwise.
+#
+sub GetCreate($$)
+{
+ my ($self,$stampp) = @_;
+
+ # Must be a real reference.
+ return -1
+ if (! ref($self));
+
+ my $imageid = $self->imageid();
+ my $version = $self->version();
+
+ my $result =
+ DBQueryWarn("select UNIX_TIMESTAMP(created) from image_versions ".
+ "where imageid='$imageid' and version='$version'");
+ if ($result && $result->numrows) {
+ my ($stamp) = $result->fetchrow_array();
+ $$stampp = $stamp;
+ return 0;
+ }
+}
+
#
# Return updated time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
@@ -2714,6 +2739,46 @@ sub SetImageMetadataURL($$)
return 0;
}
+sub SetImageFileURL($$)
+{
+ my ($self, $url) = @_;
+ $url = DBQuoteSpecial($url);
+
+ # Must be a real reference.
+ return -1
+ if (! ref($self));
+
+ my $imageid = $self->imageid();
+ my $version = $self->version();
+
+ DBQueryWarn("update image_versions set imagefile_url=$url ".
+ "where imageid='$imageid' and version='$version'")
+ or return -1;
+
+ $self->{'IMAGE'}->{'image_metadata_url'} = $url;
+ return 0;
+}
+
+sub SetFormat($$)
+{
+ my ($self, $format) = @_;
+
+ return -1
+ if ($self->Update({'format' => $format}));
+
+ return 0;
+}
+
+sub SetPath($$)
+{
+ my ($self, $path) = @_;
+
+ return -1
+ if ($self->Update({'path' => $path}));
+
+ return 0;
+}
+
sub LocalURL($)
{
my ($self) = @_;
@@ -2856,7 +2921,8 @@ sub IsDirPath($)
return 0;
}
return 1
- if ($self->path() =~ /\/$/);
+ if ($self->path() =~ /\/$/
+ && (!defined($self->format()) || $self->format() ne 'docker'));
return 0;
}
sub FullImagePath($)
@@ -2958,7 +3024,9 @@ sub IsSystemImage($)
my ($self) = @_;
# Yuck.
- return ($self->path() =~ /^\/usr\/testbed\// ? 1 : 0);
+ return ($self->path() =~ /^\/usr\/testbed\// ? 1 : 0
+ || ($self->format() eq 'docker' && $self->global()
+ && $self->path() =~ /emulab-ops\/emulab-ops/));
}
#
# When images are stored in sub directories.
diff --git a/db/OSImage.pm.in b/db/OSImage.pm.in
index 27e5c3951f..d6cbc7e1ab 100644
--- a/db/OSImage.pm.in
+++ b/db/OSImage.pm.in
@@ -1,6 +1,6 @@
#!/usr/bin/perl -wT
#
-# Copyright (c) 2007-2017 University of Utah and the Flux Group.
+# Copyright (c) 2007-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
@@ -662,6 +662,17 @@ sub Chunks($)
return $self->image()->Chunks();
}
+#
+# Return updated time for image as a UNIX timestamp via the passed ref.
+# Return 0 on success, non-zero otherwise.
+#
+sub GetCreate($$)
+{
+ my ($self,$stampp) = @_;
+
+ return $self->image()->GetCreate($stampp);
+}
+
#
# Return updated time for image as a UNIX timestamp via the passed ref.
# Return 0 on success, non-zero otherwise.
@@ -1052,6 +1063,20 @@ sub SetImageMetadataURL($$)
return $self->image()->SetImageMetadataURL($url);
}
+sub SetFormat($$)
+{
+ my ($self, $format) = @_;
+
+ return $self->image()->SetFormat($format);
+}
+
+sub SetPath($$)
+{
+ my ($self, $path) = @_;
+
+ return $self->image()->SetPath($path);
+}
+
sub LocalURL($)
{
my ($self) = @_;
diff --git a/db/OSinfo.pm.in b/db/OSinfo.pm.in
index 187100ac25..afec6e4e71 100644
--- a/db/OSinfo.pm.in
+++ b/db/OSinfo.pm.in
@@ -61,6 +61,7 @@ my %FEATURES = ( "ping" => 1,
"linktest" => 1,
"linkdelays" => 0,
"xen-host" => 0,
+ "docker-host"=> 0,
"suboses" => 0 );
# Valid OS names. Mirrored in the web interface. The value is a user-okay flag.
diff --git a/db/User.pm.in b/db/User.pm.in
index e0d1f74999..a84bce2a21 100644
--- a/db/User.pm.in
+++ b/db/User.pm.in
@@ -788,6 +788,8 @@ sub Purge($)
or return -1;
DBQueryWarn("delete from user_policies where uid='$uid_idx'")
or return -1;
+ DBQueryWarn("delete from user_token_passwords where uid_idx='$uid_idx'")
+ or return -1;
return 0;
}
@@ -2317,6 +2319,187 @@ sub UpdateExports($)
return 0;
}
+#
+# A simple interface to the user_token_passwords table. This table
+# holds non-login passwords for user access to various testbed
+# subsystems, each associated with a user account for accounting
+# purposes. If you need a non-user oauth-like token, that will need
+# another table. When users login with these usernames/passwords, they
+# should be given an oauth (or similar) token.
+#
+# Table fields:
+# * uid_idx - users.idx
+# * uid - users.uid
+# * subsystem - a name that identifies the subsystem/function/token issuer
+# * scope_type - the type of access being granted
+# * scope_value - the value of access being granted
+# * username - the login username provided to the authentication endpoint;
+# need not be the username!
+# * plaintext - the plaintext value of the password; only necessary if the
+# user should be able to retrieve it through some secure interface
+# * hash - the password hash to be checked at authentication
+# * expiration - the expiration time of this username/password; either an
+# int (a unix timestamp) or a string (a local mysql datetime).
+# * token_lifetime - the lifetime of each token generated for this
+# username/password
+# * token_onetime - 1 if each token generated for this username/password
+# should be a one-time-use token; 0 otherwise
+# * system - 1 if this is a system-created username necessary for an
+# infrastructure service (i.e. capturing a disk image); 0 otherwise.
+#
+# Obviously, not all subsystems/token issuers would require or be
+# capable of using all these fields; but this should be a good set for
+# supporting a variety of OAuth tokens.
+#
+# There is a UNIQUE KEY set:
+# UNIQUE KEY `user_token` (`subsystem`,`username`,`plaintext`)
+# We want to allow a user to have multiple, same-named passwords per
+# service; but they ought to have a different plaintext value. If we
+# included hash in the key, we could not the latter restriction due to
+# the random bits in the salt; thus we opt to only allow one empty
+# password per username. This should be a fine compromise.
+#
+
+#
+# Add a username/password, associated with an existing user account, to
+# be used for token generation during authn/authz (i.e., OAuth):
+#
+# $user->AddTokenPassword($target_user,$subsystem,$scope_type,$scope_value,
+# $username,$plaintext,$hash;$expiration,$token_lifetime,$token_onetime,
+# $system)
+#
+# where subsystem must be set; scope_type and scope_value default to ''
+# if undef; username defaults to this_user or target_user if unset. One
+# of plaintext or hash must be set; if both are set,
+# crypt(plaintext,hash) == hash must hold. plaintext defaults to '' if
+# undef. expiration should be set by the caller if necessary for that
+# subsystem; it defaults to NULL (never expires); token_lifetime, if
+# unset, will be set to a default value by the subsystem token issuer;
+# token_onetime defaults to 0 (i.e., token will be multi-user) if not
+# set; system defaults to 0 if not set.
+#
+# This is a library function, never called based on direct input from
+# the user; therefore we do not check the password strength! We are
+# zealous in escaping input ags anyway.
+#
+sub AddTokenPassword($$$$$$$$$;$$$$)
+{
+ my ($class, $this_user, $target_user, $subsystem, $scope_type, $scope_value,
+ $username, $plaintext, $hash,
+ $expiration, $token_lifetime, $token_onetime, $system) = @_;
+
+ my $this_uid = $this_user->uid();
+ my $isadmin = $this_user->IsAdmin();
+
+ my $target_uid = $target_user->uid();
+ my $target_uid_idx = $target_user->uid_idx();
+ my $status = $target_user->status();
+
+ return 1
+ if (!defined($subsystem) || !$subsystem);
+ return 1
+ if ((!defined($plaintext) || !$plaintext)
+ && (!defined($hash) || !$hash));
+ return 1
+ if (defined($plaintext) && $plaintext ne ""
+ && defined($hash) && $hash ne ""
+ && crypt($plaintext,$hash) ne $hash);
+
+ if (!defined($username) || !$username) {
+ $username = $target_uid;
+ }
+ $scope_type = ""
+ if (!defined($scope_type));
+ $scope_value = ""
+ if (!defined($scope_value));
+ if (!defined($hash) || $hash eq "") {
+ #$rs = join("",(".","/",0..9,"A".."Z","a".."z")[map { rand(64) } (0..15)]);
+ #$hash = crypt($plaintext,"\$5\$$rs\$");
+ $hash = PassWordHash($plaintext);
+ return 2
+ if (!defined($hash) || $hash eq '');
+ }
+
+ my $qpt = DBQuoteSpecial($plaintext);
+ my $qpw = DBQuoteSpecial($hash);
+ my $exp = "NULL";
+ if (defined($expiration) && $expiration ne "") {
+ if ($expiration =~ /^\d+$/) {
+ $exp = "DATE_ADD(NOW(), INTERVAL $expiration SECOND)";
+ }
+ else {
+ $exp = DBQuoteSpecial($expiration);
+ }
+ }
+ my ($lt,$os,$sys) = (3600,0,0);
+ $lt = $token_lifetime
+ if (defined($token_lifetime) && $token_lifetime ne "");
+ $os = 1
+ if (defined($token_onetime) && $token_onetime);
+ $sys = 1
+ if (defined($system) && $system);
+ my $q = "insert into user_token_passwords".
+ " (uid_idx,uid,subsystem,scope_type,scope_value,username,plaintext,".
+ " hash,issued,expiration,token_lifetime,token_onetime) values".
+ " ('$target_uid_idx','$target_uid','$subsystem','$scope_type',".
+ " '$scope_value','$username',$qpt,$qpw,NOW(),$exp,$lt,$os)";
+ return 1
+ if (!DBQueryWarn($q));
+
+ return 0;
+}
+
+sub DeleteTokenPasswords($$$$$;$)
+{
+ my ($class, $this_user, $target_user, $subsystem, $username,
+ $plaintext) = @_;
+
+ my $this_uid = $this_user->uid();
+ my $isadmin = $this_user->IsAdmin();
+
+ my $target_uid = $target_user->uid();
+ my $target_uid_idx = $target_user->uid_idx();
+ my $status = $target_user->status();
+
+ return 1
+ if (!defined($subsystem) || !$subsystem);
+ if (!defined($username) || !$username) {
+ $username = $target_uid;
+ }
+
+ my $q = "delete from user_token_passwords where".
+ " uid_idx='$target_uid_idx' and subsystem='$subsystem'".
+ " and username='$username'";
+ if (defined($plaintext)) {
+ $q .= " and plaintext=" . DBQuoteSpecial($plaintext);
+ }
+ return 1
+ if (!DBQueryWarn($q));
+
+ return 0;
+}
+
+sub DeleteTokenPasswordByIdx($$$$)
+{
+ my ($class, $this_user, $target_user, $idx) = @_;
+
+ my $this_uid = $this_user->uid();
+ my $isadmin = $this_user->IsAdmin();
+
+ my $target_uid = $target_user->uid();
+ my $target_uid_idx = $target_user->uid_idx();
+
+ return 1
+ if (!defined($idx) || !$idx);
+
+ my $q = "delete from user_token_passwords where".
+ " uid_idx='$target_uid_idx' and idx=".DBQuoteSpecial($idx);
+ return 1
+ if (!DBQueryWarn($q));
+
+ return 0;
+}
+
# _Always_ make sure that this 1 is at the end of the file...
1;
diff --git a/protogeni/lib/GeniCM.pm.in b/protogeni/lib/GeniCM.pm.in
index 60d805249e..6c4a1f6da3 100755
--- a/protogeni/lib/GeniCM.pm.in
+++ b/protogeni/lib/GeniCM.pm.in
@@ -1175,6 +1175,7 @@ sub GetTicketAuxAux($)
my $isbridge = 0;
my $isfirewall = 0;
my $xensettings;
+ my $dockersettings;
my $fwsettings;
# Always populate iface2node mapping, even if we let the node
@@ -1410,6 +1411,50 @@ sub GetTicketAuxAux($)
$fwsettings = GeniXML::GetFirewallSettings($ref);
}
}
+ elsif ($virtualization_subtype eq "emulab-docker") {
+ # Allow caller to set the image to use, but also
+ # trick to set the parent.
+ if (defined($osinfo)) {
+ if (! $osinfo->IsSubOS()) {
+ $parent_osname = $osname;
+ $osname = "DOCKER-EXT";
+ }
+ }
+ elsif (!defined($osname)) {
+ # Allow for Docker image url via node attribute.
+ $osname = "DOCKER-EXT";
+ }
+ # If not set, we will pick up the default_image below.
+ #
+ # Look for the knobs
+ #
+ if (GeniXML::HasDockerSettings($ref)) {
+ $dockersettings = GeniXML::GetDockerSettings($ref);
+ }
+ # If they want an external image (or we
+ # defaulted to one), but one isn't set, set a
+ # good default.
+ if ($osname eq 'DOCKER-EXT') {
+ if (!defined($dockersettings)) {
+ $dockersettings = {};
+ }
+ if (!exists($dockersettings->{'extimage'})
+ && !exists($dockersettings->{'dockerfile'})) {
+ $dockersettings->{'extimage'} = "ubuntu:16.04";
+ }
+ }
+ my $ptype = GeniXML::DockerPtype($ref);
+ $pctype = $ptype
+ if (defined($ptype));
+ $virtexperiment->encap_style("vlan");
+
+ #
+ # Per-vnode firewall options.
+ #
+ if (GeniXML::HasFirewallSettings($ref)) {
+ $fwsettings = GeniXML::GetFirewallSettings($ref);
+ }
+ }
elsif ($virtualization_subtype eq "emulab-spp") {
$osname = "SPPVM-FAKE";
$pctype = "sppvm";
@@ -1632,7 +1677,8 @@ sub GetTicketAuxAux($)
#
if (defined($instantiate_on)) {
if (defined($virtualization_subtype) &&
- $virtualization_subtype eq "emulab-xen") {
+ ($virtualization_subtype eq "emulab-xen" ||
+ $virtualization_subtype eq "emulab-docker")) {
if (!exists($allnodes{$instantiate_on})) {
$response =
@@ -1644,7 +1690,7 @@ sub GetTicketAuxAux($)
else {
$response =
GeniResponse->Create(GENIRESPONSE_BADARGS, undef,
- "Only Xen nodes can use 'instantiate_on'");
+ "Only Xen or Docker nodes can use 'instantiate_on'");
goto bad;
}
}
@@ -2090,6 +2136,111 @@ sub GetTicketAuxAux($)
"attrvalue" => $attrvalue });
}
}
+ if (defined($dockersettings)) {
+ foreach my $setting (keys(%$dockersettings)) {
+ my $attrvalue = $dockersettings->{$setting};
+ my $attrkey;
+ my $isshared = (!defined($exclusive) || !$exclusive ? 1 : 0);
+
+ if ($setting eq "ram") {
+ if ($attrvalue !~ /^\d*$/) {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker ram value; integers only");
+ goto bad;
+ }
+ if ($isshared && 0 && $attrvalue > 1024) {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker shared node ram value;".
+ " limited to 1024 MB");
+ goto bad;
+ }
+ $attrkey = "DOCKER_MEMSIZE";
+ }
+ elsif ($setting eq "cores") {
+ if ($attrvalue !~ /^\d*$/) {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker cores value; integers only");
+ goto bad;
+ }
+ if ($attrvalue <= 0) {
+ # ignore silly user.
+ next;
+ }
+ $attrkey = "DOCKER_CORES";
+ if ($isshared && $attrvalue > 4) {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker shared node cores value; ".
+ "limited to 1-4 cores");
+ goto bad;
+ }
+ }
+ elsif ($setting eq "extimage") {
+ $attrkey = "DOCKER_EXTIMAGE";
+ }
+ elsif ($setting eq "extusername") {
+ $attrkey = "DOCKER_EXTUSER";
+ }
+ elsif ($setting eq "extpassword") {
+ $attrkey = "DOCKER_EXTPASS";
+ }
+ elsif ($setting eq "dockerfile") {
+ $attrkey = "DOCKER_DOCKERFILE";
+ }
+ elsif ($setting eq "tbaugmentation") {
+ if ($attrvalue ne 'full' && $attrvalue ne 'buildenv'
+ && $attrvalue ne 'core' && $attrvalue ne 'basic'
+ && $attrvalue ne 'none') {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker tbaugmentation value; must be full,".
+ " buildenv, core, basic, or none");
+ goto bad;
+ }
+ $attrkey = "DOCKER_EMULABIZATION";
+ }
+ elsif ($setting eq "tbaugmentation_update") {
+ $attrkey = "DOCKER_EMULABIZATION_UPDATE";
+ }
+ elsif ($setting eq "ssh_style") {
+ if ($attrvalue ne 'direct' && $attrvalue ne 'exec') {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker ssh_style value; must be direct or exec");
+ goto bad;
+ }
+ $attrkey = "DOCKER_SSH_STYLE";
+ }
+ elsif ($setting eq "exec_shell") {
+ if ($attrvalue !~ /^[\/\w\d_\-]+$/) {
+ $response = GeniResponse->Create(
+ GENIRESPONSE_BADARGS, undef,
+ "Bad Docker exec_shell value; can only have".
+ " alphanumeric chars, _, -, or /");
+ goto bad;
+ }
+ $attrkey = "DOCKER_EXEC_SHELL";
+ }
+ elsif ($setting eq "cmd") {
+ $attrkey = "DOCKER_CMD";
+ #$attrvalue = DBQuoteSpecial($attrvalue);
+ }
+ elsif ($setting eq "env") {
+ $attrkey = "DOCKER_ENV";
+ #$attrvalue = DBQuoteSpecial($attrvalue);
+ }
+ else {
+ next;
+ }
+ $virtexperiment->NewTableRow("virt_node_attributes",
+ {"vname" => $node_nickname,
+ "attrkey" => $attrkey,
+ "attrvalue" => $attrvalue });
+ }
+ }
if (defined($fwsettings)) {
if (exists($fwsettings->{'style'})) {
$virtnode->firewall_style($fwsettings->{'style'});
@@ -3455,7 +3606,7 @@ sub GetTicketAuxAux($)
my $exclusive = (defined($node->sharing_mode()) ? "0" : "1");
#
- # Watch for dedicated XEN vhost nodes. We add those to the manifest
+ # Watch for dedicated XEN/Docker vhost nodes. We add those to the manifest
# later, and as real sliver objects when the ticket is redeemed.
#
if ($node->isvirtnode() && $exclusive && !defined($instantiate_on)) {
@@ -4658,6 +4809,11 @@ sub SliverWorkAux($)
if (defined($osinfo) && $osinfo->FeatureSupported("xen-host")){
$node->NoNFSMounts();
}
+ if (defined($osinfo)
+ && ($osinfo->FeatureSupported("docker-host"))) {
+ #|| $osinfo->osname() =~ /dock/i)) {
+ $node->NoNFSMounts();
+ }
}
}
diff --git a/protogeni/lib/GeniXML.pm.in b/protogeni/lib/GeniXML.pm.in
index 85e3cdbace..873522af0f 100755
--- a/protogeni/lib/GeniXML.pm.in
+++ b/protogeni/lib/GeniXML.pm.in
@@ -1048,6 +1048,19 @@ sub HasXenSettings($)
return 0;
}
+sub HasDockerSettings($)
+{
+ my ($node) = @_;
+
+ my $type = FindFirst("n:sliver_type", $node);
+ if (defined($type)) {
+ my $settings = FindNodesNS("n:docker", $type, $EMULAB_NS)->pop();
+ return 1
+ if (defined($settings));
+ }
+ return 0;
+}
+
sub UseTypeDefaultImage($)
{
my ($node) = @_;
@@ -1077,6 +1090,22 @@ sub XenPtype($)
return undef;
}
+sub DockerPtype($)
+{
+ my ($node) = @_;
+
+ my $type = FindFirst("n:sliver_type", $node);
+ if (defined($type)) {
+ my $ptype = FindNodesNS("n:docker_ptype", $type, $EMULAB_NS)->pop();
+ if (defined($ptype)) {
+ my $name = GetText("name", $ptype);
+ return $name
+ if (defined($name) && $name ne "");
+ }
+ }
+ return undef;
+}
+
sub MultiplexFactor($)
{
my ($rspec) = @_;
@@ -1178,6 +1207,53 @@ sub GetXenSettings($)
return $result;
}
+sub GetDockerSettings($)
+{
+ my ($node) = @_;
+ my $result = {};
+
+ my $type = FindFirst("n:sliver_type", $node);
+ return undef
+ if (!defined($type));
+
+ my $settings = FindNodesNS("n:docker", $type, $EMULAB_NS)->pop();
+ return undef
+ if (!defined($settings));
+
+ my $tmp = GetText("cores", $settings);
+ $result->{"cores"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("ram", $settings);
+ $result->{"ram"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("extimage", $settings);
+ $result->{"extimage"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("dockerfile", $settings);
+ $result->{"dockerfile"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("tbaugmentation", $settings);
+ $result->{"tbaugmentation"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("tbaugmentation_update", $settings);
+ $result->{"tbaugmentation_update"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("ssh_style", $settings);
+ $result->{"ssh_style"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("exec_shell", $settings);
+ $result->{"exec_shell"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("cmd", $settings);
+ $result->{"cmd"} = $tmp
+ if (defined($tmp));
+ $tmp = GetText("env", $settings);
+ $result->{"env"} = $tmp
+ if (defined($tmp));
+
+ return $result;
+}
+
sub GetElabInElabSettings($)
{
my ($ref) = @_;
diff --git a/protogeni/scripts/postimagedata.in b/protogeni/scripts/postimagedata.in
index 2f2cc0dc1b..78b2f3c9ce 100644
--- a/protogeni/scripts/postimagedata.in
+++ b/protogeni/scripts/postimagedata.in
@@ -303,7 +303,12 @@ sub PostImageInfo($)
$arch = "aarch64";
}
if ($type eq "pcvm") {
- my $virtualization = "emulab-xen";
+ if ($image->format() eq "docker") {
+ $virtualization = "emulab-docker";
+ }
+ else {
+ $virtualization = "emulab-xen";
+ }
}
}
}
diff --git a/tbsetup/GNUmakefile.in b/tbsetup/GNUmakefile.in
index ff3066c824..a2e38e2354 100644
--- a/tbsetup/GNUmakefile.in
+++ b/tbsetup/GNUmakefile.in
@@ -118,7 +118,9 @@ LIB_STUFF = libtbsetup.pm exitonwarn.pm libtestbed.pm \
libvtop_stable.pm libvtop_test.pm \
libosload_dell_s3048.pm libosload_dell_s4048.pm \
libosload_mlnx_sn2410.pm libosload_dell.pm \
- libosload_hp5406.pm
+ libosload_hp5406.pm \
+ libimageops.pm libimageops_ndz.pm libimageops_docker.pm \
+ libimageops_ec2.pm
# These scripts installed setuid, with sudo.
diff --git a/tbsetup/libimageops.pm.in b/tbsetup/libimageops.pm.in
new file mode 100644
index 0000000000..5e1d7afa9d
--- /dev/null
+++ b/tbsetup/libimageops.pm.in
@@ -0,0 +1,1468 @@
+#!/usr/bin/perl -w
+#
+# Copyright (c) 2000-2018 University of Utah and the Flux Group.
+#
+# {{{EMULAB-LICENSE
+#
+# This file is part of the Emulab network testbed software.
+#
+# This file is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or (at
+# your option) any later version.
+#
+# This file is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+# License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this file. If not, see <http://www.gnu.org/licenses/>.
+#
+# }}}
+#
+
+#
+# A simple package that loads and instantiates image type-specific
+# modules so that callers can invoke operations on them. Doesn't
+# provide any functionality of its own.
+#
+package libimageops;
+
+use strict;
+use Exporter;
+use vars qw(@EXPORT $AUTOLOAD);
+use base qw( Exporter );
+
+@EXPORT = qw(Factory);
+
+use libEmulab;
+use libdb;
+use libtestbed;
+use libreboot;
+use libosload;
+use OSImage;
+use Node;
+use NodeType;
+use libtblog;
+use Logfile;
+use English;
+use Data::Dumper;
+use EmulabFeatures;
+
+my $debug = 1;
+
+sub setDebug($) {
+ $debug = $_[0];
+}
+
+# Break circular reference someplace to avoid exit errors.
+sub DESTROY {
+ my $self = shift;
+
+ foreach my $key (keys(%{ $self })) {
+ $self->{$key} = undef;
+ }
+}
+
+# To avoid writing out all the methods.
+sub AUTOLOAD {
+ my $self = shift;
+ my $type = ref($self) or die("$self is not an object");
+
+ my $name = $AUTOLOAD;
+ $name =~ s/.*://; # strip fully-qualified portion
+
+ if (@_) {
+ return $self->{'HASH'}->{$name} = shift;
+ }
+ elsif (exists($self->{'HASH'}->{$name})) {
+ return $self->{'HASH'}->{$name};
+ }
+ print STDERR "$self: tried to access unknown slot $name\n";
+ return undef;
+}
+
+
+#
+# The arguments to this function are void or an inline hash.
+#
+# We accept several keys that guide us to the correct backend:
+# "format" (one of ndz or docker; default ndz).
+# "node" a Node object that the image is loaded on, or will be, or
+# that we are capturing from.
+# "image" an Image object whose format field guides the search
+#
+# If both format and image are specified, the image must have the format
+# specified, else this will return undef. If format and/or
+# iamge->format() is undef or "", we will use the default "ndz" format
+# instead.
+#
+# If node is specified, we look first for packages of form
+# libimageops_<format>_<node_type> and libimageops_<format>_<node_class>.
+# If we don't find those, we fallback to libimageops_<format>. If node
+# is specified, either format or image must be specified (and
+# image->format() must not be undef or ""!).
+#
+# If no arguments are specified, or if format (or image->format())
+# is undef or '', we return our default -- libimageops_ndz.
+#
+sub Factory(;%) {
+ my (%args) = @_;
+
+ my @packages = ();
+ my ($format,$type,$class,$image,$node);
+ if (defined($args{"format"})) {
+ $format = $args{"format"};
+ }
+ if (defined($args{"image"})) {
+ $image = $args{"image"};
+ if (!ref($image)) {
+ if (exists($args{"imagepid"})) {
+ $image = OSImage->Lookup($args{"imagepid"},$image);
+ if (!defined($image)) {
+ $@ = "No such image descriptor $args{'image'} in project".
+ " $args{'imagepid'}";
+ return undef;
+ }
+ }
+ else {
+ my $tmp = OSImage->Lookup($image);
+ if (!defined($tmp)) {
+ $tmp = OSImage->LookupByName($image);
+ }
+ if (!defined($tmp)) {
+ $@ = "No such image descriptor $args{'image'}!";
+ return undef;
+ }
+ $image = $tmp;
+ }
+ }
+ elsif (!($image->isa("Image") || $image->isa("OSImage"))) {
+ $@ = "$image is not an Image nor OSImage!";
+ return undef;
+ }
+ if (defined($image->format()) && $image->format() ne "") {
+ if (defined($format) && $format ne $image->format()) {
+ $@ = "format $format and image->format are not the same!";
+ return undef;
+ }
+ $format = $image->format();
+ }
+ }
+ if (!defined($format)) {
+ $format = "ndz";
+ print "libimageops Factory: defaulting unspecified format to default ndz!\n"
+ if ($debug);
+ $format = "ndz";
+ }
+ if (defined($args{"node"})) {
+ $node = $args{"node"};
+ if (!ref($node)) {
+ $node = Node->Lookup($node);
+ if (!defined($node)) {
+ $@ = "Invalid node name $args{'node'}!";
+ return undef;
+ }
+ }
+ $type = $node->type();
+ $class = $node->class();
+ push(@packages,"libimageops_${format}_${type}");
+ push(@packages,"libimageops_${format}_${class}");
+ }
+ push(@packages,"libimageops_$format");
+
+ if (defined($image) && $args{"imageref"}) {
+ ${$args{"imageref"}} = $image;
+ }
+
+ print "libimageops Factory searching for ".join(",",@packages)."\n"
+ if ($debug);
+
+ my @msgs = ();
+ for my $packname (@packages) {
+ # First, try to simply instantiate on the assumption it's
+ # already loaded.
+ my $obj = eval { $packname->New(); };
+ if (!$@) {
+ print "libimageops Factory created $obj\n"
+ if ($debug);
+ return $obj;
+ }
+ # Otherwise, try to load it, then try again to reinstantiate.
+ eval "require $packname";
+ if ($@) {
+ if ($@ =~ /Can't locate libimageops_/) {
+ push(@msgs,"$packname module load failed: " . $@);
+ next;
+ }
+ else {
+ print STDERR "Error: failed to load existing $packname; aborting ($@)!\n";
+ return undef;
+ }
+ }
+ print "libimageops Factory loaded $packname, running New\n"
+ if ($debug);
+ $obj = eval { $packname->New(); };
+ if (!$@) {
+ print "libimageops Factory created libimageops $obj\n"
+ if ($debug);
+ return $obj;
+ }
+ else {
+ print STDERR "Error: failed to instantiate $packname after successful load; aborting ($@)!\n";
+ return undef;
+ }
+ }
+
+# # Not loaded?
+# if ($@) {
+# push(@msgs,"$packname module load failed: " . $@);
+# }
+# elsif (defined($newtype)) {
+# return $newtype;
+# }
+# else {
+# push(@msgs,"$packname module load succeeded but New failed: $@ $!");
+# }
+# }
+ print STDERR "Error: failed to load libimageops formats:\n ".
+ join("\n ",@msgs)."\n";
+
+ return undef;
+}
+
+#
+# Base class for the format-specific packages below.
+#
+package libimageops_base;
+use base qw(libimageops);
+
+use strict;
+use English;
+use vars qw($AUTOLOAD);
+use POSIX qw(setsid :sys_wait_h);
+use Errno qw(ENOSPC);
+use libEmulab;
+use libdb;
+use libtestbed;
+use libimageops;
+use libtblog;
+use Node;
+use English;
+use Data::Dumper;
+use overload ('""' => 'Stringify');
+
+#
+# Testbed Support libraries
+#
+use lib "@prefix@/lib";
+use libdb;
+use EmulabConstants;
+use libtestbed;
+use libadminmfs;
+use Experiment;
+use Node;
+use User;
+use OSImage;
+use Image; # Cause of datasets.
+use Logfile;
+use WebTask;
+use Project;
+use EmulabFeatures;
+
+#
+# Configure variables
+#
+my $TB = "@prefix@";
+my $TBOPS = "@TBOPSEMAIL@";
+my $TBLOGS = "@TBLOGSEMAIL@";
+my $CONTROL = "@USERNODE@";
+my $NONFS = @NOSHAREDFS@;
+my $PROJROOT = "@PROJROOT_DIR@";
+my $GROUPROOT = "@GROUPSROOT_DIR@";
+my $WITHPROVENANCE= @IMAGEPROVENANCE@;
+my $WITHDELTAS = @IMAGEDELTAS@;
+my $ISFS = ("@BOSSNODE_IP@" eq "@FSNODE_IP@") ? 1 : 0;
+
+#
+# Commands.
+#
+my $CHECKQUOTA = "$TB/sbin/checkquota";
+my $TRIGGERUPDATE = "$TB/sbin/protogeni/triggerimageupdate";
+
+sub New($)
+{
+ my ($class) = @_;
+
+ my $self = {};
+ $self->{'HASH'} = {};
+ $self->{'HASH'}->{'type'} = $class;
+
+ $self->{'HASH'}->{'debug'} = $debug;
+
+ #
+ # Image Creation Tuneables.
+ #
+ # $maxwait Max total wall clock time to allow for image collection.
+ # We abort after this length of time even if we are still
+ # actively making progress collecting the image.
+ # Empirically we have observed about 1.6MB/sec on a pc850
+ # for a Windows image (the slowest to create), so figuring
+ # 1.5MB/sec for a 6GB max image works out to around 72 minutes.
+ # This value comes from sitevar images/create/maxwait if set.
+ #
+ # $idlewait Max time to allow between periods of progress.
+ # This value ensures that if we get stuck and stop making
+ # progress, we don't have to wait the potentially very long
+ # time til the $maxwait time expires to notice and abort.
+ # This value comes from sitevar images/create/idlewait if set.
+ #
+ # $checkwait Time between progress checks (must be int div of $idlewait)
+ # Hardwired here, does not come from DB.
+ #
+ # $reportwait Time between progress reports (must be multiple of $checkwait)
+ # Hardwired here, does not come from DB.
+ #
+ # $maximagesize Max size in bytes of an image. Currently this is site-wide
+ # and comes from sitevar images/create/maxsize if set. It should
+ # probably be finer-grained (per-project? per-user?) than that.
+ #
+ my $maxwait = (72 * 60);
+ my $idlewait = ( 8 * 60);
+ my $reportwait = ( 2 * 60);
+ # Check more frequently for web updates, sez Leigh.
+ my $checkwait = 5;
+ my $maximagesize = (6 * 1024**3);
+
+ #
+ # Reset default values from site variables if they exist.
+ #
+ my $tmp;
+ if (TBGetSiteVar("images/create/maxwait", \$tmp)) {
+ $maxwait = $tmp * 60;
+ }
+ if (TBGetSiteVar("images/create/idlewait", \$tmp)) {
+ $idlewait = $tmp * 60;
+ }
+ if (TBGetSiteVar("images/create/maxsize", \$tmp)) {
+ $maximagesize = $tmp * 1024**3;
+ }
+ $idlewait = $maxwait
+ if ($maxwait < $idlewait);
+ $reportwait = $idlewait
+ if ($idlewait < $reportwait);
+ $checkwait = $reportwait
+ if ($reportwait < $checkwait);
+
+ $self->{'HASH'}->{'maxwait'} = $maxwait;
+ $self->{'HASH'}->{'idlewait'} = $idlewait;
+ $self->{'HASH'}->{'reportwait'} = $reportwait;
+ $self->{'HASH'}->{'checkwait'} = $checkwait;
+ $self->{'HASH'}->{'maximagesize'} = $maximagesize;
+
+ bless($self, $class);
+ return $self;
+}
+
+# Break circular reference someplace to avoid exit errors.
+sub DESTROY {
+ my $self = shift;
+
+ foreach my $key (keys(%{ $self })) {
+ $self->{$key} = undef;
+ }
+}
+
+sub AUTOLOAD {
+ my $self = shift;
+ my $type = ref($self) or die("$self is not an object\n");
+
+ my $name = $AUTOLOAD;
+ $name =~ s/.*://; # strip fully-qualified portion
+
+ if (@_) {
+ return $self->{'HASH'}->{$name} = shift;
+ }
+ elsif (exists($self->{'HASH'}->{$name})) {
+ return $self->{'HASH'}->{$name};
+ }
+ print STDERR "$self: tried to access unknown slot $name\n";
+ return undef;
+}
+
+sub CheckImageQuota($$$$) {
+ my ($self,$pid,$gid,$user) = @_;
+ my $rc = 0;
+ my $msg;
+
+ my $copt = ($pid eq $gid) ? "-p $pid" : "-g $pid/$gid";
+ if (system("$CHECKQUOTA $copt -m 3GB ".$user->uid()) != 0) {
+ $rc = ENOSPC;
+ $msg = "You are over your disk quota on $CONTROL, or there is less".
+ " than a minimum amount (3GB) of space. Please login and cleanup!";
+ }
+
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# Check progress of image creation by periodically checking the image size.
+#
+# Called every $checkwait seconds.
+# Reports progress every $reportwait seconds.
+# Gives up after $idlewait seconds without a size change.
+#
+# XXX: replace all the refs with $state->{...} XXX
+sub _CheckProgress($$)
+{
+ my ($state, $statusp) = @_;
+
+ my $node_id = $state->{'node_id'};
+ my $runticks = $state->{'runticks'};
+ my $checkwait = $state->{'checkwait'};
+ my $maxwait = $state->{'maxwait'};
+ my $reportwait = $state->{'reportwait'};
+ my $idlewait = $state->{'idlewait'};
+ my $idleticks = $state->{'idleticks'};
+ my $maxiidleticks = $state->{'maxiidleticks'};
+ my $webtask = $state->{'webtask'};
+ my $filename = $state->{'filename'};
+ my $lastsize = $state->{'lastsize'};
+ my $maximagesize = $state->{'maximagesize'};
+ my $result;
+
+ my $maxticks = int($maxwait / $checkwait);
+ my $reportticks = int($reportwait / $checkwait);
+ my $maxidleticks = int($idlewait / $checkwait);
+
+ if ($runticks == 0) {
+ print "$node_id: started image capture for '$filename', ".
+ "waiting up to " . int($maxwait/60) . " minutes total or ".
+ int($idlewait/60) . " minutes idle.\n";
+ }
+
+ #
+ # Command has finished for better or worse, record status and finish.
+ #
+ if (defined($statusp) && $statusp->{$node_id} ne "none") {
+ $result = $state->{'result'};
+ $state->{'result'} = $result;
+ print "$node_id: image capture has completed: status='$result'\n";
+ return 0;
+ }
+
+ #
+ # Has run too long
+ #
+ $runticks++;
+ if ($runticks >= $maxticks) {
+ $result = "timeout";
+ $state->{'result'} = $result;
+ print "$node_id: image capture has completed: timeout\n";
+ return 0;
+ }
+
+ #
+ # See if imagezip on the node is making progress. If not, we need to
+ # check the idle timer and timeout if we have taken too long.
+ #
+ # Also, check to see if the (somewhat arbitrary) maximum filesize has
+ # been exceeded.
+ #
+ my $cursize = (stat($filename))[7];
+ if (!defined($cursize)) {
+ #
+ # XXX avoid an ugly uninitialized value.
+ # This should not happen, since we created the file,
+ # but just in case, if the file doesn't exist set the size to 0.
+ # We will eventually timeout.
+ #
+ $cursize = 0;
+ }
+ if (defined($webtask)) {
+ $webtask->imagesize($cursize / 1024);
+ }
+ if ($cursize > $maximagesize) {
+ $result = "toobig";
+ print "$node_id: image capture has completed: image too big\n";
+ return 0;
+ }
+ if ($cursize == $lastsize) {
+ $idleticks++;
+ $state->{'idleticks'} = $idleticks;
+ if ($idleticks >= ($cursize > 0 ? $maxidleticks : $maxiidleticks)) {
+ $result = "timeout";
+ $state->{'result'} = $result;
+ print "$node_id: image capture has completed: idle timeout\n";
+ return 0;
+ }
+ } else {
+ $idleticks = $state->{'idleticks'} = 0;
+ }
+ $lastsize = $state->{'lastsize'} = $cursize;
+
+ if (($runticks % $reportticks) == 0) {
+ my $curtdiff = int($runticks * $checkwait / 60);
+ print "$node_id: still waiting ...".
+ " it has been ". $curtdiff ." minutes.".
+ " Current image size: $cursize bytes.\n";
+ }
+ return 1;
+}
+
+sub RunWithSSH($$$$;$) {
+ my ($self,$node_id,$state,$cmd,$outputfile) = @_;
+ my $stat = undef;
+
+ $cmd = "$TB/bin/sshtb -n -host $node_id $cmd";
+ if (defined($outputfile) && $outputfile ne '') {
+ $cmd .= " >$outputfile 2>&1";
+ }
+ print STDERR "About to: '$cmd' as uid $UID\n"
+ if ($debug);
+
+ my $mypid = fork();
+ if ($mypid < 0) {
+ return "setupfailed";
+ }
+
+ if ($mypid == 0) {
+ my $stat = 0;
+ if (system($cmd)) {
+ $stat = $?;
+ }
+ if ($stat & 127) {
+ # died with a signal, return the signal
+ POSIX::_exit($stat & 127);
+ }
+ POSIX::_exit($stat >> 8);
+ }
+
+ #
+ # Parent. Wait for ssh to finish, reporting periodic progress
+ # as TBAdminMfsRunCmd would do.
+ #
+ my $endtime = time() + $self->maxwait() + $self->checkwait();
+ while (1) {
+ print "waitpid: mypid $mypid\n";
+ my $kid = waitpid($mypid, WNOHANG);
+ # ssh finished
+ if ($kid == $mypid) {
+ $stat = $?;
+ if ($stat & 127) {
+ # died with a signal, return the signal
+ $stat = $stat & 127;
+ } else {
+ # else return the exit code
+ $stat = $stat >> 8;
+ }
+ last;
+ }
+
+ # huh?
+ if ($kid == -1) {
+ $stat = -1;
+ last;
+ }
+
+ # check on progress
+ if (defined($state) && !_CheckProgress($state, undef)) {
+ $stat = $state->{'result'};
+ last;
+ }
+
+ # wait for awhile
+ sleep($self->checkwait());
+ if (time() >= $endtime) {
+ $stat = "timeout";
+ last;
+ }
+ }
+
+ return $stat;
+}
+
+sub CreateImageValidate($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my ($imagename,$imageid,$imagepid);
+ my $msg;
+
+ my $this_user = $args->{'user'};
+ #
+ # Default the image search project to TBOPSPID() if unspecified.
+ #
+ if (defined($args->{'imagepid'})) {
+ $imagepid = $args->{'imagepid'};
+ }
+ else {
+ $imagepid = $args->{'imagepid'} = TBOPSPID();
+ }
+
+ #
+ # If the image is a ref (assume to an Image), use it. Else,
+ # grab the imageid description from the DB. We do a permission check, but
+ # mostly to avoid hard to track errors that would result if the user picked
+ # the wrong one (which is likely to happen no matter what I do).
+ #
+ if (!ref($image)) {
+ $image = Image->Lookup($imagepid, $image);
+ }
+
+ # Tailor our error on undef image to the dataset case, or not.
+ if (!defined($image)) {
+ if (defined($args->{'bsname'})) {
+ $msg = "Dataset $imagename does not exist";
+ }
+ else {
+ $msg = "No such image descriptor $imagename in project $imagepid!";
+ }
+ goto validationerr;
+ }
+ $imageid = $image->imageid();
+ $imagename = $image->imagename();
+
+ if (!$this_user->IsAdmin()
+ && !$image->AccessCheck($this_user, TB_IMAGEID_ACCESS)) {
+ $msg = "You do not have permission to use imageid $imageid!";
+ goto validationerr;
+ }
+
+ # If we have a blockstore name, the image must be a dataset.
+ if (defined($args->{'bsname'}) && !$image->isdataset()) {
+ $msg = "$image is not a dataset for $args->{'bsname'}";
+ goto validationerr;
+ }
+
+ # Must have a blockstore name if the image is marked as a dataset.
+ if ($image->isdataset() && !defined($args->{'bsname'})) {
+ $msg = "You must provide a blockstore name for this image!";
+ goto validationerr;
+ }
+
+ # Also set a pid field that represents the target's container
+ # project. If this is an image capture of a node, this field will
+ # be overridden by the pid of the containing experiment; see
+ # CreateImageValidateTarget(). If it is a capture of something
+ # else, we just use the image pid; thus the default here.
+ $args->{'pid'} = $args->{'imagepid'};
+
+ if (wantarray) {
+ return ($image,);
+ }
+ else {
+ return $image;
+ }
+
+ validationerr:
+ tbwarn("$self CreateImageValidate: $msg\n");
+ if (wantarray) {
+ return (undef,$msg);
+ }
+ else {
+ return undef;
+ }
+}
+
+sub CreateImageValidateTarget($$$$) {
+ my ($self,$image,$node,$args) = @_;
+ my $msg;
+
+ if (!ref($node)) {
+ $node = Node->Lookup($node);
+ if (!defined($node)) {
+ $msg = "Invalid node name $node!";
+ goto validationerr;
+ }
+ }
+
+ if (!$node->isa("Node")) {
+ $msg = "target ($node) is not a valid Node!";
+ goto validationerr;
+ }
+
+ # Save this Node for later use.
+ $args->{'node'} = $node;
+
+ # Check node and permission
+ if (!$node->AccessCheck($args->{'user'}, TB_NODEACCESS_LOADIMAGE)) {
+ $msg = "You do not have permission to create an image from $node";
+ goto validationerr;
+ }
+
+ if ($node->IsTainted()) {
+ $msg = "$node is tainted - image creation denied";
+ goto validationerr;
+ }
+
+ if ($node->isvirtnode()) {
+ #
+ # Need to know this is a xen-host to tailor method below.
+ #
+ my $pnode = Node->Lookup($node->phys_nodeid());
+ my $osimage = OSImage->Lookup($pnode->def_boot_osid());
+ if (!defined($osimage)) {
+ $msg = "Could not get OSImage for $pnode";
+ goto errout;
+ }
+ if ($osimage->FeatureSupported("xen-host")
+ && $image->mbr_version() == 99) {
+ $args->{'doprovenance'} = $args->{'delta'} = $args->{'signature'} = 0;
+ }
+ }
+
+ #
+ # We need the project id of the container experiment later.
+ #
+ my $experiment = $node->Reservation();
+ if (!defined($experiment)) {
+ $msg = "Could not map $node to its experiment object!";
+ goto validationerr;
+ }
+ $args->{'pid'} = $experiment->pid();
+
+ if (wantarray) {
+ return ($node,);
+ }
+ else {
+ return $node;
+ }
+
+ validationerr:
+ tbwarn("$self CreateImageValidateTarget: $msg\n");
+ if (wantarray) {
+ return (undef,$msg);
+ }
+ else {
+ return undef;
+ }
+}
+
+sub CreateImageValidateArgs($$$$) {
+ my ($self,$image,$target,$args) = @_;
+
+ if (wantarray) {
+ return (0,undef);
+ }
+ else {
+ return 0;
+ }
+}
+
+#
+# This function does *not* fail, unless it fails to create a Logfile.
+#
+# When it returns, if it is the parent, it will set $childpid_ref to the
+# pid of the child. If waitmode is enabled, $childstat_ref will be set
+# to the child's exit value; if not, it will be set to undef. If the
+# child returns, it does not bother set $childstat_ref at all, and only
+# sets $$childpid_ref to 0. Thus, the caller knows what to do.
+#
+sub BackgroundImageCapture($$$$$$) {
+ my ($self,$image,$node,$args,$childpid_ref,$childstat_ref) = @_;
+ my $rc = -1;
+ my $msg;
+
+ #
+ # Go to the background since this is going to take a while.
+ #
+ my $experiment = $node->Reservation();
+ my $logfile = Logfile->Create(
+ (defined($experiment) ? $experiment->gid_idx() : $image->gid_idx()));
+ if (!defined($logfile)) {
+ $msg = "Could not create a logfile";
+ goto out;
+ }
+ # Save it off, other functions might refer to it.
+ $args->{'logfile'} = $logfile;
+ # Mark it open since we are going to start using it right away.
+ $logfile->Open();
+ # Logfile becomes the current spew.
+ $image->SetLogFile($logfile);
+
+ if (my $childpid = TBBackGround($logfile->filename())) {
+ $$childpid_ref = $childpid;
+
+ #
+ # Parent exits normally, except if in waitmode.
+ #
+ if (!$args->{'waitmode'}) {
+ print("Your image from $node is being created\n".
+ "You will be notified via email when the image has been\n".
+ "completed, and you can load the image on another node.\n");
+ $rc = 0;
+ $$childstat_ref = undef;
+ goto out;
+ }
+ # XXX fix, only if interactive.
+ print("Waiting for image creation to complete\n");
+ print("You may type ^C at anytime; you will be notified via email;\n".
+ "later; you will not actually interrupt image creation.\n");
+
+ # Give child a chance to run.
+ select(undef, undef, undef, 0.25);
+
+ #
+ # Reset signal handlers. User can now kill this process, without
+ # stopping the child.
+ #
+ $SIG{TERM} = 'DEFAULT';
+ $SIG{INT} = 'DEFAULT';
+ $SIG{QUIT} = 'DEFAULT';
+
+ #
+ # Wait until child exits or until user gets bored and types ^C.
+ #
+ waitpid($childpid, 0);
+
+ print("Done. Exited with status: $?\n");
+ $$childstat_ref = $? >> 8;
+ $rc = 0;
+ goto out;
+ }
+ else {
+ $$childpid_ref = $childpid;
+
+ # Child returns to caller and continues...
+ $rc = 0;
+ }
+
+ out:
+ tbwarn("$self BackgroundImageCapture: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# Do the low-level work of capturing the image from a target; called by
+# the high-level CreateImage function. This function should Lock() and
+# Unlock() the image -- none of the prior functions called in
+# CreateImage() should require the image to be locked.
+#
+sub CaptureImage($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my ($rc,$msg) = (-1);
+
+ $msg = "bug: $self CaptureImage must be overridden!";
+ tbwarn("$self CreateImage: $msg\n");
+
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# Do some common finalization tasks. Note, this function should be
+# invoked from CaptureImage or an override of it. It requires the image
+# to be locked.
+#
+sub CreateImageFinalize($$$$) {
+ my ($self,$image,$target,$args) = @_;
+
+ # Append bootlog (which has prepare output)
+ my $bootlog = $args->{'bootlog'};
+ if (defined($bootlog)) {
+ print "\n\n";
+ print "------------------ Prepare Output ----------------\n";
+ print "$bootlog\n";
+ }
+
+ my $this_user = $args->{'user'};
+ my $user_name = $this_user->name();
+ my $user_email = $this_user->email();
+ my $imagepid = $args->{'imagepid'};
+ my $imagename = $image->imagename();
+ my $filename = $args->{'filename'};
+ my $logfile = $args->{'logfile'};
+ my $pid = $args->{'pid'};
+
+ SENDMAIL("$user_name <$user_email>",
+ "Image Creation on $target Completed: $imagepid/$imagename",
+ "Image creation on $target has completed. As you requested, the\n".
+ "image has been written to $filename.\n".
+ "You may now os_load this image on other nodes in your experiment.\n",
+ "$user_name <$user_email>",
+ "Bcc: $TBLOGS",
+ defined($logfile) ? ($logfile->filename()) : ())
+ if (!$args->{'noemail'});
+
+ if (defined($logfile)) {
+ # Close up the log file so the webpage stops.
+ $logfile->Close();
+ $image->ClearLogFile();
+ }
+ my $webtask = $args->{'webtask'};
+ if (defined($webtask)) {
+ # Cause of the fork in run_with_ssh.
+ $webtask->Refresh();
+ $webtask->status("ready");
+ $image->Refresh();
+ if ($image->size() > 0) {
+ $webtask->imagesize($image->size() / 1024);
+ }
+ $webtask->Exited(0);
+ }
+
+ #
+ # Normal images are immediately marked as "released" (and ready),
+ # but global system images are just marked ready, and must be explicitly
+ # released.
+ #
+ $image->MarkReady();
+ if (!($image->global() && $pid eq TBOPSPID()) || !$args->{'doprovenance'}) {
+ $image->Release();
+ }
+ #
+ # If we are being told that this new image needs to be copied back to
+ # its original home, setup the copyback table. I ended up putting this
+ # here instead of clone_image, cause when provenance is turned off, the
+ # readybit is still set while a new snapshot is taken, so cannot use that
+ # to control when the copyback actually happens. It might make more sense
+ # to clear the readybit in this case, but since most sites are not doing
+ # provenance, need to be careful about a change like this.
+ #
+ my $origin_uuid = $args->{'origin_uuid'};
+ if (defined($origin_uuid)) {
+ my $imageid = $image->imageid();
+ my $version = $image->version();
+ DBQueryWarn("replace into image_notifications set ".
+ " imageid='$imageid',version='$version', ".
+ " origin_uuid='$origin_uuid',notified=now()");
+ # This can fail, we will catch it later from the CM daemon when
+ # we try again. Use the nolock option since we have it.
+ system("$TRIGGERUPDATE -l $imageid");
+ }
+
+ if (wantarray) {
+ return (0,undef);
+ }
+ else {
+ return 0;
+ }
+}
+
+sub CreateImageFailure($$$$$) {
+ my ($self,$image,$target,$args,$mesg) = @_;
+
+ if (!defined($mesg)) {
+ $mesg = "";
+ }
+
+ my $gotlock;
+ if ($image->Lock()) {
+ my $lmsg = "Image $image is already locked in $self CreateImageFailure;".
+ " manual cleanup required!";
+ tbwarn($lmsg);
+ $mesg .= "\n$lmsg\n";
+ $gotlock = 0;
+ }
+ else {
+ $gotlock = 1;
+ }
+
+ my $this_user = $args->{'user'};
+ my $user_name = $this_user->name();
+ my $user_email = $this_user->email();
+ my $imagepid = $args->{'imagepid'};
+ my $imagename = $image->imagename();
+ my $filename = $args->{'filename'};
+ my $logfile = $args->{'logfile'};
+ my $pid = $args->{'pid'};
+ my $webtask = $args->{'webtask'};
+
+ #
+ # Send a message to the testbed list.
+ #
+ SENDMAIL("$user_name <$user_email>",
+ "Image Creation Failure on $target: $imagepid/$imagename",
+ defined($mesg) ? $mesg : "",
+ "$user_name <$user_email>",
+ "Cc: $TBOPS",
+ defined($logfile) ? ($logfile->filename()) : ());
+
+ if ($gotlock) {
+ if (defined($logfile)) {
+ # Close up the log file so the webpage stops.
+ $logfile->Close();
+ $image->ClearLogFile();
+ }
+
+ # This is a temporary file.
+ if (defined($filename)) {
+ unlink($filename);
+ }
+ if (defined($webtask)) {
+ # Cause of the fork in run_with_ssh.
+ $webtask->Refresh();
+ $webtask->status("failed");
+ $webtask->imagesize(0);
+ $webtask->Exited(1);
+ }
+ $image->Unlock();
+ }
+
+ return;
+}
+
+#
+# Create an image from a node, possibly the one used in the constructor,
+# or the supplied node. This is a high-level function that calls
+# CreateImageValidate, CheckImageQuota, and CaptureImage.
+#
+sub CreateImage($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my ($rc,$msg);
+
+ if (!defined($args)) {
+ $args = {};
+ }
+ elsif (!ref($args) || ref($args) ne 'HASH') {
+ $rc = -1;
+ $msg = "bug: 'args' parameter must be a hash ref";
+ goto out;
+ }
+
+ #
+ # Verify user and get his DB uid and other info for later.
+ #
+ my $this_user = User->ThisUser();
+ if (!defined($this_user)) {
+ $rc = -1;
+ $msg = "You ($UID) do not exist!";
+ goto out;
+ }
+ $args->{'user'} = $this_user;
+
+ ($image,$msg) = $self->CreateImageValidate($image,$target,$args);
+ if (!defined($image)) {
+ $rc = -1;
+ goto out;
+ }
+
+ ($target,$msg) = $self->CreateImageValidateTarget($image,$target,$args);
+ if (!defined($target)) {
+ $rc = -1;
+ goto out;
+ }
+
+ ($rc,$msg) = $self->CreateImageValidateArgs($image,$target,$args);
+ goto out
+ if ($rc);
+
+ ($rc,$msg) = $self->CheckImageQuota($image->pid(),$image->gid(),$this_user);
+ goto out
+ if ($rc);
+
+ if (!$args->{'foreground'}) {
+ my ($childpid,$childstat);
+
+ ($rc,$msg) = $self->BackgroundImageCapture(
+ $image,$target,$args,\$childpid,\$childstat);
+ goto out
+ if ($rc);
+ if ($childpid) {
+ if (!defined($childstat)) {
+ $rc = 0;
+ $msg = undef;
+ }
+ else {
+ $rc = $childstat;
+ $msg = "$self CreateImage: waited-for child $childpid returned".
+ " nonzero $childstat";
+ }
+ goto out;
+ }
+ #
+ # And only continue if this is the child, or if we are just in
+ # the foreground.
+ #
+ }
+
+ #
+ # Catch compile errors via eval, cause if we backgrounded above,
+ # admins would otherwise have to dig out the logfile. This way they
+ # get email.
+ #
+ eval {
+ ($rc,$msg) = $self->CaptureImage($image,$target,$args);
+ };
+ if ($@ || $rc) {
+ if ($@) {
+ $msg = $@;
+ }
+ $self->CreateImageFailure($image,$target,$args,$msg);
+ goto out;
+ }
+
+ out:
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub DeleteImage($$;$) {
+ my ($self,$image,$args) = @_;
+ my $rc = -1;
+ my $msg = "";
+ my $needunlock = 0;
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ #
+ # Map invoking user to object.
+ #
+ my $this_user = User->ThisUser();
+ if (!defined($this_user)) {
+ $msg = "You ($UID) do not exist!";
+ goto errout;
+ }
+ if (!$image->AccessCheck($this_user, TB_IMAGEID_DESTROY())) {
+ $msg = "You do not have permission to delete this image!";
+ goto errout;
+ }
+ if ($image->pid() eq TBOPSPID() && $image->global() &&
+ (!exists($args->{'force_global'}) || !$args->{'force_global'})) {
+ $msg = "Refusing to delete global system image $image";
+ goto errout;
+ }
+
+ #
+ # Before we do anything destructive, we lock the descriptor.
+ #
+ if (!$args->{'impotent'}) {
+ if ($image->Lock()) {
+ $msg = "Image is locked, please try again later!";
+ goto errout;
+ }
+ $needunlock = 1;
+ }
+ my $imageid = $image->imageid();
+ my $imagename = $image->imagename();
+ my $imagepid = $image->pid();
+ my $imagevers = $image->version();
+
+ # Sanity check; cannot delete a deleted version.
+ if ($args->{'versonly'} && defined($image->deleted())) {
+ $msg = "Image version is already deleted";
+ goto errout;
+ }
+
+ #
+ # We want to send email to the creator. Also, watch for an image created
+ # on the Geni path; the creator urn tells us who the creator is, rather
+ # then who is calling the script. When PROTOGENI_LOCALUSER=0 there is no
+ # local creator, but when set there is a local shadow user we can use.
+ #
+ my $notifyuser = $image->GetCreator();
+ if (!defined($notifyuser)) {
+ $notifyuser = $this_user;
+ }
+ if (defined($image->creator_urn())) {
+ my $geniuser = GeniUser->Lookup($image->creator_urn(), 1);
+ if (defined($geniuser) && $geniuser->IsLocal()) {
+ $notifyuser = $geniuser->emulab_user();
+ }
+ else {
+ # This is okay, it is just for email below.
+ $notifyuser = $geniuser;
+ }
+ }
+ if ($debug) {
+ print STDERR "$self->DeleteImage: Will send email to $notifyuser\n";
+ }
+
+ #
+ # When IMAGEPROVENANCE is on, we never delete system images, we
+ # rename them.
+ #
+ if ($image->pid() eq TBOPSPID() && !$args->{'force'}) {
+ if ($args->{'purge'}) {
+ $args->{'purge'} = 0;
+ print STDERR "$self DeleteImage: Ignoring purge option for system image.\n"
+ if ($debug);
+ }
+ if ($WITHPROVENANCE) {
+ print STDERR "$self DeleteImage: Turning on rename option for system image.\n"
+ if ($debug);
+ $args->{'rename'} = 1;
+ }
+ }
+
+ #
+ # Give subclasses a chance to conduct final checks, ensure nothing
+ # is using the image, etc.
+ #
+ ($rc,$msg) = $self->DeleteImagePrepare($image,$args);
+ if ($rc) {
+ goto errout;
+ }
+
+ #
+ # Only purge or rename files if caller asked.
+ #
+ if ($args->{'purge'} || $args->{'rename'}) {
+ ($rc,$msg) = $self->DeleteImageFiles($image,$args);
+ if ($rc) {
+ goto errout;
+ }
+ }
+
+ #
+ # Stop here if impotent; else, proceed to delete the descriptor and
+ # notify the IMS.
+ #
+ if ($args->{'impotent'}) {
+ $rc = 0;
+ goto out;
+ }
+
+ #
+ # If using the image tracker, have to notify the IMS.
+ #
+ if (!$args->{'versonly'}) {
+ # Do this before delete().
+ if (GetSiteVar("protogeni/use_imagetracker")) {
+ if ($image->SchedIMSDeletion(1) != 0) {
+ $msg = "Could not schedule IMS deletion";
+ goto errout;
+ }
+ }
+ if ($image->Delete() != 0) {
+ $msg = "Could not delete image!";
+ goto errout;
+ }
+ $notifyuser->SendEmail("delete_image: Image has been deleted",
+ "Image $imagepid,$imagename ($imageid) has ".
+ "been deleted by $this_user\n");
+ }
+ else {
+ # Do this before delete().
+ if (GetSiteVar("protogeni/use_imagetracker")) {
+ if ($image->SchedIMSDeletion(0) != 0) {
+ $msg = "Could not schedule IMS deletion";
+ goto errout;
+ }
+ }
+ if ($image->DeleteVersion() != 0) {
+ $msg = "Could not delete image version!";
+ goto errout;
+ }
+ # I know, we are unlocking something we just deleted. Its okay, relax.
+ $image->Unlock();
+ $notifyuser->SendEmail("delete_image: Image Version has been deleted",
+ "Version $imagevers of image $imagepid,$imagename".
+ "($imageid)\nhas been deleted by $this_user\n");
+ }
+
+ # Success!
+ $image->Unlock()
+ if ($needunlock);
+
+ out:
+ if (wantarray) {
+ return ($rc,"");
+ }
+ return 0;
+
+ errout:
+ $image->Unlock()
+ if ($needunlock);
+ tbwarn("$self DeleteImage: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# DeleteImagePrepare is called by DeleteImage, prior to any permanent
+# actions being taken (i.e., before image files are moved or removed,
+# and before the descriptor(s) is deleted). Its implementation is
+# optional.
+#
+sub DeleteImagePrepare($$;$) {
+ my ($self,$image,$args,) = @_;
+
+ if (wantarray) {
+ return (0,"");
+ }
+ else {
+ return 0;
+ }
+}
+
+#
+# DeleteImageFiles is called by DeleteImage, prior to descriptor
+# deletion. It can partially fail, in which case the descriptor will
+# not be deleted, and some amount of manual cleanup may be required.
+#
+sub DeleteImageFiles($$;$) {
+ my ($self,$image,$args,) = @_;
+
+ if (wantarray) {
+ return (-1,"$self DeleteImageFiles: Not implemented!");
+ }
+ else {
+ return -1;
+ }
+}
+
+#
+# Validate ensures that an image is valid. What constitutes validity is
+# format-dependent. Returns (0,) on success; (<nonzero>,msg) on failure.
+#
+sub Validate($$;$) {
+ my ($self,$image,$args) = @_;
+
+ if (wantarray) {
+ return (-1,"$self Validate: Not implemented!");
+ }
+ else {
+ return -1;
+ }
+}
+
+#
+# UpdateHash recreates the image hash from the image file, and updates
+# the database. It returns (undef,msg) on error; or else (hash,) on
+# success.
+#
+sub UpdateHash($$;$) {
+ my ($self,$image,$args) = @_;
+
+ if (wantarray) {
+ return (undef,"$self UpdateHash: Not implemented!");
+ }
+ else {
+ return undef;
+ }
+}
+
+#
+# A helper method to Release() that ensures permissions, etc.
+#
+sub _ReleaseChecks($$;$) {
+ my ($self,$image,$args) = @_;
+ my $msg;
+ my $user;
+
+ my $force = 0;
+ my $markready = 0;
+ if (exists($args->{'force'})) {
+ $force = $args->{'force'};
+ }
+ if (exists($args->{'markready'})) {
+ $markready = $args->{'markready'};
+ }
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ if ($UID) {
+ $user = User->ThisUser();
+ if (!defined($user)) {
+ $msg = "You ($UID) do not exist!";
+ goto errout;
+ }
+ }
+
+ if ($image->released()
+ && (!exists($args->{'force'}) || $args->{'force'} == 0)) {
+ $msg = "Image is already released! ".
+ "Maybe you need to provide imageid:version";
+ goto errout;
+ }
+ if ($UID && !$user->IsAdmin()) {
+ $msg = "Only admins can release an image.";
+ goto errout;
+ }
+ if (!$image->ready() && !($force && $markready)) {
+ $msg = "Image is not ready yet!";
+ goto errout;
+ }
+
+ if (wantarray) {
+ return (0,"");
+ }
+ else {
+ return 0;
+ }
+
+ errout:
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ else {
+ return -1;
+ }
+}
+
+#
+# Release an image (version). Returns (<nonzero>,msg) on error; (0,) on
+# success.
+#
+sub Release($$;$) {
+ my ($self,$image,$args) = @_;
+
+ if (wantarray) {
+ return (-1,"$self Release: Not implemented!");
+ }
+ else {
+ return -1;
+ }
+}
+
+#
+# Downloads the image content from origin (specified in $args ref) into
+# $image. This is not a replacement for image_import; this is just
+# about pulling in the bytes.
+#
+sub ImportImageContent($$;$) {
+ my ($self,$image,$args) = @_;
+
+ if (wantarray) {
+ return (-1,"$self ImportImageContent: Not implemented!");
+ }
+ else {
+ return -1;
+ }
+}
+
+#
+# Stringify for output.
+#
+sub Stringify($) {
+ my ($self) = @_;
+
+ my $type = $self->type();
+
+ return "[$type]";
+}
+
+1;
diff --git a/tbsetup/libimageops_docker.pm.in b/tbsetup/libimageops_docker.pm.in
new file mode 100644
index 0000000000..2e3a5ffc12
--- /dev/null
+++ b/tbsetup/libimageops_docker.pm.in
@@ -0,0 +1,1347 @@
+#!/usr/bin/perl -w
+#
+# Copyright (c) 2000-2018 University of Utah and the Flux Group.
+#
+# {{{EMULAB-LICENSE
+#
+# This file is part of the Emulab network testbed software.
+#
+# This file is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or (at
+# your option) any later version.
+#
+# This file is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+# License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this file. If not, see <http://www.gnu.org/licenses/>.
+#
+# }}}
+#
+package libimageops_docker;
+use strict;
+use libimageops;
+use base qw(libimageops_base);
+
+use libdb;
+use libtestbed;
+use libtblog;
+use User;
+use Node;
+use English;
+use Data::Dumper;
+use overload ('""' => 'Stringify');
+use File::Temp qw/ :mktemp /;
+
+my $TB = "@prefix@";
+my $OURDOMAIN = "@OURDOMAIN@";
+my $CREATEDOCKERIMAGE = "/usr/local/bin/create-docker-image";
+my $DOCKREGCLI = "/usr/local/bin/docker-registry-cli";
+# Prefer dev trees for docker creds, but bail to regular place.
+my $DOCKCREDDIR = "$TB/etc/docker";
+if (! -d "$TB/etc/docker" && -d "/usr/testbed/etc/docker") {
+ $DOCKCREDDIR = "/usr/testbed/etc/docker";
+}
+my $ADMINDOCKREGCLI = "$DOCKREGCLI -C $DOCKCREDDIR/admin-boss.creds" .
+ " --cert $DOCKCREDDIR/cert.pem --key $DOCKCREDDIR/privkey.pem";
+
+sub New($)
+{
+ my ($class) = @_;
+
+ my $self = $class->SUPER::New();
+
+ $self->{'HASH'}->{'maxwait'} = 8 * 60;
+ $self->{'HASH'}->{'idlewait'} = 2 * 60;
+ $self->{'HASH'}->{'reportwait'} = 1 * 60;
+ $self->{'HASH'}->{'maximagesize'} = 1 * 1024 ** 3;
+
+ bless($self, $class);
+ return $self;
+}
+
+#
+# No quotas in Docker-land for now.
+#
+sub CheckImageQuota($$$$) {
+ my ($self,$pid,$gid,$user) = @_;
+ my $rc = 0;
+ my $msg;
+
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub CreateImageValidateArgs($$$$) {
+ my ($self,$image,$node,$args) = @_;
+ my $rc = -1;
+ my $msg;
+
+ #
+ # Need to know this is a docker-host to tailor method below.
+ #
+ if (!$node->isvirtnode()) {
+ $msg = "$node does not appear to be a virtnode!";
+ goto out;
+ }
+
+ my $pnode = Node->Lookup($node->phys_nodeid());
+ my $osimage = OSImage->Lookup($pnode->def_boot_osid());
+ if (!defined($osimage)) {
+ $msg = "Could not get OSImage for $pnode (virtnode $node)";
+ goto errout;
+ }
+
+ if ($osimage->FeatureSupported("docker-host")) {
+ $args->{'delta'} = $args->{'signature'} = 0;
+ }
+
+ $rc = 0;
+
+ out:
+ tbwarn("$self CreateImageValidateArgs: $msg ($rc)\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub GetLocalRegistryPathForImage($$;$$$) {
+ my ($self,$image,$registry_ref,$repo_ref,$tag_ref) = @_;
+ my ($rc,$msg) = (undef,"");
+
+ my $registry;
+ #
+ # Find the local cluster's registry, if any.
+ #
+ if ($OURDOMAIN eq 'utah.cloudlab.us') {
+ $registry = "ops.utah.cloudlab.us:5080";
+ }
+ elsif ($OURDOMAIN eq 'emulab.net') {
+ $registry = "ops.emulab.net:5080";
+ }
+ else {
+ $msg = "Site $OURDOMAIN does not have a local registry;".
+ " cannot capture image $image!";
+ goto out;
+ }
+ #if (TBGetSiteVar("docker/registry",\$registry)
+ # || $registry eq "") {
+ # $msg = "no local registry specified in sitevar general/docker_registry";
+ # goto out;
+ #}
+
+ #
+ # Docker registry does not accept uppercase characters in repo names.
+ #
+ my $repo = lc($image->pid() . "/" . $image->gid() . "/" .
+ $image->imagename());
+ my $tag = $image->version();
+
+ $rc = "$registry/$repo:$tag";
+
+ $$registry_ref = $registry
+ if (defined($registry_ref));
+ $$repo_ref = $repo
+ if (defined($repo_ref));
+ $$tag_ref = $tag
+ if (defined($tag_ref));
+
+ out:
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub CaptureImage($$$$) {
+ my ($self,$image,$node,$args) = @_;
+ my $rc = -1;
+ my $msg;
+
+ my $node_id = $node->node_id();
+ my $phys_node_id = $node->phys_nodeid();
+ my $webtask = $args->{'webtask'};
+ my $this_user = $args->{'user'};
+ my $imagepid = $args->{'imagepid'} || $image->pid();
+ my $imagename = $image->imagename();
+
+ my ($path,$registry,$repo,$tag);
+ my $rs = join("",(0..9,"A".."Z","a".."z")[map { rand(62) } (0..7)]);
+ my $needunlock = 0;
+
+ ($path,$msg) = $self->GetLocalRegistryPathForImage(
+ $image,\$registry,\$repo,\$tag);
+ if (!defined($path)) {
+ $rc = -1;
+ goto errout;
+ }
+
+ #
+ # Before we do anything destructive, we lock the image.
+ #
+ if ($image->Lock()) {
+ $msg = "Image is locked, please try again later ($PID)!";
+ goto out;
+ }
+ $needunlock = 1;
+
+ #
+ # Update the image to point to the repo/tag; change its format.
+ #
+ $image->SetFormat("docker");
+ $image->SetPath($path);
+ # Save off the "filename".
+ $args->{'filename'} = $path;
+
+ #
+ # Also update the image to run on type pcvm, so that we can do real
+ # osloads of this image on pcvms. This should probably be somewhere
+ # else, but no big deal.
+ #
+ $image->SetRunsOnNodeType('pcvm');
+
+ #
+ # We need a temporary user/pass credential for the docker
+ # login/commit/push on the node to login to our cluster's local
+ # Docker registry. So create that. We need to remember these to
+ # delete the token post-imaging.
+ #
+ my $experiment = $node->Reservation();
+ my ($docker_user,$docker_pass);
+ $docker_user = "exim" . $experiment->idx() . "-$rs";
+ $docker_pass = TBGenSecretKey();
+
+ if (User->AddTokenPassword(
+ $this_user,$this_user,"docker.registry","image.create",$repo,
+ $docker_user,$docker_pass,undef,3600,3600,0,1)) {
+ $msg = "Could not create temporary Docker Registry credentials";
+ goto out;
+ }
+
+ # XXX continuing... also need to fixup image descriptor creation,
+ # which is pre-this file.
+ my $debugarg = "";
+ if ($self->debug()) {
+ $debugarg = "-d 10";
+ }
+ my $command =
+ "$CREATEDOCKERIMAGE $debugarg -R $registry -r $repo -t $tag" .
+ " -u $docker_user -p $docker_pass $node_id";
+
+ # Mark webtask
+ $webtask->status("imaging")
+ if (defined($webtask));
+
+ # Clear the bootlog; see below.
+ $node->ClearBootLog();
+
+ #
+ # Big hack; we want to tell the node to update the master password
+ # files. But need to do this in a backwards compatable manner, and
+ # in way that does not require too much new plumbing. So, just touch
+ # file in /var/run, the current version of prepare looks for it.
+ #
+ my $SAVEUID;
+ if ($args->{'update_prepare'}) {
+ $SAVEUID = $UID;
+ $EUID = $UID = 0;
+ my $cmd = "$TB/bin/sshtb -n -o ConnectTimeout=10 ".
+ "-host $node_id touch /var/run/updatemasterpasswdfiles";
+ print STDERR "About to: '$cmd'\n"
+ if ($self->debug());
+ system($cmd);
+ if ($?) {
+ $msg = "'$cmd' failed";
+ goto errout;
+ }
+ $EUID = $UID = $SAVEUID;
+ }
+
+ #
+ # Now execute command and wait. If we have a webtask, capture the
+ # output of the command so we can spawn another child to process it;
+ # but do that before suid.
+ #
+ my ($fname,$rpid);
+ if (defined($webtask)) {
+ $fname = mktemp("/tmp/create-docker-image-XXXXXX");
+ open(TFD,">$fname");
+ close(TFD);
+ $rpid = fork();
+ if ($rpid == 0) {
+ $SIG{INT} = sub { exit(0); };
+ while (!stat($fname)) {
+ sleep(0.1);
+ }
+ open(FD,$fname);
+ while (1) {
+ while (my $line = <FD>) {
+ if ($self->debug()) {
+ print "CREATEIMAGE: $line";
+ }
+ chomp($line);
+ if ($line =~ /PUSHPROGRESS:\s+(\d+)\s+bytes/) {
+ $webtask->Refresh();
+ $webtask->imagesize(int($1)/(1024));
+ }
+ }
+ sleep(0.1);
+ }
+ exit(0);
+ }
+ }
+
+ $SAVEUID = $UID;
+ $EUID = $UID = 0;
+ my $result = $self->RunWithSSH($phys_node_id,undef,$command,$fname);
+ $EUID = $UID = $SAVEUID;
+
+ if (defined($webtask)) {
+ sleep(1);
+ kill(2,$rpid);
+ waitpid($rpid,0);
+ }
+
+ User->DeleteTokenPasswords($this_user,$this_user,"docker.registry",
+ $docker_user,$docker_pass);
+
+ # Grab boot log now. Node will reboot and possibly erase it. We should
+ # probably come up with a better way to handle this.
+ my $bootlog;
+ if ($node->GetBootLog(\$bootlog) == 0) {
+ $args->{'bootlog'} = $bootlog;
+ }
+ if (defined($webtask)) {
+ # Cause of the fork in run_with_ssh.
+ $webtask->Refresh();
+ $webtask->status("finishing");
+ }
+
+ #
+ # If we timed out, if the result code was bad, or if the image size
+ # grew too large.
+ #
+ if ($result eq "setupfailed") {
+ $msg = "FAILED: Node setup failed ...";
+ goto out;
+ }
+ if ($result eq "timeout") {
+ $msg = "FAILED: Timed out generating image ...";
+ goto out;
+ }
+ if ($result eq "toobig") {
+ $msg = "FAILED: Maximum image size (".$self->maximagesize()." bytes) exceeded ...";
+ goto out;
+ }
+ if ($result ne "0") {
+ $msg = "FAILED: Returned error code $result generating image ...";
+ goto out;
+ }
+
+ # We want to Validate that the image made it to the repository;
+ # also, Validate has the side-effects of updating the DB with the
+ # size/hash fields.
+ ($rc,$msg) = $self->Validate($image,{'update'=>1,'validate'=>{'all'=>1}});
+ if ($rc) {
+ $msg = "$self CaptureImage: failed validation: $msg";
+ goto out;
+ }
+ # Pick up the new size/hash info.
+ #$image->Refresh();
+
+ if (defined($webtask)) {
+ $webtask->Refresh();
+ my $sz = int($image->size());
+ $webtask->imagesize($sz / 1024)
+ if ($sz > 0);
+ }
+
+ my $cname = "$imagepid/$imagename";
+ $cname .= ":" . $image->version()
+ if ($args->{'doprovenance'});
+ print "$cname: ";
+ print "image creation succeeded, written to $path.\n";
+ "Final size: ".$image->size()." bytes.\n";
+
+ ($rc,$msg) = $self->CreateImageFinalize($image,$node,$args);
+ goto out
+ if ($rc);
+
+ $rc = 0;
+
+ out:
+ $image->Unlock()
+ if ($needunlock);
+ if ($rc) {
+ tbwarn("$self CaptureImage: $msg\n");
+ }
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub CLI_FORCE_NONE() { return 0; }
+sub CLI_FORCE_TOKEN() { return 1; }
+sub CLI_FORCE_SETUID() { return 2; }
+
+sub GetDockerRegCLICommandPrefix($$$$$$$) {
+ my ($self,$op,$repo,$context,$force,$setuidref,$cleanupref) = @_;
+ if (!defined($force)) {
+ $force = CLI_FORCE_NONE();
+ }
+
+ my $dosetuid = 0;
+ my $this_user;
+ my $username;
+ if ($UID != 0) {
+ $this_user = User->ThisUser();
+ if (!defined($this_user)) {
+ return (undef,"You ($UID) do not exist!");
+ }
+ }
+ else {
+ $username = "root";
+ }
+
+ my $debugarg = "";
+ if ($self->debug()) {
+ $debugarg = "-d";
+ }
+
+ #
+ # First, see if we should be using the setuid root path, and the
+ # admin ssl cert.
+ #
+ if (($force == CLI_FORCE_NONE()
+ && exists($ENV{"WITH_TB_ADMIN_PRIVS"})
+ && $ENV{"WITH_TB_ADMIN_PRIVS"} eq "1")
+ || $force == CLI_FORCE_SETUID()) {
+ # Check if we have setuid privs, so we can use the admin cert;
+ # or if we'll need a temp token. No other way I know to do this
+ # in perl other than to try!
+ if ($EUID != 0) {
+ my $SAVEEUID = $EUID;
+ $EUID = 0;
+ if ($EUID == 0) {
+ $dosetuid = 1;
+ }
+ $EUID = $SAVEEUID;
+
+ if ($dosetuid != 1) {
+ if ($force == CLI_FORCE_SETUID()) {
+ return (undef,"not setuid!");
+ }
+ else {
+ tbwarn("$self: GetDockerRegCLICommandPrefix: invoked with".
+ " wap but not setuid; falling back to token auth!\n");
+ $force = CLI_FORCE_TOKEN();
+ }
+ }
+ }
+ }
+ if ($dosetuid == 1 || ($force != CLI_FORCE_TOKEN()
+ && ($UID == 0 || $EUID == 0))) {
+ $$setuidref = 1;
+ return ("$ADMINDOCKREGCLI $debugarg",undef);
+ }
+ else {
+ $$setuidref = 0;
+ }
+
+ if ($this_user->uid() eq 'geniuser') {
+ # If the current user is geniuser, there might be a localuser or
+ # not; so just use token auth for this.
+ $force = CLI_FORCE_TOKEN();
+ }
+
+ if ($force == CLI_FORCE_TOKEN()) {
+ my ($docker_token_user,$docker_token_pass);
+ $docker_token_user = $this_user->uid() . "-$context";
+ $docker_token_pass = TBGenSecretKey();
+ if (User->AddTokenPassword(
+ $this_user,$this_user,"docker.registry",$op,$repo,
+ $docker_token_user,$docker_token_pass,undef,600,600,0,1)) {
+ return (undef,
+ "Could not create temporary Docker Registry credentials");
+ }
+ else {
+ $$cleanupref = sub { User->DeleteTokenPasswords(
+ $this_user,$this_user,"docker.registry",
+ $docker_token_user,$docker_token_pass); };
+ }
+
+ return ("$DOCKREGCLI $debugarg".
+ " -u $docker_token_user -p $docker_token_pass",undef);
+ }
+ else {
+ my ($uid,$homedir) = ($this_user->uid(),$this_user->HomeDir());
+ return ("$DOCKREGCLI $debugarg".
+ " -u $uid -p '' --cert $homedir/.ssl/emulab.pem",undef);
+ }
+}
+
+sub DeleteImageFiles($$$) {
+ my ($self,$image,$args) = @_;
+ if (!defined($args)) {
+ $args = {};
+ }
+ my $versonly = $args->{'versonly'} || 0;
+ my $purge = $args->{'purge'} || 0;
+ my $rename = $args->{'rename'} || 0;
+ my $impotent = $args->{'impotent'} || 0;
+ my $msg = "";
+ my $rc = -1;
+ my $cmd;
+
+ #
+ # When doing image provenance, we have to deal with all versions
+ # of the image. This will not return deleted versions.
+ #
+ my @images = ();
+ if ($image->AllVersions(\@images)) {
+ $msg = "Could not get list of image (versions)";
+ goto out;
+ }
+ #
+ # When deleting just a single version, if this is the last or only
+ # version, then turn off version only. Makes no sense to have a
+ # descriptor with no non-deleted versions.
+ #
+ if ($versonly && scalar(@images) == 1) {
+ $versonly = 0;
+ }
+ if ($versonly) {
+ @images = ($image);
+ }
+
+ my $setuid = 0;
+ my $cleanup;
+ my $SAVEEUID;
+
+ #
+ # Grab our command prefix. If that creates a token for us, we use
+ # it for all versions.
+ #
+
+ foreach my $imageversion (@images) {
+ # Split the path into server, repo, and tag.
+ my ($server,$repo,$tag);
+ if ($imageversion->path() =~ /^([^\/]+)\/([^:]+):(.*)$/) {
+ ($server,$repo,$tag) = ($1,$2,$3);
+ }
+ else {
+ $msg = "$self DeleteImageFiles: bad path ".$imageversion->path().
+ "; aborting!";
+ goto out;
+ }
+
+ if ($impotent) {
+ print STDERR "$self DeleteImageFiles: would have deleted" .
+ " $server $repo $tag\n";
+ next;
+ }
+
+ if ($rename) {
+ my $newtag = $imageversion->imageid() . "-$tag";
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.rename",$repo,"image-delete-rename",CLI_FORCE_NONE(),
+ \$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self DeleteImageFiles: $msg";
+ goto errout;
+ }
+ my $tcmd = "$cmd -s $server tag_image -r $repo -t $tag -n $newtag";
+ if ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: running '$tcmd'\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($tcmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "$self DeleteImageFiles: failed to rename tag" .
+ " $imageversion ($server $repo $tag) to $newtag!";
+ goto out;
+ }
+
+ $tcmd = "$cmd -s $server delete_tag -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: running '$tcmd'\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($tcmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "$self DeleteImageFiles: failed to delete old tag" .
+ " in rename of $imageversion ($server $repo $tag) to" .
+ " $newtag!";
+ goto out;
+ }
+ }
+ elsif ($purge) {
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$repo,"image-delete-check",CLI_FORCE_NONE(),
+ \$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self DeleteImageFiles: $msg";
+ goto errout;
+ }
+ my $tcmd = "$cmd -s $server check_image -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: running '$tcmd'\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($tcmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ if ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: $image ($server".
+ " $repo $tag) file already gone; nothing to delete.\n";
+ }
+ ($rc,$msg) = (0,0);
+ goto out;
+ }
+
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.delete",$repo,"image-delete-tag",CLI_FORCE_NONE(),
+ \$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self DeleteImageFiles: $msg";
+ goto errout;
+ }
+ $tcmd = "$cmd -s $server delete_image -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: running '$tcmd'\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($tcmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "$self DeleteImageFiles: failed to delete image" .
+ " $imageversion ($server $repo $tag)!";
+ goto out;
+ }
+ }
+ elsif ($self->debug()) {
+ print STDERR "$self DeleteImageFiles: nothing to do for" .
+ " $server $repo $tag\n";
+ }
+ }
+
+ $rc = 0;
+
+ out:
+ tbwarn("$self DeleteImageFiles: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub Validate($$;$) {
+ my ($self,$image,$args) = @_;
+ my ($rc,$msg,$cmd,$userperm);
+
+ if (!defined($args)) {
+ $args = {};
+ }
+ if ($self->debug()) {
+ use Data::Dumper;
+ print STDERR "$self Validate: args: ".Dumper($args)."\n";
+ }
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ if ($args->{'update'}) {
+ $userperm = TB_IMAGEID_MODIFYINFO();
+ }
+ else {
+ $userperm = TB_IMAGEID_READINFO();
+ }
+
+ # Split the path into server, repo, and tag.
+ my ($server,$repo,$tag);
+ if ($image->path() =~ /^([^\/]+)\/([^:]+):(.*)$/) {
+ ($server,$repo,$tag) = ($1,$2,$3);
+ }
+ else {
+ $msg = "Invalid Docker image path: ".$image->path()."!";
+ goto errout;
+ }
+
+ my $setuid = 0;
+ my $cleanup;
+ my $SAVEEUID;
+
+ #
+ # Ensure the manifest and layers are all present.
+ #
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$repo,"validate",CLI_FORCE_NONE(),\$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self Validate: $msg";
+ goto errout;
+ }
+ $cmd .= " -s $server check_image -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self: validating Docker image ($server,$repo,$tag)\n";
+ print STDERR "$self: validation command: $cmd\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($cmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "Error validating Docker image!";
+ goto errout;
+ }
+
+ # Now grab the manifest metadata to populate the DB, if we're
+ # supposed to update:
+ if ($args->{'update'}) {
+ if ($self->debug()) {
+ print STDERR "$self: updating Docker image metadata" .
+ " ($server,$repo,$tag)\n";
+ }
+ my @lines;
+
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$repo,"validate",CLI_FORCE_NONE(),\$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self Validate: $msg";
+ goto errout;
+ }
+ $cmd .= " -s $server get_image_metadata -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self: metadata command: $cmd\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ @lines = `$cmd`;
+ $rc = $?;
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "Error getting Docker image reference metadata!";
+ goto errout;
+ }
+ if ($self->debug()) {
+ print STDERR "$self Validate: ".join('',@lines)."\n";
+ }
+ for my $line (@lines) {
+ chomp($line);
+ }
+
+ if ($args->{'validate'}{'all'} || $args->{'validate'}{'hash'}) {
+ if ($self->debug()) {
+ print STDERR "$self Validate: updating hashes\n";
+ }
+ my ($alg,$hash);
+ my ($lnum,$lalg,$lhash,$lsize);
+ my $maxlayer = -1;
+ for my $line (@lines) {
+ if ($line =~ /^digest:\s+([^:]+):([\w\d]+)$/) {
+ ($alg,$hash) = ($1,$2);
+ if ($self->debug()) {
+ print STDERR "$self: digest: $alg, $hash\n";
+ }
+ }
+ elsif ($line =~ /^layer\[(\d+)\]:\s+([^:]+):(\w+),\s+(\d+)$/) {
+ if (int($1) > $maxlayer) {
+ $lnum = int($1);
+ $maxlayer = $lnum;
+ ($lalg,$lhash,$lsize) = ($2,$3,$4);
+ }
+ if ($self->debug()) {
+ print STDERR "$self: layer: $1, $2, $3, $4\n";
+ }
+ }
+ }
+ if (!defined($alg) || !defined($hash)) {
+ $msg = "Unrecognized algorithm:hash digest for tag $tag!";
+ goto errout;
+ }
+ elsif ($alg ne 'sha256') {
+ $msg = "unexpected Docker digest algorithm $alg; not inserting!";
+ goto errout;
+ }
+ if (!defined($lalg) || !defined($lhash)) {
+ $msg = "Unrecognized algorithm:hash digest for layers for" .
+ " tag $tag!";
+ goto errout;
+ }
+ elsif ($lalg ne 'sha256') {
+ $msg = "unexpected Docker digest algorithm $alg for layer" .
+ " $lhash; not inserting as delta hash!";
+ goto errout;
+ }
+
+ $image->SetHash($hash);
+ $image->SetDeltaHash($lhash);
+ if ($self->debug()) {
+ print STDERR "$self Validate: set hash $hash, delta hash $lhash\n";
+ }
+ }
+ if ($args->{'validate'}{'all'} || $args->{'validate'}{'size'}) {
+ if ($self->debug()) {
+ print STDERR "$self Validate: updating sizes\n";
+ }
+ my ($size,$lalg,$lhash,$lsize,$lnum);
+ my $maxlayer = -1;
+ for my $line (@lines) {
+ if ($line =~ /^size:\s+(\d+)$/) {
+ $size = $1;
+ if ($self->debug()) {
+ print STDERR "$self: size: $size\n";
+ }
+ }
+ elsif ($line =~ /^layer\[(\d+)\]:\s+([^:]+):(\w+),\s+(\d+)$/) {
+ if (int($1) > $maxlayer) {
+ $lnum = int($1);
+ $maxlayer = $lnum;
+ ($lalg,$lhash,$lsize) = ($2,$3,$4);
+ }
+ if ($self->debug()) {
+ print STDERR "$self: layer size: $1, $2, $3, $lsize\n";
+ }
+ }
+ }
+ if (!defined($size)) {
+ $msg = "No size for tag $tag!";
+ goto errout;
+ }
+ elsif (!defined($lsize)) {
+ $msg = "No layer size for tag $tag!";
+ goto errout;
+ }
+
+ $image->SetSize(int($size));
+ $image->SetDeltaSize(int($lsize));
+ if ($self->debug()) {
+ print STDERR "$self Validate: size $size, delta size $lsize\n";
+ }
+ }
+
+ #
+ # Sanitize the updated timestamp for any update. If it is NULL,
+ # that will break tmcd and anything else that assumes a non-null
+ # mtime. So, if it is NULL, one was never set; just use the
+ # ctime. Otherwise, do nothing.
+ #
+ my $stamp;
+ $image->GetUpdate(\$stamp);
+ if (!defined($stamp) || $stamp eq '' || $stamp == 0) {
+ $image->GetCreate(\$stamp);
+ $image->MarkUpdate(undef,$stamp);
+ }
+ }
+
+ out:
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ tbwarn("$self Validate: $msg\n");
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ return -1;
+}
+
+sub UpdateHash($$;$) {
+ my ($self,$image,$args) = @_;
+ my ($cmd,$msg,$rc);
+
+ my $debugarg = "";
+ if ($self->debug()) {
+ $debugarg = "-d";
+ }
+ my $userperm;
+ my $impotent = 0;
+ if (exists($args->{"impotent"})) {
+ $impotent = $args->{"impotent"};
+ }
+ if ($impotent) {
+ $userperm = TB_IMAGEID_MODIFYINFO();
+ }
+ else {
+ $userperm = TB_IMAGEID_READINFO();
+ }
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ # Split the path into server, repo, and tag.
+ my ($server,$repo,$tag);
+ if ($image->path() =~ /^([^\/]+)\/([^:]+):(.*)$/) {
+ ($server,$repo,$tag) = ($1,$2,$3);
+ }
+ else {
+ $msg = "Invalid Docker image path: ".$image->path()."!";
+ goto errout;
+ }
+
+ if ($self->debug()) {
+ print STDERR "$self: updating Docker image hash ($server,$repo,$tag)\n";
+ }
+
+ my $setuid = 0;
+ my $cleanup;
+ my $SAVEEUID;
+
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$repo,"updatehash",CLI_FORCE_NONE(),\$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ $msg = "$self UpdateHash: $msg";
+ goto errout;
+ }
+ $cmd .= " -s $server get_reference_metadata -r $repo --reference $tag";
+ if ($self->debug()) {
+ print STDERR "$self: validating Docker image ($server,$repo,$tag)\n";
+ print STDERR "$self: validation command: $cmd\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ my @lines = `$cmd`;
+ $rc = $?;
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ $msg = "Error getting Docker image digest!";
+ goto errout;
+ }
+
+ my ($alg,$hash);
+ for my $line (@lines) {
+ chomp($line);
+ if ($line =~ /^digest:\s+([^:]+):(\w+)$/) {
+ ($alg,$hash) = ($1,$2);
+ if ($self->debug()) {
+ print STDERR "$self: $alg, $hash\n";
+ }
+ last;
+ }
+ }
+ if (!defined($alg) || !defined($hash)) {
+ $msg = "Unrecognized algorithm:hash digest for reference $tag!";
+ goto errout;
+ }
+ elsif ($alg ne 'sha256') {
+ $msg = "unexpected Docker digest algorithm $alg; not inserting!";
+ goto errout;
+ }
+
+ if (!$impotent) {
+ if ($image->SetHash($hash) != 0) {
+ $msg = "Failed to set the hash for $image";
+ goto errout;
+ }
+ }
+
+ out:
+ if (wantarray) {
+ return ($hash,"");
+ }
+ return $hash;
+
+ errout:
+ tbwarn("$self UpdateHash: $msg\n");
+ if (wantarray) {
+ return (undef,$msg);
+ }
+ return undef;
+}
+
+sub Release($$;$) {
+ my ($self,$image,$args) = @_;
+ my ($rc,$msg) = (-1,"");
+
+ my ($quiet,$force,$markready,$impotent,$validate) = (0,0,0,0,1);
+ if (!defined($args)) {
+ $args = {};
+ }
+ if (exists($args->{'impotent'})) {
+ $impotent = $args->{'impotent'};
+ }
+ if (exists($args->{'quiet'})) {
+ $quiet = $args->{'quiet'};
+ }
+ if (exists($args->{'force'})) {
+ $force = $args->{'force'};
+ }
+ if (exists($args->{'markready'})) {
+ $markready = $args->{'markready'};
+ }
+
+ ($rc,$msg) = $self->_ReleaseChecks($image,$args);
+ if ($rc) {
+ goto errout;
+ }
+
+ if ($force && $markready) {
+ if ($impotent) {
+ print "Would mark image ready/released, but not do anything else\n"
+ if (!$quiet);
+ }
+ else {
+ print "Marking image ready/released, but not doing anything else\n"
+ if (!$quiet);
+ $image->MarkReady();
+ $image->MarkReleased();
+ }
+ goto out;
+ }
+
+ my $needunlock = 0;
+
+ #
+ # Before we do anything destructive, we lock the image.
+ #
+ if ($image->Lock()) {
+ $msg = "Image is locked, please try again later!";
+ goto errout;
+ }
+ $needunlock = 1;
+
+ if ($validate) {
+ if ($impotent) {
+ print STDERR "Would run imagevalidate on " .
+ $image->versname() . "\n" if (!$quiet);
+ }
+ else {
+ if ($self->debug()) {
+ print STDERR "Running imagevalidate on " .
+ $image->versname() . "\n";
+ }
+ ($rc,$msg) = $self->Validate($image);
+ if ($rc) {
+ $msg = "Failed to validate the image: $msg";
+ goto errout;
+ }
+ }
+ }
+ # Now mark as released.
+ if ($impotent) {
+ print STDERR "Would mark image as released\n"
+ if (!$quiet);
+ }
+ elsif ($image->Release()) {
+ $msg = "Could not mark image as released!";
+ goto errout;
+ }
+
+ out:
+ $image->Unlock()
+ if ($needunlock);
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ $image->Unlock()
+ if ($needunlock);
+ tbwarn("$self Release: $msg\n");
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ return -1;
+}
+
+sub ImportImageContent($$;$) {
+ my ($self,$image,$args) = @_;
+ if (!defined($args)) {
+ $args = {};
+ }
+ my ($rc,$msg) = (-1,"");
+
+ my ($origin_path,$newhash,$downloaded_ref,$force) = (undef,undef,undef,0);
+ my $locked = 0;
+ my $needunlock = 0;
+ if (exists($args->{'origin_path'})) {
+ $origin_path = $args->{'origin_path'};
+ }
+ else {
+ $msg = "No origin docker registry path provided";
+ goto errout;
+ }
+ if (exists($args->{'newhash'})) {
+ $newhash = $args->{'newhash'};
+ }
+ if (exists($args->{'downloaded_ref'})) {
+ $downloaded_ref = $args->{'downloaded_ref'};
+ }
+ if (exists($args->{'force'})) {
+ $force = $args->{'force'};
+ }
+ if (exists($args->{'locked'})) {
+ $locked = $args->{'locked'};
+ }
+
+ #
+ # Make sure we are already locked; lock if not.
+ #
+ if (!$locked) {
+ if ($image->Lock()) {
+ $msg = "$self ImportImageContent: $image is already locked,".
+ " please try again later ($PID)!";
+ goto out;
+ }
+ $needunlock = 1;
+ }
+
+ #
+ # Like in CaptureImage, we need to fixup the path to point to our
+ # local registry; and a few other things below:
+ #
+ my ($path,$server,$repo,$tag);
+ ($path,$msg) = $self->GetLocalRegistryPathForImage(
+ $image,\$server,\$repo,\$tag);
+ if (!defined($path)) {
+ $rc = -1;
+ goto errout;
+ }
+
+ my $doit = 0;
+ if ((defined($newhash) && $newhash ne $image->hash()) || $force) {
+ $doit = 1;
+ }
+ else {
+ #
+ # Check to see if the image already exists at the path; don't
+ # need it if it's here.
+ #
+ my $setuid = 0;
+ my $cleanup;
+ my $SAVEEUID;
+ my $cmd;
+
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$repo,"validate",CLI_FORCE_NONE(),\$setuid,\$cleanup);
+ if (!defined($cmd)) {
+ tbwarn("$self ImportImageContent: $msg; skipping validation");
+ }
+ else {
+ $cmd .= " -s $server check_image -r $repo -t $tag";
+ if ($self->debug()) {
+ print STDERR "$self: checking Docker image ($server,$repo,$tag)\n";
+ print STDERR "$self: validation command: $cmd\n";
+ }
+ if ($setuid) {
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ }
+ $rc = system($cmd);
+ $EUID = $SAVEEUID
+ if ($setuid);
+ $cleanup->()
+ if (defined($cleanup));
+ if ($rc) {
+ if ($self->debug()) {
+ print STDERR "$self ImportImageContent: $image not".
+ " present; will download\n";
+ }
+ $doit = 1;
+ }
+ }
+ }
+
+ #
+ # Just return if we already have the right image and aren't forcing.
+ #
+ if (!$doit) {
+ goto out;
+ }
+
+ #
+ # Update the image to point to the repo/tag; change its format.
+ #
+ $image->SetPath($path);
+ $image->SetFormat("docker");
+ #
+ # Also update the image to run on type pcvm, so that we can do real
+ # osloads of this image on pcvms. This should probably be somewhere
+ # else, but no big deal.
+ #
+ $image->SetRunsOnNodeType('pcvm');
+
+ my ($rserver,$rrepo,$rtag);
+ if ($origin_path =~ /^([^\/]+)\/([^:]+):(.*)$/) {
+ ($rserver,$rrepo,$rtag) = ($1,$2,$3);
+ }
+ else {
+ $msg = "$self ImportImageContent: could not extract remote docker".
+ " registry info from $origin_path!";
+ goto errout;
+ }
+
+ #
+ # Now we can pull and push the image all in one go, using our local
+ # admin credentials both times. NB: for this one, we *require*
+ # admin credential access so we can pull any image from the remote
+ # cluster.
+ #
+ my $setuid = 1;
+ my $cleanup;
+ my $SAVEEUID;
+ my $cmd;
+
+ my $idir = File::Temp->newdir(TEMPLATE=>"docker-image-XXXXXX",DIR=>"/tmp");
+ my $filebasename = $idir->dirname() . "image";
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.pull",$rrepo,"validate",CLI_FORCE_SETUID(),
+ \$setuid,\$cleanup);
+ if (!defined($cmd) || !$setuid) {
+ $msg = "$self ImportImageContent: error getting admin CLI prefix;".
+ " aborting!";
+ goto errout;
+ }
+ $cmd .= " -s $rserver pull_image -r $rrepo -t $rtag -f $filebasename";
+ if ($self->debug()) {
+ print STDERR "$self: pulling remote Docker image into $filebasename".
+ " ($rserver,$rrepo,$rtag)\n";
+ print STDERR "$self: command: $cmd\n";
+ }
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ $rc = system($cmd);
+ $EUID = $SAVEEUID;
+ if ($rc) {
+ $msg = "$self ImportImageContent: error pulling remote image!";
+ goto errout;
+ }
+
+ #
+ # And now we push it into our local registry.
+ #
+ ($setuid,$cleanup) = (0,undef);
+ ($cmd,$msg) = $self->GetDockerRegCLICommandPrefix(
+ "image.push",$repo,"push",CLI_FORCE_SETUID(),\$setuid,\$cleanup);
+ if (!defined($cmd) || !$setuid) {
+ $msg = "$self ImportImageContent: error getting admin CLI prefix;".
+ " aborting!";
+ goto errout;
+ }
+ $cmd .= " -s $server push_image -r $repo -t $tag -f $filebasename";
+ if ($self->debug()) {
+ print STDERR "$self: pushing remote Docker image from $filebasename".
+ " ($server,$repo,$tag)\n";
+ print STDERR "$self: command: $cmd\n";
+ }
+ $SAVEEUID = $EUID;
+ $EUID = 0;
+ $rc = system($cmd);
+ $EUID = $SAVEEUID;
+ if ($rc) {
+ $msg = "$self ImportImageContent: error pushing remote image to".
+ " local registry!";
+ goto errout;
+ }
+
+ #
+ # Now we validate the image, just as in CaptureImage.
+ #
+ if ($self->debug()) {
+ print STDERR "Running imagevalidate on " . $image->versname() . "\n";
+ }
+ ($rc,$msg) = $self->Validate($image,{'update'=>'1','validate'=>{'all'=>1}});
+ if ($rc) {
+ $msg = "Failed to validate $image: $msg";
+ goto errout;
+ }
+
+ if (defined($downloaded_ref)) {
+ $$downloaded_ref = 1;
+ }
+
+ out:
+ $image->Unlock()
+ if ($needunlock);
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ $image->Unlock()
+ if ($needunlock);
+ tbwarn("$self ImportImageContent: $msg\n");
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ return -1;
+}
+
+1;
diff --git a/tbsetup/libimageops_ec2.pm.in b/tbsetup/libimageops_ec2.pm.in
new file mode 100644
index 0000000000..e98c3d9d42
--- /dev/null
+++ b/tbsetup/libimageops_ec2.pm.in
@@ -0,0 +1,126 @@
+#!/usr/bin/perl -w
+#
+# Copyright (c) 2000-2018 University of Utah and the Flux Group.
+#
+# {{{EMULAB-LICENSE
+#
+# This file is part of the Emulab network testbed software.
+#
+# This file is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or (at
+# your option) any later version.
+#
+# This file is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+# License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this file. If not, see <http://www.gnu.org/licenses/>.
+#
+# }}}
+#
+package libimageops_ec2;
+use strict;
+use libimageops;
+use base qw(libimageops_ndz);
+
+use libdb;
+use libtestbed;
+use libimageops;
+use libtblog;
+use Node;
+use English;
+use Data::Dumper;
+
+#
+# Testbed Support libraries
+#
+use lib "@prefix@/lib";
+use libdb;
+use EmulabConstants;
+use libtestbed;
+use libadminmfs;
+use Experiment;
+use Node;
+use User;
+use OSImage;
+use Image; # Cause of datasets.
+use Logfile;
+use WebTask;
+use Project;
+use EmulabFeatures;
+
+my $EC2SNAP = "$TB/sbin/ec2import.proxy";
+
+sub CreateImageValidateTarget($$$$) {
+ my ($self,$image,$target,$args) = @_;
+
+ # Only warn if they explicitly specified an option
+ if ((defined($args->{'delta'}) && $args->{'delta'} == 1)
+ || (defined($args->{'signature'}) && $args->{'signature'} == 1)) {
+ tbwarn("*** WARNING: don't support delta imaging of EC2 images, ".
+ "ignoring delta/signature options.\n");
+ $args->{'delta'} = $args->{'signature'} = 0;
+ }
+
+ $args->{'pid'} = $image->pid();
+
+ return $target;
+}
+
+#
+# EC2 nodes.
+# Run on ops.
+#
+sub DoCapture($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my $rc = -1;
+ my $msg;
+
+ my $this_user = $args->{'user'};
+ my $user_uid = $this_user->uid();
+ my $ofilename = $args->{'ofilename'};
+ my $webtask = $args->{'webtask'};
+
+ my $safe_target = User::escapeshellarg($target);
+ my $pid = $image->pid();
+ my $imageid = $image->imageid();
+
+ my $cmd = "$TB/bin/sshtb -host $CONTROL $EC2SNAP -u $user_uid ".
+ "$safe_target $pid $user_uid $imageid $ofilename";
+ print STDERR "About to: '$cmd'\n"
+ if (1 || $debug);
+
+ my $SAVEUID = $UID;
+ $EUID = $UID = 0;
+
+ system($cmd);
+ $rc = $?;
+ if ($rc) {
+ $msg = "Command '$cmd' failed with $?";
+ }
+
+ $EUID = $UID = $SAVEUID;
+
+ if (defined($webtask)) {
+ if ($rc) {
+ $webtask->status("failed");
+ }
+ else {
+ $webtask->status("finishing");
+ }
+ }
+
+ tbwarn("$self DoCapture: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+1;
diff --git a/tbsetup/libimageops_ndz.pm.in b/tbsetup/libimageops_ndz.pm.in
new file mode 100644
index 0000000000..87c01afd41
--- /dev/null
+++ b/tbsetup/libimageops_ndz.pm.in
@@ -0,0 +1,1843 @@
+#!/usr/bin/perl -w
+#
+# Copyright (c) 2000-2018 University of Utah and the Flux Group.
+#
+# {{{EMULAB-LICENSE
+#
+# This file is part of the Emulab network testbed software.
+#
+# This file is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or (at
+# your option) any later version.
+#
+# This file is distributed in the hope that it will be useful, but WITHOUT
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+# FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public
+# License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this file. If not, see <http://www.gnu.org/licenses/>.
+#
+# }}}
+#
+package libimageops_ndz;
+use strict;
+use libimageops;
+use base qw(libimageops_base);
+
+use File::Basename;
+use Cwd qw(realpath);
+use libdb;
+use libtestbed;
+use libimageops;
+use libtblog;
+use Node;
+use English;
+use Data::Dumper;
+
+#
+# Testbed Support libraries
+#
+use lib "@prefix@/lib";
+use libdb;
+use EmulabConstants;
+use libtestbed;
+use libadminmfs;
+use Experiment;
+use Node;
+use User;
+use OSImage;
+use Image; # Cause of datasets.
+use Logfile;
+use WebTask;
+use Project;
+use EmulabFeatures;
+
+#
+# Configure vars.
+#
+my $BOSSIP = "@BOSSNODE_IP@";
+#my $NONFS = @NOSHAREDFS@;
+
+#
+# Commands
+#
+my $CREATEIMAGE = "/usr/local/bin/create-versioned-image";
+my $CREATEXENIMAGE = "/usr/local/bin/create-xen-image";
+my $OCREATEIMAGE = "/usr/local/bin/create-image";
+my $REBOOT_PREPARE = "@CLIENT_BINDIR@/reboot_prepare";
+my $FRISKILLER = "$TB/sbin/frisbeehelper";
+my $IMAGEVALIDATE = "$TB/sbin/imagevalidate";
+my $SHA1 = "/sbin/sha1";
+my $FSNODE = "@FSNODE@";
+my $SYSIMAGEDIR = "$TB/images";
+my $SSH = "/usr/bin/ssh";
+my $SCP = "/usr/bin/scp";
+my $SUDO = "/usr/local/bin/sudo";
+
+#
+# Options for imagezipper on the client-side. These apply only to imagezip,
+# i.e., the local node imaging process. They also only apply to the latest
+# version of the client script (create-versioned-image). For the older path,
+# options are hardwired into the create-image script.
+#
+# Note that since we cannot have spaces in the string passed to the client,
+# options are encoded; e.g.:
+# -N -z 9 -d -a SHA1
+# would be encoded as:
+# N,z=9,d,a=SHA1
+#
+# Specific options:
+#
+# By default we do not create relocations (-N) in the resulting image for a
+# couple of reasons. One is that we never did relocation support for GRUB
+# partition boot code, so modern Linux images would not have relocations
+# anyway. For FreeBSD this does mean that we cannot relocate them (we use
+# a relocation for correct disklabel construction), but we never really
+# took advantage of this anyway. The second reason is that ranges with
+# relocations are always considered different for hash comparisons, so an
+# otherwise empty FreeBSD delta image would have 64K of data in it.
+#
+# XXX change of heart: we now will generate relocations for FreeBSD
+# partition images. I *have* had occasion to relocate these (e.g., loading
+# them into a virtual disk for imagezip testing) and you just cannot use
+# them without relocations. Adding the -N flag for other images is done
+# later based on the def_boot_osid, so it won't try to do relocations for
+# Linux or Windows or anything else.
+#
+# So, only add "N" here if you absolutely, positively cannot tolerate
+# relocations anywhere!
+#
+my $ZIPPEROPTS = "";
+
+sub CreateImageValidate($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my ($msg);
+ my ($dstsigfile);
+
+ ($image,$msg) = $self->SUPER::CreateImageValidate($image,$target,$args);
+ if (!defined($image)) {
+ goto err;
+ }
+
+ #
+ # See if per-project/per-user provenance feature is set.
+ #
+ if ($WITHPROVENANCE) {
+ my $project = Project->Lookup($image->pid());
+ my $group = Group->Lookup($image->pid(), $image->gid());
+ if (!defined($project)) {
+ $msg = "Could not lookup project for $image";
+ goto err;
+ }
+
+ # But allow feature override.
+ $args->{'doprovenance'} = EmulabFeatures->FeatureEnabled(
+ "ImageProvenance", $args->{'user'}, $args->{'group'});
+
+ # Temporary override for all geni projects until we can export deltas.
+ if ($project->IsNonLocal()) {
+ $args->{'nodelta'} = 1;
+ }
+ }
+
+ #
+ # When provenance is enabled and we have delta support, we always collect
+ # signatures and we always try to create deltas. Note that we set them to
+ # a different non-zero value so we can distinguish this case and not print
+ # various warnings below.
+ #
+ # XXX We really shouldn't be doing this implicitly--our caller should just
+ # be specifying the options when provenance is enabled--but this script is
+ # called from a surprisingly large number of places, so we do!
+ #
+ if ($args->{'doprovenance'} && $WITHDELTAS) {
+ $args->{'delta'} = 2
+ if ($args->{'delta'} == 0);
+ $args->{'signature'} = 2
+ if ($args->{'signature'} == 0);
+
+ # XXX let's try this for now
+ $args->{'deltapct'} = 50
+ if ($args->{'deltapct'} == 0);
+
+ # Override delta but still collect signatures.
+ $args->{'delta'} = 0
+ if ($image->nodelta() || $args->{'nodelta'});
+ }
+
+ #
+ # Make sure that the directory exists and is writeable for the user.
+ # We test this by creating the file. Its going to get wiped anyway.
+ #
+ my $filename = $image->TempImageFile();
+ my $isglobal = $image->global();
+ my $prefixdir = $image->SaveDir();
+
+ #
+ # If we are creating a signature file for this image, get the
+ # signature file name.
+ #
+ if ($args->{'signature'}) {
+ # We want to use the temp filename.
+ $args->{'dstsigfile'} = $filename . ".sig";
+ }
+
+ #
+ # Redirect pathname for global images. See equiv code in clone_image.
+ #
+ $args->{'usepath'} = 0;
+
+ if ($isglobal && $image->IsSystemImage()) {
+ $filename = $prefixdir . basename($filename);
+ print "*** WARNING: Writing global descriptor to $filename instead!\n";
+
+ #
+ # Ditto for the signature file
+ #
+ if ($args->{'signature'}) {
+ $args->{'dstsigfile'} = $prefixdir . basename($args->{'dstsigfile'});
+ }
+
+ #
+ # XXX the Emulab config of the master server doesn't know this trick
+ # so when it tries to lookup imageid emulab-ops/<whatever> it would
+ # still map to /usr/testbed and fail because it cannot update images
+ # outside of /{users,group,proj}. So we skirt the issue by passing
+ # it the full path contructed here rather than the imageid.
+ #
+ $args->{'usepath'} = 1;
+ }
+
+ #
+ # Make sure real path is someplace that makes sense; remember that the
+ # image is created on the nodes, and it NFS mounts directories on ops.
+ # Writing the image to anyplace else is just going to break things.
+ #
+ # Use realpath on the directory part of the path to validate. If we ran
+ # realpath on the filename, it would return null since $filename (a temp
+ # file) won't exist. Note that we can use dirname/basename here since
+ # $filename is well formed (both dir and file components).
+ #
+ # We still use the original path for passing to the client-side since
+ # boss and the client may not have the same real path for a file.
+ #
+ my $ofilename = $filename;
+ my $tdir = dirname($filename);
+ my $translated = realpath($tdir);
+ if ($translated && $translated =~ /^([-\w\.\/\+:]+)$/) {
+ my $tfile = basename($filename);
+
+ $filename = $1;
+ # XXX check the last component
+ if ($tfile =~ /^([-\w\.\+:]+)$/) {
+ $filename = "$filename/$1";
+ } else {
+ $msg = "Bad characters in image filename";
+ goto err;
+ }
+ }
+ else {
+ if ($translated) {
+ $msg = "Bad characters in image pathname";
+ goto err;
+ }
+ $msg = "Image directory does not exist";
+ goto err;
+ }
+ # Make sure the result (really the final component) is not a symlink or dir
+ if (-l $filename) {
+ $msg = "$filename is a symlink! Must be a plain file.";
+ goto err;
+ }
+ if (-d $filename) {
+ $msg = "$filename is a directory! Must be a plain file.";
+ goto err;
+ }
+
+ #
+ # The file must reside in an allowed directory. Since this script
+ # runs as the caller, regular file permission checks ensure its a file
+ # the user is allowed to use.
+ #
+ if (! TBValidUserDir($filename, $ISFS)) {
+ $msg = "$filename does not resolve to an allowed directory!";
+ goto err;
+ }
+
+ $args->{'filename'} = $filename;
+ $args->{'ofilename'} = $ofilename;
+
+ if (wantarray) {
+ return ($image,);
+ }
+ else {
+ return $image;
+ }
+
+ err:
+ tbwarn("$self CreateImageValidate: $msg\n");
+ if (wantarray) {
+ return (undef,$msg);
+ }
+ else {
+ return undef;
+ }
+}
+
+sub CreateImageValidateArgs($$$$) {
+ my ($self,$image,$node,$args) = @_;
+ my $rc = -1;
+ my $msg;
+
+ ($rc,$msg) = $self->SUPER::CreateImageValidateArgs($image,$node,$args);
+
+ # Do not create a delta for system images but still collect signatures.
+ my $experiment = $node->Reservation();
+ $args->{'delta'} = 0
+ if ($experiment->pid() eq TBOPSPID());
+
+ #
+ # If we are creating a delta image, figure out what image we are
+ # deriving from so that we can grab the signature.
+ #
+ if ($args->{'delta'}) {
+ #
+ # Find the source image we are creating a delta from. When provenance
+ # is enabled, we can use the parent image. If not enabled, we attempt
+ # to determine what is already on the node via the partitions table.
+ #
+ # If we cannot determine the source, we just warn and create a full
+ # image instead.
+ #
+ my $srcimage;
+ if ($args->{'doprovenance'}) {
+ $srcimage = $image->Parent();
+ }
+ if (!defined($srcimage) && !$image->isdataset()) {
+ (undef, $srcimage) = $node->RunningOsImage();
+ }
+ if (defined($srcimage)) {
+ my $srcsigfile = $srcimage->FullImageSigFile();
+ if (! -e "$srcsigfile") {
+ # XXX user may not have direct access to a shared image
+ my $SAVEUID = $UID;
+ $EUID = $UID = 0;
+ if (! -e "$srcsigfile") {
+ $srcsigfile = undef;
+ }
+ $EUID = $UID = $SAVEUID;
+ }
+ if (!defined($srcsigfile)) {
+ if ($args->{'delta'} == 1) {
+ print "*** WARNING: no signature file for source image, ".
+ "cannot create delta; creating full image instead.\n";
+ }
+ $args->{'delta'} = 0;
+ }
+ else {
+ # Save off the srcsigfile for later use
+ $args->{'srcsigfile'} = $srcsigfile;
+ }
+ # Save it off for later use.
+ $args->{'srcimage'} = $srcimage;
+ } else {
+ if ($args->{'delta'} == 1) {
+ print "*** WARNING: no source for image, ".
+ "cannot create delta; creating full image instead.\n";
+ }
+ $args->{'delta'} = 0;
+ }
+ }
+
+ #
+ # To avoid blowing a cavernous hole ("allow all TCP ports to boss")
+ # in the per-experiment firewall, we don't use the frisbee uploader if
+ # the node is firewalled.
+ #
+ if ($args->{'usefup'} && $experiment->IsFirewalled()) {
+ tbwarn("$self CreateImageValidateTarget: $node is firewalled, not using Frisbee uploader\n");
+ $args->{'usefup'} = 0;
+ if ($NONFS) {
+ $args->{'usenfs'} = 0;
+ $args->{'usessh'} = 1;
+ } else {
+ $args->{'usenfs'} = 1;
+ $args->{'usessh'} = 0;
+ }
+ }
+
+ $rc = 0;
+
+ if (wantarray) {
+ return ($rc,);
+ }
+ else {
+ return $rc;
+ }
+
+ validationerr:
+ tbwarn("$self CreateImageValidateArgs: $msg\n");
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# This does exactly the work of capturing the ndz file and getting it
+# back to storage.
+#
+sub DoCapture($$$$) {
+ my ($self,$image,$node,$args) = @_;
+ my $rc = -1;
+ my $msg;
+ my $isxenhost = 0;
+ my $def_devtype = "ad";
+ my $def_devnum = 0;
+ my $devtype;
+ my $devnum;
+ my $device;
+
+ if (!defined($node) || !ref($node) || !$node->isa("Node")) {
+ $msg = "target ($node) is not a Node!";
+ goto errout;
+ }
+
+ my $node_id = $node->node_id();
+ my $isvirtnode = $node->isvirtnode();
+ my $isdataset = $image->isdataset();
+ my $doprovenance = $args->{'doprovenance'};
+ my $usefup = $args->{'usefup'};
+ my $usepath = $args->{'usepath'};
+ my $usessh = $args->{'usessh'};
+ my $nomfs = $args->{'nomfs'};
+ my $filename = $args->{'filename'};
+ my $ofilename = $args->{'ofilename'};
+ my $srcsigfile = $args->{'srcsigfile'};
+ my $dstsigfile = $args->{'dstsigfile'};
+ my $webtask = $args->{'webtask'};
+ my $delta = $args->{'delta'};
+ my $deltapct = $args->{'deltapct'};
+ my $signature = $args->{'signature'};
+ my $bsname = $args->{'bsname'};
+ my $logfile = $args->{'logfile'};
+ my $srcimage = $args->{'srcimage'};
+
+ my $ssh_node_id;
+ if ($isvirtnode && !$isdataset) {
+ $ssh_node_id = $node->phys_nodeid();
+ }
+ else {
+ $ssh_node_id = $node_id;
+ }
+
+ #
+ # Need to know this is a xen-host to tailor method below.
+ #
+ if ($isvirtnode) {
+ my $pnode = Node->Lookup($node->phys_nodeid());
+ my $osimage = OSImage->Lookup($pnode->def_boot_osid());
+ if (!defined($osimage)) {
+ $msg = "Could not get OSImage for $pnode";
+ goto errout;
+ }
+ $isxenhost = 1
+ if ($osimage->FeatureSupported("xen-host"));
+ }
+
+ if (! ($isvirtnode || $isdataset)) {
+ #
+ # Get the disktype for this node
+ #
+ $node->disktype(\$devtype);
+ $node->bootdisk_unit(\$devnum);
+
+ $devtype = $def_devtype
+ if (!defined($devtype));
+ $devnum = $def_devnum
+ if (!defined($devnum));
+ $device = "/dev/${devtype}${devnum}";
+ }
+
+ #
+ # Okay, we want to build up a command line that will run the script on
+ # on the client node. We use the imageid description to determine what
+ # slice (or perhaps the entire disk) is going to be zipped up. We do not
+ # allow arbitrary combos of course.
+ #
+ my $startslice;
+ my $loadlength;
+ my $command;
+
+ # Default to the package's zipperopts; we might add to them.
+ my $zipperopts = $ZIPPEROPTS;
+
+ #
+ # Virtnode images use a version of the old create-image script on the vhost
+ # XXX needs to be fixed.
+ #
+ if ($isvirtnode && !$isdataset && (!$doprovenance || !$isxenhost)) {
+ $command = "$OCREATEIMAGE";
+ if ($usefup) {
+ my $id;
+ if ($usepath) {
+ $id = $ofilename;
+ } else {
+ $id = $image->pid() . "/" . $image->imagename();
+ }
+ $command .= " -S $BOSSIP -F $id";
+ }
+
+ #
+ # XXX Need to add XEN package flag to descriptor.
+ #
+ if ($isxenhost) {
+ if ($image->mbr_version() == 99) {
+ $command .= " -p";
+ }
+ if ($image->loadpart()) {
+ $command .= " -s " . $image->loadpart();
+ }
+ }
+ $command .= " $node_id";
+
+ if ($usefup || $usessh) {
+ $command .= " -";
+ } else {
+ $command .= " $ofilename";
+ }
+ }
+ #
+ # Regular nodes with provenance tracking is turned off, use the old script.
+ #
+ elsif (!$doprovenance) {
+ $command = "$OCREATEIMAGE";
+ if ($usefup) {
+ my $id;
+ if ($usepath) {
+ $id = $ofilename;
+ } else {
+ $id = $image->pid() . "/" . $image->imagename();
+ }
+ $command .= " -S $BOSSIP -F $id";
+ }
+
+ if ($isdataset) {
+ # This is not backward compatable, but none of the BS code is.
+ $command .= " -b $bsname";
+ }
+ else {
+ $startslice = $image->loadpart();
+ $loadlength = $image->loadlength();
+ if ($startslice || $loadlength == 1) {
+ $command .= " -s $startslice";
+ }
+ $command .= " $device";
+ }
+
+ if ($usefup || $usessh) {
+ $command .= " -";
+ } else {
+ $command .= " $ofilename";
+ }
+
+ #
+ # XXX always use ssh for now to get better log info; i.e., all
+ # the log info winds up in one logfile.
+ #
+ $usessh = 1;
+ }
+ #
+ # Otherwise, use the new script with different argument syntax.
+ #
+ else {
+ $command = ($isxenhost && !$isdataset ? "$CREATEXENIMAGE" : "$CREATEIMAGE");
+
+ #
+ # XEN Hosts cannot do provenance/delta without client side update.
+ # We need to provide these arguments for backwards compat though.
+ #
+ if ($isxenhost && !$isdataset) {
+ $command .= " $node_id";
+ if ($usefup || $usessh) {
+ $command .= " -";
+ } else {
+ $command .= " $ofilename";
+ }
+ }
+
+ my $id;
+ if ($usefup) {
+ $command .= " METHOD=frisbee SERVER=$BOSSIP";
+
+ # if the node has a subboss, use that for downloads
+ my $subboss;
+ $node->GetSubboss("frisbee", \$subboss);
+ if (defined($subboss)) {
+ $command .= " DOWNSERVER=$subboss";
+ }
+
+ if ($usepath) {
+ $id = $ofilename;
+ } else {
+ $id = $image->pid() . "/" . $image->imagename() . ":" .
+ $image->version();
+ }
+ } else {
+ $id = $ofilename;
+ }
+ $command .= " IMAGENAME=$id";
+ if ($srcsigfile) {
+ if (!$usefup) {
+ $command .= " OLDSIGFILE=$srcsigfile";
+ } else {
+ my $sid = $srcimage->pid() . "/" . $srcimage->imagename() .
+ ":" . $srcimage->version();
+ $command .= " OLDSIGFILE=$sid,sig";
+ }
+ }
+ if ($dstsigfile) {
+ if (!$usefup || $usepath) {
+ $command .= " NEWSIGFILE=$dstsigfile";
+ } else {
+ $command .= " NEWSIGFILE=$id,sig";
+ }
+ }
+
+ #
+ # See whether we need the "no relocations" flag or not.
+ # We only include generate relocations for FreeBSD parititon images.
+ #
+ my $needrelocs = 0;
+ if ($image->loadpart()) {
+ my $pnode = Node->Lookup($node->phys_nodeid());
+ my $osimage = OSImage->Lookup($pnode->def_boot_osid());
+ if (defined($osimage) && $osimage->OS() eq "FreeBSD") {
+ $needrelocs = 1;
+ }
+ }
+ if (!$needrelocs) {
+ $zipperopts .= "$ZIPPEROPTS,"
+ if ($ZIPPEROPTS);
+ $zipperopts .= "N";
+ }
+
+ if ($deltapct > 0) {
+ $zipperopts .= ","
+ if ($zipperopts);
+ $zipperopts .= "P=$deltapct";
+ }
+
+ if ($isdataset) {
+ # This is not backward compatable, but none of the BS code is.
+ $command .= " BSNAME=$bsname";
+ }
+ else {
+ $startslice = $image->loadpart();
+ $loadlength = $image->loadlength();
+
+ if ($startslice || $loadlength == 1) {
+ $command .= " PART=$startslice";
+ }
+ if (!$isxenhost) {
+ # The XEN host will figure out what device on its own.
+ $command .= " DISK=$device";
+ }
+ }
+
+ if ($zipperopts) {
+ $command .= " IZOPTS=$zipperopts";
+ }
+
+ #
+ # XXX always use ssh for now to get better log info; i.e., all
+ # the log info winds up in one logfile.
+ #
+ $usessh = 1;
+ }
+
+ # This tells the master server what uploader path to use.
+ if ($image->SetUploaderPath($filename)) {
+ $msg = "Could not set the uploader path";
+ goto errout;
+ }
+
+ # Clear the bootlog; see below.
+ $node->ClearBootLog();
+
+ #
+ # Setup _CheckProgress state
+ #
+ # XXX initial idle period. This is the period before any write is performed
+ # to the file. When creating or checking signatures, it can take a long time
+ # before anything is written to the new image file. So we give them some
+ # extra time to get the ball rolling.
+ #
+ my $maxiidleticks = int($self->idlewait() / $self->checkwait());
+ if ($delta || $signature) {
+ $maxiidleticks *= 2;
+ }
+
+ my $result;
+ #
+ # We are going to pass this state array to either RunWithSSH or
+ # TBAdminMfsRunCmd.
+ #
+ my %cpstate = (
+ 'node_id' => $ssh_node_id,'runticks' => 0,'idleticks' => 0,
+ 'checkwait' => $self->checkwait(),'reportwait' => $self->reportwait(),
+ 'maxwait' => $self->maxwait(),'idlewait' => $self->idlewait(),
+ 'maxiidleticks' => $maxiidleticks,'webtask' => $webtask,
+ 'filename' => $filename,'lastsize' => 0,
+ 'maximagesize' => $self->maximagesize(),'result' => undef );
+
+ #
+ # Big hack; we want to tell the node to update the master password
+ # files. But need to do this in a backwards compatable manner, and
+ # in way that does not require too much new plumbing. So, just touch
+ # file in /var/run, the current version of prepare looks for it.
+ #
+ if ($args->{'update_prepare'}) {
+ my $SAVEUID = $UID;
+ $EUID = $UID = 0;
+ my $cmd = "$TB/bin/sshtb -n -o ConnectTimeout=10 ".
+ "-host $node_id touch /var/run/updatemasterpasswdfiles";
+ print STDERR "About to: '$cmd'\n" if ($debug);
+ system($cmd);
+ if ($?) {
+ $msg = "'$cmd' failed";
+ goto errout;
+ }
+ $EUID = $UID = $SAVEUID;
+ }
+
+ #
+ # Virtnodes.
+ # Run on vnode host.
+ #
+ if ($isvirtnode || $isdataset) {
+ my $SAVEUID = $UID;
+ $EUID = $UID = 0;
+
+ if (!$isdataset) {
+ #
+ # XEN creates a problem; the physical host cannot actually
+ # execute a command inside the guest, but we need to run
+ # reboot_prepare and reboot it. FreeBSD creates an additional
+ # problem in that shutdown has to run to invoke prepare; reboot
+ # does not run it, and a shutdown from outside the VM has the
+ # sae effect; prepare does not run. What a pain.
+ #
+ my $cmd = "$TB/bin/sshtb -n -o ConnectTimeout=10 ".
+ "-host $node_id $REBOOT_PREPARE";
+ print STDERR "About to: '$cmd'\n" if ($debug);
+ system($cmd);
+ if ($?) {
+ $msg = "'$cmd' failed";
+ goto errout;
+ }
+ }
+
+ # Mark webtask
+ $webtask->status("imaging")
+ if (defined($webtask));
+
+ #
+ # Now execute command and wait.
+ #
+ if ($NONFS) {
+ $result = $self->RunWithSSH(
+ $ssh_node_id, \%cpstate, $command, $ofilename);
+ } else {
+ $result = $self->RunWithSSH(
+ $ssh_node_id, \%cpstate, $command, undef);
+ }
+ $EUID = $UID = $SAVEUID;
+ goto done;
+ }
+
+ #
+ # Normal nodes.
+ # Reboot into admin mode and run the command.
+ # Note that without a shared FS, we just boot the node into the admin MFS
+ # and run the command via SSH, capturing the output.
+ #
+ my $me = $0;
+ my %mfsargs = ();
+ $mfsargs{'name'} = $me;
+ $mfsargs{'prepare'} = 1;
+
+ if ($usessh) {
+ #
+ # Put the node in admin mode...
+ #
+ if (!$nomfs) {
+ $mfsargs{'on'} = 1;
+ $mfsargs{'clearall'} = 0;
+ if (TBAdminMfsSelect(\%mfsargs, undef, $node_id)) {
+ $result = "setupfailed";
+ goto done;
+ }
+
+ #
+ # ...boot it...
+ #
+ $mfsargs{'reboot'} = 1;
+ $mfsargs{'retry'} = 0;
+ $mfsargs{'wait'} = 1;
+ my @failed = ();
+ if (TBAdminMfsBoot(\%mfsargs, \@failed, $node_id)) {
+ $result = "setupfailed";
+ goto done;
+ }
+ }
+
+ # Mark webtask
+ $webtask->status("imaging")
+ if (defined($webtask));
+
+ #
+ # ...execute command and wait!
+ # Note: we do not pass the filename, that is part of the key/value
+ # string we built up.
+ #
+ my $SAVEUID = $UID;
+ $EUID = $UID = 0;
+ $result = $self->RunWithSSH($ssh_node_id, \%cpstate, $command, undef);
+ $EUID = $UID = $SAVEUID;
+ if ($result eq "setupfailed") {
+ goto done;
+ }
+ } else {
+ $mfsargs{'command'} = $command;
+ $mfsargs{'timeout'} = $self->maxwait() + $self->checkwait();
+ $mfsargs{'pfunc'} = \&_CheckProgress;
+ $mfsargs{'pinterval'} = $self->checkwait();
+ $mfsargs{'pcookie'} = \%cpstate;
+
+ # Mark webtask
+ $webtask->status("imaging")
+ if (defined($webtask));
+
+ my $retry = 1;
+ while ($retry) {
+ $retry = 0;
+ if (TBAdminMfsRunCmd(\%mfsargs, undef, $node_id)) {
+ $result = "setupfailed"
+ if (!defined($result));
+ }
+ }
+ }
+
+ #
+ # XXX woeful backward compat hack.
+ # The old client-side script will not recognize the -S and -F options
+ # we pass in and will exit(-1). We detect that here and retry with ssh/nfs.
+ #
+ # Note that we only do this in the old, non-provenance world since you
+ # must have an up-to-date MFS to handle provenance.
+ #
+ if (!$doprovenance && $usefup && $result eq "255") {
+ print STDERR "MFS does not support frisbee upload, falling back on ".
+ ($NONFS ? "ssh" : "nfs")."...\n";
+
+ $command = "$OCREATEIMAGE ";
+
+ $startslice = $image->loadpart();
+ $loadlength = $image->loadlength();
+ if ($startslice || $loadlength == 1) {
+ $command .= " -s $startslice";
+ }
+ $command .= " $device";
+ if ($usessh) {
+ $command .= " -";
+ } else {
+ $command .= " $ofilename";
+ }
+
+ $usefup = 0;
+ # reset state for _CheckProgress
+ $cpstate{'runticks'} = 0;
+ $cpstate{'idleticks'} = 0;
+ $cpstate{'lastsize'} = 0;
+ $cpstate{'result'} = undef;
+
+ if ($NONFS) {
+ $result = $self->RunWithSSH(
+ $ssh_node_id, \%cpstate, $command, $ofilename);
+ } else {
+ $result = $self->RunWithSSH(
+ $ssh_node_id, \%cpstate, $command, undef);
+ }
+ }
+
+ done:
+
+ # Grab boot log now. Node will reboot and possibly erase it. We should
+ # probably come up with a better way to handle this.
+ if (!$isdataset) {
+ my $bootlog;
+ if ($node->GetBootLog(\$bootlog) == 0) {
+ $args->{'bootlog'} = $bootlog;
+ }
+ }
+ if (defined($webtask)) {
+ # Cause of the fork in run_with_ssh.
+ $webtask->Refresh();
+ $webtask->status("finishing");
+ }
+
+ #
+ # If we timed out, if the result code was bad, or if the image size
+ # grew too large.
+ #
+ if ($result eq "setupfailed") {
+ $msg = "FAILED: Node setup failed ...";
+ goto out;
+ }
+ if ($result eq "timeout") {
+ $msg = "FAILED: Timed out generating image ...";
+ goto out;
+ }
+ if ($result eq "toobig") {
+ $msg = "FAILED: Maximum image size (".$self->maximagesize()." bytes) exceeded ...";
+ goto out;
+ }
+ if ($result ne "0") {
+ $msg = "FAILED: Returned error code $result generating image ...";
+ goto out;
+ }
+
+ $rc = 0;
+ $msg = undef;
+
+ out:
+ # Mike says it is a good idea to clear this.
+ $image->ClearUploaderPath();
+
+ #
+ # Turn admin mode back off and reboot back to the old OS
+ #
+ if (!($isvirtnode || $isdataset) && !$nomfs) {
+ my %args = ();
+ $args{'name'} = $me;
+ $args{'on'} = 0;
+ $args{'clearall'} = 0;
+ if (TBAdminMfsSelect(\%args, undef, $node_id)) {
+ print("*** $me:\n".
+ " Could not turn admin mode off for $node_id!\n");
+ if (!defined($msg)) {
+ $msg = "Encountered problems cleaning up";
+ }
+ goto out2;
+ }
+
+ %args = ();
+ $args{'name'} = $me;
+ $args{'on'} = 0;
+ $args{'reboot'} = 1;
+ $args{'wait'} = 0;
+ if (TBAdminMfsBoot(\%args, undef, $node_id)) {
+ print("*** $me:\n".
+ " Failed to reboot $node_id on cleanup!\n");
+ if (!defined($msg)) {
+ $msg = "Encountered problems cleaning up";
+ }
+ goto out2;
+ }
+ }
+
+ out2:
+ tbwarn("$self DoCapture: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+#
+# Create an image from the given target. The image parameter must be an
+# Image object whose capture into was validated by CreateImageValidate.
+# The target parameter, however, is still whatever
+# CreateImageValidateTarget returned. This function does not assume it
+# is a Node.
+#
+# This does all the generic final image checks (that we couldn't do
+# before this because the image lock was not held), and other stuff, for
+# NDZ images, but does not actually run the command to capture the node.
+# It calls $self->DoCapture() to do that. This provides an abstraction
+# boundary that libimageops_ec2 relies upon. libimageops_ec2 needs
+# nearly everything that libimageops_ndz does, but it does not want
+# node-specific things (because the capture target is not a node). In
+# general, it is worthwhile to preserve our ability to capture an ndz
+# image from a non-Node target.
+#
+sub CaptureImage($$$$) {
+ my ($self,$image,$target,$args) = @_;
+ my $msg;
+ my $rc = -1;
+ my $needunlock = 0;
+ my $hacksigfile;
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+ #if (!defined($target) || !ref($target) || !$target->isa("Node")) {
+ # $msg = "target ($target) is not a Node!";
+ # goto errout;
+ #}
+
+ # Define some local vars so we don't have to refer back to args.
+ my $pid = $args->{'pid'};
+ if (!defined($pid)) {
+ $msg = "$self CaptureImage: pid not set; bug?";
+ goto errout;
+ }
+ my $isdataset = $image->isdataset();
+ my $prefixdir = $image->SaveDir();
+ my $imageid = $image->imageid();
+ my $imagepid = $args->{'imagepid'};
+ my $imagename = $image->imagename();
+ my $srcimage = $args->{'srcimage'};
+ my $doprovenance = $args->{'doprovenance'};
+ my $srcsigfile = $args->{'srcsigfile'};
+ my $dstsigfile = $args->{'dstsigfile'};
+ my $filename = $args->{'filename'};
+ my $delta = $args->{'delta'};
+ my $deltapct = $args->{'deltapct'};
+ my $logfile = $args->{'logfile'};
+
+ #
+ # Before we do anything destructive, we lock the image.
+ #
+ if ($image->Lock()) {
+ $msg = "Image is locked, please try again later ($PID)!";
+ goto errout;
+ }
+ $needunlock = 1;
+
+ # Now we can set the webtask.
+ my $webtask = $args->{'webtask'};
+ if (defined($webtask)) {
+ $image->SetWebTask($webtask);
+ }
+
+ if ($doprovenance && !$isdataset && $image->ready()) {
+ $msg = "$image ready flag is set, this is inconsistent!";
+ goto errout;
+ }
+
+ #
+ # Slight problem here; killing off the running frisbeed will cause
+ # any experiment trying to load that image, to blow up. So we
+ # do not want to do this for system images, but for project images
+ # this is generally okay to do.
+ #
+ if ($pid ne TBOPSPID()) {
+ system("$FRISKILLER -k $imageid");
+ if ($?) {
+ $msg = "Could not kill running frisbee for $imageid!";
+ goto errout;
+ }
+ }
+
+ #
+ # We want to confirm the user can create the temp file in the target
+ # directory, so create a zero length file. But first, need to make
+ # sure the target directory exists in the image path is a directory.
+ #
+ # Make sure the path directory exists.
+ if ($image->CreateImageDir()) {
+ $msg = "Could not create image directory";
+ goto errout;
+ }
+ if (!open(FILE, "> $filename")) {
+ $msg = "Could not create $filename: $!";
+ goto errout;
+ }
+ if (!close(FILE)) {
+ $msg = "Could not truncate $filename: $!";
+ goto errout;
+ }
+ #
+ # XXX this script runs as the user creating the image.
+ # However, in the uploader case, the uploader runs as the creator of
+ # the image. In the case those two are not the same, we need to make
+ # sure that the file we create here is group writable.
+ #
+ if (!chmod(0664, $filename)) {
+ $msg = "Could not make $filename group writable: $!";
+ goto errout;
+ }
+
+ #
+ # For the source signature file of global images, we actually have to copy
+ # it to somewhere where frisbee can access it (in case NFS is being used).
+ # Note that we wait to do this until after we are sure the imagedir exists.
+ #
+ if ($srcsigfile && $srcimage->IsSystemImage()) {
+ my $osrcsigfile = $srcsigfile;
+ $srcsigfile = $prefixdir . basename($srcsigfile);
+ if (system("cp -fp $osrcsigfile $srcsigfile")) {
+ $msg = "Could not copy source signature file ".
+ "$osrcsigfile to $srcsigfile";
+ goto errout;
+ }
+ # XXX remember so we can cleanup later
+ $hacksigfile = $srcsigfile;
+ }
+
+ #
+ # From here on out, we should take care to clean up the DB, and
+ # reboot the source node. DoCapture should do all of that, with the
+ # exception of the hacksigfile name cleanup.
+ #
+ ($rc,$msg) = $self->DoCapture($image,$target,$args);
+ goto errout
+ if ($rc);
+
+ #
+ # XXX ugh! If we were doing the autodelta thing, we have to check
+ # our logfile to see if imagezip reported creating a full image
+ # instead of a delta. Here we are relying on the fact that we are
+ # using SSH, that we are in the background and thus keeping a log
+ # (so the message will wind up in our log), and we depend on the
+ # format of the message itself.
+ #
+ # Ugly? Yes, but worst case one of our assumptions fails and we record an
+ # image as a delta when it isn't, which is just inefficient when it comes to
+ # loading the image.
+ #
+ if ($delta && $deltapct > 0 && defined($logfile)) {
+ if (open(FD, "<" . $logfile->filename())) {
+ # XXX should occur early in the log
+ my $maxlines = 100;
+ while ($maxlines--) {
+ my $line = <FD>;
+ if ($line =~ /^Auto image selection creating (\S+) image/) {
+ if ($1 eq "full") {
+ print "Chose to create full image rather than delta.\n";
+ $delta = $args->{'delta'} = 0;
+ }
+ last;
+ }
+ }
+ close(FD);
+ }
+ }
+
+ #
+ # The upload completed okay, so move the files into place so that
+ # imagevalidate finds them in the correct place. We have to watch for
+ # the case that usepath=1 (target is in /usr/testbed); we do not want
+ # to rename them to the target (will not work anyway), they have to
+ # stay in /proj. More succintly, we always move the new files to the
+ # prefix location.
+ #
+ my $hfilename = $prefixdir .
+ basename(($delta ? $image->DeltaImageFile() : $image->FullImageFile()));
+
+ if (system("/bin/mv -f $filename $hfilename")) {
+ $msg = "Could not move new image file ($filename) into place ($hfilename)";
+ goto errout;
+ }
+ if ($dstsigfile &&
+ system("/bin/mv -f $dstsigfile $prefixdir" .
+ basename(($delta ?
+ $image->FullImageSigFile() :
+ $image->FullImageSigFile())))) {
+ $msg = "Could not move new signature file ($dstsigfile) into place".
+ " ($prefixdir/$dstsigfile)";
+ goto errout;
+ }
+
+ #
+ # In the new world, we can have both a full and delta image.
+ # If the other version exists for the image we just created, we need to
+ # get rid of it as it is now stale.
+ #
+
+ # Use this filename from here on out
+ $args->{'filename'} = $filename = $hfilename;
+
+ #
+ # Update fields in the DB related to the image.
+ #
+ # Note that we do not do this for "standard" images since they get
+ # uploaded into /proj/emulab-ops rather than /usr/testbed. We could
+ # automatically move the image into place here, but that makes us
+ # nervous. We prefer an admin do that by hand after testing the new
+ # image!
+ #
+
+ #
+ # isdelta be gone! We now key off the individual size fields to tell
+ # whether one or both of a delta and full image exist.
+ #
+ my $fsize = (stat($filename))[7];
+ if (!defined($fsize)) {
+ #
+ # XXX exact value doesn't matter since imagevalidate will fix it.
+ # Just has to be non-zero for full/delta differentiation to work.
+ $fsize = 1;
+ }
+ if ($delta) {
+ $image->SetDeltaSize($fsize);
+ } else {
+ $image->SetFullSize($fsize);
+ }
+
+ #
+ # User and nonglobal images are immediately marked as "released", and
+ # so we must run imagevalidate on them. Global system images need to be
+ # explicity released, but for those we must not run imagevaildate when
+ # writing the image to /proj and provenance is off, since we would overwrite
+ # the values for the image that is actually in use in /usr/testbed/images
+ #
+ my $cname = "$imagepid/$imagename";
+ $cname .= ":" . $image->version()
+ if ($doprovenance);
+ my $tbopsmsg = "";
+ my $isglobal = $image->global();
+ if ($pid ne TBOPSPID() || !$isglobal || $image->version() || $doprovenance) {
+ if (system("$IMAGEVALIDATE -u $cname") != 0) {
+ $tbopsmsg =
+ "DB state update for image $cname failed, try again with:\n".
+ " $IMAGEVALIDATE -u $cname\n";
+ }
+ }
+ elsif ($isglobal && $pid eq TBOPSPID()) {
+ $tbopsmsg =
+ "Did not update DB state for global image $cname\n".
+ "since image was written to '$filename' instead of $TB/images.\n\n".
+ "Please run imagerelease when ready for release:\n".
+ " imagerelease -q $cname\n";
+ }
+ if ($tbopsmsg) {
+ SENDMAIL($TBOPS,
+ "Image DB state update failure for $cname",
+ $tbopsmsg,
+ $TBOPS,
+ undef,
+ ());
+ }
+
+ print "$cname: ";
+ print "delta "
+ if ($delta);
+ print "image creation succeeded, written to $filename.\n";
+ # "Final size: $fsize bytes.\n";
+
+ ($rc,$msg) = $self->CreateImageFinalize($image,$target,$args);
+ goto errout
+ if ($rc);
+
+ out:
+ $rc = 0;
+ $msg = undef;
+
+ errout:
+ if (defined($hacksigfile)) {
+ unlink($hacksigfile);
+ }
+ if ($needunlock) {
+ $image->Unlock();
+ }
+ tbwarn("$self CaptureImage: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub DeleteImagePrepare($$;$) {
+ my ($self,$image,$args) = @_;
+ my $rc = -1;
+ my $msg = "";
+
+ #
+ # Be sure to kill off running frisbee. If a node is trying to load that
+ # image, well tough.
+ #
+ my $imageid = $image->imageid();
+ if (!$args->{'impotent'}) {
+ system("$FRISKILLER -k $imageid");
+ if ($?) {
+ $msg = "Could not kill running frisbee for $imageid!";
+ goto errout;
+ }
+ }
+
+ return 0;
+
+ errout:
+ tbwarn("$self DeleteImagePrepare: $msg\n")
+ if ($rc);
+ if (wantarray) {
+ return ($rc,$msg);
+ }
+ else {
+ return $rc;
+ }
+}
+
+sub DeleteImageFiles($$;$) {
+ my ($self,$image,$args) = @_;
+ if (!defined($args)) {
+ $args = {};
+ }
+ my $msg;
+ my $versonly = $args->{'versonly'} || 0;
+ my $purge = $args->{'purge'} || 0;
+ my $rename = $args->{'rename'} || 0;
+ my $impotent = $args->{'impotent'} || 0;
+
+ #
+ # Since admins will often delete image descriptors for users, we are
+ # setuid root. Flip for deleting the image file.
+ #
+ my $isdirpath = $image->IsDirPath();
+
+ #
+ # When doing image provenance, we have to deal with all versions
+ # of the image. This will not return deleted versions.
+ #
+ my @images = ();
+ if ($image->AllVersions(\@images)) {
+ $msg = "Could not get list of image (versions)";
+ goto errout;
+ }
+ #
+ # When deleting just a single version, if this is the last or only
+ # version, then turn off version only. Makes no sense to have a
+ # descriptor with no non-deleted versions.
+ #
+ if ($versonly && scalar(@images) == 1) {
+ $versonly = 0;
+ }
+ if ($versonly) {
+ @images = ($image);
+ }
+
+ #
+ # If the path is a directory, we can just do a rename on it.
+ # But not if deleting just a single image version.
+ #
+ if ($isdirpath && !$versonly) {
+ my $dirname = $image->path();
+
+ if ($purge) {
+ if ($impotent) {
+ print "Would remove directory $dirname\n" if (-e $dirname);
+ }
+ else {
+ $EUID = 0;
+ system("/bin/rm -rf $dirname");
+ if ($?) {
+ $msg = "Could not remove $dirname";
+ goto errout;
+ }
+ $EUID = $UID;
+ }
+ }
+ else {
+ my $newname = dirname($dirname) . "/" . basename($dirname) .
+ "," . $image->imageid();
+
+ if ($impotent) {
+ print "Would rename $dirname to $newname\n" if (-e $dirname);
+ }
+ else {
+ if (-e $dirname) {
+ $EUID = 0;
+ system("/bin/mv -fv $dirname $newname");
+ if ($?) {
+ $msg = "Could not rename $dirname to $newname";
+ goto errout;
+ }
+ $EUID = $UID;
+ }
+ # Hmm, need an update all versions method.
+ foreach my $imageversion (@images) {
+ # Need trailing slash!
+ $imageversion->Update({"path" => $newname . "/"});
+ }
+ }
+ }
+ #
+ # Fall into the loop below to clean up stale image versions and
+ # backup files.
+ #
+ }
+ foreach my $imageversion (@images) {
+ my @todelete = ();
+ my @torename = ();
+ my $filename = $imageversion->FullImageFile();
+
+ push(@torename, $filename);
+ push(@todelete, "$filename.bak");
+ push(@todelete, "$filename.tmp");
+ push(@torename, $imageversion->FullImageSHA1File());
+ push(@torename, $imageversion->FullImageSigFile());
+
+ # Backwards compat with non-directory image paths.
+ if ($filename ne $imageversion->DeltaImageFile()) {
+ $filename = $imageversion->DeltaImageFile();
+ push(@torename, $filename);
+ push(@todelete, "$filename.bak");
+ push(@todelete, "$filename.tmp");
+ push(@torename, $imageversion->DeltaImageSHA1File());
+ }
+
+ # We throw away versions that never came ready or released.
+ if ($purge ||
+ !($imageversion->ready() && $imageversion->released())) {
+ @todelete = (@todelete, @torename);
+ @torename = ();
+ }
+ # Throw away the slot if it never came ready or released.
+ # Only if the highest numbered version, no holes please.
+ if ($imageversion->IsNewest() &&
+ !($imageversion->ready() && $imageversion->released())) {
+ if ($impotent) {
+ my $vers = $imageversion->version();
+ print STDERR "Would kill version $vers DB state since it was ".
+ "not ready/released\n";
+ }
+ else {
+ $imageversion->PurgeVersion();
+ }
+ }
+
+ $EUID = 0;
+ foreach my $file (@todelete) {
+ if (-e $file) {
+ if ($impotent) {
+ print STDERR "Would delete $file\n";
+ next;
+ }
+ if (!unlink($file)) {
+ SENDMAIL($TBOPS,
+ "delete_image: Could not remove image file",
+ "Could not remove $file\n".
+ "Someone will need to do this by hand.\n");
+ }
+ }
+ }
+ $EUID = $UID;
+
+ #
+ # Skip renames for directory based images.
+ # Note that when deleting a single version in an image directory,
+ # we do not want to do a rename. That would be confusing if some
+ # versions were deleted and then the entire image deleted later.
+ #
+ next
+ if ($isdirpath);
+
+ #
+ # Delete with rename; move the current files out of the way
+ # so that they do not conflict with a later image of the same name.
+ # We do this by creating a subdir for the files.
+ #
+ $EUID = 0;
+ if (@torename) {
+ my $dirname = dirname($imageversion->path()) .
+ "/" . $image->imagename() . "," . $image->imageid();
+
+ if (! -e $dirname) {
+ if ($impotent) {
+ print "Would mkdir($dirname)\n";
+ }
+ elsif (! mkdir("$dirname", 0775)) {
+ $msg = "Could not mkdir $dirname";
+ goto errout;
+ }
+ }
+ foreach my $file (@torename) {
+ my $newname = $dirname . "/" . basename($file);
+
+ if ($impotent) {
+ print "Would rename $file to $newname\n" if (-e $file);
+ next;
+ }
+ if (-e $file) {
+ system("/bin/mv -fv $file $newname");
+ if ($?) {
+ $msg = "Could not rename $file to $dirname";
+ goto errout;
+ }
+ }
+ if ($file eq $filename &&
+ $imageversion->Update({"path" => $newname})) {
+ $msg = "Could not update path for $imageversion";
+ goto errout;
+ }
+ }
+ }
+ $EUID = $UID;
+ }
+
+ out:
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ tbwarn("$self DeleteImageFiles: $msg\n")
+ if (defined($msg));
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ else {
+ return -1;
+ }
+}
+
+sub Validate($$;$) {
+ my ($self,$image,$args) = @_;
+ my $msg;
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ # XXX TODO
+
+ out:
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ tbwarn("$self Validate: $msg\n");
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ return -1;
+}
+
+sub UpdateHash($$;$) {
+ my ($self,$image,$args) = @_;
+ my $msg;
+
+ my $userperm;
+ my $impotent = 0;
+ if (exists($args->{"impotent"})) {
+ $impotent = $args->{"impotent"};
+ }
+ if ($impotent) {
+ $userperm = TB_IMAGEID_MODIFYINFO();
+ }
+ else {
+ $userperm = TB_IMAGEID_READINFO();
+ }
+
+ if (!defined($image) || !ref($image)
+ || !($image->isa("Image") || $image->isa("OSImage"))) {
+ $msg = "image ($image) is not an Image!";
+ goto errout;
+ }
+
+ my $this_user;
+ if ($UID != 0) {
+ $this_user = User->ThisUser();
+ if (!defined($this_user)) {
+ $msg = "You ($UID) do not exist!";
+ goto errout;
+ }
+ if (!$image->AccessCheck($this_user,$userperm)) {
+ $msg = "$image: insufficient privilege";
+ goto errout;
+ }
+ }
+
+ my @tuples = ( [ $image->FullImageFile(),$image->hash(),
+ $image->FullImageSHA1File() ],
+ [ $image->DeltaImageFile(),$image->deltahash(),
+ $image->DeltaImageSHA1File() ] );
+
+ my $filehash;
+ foreach my $tuple (@tuples) {
+ my ($path,$hash,$hashfile) = ($tuple->[0],$tuple->[1],$tuple->[2]);
+ next
+ if (! -e $path);
+
+ $filehash = `$SHA1 $path`;
+ if ($?) {
+ $msg = "Could not generate sha1 hash of $path";
+ goto errout;
+ }
+ if ($filehash =~ /^SHA1.*= (\w*)$/) {
+ $hash = $1;
+ if (!$impotent) {
+ if ($image->SetHash($1) != 0) {
+ $msg = "Failed to set the hash for $image";
+ goto errout;
+ }
+ }
+ }
+ else {
+ $msg = "Could not parse the sha1 hash: '$filehash'";
+ goto errout;
+ }
+ if (!$impotent) {
+ unlink($hashfile)
+ if (-e $hashfile);
+ open(HASH, ">$hashfile") or
+ fatal("Could not open $hashfile for writing: $!");
+ print HASH $filehash;
+ close($hashfile);
+ }
+ }
+
+ out:
+ if (wantarray) {
+ return ($filehash,"");
+ }
+ return $filehash;
+
+ errout:
+ tbwarn("$self UpdateHash: $msg\n");
+ if (wantarray) {
+ return (undef,$msg);
+ }
+ return undef;
+}
+
+sub Release($$;$) {
+ my ($self,$image,$args) = @_;
+ my ($rc,$msg);
+
+ my ($quiet,$force,$markready,$impotent,$validate) = (0,0,0,0,1);
+ if (!defined($args)) {
+ $args = {};
+ }
+ if (exists($args->{'impotent'})) {
+ $impotent = $args->{'impotent'};
+ }
+ if (exists($args->{'quiet'})) {
+ $quiet = $args->{'quiet'};
+ }
+ if (exists($args->{'force'})) {
+ $force = $args->{'force'};
+ }
+ if (exists($args->{'markready'})) {
+ $markready = $args->{'markready'};
+ }
+ if (exists($args->{'validate'})) {
+ $validate = $args->{'validate'};
+ }
+
+ ($rc,$msg) = $self->_ReleaseChecks($image,$args);
+ if ($rc) {
+ goto errout;
+ }
+
+ if ($force && $markready) {
+ if ($impotent) {
+ print "Would mark image ready/released, but not do anything else\n"
+ if (!$quiet);
+ }
+ else {
+ print "Marking image ready/released, but not doing anything else\n"
+ if (!$quiet);
+ $image->MarkReady();
+ $image->MarkReleased();
+ }
+ goto out;
+ }
+
+ my $needunlock = 0;
+ #
+ # Grab version 0 of the descriptor, which tells us if the image is really
+ # stored on boss. At some point, it would be nice to store unreleased
+ # versions of system images on boss too, but not enough disk space to
+ # support that, so we put newer versions in /proj until they are released,
+ # and then copy them over to avoid the NFS overhead when using the image.
+ # If we have to copy them back, we might also have to update the path in
+ # the database.
+ #
+ my $updatepath = undef;
+ my %copyfiles = ();
+ my $version0;
+
+ #
+ # Before we do anything destructive, we lock the image.
+ #
+ if ($image->Lock()) {
+ $msg = "Image is locked, please try again later!";
+ goto errout;
+ }
+ $needunlock = 1;
+ if ($image->version()) {
+ $version0 = OSImage->Lookup($image->imageid(), 0);
+ if (!defined($version0)) {
+ $msg = "Cannot lookup version zero of $image";
+ goto errout;
+ }
+ }
+ else {
+ $version0 = $image;
+ }
+ if ($version0->IsSystemImage()) {
+ my $vers0src = "$PROJROOT/" . $image->pid() . "/images/";
+ if ($version0->IsDirPath()) {
+ # Add in the directory.
+ $vers0src .= basename($version0->path()) . "/";
+ }
+
+ my $dstdir = $SYSIMAGEDIR . "/";
+ if ($image->IsDirPath()) {
+ $dstdir .= basename($image->path()) . "/";
+ }
+
+ #
+ # For version 0 of the image, create_image sticks the file out on /proj,
+ # but leaves the image path set to $TB/images
+ #
+ $copyfiles{($image->version() ?
+ $image->FullImageFile() :
+ $vers0src . basename($image->FullImageFile()))} =
+ $dstdir . basename($image->FullImageFile());
+ $copyfiles{($image->version() ?
+ $image->DeltaImageFile() :
+ $vers0src . basename($image->DeltaImageFile()))} =
+ $dstdir . basename($image->DeltaImageFile());
+ $copyfiles{($image->version() ?
+ $image->FullImageSigFile() :
+ $vers0src . basename($image->FullImageSigFile()))} =
+ $dstdir . basename($image->FullImageSigFile());
+ $copyfiles{($image->version() ?
+ $image->DeltaImageSigFile() :
+ $vers0src . basename($image->FullImageSigFile()))} =
+ $dstdir . basename($image->DeltaImageSigFile());
+
+ if ($image->version()) {
+ $updatepath = $dstdir;
+ if ($image->IsDirPath()) {
+ $updatepath = $dstdir;
+ }
+ else {
+ $updatepath = $SYSIMAGEDIR . "/" . basename($image->path());
+ }
+ }
+ foreach my $from (keys(%copyfiles)) {
+ my $to = $copyfiles{$from};
+
+ next
+ if (! -e $from);
+
+ if ($impotent) {
+ print STDERR "Would copy ${FSNODE}:$from $to\n"
+ if (!$quiet);
+ next;
+ }
+ if ($self->debug()) {
+ print STDERR "Copying ${FSNODE}:$from $to\n";
+ }
+ system("$SUDO $SCP -p ${FSNODE}:$from $to");
+ if ($?) {
+ $msg = "Failed to scp ${FSNODE}:$from $to";
+ goto errout;
+ }
+ }
+ if (defined($updatepath)) {
+ if ($impotent) {
+ print STDERR "Would update path to $updatepath\n"
+ if (!$quiet);
+ }
+ elsif ($image->Update({"path" => $updatepath})) {
+ $msg = "Failed to update path";
+ goto errout;
+ }
+ }
+ #
+ # XXX: convert to $self->Validate once ported.
+ #
+ if ($validate) {
+ if ($impotent) {
+ print STDERR "Would run imagevalidate on " .
+ $image->versname() . "\n" if (!$quiet);
+ }
+ else {
+ if ($self->debug()) {
+ print STDERR "Running imagevalidate on " .
+ $image->versname() . "\n";
+ }
+ system("$IMAGEVALIDATE -u " . ($self->debug() ? " " : "-q ") .
+ $image->versname());
+ if ($?) {
+ $msg = "Failed to validate the image!";
+ goto errout;
+ }
+ }
+ }
+ }
+ # Now mark as released.
+ if ($impotent) {
+ print STDERR "Would mark image as released\n"
+ if (!$quiet);
+ }
+ elsif ($image->Release()) {
+ $msg = "Could not mark image as released!";
+ goto errout;
+ }
+
+ # If everything worked, remove the copies on ops to avoid unsightly clutter
+ if (keys(%copyfiles)) {
+ my @todelete = ();
+
+ foreach my $from (keys(%copyfiles)) {
+ push(@todelete, $from);
+ push(@todelete, $from . ".sha1");
+ }
+ if ($impotent) {
+ foreach my $file (@todelete) {
+ print STDERR "Would delete $file\n"
+ if (!$quiet);
+ }
+ }
+ else {
+ if ($self->debug()) {
+ print STDERR "Removing temporary copy on $FSNODE\n";
+ }
+ system("$SUDO $SSH ${FSNODE} rm -f @todelete");
+ }
+ }
+
+ out:
+ $image->Unlock()
+ if ($needunlock);
+ if (wantarray) {
+ return (0,"");
+ }
+ return 0;
+
+ errout:
+ $image->Unlock()
+ if ($needunlock);
+ tbwarn("$self Release: $msg\n");
+ if (wantarray) {
+ return (-1,$msg);
+ }
+ return -1;
+}
+
+1;
diff --git a/tbsetup/libosload_new.pm.in b/tbsetup/libosload_new.pm.in
index 8f6e0fdb77..baa6740a3a 100644
--- a/tbsetup/libosload_new.pm.in
+++ b/tbsetup/libosload_new.pm.in
@@ -495,6 +495,10 @@ sub osload($$$) {
%nodeflags = %{$args->{'nodeflags'}};
}
+ if (defined($args->{'debug'})) {
+ $self->debug($args->{'debug'});
+ }
+
$self->{FLAGS} = \%flags;
#
@@ -1951,10 +1955,42 @@ sub AddNode($$$$)
return 0;
}
+sub _CheckImage($$$)
+{
+ my ($self,$nodeobject,$imageid,$image) = @_;
+ my $node_id = $nodeobject->node_id();
+
+ $self->dprint(1,"_CheckImage($node_id): using $image");
+
+ #
+ # We can have both a full image and/or a delta image. We
+ # always prefer the full image if we have it.
+ #
+ if (! ($image->HaveFullImage() || $image->HaveDeltaImage())) {
+ #
+ # This should be an error, but until we run imagevalidate
+ # over all images, we want to do it here.
+ #
+ if ($self->GetImageSize($image, $nodeobject)) {
+ tberror "$nodeobject: no full or delta image file!";
+ return -1;
+ }
+ }
+ if (! ($image->size() || $image->deltasize())) {
+ tberror "$image: no size info!";
+ return -1;
+ }
+
+ $self->dprint(2,"_CheckImage($node_id): imageinfo for $imageid:\n");
+
+ return 0;
+}
+
sub _CheckImages($$)
{
my ($self,$nodeobject) = @_;
my $node_id = $nodeobject->node_id();
+ my $rc = 0;
$self->dprint(0,"_CheckImages($node_id)");
@@ -1974,27 +2010,7 @@ sub _CheckImages($$)
my $imageid = $imageids[$i];
my $image = $images[$i];
- $self->dprint(1,"_CheckImages($node_id): using $image");
-
- #
- # We can have both a full image and/or a delta image. We
- # always prefer the full image if we have it.
- #
- if (! ($image->HaveFullImage() || $image->HaveDeltaImage())) {
- #
- # This should be an error, but until we run imagevalidate
- # over all images, we want to do it here.
- #
- if ($self->GetImageSize($image, $nodeobject)) {
- tberror "$nodeobject: no full or delta image file!";
- return -1;
- }
- }
- if (! ($image->size() || $image->deltasize())) {
- tberror "$image: no size info!";
- return -1;
- }
- $self->dprint(2,"_CheckImages($node_id): imageinfo for $imageid:\n");
+ $rc += $self->_CheckImage($nodeobject,$imageid,$image);
}
return 0;
@@ -2671,6 +2687,89 @@ sub _LoadDefaultImages($$)
# return 0;
#}
+sub _CheckImageDocker($$$$)
+{
+ my ($self,$nodeobject,$imageid,$image) = @_;
+ my $node_id = $nodeobject->node_id();
+
+ $self->dprint(1,"_CheckImage($node_id): using $image");
+
+ #
+ # Three cases: the image is external (we know nothing); the image is
+ # in our local Docker repo (we know things); the image is in our
+ # federation.
+ #
+ # In the first case, we ask our local docker jail to find out how
+ # large the image is (all layers).
+ #
+
+ $self->dprint(2,"_CheckImageDocker($node_id): imageinfo for $imageid:\n");
+
+ return 0;
+}
+
+sub _CheckImage($$$$)
+{
+ my ($self,$nodeobject,$imageid,$image) = @_;
+ my $node_id = $nodeobject->node_id();
+
+ if ($image->format() eq "docker") {
+ return $self->_CheckImageDocker($nodeobject,$imageid,$image);
+ }
+ else {
+ return $self->SUPER::_CheckImage($nodeobject,$imageid,$image);
+ }
+}
+
+sub SetupReloadDocker($$$)
+{
+ my ($self,$nodeobject,$image) = @_;
+ my $node_id = $nodeobject->node_id();
+ my $osimage;
+
+ $self->dprint(0,"SetupReloadDocker($node_id): setting up reload");
+
+ my $prepare = $self->nodeflag($nodeobject,'prepare');
+ my $zerofree = 0;
+
+ #
+ # Put it in the current_reloads table so that nodes can find out which
+ # OS to load. See tmcd.
+ #
+ my $query_result =
+ DBQueryWarn("delete from current_reloads where node_id = '$node_id'");
+ return -1
+ if (!$query_result);
+
+ my $imageid = $image->imageid();
+ my $version = $image->version();
+
+ $query_result =
+ DBQueryWarn("insert into current_reloads ".
+ "(node_id, idx, image_id, imageid_version,".
+ " mustwipe, prepare) values ".
+ "('$node_id', 0, '$imageid', '$version',".
+ " $zerofree, $prepare)");
+ return -1
+ if (!$query_result);
+
+ my $osid = TBNodeDiskloadOSID($node_id);
+ $osimage = OSImage->Lookup($osid);
+
+ #
+ # We used to invoke os_select here and it checks for MODIFYINFO permission
+ # on the node. Since we have already checked for LOADIMAGE permission,
+ # which requires the same user privilege, we do not need to check further.
+ #
+ if (!defined($osimage)
+ || $nodeobject->OSSelect($osimage,"next_boot_osid",$self->debug())) {
+ tberror "$self: os_select $osid failed!";
+ return -1;
+ }
+
+ return 0;
+}
+
sub SetupReload($$)
{
my ($self,$nodeobject) = @_;
@@ -2678,9 +2777,35 @@ sub SetupReload($$)
$self->dprint(0,"SetupReload($node_id): setting up reload");
- if ($self->SUPER::SetupReload($nodeobject)) {
+ my @images = @{$self->GetImages($nodeobject)};
+ my $isdocker = 0;
+ my $isother = 0;
+ for my $image (@images) {
+ if ($image->format() eq 'docker') {
+ ++$isdocker;
+ }
+ else {
+ ++$isother;
+ }
+ }
+ if ($isdocker && $isother) {
+ tberror "$self: cannot mix Docker and non-Docker images on $node_id!";
+ return -1;
+ }
+ elsif ($isdocker > 1) {
+ tberror "$self: cannot load more than one Docker image on $node_id!";
return -1;
}
+ elsif ($isdocker) {
+ if ($self->SetupReloadDocker($nodeobject,$images[0])) {
+ return -1;
+ }
+ }
+ else {
+ if ($self->SUPER::SetupReload($nodeobject)) {
+ return -1;
+ }
+ }
# Need to kick virtnodes so stated picks up the next_op_mode from os_select
TBSetNodeEventState($node_id,TBDB_NODESTATE_SHUTDOWN);
@@ -2688,6 +2813,21 @@ sub SetupReload($$)
return 0;
}
+sub ComputeMaxLoadWaitTimeDocker($$$)
+{
+ my ($self,$nodeobject,$image) = @_;
+ my $maxwait = 0;
+
+ my @images = @{$self->GetImages($nodeobject)};
+ foreach my $image (@images) {
+ # XXX: just give it a half hour for now, until we can estimate
+ # the layer size, etc.
+ $maxwait += 1800;
+ }
+
+ return $maxwait;
+}
+
#
# Compute the time to load all images on this node.
#
@@ -2695,7 +2835,32 @@ sub ComputeMaxLoadWaitTime($$)
{
my ($self,$nodeobject) = @_;
- my $maxwait = $self->SUPER::ComputeMaxLoadWaitTime($nodeobject);
+ my $maxwait = 0;
+
+ my @images = @{$self->GetImages($nodeobject)};
+ foreach my $image (@images) {
+ if ($image->format() eq 'docker') {
+ $maxwait += $self->ComputeMaxLoadWaitTimeDocker($nodeobject,$image);
+ next;
+ }
+
+ my $imageid = $image->imageid();
+ my $isfull = $image->HaveFullImage();
+ my $imagesize = ($isfull ? $image->size() : $image->deltasize());
+ #
+ # Compute a maxwait time based on the image size plus a constant
+ # factor for the reboot cycle. This is used later in
+ # WaitTillReloadDone(). Arguably, this should be part of the
+ # image DB state, so we store it in the imageinfo array too.
+ #
+ my $chunks = $imagesize >> 20;# size may be > 2^31, shift is unsigned
+
+ # XXX aren't we multi-counting reboots??
+ # ok, moved constant factor out.
+ $maxwait += int(($chunks / 100.0) * 65);
+ }
+
+ $maxwait += $TBLOADWAIT;
#
# If it's a virtnode, we need to add a bunch of time based
diff --git a/tbsetup/libvtop_stable.pm.in b/tbsetup/libvtop_stable.pm.in
index 3fd936c264..bdc817f37d 100755
--- a/tbsetup/libvtop_stable.pm.in
+++ b/tbsetup/libvtop_stable.pm.in
@@ -1438,10 +1438,12 @@ sub LoadVirtNodes($)
$attrs->{$attr->attrkey()} = $attr->attrvalue();
if ($attr->attrkey() eq "XEN_MEMSIZE" ||
+ $attr->attrkey() eq "DOCKER_MEMSIZE" ||
$attr->attrkey() eq "MEMORYSIZE") {
$vnode->_desires()->{"?+ram"} = $attr->attrvalue();
}
- elsif ($attr->attrkey() eq "XEN_CORES") {
+ elsif ($attr->attrkey() eq "XEN_CORES" ||
+ $attr->attrkey() eq "DOCKER_CORES") {
# Overridden below for shared nodes.
$self->experiment()->SetVirtNodeAttribute($vname,
"VM_VCPUS",
@@ -1471,6 +1473,13 @@ sub LoadVirtNodes($)
$minmem = 256;
$maxmem = 1024 * 16;
}
+ elsif (defined($vnode->_parent_osinfo())
+ && ($vnode->_parent_osinfo()->FeatureSupported("docker-host"))) {
+ # || $vnode->_parent_osinfo()->osname() =~ /dock/i)) {
+ $defmem = 64;
+ $minmem = 32;
+ $maxmem = 1024 * 16;
+ }
if (!exists($vnode->_desires()->{"?+ram"})) {
$self->printdb("Setting VM memsize to $defmem for $vname\n");
$vnode->_desires()->{"?+ram"} = $defmem;
diff --git a/tbsetup/ptopgen.in b/tbsetup/ptopgen.in
index 974f6e9ce6..58382e54e2 100755
--- a/tbsetup/ptopgen.in
+++ b/tbsetup/ptopgen.in
@@ -123,6 +123,7 @@ $opstate = <<'OPSTATE';
<sliver_type name="raw-pc" />
<sliver_type name="emulab-openvz" />
<sliver_type name="emulab-xen" />
+ <sliver_type name="emulab-docker" />
<state name="geni_notready">
<action name="geni_start" next="geni_configuring">
@@ -889,6 +890,7 @@ our %node_latitudes;
our %node_longitudes;
our $openvzid;
+our $dockerid;
$result = DBQueryFatal($osidquery1);
processOs($result);
@@ -909,6 +911,9 @@ sub processOs
if ($osname eq "OPENVZ-STD") {
$openvzid = $osid;
}
+ if ($osname eq "DOCKVH-STD") {
+ $dockerid = $osid;
+ }
if ($typemap{$type}) {
my $default = $typemap{$type}->{'OSID'};
if ($geni eq 1 ||
@@ -1025,6 +1030,9 @@ while (my ($osid,$nextosid) = $result->fetchrow()) {
if (defined($openvzid) && $osid == $openvzid) {
$openvzid = $nextosid;
}
+ if (defined($dockerid) && $osid == $dockerid) {
+ $dockerid = $nextosid;
+ }
#
# Check to see if they were allowed to use the real OSID
#
@@ -1504,6 +1512,8 @@ foreach $node (@nodenames) {
push(@sliver_types, "emulab-openvz");
} elsif ($features[$i] eq 'xen-host') {
push(@sliver_types, "emulab-xen");
+ } elsif ($features[$i] eq 'docker-host') {
+ push(@sliver_types, "emulab-docker");
}
}
}
@@ -1567,6 +1577,7 @@ foreach $node (@nodenames) {
{
push(@sliver_types, "emulab-openvz");
push(@sliver_types, "emulab-xen");
+ push(@sliver_types, "emulab-docker");
}
}
push(@types, $auxinfo);
@@ -2947,6 +2958,10 @@ sub print_node_types
}
} elsif ($sliverTypes[$i] eq "emulab-xen") {
print_raw_osids("pcvm");
+ } elsif ($sliverTypes[$i] eq "emulab-docker") {
+ if (defined($dockerid)) {
+ print_osids($osid_subosids{$dockerid}, undef);
+ }
}
print " </sliver_type>\n";
}
diff --git a/utils/clone_image.in b/utils/clone_image.in
index 0ce1328ae1..21ad22d5b8 100644
--- a/utils/clone_image.in
+++ b/utils/clone_image.in
@@ -509,6 +509,9 @@ if (defined($base_image)) {
$xmlfields{"loadpart"} = $base_image->loadpart();
$xmlfields{"noexport"} = $base_image->noexport();
$xmlfields{"noclone"} = $base_image->noclone();
+ if ($base_image->format() ne 'ndz') {
+ $xmlfields{"format"} = $base_image->format();
+ }
# Short form uses wholedisk instead. Should fix this.
if ($base_image->loadpart() == 0 && $base_image->loadlength() == 4) {
diff --git a/utils/create_image.in b/utils/create_image.in
index 2f81486628..c725a09706 100644
--- a/utils/create_image.in
+++ b/utils/create_image.in
@@ -344,6 +344,17 @@ if (defined($options{"A"})) {
if (defined($options{"F"})) {
$nodelta = 1;
}
+if (defined($options{"p"})) {
+ $imagepid = $options{"p"};
+
+ if ($imagepid =~ /^([-\w\.]+)$/) {
+ $imagepid = $1;
+ }
+ else {
+ die("*** $0:\n".
+ " Bad data in $imagepid.\n");
+ }
+}
if (@ARGV != 2) {
usage();
}
@@ -358,16 +369,71 @@ my $target = $ARGV[1];
#
$EUID = $UID;
-if (defined($options{"p"})) {
- $imagepid = $options{"p"};
-
- if ($imagepid =~ /^([-\w\.]+)$/) {
- $imagepid = $1;
+#
+# See if we should use libimageops instead. Note that
+# libimageops::CreateImage handles all the permissions checks, etc, so
+# no problem cutting to it straight after arg processing.
+#
+my $usenew = 0;
+my $newtarget;
+if ($target =~ /^([-\w]+)$/) {
+ $target = $1;
+
+ $newtarget = Node->Lookup($target);
+ if (defined($newtarget) && $newtarget->isvirtnode()) {
+ #
+ # Need to know this is a docker-host.
+ #
+ my $pnode = Node->Lookup($newtarget->phys_nodeid());
+ my $osimage = OSImage->Lookup($pnode->def_boot_osid());
+ if (defined($osimage) && $osimage->FeatureSupported("docker-host")) {
+ $usenew = 1;
+ }
}
- else {
- die("*** $0:\n".
- " Bad data in $imagepid.\n");
+}
+if ($usenew) {
+ my $iops = libimageops::Factory("image" => $imagename, "node" => $newtarget,
+ "imagepid" => $imagepid);
+ if (!defined($iops)) {
+ print STDERR "*** $0:\n$@\n";
+ exit(-1);
+ }
+
+ # Set up the argv:
+ my %args = (
+ 'debug' => 1,
+ 'waitmode' => $waitmode,
+ 'usessh' => $usessh,
+ 'usenfs' => $usenfs,
+ 'usefup' => $usefup,
+ 'noemail' => $noemail,
+ 'delta' => $delta,
+ 'nodelta' => $nodelta,
+ 'nomfs' => $nomfs,
+ 'quiet' => $quiet,
+ 'signature' => $signature,
+ 'deltapct' => $deltapct,
+ 'update_prepare' => $update_prepare,
+ 'imagepid' => $imagepid,
+ 'bsname' => $bsname,
+ 'origin_uuid' => $origin_uuid,
+ 'webtask' => $webtask,
+ );
+ if ($debug) {
+ use Data::Dumper;
+ print STDERR "create_image libimageops::CreateImage args: " .
+ Dumper(%args) . "\n";
+ }
+
+ #
+ # Create the image. Library does all the work.
+ #
+ my ($rc,$err) = $iops->CreateImage($imagename,$target,\%args);
+ if ($rc) {
+ print STDERR "*** $0:\n$err\n";
+ exit($rc);
}
+ exit(0);
}
#
diff --git a/utils/delete_image.in b/utils/delete_image.in
index 0f693105a5..32f77c70f9 100644
--- a/utils/delete_image.in
+++ b/utils/delete_image.in
@@ -34,6 +34,7 @@ sub usage()
{
print("Usage: delete_image [[-f | -F] -p | -r] <imagename>\n".
"Options:\n".
+ " -d Enable debug messages\n".
" -p Purge the disk image file(s)\n".
" -r Rename the disk image file(s) instead (default)\n".
" -n Impotent mode, show what would be done.\n".
@@ -138,6 +139,36 @@ usage()
if (! ($purge || $rename));
my $imageid = shift(@ARGV);
+my $image = OSImage->Lookup($imageid);
+if (!defined($image)) {
+ fatal("Image does not exist in the DB!");
+}
+
+#
+# See if we should use libimageops instead. Eventually it will be
+# a feature as well as a few special cases; but certain node types may
+# require it (i.e. Docker).
+#
+my $usenew = 0;
+if ($image->format() eq 'docker') {
+ $usenew = 1;
+}
+if ($usenew) {
+ use libimageops;
+
+ libimageops::setDebug(1)
+ if (1 || $debug);
+ my %args = ( 'purge' => $purge,'rename' => $rename,
+ 'impotent' => $impotent,'versonly' => $versonly,
+ 'force' => $force,'force_global' => $FORCE );
+
+ my $iops = libimageops::Factory("image" => $image);
+ my ($rc,$msg) = $iops->DeleteImage($image,\%args);
+ if ($rc) {
+ print STDERR "Error: $msg\n";
+ }
+ exit($rc);
+}
#
# Map invoking user to object.
@@ -146,10 +177,6 @@ my $this_user = User->ThisUser();
if (! defined($this_user)) {
fatal("You ($UID) do not exist!");
}
-my $image = OSImage->Lookup($imageid);
-if (!defined($image)) {
- fatal("Image does not exist in the DB!");
-}
if (!$image->AccessCheck($this_user, TB_IMAGEID_DESTROY())) {
fatal("You do not have permission to delete this image!");
}
diff --git a/utils/dumpdescriptor.in b/utils/dumpdescriptor.in
index cd865f1091..bae598d53c 100644
--- a/utils/dumpdescriptor.in
+++ b/utils/dumpdescriptor.in
@@ -150,6 +150,7 @@ sub DumpImage($)
my @imagelist = ();
$xmlfields{"imagename"} = $image->imagename();
+ $xmlfields{"format"} = $image->format();
if (!$export) {
$xmlfields{"pid"} = $image->pid();
$xmlfields{"gid"} = $image->gid();
diff --git a/utils/image_import.in b/utils/image_import.in
index a7f5fe289d..7b08d86333 100644
--- a/utils/image_import.in
+++ b/utils/image_import.in
@@ -456,29 +456,54 @@ if ($update && $newhash ne $image->hash()) {
# get a new copy.
#
if ($getimage) {
- # Run as root to access /proj
- $EUID = $UID = 0;
- if (! -e $image->FullImageFile() || $newhash ne $image->hash() || $force) {
- # Make sure the path directory exists.
- if ($image->CreateImageDir()) {
+ my $downloaded = 0;
+ my $versname = $image->versname();
+
+ if ($image->format() eq 'docker') {
+ use libimageops;
+
+ libimageops::setDebug($debug);
+ my %args = (
+ 'origin_path' => $xmlparse->{'attribute'}->{'path'}->{'value'},
+ 'downloaded_ref' => \$downloaded,'force' => $force,
+ 'newhash' => $newhash,'locked' => 1 );
+ my $iops = libimageops::Factory("image" => $image);
+ my ($rc,$msg) = $iops->ImportImageContent($image,\%args);
+ if ($rc) {
+ print STDERR "Error: $msg\n";
$image->Unlock();
exit(1);
}
- $EUID = $UID = $SAVEUID;
-
- if (DownLoadImage($image, $newhash, $user, $group)) {
- $image->Unlock();
- exit(1);
- }
- # Update DB info.
- my $versname = $image->versname();
+ }
+ else {
# Run as root to access /proj
$EUID = $UID = 0;
- if (system("$IMAGEVALIDATE -u $versname")) {
- # XXX should this be fatal?
- print STDERR "Could not update DB info for $image\n";
+ if (! -e $image->FullImageFile() || $newhash ne $image->hash() || $force) {
+ # Make sure the path directory exists.
+ if ($image->CreateImageDir()) {
+ $image->Unlock();
+ exit(1);
+ }
+ $EUID = $UID = $SAVEUID;
+
+ if (DownLoadImage($image, $newhash, $user, $group)) {
+ $image->Unlock();
+ exit(1);
+ }
+ # Update DB info.
+ # Run as root to access /proj
+ $EUID = $UID = 0;
+ if (system("$IMAGEVALIDATE -u $versname")) {
+ # XXX should this be fatal?
+ print STDERR "Could not update DB info for $image\n";
+ }
+ $EUID = $UID = $SAVEUID;
+
+ $downloaded = 1;
}
- $EUID = $UID = $SAVEUID;
+ }
+
+ if ($downloaded) {
$image->MarkReady();
$image->Release();
# Its more important to know when we brought the new version in.
diff --git a/utils/imagehash.in b/utils/imagehash.in
index 34160b632c..f77d4bca62 100644
--- a/utils/imagehash.in
+++ b/utils/imagehash.in
@@ -54,6 +54,8 @@ my $SHA1 = "/sbin/sha1";
# Protos
sub fatal($);
+sub ShowHash($);
+sub UpdateHash($);
#
# Untaint the path
@@ -99,7 +101,43 @@ if (!defined($image)) {
}
my $imageid = $image->imageid();
-sub UpdateHash($)
+#
+# Show the full hash if that's all we're doing.
+#
+if ($showhash) {
+ ShowHash("full");
+ exit(0);
+}
+
+#
+# See if this is a Docker image; use libimageops if so.
+#
+if ($image->format() eq 'docker') {
+ use libimageops;
+
+ libimageops::setDebug(1)
+ if ($debug);
+ my %args = ('impotent' => $impotent,'showhash' => $showhash);
+
+ my $iops = libimageops::Factory("image" => $image);
+ my ($rc,$msg) = $iops->UpdateHash($image,\%args);
+ if (!defined($rc) && defined($msg)) {
+ print STDERR "Error: $msg\n";
+ exit(1);
+ }
+ else {
+ print "$rc\n";
+ exit(0);
+ }
+}
+else {
+ UpdateHash("full");
+ UpdateHash("delta");
+
+ exit(0);
+}
+
+sub ShowHash($)
{
my ($which) = @_;
my ($path, $hash);
@@ -114,12 +152,26 @@ sub UpdateHash($)
$hash = $image->deltahash();
$hashfile = $image->DeltaImageSHA1File()
}
- if ($showhash) {
- $hash = "No hash yet" if (!defined($hash));
-
- print "$path = $hash\n";
- exit(0);
+ $hash = "No hash yet" if (!defined($hash));
+ print "$path = $hash\n";
+}
+
+sub UpdateHash($)
+{
+ my ($which) = @_;
+ my ($path, $hash);
+
+ if ($which eq "full") {
+ $path = $image->FullImageFile();
+ $hash = $image->hash();
+ $hashfile = $image->FullImageSHA1File()
}
+ else {
+ $path = $image->DeltaImageFile();
+ $hash = $image->deltahash();
+ $hashfile = $image->DeltaImageSHA1File()
+ }
+
return
if (! -e $path);
@@ -147,10 +199,6 @@ sub UpdateHash($)
}
print "$path = $hash\n";
}
-UpdateHash("full");
-UpdateHash("delta");
-
-exit(0);
sub fatal($)
{
diff --git a/utils/imagerelease.in b/utils/imagerelease.in
index 7a119faca8..0ed68174cc 100644
--- a/utils/imagerelease.in
+++ b/utils/imagerelease.in
@@ -1,6 +1,6 @@
#!/usr/bin/perl -w
#
-# Copyright (c) 2014-2017 University of Utah and the Flux Group.
+# Copyright (c) 2014-2018 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
@@ -124,10 +124,36 @@ if (!defined($image)) {
if (!defined($image)) {
fatal("No such image!");
}
+
+#
+# See if this is a Docker image; use libimageops if so.
+#
+if ($image->format() eq 'docker') {
+ use libimageops;
+
+ libimageops::setDebug(1)
+ if ($debug);
+ my %args = ('impotent' => $impotent,'quiet' => $quiet,
+ 'force' => $force,'markready' => $markready);
+
+ my $iops = libimageops::Factory("image" => $image);
+ my ($rc,$msg) = $iops->Release($image,\%args);
+ if ($rc) {
+ print STDERR "Error: $msg\n";
+ exit(1);
+ }
+ else {
+ exit(0);
+ }
+}
+
if ($image->released() && !$force) {
fatal("Image is already released! ".
"Maybe you need to provide imageid:version");
}
+if ($UID && !$user->IsAdmin()) {
+ fatal("Only admins can release an image.");
+}
if ($force && $markready) {
if ($impotent) {
print "Would mark image ready/released, but not do anything else\n";
@@ -142,9 +168,6 @@ if ($force && $markready) {
if (!$image->ready()) {
fatal("Image is not ready yet!");
}
-if ($UID && !$user->IsAdmin()) {
- fatal("Only admins can release an image.");
-}
#
# Grab version 0 of the descriptor, which tells us if the image is really
diff --git a/utils/imagevalidate.in b/utils/imagevalidate.in
index df272220f0..f64ea8ab5b 100644
--- a/utils/imagevalidate.in
+++ b/utils/imagevalidate.in
@@ -668,6 +668,32 @@ sub doimage($)
}
my $imageid = $image->imageid();
+ #
+ # See if this is a Docker image; use libimageops if so.
+ #
+ if ($image->format() eq 'docker') {
+ use libimageops;
+
+ libimageops::setDebug(1)
+ if ($debug);
+ my %args = (
+ 'update' => $update,'fastupdate' => $fastupdate,
+ 'quiet' => $quiet,'allvers' => $allvers,'nouser' => $nouser,
+ 'newhash' => $newhash,'dodeleted' => $dodeleted,
+ 'validate' => \%validate
+ );
+
+ my $iops = libimageops::Factory("image" => $image);
+ my ($rc,$msg) = $iops->Validate($image,\%args);
+ if ($rc && defined($msg)) {
+ print STDERR "Error: $msg\n";
+ return 1;
+ }
+ else {
+ return 0;
+ }
+ }
+
#
# If the user is not an admin, must have perm on the image.
#
--
GitLab