Commit 2641af4d authored by Leigh Stoller's avatar Leigh Stoller

Reorg of two aspects of node update.

* install-rpm, install-tarfile, spewrpmtar.php3, spewrpmtar.in: Pumped
  up even more! The db file we store in /var/db now records both the
  timestamp (of the file, or if remote the install time) and the MD5
  of the file that was installed. Locally, we can get this info when
  accessing the file via NFS (copymode on or off). Remote, we use wget
  to get the file, and so pass the timestamp along in the URL request,
  and let spewrpmtar.in determine if the file has changed. If the
  timestamp it gets is >= to the timestamp of the file, an error code
  of 304 (Not Modifed) is returned. Otherwise the file is returned.

  If the timestamps are different (remote, server sends back an actual
  file), the MD5 of the file is compared against the value stored. If
  they are equal, update the timestamp in the db file to avoid
  repeated MD5s (or server downloads) in the future. If the MD5 is
  different, then reinstall the tarball or rpm, and update the db file
  with the new timestamp and MD5. Presto, we have auto update capability!

  Caveat: I pass along the old MD5 in the URL, but it is currently
  ignored. I do not know if doing the MD5 on the server is a good
  idea, but obviously it is easy to add later. At the moment it
  happens on the node, which means wasted bandwidth when the timestamp
  has changed, but the file has not (probably not something that will
  happen in typical usage).

  Caveat: The timestamp used on remote nodes is the time the tarfile
  is installed (GM time of course). We could arrange to return the
  timestamp of the local file back to the node, but that would mean
  complicating the protocol (or using an http header) and I was not in
  the mood for that. In typical usage, I do not think that people will
  be changing tarfiles and rpms so rapidly that this will make a
  difference, but if it does, we can change it.

* node_update.in, client side watchdog, and various web pages:
  Deflated node_update, removing all of the older ssh code. We now
  assume that all nodes will auto update on a periodic basis, via the
  watchdog that runs on all client nodes, including plab nodes.

  Changed the permission check to look for new UPDATE permission (used
  to be UPDATEACCOUNT). As before, it requires local_root or better.
  The reason for this is that node_update now implies more than just
  updating the accounts/mounts. The web pages have been changed to
  explain that in addition to mounts/accounts, rpms and tarfiles will
  also be updated. At the moment, this is still tied to a single
  variable (update_accounts) in the nodes table, but as Kirk requested
  at the meeting, it will probably be nice to split these out in the
  future.

  Added the ability to node_update a single node in an experiment (in
  addition to all nodes option on the showexp page). This has been
  added to the shownode webpage menu options.

  Changed locking code to use the newer wrapper states, and to move
  the experiment to RUNNING_LOCKED until the update completes. This is
  to prevent mayhem in the rest of the system (which could be dealt
  with, but is not worth the trouble; people have to wait until their
  initiated update is complete, before they can swap out the
  experiment).

  Added "short" mode to shownode routine, equiv to the recently added
  short mode for showexp. I use this on the confirmation page for
  updating a single node, giving the user a couple of pertinent (feel
  good) facts before they comfirm.
parent fc0a4179
......@@ -42,7 +42,7 @@ use Exporter;
USERSTATUS_ACTIVE USERSTATUS_FROZEN
USERSTATUS_UNAPPROVED USERSTATUS_UNVERIFIED USERSTATUS_NEWUSER
TB_EXPT_READINFO TB_EXPT_MODIFY TB_EXPT_DESTROY
TB_EXPT_READINFO TB_EXPT_MODIFY TB_EXPT_DESTROY TB_EXPT_UPDATE
TB_EXPT_MIN TB_EXPT_MAX
TB_PROJECT_READINFO TB_PROJECT_MAKEGROUP
......@@ -352,8 +352,9 @@ sub TB_USERINFO_MAX() { TB_USERINFO_MODIFYINFO; }
sub TB_EXPT_READINFO() { 1; }
sub TB_EXPT_MODIFY() { 2; }
sub TB_EXPT_DESTROY() { 3; }
sub TB_EXPT_UPDATE() { 4; }
sub TB_EXPT_MIN() { TB_EXPT_READINFO; }
sub TB_EXPT_MAX() { TB_EXPT_DESTROY; }
sub TB_EXPT_MAX() { TB_EXPT_UPDATE; }
# Projects.
sub TB_PROJECT_READINFO() { 1; }
......
......@@ -10,6 +10,7 @@ OBJDIR = ..
SUBDIR = os
LBINDIR = $(DESTDIR)/usr/local/bin
SYSTEM := $(shell uname -s)
include $(OBJDIR)/Makeconf
......@@ -39,8 +40,10 @@ client-install:
$(INSTALL) -m 755 -o root -g wheel -d $(LBINDIR)
$(INSTALL_PROGRAM) $(SRCDIR)/install-tarfile $(LBINDIR)/install-tarfile
$(INSTALL_PROGRAM) $(SRCDIR)/install-rpm $(LBINDIR)/install-rpm
ifneq ($(SYSTEM),Linux)
$(INSTALL_PROGRAM) $(SRCDIR)/create-image $(LBINDIR)/create-image
$(MAKE) -C imagezip client-install
endif
$(MAKE) -C syncd client-install
remote-install:
......
#!/usr/bin/perl -wT
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2003 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
use Getopt::Std;
use POSIX 'setsid';
use POSIX qw(mktime);
# Drag in path stuff so we can find emulab stuff.
# XXX Temporary until I have the new tmcc library finished!
BEGIN { require "/etc/emulab/paths.pm"; import emulabpaths; }
#
......@@ -54,15 +51,22 @@ my $copymode = 0;
my $debug = 0;
my $copyfile;
my $nodeid;
my $keyhash;
my $filemd5;
my @identlines = ();
my $installed = 0;
#
# Load the OS independent support library. It will load the OS dependent
# library and initialize itself.
#
use libsetup;
use libtmcc;
# Protos
sub GetRPMFile($$$);
sub GetRPMFile($$$$$$$);
sub GetMD5($);
sub WriteIdentFile();
#
# Must be running as root to work.
......@@ -112,14 +116,39 @@ else {
}
#
# Check to make sure this RPM has not already been installed. Update
# the file now. If the rpm fails, we got big problems.
#
# Check to make sure this rpm has not already been installed.
# If so, we get the old timestamp and md5 so we can compare against
# current ones.
# We need to update the stamp/md5 in place in case it has changed, so
# copy out all the identlines so we can write them back later. We do not
# copyout the current one of course; we make up a new line at the end
# of this script based on the new info.
#
if (-e $IDENTFILE) {
if (! system("egrep -q -s '^${rpm}' $IDENTFILE")) {
print STDOUT "RPM $rpm has already been installed!\n";
exit(1);
if (!open(IDENT, $IDENTFILE)) {
fatal("Could not open $IDENTFILE: $!");
}
while (<IDENT>) {
if ($_ =~ /^([-\w\.\/]*) ([\d]*) ([\w]*)$/) {
my $file = $1;
my $stamp= $2;
my $md5 = $3;
if ($file eq $rpm) {
#
# Save the info and continue;
#
$oldstamp = $stamp;
$oldmd5 = $md5;
next;
}
push(@identlines, "$file $stamp $md5");
}
else {
warn("*** WARNING: Bad line in $IDENTFILE: $_\n");
}
}
close(IDENT);
}
#
......@@ -133,6 +162,28 @@ if (! $copymode) {
if (! -r $rpm) {
fatal("$rpm does not exist or is not accessible!");
}
#
# Compare timestamp. If no change, we are done.
#
(undef,undef,undef,undef,undef,undef,
undef,undef,undef,$filestamp) = stat($rpm);
if (defined($oldstamp) && $oldstamp >= $filestamp) {
print STDOUT "RPM $rpm has already been installed!\n";
exit(1);
}
#
# Otherwise compare MD5.
#
$filemd5 = GetMD5($rpm);
if (defined($oldmd5) && $filemd5 eq $oldmd5) {
print STDOUT "RPM $rpm has already been installed!\n";
# Must write a new ident file to avoid repeated checks.
push(@identlines, "$rpm $filestamp $filemd5");
WriteIdentFile();
exit(1);
}
}
else {
$copyfile = `mktemp /var/tmp/rpm.XXXXXX`;
......@@ -143,18 +194,29 @@ else {
else {
die("Bad data in copyfile name: $copyfile");
}
GetRPMFile($rpm, $copyfile, $usewget);
#
# Dies on any failure!
# Dies on any failure.
# Returns >0 if server copy has not been modifed.
# Returns =0 if okay to install, and gives us new stamp/md5.
#
if (GetRPMFile($rpm, $copyfile, $usewget,
$oldstamp, $oldmd5, \$filestamp, \$filemd5)) {
print STDOUT "RPM $rpm has already been installed!\n";
if (defined($filestamp) && $filestamp != $oldstamp) {
# Must write a new ident file to avoid repeated checks.
push(@identlines, "$rpm $filestamp $oldmd5");
WriteIdentFile();
}
unlink($copyfile)
if (-e $copyfile);
exit(1);
}
}
#
# Add to index first; if fails too bad.
#
if (system("echo \"$rpm\" >> $IDENTFILE")) {
fatal("Could not update $IDENTFILE");
}
# Okay, add new info to the list for update.
#
push(@identlines, "$rpm $filestamp $filemd5");
#
# Run the RPM.
......@@ -162,12 +224,18 @@ if (system("echo \"$rpm\" >> $IDENTFILE")) {
if ($copymode) {
$rpm = $copyfile;
}
system("rpm -i $rpm");
system("rpm -U --force --nodeps $rpm");
$exit_status = $? >> 8;
if ($copymode) {
unlink($copyfile);
}
#
# Recreate the index file if the install was okay.
#
if (!$exit_status) {
WriteIdentFile();
}
exit($exit_status);
sub fatal {
......@@ -183,19 +251,33 @@ sub fatal {
#
# Get an RPM from the server via tmcc and stash.
#
sub GetRPMFile($$$)
sub GetRPMFile($$$$$$$)
{
my ($rpm, $copyfile, $usewget) = @_;
my ($rpm, $copyfile, $usewget,
$oldstamp, $oldmd5, $filestamp, $filemd5) = @_;
my $buf;
my $bytelen;
#
# If copying via NFS, must watch for read errors and retry.
#
if (! $usewget) {
#
# Compare timestamp. If no change, we are done.
#
my (undef,undef,undef,undef,undef,undef,undef,$bytelen,
undef,$stamp) = stat($rpm);
if (defined($oldstamp) && $oldstamp >= $stamp) {
print STDOUT "Timestamp ($stamp) for $rpm unchanged!\n"
if ($debug);
return 1;
}
# Must do this for caller so that if the MD5 has not changed,
# the caller can update the timestamp in the ident file.
$$filestamp = $stamp;
open(TMCC, "< $rpm")
or fatal("Could not open rpm on server!");
$bytelen = (stat($rpm))[7];
#
# Open the target file and start dumping the data in.
......@@ -237,6 +319,17 @@ sub GetRPMFile($$$)
}
close(JFILE);
close(TMCC);
#
# Compare md5.
#
my $md5 = GetMD5($copyfile);
if (defined($oldmd5) && $oldmd5 eq $md5) {
print STDOUT "MD5 ($md5) for $rpm unchanged!\n"
if ($debug);
return 2;
}
$$filemd5 = $md5;
}
else {
#
......@@ -274,10 +367,8 @@ sub GetRPMFile($$$)
#
# Lastly, need our boss node.
#
my ($www) = split(" ", `tmcc bossinfo`);
die("Could not get bossinfo!")
if ($?);
#
my ($www) = tmccbossname();
if ($www =~ /^[-\w]+\.(.*)$/) {
$www = "www.${1}";
......@@ -285,22 +376,120 @@ sub GetRPMFile($$$)
else {
fatal("Tainted bossinfo $www!");
}
$www = "https://${www}";
#$www = "https://${www}/dev/stoller";
#$www = "http://golden-gw.ballmoss.com:9876/~stoller/testbed";
#
# Okay, run wget with the proper arguments.
#
my $cmd = "wget -q -O $copyfile ".
my $cmd = "wget -nv -O $copyfile ".
($debug ? "--server-response " : "") .
"'https://${www}/spewrpmtar.php3".
"?nodeid=${nodeid}&file=${rpm}&key=${keyhash}'";
"'${www}/spewrpmtar.php3".
"?nodeid=${nodeid}&file=${rpm}&key=${keyhash}".
(defined($oldstamp) ? "&stamp=$oldstamp" : "") .
(defined($oldmd5) ? "&md5=$oldmd5" : "") .
"'";
if ($debug) {
print STDERR "$cmd\n";
}
system($cmd);
fatal("Could not retrieve $rpm from $www")
if ($?);
#
# We need to read back the response to see if the file was
# unchanged. This is dumb; why doesn't wget exit with reasonable
# error codes?
#
my $nochange = 0;
if (!open(WGET, "$cmd 2>&1 |")) {
fatal("Cannot start wget: $!\n");
}
while (<WGET>) {
print $_
if ($debug);
# Ick!
if ($_ =~ /^.* ERROR 304.*$/i) {
$nochange = 1;
}
}
if (! close(WGET)) {
if ($?) {
fatal("Could not retrieve $rpm from $www")
if (!$nochange);
# Otherwise, not modifed.
print STDOUT "Timestamp for $rpm unchanged!\n"
if ($debug);
return 1;
}
else {
fatal("Error closing wget pipe: $!\n");
}
}
# Must do this for caller so that if the MD5 has not changed,
# the caller can update the timestamp in the ident file.
#
# Always use GM time for this. The server expects it.
$$filestamp = mktime(gmtime(time()));
#
# We got a file. Compare the MD5 now.
#
my $md5 = GetMD5($copyfile);
if (defined($oldmd5) && $oldmd5 eq $md5) {
print STDOUT "MD5 ($md5) for $rpm unchanged!\n"
if ($debug);
return 2;
}
$$filemd5 = $md5;
}
return 0;
}
#
# Get MD5 of file.
#
sub GetMD5($)
{
my ($file) = @_;
my $md5;
if ($OSNAME eq "linux") {
$md5 = `md5sum $file`;
if ($md5 =~ /^([\w]*)\s*.*$/) {
$md5 = $1;
}
else {
fatal("Bad MD5 for $file: $md5.");
}
}
elsif ($OSNAME eq "freebsd") {
$md5 = `md5 -q $file`;
if ($md5 =~ /^([\w]*)$/) {
$md5 = $1;
}
else {
fatal("Bad MD5 for $file: $md5.");
}
}
else {
fatal("Do not know how to compute MD5s!");
}
return $md5;
}
#
# Recreate the ident file.
#
sub WriteIdentFile()
{
if (!open(IDENT, "> $IDENTFILE")) {
fatal("Could not open $IDENTFILE for writing: $!");
}
foreach my $id (@identlines) {
print IDENT "$id\n";
}
close(IDENT);
}
#!/usr/bin/perl -wT
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2003 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
use Getopt::Std;
use POSIX 'setsid';
use POSIX qw(mktime);
# Drag in path stuff so we can find emulab stuff.
# XXX Temporary until I have the new tmcc library finished!
BEGIN { require "/etc/emulab/paths.pm"; import emulabpaths; }
#
......@@ -58,15 +55,23 @@ my $debug = 0;
my $copyfile;
my $nodeid;
my $keyhash;
my $filemd5;
my $filestamp;
my $oldmd5;
my $oldstamp;
my @identlines = ();
#
# Load the OS independent support library. It will load the OS dependent
# library and initialize itself.
#
use libsetup;
use libtmcc;
# Protos
sub GetTarFile($$$);
sub GetTarFile($$$$$$$);
sub GetMD5($);
sub WriteIdentFile();
#
# Must be running as root to work.
......@@ -138,14 +143,39 @@ if (! -d $installdir) {
}
#
# Check to make sure this tarfile has not already been installed. Update
# the file now. If the tar fails, we got big problems.
#
# Check to make sure this tarfile has not already been installed.
# If so, we get the old timestamp and md5 so we can compare against
# current ones.
# We need to update the stamp/md5 in place in case it has changed, so
# copy out all the identlines so we can write them back later. We do not
# copyout the current one of course; we make up a new line at the end
# of this script based on the new info.
#
if (-e $IDENTFILE) {
if (! system("egrep -q -s '^${tarfile}' $IDENTFILE")) {
print STDOUT "Tarfile $tarfile has already been installed!\n";
exit(1);
if (!open(IDENT, $IDENTFILE)) {
fatal("Could not open $IDENTFILE: $!");
}
while (<IDENT>) {
if ($_ =~ /^([-\w\.\/]*) ([\d]*) ([\w]*)$/) {
my $file = $1;
my $stamp= $2;
my $md5 = $3;
if ($file eq $tarfile) {
#
# Save the info and continue;
#
$oldstamp = $stamp;
$oldmd5 = $md5;
next;
}
push(@identlines, "$file $stamp $md5");
}
else {
warn("*** WARNING: Bad line in $IDENTFILE: $_\n");
}
}
close(IDENT);
}
#
......@@ -159,6 +189,28 @@ if (! $copymode) {
if (! -r $tarfile) {
fatal("$tarfile does not exist or is not accessible!");
}
#
# Compare timestamp. If no change, we are done.
#
(undef,undef,undef,undef,undef,undef,
undef,undef,undef,$filestamp) = stat($tarfile);
if (defined($oldstamp) && $oldstamp >= $filestamp) {
print STDOUT "Tarfile $tarfile has already been installed!\n";
exit(1);
}
#
# Otherwise compare MD5.
#
$filemd5 = GetMD5($tarfile);
if (defined($oldmd5) && $filemd5 eq $oldmd5) {
print STDOUT "Tarfile $tarfile has already been installed!\n";
# Must write a new ident file to avoid repeated checks.
push(@identlines, "$tarfile $filestamp $filemd5");
WriteIdentFile();
exit(1);
}
}
else {
$copyfile = `mktemp /var/tmp/tarball.XXXXXX`;
......@@ -169,18 +221,29 @@ else {
else {
die("Bad data in copyfile name: $copyfile");
}
GetTarFile($tarfile, $copyfile, $usewget);
#
# Dies on any failure!
# Dies on any failure.
# Returns >0 if server copy has not been modifed.
# Returns =0 if okay to install, and gives us new stamp/md5.
#
if (GetTarFile($tarfile, $copyfile, $usewget,
$oldstamp, $oldmd5, \$filestamp, \$filemd5)) {
print STDOUT "Tarfile $tarfile has already been installed!\n";
if (defined($filestamp) && $filestamp != $oldstamp) {
# Must write a new ident file to avoid repeated checks.
push(@identlines, "$tarfile $filestamp $oldmd5");
WriteIdentFile();
}
unlink($copyfile)
if (-e $copyfile);
exit(1);
}
}
#
# Add to index first; if fails too bad.
#
if (system("echo \"$tarfile\" >> $IDENTFILE")) {
fatal("Could not update $IDENTFILE");
}
# Okay, add new info to the list for update.
#
push(@identlines, "$tarfile $filestamp $filemd5");
#
# Figure what decompression flag is required, based on file extension.
......@@ -211,6 +274,12 @@ if ($copymode) {
unlink($copyfile);
}
#
# Recreate the index file if the install was okay.
#
if (!$exit_status) {
WriteIdentFile();
}
exit($exit_status);
sub fatal {
......@@ -226,19 +295,33 @@ sub fatal {
#
# Get a tarfile from the server via tmcc and stash.
#
sub GetTarFile($$$)
sub GetTarFile($$$$$$$)
{
my ($tarfile, $copyfile, $usewget) = @_;
my ($tarfile, $copyfile, $usewget,
$oldstamp, $oldmd5, $filestamp, $filemd5) = @_;
my $buf;
my $bytelen;
#
# If copying via NFS, must watch for read errors and retry.
#
if (! $usewget) {
#
# Compare timestamp. If no change, we are done.
#
my (undef,undef,undef,undef,undef,undef,undef,$bytelen,
undef,$stamp) = stat($tarfile);
if (defined($oldstamp) && $oldstamp >= $stamp) {
print STDOUT "Timestamp ($stamp) for $tarfile unchanged!\n"
if ($debug);
return 1;
}
# Must do this for caller so that if the MD5 has not changed,
# the caller can update the timestamp in the ident file.
$$filestamp = $stamp;
open(TMCC, "< $tarfile")
or fatal("Could not open tarfile on server!");
$bytelen = (stat($tarfile))[7];
#
# Open the target file and start dumping the data in.
......@@ -280,6 +363,17 @@ sub GetTarFile($$$)
}
close(JFILE);
close(TMCC);
#
# Compare md5.
#
my $md5 = GetMD5($copyfile);
if (defined($oldmd5) && $oldmd5 eq $md5) {
print STDOUT "MD5 ($md5) for $tarfile unchanged!\n"
if ($debug);
return 2;
}
$$filemd5 = $md5;
}
else {
#
......@@ -317,10 +411,8 @@ sub GetTarFile($$$)
#
# Lastly, need our boss node.
#
my ($www) = split(" ", `tmcc bossinfo`);
die("Could not get bossinfo!")
if ($?);
#
my ($www) = tmccbossname();
if ($www =~ /^[-\w]+\.(.*)$/) {
$www = "www.${1}";
......@@ -328,22 +420,121 @@ sub GetTarFile($$$)
else {
fatal("Tainted bossinfo $www!");
}
$www = "https://${www}";
#$www = "https://${www}/dev/stoller";
#$www = "http://golden-gw.ballmoss.com:9876/~stoller/testbed";
#
# Okay, run wget with the proper arguments.
#
my $cmd = "wget -q -O $copyfile ".
my $cmd = "wget -nv -O $copyfile ".
($debug ? "--server-response " : "") .
"'https://${www}/spewrpmtar.php3".
"?nodeid=${nodeid}&file=${tarfile}&key=${keyhash}'";
"'${www}/spewrpmtar.php3".
"?nodeid=${nodeid}&file=${tarfile}&key=${keyhash}" .
(defined($oldstamp) ? "&stamp=$oldstamp" : "") .
(defined($oldmd5) ? "&md5=$oldmd5" : "") .
"'";
if ($debug) {
print STDERR "$cmd\n";
}
system($cmd);
fatal("Could not retrieve $tarfile from $www")
if ($?);
#
# We need to read back the response to see if the file was
# unchanged. This is dumb; why doesn't wget exit with reasonable
# error codes?
#
my $nochange = 0;
if (!open(WGET, "$cmd 2>&1 |")) {
fatal("Cannot start wget: $!\n");
}
while (<WGET>) {
print $_
if ($debug);
# Ick!
if ($_ =~ /^.* ERROR 304.*$/i) {
$nochange = 1;
}
}
if (! close(WGET)) {
if ($?) {
fatal("Could not retrieve $tarfile from $www")
if (!$nochange);
# Otherwise, not modifed.
print STDOUT "Timestamp for $tarfile unchanged!\n"
if ($debug);
return 1;
}
else {
fatal("Error closing wget pipe: $!\n");
}
}
# Must do this for caller so that if the MD5 has not changed,
# the caller can update the timestamp in the ident file.
#
# Always use GM time for this. The server expects it.
$$filestamp = mktime(gmtime(time()));
#
# We got a file. Compare the MD5 now.
#
my $md5 = GetMD5($copyfile);
if (defined($oldmd5) && $oldmd5 eq $md5) {
print STDOUT "MD5 ($md5) for $tarfile unchanged!\n"
if ($debug);
return 2;
}
$$filemd5 = $md5;
}
return 0;
}
#
# Get MD5 of file.
#
sub GetMD5($)
{
my ($file) = @_;
my $md5;
if ($OSNAME eq "linux") {
$md5 = `md5sum $file`;
if ($md5 =~ /^([\w]*)\s*.*$/) {
$md5 = $1;
}
else {
fatal("Bad MD5 for $file: $md5.");
}
}
elsif ($OSNAME eq "freebsd") {
$md5 = `md5 -q $file`;
if ($md5 =~ /^([\w]*)$/) {
$md5 = $1;
}
else {
fatal("Bad MD5 for $file: $md5.");
}
}
else {
fatal("Do not know how to compute MD5s!");
}
return $md5;
}
#
# Recreate the ident file.
#