Commit 8371fc79 authored by Leigh Stoller's avatar Leigh Stoller

Checkpoint my cvs interface to the workbench. This first cut uses the

"rtag" directive to initiate template modify operations. So, to get started
you do a checkout:

  cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo checkout XXXXX

where XXXXX is the part of the guid (10000/1) before the slash. Might try
and roll all templates into a single project wide repo at some point, to
avoid the extraneous path stuff, but didn't want to worry that just yet.

Okay, so have a checkout. You can work along the trunk, doing commits. To
create a new template (a modify of the existing template), you tag the tree
using rtag:

  cvs -d ops.emulab.net:/proj/$pid/templates/XXXXX/cvsrepo rtag mytag XXXXX

A template modify is started at the end, and you should probably wait for
email before continuing. Eventually I will need to add locking of some
kind, but I have to do the modify in the background, or else I get deadlock
cause cvs keeps the repo locked, and the modify also needs to access it.

Each time you tag along the trunk, you get a modified template, which in
the history diagram looks like:

  10000/1 --> 10000/2 --> 10000/3 ...

If you want to branch, say at 10000/2 you can create a branch tag using rtag:

  cvs -d [cut] rtag -r T10000/2 -b mytag2 XXXXX

You can also use your own tags for -r option, but I also create a TXXXXX/YY
tag at each template modify, which is easy to remember.

Then update your sandbox to the new branch, commit changes along that
branch, and then later use rtag again to initiate a template modify
operation:

  cvs update -r mytag2
  cvs commit ...
  cvs -d [cut] rtag -r mytag2 mytag3 XXXXX

And now the history diagram looks like:

  10000/1 --> 10000/2 --> 10000/3 ...
                |
                |
                -> 10000/4 ...

You should be able to mix interaction via the web with interaction via the
cvs interface. I've tested it, although not extensively.
parent 0cb4530f
......@@ -2480,7 +2480,7 @@ outfiles="$outfiles Makeconf GNUmakefile \
tbsetup/ipassign/ipassign_wrapper tbsetup/assign_prepass \
tbsetup/panic tbsetup/tbrsync tbsetup/nfstrace \
tbsetup/checkup/GNUmakefile tbsetup/checkup/checkup_daemon \
tbsetup/libtblog.pm \
tbsetup/libtblog.pm tbsetup/template_cvsroot/GNUmakefile \
tip/GNUmakefile tip/console \
tmcd/GNUmakefile tmcd/tmcd.restart \
tmcd/common/GNUmakefile tmcd/common/config/GNUmakefile \
......
......@@ -861,7 +861,7 @@ outfiles="$outfiles Makeconf GNUmakefile \
tbsetup/ipassign/ipassign_wrapper tbsetup/assign_prepass \
tbsetup/panic tbsetup/tbrsync tbsetup/nfstrace \
tbsetup/checkup/GNUmakefile tbsetup/checkup/checkup_daemon \
tbsetup/libtblog.pm \
tbsetup/libtblog.pm tbsetup/template_cvsroot/GNUmakefile \
tip/GNUmakefile tip/console \
tmcd/GNUmakefile tmcd/tmcd.restart \
tmcd/common/GNUmakefile tmcd/common/config/GNUmakefile \
......
......@@ -15,7 +15,7 @@ SYSTEM := $(shell uname -s)
include $(OBJDIR)/Makeconf
SUBDIRS = checkpass ns2ir nsverify nseparse checkup
SUBDIRS = checkpass ns2ir nsverify nseparse checkup template_cvsroot
BIN_STUFF = power snmpit tbend tbprerun tbreport \
os_load endexp batchexp swapexp \
......@@ -158,6 +158,7 @@ endif
subdir-install:
@$(MAKE) -C checkpass install
@$(MAKE) -C ns2ir install
@$(MAKE) -C template_cvsroot install
@$(MAKE) -C nsverify install
@$(MAKE) -C nseparse install
$(PLAB_INSTALL)
......
......@@ -1463,6 +1463,21 @@ sub InstanceList($$$)
return 0;
}
#
# Change description
#
sub SetDescription($$)
{
my ($self, $description) = @_;
# Must be a real reference.
return -1
if (! ref($self));
return $self->ModifyMetadata("description", $description,
User->ThisUser(), "template_description");
}
#
# Copy the datastore from parent template to child template.
#
......@@ -1723,7 +1738,7 @@ sub CreateDirectory($)
print "*** Could not create directory $versdir: $!\n";
return $rval;
}
foreach my $token ("tbdata", "archive", "datastore", "logs") {
foreach my $token ("tbdata", "datastore") {
my $dir = "$versdir/$token";
if (! mkdir($dir, 0770) ||
......@@ -1733,7 +1748,7 @@ sub CreateDirectory($)
return $rval;
}
}
if (0) {
my $workdir = TBExptWorkDir($pid, $eid);
#
......@@ -1748,6 +1763,7 @@ sub CreateDirectory($)
print "*** Could not create $workdir: $!\n";
return -1;
}
}
return 0;
}
......
......@@ -325,41 +325,27 @@ tblog_set_info($pid,$eid,$UID);
#
# Create a directory structure for the experiment.
# There is no need to do this for the template wrapper experiment; it was
# already done when the template was created.
#
if (!defined($template) || defined($instance)) {
if ($experiment->CreateDirectory() != 0) {
if (($? >> 8) == EDQUOT()) {
# Obey exit status protocol for web page; User should see this.
$errorstat = 1;
}
fatal({type => 'secondary', severity => SEV_SECONDARY,
error => ['create_experiment_directory_failed']},
"Failed to created experiment directory");
}
if (defined($instance)) {
# Need to cross-mark the instance right away so that it is flagged.
# Would be better to do this with a plain flag.
my %args = ();
$args{'exptidx'} = $experiment->idx();
$instance->Update(0, \%args) == 0
or fatal("Could not update experiment instance record!");
#
if ($experiment->CreateDirectory() != 0) {
if (($? >> 8) == EDQUOT()) {
# Obey exit status protocol for web page; User should see this.
$errorstat = 1;
}
fatal({type => 'secondary', severity => SEV_SECONDARY,
error => ['create_experiment_directory_failed']},
"Failed to created experiment directory");
}
else {
#
# But we do need to update the experiment record with the template path.
#
if (defined($instance)) {
# Need to cross-mark the instance right away so that it is flagged.
# Would be better to do this with a plain flag.
my %args = ();
$args{'path'} = $template->path();
$args{'exptidx'} = $experiment->idx();
$experiment->Update(\%args) == 0
or fatal("Could not update experiment record!");
# And the template with the new experiment index.
$instance->Update(0, \%args) == 0
or fatal("Could not update experiment instance record!");
}
elsif (defined($template)) {
# Tell the template with the new experiment index.
%args = ();
$args{'exptidx'} = $experiment->idx();
......
......@@ -18,7 +18,7 @@ use Exporter;
TBSCRIPTLOCK_OKAY TBSCRIPTLOCK_TIMEDOUT
TBSCRIPTLOCK_IGNORE TBSCRIPTLOCK_FAILED
PROJROOT GROUPROOT USERROOT SCRATCHROOT SHAREROOT
TBValidUserDir TBValidUserDirList TBMakeTempFile NewUUID);
TBValidUserDir TBValidUserDirList TBMakeTempFile NewUUID System);
# After package decl.
use English;
......@@ -29,6 +29,7 @@ use Fcntl ':flock';
use IO::Handle;
use File::Basename;
use Time::HiRes qw(gettimeofday);
use vars qw(@EXPORT_OK);
#my $MAILTAG = "@OURDOMAIN@";
my $MAILTAG = "@THISHOMEBASE@";
......@@ -37,6 +38,10 @@ my $TIMESTAMPS = "@TIMESTAMPS@";
my $TBOPSEMAIL = "@TBOPSEMAIL@";
my $SCRIPTNAME = "Unknown";
# Exported.
$SYSTEM_DEBUG = 0;
@EXPORT_OK = qw($SYSTEM_DEBUG);
#
# Real mount points (on the fileserver) for exported directories.
# At the moment, we have no reason to export these via functions.
......@@ -684,4 +689,18 @@ sub NewUUID()
return undef;
}
sub System($)
{
my ($command) = @_;
print STDERR "Running '$command'\n"
if ($system_debug);
TBDebugTimeStamp($command);
my $retval = system($command);
TBDebugTimeStamp("Done");
return $retval;
}
1;
......@@ -34,16 +34,18 @@ use Data::Dumper;
sub usage()
{
print(STDERR
"Usage: template_checkout [-q] -f <path> <guid/vers>\n".
"Usage: template_checkout [-q] [-u] -f <path> <guid/vers>\n".
"switches and arguments:\n".
"-q - be less chatty\n".
"-u - update existing checkout\n".
"-f - path to checkout directory on ops (via NFS).\n".
"<guid/vers> - GUID and version to swapin\n");
exit(-1);
}
my $optlist = "qdf:";
my $optlist = "qdf:u";
my %options = ();
my $quiet = 0;
my $update = 0;
my $debug = 0;
my $directory;
my $guid;
......@@ -124,7 +126,7 @@ ParseArgs();
# A little cookie file in that indicates what template it refers to.
my $cookie = "$directory/.template";
if (-e $cookie) {
if (-e $cookie && !$update) {
print STDERR "$directory already contains a checkout!\n";
print STDERR "There is no 'update' operation yet; delete or move it.\n";
exit(1);
......@@ -158,6 +160,14 @@ if (! -e "$directory/exp") {
tbdie("Could not create exp symlink in $directory!");
}
#
# No update in place (yet) since we use simple zip files. Move the current
# datastore directory out of the way.
#
if ($update && -d "$directory/datastore") {
}
#
# There is a plain experiment underlying the template, which has a lot
# of stuff we do not want to show the user in the checkout cause its
......@@ -236,6 +246,9 @@ sub ParseArgs()
if (defined($options{"d"})) {
$debug = 2;
}
if (defined($options{"u"})) {
$update = 1;
}
}
#
......
......@@ -29,20 +29,25 @@ sub usage()
{
print(STDERR
"Usage: template_commit [-q] -f <path>\n".
" template_commit [-q] [-e eid] <guid/vers>\n".
" template_commit [-q] [-e eid | -r tag] <guid/vers>\n".
" template_commit [-q] -p pid -e eid\n".
"switches and arguments:\n".
"-q - be less chatty\n".
"-e <eid> - Experiment instance to commit from\n".
"-E <str> - A pithy sentence describing the new template\n".
"-t <tid> - The template name (alphanumeric, no blanks)\n".
"-p <pid> - Project for -e option\n".
"<guid/vers> - GUID and version to swapin\n");
exit(-1);
}
my $optlist = "qe:dp:f:";
my $optlist = "qe:dp:f:t:E:r:";
my %options = ();
my $quiet = 0;
my $debug = 0;
my $frompath;
my $repotag;
my $tid;
my $description;
my $eid;
my $pid;
my $guid;
......@@ -74,6 +79,7 @@ sub ParseArgs();
sub fatal($$);
sub sighandler($);
sub cleanup();
sub CommitFromRepo();
sub CommitFromInstance();
sub CommitFromTemplate();
sub CommitFromCheckout();
......@@ -177,28 +183,75 @@ if (! $template->AccessCheck($this_user, TB_EXPT_MODIFY)) {
#
$SIG{TERM} = \&sighandler;
if (defined($frompath)) {
CommitFromCheckout();
}
elsif (defined($eid)) {
CommitFromInstance();
if (defined($repotag)) {
CommitFromRepo();
}
else {
CommitFromTemplate();
}
tbdie("Unsupported template commit operation!");
if (defined($frompath)) {
CommitFromCheckout();
}
elsif (defined($eid)) {
CommitFromInstance();
}
else {
CommitFromTemplate();
}
}
exit(0);
#
# Commit from a Repo.
#
sub CommitFromRepo()
{
my $pid = $template->pid();
my $gid = $template->gid();
my $optarg = ($quiet ? "-q" : "");
# Optional override from commandline
$tid = $template->tid()
if (!defined($tid));
#
# Template modify to give us a new version. Giving it $frompath causes
# the datastore to be imported from the directory instead of the parent
# template.
#
system("$modify -w -m $guid/$version $optarg -g $gid -r $repotag ".
"$pid $tid");
if ($?) {
fatal($? >> 8, "Failed to modify $template!");
}
# Pick up changes to child guid/vers.
$template->Refresh();
my $child_guid = $template->child_guid();
my $child_vers = $template->child_vers();
my $child = Template->Lookup($child_guid, $child_vers);
if (!defined($child)) {
fatal(-1, "Lookup of child template failed!");
}
$child->SetDescription($description)
if (defined($description));
return 0;
}
#
# Commit from a checkout.
#
sub CommitFromCheckout()
{
my $pid = $template->pid();
my $tid = $template->tid();
my $gid = $template->gid();
my $nsfile = "$frompath/tbdata/nsfile.ns";
my $optarg = ($quiet ? "-q" : "");
# Optional override from commandline
$tid = $template->tid()
if (!defined($tid));
#
# The NS file is taken from the checkout.
#
......@@ -219,6 +272,12 @@ sub CommitFromCheckout()
$template->Refresh();
my $child_guid = $template->child_guid();
my $child_vers = $template->child_vers();
my $child = Template->Lookup($child_guid, $child_vers);
if (!defined($child)) {
fatal(-1, "Lookup of child template failed!");
}
$child->SetDescription($description)
if (defined($description));
# Update the cookie so that the checkout refers to the new template
# not the original template.
......@@ -241,12 +300,15 @@ sub CommitFromCheckout()
sub CommitFromTemplate()
{
my $pid = $template->pid();
my $tid = $template->tid();
my $gid = $template->gid();
my $userdir = $template->path();
my $nsfile = "$userdir/tbdata/nsfile.ns";
my $optarg = ($quiet ? "-q" : "");
# Optional override from commandline
$tid = $template->tid()
if (!defined($tid));
#
# The NS file is taken from the template.
#
......@@ -260,6 +322,19 @@ sub CommitFromTemplate()
if ($?) {
fatal($? >> 8, "Failed to modify template!");
}
# Pick up changes to child guid/vers.
$template->Refresh();
my $child_guid = $template->child_guid();
my $child_vers = $template->child_vers();
my $child = Template->Lookup($child_guid, $child_vers);
if (!defined($child)) {
fatal(-1, "Lookup of child template failed!");
}
$child->SetDescription($description)
if (defined($description));
return 0;
}
#
......@@ -268,6 +343,7 @@ sub CommitFromTemplate()
sub CommitFromInstance()
{
my $pid = $template->pid();
my $gid = $template->gid();
my $experiment = Experiment->Lookup($pid, $eid);
if (!defined($experiment)) {
......@@ -289,12 +365,13 @@ sub CommitFromInstance()
fatal(1, "There is no NS file in $userdir/tbdata!")
if (! -e $nsfile);
# Optional override from commandline
$tid = $template->tid()
if (!defined($tid));
#
# Start with a plain template modify of the current template.
#
my $tid = $template->tid();
my $gid = $template->gid();
system("$modify -m $guid/$version -w -g $gid $pid $tid $nsfile");
if ($?) {
fatal($? >> 8, "Failed to commit instance to template!");
......@@ -396,6 +473,40 @@ sub ParseArgs()
tbdie("Improper experiment name (id)!");
}
}
elsif (defined($options{"r"})) {
$repotag = $options{"r"};
if ($repotag =~ /^([-\w:]+)$/) {
$repotag = $1;
}
else {
tbdie("Bad data in argument: $repotag");
}
}
if (defined($options{"t"})) {
$tid = $options{"t"};
if ($tid =~ /^([-\w]+)$/) {
$tid = $1;
}
else {
tbdie("Bad data in argument: $tid.");
}
if (! TBcheck_dbslot($tid, "experiments", "eid",
TBDB_CHECKDBSLOT_WARN|TBDB_CHECKDBSLOT_ERROR)) {
tbdie("Improper template name (id)!");
}
}
if (defined($options{"E"})) {
if (! TBcheck_dbslot($options{"E"},
"experiment_templates", "description",
TBDB_CHECKDBSLOT_WARN|TBDB_CHECKDBSLOT_ERROR)) {
tbdie("Improper template description!");
}
$description = $options{"E"};
}
if (defined($options{"q"})) {
$quiet = 1;
......
This diff is collapsed.
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2007 University of Utah and the Flux Group.
# All rights reserved.
#
SRCDIR = @srcdir@
TESTBED_SRCDIR = @top_srcdir@
OBJDIR = ../..
SUBDIR = tbsetup/template_cvsroot
include $(OBJDIR)/Makeconf
#LIB_STUFF = commit_prep commit_logmsg commit_finish
#OTHER_STUFF = commitinfo loginfo verifymsg
LIB_STUFF = logtag
OTHER_STUFF = taginfo
CTRLSBIN_STUFF= rtag_commit
#
# Force dependencies on the scripts so that they will be rerun through
# configure if the .in file is changed.
#
all: $(LIB_STUFF) $(CTRLSBIN_STUFF)
include $(TESTBED_SRCDIR)/GNUmakerules
install: \
$(addprefix $(INSTALL_LIBDIR)/cvsroot/, $(LIB_STUFF)) \
$(addprefix $(INSTALL_LIBDIR)/cvsroot/, $(OTHER_STUFF)) \
$(addprefix $(INSTALL_DIR)/opsdir/sbin/, $(CTRLSBIN_STUFF))
$(INSTALL_LIBDIR)/cvsroot/%: %
@echo "Installing $<"
-mkdir -p $(INSTALL_LIBDIR)/cvsroot
$(INSTALL) $< $@
$(INSTALL_DIR)/opsdir/sbin/%: %
@echo "Installing $<"
-mkdir -p $(INSTALL_DIR)/opsdir/sbin
$(INSTALL) $< $@
clean:
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2006, 2007 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
use strict;
use Getopt::Std;
use File::Basename;
#
# Configure variables
#
my $TBROOT = "@prefix@";
my $template_commit = "$TBROOT/bin/template_commit";
# un-taint path
$ENV{'PATH'} = '/bin:/usr/bin:/usr/local/bin';
delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};
# CVS sets this.
my $CVSROOT = $ENV{'CVSROOT'};
#print "Finish: @ARGV\n";
#
# First argument is a space separated list of tokens, and the first token
# is the directory of the commit.
#
my @tokens = split(' ', $ARGV[0]);
my $directory = $tokens[0];
# See commit_prep
my $id = getpgrp();
my $lastdir_file = "/tmp/#cvs.files.lastdir.${id}";
my $docommit_file = "/tmp/#cvs.docommit.${id}";
my $logmsg_file = "/tmp/#cvs.logmsg.${id}";
exit(0)
if (!-e $lastdir_file);
# See if we are processing the last directory in the commit
my $lastdirectory = `cat $lastdir_file`;
chomp($lastdirectory);
#print "$lastdirectory, $directory\n";
if (! ($lastdirectory =~ /\/$directory$/)) {
exit(0);
}
# unlink($lastdir_file);
# This is the last directory. See if we are supposed to fire off a commit.
exit(0)
if (!-e $docommit_file);
# Grab the tag out of the file, which is passed along to template_commit.
open(COM, $docommit_file) or
die("Could not open $docommit_file!\n");
my $tag = <COM>;
close(COM);
chomp($tag);
#
# Suck the guid out of the magic cookie file.
#
my $guid = `cat $CVSROOT/CVSROOT/GUID`;
chomp($guid);
my $logmsg = `cat $logmsg_file`;
chomp($logmsg);
print "Running template_commit -r $tag $guid\n";
print "Starting a template modify using repository tag $tag ... Be patient!\n";
system("$template_commit -r $tag -E '$logmsg' $guid") == 0
or exit(1);
exit(0);
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2006, 2007 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
use strict;
use Getopt::Std;
# un-taint path
$ENV{'PATH'} = '/bin:/usr/bin:/usr/local/bin';
delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};
#print "Log: @ARGV\n";
#
# Simply record the log message for later.
#
my $id = getpgrp();
my $logfile = shift(@ARGV);
my $filename = "/tmp/#cvs.logmsg.${id}";
my $docommit = "/tmp/#cvs.docommit.${id}";
if (-e $docommit) {
system("cp -p $logfile $filename") == 0
or die("Could not copy $logfile $filename!\n");
}
exit(0);
#!/usr/bin/perl -wT
#
# EMULAB-COPYRIGHT
# Copyright (c) 2006, 2007 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
use strict;
use Getopt::Std;
# un-taint path
$ENV{'PATH'} = '/bin:/usr/bin:/usr/local/bin';
delete @ENV{'IFS', 'CDPATH', 'ENV', 'BASH_ENV'};
#print "Prep: @ARGV\n";
#
# Simply record the directory (first argument). The last one we get indicates
# the last file of the commit.
#
my $id = getpgrp();
my $directory = shift(@ARGV);
my $filename = "/tmp/#cvs.files.lastdir.${id}";
my $docommit = "/tmp/#cvs.docommit.${id}";
my $entries = "CVS/Entries";
open(FILE, ">$filename") or die("Cannot open $filename, stopped");
print FILE "$directory\n";
close(FILE);
#
# Scan the file list. If we are committing the magic file, setup for a
# commit later in the loginfo handler.
#
if (! grep(/^TemplateInfo$/, @ARGV)) {
exit(0);
}
#
# Read the CVS version out of the CVS/Entries file so we can match it
# up against a tag.
#
my ($name,$revision,$date,undef,$tag) =
split('/', substr(`grep TemplateInfo $entries`, 1));
if (!defined($name) || !defined($revision)) {
die("Could not find TemplateInfo in $entries!\n");
}
#
# If not operating on the trunk (revision is x.y), then it has to be a
# tagged sandbox.
#
if ($revision =~ /^\d+\.\d+$/) {
$tag = "HEAD";
}
elsif (!defined($tag)) {
die("You must tag your repository to create a new template!\n");
}
else {
chomp($tag);
$tag = substr($tag, 1);
}
#print "TemplateInfo: $revision,$tag\n";
# Cookie file that says we need to do a commit, and what the tag is.
open(COM, ">$docommit")
or die("Could not write $docommit");
print COM "$tag\n";
close(COM);
exit(0);
# The "commitinfo" file is used to control pre-commit checks.
# The filter on the right is invoked with the repository and a list
# of files to check. A non-zero exit of the filter program will
# cause the commit to be aborted.
#
# The first entry on a line is a regular expression which is tested
# against the directory that the change is being committed to, relative
# to the $CVSROOT. For the first match that is found, then the remainder
# of the line is the name of the filter to run.
#
# If the repository name does not match any of the regular expressions in this
# file, the "DEFAULT" line is used, if it is specified.
#
# If the name "ALL" appears as a regular expression it is always used
# in addition to the first matching regex or "DEFAULT".
#
ALL $CVSROOT/CVSROOT/commit_prep
# The "loginfo" file controls where "cvs commit" log information
# is sent. The first entry on a line is a regular expression which must match
# the directory that the change is being made to, relative to the
# $CVSROOT. If a match is found, then the remainder of the line is a filter
# program that should expect log information on its standard input.
#
# If the repository name does not match any of the regular expressions in this
# file, the "DEFAULT" line is used, if it is specified.
#
# If the name ALL appears as a regular expression it is always used
# in addition to the first matching regex or DEFAULT.
#
# You may specify a format string as part of the