Commit f7e53243 authored by Mike Hibler's avatar Mike Hibler

Modernize elabinelab and Emulab install support a bit.

Support FreeBSD 10.3. We will need to be moving to this before long
as 10.2 EOLs in two days.

Support setup of "Emulab-aware" ZFS use in install scripts. Note that
the core support code was already done (WITHZFS, WITHAMD). Mostly this
involves changes to setup either amd (WITHAMD==1) or autofs (WITHAMD==0)
on the boss node and to NOT add mounts of /{users,groups,proj} to
/etc/fstab. We still need to add a section to the install documentation
about setting up a zpool for Emulab to use. There was also a fix to the
firstuser script which did not do the account setup correctly.

Support setup of ZFS in elabinelab. The elabinelab attributes CONFIG_ZFS
and CONFIG_AUTOFS are used to convey intent here. Currently they can only
be used in an "ops+fs" config (e.g., the standard boss and ops config,
NOT the seperate fs node config). It should work with either the physical
or virtual node setups:

* For the physical node setup, we actually use local blockstores in the
  ops node config: a SYSVOL blockstore for /usr/testbed and a tiny 1Mib
  NONSYSVOL blockstore. The latter blockstore is not actually used, we
  just make it to force setup of a ZFS zpool which we then use for the
  inner elab.

* For the virtual node setup, we just identify the virtual EXTRADISK
  intended for "/q" and create a zpool on that device.

I would like to change all physical elabinelab setups to use blockstors
rather than the current hacky mkextrafs usage. But that is a task for
another day.

Finally, a couple of random changes in elabinelab code: change the
CentOS image downloaded to CENTOS7-64-STD, increased the default sizes
of the EXTRADISKS used in the VM config.
parent f28a1d5b
#!/bin/sh
#
# Copyright (c) 2000-2015 University of Utah and the Flux Group.
# Copyright (c) 2014-2016 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -60,6 +60,10 @@ if [ $# -eq 0 ]; then
exit 0
fi
#
# XXX if the validmounts file does not exist, we force it to try. This has
# a nasty side-effect of returning empty dirs for non-existent server dirs.
#
key=$1
if test -f "$TB/etc/validmounts.txt" &&
! grep -q "^/$map/$key\$" $TB/etc/validmounts.txt; then
......
#!/usr/bin/perl -w
#
# Copyright (c) 2004-2015 University of Utah and the Flux Group.
# Copyright (c) 2004-2016 University of Utah and the Flux Group.
#
# {{{EMULAB-LICENSE
#
......@@ -148,6 +148,11 @@ my $OPSMOUNTDIR = "/ops";
# Hardwired?
my $TBDIR = "/usr/testbed";
# ZFS support
my $ZPOOLCMD = "/sbin/zpool";
my $ZFSCMD = "/sbin/zfs";
my $ZFSPOOL = "emulab";
#
# Support for Windows nodes.
# XXX here for compat; replaced by emulabconfig option.
......@@ -214,10 +219,15 @@ my %emulabconfig = (
# CONFIG_NOSETUP debug option: don't do most of rc.mkelab activities.
# CONFIG_SCRATCHFS configure a separate "scratch" filesystem.
# CONFIG_SINGLECNET use the real cnet as the inner cnet.
# CONFIG_ELVIN support Elvin compatibility
# CONFIG_ELVIN support Elvin compatibility (obsolete)
# CONFIG_WINDOWS support Windows OS on nodes
# CONFIG_SHAREDFS support shared filesystem
# CONFIG_SHAREDFS support shared filesystem
# CONFIG_QUERIER configure mfrisbeed to act as IGMP querier
# CONFIG_OPSVM configure ops node to be jail on boss (obsolete)
# CONFIG_NODBINIT do not load DB state on boss
# CONFIG_GENIRACK configure as a GENI rack
# CONFIG_ZFS configure fs node to use ZFS
# CONFIG_AUTOFS configure autofs instead of amd for boss ZFS mounts
#
"CONFIG_NOSETUP" => $NOSETUP,
"CONFIG_SCRATCHFS" => $SCRATCHFS,
......@@ -230,6 +240,8 @@ my %emulabconfig = (
"CONFIG_OPSVM" => $OPSVM,
"CONFIG_NODBINIT" => 0,
"CONFIG_GENIRACK" => 0,
"CONFIG_ZFS" => 0,
"CONFIG_AUTOFS" => 0,
# Default protogeni package if LOAD_PROTOGENI is enabled.
"PGENI_PKG" => "emulab-protogeni-1.0",
......@@ -498,6 +510,20 @@ sub doboot()
$NTPSERVER = "ops"
if ($emulabconfig{"CONFIG_SINGLECNET"});
#
# Only support ZFS right now on ZFS-savvy ops+fs node.
#
if ($emulabconfig{"CONFIG_ZFS"} != 0) {
# XXX don't support this yet
if ($emulabconfig{"ROLE"} eq "fs") {
SetupFatal("Do not support ZFS in standalone FS yet.");
}
if ($emulabconfig{"ROLE"} eq "ops+fs" &&
(! -x $ZPOOLCMD || system("kldload -n zfs.ko"))) {
SetupFatal("Image does not support ZFS.");
}
}
#
# XXX not-so-temporary: adjust package info.
#
......@@ -631,7 +657,7 @@ sub doboot()
$emulabconfig{EXTRA_PKG} = "emulab-extras-6.1";
$emulabconfig{PACKAGE_TARBALL} = "FreeBSD-10.1-packages${suf}.tar.gz";
$emulabconfig{PGENI_PKG} = "emulab-protogeni-6.1";
} elsif ($FBSD_VERSION >= 10.2) {
} elsif ($FBSD_VERSION == 10.2) {
my $suf = ($FBSD_ARCH eq "amd64") ? "-64" : "";
$emulabconfig{FS_PKG_DIR} = "/share/freebsd/10.2/packages${suf}";
$emulabconfig{OPS_PKG_DIR} = $emulabconfig{FS_PKG_DIR};
......@@ -643,6 +669,18 @@ sub doboot()
$emulabconfig{EXTRA_PKG} = "emulab-extras-6.2";
$emulabconfig{PACKAGE_TARBALL} = "FreeBSD-10.2-packages${suf}.tar.gz";
$emulabconfig{PGENI_PKG} = "emulab-protogeni-6.2";
} elsif ($FBSD_VERSION >= 10.3) {
my $suf = ($FBSD_ARCH eq "amd64") ? "-64" : "";
$emulabconfig{FS_PKG_DIR} = "/share/freebsd/10.3/packages${suf}";
$emulabconfig{OPS_PKG_DIR} = $emulabconfig{FS_PKG_DIR};
$emulabconfig{BOSS_PKG_DIR} = $emulabconfig{FS_PKG_DIR};
$emulabconfig{EXTRA_PKG_DIR}= $emulabconfig{FS_PKG_DIR};
$emulabconfig{FS_PKG} = "emulab-fs-6.3";
$emulabconfig{OPS_PKG} = "emulab-ops-6.3";
$emulabconfig{BOSS_PKG} = "emulab-boss-6.3";
$emulabconfig{EXTRA_PKG} = "emulab-extras-6.3";
$emulabconfig{PACKAGE_TARBALL} = "FreeBSD-10.3-packages${suf}.tar.gz";
$emulabconfig{PGENI_PKG} = "emulab-protogeni-6.3";
}
#
# If there is a package tarball, prefer that and grab it now.
......@@ -1286,19 +1324,42 @@ sub SetupOpsNode($)
my $fromscratch =
($emulabconfig{"LOAD_ELABSOURCE"} && $emulabconfig{"LOAD_MFS"});
my $installpkgs = $emulabconfig{"LOAD_PACKAGES"};
my $usezfs = $emulabconfig{"CONFIG_ZFS"};
#
# Create filesystems for fs use. We need:
# * /usr/testbed => disk0s2e
# for a "from scratch" build and:
# for a "from scratch" build, and:
# * /q (proj,users,scratch) => disk0s4e
# * /share => disk0s4f
# for ops+fs and fs-only nodes.
# for ops+fs and fs-only nodes without ZFS, and:
# * /q (proj,users,scratch,share) => "emulab" zpool
# for ops+fs with ZFS.
#
if ($fromscratch) {
SetupTBDir($TBDIR);
}
if ($isfs) {
if (!$isfs) {
goto skipfs;
}
#
# For ZFS, we look for an existing "emulab" zpool and use that if
# it exists (since it was set up just for us!). Otherwise, we look
# for an extra disk targetted for /q and create the zpool there.
#
if ($usezfs) {
if (system("$ZPOOLCMD list -H -o name | grep -q '^${ZFSPOOL}\$'")) {
my $fsdev = FindExtraFSConfig($FSMOUNTDIR);
if (!$fsdev) {
SetupFatal("Could not find space for ZFS zpool.");
}
if (system("$ZPOOLCMD create -m none $ZFSPOOL /dev/$fsdev")) {
SetupFatal("Could not create ZFS zpool on /dev/$fsdev.");
}
}
}
else {
#
# Look for a specified /share, otherwise do the -2 trick to
# split $FSDIR
......@@ -1354,6 +1415,7 @@ sub SetupOpsNode($)
}
}
skipfs:
#
# Download Emulab source if necessary
#
......@@ -1489,6 +1551,14 @@ sub SetupOpsNode($)
mysystem("echo '$sshdpid' > /var/run/sshd.pid");
}
#
# XXX me again: prepare also hides our local blockstores;
# put them back!
#
if ($usezfs && -x "$BINDIR/rc/rc.storagelocal") {
mysystem("$BINDIR/rc/rc.storagelocal boot");
}
#
# Remove the outer testbed startup script.
#
......@@ -1621,33 +1691,44 @@ sub SetupOpsNode($)
RecreateDir("/groups", 0);
RecreateDir("/share", 0);
mysystem("mkdir $FSDIR/users $FSDIR/proj $FSDIR/groups $FSDIR/share");
if ($isfs) {
mysystem("ln -s $FSDIR/users /users");
mysystem("ln -s $FSDIR/proj /proj");
mysystem("ln -s $FSDIR/groups /groups");
if ($usezfs) {
my $zo = "-o setuid=off -o sharenfs='boss -maproot=root'";
#
# Mount /share. The partition and filesystem were created above.
#
if (defined($sharefsdev)) {
mysystem("mkdir /share");
mysystem("mount /share");
}
else {
mysystem("mount $FSDIR/share");
mysystem("ln -s $FSDIR/share /share");
mysystem("$ZFSCMD create $zo -o mountpoint=/users $ZFSPOOL/users");
mysystem("$ZFSCMD create $zo -o mountpoint=/proj $ZFSPOOL/proj");
mysystem("$ZFSCMD create $zo -o mountpoint=/groups $ZFSPOOL/groups");
mysystem("$ZFSCMD create $zo -o mountpoint=/share $ZFSPOOL/share");
mysystem("$ZFSCMD create $zo -o mountpoint=/scratch $ZFSPOOL/scratch")
if ($emulabconfig{"CONFIG_SCRATCHFS"});
} else {
mysystem("mkdir $FSDIR/users $FSDIR/proj $FSDIR/groups $FSDIR/share");
if ($isfs) {
mysystem("ln -s $FSDIR/users /users");
mysystem("ln -s $FSDIR/proj /proj");
mysystem("ln -s $FSDIR/groups /groups");
#
# Mount /share. The partition and filesystem were created above.
#
if (defined($sharefsdev)) {
mysystem("mkdir /share");
mysystem("mount /share");
}
else {
mysystem("mount $FSDIR/share");
mysystem("ln -s $FSDIR/share /share");
}
}
}
#
# And the optional /scratch which share the same FS as /proj, et.al.
#
if ($emulabconfig{"CONFIG_SCRATCHFS"}) {
RecreateDir("/scratch", 0);
mysystem("mkdir $FSDIR/scratch");
if ($isfs) {
mysystem("ln -s $FSDIR/scratch /scratch");
#
# And the optional /scratch which share the same FS as /proj, et.al.
#
if ($emulabconfig{"CONFIG_SCRATCHFS"}) {
RecreateDir("/scratch", 0);
mysystem("mkdir $FSDIR/scratch");
if ($isfs) {
mysystem("ln -s $FSDIR/scratch /scratch");
}
}
}
......@@ -1704,6 +1785,9 @@ sub SetupOpsNode($)
print RC "nfs_server_enable=\"YES\"\n";
print RC "nfs_server_flags=\"-u -t -n 8\"\n";
print RC "nfs_client_enable=\"YES\"\n";
if ($usezfs) {
print RC "zfs_enable=\"YES\"\n";
}
if ($isfs && $emulabconfig{"CONFIG_WINDOWS"}) {
#
# As of 10/2012, the startup for samba changed.
......@@ -3136,6 +3220,8 @@ sub CreateDefsFile($)
{
my ($defsfile) = @_;
my $opsvm = $emulabconfig{"CONFIG_OPSVM"};
my $usezfs = $emulabconfig{"CONFIG_ZFS"};
my $useautofs = $emulabconfig{"CONFIG_AUTOFS"};
print "Creating defs file from stub defs file\n";
......@@ -3478,6 +3564,27 @@ sub CreateDefsFile($)
last SWITCH;
};
/^WITHAMD$/ && do {
if (!$usezfs || $useautofs) {
print OUTDEFS "WITHAMD=0\n";
} else {
print OUTDEFS "WITHAMD=1\n";
}
last SWITCH;
};
/^WITHZFS$/ && do {
if ($usezfs) {
print OUTDEFS "WITHZFS=1\n";
} else {
print OUTDEFS "WITHZFS=0\n";
}
last SWITCH;
};
/^ZFS_ROOT$/ && do {
print OUTDEFS "ZFS_ROOT=\"$ZFSPOOL\"\n";
last SWITCH;
};
print OUTDEFS $line;
}
}
......@@ -3646,12 +3753,15 @@ sub SetupTBDir($)
{
my ($TBDIR) = @_;
RecreateDir($TBDIR, 1);
if (my $dev = FindExtraFSConfig($TBDIR)) {
mysystem("$BINDIR/mkextrafs.pl -f -s 0 -r $dev -f $TBDIR");
goto done;
# If using ZFS, then /usr/testbed should already be setup
if ($emulabconfig{"CONFIG_ZFS"} == 0) {
RecreateDir($TBDIR, 1);
if (my $dev = FindExtraFSConfig($TBDIR)) {
mysystem("$BINDIR/mkextrafs.pl -f -s 0 -r $dev -f $TBDIR");
goto done;
}
mysystem("$BINDIR/mkextrafs.pl -s 2 -f $TBDIR");
}
mysystem("$BINDIR/mkextrafs.pl -s 2 -f $TBDIR");
done:
mysystem("mkdir $TBDIR/src $TBDIR/obj");
}
......
......@@ -2298,6 +2298,9 @@ sub os_remove_storage_slice($$$)
# care so much about full images.
#
if ($teardown == 3) {
if ($bsid eq "SYSVOL") {
return 1;
}
if (get_zpool_active_datasets("emulab") == 0 &&
mysystem("$ZPOOL export emulab $redir")) {
warn("*** $lv: could not export zpool 'emulab'\n");
......
......@@ -104,6 +104,15 @@ OPSVM_MOUNTPOINT=changeme
MIN_UNIX_UID=2000
MIN_UNIX_GID=2000
# use ZFS
WITHZFS=changeme
ZFS_ROOT=changeme
ZFS_QUOTA_USER="none"
ZFS_QUOTA_PROJECT="none"
ZFS_QUOTA_GROUP="none"
WITHAMD=changeme
AMD_ROOT=/.amd_mnt/ops
# Which console to use in the MFS. We need to set this for GeniRacks.
NODECONSOLE=changeme
# Which version of the MFS.
......
......@@ -240,7 +240,8 @@ elsif ($server eq "boss") {
'boss/ports', 'boss/portfix', 'boss/patches', 'cracklib',
'apache', 'boss/rcfiles', 'ntpd', 'boss/rcconf', 'boss/syslog',
'boss/database', 'etchosts', 'resolvetest',
'exports', 'nfsmounts', 'boss/mibs', 'boss/crontab', 'sudoers',
'exports', 'automount', 'nfsmounts',
'boss/mibs', 'boss/crontab', 'sudoers',
'samba', 'boss/ssh', 'boss/hostkeys',
'boss/rndc', 'boss/loaderconf', 'boss/sysctlconf',
'boss/sslcerts', 'boss/pubsub', 'boss/software',
......@@ -265,7 +266,8 @@ elsif ($server eq "fs") {
elsif ($server eq "ops") {
@files = ('perl', 'usersgroups', 'dirs', 'etchosts', 'resolvetest',
'ops/ports', 'ops/portfix', 'ops/patches', 'ntpd', 'ops/rcconf',
'ops/sendmail', 'ops/syslog', 'exports', 'nfsmounts',
'ops/sendmail', 'ops/syslog',
'exports', 'automount', 'nfsmounts',
'ops/crontab', 'sudoers', 'samba', 'ops/ssh', 'capture',
'ops/rcfiles', 'apache', 'ops/database',
'ops/cvsd', 'ops/flyspray', 'ops/twiki', 'ops/firewall'
......
......@@ -221,6 +221,8 @@ $OPSVM_ENABLE = @OPSVM_ENABLE@;
$NEEDMROUTED = 0;
$ARCHSUPPORT = @ARCHIVESUPPORT@;
$BROWSER_CONSOLE_ENABLE = @BROWSER_CONSOLE_ENABLE@;
$WITHZFS = @WITHZFS@;
$WITHAMD = @WITHAMD@;
#
# Python/Perl paths
......
#
# The template
#
use strict;
use libinstall;
use installvars;
sub Install($$$)
{
my ($server, $isupdate, $impotent) = @_;
if ($isupdate) {
return 0;
}
Phase "Automount", "Setting up automounter", sub {
if (!$WITHZFS || ISFS($server) ||
(ISOPSNODE($server) && $OPSVM_ENABLE)) {
PhaseSkip("Unneeded");
}
if ($WITHAMD) {
my @lines = ();
Phase "amd", "Enabling AMD", sub {
foreach my $map ("groups", "proj", "users") {
Phase "amd-$map", "Create $map map", sub {
DoneIfExists("/etc/amd.$map");
@lines = (
"/defaults opts:=rw,nosuid,vers=3,proto=tcp",
"* type:=nfs;rhost:=ops;rfs:=/$map/\${key}"
);
CreateFileFatal("/etc/amd.$map", @lines);
};
}
Phase "amd-rc", "Add to $RCCONF", sub {
@lines = (
"amd_enable=\"YES\"",
"amd_flags=\"-k amd64 -x all -l syslog /users /etc/amd.users /proj /etc/amd.proj /groups /etc/amd.groups\""
);
DoneIfUpdated($RCCONF, @lines);
AppendToFileFatal($RCCONF, @lines);
};
Phase "amd-start", "(Re)starting", sub {
ExecQuietFatal("/etc/rc.d/amd restart");
};
};
}
#
# Use autofs.
#
else {
Phase "autofs", "Enabling autofs", sub {
# XXX autofs had problems with long paths before FreeBSD 10.3.
if ($FBSD_MAJOR < 10 ||
($FBSD_MAJOR == 10 && $FBSD_MINOR < 2)) {
PhaseFatal("No autofs in this version of FreeBSD; ".
"use WITHAMD instead.");
}
if ($FBSD_MAJOR == 10 && $FBSD_MINOR < 3) {
print STDERR "WARNING: autofs ".
"is buggy in this version of FreeBSD\n";
}
Phase "autofs-maps", "Creating maps", sub {
DoneIfExists("$ETCDIR/autofs_boss.sh");
ExecQuietFatal("$GMAKE -C $TOP_OBJDIR/autofs first-install");
ExecQuietFatal("touch $ETCDIR/validmounts.txt");
};
Phase "autofs-rc", "Add to $RCCONF", sub {
my @lines = (
"autofs_enable=\"YES\"",
"automountd_flags=\"-v\"",
"autounmountd_flags=\"-v\""
);
DoneIfUpdated($RCCONF, @lines);
AppendToFileFatal($RCCONF, @lines);
};
Phase "autofs-start", "(Re)starting", sub {
ExecQuietFatal("/etc/rc.d/autounmountd restart");
ExecQuietFatal("/etc/rc.d/automountd restart");
ExecQuietFatal("/etc/rc.d/automount restart");
};
};
}
};
return 0;
}
# Local Variables:
# mode:perl
# End:
......@@ -12,24 +12,32 @@ my $UTAHURL = "http://www.emulab.net/downloads";
my $DESCRIPTORS = "$TOP_SRCDIR/install/descriptors-v4.xml";
my $GENDEV = "$TOP_SRCDIR/install/descriptors-gendev.xml";
my @MBRS = ("emulab-mbr.dd", "emulab-mbr2.dd", "emulab-mbr3.dd");
my $DEFAULTIMAGE = "UBUNTU14-64-STD";
my $LINUXSTDIMAGE = "UBUNTU14-64-STD";
my $FBSDSTDIMAGE = "FBSD102-64-STD";
my $DEFAULTIMAGE = $LINUXSTDIMAGE;
#
# We now use image import.
#
my %STDIMAGES = (
$DEFAULTIMAGE => "https://www.emulab.net/image_metadata.php?uuid=0a29c738-32b3-11e4-b30a-001143e453fe",
"CENTOS71-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=c74d9fc1-e54b-11e4-97ea-38eaa71273fa",
"UBUNTU14-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=0a29c738-32b3-11e4-b30a-001143e453fe",
"CENTOS7-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=6fa68fd6-9163-11e6-ac8c-90e2ba22fee4",
"FBSD102-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=57e6bf53-4376-11e5-bf03-38eaa71273fa",
"XEN44-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=6f5e573e-83f3-11e5-bf03-38eaa71273fa",
);
my %NEWIMAGES = (
"UBUNTU16-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=7683ca8e-5e37-11e6-ac8a-90e2ba22fee4",
"FBSD103-64-STD" => "https://www.emulab.net/image_metadata.php?uuid=f3866e90-2464-11e6-bdf0-d1afad303f71",
);
#
# Old style mappings, but probably still in use someplace.
#
my %STDMAPPINGS = (
"RHL-STD" => $DEFAULTIMAGE,
"FBSD-STD" => "FBSD102-64-STD",
"RHL-STD" => $LINUXSTDIMAGE,
"FBSD-STD" => $FBSDSTDIMAGE,
);
sub Install($$$)
......
......@@ -52,6 +52,7 @@ sub Install($$$)
PhaseSkip("named not running!?");
}
}
sleep 1;
};
Phase "starting", "Starting named", sub {
#
......
......@@ -29,6 +29,11 @@ sub Install($$$)
push(@LOGFILES, "$LOGDIR/flowvisor.log");
}
# Autofs logs
if ($WITHZFS && !$WITHAMD) {
push(@LOGFILES, "$LOGDIR/autofs.log");
}
Phase "syslog", "Setting up syslog", sub {
Phase "sysconf", "Editing $SYSLOG_CONF", sub {
DoneIfEdited($SYSLOG_CONF);
......@@ -110,6 +115,14 @@ sub Install($$$)
"!foam", "*.*\t\t\t\t\t\t$LOGDIR/foam.log",
"!flowvisor", "*.*\t\t\t\t\t\t$LOGDIR/flowvisor.log");
}
if ($WITHZFS && !$WITHAMD) {
@syslog_lines = (@syslog_lines,
"!automount", "*.*\t\t\t\t\t\t$LOGDIR/autofs.log",
"!automountd", "*.*\t\t\t\t\t\t$LOGDIR/autofs.log",
"!autounmountd", "*.*\t\t\t\t\t\t$LOGDIR/autofs.log");
}
AppendToFileFatal($SYSLOG_CONF, @syslog_lines);
};
......@@ -141,60 +154,70 @@ sub Install($$$)
Phase "newsyslog", "Setting up $NEWSYSLOG_CONF", sub {
DoneIfEdited($NEWSYSLOG_CONF);
AppendToFileFatal($NEWSYSLOG_CONF,
"$LOGDIR/tmcd.log 640 9 1000 * Z",
"$LOGDIR/tmcd-meta.log 640 9 1000 * Z",
"$LOGDIR/stated.log 640 9 1000 * Z ".
"/var/run/stated.pid 31",
"$LOGDIR/stated-mail.log 640 9 1000 * Z ".
"/var/run/stated.pid 31",
"$LOGDIR/checknodes.log 640 14 300 * Z ".
"/var/run/checknodes_daemon.pid",
"$LOGDIR/osselect.log 640 9 300 * Z",
"$LOGDIR/power.log 640 7 300 * Z",
"$LOGDIR/frisbeed.log 640 7 300 * Z",
"$LOGDIR/frisuploadd.log 640 7 300 * Z",
"$LOGDIR/mfrisbeed.log 640 7 1000 * Z",
"$LOGDIR/tftpd.log 640 7 200 * Z",
"$LOGDIR/named.log 640 7 200 * Z",
"$LOGDIR/dhcpd.log 640 7 200 * Z",
"$LOGDIR/bootinfo.log 640 7 200 * Z",
"$LOGDIR/capserver.log 640 5 200 * Z",
"$LOGDIR/pubsubd.log 640 5 1000 * Z",
"$LOGDIR/suexec.log 640 3 200 * Z",
"$LOGDIR/genlastlog.log 640 3 200 * Z",
"$LOGDIR/genlastlog 640 3 200 * Z " .
"/var/run/lastlog_daemon.pid",
"$LOGDIR/xmlrpcbag.log 640 7 300 * Z",
"$LOGDIR/sslxmlrpc.log 640 7 300 * Z",
"$LOGDIR/idlemail.log 640 14 1000 * Z",
"$LOGDIR/reloadlog 640 9 1000 * Z ".
"/var/run/reload_daemon.pid",
"$LOGDIR/checkuplog 640 9 1000 * Z ".
"/var/run/checkup_daemon.pid",
"$LOGDIR/poollog 640 9 1000 * Z ".
"/var/run/pool_daemon.pid",
"$LOGDIR/expire_daemon.log 640 9 1000 * Z ".
"/var/run/expire_daemon.pid",
"$LOGDIR/sa_daemon.log 640 9 1000 * Z ".
"/var/run/sa_daemon.pid",
"$LOGDIR/foam.log 640 7 * 168 Z",
"$LOGDIR/flowvisor.log 640 7 1000 * Z",
"$LOGDIR/poolmonitor.log 644 7 2000 * Z ".
"/var/run/poolmonitor.pid",
"$LOGDIR/lease_daemon.log 644 14 1000 * Z ".
"/var/run/lease_daemon.pid",
"$LOGDIR/apache_access_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_error_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_engine_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_request_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_access_log 644 7 10000 * BZ ".
"/var/run/httpd.pid"
);
my @nsyslog_lines = (
"$LOGDIR/tmcd.log 640 9 1000 * Z",
"$LOGDIR/tmcd-meta.log 640 9 1000 * Z",
"$LOGDIR/stated.log 640 9 1000 * Z ".
"/var/run/stated.pid 31",
"$LOGDIR/stated-mail.log 640 9 1000 * Z ".
"/var/run/stated.pid 31",
"$LOGDIR/checknodes.log 640 14 300 * Z ".
"/var/run/checknodes_daemon.pid",
"$LOGDIR/osselect.log 640 9 300 * Z",
"$LOGDIR/power.log 640 7 300 * Z",
"$LOGDIR/frisbeed.log 640 7 300 * Z",
"$LOGDIR/frisuploadd.log 640 7 300 * Z",
"$LOGDIR/mfrisbeed.log 640 7 1000 * Z",
"$LOGDIR/tftpd.log 640 7 200 * Z",
"$LOGDIR/named.log 640 7 200 * Z",
"$LOGDIR/dhcpd.log 640 7 200 * Z",
"$LOGDIR/bootinfo.log 640 7 200 * Z",
"$LOGDIR/capserver.log 640 5 200 * Z",
"$LOGDIR/pubsubd.log 640 5 1000 * Z",
"$LOGDIR/suexec.log 640 3 200 * Z",
"$LOGDIR/genlastlog.log 640 3 200 * Z",
"$LOGDIR/genlastlog 640 3 200 * Z ".
"/var/run/lastlog_daemon.pid",
"$LOGDIR/xmlrpcbag.log 640 7 300 * Z",
"$LOGDIR/sslxmlrpc.log 640 7 300 * Z",
"$LOGDIR/idlemail.log 640 14 1000 * Z",
"$LOGDIR/reloadlog 640 9 1000 * Z ".
"/var/run/reload_daemon.pid",
"$LOGDIR/checkuplog 640 9 1000 * Z ".
"/var/run/checkup_daemon.pid",
"$LOGDIR/poollog 640 9 1000 * Z ".
"/var/run/pool_daemon.pid",
"$LOGDIR/expire_daemon.log 640 9 1000 * Z ".
"/var/run/expire_daemon.pid",
"$LOGDIR/sa_daemon.log 640 9 1000 * Z ".
"/var/run/sa_daemon.pid",
"$LOGDIR/poolmonitor.log 644 7 2000 * Z ".
"/var/run/poolmonitor.pid",
"$LOGDIR/lease_daemon.log 644 14 1000 * Z ".
"/var/run/lease_daemon.pid",
"$LOGDIR/apache_access_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_error_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_engine_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_request_log 644 7 10000 * BZ ".
"/var/run/httpd.pid",
"$LOGDIR/apache_ssl_access_log 644 7 10000 * BZ ".
"/var/run/httpd.pid");
if ($PROTOGENI_GENIRACK) {
@nsyslog_lines = (@nsyslog_lines,
"$LOGDIR/foam.log 640 7 * 168 Z",
"$LOGDIR/flowvisor.log 640 7 1000 * Z");
}
if ($WITHZFS && !$WITHAMD) {
@nsyslog_lines = (@nsyslog_lines,
"$LOGDIR/autofs.log 640 14 1000 * Z");
}
AppendToFileFatal($NEWSYSLOG_CONF, @nsyslog_lines);
};
};
......
......@@ -51,7 +51,12 @@ sub Install($$$)
}
if (ISFS($server)) {
# The fileserver exports these too.
@dirs = (@dirs, @MOUNTPOINTS);
# But only /share if ZFS.
if ($WITHZFS) {
@dirs = (@dirs, $SHAREROOT);
} else {
@dirs = (@dirs, @MOUNTPOINTS);
}
}
foreach my $dir (@dirs) {
if ($dir ne $SHAREROOT) {
......
......@@ -29,6 +29,7 @@ sub Install($$$)
PhaseSkip("FSes are mounted from outside the VM");
}
my @dirs = ();
my @mkdirs = ();
#
# If we are the FS then we do not need to mount the FS file systems.
......@@ -38,16 +39,22 @@ sub Install($$$)
# unless ops is a VM on boss (see skip above).
#
if (!ISFS($server)) {
@dirs = @MOUNTPOINTS;
if ($WITHZFS) {
@dirs = ($SHAREROOT);
} else {
@dirs = @MOUNTPOINTS;
}
@mkdirs = @MOUNTPOINTS;
}
if (ISBOSSNODE($server) && !$OPSVM_ENABLE) {
@dirs = (@dirs, $USERSVAR_MOUNTPOINT, $OPSDIR_MOUNTPOINT);
@mkdirs = (@mkdirs, $USERSVAR_MOUNTPOINT, $OPSDIR_MOUNTPOINT);
}
PhaseSkip("Nothing to mount")
if (!@dirs);
if (!@mkdirs);
Phase "mountpoints", "Creating mountpoints", sub {
foreach my $dir (@dirs) {
foreach my $dir (@mkdirs) {
Phase $dir, $dir, sub {
DoneIfExists($dir);
mkdir $dir, 0777 or
......@@ -79,7 +86,12 @@ sub Install($$$)
"$USERNODE:/usr/testbed\t\t$OPSDIR_MOUNTPOINT\tnfs\t".
"rw,soft,-b,$nfsopt\t0\t0");
}
else {
#
# For ZFS, we do not mount the roots of the {users,proj,groups}
# filesystems as each {user,proj,group} has their own FS.
# We only mount /share.
#
elsif (!$WITHZFS || $dir eq $SHAREROOT) {
push(@lines,
"$FSNODE:$dir\t\t$dir\tnfs\trw,$nfsopt\t0\t0");
}
......
......@@ -213,17 +213,30 @@ if { $opsfw != {} } {
#
# We need to tell the XEN host to add some disks to the guests.
#
# There is no particular reason to be conservative here. We use only
# dedicated nodes for elabinelab VMs, and at a minimum we should have
# 120GB of disk available.
#
# Boss:
# disk1: 10G (/usr/testbed)
# disk2: 40G (/usr/testbed/data, primarily images)
# disk3: 10G (/usr/testbed/log)
# Ops:
# disk1: 10G (/usr/testbed)
# disk2: 40G (/q, including users/proj/groups)
# disk3: 10G (/share)
#
set bossdisks [tb-get-elabinelab-role-attribute "boss" "XEN_EXTRADISKS"]
if { $bossdisks != {} } {
$myboss add-attribute XEN_EXTRADISKS $bossdisks
} else {
$myboss add-attribute XEN_EXTRADISKS "disk1:10G,disk2:10G,disk3:10G"
$myboss add-attribute XEN_EXTRADISKS "disk1:10G,disk2:40G,disk3:10G"
}
set opsdisks [tb-get-elabinelab-role-attribute "ops" "XEN_EXTRADISKS"]
if { $opsdisks != {} } {
$myops add-attribute XEN_EXTRADISKS $opsdisks