setup.txt 7.21 KB
Newer Older
1
#####
2
##### Setting up the Utah Network Testbed software on a boss node
3
##### Tested on FreeBSD 4.3, FreeBSD 4.5, FreeBSD 4.6, and FreeBSD 4.7
4
#####
5

Jay Lepreau's avatar
Jay Lepreau committed
6 7 8
##### Step -1 - Set up "ops"

Follow the instructions in the setup-ops.txt file before the ones in this file!
9

10
##### Step 0 - OS installation and setup
11

12
Install FreeBSD on the machine you'll be using for your boss node, using the
13 14 15 16 17 18 19 20 21 22 23
standard FreeBSD installation process.  When asked by the installer, it's best
to choose the 'Developer' distribution set - this gets you full sources. The
'X-Developer' distribution set would be fine too, if you want to be able to run
X clients from the boss node.  When it asks if you want to install the ports
collection, answer yes.  You don't need to worry about which packages to
install (of course, grab your favorite editors, etc.) - the ones our software
needs will be installed automatically later.  You, will, however, have to make
sure that you create a partition large enough to hold /usr/testbed - in
addition to the testbed software, this is where many disk images will get
stored. Also, the /var partition will need to be large enough to hold the
database - 100MB extra for the database should be sufficient.
24

25 26
Make sure that you have the network correctly configured, and make sure that it
has a hostname that resolves. The silly mysql-server port requires this.
27

28 29 30 31
If you want, you can go ahead and create an account for yourself on boss. For
now, just stick the home directory somewhere local, and move it to /users/ once
you've got it mounted from ops (the boss-install script will set this up). In
general, it's probably simpler to just use 'root' for now.
32

33
##### Step 1 - Create a defs file
34

35 36 37
The defs file will describe some of your setup, such as the hostnames of your
boss and ops nodes, and email addresses that certain types of mail will be sent
to.
38

39 40
Use the 'defs-example' file in the root of our source distribution as a
template. It contains comments explaining the important variables to set.
41

42 43 44 45
You'll also need to make a defs file for the web interface. This is found in
the www/ directory of the testbed source tree. Copy example-defs.php3 to
<name>-defs.php3, and put <name> into WWWDEFS in the main defs file. Important
variables in this file are commented as well.
46

47
##### Step 2 -  Unpacking and running configure
48

49
This works the same as it did on ops:
50
cd ~/tbobj
51
~/testbed/configure --with-TBDEFS=/users/ricci/testbed/defs-ricci
52

53
##### Step 3 - Running the boss installation script
54

Jay Lepreau's avatar
Jay Lepreau committed
55
Again, this works the same as it did on ops, except that you run
56
install/boss-install in the object tree, instead of ops-install.
57

58 59 60
Part way through, this script will bail out and prompt you to install some
ports. Some of the ports we need on boss have interactive installations, and
this generally takes a long time, and you want to see what's going on, right?
61

62 63 64 65 66 67 68 69
So, just cd to /usr/ports/misc/emulab-boss/ and run (as root) a 'make install'.
There are a few things you'll need to know about the ports:
* Installation of the mysql323-server port will fail if the name returned by
  'hostname' doesn't resolve.
* The php4 port will bring up a funky menu system to let you choose some
  features to enable and disable. Enable 'mhash', and leave the rest alone
* The snmp port asks all kinds of silly questions that don't matter, since
  we're not going to be running snmpd on boss. Just take the defaults.
70

71
When you're done, re-run the boss-install script.
72

73 74
Like the ops-install script, boss-install sets up paswordless sudo for anyone
in the wheel group.
75

76 77 78 79 80 81 82 83 84
There is one bootstrapping problem we have that needs to be worked around - we
put fully-qualified names for the ops/users node into /etc/fstab on boss. But,
if you're running the nameserver for this domain on boss, those names won't be
resolvable yet. Since we don't yet have a way to auto-generate DNS
configuration files, the suggested work-around is to add addresses for the
FSNODE and USERNODE that you specified in your defs file (which may be the same
thing), to /etc/hosts on boss. Remebember to remove them once you really have
DNS set up.

85
##### Step 4 - Installing from source.
86

87 88
In your object directory, do a 'gmake && gmake boss-install'. Then, as root, do
a 'gmake post-install'. The post-install target needs to run as root, so that
89
it can make certain scripts setuid, etc.
90

91
##### Step 5 - Setting up root ssh from boss to ops
92

93 94 95 96
The boss node needs to be able to ssh in, as root, to the ops node. To set this
up, copy root's public identity from boss (created by the boss-install script)
to ops's authorized_keys file:
scp /root/.ssh/identity.pub ops:/root/.ssh/authorized_keys
97

98
##### Step 6 - Filling the database
99

100 101
See the file setup-db.txt in this directory for instructions on getting the
proper information about your site and nodes into the database.
Leigh B. Stoller's avatar
Leigh B. Stoller committed
102

103
##### Step 7 - Other miscellaneous things to set up
104

105 106
There are a few things we haven't been able to completely automate just yet,
though we hope to soon. 
107

108 109 110 111
DNS configuration - boss-install enables bind on your boss node, but does not
set it up as a server for your domain. We don't have documentation for creating
the zone files, etc yet. Best bet right now is to ask Utah for a copy of
theirs.  Hopefully, there will be a script to auto-generate large portions of
112 113 114 115 116 117
them from the database. Once done, add 127.0.0.1 as the first nameserver in
/etc/resolv.conf . Also, add boss as the first nameserver on ops, too. If
you're running under the .emulab.net domain, too, make sure to get Utah to
delegate the subdomain to you. If you have any hardware (eg. switches or power
controllers) on a private subnet, add them to /etc/hosts, with the same name
they've been give in the database.
118

119 120 121 122 123 124 125
SSL certificates - Our apache config file expects to find SSL certificates in:
/usr/local/etc/apache/ssl.crt/www.<sitename>.crt and
/usr/local/etc/apache/ssl.key/www.<sitename>.net.key
(where <sitename> is OURDOMAIN from the configure defs file.) 
Generate a passwordless certificate (up to you if you want to get a 'real' one
from Verisign, etc., or sign your own), and place the files from it in the
above locations.
126 127 128

cvsupd - Minor changes to images can be distributed at boot time with cvsup.
See doc/newimage.txt for an overview of setting up a sup tree. Make sure to
129 130
copy over the old one (if it exists).

131
dhcpd - Once you've filled out the nodes and interfaces tables (instructions in
132 133 134 135 136
setup-db.txt), you need to generate a config file for dhcpd. For this, you use
the dhcpd_makeconf script (installed into /usr/testbed/sbin), which uses the
template file in the dhcpd directory of the source distribution. Make sure to
edit the template file to reflect your local IP addresses, and include all node
types. Place the resulting config file (which dhcpd_makeconf spits to the
137 138 139 140 141 142 143 144 145 146 147
standard output), in /usr/local/etc/dhcpd.conf .  NOTE: This template file is
for isc-dhcpd 2.x . It needs to be updated to work with isc-dhcpd 3.x .

tftpboot - There are a few bootloaders, mini-kernels, and MFSes that are used
to boot, reload, etc. testbed machines, which live in /tftpboot . For the time
being, these are not distributed with our source, and require some site
customizations, so ask Utah for the boot loaders, etc.

disk images - You'll also, of course, need disk iamges to go on your nodes.
Right now, we have no automatic way of generating these, so you'll have to ask
Utah for some.