setup.txt 6.78 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
##### Setting up the Utah Network Testbed software
##### Robert Ricci
##### Last updated June 5, 2001
##### Tested on FreeBSD 4.3

##### Step 0

First of all, the machine should be configured correctly for the network it is
on, have the root password set, etc.

##### Step 1 - Package installation

Install the necessary packages. The following are necessary, and available as
FreeBSD ports:

apache+mod_ssl-1.3.19+2.8.2 The Apache 1.3 webserver with SSL/TLS functionality
cvsupd-bin-16.1     A general network file distribution system optimized for CV
18
fping-2.2b1         Quickly ping N hosts w/o flooding the network
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
gmake-3.79.1        GNU version of 'make' utility
isc-dhcp-2.0.5      ISC Dynamic Host Configuration Protocol client and server c
mod_php3-3.0.18     PHP3 module for Apache
mysql-client-3.23.36 Multithreaded SQL database (client)
mysql-server-3.23.36 Multithreaded SQL database (server)
otcl-1.0a6          MIT Object Tcl
p5-DBI-1.15         The perl5 Database Interface.  Required for DBD::* modules
p5-Mysql-modules-1.2215 Perl5 modules for accessing MySQL databases
p5-SNMP-4.2.0       A perl5 module for interfacing with the CMU SNMP library
p5-SNMP_Session-0.83 A perl5 module for providing rudimentary access to SNMPv1 a
rpm-3.0.6_5         The Red Hat Package Manager
tcl-8.3.3           Tool Command Language
tcl-sql-20000114_1  TCL module for accessing MySQL databases
ucd-snmp-4.2        An extendable SNMP implimentation

##### Step 2 - LEDA

Currently, the LEDA library is required to compile some testbed software (we
hope to remove this dependency at some point.) The simplest place to install it
is /usr/testbed/LEDA. If another location is used, be sure to use the
--with-LEDA=<dir> option to configure in the next step. The home page for LEDA
is at:

http://www.algorithmic-solutions.com/as_html/products/products.html

##### Step 3 - Testbed tree configuration/installation

Configure the testbed tree. For example, I have the testbed source in ~/testbed,
and use the ~/tbobj directory to do my builds in.

cd ~/tbobj
~/testbed/configure
gmake
gmake boss-install
gmake post-install

The 'post-install' target needs to be done as root, because certain scripts
needs to be setuid root.

##### Step 4 - Database Creation

Create the database. Use a dump of an old database, or use the SQL commands in
testbed/sql/database-create.sql to create an empty one with the right tables.
(Sorry, no help on filling them in yet). You'll also need to create some special
users. do with by running:
mysql mysql < testbed/aql/users-create.sql

##### Step 5 - Misc. Files and Services

SNMP MIBs - MIBs go in /usr/local/share/snmp/mibs. In addition to the ones
installed by the ucd-snmp package, you'll need MIBs for Cisco and Intel
switches. You can grab the Cisco MIBs from:
ftp.cisco.com/pub/mibs
I'm not sure where to grab the INTEL ones. 

DNS zones - Make sure to include the DNS configuration files from /etc/named/
Stick:
named-enable="YES"
in /etc/rc.conf

NFS - Make the machine an NFS server and client with the following in /etc/rc.conf:
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 16"
nfs_client_enable="YES"

tftp - Should have the following line in /etc/inetd.conf
tftp    dgram   udp wait    nobody  /usr/libexec/tftpd  tftpd /tftpboot /proj
(make sure to HUP inetd)

SSH - If possible, grab the old machine's SSH host keys (from
/etc/ssh/ssh_host*) and HUP sshd. Also, get the root identity and known_hosts
files from the old machine (/root/.ssh/{identity,identity.pub,known_hosts}) - Make
sure to preserve file and directory permissions. You'll probably also want to add
PermitRootLogin yes
to /etc/ssh/sshd_config (and HUP sshd) so that you can log in a root remotely

Grab the old /etc/master.passwd file, and run 'cd /etc && pwd_mkdb -p master.passwd'
Also grab the old /etc/groups file
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117

/etc/syslog.conf needs entries for some of our own services. Example:
!bootinfo
*.*                     /var/log/testbed/bootinfo.log
!tmcd
*.*                     /var/log/testbed/tmcd.log
!capture
*.*                     /var/log/tiplogs/capture.log
!dhcpd
*.*                     /usr/testbed/log/dhcpd.log

DHCP - Need to install the dhcpd config file. The old (deprecated) location was
/usr/site/bin/dhcp/dhcpd.conf. The new location (and the place you should
install it if you used the 'isc-dhcpd' port) is /usr/local/etc/dhcpd.conf

RC scripts - The mysql-client rc script needs to run before ANY testbed
services are started! The mysql server should also be started early in the
process.  boot process. You can ensure this by changing directories to
/usr/local/etc/rc.d and renaming 'mysql-client.sh' to '1.mysql-client.sh' and
'mysql-server.sh' to '2.mysql-server.sh'. You will also need to install the
'3.testbed.sh' script (in the rc.d directory of the testbed tree)
118
119
120
121
122
123
124
125

Logs - To avoid filling up /var, link /var/log/testbed to /usr/testbed/log

CVSUPD - Minor changes to images can be distributed at boot time with cvsup. See
doc/newimage.txt for an overview of setting up a sup tree. Make sure to copy over
the old one (if it exists), and make sure cvsupd is running (there's an example
rc.d script in the rc.d/ directory of the testbed CVS tree)

Robert Ricci's avatar
Robert Ricci committed
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
Apache - You should have installed apache with mod_ssl, and php3 (NOTE: Version
3.0.17 is known to have broken file uploading support. Use 3.0.16 or 3.0.18 -
newer versions are likely to work as well.) Make sure to copy over the old
apache.conf file, and SSL certificates. If generating new ones, the importnat
Apache configuration options to set (and our values for them) are: (NOTE: these
were taken directly from our config file, but they should make sense if you
have experience with Apache config files)
ServerAdmin	webadmin@fast.cs.utah.edu
DocumentRoot	"/usr/testbed/www"
<Directory "/usr/testbed/www">
	Options Indexes FollowSymLinks MultiViews
	AllowOverride All
	Order allow,deny
	Allow from all
</Directory>
ErrorLog /usr/testbed/log/apache_error_log
CustomLog /usr/testbed/log/apache_access_log combined
<VirtualHost _default_:443>
DocumentRoot "/usr/testbed/www"
ServerName www.emulab.net
ServerAdmin testbed-ops@fast.cs.utah.edu
ErrorLog /usr/testbed/log/apache_error_log
TransferLog /usr/testbed/log/apache_access_log
...
</VirtualHost>
Make sure SSL and mod_php are enabled (which is done for you if you install the
FreeBSD port)

154
155
156
157
158
159
160
161
Cron jobs: We currently have two cron jobs running for the testbed. Both can be
run out of /etc/crontab
45	1	*	*	*	root	/usr/testbed/sbin/backup
*/5	*	*	*	*	tbroot	/usr/testbed/sbin/node_status
(note that the second one is run as the 'tbroot' user, but can be run as any
user that has read/write access to the database.)
Don't forget to HUP cron!

162
163
164
165
166
167
168
##### Last-minute synching
Right before bringing the new boss node online (if copying from an old boss node),
make sure to have copy over the latest versions of:
* The database
* The sup tree
* The dhcpd.conf file
* The DNS records
Robert Ricci's avatar
Robert Ricci committed
169
* The password file