setup.txt 8.8 KB
Newer Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
##### Setting up the Utah Network Testbed software
##### Robert Ricci
##### Last updated June 5, 2001
##### Tested on FreeBSD 4.3

##### Step 0

First of all, the machine should be configured correctly for the network it is
on, have the root password set, etc.

##### Step 1 - Package installation

Install the necessary packages. The following are necessary, and available as
FreeBSD ports:

apache+mod_ssl-1.3.19+2.8.2 The Apache 1.3 webserver with SSL/TLS functionality
cvsupd-bin-16.1     A general network file distribution system optimized for CV
18
fping-2.2b1         Quickly ping N hosts w/o flooding the network
19
20
21
22
23
24
25
26
27
28
29
gmake-3.79.1        GNU version of 'make' utility
isc-dhcp-2.0.5      ISC Dynamic Host Configuration Protocol client and server c
mod_php3-3.0.18     PHP3 module for Apache
mysql-client-3.23.36 Multithreaded SQL database (client)
mysql-server-3.23.36 Multithreaded SQL database (server)
otcl-1.0a6          MIT Object Tcl
p5-DBI-1.15         The perl5 Database Interface.  Required for DBD::* modules
p5-Mysql-modules-1.2215 Perl5 modules for accessing MySQL databases
p5-SNMP-4.2.0       A perl5 module for interfacing with the CMU SNMP library
p5-SNMP_Session-0.83 A perl5 module for providing rudimentary access to SNMPv1 a
rpm-3.0.6_5         The Red Hat Package Manager
30
tcl-8.2.3           Tool Command Language
31
32
33
tcl-sql-20000114_1  TCL module for accessing MySQL databases
ucd-snmp-4.2        An extendable SNMP implimentation

34
35
36
37
38
39
Note on TCL: Do NOT install tcl83 - otcl, which is used by some testbed scripts,
requires tcl82. When you install the tcl-sql package, it will be put in the library
directory for the latest version of tcl you have installed, so if you have tcl83
installed at the time, you will have tcl-sql support under 8.3.X, but not under
8.2.X (which testbed scripts use)

40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
##### Step 2 - LEDA

Currently, the LEDA library is required to compile some testbed software (we
hope to remove this dependency at some point.) The simplest place to install it
is /usr/testbed/LEDA. If another location is used, be sure to use the
--with-LEDA=<dir> option to configure in the next step. The home page for LEDA
is at:

http://www.algorithmic-solutions.com/as_html/products/products.html

##### Step 3 - Testbed tree configuration/installation

Configure the testbed tree. For example, I have the testbed source in ~/testbed,
and use the ~/tbobj directory to do my builds in.

cd ~/tbobj
~/testbed/configure
gmake
gmake boss-install
gmake post-install

The 'post-install' target needs to be done as root, because certain scripts
needs to be setuid root.

##### Step 4 - Database Creation

Create the database. Use a dump of an old database, or use the SQL commands in
testbed/sql/database-create.sql to create an empty one with the right tables.
(Sorry, no help on filling them in yet). You'll also need to create some special
users. do with by running:
mysql mysql < testbed/aql/users-create.sql

##### Step 5 - Misc. Files and Services

SNMP MIBs - MIBs go in /usr/local/share/snmp/mibs. In addition to the ones
installed by the ucd-snmp package, you'll need MIBs for Cisco and Intel
switches. You can grab the Cisco MIBs from:
ftp.cisco.com/pub/mibs
78
79
80
81
82
83
84
The Intel ones can be found from the site for the 510T switches at:
http://www.intel.com/network/connectivity/products/exp510t.htm
Now, a step that involves some voodoo I don't quite understand: make sure that
/usr/local/share/snmp/mibs/.index exists (touch it if it doesn't), and chmod it
to 666. Now, do an snmpwalk of some device (eg. 'snmpwalk cisco1 public') -
this will force the .index file to get rebuilt. Suggestions of better ways to
rebuild this file are welcome!
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

DNS zones - Make sure to include the DNS configuration files from /etc/named/
Stick:
named-enable="YES"
in /etc/rc.conf

NFS - Make the machine an NFS server and client with the following in /etc/rc.conf:
nfs_server_enable="YES"
nfs_server_flags="-u -t -n 16"
nfs_client_enable="YES"

tftp - Should have the following line in /etc/inetd.conf
tftp    dgram   udp wait    nobody  /usr/libexec/tftpd  tftpd /tftpboot /proj
(make sure to HUP inetd)

100
101
102
103
104
inetd - In FreeBSD, you need to prevent inetd from rate-limiting connections
(an attempt to defend against DOS attacks, but very annoying in a testbed
environment). Put the following in /etc/rc.conf:
inetd_flags="-wW -R 0"

105
106
107
108
109
SSH - If possible, grab the old machine's SSH host keys (from
/etc/ssh/ssh_host*) and HUP sshd. Also, get the root identity and known_hosts
files from the old machine (/root/.ssh/{identity,identity.pub,known_hosts}) - Make
sure to preserve file and directory permissions. You'll probably also want to add
PermitRootLogin yes
110
to /etc/ssh/sshd_config (and HUP sshd) so that you can log in as root remotely
111
112
113

Grab the old /etc/master.passwd file, and run 'cd /etc && pwd_mkdb -p master.passwd'
Also grab the old /etc/groups file
114
115
116

/etc/syslog.conf needs entries for some of our own services. Example:
!bootinfo
117
*.*                     /usr/testbed/log/bootinfo.log
118
!tmcd
119
*.*                     /usr/testbed/log/tmcd.log
120
!capture
121
*.*                     /usr/testbed/log/tiplogs/capture.log
122
123
!dhcpd
*.*                     /usr/testbed/log/dhcpd.log
124
125
All of these logs should be created before you HUP syslogd or reboot - All of them
can be world-readable
126
127
128

DHCP - Need to install the dhcpd config file. The old (deprecated) location was
/usr/site/bin/dhcp/dhcpd.conf. The new location (and the place you should
129
install it if you used the 'isc-dhcpd' port) is /usr/local/etc/dhcpd.conf .
130
131
132
133
134

RC scripts - The mysql-client rc script needs to run before ANY testbed
services are started! The mysql server should also be started early in the
process.  boot process. You can ensure this by changing directories to
/usr/local/etc/rc.d and renaming 'mysql-client.sh' to '1.mysql-client.sh' and
135
136
137
138
'mysql-server.sh' to '2.mysql-server.sh'. Furthermore, dhcpd needs to start
before proxydhcp, so rename 'dhcpd.sh' to '2.dhcpd.sh'. You will also need to
install the '3.testbed.sh' and 'cvsupd.sh'  scripts (in the rc.d directory of
the testbed tree)
139
140
141
142
143
144
145
146

Logs - To avoid filling up /var, link /var/log/testbed to /usr/testbed/log

CVSUPD - Minor changes to images can be distributed at boot time with cvsup. See
doc/newimage.txt for an overview of setting up a sup tree. Make sure to copy over
the old one (if it exists), and make sure cvsupd is running (there's an example
rc.d script in the rc.d/ directory of the testbed CVS tree)

Robert Ricci's avatar
Robert Ricci committed
147
148
Apache - You should have installed apache with mod_ssl, and php3 (NOTE: Version
3.0.17 is known to have broken file uploading support. Use 3.0.16 or 3.0.18 -
149
150
151
152
153
154
155
156
157
158
newer versions are likely to work as well.) We have an auto-generated config
file that you can install by changing to the apache subdir of your build tree
and running 'gmake install'. Some apache installtions may expect to find their
config file at /usr/local/etc/apache/httpd.conf, rather than apache.conf (where
ours gets installed.) Just symlink apache.conf to httpd.conf if you have
trouble with this. Also our config file expects to find SSL certificates in:
/usr/local/etc/apache/ssl.crt/www.<sitename>.crt and
SSLCertificateKeyFile /usr/local/etc/apache/ssl.key/www.<sitename>.net.key
(where <sitename> is OURSITE from the configure defs file.) Make sure yours
go there, or edit the apache.conf file appropriately.
Robert Ricci's avatar
Robert Ricci committed
159

160
161
162
Cron jobs: We currently have two cron jobs running for the testbed. Both can be
run out of /etc/crontab
45	1	*	*	*	root	/usr/testbed/sbin/backup
163
*/5	*	*	*	*	root	/usr/testbed/sbin/node_status
164
165
Don't forget to HUP cron!

Robert Ricci's avatar
Robert Ricci committed
166
167
168
169
170
ntpd: The boss node should be running ntpd. In FreeBSD, you can enable this with
the line
xntpd_enable="YES"
in /etc/rc.conf. Check out the ntpd man page for configuration information.

171
You may want a program to allow administrator-types to run stuff easily as root.
172
Here at Utah, we have two: su1 (developed locally) and sudo (installed from
173
174
175
FreeBSD ports) - don't forget to get it set up! Our strategy on boss was to
give everyone in the weel group unrestricted sudo access with:
%wheel  ALL=(ALL) NOPASSWD: ALL
176
177
178
179
180

checkpass - Need to get a copy of the dictionary that checkpass uses to check
for bad passwords when creating a new account. I'll fill out this section
soon when I figure out the best way to handle this...

181
182
183
184
185
186
syslogd - Normally, sylogd on FreeBSD is run with the '-s' flag to prevent
logging to it over the network. We use network logging, so we need this
feature. Re-enable it by putting:
syslogd_flags=""
in /etc/rc.conf

187
188
189
190
191
192
193
194
195
196
197
198
##### Step 6 - Stuff to copy from an old boss node

If you're simply moving from one boss node to another, there are a few files
and trees you'll want to make sure to copy over:
/usr/testbed/images/
/tftpboot/
/etc/namedb/
/etc/master.password
/etc/group
/usr/testbed/sup/
/usr/site/

199
200
201
202
203
204
205
##### Last-minute synching
Right before bringing the new boss node online (if copying from an old boss node),
make sure to have copy over the latest versions of:
* The database
* The sup tree
* The dhcpd.conf file
* The DNS records
Robert Ricci's avatar
Robert Ricci committed
206
* The password file