The following is a guide to setting up power control and serial lines. Overview: 0) Setup capture logging 1) Enter all the power controller's info into the emulab DB and "capture" startup script (if applicable). 2) Enter all the nodes' outlet info into the DB 3) Enter all the nodes' serial line info into the DB and "capture" startup 4) Enter all tipserver machines into the database. 5) Collect interface mapping from nodes Details: Steps two and three can be easily scripted (automated) if you have an online mapping of node to outlet and node to serial line. Note that your boss node should *NOT* also be a tipserver node! You can host tiplines on your users node if you like, or on other dedicated tipserver nodes. Hosting tiplines on boss (and thus exporting them via capture) is generally a bad idea because of security issues, not to mention that it just will plain not work. 0) Setup capture logging Before you can start capture for any power controllers or nodes, you should setup the logging files. Capture is hardwired to log into /var/log/tiplogs. We usually create this as a symlink to /usr/testbed/log/tiplogs (yes, yes, we should change the path in the source...). So for each serial line server you should: mkdir -p /usr/testbed/log/tiplogs ln -s /usr/testbed/log/tiplogs /var/log/tiplogs cp /dev/null /usr/testbed/log/tiplogs/capture.log To setup logging for the capture processes themselves, add the following to each /etc/syslog.conf: # testbed stuff !capture *.* /usr/testbed/log/tiplogs/capture.log and then HUP syslog. To ensure the logfile gets "rolled" periodically, put this in /etc/newsyslog.conf do: # testbed stuff /usr/testbed/log/tiplogs/capture.log 664 7 1000 * Z *** export log filesystem to ops node. So log dir should really be a distinct filesystem, else you export more than you intend! # rc.conf: added by Emulab setup: allow export of log filesystem rpcbind_enable="YES" nfs_server_enable="YES" nfs_reserved_port_only="YES" mountd_enable="YES" mountd_flags="-r -p 900" # exports: added by Emulab setup: allow export of log filesystem /var/log/tiplogs ops -ro -alldirs *** install boss rootkey in tipserv authorized_keys *** install script(s) to keep groups file in sync *** add NFS mount to ops fstab tipserv:/var/log/tiplogs /var/log/tiplogs nfs ro,-b,-i,nodev,nosuid 0 0 note that ops-install script assumes that ops is a tipserv machine if mounting remote tipserv directory on ops:/var/log/tiplogs, be sure to disable newsyslog.conf entry. # exports: added by Emulab setup: allow export of log filesystem /var/log/tiplogs ops -ro -alldirs *** install script(s) to keep groups file in sync *** note that ops-install script assumes that ops is a tipserv machine if mounting remote tipserv directory on ops:/var/log/tiplogs, be sure to disable newsyslog.conf entry. 1) Enter all the power controller's serial line info into the emulab DB and add a line to the "capture" startup script. For each power controller, do the following: * Create an entry in the "nodes" table. insert into nodes (node_id, type, phys_nodeid, role) values ("power", "", "power", "powerctrl"); Where "" is the power controller designation you pick. I suggest just using a simple number. You can, of course, change this name if you wish, just use the same designation below. is the type of power controller; valid values are "RPC27" and "APC". If this power controller of type "APC", make sure it has an entry in your /ets/hosts file on your boss node, and skip to step 2. * Create an entry in the "tiplines" table. insert into tiplines (tipname, node_id, server) values ("power", "power", ""); Where "" is the same value as above and is the hostname of the machine that has the serial connection to the power controller. * Create a capture startup line in /usr/local/etc/rc.d/capture.sh on the machine that has the serial connection to "power". If the file doesn't exist yet, create it, and put #!/bin/sh at the top, then change the file's mode to 0755. Add a line like this to the above file: /usr/testbed/sbin/capture -T 15 -r -s 9600 power >/dev/null 2>&1 & Where is as above, and is the tty device file associated with this power controller. To make sure the console log for the power controller gets "rolled", add the following to /etc/newsyslog.conf: /usr/testbed/log/tiplogs/power.log 640 7 1000 * Z /usr/testbed/log/tiplogs/power.pid * Disable the status detail and confirmation prompt on the power controller. Manually execute the command line you added above, then run "console power". You will now have access to this device's serial line. Navigate the menus to turn off status reporting and confirmation. 2) Enter all the nodes' outlet info into the DB For each node, do the following: * insert row into outlets table insert into outlets values ("pc", "power", , NULL); Where is the testbed node unique suffix (e.g., number), is the power controller that its connected to, and is the outlet number on the power controller. * Try power cycling the node (perhaps wait to do this until you get to the end of step 3) on your boss node, execute sudo /usr/testbed/bin/power cycle pc You shouldn't get any errors, and the machine should go through a cold reboot cycle. 3) Enter all the nodes' serial line info into the DB and "capture" startup This process is nearly identical to step one. For each node, do the following: * Insert tipline row into DB insert into tiplines (tipname, node_id, server) values ("pc", "pc", ""); * Create a capture startup line in /usr/local/etc/rc.d/capture.sh on the machine with the serial conection to "pc". If the file doesn't exist, follow the script creation step outlined above in step 1. Open the above file, and add a line like this: /usr/testbed/sbin/capture -T 15 -r -s 115200 pc >/dev/null 2>&1 & Where is as above, and is the tty device file associated with this pc. To make sure the console log for the node gets "rolled", add the following to /etc/newsyslog.conf: /usr/testbed/log/tiplogs/pc.log 640 7 1000 * Z /usr/testbed/log/tiplogs/pc.pid * Test the console line Run "console pc" on serial 2. Hit enter a couple of times, or reboot (using "power"!) the machine and wait to see if the PXE and MFS kernel console messages appear. 4) Enter all tipserver machines into the database. For each independent tipserver machine you have, do the following in mysql: insert info tipservers values (""); 5) Collect interface mapping from nodes XXXX this should move elsewhere XXXX Once you have the consoles up and running, it would be a good time to collect interface mapping information from the nodes. Unfortunately, FreeBSD and Linux might number interfaces in different orders and they do give them different names. Thus, Emulab requires a mapping from FreeBSD interface number to Linux interface number. You'll need to boot one of each different node type into both freebsd and linux, recording the interface number to MAC address mapping. Use these two mappings to correlate the FreeBSD and Linux interface numbers. You'll use this information in conjunction with the instructions for adding nodes in setup-nodes.txt