setup-db.txt 12.5 KB
Newer Older
##### Setting up tbdb for a new boss node
3 4 5 6

Note: we are working on better automating many of the procedures in this
document - for now, though, a few of them are still manual.

8 9 10
Note: steps labeled "Local Only" are only required when setting up a testbed
with local nodes - they can be skipped in a widearea-only testbed.

11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104
##### Step 1 - Setup users, projects, and experiments

In order to proceed, you should have the following working (from the boss and
ops setup documentation):
NFS mounts between boss and ops
Root ssh keys (so that root on boss can ssh to ops without a password)
The web interface (you don't need to be able to log in yet)

Run the 'firstuser' script on boss. This will get put in the utils subdirectory
of your object tree. (Note: configure will not set the executable bit on the
script, so you'll need to set it yourself, or run 'perl firstuser'.) This
script will create a dummy user, which will be used only to bootstrap 'real'
users. The firstuser script will prompt you for a password for this user, named

Make sure you can log into the web interface using the 'elabman' account. This
account is created as a testbed administrator, but there is one thing you will
need to do in order to use your admin powers. For the same reason that you use
'su' and/or 'sudo' on your UNIX boxes instead of logging in as root, you must
explicitly enable admin privileges after you log in. When logged in as a user
who is allowed to become an admin, you will see a green dot one the left side
of the header above the main page content. The green dot means that although
you are allowed admin powers, they are currently turned off, and you see the
same web pages that a regular user sees, and can use the sam actions. If you
click on the dot, it will turn red, and you will have full administrator
privileges; we call this 'going red dot'.  Many of the procedures in this file
require you to be in red dot mode.

Now, we'll use the elabman user to bootstrap your own account. Log out of the
web interface, and use the 'Request Account' button to start a new project.
Create a project that you can use for the faculty/students/staff involved in
running your testbed - you can name it anything you'd like. Fill in your own
information in the 'Project Head Information' section.  If you have created an
account for yourself by hand on boss and/or ops, it is important that you
either user a different username on this form, or delete your handmade account
(using 'pw userdel <username>') on BOTH boss AND ops. Shortly after submitting
this form, you should get email at the address you included on the form -
follow the instructions it gives for verifying your email address. After
entering your password, while still logged in to your new account, use the
'Join project' link in the menu to join the 'emulab-ops' project (leave the
'group' field blank.)

Okay, now log out of the web interface, log back on as elabman, and go 'red
dot'.  Use the 'Approve New Projects' link on the menu to approve the project
you just started. Also, use the 'New User Approval' page to approve your new
account in the the 'emulab-ops' project - give yourself trust 'group root', so
that you can approve others into this project too.

Go to the web interface, and log in as the admin account you created. Create a
project for the administrators at your site (we call ours 'testbed', you can
call yours whatever you call your project or group).  As soon as you've sent
the request to sart the project, you can use the 'New Project Approval' link to
make it active.

Now, that you have an account, there a few things you're going to want to do -
you'll also want to do these things to other admin-types too, when their
accounts get created:
1) Set your 'admin bit' - set the 'admin' column for your account in the users
   table to '1' - you can do this, on boss, with:
   echo 'update users set admin=1 where uid="<username>"' | mysql tbdb

2) Give yourself the ability to log in to boss - most users have a restricted
   shell on boss, and are not allowed to log in using a password. Edit the
   password file (use 'vipw', FreeBSD requires some special processing on the
   password file after editing). Give yourself a real shell, and paste in your
   encrypted password string from ops.

3) Add yourself to important groups - If you want to be a member of any UNIX groups
   on boss, use the 'unixgroups' command (our automated account tools may wipe
   out groups added by hand.) You will need to be logged in as yourself (not
   root) on boss to run this command. You will want to be a member of at least
   the 'tbadmin' and 'wheel' groups. The syntax for this command is:

   unixgroups -a <username> <groupname> ...

Just as you need to go 'red dot' to use admin privileges on the web interface,
you must also explicitly enable them on the command line. To do this, prefix
the command you want to run with 'withadminprivs'. For example, I might invoke
unixgroups like this:
withadminprivs unixgroups -a ricci tbadmin wheel

(Note: withadminprivs and many other admin-type commands live in
/usr/testbed/sbin - you'll want to put this and /usr/testbed/bin in your

Now that you are an admin, you don't need the elabman account anymore. Log into
the web interface as yourself, go red dot, and go to the user list. Find
elabman (you may need to click on 'Show: all' a the top of the page). Go to
elabman's profile page  (by clicking on the uid) and freeze the user.

Others at your site can now apply to join your project, or start their own.

##### Step 2 - Setup web sql editor
105 106 107 108 109 110 111 112 113 114 115 116 117 118

Several of the steps below require you to add data to the database by hand. If
you're comfortable with SQL, you can do this directly with SQL queries, or you
can use the generic web-based SQL table editor provided with the testbed
software. If you plan to use the former method, you can skip this step.

********************************** WARNING **********************************
* Many tables depend on data in other tables, or depend on programs running *
* to effect a change. Thus, you should not edit tables other than the ones  *
* described in this document.                                               *
* You have been warned......                                                *
********************************** WARNING **********************************

First, you'll want to protect the webdb script from outside browsers. Because
of its flexibility, it would be quite dangerous if it were broken into. So, we
add an additional layer of protection by limiting the IP addresses it may be
used from. Open your httpd.conf file (created as part of the boss
122 123 124 125 126 127 128 129 130
installation), and find the 'Directory' directives. Add a section such as
<Directory /usr/testbed/www/webdb>
	AllowOverride None
	order deny,allow
	deny from all
	allow from 155.99.212.
If you installed the testbed tree somewhere other than /usr/testbed, fix the
directory. Change the 'allow from' line to match your IP subnet (note the '.'
132 133 134 135
on the end of the address, to match the entire subnet). You can have as
many 'allow' lines as you want. Restart apache.

Next, you'll need to specify which users have the ability to edit the database.
136 137
This is done with the 'dbedit' column in the users table. You can turn on
a user's dbedit bit like so:

   echo 'update users set dbedit=1 where uid="<username>"' | mysql tbdb

##### Step 3 - Setup switches
##### Local Only
143 144 145 146

1) Create node types for switches

Add entries to the node_types table for each type of switch you'll be using.
147 148 149 150 151
The class column should be 'switch'. The type for Cisco switches should be
something like 'cisco6509', or 'catalyst2950'.  Most of the other columns
aren't important for switches (so you can leave them blank), but putting in
max_cards (if the switch is expandable) and max_ports can be useful for your
own information.

2) Create interface types for switch interconnects (if any)

155 156 157 158 159
If you'll be connecting the experimental switches together, you'll add
interface types for the ports you'll be using to link them together.  These go
into the interface_types table. Make up something descriptive for the 'type'
column (no spaces.) Make sure to set the max_speed (in Kbps) and full_duplex
(1 for full duplex, 0 for half duplex) columns correctly.

3) Create switches in nodes table
162 163

Next, add the switches to the nodes table. The only necessary fields here are
164 165
node_id (choose one appropriate to the switch), type (use one of the ones you
created earlier,) and role. Role, for switches, should be either 'testswitch'
166 167
or 'controlswitch', depending on whether you're using it for the experimental
or control network. If it's used for both, call it a 'testswitch'.

169 170 171
The node_id you select must resolve in DNS and/or boss's /etc/host file - we'll
use this name to contact the switch for SNMP. Ie., you must be able to ping the
name you select as the switch's node_id.

4) Add state for switch interconnects

175 176 177 178 179 180
If you'll be connecting the experimental switches together, add interfaces to
each of them.  Use the interface types you created above for them, and be sure
to get the correct card and port numbers. In modular switches, such as the
Cisco Cat6500 series, use the switch's 'native' module and port numbers.  For
switches that do not have expansion modules, the card number is assumed to be
0. Make sure to get the current_speed (this time, in Mbps) and duplex correct.
181 182 183 184

Now, go into the wires table and add wires connecting the switch interfaces you
just created. Make sure to set the type to 'Trunk'. In this case, it doesn't
matter which switch you use as node 1 (the node_id1, card1, port1 columns), and
185 186
which you use as node 2. If you have a non-modular switch, all ports are
considered to be on card 0.
187 188 189 190 191 192

7) Create switch stacks

Into the switch_stack_types table, create entries for each stack in your
network. A stack is set of switches that share common VLANs. Usually, your
experimental switches will be one stack, and the control network another. You
193 194 195 196 197 198
should create a stack even if there is only one switch in it. The control
stack should be named 'Control', and the experimental stack 'Experiment' (make
sure to get the capitalization right). The 'leader' field should be set to the
node_id of the master switch in the stack.  The stack_type column is used by
snmpit to determine which module to use to talk to the stack. The currently
supported values are 'cisco' and 'intel'. Making a control network stack is
199 200 201 202 203 204 205 206 207 208 209 210
optional. There are a few columns in this table that you will need to set:
supports_private: (Cisco only) This switch can make private VLANs - probably
    best to leave it as 0
single_domain: (Cisco only) Means that all switches in the stack use VTP to
    maintain a common set of VLANs. This means that we will only create VLANs
    on the stack leader - if set to 0, we create VLANs on each switch
snmp_community: The SNMP community string we use for read-write access to the
    switch. Most switches use 'private' by default.
min_vlan: The smallest VLAN number that the testbed software will use. If left
    NULL, defaults to 2.
max_vlan: Ditto, but the biggest number. Defaults to 1000.
211 212 213

Finally, add switches to these stacks by putting entries in the switch_stacks
table, listing the node_id of each switch and the stack_id of the stack it
214 215
belongs to. You can leave the 'is_primary' column of these rows with its default
value (1).

##### Step 4 - Setup control hardware
##### Local Only

1) Power controllers
221 222 223 224 225 226 227 228 229 230 231 232 233 234

The only two supported power controllers at this time are the APC AP9210 and
the BayTech RPC27. Theoretically, other SNMP and serial-controlled power
controllers should not be hard to support.

If your power controllers are controlled via IP and SNMP, add a node type ander
interface type for them. Then, give them an entries in the nodes, interfaces,
and wires tables. Make sure to get the IP address correct in the interfaces
table. The role listed in the nodes table should be 'powerctrl'.

If you have serial power controllers, just create entries node_types and nodes
- no interfaces or wires. Again, the role in the nodes table should be

##### Step 5 - Images and OSids
236 237 238 239

These will depend on the image(s) and any OSKit kernels you've recieved, or
built yourself. Since you're probably using disk images from Utah, the best
thing to do is ask us for the appropriate lines for the os_info and images
240 241 242 243 244 245 246 247
tables. Make sure to get the PXEBOOT, PXEFRISBEE, and PXEFBSD OSIDs, which are
required to load disk images.  For a widearea-only testbed, no images entries
in the images table a required, but OSIDs are still useful, to determine what a
node's capabilities are. 

##### Step 6 - Setup nodes

To add nodes to the testbed, see setup-nodes.txt .