|
|
This document contains some configuration guidelines that we (Utah) have
|
|
|
found useful to improve the performance of our Cisco switches.
|
|
|
|
|
|
All commands given are to be entered at the `(enable)` prompt on your Cisco
|
|
|
switches. They are for CatOS -- switches that run IOS will likely have
|
|
|
alternate versions of these commands, most of which can be found at the
|
|
|
bottom.
|
|
|
|
|
|
<ports> means a list of ports, which on the CatOS command line can include
|
|
|
lists and ranges, such as "3/1,3/2" or "3/1-48" or "3/1-48,4/1-48,5/1-48".
|
|
|
|
|
|
### Allowing ports to come up quicker
|
|
|
|
|
|
This is useful on both the experimental and control networks:
|
|
|
|
|
|
```
|
|
|
set spantree portfast <ports> enable
|
|
|
```
|
|
|
|
|
|
Use this on all ports that are directly connected to nodes, servers, power
|
|
|
controllers - anything this is not another switch. Normally, the switch
|
|
|
waits a while (several seconds) when a port first comes up before
|
|
|
forwarding traffic from the port - it does so to prevent loops in the
|
|
|
switch topology. The main place you will see the benefit of this is on the
|
|
|
control net - with portfast disabled, the first several DHCP packets sent
|
|
|
by booting nodes will be dropped, causing the DHCP to take much longer than
|
|
|
necessary.
|
|
|
|
|
|
### Reducing stray traffic
|
|
|
|
|
|
Disable spanning tree protocol (STP). If enabled, STP sends out packets
|
|
|
approximately every two seconds on every port. You can disable it on all
|
|
|
VLANs with the command:
|
|
|
|
|
|
```
|
|
|
set spantree disable all
|
|
|
```
|
|
|
|
|
|
There are two major consequences (for our purposes) of disabling STP:
|
|
|
|
|
|
1. You cannot have **any** loops in your switch topology, or bad things will happen.
|
|
|
2. VLAN prunning on trunks will not work, causing broadcast traffic to be
|
|
|
forwarded across trunks that it does not need to cross. We've added
|
|
|
features to snmpit to manually do STP's job in this case, so this
|
|
|
problem is taken care of.
|
|
|
|
|
|
You must have STP disabled on **all** switches that are trunked together!
|
|
|
If it is enabled on even one, STP traffic will be seen on all of them.
|
|
|
|
|
|
The switch does not trust you to use portfast responsibly. So, it has a
|
|
|
'bpdu-guard' feature that helps guard against loops. Turn this feature off
|
|
|
with the command:
|
|
|
|
|
|
```
|
|
|
set spantree portfast bpdu-guard disable
|
|
|
```
|
|
|
|
|
|
Cisco uses a protocol called 'CDP' to discover other Cisco devices. This
|
|
|
sends out small packets every two minutes. You can disable it with:
|
|
|
|
|
|
```
|
|
|
set cdp disable <ports>
|
|
|
```
|
|
|
|
|
|
Ideally, you should only disable CDP on ports that do not have other Cisco
|
|
|
devices attached, but in practice, running with CDP disabled on all ports
|
|
|
is fine.
|
|
|
|
|
|
Switch ports will, by default, try to negotiate trunking and
|
|
|
channeling. Cisco provides a handy macro to disable both of these and also
|
|
|
enable portfast on the ports:
|
|
|
|
|
|
```
|
|
|
set port host <ports>
|
|
|
```
|
|
|
|
|
|
### Setting MAC address aging time
|
|
|
|
|
|
We have found that some experimenters use applications, kernels, etc. that
|
|
|
only receive traffic, not send it. This presents a problem, because it
|
|
|
prevents the switch from learning which port the node is on, and thus
|
|
|
broadcasting traffic for it to every port in the VLAN. This can be solved
|
|
|
by 'priming' - i.e. having the receive-only node send some traffic (like an
|
|
|
ARP response) at the beginning of the experiment. However, the default
|
|
|
aging time of 300 seconds makes this impractical. So, we have disabled this
|
|
|
aging, making learned MACs permanent (until the VLAN is torn down.)
|
|
|
|
|
|
You must do this for each VLAN, with the command:
|
|
|
|
|
|
```
|
|
|
set cam agingtime <vlan> 0
|
|
|
```
|
|
|
|
|
|
For convenience, we've supplied a file in the 'doc' folder of the emulab
|
|
|
source repository, called ['no-cam-aging.cfg'](https://gitlab.flux.utah.edu/emulab/emulab-stable/blob/master/doc/no-cam-aging.cfg)
|
|
|
that disables aging on VLANs 2-999 (those potentially used by our
|
|
|
software). Transfer this file to the switch with the `copy tftp config`
|
|
|
command.
|
|
|
|
|
|
We also suggest that you do this on your control network as well. Part of
|
|
|
the booting process leaved the nodes sitting dormant at a boot loader for
|
|
|
extended periods of time, so the switch will tend to forget their
|
|
|
MACs. Turning off aging is not critical, but we suggest it because it will
|
|
|
reduce stray traffic while the switch relearns MAC addresses.
|
|
|
|
|
|
### IGMP snooping on the control network
|
|
|
|
|
|
In order for multicast to work correctly, you need to make sure that IGMP
|
|
|
snooping is enabled on the control switch. This is needed for frisbee, our
|
|
|
disk-loading system. It's up to you whether you want to enable this on the
|
|
|
experimental switches. In general, we recommend it so that your
|
|
|
experimenters can use multicast, but it does seem that unexpected or
|
|
|
malformed multicast packets have an easier time DOSing the switch control
|
|
|
processor than unicast traffic. On CatOS, the command is:
|
|
|
|
|
|
```
|
|
|
set igmp enable
|
|
|
```
|
|
|
or on IOS:
|
|
|
```
|
|
|
ip igmp snooping
|
|
|
```
|
|
|
|
|
|
Cisco has a [good reference](http://www.cisco.com/en/US/products/hw/switches/ps708/products_tech_note09186a00800b0871.shtml)
|
|
|
for IGMP snooping on Ciscos.
|
|
|
|
|
|
### Setting up multicast between multiple switches
|
|
|
|
|
|
If you have more than one switch on the experimental or control networks,
|
|
|
you may need to do a little setup to get multicast between them. The
|
|
|
symptom of this problem is that multicast doesn't work between two nodes on
|
|
|
different switches, and if you run 'show multicast groups' on each switch,
|
|
|
some will show the group as existing, and others will not.
|
|
|
|
|
|
Run the following command for both sides (i.e. on both switches) of every
|
|
|
trunk link:
|
|
|
|
|
|
```
|
|
|
set multicast router 1/1
|
|
|
```
|
|
|
|
|
|
(This assumes that port 1/1 is your trunk link.) If you are using
|
|
|
EtherChannel to bond multiple links together to form a single trunk, you
|
|
|
only need to run this command for the first port in the channel.
|
|
|
|
|
|
We had some problems running this command on the trunk on one of our
|
|
|
switches. It failed with the error:
|
|
|
|
|
|
```
|
|
|
Failed to add port 2/1 to multicast router port list
|
|
|
```
|
|
|
|
|
|
What we finally did to resolve this was to tear down the trunk link and
|
|
|
EtherChannel that port was a part of, run the command on it (which
|
|
|
succeeded this time), and then build the EtherChannel and trunk back up.
|
|
|
|
|
|
### Setting the clock
|
|
|
|
|
|
Since boss is an NTP server, you should set your switches to sync time with
|
|
|
it. On CatOS, this is accomplished with:
|
|
|
|
|
|
```
|
|
|
set ntp server 10.11.12.1
|
|
|
set ntp timezone MST -7
|
|
|
set ntp summertime MDT
|
|
|
set ntp summertime enable
|
|
|
set ntp summertime recurring
|
|
|
set ntp client enable
|
|
|
show time
|
|
|
```
|
|
|
|
|
|
Of course, you'll need to replace 10.11.12.1 with the IP address your boss
|
|
|
node uses to talk to the switches (usually its control-hardware interface),
|
|
|
and 'MST', -7, and 'MDT' with the names of your timezone and its offset
|
|
|
from GMT. If you don't use daylight savings time, leave out the
|
|
|
'summertime' steps, and instead do:
|
|
|
|
|
|
```
|
|
|
set ntp summertime enable
|
|
|
```
|
|
|
|
|
|
Watch the output of `show time` for a while to make sure the clock syncs
|
|
|
up. It may take a few minutes.
|
|
|
|
|
|
### Setting the login password and the enable (superuser) password.
|
|
|
|
|
|
```
|
|
|
set password
|
|
|
set enablepass
|
|
|
```
|
|
|
|
|
|
### IOS commands
|
|
|
|
|
|
If you are running IOS on your switches, here are a few notes that may help
|
|
|
you 'translate' the above commands. Note that to save changes (stick across
|
|
|
reboots) you must do this:
|
|
|
|
|
|
```
|
|
|
copy running-config startup-config
|
|
|
```
|
|
|
|
|
|
Interfaces in CatOS are named as module/port, while interfaces in IOS are
|
|
|
named as TypeModule/Port. For example, if module 1 has gigabit interfaces,
|
|
|
what you would call 1/1 in CatOS is Gi1/1 in IOS. 100Mbit ethernet is
|
|
|
'Fa'. (Really, these are 'GigabitEthernet' and 'FastEthernet' respectively,
|
|
|
but you can abbreviate them.)
|
|
|
|
|
|
In order to operate on many interfaces at once, you can issue configuration
|
|
|
commands like this:
|
|
|
|
|
|
```
|
|
|
range gi1/1 - 48, gi2/1 - 48, gi3/1 - 48
|
|
|
```
|
|
|
|
|
|
which would configure all 48 gigabit interfaces on modules 1, 2, and 3.
|
|
|
|
|
|
The equivalent of 'set port host' (which sets portfast, disables BPDU
|
|
|
guard, etc.) is:
|
|
|
|
|
|
```
|
|
|
switchport host
|
|
|
```
|
|
|
|
|
|
applied to an interface or a range of interfaces, as in:
|
|
|
|
|
|
```
|
|
|
interface range gi1/1 - 48, gi2/1 - 48, gi3/1 - 48
|
|
|
switchport host
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
In order to disable the Spanning Tree Protocol, you would use:
|
|
|
|
|
|
```
|
|
|
no spanning-tree vlan 1-1005
|
|
|
```
|
|
|
|
|
|
In order to create a VLAN and set its name:
|
|
|
|
|
|
```
|
|
|
vlan 10
|
|
|
name control-hardware
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
In order to set the IP address of the interface in VLAN 10:
|
|
|
|
|
|
```
|
|
|
interface vlan 10
|
|
|
ip address 10.11.13.183 255.255.255.0
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
In order to enable an interface:
|
|
|
|
|
|
```
|
|
|
interface vlan10
|
|
|
no shutdown
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
In order to remove a VLAN:
|
|
|
|
|
|
```
|
|
|
no vlan 1000
|
|
|
```
|
|
|
|
|
|
To put an interface into a VLAN:
|
|
|
|
|
|
```
|
|
|
interface gi0/1
|
|
|
switchport access vlan 10
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
To turn on trunking for an interface:
|
|
|
|
|
|
```
|
|
|
interface gi0/1
|
|
|
switchport mode trunk
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
To turn off trunking for an interface:
|
|
|
|
|
|
```
|
|
|
interface gi0/1
|
|
|
switchport mode access
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
To put interfaces into an EtherChannel:
|
|
|
|
|
|
```
|
|
|
interface range gi1/41 - 48
|
|
|
channel-group 1 mode on
|
|
|
exit
|
|
|
```
|
|
|
|
|
|
(Notes: If you want to make more than one channel, give each set of ports a
|
|
|
different channel number. And, now, you will configure the whole channel as
|
|
|
'interface port-channel 1'.)
|
|
|
|
|
|
To set the native VLAN on a trunk:
|
|
|
|
|
|
```
|
|
|
interface gi0/1
|
|
|
switchport trunk native vlan 1
|
|
|
```
|
|
|
|
|
|
To set the read-write SNMP community string to 'public':
|
|
|
|
|
|
```
|
|
|
snmp-server community public rw
|
|
|
```
|
|
|
|
|
|
To globally disable the Cisco Router Discovery (CDP) protocol:
|
|
|
|
|
|
```
|
|
|
no cdp run
|
|
|
```
|
|
|
|
|
|
To enable NTP on the switch:
|
|
|
|
|
|
```
|
|
|
configure terminal
|
|
|
ntp server 10.11.12.1
|
|
|
clock timezone MST -7
|
|
|
clock summer-time MDT recurring
|
|
|
exit
|
|
|
show clock
|
|
|
```
|
|
|
|
|
|
To set the encrypted password for privileged access:
|
|
|
|
|
|
```
|
|
|
enable secret password_here
|
|
|
```
|
|
|
|
|
|
To set the password for privileged access (used when there is no enable
|
|
|
secret and when using older software):
|
|
|
|
|
|
```
|
|
|
enable password password_here
|
|
|
```
|
|
|
|
|
|
Set password for console access:
|
|
|
|
|
|
```
|
|
|
(config)#line console 0
|
|
|
(config-line)#login
|
|
|
(config-line)#password password_here
|
|
|
```
|
|
|
|
|
|
Set password for telnet access (password must be set to access router
|
|
|
through telnet):
|
|
|
|
|
|
```
|
|
|
(config)#line vty 0 4
|
|
|
(config-line)#login
|
|
|
(config-line)#password password_here
|
|
|
```
|
|
|
|
|
|
* [Prev](install/Power Controllers)
|
|
|
* [Next](install/Setting up the Console Server)
|
|
|
* [Home](install/Installing Emulab) |