Commit 9ebf916a authored by Robert Ricci's avatar Robert Ricci

Remove vlans.txt, because it's out of date. The acl files themselves

on boss are the best 'documentation' of the firewall rules. Update
setup-network.txt to reflect this.

Also fixed up some stuff in setup-network.txt that we listed as
alternate, untried, configurations. In fact, we use them now, and
they are the recommended way to do things.
parent ff474617
...@@ -21,12 +21,15 @@ these switches if they are Ciscos.) ...@@ -21,12 +21,15 @@ these switches if they are Ciscos.)
##### Splitting up the control net ##### Splitting up the control net
We basically have 4 VLANs on the control network: We basically have 5 VLANs on the control network:
'external' contains our connection to the outside world 'external' contains our connection to the outside world
'private' contains the boss node, and all IP-controllable devices 'private' contains the boss node, and our backup server
(namely, power controllers and switch IP interfaces)
'public' contains our ops node 'public' contains our ops node
'control' contains the control net interfaces of all experimental nodes 'control' contains the control net interfaces of all experimental nodes
'control-hardware' contains all IP-controllable devices (namely, power
controllers and switch IP interfaces, as well as a second
interface on the boss node. This VLAN uses private IP addresses,
and does NOT contain a router interface.
This is done for security - we route (using a module in our control-net switch) This is done for security - we route (using a module in our control-net switch)
between these VLANs, and do some firewalling between each of them. The main between these VLANs, and do some firewalling between each of them. The main
...@@ -62,8 +65,8 @@ I'd recommend at least going after #1, #2, and #3. We've put a lot of thought ...@@ -62,8 +65,8 @@ I'd recommend at least going after #1, #2, and #3. We've put a lot of thought
into this, so if you're wondering how your choice here will affect other into this, so if you're wondering how your choice here will affect other
aspects of security, ask Utah, and we can probably help you. aspects of security, ask Utah, and we can probably help you.
A (somewhat outdated) overview of the firewall rules we have in place can be Since our firewall rules change frequently as we add new services, it's best to
found in vlans.txt. contact Utah and ask us for the current set.
##### Connecting the contol net to the experimental net ##### Connecting the contol net to the experimental net
...@@ -73,13 +76,12 @@ your control net to the experimental net. However, you will need to be _very_ ...@@ -73,13 +76,12 @@ your control net to the experimental net. However, you will need to be _very_
careful that no experimental traffic can leak across this line, and that it careful that no experimental traffic can leak across this line, and that it
cannot be used as a 'back door' into the private net by nodes. cannot be used as a 'back door' into the private net by nodes.
On the control switch, it should be in the same VLAN as your boss node. On the control switch, it should be put into the 'control-hardware' VLAN.
On the experimental switch, it should be in the same VLAN as the switch's IP On the experimental switch, it should be in the same VLAN as the switch's IP
interface. The simplest way to accomplish this is just to leave it in VLAN 1, interface. The best thing to do is create a new VLAN, and move the switch's IP
the default one. However, you want to make sure that no ports on the nodes can interface into it. On CatOS, you accomplish this with:
talk to it, so you'll want to disable them all. (Our software will re-enable set interface sc0 <vlan #>
them, and disable them as needed.)
You will also need to give the experimental switch(es) IP addresses. With You will also need to give the experimental switch(es) IP addresses. With
CatOS, you do it like this: CatOS, you do it like this:
...@@ -126,24 +128,3 @@ after the master switch.) On this one, run ...@@ -126,24 +128,3 @@ after the master switch.) On this one, run
set vtp mode server set vtp mode server
On all the others, run On all the others, run
set vtp mode client set vtp mode client
##### Some variations on the above configuration
There are some variations on the above configuration that we haven't tried, but
may work for you.
First, you may try putting all IP-controllable control hardware on a VLAN (or
another switch/hub) hanging directly off another interface on the boss node. In
the example above, the control hardware is given 'real' IP addresses, but
prevented from talking to the outside worly by firewall rules. An alternative
to this would be to give them 'private' (ie. 10.0.0.X) addresses, and put
another interface in the boss node that talks only to these devices.
As an added measure of security on the experimental network, you could move
your switch IP interfaces (and the wire that runs from them to the control net)
out of VLAN 1. When ports are not in use, we put them in VLAN 1 and disable
them. But, if a port accidentally ends up in VLAN 1 still enabled, it could
theoretically talk to the switch and change configurations. Furthermore, we've
had circumstances in which delay nodes accidentally ended up in tis VLAN, still
forwarding packets, which created a very nasty forwarding loop. Using some
VLAN other than #1 prevents such problems from impacting the boss node.
Plans for the control switch VLANs, and description of which traffic should
(and should not) be allowed to cross between VLANs.
***** VLAN Mebers
* External: Just the wire to the Netcomm router
* Public: Plastic
* Private: paper, power controllers, IP interfaces to switches
* Control: All testbed nodes control network interfaces
***** What to allow
NOTE: ICMP gets to go from anywhere to anywhere
1) External <-> Private
ssh for logging in
http web server
smtp outbound, mail sent from the web server
ntp time synch
dns name server
nsr networker filesave to/from envy
2) External <-> Public
ssh for logging in
high ports for user proxies
nsr networker filesave to/from envy
smtp outbound, mail
nfs only to .212 subnet
snmp get machine information
3) External <-> Control
ssh for logging in
hight ports allow users to run their own
snmp get machine information
4) Private <-> Public
ssh as always
nfs plastic:/proj mounted on paper
plastic:/users mounted on paper
paper:/usr/testbed/tftpboot/proj on plastic
ntp time
dns name server
capserver capture server (tcp port 855) on paper.
5) Private <-> Control
ssh as always
ntp time synch
dns name server
dhcp including forwarded broadcasts
proxydhcp extended DHCP support (for PXE)
tftp for loading pxeboot, pxeboot loading multiboot kernels
bootinfo pxeboot <-> bootinfo daemon
tmcd the testbed master control daemon
nfs images from netdisk
cvsup cvsup client talks to cvsupd on paper.
6. Public <-> Control
ssh as always
nfs /users, all FSes for dnards
plastic:/proj on node:/proj
plastic:/q/tftpboot/proj from netdisk
high ports talk to user proxies on plastic
snmp get machine information
***** Implemenation notes:
In practice, we don't get to enter the router configuration in the above
manner. What I've tried to do, then, put fairly restrictive ACLs on the private
and public VLANs, to prevent nastiness from coming in from ANYWHERE. The
control VLAN gets few restrictions (just prevents any non-155.99.132s from
getting out)- the restrictions preventing it from using low ports to the
outside world are done on the external VLAN.
Note that most rules have to be entered symmetrically, like:
permit tcp any to any on port 22
permit tcp any on port 22 to any
This annoyance is exacerbated by the fact than many protocols have to be done
twice, once for udp, and once for tcp. This is ignored in the rules below
Also note that, due to Cisco's funky syntax, to match 155.101.128/20, we enter
155.101.128.0 0.0.15.255
And to match 155.101.132/22, it's
155.101.132.0 0.0.0.3.255
One more note: It may be possible to reduce the number of rules due to the fact
that I was learning about the process as I was doing it....
private ACL:
permit port 22 (ssh) from anywhere to anywhere
permit port 80 (http) from anywhere to anywhere
permit port 443 (https) from anywhere to anywhere
permit port 53 (domains) from anywhere to anywhere
permit port 25 (smtp) from anywhere to anywhere
permit port 123 (ntp) anywhere to anywhere
permit port 69 (tftp) from 155.101.128/24 to 155.101.128/20
permit port 6969 (bootinfo) from 155.101.128/24 to 155.101.128/20
permit port 7777 (tmcd) from 155.101.128/24 to 155.101.128/20
permit NFS ports from 155.101.128/24 to 155.101.128/20
permit NFS ports from 155.101.128/20 to 155.101.128/24
permit icmp from anywhere to anywhere
permit port 67 (bootps) to 155.101.128/24 from 155.101.128/20
permit port 68 (bootpc) to 155.101.128/20 from 155.101.128/24
permit all outgoing UDP packets from 155.101.128/24 (needed for tftp)
deny everything else
public ACL:
permit port 22 (ssh) from anywhere to anywhere
permit port 161 (snmp) from anywhere to anywhere
permit port 25 (smtp) from 155.101.129/24 to anywhere
permit port 123 (ntp) anywhere to anywhere
permit port 53 (domains) from anywhere to anywhere
permit NFS ports from 155.101.129/24 to 155.101.128/20
permit NFS ports from 155.101.128/20 to 155.101.129/24
permit icmp from anywhere to anywhere
permit all tcp ports > 1024 from anywhere to anywhere
permit all udp ports > 1024 from anywhere to anywhere
deny everything else
control ACL: (just prevent IP spoofing)
permit icmp from 155.101.132/22 to any
permit icmp froma any to 155.101.132/22
permit tcp from 155.101.132/22 to any
permit tcp form any to 155.101.132/22
permit udp from 155.101.132/22 to any
permit udp form any to 155.101.132/22
deny everything else
external ACL: (just prevent testbed machines from using low ports, except ssh)
permit port 22 (ssh) from anywhere to anywhere
permit port 161 (snmp) from anywhere to anywhere
deny all tcp ports < 1024 from 155.101.132/22 to any
***** Misc. notes:
NFS needs ports 2049 (nfsd), 4045 (lockd), and 111 (surpc)
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment