Hardware infrastructure upgrades for CloudLab
A few semi-related things that need to be done:
- Subboss for node imaging (frisbee). There are occasional protracted delays when staring up frisbee sessions. Not a huge problem, but if we can get a dedicated 10Gb link directly on the node control net, it will help. I have a dual-port X520 card, and we can probably use one of the old 1U HP boxes or a 2U Dell box for this purpose. Just need to make sure it has a PCIe 2.0 (?) slot. The ulterior motive for a subbos box is...
- Provide a machine to hook the USB based serial port hub (for user allocatable switch consoles) to. The hub works fine under FreeBSD (11.1) which is what the subboss will be. See #382 (closed), last few comments.
- Get dedicated 10Gb links into the boss and ops VMs. Right now they (along with the control node) share a single 10Gb link The server has a dual-port 10Gb card, and we can use a spare 1Gb interface for the control interface for the control node itself. This is the way the mothership control node is setup.
- Get the filesave box hooked up and running. This box is intended primarily as an offsite backup for Emulab/Flux, but we have designed things such that it will also backup the critical CloudLab/Apt servers. The box is setup, we just need to get it down there and running. This will hook in via 1-2 10Gb interfaces.
- Scare up enough 10Gb ports on
bighp1
to do all this. We currently have two broken out 10Gb ports. The layout:- 2/0/19:1: currently control node, use for passthrough to boss (requires re-config of boss VM, need to have control node 1Gb and ops 10Gb broken out first)
- 2/0/19:2: currently unused, use for passthrough to ops (can be setup anytime, requires reconfig of ops VM)
- 2/0/19:3: currently unused, use for ms-subboss (can be setup anytime)
- 2/0/19:4: currently unused, use for ms-procurve (maybe issues here since this link is currently fiber, have to see if Dell DAC cable will work on Procurve switch, I have reason to believe it will not--test this first)
- 2/0/20:1: currently the fiber uplink to ms-procurve, move to 2/0/19:4 and use this port for filesave box.
- 2/0/20:2: currently unused, second link to filesave box?
- 2/0/20:3: currently unused, future expansion
- 2/0/20:3: currently unused, future expansion
We only have one 3m breakout cable right now, we will have to order a second. I am hoping that bighp1 will be okay with the Dell DAC cable. We do have 1m HP (Mellanox?) breakouts. Those are too short to use here, but maybe we could swap one of the 3m netscout<->node breakouts for a one meter for nearby nodes.