1. 30 Jul, 2018 1 commit
    • Leigh Stoller's avatar
      More OPSVM changes; now boss can be a XEN VM. The wrinkle here is that · fb85c44d
      Leigh Stoller authored
      if we need to assign a routable IP to boss, we also need a routable IP
      for the OPS VM, and the easiest way to do that is with an address
      pool (count=1). Also changes to pass that IP address to the XEN
      clientside code so that it alters the antispoofing firewall rules
      that every XEN VM gets.
      fb85c44d
  2. 12 Sep, 2017 1 commit
  3. 26 Jul, 2017 1 commit
    • Mike Hibler's avatar
      Support for per-experiment root keypairs (Round 1). See issue #302. · c6150425
      Mike Hibler authored
      Provide automated setup of an ssh keypair enabling root to login without
      a password between nodes. The biggest challenge here is to get the private
      key onto nodes in such a way that a non-root user on those nodes cannot
      obtain it. Otherwise that user would be able to ssh as root to any node.
      This precludes simple distribution of the private key using tmcd/tmcc as
      any user can do a tmcc (tmcd authentication is based on the node, not the
      user).
      
      This version does a post-imaging "push" of the private key from boss using
      ssh. The key is pushed from tbswap after nodes are imaged but before the
      event system, and thus any user startup scripts, are started. We actually
      use "pssh" (really "pscp") to scale a bit better, so YOU MUST HAVE THE
      PSSH PACKAGE INSTALLED. So be sure to do a:
      
          pkg install -r Emulab pssh
      
      on your boss node. See the new utils/pushrootkeys.in script for more.
      
      The public key is distributed via the "tmcc localization" command which
      was already designed to handle adding multiple public keys to root's
      authorized_keys file on a node.
      
      This approach should be backward compatible with old images. I BUMPED THE
      VERSION NUMBER OF TMCD so that newer clients can also get back (via
      rc.localize) a list of keys and the names of the files they should be stashed
      in. This is used to allow us to pass along the SSL and SSH versions of the
      public key so that they can be placed in /root/.ssl/<node>.pub and
      /root/.ssh/id_rsa.pub respectively. Note that this step is not necessary for
      inter-node ssh to work.
      
      Also passed along is an indication of whether the returned key is encrypted.
      This might be used in Round 2 if we securely implant a shared secret on every
      node at imaging time and then use that to encrypt the ssh private key such
      that we can return it via rc.localize. But the client side script currently
      does not implement any decryption, so the client side would need to be changed
      again in this future.
      
      The per experiment root keypair mechanism has been exposed to the user via
      old school NS experiments right now by adding a node "rootkey" method. To
      export the private key to "nodeA" and the public key to "nodeB" do:
      
          $nodeA rootkey private 1
          $nodeB rootkey public 1
      
      This enables an asymmetric relationship such that "nodeA" can ssh into
      "nodeB" as root but not vice-versa. For a symmetric relationship you would do:
      
          $nodeA rootkey private 1
          $nodeB rootkey private 1
          $nodeA rootkey public 1
          $nodeB rootkey public 1
      
      These user specifications will be overridden by hardwired Emulab restrictions.
      The current restrictions are that we do *not* distribute a root pubkey to
      tainted nodes (as it opens a path to root on a node where no one should be
      root) or any keys to firewall nodes, virtnode hosts, delay nodes, subbosses,
      storagehosts, etc. which are not really part of the user topology.
      
      For more on how we got here and what might happen in Round 2, see:
      
          #302
      c6150425
  4. 25 Jul, 2017 1 commit
  5. 05 Jun, 2017 1 commit
    • Leigh Stoller's avatar
      Working on issue #269 ... · ad2a3e70
      Leigh Stoller authored
      Add new script to "deprecate" images:
      
      	boss> wap deprecate_image
      	Usage: deprecate_image [-e|-w] <image> [warning message to users]
      	Options:
      	       -e     Use of image is an error; default is warning
      	       -w     Use of image is a warning
      
      When an image is deprecated with just warnings, new classic experiments
      generate warnings in the output. Swapping in an experiment also
      generates warnings in the output, but also sends email to the user.
      When the image set for error, both new experiment and swapin will fail
      with prejudice.
      
      Same deal on the Geni path; we generate warnings/errors and send email.
      Errors are reflected back in the Portal interface.
      
      At the moment the image server knows nothing about deprecated images, so
      the Portal constraint checker will not be bothered nor tell the user
      until later when the cluster throws an error. As a result, when we
      deprecate an image, we need to do it on all clusters. Needs to think
      about this a bit more.
      ad2a3e70
  6. 01 Jun, 2017 1 commit
  7. 31 May, 2017 1 commit
  8. 03 Jan, 2017 1 commit
  9. 07 Apr, 2016 2 commits
  10. 18 May, 2015 1 commit
  11. 15 May, 2015 1 commit
    • Leigh Stoller's avatar
      Directory based image paths. · 3a21f39e
      Leigh Stoller authored
      Soon, we will have images with both full images and deltas, for the same
      image version. To make this possible, the image path will now be a
      directory instead of a file, and all of the versions (ndz,sig,sha1,delta)
      files will reside in the directory.
      
      A new config variable IMAGEDIRECTORIES turns this on, there is also a check
      for the ImageDiretories feature. This is applied only when a brand new
      image is created; a clone version of the image inherits the path it started
      with. Yes, you can have a mix of directory based and file based image
      descriptors.
      
      When it is time to convert all images over, there is a script called
      imagetodir that will go through all image descriptors, create the
      directory, move/rename all the files, and update the descriptors.
      Ultimately, we will not support file based image paths.
      
      I also added versioning to the image metadata descriptors so that going
      forward, old clients can handle a descriptor from a new server.
      3a21f39e
  12. 13 Mar, 2015 1 commit
  13. 04 Aug, 2014 1 commit
    • Kirk Webb's avatar
      Add support for disabling dhcp for an experiment/user/group · ca561d42
      Kirk Webb authored
      In support of the collaboration effort with CHPC, I've added a check
      and actions for a new feature, ExternalNodeManagment.  When swapping in,
      the swapper, group, and experiment are checked for the feature.  Since
      there doesn't appear to be a way to ask for an experiment feature in the
      parser, I'm punting for now and just letting the experiment inherit this
      feature from the user and/or project/group.
      
      The effect of this feature is that nodes will be omitted from the
      dhcpd.conf file when allocated to a marked experiment, and put back
      in when freed from that experiment.
      
      We will also use this feature to force nodes to reboot even when they
      are in PXEWAIT during the OS setup phase.  Ultimately the point is for
      something else to take over the boot process and manage the experiment's
      nodes for the duration of the experiment.
      ca561d42
  14. 05 Jun, 2013 1 commit
  15. 14 May, 2013 1 commit
    • Leigh Stoller's avatar
      Add new script to do arp lockdown on boss. · f5cc889a
      Leigh Stoller authored
      The other version is only for the client side (subboss,ops), but does
      not work on real boss. Also hooked into tbswap so that the arps are
      updated during swapin/swapout. Also change tmcd to return arp
      directives for all containers, not just on shared nodes.
      f5cc889a
  16. 27 Nov, 2012 1 commit
    • Leigh Stoller's avatar
      Start saving the virtual and physical state after each swapin/swapmod · 229d2bcc
      Leigh Stoller authored
      so that we can do post-mortem debugging. I'm adding this so I can
      track down the current snmpit failures, but might useful later too.
      
      Sadly, mysql dump files have no table metadata so once the schema
      changes, these files will become less useful if we ever want to load
      them into a running DB. It would be nie to dump them in a format that
      allows this, but will be a bunch more work.
      229d2bcc
  17. 26 Nov, 2012 1 commit
  18. 01 Oct, 2012 1 commit
  19. 26 Sep, 2012 1 commit
  20. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  21. 04 Sep, 2012 1 commit
  22. 27 Aug, 2012 1 commit
  23. 01 Aug, 2012 1 commit
    • Leigh Stoller's avatar
      This commit adds some simple support for using the Infiniband on the · 997b21b5
      Leigh Stoller authored
      Probe Cluster. The problem is that the IFB is a shared network that
      every node attaches to, which can looks like an ethernet device that
      can ifconfig'ed. In other words, one big lan.
      
      But we still want the user to be able to create a lan so that they can
      interact with it in thei NS file like any other network.
      
      The NS syntax is:
      
      	set lan2 [$ns make-lan "node1 node2 node3" * 0ms]
      	tb-set-switch-fabric $lan2 "infiniband"
      
      The switch fabric tells the backend to do IP assignment for the
      specific global network. Yes, I tried to be a little but general
      purpose. Lets see how this actually turns out.
      
      This first commit treats the fabric as a single big lan on the same
      subnet.
      
      NOTE 1: Since the unroutable IP space is kinda small, but the Probe
      Cluster is really big, we can easily run out of bits if we tried to do
      assignment on virtual topos. Instead, fabrics get their IP allocation
      at swapin time, and the allocations are deleted when the experiment is
      swapped out. The rationale is that the number of swapped in
      experiments is much much smaller then the number of possible topos
      that can be loaded into the DB. Still might run out, but less likely.
      
      The primary impact of above is that IP assignments can change from
      one swap to another, but this is easy to deal with if the user is
      scripting their experiment; the IP allocation is available via the
      XMLRPC interface.
      
      NOTE 2: The current code allocates from a single big network, which
      makes it easy for users to mess each other up if they start doing
      things by hand. Ultimately, we want each lan in each experinent to use
      their own subnet, but that is going to take more work, so lets do it
      in the second phase.
      
      The definition of "network fabrics" is in the new network_fabrics
      tables. As an example for probe:
      
      	INSERT INTO `network_fabrics` set
      		idx=NULL,
      		name='ifband',
      		created=now(),
      		ipalloc=1, ipalloc_onenet=1,
      		ipalloc_subnet='192.168.0.0',ipalloc_netmask='255.255.0.0'
      997b21b5
  24. 23 Jul, 2012 1 commit
  25. 26 Jun, 2012 1 commit
  26. 06 Jun, 2012 1 commit
  27. 04 Jun, 2012 3 commits
  28. 30 Apr, 2012 1 commit
    • Mike Hibler's avatar
      More minor cleanups exposed by FBSD9 install. · ffb52cd0
      Mike Hibler authored
      Bogus param list for AddPerExperimentSwitchStack; can't have scalar args
      after a list arg.
      
      Missing mailing list TBERRORSEMAIL in defs-elabinelab. Caused redundant
      "testbed-ops" addresses in /etc/aliases.
      
      Weed out said redundant mailing list names when creating aliases during
      ops-install.
      
      named.reload script has gone away. Use rndc instead.
      ffb52cd0
  29. 24 Apr, 2012 1 commit
  30. 11 Apr, 2012 1 commit
    • Leigh Stoller's avatar
      So this commit allows a vlan to be "shared" bewteen experiments. By · dae29101
      Leigh Stoller authored
      shared, I mean that an experiment can request that a port be put into
      a vlan belonging to another experiment. This started out as a hack to
      support openflow enabled vlans in Geni, but then I got a request to
      make it a little more general purpose. You all know how that goes.
      
      Okay, say you have an experiment E1 in some project and that
      experiment has a link or lan call "lan0". You want other experiments
      to be able to stick ports in that vlan. On boss, you would do this
      after E1 is swapped in:
      
      boss> wap sharevlan -o testbed,E1 lan0 mysharedlan
      
      The -o option says to make the vlan open to anyone; without that
      option, only admins can swap in an experiment that requests a port in
      lan0.  The token "mysharedlan" is just a level of indirection for the
      NS file (or rspec).
      
      Next you create a new experiment E2, and in your NS file:
      
      	$ns make-portinvlan $n1 "mysharedlan"
      
      which says to create a lan with a interface on node n1, in the vlan
      named by the token mysharedlan. The token keeps specific pid/eids out
      of the NS file. 
      
      When E2 is swapped in, assign does its thing, and the selected port is
      added to the members list for lan0 in testbed,E1 and then we call
      snmpit with the syncvlansfromtables (-X) option to get the port added.
      
      When E2 is swapped out, we undo the members list and call snmpit with
      the -X option again.
      
      The access issue is a bit of hack of course (open or admins) but I did
      not want to invent a new permission mechanism (yet).
      
      And of course, this is still a work in progress.
      dae29101
  31. 01 Feb, 2012 1 commit
  32. 28 Nov, 2011 1 commit
  33. 11 Nov, 2011 1 commit
  34. 22 Sep, 2011 1 commit
    • Leigh Stoller's avatar
      Changes to "panic" mode. · 5c1ced81
      Leigh Stoller authored
      * Any experiment can now be paniced; formerlly, only firewalled
        experiments could be paniced. There are new options in the admin
        menu section of the Show Experiment page. ProtoGeni experiments can
        also be paniced.
      
        At the moment, this is an admin-only option. Should it be?
      
      * There are now two levels of panic. The first level (and the default)
        is to throw all the machines into the Admin MFS and reboot them.
      
        Level 2 (OH NO!) disables the control network. Not sure we need two
        levels, but hey, it was easy.
      
      * Admins can swapout a paniced experiment. There is new code in tbswap
        to watch for a paniced experiment, and zaps the disks.
      
      * Add menu option to clear the panic. Also admin-only. 
      
      * Minor tweaks to Show Experiment when experiment is paniced.
      5c1ced81
  35. 20 Jun, 2011 1 commit
    • Leigh Stoller's avatar
      "Improvements" to protogeni tunnels and UpdateSliver(); previously we · b9b443a6
      Leigh Stoller authored
      were tearing down and recreating tunnels during UpdateSliver(), cause
      Experiment->RemovePhysicalState() assumes that all physical state is
      created in the mapper, but in Protogeni tunnels are created outside
      the mapper, and so we were losing the tunnels. My original hack was to
      notice that and recreate them. But then I added sliver ids to tunnel
      interfaces, and if the tunnels are recreated, the sliver ids are going
      to change, and that will annoy users. So this change deals with this
      by preventing protogeni tunnels from getting removed by the mapper.
      b9b443a6
  36. 18 May, 2011 1 commit
    • David Johnson's avatar
      Bug fixes and per-experiment VLAN removal. · 8fbe1ab3
      David Johnson authored
      Now that we have logical wires, we *must* actually make sure that the
      VLANs created on the per-experiment switch are removed at swapout (i.e.,
      are at least removed from the vlans table), so that any logical wires can
      be safely removed.  For now, we actually invoke snmpit to remove them.
      This is probably a bad idea, because who knows what state the user left
      the switch in prior to swapout.
      
      The bugfixes are just me forgetting that \ does not continue a line in
      perl :).
      8fbe1ab3
  37. 17 May, 2011 1 commit
    • David Johnson's avatar
      Add per-experiment switch support. · ea496d44
      David Johnson authored
      Per-experiment switch stacks only come into being if the experiment
      actually has a switch allocated to it.  If not, tbswap and snmpit
      should function unchanged.  If there is a per-experiment stack that needs
      configuration, we first invoke normal snmpit in the normal place, but we
      use the new snmpit option `--skip-supplied' in combination with -S to skip
      the per-experiment stack.  We then configure the per-experiment stack by
      itself with -S after os_setup has completed.
      
      There are some new functions in the db backend stuff to create, modify,
      and remove per-experiment switches.
      
      There is some new code in snmpit to do the --skip-supplied filtering.  I
      also put all the -S, -i, and --skip-supplied stuff into portstats...
      because we also can't call portstats on a per-experiment switch in tbswap;
      otherwise it will hang and/or fail.
      ea496d44