1. 15 Apr, 2019 1 commit
    • Mike Hibler's avatar
      Initial steps to enable jumbo frames on experiment interfaces. · 33beb373
      Mike Hibler authored
      This is just mods to the tmcd "ifconfig" command to include an MTU= arg.
      Right now we don't have anything in the DB for MTU, so tmcd is just returning
      "MTU=" which says to not explicitly set the MTU.
      
      It also includes the basic client-side support which I have tested on a
      physical interface with MTU=1500. Further changes will be needed to DTRT
      on virtual interfaces and their physical carrier interface.
      
      But the hope is to get the client-side part nailed down before the next
      set of images are rolled, so that we will be ready when support for the
      front-side (UI and DB state) get added.
      33beb373
  2. 11 Dec, 2018 1 commit
    • Leigh Stoller's avatar
      Changes for building/installing capture/console on control nodes: · fabd07a7
      Leigh Stoller authored
      * Makefile changes to build and install nossl versions of capture and
        console on a rack control node (or more generally, a physical node
        hosting boss/ops VMs that are not built on our XEN49 image).
      
      * Add -I (insecure) option to capture, that listens on localhost only.
      
      * Add systemd startup files for capture on ops and boss, I tested these
        on Ubuntu18.
      
      Basic instructions:
      
      * Clone the emulab-devel repo to the control node.
      
        git clone https://gitlab.flux.utah.edu/emulab/emulab-devel.git
      
      * On the control node, install the libssl devel code:
      
        sudo apt-get update
        sudo apt-get install libssl-dev
      
      * configure and build capture. Note that the obj-clientside directory might
        already exist, you can just rm -rf the directory.
      
        control> cd ~elabman
        control> mkdir obj-clientside
        control> cd obj-clientside
        control> /path/to/emulab-devel/clientside/configure
        control> make rack-control
        control> sudo make rack-control-install
        control> (cd os/capture; sudo make rack-control-startup-install)
      
      * start capture.
      
        control> sudo systemctl daemon-reload
        control> sudo systemctl start capture-boss
        control> sudo systemctl start capture-ops
      fabd07a7
  3. 25 May, 2018 1 commit
  4. 04 May, 2018 1 commit
    • David Johnson's avatar
      Clientside support for openssl 1.1.0. · 2204eda9
      David Johnson authored
      Many structs were made opaque in 1.1.0.  It's not always possible to
      do any better than a large three-way #if either because that is the
      only way to be backwards-compat in some cases.
      
      I also fixed a few minor possible leak-on-error cases in event.c.
      2204eda9
  5. 24 Apr, 2018 1 commit
  6. 11 Apr, 2018 1 commit
    • Leigh Stoller's avatar
      Initial checkin of ONIE clientside. · 72d6a8e6
      Leigh Stoller authored
      * Add onie-dongle and onie-dongle-install targets, which builds and
        installs (DESTDIR required) the bits and pieces we need. This install
        is intended to update the initram FS. ONIE operates as the admin MFS
        and the "frisbee" MFS, bootinfoclient used to emulate PXEWAIT
        waitmode.
      
      * Need to be build in the ONIE cross compiler environment, see the
        ftos.env and mlnx.env for the environment variables before config and
        build.
      
      * Basic operation is like the old CDROM; use bootinfoclient and tmcc
        bootwhat to drop into "admin" or "frisbee" mode, or boot the NOS. Use
        tmcc loadinfo and call onie-nos-install. Use a grub environment
        variable to tell grub to either boot the NOS (and then clear the
        variable) or boot into ONIE.
      72d6a8e6
  7. 02 Apr, 2018 1 commit
  8. 16 Jan, 2018 2 commits
    • Leigh Stoller's avatar
      Minor fix. · a9f92d1b
      Leigh Stoller authored
      a9f92d1b
    • Leigh Stoller's avatar
      Lots of changes for SSL enabled pubsub: · e44fc90d
      Leigh Stoller authored
      Pubsub libraries are now SSL enabled by default, so that we can talk SSL
      from a perl client. To do this we need another entry point from SWIG
      into the event code, event_register_withssl. At the same time there is a
      new entry point called event_set_sockbufsizes that calls a new pubsub
      entry point pubsub_set_sockbufsizes.
      
      The problem is that current swig generates code that does not compile,
      and since I don't know nothing about swig, I just hand crafted the two
      new routines that needed in event_wrap.c and the few extra lines that go
      into event.pm.
      
      Also change all the link lines to include the ssl/crypto libraries when
      linking.
      e44fc90d
  9. 09 Jan, 2018 1 commit
  10. 26 Jul, 2017 1 commit
    • Mike Hibler's avatar
      Support for per-experiment root keypairs (Round 1). See issue #302. · c6150425
      Mike Hibler authored
      Provide automated setup of an ssh keypair enabling root to login without
      a password between nodes. The biggest challenge here is to get the private
      key onto nodes in such a way that a non-root user on those nodes cannot
      obtain it. Otherwise that user would be able to ssh as root to any node.
      This precludes simple distribution of the private key using tmcd/tmcc as
      any user can do a tmcc (tmcd authentication is based on the node, not the
      user).
      
      This version does a post-imaging "push" of the private key from boss using
      ssh. The key is pushed from tbswap after nodes are imaged but before the
      event system, and thus any user startup scripts, are started. We actually
      use "pssh" (really "pscp") to scale a bit better, so YOU MUST HAVE THE
      PSSH PACKAGE INSTALLED. So be sure to do a:
      
          pkg install -r Emulab pssh
      
      on your boss node. See the new utils/pushrootkeys.in script for more.
      
      The public key is distributed via the "tmcc localization" command which
      was already designed to handle adding multiple public keys to root's
      authorized_keys file on a node.
      
      This approach should be backward compatible with old images. I BUMPED THE
      VERSION NUMBER OF TMCD so that newer clients can also get back (via
      rc.localize) a list of keys and the names of the files they should be stashed
      in. This is used to allow us to pass along the SSL and SSH versions of the
      public key so that they can be placed in /root/.ssl/<node>.pub and
      /root/.ssh/id_rsa.pub respectively. Note that this step is not necessary for
      inter-node ssh to work.
      
      Also passed along is an indication of whether the returned key is encrypted.
      This might be used in Round 2 if we securely implant a shared secret on every
      node at imaging time and then use that to encrypt the ssh private key such
      that we can return it via rc.localize. But the client side script currently
      does not implement any decryption, so the client side would need to be changed
      again in this future.
      
      The per experiment root keypair mechanism has been exposed to the user via
      old school NS experiments right now by adding a node "rootkey" method. To
      export the private key to "nodeA" and the public key to "nodeB" do:
      
          $nodeA rootkey private 1
          $nodeB rootkey public 1
      
      This enables an asymmetric relationship such that "nodeA" can ssh into
      "nodeB" as root but not vice-versa. For a symmetric relationship you would do:
      
          $nodeA rootkey private 1
          $nodeB rootkey private 1
          $nodeA rootkey public 1
          $nodeB rootkey public 1
      
      These user specifications will be overridden by hardwired Emulab restrictions.
      The current restrictions are that we do *not* distribute a root pubkey to
      tainted nodes (as it opens a path to root on a node where no one should be
      root) or any keys to firewall nodes, virtnode hosts, delay nodes, subbosses,
      storagehosts, etc. which are not really part of the user topology.
      
      For more on how we got here and what might happen in Round 2, see:
      
          emulab/emulab-devel#302
      c6150425
  11. 09 Feb, 2017 1 commit
  12. 17 Jan, 2017 1 commit
    • Mike Hibler's avatar
      Implement heartbeat/status reports in Frisbee. · 2be46ba4
      Mike Hibler authored
      There are three pieces here, a change to the frisbee protocol itself, an
      Emulab event component to get status back to the portal, and the surrounding
      infrastructure to make it all work.
      
      Frisbee heartbeat messages:
      
      Added a new message type to the frisbee protocol, "Progress". In theory it
      operates by having the server send a multicast progress request to its clients
      which includes an interval at which to report (or "just once") and an
      indication of what to report (nothing, progress summary, or full stats). The
      client then sends unicast "fire and forget" UDP replies according to that
      schedule. However, I took a shortcut for the moment and just added a command
      line option to the client to tell it to report a summary at the indicated
      interval (-H <interval>).  So the server never sends requests.
      
      This is implemented in the client by a fourth thread since I wanted it to
      operate independent of packet reception (which would cause clients to report
      in a highly synchronized fashion due to multicast). The server instance just
      logs progress reports into its log.
      
      This protocol addition should be fully backward compatible as both client and
      server ignore (but log) unknown messages.
      
      Emulab progress report events:
      
      When this is compiled in (-DEMULAB_EVENTS) and turned on (-E <server>), the
      frisbee server instances will send a FRISBEEPROGRESS event to the indicated
      event server for every progress report it receives (in addition to logging the
      events to its own log). Right now it will create an event with key/value pairs
      for the information in a client summary reply:
      
      TSTAMP is the client's time at which it sends the event. Could be used by the
      received to determine latency of the report if it cared (and if it assumed
      that the clocks are in sync). We don't care about this.
      
      SEQUENCE is the report number. Again, could be used by the receiver, in this
      case to detect loss, if it cared. We don't.
      
      CHUNKS_RECV is complete chunks that the client has received from the network.
      CHUNKS_DECOMP is chunks decompressed by the client.  BYTES_WRITTEN is bytes
      written to disk by the client.
      
      Any of the three can be used by the event receiver as an indication of life
      and/or progress. However, only the last would be a reasonable indicator of
      time remaining since it is the last (and slowest) phase of imaging. To
      estimate time remaining we could compare that value to the amount of
      uncompressed data that is in the image. This makes the sketchy assumptions
      that time for writes to the disk are uniform and that the number and distance
      of seeks is uniform, but it is better than a sharp stick in the eye.
      
      Emulab infrastructure:
      
      There is a new sitevar "images/frisbee/heartbeat" which can be set to a
      non-zero value to tell the frisbee MFS to fire off frisbee with -H <value>
      and thus make reports. The default value of zero means to not make reports.
      The tmcd "loadinfo" command sends this through via the HEARTBEAT=<value>
      param.
      
      REQUIRED A TMCD VERSION BUMP TO 41.
      2be46ba4
  13. 14 Oct, 2016 1 commit
  14. 08 Feb, 2016 1 commit
  15. 27 Jan, 2016 2 commits
  16. 21 Sep, 2015 1 commit
  17. 01 Sep, 2015 1 commit
  18. 24 Jun, 2015 1 commit
    • Mike Hibler's avatar
      Updates for new FreeBSD 10.1 based servers. · 480fdc70
      Mike Hibler authored
      Big changes a comin' to try to get us back on the supported path.
      
       * perl 5.14 -> 5.20
       * mysql 5.1 -> 5.5
       * php 5.4   -> 5.6
       * tcl 8.4   -> 8.6
       * number of vim patches up to 683.
      
      Not everything tested yet, but getting there.
      
      Specific changes:
      
       * New install/ports directory. New packages for FreeBSD 10.1 are version
         6.1. Cleaned up the ports' Makefiles getting rid of conditionals for
         all older versions. Also got rid of ports we don't use. Old ports tree
         is now install/oports.
      
       * Install script changes. Make sure /usr/bin/perl and /usr/local/bin/python
         links exist. Ports no longer make these but we use them in '#!'. Changes
         to mysql install and startup script--mysql has changed a LOT since we did
         the support in 4.x. Create syslog entry for named.log. Make sure php.conf
         loads the legacy "mysql" module rather than using "mysqli".
      
       * Elabinelab support. reflect new packages, remove all old packages
         (except perl) before installing new versions, install "extras" package,
         make sure sendmail cert get regenerated, make sure /usr/bin/perl link
         exists, make sure /usr/local/bin/python link exists.
      
       * Custom ports. otcl and xerces-c2 have both been removed from the ports
         tree as of Q2 2015. ipmitool-devel is a port for the latest version of
         ipmitool. The FreeBSD port is still a rev behind here. We need the
         newer version as it appears to make our SOL consoles more stable.
      
       * Random. Fixed prerender as neato output has changed again. Tweak to
         sslxmlrpc_server to reflect change in an underlying library. Tweak to
         db/libdb.py.in to turn on autocommit which matters now as mysql 5.5 will
         hang on a metadata lock otherwise. Remade eventsys perl/python stubs
         with SWIG 2.0. SWIG 1.3 did not produce working stubs for perl 5.20.
      
      Specific un-changes:
      
       * Apache is still at 2.2. I lack the guts and skilz to upgrade to 2.4.
      
       * Xerces library is still at (now unsupported) 2.8. Assign will need
         changes before we can move to 3.x.
      
       * Python is still 2.7.
      
      Thanks to Keith Sklower for all the work he did converting ports!
      480fdc70
  19. 26 Jan, 2015 1 commit
  20. 13 Jan, 2015 1 commit
    • Mike Hibler's avatar
      Fix secure-boot nonce calculation. · f131e866
      Mike Hibler authored
      On a 64-bit FreeBSD, struct timeval is 16 bytes rather than 8 and
      that is too big to fit in our nonce. We use the low 32-bits of those now.
      f131e866
  21. 19 Nov, 2014 1 commit
    • Kirk Webb's avatar
      Sprinkle taint checks throughout tmcd to avert privilege escalation. · d9c27fac
      Kirk Webb authored
      Also add utility function to allow the node to get the exact details of
      the image it is running ('imageinfo').
      
      Some of the taint checks are rather heavy-handed presently.  Pretty much
      any vector that could be used by the user to do something as root has
      been severed right at the top of the relevant tmcd calls.
      
      Calls affected:
      
      manifest ('blackbox' and 'useronly' taintstates)
      rpms ('blackbox' and 'useronly' taintstates)
      tarballs ('blackbox' and 'useronly' taintstates)
      blobs ('blackbox' and 'useronly' taintstates)
      startupcmd ('blackbox' taintstate)
      mounts ('blackbox' taintstate)
      programs ('blackbox' taintstate)
      
      Taint handling for the 'accounts' call was dealt with in a prior commit.
      d9c27fac
  22. 06 Oct, 2014 1 commit
  23. 31 Jul, 2014 1 commit
  24. 25 Jul, 2014 1 commit
  25. 16 May, 2014 1 commit
  26. 07 May, 2014 1 commit
    • Mike Hibler's avatar
      Introducing TMCD version 38! Returns additional "loadinfo" info. · 4a8604b1
      Mike Hibler authored
      New loadinfo returns:
      
      IMAGELOW, IMAGEHIGH: range of sectors covered by the image.
          This is NOT the same as what imageinfo or imagedump will show.
          For partition images, these low and high values are adjusted
          for the MBR offset of the partition in question. So when loading
          a Linux image, expect values like 6G and 12G. The intent here
          (not yet realized) is that these values will be used to construct
          an MBR/GPT on the fly, rather than using hardcode magic MBR versions.
          You can get the uncompressed size of the image with (high - low + 1).
      
      IMAGESSIZE: the units of the low/high values.
          Always 512 right now, may be 4096 someday.
      
      IMAGERELOC: non-zero if the image can be placed at an offset other
          than IMAGELOW (i.e., it can be relocated). This may or may not
          prove useful for dynamic MBR construction...we will see.
      
      Probably didn't need to bump the version here, but I am playing it safe.
      4a8604b1
  27. 20 Feb, 2014 1 commit
  28. 12 Oct, 2013 2 commits
  29. 09 Sep, 2013 1 commit
    • Mike Hibler's avatar
      Build libtb and tmcd lib for frisbee-mfs. · bd9cc69b
      Mike Hibler authored
      Side-effect of change that added full tmcc build (aka, with SSL) to client.
      Even though we currently don't install full tmcc in frisbee MFS (due to
      other missing libraries) we at least make sure it builds.
      bd9cc69b
  30. 22 Jul, 2013 1 commit
  31. 18 Jun, 2013 1 commit
  32. 16 May, 2013 2 commits
  33. 07 May, 2013 1 commit
  34. 30 Apr, 2013 1 commit
    • Kirk Webb's avatar
      Add complete local node storage support from parser down to tcmd. · dab52801
      Kirk Webb authored
      Doing this required adding columns to the virt and physical blockstores
      tables to mark the attributes that will be considered for mapping.
      Unmarked entries just flow through to the client-side.
      
      This commit also introduces filesystem support in the form of passing
      through a mount point to the client-side.  It is left to the client to
      decide what filesystem and fs options to use to setup the space, including
      any logical volume aggregation required to support the request.
      dab52801
  35. 15 Apr, 2013 1 commit
  36. 27 Feb, 2013 1 commit
    • Kirk Webb's avatar
      Fixes to the DB queries for storage config. · 11350134
      Kirk Webb authored
      I had it looking in the wrong tables given that I wanted to find the
      set of lans a node is connected to which contain blockstores (physical
      instead of virtual).
      
      Also squash a couple of bugs.
      11350134