1. 09 Apr, 2019 1 commit
    • Mike Hibler's avatar
      Hack support for a "local" (non-boss) version of the power command. · 5b9079b9
      Mike Hibler authored
      Uses text files for getting the controller info and outlet map.
      Only supported for the APC and Raritan modules. Some of the other
      modules might work...but we don't care. Others would not work without
      more effort as they use the Emulab DB directly.
      
      Included hack makefile targets "localboss" and "localboss-install"
      to install this version and the minimum number of modules.
      5b9079b9
  2. 04 Mar, 2019 1 commit
  3. 14 Nov, 2018 1 commit
    • Leigh B Stoller's avatar
      Use sunlink flag to prevent users from removing critical directories in · 982f3f59
      Leigh B Stoller authored
      /proj. Applied to top level only for now, since that was reasonably easy
      to do, since projects and group stuff is all done on ops already (where
      the chflags has to run). We could apply this to experiment and image
      directories too, but we all know the better approach is to stop mounting
      /proj on experimental nodes, right?
      
      Also a new script mkprojdirs to create/recreate missing project
      directories and do the chflags (calls over to ops and uses the
      existing proxy script).
      982f3f59
  4. 29 Aug, 2018 1 commit
  5. 04 Jun, 2018 1 commit
    • David Johnson's avatar
      Docker server-side core, esp new libimageops support for Docker images. · 66366489
      David Johnson authored
      The docker VM server-side goo is mostly identical to Xen, with slightly
      different handling for parent images.  We also support loading external
      Docker images (i.e. those without a real imageid in our DB; in that
      case, user has to set a specific stub image, and some extra per-vnode
      metadata (a URI that points to a Docker registry/image repo/tag);
      the Docker clientside handles the rest.
      
      Emulab Docker images map to a Emulab imageid:version pretty seamlessly.
      For instance, the Emulab `emulab-ops/docker-foo-bar:1` image would map
      to `<local-registry-URI>/emulab-ops/emulab-ops/docker-foo-bar:1`; the
      mapping is `<local-registry-URI>/pid/gid/imagename:version`.  Docker
      repository names are lowercase-only, so we handle that for the user; but
      I would prefer that users use lowercase Emulab imagenames for all Docker
      images; that will help us.  That is not enforced in the code; it will
      appear in the documentation, and we'll see.
      
      Full Docker imaging relies on several other libraries
      (https://gitlab.flux.utah.edu/emulab/pydockerauth,
      https://gitlab.flux.utah.edu/emulab/docker-registry-py).  Each
      Emulab-based cluster must currently run its own private registry to
      support image loading/capture (note however that if capture is
      unnecessary, users can use the external images path instead).  The
      pydockerauth library is a JWT token server that runs out of boss's
      Apache and implements authn/authz for the per-Emulab Docker registry
      (probably running on ops, but could be anywhere) that stores images and
      arbitrates upload/download access.  For instance, nodes in an experiment
      securely pull images using their pid/eid eventkey; and the pydockerauth
      emulab authz module knows what images the node is allowed to pull
      (i.e. sched_reloads, the current image the node is running, etc).  Real
      users can also pull images via user/pass, or bogus user/pass + Emulab
      SSL cert.  GENI credential-based authn/z was way too much work, sadly.
      There are other auth/z paths (i.e. for admins, temp tokens for secure
      operations) as well.
      
      As far as Docker image distribution in the federation, we use the same
      model as for regular ndz images.  Remote images are pulled in to the
      local cluster's Docker registry on-demand from their source cluster via
      admin token auth (note that all clusters in the federation have
      read-only access to the entire registries of any other cluster in the
      federation, so they can pull images).  Emulab imageid handling is the
      same as the existing ndz case.  For instance, image versions are lazily
      imported, on-demand; local version numbers may not match the remote
      image source cluster's version numbers.  This will potentially be a
      bigger problem in the Docker universe; Docker users expect to be able to
      reference any image version at any time anywhere.  But that is of course
      handleable with some ex post facto synchronization flag day, at least
      for the Docker images.
      
      The big new thing supporting native Docker image usage is the guts of a
      refactor of the utils/image* scripts into a new library, libimageops;
      this is necessary to support Docker images, which are stored in their
      own registry using their own custom protocols, so not amenable to our
      file-based storage.  Note: the utils/image* scripts currently call out
      to libimageops *only if* the image format is docker; all other images
      continue on the old paths in utils/image*, which all still remain
      intact, or minorly-changed to support libimageops.
      
      libimageops->New is the factory-style mechanism to get a libimageops
      that works for your image format or node type.  Once you have a
      libimageops instance, you can invoke normal image logical operations
      (CreateImage, ImageValidate, ImageRelease, et al).  I didn't do every
      single operation (for instance, I haven't yet dealt with image_import
      beyond essentially generalizing DownLoadImage by image format).
      Finally, each libimageops is stateless; another design would have been
      some statefulness for more complicated operations.   You will see that
      CreateImage, for instance, is written in a helper-subclass style that
      blurs some statefulness; however, it was the best match for the existing
      body of code.  We can revisit that later if the current argument-passing
      convention isn't loved.
      
      There are a couple outstanding issues.  Part of the security model here
      is that some utils/image* scripts are setuid, so direct libimageops
      library calls are not possible from a non-setuid context for some
      operations.  This is non-trivial to resolve, and might not be worthwhile
      to resolve any time soon.  Also, some of the scripts write meaningful,
      traditional content to stdout/stderr, and this creates a tension for
      direct library calls that is not entirely resolved yet.  Not hard, just
      only partly resolved.
      
      Note that tbsetup/libimageops_ndz.pm.in is still incomplete; it needs
      imagevalidate support.  Thus, I have not even featurized this yet; I
      will get to that as I have cycles.
      66366489
  6. 11 Apr, 2018 1 commit
  7. 01 Jan, 2018 1 commit
    • Leigh B Stoller's avatar
      Changes to reservation system wrt classic interface: · dc90a087
      Leigh B Stoller authored
      1. Reservation system now groks experiment lockdown and swappable. When
         swapping in, lockdown and swappable mean the expected end of the
         experiment is never.
      
      2. Reservation library now handles changes to lockdowm, swappable, and
         autoswap (timeout). editexp now hands these changes off to a new
         script called manage_expsettings, which can be called by hand since
         we might need to force a change (I am not changing the classic UI, if
         a change is not allowed by the res system, we have to do it by hand).
      
      3. Minor fixes to reservation library.
      dc90a087
  8. 23 Aug, 2017 2 commits
  9. 15 Dec, 2016 1 commit
  10. 17 Oct, 2016 1 commit
  11. 14 Oct, 2016 1 commit
    • Leigh B Stoller's avatar
      Attempt to address the problem described in issue #166; that nodes fail · 5d7164b3
      Leigh B Stoller authored
      to go from PXEBOOTING (pxewakeup) to actually booting, but we do not
      know that for a really long time cause we send a BOOTING event from
      bootinfo right after PXEBOOTING, since that was the only place to hook
      it in. Well Mike discovered the "on commit" support in dhcpd, and so
      that is what we are going to use now. Note that uboot nodes have been
      using on commit, now all nodes will when BOOTINFO_EVENTS=0.
      
      Mike's reportboot program is now a daemon, renamed to report_daemon.
      The original reportboot program is a little script that writes the
      arguments from dhcpd to a unix socket to be picked up by the daemon,
      which does the original work of mapping the IP/Mac to a node id and
      sending an event. The code has also been modified to run on a subboss
      using the same node mapping given to to dhcpd, reconstituted as DBM
      file by subboss_dhcpd_makeconf.
      
      The reason for using a daemon this way is so that we do not hang up
      dhcpd in case we cannot get to the event system. The unix domain
      socket will give us some amount of buffering, but I suspect that any
      event problem will eat that space up quickly, and I will be back to
      revisit this (probably want reportboot to not block on its write
      to the socket).
      
      pxeboot changed to not send PXEBOOTING or BOOTING when
      BOOTINFO_EVENTS=0.
      5d7164b3
  12. 03 Oct, 2016 1 commit
  13. 07 Apr, 2016 1 commit
  14. 21 Mar, 2016 1 commit
  15. 19 Oct, 2015 1 commit
  16. 07 Jan, 2015 1 commit
  17. 05 Dec, 2014 1 commit
    • Mike Hibler's avatar
      Support dynamically created NFS-root filesystems for admin MFS. · f36bcfab
      Mike Hibler authored
      Significant hackary involved. Similar to exports_setup, there is a boss-side
      script and an ops-side script to handle creation and destruction of the ZFS
      clones that are used for the NFS filesystem. The rest was all about when to
      invoke said scripts.
      
      Creation is easy, we just do a clone whenever the TBAdminMfsSelect is called
      to "turn on" node admin mode. Destruction is not so simple. If we destroyed
      the clone on the corresponding TBAdminMfsSelect "off" call, then we could
      yank the filesystem out from under the node if it was still running in the
      MFS (e.g., "node_admin -n off node"). While that would probably be okay in
      most uses, where at worst we would have to apod or power cycle the node, we
      try to do better. TBAdminMfsSelect "off" instead just renames the clone
      (to "<nodeid>-DEAD") so that it stays available if the node is running on
      it at the time, but ensures that it will not get accidentally used by any
      future boot. We check for, and destroy, any previous versions for a node
      every time we invoke the nfsmfs_setup code for that node. We also destroy
      live or dead clones whenever we call nfree. This ensures that all MFSes
      get cleaned up at experiment swapout time.
      f36bcfab
  18. 23 Nov, 2014 1 commit
  19. 06 Jan, 2014 1 commit
    • Mike Hibler's avatar
      First reasonably functional lease_daemon. · 595a118c
      Mike Hibler authored
      Walks leases through their various states, enforces expiration and idle
      times and grace periods, and expires/locks leases.
      
      As always, probably sends too much email.
      595a118c
  20. 11 Dec, 2013 1 commit
  21. 04 Nov, 2013 2 commits
    • Kirk Webb's avatar
      7cb7effc
    • Kirk Webb's avatar
      Commit portfix version of snmpit and the Port module. · 264500e2
      Kirk Webb authored
      This commit pushes the changes located in the temporary snmpit_portfix
      subdirectory into the mainline snmpit_test directory, and updates
      db/Ports.pm with the 'portfix' changes.  These changes ensure better
      consistency with port lookup/resolution.  In particular, interswitch
      trunk lookup will return the switch-local side of the link.
      
      This commit also adds an installer script to cleanup the temporary files
      introduced for burning in the snmpit_portfix changes.
      264500e2
  22. 17 Sep, 2013 2 commits
  23. 14 May, 2013 1 commit
    • Leigh B Stoller's avatar
      Add new script to do arp lockdown on boss. · f5cc889a
      Leigh B Stoller authored
      The other version is only for the client side (subboss,ops), but does
      not work on real boss. Also hooked into tbswap so that the arps are
      updated during swapin/swapout. Also change tmcd to return arp
      directives for all containers, not just on shared nodes.
      f5cc889a
  24. 14 Nov, 2012 1 commit
    • Leigh B Stoller's avatar
      Move rpm/tar download from boss to ops, to avoid wasted network traffic. · f37cd9dc
      Leigh B Stoller authored
      To turn this option on, define SPEWFROMOPS=1 in your defs file. This
      will result in a redirect message from boss which will send the wget
      client over to ops. 
      
      A perl setuid root cgi script is run from the webserver on ops when a
      /spewrpmtar request is made. This script sends the key,nodeid,file
      over to boss via XMLRPC (as elabman). The return is simple yes or no,
      the caller is allowed (not allowed) to have that file. Since the
      ops script runs as root, it can spew the file back to the caller.
      
      Note that the elabinelab checks for the elabinelab source code are
      gone; we are now open source. Also, we spew that file from /share now,
      to be consistent.
      f37cd9dc
  25. 26 Sep, 2012 1 commit
  26. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  27. 04 Sep, 2012 1 commit
    • Leigh B Stoller's avatar
      Add image import utilities. · e468f885
      Leigh B Stoller authored
      image_setup is run from tbprerun to verify and create image
      descriptors, and then later from tbswap to actually download
      and verify the image (ndz) file.
      
      image_import does the actual work for a specific image (url).
      e468f885
  28. 29 Aug, 2012 1 commit
  29. 18 Jun, 2012 1 commit
  30. 04 Jun, 2012 3 commits
  31. 30 Apr, 2012 1 commit
  32. 07 Nov, 2011 1 commit
  33. 20 Sep, 2011 1 commit
  34. 11 Aug, 2011 1 commit
    • Mike Hibler's avatar
      Initial support for loading Windows7 .wim images via WinPE/ImageX. · ac711ea5
      Mike Hibler authored
      1. Support for "one-shot" PXE booting ala the one-shot osid. Switches to
         pxelinux to boot WinPE and then switch back after done. Painful now
         because we have to HUP dhcpd everytime we change the PXE path, but we
         may be able to fix this in the future by going all-pxelinux-all-the-time.
      
      2. Added pxe_select, analogous to os_select, for changing the pxe_boot_path
         including the one time path.
      
      3. Added the WIMRELOAD state machine to shepherd a node through the process.
         Still has some rough edges and may need refining.
      ac711ea5
  35. 28 Jul, 2011 1 commit
    • Leigh B Stoller's avatar
      Power "saving" additions from Barry Trent, who got them from Kevin · 03478fb9
      Leigh B Stoller authored
      Lahey.
      
      Power saving turns off nodes that have been sitting in PXEWAIT (and
      are thus free) for more then a set amount of time (see sitevar
      general/idlepower_idletime, which defaults to 3600 seconds).
      
      The driver script is tbsetup/idlepower.in and needs to be added to
      /etc/crontab at sites that want to do this. Even so, operation is
      enabled by the sitevar general/idlepower_enable. Each time it runs, it
      checks for nodes that need to be turned off, and then calls power.
      Note: This should be a daemon not a cron job.
      
      To be considered for power saving, you must add an attribute to the
      node_type_attributes table called 'idlepower_enable', set to 1.
      
      Locally, I hacked up stated and power to make the state transitions
      legal so that stated does not whine. I added POWEROFF as a valid
      transition from any state, to opmodes NORMAL, NORMALv1, and NORMALv2.
      Barry's original patch already had a state transition for PXEKERNEL.
      In power, I added code to look at the actual operation, and in the
      case of "on", do not send an event if the node is not in POWEROFF,
      since a user can foolishly say power on anytime, and the node is on
      nothing is every going to change, and the state transition would be
      wrong.
      
      node_reboot takes of powering nodes on, when they are in POWEROFF.
      
      Barry on copyright issues:
       "I'm not sure those rights are mine to grant! Remember that this code
       came originally from Kevin Lahey (kml@patheticgeek.net) and
       originated at DETER (although he's apparently not there anymore). I
       don't foresee a problem from our point of view (but I'll double
       check, of course). Shall I try to contact Kevin try to sort this mess
       out, or do you think it's better to coordinate from your end?"
      03478fb9