1. 24 May, 2018 1 commit
  2. 14 Dec, 2017 1 commit
  3. 11 Dec, 2017 1 commit
  4. 07 Dec, 2017 2 commits
  5. 05 Dec, 2017 1 commit
  6. 04 Dec, 2017 2 commits
  7. 01 Dec, 2017 1 commit
  8. 30 Nov, 2017 3 commits
  9. 28 Nov, 2017 1 commit
  10. 27 Nov, 2017 1 commit
  11. 11 Sep, 2017 1 commit
  12. 08 Sep, 2017 1 commit
  13. 01 Sep, 2017 1 commit
    • David Johnson's avatar
      Fix bug from the split ctl/nm days; also parse routable_pools "safely". · c8f2eb34
      David Johnson authored
      This was a holdover from the original profile with split ctl and nm
      management nodes.  Now there is just a single ctl node by default, but
      the IPs are still requested tied to the nm node.  So we were ignoring
      the split and the custom naming options.
      Then, the hunk o' perl that was pulling the addrs out of the manifest
      was first just assuming the first cluster manifest had the proper addrs,
      which is wrong; we have to look at all cluster manifests.  Then, we have
      to ensure we are extracting public addrs for the NM node (which in the
      non-split world is the ctl node).
      So now the hunk o' perl is really large and is "safer".
  14. 13 Jan, 2017 1 commit
  15. 13 Nov, 2016 1 commit
  16. 13 Oct, 2016 1 commit
  17. 11 Oct, 2016 4 commits
    • David Johnson's avatar
      Improvements in ovs/linuxbridge iface config and hosts file setup. · 938d5fe0
      David Johnson authored
      Also, I had to change the linuxbridge configuration around.  I'd been
      using statically-configured linux bridges, and I'd assumed I could tell
      Neutron to use them.  Unfortunately, this is sort of true, but not
      enough; from the comments:
       NB: We can only control the name of the external br-ex bridge,
       because only the Neutron linuxbridge driver accepts both a map of
       physical networks to physical interfaces; and physical network to
       bridge names.  Nova assumes that the bridge it must plug a device
       into is named according to the physical network uuid.  Thus, for
       the linuxbridge case, we only setup bridge_mappings for
       br-ex... modulo a flag.  Hopefully in the future they will see the
       sense in allowing static bridge configurations.
      So for now, only br-ex is static; the others are dynamic.  What a pain
      for debugging!  Stupid.
      More importantly, this commit also takes integrates correctly with the
      new Emulab clientside improvements that let the user customize interface
      config and /etc/hosts file generation.  To handle hosts, we create a
      static manifest file that tells Emulab to call an rc.hostnames
      pre-hook.  This hook basically grabs our latest special openstack hosts
      entries, and ensures they make it into /etc/hosts.head, so that
      genhostsfile prepends our special names.  To handle interfaces, we
      further customize /etc/network/interfaces so that Emulab's rc.ifconfig
      only tries to configure interfaces we haven't handled.
      Thus, if the clientside of the disk image the scripts are operating on
      includes the new clientside hookable support, we no longer move
      rc.ifconfig and rc.hostnames out of our way --- we let them run, secure
      in the knowledge that our customizations won't get trampled.
      (All this improvement was necessary so that blockstores and event system
      stuff would work.)
    • David Johnson's avatar
    • David Johnson's avatar
    • David Johnson's avatar
  18. 09 Sep, 2016 1 commit
    • David Johnson's avatar
      Refactor VM image setup; support extra images; add user import script. · 3b3a8d50
      David Johnson authored
      This cleanly refactors everything we do to VM images (asserting the
      random passwd, disabling root passwd, changing sshd config, etc).  This
      allows us to support adding extra images based on URL/name user provides
      us in params, and to allow them to call our script after profile
      instantiation to add an image.  It's fairly comprehensive and it
      certainly works for the common cloud images from various linux vendors.
      It also rolls multi-nic support into each image.  We do this via
      boot-time udev scripts and dhcp hooks that ensure we don't add routes
      for interfaces other than eth0 (this ensures that the default gateway is
      always attached to eth0).  The old, hacky, sometimes-broken multi-nic
      support is gone, as is the special image.  I have no idea why cloud
      images don't just include this feature by default... it's not hard at
      We support Ubuntu and Fedora/Centos.  We support basically the image
      formats that qemu-nbd supports (i.e., qcow, vmware, vdi, raw), and gz or
      xz compression.  That seems to cover the core spectrum.
      On aarch64, we yank the kernel and initrd out of the image's /boot and
      create an AMI/AKI/ARI image tuple, instead of uploading the raw disk
      image.  I have never figured out how to boot a raw Ubuntu cloud image on
      KVM/aarch64, and the HP guys never got back to me.  So this is the only
      way I know (well, there's UEFI, and there's a UEFI aarch64 BIOS, so the
      UEFI cloud images might work... but life is way too short for all that
  19. 19 Aug, 2016 1 commit
    • David Johnson's avatar
      Add Mitaka; unified controller/networkmanager; Manila; linuxbridge. · 6d23a989
      David Johnson authored
      The feature notes:
        * Mitaka is now the default OpenStack release configured by this
          profile.  Kilo and Juno are deprecated, and we are no longer testing
          the profile's functionality under those versions (although we have
          no concrete plans to remove the code at this point).  They may
          continue to work, or they may not.  You should update to Mitaka if
          possible, of course.
        * The default topology is now down to two nodes: a controller (`ctl`)
          node and a compute (`cp-1`) node; the networkmanager node's
          functionality has been moved to the controller, as is the default in
          the OpenStack Ubuntu/Apt documentation.  You can return to the old
          three-node configuration by changing the name of the
          "networkmanager" node in the Advanced Parameters from `ctl` to `nm`.
        * One of the bigger Mitaka features is shared filesystem support
          (Manila).  We download a shared filesystem image and configure
          Manila so that you can immediately create a share and connect it to
        * We have added support for the Neutron ML2 "Linuxbridge" driver,
          although we continue to install the "OpenVSwitch" ML2 driver by
          default.  The Linuxbridge driver is not as well-tested as the
          OpenVSwitch driver, in all possible configurations of this profile.
          Although OpenStack has switched to the linuxbridge driver as its
          default, we have no plans to do that yet.
        * You can now choose an Apt mirror and set a custom mirror path if you
          require fast localized access to a mirror.
        * The MTU that dnsmasq pushes to your OpenStack VMs has been reduced
          from 1454 bytes to 1450 bytes.  1454 is an adequate setting for GRE
          tunnels, of course, but not for VXLAN networks, which require 1450
          on a normal physical network with 1500-byte MTU.  Somehow this
          mistake escaped prior testing.
      A few details:
        * I refactored the Neutron ML2 plugin setup code, since all nodes
          have to be configured essentially the same way.  Moreover, it
          supports either openvswitch or linuxbridge.
        * I haven't setup Manila for aarch64 because there is no available
          Manila service image for aarch64.  Have to build one of my own.
  20. 14 Jul, 2016 2 commits
    • David Johnson's avatar
      Use a different mechanism to tell dpkg automatic conffile settings. · 4ace7c90
      David Johnson authored
      Now I see why I hadn't enabled the
        -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"
      directly on the apt-get command lines.  apt-get must have a bug, because
      when you specify this option in noninteractive (and non-pty, I assume,
      because this is via startup command-then-ssh), at least one of the dpkg
      commands invoked by apt-get has no dpkg action.
      So, put these two options into /etc/dpkg/dpkg.cfg/cloudlab, and then
      there are no problems.
      Of course, this means this same behavior will happen to the users if
      they try apt-get or dpkg later on.  This is on the one hand, preferable,
      because then they can't possibly screw up openstack config files through
      package upgrades.  On the other hand, they might get fooled they're
      upgrading some other package.
      Probably will just document this and call it good :).
    • David Johnson's avatar
  21. 13 Jul, 2016 1 commit
    • David Johnson's avatar
      First step in Mitaka support: support ctl-only openstacks (2 nodes). · 8167c83f
      David Johnson authored
      The docs have been recommending a unified networkmanager-ctl physical
      node now since Liberty, instead of the old 3-node approach.  Obviously
      this has appeal for us, so get this done before doing any
      Mitaka-specific stuff.
      It really wasn't as hard as I thought... basically just worked with
      these few changes.  We no longer assign an 'nm' name at all.
      (Also, this commit has my dev tarball in osp.py instead of the real
      thing, because there is no real thing yet.)
  22. 26 May, 2016 1 commit
  23. 04 May, 2016 1 commit
  24. 19 Apr, 2016 1 commit
    • David Johnson's avatar
      Add a very simple extension mechanism. · 0e98185b
      David Johnson authored
      User has to include a tarball containing a single dir which in turn
      contains a script called setup.sh .  This tarball must be installed into
      /tmp/setup/ext .  Minimal, but who cares, it works for now.
  25. 27 Feb, 2016 2 commits
  26. 22 Feb, 2016 1 commit
  27. 17 Feb, 2016 2 commits
  28. 16 Feb, 2016 1 commit
    • David Johnson's avatar
      Add Liberty support and configurable Keystone API version support. · 0dd7c424
      David Johnson authored
      Add Liberty support.
      Add keystone v3 support.  Now you can choose which version of keystone
      to run... all combinations tested exception Juno with v3.
      Make node type and link speed configurable.
      Make token and session timeouts much longer by default (so people don't
      get logged out so quickly), but also configurable.
      Keystone is now served by WSGI through Apache on Kilo and Liberty.
      Memcached keystone token caching is disabled for now; it causes
      intermittent problems; so using SQL for now.
      Add localhost to /etc/hosts file.  This doesn't cause problems anymore,
      if it ever did.
      We now use the `openstack' CLI command for >= Kilo, instead of the
      per-service client CLI tools.
      Stick with ovs agent even in Liberty -- even though the default is now
      linuxbridge, it seems.
      In general, get rid of nearly all the rest of the cat <<EOF ... EOF
      stuff and replace it with crudini --set/--del.  A touch slower, but much
      Also in general, improve the Kilo support so that it more closely
      matches the docs.
  29. 01 Feb, 2016 1 commit
    • David Johnson's avatar
      Use a less lame swapper == geniuser check. · 8a8fa109
      David Johnson authored
      If you instantiate a portal expt on Emulab (where you might have a real
      account), the swapper is you, not geniuser.  So, check geniuser via
      geni-get slice_urn success/failure.
  30. 23 Dec, 2015 1 commit
    • David Johnson's avatar
      Support dynamic addition/deletion of compute nodes. · 1dc78db1
      David Johnson authored
      Also, adds a geni-lib script that generates an rspec instead of printing
      it (although print still works at portal) and generates input for
      CM::AddNodes() when requested.  This generator is stateful; it tries
      to avoid generating new nodes with previously-used IPs or client_ids;
      thus it is a separate object.  It is designed so that it can be imported
      into a script, and the importing script can look for special
      DYNSLICE_GENERATOR variables to use its rspec foo to create a slice and
      add nodes in some semantic way.