All new accounts created on Gitlab now require administrator approval. If you invite any collaborators, please let Flux staff know so they can approve the accounts.

  1. 24 Jun, 2016 1 commit
  2. 23 Jun, 2016 1 commit
  3. 16 Jun, 2016 1 commit
  4. 13 Jun, 2016 4 commits
  5. 14 May, 2016 1 commit
    • David Johnson's avatar
      Simplify the hadoop image creation and track modified conf files. · 06e1878b
      David Johnson authored
      Lots of little changes here (but the experiment configuration for the
      paper experiments is preserved)... now all the hadoop setup scripts and
      config files that are baked into the hadoop VM image are stored here in
      the source tree, and then are also placed in the overall capnet ext
      tarball that references.  Thus, no need for all the extra
      and conf tarballs.  Now we only download hadoop and a wordfile (for
      reproducibility of input) from when we create the hadoop
      The hadoop config files included here are the ones that we need and are
      working.  During image creation, they get baked into a tarball in the
      image, and then extracted at VM runtime once the hadoop install scripts
      have unpacked the hadoop tarball.  We wait til runtime to unpack hadoop
      because it's huge.  But the conf dir we use is in the unpacked dir,
      hence the need to wait to unpack our overlay conf tarball.
      The hadoop config files here are slightly different than Unni's (but of
      course they are the ones we used for the paper); there
      are changes so that the slaves can contact the tracker on the master (I
      think that's what it was); and more imporantly JVM and hadoop memory
      limit adjustments to make the wordcount case work for our experiments.
      I don't know how well they'll work for others... I might have
      inadvertently required that VMs have 4096MB of memory minimum :(.  But
      that is ok for us.
  6. 11 May, 2016 1 commit
  7. 10 May, 2016 10 commits
  8. 09 May, 2016 10 commits
  9. 08 May, 2016 6 commits
    • David Johnson's avatar
      Whoops, no host key checking. · 8974bfbc
      David Johnson authored
    • David Johnson's avatar
      Again? · ff885e37
      David Johnson authored
    • David Johnson's avatar
      Bug. · f4259588
      David Johnson authored
    • David Johnson's avatar
      Bug. · 3a3f729f
      David Johnson authored
    • David Johnson's avatar
      A simple port of Unni's Hadoop expect scripts to sh. · 25f12d8c
      David Johnson authored
      Also some Hadoop utility scripts, and wrappers to run jobs we care about
      for testing or whatever.
      These don't have the password login Unni had done in expect.  Right now
      we don't need that.  If we need that, we can add it as the "frontend"
      script and then use these for the rest.
      The other part of these scripts is the part that bakes the hadoop image,
      in ../ .
    • David Johnson's avatar
      Hadoop workflow agent tandem. · 51100be1
      David Johnson authored
      This is a bit odd, because the hadoop service wfa does not know when
      it's done receiving node caps from the user/tenant wfa, and it has no
      way to signal the user when it's done setting up.  Oh well.
  10. 07 May, 2016 2 commits
  11. 06 May, 2016 1 commit
  12. 05 May, 2016 2 commits
    • David Johnson's avatar
      Add a Capnet dhcp Dnsmasq wrapper to stop DNS recursive resolution. · fc2350ec
      David Johnson authored
      Capnet networks cannot get to the external world.  However, the default
      Cloudlab/OpenStack dnsmasq arrangment (of course) specifies an external
      resolver.  This slows all kinds of queries from the VMs, and slows bootup,
      while the local resolver waits for the remote one to timeout.
      Dnsmasq in openstack doesn't give up per-network config ability, so we
      add some of our own.  There is now a custom capnet dnsmasq config file
      sans external resolver; and the wrapper class strips out any --server
      CLI options that the base class might have added due to the dhcp/dnsmasq
      config file opts.  It warns when it does this.
      We may not want that behavior in the future; hopefully we remember to
      get rid of it then.  But there's no other way to allow recursive public
      resolution for non-capnet networks, and then disallow it for Capnet
      networks, without this.
    • David Johnson's avatar