1. 24 Apr, 2018 1 commit
  2. 26 Mar, 2018 1 commit
  3. 16 Feb, 2018 1 commit
    • Leigh B Stoller's avatar
      A lot of work on the RPC code, among other things. · 56f6d601
      Leigh B Stoller authored
      I spent a fair amount of improving error handling along the RPC path,
      as well making the code more consistent across the various files. Also
      be more consistent in how the web interface invokes the backend and gets
      errors back, specifically for errors that are generated when taking to a
      remote cluster.
      Add checks before every RPC to make sure the cluster is not disabled in
      the database. Also check that we can actually reach the cluster, and
      that the cluster is not offline (NoLogins()) before we try to do
      anything. I might have to relax this a bit, but in general it takes a
      couple of seconds to check, which is a small fraction of what most RPCs
      take. Return precise errors for clusters that are not available, to the
      web interface and show them to user.
      Use webtasks more consistently between the web interface and backend
      scripts. Watch specifically for scripts that exit abnormally (exit
      before setting the exitcode in the webtask) which always means an
      internal failure, do not show those to users.
      Show just those RPC errors that would make sense users, stop spewing
      script output to the user, send it just to tbops via the email that is
      already generated when a backend script fails fatally.
      But do not spew email for clusters that are not reachable or are
      offline. Ditto for several other cases that were generating mail to
      tbops instead of just showing the user a meaningful error message.
      Stop using ParRun for single site experiments; 99% of experiments.
      For create_instance, a new "async" mode that tells CreateSliver() to
      return before the first mapper run, which is typically very quickly.
      Then watch for errors or for the manifest with Resolve or for the slice
      to disappear. I expect this to be bounded and so we do not need to worry
      so much about timing this wait out (which is a problem on very big
      topologies). When we see the manifest, the RedeemTicket() part of the
      CreateSliver is done and now we are into the StartSliver() phase.
      For the StartSliver phase, watch for errors and show them to users,
      previously we mostly lost those errors and just sent the experiment into
      the failed state. I am still working on this.
  4. 25 Jan, 2018 1 commit
  5. 03 Nov, 2016 1 commit
  6. 29 Aug, 2016 1 commit
    • Leigh B Stoller's avatar
      Add support for secure image download using a credential to grant · 10b1dbd2
      Leigh B Stoller authored
      permission to the image being requested. Added for image backed
      datasets. The key to this commit is being able to abort the logging and
      return STDOUT/STDERR to apache so that the image can be streamed.
      There is a corresponding change on the caller side, that adds a new
      method CallMethodStream() that takes a file descriptor to send the
      data to.
  7. 20 Jul, 2016 1 commit
    • Leigh B Stoller's avatar
      This change is to support switching urn in data structure from strings · 9d2cc009
      Leigh B Stoller authored
      to objects (see GeniHRN, look for new()). Much easier, less typing.  But
      in order to do that, we have make sure that if we send one on the wire,
      it gets converted properly (converted to it its plain string).
      But, Frontier and XML::RPC do not let you hook in so that if you have a
      blessed reference, it will call its stringify method for encoding.  Odd,
      cause encode_json (JSON) supports that.
      As it turns out, Frontier is structured so that it is easy to hook into
      it, and its only mildly sleazy. And Frontier has not changed in years,
      so probably not going to change much in the next few years. XML::RPC was
      too messy, so I switched Genixmlrpc CallMethod() to use Frontier
      instead.  Does not seem to risky, we use Frontier on the receiving end
      Lets see how this goes, its been running in my devel tree for a while.
  8. 22 Feb, 2016 1 commit
  9. 23 Sep, 2015 1 commit
  10. 10 Mar, 2015 1 commit
  11. 28 Oct, 2014 1 commit
  12. 14 May, 2014 1 commit
  13. 03 Dec, 2013 1 commit
  14. 19 Jun, 2013 1 commit
  15. 22 May, 2013 1 commit
  16. 27 Dec, 2012 2 commits
  17. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      This commit is intended to clear up that confusion.
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
  18. 09 May, 2012 2 commits
    • Leigh B Stoller's avatar
    • Leigh B Stoller's avatar
      Deal with a problem that surfaced on FreeBSD 9.0 and associated ports. · 2e1ba24a
      Leigh B Stoller authored
      The problem is with the chunking that apache does to avoid having
      to read the entire response, so it can set the Content-Length header
      properly. I have no idea who apache does chunking, but on the 9.0
      client it was getting confused and trash characters were leaking into
      the data. The trash is probably the result of the client getting out
      of sync. 
      Anyway, I do not know who is at fault (client code or our ancient
      apache server), but after many hours of screwing around I found out
      how to prevent the client from getting chunked data from the server.
      Basically, you have to force the request to be sent as HTTP/1.0
      instead of 1.1, which tells apache to not chunk the data. In general
      this is not a good thing to do, but in this case the RPCs are pretty
      small and not on a critical performance path. We can revisit when we
      upgrade our boss to apache 2.X ...
  19. 21 Mar, 2011 1 commit
  20. 06 Jan, 2011 1 commit
  21. 07 Dec, 2010 1 commit
  22. 11 Oct, 2010 1 commit
    • Leigh B Stoller's avatar
      Work on an optimization to the perl code. Maybe you have noticed, but · 92f83e48
      Leigh B Stoller authored
      starting any one of our scripts can take a second or two. That time is
      spent including and compiling 10000s of thousands of lines of perl
      code, both from our libraries and from the perl libraries.
      Mostly this is just a maintenance thing; we just never thought about
      it much and we have a lot more code these days.
      So I have done two things.
      1) I have used SelfLoader() on some of our biggest perl modules.
         SelfLoader delays compilation until code is used. This is not as
         good as AutoLoader() though, and so I did it with just a few 
         modules (the biggest ones).
      2) Mostly I reorganized things:
        a) Split libdb into an EmulabConstants module and all the rest of
           the code, which is slowly getting phased out.
        b) Move little things around to avoid including libdb or Experiment
           (the biggest files).
        c) Change "use foo" in many places to a "require foo" in the
           function that actually uses that module. This was really a big
           win cause we have dozens of cases where we would include a
           module, but use it in only one place and typically not all.
      Most things are now starting up in 1/3 the time. I am hoping this will
      help to reduce the load spiking we see on boss, and also help with the
      upcoming Geni tutorial (which kill boss last time).
  23. 16 Mar, 2010 1 commit
  24. 02 Dec, 2009 1 commit
  25. 24 Sep, 2009 1 commit
  26. 04 Mar, 2009 1 commit
    • Leigh B. Stoller's avatar
      Change EMULAB-COPYRIGHT to GENIPUBLIC-COPYRIGHT, for future expansions · bb878eff
      Leigh B. Stoller authored
      to the Geni Public License at http://www.geni.net/docs/GENIPubLic.pdf,
      whose expansion at this time is:
      Permission is hereby granted, free of charge, to any person obtaining
      a copy of this software and/or hardware specification (the "Work") to
      deal in the Work without restriction, including without limitation the
      rights to use, copy, modify, merge, publish, distribute, sublicense,
      and/or sell copies of the Work, and to permit persons to whom the Work
      is furnished to do so, subject to the following conditions:
      The above copyright notice and this permission notice shall be
      included in all copies or substantial portions of the Work.
      IN THE WORK.
  27. 28 Jan, 2009 1 commit
  28. 27 Oct, 2008 1 commit
  29. 16 Oct, 2008 1 commit
  30. 08 Sep, 2008 1 commit
  31. 14 Aug, 2008 1 commit
  32. 12 Aug, 2008 1 commit
  33. 31 Jul, 2008 1 commit
  34. 03 Jun, 2008 1 commit
  35. 30 May, 2008 1 commit
  36. 23 May, 2008 1 commit
  37. 20 May, 2008 1 commit
  38. 13 May, 2008 1 commit