1. 07 Nov, 2018 1 commit
    • Leigh Stoller's avatar
      Quick fix for watchdog/backup interaction; use a script lock. · 72b4ba32
      Leigh Stoller authored
      From Slack:
      
      What I notice is that mysqldump is read locking all of the tables for a
      long time. This time gets longer and longer of course as the DB gets
      bigger. Last night enough stuff backed up (trying to get various write
      locks) that we hit the 500 thread limit. I only know this cause mysql
      prints "killing 501" threads at 2:03am. Which makes me wonder if our
      thread limit is too small (but seems like it would have to be much
      bigger) or if our backup strategy is inappropriate for how big the DB is
      and how busy the system is. But to be clear, I am not even sure if
      mysqld throws in the towel when it hits 500 threads, I am in the midst
      of reading obtuse mysql documentation. (edited) There a bunch of other
      error messages that I do not understand yet.
      
      I can reproduce this in my elabinelab with a 10 line perl script. Two
      problems; one is that we do not use the permission system, so we cannot
      use dynamic permissions, which means that the single thread that is left
      for just this case, can be used by anyone, and so the server is fully
      out of threads. And 2) then the Emulab mysql watchdog cannot perform its
      query, and so it thinks mysqld has gone catatonic and kills it, right in
      the middle of the backup. Yuck * 2. (edited)
      
      And if anyone is curious about a more typical approach: "If you want to
      do this for MyISAM or mixed tables without any downtime from locking the
      tables, you can set up a slave database, and take your snapshots from
      there. Setting up the slave database, unfortunately, causes some
      downtime to export the live database, but once it's running, you should
      be able to lock it's tables, and export using the methods others have
      described. When this is happening, it will lag behind the master, but
      won't stop the master from updating it's tables, and will catch up as
      soon as the backup is complete"
      72b4ba32
  2. 05 May, 2017 1 commit
  3. 10 Oct, 2016 1 commit
  4. 10 Jul, 2015 1 commit
  5. 26 May, 2015 1 commit
  6. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  7. 08 Mar, 2010 1 commit
  8. 10 Nov, 2009 1 commit
  9. 11 Apr, 2008 1 commit
  10. 28 Apr, 2006 1 commit
  11. 31 May, 2005 1 commit
  12. 04 Nov, 2003 1 commit
  13. 18 Sep, 2003 1 commit
  14. 06 Aug, 2003 1 commit
    • Leigh Stoller's avatar
      Clean up temporary files used in modify. The temp dirs were being · 05bd80ff
      Leigh Stoller authored
      created in /tmp and left behind. I've moved them to the expwork
      directory instead, and added a routine in the library to clear them
      out.
      
      Clear out the nsfile (stored in /tmp) used in modify. The web page was
      creating a temp file, but never removing it. swapexp now copies the
      nsfile in so that the web page can remove the temporary after the
      script exits. The temp is placed in the expwork directory as well, but
      left behind for debugging.
      
      When swapmod fails, send along the nsfile in the email message.
      05bd80ff
  15. 05 Jun, 2003 1 commit
    • Leigh Stoller's avatar
      Add roll of the slowqueries log. · 9d416c02
      Leigh Stoller authored
      Add commented out find -delete command to remove all mysql logs older
      than 35 days.  I just run it by hand at the moment, but I'll turn it
      on at some point.
      9d416c02
  16. 25 Oct, 2002 1 commit
  17. 04 Jul, 2002 1 commit
  18. 12 Feb, 2002 1 commit
  19. 11 Feb, 2002 1 commit
    • Leigh Stoller's avatar
      Changes to how the backup files are named and saved. The existing · b31fdcdc
      Leigh Stoller authored
      naming scheme was not very intuitive and the names sucked. Also, I
      want to increase the frequency with which we run the backups, and I
      want to implement an automated roll so that we only keep about a
      months worth of history around.
      
      Anyway, the new approach is to open up the index file and see what the
      name of the current update file is. It has a numeric extension. Rename
      the base log to base.XXX, and then snapshot the DB into backup.XXX So,
      the combination of update.XXX and base.XXX is the DB history since the
      last time the script was run. The file backup.XXX corresponds to the
      DB at this point in time.
      
      To restore (or track the changes of) a DB, simply take backup.XXX and
      apply the changes that are stored in update.XXX+1 (which are the changes
      made since backup.XXX was made). This should give you a DB that is the
      same as backup.XXX+1. You can go back further, and just apply all the
      subsequent update.XXX files.
      b31fdcdc
  20. 06 Dec, 2001 2 commits
  21. 21 Feb, 2001 1 commit
  22. 16 Jan, 2001 1 commit
  23. 08 Jan, 2001 1 commit