1. 06 May, 2014 1 commit
    • Mike Hibler's avatar
      Add "relocatable" flag to images table to indicate that an image can be moved. · 65de520b
      Mike Hibler authored
      Hopefully, my last schema change related to images. If relocatable is not
      set then an image must be loaded at the lba_low offset. If set, then the
      image can be loaded at other offsets. Currently, all FBSD images are
      relocatable courtesy of the relocation mechanism in imagezip (which can
      fix up otherwise absolute offsets in an image). Sadly, Linux images are
      not relocatable due to absolute block numbers in the grub partition
      bootblock that we require. Ryan "taught" imagezip to relocate these, but
      I need to find his changes.
      65de520b
  2. 02 May, 2014 1 commit
    • Mike Hibler's avatar
      Add low/high sector numbers to the images table. · c345f7cf
      Mike Hibler authored
      These are computed by imagedump for .ndz images. The plan is to
      pass this info on to clients via tmcc so they can know the max disk
      size required.
      
      There will shortly be a utility to automatically update these values
      when an image is created or updated. Stay tuned.
      c345f7cf
  3. 24 Jan, 2014 1 commit
  4. 28 May, 2013 1 commit
  5. 14 May, 2013 1 commit
  6. 23 Jan, 2013 1 commit
  7. 14 Jan, 2013 1 commit
  8. 14 Nov, 2012 1 commit
  9. 24 Sep, 2012 1 commit
    • Eric Eide's avatar
      Replace license symbols with {{{ }}}-enclosed license blocks. · 6df609a9
      Eric Eide authored
      This commit is intended to makes the license status of Emulab and
      ProtoGENI source files more clear.  It replaces license symbols like
      "EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
      blocks that contain actual license statements.
      
      This change was driven by the fact that today, most people acquire and
      track Emulab and ProtoGENI sources via git.
      
      Before the Emulab source code was kept in git, the Flux Research Group
      at the University of Utah would roll distributions by making tar
      files.  As part of that process, the Flux Group would replace the
      license symbols in the source files with actual license statements.
      
      When the Flux Group moved to git, people outside of the group started
      to see the source files with the "unexpanded" symbols.  This meant
      that people acquired source files without actual license statements in
      them.  All the relevant files had Utah *copyright* statements in them,
      but without the expanded *license* statements, the licensing status of
      the source files was unclear.
      
      This commit is intended to clear up that confusion.
      
      Most Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the Affero GNU General Public License, version 3
      (AGPLv3).
      
      Most Utah-copyrighted files related to ProtoGENI are distributed under
      the terms of the GENI Public License, which is a BSD-like open-source
      license.
      
      Some Utah-copyrighted files in the Emulab source tree are distributed
      under the terms of the GNU Lesser General Public License, version 2.1
      (LGPL).
      6df609a9
  10. 04 Sep, 2012 1 commit
  11. 29 Aug, 2012 1 commit
  12. 27 Aug, 2012 1 commit
  13. 07 Jun, 2012 1 commit
    • Leigh B Stoller's avatar
      New script, clone_image to simplify create/snapshot from a node. · b01c991d
      Leigh B Stoller authored
      clone_image is a wrapper around newimageid_ez and create_image, that
      simplifies the most common operation; creating a new imageid derived
      from the image/os that is currently running in the node, and then
      taking a snapshot of the node. So for example, if node pcXXX is
      running image FREEBSD, and you want to create a custom image from that
      node, all you need to do:
      
      	boss> clone_image myfreebsd pcXXX
      
      which will create the new descriptor, deriving everything from the
      FREEBSD image on the node, and then take a snapshot from pcXXX. If
      the descriptor already exists, just take the snapshot.
      
      So what if you do:
      
      	boss> clone_image FREEBSD pcXXX
      
      well, the image is always looked up in the project the node is
      currently attached to, so in fact a new descriptor is created in that
      project, and you do not actually overwrite an image from some other
      project. 
      
      I've added some locking to images to prevent concurrent snapshots.
      This seemed like a good idea since this script is going to be used
      from the ProtoGeni interface. More on this in another commit.
      b01c991d
  14. 06 Jun, 2012 1 commit
  15. 15 May, 2012 1 commit
  16. 01 May, 2012 1 commit
  17. 06 Mar, 2012 1 commit
  18. 22 Dec, 2011 1 commit
  19. 11 Oct, 2011 1 commit
  20. 10 Oct, 2011 1 commit
    • Leigh B Stoller's avatar
      Add support for sharing images between projects. New table called · 646b64f6
      Leigh B Stoller authored
      image_permissions stores access info for images. You can share an
      image with a user or a group (project), and you can specify write
      access to allow updating the image in place. Note that write access
      does not allow the descriptor to be modified, only the image itself.
      Well, that is how it will be after Mike changes mfrisbeed.
      
      The front end script to modify permissions is grantimage:
      
      	boss> grantimage -u stoller -w tbres,myimage
      	boss> grantimage -u stoller -w tbres,myimage
      
      which grants write access to stoller. Or:
      
      	boss> grantimage -g testbed,testbed tbres,myimage
      
      which grants access to the testbed project. Notice that you can
      specify subgroups this way.
      
      	boss> grantimage -l tbres,myimage
      
      will give you a list of current permissions. To revoke, just add -r
      option:
      
      	boss> grantimage -g testbed,testbed -r tbres,myimage
      
      Who is allowed to grant access to an image? 1) An adminstrator of
      course, 2) the image creator, and 3) any group_root in the group that
      the image belongs to. Being granted access to use an image does not
      confer permission to grant access to others.
      
      One last task; while the web interface displays the permissions, there
      is no web interface to modify the permissions; users will still have
      to ask us for now.
      646b64f6
  21. 18 Jan, 2011 1 commit
  22. 22 Nov, 2010 1 commit
  23. 16 Nov, 2010 1 commit
    • Kevin Atkinson's avatar
      Add support for all node "tb-set-tarfiles". · a0d0c95e
      Kevin Atkinson authored
      "tb-set-tarfiles" is like "tb-set-node-tarfiles" except that it
      distributes the tarfile to all nodes rather than just one and that it
      uses frisbee to distribute the file.
      
      These changes involved 1) refactoring frisbee info from images table
      into a new table, frisbee_blobs, 2) a new experiment_blobs table, and
      3) a new tmcd command so the node knows how to get the files from the
      server.
      
      The changes where designed to be general purpose enough to eventually
      support:
        1) Distributing arbitrary files (not just tarfiles) to nodes
        2) Perform arbitrary actions on those files
        3) Use arbitrary methods to get the files
      
      As such the tmcd line is as follows:
        URL=* ACTION=*
      
      where URL is currently:
        frisbee.mcast://<ADDR>/<FILE>
      for example
        frisbee.mcast://234.16.184.192:18092/users/kevina/home-dir.tar.gz
      and when we get around to using a master Frisbee server it could be
        frisbee://*
      or it could be a file://, http://, etc.
      
      and ACTION is currently:
        unpack:<LOCATION>
      for example
        unpackt:/users
      with future syntax to be determined.
      a0d0c95e
  24. 12 Nov, 2010 1 commit
  25. 11 Oct, 2010 1 commit
    • Leigh B Stoller's avatar
      Work on an optimization to the perl code. Maybe you have noticed, but · 92f83e48
      Leigh B Stoller authored
      starting any one of our scripts can take a second or two. That time is
      spent including and compiling 10000s of thousands of lines of perl
      code, both from our libraries and from the perl libraries.
      
      Mostly this is just a maintenance thing; we just never thought about
      it much and we have a lot more code these days.
      
      So I have done two things.
      
      1) I have used SelfLoader() on some of our biggest perl modules.
         SelfLoader delays compilation until code is used. This is not as
         good as AutoLoader() though, and so I did it with just a few 
         modules (the biggest ones).
      
      2) Mostly I reorganized things:
      
        a) Split libdb into an EmulabConstants module and all the rest of
           the code, which is slowly getting phased out.
      
        b) Move little things around to avoid including libdb or Experiment
           (the biggest files).
      
        c) Change "use foo" in many places to a "require foo" in the
           function that actually uses that module. This was really a big
           win cause we have dozens of cases where we would include a
           module, but use it in only one place and typically not all.
      
      Most things are now starting up in 1/3 the time. I am hoping this will
      help to reduce the load spiking we see on boss, and also help with the
      upcoming Geni tutorial (which kill boss last time).
      92f83e48
  26. 08 Apr, 2010 1 commit
  27. 12 Oct, 2009 1 commit
    • David Johnson's avatar
      Add the ability to load images on virtnodes. For now, we just overload · c6c57bc9
      David Johnson authored
      the tb-set-node-os command with a second optional argument; if that is
      present, the first arg is the child OS and the second is the parent OS.
      We add some new features in ptopgen (OS-parentOSname-childOSname) based
      off a new table that maps which child OSes can run on which parents, and
      the right desires get added to match.  We setup the reloads in os_setup
      along with the parents.  Also needed a new opmode, RELOAD-PCVM, to handle
      all this.
      
      For now, users only have to specify that their images can run on pcvms, a
      special hack for which type the images can run on.  This makes sense in
      general since there is no point conditionalizing childOS loading on
      hardware type at the moment, but rather on parentOS.  Hopefully this stuff
      wiill mostly work on shared nodes too, although we'll have to be more
      aggressive on the client side garbage collecting old frisbee'd images for
      long-lived shared hosts.
      
      I only made these changes in libvtop, so assi...
      c6c57bc9
  28. 10 Sep, 2008 1 commit
    • Mike Hibler's avatar
      Slight beefing up of support for alternate MBRs: · 31009d09
      Mike Hibler authored
       * when creating an image from a node, make sure the new image
         gets the MBR version used by the existing image
       * when loading a single-partition image that requires a different
         MBR, invalidate all other existing partition ("invalidate" in the
         sense that we remove any partitions table entries, we don't do anything
         to the disk)
      31009d09
  29. 06 May, 2008 1 commit
    • Kevin Atkinson's avatar
      · f383ecf2
      Kevin Atkinson authored
      Fix bug in Image->Create where the ezid field wasn't correctly being set.
      f383ecf2
  30. 01 May, 2008 1 commit
    • Kevin Atkinson's avatar
      Implemt FS#187 -- Show admin history of projects: · 8054f5f8
      Kevin Atkinson authored
        When a project is initially created a new mailing list is created,
        PROJ-admin@emulab.net.
      
        testbed-approval is subscribed to the list
      
        Several emails that originally went to testbed-approval now go to the
        mailing list instead.  The From, To, fields are basically the same
        with testbed-approval becoming PROJ-admin.  This means some mail
        is sent with a From PROJ-admin and Bcc the mailing list.  Note that
        some mail still goes to testbed-approval directly, in particular
        ones where there is no clear project involved, and when a project is
        denied.
      
        In addition notifications of approval status of new members is also
        sent to the list.  These emails use to only go to testbed-audit@.
      
        Currently All mail sent to PROJ-admin is also sent to testbed-audit
        (via a Bcc).  This means that some mail that didn't use to go to
        testbed-audit now does.
      
        The mailing list is deleted when a project is deleted with out first
        being approved.  Becuase of this notified that a project is denied
        is sent to testbed-approval instead of PROJ-admin.
      
        Admins can access the mailing list from the Project Profile page.
      
        The mailing list is open in order to allow users to reply to the
        mailing list, in addition the check that PROJ-admin is in the To or
        CC field is disabled.  There is currently no spam control on the
        mailing lists.  However, since the mailing list address is not posted
        anywhere it should't pick up to much spam.  If it does we can deal
        with it then.
      
        Finally, a new script is created to create the per-project admin
        mailing list.  See doc/UPDATING.
      
      Also add DBQuerySingleFatal to libdb, which is like DBQueryFatal but
      also fails if the query didn't return any results.  Basically
      identical to he version in libtblog.  Eventually libtblog should be
      modified to use this version.
      8054f5f8
  31. 31 Jan, 2008 1 commit
  32. 19 Oct, 2007 1 commit
    • Russ Fish's avatar
      Move newimageid_ez page form logic to a backend Perl script. · 9a586d0b
      Russ Fish authored
           www/newimageid_ez.php3 - The reworked PHP page.  Calls Image::NewImageId with ez=1.
           www/newimageid.php3 - Call Image::NewImageId with ez=0.
           www/imageid_defs.php - Re-use the Image::NewImageId class method, adding an 'ez' arg.
           backend/{newimageid_ez,GNUmakefile}.in configure configure.in - New backend script.
               After checks, calls OSinfo->Create and Image->Create with the same XML args array.
           db/Image.pm.in - Re-use the Image->Create method, adding an imageid over-ride arg.
           db/OSinfo.pm.in - Filter out extraneous db slot args from XML in the Create method.
           db/libdb.pm.in - Add TB_{OS,IMAGE}ID_* constants from dbdefs.php3 .
           sql/database-fill.sql - Add OS entries to the table_regex 'images' pattern set.
      9a586d0b
  33. 12 Oct, 2007 1 commit
    • Russ Fish's avatar
      Move newimageid page form logic to a backend Perl script. · 3f5e4ef8
      Russ Fish authored
           www/newimageid.php3 - The reworked PHP page.
           www/imageid_defs.php - Add an Image::NewImageId class method
                                  bridging to the script via XML.
           backend/{newimageid,GNUmakefile}.in configure configure.in - New backend script.
           db/Image.pm.in - Add a Create class method for script arg checking.
           sql/database-fill.sql - Add to the table_regex 'images' checking patterns.
      3f5e4ef8
  34. 03 Oct, 2007 1 commit
    • Russ Fish's avatar
      Move editimageid page form logic to a backend Perl script. · 9dcf6297
      Russ Fish authored
           www/editimageid.php3 - The reworked PHP page.
           www/imageid_defs.php - Add an Experiment::EditImageid class method
                                  bridging to the script via XML.
           backend/{editimageid,GNUmakefile}.in configure configure.in - New backend script.
           db/Image.pm.in - Add an EditImageid worker class method for script arg checking.
           sql/database-fill.sql - Add to the table_regex 'images' checking patterns.
      9dcf6297
  35. 21 Aug, 2007 1 commit
    • Leigh B. Stoller's avatar
      Another round of widearea node hacking for CMU. These changes add · 99346dc0
      Leigh B. Stoller authored
      widearea reloading support.
      
      * New slot in the images table to store an access key which remote
        sites must provide in order to download an image (via https).
      
      * tmcd returns a different kind of ADDRESS field from doloadinfo.
        Instead of the multicast stuff, return a URL that points to boss'
        web server. The URL is of the form:
      
         https://www.myemulab.net/spewimage.php?imageid=10013&access_key=abcdef
      
        which as you can see is fully specified; the client does not need
        to know anything else.
      
      * New webpage and backend scripts appropriately called "spewimage"
        which also includes support for the http HEAD request (from wget) to
        avoid downloading images that are already on the node. I just
        learned about this HEAD request stuff today ... but otherwise these
        operate as expected, spewing the image if the access key is provided.
      
      * Changes to rc.frisbee to deal with remote loading. In addition to
        URL support, I also added support for simple paths, the intent being
        that we will probably distribute images offline (say, at night) so
        that when a node reboots it doesn't actually have to wait 60 minutes
        for an image to download. I have not added any server side support
        for this yet though. Maybe later this week.
      
      * Other bits and pieces and fixes to make this work.
      99346dc0
  36. 02 Aug, 2007 1 commit
  37. 26 Apr, 2007 1 commit
  38. 13 Mar, 2007 1 commit