1. 19 Dec, 2005 2 commits
    • Kevin Atkinson's avatar
      · 45f997fd
      Kevin Atkinson authored
      Updates to to Error Logging API Code.
      
      You should start seeing much better error messages coming from my
      system.  Errors coming from parse.proxy and assign (the two most
      frequent sources of errors) should now be concise and to the point.
      Errors coming from libosload/libreboot (the next most frequent source
      of errors) should now also be much better, but not perfect.  Getting
      perfect errors will likely a rework of how errors are handled in
      libosload/libreboot, just adding tberror/tbwarn/tbnotice calls is not
      enough.  I can do this at a latter date if necessary.
      
      A few minor database changes.
      
      Some changes to the API.  A few bug fixes. Lots of tberror/tbwarn/tbnotice
      added to scripts.
      
      Since assign is a C program, and at this time my API is perl only, I wrote a
      second wrapper around assign, assign_wrapper2.  When assign fails errors are
      now parsed in assign_wrapper2, sent to stderr and logged.  This means that
      RunAssign() just returns when assign fails rather than echoing some of
      assign.log output and then quiting.  The output to the activity log remains
      unchanged.
      
      Since "parse.proxy" is run from ops I couldn't use my API in it, even though
      it is a perl program.  Instead I parse the errors coming form it in
      parse-ns.
      45f997fd
    • Leigh B. Stoller's avatar
      Add support for moving deleted users to a deleted users table. This · b4231fbf
      Leigh B. Stoller authored
      would be no big deal, except that we want to retain user_stats for
      deleted users, and rather then a deleted_user_stats table, I want to
      retain stats for deleted users in the user_stats table, since that
      is a more natural place for them.
      
      The main problem is that we use the login (uid) as the cross table
      reference slot all over the DB, which is fundamentally incorrect, if
      we want to be able reuse uids and still know what historical data
      refers to.
      
      So, I have taken a few baby steps towards weaning us off the uid, and
      towards permanently unique key for users, using the unix_uid integer
      for now, but probably something slightly different later.
      
      The user_stats is now indexed on this new key (called uid_idx in the
      users_stats table) instead of the plain uid.
      
      The unix_uid slot in the users table is no longer an auto_increment
      field, but instead uses the emulab_indicies table for the next
      available index.
      b4231fbf
  2. 15 Dec, 2005 1 commit
  3. 13 Dec, 2005 2 commits
  4. 07 Dec, 2005 1 commit
  5. 06 Dec, 2005 1 commit
  6. 28 Nov, 2005 1 commit
    • Kirk Webb's avatar
      · 2483e886
      Kirk Webb authored
      Added plab_mapping table to the DB to help track plab node identity.
      2483e886
  7. 08 Nov, 2005 1 commit
  8. 04 Nov, 2005 1 commit
  9. 20 Oct, 2005 1 commit
    • Kirk Webb's avatar
      · 5326988f
      Kirk Webb authored
      New node_attributes facility and table.
      
      Auxiliary node attributes, such as service tag #, BIOS version, etc., are
      should now be placed into the node_attributes table.  This can be accomplished
      by either using the node_attributes command line tool, or by using the
      modnodeattributes_form.php3 form (not linked in anywhere yet, but will be
      in a moment).  Attribute names and values are checked for sanity using
      table_regex entries.  Also note that I started with the nodecontrol stuff
      as a template.
      
      The command line tool and web form (which simply calls the command line tool
      to actually do the modifications) can add, delete, and/or remove attributes.
      
      Finally, note that the bios_version column has been moved from the nodes
      table to the node_attributes table.  The Node Information page will show
      the list of current attributes at the bottom of the info table.
      5326988f
  10. 19 Oct, 2005 1 commit
  11. 17 Oct, 2005 1 commit
    • Kirk Webb's avatar
      · c045b869
      Kirk Webb authored
      Add serial and service_tag columns to the database for tracking machines.
      c045b869
  12. 11 Oct, 2005 1 commit
  13. 20 Sep, 2005 1 commit
  14. 07 Sep, 2005 2 commits
    • Mike Hibler's avatar
      Inner-elab role changes: · 88926d1c
      Mike Hibler authored
      	'boss' -> 'boss+router'
      	'ops'  -> 'ops+fs'
      	'fs'      (new role)
      	'router'  (new role)
      
      These are in preparation for allowing configurations with split ops and fs
      nodes (and sometime down the road, split boss and router nodes).
      
      This checkin is just the DB state changes along with the scripts that look
      at that state.  The Big One, which actually sets up separate nodes
      automatically, is undergoing more testing but will be Coming Soon.
      88926d1c
    • Robert Ricci's avatar
      Added a 'global_vtypes' table, which contains vtypes that get · 970f7244
      Robert Ricci authored
      defined for everyone. Admins can use this, for example, to make
      type aliases.
      970f7244
  15. 01 Sep, 2005 1 commit
  16. 19 Aug, 2005 2 commits
  17. 17 Aug, 2005 3 commits
    • Leigh B. Stoller's avatar
      The Emulab Knowledge Base! · 6f08c442
      Leigh B. Stoller authored
      Okay, I implemented a primitive Knowledge Base! The current contents are
      *all* the existing FAQ entries, which I entered manually. Here are the
      details.
      
      * My reason for doing this is that we need something very simple. The wiki
        is too much of a barrier, and its search capabilities are pathetic.
      
      * The search page for the Knowledge Base is:
      
      	https://www.emulab.net/kb-search.php3
      
        Fairly primitive keyword search. Turns out that mysql 4.0 has a bunch for
        really good text searching functions built in, but we run 3.23 ... so I
        had to roll it myself. So, its a simple keyword (space or comma
        separated) search, no regular expressions.
      
      * Each DB record has a "faq_entry" flag, so creating the current FAQ on the
        fly from the DB is easy. See:
      
      	https://www.emulab.net/kb-faq.php3
      
      * In reddot mode, you can add new KB entries:
      
      	https://www.emulab.net/kb-manage.php3
      
        The form is fairly obvious but here are details anyway:
      
          Section Name: Choose an existing title, or make up a new one.
          Title:        The title of the KB (or FAQ) entry.
          Faq Entry:    Check this box if the new entry should show up in the FAQ.
          X Ref Tag:    A token so you can refer to other KB entries by name,
                        instead of by its index. Within the KB entry you would
                        write: <a href=kb-show.php3?xref_tag=sometag>
          Body:         Whatever you like. I took the existing FAQ entries and
                        stuck them with no changes except for the xref_tag
                        mentioned about (since some entries referenced other
                        entries).
      
      * Once you click on sumbit, you will see the entry as it will appear to
        users, along with a submenu to Modify/Delete/Add entries. You can modify
        the current entry from that menu. Mere users do not see this menu, only
        when in reddot mode.
      
      * The intent here is that we can generate new entries really easy, right
        from email if you like (with appropriate <pre> or <xmp> tags around it).
      
      * I have added sql/knowlbase-create.sql and a makefile target to
        generate that file when creating a distribution. I also added a section
        to install/boss-install to insert the entries into the new DB.
      
      * I hooked the search function into the existing Search Documentation link.
        We know search both the Knowledge Base *and* the Documentation on doc
        searches. This probably needs a little more work to get right.
      
      * I changed a lot of faq links to be more consistent and to reference
        the proper xref_tags (#swapping instead of #UTT-34).
      6f08c442
    • Leigh B. Stoller's avatar
    • Leigh B. Stoller's avatar
      Add a last_activity field to user_stats table, and set that from libdb · eda71c58
      Leigh B. Stoller authored
      where we insert the testbed_stats record. This will make it much much faster
      to generate the recent users list in genelists (currently taking 3 seconds!).
      eda71c58
  18. 15 Aug, 2005 1 commit
    • Leigh B. Stoller's avatar
      The bulk of the mailman support. Still not turned on by default (cause · a64593f3
      Leigh B. Stoller authored
      Jay has "comments"), but I do not want it hanging around in my source
      tree. Here is my mail message:
      
      * The "My Mailing Lists" is context sensitive (copied from Tim's
        changes to the My Bug Databases). It takes you to the *archives* for
        the current project (or subgroup) list. Or it takes you to your
        first joined project.
      
      * The showproject and showgroup pages have direct links to the project
        and group specific archives. If you are in reddot mode, you also
        get a link to the admin page for the list. Note that project and
        group leaders are just plain members of these lists.
      
      * The interface to create a new "user" list is:
      
      	https://www.emulab.net/dev/stoller/newmmlist.php3
      
        We do not store the password, but just fire it over in the list
        creation process.
      
        Anyone can create their own mailing lists. They are not associated
        with projects, but just the person creating the list. That person
        is the list administrator and is given permission to access the
        configuration page.
      
        This page is not hooked in yet; not sure where.
      
      * Once you have your own lists, you user profile page includes a link
        in the sub menu: Show Mailman Lists. From this page you can delete
        lists, zap to the admin page, or change the admin password (which is
        really just a subpage of the admin page).
      
      * As usual, in reddot mode you can mess with anyone else's mailman lists,
        (via the magic of mailman cookies).
      
      * Note on cross machine login. The mailman stuff has a really easy way
        to generate the right kind of cookie to give users access. You can
        generate a cookie to give user access, or to the admin interface for
        a list (a different cookie). Behind the scenes, I ssh over and get
        the cookie, and set it in the user's browser from boss. When the
        browser is redirected over to ops, that cookie goes along and gives
        the user the requested access. No passwords need be sent around,
        since we do the authentication ourselves.
      a64593f3
  19. 08 Aug, 2005 1 commit
    • Leigh B. Stoller's avatar
      Add mailman passwords. One to the groups table to hold the admin · 092a4fdc
      Leigh B. Stoller authored
      password for each project/group list, and another to the users
      table to hold each users password.
      
      These passwords are stored in cleartext. This is okay since the
      mailman password is deemed to be throw away since the mailman
      system is happy to mail them out to people in cleartext.
      
      We initially set them to a random value, and will provide a web
      interface on the Emulab side to change them. This is done in the
      mailman setup script though.
      092a4fdc
  20. 24 Jun, 2005 1 commit
  21. 22 Jun, 2005 1 commit
    • Leigh B. Stoller's avatar
      Added my simplistic link tracing and monitoring. Example usage and · 7942119e
      Leigh B. Stoller authored
      some details can be found in the advanced tutorial that I wrote up.
      See this link:
      
      http://www.emulab.net/tutorial/docwrapper.php3?docname=advanced.html#Tracing
      
      The basic idea is that each virt_lan entry gets a couple of new slots
      describing the type of tracing that is desired.
      
        traced tinyint(1) default '0',
        trace_type enum('header','packet','monitor') NOT NULL default 'header',
        trace_expr tinytext,
        trace_snaplen int(11) NOT NULL default '0',
        trace_endnode tinyint(1) NOT NULL default '0',
      
      There is a new physical table called "traces" that is a little bit
      like the current delays table. A new tmcd command returns the trace
      configuration to the client nodes (tmcd/common/config/rc.trace).
      
      The delays table got a new boolean called "noshaping" that tells the
      delay node to bridge, but not set up any pipes. This allows us to
      capture traffic at the delay node, but without much less overhead on
      the packets.
      
      The pcapper got bloated up to do packet capture and more event stuff.
      I also had to add some mutex locking around calls into the pcap
      library and around malloc, since the current setup used linuxthreads,
      which is not compatable with the standard libc_r library. I was
      getting all kinds of memory corruption, and I am sure that if someone
      breathes on the pcapper again, it will break in some new way.
      7942119e
  22. 13 Jun, 2005 1 commit
    • Timothy Stack's avatar
      · 5e43a771
      Timothy Stack authored
      Initial checkin of a "repositioning" daemon that moves robots back to
      their pens on swapout.
      
      	* configure, configure.in: Add tbsetup/repos_daemon.
      
      	* db/libdb.pm.in: Add constants for the
      	repositionpending/repositioning experiments.
      
      	* db/nfree.in: When freeing garcias, send them to
      	repositionpending instead of reloadpending.
      
      	* event/sched/event-sched.c: Deal with the rare case of no
      	SIMULATOR object being in the agent list for an experiment.
      
      	* robots/emc/emcd.c, robots/emc/locpiper.in: Fix some typos.
      
      	* robots/rmcd/masterController.h, robots/rmcd/masterController.c,
      	robots/rmcd/obstacles.h, robots/rmcd/obstacles.c: Ignore dynamic
      	obstacles that are far away and remove dynamic obstacles where the
      	robot is inside the natural obstacle area.
      
      	* sql/database-create.sql, sql/database-migrate.txt: Add a
      	reposition_status table that tracks the status of robots that are
      	being moved back to their pens.
      
      	* tbsetup/GNUmakefile.in: Install the repos_daemon script.
      
      	* tbsetup/reload_daemon.in: Move robots to the repositionpending
      	experiment, if they haven't already reached their pen.
      
      	* tbsetup/repos_daemon.in: Daemon that takes care of seeing robots
      	back to their pens after they are freed from an experiment.
      5e43a771
  23. 25 May, 2005 1 commit
    • Leigh B. Stoller's avatar
      Add a "nosetup" option to elabinelab experiments. In the experiments · eed85271
      Leigh B. Stoller authored
      table, if elabinelab_nosetup is non-zero, boss and ops setup will do
      just enough to get the nodes into a state that hopefully approximates
      what a real installation might look like before installing our stuff.
      I do install the packages cause there is no point in waiting for that
      to finish interactively.
      
      From this point, you can log into the console(s) and run the setup
      instructions verbatim, although I have not actually tried that yet.
      
      The nice thing is that if you manage to get things properly setup, it
      can function as a real elabinelab since the outer environment has been
      setup. This is quite a bit different then how we tested during the
      last release frenzy.
      
      Its not quite perfect of course, since the images are not "clean", but
      I think this is okay for testing.
      eed85271
  24. 20 May, 2005 1 commit
    • Leigh B. Stoller's avatar
      Checkpoint some robot changes. · 5f67fe09
      Leigh B. Stoller authored
      * New robot event listener:
      
          * It is intended to be started and stopped from the experiment
            swapin path instead of as a global daemon. It takes the pid/eid
            of the experiment, and will deal with events only for those
            nodes that are allocated to the experiment. We have some long
            range plans of time sharing the robot lab, so I figured we might
            as get a little bit of a start on that.
      
          * Once it fires up, it subscribes to the usual assortment of
            events, just like the loclistener does.
      
          * It then binds a socket on which to listen for connections from
            the web server.
      
          * Then it loops, looking for events and for connections from the
            web server. Connections from the web server are for forwarding
            the event stream in real time to whatever applets are currently
            viewing the robot lab.
      
          * As each event comes in, it is parsed, entered into the DB (nodes
            and location_info table), and fired out (in a textual form) to
            all the applet listeners. The web interface just acts as pipe in
            this case for the data.
      
          * The event stream is also duplicated to a file in the experiment
            directory (the same stuff that is piped to the applet), named by
            the current resource record ID. I hope to use this stream to
            playback the motion in the applet (coupled with webcam images
            once I figure out how to sync them).
      
      * tbswap: Start and stop the new listener.
      
      * Robotrack: I changed the interface for how we actually communicate
        the event data. Much more reasonable then it was (a comma separated
        string of numbers!).
      
      * new database fields in the experiments table to hold the process ID
        of the listener and the port it is listening on. The port is not
        used yet, as the robot lab is still not shared. Will need to revist
        this later. Currently uses a fixed port number.
      
      * www/robotrack/robopipe.php3: Killed most of the old code and replace
        with simple socket connect to the listener.
      5f67fe09
  25. 16 May, 2005 1 commit
    • Leigh B. Stoller's avatar
      Add support for specifying the CVS tag to use when getting the source code · f1863cfd
      Leigh B. Stoller authored
      for the inner elab.
      
      	tb-set-elabinelab-cvstag dist-foo
      
      Will result in this branch getting checked out from the mirrored repository
      (updated nightly) on boss and sent back to the node, instead of the usual
      source tarball that we keep in /usr/testbed/src (still the default
      behaviour if no tag is specified. You can also do this if you like:
      
      	tb-set-elabinelab-cvstag HEAD
      
      which of course is a special tag to CVS.
      f1863cfd
  26. 12 May, 2005 1 commit
    • Leigh B. Stoller's avatar
      Checkpoint the rest of my changes to support swapmod of both ElabInElab and · 6eff9de6
      Leigh B. Stoller authored
      Firewalled experiments (see tbsetup/elabinelab.in for the other stuff).
      
      * To support firewalled experiments, needed to add a new virt_firewalls
        table to split the existing firewalls table up, which included both
        virtual and physical stuff. There are the usual frontend changes and a
        few other things scattered around, including tmcd.c.
      
      * The firewall code in tbswap got some beefing up to support adding and
        deleting nodes from the its special control net vlan. Note that I have
        not made any progress on containment of deleted nodes, just as we do not
        do anything now for teardown (unless its paniced, in which case the
        experiment cannot be modified anyway).
      
      * ptopgen and assign_wrapper got some interesting modifications: Unlike
        regular swapmod, we cannot just tear down all the vlans since that would
        interrupt everything inside the inner elab. Instead, leave the vlans as
        is. The problem is that when assign runs, it can just as easily pick
        different interfaces on the same nodes, which would be a royal pain in
        the ass to deal with! So, ptopgen got a new option (-u) that assign
        wrapper uses to tell ptopgen that it should prune out unused interfaces
        from nodes that are already allocated to the experiment. This is, at
        best, as pathetically gross hack, but it makes sure that all the
        interfaces stay the same across swapmods.
      
      * The unrelated revision of elabinelab has a bunch of new code for adding
        and deleting nodes from the inner elab. Mostly it deals with dhcpd (inner
        and outer, waiting for nodes to reboot, etc). It also deals with updating
        the vlans table in the DB, pruning out any nodes (ports) that are deleted
        but for which there are still interfaces in existing vlans. Said ports
        are them moved back to the default vlan with calls to snmpit. Also under
        another revision a a couple of weeks ago are the web interface changes to
        support the newnode MFS inside an inner Emulab.
      
      * swapexp and endexp got some more checks for firewalled and paniced
        experiments, which were missing.
      6eff9de6
  27. 15 Apr, 2005 1 commit
  28. 13 Apr, 2005 1 commit
  29. 06 Apr, 2005 1 commit
  30. 05 Apr, 2005 1 commit
  31. 30 Mar, 2005 1 commit
  32. 29 Mar, 2005 1 commit
  33. 28 Mar, 2005 1 commit
  34. 25 Mar, 2005 1 commit