GitLab

directory.

The biggest changes were related to 10.0's replacing gcc with clang
and the switch to a new package system. The former required various
fixes to Makefiles that hardwired "gcc" and also missing prototypes
in source files that clang is particularly obnoxious about.

There was also accomodating the normal evolution of the ports tree.
New variables were needed in many of the Makefiles for this. Hopefully,
all the changes I made are backward compatible, but I don't actually
care that much since I hope to never, ever have to remake those older
packages! A new emulab version of the m2crypto port was needed to
reflect that m2crypto is no longer installed as an .egg file.

Casualties:

  * Emulab assign. As of this commit, assign does not build with
    clang. It will build, but immediately core dumps if built with
    a gcc port (gcc46 below). THIS NEEDS TO BE FIXED!

  * Emulab nsverify. This requires building a pure ns-2.34, which
    is old and does not compile with clang. You will need to set
    NSVERIFY=0 in your defs file. I did this in defs-elabinelab
    (iff FBSD==10) but no where else right now.

  * Bind in the base distro. Bind is no longer part of FreeBSD
    (it has been replaced with something called "unbound"). So we
    install bind99 from a port and we make the /etc/namedb symlink
    to /usr/local/etc/namedb which is where it now lives.

  * Perl 5.12. It has been totally removed from the ports tree.
    We now install 5.14, the next-most obsolete version of Perl!

  * pkg_{add,delete,info}. As mentioned, the packages tools have
    changed. For the most part "pkg <cmd>" is the same as "pkg_<cmd>"
    but not always. This required considerable violence in the
    install/phases code. But it is actually cleaner now.

  * GCC in the base distro. I added installation of the gcc46 port
    to the boss and ops meta packages, just for old-times sake
    (we might wind up needing it, if still more stuff doesn't work
    with clang).

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* statements, the licensing status of
the source files was unclear.

This commit is intended to clear up that confusion.

Most Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the Affero GNU General Public License, version 3
(AGPLv3).

Most Utah-copyrighted files related to ProtoGENI are distributed under
the terms of the GENI Public License, which is a BSD-like open-source
license.

Some Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the GNU Lesser General Public License, version 2.1
(LGPL).

For now, I made it possible to set speed of the capture with
tiptunnel.  Later we could expose more options... but the right
thing to do is build a ConsoleAgent that is much more flexible.
I think we now know what we need as far as a good feature set
for that program.

gets the stuff from the DB instead of the xmlrpc server.

things for which we previously linked -static.

Fixup some clean targets in a couple of cases as well.

Get capture to compile without SSL (capture-nossl).  This is untested.

of adding copyrights to them.

strange cause it exists when the surrounding shell exits, so I changed
things a bit to avoid the problem. tiptunnel is a train wreck that
needs a total redo.

Pass through some args.

XMLRPC server on boss.

daemonize after making the connection to capture so an error is
reported if it cannot connect.

so needed to resplit the install/control-install targets.

Mote and robot related stuff.  The main thing is the addition of relay
capabilities to capture and related things.

	* GNUmakefile.in: Add the capture and tip subdirectories to the
	client and client-install targets.

	* configure, configure.in, config.h.in: Detect srandomdev() for
	capture and add "mote/newmote" script.

	* capture/GNUmakefile.in, capture/capture.c: Add "relay"
	capabilities to capture.

	* capture/capquery.c: Query the capserver for the relay receiver's
	port number.

	* capture/capserver.c: Small hack to return the port number
        for a node.

	* db/libdb.pm.in, db/xmlconvert.in: Add virt_tiptunnels table.

	* event/program-agent/program-agent.c: Change log file names to
	something a little more user-friendly.  Add a "MODIFY" event
	handler that lets the user set agent attributes (command, tag,
	timeout) without having to run a program.

	* event/sched/GNUmakefile.in, event/sched/console-agent.cc,
	event/sched/console-agent.h, event/sched/event-sched.c: Add
	console agents that can be used to snapshot a section of the
	capture log file.

	* event/sched/node-agent.cc: Some minor cleanup.

	* event/sched/simulator-agent.cc, event/sched/simulator-agent.h:
	Add the config data to the report mail.  Add a "RESET" event
	handler that runs "loghole clean".  Save the report mail in a file
	so it gets archived with the rest of the logs.

	* lib/libtb/tbdefs.h: Add CONSOLE object type.

	* mote/GNUmakefile.in, mote/newmote: Add newmote script, just a
	quick hack to add motes to the DB.

	* mote/tbuisp.in: Add another backend for loading motes through
	their relay capture server.

	* robots/mtp/mtp_dump.c: Dump the min/max values for x and y,
	handy for figuring out the bounds of the camera.

	* sql/database-fill.sql: Change the RELOAD-MOTE/SHUTDOWN ->
	ALWAYSUP/SHUTDOWN mode transition to ALWAYSUP/ISUP since stated
	doesn't seem to run triggers after a state change by a mode
	transition.

	* tbsetup/tbreport.in: Change the ordering of the eventlist so it
	displays event-sequences appropriately.

	* tbsetup/ns2ir/GNUmakefile.in, tbsetup/ns2ir/console.tcl,
	tbsetup/ns2ir/node.tcl, tbsetup/ns2ir/parse.tcl.in,
	tbsetup/ns2ir/sim.tcl.in: Add a "console" agent that represents
	the serial console for a node.

	* tbsetup/ns2ir/sequence.tcl: Add an "append" method so it is
	easier to build sequences dynamically.

	* tbsetup/ns2ir/topography.tcl: Make checkdest available to
	regular users.

	* tip/GNUmakefile.in, tip/tiptunnel.c: Add support for uploading a
	file to a relay version of capture and exporting the end
	connection as a pty.

	* tmcd/decls.h, tmcd/common/libsetup.pm: Bump version number since
	the dosubnodelist change is not backwards compatible.

	* tmcd/tmcd.c: Make dosubnodelist and dosubconfig callable even
	when a node isn't allocated.  Add dotiptunnels command that
	returns which serial consoles are to be mounted on a node.  Add
	mote version of subconfig that returns information needed to
	startup the relay version of capture.

	* tmcd/common/bootsubnodes: For motes, startup the relay version
	of capture (XXX stargate specific).

	* tmcd/common/libsetup.pm, tmcd/common/libtmcc.pm,
	tmcd/common/config/rc.config, tmcd/common/config/rc.tiptunnels:
	Client side changes for mounting another nodes serial line.

	* tmcd/common/rc.bootsetup: Always boot the subnodes, even when
	free.  This is used for motes since their capture needs to be up
	for reloading at the time.

	* tmcd/linux/ixpboot: Shuffle some code around so the script
	doesn't fail if the ixp isn't allocated.

	* utils/loghole.in: Add "digest.out" and "report.mail" as global
	logs to be saved in archives and display the "report.mail" file
	when showing a loghole archive.

	* xmlrpc/emulabserver.py.in: Scrub more of the return values to
	get rid of "None"s.

Added makefile mojo to install "tip has been deprecated" script
in place of tip in testbed.

Changed format for passing in command to execute
to use @s instead of %s..
also removed exteraneous output when not using -v.

Added "console" binary target for tiptunnel... use in the same way as "tip"
(e.g. logged into plastic).

all ".gz"s are installed from /www/downloads

(Client was sending NULLs, which were echoed back

by the server as literally a '^' and a '@'),

To tip (or tiptunnel on a normal acl,) capture behaves the same.
However, if a client connects and presents "USESSL" as the first six characters of their
connection key, both sides initiate SSL negotiation.
The server then attempts to get the key again. The second one is used for the check.

SSL initialization is done on the first attempt by a client to connect via SSL.
Capture assumes $(prefix)/etc/capture/cert.pem contains its certificate unless
the '-c <certfile>' option is used.. if the certificate is not found or invalid, that
connection fails, but normal connections will still succeed (and it will try to find the file
again, next time an SSL connection is attempted.)

On the client side, tiptunnel only uses ssl if there is a "ssl-server-cert:"
property in the acl file. This is the SHA hash of the certificate that the capture server is
expected to have (in hex.) If the certificate presented by the server does not hash to the
same value, the connection is dropped.

Added "fakie telnet" to tunnel; tells client to not act stupid (no local echo and no line-at-a-time,)
and filters out client telnet replies so they don't blow the server's mind.

Tiptunnel can now take "ssl-server-cert:" property from the ACL file,
which is a SHA hash of the expected server certificate.

(this is used to verify the server's identity,
thus precluding man-in-the-middle attacks.)

If no "ssl-server-cert:" is in the ACL,
it will revert to using a normal TCP connection.

In this version, authentication is still the same (even over SSL.)
(next step: add SSL to capture server.)

Essentially tip, but instead of presenting a tty, it opens a tunnel port that (for instance)
telnet can talk to. Example (on credit):

tiptunnel /var/log/tiplogs/pc1.acl telnet

Will open up a local port then fork/exec telnet with "localhost" and the tunnel port number as arguments.
(Functionally equivalent to "tip pc1", only with telnet escape sequences)

A later version of this program is what users will likely download for the quick-tip-through-the-web scheme.
(next step: SSL)

ack/nak for a connection so that the connecting process knows what the
hell is going on. Turned out to be necessary for power control since
we do that in parallel, and because it stays busy for 10 seconds on
each power control. I think we will end up revisiting this at some
point, adding blocking connections instead of connect/fail status.

attempted. Good for debugging.

capable of rerunning hunt, and it was too confusing anyway.

Need to reset the getremote state so that we can do a second search
in /etc/remote file.

is actually used anymore.
Added a "generic" entry to /etc/remote so that we do not need tip
entries for each node; they all look the same anyway.
Change tip to lookup up generic /etc/remote entry, just to make
tip happy. The acl file comes from the tiplogs directory, as
set in the header file.

machine.

distributed tiplines (and capture processes).

defaulting to sockets based first. This is handy cause we oftern want
to talk to a real tty device (like the power controllers).

of pty/tty based (since they have several annoying problems
associated). Note that permission is granted via the use of an "acl"
file; /dev/tip/machine.acl, which must be set to the group of the
project the node is in, so the user can read out the process id number
and the random bits that are used by capture to grant permission to
use (tip sends the random bits across first thing). This handshake is
due to change to a request/challenge scheme as described by Dave in
email to the testbed list.