GitLab

Yeah, this only delays the inevitable, but doing better will require
more careful re-architecture

When set, assign is not allowed to map this link to an interswitch link;
this is to force a link to be resolved by two nodes connected to the
same switch.

Part of ticket #64

Now, we treat values < 0 as meaning 'no limit', and 0 means 'limit to
none'

Allowing oversubscription can push the current_load of a pnode
over its max_load, which causes us to try to take the square
root of a negative number. So, add an explicit check for that case.

Looks like we are going to want the ability to tweak vnode placement per
run rather than by building new assign binaries. So, add new '-S
balance' and '-S pack' options. I suspect it would also be useful to add
'-S spread' to try to spread vnodes out across as many hosts as
possible, some people want this when using shared hosts.

When set, the goal is to pack vnodes as tightly as possible - that is,
to penalize wasted slots in assignments.

One downside is that this will tend to make really lopsided mappings:
eg. if you have 50 slots per pnode, and 51 vodes, you will end up
mapping 50 to one and 1 to the other.

The other downside is that I have not yet thought about how to balance
this with link weights.

...bringing assign kicking and screaming into the century of the fruitbat.

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* statements, the licensing status of
the source files was unclear.

This commit is intended to clear up that confusion.

Most Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the Affero GNU General Public License, version 3
(AGPLv3).

Most Utah-copyrighted files related to ProtoGENI are distributed under
the terms of the GENI Public License, which is a BSD-like open-source
license.

Some Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the GNU Lesser General Public License, version 2.1
(LGPL).

When set, randomizes the order of nodes before making pclasses. This
has the effect of putting different nodes at the 'front' of the pclass,
The idea is to avoid 'hot spots' of nodes that get over-used simply
because they happen to appear earlier in the .ptop file.

NB: because STL seems to use a different source of randomness, using
this with '-s' to set a seed does *not* make assign deterministic.

Detect location of boost headers, since they moved in some recent
version.
(cherry picked from commit b7f6ae13)

Have I mentioned that I hate it when library developers move their
headers around? Change the location of the property_list.hpp header
to a macro and add a new -DNEWER_BOOT define that swtiches this to
the newer location.

Not automatically detected yet; should be checked in configure.
(cherry picked from commit 2403b264)

Detect location of boost headers, since they moved in some recent
version.

Have I mentioned that I hate it when library developers move their
headers around? Change the location of the property_list.hpp header
to a macro and add a new -DNEWER_BOOT define that swtiches this to
the newer location.

Not automatically detected yet; should be checked in configure.

This renames, rather than removes, ptop/vtop files that fail to
parse, so that they can be looked at later.

Handle deeper directory structures for topfiles, and when
using -n, restore the mtime of the original file (so that
we can use it for determining trends over time)

Add support for a few features that have been added since this was
originally written

Fix a bug where the total bandwidth for pnodes that are directly
connected (without an intervening switch) is not kept track of
correctly.

Originally introduced into parse_ptop.cc, and spread to the other
parsers via copy and paste.

Each pclass hold a mapping from type names to nodes of that type
that are currently available for use. This structure was not being
properly maintained; when a mapping was removed, only the entry
for the pnodes *current* type was being put back. When the pnode
goes empty, *all* of its possible types should be marked as available.

The consequence was that for pnodes that could take on multiple types
(which is the common case in Emulab), once a node had been considered
for a certain type (say, 'pc', 'pcvm', or 'delay'), it would *never*
be considered for another type again.

Add a new function that gets called when a pnode goes empty, to reset
*all* of its type mappings in its pclass.

Fixing this bug could have some serious positive consequences.

Code didn't match comment; comment was right.

PNODE_ALWAYS_FRONT wasn't used for anything

PNODE_SWITCH_LOAD was a bugfix, which we had turned on by default, and
there was no point in being able to turn the bug back on.

It was a good idea, but we've never used it in production, it hasn't
compiled in years, and it was just cluttering up the code.

The 'best' variable didn't really store the best score seen so far,
so rename it to prev_score. Likewise for the bestviolated variable.

Re-think a very old idea in assign - that no_connect violations for
unmapped links should not be assessed against links whose endpoints are
not mapped. The problem with this is that assign is too hesitant to map
nodes - if there is more than one link that can't be mapped or whose
other end isn't mapped yet, then assign thinks it's better to leave the
node unmapped. This leads to confusing solutions (in which nodes remain
unmapped when really it's the links that are the problem), and I beleive
it keeps assign from exploring valuable parts of the solution space.

This patch chanes the behavior, so that no_connect violations get scored
even when the endpoints of the link are not mapped.

Not well enough tested to run in production.

I think this must get the award for oldest (frequently excercised) bug
in the Emulab source. Thanks to the magic of the git pickaxe, I
discovered that code with this problem was first committed Monday, May
22, 2000, when it used to be in assign_hw/assign.cc

assign used to use a priority queue to 'randomly' select unassigned
nodes to try assigning. Turns out this is not very random! A node can
get unlucky, get a very low random priority assigned to it, and get
stuck in the queue for a very, very, very long time. As a result, I've
seen assign try to re-assign the same node several thousand times in a
row, when there are others waiting to be re-assigned. Of course, this is
going to result in very poor exploration of the state space.

I changed the unassigned node selection process to use a regular list,
and select an item from the list at random at the time of item removal.
This gets a much more even distribution of nodes, at the small cost of
linear iteration over the list of unassigned nodes.  This should be a
pretty minor cost, though, as iteration over this list should be cheap
and the list itself should generally be pretty small.

Also added some debugging statements.

Add debugging statements useful in tracking down which vnodes assign attempts
to assign to which pnodes.

Was returning a structure with a pointer to a stack-allocated string in
it. Use knowledge about where that string came from to avoid having to
make a copy of it, instead.

There is still a minor memory leak, but left behind a note rather than
fixing it (which would be quite involved).

Because it's operating on char*s, the STL list remove() function
doesn't work; it needs to do a strcmp, not a simple comparison. So, hand-write
the loop to do this.

Also add a santity check to produce a fatal error rather than allowing this
function to go into an infinite loop.

Think I got it this time (yeah, I've said that four times before).
It requires some extra bookkeeping; we were recording source and
dest interface names in tb_plink, but not source and dest node
names. With this new information, find_best_link() now has sole
responsibility for figuring out if the vlink and plink go in the
same direction, and the callers of find_best_link() are now
responsible for knowing which direction they are traversing the
vlink in (and thus whether they should tell find_best_link() to
check the source or dest of the vlink.)

Okay, I know I said this the first three times, but I think I've got it
this time - the problem had to do with how I was handling the flipped
flag (of course). Many of the calls find_best_link() give the dest_pv
of a link instead of the pv (source pnode). When we do this, we have to
invert the flipped flag, 'cause we're effectively reversing the order
of the link.

Normalized the error message for excessive link properties and corrected the behaviour of the parser when no link properties are specified.

than earlier.