Commit d17d8f30 authored by Jonathon Duerig's avatar Jonathon Duerig
Browse files

Documentation for various files and source cleanup in

parent 774a197b
......@@ -6,7 +6,7 @@ certain check results and progress messages are printed to standard error
as well. When the program is integrated with the rest of Emulab, these will
be removed.
Note that for now, the command line option should include '-c ', then either '-s ' or '-p<number> ', then one of '-h ', '-l ', or '-n '.
Note that for now, the command line option should include '-c ', then either '-s ' or '-p<something> ', then one of '-h ', '-l ', or '-n '.
Example: "bin/ipassign -c -p2 -n < graph/pastry44.graph"
Use conservative bitspace allocation for 2 partitions with
......@@ -140,32 +140,101 @@ Framework -- The framework processes the command line arguments and uses
as an intermediary for feeding the results of ip assignment
into the router.
ConservativeAssigner --
HostRouter --
LanRouter --
NetRouter --
SquareRootPartition -- the square root of the number of LANs is calculated
and this number is used to determine how many partitions METIS should
SearchPartition -- METIS is called repeatedly with different numbers of
partitions and the result is scored. The best scored partitioning is then
used. The scoring function optimizes for quickest routing speed by
cubing the number of border LANs and adding the result to the sum of
the cubes of each partition size.
FixedPartition -- METIS is used to create a specific number of partitions
ConservativeAssigner -- The largest LAN size is used to determine the number
of bits used to represent hosts in a LAN. The largest partition size is
used to determine the number of bits to represent LANs in a partition,
and the remaining bits are used to represent partition number. This
causes waste in bitspace usage, but the bitspace is large enough that this
should only cause problems when tens or hundreds of thousands of nodes
are in a network. If the largest LAN, the largest partition, and the number
of partitions can each fit within their own 8-bitspace, then 8-bit
boundaries are used because of human readability. Note that in addition
to IP assignment, disconnected partitions are seperated here.
HostRouter -- Ignore any hierarchy that was given to us by ip assignment.
Use routecalc to get the shortest path from every node to every other node
and use this information to calculate a route from every node to every
LanRouter -- Ignore any hierarchy that was given to us by ip assignment.
Use routecalc to get the shortest path from every LAN to every other LAN,
and use the result to calculate a route from every node to every LAN.
NetRouter -- This is a generalized algorithm that takes advantage of all levels
of hierarchy that ip assignment provides. Here is the outline of that
A) Algorithm
1) for each level starting at just above the LAN level
a) for each partition in the current level
i) calculateBorders()
ii) calculateBorderToBorderPaths()
iii) calculateNodeToBorderPaths()
iv) calculatePartitionRoutes()
B) calculateBorders
1) The border LANs at each level are a subset of the border LANs of
the level below.
2) Therefore, only border LANs of the level below need
to be checked to see if they are still border LANs.
3) To check if a LAN is a border LAN, go through every host that it
connects to. If any host connects to more than one partition on
this level, this is a border LAN.
C) calculateBorderToBorderPaths
1) Take all the border nodes within the current partition and
temporarily renumber them so that they are contiguous and small.
2) Feed them into routecalc
3) Take the results and update the borderConnection structure to
reflect the known routes.
D) calculateNodeToBorderPaths
1) From any host, we already know the routes to the border LANs in its
partition from the level below.
2) We also know the routes between the old border LANs and the new
ones from G.
3) Therefore, we can find the shortest path to the new border LANs by
systematically combining pairs of the two routes.
E) calculatePartitionRoutes
1) We know the routes from each host to every border LAN in its
sub-partition and routes from each border LAN in that sub-partition
to every other border LAN in the partition. Calculating routes is
similar to the above.
Non-numeric and non-whitespace characters in the input are invalid.
LANs must have at least two nodes connected to them.
Input graphs should be connected
Each LAN must have at least two nodes connected to it.
Input graphs should be connected.
Some graph configurations could cause the bitspace to be used up.
One of the input arguments may be invalid.
Various impossible conditions should never happen.
Room for Improvement
Take the adaptive ip-assignment algorithm and turn it into an ip assignment
module which works with the framework.
Create an ip-assignment module that provides more than one additional level
of hierarchy.
Change output format to be more compact. Possible change it to some form of
binary representation.
Modify both ip assignment and routing to allow for disconnected graphs.
Find a better scoring algorithm for METIS.
Find a replacement for METIS which is more suited to this particular problem.
I have talked with Scott a bit about a suggestion for using spanning-trees
in partitioning.
Improve logging/testing automation.
In the LAN-router, circular topologies are generated for large graphs. I am
investigating it. This is not that critical as LAN routing is not used
......@@ -7,7 +7,6 @@
#include "lib.h"
#include "Exception.h"
#include "Assigner.h"
......@@ -9,6 +9,11 @@
// This is the base class for all ip assignment algorithms. Using
// this class, the ip assignment algorithm can be selected dynamically.
// To create a new ip assignment module, create a class which inherits from
// Assigner and implements all of the below methods as described.
// Don't forget to change the command-line interface to actually use your
// new ip assignment strategy.
......@@ -17,25 +22,65 @@
class Assigner
// Note that I refer to 'partitions' in general. Each level is a
// partitioning up of the level below. On the lowest level, each LAN is
// a partition of its own.
// A NodeLookup table allows one to find the partitions which a particular
// node belongs too. The outer vector is a selector based on the node
// number. The inner vector is a list of partition numbers that node
// belongs too. In order to have node referencing to multiple levels of
// hierarchy, there should be a vector of these.
typedef std::vector< std::vector<size_t> > NodeLookup;
// A MaskTable associates each partition number with the size of the
// subnet mask for that partition. For instance, a LAN with 8 bits of
// bitspace allocated to it would have '24' as its entry in the MaskTable.
// To have Masks on different levels, it is necessary to have a vector of
// these tables.
typedef std::vector<int> MaskTable;
// Each partition has a prefix associated with it. This prefix represents
// the the first n bits of any IP address for all interfaces that are
// contained within the partition. 'n', in this case, is the size of the
// subnet mask for that partition. To have more than one level of prefixes,
// a vector of PrefixTables is required.
typedef std::vector<IPAddress> PrefixTable;
// A partition on each level is made up of sub-partitions on the level
// below. (On the bottom level, each LAN is made up of nodes). This table
// relates each partition number to a list of partition numbers for the
// level below. To represent many levels of partitioning, a vector of
// LevelLookups is necessary.
typedef std::vector< std::vector< size_t > > LevelLookup;
// As you can see, the final data structure representing one part of the
// hierarchy can seem pretty complex. One can have vectors of vectors of
// vectors. Because of this, whenever possible, you should use references
// as named temporaries to try to make this conceptually simple.
virtual ~Assigner();
// Create a copy of the current object polymorphically.
virtual std::auto_ptr<Assigner> clone(void) const=0;
// This may be called more than once to set the partitioning method
// selected by the command line arguments. Note that only a handle is
// being passed in. Ownership remains outside of this object.
virtual void setPartition(Partition & newPartition)=0;
// This is called repeatedly to form the graph. Raw parsing
// of the command line is done outside.
// Any numbers between [0..MaxNodeNumber] that are unused
// should be represented with the constant INVALID_NODE
virtual void addLan(int, int, std::vector<size_t>)=0;
// The main processing function. After the graph has been populated,
// assign IP addresses to the nodes.
virtual void ipAssign(void)=0;
// Output the network and associated IP addresses to the specified stream.
virtual void print(std::ostream &) const=0;
// Populate the argument vectors with our state so that routes can
// be calculated.
// for the explanation of these arguments, see Router.h
......@@ -18,7 +18,8 @@ ConservativeAssigner::ConservativeAssigner(Partition & newPartition)
, m_largestLan(0)
, m_lanMaskSize(0)
, m_netMaskSize(0)
, m_partition(newPartition)
, m_partition(&newPartition)
// m_lanList, m_nodeToLan, and m_partitionIPList initialize themselves.
......@@ -31,35 +32,49 @@ auto_ptr<Assigner> ConservativeAssigner::clone(void) const
return auto_ptr<Assigner>(new ConservativeAssigner(*this));
void ConservativeAssigner::setPartition(Partition & newPartition)
void ConservativeAssigner::setPartition(Partition & newPartition)
m_partition = &newPartition;
void ConservativeAssigner::addLan(int bits, int weight, vector<size_t> nodes)
// add the new LAN to m_lanList
Lan temp;
temp.number = m_lanList.size();
temp.weight = weight;
temp.nodes = nodes;
// if this LAN is the largest we've encountered, note that fact.
m_largestLan = max(m_largestLan, static_cast<int>(nodes.size()));
// Update m_nodeToLan such that the nodes with interfaces in this LAN
// know that fact.
for (size_t i = 0; i < nodes.size(); ++i)
if (nodes[i] >= m_nodeToLan.size())
m_nodeToLan.resize(nodes[i] + 1);
vector<size_t> & currentNode = m_nodeToLan[nodes[i]];
void ConservativeAssigner::ipAssign(void)
// TODO: When I fix the disconnected graphs problem, remove this.
// The reason that this is here is that disconnected graphs don't
// cause an error until route calculation. The error that is caused
// is not obviously due to the graph not being connected. Therefore,
// this is provided as a hint until that is fixed.
if (!(isConnected(0)))
cerr << "Graph is not connected" << endl;
// These hold a METIS-ready conversion of the LAN-graph data.
std::vector<int> partitionArray;
std::vector<int> graphIndexArray;
std::vector<int> graphNeighborArray;
......@@ -69,10 +84,11 @@ void ConservativeAssigner::ipAssign(void)
convert(graphIndexArray, graphNeighborArray, weightArray);
// partition the graph
m_partition.partition(graphIndexArray, graphNeighborArray, weightArray,
m_partition->partition(graphIndexArray, graphNeighborArray, weightArray,
m_partitionCount = m_partition.getPartitionCount();
m_partitionCount = m_partition->getPartitionCount();
// This is the code for calling METIS and changing its tolerances.
/* if (numPartitions > 1)
int numConstraints = 2;
......@@ -100,6 +116,7 @@ void ConservativeAssigner::ipAssign(void)
&(partitionArray[0])); // return the partition of each vertex
// Each number in the partitions vector is the partition that
// the associated Lan in m_lanList belongs too.
// Let each Lan know where it belongs.
......@@ -111,41 +128,20 @@ void ConservativeAssigner::ipAssign(void)
pos->partition = *partPos;
// If there are disconnected partitions, turn them into different
// partitions.
// m_partitionCount may increase during this loop
int loop = 0;
while (loop < m_partitionCount)
// Each time a breadth-first walk from a LAN in a partition fails
// to reach every LAN in that partition, the remainder is slopped
// together into a new partition which is m_partitionCount+1.
// Then m_partitionCount is incremented.
for (int i = 0; i < m_partitionCount; ++i)
if (!(isConnected(i)))
cerr << "Not connected " << i << endl;
// output how many border LANs there are.
vector<bool> border;
fill(border.begin(), border.end(), false);
for (size_t i = 0; i < border.size(); ++i)
for (size_t j = 0; j < m_lanList[i].nodes.size(); ++j)
size_t node = (m_lanList[i].nodes)[j];
for (size_t k = 0; k < m_nodeToLan[node].size(); ++k)
size_t destLan = m_nodeToLan[node][k];
if (m_lanList[i].partition != m_lanList[destLan].partition)
border[i] = true;
// find out which bits go to which subnet
......@@ -180,6 +176,8 @@ void ConservativeAssigner::print(ostream & output) const
output << lanPos->number << " ";
output << lanPos->nodes[i] << " ";
// The ip address of a particular interface is just an offset
// from the prefix associated with its LAN.
output << ipToString(lanPos->ip + i + 1);
output << endl;
......@@ -223,10 +221,6 @@ void ConservativeAssigner::graph(vector<Assigner::NodeLookup> & nodeToLevel,
/* for (size_t j = 0; j < m_nodeToLan[i].size(); ++j)
......@@ -353,6 +347,9 @@ int ConservativeAssigner::getBits(int num)
int ConservativeAssigner::getLanBits(vector<int> const & partitionArray,
int numPartitions)
// Create an array of counters, adding to the appropriate counter
// whenever we encounter a LAN in that partition. Then take the largest
// and voila! instant largest partition!
vector<int> counter;
int current;
......@@ -397,7 +394,7 @@ void ConservativeAssigner::assignNumbers(int networkSize, int lanSize,
fill(counter.begin(), counter.end(), 0);
// we want to set up intuitive bit boundaries if possible
// we want to set up human readable bit boundaries if possible
if (networkSize <= 8 && lanSize <= 8 && hostSize <= 8)
networkSize = 8;
......@@ -433,13 +430,14 @@ void ConservativeAssigner::assignNumbers(int networkSize, int lanSize,
bool ConservativeAssigner::isConnected(int whichPartition)
// size_t orig = m_partitionCount;
// makeConnected(whichPartition);
// return orig >= m_partitionCount;
// Use a queue for a breadth-first search
queue<size_t> nextConnection;
size_t first = 0;
size_t numInPartition = 0;
// This extra scoping is to make sure that i can be re-used.
// Find out how many LANs are in whichPartition and find an arbitrary
// starting LAN for the search.
for (size_t i = 0; i < m_lanList.size(); ++i)
if (m_lanList[i].partition == whichPartition)
......@@ -449,40 +447,36 @@ bool ConservativeAssigner::isConnected(int whichPartition)
if (whichPartition == 0)
cerr << whichPartition << ':' << numInPartition << endl;
if (numInPartition == 0)
// trivial case
if (numInPartition == 0 || m_lanList.size() == 0)
cerr << "Empty partition " << whichPartition << endl;
return true;
vector<bool> connected;
fill(connected.begin(), connected.end(), false);
connected[first] = true;
if (m_lanList.size() > 0)
// breadth-first search
while (!(nextConnection.empty()))
while (!(nextConnection.empty()))
size_t current = nextConnection.front();
// add all adjascent LANs in this partition
for (size_t i = 0; i < m_lanList[current].nodes.size(); ++i)
size_t current = nextConnection.front();
for (size_t i = 0; i < m_lanList[current].nodes.size(); ++i)
size_t currentNode = m_lanList[current].nodes[i];
for (size_t j = 0; j < m_nodeToLan[currentNode].size(); ++j)
size_t currentNode = m_lanList[current].nodes[i];
for (size_t j = 0; j < m_nodeToLan[currentNode].size(); ++j)
size_t destLan = m_nodeToLan[currentNode][j];
if (m_lanList[destLan].partition == whichPartition
&& !(connected[destLan]))
size_t destLan = m_nodeToLan[currentNode][j];
if (m_lanList[destLan].partition == whichPartition)
if (!(connected[destLan]))
connected[destLan] = true;
connected[destLan] = true;
......@@ -490,11 +484,14 @@ bool ConservativeAssigner::isConnected(int whichPartition)
vector<bool>::difference_type num = count(connected.begin(),
// iff we got to every node in this partition during our search,
// then the partition is connected.
return num == numInPartition;
void ConservativeAssigner::makeConnected(int whichPartition)
// breadth first search. This is very similar to isConnected()
queue<size_t> nextConnection;
size_t first = 0;
size_t numInPartition = 0;
......@@ -24,15 +24,24 @@ private:
: weight(0), partition(0), number(0), ip(0)
// The partition number associated with this LAN. This is the index
// of the LAN in m_lanList. Not to be confused with 'partition'.
size_t number;
// The weight of this LAN.
int weight;
// The partition number of the super-partition this LAN is a part of.
int partition;
// The IP Address prefix that this LAN provides. Note that with
// conservative IP assignment, every LAN has the same netmask.
IPAddress ip;
// The list of nodes which have interfaces in this LAN.
std::vector<size_t> nodes;
ConservativeAssigner(Partition & newPartition);
virtual ~ConservativeAssigner();
// For explanation of the purpose of these functions, see 'Assigner.h'
virtual std::auto_ptr<Assigner> clone(void) const;
virtual void setPartition(Partition & newPartition);
......@@ -59,24 +68,47 @@ private:
void convertAddNode(Lan const & info, int currentNode,
std::vector<int> & neighbors,
std::vector<int> & weights) const;
// Given a number, how many bits are required to represent up to and
// including that number?
static int getBits(int num);
// How many bits are required to represent each LAN in each partition?
static int getLanBits(std::vector<int> const & partitionArray,
int numPartitions);
static int roundUp(double num);
// If we don't have enough bitspace to represent every partition, LAN,
// and interface, throw an exception and abort.
static void checkBitOverflow(int numBits);
// Actually give ip prefixes to each LAN and partition.
void assignNumbers(int networkSize, int lanSize, int hostSize,
int numPartitions);
// Do the LANs withing a particular partition have a path to each other?
bool isConnected(int whichPartition);
// If a particular partition is disconnected, seperate it into two
// partitions.
void makeConnected(int whichPartition);
// Information about each LAN, indexed by the number assigned to each
// LAN.
std::vector<Lan> m_lanList;
// At this point, we only have to worry about one level of hierarchy.
// Given a node number, what are the LANs to which it belongs?
// See Assigner.h for a definition of 'NodeLookup'
NodeLookup m_nodeToLan;
// The number of partitions the LAN-graph was divided up into.
int m_partitionCount;
// The number of nodes in the largest LAN.
int m_largestLan;
// The number of bits in the netmask of each LAN.
int m_lanMaskSize;
// The number of bits in the netmask of each partition.
int m_netMaskSize;
// IP prefixes for every partition.
PrefixTable m_partitionIPList;
Partition & m_partition;
// This pointer is used polymorphically to determine which partitioning
// method to use.
Partition * m_partition;
......@@ -10,12 +10,6 @@
// the program. Each one has a string associated with it that is printed
// to the user as output.
// Note that as of now, the strings are completely determined by the
// creator of an Exception object. I ought to go back and change this
// so that each Exception contributes its exception name to the beginning
// of the string and the Exception creator just provides the incidental
// information.
......@@ -42,6 +36,7 @@ private:
string message;
// Can be thrown during IP assignment
class BitOverflowException : public StringException
......@@ -51,6 +46,7 @@ public:
// Can be thrown during input phase
class InvalidCharacterException : public StringException
......@@ -60,6 +56,7 @@ public:
// Can be thrown during input phase
class MissingWeightException : public StringException
......@@ -69,6 +66,7 @@ public:
// Can be thrown during input phase
class NotEnoughNodesException : public StringException
......@@ -78,6 +76,8 @@ public:
// Should not be thrown.
// This is deprecated and will be removed soon.
class NoHeaderException : public StringException
......@@ -87,6 +87,7 @@ public:
// Can be thrown during startup
class InvalidArgumentException : public StringException
......@@ -96,6 +97,8 @@ public:
// Can be thrown in various places. I put this here whenever there was
// a dangling 'else' or 'default' that should never be reached.
class ImpossibleConditionException : public StringException
......@@ -105,6 +108,8 @@ public: