Commit 2602e6b9 authored by Jonathon Duerig's avatar Jonathon Duerig

Merge branch 'master' of

parents 4c28b1bb 0b83c91a
......@@ -56,7 +56,7 @@ open(STDOUT, ">> $logname") or die("opening $logname for STDOUT: $!");
# Clear the frisbee_blobs table since any frisbee's
# that were running are obviously not running anymore!
DBQueryFatel("delete from frisbee_blobs");
DBQueryFatal("delete from frisbee_blobs");
# Clear the event scheduler pids. Its okay to set them to zero since
......@@ -46,6 +46,10 @@ NEW FEATURES:
data will go into every chunk and still compress that, but there
is no point since we pad out every chunk to 1MB.
[ The discovery protocol is now done. On-the-fly images has not
though we can distribute arbitrary files, we do so without creating
imagezip-format images. ]
Now one could imagine a super, caching frisbee server that creates
compressed images on the fly and caches them for later use. Perhaps
it would become more of a chunk server where it caches every chunk
......@@ -137,6 +141,13 @@ ENHANCEMENTS:
chunks. So maybe 1448B/blk * 768 blks/chunk == 1.06MB/chunk. PREQUEST
BlockMaps come down from 128 bytes to 96.
[ Support for jumbo packets has been done. This increases the block
size to 8192 and reduces the blocks/chunk to 128 to keep the chunk
size constant (so that we can continue to distribute our existing
images). Currently this requires static re-compilation of both the
client and server, though some support has been put in for negotiating
the blocksize (in join v2 messages). ]
6. Dynamic rate pacing in the server.
Our attempts to date have been pretty feeble. I think we have a
......@@ -152,6 +163,36 @@ ENHANCEMENTS:
and frisbeed pad it out, we can save a lot of disk space for these
small images.
[ DONE ]
8. Allow a server to serve multiple unicast clients.
Right now an instance of the server not only serves just a single
image, but only to a single destination address. This is reasonable
for broadcast/multicast but is overly restrictive for unicast. Changing
this should be minor, we just need to keep track of destinations
(addr/port) in the queue along with block ranges to send out. We would
need to back off the queue optimizations where we combine incoming
requests with those already in the queue (i.e., now we would also have
to make sure that they are for the same destination before we combine).
Minor changes would be needed to PacketSend/Recv to track the client
IP/port rather than just assuming/using the global mcastaddr/portnum.
9. Allow the frisbee client to be used in a pipe.
If we could pipe the output of frisbee into another utility, this would
make it more useful for arbitrary file distribution. For example:
frisbee -m <addr> -p <port> - | tar xzf
to download and unpack a tarfile. The problem is out-of-order processing
of chunks and there are a couple of ways around it. Frisbee can already
request chunks in-order, but it is also opportunistic and saves other
chunk data it needs that other clients have requested. We could just
ignore that data and keep re-requesting blocks in order as we need them,
or we could do some limited memory caching of incoming data; i.e., save
but don't decompress chunks until we need them. We could cache to disk
as well, but then we don't really save anything over just frisbeeing into
a tmp file and giving that to the next util in the pipeline.
1. Have seen the clients run out of socket buffer space causing them
......@@ -12,7 +12,8 @@ use vars qw(@ISA @EXPORT);
@ISA = "Exporter";
@EXPORT = qw(Parse ParseFile IsVersion0 FindNodes FindNodesNS
FindFirst FindElement FindAttr IsLanNode IsLocalNode IsTunnel GetExpires
FindFirst FindElement FindAttr GetNodeByVirtualId GetLinkByVirtualId
IsLanNode IsLocalNode IsTunnel GetExpires
GetNodeId GetVirtualId GetInterfaceId GetInterfaceNodeId GetSliverId
GetManagerId GetColocate GetSubnodeOf
GetVirtualizationType SetVirtualizationType GetVirtualizationSubtype
......@@ -189,6 +190,35 @@ sub FindAttr($$)
return FindNodeType($path, $node, XML_ATTRIBUTE_NODE);
sub GetElementByVirtualId($$$)
my ($name, $type, $node) = @_;
my @list = FindNodes('n:'.$type.'[@virtual_id = "'.$name.'"]',
my $result = undef;
if (scalar(@list) > 0)
$result = $list[0];
if (scalar(@list) > 1)
cluck("WARNING: Multiple $type with virtual_id $name found");
return $result;
sub GetNodeByVirtualId($$)
my ($name, $node) = @_;
return GetElementByVirtualId($name, 'node', $node);
sub GetLinkByVirtualId($$)
my ($name, $node) = @_;
return GetElementByVirtualId($name, 'link', $node);
# Returns true if a given XML Node is an RSpec node and is of type lan
sub IsLanNode($)
......@@ -526,6 +556,14 @@ sub RemoveChild($$)
# Replaces a node (oldnode) with a copy of another node (newnode)
sub ReplaceNode($$)
my ($oldnode, $newnode) = @_;
my $copy = $newnode->cloneNode(1);
# checks for the existense of policy in extensions of the given
# credential.
sub PolicyExists($$)
......@@ -99,6 +99,7 @@ use libtblog;
use libvtop;
use libadminctrl;
use User;
use EmulabFeatures;
# Protos
sub fatal(@);
......@@ -220,6 +221,11 @@ if (!TBAdmin() &&
# multiplex_factor default.
$mfactor = $experiment->multiplex_factor()
if (!defined($mfactor) && defined($experiment->multiplex_factor()));
my $newassign =
$experiment->GetGroup(), $experiment);
# These are the flags to the vtop creation code.
......@@ -557,6 +563,8 @@ sub RunAssign($$)
# Run assign
my $cmd = "assign";
$cmd .= "-new"
if ($newassign);
my $args = "";
$args .= "-f rspec "
if ($userspec == 1);
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment