All new accounts created on Gitlab now require administrator approval. If you invite any collaborators, please let Flux staff know so they can approve the accounts.

Commit 8fd5c467 authored by Leigh B. Stoller's avatar Leigh B. Stoller

Add "distributed" calculation of the hosts file. If a topomap file is

present (gettopmap() returns valid) turn the topomap into input to feed
into the genhostsfile program (see initial revision for os/genhostsfile),
Take the output and feed that into the parsing code.

NB: If we change the output of tmcd hostnames call, we have to change the
output of this program too. This will likely become a problem at some
point, although the format of the hostnames command has not changed for
about 2 years now.

The point of doing this is to remove the bottleneck of computing hostsfiles
from tmcd, which centrally computes them for every node, and in a big topo
(100s of nodes) that takes a lot of time and bw.
parent 8e73e24d
......@@ -96,6 +96,9 @@ SWITCH: for ($action) {
}
exit(0);
# More protos
sub fromtopo($);
#
# Boot Action.
#
......@@ -106,7 +109,12 @@ sub doboot()
print STDOUT "Checking Testbed hostnames configuration ... \n";
if (tmcc(TMCCCMD_HOSTS, undef, \@tmccresults) < 0) {
#
# First see if we have a topo file; we can generate our own hosts
# file if we do, saving a lot of load on tmcd in big experiments.
#
if (fromtopo(\@tmccresults) < 0 &&
tmcc(TMCCCMD_HOSTS, undef, \@tmccresults) < 0) {
fatal("Could not get hosts file from server!");
}
# Important; if no results then do nothing. Do not want to kill
......@@ -195,3 +203,96 @@ sub docleanup()
fatal("Could not copy default $HOSTSFILE into place!");
}
}
#
# Generate hosts file (as if it came from tmcd) locally if we have a topo file.
#
sub fromtopo($)
{
my ($rptr) = @_;
my @results = ();
my $topomap;
my $mapfile = "$BOOTDIR/hostmap";
my ($pid, $eid, $vname) = check_nickname();
my %nodes = ();
my %lans = ();;
if (gettopomap(\$topomap)) {
return -1;
}
# The nodes section tells us the name of each node, and all its links.
foreach my $noderef (@{ $topomap->{"nodes"} }) {
my $vname = $noderef->{"vname"};
my $links = $noderef->{"links"};
my $count = 0;
$nodes{$vname} = [];
# Links is a string of "$lan1:$ip1 $lan2:$ip2 ..."
foreach my $link (split(" ", $links)) {
my ($lan,$ip) = split(":", $link);
push(@{ $nodes{$vname} }, "$count:$ip");
$lans{"$vname:$count"} = $lan;
$count++;
}
}
#
# Construct input for external program.
#
if (! open(MAP, ">$mapfile")) {
warn("*** WARNING: Could not create $mapfile!\n");
@$rptr = ();
return -1;
}
#
# First spit out virt_nodes
#
print MAP scalar(keys(%nodes)) . "\n";
foreach my $node (keys(%nodes)) {
my @members = @{ $nodes{$node} };
print MAP "$node,";
print MAP join(" ", @members);
print MAP "\n";
}
#
# Then spit out virt_lans.
#
print MAP scalar(keys(%lans)) . "\n";
foreach my $member (keys(%lans)) {
my $lan = $lans{$member};
print MAP "$lan,$member\n";
}
close(MAP);
#
# Now run the dijkstra program on the input.
#
if (!open(GENH, "cat $mapfile | $BINDIR/genhostsfile $vname |")) {
warn("*** WARNING: Could not invoke genhostsfile on mapfile!\n");
@$rptr = ();
return -1;
}
while (<GENH>) {
push(@results, $_);
}
if (! close(GENH)) {
if ($?) {
warn("*** WARNING: genhostsfile exited with status $?!\n");
}
else {
warn("*** WARNING: Error closing genhostsfile pipe: $!\n");
}
@$rptr = ();
return -1;
}
@$rptr = @results;
return 0;
}
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment