Skip to content

GitLab

emulab-devel
emulab-devel
Commit fd4eaa8d authored by Mac Newbold's avatar Mac Newbold
Browse files
Options

Remove idlecheck. Add idletimes and idlemail (and webidlemail). 

(webidlecheck was removed in a separate commit.)

idletimes is the new commandline tool for checking idleness. It shows a
table like this:

PID          EID          Nodes  HrsIdle     Last Activity     Act. type
------------------------------------------------------------------------
AVQ          Tunnel           4   15.18   2003-04-07 21:18:20  tty
pces         toy              1    3.92   2003-04-08 08:34:02  tty
Spinglass    fawaz            1   13.76   2003-04-07 22:43:48  tty
TempleSCTP   FTP-REAL         1   26.40   2003-04-07 10:05:18  tty,net
testbed      bvclass          4   24.58   2003-04-07 11:54:13  net
testbed      ltest            1   14.37   2003-04-07 22:07:02  tty
testbed      ron-image        1  501.88   2003-03-18 13:36:28  cpu,ext,tty,net
testbed      Tone             1  501.88   2003-03-18 13:36:30  cpu,ext,tty,net

By default it only shows things that are over an idleness threshold. It
also has a mode to show all expts. Also can show idleness/activity by node
instead of by expt, which has a slightly different table. (s/Nodes/NodeID/)

idlemail isn't finished yet, but I don't want it to hold up idletimes and
the other changes any longer. It doesn't get used by anything yet, of
course.
parent 16321a1e
Hide whitespace changes
Inline Side-by-side
Showing with 300 additions and 235 deletions
configure
View file @ fd4eaa8d
......@@ -1338,7 +1338,8 @@ outfiles="$outfiles Makeconf GNUmakefile \
db/libdb.pm db/inuse db/avail db/nodeip db/showgraph \
db/dhcpd_makeconf db/nodelog db/webnodelog db/unixgroups \
db/dbcheck db/interswitch db/dbboot db/schemacheck \
db/grabron db/idlecheck db/webnfree db/stategraph db/readycount \
db/grabron db/webnfree db/stategraph db/readycount \
db/idletimes db/idlemail db/webidlemail \
discvr/GNUmakefile \
ipod/GNUmakefile \
lib/GNUmakefile lib/libtb/GNUmakefile \
......
configure.in
View file @ fd4eaa8d
......@@ -381,7 +381,8 @@ outfiles="$outfiles Makeconf GNUmakefile \
db/libdb.pm db/inuse db/avail db/nodeip db/showgraph \
db/dhcpd_makeconf db/nodelog db/webnodelog db/unixgroups \
db/dbcheck db/interswitch db/dbboot db/schemacheck \
db/grabron db/idlecheck db/webnfree db/stategraph db/readycount \
db/grabron db/webnfree db/stategraph db/readycount \
db/idletimes db/idlemail db/webidlemail \
discvr/GNUmakefile \
ipod/GNUmakefile \
lib/GNUmakefile lib/libtb/GNUmakefile \
......
db/GNUmakefile.in
View file @ fd4eaa8d
......@@ -11,11 +11,12 @@ UNIFIED = @UNIFIED_BOSS_AND_OPS@
include $(OBJDIR)/Makeconf
BIN_SCRIPTS = nalloc nfree nodeip idlecheck readycount
BIN_SCRIPTS = nalloc nfree nodeip readycount
SBIN_SCRIPTS = avail inuse showgraph if2port backup webcontrol node_status \
genelists genelists.proxy dhcpd_makeconf nodelog unixgroups \
dbcheck interswitch dbboot grabron stategraph newwanode
LIBEXEC_SCRIPTS = webnodelog webnfree webnewwanode
dbcheck interswitch dbboot grabron stategraph newwanode \
idletimes idlemail
LIBEXEC_SCRIPTS = webnodelog webnfree webnewwanode webidlemail
LIB_SCRIPTS = libdb.pm
# Stuff installed on plastic.
......
db/idlecheck.in deleted 100644 → 0
View file @ 16321a1e
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2002 University of Utah and the Flux Group.
# All rights reserved.
#
# idlecheck - See if node/expt is active
# Configure variables
use lib '@prefix@/lib';
use libdb;
use English;
# Turn off line buffering on output
$| = 1;
$h = 0; #help mode
$t=0;
$p=0;
$f=0;
$u=0;
$s=0;
$d = 0; #debug mode
sub help {
die("Usage:
idlecheck [-h] [-t hours] [-p pph] [-f min] [-u] [-s] [-d]
-h\tThis help message
-t\tTime (in hours) of inactivity required to be considered idle
-p\tPackets per hour required on average to be considered active
-f\tMaximum minutes elapsed for data to be considered fresh
-u\tShow unswappable experiments
-s\tShow experiments with stale data
-d\tDebug/verbose mode
Check activity for all experiments.
Currently the following qualify as activity:
* Packets sent/received on the experimental network
* Use of a tty
Activity does not currently include:
* Network traffic for nodes using automatic routing or trafgen daemons
* Computation (non-idle load averages)
* Control net traffic
* Control without the use of a tty\n"
);
}
my $defidlehours = 2;
print "Got ARGV = ",join(" ",@ARGV),"\n" if $d > 1;
my %args = ();
while ($#ARGV >= 0) {
$_ = shift;
$_ =~ /^(.)(.)/;
if ($1 ne '-') { next; }
if ($2 ne 'h' && $2 ne 't' && $2 ne 'p' && $2 ne 'f' &&
$2 ne 'u' && $2 ne 's' && $2 ne 'd') { next; }
my $opt="\L$2";
my $val = shift if ($#ARGV >= 0 && ($ARGV[0] =~ /^[^-]/));
if (!defined($val)) { $val=1; }
# Let -d be used multiple times
if ($opt eq 'd') { $d++; } else {
$$opt=$val;
}
if ($d>1) { print "$opt = $$opt\n"; }
}
if ($h) { help(); }
# Only root or admin types!
if (($UID != 0) && (!TBAdmin($UID))) {
die("Only root or TB administrators can run idlecheck.\n");
}
# Configurable parameters
my $idlehours = $t || $defidlehours;
my $minpph = $p || 1; # Min packets per hour a non-idle interface sends/rcvs
my $stalemin = $f || 65; # Max minutes of staleness for latest report
# Derived params
my $minpkts = $idlehours * $minpph;
my $idlesec = $idlehours * 3600;
my $stalesec = $stalemin * 60;
my $rtype="ospf";
my $node1="";
my $node2="";
my $node3="";
if ($n) {
$node1="a.node_id,";
$node2="and a.node_id=\"$n\"";
$node3=",node_id";
}
my %active=();
my %fresh=();
my %router=();
# This query finds how many packets the non-control net interfaces
# have sent in the last $idlesec seconds, and saves it in a temporary
# table. It then aggregates that and shows a list of pid/eid (and
# maybe nodes) that have an interface that sent/rcvd at least $minpkts
# packets in the time period. It also makes sure to only count entries
# since they've been swapped in, and makes sure they were swapped in
# at least $idlesec seconds ago.
# The last query below finds the last reported tty use for each
# expt. It only counts reports that have been received in the last
# $stalemin minutes and that have been since the expt was swapped in,
# and only shows expts that have been used in the last $idlesec seconds.
for my $cmd ("drop table if exists idletemp;",
"create temporary table idletemp
select r.pid,r.eid, routertype as router, $node1
max(ipkts)-min(ipkts) as idiff , max(opkts)-min(opkts) as odiff
from iface_counters as a
left join reserved as r on a.node_id = r.node_id
left join nodes as n on a.node_id=n.node_id
left join virt_trafgens as v on r.vname=v.vnode
left join interfaces as i on a.mac=i.mac
left join experiments as e on e.pid=r.pid and e.eid=r.eid
where tstamp >= expt_swapped
and v.vnode is null
and (unix_timestamp(now())-unix_timestamp(tstamp) <= $idlesec)
and (unix_timestamp(now())-unix_timestamp(expt_swapped) >= $idlesec)
and (routertype=\"$rtype\" or i.IP not like \"155.101.%\") $node2
group by r.pid,r.eid,a.mac;",
"select pid,eid, max(idiff), max(odiff) from idletemp
group by pid,eid $node3
having (max(idiff) >= $minpkts) or (max(odiff) >= $minpkts);",
# Ordering matters here! Make sure that the router query comes
# _after_ the idletemp query and _before_ all the other select queries!
"select pid, eid, router from idletemp
where router=\"$rtype\"
group by pid,eid ,router;",
"select r.pid,r.eid,max(last_tty) as lastuse ,max(tstamp) as t
from node_idlestats as n
left join reserved as r on n.node_id=r.node_id
left join experiments as e on e.pid=r.pid and e.eid=r.eid
where tstamp >= expt_swapped and r.pid is not null and r.eid is not null
group by pid,eid
having (unix_timestamp(now())-unix_timestamp(lastuse) <= $idlesec)
order by pid,eid,last_tty,tstamp;") {
print "Sending cmd:\n$cmd\n" if $d>1;
my $result = DBQueryFatal($cmd);
if ($cmd =~ /^select /i && $result->numrows() > 0) {
# Add the pid/eid to our list of active expts
while(@r=$result->fetchrow()) {
$pid=$r[0];
$eid=$r[1];
if (!($cmd =~ /where router/i)) {
print "Adding $pid/$eid to active list\n" if $d;
$active{"$pid/$eid"} = 1;
} else {
print "Removing $pid/$eid from active list - it has $rtype routers!\n" if $d;
$active{"$pid/$eid"} = 0;
$router{"$pid/$eid"} = 1;
}
}
}
print $result->as_string() if ($d>1);
}
# These queries find all experiments that have had all of their nodes
# report in the last $stalesec seconds.
my $result="";
foreach $cmd ("drop table if exists idletemp2",
"create temporary table idletemp2
select pid,eid,n.node_id,max(tstamp) as t from reserved as r
left join node_idlestats as n on r.node_id=n.node_id
where r.node_id not like 'sh%' and r.node_id not like 'wireless%'
and r.node_id not like '%ron%'
group by n.node_id order by pid,eid,t;",
"select pid,eid,min(t) as oldest from idletemp2
where t is not null group by pid,eid
having (unix_timestamp(now())-unix_timestamp(oldest)<=$stalesec)
order by pid,eid;") {
print "Sending cmd:\n$cmd\n" if $d>1;
$result = DBQueryFatal($cmd);
}
while(@r=$result->fetchrow()) {
$pid=$r[0];
$eid=$r[1];
print "Adding $pid/$eid to fresh data list\n" if $d;
$fresh{"$pid/$eid"} = 1;
}
print $result->as_string() if ($d>1);
my @list=();
# Find expts that have nodes reserved, and look up their bits.
$cmd = "select r.pid,r.eid,swappable,expt_swapped from reserved as r
left join experiments as e on e.pid=r.pid and e.eid=r.eid
where (unix_timestamp(now())-unix_timestamp(expt_swapped) >= $idlesec)
and idle_ignore=0 group by r.pid,r.eid order by r.pid,r.eid";
print "Sending cmd:\n$cmd\n" if $d>1;
$result = DBQueryFatal($cmd);
while(@r=$result->fetchrow()) {
$pid=$r[0];
$eid=$r[1];
$swap=$r[2];
$idle=!(defined($active{"$pid/$eid"}) && $active{"$pid/$eid"});
$router=defined($active{"$pid/$eid"}) && $router{"$pid/$eid"};
$stale=!(defined($fresh{"$pid/$eid"}) && $fresh{"$pid/$eid"});
print "Checking for $pid/$eid in active list\n" if $d;
# Now output the results
# IMPORTANT: If you make changes to output format, be sure to update
# testbed/www/showexp_list.php3 as well, since it reads this output
my $str= "$pid/$eid";
$str = $str . " " x (28-length($str))." ";
$str .= ($idle? "inactive\t" : "\t\t" );
$str .= ($stale?"stale\t" : "\t" );
$str .= (!$swap? "unswappable\t" : "\t\t" );
$str .= ($router? "$rtype\n" : "\n" );
if (($idle && !$stale && $swap) ||
($stale && $s) ||
(!$swap && $u)) { print $str; }
if ($idle) { push(@list,"(pid='$pid' and eid='$eid')"); }
}
print $result->as_string() if ($d>1);
if (($idlehours==$defidlehours) && (@list > 0)) {
# Don't clear flags when we're using a time other than the normal one
$cmd = "update experiments set swap_requests=0 where not (".
join(" or ",@list).")";
print "Sending cmd:\n$cmd\n" if $d>1;
DBQueryWarn($cmd);
}
exit(0);
db/idlemail.in 0 → 100644
View file @ fd4eaa8d
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2002 University of Utah and the Flux Group.
# All rights reserved.
#
# idlecheck - See if node/expt is active
# Configure variables
use lib '@prefix@/lib';
use libdb;
use English;
use Getopt::Std;
# Turn off line buffering on output
$| = 1;
# Defaults
my $defidlehours = 2;
# Don't put 'my' on these, or they won't be settable with ${$var}
$h = 0; # help mode
$d = 0; # debug mode
$f = 0; # force mode
$t = $defidlehours; # Threshold idle time
print "Got ARGV = ",join(" ",@ARGV),"\n" if $d;
sub help {
die("Usage:
idlemail [-h] [-d] [-f] [<pid> <eid>]
-h Show this help message
-d Enable debugging/verbose output
-f Force sending a message for <pid> <eid>
If <pid> and <eid> are supplied, send a swap request for that experiment.
Without -f, the message won't be sent if pid/eid hasn't been idle for at
least $t hours.
idlemail runs periodically (via cron(8)) to send email messages
regarding experiments that are idle for over $t hours.\n");
}
my $optlist = "hdf";
my %opt = ();
if (! getopts($optlist,\%opt)) { help(); }
# Copy the options into global vars
foreach $var (keys %opt) {
${$var} = $opt{$var};
print "\$$var = $opt{$var} (".${$var}.")\n" if $d;
}
my $pid = shift || "";
my $eid = shift || "";
print "Settings: h=$h d=$d f=$f pid=$pid eid=$eid\n" if $d;
if ($h) { help(); }
# Only root or admin types!
if (($UID != 0) && (!TBAdmin($UID))) {
die("Only root or TB administrators can run idlemail.\n");
}
if ($pid eq "" || $eid eq "") {
# Normal mode
# Construct the query
my $lastact_query = "greatest(last_tty_act, last_net_act, ".
"last_cpu_act, last_ext_act)";
my $sql = "select pid, eid, last_report,
max(last_tty_act) as last_tty_act, max(last_net_act) as last_net_act,
max(last_cpu_act) as last_cpu_act, max(last_ext_act) as last_ext_act,
max($lastact_query) as last_act, round((unix_timestamp(now()) -
unix_timestamp(max($lastact_query)))/3600,2) as idle_time,
count(r.node_id) as nodes
from node_activity as n left join reserved as r on n.node_id=r.node_id
where pid is not null and eid is not null
group by pid,eid order by pid,eid";
my $q = DBQueryFatal($sql);
if ($d) { print $q->as_string; $q->dataseek(0); }
while (%r = $q->fetchhash()) {
$pid = $r{'pid'};
$eid = $r{'eid'};
#rep = $r{'last_report'};
$tty = $r{'last_tty_act'};
$net = $r{'last_net_act'};
$cpu = $r{'last_cpu_act'};
$ext = $r{'last_ext_act'};
$act = $r{'last_act'};
$time= $r{'idle_time'};
$nodes=0;
$id="";
if (!$n) { $nodes = $r{'nodes'}; }
else { $id = $r{'node_id'}; }
%type = ();
if ($tty eq $act) { $type{"tty"} = 1; }
if ($net eq $act) { $type{"net"} = 1; }
if ($cpu eq $act) { $type{"cpu"} = 1; }
if ($ext eq $act) { $type{"ext"} = 1; }
$typestr = join(",",keys %type);
if (!$a && ($time < $t)) { next; }
# Do something
}
} else {
# pid/eid mode - send one
}
exit(0);
sub SendMessage {
my ($
db/idletimes.in 0 → 100644
View file @ fd4eaa8d
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2002 University of Utah and the Flux Group.
# All rights reserved.
#
# idletimes - show how long nodes/expts have been idle
# Configure variables
use lib '@prefix@/lib';
use libdb;
use English;
use Getopt::Std;
# Turn off line buffering on output
$| = 1;
# Defaults
my $defidlehours = 2;
# Don't put 'my' on these, or they won't be settable with ${$var}
$h = 0; # help mode
$d = 0; # debug mode
$a = 0; # show all
$n = 0; # show nodes
$t = $defidlehours; # Threshold idle time
print "Got ARGV = ",join(" ",@ARGV),"\n" if $d;
sub help {
die("Usage:
idletimes [-h] [-d] [-a] [-n] [-t hrs]
-h Show this help message
-d Enable debugging/verbose output
-a Show all expts (or nodes with -n)
-n Show nodes instead of experiments
-t hrs Threshold idle time (in hours, default $defidlehours)
Note: idletimes shows experiments that are set to idle_ignore.
");
}
my $optlist = "hdant:";
my %opt = ();
if (! getopts($optlist,\%opt)) { help(); }
# Copy the options into global vars
foreach $var (keys %opt) {
${$var} = $opt{$var};
print "\$$var = $opt{$var} (".${$var}.")\n" if $d;
}
print "Settings: h=$h d=$d a=$a n=$n t=$t\n" if $d;
if ($h) { help(); }
# Only root or admin types!
if (($UID != 0) && (!TBAdmin($UID))) {
die("Only root or TB administrators can run idletimes.\n");
}
# Construct the query
my $lastact_query = "greatest(last_tty_act, last_net_act, ".
"last_cpu_act, last_ext_act)";
my $sql = "";
if (!$n) {
$sql .= "select pid, eid, last_report,
max(last_tty_act) as last_tty_act, max(last_net_act) as last_net_act,
max(last_cpu_act) as last_cpu_act, max(last_ext_act) as last_ext_act,
max($lastact_query) as last_act, round((unix_timestamp(now()) -
unix_timestamp(max($lastact_query)))/3600,2) as idle_time,
count(r.node_id) as nodes
from node_activity as n left join reserved as r on n.node_id=r.node_id
where pid is not null and eid is not null
group by pid,eid order by pid,eid";
} else {
$sql .= "select pid, eid, n.node_id, last_report,
last_tty_act, last_net_act, last_cpu_act, last_ext_act,
$lastact_query as last_act, round((unix_timestamp(now()) -
unix_timestamp($lastact_query))/3600,2) as idle_time
from node_activity as n left join reserved as r on n.node_id=r.node_id
where pid is not null and eid is not null
order by pid,eid,n.node_id";
}
my $q = DBQueryFatal($sql);
if ($d) { print $q->as_string; $q->dataseek(0); }
if (!$n) {
$FORMAT_NAME = "ByExpt";
print <<EOT;
PID EID Nodes HrsIdle Last Activity Act. type
------------------------------------------------------------------------
EOT
} else {
$FORMAT_NAME = "ByNode";
print <<EOT;
PID EID NodeId HrsIdle Last Activity Act. type
-------------------------------------------------------------------------
EOT
}
format ByExpt =
@<<<<<<<<<<< @<<<<<<<<<<<<< @## @##.## @<<<<<<<<<<<<<<<<<< @<<<<<<<<<<<<<<
$pid, $eid, $nodes, $time, $act, $typestr
.
format ByNode =
@<<<<<<<<<<< @<<<<<<<<<<<<< @<<<< @##.## @<<<<<<<<<<<<<<<<<< @<<<<<<<<<<<<<<
$pid, $eid, $id, $time, $act, $typestr
.
while (%r = $q->fetchhash()) {
$pid = $r{'pid'};
$eid = $r{'eid'};
#rep = $r{'last_report'};
$tty = $r{'last_tty_act'};
$net = $r{'last_net_act'};
$cpu = $r{'last_cpu_act'};
$ext = $r{'last_ext_act'};
$act = $r{'last_act'};
$time= $r{'idle_time'};
$nodes=0;
$id="";
if (!$n) { $nodes = $r{'nodes'}; }
else { $id = $r{'node_id'}; }
%type = ();
if ($tty eq $act) { $type{"tty"} = 1; }
if ($net eq $act) { $type{"net"} = 1; }
if ($cpu eq $act) { $type{"cpu"} = 1; }
if ($ext eq $act) { $type{"ext"} = 1; }
$typestr = join(",",keys %type);
if (!$a && ($time < $t)) { next; }
write();
}
exit(0);
db/webidlemail.in 0 → 100644
View file @ fd4eaa8d
#!/usr/bin/perl -w
#
# EMULAB-COPYRIGHT
# Copyright (c) 2000-2002 University of Utah and the Flux Group.
# All rights reserved.
#
use English;
#
# This gets invoked from the Web interface. Simply a wrapper ...
#
# usage: webidlemail arguments ...
#
#
# Configure variables
#
my $TB = "@prefix@";
#
# Run the real thing, and never return.
#
exec "$TB/bin/idlemail", @ARGV;
die("webidlemail: Could not exec idlemail: $!");
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment