• Leigh Stoller's avatar
    Rework how we store the sliver/slice status from the clusters: · e5d36e0d
    Leigh Stoller authored
    In the beginning, the number and size of experiments was small, and so
    storing the entire slice/sliver status blob as json in the web task was
    fine, even though we had to lock tables to prevent races between the
    event updates and the local polling.
    
    But lately the size of those json blobs is getting huge and the lock is
    bogging things down, including not being able to keep up with the number
    of events coming from all the clusters, we get really far behind.
    
    So I have moved the status blobs out of the per-instance web task and
    into new tables, once per slice and one per node (sliver). This keeps
    the blobs very small and thus the lock time very small. So now we can
    keep up with the event stream.
    
    If we grow big enough that this problem comes big enough, we can switch
    to innodb for the per-sliver table and do row locking instead of table
    locking, but I do not think that will happen
    e5d36e0d
Name
Last commit
Last update
..
aptui Loading commit data...
autostatus-icons Loading commit data...
blob Loading commit data...
buildui Loading commit data...
cvsweb Loading commit data...
doc Loading commit data...
downloads Loading commit data...
floormap Loading commit data...
garcia-telemetry Loading commit data...
hyperviewer Loading commit data...
jabber Loading commit data...
js Loading commit data...
netlab Loading commit data...
pix Loading commit data...
protogeni Loading commit data...
robotrack Loading commit data...
sdr Loading commit data...
sec-check Loading commit data...
timetree Loading commit data...
tutorial Loading commit data...
usermap Loading commit data...
webdb Loading commit data...
wikidocs/wiki Loading commit data...
wireless-stats Loading commit data...
.htaccess Loading commit data...
1px.gif Loading commit data...
BlinkenLichten$1.class Loading commit data...
BlinkenLichten.class Loading commit data...
BlinkenLichten.java Loading commit data...
GNUmakefile.in Loading commit data...
JSON.php Loading commit data...
Sajax.php Loading commit data...
TOCHECK Loading commit data...
TODO Loading commit data...
WebCamApplet.jar Loading commit data...
accept_cert.html Loading commit data...
adminlogin.php Loading commit data...
alternate_domains_emulab.net.php Loading commit data...
approveproject.php3 Loading commit data...
approveproject_form.php3 Loading commit data...
approveproject_list.php3 Loading commit data...
approveuser.php3 Loading commit data...
approveuser_form.php3 Loading commit data...
archive_control.php3 Loading commit data...
archive_list.php Loading commit data...
archive_missing.php3 Loading commit data...
archive_tag.php3 Loading commit data...
archive_tags.php3 Loading commit data...
archive_view.php3 Loading commit data...
arrow4.ico Loading commit data...
banner.svg Loading commit data...
beginexp.php Loading commit data...
beginexp_form.php3 Loading commit data...
blockstore_defs.php Loading commit data...
blueball.gif Loading commit data...
blueball.png Loading commit data...
boot.php3 Loading commit data...
bootlog.php3 Loading commit data...
builtwith.eps Loading commit data...
builtwith.png Loading commit data...
builtwith.sxd Loading commit data...
bullet-aqua.png Loading commit data...
bullet-blue.png Loading commit data...
bullet-bluegreen.png Loading commit data...
bullet-green.png Loading commit data...
bullet-orange.png Loading commit data...
bullet-purple.png Loading commit data...
bullet-red.png Loading commit data...
bullet-red.svg Loading commit data...
bullet-yellow.png Loading commit data...
busy.gif Loading commit data...
cdromcheckin.php3 Loading commit data...
changeall.csh Loading commit data...
changeuid.php Loading commit data...
chpasswd.php3 Loading commit data...
clientui.php3 Loading commit data...
coming-soon-thumb.gif Loading commit data...
common-style.css Loading commit data...
console.gif Loading commit data...
cool.gif Loading commit data...
copyright-local.txt Loading commit data...
copyright-standard.html Loading commit data...
copyright.html Loading commit data...
copyright.php Loading commit data...
create-dataset.php Loading commit data...
csshover.htc Loading commit data...
cssmenu-new.css Loading commit data...
cssmenu.css Loading commit data...
currentusage.php3 Loading commit data...
dbcheck.php3 Loading commit data...
dbdefs.php3.in Loading commit data...
defs.php3.in Loading commit data...
delaycontrol.php3 Loading commit data...
deleted_pubs.php Loading commit data...
deletegroup.php3 Loading commit data...
deleteimageid.php3 Loading commit data...
deletenodelog.php3 Loading commit data...
deleteosid.php3 Loading commit data...
deleteproject.php3 Loading commit data...
deleteuser.php3 Loading commit data...
delmmlist.php3 Loading commit data...
doc.php3 Loading commit data...
docwrapper.php3 Loading commit data...
dynselect.js Loading commit data...
edit-dataset.php Loading commit data...
editexp.php3 Loading commit data...
editgroup.php3 Loading commit data...
editimageid.php3 Loading commit data...
editnodetype.php3 Loading commit data...
editsitevars.php3 Loading commit data...
elabinelab.pdf Loading commit data...
elabinelab.sxd Loading commit data...
emailus.php3 Loading commit data...
emulab-mime-winxp.reg Loading commit data...
emulab-mime-winxp.zip Loading commit data...
emulab-powered-research.png Loading commit data...
emulab-powered-research.svg Loading commit data...
emulab.app Loading commit data...
emulab.scpt Loading commit data...
emulab_sup.js Loading commit data...
emusearch.xml.in Loading commit data...
endexp.php3 Loading commit data...
epc-diagram.png Loading commit data...
error.php3 Loading commit data...
error.shtml Loading commit data...
experiment_defs.php Loading commit data...
experimentrun_show.php Loading commit data...
explist.php3 Loading commit data...
expubs.php Loading commit data...
fancy-banner.svg Loading commit data...
fancy-builtwith.jpg Loading commit data...
fancy-builtwith.png Loading commit data...
fancy-builtwith.svg Loading commit data...
fancy-header-CMULAB.svg Loading commit data...
fancy-header-apt.png Loading commit data...
fancy-header-background-short.jpg Loading commit data...
fancy-header-background.jpg Loading commit data...
fancy-header-cloudlab.png Loading commit data...
fancy-header-cmuemulab.png Loading commit data...
fancy-header-emulab.net.png Loading commit data...
fancy-header-genirack.png Loading commit data...
fancy-header-phantomnet.png Loading commit data...
fancy-header.jpg Loading commit data...
fancy-sheader-CMULAB.svg Loading commit data...
fancy-sheader-apt.png Loading commit data...
fancy-sheader-cloudlab.png Loading commit data...
fancy-sheader-cmuemulab.png Loading commit data...
fancy-sheader-emulab.net.png Loading commit data...
fancy-sheader-genirack.png Loading commit data...
fancy-sheader-phantomnet.png Loading commit data...
faq.html Loading commit data...
faq.php3 Loading commit data...
favicon.ico Loading commit data...
favicon.svg Loading commit data...
feedback.php3 Loading commit data...
fetchlogfile.html Loading commit data...
floormap.php3 Loading commit data...
floormap_aux.php3 Loading commit data...
form_defs.php Loading commit data...
form_example.php Loading commit data...
freenode.php3 Loading commit data...
freezeuser.php3 Loading commit data...
geni-auth.js Loading commit data...
geni-login.php Loading commit data...
geni_defs.php Loading commit data...
genihistory.php Loading commit data...
genislices.php Loading commit data...
gensslcert.php3 Loading commit data...
getsslcert.php3 Loading commit data...
getsslcertjs.php3 Loading commit data...
gotobugdb.php3 Loading commit data...
gotochat.php3 Loading commit data...
gotommlist.php3 Loading commit data...
gotopeer.php Loading commit data...
gototrac.php3 Loading commit data...
gotowiki.php3 Loading commit data...
gradient-black-flipped.png Loading commit data...
gradient-black.png Loading commit data...
greenball.gif Loading commit data...
greenball.png Loading commit data...
group_defs.php Loading commit data...
hardware-cmuemlab.html Loading commit data...
hardware.php Loading commit data...
headerbgbb.jpg Loading commit data...
htmlinstall.in Loading commit data...
image_metadata.php Loading commit data...
imageid_defs.php Loading commit data...
index-cmuemulab.html Loading commit data...
index-mainsite.html Loading commit data...
index-nonmain.html Loading commit data...
index-phantomnet.html Loading commit data...
index.html Loading commit data...
index.php3 Loading commit data...
installwikidocs.in Loading commit data...
instance_show.php Loading commit data...
java.html Loading commit data...
jay.jpg Loading commit data...
jay.php Loading commit data...
joinproject.php3 Loading commit data...
json.js Loading commit data...
kb-browse.php3 Loading commit data...
kb-faq.php3 Loading commit data...
kb-show.php3 Loading commit data...
lanicon.png Loading commit data...
lease_defs.php Loading commit data...
ledpipe.php3 Loading commit data...
linkgraph_image.php Loading commit data...
linkmon_ctl.php3 Loading commit data...
linkmon_list.php3 Loading commit data...
linkmon_mon.php3 Loading commit data...
linktest.php3 Loading commit data...
list-datasets.php Loading commit data...
listrepos.php3 Loading commit data...
loadimage.php3 Loading commit data...
logger.css Loading commit data...
logger.js Loading commit data...
login.php3 Loading commit data...
login_plab.php3 Loading commit data...
login_redirect.php Loading commit data...
logo-background.gif Loading commit data...
logo-emulab.gif Loading commit data...
logo.sxd Loading commit data...
logoff.gif Loading commit data...
logon.gif Loading commit data...
logout.php3 Loading commit data...
manifesthistory.php Loading commit data...
menu-collapsed.png Loading commit data...
menu-expanded.png Loading commit data...
menu.php3 Loading commit data...
modifyexp.php3 Loading commit data...
modnodeattributes.php3 Loading commit data...
modnodeattributes_form.php3 Loading commit data...
moduserinfo.php3 Loading commit data...
moteleds.php3 Loading commit data...
mtp.jar Loading commit data...
mungelog.js Loading commit data...
mychat.php3 Loading commit data...
netbed-icon-howto.txt Loading commit data...
netbed-large.png Loading commit data...
netbed.ico Loading commit data...
netbed.png Loading commit data...
netemu.php3 Loading commit data...
netlab-client.jar Loading commit data...
new.gif Loading commit data...
newgroup.php3 Loading commit data...
newimageid.php3 Loading commit data...
newimageid_ez.php3 Loading commit data...
newmmlist.php3 Loading commit data...
newnode-defs.php3 Loading commit data...
newnode_edit.php3 Loading commit data...
newnodecheckin.php Loading commit data...
newnodelog.php3 Loading commit data...
newnodelog_form.php3 Loading commit data...
newnodes_list.php3 Loading commit data...
newosid.php3 Loading commit data...
newproject.php3 Loading commit data...
news-rss.php3 Loading commit data...
news.php3 Loading commit data...
node_defs.php Loading commit data...
nodecontrol.php3 Loading commit data...
nodecontrol_form.php3 Loading commit data...
nodecontrol_list.php3 Loading commit data...
nodeicon.png Loading commit data...
noderdp.php3 Loading commit data...
nodessh.php3 Loading commit data...
nodetipacl.php3 Loading commit data...
nodeutilization.php Loading commit data...
nologins.php3 Loading commit data...
nscheck.php3 Loading commit data...
nscheck_form.php3 Loading commit data...
obstacle_list.php3 Loading commit data...
oncrpc.jar Loading commit data...
onload.js Loading commit data...
osconfig_dump.php Loading commit data...
osiddefs.php3 Loading commit data...
osinfo_defs.php Loading commit data...
overlay.elabinelab.gif Loading commit data...
overlay.emulab.net.gif Loading commit data...
overlay.gif Loading commit data...
overlay.mac.emulab.net.gif Loading commit data...
overlay.mini.emulab.net.gif Loading commit data...
overlay.netlab.cc.gatech.edu.gif Loading commit data...
overlay.stoller.emulab.net.gif Loading commit data...
overlay.uky.emulab.net.gif Loading commit data...
override-phantomnet.css Loading commit data...
pagenotworking.php Loading commit data...
panicbutton.gif Loading commit data...
panicbutton.php3 Loading commit data...
password.php3 Loading commit data...
peer_login.php Loading commit data...
people.php3 Loading commit data...
pgeninews.php Loading commit data...
pnet-background-short.png Loading commit data...
pnet-background.png Loading commit data...
pnet-index.php Loading commit data...
pnet-login.png Loading commit data...
pnet-rqaccount.png Loading commit data...
powertime.php3 Loading commit data...
prereserve_node.php3 Loading commit data...
profile.php Loading commit data...
project_defs.php Loading commit data...
projectlist.php3 Loading commit data...
protogeni.png Loading commit data...
pub_defs.php Loading commit data...
pubs.html Loading commit data...
pubs.php3 Loading commit data...
rdp-mime-winxp.c Loading commit data...
rdp-mime-winxp.exe Loading commit data...
rdp-mime-winxp.pl Loading commit data...
rdp-mime.pl Loading commit data...
rdp.gif Loading commit data...
record_revise.php Loading commit data...
redball.gif Loading commit data...
redball.png Loading commit data...
remapexp.php3 Loading commit data...
replayexp.php3 Loading commit data...
reportnode.php3 Loading commit data...
reqaccount.php3 Loading commit data...
reqpnetaccount.php Loading commit data...
request_idleinfo.php3 Loading commit data...
request_swapexp.php3 Loading commit data...
requestaccount.gif Loading commit data...
resendapproval.php Loading commit data...
resendkey.php3 Loading commit data...
robots.txt Loading commit data...
rss.png Loading commit data...
rusage_viz.css Loading commit data...
rusage_viz.php Loading commit data...
sc2002tut.php3 Loading commit data...
search.php3 Loading commit data...
search_cse.php Loading commit data...
sendemail.php3 Loading commit data...
sendtestmsg.php3 Loading commit data...
servicepipe.php3 Loading commit data...
session.php Loading commit data...
setnodeloc.php3 Loading commit data...
show-dataset.php Loading commit data...
showconlog.php3 Loading commit data...
showevents.php Loading commit data...
showexp.js Loading commit data...
showexp.php3 Loading commit data...
showexp_list.php3 Loading commit data...
showexpstats.php3 Loading commit data...
showgroup.php3 Loading commit data...
showimageid.php3 Loading commit data...
showimageid_list.php3 Loading commit data...
showlasterror.php3 Loading commit data...
showlogfile.php3 Loading commit data...
showlogfile_sup.php3 Loading commit data...
showmmlists.php3 Loading commit data...
shownode.php3 Loading commit data...
shownodehistory.php3 Loading commit data...
shownodelog.php3 Loading commit data...
shownodetype.php3 Loading commit data...
shownodetype_list.php Loading commit data...
shownsfile.php3 Loading commit data...
showobstacle.php3 Loading commit data...
showosid_list.php3 Loading commit data...
showosinfo.php3 Loading commit data...
showpolicies.php3 Loading commit data...
showpool.php Loading commit data...
showproject.php3 Loading commit data...
showproject_list.php3 Loading commit data...
showsite_list.php Loading commit data...
showslice.php Loading commit data...
showslicelogs.php Loading commit data...
showslicepub.php Loading commit data...
showstats.php3 Loading commit data...
showstuff.php3 Loading commit data...
showsumstats.php3 Loading commit data...
showthumb.php3 Loading commit data...
showuser.php3 Loading commit data...
showuser_list.php3 Loading commit data...
showvlantaghistory.php Loading commit data...
simple-emulab-logo.svg Loading commit data...
sitecheckin.php Loading commit data...
software.php3 Loading commit data...
sorttable.js Loading commit data...
spewconlog.php3 Loading commit data...
spewevents.php Loading commit data...
spewimage.php Loading commit data...
spewlogfile.php3 Loading commit data...
spewrpmtar.php3 Loading commit data...
spitnsdata.php3 Loading commit data...
spitreport.php Loading commit data...
sponsors.html Loading commit data...
ssh-keys.php Loading commit data...
ssh-mime-windows.pl Loading commit data...
ssh-mime-winxp.c Loading commit data...
ssh-mime-winxp.exe Loading commit data...
ssh-mime-winxp.pl Loading commit data...
ssh-mime.c Loading commit data...
ssh-mime.pl Loading commit data...
ssh.gif Loading commit data...
start.php3 Loading commit data...
statechange.php Loading commit data...
style-doc.css Loading commit data...
style-new.css Loading commit data...
style-nonmain.css Loading commit data...
style.css Loading commit data...
submitpub.php Loading commit data...
suuser.php Loading commit data...
swapexp.php3 Loading commit data...
swish.conf.in Loading commit data...
table_defs.php Loading commit data...
tbauth.php3 Loading commit data...
tbstyle-plain.css Loading commit data...
telemetry.php3 Loading commit data...
template_analyze.php Loading commit data...
template_commit.php Loading commit data...
template_create.php Loading commit data...
template_defs.php Loading commit data...
template_editevents.php Loading commit data...
template_export.php Loading commit data...
template_exprun.php Loading commit data...
template_graph.php Loading commit data...
template_history.php Loading commit data...
template_metadata.php Loading commit data...
template_modify.php Loading commit data...
template_search.php Loading commit data...
template_show.php Loading commit data...
template_sup.js Loading commit data...
template_swapin.php Loading commit data...
textbox.js Loading commit data...
thinlet.jar Loading commit data...
thumbnail-siteloc.png Loading commit data...
thumbnail-usage.png Loading commit data...
thumbnail-usermap.png Loading commit data...
toggle.php Loading commit data...
top2image.php3 Loading commit data...
transparent.gif Loading commit data...
trash.jpg Loading commit data...
unregister_globals.php Loading commit data...
updateaccounts.php3 Loading commit data...
updown.php3 Loading commit data...
url_defs.php Loading commit data...
user_defs.php Loading commit data...
verifyusr.php3 Loading commit data...
verifyusr_form.php3 Loading commit data...
wallpaper0.jpg Loading commit data...
wallpaper1.png Loading commit data...
wanodecheckin.php Loading commit data...
webcam.php3 Loading commit data...
webcamimg.php3 Loading commit data...
websearch.in Loading commit data...
webtask.php Loading commit data...
widearea_info.php3 Loading commit data...
widearea_nodeinfo.php3 Loading commit data...
widearea_redirect.php Loading commit data...
wideareakeys.php3 Loading commit data...
wikiregister.php3 Loading commit data...
xmlrpc.php3.in Loading commit data...
xmlrpcpipe.php3.in Loading commit data...