Make benchmark scheduler into a "production" service
Prerequisite Tasks:
Server-level tasks:
-
Combine the parts from two PG nodes into one to make a machine for benchmark scheduling/CONFIRM. -
Install base Ubuntu 18.04 image on the machine. -
Install database (mariadb). -
Make non-root user on DB to handle benchmark results inserts. -
Install Python2 and requisite libraries for orchestration. -
Create a user account to be used for benchmark orchestration. -
Make c6420
servers able to set CPU frequency scaling.
Flux-level tasks:
-
Move code to the emulab group on gitlab. -
Create a service account for geni-lib.
Benchmark-level tasks:
(The following will be done on the currently non-live code in the emulab repo!)
-
Add source for NIC -> NIC latency benchmarks to benchmark repo. -
Add invocation of, and results parsing for, NIC -> NIC latency benchmark to benchmark script (Utah only). -
Check if APT can run NIC -> NIC latency benchmark. If so, add invocation. -
Add source for NASA NPB CPU benchmarks to benchmark repo. -
Add invocation of, and results parsing for, NASA NPB CPU benchmarks to benchmark script. -
Disable DVFS section for c6420
machines temporarily -
Fix benchmark script to support Ubuntu 18.04 output for ifconfig
(or switch toip link
/ip addr
). -
Pick an r320
machine to be the network destination node for APT. -
Configure the r320
machine to be the network destination node for APT. -
Pick a d430
machine to be the network destination node for Emulab. -
Configure the d430
machine to be the network destination node for Emulab. -
Start dumping OS version (16.04/18.04) in env_info. -
Start dumping CPU model in env_info. -
Do a final pass to make sure we don't need to add any new fields to the DB. -
Tune iperf3 params for xl170
s (25 Gbps links)
Orchestration-level tasks:
(The following will be done on the currently non-live code in the emulab repo!)
-
Add support for emulab/APT in orchestration. -
Add support for CPU benchmarks in results processing. -
Add support for NIC->NIC latency benchmark in results processing. -
Make created experiments use Ubuntu 18.04 instead of 16.04. -
Update the db schema to support the CPU benchmarks. -
Update the db schema to support the NIC->NIC latency benchmark. -
Update the db schema to support OS version. -
Update the db schema to support CPU model. -
Update DB access sections to use non-root DB user. -
Make the code more environment agnostic (move hard-coded vars to cmdline args). -
Rewrite arg parser to use argparse instead of getopt. -
Properly handle thrown exceptions due to resource reservations. -
Make a pass through the code to check refactor potential.
Verification tasks:
Benchmark-level tasks:
-
Test the benchmark script on all Cloudlab Phase II hardware types. -
Test the benchmark script on all APT hardware types. -
Test the benchmark script on all Emulab hardware types.
Orchestration-level tasks:
-
Test the orchestration script on Cloudlab Utah. -
Test the orchestration script on Cloudlab Wisconsin. -
Test the orchestration script on Cloudlab Clemson. -
Test the orchestration script on APT. -
Test the orchestration script on Emulab.
Migration tasks:
-
Stop collection cronjobs on ms1102
. -
Pick an xl170
machine to be the new network destination node for CloudLab Utah. -
Pick a c220g5
machine to be the new network destination node for CloudLab Wisconsin. -
Pick a c6420
machine to be the new network destination node for CloudLab Clemson. -
Import old DB data into new database. -
Update NULL fields as needed for old data. -
Set up crontab to run scheduled benchmark orchestration (with proper arguments). -
Find a better system for DB backups. -
Set up cronlog rollover using savelog.
Edited by Aleksander Maricq