Parameterize hadoop wfas; change master flow setup; check master flows.
First, we add key-val parameters. broker_name=foo sets the broker name to foo for registration and lookup (defaults to 'hadoop'). runhadoop=False (or other False equivalents) will force the wfas to skip Hadoop config and run. We also separate out master<->slave flow setup and put it at the beginning of the allpairs flow mesh setup; this was for debugging. Finally, we also try and retry a simple parallel ssh task from the Hadoop master to all slaves, to "verify" and "wait" for them to all be up and reachable. This helps us catch problems like when a VM can't fully boot because it failed to get its metadata, or whatever.
Showing with 154 additions and 61 deletions