Skip to content
  • Leigh B. Stoller's avatar
    The panic button ... · 87dd2e60
    Leigh B. Stoller authored
    * tbsetup/ New backend script to implement the panic button
      feature. When used, it will cut the severe the connection to the
      firewall node by using snmpit to disable the port. Sets the panic
      bit (and date) in the experiments table, and changes the state of
      the experiment from "active" to "paniced" to ensure that the
      experiment cannot be messed with (swapped out or modified). Sends
      email to tbops when the panic button is pressed.
      Used with -r option, reverses the above. State is set back to
      active, the panic bit is cleared, and the port is renabled with
    * tbsetup/ During swapout, a firewalled experiment that has
      been paniced will get a cleaning; The nodes are powered off, then
      the osids for all the nodes are reset (with os_select) so that they
      will boot the MFS, and then the nodes are powered on. Then the
      control network is turned back on, and then I wait for the nodes to
      reboot (this is simply cause we do not record in the DB that a node
      is turned off, and if I do not wait, the reload daemon will end
      hitting the power button again if they do not reboot in time. We can
      fix this later.
      I am not planning to apply this to general firewalled experiments
      yet as the power cycling is going to be hard on the nodes, so would
      rather that we at least have a 1/2 baked plan before we do that.
    * www/showexp.php3: If experiment is firewalled, show the Panic
      Button, linked to the panic button web script. If the experiment has
      already had the panic button pressed, show a big warning message and
      explain that user must talk to tbops to swap the experiment out.
      Also fiddle with menu options so that the terminate link is gone,
      and the swap link is visible only in admin mode. In other words, only
      an admin person can swap an experiment once it is paniced. And of
      course, an admin person can the backend panic script above with the
      -r option, but thats not something to be done lightly.
    * db/ Add "paniced" as an experiment state (EXPTSTATE_PANICED).
      Add utility functions: TBExptSetPanicBit(), TBExptGetPanicBit(), and
    * tbsetup/ Minor state fiddling so that an experiment can
      be swapped while in paniced state, but only when in admin mode. Also
      clear the panic bit when experiment is swapped out.
    * www/ Add "paniced" as an experiment state. Add a
      utility function TBExptFirewall() to see if experiment is firewalled.
    * www/panicbutton.php3: New web script to invoke the backend panic
      script mentioned above, after the usual confirm song and dance.
    * www/panicbutton.gif: New gif of a red panic button that I stole off
      the net. If anyone has sees/has a better one, feel free to replace
      this one.
    * utils/ Add -s option so that I can pass in the
      state I want to wait for (used from tbswap above to wait for nodes
      to reach ISUP after power on).