SOL console issues at Clemson
We have experienced a variety of problems with IPMI SOL-based consoles and capture.
Most recently, we found that the new c6320 Clemson nodes tend to get stuck coming out of PXEWAIT. They just stop sending console output at some point and you don't know what is going on til you type something at the console. We have observed a couple of behaviors.
One is that when you type something you get back "|||p|: command not found" (or something like that--lots of pipes and occasional other chars) indicating that the node has fallen into interactive mode for pxeboot. These nodes are running the most recent pxeboot which wants you to specifically type ^C (not just any char) to get into interactive mode. Nonetheless, they seem to wind up there and since they haven't been reporting output, we don't know exactly how they got there.
Another is where the console just stops logging at some point and becomes totally unresponsive. We notice these when a node fails to come out of PXEWAIT after being sent a wakeup and we cannot figure out why because the console it completely unresponsive.