Now I see why I hadn't enabled the

  -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold"

directly on the apt-get command lines.  apt-get must have a bug, because
when you specify this option in noninteractive (and non-pty, I assume,
because this is via startup command-then-ssh), at least one of the dpkg
commands invoked by apt-get has no dpkg action.

So, put these two options into /etc/dpkg/dpkg.cfg/cloudlab, and then
there are no problems.

Of course, this means this same behavior will happen to the users if
they try apt-get or dpkg later on.  This is on the one hand, preferable,
because then they can't possibly screw up openstack config files through
package upgrades.  On the other hand, they might get fooled they're
upgrading some other package.

Probably will just document this and call it good :).