Drop 99-emulab-networkd.rules into /etc/udev/rules.d,
emulab-networkd@.service into /lib/systemd/system, and do
`systemctl enable systemd-networkd.service`.  I would also do
`ln -sf /run/systemd/resolve/resolv.conf /etc/resolv.conf`; we don't
want any part of systemd-resolved.

This is the only way I could come up with to make systemd-networkd
listen on all wired ethernet ifaces, and get it into the right state.
systemd-networkd is like NetworkManager in some ways.  I don't know if
it's better or worse.  The main problem for us is that it doesn't
support hooks, and that it runs its own dhcp client (which is crappy and
inflexible compared to dhclient).  We don't need anything super flexible
though since we don't use custom options, and it has an option to ignore
the lease lifetime and leave the iface up even if expiry happens before
renewal can take place.

Anyway, here's the flow: udev rules tell systemd to require a unit
(emulab-networkd@$iface.service) for each physical wired ethernet
device.  These files write runtime .network files into
/run/systemd/network, which are read by systemd-networkd.service if it
is enabled.  We just tell systemd-networkd to listen on them all, then
we use its waiter script.  However, its wait script is broken and/or the
documentation is contradictory, so we don't trust it to tell us which
interface comes up; we invoke its networkctl client to tell us.  We then
write the up iface to /run/emulab/cnet, rm -f the .network files for all
ifaces other than the control net device (cause we don't want
systemd-networkd to manage expt network devices), and restart
systemd-networkd so it can appropriately report that it is "managing"
the control net device, but not the expt devices.  It is very
unfortunate that we have to restart systemd-networkd, but there is no
other way to tell it about new configuration.  If you HUP it, it dies.
These systemd folks must live in a box where all use cases look alike.

Note that if the user overrides with their own .network scripts in
/etc/systemd/network that Match any of the physical NICs, those take
precedence over our per-interface scripts, which is what we want.

Note also that to do something like move the real control net into a
bridge, you can just remove or update the .network file in
/run/systemd/network, and restart systemd-networkd.  It should re-dhcp
on the bridge.  It does not remove NIC configuration at exit; it just
"catches up" with any NIC address when it is restarted.

I have not done all the dhclient-exit-hook normal goo (i.e. writing the
control net controlif/IP into the right place, etc), but that should all
be possible.

Finally: I haven't yet decided that this is the path for Ubuntu 18.  We
might well ignore this and fall back to the old /etc/network/interfaces
ifupdown strategy.  At least there it is possible to use a real
dhclient.