GitLab

(This var just had to be defined for the blockstore case.)

This was to try to solve the performance problem; no luck here.

Memcache + Keystone + WSGI/Apache seems to cause a problem where
Keystone is effectively unavailable (internal errors) for about a
minute... then it comes back by itself.

So we disable it by default.  The docs default to using it, but this is
far from the first time the doc defaults trigger bugs or are simply bad
configuration!

Add Liberty support.

Add keystone v3 support.  Now you can choose which version of keystone
to run... all combinations tested exception Juno with v3.

Make node type and link speed configurable.

Make token and session timeouts much longer by default (so people don't
get logged out so quickly), but also configurable.

Keystone is now served by WSGI through Apache on Kilo and Liberty.

Memcached keystone token caching is disabled for now; it causes
intermittent problems; so using SQL for now.

Add localhost to /etc/hosts file.  This doesn't cause problems anymore,
if it ever did.

We now use the `openstack' CLI command for >= Kilo, instead of the
per-service client CLI tools.

Stick with ovs agent even in Liberty -- even though the default is now
linuxbridge, it seems.

In general, get rid of nearly all the rest of the cat <<EOF ... EOF
stuff and replace it with crudini --set/--del.  A touch slower, but much
cleaner.

Also in general, improve the Kilo support so that it more closely
matches the docs.

If you instantiate a portal expt on Emulab (where you might have a real
account), the swapper is you, not geniuser.  So, check geniuser via
geni-get slice_urn success/failure.

Also, adds a geni-lib script that generates an rspec instead of printing
it (although print still works at portal) and generates input for
CM::AddNodes() when requested.  This generator is stateful; it tries
to avoid generating new nodes with previously-used IPs or client_ids;
thus it is a separate object.  It is designed so that it can be imported
into a script, and the importing script can look for special
DYNSLICE_GENERATOR variables to use its rspec foo to create a slice and
add nodes in some semantic way.

We want to use only local AM tmcd info in this case...

(Of course this will suffer from the problem of the limited tmcd buffer
for getmanifest.)

Quit trying to apt-get packages if they're installed, unless the
user selects the new DO_APT_UPGRADE option.  Always install was nice
in the beginning, but it is no longer the best use case, and it can
cause uncertainty when failures happen (i.e., if new versions of
packages get installed that the scripts can't handle).  So now there
are three apt options in the scripts and in the geni-lib script:

DO_APT_UPDATE -- updates the apt cache (often hard to do pkg
  install/upgrade if the cache is out of date); defaults to 1
DO_APT_INSTALL -- if this is set 0, we don't install *anything*
  other than critical deps (think python-m2crypto); defaults to 1
DO_APT_UPGRADE -- if this is set 1, we always run apt-get install
  to either install and/or upgrade OpenStack packages and deps.
  The big change is that this now defaults to 0 -- so packages are
  not upgraded from their current versions if they exist.

There's a fallback to this too -- if for whatever reason, we don't get
a password, we generate a random one and email it to the user.  Not
perfect, but still better than letting a null passwd through.

So far only have to handle utopic.

This allows us to operate on images with the openstack packages
installed, but services disabled to speed up boot.

Can't suck up all the disk with these...

(Also, had never committed the manifest getter support programs.)

Openstack doesn't yet have 1) serial console client builtin to
dashboard, nor 2) serial console log + r/w serial console access.

So right now, you have to choose if you want to enable r/w console
access via CLI client, or if you want to be able to view serial logs in
the dashboard web UI; default is the latter (logs).

If you enable r/w consoles, we download a simple websocket console
client, setup a little frontend script, so users can type

  $ /root/setup/novaconsole.sh <instancename>

and get to the console.  Escape is ~.  In addition to the console CLI
tool, we have to grab the latest version of the python websocket
library, cause the one in Ubuntu 15.04 is horribly out of date and
doesn't seem to support binary/base64 websockets, which the console CLI
tool requires (as does the server).

For our flat lan case, we run lans directly over the top of real (or
emulab vlan) networks.  So we extend those emulab IP networks into
the openstack flat network associated with each emulab network.  Well,
openstack assumes it controls the subnet and can just allocate special
addresses anywhere -- like for the local dhcp agent IP, or a router
interface -- it doesn't respect the allocation_pool value we give the
network (so that must just be for compute nodes).

Further, there's no way to set these special IPs when creating networks,
subnets, and routers, so we retroactively find those ports and change
their IP addrs to something that won't stomp on emulab IPs, nor on the
openstack allocation_pool.

This seems to work great, except that packets outbound from an openstack
instance don't get SNAT'd so they appear they came from the real
external world address.  Inbound packets make it all the way, so at
least DNAT is working.

Setup several kinds of networks: tunnels, flat networks, flat networks
multiplexed via vlans over physical networks (where openstack doesn't
manage the vlan ids), and real vlan networks (where openstack *does*
manage the vlan ids).  Tunnels always go over the first flat data net.

Be very flexible in terms of assigning IPs; generate them ourselves
if they dind't come to us, or if user wants to use our generated ones.
I tried to be smart (enough) with this.

Setup VNC-based consoles on x86-64; working in dashboard.

Don't put plaintext admin password in profile anymore; instead, expect
a hash of the admin password.  Replace the temp admin password in the
keystone database with the hash we get.  But, since the CLI tools
all require real user auth, setup a secondary 'adminapi' account
that is a real admin, and use that to see admin-openrc.sh for CLI
tools, and for all our configuration, and places where the services
use a real admin account to auth.  Also, push the admin password
hash all the way into our instance images.

Also add some params, like setting the admin password and opting
not to install or update any of the installed openstack packages...

Have to notice that the experiment lans are atop vlans so we can use
the right regexes to collect info, and so we can save their
configuration appropriately in our static rewrite of
/etc/network/interfaces .

We now have a prebuilt tarball that we download from boss, that
has sshd, cloud-guest-utils, etc, preinstalled, to save time.  Use
it by default.

Also, create (at least) two images by default: trusty-server and
trusty-server-multi-nic (which has support for up to 8 virtual NICs).
Turns out Ubuntu distros have horrible support for "variable" NICs;
if the NIC is listed in /etc/network/interfaces, the setup scripts
will refuse to generate the right upstart event, which hangs the
cloud-guest-utils init script for a long, long time.  So I've hacked
a little service together that notes nonexistent NICs in
/etc/network/interfaces and fires a fake complete event at the right
time.  I didn't move this hack into the default image because I don't
trust it enough.

Also, create trusty-server first, so the first image is sane.

And also, don't add extra network public gateways via
ext-net if there aren't enough public IPs.  We need at
least 2 so that there is 1 floater and 1 for the private-to-public
router interface :(