GitLab

support the genidesktop.

From: Leigh Stoller <lbstoller@gmail.com>
Date: Wed, 22 May 2013 13:49:33 -0700
Cc: instageni-design@geni.net

So far we have been pretty loose about checking to make sure the
certificate chains obey the Geni rules. These rules include checking to
make sure that only approved entities can sign particular kinds of
credentials. For example; only something known to be a Slice Authority
should be allowed to create a slice and return a slice credential.

The other check we have been lax about, is verifying that the URN namespace
is consistent along the chain from CA to the target. For example, a chain
that starts in Utah:

	URI:urn:publicid:IDN+emulab.net+authority+root

should not be able to sign anything outside its namespace. That is, Utah
should not be able to sign a user or slice credential like:

	urn:publicid:IDN+panther+user+shufeng

This is made more complicated when we introduce subsa certs along the way,
where Utah signs its SA cert and that signs a project slice. In this case
the chain would look something like:

	URI:urn:publicid:IDN+emulab.net+authority+root
	URI:urn:publicid:IDN+emulab.net+authority+sa
        URI:urn:publicid:IDN+emulab.net:testbed+authority+sa
        URI:urn:publicid:IDN+emulab.net:testbed+slice+myslice

There are also scoping rules; A subsa like:

        URI:urn:publicid:IDN+emulab.net:testbed+authority+sa

should not be able to sign:

        URI:urn:publicid:IDN+emulab.net:someotherproject+slice+myslice

The entire cert chain is require to verify this. The CA roots are in the
bundle, and the intermediate certs should be enclosed in the signature
section of the XML document.

We have to make the same check against the user certificate after apache
verifies the chain. For apache (or any SSL server) you have to load the
chain, and as I mentioned in earlier email, this is easy with perl and
python based clients.

With all that said, we do not plan to start rigorous enforcement of the
first check above, and for the second class of checks, we just want to
enforce a simple prefix check until we get our subsa house in order (since
we don't even conform properly yet!).

In GetCredential() and RenewSlice() now, 'cos I forgot them the first time.

This commit is intended to makes the license status of Emulab and
ProtoGENI source files more clear.  It replaces license symbols like
"EMULAB-COPYRIGHT" and "GENIPUBLIC-COPYRIGHT" with {{{ }}}-delimited
blocks that contain actual license statements.

This change was driven by the fact that today, most people acquire and
track Emulab and ProtoGENI sources via git.

Before the Emulab source code was kept in git, the Flux Research Group
at the University of Utah would roll distributions by making tar
files.  As part of that process, the Flux Group would replace the
license symbols in the source files with actual license statements.

When the Flux Group moved to git, people outside of the group started
to see the source files with the "unexpanded" symbols.  This meant
that people acquired source files without actual license statements in
them.  All the relevant files had Utah *copyright* statements in them,
but without the expanded *license* statements, the licensing status of
the source files was unclear.

This commit is intended to clear up that confusion.

Most Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the Affero GNU General Public License, version 3
(AGPLv3).

Most Utah-copyrighted files related to ProtoGENI are distributed under
the terms of the GENI Public License, which is a BSD-like open-source
license.

Some Utah-copyrighted files in the Emulab source tree are distributed
under the terms of the GNU Lesser General Public License, version 2.1
(LGPL).

This is necessary to comply with GENI AM API v3.

Move the expiration test into verifygenicred. Change the invocation to
capture the output so that we can say something useful in the error
response, instead of what we do now which is just tell the user there
is an error.

projects to which they belong.

Generate authority certificates for local sub-authorities (i.e. authorities
corresponding to a local project) on demand.

Map per-project URLs to the same XMLRPC server handled in the context of
the authority for the specified project.

Make the SA give out per-project credentials when it's asked for a
GetCredential in a sub-authority.

something.

CM) and max_slice_lifetime (for the SA) so that slice credentials and
slivers can have max different expirations. This is so a CM can can
limit easily limit slivers so they do not hang out for months doing
nothing.

CreateSliver(), to handle multiple accounts.  This somewhat reflects
the Geni AM API for keys, which allows the client to specify multiple
users, each with a set of ssh keys.

The keys argument to the CM now looks like the following (note that
the old format is still accepted and will be for a while).

[{'urn'   => 'urn:blabla'
  'login' => 'dopey',
  'keys'  => [ list of keys like before ]},
 {'login' => "leebee",
  'keys'  => [ list of keys ... ]}];

Key Points:

1. You can supply a urn or a login or both. Typically, it is going to
   be the result of getkeys() at the PG SA, and so it will include
   both.

2. If a login is provided, use that. Otherwise use the id from the urn.

3. No matter what, verify that the token is valid for Emulab an uid
   (standard 8 char unix login that is good on just about any unix
   variant), and transform it if not.

4. For now, getkeys() at the SA will continue to return the old format
   (unless you supply version=2 argument) since we do not want to
   default to a keylist that most CMs will barf on.

5. I have modified the AM code to transform the Geni AM version of the
   "users" argument into the above structure. Bottom line here, is
   that users of the AM interface will not actually need to do
   anything, although now multiple users are actually supported
   instead of ignored.

Still to be done are the changes to the login services structure in
the manifest. We have yet to settle on what these changes will look
like, but since people generally supply valid login ids, you probably
will not need this, since no transformation will take place.

Lbs

to the GeniRegistry file; move the slice part back into GeniSlice
where it belongs.

that offline a couple of minutes later. Reduces the load on the CH
when doing parallel registratons.

Add caching of user and slice credentials, to avoid regenerating the
same credentials over and over for the test scripts. Stored in the
geni_credentials table, they are checked for expiration before handing
them out. Also check to make sure that the user certificate has not
changed, and regen/cache if it has.

starting any one of our scripts can take a second or two. That time is
spent including and compiling 10000s of thousands of lines of perl
code, both from our libraries and from the perl libraries.

Mostly this is just a maintenance thing; we just never thought about
it much and we have a lot more code these days.

So I have done two things.

1) I have used SelfLoader() on some of our biggest perl modules.
   SelfLoader delays compilation until code is used. This is not as
   good as AutoLoader() though, and so I did it with just a few 
   modules (the biggest ones).

2) Mostly I reorganized things:

  a) Split libdb into an EmulabConstants module and all the rest of
     the code, which is slowly getting phased out.

  b) Move little things around to avoid including libdb or Experiment
     (the biggest files).

  c) Change "use foo" in many places to a "require foo" in the
     function that actually uses that module. This was really a big
     win cause we have dozens of cases where we would include a
     module, but use it in only one place and typically not all.

Most things are now starting up in 1/3 the time. I am hoping this will
help to reduce the load spiking we see on boss, and also help with the
upcoming Geni tutorial (which kill boss last time).

(this time for the urn). Also change the protoype to allow an error to
be returned.

internally, as the primary key in the tables, but the CM/SA APIs no
longer use them. The CH still accepts them for now. We can probably
stop putting them into manifests and advertisements at this point as
well. 

For slivers, stop using the uuid of the node as the uuid of the sliver
itself; generate a new one. As above, this is cause the uuid is the
primary key in the table, but the URN is what we use for lookups,
etc.

backwards compatible with old SAs and CMs until new code makes it
out to everyone. So the CM now does a version check at the target SA,
and if an old version 1, use the bogus self signed cred. If the SA is
version 1.01, send a proper sliver credential.

In the SA, accept older bogus credential for now, but start accepting
the new sliver credential, and apply more stringent checks.

failure; leave it for offline by the sa_daemon.

Reregister slices when they are renewed so that the clearing house has
the updated expiration time.

from slicename(), which breaks on names with a dash in them.

to all of the API functions, and prefer that to any UUID argument.
There are a lot of little changes.

At this point, the CH and SA will no longer accept certificates that
do not have URNs in them. The CH will send email to the email address
listed in the certificate.

It is now added as a fix.

The component managers are those which have reported having a sliver to the SA.