GitLab

of the aggregate_history table.

"testing" variable, which of course, makes things break in totally
obscure ways.

(orphaned) by the protogeni code.

to supply the fully specified rspec to both CMs, my changes permit you
to specify only the component_manager_urn for the remote side.

When you start the sliver, the CMs contact each other to resolve the
slice (get the manifest), find the actual node in the manifest, and
resolve that node to get the control network.

This even allows for fully unbound rspecs to be supplied, as long as
the component_manager_urn is specified.

although the default is currently to do cooked mode in "basic" mode,
rather then "full" mode.

(this time for the urn). Also change the protoype to allow an error to
be returned.

internally, as the primary key in the tables, but the CM/SA APIs no
longer use them. The CH still accepts them for now. We can probably
stop putting them into manifests and advertisements at this point as
well. 

For slivers, stop using the uuid of the node as the uuid of the sliver
itself; generate a new one. As above, this is cause the uuid is the
primary key in the table, but the URN is what we use for lookups,
etc.

StartSliver() to try to power on nodes instead of rebooting them.
This is why we need to actually track the power on/off state of
nodes.

node to see if the desired os is loaded, and if not initate an async
osload. This code mostly mirrors code in os_setup, but has been
reorganized to be more concise and easier to understand.

down mode) is powered back on when starting or
deleting the sliver.

in the desired state. It is an aggregate after all, and things could
be "mixed".

Fix XML. Don't try to canonicalize to print. That has too much unwanted baggage. Create a copy of the node instead which will automatically add namespace declarations to it.

Print canonicalized strings as output. If you don't the namespaces->prefix bindings won't always be printed if you print out a fragment.

since it has to be aware of the CM version. Add a Version() call to
GeniAuthority with goes asks the CM what version it is exporting.
Based on that, we know how to do a resolve of a component. Refactored
the code that was used in GeniAggregate when creating tunnels, since
that is where we have to Resolve components. This also turns up in
cooked mode.

Continuine moving towards a urn-only world. If a GeniAuthority or a
GeniComponent does not have the URN set locally in the DB, go back to
the clearinghouse and get it. Error if it is not known, and go bang on
the remote site to update and rerun register_resources.

Checkpoint. All the conversion is done, but I need to refactor the xpaths to account for brittleness in the xpath specification.

1. You cannot unregister a slice at the SA before it has expired. This
   will be annoying at times, but the alphanumeric namespace for slice
   ames is probably big enough for us.

2. To renew a slice, the easiest approach is to call the Renew method
   at the SA, get a new credential for the slice, and then pass that
   to renew on the CMs where you have slivers.

The changes address the problem of slice expiration.  Before this
change, when registering a slice at the Slice Authority, there was no
way to give it an expiration time. The SA just assigns a default
(currently one hour). Then when asking for a ticket at a CM, you can
specify a "valid_until" field in the rspec, which becomes the sliver
expiration time at that CM. You can later (before it expires) "renew"
the sliver, extending the time. Both the sliver and the slice will
expire from the CM at that time.

Further complicating things is that credentials also have an
expiration time in them so that credentials are not valid forever. A
slice credential picks up the expiration time that the SA assigned to
the slice (mentioned in the first paragraph).

A problem is that this arrangement allows you to extend the expiration
of a sliver past the expiration of the slice that is recorded at the
SA. This makes it impossible to expire slice records at the SA since
if we did, and there were outstanding slivers, you could get into a
situation where you would have no ability to access those slivers. (an
admin person can always kill off the sliver).

Remember, the SA cannot know for sure if there are any slivers out
there, especially if they can exist past the expiration of the slice.

The solution:

* Provide a Renew call at the SA to update the slice expiration time.
  Also allow for an expiration time in the Register() call.

  The SA will need to abide by these three rules:
  1. Never issue slice credentials which expire later than the
     corresponding slice
  2. Never allow the slice expiration time to be moved earlier
  3. Never deregister slices before they expire [*].

* Change the CM to not set the expiration of a sliver past the
  expiration of the slice credential; the credential expiration is an
  upper bound on the valid_until field of the rspec. Instead, one must
  first extend the slice at the SA, get a new slice credential, and
  use that to extend the sliver at the CM.

* For consistency with the SA, the CM API will changed so that
  RenewSliver() becomes RenewSlice(), and it will require the
  slice credential.

* More URN issues dealt with.

* Sliver registration and unregistraton (CM to SA).

* More V2 status stuff.

* Other fixes.

the tables never conflict on the index.

communication with the clearinghouse, and modifications to the
certificate format (put the URN in subjectAltName and move any
URL to a private OID under subjectInfoAccess).

replace the "status" field, but this will allow me to work in parallel
on version two of the API.

Minor bug fixes.

Also add a GetManifest method to avoid code duplication.

an optional manifest argument. Only processing the startup command
right now, don't want to put much effort into this until we work out
how the manifest is going to be used and processed in the V2 API.

all the vnodes and reboots at once, since those scripts parallelize
thier operation.

physical nodes that are not shared (permentantly allocated).

a missing url in the certificate for component (node). Why was that?
Well, when I create a sliver, I use the same uuid as the node, and
then later when I need to find the node for that sliver, I look for it
in the nodes table using the that uuid. This was bad cause for each
sliver I create a new certificate pair, and thus a new uuid. This
overwrites the original certificate bound to that node, only the new
certificate is not created with a URL. This is bad all around, but
with uuids being replaced by URNs and so close to the demo, I am not
going to fix this properly, but rather just avoid the problem by
reusing the existing certificate for the node when creating a sliver.
Revisit later this week.

1. Allow use of the shared nodes via the "exclusive" tag in the rspec.

2. Switch to using the mapper in GetTicket() and in RedeemTicket() as
   per this email I sent:

* GetTicket(): New rspec comes in and I build a virtual topology as I
 parse the rspec. Basically, virt_nodes and virt_lans table entries,
 which are stored into the DB. The rspec can include wildcards or
 specific nodes; I use the "fixed" slot of the virt_nodes table. Note
 that I am not yet handling fixed ifaces. Nice thing about this is
 that all of the show exp tools work.

 I run the (new) mapper on it in "solution" mode. This does two
 things; wildcards are mapped, and 2) it verifies that the rspec is
 mappable on the local hardware. Solution mode does not actually
 change the DB, but rather it spits out an XML file that I parse
 (note, we eventually will pass the rspec through, but I am not ready
 for that yet). I then allocate the nodes to the holding area, update
 the rspec, create a ticket, and return it.

* RedeemTicket(): I run the new mapper again, only this time in real
 mode with -update. This is basically a redo of the run above since
 all the nodes are reserved already, but the DB is actually filled
 out this time. I then create the slivers and such.

 The other difference is that instead of creating the vlans by hand,
 I can now run snmpit -t to do the work for me. Ditto for tear down
 with snmpit -r.

 Another bonus is that I can add (missing) IP addressess during the
 initial rspec parse, and the nodes now boot and have their
 interfaces configured. Virtual interfaces too, including the ones
 inside of virtual nodes.

* All of the above works with shared nodes too:

"<rspec xmlns=\"http://protogeni.net/resources/rspec/0.1\"> " +\
" <node virtual_id=\"geni1\" "+\
"       virtualization_type=\"emulab-vnode\" " +\
"       virtualization_subtype=\"emulab-openvz\" " +\
"       exclusive=\"0\"> " +\
"   <interface virtual_id=\"virt0\"/> " +\
" </node>" +\
" <node virtual_id=\"geni2\" "+\
"       virtualization_type=\"emulab-vnode\" " +\
"       virtualization_subtype=\"emulab-openvz\" " +\
"       exclusive=\"0\"> " +\
"   <interface virtual_id=\"virt0\"/> " +\
" </node>" +\
" <link virtual_id=\"link0\"> " +\
"  <interface_ref " +\
"            virtual_interface_id=\"virt0\" " +\
"            virtual_node_id=\"geni1\" " +\
"            /> " +\
"  <interface_ref " +\
"            virtual_interface_id=\"virt0\" " +\
"            virtual_node_id=\"geni2\" " +\
"            /> " +\
" </link> " +\
"</rspec>"

The shared nodes boot, and you can ping on the experimental networks.

Caveats:

* UpdateTicket and UpdateSliver need work as per the mail I sent the
 other day about the state of the sliver between the ticket and the
 sliver operations.

* Collocation specifications are ignored since we do not have any way
 to specify this to assign when wildcards are used. Rob, I am
 wondering if assign has any tricks we can take advantage of.

* Still need to commit all the snmpit changes and get that hooked into
 the CM.

https://www.protogeni.net/trac/protogeni/wiki/ComponentManagerAPI

not to come back with the control interface in the interfaces list.