1. 26 Oct, 2016 40 commits
    • Charles Jacobsen's avatar
      build-refactor: Incorporate common sources into kliblcd build. · 528cecbd
      Charles Jacobsen authored
      Build bug fixes for resource tree and module create code.
    • Charles Jacobsen's avatar
      build-refactor: New arch directory, header shuffling. · 1abb5ae7
      Charles Jacobsen authored
      Following the Linux kernel src layout, moved the vt-x code
      into a separate arch folder. Moved headers around:
         -- Microkernel/kliblcd arch-independent headers are in
         -- Microkernel arch-dependent headers are in arch/x86/include
         -- liblcd headers are in include/liblcd
      Switched some file/dir names to use `_` instead of `-`.
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
      build-refactor: Build setup for microkernel and kliblcd. · ab53f6cd
      Charlie Jacobsen authored
      Set up out-of-source build for microkernel and kliblcd. This is
      necessary because the common/ sources will be built for non-isolated
      and isolated worlds (hence we can't do the in-source build - can't
      build same source files with different configs in the same directory).
      Developing a better solution for the liblcd config and hacks
      so we can abstract the build over whether the source file is
      built for non-isolated or isolated. Each environment defines a
      pre- and post- hook header that should be placed appropriately
      in source files for which we want to build for both environments.
      For now, non-isolated pre- and post-hooks are empty since we don't
      need to do any kernel build config hacks.
    • Charlie Jacobsen's avatar
      build-refactor: Kconfig deleted. · 0a075cda
      Charlie Jacobsen authored
      We don't need this anymore since we're not hooked in to the
      kernel's config system.
    • Charles Jacobsen's avatar
      libcap-integration: Microkernel and kliblcd build bug fixes. · c94e9745
      Charles Jacobsen authored
      Microkernel and kliblcd compile. Some linker errors due to
      missing resource tree and create objects. Going to refactor
      build now, more code shuffling fun.
    • Anton Burtsev's avatar
      Support for LCD strack trace · 0be0f634
      Anton Burtsev authored
    • Anton Burtsev's avatar
      Support for stack dump · 0ddd65f0
      Anton Burtsev authored
    • Charlie Jacobsen's avatar
      lcd-create-v2: Re-factor klcd create with modules. · 22c4719d
      Charlie Jacobsen authored
      Not much changed here.
    • Anton Burtsev's avatar
    • Anton Burtsev's avatar
      Support for printing register state · 22c477ab
      Anton Burtsev authored
    • Charlie Jacobsen's avatar
      lcd-create-v2: LCD destroy module lcd no longer needed. · 87838506
      Charlie Jacobsen authored
      This completes the isolated LCD creation logic, v2. Theoretically,
      this is now designed (unlike v1) so that isolated LCDs can
      create other isolated LCDs (once I get the liblcd updates in
    • Charlie Jacobsen's avatar
      lcd-create-v2: First draft of lcd create v2 is done. · a4ecc8ed
      Charlie Jacobsen authored
      And a hell of a lot simpler.
    • Charlie Jacobsen's avatar
      lcd-create-v2: Guest virtual address space set up. · c28ba3f0
      Charlie Jacobsen authored
      Map everything except the ioremap region as write back. ioremap
      region is uncacheable. Only high 512 GBs are mapped to the first
      512 GBs of guest physical.
      No more dynamic build up of the guest virtual address space during
      page walks. That code was just so damn messy.
    • Charlie Jacobsen's avatar
      lcd-create-v2: Physical address space setup (except UTCB). · fc7bf418
      Charlie Jacobsen authored
      Wow, this is so much easier with slightly coarser-grained
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      lcd-create-v2: Update address space layout for v2. · 39dfe81c
      Charlie Jacobsen authored
      The boot physical address space fits in the first 512 GBs.
      The boot virtual address maps this to the high 512 GBs. In this
      way, the kernel module code will be loaded at the address it was
      linked for.
      Address spaces designed for 1 GB huge guest virtual pages. The
      hope is this address will be big enough for a lot of use cases,
      so the code inside the LCD won't ever need to touch the guest
      virtual address space (just physical).
    • Sarah Spall's avatar
      code to allocate and initialize trampoline structures. fixed bug where regular... · 427b971a
      Sarah Spall authored
      code to allocate and initialize trampoline structures. fixed bug where regular rpcs were beginning ignored if declared in an unnamed scope
    • Anton Burtsev's avatar
      Changed regs to be a data structure vs an array · d87f9b5e
      Anton Burtsev authored
      Not tested, can't test because of the PAT problems
    • Anton Burtsev's avatar
    • Charles Jacobsen's avatar
      Fix PAT initialization, shrink ioremap region for now. · 2aedcbde
      Charles Jacobsen authored
      PAT appears to work now with write back memory. Added an
      arch check to ensure PAT is valid before VM entry.
      The ioremap region was too big (the amount of metadata - bitmaps,
      etc. - was too much). I shrunk it for now to 16 MBs. Later, we
      can make it slightly larger.
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      lcd-create-v2: Add low- and high-level vmalloc to kliblcd. · cf3f9e4c
      Charlie Jacobsen authored
      Similar to alloc pages. Call into the microkernel to vmalloc
      memory. Then, add it to the vmalloc resource tree so we can
      do address -> cptr translation.
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      lcd-create-v2: Update microkernel, kliblcd memory code for vmalloc. · a3fd6768
      Charlie Jacobsen authored
      Add vmalloc memory type in places where memory objects are
      used (memory object lookup, map/unmap, resource trees, global
      memory interval tree).
    • Charlie Jacobsen's avatar
      libcap-create-v2: Add vmalloc memory type to capability types. · e9246e5a
      Charlie Jacobsen authored
      This is different from *volunteered* vmalloc memory. Volunteered
      vmalloc memory is not freed when the last capability goes
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      lcd-create-v2: Kernel module load/unload for kliblcd. · ff505da5
      Charlie Jacobsen authored
      Adds two new functions to liblcd interface - lcd_module_load and
      lcd_module_unload - for loading a kernel module from disk and into
      the caller's address space. (Interface designed so that at some
      point in the future, liblcd can implement these for isolated
      The kliblcd implementation uses the host module loader to bring
      in the kernel module from disk (as before), but now we duplicate
      the entire module (rather than just the struct module pages).
      Motivation: There are numerous bits embedded in the kernel module's
      image (struct module, symbol tables, string tables, and possibly
      more) that the host uses during kallsyms lookup, module linked-list
      traversal, and so on, and it wouldn't be safe for the module loader
      to share that with the LCD. (The module loader wasn't designed
      to share bits with a potentially malicious or buggy peer.) It's
      simpler to just duplicate the entire module image (init and core).
      The other advantage is once we've made the duplicate, we can
      unload the copy in the host module loader (all we needed were the
      program bits).
      IMPORTANT: This has some significant implications:
        1 - It means means we no longer have to look up the host's copy
            of the struct module in the receiving side of glue code for
            e.g. register_filesystem. (The struct module will be treated
            just like any other struct.)
            I think overall this will be a good thing, and we won't run into
            serious problems in the near future. Functions like
            register_filesystem bump the reference count on the struct
            module. When the user tried to unload the kernel module, the host
            checks that the reference count goest to zero. Now, we need to
            ensure instead that the LCD that contains the module won't go
            away (e.g. before all file systems it is responsible for have
            been unmounted). We'll have to cross that bridge when we get there.
        2 - Some of the host utility functions for e.g. symbol lookup will
            not work as is. But I see some workarounds.
      The other big change is now we're using more coarse-grained
      capability management for the module pages (only two capabilities -
      one for init vmalloc mem, one for core vmalloc mem, rather than
      a capability for each page). I need to now implement
      lcd_vmalloc/lcd_vfree in order for this code to be fully ready.
      Note: I considered linking the kernel module at a lower address
      so that we can use a direct map in the LCD's virtual address
      space. And this may be possible now that we're unloading the
      module right after we load it. But it's a bit risky since the
      host uses some of the section addresses for symbol and exception
      table lookup, and maybe others (if I linked it for a lower
      address, the host would crash or use random memory). The benefits
      didn't seem to outweigh the risks.
    • Charlie Jacobsen's avatar
      host-resource-trees: Volunteer host memory. · ac556704
      Charlie Jacobsen authored
      When non-isolated code wants to "volunteer" host memory to the
      microkernel's capability system, it invokes lcd_volunteer_pages,
      lcd_volunteer_dev_mem, or lcd_volunteer_vmalloc_mem, depending on
      the type of memory.
      Internally, these check to see if the memory has already been
      volunteered (so we don't get duplicates -- this is checked via the
      global memory interval tree). If not, it inserts the memory into
      the caller's cspace. The caller can subsequently share the memory
      with e.g. an isolated LCD via the capability mediated interfaces.
      I had support for this before, but there was no checking for
      duplicate inserts (and this is a real problem with the pmfs example
      for string sharing): Non-isolated code has no way of knowing
      (without implementing data structures on its own) whether it inserted
      host memory already or not, or whether some other non-isolated
      code has.
      Furthermore, now we have full support for address -> cptr translation
      in the non-isolated side. This is also needed for the pmfs example
      with string sharing: before, the non-isolated code just always
      inserted memory every time to share strings, even if this lead
      to duplicate inserts.
      I think this is one of the "friction points" of embedding a
      capability system inside a kernel: translation from host objects
      to capabilities and back. For some objects, you can just embed
      the cptr in the object itself (our "container structs"). But for
      some things -- like memory -- it's not so easy. (For device memory,
      the host kernel doesn't use a struct page to represent it. So we're
      faced with creating our own giant array of data structures to
      represent each page of device memory, and embedding the cptr in
      that. Or instead -- as I have done -- use a data structure like
      a tree to do a reverse lookup.)
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      host-resource-trees: Update higher level alloc, map, unmap for trees. · 647e7c9b
      Charlie Jacobsen authored
      To mirror what will happen inside an isolated container, when
      code does a higher-level page alloc, it expects the pages to
      be mapped in its virtual and physical address spaces. It also
      expects to be able to do address -> cptr translations. This means
      we need to insert the page memory object into the resource tree
      during page alloc (and remove it when the pages are freed).
      Similarly, for higher-level map/unmap, we need to update the resource
      trees - as this is what will happen inside an isolated LCD, and
      non-isolated code expects to be able to do address -> cptr
      Almost done with this boring but important feature ...
    • Charlie Jacobsen's avatar
      host-resource-trees: Add/remove resource nodes from trees in kliblcd. · d39579d0
      Charlie Jacobsen authored
      Motivation: An LCD needs to keep track of address -> cptr
      correspondences. The resource trees fulfill that role. Each
      kLCD has two resource trees: one for physical memory (RAM, dev mem,
      etc.) and one for vmalloc memory (non-contiguous physical
      memory that is contiguous in the virtual address space).
      To mirror isolated code, when the kLCD maps/unmaps a memory
      object in host physical, we update its resource trees. (Of course,
      we don't bother / can't modify the host's physical mappings, so
      this is all that happens.) It gives kliblcd a chance to update
      the trees.
    • Charlie Jacobsen's avatar
      host-resource-trees: Remove memory object from tree in cap delete. · 659122b2
      Charlie Jacobsen authored
      When the last capability to a memory object goes away, the object
      will no longer be in the microkernel's capability system. Remove
      it from the memory interval tree.
    • Charlie Jacobsen's avatar
      host-resource-trees: Page alloc inserts into memory interval tree. · ccbce7bb
      Charlie Jacobsen authored
      Microkernel's page allocation code will insert the memory object
      that represents the allocated pages into the interval tree.
      Fixes interval tree insertion to use memory object range (start
      and last).
      Adds locks for tree traversal and per-node locks.
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
      host-resource-trees: Add global memory interval tree. · ad373505
      Charlie Jacobsen authored
      This is for tracking what host memory is in the microkernel's
      capability system. This prevents us from inserting the same memory
      multiple times into the capability system.
    • Charlie Jacobsen's avatar
      host-resource-trees: Clean up internal header, documentation. · e43ec0a6
      Charlie Jacobsen authored
      Clean up microkernel's module init/exit file, add init/exits for
      each subsystem.
    • Abhiram Balasubramanian's avatar
      Fix compilation error · 4acf9cbb
      Abhiram Balasubramanian authored
      - moved dependancies accordingly
      Signed-off-by: Abhiram Balasubramanian's avatarAbhiram Balasubramanian <abhiram@cs.utah.edu>