1. 26 Oct, 2016 40 commits
    • Charlie Jacobsen's avatar
      liblcd-v2: Heap allocator internals in place, first draft. · fa0aea92
      Charlie Jacobsen authored
      Uses the "generalized" allocator to manage 16 MBs of guest physical
      address space for a heap. Only the initial 4 MBs will be backed,
      and host memory is sucked in at 4 MB granularity.
      
      Some tuning of parameters will be necessary soon.
      fa0aea92
    • Charlie Jacobsen's avatar
      liblcd-v2: Low-level mmap done. Tweaked low-level mmap interface. · 9829af71
      Charlie Jacobsen authored
      Easier to have caller pass memory object order and gpa base for
      _lcd_mmap and _lcd_munmap, since in liblcd we can look those
      things up easily.
      9829af71
    • Charlie Jacobsen's avatar
      liblcd-v2: Fix kliblcd mapping, address -> cptr translation. · 89df5958
      Charlie Jacobsen authored
      I needed to clarify this part of the liblcd interface and ensure
      I have it right.
      
      Non-isolated code *must* invoke lcd_map_phys, lcd_map_virt, or
      _lcd_mmap on memory objects it either volunteers or is granted.
      This gives kliblcd an opportunity to insert the memory object
      into one of the two resource trees used for address -> cptr
      translation.
      
      I changed the names of the two trees to clarify what is stored
      in there - contiguous vs non-contiguous. (This is only a kliblcd
      internal thing.) It doesn't matter if you map contiguous memory
      via lcd_map_phys or lcd_map_virt; it always goes in the contiguous
      resource tree. Similar or non-contiguous (e.g. vmalloc) mem.
      89df5958
    • Charlie Jacobsen's avatar
      liblcd-v2: Low-level system call wrappers. · 7fe276a9
      Charlie Jacobsen authored
      I need to pause liblcd-v2 and redo the resource tree logic
      for kliblcd. Now that I'm working on liblcd, I'm seeing the way
      it should be.
      7fe276a9
    • Charlie Jacobsen's avatar
      liblcd-v2: Add memory itree code, simple LCD create code. · c99648ed
      Charlie Jacobsen authored
      Memory itree code initializes resource tree needed for
      address -> cptr translation. Provides interface for other liblcd
      code to add/remove from tree (page allocator will use that).
      
      Removed redundant nr_pages_order field from resource tree node. Its
      size can be computed from the interval tree node.
      
      Filled in some simple functions that are part of the liblcd
      interface. Some are no-ops for isolated code.
      c99648ed
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      liblcd-v2: Add missing syscalls, adjust liblcd syscall interfacing. · 3b8910b9
      Charlie Jacobsen authored
      I'm using registers now instead of cheating with the utcb. This
      is good in the end because it was getting ugly using the utcb
      for syscall arguments (e.g., during a sync ipc send, the higher
      level code needed to know that one of the utcb registers would
      be used for the actual syscall - that's pretty bad).
      3b8910b9
    • Sarah Spall's avatar
      ff2560df
    • Charles Jacobsen's avatar
      build-refactor: Fix interval tree missing symbols. · 8585db69
      Charles Jacobsen authored
      Linux kernel doesn't export interval tree symbols. So we just
      define our own (using the fancy generic macros Google made). This
      is necessary for isolated liblcd anyway.
      8585db69
    • Charles Jacobsen's avatar
      build-refactor: Incorporate common sources into kliblcd build. · 528cecbd
      Charles Jacobsen authored
      Build bug fixes for resource tree and module create code.
      528cecbd
    • Charles Jacobsen's avatar
      build-refactor: New arch directory, header shuffling. · 1abb5ae7
      Charles Jacobsen authored
      Following the Linux kernel src layout, moved the vt-x code
      into a separate arch folder. Moved headers around:
      
         -- Microkernel/kliblcd arch-independent headers are in
            include/lcd_domains
         -- Microkernel arch-dependent headers are in arch/x86/include
         -- liblcd headers are in include/liblcd
      
      Switched some file/dir names to use `_` instead of `-`.
      1abb5ae7
    • Charles Jacobsen's avatar
      f7983ab5
    • Charlie Jacobsen's avatar
      build-refactor: Build setup for microkernel and kliblcd. · ab53f6cd
      Charlie Jacobsen authored
      Set up out-of-source build for microkernel and kliblcd. This is
      necessary because the common/ sources will be built for non-isolated
      and isolated worlds (hence we can't do the in-source build - can't
      build same source files with different configs in the same directory).
      
      Developing a better solution for the liblcd config and hacks
      so we can abstract the build over whether the source file is
      built for non-isolated or isolated. Each environment defines a
      pre- and post- hook header that should be placed appropriately
      in source files for which we want to build for both environments.
      
      For now, non-isolated pre- and post-hooks are empty since we don't
      need to do any kernel build config hacks.
      ab53f6cd
    • Charlie Jacobsen's avatar
      build-refactor: Kconfig deleted. · 0a075cda
      Charlie Jacobsen authored
      We don't need this anymore since we're not hooked in to the
      kernel's config system.
      0a075cda
    • Charles Jacobsen's avatar
      libcap-integration: Microkernel and kliblcd build bug fixes. · c94e9745
      Charles Jacobsen authored
      Microkernel and kliblcd compile. Some linker errors due to
      missing resource tree and create objects. Going to refactor
      build now, more code shuffling fun.
      c94e9745
    • Anton Burtsev's avatar
      Support for LCD strack trace · 0be0f634
      Anton Burtsev authored
      0be0f634
    • Anton Burtsev's avatar
      Support for stack dump · 0ddd65f0
      Anton Burtsev authored
      0ddd65f0
    • Charlie Jacobsen's avatar
      lcd-create-v2: Re-factor klcd create with modules. · 22c4719d
      Charlie Jacobsen authored
      Not much changed here.
      22c4719d
    • Anton Burtsev's avatar
      2410a51f
    • Anton Burtsev's avatar
      Support for printing register state · 22c477ab
      Anton Burtsev authored
      22c477ab
    • Charlie Jacobsen's avatar
      lcd-create-v2: LCD destroy module lcd no longer needed. · 87838506
      Charlie Jacobsen authored
      This completes the isolated LCD creation logic, v2. Theoretically,
      this is now designed (unlike v1) so that isolated LCDs can
      create other isolated LCDs (once I get the liblcd updates in
      place).
      87838506
    • Charlie Jacobsen's avatar
      lcd-create-v2: First draft of lcd create v2 is done. · a4ecc8ed
      Charlie Jacobsen authored
      And a hell of a lot simpler.
      a4ecc8ed
    • Charlie Jacobsen's avatar
      lcd-create-v2: Guest virtual address space set up. · c28ba3f0
      Charlie Jacobsen authored
      Map everything except the ioremap region as write back. ioremap
      region is uncacheable. Only high 512 GBs are mapped to the first
      512 GBs of guest physical.
      
      No more dynamic build up of the guest virtual address space during
      page walks. That code was just so damn messy.
      c28ba3f0
    • Charlie Jacobsen's avatar
      lcd-create-v2: Physical address space setup (except UTCB). · fc7bf418
      Charlie Jacobsen authored
      Wow, this is so much easier with slightly coarser-grained
      capabilities.
      fc7bf418
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      lcd-create-v2: Update address space layout for v2. · 39dfe81c
      Charlie Jacobsen authored
      The boot physical address space fits in the first 512 GBs.
      The boot virtual address maps this to the high 512 GBs. In this
      way, the kernel module code will be loaded at the address it was
      linked for.
      
      Address spaces designed for 1 GB huge guest virtual pages. The
      hope is this address will be big enough for a lot of use cases,
      so the code inside the LCD won't ever need to touch the guest
      virtual address space (just physical).
      39dfe81c
    • Sarah Spall's avatar
      code to allocate and initialize trampoline structures. fixed bug where regular... · 427b971a
      Sarah Spall authored
      code to allocate and initialize trampoline structures. fixed bug where regular rpcs were beginning ignored if declared in an unnamed scope
      427b971a
    • Anton Burtsev's avatar
      Changed regs to be a data structure vs an array · d87f9b5e
      Anton Burtsev authored
      Not tested, can't test because of the PAT problems
      d87f9b5e
    • Anton Burtsev's avatar
      d41ab5b9
    • Charles Jacobsen's avatar
      Fix PAT initialization, shrink ioremap region for now. · 2aedcbde
      Charles Jacobsen authored
      PAT appears to work now with write back memory. Added an
      arch check to ensure PAT is valid before VM entry.
      
      The ioremap region was too big (the amount of metadata - bitmaps,
      etc. - was too much). I shrunk it for now to 16 MBs. Later, we
      can make it slightly larger.
      2aedcbde
    • Charlie Jacobsen's avatar
      1e5c02ad
    • Charlie Jacobsen's avatar
      lcd-create-v2: Add low- and high-level vmalloc to kliblcd. · cf3f9e4c
      Charlie Jacobsen authored
      Similar to alloc pages. Call into the microkernel to vmalloc
      memory. Then, add it to the vmalloc resource tree so we can
      do address -> cptr translation.
      cf3f9e4c
    • Charlie Jacobsen's avatar
      175b98f6
    • Charlie Jacobsen's avatar
      lcd-create-v2: Update microkernel, kliblcd memory code for vmalloc. · a3fd6768
      Charlie Jacobsen authored
      Add vmalloc memory type in places where memory objects are
      used (memory object lookup, map/unmap, resource trees, global
      memory interval tree).
      a3fd6768
    • Charlie Jacobsen's avatar
      libcap-create-v2: Add vmalloc memory type to capability types. · e9246e5a
      Charlie Jacobsen authored
      This is different from *volunteered* vmalloc memory. Volunteered
      vmalloc memory is not freed when the last capability goes
      away.
      e9246e5a
    • Charlie Jacobsen's avatar
      b49acd30
    • Charlie Jacobsen's avatar
      lcd-create-v2: Kernel module load/unload for kliblcd. · ff505da5
      Charlie Jacobsen authored
      Adds two new functions to liblcd interface - lcd_module_load and
      lcd_module_unload - for loading a kernel module from disk and into
      the caller's address space. (Interface designed so that at some
      point in the future, liblcd can implement these for isolated
      code.)
      
      The kliblcd implementation uses the host module loader to bring
      in the kernel module from disk (as before), but now we duplicate
      the entire module (rather than just the struct module pages).
      
      Motivation: There are numerous bits embedded in the kernel module's
      image (struct module, symbol tables, string tables, and possibly
      more) that the host uses during kallsyms lookup, module linked-list
      traversal, and so on, and it wouldn't be safe for the module loader
      to share that with the LCD. (The module loader wasn't designed
      to share bits with a potentially malicious or buggy peer.) It's
      simpler to just duplicate the entire module image (init and core).
      
      The other advantage is once we've made the duplicate, we can
      unload the copy in the host module loader (all we needed were the
      program bits).
      
      IMPORTANT: This has some significant implications:
      
        1 - It means means we no longer have to look up the host's copy
            of the struct module in the receiving side of glue code for
            e.g. register_filesystem. (The struct module will be treated
            just like any other struct.)
      
            I think overall this will be a good thing, and we won't run into
            serious problems in the near future. Functions like
            register_filesystem bump the reference count on the struct
            module. When the user tried to unload the kernel module, the host
            checks that the reference count goest to zero. Now, we need to
            ensure instead that the LCD that contains the module won't go
            away (e.g. before all file systems it is responsible for have
            been unmounted). We'll have to cross that bridge when we get there.
      
        2 - Some of the host utility functions for e.g. symbol lookup will
            not work as is. But I see some workarounds.
      
      The other big change is now we're using more coarse-grained
      capability management for the module pages (only two capabilities -
      one for init vmalloc mem, one for core vmalloc mem, rather than
      a capability for each page). I need to now implement
      lcd_vmalloc/lcd_vfree in order for this code to be fully ready.
      
      Note: I considered linking the kernel module at a lower address
      so that we can use a direct map in the LCD's virtual address
      space. And this may be possible now that we're unloading the
      module right after we load it. But it's a bit risky since the
      host uses some of the section addresses for symbol and exception
      table lookup, and maybe others (if I linked it for a lower
      address, the host would crash or use random memory). The benefits
      didn't seem to outweigh the risks.
      ff505da5
    • Charlie Jacobsen's avatar
      host-resource-trees: Volunteer host memory. · ac556704
      Charlie Jacobsen authored
      When non-isolated code wants to "volunteer" host memory to the
      microkernel's capability system, it invokes lcd_volunteer_pages,
      lcd_volunteer_dev_mem, or lcd_volunteer_vmalloc_mem, depending on
      the type of memory.
      
      Internally, these check to see if the memory has already been
      volunteered (so we don't get duplicates -- this is checked via the
      global memory interval tree). If not, it inserts the memory into
      the caller's cspace. The caller can subsequently share the memory
      with e.g. an isolated LCD via the capability mediated interfaces.
      
      I had support for this before, but there was no checking for
      duplicate inserts (and this is a real problem with the pmfs example
      for string sharing): Non-isolated code has no way of knowing
      (without implementing data structures on its own) whether it inserted
      host memory already or not, or whether some other non-isolated
      code has.
      
      Furthermore, now we have full support for address -> cptr translation
      in the non-isolated side. This is also needed for the pmfs example
      with string sharing: before, the non-isolated code just always
      inserted memory every time to share strings, even if this lead
      to duplicate inserts.
      
      I think this is one of the "friction points" of embedding a
      capability system inside a kernel: translation from host objects
      to capabilities and back. For some objects, you can just embed
      the cptr in the object itself (our "container structs"). But for
      some things -- like memory -- it's not so easy. (For device memory,
      the host kernel doesn't use a struct page to represent it. So we're
      faced with creating our own giant array of data structures to
      represent each page of device memory, and embedding the cptr in
      that. Or instead -- as I have done -- use a data structure like
      a tree to do a reverse lookup.)
      ac556704