1. 29 Apr, 2018 1 commit
  2. 27 Nov, 2016 2 commits
  3. 26 Oct, 2016 1 commit
    • Charlie Jacobsen's avatar
      Major overhaul of build process. · 8198c2fb
      Charlie Jacobsen authored
      Full kernel build no longer required. Yay! This should
      cut down on dev time a lot.
      I moved all of the LCD source into $(kernel-src)/lcd-domains/,
      so it's all in one spot. There is now a top-level makefile in
      there that triggers building liblcd, the microkernel, and the
      examples. This is built as an *external* build now, even
      though the directory is in the kernel source. The build now takes
      under a minute to do everything LCD related.
      This should also make verification easier in the future (e.g.
      building with clang) if we aren't ensnared in the kernel
      Of course, to use the microkernel and examples, you have to
      build the patched kernel and install it. But now when you
      make a few lines of changes in e.g. an example, you don't have
      to trigger a top-level kernel build to rebuild it. Running
      the full kernel build takes on average about 3 - 4 minutes
      (some files are generated everytime, linking is done, and so
      on), and can take upwards of 30 minutes for a full build if you
      Which brings me to my other change: no more config for LCDs
      in menuconfig. If we create menu entries for every example
      and so on, we end up changing the config too often, and this
      triggers full kernel rebuilds == waste of time. We can use
      macros by setting them via compiler flags (e.g., -DSOME_FLAG).
      Furthermore, it wasn't making sense to me to do conditional
      compilation for LCD support (we always want to compile for that).
      Yes, changes aren't clearly delineated with macros, but you can
      see changes made by just doing 'git diff v3.10.14 some-file-or-dir'.
      The wiki has been fully updated with instructions for building,
      and other relevant parts (updated paths to files).
      I also took the opportunity to clean up some old stuff lying around
      that is dead (like lcdguest). I incorporated all of the documentation
      in Documentation/lcd-domains into the wiki so it's all in one
      spot now (including some helpful debug tips).
  4. 25 Oct, 2016 36 commits
    • Charlie Jacobsen's avatar
      Installs PMFS filesystem. · 197888d7
      Charlie Jacobsen authored
      This was done by applying pmfs.patch from our version
      of the PMFS git repository (currently here -
      https://gitlab.flux.utah.edu/xcap/pmfs), and squashing
      all of the commits from the patch into this one.
      pmfs.patch was generated by essentially doing a diff
      on the v3.10-compatible PMFS (the head of the master
      branch in our repo) and linux v3.10 (tagged in our
      repo as linux-v3.10). There were a
      few minor patch conflicts I had to manually fix
      in the patch. Those changes are reflected in the
      pmfs.patch that is in our repo.
    • Charlie Jacobsen's avatar
      Breaks apart microkernel into smaller pieces, and refactors regression tests. · 40bfcbdb
      Charlie Jacobsen authored
      Trying to make the code base saner for others to read through.
      Tests are no longer ran every time you insert the microkernel. They are built
      as a separate kernel module (in virt/lcd-domains/tests/lcd-tests.ko).
      Ran tests again, and some of the LCD functional tests. All appears OK.
    • Charlie Jacobsen's avatar
      Linux slab allocator and page allocator inside LCDs. · 35ed57d7
      Charlie Jacobsen authored
      Single threaded, no locks, no fancy NUMA/percpu.
      Passing some simple examples. Added a memory management
      example module, in test-mods/mm, that exercises a lot
      of this new code.
      I moved in and adapted our existing guest virtual
      paging code from kliblcd.c. I'm using statically
      allocated bitmaps and arrays for tracking allocations
      in the guest virtual and physical address spaces.
      Using identity mapping for ease. (I decided not to
      use Linux's page allocator since it's too intertwined
      with the boot process - percpu variables, freeing
      init mem, boot allocator, all kinds of complexity ...)
      It might not be too hard to reimplement the buddy
      allocator algorithms, since I had to include a
      statically allocated array of struct pages anyway.
      I've set aside about 16 MBs for dynamic page allocations,
      but this can be changed using macros. You can allocate
      1, 2, 4, 8, etc. pages at a time. (The slab allocator
      requires this.)
      I finally broke down and set up boot info pages - 4
      boot pages right now, can be adjusted with a macro.
      Whoever boots the lcd needs to pack in information about the
      lcd's address space, initial cspace, and so on. 4 pages
      is enough to pack in information for larger modules
      like the mm example.
      I moved liblcd to a separate directory, and hacked the
      build system so that we can build liblcd as a static
      library and link it with example modules.
      liblcd/ contains lcd/, which has code for interacting
      with the microkernel and my simple page allocator.
      The Linux slab allocator is inside mm/, and some
      needed dependencies are in lib/. I made very few
      changes to the source code itself, but used some
      preprocessor/compiler hacks to make everything work. See
      Documentation/lcd-domains/liblcd.txt. I elided all of
      the locking and made it single core, single NUMA node.
      It's possible we'll see some bigs in the future, in code I haven't
      excerised yet (will probably manifest themselves as
      page faults).
      Ideally, we should have a separate tree for liblcd
      and building modules. That way we can avoid some of these
      hacks (maybe not all).
      Updated a lot of the documentation in
    • Charlie Jacobsen's avatar
      Simple printk, adapted from Linux's vsprintf. · 41e1d4fd
      Charlie Jacobsen authored
      Updated liblcd and the microkernel. Doesn't support
      all escapes, like %p.
      Updated documentation. Stack protection should be turned off.
    • Charlie Jacobsen's avatar
      Two simple IPC tests are passing. Still getting mysterious hang. · c667fea5
      Charlie Jacobsen authored
      Working on moving Linux's mm into lcd's.
      I gave up trying to debug the hang. Confirmed the pages for the lcd's
      vm are the ones I expect. Turned on red zones. All tests are passing.
      Hang happens after insmod/rmmod of the lcd module about 10 - 20 times, it
      varies. Sometimes one core just silently dies / doesn't even respond to
      an NMI. Sometimes the ethernet driver complains (this could be an
      unrelated bug that was fixed upstream).
      Few things in this commit:
      Updated documentation in Documentation/lcd-domains/.
      Baby version of lib kernel, inside arch/x86/lcd-domains/liblcd.c.
      Unfortunately due to the recursive make, this needs to be textually
      included inside the modules destined for lcd's, for now.
      Added new test modules and modified directory structure and
      build system. See documentation in Documentation/lcd-domains.
      A few tweaks to the nmi handler to print a backtrace. May remove that in
      the future, as it's probably not safe to do inside an nmi handler (but if
      we're in that error state, we might be desperate to know what's happening ...).
      Changed interrupt handling in arch-dependent code. The KVM code we were using
      is probably wrong for 64-bit - it doesn't properly switch stacks, etc., which
      is super important for 64-bit and may be impossible to emulate in
      software. I think this could be stale code inside KVM, but not sure. Dune
      doesn't use it. KVM doesn't ack external interrutps on vm exit, so I think
      this interrupt emulation code is always skipped (at least for non-nested
      Instead, we're not ack'ing interrupts on exit, and letting the native code
      do the right thing, like Dune.
      I was thinking this might be the source of the bad hang (stack
      overflow, e.g.), but not true.
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <vikram186@gmail.com>
    • Charlie Jacobsen's avatar
      Except IPC, kliblcd fully tested. Everything is working. · 664628a4
      Charlie Jacobsen authored
      Documentation in Documentation/lcd-domains/...
      Loading, mapping, and running a module is working correctly, using
      all of the capability code that interposes on each operation (mapping,
      freeing pages, etc.).
      cptr allocation and indexing into cspaces is working correctly.
      IPC testing and debugging is coming next.
    • Charlie Jacobsen's avatar
      Muktesh's capabilities fully incorporated. Capsicum-style enter/exit. · 6ee9a51f
      Charlie Jacobsen authored
      Builds, but not fully tested. Good tests for capability subsystem, some tests
      for kliblcd.
      Non-isolated kernel threads can "enter" the lcd system by doing
      klcd_enter / klcd_exit. They can create other lcd's, set them up, etc. They
      use the same interface that regular lcd's will use, so such code could be
      moved to an lcd, as we had planned. Will document this in Documentation folder
      tomorrow ( == today ).
      Capability system does checks now when a capability is deleted/revoked: for
      example, if it's for a page, the microkernel checks if the page is mapped, and
      unmaps it. If the last capability goes away, the page is freed. Documentation
      is in Documentation/lcd-domains/cap.txt.
      IPC code is in place, but not tested yet (pray for me).
      Debug is taking some time. Sometimes requires a power cycle which adds an
      extra 5 - 10 minutes. Build is slow the first time after reboot. Give me a user
      level program and I'll debug it in 30 seconds! argc
      Main arch-independent files:
          include/lcd-domains/kliblcd.h, types.h
             This is what non-isolated kernel code should include to use the
             kliblcd interface to the microkernel.
          virt/lcd-domains/main.c, kliblcd.c, cap.c, ipc.c, internal.h
             The microkernel, broken up into pieces.
             The tests, in progress.
      Some old files are still hanging around in virt/lcd-domains and will be
      incorprated/cleaned up soon.
      I couldn't squash over the merge from the decomposition branch, so there's a
      bunch of junk commits coming over. (I should've just copied Muktesh's files.)
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <vikram186@gmail.com>
    • Charles Jacobsen's avatar
      Works! · b9e42c88
      Charles Jacobsen authored
    • Charlie Jacobsen's avatar
    • Charles Jacobsen's avatar
      Wasn't doing a get/put on kthread. · 7ac8a242
      Charles Jacobsen authored
      Still haven't figured out page fault bug.
    • Charles Jacobsen's avatar
      Fixed build errors, module init tools. · 0f690a86
      Charles Jacobsen authored
      Getting nasty runtime bugs though.
    • Charlie Jacobsen's avatar
      Finished the majority of the arch-indep code changes needed. · d71afeb4
      Charlie Jacobsen authored
      Refactored lcd's into lcd and lcd_thread. Still need to test/update
    • Charles Jacobsen's avatar
      All (simple) lcd arch tests passing. · bee0375b
      Charles Jacobsen authored
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
      Separated lcd into container and thread objects. · dbc4e40c
      Charlie Jacobsen authored
      Updated code. Removed gdt/tss/idt for now. Added doc directory
      and some initial doc.
    • Charles Jacobsen's avatar
    • Charles Jacobsen's avatar
      Switched to a kmem cache for allocating lcd_arch's. · 0f6ce597
      Charles Jacobsen authored
      Was getting bad load/store msr addresses for vm exits.
      Properly detected by the lcd_arch_check code.
      In struct lcd_arch, the msr_autoload fields were not
      word aligned in some instances (see requirement
      in Intel SDM V3 Even with compiler align
      attributes, if we use kmalloc, proper alignment is
      not guaranteed - we can allocate a struct lcd_arch
      on the wrong byte and it throws off the alignment
      of the entire struct.
      kmem cache allows specifying alignment. Appears to be
      working correctly.
    • Charlie Jacobsen's avatar
      Basic lcd module create, run, and destroy. · e0193fa4
      Charlie Jacobsen authored
      This code is ugly, but it's working.
      Tested with basic module, and appears to be working
      properly. I will soon incorporate the patched
      modprobe into the kernel tree, and then this code
      will be usable by everyone.
      The ipc code is still unimplemented. The only
      hypercall handled is yield. Also note that other
      exit conditions (e.g. external interrupt) have not
      been fully tested.
      -- kernel code calls lcd_create_as_module with
         the module's name
      -- lcd_create_as_module loads the module using
         request_lcd_module (request_lcd_module calls
         the patched modprobe to load the module, and
         the patched modprobe calls back into the lcd
         driver via the ioctrl interface to load the
      -- lcd_create_as_module then finds the loaded
         module, spawns a kernel thread and passes off
         the module to it
      -- the kernel thread initializes the lcd and
         maps the module inside it, then suspends itself
      -- lcd_run_as_module wakes up the kernel thread
         and tells it to run
      -- lcd_delete_as_module stops the kernel thread
         and deletes the module from the host kernel
      File-by-file details:
      -- lcd was not running in 64-bit mode, and my
         checks had one subtle bug
      -- fixed %cr3 load to properly load vmcs first
      -- fixed set program counter to use guest virtual
         rather than guest physical address
      -- added struct lcd to task_struct
      -- lcd pointer set to null when task_struct is
      -- made init_module and delete_module system calls
         callable from kernel code
      -- available in module.h via do_sys_init_module and
      -- simply moved the majority of the guts of the
         system calls into a non-system call, exported
      -- take an extra flag, for_lcd; when set, the init
         code skips over running (and deallocating) the
         module's init code, and the delete code skips
         over running the module exit
      -- system calls from user code set for_lcd = 0; this
         ensures existing code still works
      -- changed __request_module to __do_request_module; takes
         one extra argument, for_lcd
      -- __request_module   ==>  __do_request_module with for_lcd = 0
      -- request_lcd_module ==>  __do_request_module with for_lcd = 1
      -- call_modprobe conditionally uses lcd_modprobe_path, the path
         to a patched modprobe accessible via sysfs
      -- added lcd status enum; see source code doc
      -- three routines for creating/running/destroying
         lcd's that use modules; see source code doc
      -- added interface defns for patched modprobe to call into
         lcd driver for module init; lcd driver loads
         module (via slightly refactored module.c code) on behalf
         of modprobe
      -- implementation of routines for modules inside lcd's
      -- implementation of module init / delete for lcd's
         (uses patched module.c code)
      -- added test module for lcd module code
      -- test runs automatically when lcd module is inserted
    • Charles Jacobsen's avatar
      Simple blob successfully runs and returns in vmx. · c4fc774c
      Charles Jacobsen authored
      In the process of debugging, major corrections and
      debug checks implemented (this is a big squash):
      -- coded up entry checks documented in Intel SDM
         V3 chp 26 (this should help later if settings
         are changed, make it less risky to experiment
         and give more confidence)
      -- fixed host tr base addr bug (the worst bug! caused
         system to hang and then crash, since the tss
         was erroneous...)
      -- fixed vmx_entry to properly set host rsp before
      -- setting host sysenter and idt info
      -- fixed cs ar bytes bug
      -- fixed gdt limit bug
      -- fixed tr type bug
      -- extra settings added to cr0 and cr4, but may
         not be needed ... (debug fix attempt)
      -- lstar mstar autoload, may not be needed ...
         (debug fix attempt)
    • Anton Burtsev's avatar
      LCD API client and server code · fc2675ff
      Anton Burtsev authored
        -- LCD export an API to its domains that is accessible via
        capability invocations (well, syscalls if you like)
        -- Kernel runs a thread that implement this API, e.g. serves
        capability invocations
    • Anton Burtsev's avatar
      Simple, list-based capability allocator · 75340d2a
      Anton Burtsev authored
        -- I've split capability allocation and cspaces, this makes much more
           sense -- complex domains can implement custom allocation policies,
           simple domains go with a static set of caps
        -- This is a list-based (zone like) allocator
    • Charles Jacobsen's avatar
      Fixed build errors, all tests passing. · 78569fe2
      Charles Jacobsen authored
      Fixed (another) nasty casting bug in the
      code that initializes the ept pointer. I
      was using the old way of casting, and overlooking
      that I was storing a host virtual pointer
      inside a host physical pointer. This caused
      invept to fail (invept checks if the eptp is
      I also fixed invept and invvpid to print an
      error message (like vmload and vmclear), rather
      than generate an invalid opcode exception. It's
      easier to debug and understand the problem.
      In the future, we could more carefully parse the
      error returned.
    • Charles Jacobsen's avatar
      Minor build bug fixes. · f9875eab
      Charles Jacobsen authored
      (Sorry for the barrage of pushes, but I want
      to make sure I push changes in case I crash
      the machine...)
    • Charles Jacobsen's avatar
      Most build errors fixed. · e6442a9d
      Charles Jacobsen authored
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      Introducing types for each gpa, gva, hva, hpa. · 82aa94ed
      Charlie Jacobsen authored
      Constructors and casting inlines are in arch-dep
      code header.
    • Charles Jacobsen's avatar
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
      Shifted gpa map / unmap range to arch-dep code. · f9b52a46
      Charlie Jacobsen authored
      Updated tests.
    • Charles Jacobsen's avatar
      Fixing up ept destroy to not free mapped mem. · 8a977687
      Charles Jacobsen authored
      Originally, destroying the ept structures also
      freed the mapped host physical memory. This is
      leading to bad double free errors. Instead, I
      will report a potential memory leak if an ept
      pte is present.
      Users of the ept should unmap any host phys
      memory before destroying the ept (will shift
      over ept unmap range from arch-indep to
      arch-dep next).
      Updated arch test04 to use vmx_free_ept, now
      that it doesn't try to free mapped host phys
    • Charles Jacobsen's avatar
      All lcd tests through 5 are passing. · a81d9641
      Charles Jacobsen authored
      Added pgd_pfn to asm/pgtables.h.
    • Charlie Jacobsen's avatar
      Small fix to arch set pc. · 32c30a68
      Charlie Jacobsen authored
      Need to load vmcs in order to modify it.