All new accounts created on Gitlab now require administrator approval. If you invite any collaborators, please let Flux staff know so they can approve the accounts.

  1. 15 May, 2017 1 commit
  2. 27 Mar, 2017 1 commit
  3. 27 Nov, 2016 1 commit
  4. 26 Oct, 2016 13 commits
    • Charlie Jacobsen's avatar
    • Charles Jacobsen's avatar
      stack-trace: Fix module loader to not 'erase' init symbols. · eac08288
      Charles Jacobsen authored
      For normal module loads, __init code is freed, so it makes sense
      we update the symbol table to remove those symbols. But for
      LCDs, we want them to hang around.
    • Charles Jacobsen's avatar
      stack-trace: Stack trace 90 percent there. Wow, that was ugly. · d8f3540a
      Charles Jacobsen authored
      I fought this thing until late in the morning. Since we're
      working with a copy of the module image (.ko), some of the fields
      in the struct module have to be updated (symbol and string table
      fields). Then, everything works.
      Still getting weird trace symbol for return address at the
      bottom of the stack.
    • Charles Jacobsen's avatar
    • Charlie Jacobsen's avatar
      libcap-integration: Resource trees partially integrated into kliblcd. · be7b3caa
      Charlie Jacobsen authored
      Enter/exit code sets up/tears down the thread's tree.
      Fixed a few spotted bugs in allocation and tree code.
    • Charlie Jacobsen's avatar
      Moves async/thc into liblcd. Removes old files. · 34760078
      Charlie Jacobsen authored
      Not integrated into liblcd yet.
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <>
    • Charlie Jacobsen's avatar
      Major overhaul of build process. · 8198c2fb
      Charlie Jacobsen authored
      Full kernel build no longer required. Yay! This should
      cut down on dev time a lot.
      I moved all of the LCD source into $(kernel-src)/lcd-domains/,
      so it's all in one spot. There is now a top-level makefile in
      there that triggers building liblcd, the microkernel, and the
      examples. This is built as an *external* build now, even
      though the directory is in the kernel source. The build now takes
      under a minute to do everything LCD related.
      This should also make verification easier in the future (e.g.
      building with clang) if we aren't ensnared in the kernel
      Of course, to use the microkernel and examples, you have to
      build the patched kernel and install it. But now when you
      make a few lines of changes in e.g. an example, you don't have
      to trigger a top-level kernel build to rebuild it. Running
      the full kernel build takes on average about 3 - 4 minutes
      (some files are generated everytime, linking is done, and so
      on), and can take upwards of 30 minutes for a full build if you
      Which brings me to my other change: no more config for LCDs
      in menuconfig. If we create menu entries for every example
      and so on, we end up changing the config too often, and this
      triggers full kernel rebuilds == waste of time. We can use
      macros by setting them via compiler flags (e.g., -DSOME_FLAG).
      Furthermore, it wasn't making sense to me to do conditional
      compilation for LCD support (we always want to compile for that).
      Yes, changes aren't clearly delineated with macros, but you can
      see changes made by just doing 'git diff v3.10.14 some-file-or-dir'.
      The wiki has been fully updated with instructions for building,
      and other relevant parts (updated paths to files).
      I also took the opportunity to clean up some old stuff lying around
      that is dead (like lcdguest). I incorporated all of the documentation
      in Documentation/lcd-domains into the wiki so it's all in one
      spot now (including some helpful debug tips).
    • Charlie Jacobsen's avatar
      PMFS isolation: full init operational (minus function pointers). · af74008b
      Charlie Jacobsen authored
      bdi init/destroy in place. Fixed some data store bugs.
      file system type function pointers not set up yet. Need to figure out
      strategy for handling them (during mount).
    • Charlie Jacobsen's avatar
      Starts glue for PMFS. · ce03d0a2
      Charlie Jacobsen authored
      Sets up vfs klcd, and most of the code in place in boot module
      for setting up pmfs and vfs.
      (Need to squash and merge some code from the mainline.)
    • Charlie Jacobsen's avatar
      Fixes security bug: LCD could modify host's struct. · 18fc3b79
      Charlie Jacobsen authored
      This is a trivial commit, but I want to get this fixed before
      it bites us in the future.
      The problem: the host kernel uses a struct to represent an
      installed kernel module. But this struct is literally embedded
      in the module's program bits. We're mapping the entire module
      inside the LCD. Thus, the LCD can do whatever it wants to the
      struct, and this could affect the host.
      Yes, all of the other parts of the module are technically exposed
      to the host. But it's important we isolate this struct because this
      is something the host will actually touch.
      The solution: I duplicate the page that contains the struct, so
      that the LCD has a separate copy of the struct.
      Open issues: we still need to come up with a solution for an LCD
      to pass its struct module as an argument, via rpc. I'm going to
      hack it for now with PMFS.
    • Charlie Jacobsen's avatar
      Pulls dispatch loop definitions into headers. · bd924ac1
      Charlie Jacobsen authored
      This is a simple commit that pulls some common dispatch
      loop infrastructure (struct defs, etc.) into
      Updated glue example, all working.
    • Charlie Jacobsen's avatar
      Improves/abstracts over trampoline glue code (for function pointers). · 65db6aff
      Charlie Jacobsen authored
      I've hidden the nasty bits of the trampoline code as
      much as I can for now behind some macros, defined
      in include/lcd-domains/trampoline.h and trampoline_link.h.
      I updated the glue code example. Everything is working.
      Hopefully this should make trampolines easier to write,
      generate, and eventually verify (the macros I wrote
      will "expand" to some kind of models when we verify).
    • Charlie Jacobsen's avatar
      Adds full functioning example with glue code. · 424ac36d
      Charlie Jacobsen authored
      Example is in virt/lcd-domains/test-mods/glue-example. It consists
      of a fake minix and fake vfs. The original "unmodified" code is
      in minix/original/main.c, vfs/original/main.c, and include/vfs.h.
      The glue code (written by hand) is in minix/glue and vfs/glue. The
      IDL is in idl/.
      You can build the fake minix and vfs for isolation or as regular
      modules. To build for isolation, run `make menuconfig' and under
      Test Modules --> Example Exercising IDL and Glue.
      To run minix and vfs in isolation, install the "boot" module,
      lcd-test-mod-glue-example-boot.ko. (Its code is under isol-boot/main.c.)
      The example will print status messages to the kernel logs so you
      can see a trace of the interaction.
      A few hacks were necessary to fully exercise all of the code. For example,
      the fake vfs is the one that invokes new_file and rm_file, in the middle
      of its dispatch loop.
      The code for the trampolines in vfs/glue/vfs_caller.{c,lds} is probably
      the ugliest part. That required a little bit of trickery and low-level
  5. 25 Oct, 2016 24 commits
    • Michael Quigley's avatar
      exported symbols needed for use in kernel modules. Also patched genksyms so... · 9432f609
      Michael Quigley authored
      exported symbols needed for use in kernel modules. Also patched genksyms so that it doesn't crash on a null name which is generated from the inline asembly I think.
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <>
    • Charlie Jacobsen's avatar
      Installs PMFS filesystem. · 197888d7
      Charlie Jacobsen authored
      This was done by applying pmfs.patch from our version
      of the PMFS git repository (currently here -, and squashing
      all of the commits from the patch into this one.
      pmfs.patch was generated by essentially doing a diff
      on the v3.10-compatible PMFS (the head of the master
      branch in our repo) and linux v3.10 (tagged in our
      repo as linux-v3.10). There were a
      few minor patch conflicts I had to manually fix
      in the patch. Those changes are reflected in the
      pmfs.patch that is in our repo.
    • Charlie Jacobsen's avatar
      Adds support for creating a kLCD from another kLCD. · e49732d2
      Charlie Jacobsen authored
      This is a boring but important commit that we need for
      our glue code example (that is still in progress).
      The main motiviation is: One kLCD (like a booting kLCD) may
      want to create other threads to run kernel modules, and it
      may want to grant them capabilities to resources. (Before
      this, this wasn't possible - all kLCDs were isolated from
      each other.) Another alternative (possibly better) for the
      future is to allow threads to share cspaces. But we're
      probably not there yet.
      Adds the following to the kLIBLCD interface:
      -- klcd_create_klcd - essentially creates another kernel
         thread and initializes the lcd-specific stuff like its
         cspace; the creator can then use lcd_cap_grant to grant
         this thread capabilities
      -- klcd_create_module_klcd - creates a klcd (using
         klcd_create_klcd) and loads a kernel module for the klcd
         to run
      -- klcd_destroy_module_klcd - destroys klcd (the kernel
         thread, etc.) and unloads the module
      Adds test and an example group of modules that show how to use
    • Charlie Jacobsen's avatar
      Adds 'data store' to non-isolated kliblcd. · 730fd7d4
      Charlie Jacobsen authored
      Kind of boring, but important. Now poised for doing some non-isolated
      glue code.
      Kind of bummer we have essentially three copies of the cspace-related
      code - one in liblcd (isolated data store), one in kliblcd, and one in
      the microkernel (real cspaces). Hopefully we can coalesce and reduce
      the redundancy at some point.
      The data store code for kliblcd is in virt/lcd-domains/kliblcd/dstore.c.
      The interface is in include/lcd-domains/kliblcd.h.
      Moved dstore into kliblcd. No build yet.
      Small typo in cap doc.
      Debug something else.
      Updated dstore to do get/put.
      Fixes build. liblcd tests pass.
    • Charlie Jacobsen's avatar
      Breaks apart microkernel into smaller pieces, and refactors regression tests. · 40bfcbdb
      Charlie Jacobsen authored
      Trying to make the code base saner for others to read through.
      Tests are no longer ran every time you insert the microkernel. They are built
      as a separate kernel module (in virt/lcd-domains/tests/lcd-tests.ko).
      Ran tests again, and some of the LCD functional tests. All appears OK.
    • Charlie Jacobsen's avatar
      Basic 'data store' code added to libkernel. · ef040dc7
      Charlie Jacobsen authored
      Just the capability code adapted for use inside the libkernel.
      Basic tests passing. Coalesced a lot of the liblcd tests into
      one spot (in liblcd/lcd/test.c).
    • Muktesh Khole's avatar
    • Muktesh Khole's avatar
      Fixed crash, basic test passed · d57d12c0
      Muktesh Khole authored
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <>
    • Charlie Jacobsen's avatar
      Linux slab allocator and page allocator inside LCDs. · 35ed57d7
      Charlie Jacobsen authored
      Single threaded, no locks, no fancy NUMA/percpu.
      Passing some simple examples. Added a memory management
      example module, in test-mods/mm, that exercises a lot
      of this new code.
      I moved in and adapted our existing guest virtual
      paging code from kliblcd.c. I'm using statically
      allocated bitmaps and arrays for tracking allocations
      in the guest virtual and physical address spaces.
      Using identity mapping for ease. (I decided not to
      use Linux's page allocator since it's too intertwined
      with the boot process - percpu variables, freeing
      init mem, boot allocator, all kinds of complexity ...)
      It might not be too hard to reimplement the buddy
      allocator algorithms, since I had to include a
      statically allocated array of struct pages anyway.
      I've set aside about 16 MBs for dynamic page allocations,
      but this can be changed using macros. You can allocate
      1, 2, 4, 8, etc. pages at a time. (The slab allocator
      requires this.)
      I finally broke down and set up boot info pages - 4
      boot pages right now, can be adjusted with a macro.
      Whoever boots the lcd needs to pack in information about the
      lcd's address space, initial cspace, and so on. 4 pages
      is enough to pack in information for larger modules
      like the mm example.
      I moved liblcd to a separate directory, and hacked the
      build system so that we can build liblcd as a static
      library and link it with example modules.
      liblcd/ contains lcd/, which has code for interacting
      with the microkernel and my simple page allocator.
      The Linux slab allocator is inside mm/, and some
      needed dependencies are in lib/. I made very few
      changes to the source code itself, but used some
      preprocessor/compiler hacks to make everything work. See
      Documentation/lcd-domains/liblcd.txt. I elided all of
      the locking and made it single core, single NUMA node.
      It's possible we'll see some bigs in the future, in code I haven't
      excerised yet (will probably manifest themselves as
      page faults).
      Ideally, we should have a separate tree for liblcd
      and building modules. That way we can avoid some of these
      hacks (maybe not all).
      Updated a lot of the documentation in
    • Charlie Jacobsen's avatar
      Simple printk, adapted from Linux's vsprintf. · 41e1d4fd
      Charlie Jacobsen authored
      Updated liblcd and the microkernel. Doesn't support
      all escapes, like %p.
      Updated documentation. Stack protection should be turned off.
    • Muktesh Khole's avatar
      Async Code, builds successfully - untested · 71bacb46
      Muktesh Khole authored
    • Charlie Jacobsen's avatar
      Two simple IPC tests are passing. Still getting mysterious hang. · c667fea5
      Charlie Jacobsen authored
      Working on moving Linux's mm into lcd's.
      I gave up trying to debug the hang. Confirmed the pages for the lcd's
      vm are the ones I expect. Turned on red zones. All tests are passing.
      Hang happens after insmod/rmmod of the lcd module about 10 - 20 times, it
      varies. Sometimes one core just silently dies / doesn't even respond to
      an NMI. Sometimes the ethernet driver complains (this could be an
      unrelated bug that was fixed upstream).
      Few things in this commit:
      Updated documentation in Documentation/lcd-domains/.
      Baby version of lib kernel, inside arch/x86/lcd-domains/liblcd.c.
      Unfortunately due to the recursive make, this needs to be textually
      included inside the modules destined for lcd's, for now.
      Added new test modules and modified directory structure and
      build system. See documentation in Documentation/lcd-domains.
      A few tweaks to the nmi handler to print a backtrace. May remove that in
      the future, as it's probably not safe to do inside an nmi handler (but if
      we're in that error state, we might be desperate to know what's happening ...).
      Changed interrupt handling in arch-dependent code. The KVM code we were using
      is probably wrong for 64-bit - it doesn't properly switch stacks, etc., which
      is super important for 64-bit and may be impossible to emulate in
      software. I think this could be stale code inside KVM, but not sure. Dune
      doesn't use it. KVM doesn't ack external interrutps on vm exit, so I think
      this interrupt emulation code is always skipped (at least for non-nested
      Instead, we're not ack'ing interrupts on exit, and letting the native code
      do the right thing, like Dune.
      I was thinking this might be the source of the bad hang (stack
      overflow, e.g.), but not true.
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <>
    • Muktesh Khole's avatar
      Integrated Async Code · d1d6b9a0
      Muktesh Khole authored
    • Charlie Jacobsen's avatar
      Except IPC, kliblcd fully tested. Everything is working. · 664628a4
      Charlie Jacobsen authored
      Documentation in Documentation/lcd-domains/...
      Loading, mapping, and running a module is working correctly, using
      all of the capability code that interposes on each operation (mapping,
      freeing pages, etc.).
      cptr allocation and indexing into cspaces is working correctly.
      IPC testing and debugging is coming next.
    • Charlie Jacobsen's avatar
      Muktesh's capabilities fully incorporated. Capsicum-style enter/exit. · 6ee9a51f
      Charlie Jacobsen authored
      Builds, but not fully tested. Good tests for capability subsystem, some tests
      for kliblcd.
      Non-isolated kernel threads can "enter" the lcd system by doing
      klcd_enter / klcd_exit. They can create other lcd's, set them up, etc. They
      use the same interface that regular lcd's will use, so such code could be
      moved to an lcd, as we had planned. Will document this in Documentation folder
      tomorrow ( == today ).
      Capability system does checks now when a capability is deleted/revoked: for
      example, if it's for a page, the microkernel checks if the page is mapped, and
      unmaps it. If the last capability goes away, the page is freed. Documentation
      is in Documentation/lcd-domains/cap.txt.
      IPC code is in place, but not tested yet (pray for me).
      Debug is taking some time. Sometimes requires a power cycle which adds an
      extra 5 - 10 minutes. Build is slow the first time after reboot. Give me a user
      level program and I'll debug it in 30 seconds! argc
      Main arch-independent files:
          include/lcd-domains/kliblcd.h, types.h
             This is what non-isolated kernel code should include to use the
             kliblcd interface to the microkernel.
          virt/lcd-domains/main.c, kliblcd.c, cap.c, ipc.c, internal.h
             The microkernel, broken up into pieces.
             The tests, in progress.
      Some old files are still hanging around in virt/lcd-domains and will be
      incorprated/cleaned up soon.
      I couldn't squash over the merge from the decomposition branch, so there's a
      bunch of junk commits coming over. (I should've just copied Muktesh's files.)
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <>
    • Charlie Jacobsen's avatar
      Missed common.h. · 11325d8e
      Charlie Jacobsen authored
    • Charlie Jacobsen's avatar
      Partially through starting ipc. · 9edaa6a6
      Charlie Jacobsen authored
      Will probably switch back to single lcd, multiple addr space, etc.
      Merging muktesh's caps.
    • Charles Jacobsen's avatar
      Support for multiple threads in lcd (limited). Major clean up of code. · ded1cd32
      Charles Jacobsen authored
      In line with more recent design discussions, we now have (limited)
      support for running multiple threads inside an lcd.
      Each thread will have its own hardware vm, but share a guest
      physical address space and cspace.
      It's limited for now because threads cannot handle interrupts/exceptions
      internally in the lcd. This will require a per-thread TSS (much like
      Linux's per-core TSS/interrupt stack). I removed the gdt/idt/tss for
      now (Cf. with Dune, they don't use gdt/idt/tss) and will tackle that later
      after finishing more important stuff.
      I have only tested the code for running one hardware vm inside an lcd. Some
      code is missing proper locking for the future when we have multiple threads
      inside an lcd. I'm leaving this for now.
      The microkernel uses a simple bitmap for guest physical page allocation.
      Removed blob loading - code is set up for running modules exclusively.
      See the headers and Documentation/lcd-domains for more info.
      I put a flag at the top of files that are not currently in use, and will
      probably be deleted/incorporated later.
    • Charlie Jacobsen's avatar
    • Charles Jacobsen's avatar
      Fixed build errors, module init tools. · 0f690a86
      Charles Jacobsen authored
      Getting nasty runtime bugs though.
    • Charlie Jacobsen's avatar
      Finished the majority of the arch-indep code changes needed. · d71afeb4
      Charlie Jacobsen authored
      Refactored lcd's into lcd and lcd_thread. Still need to test/update
    • Charlie Jacobsen's avatar
      Separated lcd into container and thread objects. · dbc4e40c
      Charlie Jacobsen authored
      Updated code. Removed gdt/tss/idt for now. Added doc directory
      and some initial doc.
    • Charles Jacobsen's avatar
      Starting ipc. · 4d60fe0b
      Charles Jacobsen authored
    • Charlie Jacobsen's avatar