1. 03 Apr, 2017 2 commits
  2. 02 Apr, 2017 2 commits
  3. 19 Dec, 2016 2 commits
    • Charles Jacobsen's avatar
      pmfs: mount: Fix pmfs dispatch loop, add pre/post hook headers, and more. · 05521451
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
         -- Needed to move ASYNC macro inside loop in pmfs dispatch loop
         -- Add pre_hook.h and post_hook.h headers to pmfs source files
         -- Fix get_current to return current_task global in liblcd/resolve.c,
            rather than use %gs percpu variable (was causing page fault)
      
      Switching gears for a moment to fix some things in libasync.
      05521451
    • Charles Jacobsen's avatar
      pmfs: mount: Fixed heap exhaustion due to too many stacks. · 30b2f271
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      You have to be careful where you put your asyncs, especially
      with poll recv. If you wrap the call to poll recv in an async,
      you will consume lots of memory. I moved the async so it surrounds
      the dispatch function only.
      
      In the process, to debug, I updated libasync with THCDumpAllStacks
      (I was trying to see what all the stacks were - they were all coming
      from poll recv).
      
      NOTE: While debugging, I tweaked the microkernel dump stack code
      to work for THCDumpAllStacks (continue walking until we hit a
      frame or return address). If you don't make this change,
      THCDumpAllStacks will not crash, but it won't be informative either
      (the stack walk will stop too soon).
      
      I'm able to run at least 100 iterations now of mount/unmount without
      any problems.
      30b2f271
  4. 26 Oct, 2016 8 commits
    • Charles Jacobsen's avatar
      test-v2: updated string_example builds, need to tweak interface. · 5c5c6f88
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Resource node lookup is insufficient. Caller doesn't always know
      how to calculate offsets (are the addresses in the node virtual
      or physical?).
      5c5c6f88
    • Charles Jacobsen's avatar
      test-v2: Fix some allocator bugs, fix cptr cache bug. · 3710d903
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      I was using PAGE_SIZE instead of PAGE_SHIFT in some places.
      
      Also, non-isolated code sets up the cptr cache for isolated
      code; but the size of the mutex is different for each environment
      (16 bytes versus 8 bytes). So, I hacked libcap to take care of
      the difference (using padding).
      3710d903
    • Charles Jacobsen's avatar
      test-v2: Add rb tree code to liblcd, fix unresolved mutex symbols. · 87fbf2ff
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      There is a limitation with my preprocessor hacks: If a header
      invokes a function inside another static inline function, I
      can't modify the call site (that is the key) using a macro
      undef/re-define. Instead, I need to just define the function
      in another source file.
      
      Another limitation (yet to be encountered) is a problematic
      macro function invoked in a header. I can't re-define the
      macro function before the call site using the post hooks. Will
      cross that bridge when I get there.
      87fbf2ff
    • Charles Jacobsen's avatar
      build-refactor-2: Set up liblcd build, and libcap build for liblcd. · 2c945e8c
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      The libcap build won't work quite yet, or won't be set up properly
      for liblcd rather. I need to tweak the libcap kernel build and/or
      add a new platform.
      2c945e8c
    • Charles Jacobsen's avatar
      build-refactor: New arch directory, header shuffling. · 1abb5ae7
      Charles Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Following the Linux kernel src layout, moved the vt-x code
      into a separate arch folder. Moved headers around:
      
         -- Microkernel/kliblcd arch-independent headers are in
            include/lcd_domains
         -- Microkernel arch-dependent headers are in arch/x86/include
         -- liblcd headers are in include/liblcd
      
      Switched some file/dir names to use `_` instead of `-`.
      1abb5ae7
    • Charlie Jacobsen's avatar
      libcap-integration: Start simultaneous interface refactoring. · 09650d18
      Charlie Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      09650d18
    • Charlie Jacobsen's avatar
      Major overhaul of build process. · 8198c2fb
      Charlie Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Full kernel build no longer required. Yay! This should
      cut down on dev time a lot.
      
      I moved all of the LCD source into $(kernel-src)/lcd-domains/,
      so it's all in one spot. There is now a top-level makefile in
      there that triggers building liblcd, the microkernel, and the
      examples. This is built as an *external* build now, even
      though the directory is in the kernel source. The build now takes
      under a minute to do everything LCD related.
      
      This should also make verification easier in the future (e.g.
      building with clang) if we aren't ensnared in the kernel
      source.
      
      Of course, to use the microkernel and examples, you have to
      build the patched kernel and install it. But now when you
      make a few lines of changes in e.g. an example, you don't have
      to trigger a top-level kernel build to rebuild it. Running
      the full kernel build takes on average about 3 - 4 minutes
      (some files are generated everytime, linking is done, and so
      on), and can take upwards of 30 minutes for a full build if you
      re-config'd.
      
      Which brings me to my other change: no more config for LCDs
      in menuconfig. If we create menu entries for every example
      and so on, we end up changing the config too often, and this
      triggers full kernel rebuilds == waste of time. We can use
      macros by setting them via compiler flags (e.g., -DSOME_FLAG).
      Furthermore, it wasn't making sense to me to do conditional
      compilation for LCD support (we always want to compile for that).
      Yes, changes aren't clearly delineated with macros, but you can
      see changes made by just doing 'git diff v3.10.14 some-file-or-dir'.
      
      The wiki has been fully updated with instructions for building,
      and other relevant parts (updated paths to files).
      
      I also took the opportunity to clean up some old stuff lying around
      that is dead (like lcdguest). I incorporated all of the documentation
      in Documentation/lcd-domains into the wiki so it's all in one
      spot now (including some helpful debug tips).
      8198c2fb
    • Charlie Jacobsen's avatar
      Adds full functioning example with glue code. · 424ac36d
      Charlie Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Example is in virt/lcd-domains/test-mods/glue-example. It consists
      of a fake minix and fake vfs. The original "unmodified" code is
      in minix/original/main.c, vfs/original/main.c, and include/vfs.h.
      The glue code (written by hand) is in minix/glue and vfs/glue. The
      IDL is in idl/.
      
      You can build the fake minix and vfs for isolation or as regular
      modules. To build for isolation, run `make menuconfig' and under
      Test Modules --> Example Exercising IDL and Glue.
      
      To run minix and vfs in isolation, install the "boot" module,
      lcd-test-mod-glue-example-boot.ko. (Its code is under isol-boot/main.c.)
      
      The example will print status messages to the kernel logs so you
      can see a trace of the interaction.
      
      A few hacks were necessary to fully exercise all of the code. For example,
      the fake vfs is the one that invokes new_file and rm_file, in the middle
      of its dispatch loop.
      
      The code for the trampolines in vfs/glue/vfs_caller.{c,lds} is probably
      the ugliest part. That required a little bit of trickery and low-level
      hacking.
      424ac36d
  5. 25 Oct, 2016 2 commits
    • Charlie Jacobsen's avatar
      Basic 'data store' code added to libkernel. · ef040dc7
      Charlie Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Just the capability code adapted for use inside the libkernel.
      
      Basic tests passing. Coalesced a lot of the liblcd tests into
      one spot (in liblcd/lcd/test.c).
      ef040dc7
    • Charlie Jacobsen's avatar
      Linux slab allocator and page allocator inside LCDs. · 35ed57d7
      Charlie Jacobsen authored and Vikram Narayanan's avatar Vikram Narayanan committed
      Single threaded, no locks, no fancy NUMA/percpu.
      
      Passing some simple examples. Added a memory management
      example module, in test-mods/mm, that exercises a lot
      of this new code.
      
      1:
      
      I moved in and adapted our existing guest virtual
      paging code from kliblcd.c. I'm using statically
      allocated bitmaps and arrays for tracking allocations
      in the guest virtual and physical address spaces.
      Using identity mapping for ease. (I decided not to
      use Linux's page allocator since it's too intertwined
      with the boot process - percpu variables, freeing
      init mem, boot allocator, all kinds of complexity ...)
      It might not be too hard to reimplement the buddy
      allocator algorithms, since I had to include a
      statically allocated array of struct pages anyway.
      
      I've set aside about 16 MBs for dynamic page allocations,
      but this can be changed using macros. You can allocate
      1, 2, 4, 8, etc. pages at a time. (The slab allocator
      requires this.)
      
      2:
      
      I finally broke down and set up boot info pages - 4
      boot pages right now, can be adjusted with a macro.
      Whoever boots the lcd needs to pack in information about the
      lcd's address space, initial cspace, and so on. 4 pages
      is enough to pack in information for larger modules
      like the mm example.
      
      3:
      
      I moved liblcd to a separate directory, and hacked the
      build system so that we can build liblcd as a static
      library and link it with example modules.
      
      liblcd/ contains lcd/, which has code for interacting
      with the microkernel and my simple page allocator.
      
      The Linux slab allocator is inside mm/, and some
      needed dependencies are in lib/. I made very few
      changes to the source code itself, but used some
      preprocessor/compiler hacks to make everything work. See
      Documentation/lcd-domains/liblcd.txt. I elided all of
      the locking and made it single core, single NUMA node.
      It's possible we'll see some bigs in the future, in code I haven't
      excerised yet (will probably manifest themselves as
      page faults).
      
      Ideally, we should have a separate tree for liblcd
      and building modules. That way we can avoid some of these
      hacks (maybe not all).
      
      4:
      
      Updated a lot of the documentation in
      Documentation/lcd-domains/.
      35ed57d7