1. 25 Oct, 2016 40 commits
    • Charlie Jacobsen's avatar
      Simple printk, adapted from Linux's vsprintf. · 41e1d4fd
      Charlie Jacobsen authored
      Updated liblcd and the microkernel. Doesn't support
      all escapes, like %p.
      
      Updated documentation. Stack protection should be turned off.
      41e1d4fd
    • Muktesh Khole's avatar
      Async Code, builds successfully - untested · 71bacb46
      Muktesh Khole authored
      71bacb46
    • Charlie Jacobsen's avatar
      Two simple IPC tests are passing. Still getting mysterious hang. · c667fea5
      Charlie Jacobsen authored
      Working on moving Linux's mm into lcd's.
      
      I gave up trying to debug the hang. Confirmed the pages for the lcd's
      vm are the ones I expect. Turned on red zones. All tests are passing.
      Hang happens after insmod/rmmod of the lcd module about 10 - 20 times, it
      varies. Sometimes one core just silently dies / doesn't even respond to
      an NMI. Sometimes the ethernet driver complains (this could be an
      unrelated bug that was fixed upstream).
      
      Few things in this commit:
      
        1
      =====
      
      Updated documentation in Documentation/lcd-domains/.
      
        2
      =====
      
      Baby version of lib kernel, inside arch/x86/lcd-domains/liblcd.c.
      Unfortunately due to the recursive make, this needs to be textually
      included inside the modules destined for lcd's, for now.
      
        3
      =====
      
      Added new test modules and modified directory structure and
      build system. See documentation in Documentation/lcd-domains.
      
        4
      =====
      
      A few tweaks to the nmi handler to print a backtrace. May remove that in
      the future, as it's probably not safe to do inside an nmi handler (but if
      we're in that error state, we might be desperate to know what's happening ...).
      
        5
      =====
      
      Changed interrupt handling in arch-dependent code. The KVM code we were using
      is probably wrong for 64-bit - it doesn't properly switch stacks, etc., which
      is super important for 64-bit and may be impossible to emulate in
      software. I think this could be stale code inside KVM, but not sure. Dune
      doesn't use it. KVM doesn't ack external interrutps on vm exit, so I think
      this interrupt emulation code is always skipped (at least for non-nested
      VMs).
      
      Instead, we're not ack'ing interrupts on exit, and letting the native code
      do the right thing, like Dune.
      
      I was thinking this might be the source of the bad hang (stack
      overflow, e.g.), but not true.
      
      Conflicts:
      	include/linux/sched.h
      	kernel/watchdog.c
      	virt/lcd-domains/lcd-cspace-tests2.c
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <vikram186@gmail.com>
      c667fea5
    • Muktesh Khole's avatar
      Integrated Async Code · d1d6b9a0
      Muktesh Khole authored
      d1d6b9a0
    • Charlie Jacobsen's avatar
      Except IPC, kliblcd fully tested. Everything is working. · 664628a4
      Charlie Jacobsen authored
      Documentation in Documentation/lcd-domains/...
      
      Loading, mapping, and running a module is working correctly, using
      all of the capability code that interposes on each operation (mapping,
      freeing pages, etc.).
      
      cptr allocation and indexing into cspaces is working correctly.
      
      IPC testing and debugging is coming next.
      664628a4
    • Charlie Jacobsen's avatar
      Muktesh's capabilities fully incorporated. Capsicum-style enter/exit. · 6ee9a51f
      Charlie Jacobsen authored
      Builds, but not fully tested. Good tests for capability subsystem, some tests
      for kliblcd.
      
      Non-isolated kernel threads can "enter" the lcd system by doing
      klcd_enter / klcd_exit. They can create other lcd's, set them up, etc. They
      use the same interface that regular lcd's will use, so such code could be
      moved to an lcd, as we had planned. Will document this in Documentation folder
      tomorrow ( == today ).
      
      Capability system does checks now when a capability is deleted/revoked: for
      example, if it's for a page, the microkernel checks if the page is mapped, and
      unmaps it. If the last capability goes away, the page is freed. Documentation
      is in Documentation/lcd-domains/cap.txt.
      
      IPC code is in place, but not tested yet (pray for me).
      
      Debug is taking some time. Sometimes requires a power cycle which adds an
      extra 5 - 10 minutes. Build is slow the first time after reboot. Give me a user
      level program and I'll debug it in 30 seconds! argc
      
      Main arch-independent files:
      
          include/lcd-domains/kliblcd.h, types.h
      
             This is what non-isolated kernel code should include to use the
             kliblcd interface to the microkernel.
      
          virt/lcd-domains/main.c, kliblcd.c, cap.c, ipc.c, internal.h
      
             The microkernel, broken up into pieces.
      
          virt/lcd-domains/tests/
      
             The tests, in progress.
      
      Some old files are still hanging around in virt/lcd-domains and will be
      incorprated/cleaned up soon.
      
      I couldn't squash over the merge from the decomposition branch, so there's a
      bunch of junk commits coming over. (I should've just copied Muktesh's files.)
      
      Conflicts:
      	drivers/Kconfig
      	drivers/lcd-cspace/test.h
      	include/lcd-domains/cap.h
      	include/lcd-prototype/lcd.h
      	include/lcd/console.h
      	include/lcd/elfnote.h
      	include/linux/init_task.h
      	include/linux/module.h
      	include/linux/sched.h
      	virt/lcd-domains/cap.c
      	virt/lcd-domains/ipc.c
      	virt/lcd-domains/lcd-cspace-tests2.c
      Resolved-by: Vikram Narayanan's avatarVikram Narayanan <vikram186@gmail.com>
      6ee9a51f
    • Charlie Jacobsen's avatar
      Missed common.h. · 11325d8e
      Charlie Jacobsen authored
      11325d8e
    • Charlie Jacobsen's avatar
      Partially through starting ipc. · 9edaa6a6
      Charlie Jacobsen authored
      Will probably switch back to single lcd, multiple addr space, etc.
      
      Merging muktesh's caps.
      9edaa6a6
    • Charles Jacobsen's avatar
      Support for multiple threads in lcd (limited). Major clean up of code. · ded1cd32
      Charles Jacobsen authored
      In line with more recent design discussions, we now have (limited)
      support for running multiple threads inside an lcd.
      
      Each thread will have its own hardware vm, but share a guest
      physical address space and cspace.
      
      It's limited for now because threads cannot handle interrupts/exceptions
      internally in the lcd. This will require a per-thread TSS (much like
      Linux's per-core TSS/interrupt stack). I removed the gdt/idt/tss for
      now (Cf. with Dune, they don't use gdt/idt/tss) and will tackle that later
      after finishing more important stuff.
      
      I have only tested the code for running one hardware vm inside an lcd. Some
      code is missing proper locking for the future when we have multiple threads
      inside an lcd. I'm leaving this for now.
      
      The microkernel uses a simple bitmap for guest physical page allocation.
      
      Removed blob loading - code is set up for running modules exclusively.
      
      See the headers and Documentation/lcd-domains for more info.
      
      I put a flag at the top of files that are not currently in use, and will
      probably be deleted/incorporated later.
      ded1cd32
    • Charlie Jacobsen's avatar
    • Charles Jacobsen's avatar
      Fixed build errors, module init tools. · 0f690a86
      Charles Jacobsen authored
      Getting nasty runtime bugs though.
      0f690a86
    • Charlie Jacobsen's avatar
      Finished the majority of the arch-indep code changes needed. · d71afeb4
      Charlie Jacobsen authored
      Refactored lcd's into lcd and lcd_thread. Still need to test/update
      tests.
      d71afeb4
    • Charlie Jacobsen's avatar
      Separated lcd into container and thread objects. · dbc4e40c
      Charlie Jacobsen authored
      Updated code. Removed gdt/tss/idt for now. Added doc directory
      and some initial doc.
      dbc4e40c
    • Charles Jacobsen's avatar
      Starting ipc. · 4d60fe0b
      Charles Jacobsen authored
      4d60fe0b
    • Charlie Jacobsen's avatar
    • Charlie Jacobsen's avatar
      Almost all mk code in place. · 0c27d3a5
      Charlie Jacobsen authored
      Just need to handle failures gracefully.
      0c27d3a5
    • Charlie Jacobsen's avatar
      Finished a great big chunk of the microkernel. · 342151de
      Charlie Jacobsen authored
      IPC in place. Most api calls in place, finishing implementation of
      api kill / tear down of lcds.
      342151de
    • Charlie Jacobsen's avatar
      f69072d9
    • Charlie Jacobsen's avatar
      62815dba
    • Charlie Jacobsen's avatar
      42ca2030
    • Muktesh Khole's avatar
      added memory reclaim feature · ec790197
      Muktesh Khole authored
      ec790197
    • Muktesh Khole's avatar
      added revoke feature · 32ecf3ff
      Muktesh Khole authored
      32ecf3ff
    • Muktesh Khole's avatar
      added feature to grant capabilities · e1045ef9
      Muktesh Khole authored
      e1045ef9
    • root's avatar
      Working lookup, insert, and delete · a5d6ea85
      root authored
      a5d6ea85
    • Charlie Jacobsen's avatar
      First draft of lcd prototype in place. · f9c94ba0
      Charlie Jacobsen authored
      drivers/lcd-prototype contains a README with build and run information,
      the overall design, code layout, etc.
      
      include/lcd-prototype/lcd.h contains the struct lcd data structure,
      included in struct task_struct.
      
      The old client/server test is still present, inside
      drivers/lcd-prototype/test/old, but is not runnable.
      
      I removed the old ping-pong test, and the old include/lcd/lcd.h header.
      
      Aside from the lcd.h header and makefile/kconfig modifications, Anton's draft
      is still intact, inside virt/lcd. This draft of the lcd prototype does not
      have any dependencies on Anton's code right now.
      f9c94ba0
    • Charlie Jacobsen's avatar
      First draft of ipc done. · 547aeabd
      Charlie Jacobsen authored
      Abandoned many-to-many.
      547aeabd
    • Charlie Jacobsen's avatar
      Finished ipc send, recv, call, reply. · 173b9e47
      Charlie Jacobsen authored
      173b9e47
    • Charlie Jacobsen's avatar
      Adjusted cap tests, starting ipc. · 5b42e788
      Charlie Jacobsen authored
      5b42e788
    • Charlie Jacobsen's avatar
      Starting api code. · 25949c4a
      Charlie Jacobsen authored
      25949c4a
    • Charlie Jacobsen's avatar
      Restructuring prototype code. · 963eb782
      Charlie Jacobsen authored
      963eb782
    • Charlie Jacobsen's avatar
      Basic lcd module create, run, and destroy. · e0193fa4
      Charlie Jacobsen authored
      This code is ugly, but it's working.
      
      Tested with basic module, and appears to be working
      properly. I will soon incorporate the patched
      modprobe into the kernel tree, and then this code
      will be usable by everyone.
      
      The ipc code is still unimplemented. The only
      hypercall handled is yield. Also note that other
      exit conditions (e.g. external interrupt) have not
      been fully tested.
      
      Overview:
      -- kernel code calls lcd_create_as_module with
         the module's name
      -- lcd_create_as_module loads the module using
         request_lcd_module (request_lcd_module calls
         the patched modprobe to load the module, and
         the patched modprobe calls back into the lcd
         driver via the ioctrl interface to load the
         module)
      -- lcd_create_as_module then finds the loaded
         module, spawns a kernel thread and passes off
         the module to it
      -- the kernel thread initializes the lcd and
         maps the module inside it, then suspends itself
      -- lcd_run_as_module wakes up the kernel thread
         and tells it to run
      -- lcd_delete_as_module stops the kernel thread
         and deletes the module from the host kernel
      
      File-by-file details:
      
      arch/x86/include/asm/lcd-domains-arch.h
      arch/x86/lcd-domains/lcd-domains-arch-tests.c
      arch/x86/lcd-domains/lcd-domains-arch.c
      -- lcd was not running in 64-bit mode, and my
         checks had one subtle bug
      -- fixed %cr3 load to properly load vmcs first
      -- fixed set program counter to use guest virtual
         rather than guest physical address
      
      include/linux/sched.h
      -- added struct lcd to task_struct
      
      include/linux/init_task.h
      -- lcd pointer set to null when task_struct is
         initialized
      
      include/linux/module.h
      kernel/module.c
      -- made init_module and delete_module system calls
         callable from kernel code
      -- available in module.h via do_sys_init_module and
         do_sys_delete_module
      -- simply moved the majority of the guts of the
         system calls into a non-system call, exported
         routine
      -- take an extra flag, for_lcd; when set, the init
         code skips over running (and deallocating) the
         module's init code, and the delete code skips
         over running the module exit
      -- system calls from user code set for_lcd = 0; this
         ensures existing code still works
      
      include/linux/kmod.h
      kernel/kmod.c
      kernel/sysctl.c
      -- changed __request_module to __do_request_module; takes
         one extra argument, for_lcd
      -- __request_module   ==>  __do_request_module with for_lcd = 0
      -- request_lcd_module ==>  __do_request_module with for_lcd = 1
      -- call_modprobe conditionally uses lcd_modprobe_path, the path
         to a patched modprobe accessible via sysfs
      
      include/lcd-domains/lcd-domains.h
      -- added lcd status enum; see source code doc
      -- three routines for creating/running/destroying
         lcd's that use modules; see source code doc
      
      include/uapi/linux/lcd-domains.h
      -- added interface defns for patched modprobe to call into
         lcd driver for module init; lcd driver loads
         module (via slightly refactored module.c code) on behalf
         of modprobe
      
      virt/lcd-domains/lcd-domains.c
      -- implementation of routines for modules inside lcd's
      -- implementation of module init / delete for lcd's
         (uses patched module.c code)
      
      virt/lcd-domains/Kconfig
      virt/lcd-domains/Makefile
      virt/lcd-domains/lcd-module-load-test.c
      virt/lcd-domains/lcd-tests.c
      -- added test module for lcd module code
      -- test runs automatically when lcd module is inserted
      e0193fa4
    • Anton Burtsev's avatar
      Create LCD code inside LCD API layer · ea133a9d
      Anton Burtsev authored
       -- Still need an actual function to load the module into LCD
      ea133a9d
    • Anton Burtsev's avatar
      d3811abf
    • Anton Burtsev's avatar
      e7a70c32
    • Anton Burtsev's avatar
      f2609e2e
    • Anton Burtsev's avatar
    • Anton Burtsev's avatar
      First API call: create sync endpoint · 75c72eaa
      Anton Burtsev authored
      75c72eaa
    • Anton Burtsev's avatar
      LCD API client and server code · fc2675ff
      Anton Burtsev authored
        -- LCD export an API to its domains that is accessible via
        capability invocations (well, syscalls if you like)
      
        -- Kernel runs a thread that implement this API, e.g. serves
        capability invocations
      fc2675ff
    • Anton Burtsev's avatar
      Simple, list-based capability allocator · 75340d2a
      Anton Burtsev authored
        -- I've split capability allocation and cspaces, this makes much more
           sense -- complex domains can implement custom allocation policies,
           simple domains go with a static set of caps
        -- This is a list-based (zone like) allocator
      75340d2a
    • Anton Burtsev's avatar
      Kicked out Muktesh's code, wrote my own stub · 6085bcd5
      Anton Burtsev authored
      -- Mine is primitive but works, Muktesh will have to come
         back and rewrite
      -- Cspace is an array, only insert and lookup are supported
      6085bcd5