GitLab

Define helpers for __get_free_page(s) and free_page(s)
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Also remove spin_unlock_restore stub
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

checksum and DQL (dynamic queue limits) are used by ixgbe driver. Add this to
liblcd as this may be used by other modules too in the future.
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

   -- Needed to move ASYNC macro inside loop in pmfs dispatch loop
   -- Add pre_hook.h and post_hook.h headers to pmfs source files
   -- Fix get_current to return current_task global in liblcd/resolve.c,
      rather than use %gs percpu variable (was causing page fault)

Switching gears for a moment to fix some things in libasync.

ioremap_cache and friends just assume the memory is already
mapped in the address spaces, and does a phys -> virt translation.

request/release memory regions is a no-op inside the LCD. The
non-isolated glue for mount will need to invoke these functions.

We need a way to temporarily prevent external interrupts from
causing an LCD exit.

You have to be careful where you put your asyncs, especially
with poll recv. If you wrap the call to poll recv in an async,
you will consume lots of memory. I moved the async so it surrounds
the dispatch function only.

In the process, to debug, I updated libasync with THCDumpAllStacks
(I was trying to see what all the stacks were - they were all coming
from poll recv).

NOTE: While debugging, I tweaked the microkernel dump stack code
to work for THCDumpAllStacks (continue walking until we hit a
frame or return address). If you don't make this change,
THCDumpAllStacks will not crash, but it won't be informative either
(the stack walk will stop too soon).

I'm able to run at least 100 iterations now of mount/unmount without
any problems.

I forgot I was memset'ing it to 0 in slab.c (and it wasn't necessary).

Command line string parsing functions are sucked in.

kthread spawning and waitqueue functions are no-op'd for now (these
are for pmfs's log cleaner). In the future, could redef these to
use libasync.

Use our old mm codebase (based on v3.10) and push in changes that are required
to make it compile with v4.8's header files. To make this work I pulled in
3.10's slab_def.h to kick out 4.8's slab_def.h

The changes are not that significant
* Removed struct slab, struct slab_rcu (Use struct page instead)
* Few kmem_chg related APIs were changed
* Some minor optimizations

Merged from
45eed508 slab: remove SLAB_LIMIT
16025177 slab: remove kmem_bufctl_t
b1cb0982 slab: change the management method of free objects of the slab
a57a4988 slab: use __GFP_COMP flag for allocating slab pages
56f295ef slab: use well-defined macro, virt_to_slab()
68126702 slab: overloading the RCU head over the LRU for RCU free
07d417a1 slab: remove cachep in struct slab_rcu
1ea991b0 slab: remove nodeid in struct slab
ac2b54ed slab: remove colouroff in struct slab
0c3aa83e slab: change return type of kmem_getpages() to struct page
73293c2f slab: correct pfmemalloc check
276a2439 mm/slab: Give s_next and s_stop slab-specific names
e25839f6 mm/slab: Sharing s_next and s_stop between slab and slub
0fa8103b mm/slab: Fix drain freelist excessively
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Also remove pmfs from build
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

vmcs and resolve.c
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

- rename functions
- add them to mem.h
Signed-off-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>

Signed-off-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>

Signed-off-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>

- adhere to Charlie's recent code modifications
Signed-off-by: Abhiram Balasubramanian <abhiram@cs.utah.edu>

We need this for the dispatch loop. A receiver needs to be able
to poll on synchronous IPC channels (in addition to async
channels).

We developed an example to test a clang/llvm/smack build before;
that example is now incorporated with the rest of the examples.

To experiment with weak symbols, I moved a few of the redefines
into libdeker (lcd_enter, lcd_exit, lcd_printk). These override
the liblcd versions. I also left the lcd_sync_call and
lcd_sync_reply redefines in environment.c.

There were a couple more clang fixes - one in the trampoline
assembly. The gnu assembler accepts "movq mem, mem" as a pseudo
instruction, but clang does not.

Fixed the "make clean" target. Made some extra scripts so it's easier
to build larger components (e.g. the entire microkernel, or all of
libdeker - including the special liblcd build).

The "main" required for Smack is in a "runtime" object file (sort of
like the C runtime). This is in libdeker/deker_rt.c.

Finally, I tweaked the LCD test module configuration so that you can
build a subset of the test modules (and choose how to build them...

If the module's init isn't marked with __init, it doesn't go in
the .init.text section. I was starting the LCD at the module_init
address; but this is the address of the beginning of .init.text,
not necessarily the module's init function. If the module's init
function is marked with __init, then it was going at the beginning
of that section, and so I was getting lucky.

Now I'm using the struct module.init pointer. Shouldn't matter
whether module's init function is in .init or regular .text (core)
now.

I fought this thing until late in the morning. Since we're
working with a copy of the module image (.ko), some of the fields
in the struct module have to be updated (symbol and string table
fields). Then, everything works.

Still getting weird trace symbol for return address at the
bottom of the stack.

We unload the kernel module from the host after we've dup'd it
and mapped it in the LCD. This means the struct module the host
uses goes bye-bye, and address->symbol resolution for addresses
in the LCD's .ko will return "not found" or garbage if some other
.ko was loaded in its place.

Therefore, we need a way for non-isolated code to remember where
the struct module copy is. We store this in the lcd_arch so that
it can man-handle it when it does stack tracing.

Going to take a bit more work because the struct module is no
longer directly available on the host side, so starting a new branch ...

Basic load all the way through pmfs.

The solution for struct module for now is - this struct is
treated as caller allocated (non-isolated no longer has a copy
since I dup the module bits and then unload the module).

The tweaks we have to make to "unmodified" code are not pretty
(setting up the container structs), but they work for now.

New resource tree design appears to work. glue_example and
liblcd_test work.

Updated kliblcd, liblcd, and string example.

This allows me to get all information - base address, size,
cptr, and so on.

Missing lcd_get_utcb function; added it.

Another cptr_cache ABI issue - need to compile isolated code with
padding flag; works now.

Couple small bugs in microkernel (return value not initialized for
certain error case).

Couple duplicate memory interval tree inserts/deletes were leading
to some use-after-frees/page faults. Cleaned that up. Added some
resource tree debug code along the way.

Also, caught something subtle (noted in code). I didn't consider
the following scenario: Heap tries to allocate pages; the allocator
notices it needs to bring in more fresh pages, and notifies the
heap (via a callback); the heap allocs the fresh pages (from the
microkernel), maps them, and inserts those pages into the memory interval
tree; the memory interval tree kmallocs a tree node; kmalloc
calls back into the heap to grow a slab cache.

That last bit could be a potential problem (recursive call back
into the heap before we finish the original call). Lucky for me,
I designed the heap/allocator so that (1) the pages from the first
call are already marked as in use (not on a free list); (2) the
fresh pages are mapped first *before* inserting the corresponding
cptr into the memory interval tree.

The Linux kernel deals with these same recursive issues (they
resolve them using special GFP_ flags so that you don't get
recursion). In my case, the recursion is risky, but works.

Caught a bug in heap allocator - wasn't setting an out
parameter (that design pattern I use - out params - will be the
death of me).

Updated max page allocation order to match the host. The microkernel
uses the host page allocator, so that is the only limiting
factor now (so e.g., on x86_64, you can now allocate up to
2^10 = 1024 pages = 4 MBs inside the LCD, and this is the maximum
you could allocate in non-isolated code too). Before, the allocation
size was limited to about 2^6 = 64 pages = 256 KBs.