GitLab

Define helpers for __get_free_page(s) and free_page(s)
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Also remove pmfs from build
Signed-off-by: Vikram Narayanan <vikram186@gmail.com>

Going to take a bit more work because the struct module is no
longer directly available on the host side, so starting a new branch ...

Updated kliblcd, liblcd, and string example.

Couple duplicate memory interval tree inserts/deletes were leading
to some use-after-frees/page faults. Cleaned that up. Added some
resource tree debug code along the way.

Also, caught something subtle (noted in code). I didn't consider
the following scenario: Heap tries to allocate pages; the allocator
notices it needs to bring in more fresh pages, and notifies the
heap (via a callback); the heap allocs the fresh pages (from the
microkernel), maps them, and inserts those pages into the memory interval
tree; the memory interval tree kmallocs a tree node; kmalloc
calls back into the heap to grow a slab cache.

That last bit could be a potential problem (recursive call back
into the heap before we finish the original call). Lucky for me,
I designed the heap/allocator so that (1) the pages from the first
call are already marked as in use (not on a free list); (2) the
fresh pages are mapped first *before* inserting the corresponding
cptr into the memory interval tree.

The Linux kernel deals with these same recursive issues (they
resolve them using special GFP_ flags so that you don't get
recursion). In my case, the recursion is risky, but works.

Caught a bug in heap allocator - wasn't setting an out
parameter (that design pattern I use - out params - will be the
death of me).

Updated max page allocation order to match the host. The microkernel
uses the host page allocator, so that is the only limiting
factor now (so e.g., on x86_64, you can now allocate up to
2^10 = 1024 pages = 4 MBs inside the LCD, and this is the maximum
you could allocate in non-isolated code too). Before, the allocation
size was limited to about 2^6 = 64 pages = 256 KBs.

I needed to re-think the free lists initialization.
Also bug in demand paging.

Heap and kmalloc now initialize, but I still crash afterward.

There is a limitation with my preprocessor hacks: If a header
invokes a function inside another static inline function, I
can't modify the call site (that is the key) using a macro
undef/re-define. Instead, I need to just define the function
in another source file.

Another limitation (yet to be encountered) is a problematic
macro function invoked in a header. I can't re-define the
macro function before the call site using the post hooks. Will
cross that bridge when I get there.

Host can successfully load LCD, using new interval tree/memory
object logic. (Still get EPT fault while booting LCD.)

Some fixes/changes:

    1 - Need to use page-level granularity for memory objects.
        Unlike alloc_pages, vmalloc is page-granularity (rather
        than power-of-2 page granularity).
    2 - The kernel module region base virtual address needs to
        match the host's upper 2GB address, because of how address
        calculations are done internally and because we use the
        host module loader with no linkage changes (it's possible,
        but not worth it right now). Add assertion to make sure
        this is the case.
    3 - Needed to make LCD the owner of its cspace (set owner
        pointer).

It seemed dumb to have the generalized page allocator code
repeatedly call out to the user to get memory for metadata.
For one thing, it was buggy. For another, in many use cases
we can alloc the metadata in one shot.

RAM map allocator partially started.

This is kind of like an mmap region, but only for RAM. (The guest
virtual page tables will be configured for write back).

Use case: You get a RAM capability from some other guy, and you
want to map it into your physical/virtual address space. This is
reasonable to figure out manually for big RAM chunks that are
mapped one/two times. But for little tedious ones (string sharing),
it's helpful to have an allocator to assist and track free parts
of guest physical.

The RAM map region uses very course-grained physical address space
allocation (minimum 64 MBs). This leads to a lot of internal
fragmentation, but it should be tolerable since the region is big.
It also means we have a smaller bit of metadata for tracking the
region.

vmalloc is not implemented for now.

Uses the "generalized" allocator to manage 16 MBs of guest physical
address space for a heap. Only the initial 4 MBs will be backed,
and host memory is sucked in at 4 MB granularity.

Some tuning of parameters will be necessary soon.

Easier to have caller pass memory object order and gpa base for
_lcd_mmap and _lcd_munmap, since in liblcd we can look those
things up easily.

I need to pause liblcd-v2 and redo the resource tree logic
for kliblcd. Now that I'm working on liblcd, I'm seeing the way
it should be.

Memory itree code initializes resource tree needed for
address -> cptr translation. Provides interface for other liblcd
code to add/remove from tree (page allocator will use that).

Removed redundant nr_pages_order field from resource tree node. Its
size can be computed from the interval tree node.

Filled in some simple functions that are part of the liblcd
interface. Some are no-ops for isolated code.