Skip to content
  • David Johnson's avatar
    Add memcache; and a set of builtin x86 mem access ops in xen_vm backend. · b5a40933
    David Johnson authored
    Summary: 1) I added our own internal memory access operations for the
    xen_vm backend, so we no longer need libvmi nor xenaccess.  This is
    very good; there was a lot of interface work that would have been
    required to optimize the gap between us and libvmi, to remove duplicated
    work that both libraries do.  Moreover, the new builtin mem access
    functions cache mmaps of the VM across debug exceptions... a big
    performance win for live analysis.  2) I refactored the xen_vm
    backend once again to split out all the memory access functions.  Thus,
    you can theoretically build with support for libvmi, xenaccess, and
    the builtin stuff (builtin is always built); and then choose which to
    use at runtime.  I haven't tested any of the xenaccess code... we
    would only use that on older Xen 3.x VMs, and we're beyond that
    mostly.  But switching between libvmi and the builtin stuff is fine.
    
    Details:
    
    libvmi caches v2p translations on a per-pid basis; but mmap's pages via
    libxc on each read/write.  This is a big hit for live analyses.  The
    builtin functions cache both v2p translations and mmaps.  The caches
    *can* be cleared each debug exception or user target_pause(); but
    realistically the physical page cache especially never needs to be
    cleared (unless you're dealing with a VM that is dynamically shrinking/
    growing its physical page allocation, i.e. via ballooning) -- we just
    don't bother with that case right now.  I believe it's possible, but
    it's for another day!
    
    I added a generic x86/x86_64 page table walker that can be used for
    any VM.  A helper function takes several x86 registers as input,
    figures out what paging mode (if any) the CPU is in; and walks the
    page tables to translate virtual to physical.  I believe it supports
    all the possible modes: native 64-bit; 32-bit; PAE; PSE; PSE-36;
    PSE-40 (although it doesn't check for the AMD cpu markings like it
    should for PSE-40); it supports hugepages too (of the right sizes,
    depending on mode).
    
    The memcache API provides a tagged cache of v2p translations and
    virtual and/or physical mmaps.  Tag is just an index into the cache;
    the xen_vm builtin memops use page_dir phys addrs.
    
    The builtin xen_vm memops use the target-generic x86 v2p function,
    and memcache, to read/write phys/virt target memory, and to cache
    the translations.  This code should be trivially stealable or
    refactorable to support another VM backend like for KVM; but I don't
    know what else the interface needs to support, so for now I kept
    that local in the xen_vm backend; the builtin, xenaccess, and
    libvmi memops backends implement that interface.
    
    As far as how we cache and ensure mmaps are valid... All mmaps are
    mmaps of physical pages, *but* we translate contiguous virt ranges
    to single mmaps if requested, and we can cache them.  It's just that
    for that case, the v2p mapping could obvious change, and we don't
    provide any code for verifying existing mappings.  This could be done,
    and it's still a win (because the physical pages making up the page
    table remain mapped, so walking them is faster than munmap'ing virt
    ranges unnecessarily).  The easiest thing to do is cache individual
    physical pages.  So, we support both styles at the moment, but
    don't yet validate anything.
    
    From the user interface side... I initially had started down the
    path of exposing mmaps to the user via the Target and Value APIs.
    This would have 1) involved lots of code change; and 2) by giving
    the user raw mmap addresses, we force them to deal with the mmap
    validity problem *for mmap'd virt ranges*.  It's only virt ranges
    that are concerning, because if we hand those mmaps out, we *have*
    to validate the v2p mapping underneath.  We don't have to do that
    for phys range mmaps.  Since this is not clear, and since exposing
    it to the user is harder, we don't bother for now.
    
    That means that the user interface remains the same; but there are
    memcpys between the mmaps and Value buffers.  Future work, perhaps.
    
    I think that's it!
    b5a40933