1. 09 Jul, 2010 1 commit
    • Kenji Kaneshige's avatar
      x86, ioremap: Fix incorrect physical address handling in PAE mode · ffa71f33
      Kenji Kaneshige authored
      
      
      Current x86 ioremap() doesn't handle physical address higher than
      32-bit properly in X86_32 PAE mode. When physical address higher than
      32-bit is passed to ioremap(), higher 32-bits in physical address is
      cleared wrongly. Due to this bug, ioremap() can map wrong address to
      linear address space.
      
      In my case, 64-bit MMIO region was assigned to a PCI device (ioat
      device) on my system. Because of the ioremap()'s bug, wrong physical
      address (instead of MMIO region) was mapped to linear address space.
      Because of this, loading ioatdma driver caused unexpected behavior
      (kernel panic, kernel hangup, ...).
      Signed-off-by: default avatarKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
      LKML-Reference: <4C1AE680.7090408@jp.fujitsu.com>
      Signed-off-by: default avatarH. Peter Anvin <hpa@linux.intel.com>
      ffa71f33
  2. 02 Feb, 2010 2 commits
    • Nick Piggin's avatar
      mm: purge fragmented percpu vmap blocks · 02b709df
      Nick Piggin authored
      
      
      Improve handling of fragmented per-CPU vmaps.  We previously don't free
      up per-CPU maps until all its addresses have been used and freed.  So
      fragmented blocks could fill up vmalloc space even if they actually had
      no active vmap regions within them.
      
      Add some logic to allow all CPUs to have these blocks purged in the case
      of failure to allocate a new vm area, and also put some logic to trim
      such blocks of a current CPU if we hit them in the allocation path (so
      as to avoid a large build up of them).
      
      Christoph reported some vmap allocation failures when using the per CPU
      vmap APIs in XFS, which cannot be reproduced after this patch and the
      previous bug fix.
      
      Cc: linux-mm@kvack.org
      Cc: stable@kernel.org
      Tested-by: default avatarChristoph Hellwig <hch@infradead.org>
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      --
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      02b709df
    • Nick Piggin's avatar
      mm: percpu-vmap fix RCU list walking · de560423
      Nick Piggin authored
      
      
      RCU list walking of the per-cpu vmap cache was broken.  It did not use
      RCU primitives, and also the union of free_list and rcu_head is
      obviously wrong (because free_list is indeed the list we are RCU
      walking).
      
      While we are there, remove a couple of unused fields from an earlier
      iteration.
      
      These APIs aren't actually used anywhere, because of problems with the
      XFS conversion.  Christoph has now verified that the problems are solved
      with these patches.  Also it is an exported interface, so I think it
      will be good to be merged now (and Christoph wants to get the XFS
      changes into their local tree).
      
      Cc: stable@kernel.org
      Cc: linux-mm@kvack.org
      Tested-by: default avatarChristoph Hellwig <hch@infradead.org>
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      --
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      de560423
  3. 21 Jan, 2010 1 commit
    • Yongseok Koh's avatar
      vmalloc: remove BUG_ON due to racy counting of VM_LAZY_FREE · 88f50044
      Yongseok Koh authored
      In free_unmap_area_noflush(), va->flags is marked as VM_LAZY_FREE first, and
      then vmap_lazy_nr is increased atomically.
      
      But, in __purge_vmap_area_lazy(), while traversing of vmap_are_list, nr
      is counted by checking VM_LAZY_FREE is set to va->flags.  After counting
      the variable nr, kernel reads vmap_lazy_nr atomically and checks a
      BUG_ON condition whether nr is greater than vmap_lazy_nr to prevent
      vmap_lazy_nr from being negative.
      
      The problem is that, if interrupted right after marking VM_LAZY_FREE,
      increment of vmap_lazy_nr can be delayed.  Consequently, BUG_ON
      condition can be met because nr is counted more than vmap_lazy_nr.
      
      It is highly probable when vmalloc/vfree are called frequently.  This
      scenario have been verified by adding delay between marking VM_LAZY_FREE
      and increasing vmap_lazy_nr in free_unmap_area_noflush().
      
      Even the vmap_lazy_nr is for checking high watermark, it never be the
      strict watermark.  Although the BUG_ON condition is to prevent
      vmap_lazy_nr from being negative, vmap_lazy_nr is signed variable.  So,
      it could go down to negative value temporarily.
      
      Consequently, removing the BUG_ON condition is proper.
      
      A possible BUG_ON message is like the below.
      
         kernel BUG at mm/vmalloc.c:517!
         invalid opcode: 0000 [#1] SMP
         EIP: 0060:[<c04824a4>] EFLAGS: 00010297 CPU: 3
         EIP is at __purge_vmap_area_lazy+0x144/0x150
         EAX: ee8a8818 EBX: c08e77d4 ECX: e7c7ae40 EDX: c08e77ec
         ESI: 000081fe EDI: e7c7ae60 EBP: e7c7ae64 ESP: e7c7ae3c
         DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
         Call Trace:
         [<c0482ad9>] free_unmap_vmap_area_noflush+0x69/0x70
         [<c0482b02>] remove_vm_area+0x22/0x70
         [<c0482c15>] __vunmap+0x45/0xe0
         [<c04831ec>] vmalloc+0x2c/0x30
         Code: 8d 59 e0 eb 04 66 90 89 cb 89 d0 e8 87 fe ff ff 8b 43 20 89 da 8d 48 e0 8d 43 20 3b 04 24 75 e7 fe 05 a8 a5 a3 c0 e9 78 ff ff ff <0f> 0b eb fe 90 8d b4 26 00 00 00 00 56 89 c6 b8 ac a5 a3 c0 31
         EIP: [<c04824a4>] __purge_vmap_area_lazy+0x144/0x150 SS:ESP 0068:e7c7ae3c
      
      [ See also http://marc.info/?l=linux-kernel&m=126335856228090&w=2
      
       ]
      Signed-off-by: default avatarYongseok Koh <yongseok.koh@samsung.com>
      Reviewed-by: default avatarMinchan Kim <minchan.kim@gmail.com>
      Cc: Nick Piggin <npiggin@suse.de>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      88f50044
  4. 15 Dec, 2009 1 commit
  5. 29 Oct, 2009 1 commit
  6. 11 Oct, 2009 1 commit
  7. 08 Oct, 2009 2 commits
  8. 23 Sep, 2009 1 commit
  9. 22 Sep, 2009 4 commits
  10. 14 Aug, 2009 2 commits
    • Tejun Heo's avatar
      vmalloc: implement pcpu_get_vm_areas() · ca23e405
      Tejun Heo authored
      
      
      To directly use spread NUMA memories for percpu units, percpu
      allocator will be updated to allow sparsely mapping units in a chunk.
      As the distances between units can be very large, this makes
      allocating single vmap area for each chunk undesirable.  This patch
      implements pcpu_get_vm_areas() and pcpu_free_vm_areas() which
      allocates and frees sparse congruent vmap areas.
      
      pcpu_get_vm_areas() take @offsets and @sizes array which define
      distances and sizes of vmap areas.  It scans down from the top of
      vmalloc area looking for the top-most address which can accomodate all
      the areas.  The top-down scan is to avoid interacting with regular
      vmallocs which can push up these congruent areas up little by little
      ending up wasting address space and page table.
      
      To speed up top-down scan, the highest possible address hint is
      maintained.  Although the scan is linear from the hint, given the
      usual large holes between memory addresses between NUMA nodes, the
      scanning is highly likely to finish after finding the first hole for
      the last unit which is scanned first.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Nick Piggin <npiggin@suse.de>
      ca23e405
    • Tejun Heo's avatar
      vmalloc: separate out insert_vmalloc_vm() · cf88c790
      Tejun Heo authored
      
      
      Separate out insert_vmalloc_vm() from __get_vm_area_node().
      insert_vmalloc_vm() initializes vm_struct from vmap_area and inserts
      it into vmlist.  insert_vmalloc_vm() only initializes fields which can
      be determined from @vm, @flags and @caller The rest should be
      initialized by the caller.  For __get_vm_area_node(), all other fields
      just need to be cleared and this is done by using kzalloc instead of
      kmalloc.
      
      This will be used to implement pcpu_get_vm_areas().
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Nick Piggin <npiggin@suse.de>
      cf88c790
  11. 11 Jun, 2009 2 commits
  12. 06 May, 2009 1 commit
  13. 01 Apr, 2009 1 commit
  14. 27 Feb, 2009 2 commits
    • Vegard Nossum's avatar
      mm: fix lazy vmap purging (use-after-free error) · cbb76676
      Vegard Nossum authored
      
      
      I just got this new warning from kmemcheck:
      
          WARNING: kmemcheck: Caught 32-bit read from freed memory (c7806a60)
          a06a80c7ecde70c1a04080c700000000a06709c1000000000000000000000000
           f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f f
           ^
      
          Pid: 0, comm: swapper Not tainted (2.6.29-rc4 #230)
          EIP: 0060:[<c1096df7>] EFLAGS: 00000286 CPU: 0
          EIP is at __purge_vmap_area_lazy+0x117/0x140
          EAX: 00070f43 EBX: c7806a40 ECX: c1677080 EDX: 00027b66
          ESI: 00002001 EDI: c170df0c EBP: c170df00 ESP: c178830c
           DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068
          CR0: 80050033 CR2: c7806b14 CR3: 01775000 CR4: 00000690
          DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
          DR6: 00004000 DR7: 00000000
           [<c1096f3e>] free_unmap_vmap_area_noflush+0x6e/0x70
           [<c1096f6a>] remove_vm_area+0x2a/0x70
           [<c1097025>] __vunmap+0x45/0xe0
           [<c10970de>] vunmap+0x1e/0x30
           [<c1008ba5>] text_poke+0x95/0x150
           [<c1008ca9>] alternatives_smp_unlock+0x49/0x60
           [<c171ef47>] alternative_instructions+0x11b/0x124
           [<c171f991>] check_bugs+0xbd/0xdc
           [<c17148c5>] start_kernel+0x2ed/0x360
           [<c171409e>] __init_begin+0x9e/0xa9
           [<ffffffff>] 0xffffffff
      
      It happened here:
      
          $ addr2line -e vmlinux -i c1096df7
          mm/vmalloc.c:540
      
      Code:
      
      	list_for_each_entry(va, &valist, purge_list)
      		__free_vmap_area(va);
      
      It's this instruction:
      
          mov    0x20(%ebx),%edx
      
      Which corresponds to a dereference of va->purge_list.next:
      
          (gdb) p ((struct vmap_area *) 0)->purge_list.next
          Cannot access memory at address 0x20
      
      It seems that we should use "safe" list traversal here, as the element
      is freed inside the loop. Please verify that this is the right fix.
      Acked-by: default avatarNick Piggin <npiggin@suse.de>
      Signed-off-by: default avatarVegard Nossum <vegard.nossum@gmail.com>
      Cc: Pekka Enberg <penberg@cs.helsinki.fi>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
      Cc: <stable@kernel.org>		[2.6.28.x]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      cbb76676
    • Nick Piggin's avatar
      mm: vmap fix overflow · 7766970c
      Nick Piggin authored
      
      
      The new vmap allocator can wrap the address and get confused in the case
      of large allocations or VMALLOC_END near the end of address space.
      
      Problem reported by Christoph Hellwig on a 32-bit XFS workload.
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      Reported-by: default avatarChristoph Hellwig <hch@lst.de>
      Cc: <stable@kernel.org>		[2.6.28.x]
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      7766970c
  15. 25 Feb, 2009 1 commit
  16. 23 Feb, 2009 1 commit
    • Tejun Heo's avatar
      vmalloc: add @align to vm_area_register_early() · c0c0a293
      Tejun Heo authored
      
      
      Impact: allow larger alignment for early vmalloc area allocation
      
      Some early vmalloc users might want larger alignment, for example, for
      custom large page mapping.  Add @align to vm_area_register_early().
      While at it, drop docbook comment on non-existent @size.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Cc: Nick Piggin <nickpiggin@yahoo.com.au>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      c0c0a293
  17. 20 Feb, 2009 4 commits
  18. 18 Feb, 2009 1 commit
  19. 15 Jan, 2009 2 commits
    • Andrew Morton's avatar
      revert "mm: vmalloc use mutex for purge" · 46666d8a
      Andrew Morton authored
      Revert commit e97a630e
      
       ("mm: vmalloc use
      mutex for purge")
      
      Bryan Donlan reports:
      
      : After testing 2.6.29-rc1 on xen-x86 with a btrfs root filesystem, I
      : got the OOPS quoted below and a hard freeze shortly after boot.
      : Boot messages and config are attached.
      :
      : ------------[ cut here ]------------
      : Kernel BUG at c05ef80d [verbose debug info unavailable]
      : invalid opcode: 0000 [#1] SMP
      : last sysfs file: /sys/block/xvdc/size
      : Modules linked in:
      :
      : Pid: 0, comm: swapper Not tainted (2.6.29-rc1 #6)
      : EIP: 0061:[<c05ef80d>] EFLAGS: 00010087 CPU: 2
      : EIP is at schedule+0x7cd/0x950
      : EAX: d5aeca80 EBX: 00000002 ECX: 00000000 EDX: d4cb9a40
      : ESI: c12f5600 EDI: d4cb9a40 EBP: d6033fa4 ESP: d6033ef4
      :  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
      : Process swapper (pid: 0, ti=d6032000 task=d6020b70 task.ti=d6032000)
      : Stack:
      :  000d85bc 00000000 000186a0 00000000 0dd11410 c0105417 c12efe00 0dc367c3
      :  00000011 c0105d46 d5a5d310 deadbeef d4cb9a40 c07cc600 c05f1340 c12e0060
      :  deadbeef d6020b70 d6020d08 00000002 c014377d 00000000 c12f5600 00002c22
      : Call Trace:
      :  [<c0105417>] xen_force_evtchn_callback+0x17/0x30
      :  [<c0105d46>] check_events+0x8/0x12
      :  [<c05f1340>] _spin_unlock_irqrestore+0x20/0x40
      :  [<c014377d>] hrtimer_start_range_ns+0x12d/0x2e0
      :  [<c014c4f6>] tick_nohz_restart_sched_tick+0x146/0x160
      :  [<c0107485>] cpu_idle+0xa5/0xc0
      
      and bisected it to this commit.
      
      Let's remove it now while we have a think about the problem.
      Reported-by: default avatarBryan Donlan <bdonlan@gmail.com>
      Tested-by: default avatarChristophe Saout <christophe@saout.de>
      Cc: Nick Piggin <nickpiggin@yahoo.com.au>
      Cc: Ingo Molnar <mingo@elte.hu>
      Cc: Jeremy Fitzhardinge <jeremy@goop.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      46666d8a
    • Ivan Kokshaysky's avatar
      alpha: fix vmalloc breakage · 822c18f2
      Ivan Kokshaysky authored
      
      
      On alpha, we have to map some stuff in the VMALLOC space very early in the
      boot process (to make SRM console callbacks work and so on, see
      arch/alpha/mm/init.c).  For old VM allocator, we just manually placed a
      vm_struct onto the global vmlist and this worked for ages.
      
      Unfortunately, the new allocator isn't aware of this, so it constantly
      tries to allocate the VM space which is already in use, making vmalloc on
      alpha defunct.
      
      This patch forces KVA to import vmlist entries on init.
      
      [akpm@linux-foundation.org: remove unneeded check (per Johannes)]
      Signed-off-by: default avatarIvan Kokshaysky <ink@jurassic.park.msu.ru>
      Cc: Nick Piggin <npiggin@suse.de>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Cc: Richard Henderson <rth@twiddle.net>
      Cc: Johannes Weiner <hannes@cmpxchg.org>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      822c18f2
  20. 06 Jan, 2009 4 commits
  21. 04 Jan, 2009 1 commit
  22. 10 Dec, 2008 1 commit
  23. 01 Dec, 2008 1 commit
    • Nick Piggin's avatar
      mm: vmalloc fix lazy unmapping cache aliasing · b29acbdc
      Nick Piggin authored
      
      
      Jim Radford has reported that the vmap subsystem rewrite was sometimes
      causing his VIVT ARM system to behave strangely (seemed like going into
      infinite loops trying to fault in pages to userspace).
      
      We determined that the problem was most likely due to a cache aliasing
      issue.  flush_cache_vunmap was only being called at the moment the page
      tables were to be taken down, however with lazy unmapping, this can happen
      after the page has subsequently been freed and allocated for something
      else.  The dangling alias may still have dirty data attached to it.
      
      The fix for this problem is to do the cache flushing when the caller has
      called vunmap -- it would be a bug for them to write anything else to the
      mapping at that point.
      
      That appeared to solve Jim's problems.
      Reported-by: default avatarJim Radford <radford@blackbean.org>
      Signed-off-by: default avatarNick Piggin <npiggin@suse.de>
      Cc: Russell King <rmk@arm.linux.org.uk>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      b29acbdc
  24. 19 Nov, 2008 2 commits