1. 27 Jan, 2013 1 commit
    • Frederic Weisbecker's avatar
      cputime: Generic on-demand virtual cputime accounting · abf917cd
      Frederic Weisbecker authored
      
      
      If we want to stop the tick further idle, we need to be
      able to account the cputime without using the tick.
      
      Virtual based cputime accounting solves that problem by
      hooking into kernel/user boundaries.
      
      However implementing CONFIG_VIRT_CPU_ACCOUNTING require
      low level hooks and involves more overhead. But we already
      have a generic context tracking subsystem that is required
      for RCU needs by archs which plan to shut down the tick
      outside idle.
      
      This patch implements a generic virtual based cputime
      accounting that relies on these generic kernel/user hooks.
      
      There are some upsides of doing this:
      
      - This requires no arch code to implement CONFIG_VIRT_CPU_ACCOUNTING
      if context tracking is already built (already necessary for RCU in full
      tickless mode).
      
      - We can rely on the generic context tracking subsystem to dynamically
      (de)activate the hooks, so that we can switch anytime between virtual
      and tick based accounting. This way we don't have the overhead
      of the virtual accounting when the tick is running periodically.
      
      And one downside:
      
      - There is probably more overhead than a native virtual based cputime
      accounting. But this relies on hooks that are already set anyway.
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Li Zhong <zhong@linux.vnet.ibm.com>
      Cc: Namhyung Kim <namhyung.kim@lge.com>
      Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
      Cc: Paul Gortmaker <paul.gortmaker@windriver.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      abf917cd
  2. 28 Mar, 2012 1 commit
  3. 03 Mar, 2010 1 commit
  4. 08 Feb, 2010 1 commit
    • Tony Luck's avatar
      [IA64] Remove COMPAT_IA32 support · 32974ad4
      Tony Luck authored
      
      
      This has been broken since May 2008 when Al Viro killed altroot support.
      Since nobody has complained, it would appear that there are no users of
      this code (A plausible theory since the main OSVs that support ia64 prefer
      to use the IA32-EL software emulation).
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      32974ad4
  5. 26 Mar, 2009 1 commit
  6. 17 Oct, 2008 1 commit
  7. 04 Aug, 2008 1 commit
    • Isaku Yamahata's avatar
      [IA64] pv_ops: fix ivt.S paravirtualization · 9b3cbf72
      Isaku Yamahata authored
      Recent kernels are not booting on some HP systems (though
      it does boot on others). James and Willy reported the
      problem.  James did the bisection to find the commit
      that caused the problem:
      	498c5170
      
      .
      	[IA64] pvops: paravirtualize ivt.S
      
      Two instructions were wrongly paravirtualized such that
      _FROM_ macro had been used where _TO_ was intended
      
      Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
      Cc: "Wilcox, Matthew  R" <matthew.r.wilcox@intel.com>
      Signed-off-by: default avatarIsaku Yamahata <yamahata@valinux.co.jp>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      9b3cbf72
  8. 28 May, 2008 1 commit
  9. 27 May, 2008 2 commits
    • Isaku Yamahata's avatar
      [IA64] pvops: paravirtualize ivt.S · 498c5170
      Isaku Yamahata authored
      
      
      paravirtualize ivt.S which implements fault handler in hand written
      assembly code.
      They includes sensitive or performance critical privileged instructions.
      So they need paravirtualization.
      
      Cc: Keith Owens <kaos@ocs.com.au>
      Cc: tgingold@free.fr
      Cc: Akio Takebe <takebe_akio@jp.fujitsu.com>
      Signed-off-by: default avatarYaozu (Eddie) Dong <eddie.dong@intel.com>
      Signed-off-by: default avatarIsaku Yamahata <yamahata@valinux.co.jp>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      498c5170
    • Tony Luck's avatar
      [IA64] Workaround for RSE issue · 4dcc29e1
      Tony Luck authored
      
      
      Problem: An application violating the architectural rules regarding
      operation dependencies and having specific Register Stack Engine (RSE)
      state at the time of the violation, may result in an illegal operation
      fault and invalid RSE state.  Such faults may initiate a cascade of
      repeated illegal operation faults within OS interruption handlers.
      The specific behavior is OS dependent.
      
      Implication: An application causing an illegal operation fault with
      specific RSE state may result in a series of illegal operation faults
      and an eventual OS stack overflow condition.
      
      Workaround: OS interruption handlers that switch to kernel backing
      store implement a check for invalid RSE state to avoid the series
      of illegal operation faults.
      
      The core of the workaround is the RSE_WORKAROUND code sequence
      inserted into each invocation of the SAVE_MIN_WITH_COVER and
      SAVE_MIN_WITH_COVER_R19 macros.  This sequence includes hard-coded
      constants that depend on the number of stacked physical registers
      being 96.  The rest of this patch consists of code to disable this
      workaround should this not be the case (with the presumption that
      if a future Itanium processor increases the number of registers, it
      would also remove the need for this patch).
      
      Move the start of the RBS up to a mod32 boundary to avoid some
      corner cases.
      
      The dispatch_illegal_op_fault code outgrew the spot it was
      squatting in when built with this patch and CONFIG_VIRT_CPU_ACCOUNTING=y
      Move it out to the end of the ivt.
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      4dcc29e1
  10. 20 Feb, 2008 1 commit
    • Hidetoshi Seto's avatar
      [IA64] VIRT_CPU_ACCOUNTING (accurate cpu time accounting) · b64f34cd
      Hidetoshi Seto authored
      
      
      This patch implements VIRT_CPU_ACCOUNTING for ia64,
      which enable us to use more accurate cpu time accounting.
      
      The VIRT_CPU_ACCOUNTING is an item of kernel config, which s390
      and powerpc arch have.  By turning this config on, these archs
      change the mechanism of cpu time accounting from tick-sampling
      based one to state-transition based one.
      
      The state-transition based accounting is done by checking time
      (cycle counter in processor) at every state-transition point,
      such as entrance/exit of kernel, interrupt, softirq etc.
      The difference between point to point is the actual time consumed
      during in the state. There is no doubt about that this value is
      more accurate than that of tick-sampling based accounting.
      Signed-off-by: default avatarHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      b64f34cd
  11. 06 Feb, 2007 1 commit
    • Chen, Kenneth W's avatar
      [IA64] relax per-cpu TLB requirement to DTC · 00b65985
      Chen, Kenneth W authored
      
      
      Instead of pinning per-cpu TLB into a DTR, use DTC.  This will free up
      one TLB entry for application, or even kernel if access pattern to
      per-cpu data area has high temporal locality.
      
      Since per-cpu is mapped at the top of region 7 address, we just need to
      add special case in alt_dtlb_miss.  The physical address of per-cpu data
      is already conveniently stored in IA64_KR(PER_CPU_DATA).  Latency for
      alt_dtlb_miss is not affected as we can hide all the latency.  It was
      measured that alt_dtlb_miss handler has 23 cycles latency before and
      after the patch.
      
      The performance effect is massive for applications that put lots of tlb
      pressure on CPU.  Workload environment like database online transaction
      processing or application uses tera-byte of memory would benefit the most.
      Measurement with industry standard database benchmark shown an upward
      of 1.6% gain.  While smaller workloads like cpu, java also showing small
      improvement.
      Signed-off-by: default avatarKen Chen <kenneth.w.chen@intel.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      00b65985
  12. 30 Jun, 2006 1 commit
  13. 24 Mar, 2006 1 commit
    • Russ Anderson's avatar
      [IA64] MCA recovery: kernel context recovery table · d2a28ad9
      Russ Anderson authored
      
      
      Memory errors encountered by user applications may surface
      when the CPU is running in kernel context.  The current code
      will not attempt recovery if the MCA surfaces in kernel
      context (privilage mode 0).  This patch adds a check for cases
      where the user initiated the load that surfaces in kernel
      interrupt code.
      
      An example is a user process lauching a load from memory
      and the data in memory had bad ECC.  Before the bad data
      gets to the CPU register, and interrupt comes in.  The
      code jumps to the IVT interrupt entry point and begins
      execution in kernel context.  The process of saving the
      user registers (SAVE_REST) causes the bad data to be loaded
      into a CPU register, triggering the MCA.  The MCA surfaces in
      kernel context, even though the load was initiated from
      user context.
      
      As suggested by David and Tony, this patch uses an exception
      table like approach, puting the tagged recovery addresses in
      a searchable table.  One difference from the exception table
      is that MCAs do not surface in precise places (such as with
      a TLB miss), so instead of tagging specific instructions,
      address ranges are registers.  A single macro is used to do
      the tagging, with the input parameter being the label
      of the starting address and the macro being the ending
      address.  This limits clutter in the code.
      
      This patch only tags one spot, the interrupt ivt entry.
      Testing showed that spot to be a "heavy hitter" with
      MCAs surfacing while saving user registers.  Other spots
      can be added as needed by adding a single macro.
      
      Signed-off-by: Russ Anderson (rja@sgi.com)
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      d2a28ad9
  14. 08 Mar, 2006 1 commit
    • Christoph Lameter's avatar
      [IA64] Fix race in the accessed/dirty bit handlers · d8117ce5
      Christoph Lameter authored
      
      
      A pte may be zapped by the swapper, exiting process, unmapping or page
      migration while the accessed or dirty bit handers are about to run. In that
      case the accessed bit or dirty is set on an zeroed pte which leads the VM to
      conclude that this is a swap pte. This may lead to
      
      - Messages from the vm like
      
      swap_free: Bad swap file entry 4000000000000000
      
      - Processes being aborted
      
      swap_dup: Bad swap file entry 4000000000000000
      VM: killing process ....
      
      Page migration is particular suitable for the creation of this race since
      it needs to remove and restore page table entries.
      
      The fix here is to check for the present bit and simply not update
      the pte if the page is not present anymore. If the page is not present
      then the fault handler should run next which will take care of the problem
      by bringing the page back and then mark the page dirty or move it onto the
      active list.
      Signed-off-by: default avatarChristoph Lameter <clameter@sgi.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      d8117ce5
  15. 27 Feb, 2006 1 commit
  16. 17 Nov, 2005 2 commits
    • Chen, Kenneth W's avatar
      [IA64] polish comments for tlb fault handler in ivt.S · e8aabc47
      Chen, Kenneth W authored
      
      
      Polish the comments specifically in vhpt_miss and nested_dtlb_miss
      handlers.  I think it's better to explicitly name each page table
      level with its name instead of numerically name them.  i.e., use
      pgd, pud, pmd, and pte instead of referring as L1, L2, L3 etc.
      Along the line, remove some magic number in the comments like:
      "PTA + (((IFA(61,63) << 7) | IFA(33,39))*8)".  No code change at
      all, pure comment update.  Feel free to shoot anything you have,
      darts or tomahawk cruise missile.  I will duck behind a bunker ;-)
      Signed-off-by: default avatarKen Chen <kenneth.w.chen@intel.com>
      Acked-by: default avatarRobin Holt <holt@sgi.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      e8aabc47
    • Chen, Kenneth W's avatar
      [IA64] 4 level page table bug fix in vhpt_miss · fedb25fa
      Chen, Kenneth W authored
      
      
      From source code inspection, I think there is a bug with 4 level
      page table with vhpt_miss handler.  In the code path of rechecking
      page table entry against previously read value after tlb insertion,
      *pte value in register r18 was overwritten with value newly read
      from pud pointer, render the check of new *pte against previous
      *pte completely wrong.  Though the bug is none fatal and the penalty
      is to purge the entry and retry.  For functional correctness, it
      should be fixed.  The fix is to use a different register so new
      *pud don't trash *pte.  (btw, the comments in the cmp statement is
      wrong as well, which I will address in the next patch).
      Signed-off-by: default avatarKen Chen <kenneth.w.chen@intel.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      fedb25fa
  17. 11 Nov, 2005 1 commit
    • Robin Holt's avatar
      [IA64] 4-level page tables · 837cd0bd
      Robin Holt authored
      
      
      This patch introduces 4-level page tables to ia64.  I have run
      some benchmarks and found nothing interesting.  Performance has
      consistently fallen within the noise range.
      
      It also introduces a config option (setting the default to 3
      levels).  The config option prevents having 4 level page
      tables with 64k base page size.
      Signed-off-by: default avatarRobin Holt <holt@sgi.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      837cd0bd
  18. 11 Sep, 2005 1 commit
  19. 09 Sep, 2005 1 commit
  20. 28 Jun, 2005 1 commit
    • David Mosberger-Tang's avatar
      [IA64] Speed up lfetch.fault [NULL] · 458f9355
      David Mosberger-Tang authored
      
      
      This patch greatly speeds up the handling of lfetch.fault instructions
      which result in NaT consumption. Due to the NaT-page mapped at address
      0, this is guaranteed to happen when lfetch.fault'ing a NULL pointer.
      With this patch in place, we can even define prefetch()/prefetchw() as
      lfetch.fault without significant performance degradation.  More
      importantly, it allows compilers to be more aggressive with using
      lfetch.fault on pointers that might be NULL.
      Signed-off-by: default avatarDavid Mosberger-Tang <davidm@hpl.hp.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      458f9355
  21. 21 Jun, 2005 1 commit
    • Ken Chen's avatar
      [IA64] fix nested_dtlb_miss handler for hugetlb address · 0393eed5
      Ken Chen authored
      
      
      The nested_dtlb_miss handler currently does not handle fault from
      hugetlb address correctly.  It walks the page table assuming PAGE_SIZE.
      Thus when taking a fault triggered from hugetlb address, it would not
      calculate the pgd/pmd/pte address correctly and thus result an incorrect
      invocation of ia64_do_page_fault().  In there, kernel will signal SIGBUS
      and application dies (The faulting address is perfectly legal and we
      have a valid pte for the corresponding user hugetlb address as well).
      This patch fix the described kernel bug.  Since nested_dtlb_miss is a
      rare event and a slow path anyway, I'm making the change without #ifdef
      CONFIG_HUGETLB_PAGE for code readability.  Tony, please apply.
      Signed-off-by: default avatarKen Chen <kenneth.w.chen@intel.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      0393eed5
  22. 27 Apr, 2005 2 commits
    • David Mosberger-Tang's avatar
      [IA64] Reschedule break_fault() for better performance. · f8fa5448
      David Mosberger-Tang authored
      
      
      This patch reorganizes break_fault() to optimistically assume that a
      system-call is being performed from user-space (which is almost always
      the case).  If it turns out that (a) we're not being called due to a
      system call or (b) we're being called from within the kernel, we fixup
      the no-longer-valid assumptions in non_syscall() and .break_fixup(),
      respectively.
      
      With this approach, there are 3 major phases:
      
       - Phase 1: Read various control & application registers, in
      	    particular the current task pointer from AR.K6.
       - Phase 2: Do all memory loads (load system-call entry,
      	    load current_thread_info()->flags, prefetch
      	    kernel register-backing store) and switch
      	    to kernel register-stack.
       - Phase 3: Call ia64_syscall_setup() and invoke
      	    syscall-handler.
      
      Good for 26-30 cycles of improvement on break-based syscall-path.
      Signed-off-by: default avatarDavid Mosberger-Tang <davidm@hpl.hp.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      f8fa5448
    • David Mosberger-Tang's avatar
      [IA64] In syscall-entry, use st8 instead of stf8 to clear pt_regs.r8 · 060561ff
      David Mosberger-Tang authored
      
      
      Using stf8 seemed like a clever idea at the time, but stf8 forces
      the cache-line to be invalidated in the L1D (if it happens to be
      there already).  This patch eliminates a guaranteed L1D cache-miss
      and, by itself, is good for a 1-2 cycle improvement for heavy-weight
      syscalls.
      Signed-off-by: default avatarDavid Mosberger-Tang <davidm@hpl.hp.com>
      Signed-off-by: default avatarTony Luck <tony.luck@intel.com>
      060561ff
  23. 16 Apr, 2005 1 commit
    • Linus Torvalds's avatar
      Linux-2.6.12-rc2 · 1da177e4
      Linus Torvalds authored
      Initial git repository build. I'm not bothering with the full history,
      even though we have it. We can create a separate "historical" git
      archive of that later if we want to, and in the meantime it's about
      3.2GB when imported into git - space that would just make the early
      git days unnecessarily complicated, when we don't have a lot of good
      infrastructure for it.
      
      Let it rip!
      1da177e4