1. 13 Apr, 2010 1 commit
    • Eric Dumazet's avatar
      net: sk_dst_cache RCUification · b6c6712a
      Eric Dumazet authored
      With latest CONFIG_PROVE_RCU stuff, I felt more comfortable to make this
      sk->sk_dst_cache is currently protected by a rwlock (sk_dst_lock)
      This rwlock is readlocked for a very small amount of time, and dst
      entries are already freed after RCU grace period. This calls for RCU
      again :)
      This patch converts sk_dst_lock to a spinlock, and use RCU for readers.
      __sk_dst_get() is supposed to be called with rcu_read_lock() or if
      socket locked by user, so use appropriate rcu_dereference_check()
      condition (rcu_read_lock_held() || sock_owned_by_user(sk))
      This patch avoids two atomic ops per tx packet on UDP connected sockets,
      for example, and permits sk_dst_lock to be much less dirtied.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  2. 30 Mar, 2010 1 commit
    • Tejun Heo's avatar
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo authored
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      The script does the followings.
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
      The conversion was done in the following steps.
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
      6. percpu.h was updated not to include slab.h.
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Guess-its-ok-by: default avatarChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
  3. 08 Mar, 2010 1 commit
  4. 05 Mar, 2010 2 commits
    • Zhu Yi's avatar
      net: backlog functions rename · a3a858ff
      Zhu Yi authored
      sk_add_backlog -> __sk_add_backlog
      sk_add_backlog_limited -> sk_add_backlog
      Signed-off-by: default avatarZhu Yi <yi.zhu@intel.com>
      Acked-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Zhu Yi's avatar
      net: add limit for socket backlog · 8eae939f
      Zhu Yi authored
      We got system OOM while running some UDP netperf testing on the loopback
      device. The case is multiple senders sent stream UDP packets to a single
      receiver via loopback on local host. Of course, the receiver is not able
      to handle all the packets in time. But we surprisingly found that these
      packets were not discarded due to the receiver's sk->sk_rcvbuf limit.
      Instead, they are kept queuing to sk->sk_backlog and finally ate up all
      the memory. We believe this is a secure hole that a none privileged user
      can crash the system.
      The root cause for this problem is, when the receiver is doing
      __release_sock() (i.e. after userspace recv, kernel udp_recvmsg ->
      skb_free_datagram_locked -> release_sock), it moves skbs from backlog to
      sk_receive_queue with the softirq enabled. In the above case, multiple
      busy senders will almost make it an endless loop. The skbs in the
      backlog end up eat all the system memory.
      The issue is not only for UDP. Any protocols using socket backlog is
      potentially affected. The patch adds limit for socket backlog so that
      the backlog size cannot be expanded endlessly.
      Reported-by: default avatarAlex Shi <alex.shi@intel.com>
      Cc: David Miller <davem@davemloft.net>
      Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
      Cc: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru
      Cc: "Pekka Savola (ipv6)" <pekkas@netcore.fi>
      Cc: Patrick McHardy <kaber@trash.net>
      Cc: Vlad Yasevich <vladislav.yasevich@hp.com>
      Cc: Sridhar Samudrala <sri@us.ibm.com>
      Cc: Jon Maloy <jon.maloy@ericsson.com>
      Cc: Allan Stephens <allan.stephens@windriver.com>
      Cc: Andrew Hendry <andrew.hendry@gmail.com>
      Signed-off-by: default avatarZhu Yi <yi.zhu@intel.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Acked-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  5. 22 Feb, 2010 1 commit
  6. 14 Feb, 2010 1 commit
  7. 10 Feb, 2010 1 commit
  8. 08 Nov, 2009 2 commits
    • Eric Dumazet's avatar
      udp: secondary hash on (local port, local address) · 512615b6
      Eric Dumazet authored
      Extends udp_table to contain a secondary hash table.
      socket anchor for this second hash is free, because UDP
      doesnt use skc_bind_node : We define an union to hold
      both skc_bind_node & a new hlist_nulls_node udp_portaddr_node
      udp_lib_get_port() inserts sockets into second hash chain
      (additional cost of one atomic op)
      udp_lib_unhash() deletes socket from second hash chain
      (additional cost of one atomic op)
      Note : No spinlock lockdep annotation is needed, because
      lock for the secondary hash chain is always get after
      lock for primary hash chain.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      udp: split sk_hash into two u16 hashes · d4cada4a
      Eric Dumazet authored
      Union sk_hash with two u16 hashes for udp (no extra memory taken)
      One 16 bits hash on (local port) value (the previous udp 'hash')
      One 16 bits hash on (local address, local port) values, initialized
      but not yet used. This second hash is using jenkin hash for better
      Because the 'port' is xored later, a partial hash is performed
      on local address + net_hash_mix(net)
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  9. 20 Oct, 2009 1 commit
  10. 12 Oct, 2009 2 commits
    • Neil Horman's avatar
      net: Generalize socket rx gap / receive queue overflow cmsg · 3b885787
      Neil Horman authored
      Create a new socket level option to report number of queue overflows
      Recently I augmented the AF_PACKET protocol to report the number of frames lost
      on the socket receive queue between any two enqueued frames.  This value was
      exported via a SOL_PACKET level cmsg.  AFter I completed that work it was
      requested that this feature be generalized so that any datagram oriented socket
      could make use of this option.  As such I've created this patch, It creates a
      new SOL_SOCKET level option called SO_RXQ_OVFL, which when enabled exports a
      SOL_SOCKET level cmsg that reports the nubmer of times the sk_receive_queue
      overflowed between any two given frames.  It also augments the AF_PACKET
      protocol to take advantage of this new feature (as it previously did not touch
      sk->sk_drops, which this patch uses to record the overflow count).  Tested
      successfully by me.
      1) Unlike my previous patch, this patch simply records the sk_drops value, which
      is not a number of drops between packets, but rather a total number of drops.
      Deltas must be computed in user space.
      2) While this patch currently works with datagram oriented protocols, it will
      also be accepted by non-datagram oriented protocols. I'm not sure if thats
      agreeable to everyone, but my argument in favor of doing so is that, for those
      protocols which aren't applicable to this option, sk_drops will always be zero,
      and reporting no drops on a receive queue that isn't used for those
      non-participating protocols seems reasonable to me.  This also saves us having
      to code in a per-protocol opt in mechanism.
      3) This applies cleanly to net-next assuming that commit
       (my af packet cmsg patch) is reverted
      Signed-off-by: default avatarNeil Horman <nhorman@tuxdriver.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      net: Fix struct sock bitfield annotation · 5fdb9973
      Eric Dumazet authored
      Since commit a98b65a3
       (net: annotate struct sock bitfield), we lost
      8 bytes in struct sock on 64bit arches because of
      kmemcheck_bitfield_end(flags) misplacement.
      Fix this by putting together sk_shutdown, sk_no_check, sk_userlocks,
      sk_protocol and sk_type in the 'flags' 32bits bitfield
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  11. 06 Oct, 2009 1 commit
    • Eric Dumazet's avatar
      net: speedup sk_wake_async() · bcdce719
      Eric Dumazet authored
      An incoming datagram must bring into cpu cache *lot* of cache lines,
      in particular : (other parts omitted (hash chains, ip route cache...))
      On 32bit arches :
      offsetof(struct sock, sk_rcvbuf)       =0x30    (read)
      offsetof(struct sock, sk_lock)         =0x34   (rw)
      offsetof(struct sock, sk_sleep)        =0x50 (read)
      offsetof(struct sock, sk_rmem_alloc)   =0x64   (rw)
      offsetof(struct sock, sk_receive_queue)=0x74   (rw)
      offsetof(struct sock, sk_forward_alloc)=0x98   (rw)
      offsetof(struct sock, sk_callback_lock)=0xcc    (rw)
      offsetof(struct sock, sk_drops)        =0xd8 (read if we add dropcount support, rw if frame dropped)
      offsetof(struct sock, sk_filter)       =0xf8    (read)
      offsetof(struct sock, sk_socket)       =0x138 (read)
      offsetof(struct sock, sk_data_ready)   =0x15c   (read)
      We can avoid sk->sk_socket and socket->fasync_list referencing on sockets
      with no fasync() structures. (socket->fasync_list ptr is probably already in cache
      because it shares a cache line with socket->wait, ie location pointed by sk->sk_sleep)
      This avoids one cache line load per incoming packet for common cases (no fasync())
      We can leave (or even move in a future patch) sk->sk_socket in a cold location
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  12. 30 Sep, 2009 1 commit
  13. 16 Jul, 2009 1 commit
    • Eric Dumazet's avatar
      net: sock_copy() fixes · 4dc6dc71
      Eric Dumazet authored
      Commit e912b114
      (net: sk_prot_alloc() should not blindly overwrite memory)
      took care of not zeroing whole new socket at allocation time.
      sock_copy() is another spot where we should be very careful.
      We should not set refcnt to a non null value, until
      we are sure other fields are correctly setup, or
      a lockless reader could catch this socket by mistake,
      while not fully (re)initialized.
      This patch puts sk_node & sk_refcnt to the very beginning
      of struct sock to ease sock_copy() & sk_prot_alloc() job.
      We add appropriate smp_wmb() before sk_refcnt initializations
      to match our RCU requirements (changes to sock keys should
      be committed to memory before sk_refcnt setting)
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  14. 09 Jul, 2009 2 commits
    • Jiri Olsa's avatar
      memory barrier: adding smp_mb__after_lock · ad462769
      Jiri Olsa authored
      Adding smp_mb__after_lock define to be used as a smp_mb call after
      a lock.
      Making it nop for x86, since {read|write|spin}_lock() on x86 are
      full memory barriers.
      Signed-off-by: default avatarJiri Olsa <jolsa@redhat.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Jiri Olsa's avatar
      net: adding memory barrier to the poll and receive callbacks · a57de0b4
      Jiri Olsa authored
      Adding memory barrier after the poll_wait function, paired with
      receive callbacks. Adding fuctions sock_poll_wait and sk_has_sleeper
      to wrap the memory barrier.
      Without the memory barrier, following race can happen.
      The race fires, when following code paths meet, and the tp->rcv_nxt
      and __add_wait_queue updates stay in CPU caches.
      CPU1                         CPU2
      sys_select                   receive packet
        ...                        ...
        __add_wait_queue           update tp->rcv_nxt
        ...                        ...
        tp->rcv_nxt check          sock_def_readable
        ...                        {
        schedule                      ...
                                      if (sk->sk_sleep && waitqueue_active(sk->sk_sleep))
      If there was no cache the code would work ok, since the wait_queue and
      rcv_nxt are opposit to each other.
      Meaning that once tp->rcv_nxt is updated by CPU2, the CPU1 either already
      passed the tp->rcv_nxt check and sleeps, or will get the new value for
      tp->rcv_nxt and will return with new data mask.
      In both cases the process (CPU1) is being added to the wait queue, so the
      waitqueue_active (CPU2) call cannot miss and will wake up CPU1.
      The bad case is when the __add_wait_queue changes done by CPU1 stay in its
      cache, and so does the tp->rcv_nxt update on CPU2 side.  The CPU1 will then
      endup calling schedule and sleep forever if there are no more data on the
      Calls to poll_wait in following modules were ommited:
      Signed-off-by: default avatarJiri Olsa <jolsa@redhat.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  15. 23 Jun, 2009 1 commit
    • Herbert Xu's avatar
      net: Move rx skb_orphan call to where needed · d55d87fd
      Herbert Xu authored
      In order to get the tun driver to account packets, we need to be
      able to receive packets with destructors set.  To be on the safe
      side, I added an skb_orphan call for all protocols by default since
      some of them (IP in particular) cannot handle receiving packets
      destructors properly.
      Now it seems that at least one protocol (CAN) expects to be able
      to pass skb->sk through the rx path without getting clobbered.
      So this patch attempts to fix this properly by moving the skb_orphan
      call to where it's actually needed.  In particular, I've added it
      to skb_set_owner_[rw] which is what most users of skb->destructor
      This is actually an improvement for tun too since it means that
      we only give back the amount charged to the socket when the skb
      is passed to another socket that will also be charged accordingly.
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      Tested-by: default avatarOliver Hartkopp <olver@hartkopp.net>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  16. 17 Jun, 2009 1 commit
  17. 15 Jun, 2009 1 commit
    • Vegard Nossum's avatar
      net: annotate struct sock bitfield · a98b65a3
      Vegard Nossum authored
      2009/2/24 Ingo Molnar <mingo@elte.hu>:
      > ok, this is the last warning i have from today's overnight -tip
      > testruns - a 32-bit system warning in sock_init_data():
      > [    2.610389] NET: Registered protocol family 16
      > [    2.616138] initcall netlink_proto_init+0x0/0x170 returned 0 after 7812 usecs
      > [    2.620010] WARNING: kmemcheck: Caught 32-bit read from uninitialized memory (f642c184)
      > [    2.624002] 010000000200000000000000604990c000000000000000000000000000000000
      > [    2.634076]  i i i i i i u u i i i i i i i i i i i i i i i i i i i i i i i i
      > [    2.641038]          ^
      > [    2.643376]
      > [    2.644004] Pid: 1, comm: swapper Not tainted (2.6.29-rc6-tip-01751-g4d1c22c-dirty #885)
      > [    2.648003] EIP: 0060:[<c07141a1>] EFLAGS: 00010282 CPU: 0
      > [    2.652008] EIP is at sock_init_data+0xa1/0x190
      > [    2.656003] EAX: 0001a800 EBX: f6836c00 ECX: 00463000 EDX: c0e46fe0
      > [    2.660003] ESI: f642c180 EDI: c0b83088 EBP: f6863ed8 ESP: c0c412ec
      > [    2.664003]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
      > [    2.668003] CR0: 8005003b CR2: f682c400 CR3: 00b91000 CR4: 000006f0
      > [    2.672003] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
      > [    2.676003] DR6: ffff4ff0 DR7: 00000400
      > [    2.680002]  [<c07423e5>] __netlink_create+0x35/0xa0
      > [    2.684002]  [<c07443cc>] netlink_kernel_create+0x4c/0x140
      > [    2.688002]  [<c072755e>] rtnetlink_net_init+0x1e/0x40
      > [    2.696002]  [<c071b601>] register_pernet_operations+0x11/0x30
      > [    2.700002]  [<c071b72c>] register_pernet_subsys+0x1c/0x30
      > [    2.704002]  [<c0bf3c8c>] rtnetlink_init+0x4c/0x100
      > [    2.708002]  [<c0bf4669>] netlink_proto_init+0x159/0x170
      > [    2.712002]  [<c0101124>] do_one_initcall+0x24/0x150
      > [    2.716002]  [<c0bbf3c7>] do_initcalls+0x27/0x40
      > [    2.723201]  [<c0bbf3fc>] do_basic_setup+0x1c/0x20
      > [    2.728002]  [<c0bbfb8a>] kernel_init+0x5a/0xa0
      > [    2.732002]  [<c0103e47>] kernel_thread_helper+0x7/0x10
      > [    2.736002]  [<ffffffff>] 0xffffffff
      We fix this false positive by annotating the bitfield in struct
      Reported-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarVegard Nossum <vegard.nossum@gmail.com>
  18. 11 Jun, 2009 1 commit
    • Eric Dumazet's avatar
      net: No more expensive sock_hold()/sock_put() on each tx · 2b85a34e
      Eric Dumazet authored
      One of the problem with sock memory accounting is it uses
      a pair of sock_hold()/sock_put() for each transmitted packet.
      This slows down bidirectional flows because the receive path
      also needs to take a refcount on socket and might use a different
      cpu than transmit path or transmit completion path. So these
      two atomic operations also trigger cache line bounces.
      We can see this in tx or tx/rx workloads (media gateways for example),
      where sock_wfree() can be in top five functions in profiles.
      We use this sock_hold()/sock_put() so that sock freeing
      is delayed until all tx packets are completed.
      As we also update sk_wmem_alloc, we could offset sk_wmem_alloc
      by one unit at init time, until sk_free() is called.
      Once sk_free() is called, we atomic_dec_and_test(sk_wmem_alloc)
      to decrement initial offset and atomicaly check if any packets
      are in flight.
      skb_set_owner_w() doesnt call sock_hold() anymore
      sock_wfree() doesnt call sock_put() anymore, but check if sk_wmem_alloc
      reached 0 to perform the final freeing.
      Drawback is that a skb->truesize error could lead to unfreeable sockets, or
      even worse, prematurely calling __sk_free() on a live socket.
      Nice speedups on SMP. tbench for example, going from 2691 MB/s to 2711 MB/s
      on my 8 cpu dev machine, even if tbench was not really hitting sk_refcnt
      contention point. 5 % speedup on a UDP transmit workload (depends
      on number of flows), lowering TX completion cpu usage.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  19. 17 Feb, 2009 1 commit
    • David S. Miller's avatar
      net: Kill skb_truesize_check(), it only catches false-positives. · 92a0acce
      David S. Miller authored
      A long time ago we had bugs, primarily in TCP, where we would modify
      skb->truesize (for TSO queue collapsing) in ways which would corrupt
      the socket memory accounting.
      skb_truesize_check() was added in order to try and catch this error
      more systematically.
      However this debugging check has morphed into a Frankenstein of sorts
      and these days it does nothing other than catch false-positives.
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  20. 15 Feb, 2009 1 commit
  21. 12 Feb, 2009 1 commit
    • Andrew Morton's avatar
      net: don't use in_atomic() in gfp_any() · 99709372
      Andrew Morton authored
      The problem is that in_atomic() will return false inside spinlocks if
      CONFIG_PREEMPT=n.  This will lead to deadlockable GFP_KERNEL allocations
      from spinlocked regions.
      Secondly, if CONFIG_PREEMPT=y, this bug solves itself because networking
      will instead use GFP_ATOMIC from this callsite.  Hence we won't get the
      might_sleep() debugging warnings which would have informed us of the buggy
      Solve both these problems by switching to in_interrupt().  Now, if someone
      runs a gfp_any() allocation from inside spinlock we will get the warning
      if CONFIG_PREEMPT=y.
      I reviewed all callsites and most of them were too complex for my little
      brain and none of them documented their interface requirements.  I have no
      idea what this patch will do.
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  22. 04 Feb, 2009 1 commit
  23. 25 Nov, 2008 2 commits
  24. 16 Nov, 2008 1 commit
  25. 14 Nov, 2008 1 commit
    • Ingo Molnar's avatar
      lockdep: include/linux/lockdep.h - fix warning in net/bluetooth/af_bluetooth.c · e8f6fbf6
      Ingo Molnar authored
      fix this warning:
        net/bluetooth/af_bluetooth.c:60: warning: ‘bt_key_strings’ defined but not used
        net/bluetooth/af_bluetooth.c:71: warning: ‘bt_slock_key_strings’ defined but not used
      this is a lockdep macro problem in the !LOCKDEP case.
      We cannot convert it to an inline because the macro works on multiple types,
      but we can mark the parameter used.
      [ also clean up a misaligned tab in sock_lock_init_class_and_name() ]
      [ also remove #ifdefs from around af_family_clock_key strings - which
        were certainly added to get rid of the ugly build warnings. ]
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  26. 13 Nov, 2008 1 commit
  27. 12 Nov, 2008 1 commit
    • Ingo Molnar's avatar
      lockdep: include/linux/lockdep.h - fix warning in net/bluetooth/af_bluetooth.c · e25cf3db
      Ingo Molnar authored
      fix this warning:
        net/bluetooth/af_bluetooth.c:60: warning: ‘bt_key_strings’ defined but not used
        net/bluetooth/af_bluetooth.c:71: warning: ‘bt_slock_key_strings’ defined but not used
      this is a lockdep macro problem in the !LOCKDEP case.
      We cannot convert it to an inline because the macro works on multiple types,
      but we can mark the parameter used.
      [ also clean up a misaligned tab in sock_lock_init_class_and_name() ]
      [ also remove #ifdefs from around af_family_clock_key strings - which
        were certainly added to get rid of the ugly build warnings. ]
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
  28. 04 Nov, 2008 1 commit
  29. 31 Oct, 2008 1 commit
    • Randy Dunlap's avatar
      net: delete excess kernel-doc notation · ad1d967c
      Randy Dunlap authored
      Remove excess kernel-doc function parameters from networking header
      & driver files:
      Warning(include/net/sock.h:946): Excess function parameter or struct member 'sk' description in 'sk_filter_release'
      Warning(include/linux/netdevice.h:1545): Excess function parameter or struct member 'cpu' description in 'netif_tx_lock'
      Warning(drivers/net/wan/z85230.c:712): Excess function parameter or struct member 'regs' description in 'z8530_interrupt'
      Signed-off-by: default avatarRandy Dunlap <randy.dunlap@oracle.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  30. 29 Oct, 2008 3 commits
    • Eric Dumazet's avatar
      udp: introduce sk_for_each_rcu_safenext() · 96631ed1
      Eric Dumazet authored
      Corey Minyard found a race added in commit 271b72c7
      (udp: RCU handling for Unicast packets.)
       "If the socket is moved from one list to another list in-between the
       time the hash is calculated and the next field is accessed, and the
       socket has moved to the end of the new list, the traversal will not
       complete properly on the list it should have, since the socket will
       be on the end of the new list and there's not a way to tell it's on a
       new list and restart the list traversal.  I think that this can be
       solved by pre-fetching the "next" field (with proper barriers) before
       checking the hash."
      This patch corrects this problem, introducing a new
      sk_for_each_rcu_safenext() macro.
      Signed-off-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      udp: RCU handling for Unicast packets. · 271b72c7
      Eric Dumazet authored
      Goals are :
      1) Optimizing handling of incoming Unicast UDP frames, so that no memory
       writes should happen in the fast path.
       Note: Multicasts and broadcasts still will need to take a lock,
       because doing a full lockless lookup in this case is difficult.
      2) No expensive operations in the socket bind/unhash phases :
        - No expensive synchronize_rcu() calls.
        - No added rcu_head in socket structure, increasing memory needs,
        but more important, forcing us to use call_rcu() calls,
        that have the bad property of making sockets structure cold.
        (rcu grace period between socket freeing and its potential reuse
         make this socket being cold in CPU cache).
        David did a previous patch using call_rcu() and noticed a 20%
        impact on TCP connection rates.
        Quoting Cristopher Lameter :
         "Right. That results in cacheline cooldown. You'd want to recycle
          the object as they are cache hot on a per cpu basis. That is screwed
          up by the delayed regular rcu processing. We have seen multiple
          regressions due to cacheline cooldown.
          The only choice in cacheline hot sensitive areas is to deal with the
          complexity that comes with SLAB_DESTROY_BY_RCU or give up on RCU."
        - Because udp sockets are allocated from dedicated kmem_cache,
        use of SLAB_DESTROY_BY_RCU can help here.
      Theory of operation :
      As the lookup is lockfree (using rcu_read_lock()/rcu_read_unlock()),
      special attention must be taken by readers and writers.
      Use of SLAB_DESTROY_BY_RCU is tricky too, because a socket can be freed,
      reused, inserted in a different chain or in worst case in the same chain
      while readers could do lookups in the same time.
      In order to avoid loops, a reader must check each socket found in a chain
      really belongs to the chain the reader was traversing. If it finds a
      mismatch, lookup must start again at the begining. This *restart* loop
      is the reason we had to use rdlock for the multicast case, because
      we dont want to send same message several times to the same socket.
      We use RCU only for fast path.
      Thus, /proc/net/udp still takes spinlocks.
      Signed-off-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      udp: introduce struct udp_table and multiple spinlocks · 645ca708
      Eric Dumazet authored
      UDP sockets are hashed in a 128 slots hash table.
      This hash table is protected by *one* rwlock.
      This rwlock is readlocked each time an incoming UDP message is handled.
      This rwlock is writelocked each time a socket must be inserted in
      hash table (bind time), or deleted from this table (close time)
      This is not scalable on SMP machines :
      1) Even in read mode, lock() and unlock() are atomic operations and
       must dirty a contended cache line, shared by all cpus.
      2) A writer might be starved if many readers are 'in flight'. This can
       happen on a machine with some NIC receiving many UDP messages. User
       process can be delayed a long time at socket creation/dismantle time.
      This patch prepares RCU migration, by introducing 'struct udp_table
      and struct udp_hslot', and using one spinlock per chain, to reduce
      contention on central rwlock.
      Introducing one spinlock per chain reduces latencies, for port
      randomization on heavily loaded UDP servers. This also speedup
      bindings to specific ports.
      udp_lib_unhash() was uninlined, becoming to big.
      Some cleanups were done to ease review of following patch
      (RCUification of UDP Unicast lookups)
      Signed-off-by: default avatarEric Dumazet <dada1@cosmosbay.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  31. 28 Oct, 2008 1 commit
  32. 07 Oct, 2008 2 commits