1. 10 Nov, 2010 1 commit
  2. 25 Oct, 2010 1 commit
  3. 26 Sep, 2010 1 commit
    • Eric Dumazet's avatar
      net: update SOCK_MIN_RCVBUF · 7a91b434
      Eric Dumazet authored
      SOCK_MIN_RCVBUF current value is 256 bytes
      It doesnt permit to receive the smallest possible frame, considering
      socket sk_rmem_alloc/sk_rcvbuf account skb truesizes. On 64bit arches,
      sizeof(struct sk_buff) is 240 bytes. Add the typical 64 bytes of
      headroom, and we go over the limit.
      With old kernels and 32bit arches, we were under the limit, if netdriver
      was doing copybreak.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  4. 08 Sep, 2010 1 commit
    • Eric Dumazet's avatar
      udp: add rehash on connect() · 719f8358
      Eric Dumazet authored
      commit 30fff923 introduced in linux-2.6.33 (udp: bind() optimisation)
      added a secondary hash on UDP, hashed on (local addr, local port).
      Problem is that following sequence :
      fd = socket(...)
      connect(fd, &remote, ...)
      not only selects remote end point (address and port), but also sets
      local address, while UDP stack stored in secondary hash table the socket
      while its local address was INADDR_ANY (or ipv6 equivalent)
      Sequence is :
       - autobind() : choose a random local port, insert socket in hash tables
                    [while local address is INADDR_ANY]
       - connect() : set remote address and port, change local address to IP
                    given by a route lookup.
      When an incoming UDP frame comes, if more than 10 sockets are found in
      primary hash table, we switch to secondary table, and fail to find
      socket because its local address changed.
      One solution to this problem is to rehash datagram socket if needed.
      We add a new rehash(struct socket *) method in "struct proto", and
      implement this method for UDP v4 & v6, using a common helper.
      This rehashing only takes care of secondary hash table, since primary
      hash (based on local port only) is not changed.
      Reported-by: default avatarKrzysztof Piotr Oledzki <ole@ans.pl>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Tested-by: default avatarKrzysztof Piotr Oledzki <ole@ans.pl>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  5. 19 Aug, 2010 1 commit
  6. 10 Aug, 2010 1 commit
  7. 14 Jul, 2010 1 commit
    • Tom Herbert's avatar
      net: fix problem in reading sock TX queue · b0f77d0e
      Tom Herbert authored
      Fix problem in reading the tx_queue recorded in a socket.  In
      dev_pick_tx, the TX queue is read by doing a check with
      sk_tx_queue_recorded on the socket, followed by a sk_tx_queue_get.
      The problem is that there is not mutual exclusion across these
      calls in the socket so it it is possible that the queue in the
      sock can be invalidated after sk_tx_queue_recorded is called so
      that sk_tx_queue get returns -1, which sets 65535 in queue_index
      and thus dev_pick_tx returns 65536 which is a bogus queue and
      can cause crash in dev_queue_xmit.
      We fix this by only calling sk_tx_queue_get which does the proper
      checks.  The interface is that sk_tx_queue_get returns the TX queue
      if the sock argument is non-NULL and TX queue is recorded, else it
      returns -1.  sk_tx_queue_recorded is no longer used so it can be
      completely removed.
      Signed-off-by: default avatarTom Herbert <therbert@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  8. 12 Jul, 2010 1 commit
  9. 16 Jun, 2010 1 commit
  10. 02 Jun, 2010 1 commit
  11. 01 Jun, 2010 1 commit
  12. 27 May, 2010 1 commit
    • Eric Dumazet's avatar
      net: fix lock_sock_bh/unlock_sock_bh · 8a74ad60
      Eric Dumazet authored
      This new sock lock primitive was introduced to speedup some user context
      socket manipulation. But it is unsafe to protect two threads, one using
      regular lock_sock/release_sock, one using lock_sock_bh/unlock_sock_bh
      This patch changes lock_sock_bh to be careful against 'owned' state.
      If owned is found to be set, we must take the slow path.
      lock_sock_bh() now returns a boolean to say if the slow path was taken,
      and this boolean is used at unlock_sock_bh time to call the appropriate
      unlock function.
      After this change, BH are either disabled or enabled during the
      lock_sock_bh/unlock_sock_bh protected section. This might be misleading,
      so we rename these functions to lock_sock_fast()/unlock_sock_fast().
      Reported-by: default avatarAnton Blanchard <anton@samba.org>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Tested-by: default avatarAnton Blanchard <anton@samba.org>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  13. 25 May, 2010 1 commit
  14. 24 May, 2010 1 commit
    • Herbert Xu's avatar
      cls_cgroup: Store classid in struct sock · f8451725
      Herbert Xu authored
      Up until now cls_cgroup has relied on fetching the classid out of
      the current executing thread.  This runs into trouble when a packet
      processing is delayed in which case it may execute out of another
      thread's context.
      Furthermore, even when a packet is not delayed we may fail to
      classify it if soft IRQs have been disabled, because this scenario
      is indistinguishable from one where a packet unrelated to the
      current thread is processed by a real soft IRQ.
      In fact, the current semantics is inherently broken, as a single
      skb may be constructed out of the writes of two different tasks.
      A different manifestation of this problem is when the TCP stack
      transmits in response of an incoming ACK.  This is currently
      As we already have a concept of packet ownership for accounting
      purposes in the skb->sk pointer, this is a natural place to store
      the classid in a persistent manner.
      This patch adds the cls_cgroup classid in struct sock, filling up
      an existing hole on 64-bit :)
      The value is set at socket creation time.  So all sockets created
      via socket(2) automatically gains the ID of the thread creating it.
      Whenever another process touches the socket by either reading or
      writing to it, we will change the socket classid to that of the
      process if it has a valid (non-zero) classid.
      For sockets created on inbound connections through accept(2), we
      inherit the classid of the original listening socket through
      sk_clone, possibly preceding the actual accept(2) call.
      In order to minimise risks, I have not made this the authoritative
      classid.  For now it is only used as a backup when we execute
      with soft IRQs disabled.  Once we're completely happy with its
      semantics we can use it as the sole classid.
      Footnote: I have rearranged the error path on cls_group module
      creation.  If we didn't do this, then there is a window where
      someone could create a tc rule using cls_group before the cgroup
      subsystem has been registered.
      Signed-off-by: default avatarHerbert Xu <herbert@gondor.apana.org.au>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  15. 17 May, 2010 1 commit
    • Eric Dumazet's avatar
      net: add a noref bit on skb dst · 7fee226a
      Eric Dumazet authored
      Use low order bit of skb->_skb_dst to tell dst is not refcounted.
      Change _skb_dst to _skb_refdst to make sure all uses are catched.
      skb_dst() returns the dst, regardless of noref bit set or not, but
      with a lockdep check to make sure a noref dst is not given if current
      user is not rcu protected.
      New skb_dst_set_noref() helper to set an notrefcounted dst on a skb.
      (with lockdep check)
      skb_dst_drop() drops a reference only if skb dst was refcounted.
      skb_dst_force() helper is used to force a refcount on dst, when skb
      is queued and not anymore RCU protected.
      Use skb_dst_force() in __sk_add_backlog(), __dev_xmit_skb() if
      !IFF_XMIT_DST_RELEASE or skb enqueued on qdisc queue, in
      sock_queue_rcv_skb(), in __nf_queue().
      Use skb_dst_force() in dev_requeue_skb().
      Note: dst_use_noref() still dirties dst, we might transform it
      later to do one dirtying per jiffies.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  16. 16 May, 2010 1 commit
    • Eric Dumazet's avatar
      net: Introduce sk_route_nocaps · a465419b
      Eric Dumazet authored
      TCP-MD5 sessions have intermittent failures, when route cache is
      invalidated. ip_queue_xmit() has to find a new route, calls
      sk_setup_caps(sk, &rt->u.dst), destroying the 
      sk->sk_route_caps &= ~NETIF_F_GSO_MASK
      that MD5 desperately try to make all over its way (from
      tcp_transmit_skb() for example)
      So we send few bad packets, and everything is fine when
      tcp_transmit_skb() is called again for this socket.
      Since ip_queue_xmit() is at a lower level than TCP-MD5, I chose to use a
      socket field, sk_route_nocaps, containing bits to mask on sk_route_caps.
      Reported-by: default avatarBhaskar Dutta <bhaskie@gmail.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  17. 02 May, 2010 1 commit
  18. 01 May, 2010 1 commit
    • Eric Dumazet's avatar
      net: sock_def_readable() and friends RCU conversion · 43815482
      Eric Dumazet authored
      sk_callback_lock rwlock actually protects sk->sk_sleep pointer, so we
      need two atomic operations (and associated dirtying) per incoming
      RCU conversion is pretty much needed :
      1) Add a new structure, called "struct socket_wq" to hold all fields
      that will need rcu_read_lock() protection (currently: a
      wait_queue_head_t and a struct fasync_struct pointer).
      [Future patch will add a list anchor for wakeup coalescing]
      2) Attach one of such structure to each "struct socket" created in
      3) Respect RCU grace period when freeing a "struct socket_wq"
      4) Change sk_sleep pointer in "struct sock" by sk_wq, pointer to "struct
      5) Change sk_sleep() function to use new sk->sk_wq instead of
      6) Change sk_has_sleeper() to wq_has_sleeper() that must be used inside
      a rcu_read_lock() section.
      7) Change all sk_has_sleeper() callers to :
        - Use rcu_read_lock() instead of read_lock(&sk->sk_callback_lock)
        - Use wq_has_sleeper() to eventually wakeup tasks.
        - Use rcu_read_unlock() instead of read_unlock(&sk->sk_callback_lock)
      8) sock_wake_async() is modified to use rcu protection as well.
      9) Exceptions :
        macvtap, drivers/net/tun.c, af_unix use integrated "struct socket_wq"
      instead of dynamically allocated ones. They dont need rcu freeing.
      Some cleanups or followups are probably needed, (possible
      sk_callback_lock conversion to a spinlock for example...).
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  19. 30 Apr, 2010 1 commit
  20. 28 Apr, 2010 1 commit
    • Eric Dumazet's avatar
      net: speedup udp receive path · 4b0b72f7
      Eric Dumazet authored
      Since commit 95766fff ([UDP]: Add memory accounting.), 
      each received packet needs one extra sock_lock()/sock_release() pair.
      This added latency because of possible backlog handling. Then later,
      ticket spinlocks added yet another latency source in case of DDOS.
      This patch introduces lock_sock_bh() and unlock_sock_bh()
      synchronization primitives, avoiding one atomic operation and backlog
      skb_free_datagram_locked() uses them instead of full blown
      lock_sock()/release_sock(). skb is orphaned inside locked section for
      proper socket memory reclaim, and finally freed outside of it.
      UDP receive path now take the socket spinlock only once.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  21. 27 Apr, 2010 3 commits
  22. 22 Apr, 2010 1 commit
    • Eric Dumazet's avatar
      dst: rcu check refinement · f68c224f
      Eric Dumazet authored
      __sk_dst_get() might be called from softirq, with socket lock held.
      [  159.026180] include/net/sock.h:1200 invoked rcu_dereference_check()
      without protection!
      [  159.026261] 
      [  159.026261] other info that might help us debug this:
      [  159.026263] 
      [  159.026425] 
      [  159.026426] rcu_scheduler_active = 1, debug_locks = 0
      [  159.026552] 2 locks held by swapper/0:
      [  159.026609]  #0:  (&icsk->icsk_retransmit_timer){+.-...}, at:
      [<ffffffff8104fc15>] run_timer_softirq+0x105/0x350
      [  159.026839]  #1:  (slock-AF_INET){+.-...}, at: [<ffffffff81392b8f>]
      [  159.027063] 
      [  159.027064] stack backtrace:
      [  159.027172] Pid: 0, comm: swapper Not tainted
      2.6.34-rc5-03707-gde498c89-dirty #36
      [  159.027252] Call Trace:
      [  159.027306]  <IRQ>  [<ffffffff810718ef>] lockdep_rcu_dereference
      [  159.027411]  [<ffffffff8138e4f7>] tcp_current_mss+0xa7/0xb0
      [  159.027537]  [<ffffffff8138fa49>] tcp_write_wakeup+0x89/0x190
      [  159.027600]  [<ffffffff81391936>] tcp_send_probe0+0x16/0x100
      [  159.027726]  [<ffffffff81392cd9>] tcp_write_timer+0x179/0x1e0
      [  159.027790]  [<ffffffff8104fca1>] run_timer_softirq+0x191/0x350
      [  159.027980]  [<ffffffff810477ed>] __do_softirq+0xcd/0x200
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  23. 20 Apr, 2010 1 commit
  24. 13 Apr, 2010 1 commit
    • Eric Dumazet's avatar
      net: sk_dst_cache RCUification · b6c6712a
      Eric Dumazet authored
      With latest CONFIG_PROVE_RCU stuff, I felt more comfortable to make this
      sk->sk_dst_cache is currently protected by a rwlock (sk_dst_lock)
      This rwlock is readlocked for a very small amount of time, and dst
      entries are already freed after RCU grace period. This calls for RCU
      again :)
      This patch converts sk_dst_lock to a spinlock, and use RCU for readers.
      __sk_dst_get() is supposed to be called with rcu_read_lock() or if
      socket locked by user, so use appropriate rcu_dereference_check()
      condition (rcu_read_lock_held() || sock_owned_by_user(sk))
      This patch avoids two atomic ops per tx packet on UDP connected sockets,
      for example, and permits sk_dst_lock to be much less dirtied.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  25. 30 Mar, 2010 1 commit
    • Tejun Heo's avatar
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo authored
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      The script does the followings.
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
      The conversion was done in the following steps.
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
      6. percpu.h was updated not to include slab.h.
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Guess-its-ok-by: default avatarChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
  26. 08 Mar, 2010 1 commit
  27. 05 Mar, 2010 2 commits
    • Zhu Yi's avatar
      net: backlog functions rename · a3a858ff
      Zhu Yi authored
      sk_add_backlog -> __sk_add_backlog
      sk_add_backlog_limited -> sk_add_backlog
      Signed-off-by: default avatarZhu Yi <yi.zhu@intel.com>
      Acked-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Zhu Yi's avatar
      net: add limit for socket backlog · 8eae939f
      Zhu Yi authored
      We got system OOM while running some UDP netperf testing on the loopback
      device. The case is multiple senders sent stream UDP packets to a single
      receiver via loopback on local host. Of course, the receiver is not able
      to handle all the packets in time. But we surprisingly found that these
      packets were not discarded due to the receiver's sk->sk_rcvbuf limit.
      Instead, they are kept queuing to sk->sk_backlog and finally ate up all
      the memory. We believe this is a secure hole that a none privileged user
      can crash the system.
      The root cause for this problem is, when the receiver is doing
      __release_sock() (i.e. after userspace recv, kernel udp_recvmsg ->
      skb_free_datagram_locked -> release_sock), it moves skbs from backlog to
      sk_receive_queue with the softirq enabled. In the above case, multiple
      busy senders will almost make it an endless loop. The skbs in the
      backlog end up eat all the system memory.
      The issue is not only for UDP. Any protocols using socket backlog is
      potentially affected. The patch adds limit for socket backlog so that
      the backlog size cannot be expanded endlessly.
      Reported-by: default avatarAlex Shi <alex.shi@intel.com>
      Cc: David Miller <davem@davemloft.net>
      Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
      Cc: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru
      Cc: "Pekka Savola (ipv6)" <pekkas@netcore.fi>
      Cc: Patrick McHardy <kaber@trash.net>
      Cc: Vlad Yasevich <vladislav.yasevich@hp.com>
      Cc: Sridhar Samudrala <sri@us.ibm.com>
      Cc: Jon Maloy <jon.maloy@ericsson.com>
      Cc: Allan Stephens <allan.stephens@windriver.com>
      Cc: Andrew Hendry <andrew.hendry@gmail.com>
      Signed-off-by: default avatarZhu Yi <yi.zhu@intel.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Acked-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  28. 22 Feb, 2010 1 commit
  29. 14 Feb, 2010 1 commit
  30. 10 Feb, 2010 1 commit
  31. 08 Nov, 2009 2 commits
    • Eric Dumazet's avatar
      udp: secondary hash on (local port, local address) · 512615b6
      Eric Dumazet authored
      Extends udp_table to contain a secondary hash table.
      socket anchor for this second hash is free, because UDP
      doesnt use skc_bind_node : We define an union to hold
      both skc_bind_node & a new hlist_nulls_node udp_portaddr_node
      udp_lib_get_port() inserts sockets into second hash chain
      (additional cost of one atomic op)
      udp_lib_unhash() deletes socket from second hash chain
      (additional cost of one atomic op)
      Note : No spinlock lockdep annotation is needed, because
      lock for the secondary hash chain is always get after
      lock for primary hash chain.
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      udp: split sk_hash into two u16 hashes · d4cada4a
      Eric Dumazet authored
      Union sk_hash with two u16 hashes for udp (no extra memory taken)
      One 16 bits hash on (local port) value (the previous udp 'hash')
      One 16 bits hash on (local address, local port) values, initialized
      but not yet used. This second hash is using jenkin hash for better
      Because the 'port' is xored later, a partial hash is performed
      on local address + net_hash_mix(net)
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  32. 20 Oct, 2009 1 commit
  33. 12 Oct, 2009 2 commits
    • Neil Horman's avatar
      net: Generalize socket rx gap / receive queue overflow cmsg · 3b885787
      Neil Horman authored
      Create a new socket level option to report number of queue overflows
      Recently I augmented the AF_PACKET protocol to report the number of frames lost
      on the socket receive queue between any two enqueued frames.  This value was
      exported via a SOL_PACKET level cmsg.  AFter I completed that work it was
      requested that this feature be generalized so that any datagram oriented socket
      could make use of this option.  As such I've created this patch, It creates a
      new SOL_SOCKET level option called SO_RXQ_OVFL, which when enabled exports a
      SOL_SOCKET level cmsg that reports the nubmer of times the sk_receive_queue
      overflowed between any two given frames.  It also augments the AF_PACKET
      protocol to take advantage of this new feature (as it previously did not touch
      sk->sk_drops, which this patch uses to record the overflow count).  Tested
      successfully by me.
      1) Unlike my previous patch, this patch simply records the sk_drops value, which
      is not a number of drops between packets, but rather a total number of drops.
      Deltas must be computed in user space.
      2) While this patch currently works with datagram oriented protocols, it will
      also be accepted by non-datagram oriented protocols. I'm not sure if thats
      agreeable to everyone, but my argument in favor of doing so is that, for those
      protocols which aren't applicable to this option, sk_drops will always be zero,
      and reporting no drops on a receive queue that isn't used for those
      non-participating protocols seems reasonable to me.  This also saves us having
      to code in a per-protocol opt in mechanism.
      3) This applies cleanly to net-next assuming that commit
      97775007 (my af packet cmsg patch) is reverted
      Signed-off-by: default avatarNeil Horman <nhorman@tuxdriver.com>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    • Eric Dumazet's avatar
      net: Fix struct sock bitfield annotation · 5fdb9973
      Eric Dumazet authored
      Since commit a98b65a3 (net: annotate struct sock bitfield), we lost
      8 bytes in struct sock on 64bit arches because of
      kmemcheck_bitfield_end(flags) misplacement.
      Fix this by putting together sk_shutdown, sk_no_check, sk_userlocks,
      sk_protocol and sk_type in the 'flags' 32bits bitfield
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  34. 06 Oct, 2009 1 commit
    • Eric Dumazet's avatar
      net: speedup sk_wake_async() · bcdce719
      Eric Dumazet authored
      An incoming datagram must bring into cpu cache *lot* of cache lines,
      in particular : (other parts omitted (hash chains, ip route cache...))
      On 32bit arches :
      offsetof(struct sock, sk_rcvbuf)       =0x30    (read)
      offsetof(struct sock, sk_lock)         =0x34   (rw)
      offsetof(struct sock, sk_sleep)        =0x50 (read)
      offsetof(struct sock, sk_rmem_alloc)   =0x64   (rw)
      offsetof(struct sock, sk_receive_queue)=0x74   (rw)
      offsetof(struct sock, sk_forward_alloc)=0x98   (rw)
      offsetof(struct sock, sk_callback_lock)=0xcc    (rw)
      offsetof(struct sock, sk_drops)        =0xd8 (read if we add dropcount support, rw if frame dropped)
      offsetof(struct sock, sk_filter)       =0xf8    (read)
      offsetof(struct sock, sk_socket)       =0x138 (read)
      offsetof(struct sock, sk_data_ready)   =0x15c   (read)
      We can avoid sk->sk_socket and socket->fasync_list referencing on sockets
      with no fasync() structures. (socket->fasync_list ptr is probably already in cache
      because it shares a cache line with socket->wait, ie location pointed by sk->sk_sleep)
      This avoids one cache line load per incoming packet for common cases (no fasync())
      We can leave (or even move in a future patch) sk->sk_socket in a cold location
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
  35. 30 Sep, 2009 1 commit