1. 29 Nov, 2011 1 commit
    • Eric Dumazet's avatar
      tcp: avoid frag allocation for small frames · f07d960d
      Eric Dumazet authored
      
      
      tcp_sendmsg() uses select_size() helper to choose skb head size when a
      new skb must be allocated.
      
      If GSO is enabled for the socket, current strategy is to force all
      payload data to be outside of headroom, in PAGE fragments.
      
      This strategy is not welcome for small packets, wasting memory.
      
      Experiments show that best results are obtained when using 2048 bytes
      for skb head (This includes the skb overhead and various headers)
      
      This patch provides better len/truesize ratios for packets sent to
      loopback device, and reduce memory needs for in-flight loopback packets,
      particularly on arches with big pages.
      
      If a sender sends many 1-byte packets to an unresponsive application,
      receiver rmem_alloc will grow faster and will stop queuing these packets
      sooner, or will collapse its receive queue to free excess memory.
      
      netperf -t TCP_RR results are improved by ~4 %, and many workloads are
      improved as well (tbench, mysql...)
      
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      f07d960d
  2. 28 Nov, 2011 1 commit
  3. 16 Nov, 2011 1 commit
  4. 24 Oct, 2011 1 commit
  5. 23 Oct, 2011 1 commit
  6. 21 Oct, 2011 1 commit
  7. 19 Oct, 2011 1 commit
  8. 03 Oct, 2011 1 commit
    • Eric Dumazet's avatar
      tcp: report ECN_SEEN in tcp_info · b5c5693b
      Eric Dumazet authored
      
      
      Allows ss command (iproute2) to display "ecnseen" if at least one packet
      with ECT(0) or ECT(1) or ECN was received by this socket.
      
      "ecn" means ECN was negotiated at session establishment (TCP level)
      
      "ecnseen" means we received at least one packet with ECT fields set (IP
      level)
      
      ss -i
      ...
      ESTAB      0      0   192.168.20.110:22  192.168.20.144:38016
      ino:5950 sk:f178e400
      	 mem:(r0,w0,f0,t0) ts sack ecn ecnseen bic wscale:7,8 rto:210
      rtt:12.5/7.5 cwnd:10 send 9.3Mbps rcv_space:14480
      
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      b5c5693b
  9. 27 Sep, 2011 1 commit
  10. 16 Sep, 2011 1 commit
  11. 24 Aug, 2011 1 commit
  12. 07 Jul, 2011 1 commit
  13. 04 Apr, 2011 1 commit
    • Tom Herbert's avatar
      net: Allow no-cache copy from user on transmit · c6e1a0d1
      Tom Herbert authored
      
      
      This patch uses __copy_from_user_nocache on transmit to bypass data
      cache for a performance improvement.  skb_add_data_nocache and
      skb_copy_to_page_nocache can be called by sendmsg functions to use
      this feature, initial support is in tcp_sendmsg.  This functionality is
      configurable per device using ethtool.
      
      Presumably, this feature would only be useful when the driver does
      not touch the data.  The feature is turned on by default if a device
      indicates that it does some form of checksum offload; it is off by
      default for devices that do no checksum offload or indicate no checksum
      is necessary.  For the former case copy-checksum is probably done
      anyway, in the latter case the device is likely loopback in which case
      the no cache copy is probably not beneficial.
      
      This patch was tested using 200 instances of netperf TCP_RR with
      1400 byte request and one byte reply.  Platform is 16 core AMD x86.
      
      No-cache copy disabled:
         672703 tps, 97.13% utilization
         50/90/99% latency:244.31 484.205 1028.41
      
      No-cache copy enabled:
         702113 tps, 96.16% utilization,
         50/90/99% latency 238.56 467.56 956.955
      
      Using 14000 byte request and response sizes demonstrate the
      effects more dramatically:
      
      No-cache copy disabled:
         79571 tps, 34.34 %utlization
         50/90/95% latency 1584.46 2319.59 5001.76
      
      No-cache copy enabled:
         83856 tps, 34.81% utilization
         50/90/95% latency 2508.42 2622.62 2735.88
      
      Note especially the effect on latency tail (95th percentile).
      
      This seems to provide a nice performance improvement and is
      consistent in the tests I ran.  Presumably, this would provide
      the greatest benfits in the presence of an application workload
      stressing the cache and a lot of transmit data happening.
      
      Signed-off-by: default avatarTom Herbert <therbert@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      c6e1a0d1
  14. 09 Mar, 2011 1 commit
  15. 20 Feb, 2011 1 commit
  16. 24 Jan, 2011 1 commit
  17. 24 Nov, 2010 1 commit
  18. 10 Nov, 2010 2 commits
  19. 09 Nov, 2010 1 commit
  20. 27 Sep, 2010 1 commit
  21. 20 Sep, 2010 1 commit
  22. 30 Aug, 2010 1 commit
    • Jerry Chu's avatar
      tcp: Add TCP_USER_TIMEOUT socket option. · dca43c75
      Jerry Chu authored
      
      
      This patch provides a "user timeout" support as described in RFC793. The
      socket option is also needed for the the local half of RFC5482 "TCP User
      Timeout Option".
      
      TCP_USER_TIMEOUT is a TCP level socket option that takes an unsigned int,
      when > 0, to specify the maximum amount of time in ms that transmitted
      data may remain unacknowledged before TCP will forcefully close the
      corresponding connection and return ETIMEDOUT to the application. If
      0 is given, TCP will continue to use the system default.
      
      Increasing the user timeouts allows a TCP connection to survive extended
      periods without end-to-end connectivity. Decreasing the user timeouts
      allows applications to "fail fast" if so desired. Otherwise it may take
      upto 20 minutes with the current system defaults in a normal WAN
      environment.
      
      The socket option can be made during any state of a TCP connection, but
      is only effective during the synchronized states of a connection
      (ESTABLISHED, FIN-WAIT-1, FIN-WAIT-2, CLOSE-WAIT, CLOSING, or LAST-ACK).
      Moreover, when used with the TCP keepalive (SO_KEEPALIVE) option,
      TCP_USER_TIMEOUT will overtake keepalive to determine when to close a
      connection due to keepalive failure.
      
      The option does not change in anyway when TCP retransmits a packet, nor
      when a keepalive probe will be sent.
      
      This option, like many others, will be inherited by an acceptor from its
      listener.
      
      Signed-off-by: default avatarH.K. Jerry Chu <hkchu@google.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      dca43c75
  23. 26 Aug, 2010 2 commits
    • KOSAKI Motohiro's avatar
      tcp: select(writefds) don't hang up when a peer close connection · d84ba638
      KOSAKI Motohiro authored
      This issue come from ruby language community. Below test program
      hang up when only run on Linux.
      
      	% uname -mrsv
      	Linux 2.6.26-2-486 #1 Sat Dec 26 08:37:39 UTC 2009 i686
      	% ruby -rsocket -ve '
      	BasicSocket.do_not_reverse_lookup = true
      	serv = TCPServer.open("127.0.0.1", 0)
      	s1 = TCPSocket.open("127.0.0.1", serv.addr[1])
      	s2 = serv.accept
      	s2.close
      	s1.write("a") rescue p $!
      	s1.write("a") rescue p $!
      	Thread.new {
      	  s1.write("a")
      	}.join'
      	ruby 1.9.3dev (2010-07-06 trunk 28554) [i686-linux]
      	#<Errno::EPIPE: Broken pipe>
      	[Hang Here]
      
      FreeBSD, Solaris, Mac doesn't. because Ruby's write() method call
      select() internally. and tcp_poll has a bug.
      
      SUS defined 'ready for writing' of select() as following.
      
      |  A descriptor shall be considered ready for writing when a call to an output
      |  function with O_NONBLOCK clear would not block, whether or not the function
      |  would transfer data successfully.
      
      That said, EPIPE situation is clearly one of 'ready for writing'.
      
      We don't have read-side issue because tcp_poll() already has read side
      shutdown care.
      
      |        if (sk->sk_shutdown & RCV_SHUTDOWN)
      |                mask |= POLLIN | POLLRDNORM | POLLRDHUP;
      
      So, Let's insert same logic in write side.
      
      - reference url
        http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31065
        http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/31068
      
      
      
      Signed-off-by: default avatarKOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      d84ba638
    • Eric Dumazet's avatar
      tcp: fix three tcp sysctls tuning · c5ed63d6
      Eric Dumazet authored
      
      
      As discovered by Anton Blanchard, current code to autotune 
      tcp_death_row.sysctl_max_tw_buckets, sysctl_tcp_max_orphans and
      sysctl_max_syn_backlog makes little sense.
      
      The bigger a page is, the less tcp_max_orphans is : 4096 on a 512GB
      machine in Anton's case.
      
      (tcp_hashinfo.bhash_size * sizeof(struct inet_bind_hashbucket))
      is much bigger if spinlock debugging is on. Its wrong to select bigger
      limits in this case (where kernel structures are also bigger)
      
      bhash_size max is 65536, and we get this value even for small machines. 
      
      A better ground is to use size of ehash table, this also makes code
      shorter and more obvious.
      
      Based on a patch from Anton, and another from David.
      
      Reported-and-tested-by: default avatarAnton Blanchard <anton@samba.org>
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      c5ed63d6
  24. 25 Aug, 2010 1 commit
  25. 02 Aug, 2010 1 commit
  26. 31 Jul, 2010 1 commit
  27. 14 Jul, 2010 1 commit
  28. 12 Jul, 2010 2 commits
  29. 29 Jun, 2010 1 commit
  30. 24 Jun, 2010 1 commit
    • Konstantin Khorenko's avatar
      tcp: do not send reset to already closed sockets · 565b7b2d
      Konstantin Khorenko authored
      
      
      i've found that tcp_close() can be called for an already closed
      socket, but still sends reset in this case (tcp_send_active_reset())
      which seems to be incorrect.  Moreover, a packet with reset is sent
      with different source port as original port number has been already
      cleared on socket.  Besides that incrementing stat counter for
      LINUX_MIB_TCPABORTONCLOSE also does not look correct in this case.
      
      Initially this issue was found on 2.6.18-x RHEL5 kernel, but the same
      seems to be true for the current mainstream kernel (checked on
      2.6.35-rc3).  Please, correct me if i missed something.
      
      How that happens:
      
      1) the server receives a packet for socket in TCP_CLOSE_WAIT state
         that triggers a tcp_reset():
      
      Call Trace:
       <IRQ>  [<ffffffff8025b9b9>] tcp_reset+0x12f/0x1e8
       [<ffffffff80046125>] tcp_rcv_state_process+0x1c0/0xa08
       [<ffffffff8003eb22>] tcp_v4_do_rcv+0x310/0x37a
       [<ffffffff80028bea>] tcp_v4_rcv+0x74d/0xb43
       [<ffffffff8024ef4c>] ip_local_deliver_finish+0x0/0x259
       [<ffffffff80037131>] ip_local_deliver+0x200/0x2f4
       [<ffffffff8003843c>] ip_rcv+0x64c/0x69f
       [<ffffffff80021d89>] netif_receive_skb+0x4c4/0x4fa
       [<ffffffff80032eca>] process_backlog+0x90/0xec
       [<ffffffff8000cc50>] net_rx_action+0xbb/0x1f1
       [<ffffffff80012d3a>] __do_softirq+0xf5/0x1ce
       [<ffffffff8001147a>] handle_IRQ_event+0x56/0xb0
       [<ffffffff8006334c>] call_softirq+0x1c/0x28
       [<ffffffff80070476>] do_softirq+0x2c/0x85
       [<ffffffff80070441>] do_IRQ+0x149/0x152
       [<ffffffff80062665>] ret_from_intr+0x0/0xa
       <EOI>  [<ffffffff80008a2e>] __handle_mm_fault+0x6cd/0x1303
       [<ffffffff80008903>] __handle_mm_fault+0x5a2/0x1303
       [<ffffffff80033a9d>] cache_free_debugcheck+0x21f/0x22e
       [<ffffffff8006a263>] do_page_fault+0x49a/0x7dc
       [<ffffffff80066487>] thread_return+0x89/0x174
       [<ffffffff800c5aee>] audit_syscall_exit+0x341/0x35c
       [<ffffffff80062e39>] error_exit+0x0/0x84
      
      tcp_rcv_state_process()
      ...  // (sk_state == TCP_CLOSE_WAIT here)
      ...
              /* step 2: check RST bit */
              if(th->rst) {
                      tcp_reset(sk);
                      goto discard;
              }
      ...
      ---------------------------------
      tcp_rcv_state_process
       tcp_reset
        tcp_done
         tcp_set_state(sk, TCP_CLOSE);
           inet_put_port
            __inet_put_port
             inet_sk(sk)->num = 0;
      
         sk->sk_shutdown = SHUTDOWN_MASK;
      
      2) After that the process (socket owner) tries to write something to
         that socket and "inet_autobind" sets a _new_ (which differs from
         the original!) port number for the socket:
      
       Call Trace:
        [<ffffffff80255a12>] inet_bind_hash+0x33/0x5f
        [<ffffffff80257180>] inet_csk_get_port+0x216/0x268
        [<ffffffff8026bcc9>] inet_autobind+0x22/0x8f
        [<ffffffff80049140>] inet_sendmsg+0x27/0x57
        [<ffffffff8003a9d9>] do_sock_write+0xae/0xea
        [<ffffffff80226ac7>] sock_writev+0xdc/0xf6
        [<ffffffff800680c7>] _spin_lock_irqsave+0x9/0xe
        [<ffffffff8001fb49>] __pollwait+0x0/0xdd
        [<ffffffff8008d533>] default_wake_function+0x0/0xe
        [<ffffffff800a4f10>] autoremove_wake_function+0x0/0x2e
        [<ffffffff800f0b49>] do_readv_writev+0x163/0x274
        [<ffffffff80066538>] thread_return+0x13a/0x174
        [<ffffffff800145d8>] tcp_poll+0x0/0x1c9
        [<ffffffff800c56d3>] audit_syscall_entry+0x180/0x1b3
        [<ffffffff800f0dd0>] sys_writev+0x49/0xe4
        [<ffffffff800622dd>] tracesys+0xd5/0xe0
      
      3) sendmsg fails at last with -EPIPE (=> 'write' returns -EPIPE in userspace):
      
      F: tcp_sendmsg1 -EPIPE: sk=ffff81000bda00d0, sport=49847, old_state=7, new_state=7, sk_err=0, sk_shutdown=3
      
      Call Trace:
       [<ffffffff80027557>] tcp_sendmsg+0xcb/0xe87
       [<ffffffff80033300>] release_sock+0x10/0xae
       [<ffffffff8016f20f>] vgacon_cursor+0x0/0x1a7
       [<ffffffff8026bd32>] inet_autobind+0x8b/0x8f
       [<ffffffff8003a9d9>] do_sock_write+0xae/0xea
       [<ffffffff80226ac7>] sock_writev+0xdc/0xf6
       [<ffffffff800680c7>] _spin_lock_irqsave+0x9/0xe
       [<ffffffff8001fb49>] __pollwait+0x0/0xdd
       [<ffffffff8008d533>] default_wake_function+0x0/0xe
       [<ffffffff800a4f10>] autoremove_wake_function+0x0/0x2e
       [<ffffffff800f0b49>] do_readv_writev+0x163/0x274
       [<ffffffff80066538>] thread_return+0x13a/0x174
       [<ffffffff800145d8>] tcp_poll+0x0/0x1c9
       [<ffffffff800c56d3>] audit_syscall_entry+0x180/0x1b3
       [<ffffffff800f0dd0>] sys_writev+0x49/0xe4
       [<ffffffff800622dd>] tracesys+0xd5/0xe0
      
      tcp_sendmsg()
      ...
              /* Wait for a connection to finish. */
              if ((1 << sk->sk_state) & ~(TCPF_ESTABLISHED | TCPF_CLOSE_WAIT)) {
                      int old_state = sk->sk_state;
                      if ((err = sk_stream_wait_connect(sk, &timeo)) != 0) {
      if (f_d && (err == -EPIPE)) {
              printk("F: tcp_sendmsg1 -EPIPE: sk=%p, sport=%u, old_state=%d, new_state=%d, "
                      "sk_err=%d, sk_shutdown=%d\n",
                      sk, ntohs(inet_sk(sk)->sport), old_state, sk->sk_state,
                      sk->sk_err, sk->sk_shutdown);
              dump_stack();
      }
                              goto out_err;
                      }
              }
      ...
      
      4) Then the process (socket owner) understands that it's time to close
         that socket and does that (and thus triggers sending reset packet):
      
      Call Trace:
      ...
       [<ffffffff80032077>] dev_queue_xmit+0x343/0x3d6
       [<ffffffff80034698>] ip_output+0x351/0x384
       [<ffffffff80251ae9>] dst_output+0x0/0xe
       [<ffffffff80036ec6>] ip_queue_xmit+0x567/0x5d2
       [<ffffffff80095700>] vprintk+0x21/0x33
       [<ffffffff800070f0>] check_poison_obj+0x2e/0x206
       [<ffffffff80013587>] poison_obj+0x36/0x45
       [<ffffffff8025dea6>] tcp_send_active_reset+0x15/0x14d
       [<ffffffff80023481>] dbg_redzone1+0x1c/0x25
       [<ffffffff8025dea6>] tcp_send_active_reset+0x15/0x14d
       [<ffffffff8000ca94>] cache_alloc_debugcheck_after+0x189/0x1c8
       [<ffffffff80023405>] tcp_transmit_skb+0x764/0x786
       [<ffffffff8025df8a>] tcp_send_active_reset+0xf9/0x14d
       [<ffffffff80258ff1>] tcp_close+0x39a/0x960
       [<ffffffff8026be12>] inet_release+0x69/0x80
       [<ffffffff80059b31>] sock_release+0x4f/0xcf
       [<ffffffff80059d4c>] sock_close+0x2c/0x30
       [<ffffffff800133c9>] __fput+0xac/0x197
       [<ffffffff800252bc>] filp_close+0x59/0x61
       [<ffffffff8001eff6>] sys_close+0x85/0xc7
       [<ffffffff800622dd>] tracesys+0xd5/0xe0
      
      So, in brief:
      
      * a received packet for socket in TCP_CLOSE_WAIT state triggers
        tcp_reset() which clears inet_sk(sk)->num and put socket into
        TCP_CLOSE state
      
      * an attempt to write to that socket forces inet_autobind() to get a
        new port (but the write itself fails with -EPIPE)
      
      * tcp_close() called for socket in TCP_CLOSE state sends an active
        reset via socket with newly allocated port
      
      This adds an additional check in tcp_close() for already closed
      sockets. We do not want to send anything to closed sockets.
      
      Signed-off-by: default avatarKonstantin Khorenko <khorenko@openvz.org>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      565b7b2d
  31. 15 Jun, 2010 1 commit
    • Changli Gao's avatar
      tcp: unify tcp flag macros · a3433f35
      Changli Gao authored
      
      
      unify tcp flag macros: TCPHDR_FIN, TCPHDR_SYN, TCPHDR_RST, TCPHDR_PSH,
      TCPHDR_ACK, TCPHDR_URG, TCPHDR_ECE and TCPHDR_CWR. TCBCB_FLAG_* are replaced
      with the corresponding TCPHDR_*.
      
      Signed-off-by: default avatarChangli Gao <xiaosuo@gmail.com>
      ----
       include/net/tcp.h                      |   24 ++++++-------
       net/ipv4/tcp.c                         |    8 ++--
       net/ipv4/tcp_input.c                   |    2 -
       net/ipv4/tcp_output.c                  |   59 ++++++++++++++++-----------------
       net/netfilter/nf_conntrack_proto_tcp.c |   32 ++++++-----------
       net/netfilter/xt_TCPMSS.c              |    4 --
       6 files changed, 58 insertions(+), 71 deletions(-)
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      a3433f35
  32. 31 May, 2010 1 commit
  33. 17 May, 2010 1 commit
  34. 16 May, 2010 1 commit
  35. 27 Apr, 2010 1 commit
  36. 20 Apr, 2010 2 commits