Skip to content
  • Tom Herbert's avatar
    xps: Transmit Packet Steering · 1d24eb48
    Tom Herbert authored
    
    
    This patch implements transmit packet steering (XPS) for multiqueue
    devices.  XPS selects a transmit queue during packet transmission based
    on configuration.  This is done by mapping the CPU transmitting the
    packet to a queue.  This is the transmit side analogue to RPS-- where
    RPS is selecting a CPU based on receive queue, XPS selects a queue
    based on the CPU (previously there was an XPS patch from Eric
    Dumazet, but that might more appropriately be called transmit completion
    steering).
    
    Each transmit queue can be associated with a number of CPUs which will
    use the queue to send packets.  This is configured as a CPU mask on a
    per queue basis in:
    
    /sys/class/net/eth<n>/queues/tx-<n>/xps_cpus
    
    The mappings are stored per device in an inverted data structure that
    maps CPUs to queues.  In the netdevice structure this is an array of
    num_possible_cpu structures where each structure holds and array of
    queue_indexes for queues which that CPU can use.
    
    The benefits of XPS are improved locality in the per queue data
    structures.  Also, transmit completions are more likely to be done
    nearer to the sending thread, so this should promote locality back
    to the socket on free (e.g. UDP).  The benefits of XPS are dependent on
    cache hierarchy, application load, and other factors.  XPS would
    nominally be configured so that a queue would only be shared by CPUs
    which are sharing a cache, the degenerative configuration woud be that
    each CPU has it's own queue.
    
    Below are some benchmark results which show the potential benfit of
    this patch.  The netperf test has 500 instances of netperf TCP_RR test
    with 1 byte req. and resp.
    
    bnx2x on 16 core AMD
       XPS (16 queues, 1 TX queue per CPU)  1234K at 100% CPU
       No XPS (16 queues)                   996K at 100% CPU
    
    Signed-off-by: default avatarTom Herbert <therbert@google.com>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
    1d24eb48