• Mike Marciniszyn's avatar
    staging/rdma/hfi1: Adaptive PIO for short messages · 14553ca1
    Mike Marciniszyn authored
    The change requires a new pio_busy field in the iowait structure to
    track the number of outstanding pios.  The new counter together
    with the sdma counter serve as the basis for a packet by packet decision
    as to which egress mechanism to use.  Since packets given to different
    egress mechanisms are not ordered, this scheme will preserve the order.
    
    The iowait drain/wait mechanisms are extended for a pio case.  An
    additional qp wait flag is added for the PIO drain wait case.
    
    Currently the only pio wait is for buffers, so the no_bufs_available()
    routine name is changed to pio_wait() and a third argument is passed
    with one of the two pio wait flags to generalize the routine.  A module
    parameter is added to hold a configurable threshold. For now, the
    module parameter is zero.
    
    A heuristic routine is added to return the func pointer of the proper
    egress routine to use.
    
    The heuristic is as follows:
    - SMI always uses pio
    - GSI,UD qps <= threshold use pio
    - UD qps > threadhold use sdma
      o No coordination with sdma is required because order is not required
        and this qp pio count is not maintained for UD
    - RC/UC ONLY packets <= threshold chose as follows:
      o If sdmas pending, use SDMA
      o Otherwise use pio and enable the pio tracking count at
        the time the pio buffer is allocated
    - RC/UC ONLY packets > threshold use SDMA
      o If pio's are pending the pio_wait with the new wait flag is
        called to delay for pios to drain
    
    The threshold is potentially reduced by the QP's mtu.
    
    The sc_buffer_alloc() has two additional args (a callback, a void *)
    which are exploited by the RC/UC cases to pass a new complete routine
    and a qp *.
    
    When the shadow ring completes the credit associated with a packet,
    the new complete routine is called.  The verbs_pio_complete() will then
    decrement the busy count and trigger any drain waiters in qp destroy
    or reset.
    Reviewed-by: default avatarJubin John <jubin.john@intel.com>
    Reviewed-by: default avatarDennis Dalessandro <dennis.dalessandro@intel.com>
    Signed-off-by: default avatarMike Marciniszyn <mike.marciniszyn@intel.com>
    Signed-off-by: default avatarDoug Ledford <dledford@redhat.com>
    14553ca1
Name
Last commit
Last update
..
ib.h Loading commit data...
ib_addr.h Loading commit data...
ib_cache.h Loading commit data...
ib_cm.h Loading commit data...
ib_fmr_pool.h Loading commit data...
ib_mad.h Loading commit data...
ib_marshall.h Loading commit data...
ib_pack.h Loading commit data...
ib_pma.h Loading commit data...
ib_sa.h Loading commit data...
ib_smi.h Loading commit data...
ib_umem.h Loading commit data...
ib_umem_odp.h Loading commit data...
ib_verbs.h Loading commit data...
iw_cm.h Loading commit data...
iw_portmap.h Loading commit data...
opa_port_info.h Loading commit data...
opa_smi.h Loading commit data...
rdma_cm.h Loading commit data...
rdma_cm_ib.h Loading commit data...
rdma_netlink.h Loading commit data...
rdma_vt.h Loading commit data...
rdmavt_cq.h Loading commit data...
rdmavt_mr.h Loading commit data...
rdmavt_qp.h Loading commit data...