Skip to content
  • Vasu Dev's avatar
    [SCSI] fcoe, libfc: adds per cpu exch pool within exchange manager(EM) · e4bc50be
    Vasu Dev authored
    
    
    Adds per cpu exch pool for these reasons:-
    
     1. Currently an EM instance is shared across all cpus to manage
        all exches for all cpus. This required em_lock across all
        cpus for an exch alloc, free, lookup and reset each frame
        and that made em_lock expensive, so instead having per cpu
        exch pool with their own per cpu pool lock will likely reduce
        locking contention in fast path for an exch alloc, free and
        lookup.
    
     2. Per cpu exch pool will likely improve cache hit ratio since
        all frames of an exch will be processed on the same cpu on
        which exch originated.
    
    This patch is only prep work to help in keeping complexity of next
    patch low, so this patch only sets up per cpu exch pool and related
    helper funcs to be used by next patch. The next patch fully makes
    use of per cpu exch pool in all code paths ie. tx, rx and reset.
    
    Divides per EM exch id range equally across all cpus to setup per
    cpu exch pool. This division is such that lower bits of exch id
    carries cpu number info on which exch originated, later a simple
    bitwise AND operation on exch id of incoming frame with fc_cpu_mask
    retrieves cpu number info to direct all frames to same cpu on which
    exch originated. This required a global fc_cpu_mask and fc_cpu_order
    initialized to max possible cpus number nr_cpu_ids rounded up to 2's
    power, this will be used in mapping exch id and exch ptr array
    index in pool during exch allocation, find or reset code paths.
    
    Adds a check in fc_exch_mgr_alloc() to ensure specified min_xid
    lower bits are zero since these bits are used to carry cpu info.
    
    Adds and initializes struct fc_exch_pool with all required fields
    to manage exches in pool.
    
    Allocates per cpu struct fc_exch_pool with memory for exches array
    for range of exches per pool. The exches array memory is followed
    by struct fc_exch_pool.
    
    Adds fc_exch_ptr_get/set() helper functions to get/set exch ptr in
    pool exches array at specified array index.
    
    Increases default FCOE_MAX_XID to 0x0FFF from 0x07EF, so that more
    exches are available per cpu after above described exch id range
    division across all cpus to each pool.
    
    Signed-off-by: default avatarVasu Dev <vasu.dev@intel.com>
    Signed-off-by: default avatarRobert Love <robert.w.love@intel.com>
    Signed-off-by: default avatarJames Bottomley <James.Bottomley@suse.de>
    e4bc50be