1. 31 Jan, 2013 1 commit
  2. 19 Dec, 2012 1 commit
    • Jack Morgenstein's avatar
      mlx4_core: Adjustments to Flow Steering activation logic for SR-IOV · 7b8157be
      Jack Morgenstein authored
      
      
      Separate flow steering capability detection from the decision to activate.
      
      For the master (and for native), detect the flow steering capability
      in mlx4_dev_cap, but activate the appropriate steering type in a new
      function choose_flow_steering() based on detected data.
      
      For VFs, activate flow steering based on what was actually activated
      by the master, where that info is obtained via QUERY_HCA. This fixes
      the current VF detection which is wrongly based on QUERY_DEV_CAP.
      
      Also, for SR-IOV mode, if flow steering may be activated, do so only
      if the max number of QPs per rule is sufficient to satisfy one
      subscription per VF.  If not, fall back to B0 mode. This is needed to
      serve registrations done by L2 network drivers such as mlx4_en and
      IPoIB when the network stack attempts to join to multicast groups such
      as all-hosts or the IPoIB broadcast group.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@dev.mellanox.co.il>
      Signed-off-by: default avatarOr Gerlitz <ogerlitz@mellanox.com>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      7b8157be
  3. 26 Nov, 2012 1 commit
    • Or Gerlitz's avatar
      mlx4: 64-byte CQE/EQE support · 08ff3235
      Or Gerlitz authored
      
      
      ConnectX-3 devices can use either 64- or 32-byte completion queue
      entries (CQEs) and event queue entries (EQEs).  Using 64-byte
      EQEs/CQEs performs better because each entry is aligned to a complete
      cacheline.  This patch queries the HCA's capabilities, and if it
      supports 64-byte CQEs and EQES the driver will configure the HW to
      work in 64-byte mode.
      
      The 32-byte vs 64-byte mode is global per HCA and not per CQ or EQ.
      
      Since this mode is global, userspace (libmlx4) must be updated to work
      with the configured CQE size, and guests using SR-IOV virtual
      functions need to know both EQE and CQE size.
      
      In case one of the 64-byte CQE/EQE capabilities is activated, the
      patch makes sure that older guest drivers that use the QUERY_DEV_FUNC
      command (e.g as done in mlx4_core of Linux 3.3..3.6) will notice that
      they need an update to be able to work with the PPF. This is done by
      changing the returned pf_context_behaviour not to be zero any more. In
      case none of these capabilities is activated that value remains zero
      and older guest drivers can run OK.
      
      The SRIOV related flow is as follows
      
      1. the PPF does the detection of the new capabilities using
         QUERY_DEV_CAP command.
      
      2. the PPF activates the new capabilities using INIT_HCA.
      
      3. the VF detects if the PPF activated the capabilities using
         QUERY_HCA, and if this is the case activates them for itself too.
      
      Note that the VF detects that it must be aware to the new PF behaviour
      using QUERY_FUNC_CAP.  Steps 1 and 2 apply also for native mode.
      
      User space notification is done through a new field introduced in
      struct mlx4_ib_ucontext which holds device capabilities for which user
      space must take action. This changes the binary interface so the ABI
      towards libmlx4 exposed through uverbs is bumped from 3 to 4 but only
      when **needed** i.e. only when the driver does use 64-byte CQEs or
      future device capabilities which must be in sync by user space. This
      practice allows to work with unmodified libmlx4 on older devices (e.g
      A0, B0) which don't support 64-byte CQEs.
      
      In order to keep existing systems functional when they update to a
      newer kernel that contains these changes in VF and userspace ABI, a
      module parameter enable_64b_cqe_eqe must be set to enable 64-byte
      mode; the default is currently false.
      
      Signed-off-by: default avatarEli Cohen <eli@mellanox.com>
      Signed-off-by: default avatarOr Gerlitz <ogerlitz@mellanox.com>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      08ff3235
  4. 30 Sep, 2012 2 commits
    • Jack Morgenstein's avatar
      mlx4: Modify proxy/tunnel QP mechanism so that guests do no calculations · 47605df9
      Jack Morgenstein authored
      
      
      Previously, the structure of a guest's proxy QPs followed the
      structure of the PPF special qps (qp0 port 1, qp0 port 2, qp1 port 1,
      qp1 port 2, ...).  The guest then did offset calculations on the
      sqp_base qp number that the PPF passed to it in QUERY_FUNC_CAP().
      
      This is now changed so that the guest does no offset calculations
      regarding proxy or tunnel QPs to use.  This change frees the PPF from
      needing to adhere to a specific order in allocating proxy and tunnel
      QPs.
      
      Now QUERY_FUNC_CAP provides each port individually with its proxy
      qp0, proxy qp1, tunnel qp0, and tunnel qp1 QP numbers, and these are
      used directly where required (with no offset calculations).
      
      To accomplish this change, several fields were added to the phys_caps
      structure for use by the PPF and by non-SR-IOV mode:
      
          base_sqpn -- in non-sriov mode, this was formerly sqp_start.
          base_proxy_sqpn -- the first physical proxy qp number -- used by PPF
          base_tunnel_sqpn -- the first physical tunnel qp number -- used by PPF.
      
      The current code in the PPF still adheres to the previous layout of
      sqps, proxy-sqps and tunnel-sqps.  However, the PPF can change this
      layout without affecting VF or (paravirtualized) PF code.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@dev.mellanox.co.il>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      47605df9
    • Jack Morgenstein's avatar
      mlx4_core: Add proxy and tunnel QPs to the reserved QP area · e2c76824
      Jack Morgenstein authored
      
      
      In addition, pass the proxy and tunnel QP numbers to slaves so the
      driver can perform special QP paravirtualization.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@dev.mellanox.co.il>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      e2c76824
  5. 07 Jul, 2012 1 commit
    • Hadar Hen Zion's avatar
      {NET, IB}/mlx4: Add device managed flow steering firmware API · 0ff1fb65
      Hadar Hen Zion authored
      
      
      The driver is modified to support three operation modes.
      
      If supported by firmware use the device managed flow steering
      API, that which we call device managed steering mode. Else, if
      the firmware supports the B0 steering mode use it, and finally,
      if none of the above, use the A0 steering mode.
      
      When the steering mode is device managed, the code is modified
      such that L2 based rules set by the mlx4_en driver for Ethernet
      unicast and multicast, and the IB stack multicast attach calls
      done through the mlx4_ib driver are all routed to use the device
      managed API.
      
      When attaching rule using device managed flow steering API,
      the firmware returns a 64 bit registration id, which is to be
      provided during detach.
      
      Currently the firmware is always programmed during HCA initialization
      to use standard L2 hashing. Future work should be done to allow
      configuring the flow-steering hash function with common, non
      proprietary means.
      
      Signed-off-by: default avatarHadar Hen Zion <hadarh@mellanox.co.il>
      Signed-off-by: default avatarOr Gerlitz <ogerlitz@mellanox.com>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      0ff1fb65
  6. 08 May, 2012 1 commit
    • Shlomo Pongratz's avatar
      mlx4_core: Add second capabilities flags field · b3416f44
      Shlomo Pongratz authored
      
      
      This patch adds a 64-bit flags2 features member to struct mlx4_dev to
      export further features of the hardware.  The original flags field
      tracks features whose support bits are advertised by the firmware in
      offsets 0x40 and 0x44 of the query device capabilities command.
      flags2 will track features whose support bits are scattered at various
      offsets.
      
      RSS support is the first feature to be exported through flags2.  RSS
      capabilities are located at offset 0x2e.  The size of the RSS
      indirection table is also given in this offset.
      
      Signed-off-by: default avatarShlomo Pongratz <shlomop@mellanox.com>
      Signed-off-by: default avatarRoland Dreier <roland@purestorage.com>
      b3416f44
  7. 22 Jan, 2012 1 commit
  8. 19 Dec, 2011 1 commit
    • Yevgeny Petrilin's avatar
      mlx4_core: Changing link sensing logic · 8d0fc7b6
      Yevgeny Petrilin authored
      
      
      New FW can give clues to driver regarding default port type
      and whether or not we should default to link sensing on the port.
      
      2 bits are added to QUERY_PORT command:
      1. suggested_type: This bit gives a hint whether the default port type should be
         IB or Ethernet.
         The driver will use this hint in case the user didn't specify explicitly the link layer
         type he wants to set.
      2. default_sense: If this bit is set, we would sense the port type on start-up
         and default the port to link sensing
      
      Signed-off-by: default avatarYevgeny Petrilin <yevgenyp@mellanox.co.il>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      8d0fc7b6
  9. 13 Dec, 2011 2 commits
    • Jack Morgenstein's avatar
      mlx4_core: Modify driver initialization flow to accommodate SRIOV for Ethernet · ab9c17a0
      Jack Morgenstein authored
      
      
      1. Added module parameters sr_iov and probe_vf for controlling enablement of
         SRIOV mode.
      2. Increased default max num-qps, num-mpts and log_num_macs to accomodate
         SRIOV mode
      3. Added port_type_array as a module parameter to allow driver startup with
         ports configured as desired.
         In SRIOV mode, only ETH is supported, and this array is ignored; otherwise,
         for the case where the FW supports both port types (ETH and IB), the
         port_type_array parameter is used.
         By default, the port_type_array is set to configure both ports as IB.
      4. When running in sriov mode, the master needs to initialize the ICM eq table
         to hold the eq's for itself and also for all the slaves.
      5. mlx4_set_port_mask() now invoked from mlx4_init_hca, instead of in mlx4_dev_cap.
      6. Introduced sriov VF (slave) device startup/teardown logic (mainly procedures
         mlx4_init_slave, mlx4_slave_exit, mlx4_slave_cap, mlx4_slave_exit and flow
         modifications in __mlx4_init_one, mlx4_init_hca, and mlx4_setup_hca).
         VFs obtain their startup information from the PF (master) device via the
         comm channel.
      7. In SRIOV mode (both PF and VF), MSI_X must be enabled, or the driver
         aborts loading the device.
      8. Do not allow setting port type via sysfs when running in SRIOV mode.
      9. mlx4_get_ownership:  Currently, only one PF is supported by the driver.
         If the HCA is burned with FW which enables more than one PF, only one
         of the PFs is allowed to run.  The first one up grabs a FW ownership
         semaphone -- all other PFs will find that semaphore taken, and the
         driver will not allow them to run.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@dev.mellanox.co.il>
      Signed-off-by: default avatarYevgeny Petrilin <yevgenyp@mellanox.co.il>
      Signed-off-by: default avatarLiran Liss <liranl@mellanox.co.il>
      Signed-off-by: default avatarMarcel Apfelbaum <marcela@mellanox.co.il>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      ab9c17a0
    • Marcel Apfelbaum's avatar
      mlx4_core: Added FW commands and their wrappers for supporting SRIOV · 5cc914f1
      Marcel Apfelbaum authored
      
      
      The following commands are added here:
      1. QUERY_FUNC_CAP and its wrapper.  This function is used by VFs when
         they start up to receive configuration information from the PF, such
         as resource quotas for this VF, which ports should be used (currently
         two), what protocol is running on the port (currently Ethernet ONLY,
         or port not active).
      
      2. QUERY_PORT and its wrapper. Previously, this FW command was invoked directly
         by the ETH driver (en_port.c) using mlx4_cmd_box. Virtualization is now
         required here (the VF's MAC address must be substituted for the PFs
         MAC address returned by the FW). We changed the invocation
         in the ETH driver to use mlx4_QUERY_PORT, and added the wrapper.
      
      3. QUERY_HCA. Used by the VF to determine how the HCA was initialized.
         For now, we need only the multicast table member entry size
         (log2_mc_table_entry_sz, in the ConnectX PRM).  No wrapper is needed
         here, because the data may be passed as is to the VF without modification).
      
         In this command, we have added a GLOBAL_CAPS field for passing required
         configuration information from FW to a VF (this field is to allow safely
                         adding new SRIOV capabilities which require support in VF drivers, too).
         Bits will set here by FW in response to PF-driver configuration commands which
         will activate as yet undefined new SRIOV features. The VF will test to see that
         all required capabilities indicated by this field are supported (i.e., if a bit
         is set and the VF driver does not recognize that bit, it must abort
         its initialization).  Currently, no bits are set.
      
      4. Added a CLOSE_PORT wrapper.  The PF context needs to keep track of how many VF contexts
         have the port open.  The PF context will not actually issue the FW close port command
         until the last port user issues a CLOSE_PORT request.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@dev.mellanox.co.il>
      Signed-off-by: default avatarYevgeny Petrilin <yevgenyp@mellanox.co.il>
      Signed-off-by: default avatarMarcel Apfelbaum <marcela@mellanox.co.il>
      Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
      5cc914f1
  10. 11 Aug, 2011 1 commit
  11. 18 Jul, 2011 3 commits
  12. 23 Mar, 2011 2 commits
  13. 24 Aug, 2010 3 commits
  14. 22 Oct, 2008 3 commits
  15. 25 Jul, 2008 1 commit
  16. 23 Jul, 2008 1 commit
  17. 15 Jul, 2008 1 commit
    • Vladimir Sokolovsky's avatar
      mlx4_core: Use MOD_STAT_CFG command to get minimal page size · 2d928651
      Vladimir Sokolovsky authored
      
      
      There was a bug in some versions of the mlx4 driver in
      mlx4_alloc_fmr(), which hardcoded the minimum acceptable page_shift to
      be 12.  However, new ConnectX firmware can support a minimum
      page_shift of 9 (log_pg_sz of 9 returned by QUERY_DEV_LIM) -- so with
      old drivers, ib_fmr_alloc() would fail for ULPs using the device
      minimum when creating FMRs.
      
      To preserve firmware compatibility with released mlx4 drivers, the
      firmware will continue to return 12 as before for log_page_sz in
      QUERY_DEV_CAP for these drivers.  However, to enable new drivers to
      take advantage of the available smaller page size, the mlx4 driver now
      first sets the log_pg_sz to the device minimum by setting a
      log_page_sz value to 0 via the MOD_STAT_CFG command and then reading
      the real minimum via QUERY_DEV_CAP.
      
      Signed-off-by: default avatarJack Morgenstein <jackm@mellanox.co.il>
      Signed-off-by: default avatarVladimir Sokolovsky <vlad@mellanox.co.il>
      Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      2d928651
  18. 16 Apr, 2008 1 commit
  19. 04 Feb, 2008 1 commit
  20. 09 Jul, 2007 1 commit
  21. 18 Jun, 2007 1 commit
    • Roland Dreier's avatar
      IB/mlx4: Handle FW command interface rev 3 · 5ae2a7a8
      Roland Dreier authored
      
      
      Upcoming firmware introduces command interface revision 3, which
      changes the way port capabilities are queried and set.  Update the
      driver to handle both the new and old command interfaces by adding a
      new MLX4_FLAG_OLD_PORT_CMDS that it is set after querying the firmware
      interface revision and then using the correct interface based on the
      setting of the flag.
      
      Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      5ae2a7a8
  22. 08 May, 2007 1 commit
    • Roland Dreier's avatar
      IB/mlx4: Add a driver Mellanox ConnectX InfiniBand adapters · 225c7b1f
      Roland Dreier authored
      
      
      Add an InfiniBand driver for Mellanox ConnectX adapters.  Because
      these adapters can also be used as ethernet NICs and Fibre Channel 
      HBAs, the driver is split into two modules: 
       
        mlx4_core: Handles low-level things like device initialization and 
          processing firmware commands.  Also controls resource allocation 
          so that the InfiniBand, ethernet and FC functions can share a 
          device without stepping on each other. 
       
        mlx4_ib: Handles InfiniBand-specific things; plugs into the 
          InfiniBand midlayer. 
      
      Signed-off-by: default avatarRoland Dreier <rolandd@cisco.com>
      225c7b1f