Commit b5b131c7 authored by Linus Torvalds's avatar Linus Torvalds
Browse files

Merge tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dma

Pull dmaengine updates from Vinod Koul:
 "This is smallish update with minor changes to core and new driver and
  usual updates.  Nothing super exciting here..

   - We have made slave address as physical to enable driver to do the
     mapping.

   - We now expose the maxburst for slave dma as new capability so
     clients can know this and program accordingly

   - addition of device synchronize callbacks on omap and edma.

   - pl330 updates to support DMAFLUSHP for Rockchip platforms.

   - Updates and improved sg handling in Xilinx VDMA driver.

   - New hidma qualcomm dma driver, though some bits are still in
     progress"

* tag 'dmaengine-4.6-rc1' of git://git.infradead.org/users/vkoul/slave-dma: (40 commits)
  dmaengine: IOATDMA: revise channel reset workaround on CB3.3 platforms
  dmaengine: add Qualcomm Technologies HIDMA channel driver
  dmaengine: add Qualcomm Technologies HIDMA management driver
  dmaengine: hidma: Add Device Tree binding
  dmaengine: qcom_bam_dma: move to qcom directory
  dmaengine: tegra: Move of_device_id table near to its user
  dmaengine: xilinx_vdma: Remove unnecessary variable initializations
  dmaengine: sirf: use __maybe_unused to hide pm functions
  dmaengine: rcar-dmac: clear pertinence number of channels
  dmaengine: sh: shdmac: don't open code of_device_get_match_data()
  dmaengine: tegra: don't open code of_device_get_match_data()
  dmaengine: qcom_bam_dma: Make driver work for BE
  dmaengine: sun4i: support module autoloading
  dma/mic_x100_dma: IS_ERR() vs PTR_ERR() typo
  dmaengine: xilinx_vdma: Use readl_poll_timeout instead of do while loop's
  dmaengine: xilinx_vdma: Simplify spin lock handling
  dmaengine: xilinx_vdma: Fix issues with non-parking mode
  dmaengine: xilinx_vdma: Improve SG engine handling
  dmaengine: pl330: fix to support the burst mode
  dmaengine: make slave address physical
  ...
parents c7eec380 896e041e
What: /sys/devices/platform/hidma-mgmt*/chanops/chan*/priority
/sys/devices/platform/QCOM8060:*/chanops/chan*/priority
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains either 0 or 1 and indicates if the DMA channel is a
low priority (0) or high priority (1) channel.
What: /sys/devices/platform/hidma-mgmt*/chanops/chan*/weight
/sys/devices/platform/QCOM8060:*/chanops/chan*/weight
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains 0..15 and indicates the weight of the channel among
equal priority channels during round robin scheduling.
What: /sys/devices/platform/hidma-mgmt*/chreset_timeout_cycles
/sys/devices/platform/QCOM8060:*/chreset_timeout_cycles
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains the platform specific cycle value to wait after a
reset command is issued. If the value is chosen too short,
then the HW will issue a reset failure interrupt. The value
is platform specific and should not be changed without
consultance.
What: /sys/devices/platform/hidma-mgmt*/dma_channels
/sys/devices/platform/QCOM8060:*/dma_channels
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains the number of dma channels supported by one instance
of HIDMA hardware. The value may change from chip to chip.
What: /sys/devices/platform/hidma-mgmt*/hw_version_major
/sys/devices/platform/QCOM8060:*/hw_version_major
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Version number major for the hardware.
What: /sys/devices/platform/hidma-mgmt*/hw_version_minor
/sys/devices/platform/QCOM8060:*/hw_version_minor
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Version number minor for the hardware.
What: /sys/devices/platform/hidma-mgmt*/max_rd_xactions
/sys/devices/platform/QCOM8060:*/max_rd_xactions
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains a value between 0 and 31. Maximum number of
read transactions that can be issued back to back.
Choosing a higher number gives better performance but
can also cause performance reduction to other peripherals
sharing the same bus.
What: /sys/devices/platform/hidma-mgmt*/max_read_request
/sys/devices/platform/QCOM8060:*/max_read_request
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Size of each read request. The value needs to be a power
of two and can be between 128 and 1024.
What: /sys/devices/platform/hidma-mgmt*/max_wr_xactions
/sys/devices/platform/QCOM8060:*/max_wr_xactions
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Contains a value between 0 and 31. Maximum number of
write transactions that can be issued back to back.
Choosing a higher number gives better performance but
can also cause performance reduction to other peripherals
sharing the same bus.
What: /sys/devices/platform/hidma-mgmt*/max_write_request
/sys/devices/platform/QCOM8060:*/max_write_request
Date: Nov 2015
KernelVersion: 4.4
Contact: "Sinan Kaya <okaya@cudeaurora.org>"
Description:
Size of each write request. The value needs to be a power
of two and can be between 128 and 1024.
......@@ -15,6 +15,7 @@ Optional properties:
cells in the dmas property of client device.
- dma-channels: contains the total number of DMA channels supported by the DMAC
- dma-requests: contains the total number of DMA requests supported by the DMAC
- arm,pl330-broken-no-flushp: quirk for avoiding to execute DMAFLUSHP
Example:
......
Qualcomm Technologies HIDMA Management interface
Qualcomm Technologies HIDMA is a high speed DMA device. It only supports
memcpy and memset capabilities. It has been designed for virtualized
environments.
Each HIDMA HW instance consists of multiple DMA channels. These channels
share the same bandwidth. The bandwidth utilization can be parititioned
among channels based on the priority and weight assignments.
There are only two priority levels and 15 weigh assignments possible.
Other parameters here determine how much of the system bus this HIDMA
instance can use like maximum read/write request and and number of bytes to
read/write in a single burst.
Main node required properties:
- compatible: "qcom,hidma-mgmt-1.0";
- reg: Address range for DMA device
- dma-channels: Number of channels supported by this DMA controller.
- max-write-burst-bytes: Maximum write burst in bytes that HIDMA can
occupy the bus for in a single transaction. A memcpy requested is
fragmented to multiples of this amount. This parameter is used while
writing into destination memory. Setting this value incorrectly can
starve other peripherals in the system.
- max-read-burst-bytes: Maximum read burst in bytes that HIDMA can
occupy the bus for in a single transaction. A memcpy request is
fragmented to multiples of this amount. This parameter is used while
reading the source memory. Setting this value incorrectly can starve
other peripherals in the system.
- max-write-transactions: This value is how many times a write burst is
applied back to back while writing to the destination before yielding
the bus.
- max-read-transactions: This value is how many times a read burst is
applied back to back while reading the source before yielding the bus.
- channel-reset-timeout-cycles: Channel reset timeout in cycles for this SOC.
Once a reset is applied to the HW, HW starts a timer for reset operation
to confirm. If reset is not completed within this time, HW reports reset
failure.
Sub-nodes:
HIDMA has one or more DMA channels that are used to move data from one
memory location to another.
When the OS is not in control of the management interface (i.e. it's a guest),
the channel nodes appear on their own, not under a management node.
Required properties:
- compatible: must contain "qcom,hidma-1.0"
- reg: Addresses for the transfer and event channel
- interrupts: Should contain the event interrupt
- desc-count: Number of asynchronous requests this channel can handle
- iommus: required a iommu node
Example:
Hypervisor OS configuration:
hidma-mgmt@f9984000 = {
compatible = "qcom,hidma-mgmt-1.0";
reg = <0xf9984000 0x15000>;
dma-channels = <6>;
max-write-burst-bytes = <1024>;
max-read-burst-bytes = <1024>;
max-write-transactions = <31>;
max-read-transactions = <31>;
channel-reset-timeout-cycles = <0x500>;
hidma_24: dma-controller@0x5c050000 {
compatible = "qcom,hidma-1.0";
reg = <0 0x5c050000 0x0 0x1000>,
<0 0x5c0b0000 0x0 0x1000>;
interrupts = <0 389 0>;
desc-count = <10>;
iommus = <&system_mmu>;
};
};
Guest OS configuration:
hidma_24: dma-controller@0x5c050000 {
compatible = "qcom,hidma-1.0";
reg = <0 0x5c050000 0x0 0x1000>,
<0 0x5c0b0000 0x0 0x1000>;
interrupts = <0 389 0>;
desc-count = <10>;
iommus = <&system_mmu>;
};
......@@ -341,12 +341,13 @@ config MV_XOR
config MXS_DMA
bool "MXS DMA support"
depends on SOC_IMX23 || SOC_IMX28 || SOC_IMX6Q
depends on SOC_IMX23 || SOC_IMX28 || SOC_IMX6Q || SOC_IMX6UL
select STMP_DEVICE
select DMA_ENGINE
help
Support the MXS DMA engine. This engine including APBH-DMA
and APBX-DMA is integrated into Freescale i.MX23/28/MX6Q/MX6DL chips.
and APBX-DMA is integrated into Freescale
i.MX23/28/MX6Q/MX6DL/MX6UL chips.
config MX3_IPU
bool "MX3x Image Processing Unit support"
......@@ -408,15 +409,6 @@ config PXA_DMA
16 to 32 channels for peripheral to memory or memory to memory
transfers.
config QCOM_BAM_DMA
tristate "QCOM BAM DMA support"
depends on ARCH_QCOM || (COMPILE_TEST && OF && ARM)
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
---help---
Enable support for the QCOM BAM DMA controller. This controller
provides DMA capabilities for a variety of on-chip devices.
config SIRF_DMA
tristate "CSR SiRFprimaII/SiRFmarco DMA support"
depends on ARCH_SIRF
......@@ -539,6 +531,8 @@ config ZX_DMA
# driver files
source "drivers/dma/bestcomm/Kconfig"
source "drivers/dma/qcom/Kconfig"
source "drivers/dma/dw/Kconfig"
source "drivers/dma/hsu/Kconfig"
......
......@@ -52,7 +52,6 @@ obj-$(CONFIG_PCH_DMA) += pch_dma.o
obj-$(CONFIG_PL330_DMA) += pl330.o
obj-$(CONFIG_PPC_BESTCOMM) += bestcomm/
obj-$(CONFIG_PXA_DMA) += pxa_dma.o
obj-$(CONFIG_QCOM_BAM_DMA) += qcom_bam_dma.o
obj-$(CONFIG_RENESAS_DMA) += sh/
obj-$(CONFIG_SIRF_DMA) += sirf-dma.o
obj-$(CONFIG_STE_DMA40) += ste_dma40.o ste_dma40_ll.o
......@@ -67,4 +66,5 @@ obj-$(CONFIG_TI_EDMA) += edma.o
obj-$(CONFIG_XGENE_DMA) += xgene-dma.o
obj-$(CONFIG_ZX_DMA) += zx296702_dma.o
obj-y += qcom/
obj-y += xilinx/
......@@ -438,7 +438,7 @@ struct dma_chan *acpi_dma_request_slave_chan_by_name(struct device *dev,
return ERR_PTR(-ENODEV);
}
dev_dbg(dev, "found DMA channel \"%s\" at index %d\n", name, index);
dev_dbg(dev, "Looking for DMA channel \"%s\" at index %d...\n", name, index);
return acpi_dma_request_slave_chan_by_index(dev, index);
}
EXPORT_SYMBOL_GPL(acpi_dma_request_slave_chan_by_name);
......
......@@ -496,6 +496,7 @@ int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps)
caps->src_addr_widths = device->src_addr_widths;
caps->dst_addr_widths = device->dst_addr_widths;
caps->directions = device->directions;
caps->max_burst = device->max_burst;
caps->residue_granularity = device->residue_granularity;
caps->descriptor_reuse = device->descriptor_reuse;
......
......@@ -150,7 +150,7 @@ enum dw_dma_msize {
#define DWC_CTLL_DST_INC (0<<7) /* DAR update/not */
#define DWC_CTLL_DST_DEC (1<<7)
#define DWC_CTLL_DST_FIX (2<<7)
#define DWC_CTLL_SRC_INC (0<<7) /* SAR update/not */
#define DWC_CTLL_SRC_INC (0<<9) /* SAR update/not */
#define DWC_CTLL_SRC_DEC (1<<9)
#define DWC_CTLL_SRC_FIX (2<<9)
#define DWC_CTLL_DST_MSIZE(n) ((n)<<11) /* burst, #elements */
......
......@@ -869,6 +869,13 @@ static int edma_terminate_all(struct dma_chan *chan)
return 0;
}
static void edma_synchronize(struct dma_chan *chan)
{
struct edma_chan *echan = to_edma_chan(chan);
vchan_synchronize(&echan->vchan);
}
static int edma_slave_config(struct dma_chan *chan,
struct dma_slave_config *cfg)
{
......@@ -1365,36 +1372,36 @@ static struct dma_async_tx_descriptor *edma_prep_dma_cyclic(
static void edma_completion_handler(struct edma_chan *echan)
{
struct device *dev = echan->vchan.chan.device->dev;
struct edma_desc *edesc = echan->edesc;
if (!edesc)
return;
struct edma_desc *edesc;
spin_lock(&echan->vchan.lock);
if (edesc->cyclic) {
vchan_cyclic_callback(&edesc->vdesc);
spin_unlock(&echan->vchan.lock);
return;
} else if (edesc->processed == edesc->pset_nr) {
edesc->residue = 0;
edma_stop(echan);
vchan_cookie_complete(&edesc->vdesc);
echan->edesc = NULL;
dev_dbg(dev, "Transfer completed on channel %d\n",
echan->ch_num);
} else {
dev_dbg(dev, "Sub transfer completed on channel %d\n",
echan->ch_num);
edma_pause(echan);
/* Update statistics for tx_status */
edesc->residue -= edesc->sg_len;
edesc->residue_stat = edesc->residue;
edesc->processed_stat = edesc->processed;
edesc = echan->edesc;
if (edesc) {
if (edesc->cyclic) {
vchan_cyclic_callback(&edesc->vdesc);
spin_unlock(&echan->vchan.lock);
return;
} else if (edesc->processed == edesc->pset_nr) {
edesc->residue = 0;
edma_stop(echan);
vchan_cookie_complete(&edesc->vdesc);
echan->edesc = NULL;
dev_dbg(dev, "Transfer completed on channel %d\n",
echan->ch_num);
} else {
dev_dbg(dev, "Sub transfer completed on channel %d\n",
echan->ch_num);
edma_pause(echan);
/* Update statistics for tx_status */
edesc->residue -= edesc->sg_len;
edesc->residue_stat = edesc->residue;
edesc->processed_stat = edesc->processed;
}
edma_execute(echan);
}
edma_execute(echan);
spin_unlock(&echan->vchan.lock);
}
......@@ -1837,6 +1844,7 @@ static void edma_dma_init(struct edma_cc *ecc, bool legacy_mode)
s_ddev->device_pause = edma_dma_pause;
s_ddev->device_resume = edma_dma_resume;
s_ddev->device_terminate_all = edma_terminate_all;
s_ddev->device_synchronize = edma_synchronize;
s_ddev->src_addr_widths = EDMA_DMA_BUSWIDTHS;
s_ddev->dst_addr_widths = EDMA_DMA_BUSWIDTHS;
......@@ -1862,6 +1870,7 @@ static void edma_dma_init(struct edma_cc *ecc, bool legacy_mode)
m_ddev->device_pause = edma_dma_pause;
m_ddev->device_resume = edma_dma_resume;
m_ddev->device_terminate_all = edma_terminate_all;
m_ddev->device_synchronize = edma_synchronize;
m_ddev->src_addr_widths = EDMA_DMA_BUSWIDTHS;
m_ddev->dst_addr_widths = EDMA_DMA_BUSWIDTHS;
......
......@@ -421,23 +421,25 @@ static int m2p_hw_interrupt(struct ep93xx_dma_chan *edmac)
desc->size);
}
switch (irq_status & (M2P_INTERRUPT_STALL | M2P_INTERRUPT_NFB)) {
case M2P_INTERRUPT_STALL:
/* Disable interrupts */
control = readl(edmac->regs + M2P_CONTROL);
control &= ~(M2P_CONTROL_STALLINT | M2P_CONTROL_NFBINT);
m2p_set_control(edmac, control);
return INTERRUPT_DONE;
case M2P_INTERRUPT_NFB:
if (ep93xx_dma_advance_active(edmac))
m2p_fill_desc(edmac);
/*
* Even latest E2 silicon revision sometimes assert STALL interrupt
* instead of NFB. Therefore we treat them equally, basing on the
* amount of data we still have to transfer.
*/
if (!(irq_status & (M2P_INTERRUPT_STALL | M2P_INTERRUPT_NFB)))
return INTERRUPT_UNKNOWN;
if (ep93xx_dma_advance_active(edmac)) {
m2p_fill_desc(edmac);
return INTERRUPT_NEXT_BUFFER;
}
return INTERRUPT_UNKNOWN;
/* Disable interrupts */
control = readl(edmac->regs + M2P_CONTROL);
control &= ~(M2P_CONTROL_STALLINT | M2P_CONTROL_NFBINT);
m2p_set_control(edmac, control);
return INTERRUPT_DONE;
}
/*
......
......@@ -289,6 +289,9 @@ static void idma64_desc_fill(struct idma64_chan *idma64c,
/* Trigger an interrupt after the last block is transfered */
lli->ctllo |= IDMA64C_CTLL_INT_EN;
/* Disable LLP transfer in the last block */
lli->ctllo &= ~(IDMA64C_CTLL_LLP_S_EN | IDMA64C_CTLL_LLP_D_EN);
}
static struct dma_async_tx_descriptor *idma64_prep_slave_sg(
......
......@@ -71,7 +71,7 @@
#define IDMA64C_CFGH_SRC_PER(x) ((x) << 0) /* src peripheral */
#define IDMA64C_CFGH_DST_PER(x) ((x) << 4) /* dst peripheral */
#define IDMA64C_CFGH_RD_ISSUE_THD(x) ((x) << 8)
#define IDMA64C_CFGH_RW_ISSUE_THD(x) ((x) << 18)
#define IDMA64C_CFGH_WR_ISSUE_THD(x) ((x) << 18)
/* Interrupt registers */
......
......@@ -31,6 +31,7 @@
#include <linux/dma-mapping.h>
#include <linux/workqueue.h>
#include <linux/prefetch.h>
#include <linux/sizes.h>
#include "dma.h"
#include "registers.h"
#include "hw.h"
......@@ -290,24 +291,30 @@ static dma_cookie_t ioat_tx_submit_unlock(struct dma_async_tx_descriptor *tx)
}
static struct ioat_ring_ent *
ioat_alloc_ring_ent(struct dma_chan *chan, gfp_t flags)
ioat_alloc_ring_ent(struct dma_chan *chan, int idx, gfp_t flags)
{
struct ioat_dma_descriptor *hw;
struct ioat_ring_ent *desc;
struct ioatdma_device *ioat_dma;
struct ioatdma_chan *ioat_chan = to_ioat_chan(chan);
int chunk;
dma_addr_t phys;
u8 *pos;
off_t offs;
ioat_dma = to_ioatdma_device(chan->device);
hw = pci_pool_alloc(ioat_dma->dma_pool, flags, &phys);
if (!hw)
return NULL;
chunk = idx / IOAT_DESCS_PER_2M;
idx &= (IOAT_DESCS_PER_2M - 1);
offs = idx * IOAT_DESC_SZ;
pos = (u8 *)ioat_chan->descs[chunk].virt + offs;
phys = ioat_chan->descs[chunk].hw + offs;
hw = (struct ioat_dma_descriptor *)pos;
memset(hw, 0, sizeof(*hw));
desc = kmem_cache_zalloc(ioat_cache, flags);
if (!desc) {
pci_pool_free(ioat_dma->dma_pool, hw, phys);
if (!desc)
return NULL;
}
dma_async_tx_descriptor_init(&desc->txd, chan);
desc->txd.tx_submit = ioat_tx_submit_unlock;
......@@ -318,32 +325,63 @@ ioat_alloc_ring_ent(struct dma_chan *chan, gfp_t flags)
void ioat_free_ring_ent(struct ioat_ring_ent *desc, struct dma_chan *chan)
{
struct ioatdma_device *ioat_dma;
ioat_dma = to_ioatdma_device(chan->device);
pci_pool_free(ioat_dma->dma_pool, desc->hw, desc->txd.phys);
kmem_cache_free(ioat_cache, desc);
}
struct ioat_ring_ent **
ioat_alloc_ring(struct dma_chan *c, int order, gfp_t flags)
{
struct ioatdma_chan *ioat_chan = to_ioat_chan(c);
struct ioat_ring_ent **ring;
int descs = 1 << order;
int i;
if (order > ioat_get_max_alloc_order())
return NULL;
int total_descs = 1 << order;
int i, chunks;
/* allocate the array to hold the software ring */
ring = kcalloc(descs, sizeof(*ring), flags);
ring = kcalloc(total_descs, sizeof(*ring), flags);
if (!ring)
return NULL;
for (i = 0; i < descs; i++) {
ring[i] = ioat_alloc_ring_ent(c, flags);
ioat_chan->desc_chunks = chunks = (total_descs * IOAT_DESC_SZ) / SZ_2M;
for (i = 0; i < chunks; i++) {
struct ioat_descs *descs = &ioat_chan->descs[i];
descs->virt = dma_alloc_coherent(to_dev(ioat_chan),
SZ_2M, &descs->hw, flags);
if (!descs->virt && (i > 0)) {
int idx;
for (idx = 0; idx < i; idx++) {
dma_free_coherent(to_dev(ioat_chan), SZ_2M,
descs->virt, descs->hw);
descs->virt = NULL;
descs->hw = 0;
}
ioat_chan->desc_chunks = 0;
kfree(ring);
return NULL;
}
}
for (i = 0; i < total_descs; i++) {
ring[i] = ioat_alloc_ring_ent(c, i, flags);
if (!ring[i]) {
int idx;
while (i--)
ioat_free_ring_ent(ring[i], c);
for (idx = 0; idx < ioat_chan->desc_chunks; idx++) {
dma_free_coherent(to_dev(ioat_chan),
SZ_2M,
ioat_chan->descs[idx].virt,
ioat_chan->descs[idx].hw);
ioat_chan->descs[idx].virt = NULL;
ioat_chan->descs[idx].hw = 0;
}
ioat_chan->desc_chunks = 0;
kfree(ring);
return NULL;
}
......@@ -351,7 +389,7 @@ ioat_alloc_ring(struct dma_chan *c, int order, gfp_t flags)
}
/* link descs */
for (i = 0; i < descs-1; i++) {
for (i = 0; i < total_descs-1; i++) {
struct ioat_ring_ent *next = ring[i+1];
struct ioat_dma_descriptor *hw = ring[i]->hw;
......@@ -362,114 +400,6 @@ ioat_alloc_ring(struct dma_chan *c, int order, gfp_t flags)
return ring;
}
static bool reshape_ring(struct ioatdma_chan *ioat_chan, int order)
{
/* reshape differs from normal ring allocation in that we want
* to allocate a new software ring while only
* extending/truncating the hardware ring
*/
struct dma_chan *c = &ioat_chan->dma_chan;
const u32 curr_size = ioat_ring_size(ioat_chan);
const u16 active = ioat_ring_active(ioat_chan);
const u32 new_size = 1 << order;
struct ioat_ring_ent **ring;
u32 i;
if (order > ioat_get_max_alloc_order())
return false;
/* double check that we have at least 1 free descriptor */
if (active == curr_size)
return false;
/* when shrinking, verify that we can hold the current active
* set in the new ring
*/
if (active >= new_size)
return false;
/* allocate the array to hold the software ring */
ring = kcalloc(new_size, sizeof(*ring), GFP_NOWAIT);
if (!ring)
return false;
/* allocate/trim descriptors as needed */
if (new_size > curr_size) {
/* copy current descriptors to the new ring */
for (i = 0; i < curr_size; i++) {
u16 curr_idx = (ioat_chan->tail+i) & (curr_size-1);
u16 new_idx = (ioat_chan->tail+i) & (new_size-1);
ring[new_idx] = ioat_chan->ring[curr_idx];
set_desc_id(ring[new_idx], new_idx);
}
/* add new descriptors to the ring */
for (i = curr_size; i < new_size; i++) {
u16 new_idx = (ioat_chan->tail+i) & (new_size-1);
ring[new_idx] = ioat_alloc_ring_ent(c, GFP_NOWAIT);
if (!ring[new_idx]) {
while (i--) {
u16 new_idx = (ioat_chan->tail+i) &
(new_size-1);
ioat_free_ring_ent(ring[new_idx], c);