| Age | Commit message (Collapse) | Author |
|---|
|
Implement UC QP creation in the RNIC HW for user API. An UC QP is exposed
as three work queues: send, receive, and memory management. The latter is
used for bind and invalidate WQEs to support memory windows.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260723115955.1859519-3-kotaranov@linux.microsoft.com
Reviewed-by: Long Li <longli@microsoft.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add a leading "struct" keyword to struct descriptions:
Warning: ../include/uapi/rdma/ib_user_mad.h:72 cannot understand function prototype: 'struct ib_user_mad_hdr_old'
Warning: ../include/uapi/rdma/ib_user_mad.h:116 cannot understand function prototype: 'struct ib_user_mad_hdr'
Warning: ../include/uapi/rdma/ib_user_mad.h:143 cannot understand function prototype: 'struct ib_user_mad'
Warning: ../include/uapi/rdma/ib_user_mad.h:184 cannot understand function prototype: 'struct ib_user_mad_reg_req'
Warning: ../include/uapi/rdma/ib_user_mad.h:216 cannot understand function prototype: 'enum'
One enum and #define are moved out of line so that the struct description
and the struct block are not split (separated).
When this is done, more warnings are exposed:
Warning: include/uapi/rdma/ib_user_mad.h:89 struct member 'length' not described in 'ib_user_mad_hdr_old'
Warning: include/uapi/rdma/ib_user_mad.h:135 struct member 'length' not described in 'ib_user_mad_hdr'
Warning: include/uapi/rdma/ib_user_mad.h:135 struct member 'reserved' not described in 'ib_user_mad_hdr'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'id' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'qpn' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'mgmt_class' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'mgmt_class_version' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'res' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'flags' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'method_mask' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'oui' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'rmpp_version' not described in 'ib_user_mad_reg_req2'
Warning: include/uapi/rdma/ib_user_mad.h:227 struct member 'reserved' not described in 'ib_user_mad_reg_req2'
These are all repaired by using ':' as the separator between the struct
member @name and its description (instead of '-').
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Link: https://patch.msgid.link/20260728183403.1136827-1-rdunlap@infradead.org
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
The current GET_TOGGLE_MEM ioctl requires the caller to supply
a type enum and a raw hardware queue ID (RES_ID). The kernel
looks up the CQ or SRQ by that ID without verifying that the
caller owns the resource.
Add a new, preferred code path that accepts standard uverbs
object handles (BNXT_RE_TOGGLE_MEM_CQ_HANDLE /
BNXT_RE_TOGGLE_MEM_SRQ_HANDLE) instead. The uverbs core validates
that the handle belongs to the calling context as part of resolving
it, so this path no longer needs the driver's own XArray lookup for
ownership checking. As with the legacy path, the toggle_entry's own
mmap-entry refcount (not a CQ/SRQ uobject reference) is what pins
the toggle page for the life of the GET_TOGGLE_MEM handle.
Only newer rdma-core versions support this path, if the
driver reports the supported resp mask
(BNXT_RE_UCNTX_CMASK_TOGGLE_MEM_UOBJ_SUPPORT).
The existing TYPE + RES_ID path is retained for backward
compatibility with older rdma-core.
Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Accept RDMA_NLDEV_ATTR_DEV_NAME together with RDMA_NLDEV_NET_NS_FD so a
netlink move can rename the device in the destination namespace. Keep the
name semantics aligned with the existing RDMA rename path.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260716132316.1495242-6-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add an optional mlx5 driver-namespace UMEM attribute on SRQ create so
userspace can supply the doorbell record umem explicitly, symmetric to
the CQ and QP sides. Resolve it inside mlx5_ib_db_map_user() and use it
as a private DBR page when present; otherwise take the existing UHW
share-or-pin path that preserves per-page DBR sharing across CQ/QP/SRQ
in the same process.
Add mlx5's first UVERBS_OBJECT_SRQ UAPI definition chain to attach the
new attr.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260701124015.64350-4-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Apply the per-attribute UMEM model to the SRQ create method. Add an
optional UMEM attribute that backs the SRQ WQE buffer, so userspace can
supply it as either a VA or a dma-buf through a single descriptor,
consistent with the CQ and QP create methods.
mlx5 is the only driver that pins an SRQ WQE buffer via umem; it maps a
single ucmd->buf_addr region through this attribute. No other driver
implements a user SRQ buffer, so none of them use the attribute.
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Link: https://patch.msgid.link/20260701124015.64350-2-jiri@resnulli.us
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Implement returning to userspace applications PDNs of created PDs.
The PDN is used by applications that build work requests outside of the
rdma-core code base. The PDN is used to build work requests that require
mentioning the PD. The HW still ensures PD isolation using PDN attached
to MRs and WRs, therefore the PDN mentioned in the work request must match
the PDN of the used work queue. The work requests can fit only 16 bit PDNs.
Allow users to request short PDNs which are 16 bits.
The capability to request short PDN and get PDN is encoded in the response
of alloc user context IOCTL.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260717200839.495327-1-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Currently EFA WRs support 16-bit request ID, this requires EFA to manage
a translation table to translate the IB WR ID from 64-bits to 16-bits
and translating it back on CQ completion.
Expose a new device capability to handle 64-bit request ID for SQ WRs
allowing userspace to directly post the 64-bit ID to the device.
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Reviewed-by: Tom Sela <tomsela@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260722113331.2515247-3-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add the support needed to propagate the user requested flags to config
the CQ/QP to support 64-bit SQ request ID to the device.
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Reviewed-by: Tom Sela <tomsela@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260722113331.2515247-2-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Implement completion counters for the EFA device. Each completion
counter is backed by two EFA event counters, one for success
completions and one for error completions.
The driver creates umem for counters from private descriptor ioctl
attributes using core utility.
Read operations are not implemented as the counter values are accessed
directly from userspace through the mapped memory.
Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
Link: https://patch.msgid.link/20260722083603.30334-7-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add a dedicated query interface for completion counter capabilities
via UVERBS_METHOD_QUERY_COMP_CNTR_CAPS on the device object.
The query returns the maximum number of counters, maximum counter
value, and a bitmask of supported QP attach operations. Each field
is an optional ioctl attribute, allowing userspace to request only
the capabilities it needs.
Drivers implement the query_comp_cntr_caps operation to report
device-specific capabilities.
Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
Link: https://patch.msgid.link/20260722083603.30334-4-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add core infrastructure for Completion Counters, a light-weight
alternative to polling CQ for tracking operation completions.
Define the UVERBS_OBJECT_COMP_CNTR ioctl object with create, destroy,
modify and read methods for both success and error counters. Add a QP
attach method on the QP object to associate a completion counter with a
queue pair.
Add ib_comp_cntr struct, ib_comp_cntr_attach_attr, device ops, and
DECLARE_RDMA_OBJ_SIZE for driver object allocation.
Only userspace Completion Counters are supported at this stage.
Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
Link: https://patch.msgid.link/20260722083603.30334-2-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Use struct ib_uverbs_clock_info from ib_user_verbs.h for clock info.
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Link: https://patch.msgid.link/20260610154216.712374-6-abhijit.gangurde@amd.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Enable user space applications to access the PHC state page when
firmware RDMA completion timestamp is supported.
This mapping allows user space to convert RDMA completion timestamps
to system wall time without kernel transitions, minimizing latency
overhead. Applications can directly read the PHC state through mmap,
enabling efficient timestamp correlation for precision timing
applications.
Co-developed-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Link: https://patch.msgid.link/20260610154216.712374-4-abhijit.gangurde@amd.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add a page associated with the PHC that can be mapped to user space,
allowing applications to access hardware timestamp information.
In order to synchronize between kernel and user space, a sequence
number is incremented at the beginning and end of each update.
An odd number means the data is being updated while an even number
means the update is complete. To guarantee that the data structure
was accessed atomically, user space will:
repeat:
seq1 = <read sequence>
goto <repeat> if odd
<read PHC state>
seq2 = <read sequence>
if seq1 != seq2 goto repeat
This mechanism acts as a guard against reading invalid state during
concurrent updates.
Co-developed-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Link: https://patch.msgid.link/20260610154216.712374-3-abhijit.gangurde@amd.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Enable the uverbs robust udata interface in mana_ib by setting
uverbs_robust_udata and converting the driver to the new udata
handling model.
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Replace the manual udata input copy and validation during
QP creation with the robust helper.
The irdma driver is backwards compatible with the legacy
i40iw userspace provider. The current create_qp ABI contains
two 8 byte fields. The legacy i40iw ABI was the same but
also contained two additional fields which were never actually
used. Furthermore, the i40iw userspace provider never explicitly
zero-initialized those extra fields, so there is a chance that
existing binaries are passing non-zero garbage values down
to the kernel.
Previously, the irdma driver only copied out the first 16
bytes and did not have any check for the rest of the buffer
being zero, so that additional garbage didn't matter.
By switching to ib_copy_validate_udata_in(), we will now be
checking to ensure that data beyond the kernel's definition
of the request is all zero.
In order to avoid breaking legacy binaries, we therefore need
to increase the request structure size to cover those garbage
fields.
- Legacy binaries will continue to pass down a 32 byte request,
with the driver copying the entire 32 bytes out but ignoring
the second 16 bytes, just as before.
- Newer binaries will pass down the normal 16 byte request. The
ib_copy_validate_udata_in() call will allow this to succeed
because we use user_compl_ctx as our minimum length (16 bytes).
- If the request is ever extended, the new fields would be
added after the "don't use" fields and would work as per
the normal uAPI mechanism.
Signed-off-by: Jacob Moroni <jmoroni@google.com>
Link: https://patch.msgid.link/20260713171257.3131493-5-jmoroni@google.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add RDMA_NLDEV_ATTR_RES_SUMMARY_ENTRY_MAX netlink attribute to expose
device resource limits (max_qp, max_cq, max_mr, max_pd, max_srq) in
the resource summary alongside the existing current count. This allows
userspace tools like iproute2's rdma to display resource usage in
curr/max format.
Expected output from "rdma resource show":
Before: 0: mlx5_0: qp 123 cq 45 mr 200 pd 10
After: 0: mlx5_0: qp 123/131072 cq 45/65536 mr 200/1000000 pd 10/32768
In JSON output, both "curr" and "max" fields will be provided so that
scripts can compute percentages if needed.
The new attribute is optional and backward compatible - old userspace
tools will simply ignore it.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Link: https://patch.msgid.link/20260615003646.168704-1-cui.tao@linux.dev
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
In CoCo guests, guest memory is encrypted and untrusted (T=0) devices
cannot DMA to it directly; such transfers must go through unencrypted
bounce buffers. RDMA registers user pages for direct device access,
bypassing the DMA layer and thus any bouncing, so registered memory does
not work in this configuration.
Until trusted (T=1) device detection is available, conservatively flag
every device attached to a CoCo guest. Expose the condition to userspace
as IB_UVERBS_DEVICE_CC_DMA_BOUNCE in device_cap_flags_ex so applications
can avoid memory registration and fall back to copying buffers through
send/recv.
Link: https://patch.msgid.link/r/20260517141311.2409230-2-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Add an optional mlx5 driver-namespace UMEM attribute on QP
create so userspace can supply the doorbell record umem
explicitly, symmetric to the CQ side. Resolve it inside
mlx5_ib_db_map_user() and use it as a private DBR page when
present; otherwise take the existing UHW share-or-pin path
that preserves per-page DBR sharing across CQ/QP/SRQ in the
same process.
Add mlx5's first UVERBS_OBJECT_QP UAPI definition chain to
attach the new attr.
Link: https://patch.msgid.link/r/20260529134312.2836341-17-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Add an optional mlx5 driver-namespace UMEM attribute on CQ
create so userspace can supply the doorbell record buffer
explicitly. mlx5_ib_db_map_user() resolves the attribute (or
falls back to the legacy UHW VA) into a struct
ib_uverbs_buffer_desc and runs a unified lookup-then-pin:
VA-typed descriptors share a per-page umem across CQ/QP/SRQ
in the same process, FD-typed descriptors are pinned per call.
Link: https://patch.msgid.link/r/20260529134312.2836341-16-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Apply the per-attribute UMEM model to the QP create method. Add
three optional UMEM attributes that drivers pick from based on
how their user ABI lays out the QP rings:
- CREATE_QP_BUF_UMEM is a single user buffer that backs both
the SQ and RQ of one QP. This is the common case where
userspace pins one contiguous WQE region for the QP.
- CREATE_QP_SQ_BUF_UMEM and CREATE_QP_RQ_BUF_UMEM are a pair
of user buffers backing the SQ and RQ independently, used
when the two rings live in physically distinct user
allocations and must be pinned and addressed separately.
Existing drivers would map their current umems as follows:
- mlx5: BUF for normal QPs (one ucmd->buf_addr covers SQ+RQ);
for IB_QPT_RAW_PACKET and IB_QP_CREATE_SOURCE_QPN, the RQ
side comes from ucmd->buf_addr (RQ-sized) via RQ_BUF and
the SQ from ucmd->sq_buf_addr via SQ_BUF.
- mlx4: BUF, single ucmd.buf_addr covering SQ+RQ.
- hns: BUF, single ucmd.buf_addr covering SQ + ext-SGE + RQ.
- erdma: BUF, single ureq.qbuf_va sliced by the kernel into
SQ at offset 0 and RQ at rq_offset.
- bnxt_re: SQ_BUF (ureq->qpsva) + RQ_BUF (ureq->qprva, the
RQ side is skipped when the QP uses an SRQ).
- vmw_pvrdma: SQ_BUF (sbuf_addr) + RQ_BUF (rbuf_addr, the RQ
side is skipped when the QP uses an SRQ).
- qedr: SQ_BUF (sq_addr) + RQ_BUF (rq_addr) for whichever
side the QP type actually has (no SQ for XRC_TGT/GSI; no
RQ for XRC_INI/XRC_TGT/SRQ).
- ionic: SQ_BUF (req.sq.addr) + RQ_BUF (req.rq.addr); both
are skipped when the rings are placed in CMB instead of
host memory.
- mana: raw-packet QP uses SQ_BUF (sq_buf_addr) only; the RC
path uses multiple per-queue user buffers (ucmd.queue_buf[])
that do not fit the SQ/RQ pair semantics of these attrs and
stays on the legacy UHW path.
- efa, irdma, hfi1, ocrdma, mthca, cxgb4 and usnic do not pin
a QP WQE buffer via umem; none of these attributes apply.
Link: https://patch.msgid.link/r/20260529134312.2836341-13-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Add UVERBS_ATTR_CREATE_CQ_BUF_UMEM and two driver-facing
wrappers, ib_umem_get_cq_buf() and ib_umem_get_cq_buf_or_va(),
that pin a CQ buffer umem from it. The wrappers reuse the
existing legacy CQ buffer-attr filler.
Link: https://patch.msgid.link/r/20260529134312.2836341-7-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Introduce a per-attribute UVERBS_ATTR_UMEM model so each uverbs
command's umem set is explicit in its UAPI definition. Add
driver-facing wrapper helpers that pin a umem on demand from an
attribute or a VA addr; the driver owns the returned umem and
releases it from its destroy/error paths.
Link: https://patch.msgid.link/r/20260529134312.2836341-4-jiri@resnulli.us
Signed-off-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The driver supports a new comp_mask: REQ_MASK_FIXED_QUE_ATTR.
The application sets this comp_mask bit in the CREATE_QP ureq
to indicate direct control of the QP. The driver goes through
the required processing for app allocated QPs (previous patches).
Only variable WQE mode is supported for these QPs.
This patch removes an unused comp_mask:
BNXT_RE_QP_REQ_MASK_VAR_WQE_SQ_SLOTS
Link: https://patch.msgid.link/r/20260519150041.7251-10-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
App allocated QPs can use a separate doorbell for each QP.
This doorbell region can be passed through a new driver specific
DBR_HANDLE attribute, during QP creation. When this attribute
is set, associate the QP with the given doorbell region.
While the QP holds a reference to the dbr, the dbr itself
cannot be destroyed and is rejected with EBUSY error.
Link: https://patch.msgid.link/r/20260519150041.7251-9-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
For app allocated QPs, the driver shouldn't use slots/round-up logic
to compute the msn table size. The application handles this logic
and computes 'sq_npsn' and passes it to the driver using a new uapi
parameter.
Link: https://patch.msgid.link/r/20260519150041.7251-5-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Following discussion [1], rename the comp_mask field in
efa_ibv_alloc_ucontext_cmd to supported_caps to reflect its actual
usage as a capabilities handshake mechanism rather than a standard
comp_mask. Rename related constants and align function and macro names.
[1] https://lore.kernel.org/linux-rdma/20260312120858.GH1448102@nvidia.com/
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
Link: https://patch.msgid.link/20260316180846.30273-1-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
This patch supports application allocated memory for CQs.
The application allocates and manages the CQs directly. To support
this, the driver exports a new comp_mask to indicate direct control
of the CQ. When this comp_mask bit is set in the ureq, the driver
maps this application allocated CQ memory into hardware. As the
application manages this memory, the CQ depth ('cqe') passed by it
must be used as is and the driver shouldn't update it.
For CQs, ib_core supports pinning dmabuf based application memory,
specified through provider attributes. This umem is mananged by the
ib_core and is available in ib_cq. Register 'create_cq_user' devop
to process this umem. The driver also supports the legacy interface
that allocates umem internally.
Link: https://patch.msgid.link/r/20260302110036.36387-7-sriharsha.basavapatna@broadcom.com
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Some applications may need multiple doorbells to support parallel
processing of threads that each operate on a group of resources.
The following uapi methods have been implemented in this patch.
- BNXT_RE_METHOD_DBR_ALLOC:
This will allow the appliation to create extra doorbell regions
and use the associated doorbell page index in CREATE_QP and
use the associated DB address while ringing the doorbell.
- BNXT_RE_METHOD_DBR_FREE:
Free the allocated doorbell region.
- BNXT_RE_METHOD_GET_DEFAULT_DBR:
Return the default doorbell page index and doorbell page address
associated with the ucontext.
Link: https://patch.msgid.link/r/20260302110036.36387-4-sriharsha.basavapatna@broadcom.com
Co-developed-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Reviewed-by: Selvin Xavier <selvin.xavier@broadcom.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
This flag can be set by drivers once they have finished auditing and
implementing the full udata support on every udata operation.
My intention going forward is that driver authors proposing new udata uAPI
for their drivers must first do the work and set this flag.
If this flag is not set the userspace should not try to use udata based
uAPI newer than this commit, though on a case by case basis it may be OK
based on what checks historical kernels performed on the specific call.
Since bnxt_re is audited now, it is the first driver to set the flag.
Link: https://patch.msgid.link/r/13-v3-bd56dd443069+49-bnxt_re_uapi_jgg@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Extend the VAR allocation UAPI to accept an optional flags attribute,
allowing userspace to request TLP VAR allocation via the
MLX5_IB_UAPI_VAR_ALLOC_FLAG_TLP flag.
When the TLP flag "MLX5_IB_UAPI_VAR_ALLOC_FLAG_TLP" is specified,
the driver selects the TLP VAR region for allocation instead of the
regular VirtIO VAR region.
Signed-off-by: Maher Sanalla <msanalla@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
|
|
Allow netlink users, through the usage of driver-details netlink
attribute, to get information about internal FRMR pools that use the
kernel_vendor_key FRMR key member.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-11-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Allow users to set through netlink, for a specific FRMR pool, the amount
of handles that are not aged, and fill the pool to this amount.
This allows users to warm-up the FRMR pools to an expected amount of
handles with specific attributes that fits their expected usage.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-10-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Allow users to set FRMR pools aging timer through netlink.
This functionality will allow user to control how long handles reside in
the kernel before being destroyed, thus being able to tune the tradeoff
between memory and HW object consumption and memory registration
optimization.
Since FRMR pools is highly beneficial for application restart scenarios,
this command allows users to modify the aging timer to their application
restart time, making sure the FRMR handles deregistered on application
teardown are kept for long enough in the pools for reuse in the
application startup.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-9-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add support for a new command in netlink to dump to user the state of
the FRMR pools on the devices.
Expose each pool with its key and the usage statistics for it.
Signed-off-by: Michael Guralnik <michaelgur@nvidia.com>
Reviewed-by: Patrisious Haddad <phaddad@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260226-frmr_pools-v4-8-95360b54f15e@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add new extended max inline query and report the new value to userspace.
Reviewed-by: Firas Jahjah <firasj@amazon.com>
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Signed-off-by: Yonatan Nachum <ynachum@amazon.com>
Link: https://patch.msgid.link/20260217112304.36849-3-ynachum@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Pull rdma updates from Jason Gunthorpe:
"Usual smallish cycle. The NFS biovec work to push it down into RDMA
instead of indirecting through a scatterlist is pretty nice to see,
been talked about for a long time now.
- Various code improvements in irdma, rtrs, qedr, ocrdma, irdma, rxe
- Small driver improvements and minor bug fixes to hns, mlx5, rxe,
mana, mlx5, irdma
- Robusness improvements in completion processing for EFA
- New query_port_speed() verb to move past limited IBA defined speed
steps
- Support for SG_GAPS in rts and many other small improvements
- Rare list corruption fix in iwcm
- Better support different page sizes in rxe
- Device memory support for mana
- Direct bio vec to kernel MR for use by NFS-RDMA
- QP rate limiting for bnxt_re
- Remote triggerable NULL pointer crash in siw
- DMA-buf exporter support for RDMA mmaps like doorbells"
* tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (66 commits)
RDMA/mlx5: Implement DMABUF export ops
RDMA/uverbs: Add DMABUF object type and operations
RDMA/uverbs: Support external FD uobjects
RDMA/siw: Fix potential NULL pointer dereference in header processing
RDMA/umad: Reject negative data_len in ib_umad_write
IB/core: Extend rate limit support for RC QPs
RDMA/mlx5: Support rate limit only for Raw Packet QP
RDMA/bnxt_re: Report QP rate limit in debugfs
RDMA/bnxt_re: Report packet pacing capabilities when querying device
RDMA/bnxt_re: Add support for QP rate limiting
MAINTAINERS: Drop RDMA files from Hyper-V section
RDMA/uverbs: Add __GFP_NOWARN to ib_uverbs_unmarshall_recv() kmalloc
svcrdma: use bvec-based RDMA read/write API
RDMA/core: add rdma_rw_max_sge() helper for SQ sizing
RDMA/core: add MR support for bvec-based RDMA operations
RDMA/core: use IOVA-based DMA mapping for bvec RDMA operations
RDMA/core: add bio_vec based RDMA read/write API
RDMA/irdma: Use kvzalloc for paged memory DMA address array
RDMA/rxe: Fix race condition in QP timer handlers
RDMA/mana_ib: Add device‑memory support
...
|
|
Expose DMABUF functionality to userspace through the uverbs interface,
enabling InfiniBand/RDMA devices to export PCI based memory regions
(e.g. device memory) as DMABUF file descriptors. This allows
zero-copy sharing of RDMA memory with other subsystems that support the
dma-buf framework.
A new UVERBS_OBJECT_DMABUF object type and allocation method were
introduced.
During allocation, uverbs invokes the driver to supply the
rdma_user_mmap_entry associated with the given page offset (pgoff).
Based on the returned rdma_user_mmap_entry, uverbs requests the driver
to provide the corresponding physical-memory details as well as the
driver’s PCI provider information.
Using this information, dma_buf_export() is called; if it succeeds,
uobj->object is set to the underlying file pointer returned by the
dma-buf framework.
The file descriptor number follows the standard uverbs allocation flow,
but the file pointer comes from the dma-buf subsystem, including its own
fops and private data.
When an mmap entry is removed, uverbs iterates over its associated
DMABUFs, marks them as revoked, and calls dma_buf_move_notify() so that
their importers are notified.
The same procedure applies during the disassociate flow; final cleanup
occurs when the application closes the file.
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Link: https://patch.msgid.link/20260201-dmabuf-export-v3-2-da238b614fe3@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Enable the support to report packet pacing capabilities
from kernel to user space. Packet pacing allows to limit
the rate to any number between the maximum and minimum.
The capabilities are exposed to user space through query_device.
The following capabilities are reported:
1. The maximum and minimum rate limit in kbps.
2. Bitmap showing which QP types support rate limit.
Signed-off-by: Damodharam Ammepalli <damodharam.ammepalli@broadcom.com>
Signed-off-by: Kalesh AP <kalesh-anakkur.purayil@broadcom.com>
Link: https://patch.msgid.link/20260202133413.3182578-3-kalesh-anakkur.purayil@broadcom.com
Reviewed-by: Anantha Prabhu <anantha.prabhu@broadcom.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Get CQ type from the used gdma device. The MANA_IB_CREATE_RNIC_CQ
flag is ignored. It was used in older kernel versions where
the mana_ib was shared between ethernet and rnic.
Fixes: d4293f96ce0b ("RDMA/mana_ib: unify mana_ib functions to support any gdma device")
Signed-off-by: Konstantin Taranov <kotaranov@microsoft.com>
Link: https://patch.msgid.link/20260115093625.177306-1-kotaranov@linux.microsoft.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add new ibv_query_port_speed() verb to enable applications to query
the effective bandwidth of a port.
This verb is particularly useful when the speed is not a multiplication
of IB speed and width where width is 2^n.
Signed-off-by: Or Har-Toov <ohartoov@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Edward Srouji <edwards@nvidia.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
A recent commit modified struct irdma_alloc_ucontext_resp by adding a
member with implicit padding in front of it, though this does not change
the offset of the data members other than m68k. Reported by
scripts/check-uapi.sh:
==== ABI differences detected in include/rdma/irdma-abi.h from 1dd7bde2e91c -> HEAD ====
[C] 'struct irdma_alloc_ucontext_resp' changed:
type size changed from 704 to 640 (in bits)
1 data member deletion:
'__u8 rsvd3[2]', at offset 640 (in bits) at irdma-abi.h:61:1
1 data member insertion:
'__u8 revd3[2]', at offset 592 (in bits) at irdma-abi.h:60:1
Change the size back to the previous version, and remove the implicit
padding by making it explicit and matching what x86-64 would do by placing
max_hw_srq_quanta member into a naturally aligned location.
Fixes: 563e1feb5f6e ("RDMA/irdma: Add SRQ support")
Link: https://patch.msgid.link/r/20251208133849.315451-1-arnd@kernel.org
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Geert Uytterhoeven <geert@linux-m68k.org>
Tested-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
On a few 32-bit architectures, the newly added ib_user_service_rec
structure is not 64-bit aligned the way it is on most regular ones.
Add explicit padding into the rdma_ucm_query_ib_service_resp and
rdma_ucm_resolve_ib_service structures that embed it, so that the layout
is compatible across all of them.
This is an ABI change on i386, aligning it with x86_64 and the other
64-bit architectures to avoid having to use a compat ioctl handler.
Fixes: 810f874eda8e ("RDMA/ucma: Support query resolved service records")
Link: https://patch.msgid.link/r/20251208133311.313977-1-arnd@kernel.org
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Implement verb API and UAPI changes to support SRQ functionality in GEN3
devices.
Signed-off-by: Faisal Latif <faisal.latif@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Link: https://patch.msgid.link/20250827152545.2056-13-tatyana.e.nikolova@intel.com
Tested-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
In the IPU model, a function can host one or more logical network
endpoints called vPorts. Each vPort may be associated with either a
physical or an internal communication port, and can be RDMA capable. A
vPort features a netdev and, if RDMA capable, must have an associated
ib_dev.
This change introduces a GEN3 auxiliary vPort driver responsible for
registering a verbs device for every RDMA-capable vPort. Additionally,
the UAPI is updated to prevent the binding of GEN3 devices to older
user-space providers.
Signed-off-by: Mustafa Ismail <mustafa.ismail@intel.com>
Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Link: https://patch.msgid.link/20250827152545.2056-8-tatyana.e.nikolova@intel.com
Tested-by: Jacob Moroni <jmoroni@google.com>
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Implement device supported verb APIs for control path.
Co-developed-by: Andrew Boyer <andrew.boyer@amd.com>
Signed-off-by: Andrew Boyer <andrew.boyer@amd.com>
Co-developed-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Allen Hubbe <allen.hubbe@amd.com>
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Link: https://patch.msgid.link/20250903061606.4139957-11-abhijit.gangurde@amd.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Define RDMA_DRIVER_IONIC in enum rdma_driver_id.
Signed-off-by: Abhijit Gangurde <abhijit.gangurde@amd.com>
Link: https://patch.msgid.link/20250903061606.4139957-8-abhijit.gangurde@amd.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Enable user-space to inject an event into a CM through it's event
channel. Two new events are added and supported: RDMA_CM_EVENT_USER and
RDMA_CM_EVENT_INTERNAL. With these 2 events a new event parameter "arg"
is supported, which is passed from sender to receiver transparently.
With this feature an application is able to write an event into a CM
channel with a new user-space rdmacm API. For example thread T1 could
write an event with the API:
rdma_write_cm_event(cm_id, RDMA_CM_EVENT_USER, status, arg);
and thread T2 could receive the event with rdma_get_cm_event().
Signed-off-by: Mark Zhang <markzhang@nvidia.com>
Reviewed-by: Vlad Dumitrescu <vdumitrescu@nvidia.com>
Link: https://patch.msgid.link/fdf49d0b17a45933c5d8c1d90605c9447d9a3c73.1751279794.git.leonro@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Enable user-space to query resolved service records through a ucma
command when a RDMA_CM_EVENT_ADDRINFO_RESOLVED event is received.
Signed-off-by: Or Har-Toov <ohartoov@nvidia.com>
Signed-off-by: Mark Zhang <markzhang@nvidia.com>
Reviewed-by: Vlad Dumitrescu <vdumitrescu@nvidia.com>
Link: https://patch.msgid.link/1090ee7c00c3f8058c4f9e7557de983504a16715.1751279794.git.leonro@nvidia.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
