kernel/git/stable/linux.git/include/net/libeth, branch master

xsk: honor XDP_TX_METADATA in zero-copy path

2026-08-24T18:12:00+00:00

The zero-copy path reads TX metadata whenever the UMEM has metadata space, even if the descriptor does not set XDP_TX_METADATA. Pass descriptor options through the metadata helpers and ignore metadata unless the option is set. This does not fix the existing per-WQE metadata handling for mlx5 MPWQEs. Only the descriptor that starts a session passes through xsk_tx_metadata_request() and configures offload state shared by the batch. Metadata on descriptors joining an open session is therefore not validated and does not configure its requested offloads. In addition, a non-NULL metadata pointer from such a descriptor is treated as a timestamp completion request even when XDP_TXMD_FLAGS_TIMESTAMP is not set, so its metadata union can be overwritten with an unrequested timestamp. Fixing mixed metadata states within one MPWQE requires a separate change. Fixes: 48eb03dd2630 ("xsk: Add TX timestamp and TX checksum offload support") Reviewed-by: Alexander Lobakin Signed-off-by: Stanislav Fomichev Reviewed-by: Maciej Fijalkowski Reviewed-by: Aleksandr Loktionov Link: https://patch.msgid.link/20260819160535.1472459-3-sdf@fomichev.me Signed-off-by: Jakub Kicinski

libeth: allow to create fill queues without NAPI

2026-08-12T20:56:50+00:00

Control queues can utilize libeth_rx fill queues, despite working outside of NAPI context. The only problem is standard fill queues requiring NAPI that provides them with the device pointer. Introduce a way to provide the device directly without using NAPI. Suggested-by: Alexander Lobakin Reviewed-by: Maciej Fijalkowski Signed-off-by: Pavan Kumar Linga Tested-by: Bharath R Tested-by: Samuel Salin Signed-off-by: Larysa Zaremba Signed-off-by: Tony Nguyen

xsk: validate metadata when processing requests

2026-08-03T23:43:54+00:00

The zero-copy path validates TX metadata while obtaining the descriptor context, then reads it again later when preparing the hardware request. User space can change the metadata between those operations and bypass the original validation. Validate the metadata in xsk_tx_metadata_request() and use the resulting flags snapshot for every feature check. Read request fields once so all zero-copy drivers process only values observed after successful validation. Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata") Cc: Cen Zhang (Microsoft) Signed-off-by: Stanislav Fomichev Link: https://patch.msgid.link/20260727161959.885642-7-sdf@fomichev.me Signed-off-by: Jakub Kicinski

xsk: pass TX metadata pointer by reference

2026-08-03T23:43:53+00:00

Completion handling needs to know whether a timestamp was requested when the metadata was processed. Let xsk_tx_metadata_request() update the caller's metadata pointer so that decision can be carried forward without rereading user-controlled flags. This only changes the interface; behavior remains unchanged. Fixes: ca4419f15abd ("xsk: Add launch time hardware offload support to XDP Tx metadata") Cc: Cen Zhang (Microsoft) Signed-off-by: Stanislav Fomichev Link: https://patch.msgid.link/20260727161959.885642-3-sdf@fomichev.me Signed-off-by: Jakub Kicinski

libeth, idpf: use truesize as XDP RxQ info frag_size

2026-03-05T16:02:05+00:00

The only user of frag_size field in XDP RxQ info is bpf_xdp_frags_increase_tail(). It clearly expects whole buffer size instead of DMA write size. Different assumptions in idpf driver configuration lead to negative tailroom. To make it worse, buffer sizes are not actually uniform in idpf when splitq is enabled, as there are several buffer queues, so rxq->rx_buf_size is meaningless in this case. Use truesize of the first bufq in AF_XDP ZC, as there is only one. Disable growing tail for regular splitq. Fixes: ac8a861f632e ("idpf: prepare structures to support XDP") Reviewed-by: Aleksandr Loktionov Signed-off-by: Larysa Zaremba Link: https://patch.msgid.link/20260305111253.2317394-8-larysa.zaremba@intel.com Signed-off-by: Jakub Kicinski

libeth: xdp: Disable generic kCFI pass for libeth_xdp_tx_xmit_bulk()

2025-10-30T03:04:55+00:00

When building drivers/net/ethernet/intel/idpf/xsk.c for ARCH=arm with CONFIG_CFI=y using a version of LLVM prior to 22.0.0, there is a BUILD_BUG_ON failure: $ cat arch/arm/configs/repro.config CONFIG_BPF_SYSCALL=y CONFIG_CFI=y CONFIG_IDPF=y CONFIG_XDP_SOCKETS=y $ make -skj"$(nproc)" ARCH=arm LLVM=1 clean defconfig repro.config drivers/net/ethernet/intel/idpf/xsk.o In file included from drivers/net/ethernet/intel/idpf/xsk.c:4: include/net/libeth/xsk.h:205:2: error: call to '__compiletime_assert_728' declared with 'error' attribute: BUILD_BUG_ON failed: !__builtin_constant_p(tmo == libeth_xsktmo) 205 | BUILD_BUG_ON(!__builtin_constant_p(tmo == libeth_xsktmo)); | ^ ... libeth_xdp_tx_xmit_bulk() indirectly calls libeth_xsk_xmit_fill_buf() but these functions are marked as __always_inline so that the compiler can turn these indirect calls into direct ones and see that the tmo parameter to __libeth_xsk_xmit_fill_buf_md() is ultimately libeth_xsktmo from idpf_xsk_xmit(). Unfortunately, the generic kCFI pass in LLVM expands the kCFI bundles from the indirect calls in libeth_xdp_tx_xmit_bulk() in such a way that later optimizations cannot turn these calls into direct ones, making the BUILD_BUG_ON fail because it cannot be proved at compile time that tmo is libeth_xsktmo. Disable the generic kCFI pass for libeth_xdp_tx_xmit_bulk() to ensure these indirect calls can always be turned into direct calls to avoid this error. Closes: https://github.com/ClangBuiltLinux/linux/issues/2124 Fixes: 9705d6552f58 ("idpf: implement Rx path for AF_XDP") Signed-off-by: Nathan Chancellor Reviewed-by: Aleksandr Loktionov Acked-by: Alexander Lobakin Link: https://patch.msgid.link/20251025-idpf-fix-arm-kcfi-build-error-v1-3-ec57221153ae@kernel.org Signed-off-by: Kees Cook

xdp, libeth: make the xdp_init_buff() micro-optimization generic

2025-09-08T17:26:25+00:00

Often times the compilers are not able to expand two consecutive 32-bit writes into one 64-bit on the corresponding architectures. This applies to xdp_init_buff() called for every received frame (or at least once per each 64 frames when the frag size is fixed). Move the not-so-pretty hack from libeth_xdp straight to xdp_init_buff(), but using a proper union around ::frame_sz and ::flags. The optimization is limited to LE architectures due to the structure layout. One simple example from idpf with the XDP series applied (Clang 22-git, CONFIG_CC_OPTIMIZE_FOR_PERFORMANCE => -O2): add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-27 (-27) Function old new delta idpf_vport_splitq_napi_poll 5076 5049 -27 The perf difference with XDP_DROP is around +0.8-1% which I see as more than satisfying. Suggested-by: Simon Horman Signed-off-by: Alexander Lobakin Tested-by: Ramu R Signed-off-by: Tony Nguyen

libeth: xdp: access ->pp through netmem_desc instead of page

2025-07-24T00:46:56+00:00

To eliminate the use of struct page in page pool, the page pool users should use netmem descriptor and APIs instead. Make xdp access ->pp through netmem_desc instead of page. Signed-off-by: Byungchul Park Link: https://patch.msgid.link/20250721021835.63939-13-byungchul@sk.com Signed-off-by: Jakub Kicinski

libeth: xdp, xsk: access adjacent u32s as u64 where applicable

2025-06-16T18:40:15+00:00

On 64-bit systems, writing/reading one u64 is faster than two u32s even when they're are adjacent in a struct. The compilers won't guarantee they will combine those; I observed both successful and unsuccessful attempts with both GCC and Clang, and it's not easy to say what it depends on. There's a few places in libeth_xdp winning up to several percent from combined access (both performance and object code size, especially when unrolling). Add __LIBETH_WORD_ACCESS and use it there on LE. Drivers are free to optimize HW-specific callbacks under the same definition. Signed-off-by: Alexander Lobakin Signed-off-by: Tony Nguyen

libeth: xsk: add XSkFQ refill and XSk wakeup helpers

2025-06-16T18:40:15+00:00

XSkFQ refill is pretty generic across the drivers minus FQ descriptor filling and can easily be unified with one inline callback. XSk wakeup is usually not, but here, instead of commonly used "SW interrupts", I picked firing an IPI. In most tests, it showed better performance; it also provides better control for userspace on which CPU will handle the xmit, as SW interrupts honor IRQ affinity no matter which core produces XSk xmit descs (while XDPSQs are associated 1:1 with cores having the same ID). Signed-off-by: Alexander Lobakin Signed-off-by: Tony Nguyen