summaryrefslogtreecommitdiff
path: root/net/sunrpc
AgeCommit message (Collapse)Author
3 daysMerge branch 'nfsd-next' of ↵Mark Brown
https://git.kernel.org/pub/scm/linux/kernel/git/cel/linux
5 daysSUNRPC: pin upper rpc_clnt across the TLS connect_workerChuck Lever
The TLS connect path has a use-after-free: nothing pins the upper rpc_clnt across the delayed connect_worker. xs_connect() stores task->tk_client in sock_xprt::clnt as a raw pointer and queues the worker; for TLS-secured transports that worker is xs_tcp_tls_setup_socket(), which reads several fields out of the saved pointer (cl_timeout, cl_program, cl_prog, cl_vers, cl_cred, cl_stats) to construct the args for the inner handshake rpc_clnt. The xprt does not reference the rpc_clnt; the rpc_clnt references the xprt. xs_destroy() does cancel the connect_worker, but it runs only when the xprt's refcount drops to zero, which cannot happen until the rpc_clnt releases its cl_xprt reference in rpc_free_client_work(). When a TLS handshake fails fatally (for example, an mTLS mount whose client cert does not match the server), the connecting task is woken with -EACCES and exits, the mount caller invokes rpc_shutdown_client(), and the upper rpc_clnt is freed before the queued connect_worker fires. xs_tcp_tls_setup_socket() then dereferences the freed clnt, producing the refcount_t underflow Michael Nemanov reported. Take a reference on the upper rpc_clnt in xs_connect() for TLS transports via a new rpc_hold_client() helper, and drop it in the connect_worker's exit path with rpc_release_client(). The xprt_lock_connect() / xprt_unlock_connect() pairing already serialises xs_connect() with xs_tcp_tls_setup_socket(), so the take and release are balanced one-for-one. The non-TLS connect worker (xs_tcp_setup_socket) never reads sock_xprt::clnt, so leave that path alone and avoid the clnt-holds-xprt-holds-clnt cycle that would otherwise prevent xprt destruction. Reported-by: Michael Nemanov <michael.nemanov@vastdata.com> Closes: https://lore.kernel.org/linux-nfs/40e3d522-dfcf-4fc1-9c55-b5e81f1536d5@vastdata.com/ Fixes: 75eb6af7acdf ("SUNRPC: Add a TCP-with-TLS RPC transport class") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Tested-by: Michael Nemanov <michael.nemanov@vastdata.com> Reviewed-by: Michael Nemanov <michael.nemanov@vastdata.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
5 daysSUNRPC: release lower rpc_clnt if killed waiting for XPRT_LOCKEDChuck Lever
xs_tcp_tls_setup_socket() creates a temporary "lower" rpc_clnt with rpc_create() to drive the inner TLS handshake, then waits for XPRT_LOCKED on its xprt with TASK_KILLABLE so a stuck handshake can be aborted by signal. When the wait is interrupted, the function jumps to out_unlock without releasing lower_clnt. The success path and the out_close error path both call rpc_shutdown_client(lower_clnt); only the killed-wait path skips it, leaking the clnt and its underlying xprt. Call rpc_shutdown_client() on this path before joining out_unlock. xprt_release_write() is not needed here because XPRT_LOCKED was never acquired. Fixes: 26e8bfa30dac ("SUNRPC/TLS: Lock the lower_xprt during the tls handshake") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Tested-by: Michael Nemanov <michael.nemanov@vastdata.com> Reviewed-by: Michael Nemanov <michael.nemanov@vastdata.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
7 dayssunrpc: defer rq_argp and rq_resp free until after RCU grace periodJeff Layton
svc_rqst_free() frees rqstp->rq_argp and rqstp->rq_resp synchronously via kfree(), but defers the rqstp struct free via kfree_rcu(). After svc_exit_thread() calls list_del_rcu() and svc_rqst_free(), there is a window where RCU readers that started before list_del_rcu() can still traverse the thread list and find the rqstp. These readers (e.g. nfsd_nl_rpc_status_get_dumpit()) dereference rqstp->rq_argp, which has already been freed — a use-after-free. Fix this by moving the kfree of rq_argp and rq_resp into an explicit call_rcu() callback alongside the struct free. Resources not accessed by RCU readers (bvec, buffer pages, scratch folio, auth_data) remain synchronously freed. Fixes: 812443865c5f ("sunrpc: add a rcu_head to svc_rqst and use kfree_rcu to free it") Cc: stable@vger.kernel.org Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260611-nfsd-testing-v2-4-5b90e276f2d9@kernel.org Signed-off-by: Chuck Lever <cel@kernel.org>
7 daysSUNRPC: Add svc_serv_maxthreads() to report the thread ceilingChuck Lever
A pooled RPC service sizes its threads dynamically, growing and shrinking each pool between its minimum and maximum bounds as load varies. The count of running threads therefore reflects recent demand, not the service's capacity. A consumer that sizes a data structure against the concurrency the service can sustain -- NFSD's NFSv4 session slot tables, for one -- needs that stable ceiling, and computing it means summing sp_nrthrmax across every pool. Add svc_serv_maxthreads() so the summation, and its dependence on the layout of struct svc_serv and struct svc_pool, stays within sunrpc. The read is lock-free: pool maxima change only when a service is reconfigured, a path callers already serialize against startup and shutdown, so a racing reader observes at worst a transient value. This is acceptable for the sizing heuristics that will consume it. nfsd_nrthreads() already sums sp_nrthrmax across pools by hand; convert it to svc_serv_maxthreads(), giving the new export an in-tree consumer and removing a copy of the dependence on svc_serv internals. Reviewed-by: NeilBrown <neil@brown.name> Reviewed-by: Jeff Layton <jlayton@kernel.org> Reviewed-by: Benjamin Coddington <bcodding@hammerspace.com> Link: https://patch.msgid.link/20260610-nfsd-slot-growth-clamp-v1-1-7b966700df0b@kernel.org Signed-off-by: Chuck Lever <cel@kernel.org>
7 daysnet/sunrpc/svcauth_unix: Use strscpy() to copy strings into arraysDavid Laight
Replacing strcpy() with strscpy() ensures that overflow of the target buffer cannot happen. Signed-off-by: David Laight <david.laight.linux@gmail.com> Link: https://patch.msgid.link/20260608095523.2606-16-david.laight.linux@gmail.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysrpcrdma: arm rn_done before publishing the notificationChuck Lever
rpcrdma_rn_register() inserts @rn into rd_xa with xa_alloc() before storing the caller's callback in rn->rn_done. The xarray makes @rn reachable to rpcrdma_remove_one(), which walks rd_xa and invokes rn->rn_done(rn) for every registered notification. A device removal that races a fresh registration can therefore observe @rn with rn_done still NULL, because the notification objects are zero allocated by their owners, and call through a NULL function pointer. Store rn->rn_done before xa_alloc() publishes @rn. The xarray's store-side and load-side ordering then guarantees that any CPU which finds @rn in rd_xa also observes the armed callback. rpcrdma_rn_unregister() treats a non-NULL rn_done as the sentinel for a completed registration, so the early store must not survive a failed registration. Clear rn_done again when xa_alloc() fails. Were it left set, the failed-accept cleanup path would call rpcrdma_rn_unregister() on an @rn that was never inserted, erasing an unrelated rd_xa slot and underflowing rd_kref. Fixes: 7e86845a0346 ("rpcrdma: Implement generic device removal") Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260601201703.46078-1-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: Check svc pool percpu counter allocationChuck Lever
__svc_create() initializes three per-pool percpu_counter stats and ignores every return value. On SMP, percpu_counter_init() fails when __alloc_percpu_gfp() cannot satisfy the allocation, leaving the failed counter with fbc->counters == NULL and its embedded raw_spinlock_t, list_head, and count never initialized. __svc_create() returns the half-constructed svc_serv to nfsd, lockd, or the NFS callback service anyway. Once that service is live, the hot-path increments in svc_xprt_enqueue(), svc_handle_xprt(), and svc_pool_wake_idle_thread() reach a counter whose backing pointer is NULL. The pointer is a per-cpu offset, so the access does not fault: it resolves to offset zero of the current CPU's per-cpu area and silently corrupts whatever variable lives there. A /proc/fs/nfsd/pool_stats read walks the same NULL per-cpu storage and returns garbage, and on CONFIG_DEBUG_SPINLOCK or lockdep it splats on the never-initialized lock. Creating the broken service requires a percpu allocation failure during RPC server startup, so it is reachable only by a local administrator under memory pressure or fault injection; a remote peer cannot induce the bad state on its own. Check each percpu_counter_init() return value in __svc_create() and fail when an allocation fails, unwinding the counters already set up in the current pool and in every pool initialized before it. A discrete percpu_counter_destroy() per counter at teardown frees each per-cpu allocation exactly once. Fixes: ccf08bed6e7a ("SUNRPC: Replace pool stats with per-CPU variables") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260530-tier2-local-v2-2-5a0fd532db57@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssunrpc: init gssp_lock before publishing proc entryChris Mason
create_use_gss_proxy_proc_entry() publishes /proc/net/rpc/use-gss-proxy via proc_create_data() before init_gssp_clnt() runs mutex_init() on sn->gssp_lock. Once the dentry is linked under proc_subdir_lock it is immediately reachable from userspace, so a write that lands in the window drives set_gssp_clnt() into mutex_lock() on a zero-initialized struct mutex. create_use_gss_proxy_proc_entry(net) proc_create_data("use-gss-proxy", ...) /* dentry live */ init_gssp_clnt(sn) mutex_init(&sn->gssp_lock) /* too late */ write_gssp() set_gssp_clnt(net) mutex_lock(&sn->gssp_lock) /* uninitialized */ gssp_rpc_create(...) sn->gssp_clnt = clnt mutex_unlock(&sn->gssp_lock) The window spans only the two statements between proc_create_data() returning and init_gssp_clnt(), so a writer reaches it only if the registering thread is preempted there while another task is already opening the freshly published file. register_pernet_subsys() runs in preemptible context under pernet_ops_rwsem, so that preemption is possible, and the window widens on auth_rpcgss module load, when the proc entry is created for every live net namespace whose tasks are already running. A writer that wins the race locks a zero-filled struct mutex. On CONFIG_DEBUG_MUTEXES the missing magic value trips a "lock used without init" splat; on a production kernel the fast path acquires the lock via CMPXCHG(owner, 0, current). In the latter case a second writer that arrives before init_gssp_clnt() re-zeroes owner can enter set_gssp_clnt() concurrently, shut down the first writer's clnt while it is still in use, and leak the loser's clnt. Fix by initializing sn->gssp_lock in sunrpc_init_net() so its lifetime matches the sunrpc_net it lives in. sn->gssp_clnt is already NULL from the kzalloc that backs net_generic storage, so the lazy helper is no longer needed; drop init_gssp_clnt(), its prototype, and the call from create_use_gss_proxy_proc_entry(). sunrpc.ko is a build-time dependency of auth_rpcgss.ko, so sunrpc_init_net() has always run on every netns before any auth_gss pernet init can publish the proc entry. Fixes: 030d794bf498 ("SUNRPC: Use gssproxy upcall for server RPCGSS authentication.") Cc: stable@vger.kernel.org Assisted-by: kres:claude-opus-4-7 Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260530-tier2-local-v2-1-5a0fd532db57@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: close backchannel before destroying callback serviceChuck Lever
A backchannel receive can complete a request while the NFS callback service is being torn down. xprt_complete_bc_request() removes the request from bc_pa_list, drops bc_alloc_count, marks the request in use, and then asks xprt_enqueue_bc_request() to hand it to the callback service. If teardown has already cleared xprt->bc_serv, xprt_enqueue_bc_request() currently returns without enqueueing or freeing the committed request. The xprt_get() taken on entry is leaked as well. If the producer wins the race before bc_serv is cleared, it can also enqueue onto sv_cb_list after nfs_callback_down() has stopped the callback threads, leaving the request linked to a svc_serv that is about to be freed. Close the producer side before callback threads are stopped. Add xprt_svc_shutdown_bc() to clear xprt->bc_serv under bc_pa_lock, and call it on callback shutdown and callback-start failure before stopping the service threads. Requests that lose the NULL transition in xprt_enqueue_bc_request() are released through the normal backchannel free path after balancing bc_slot_count. Finally, drain any remaining sv_cb_list requests after the callback threads have stopped and before svc_destroy() frees the service. Fixes: 441244d4273a ("SUNRPC: cleanup common code in backchannel request") Fixes: 9e9fdd0ad0fb ("NFSv4.1: protect destroying and nullifying bc_serv structure") Cc: stable@vger.kernel.org Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-6-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: Zero rpc_gss_wire_cred at svcauth_gss_decode_credbody() entryChris Mason
svcauth_gss_decode_credbody() writes the caller's rpc_gss_wire_cred field by field and assigns gc_ctx.len only on the success tail. The caller storage is svcdata->clcred, which lives in the per-svc_rqst gss_svc_data and is reused across requests. Early decode failures leave partially decoded state mixed with residue from the prior request. The trailing body_len tightness check is the sharpest case: xdr_stream_decode_opaque_inline() has already written gc_ctx.data with a borrowed inline pointer into the current request's XDR pages, but gc_ctx.len retains its prior value. Once the request pages are released the pooled clcred carries a dangling pointer paired with a stale length. Zero the caller's rpc_gss_wire_cred at function entry so that every early-return path leaves a deterministic all-zero cred. On the trailing tightness-check path, gc_ctx.len is now zero instead of stale, which neuters length-driven consumers such as gss_svc_searchbyctx() that would otherwise walk the dangling data pointer. Fixes: b0bc53470d1a ("SUNRPC: Convert the svcauth_gss_accept() pre-amble to use xdr_stream") Cc: stable@vger.kernel.org Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-5-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: Guard svcauth_gss_release() dispatch on rq_auth_statChris Mason
svcauth_gss_release() reads gc_proc and switches on gc_svc before consulting rq_auth_stat. On the SVC_DENIED path after a failed svcauth_gss_accept(), those fields may hold stale values from a prior request or uninitialized slab residue: svcauth_gss_accept() allocates gss_svc_data with non-zeroing kmalloc and clears only gsd_databody_offset and rsci per request, not clcred. Because RPC_GSS_PROC_DATA is zero, a zeroed or stale-zero gc_proc passes the existing guard and falls through into the gc_svc switch, which can dispatch to svcauth_gss_wrap_integ() or svcauth_gss_wrap_priv(). Both wrap helpers call svcauth_gss_prepare_to_wrap() before any rsci->mechctx dereference, and that helper already returns early when rq_auth_stat is not rpc_auth_ok, so the downstream NULL dereference is blocked. The dispatch itself remains structurally wrong: it reads scalars that the caller has no contract to have initialized after a failed authentication. Mirror the existing rq_auth_stat gate in svcauth_gss_prepare_to_wrap() one frame up, so svcauth_gss_release() skips the clcred dispatch entirely when authentication has not succeeded. The cleanup tail that releases rq_client, rq_gssclient, cr_group_info, and rsci still runs. Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Cc: stable@vger.kernel.org Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-4-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: reject duplicate CREDS_VALUE optionsChris Mason
gssx_dec_option_array() walks the wire-supplied option array and, for every entry whose name matches CREDS_VALUE, calls gssx_dec_linux_creds() on the same struct svc_cred. That helper unconditionally installs a fresh groups_alloc() result into creds->cr_group_info without releasing whatever pointer was already there: for (i = 0; i < count; i++) { ... decode name ... if (length == sizeof(CREDS_VALUE) && memcmp(p, CREDS_VALUE, sizeof(CREDS_VALUE)) == 0) { err = gssx_dec_linux_creds(xdr, creds); ... } } A reply that carries two CREDS_VALUE entries therefore overwrites cr_group_info on the second iteration and orphans the group_info allocated by the first call. The earlier free_creds path only releases the last cr_group_info via free_svc_cred(), so the first allocation's refcount stays at one and its kvmalloc-backed storage is leaked. No in-tree caller of gssp_accept_sec_context_upcall() expects more than one CREDS_VALUE per reply. Fix by tracking whether a CREDS_VALUE option has already been decoded and returning -EINVAL on any subsequent match, so the free_creds path releases the single group_info that was installed. Fixes: 1d658336b05f ("SUNRPC: Add RPC based upcall mechanism for RPCGSS auth") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-3-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: fix gssx_dec_option_array error path bugsChris Mason
Four coupled defects in the gssx XDR option-array decoder make the error paths unsafe: a NULL deref in the caller, a refcount leak on the decoded group_info, and a latent use-after-free that the leak fix would otherwise expose. gssx_dec_option_array() sets oa->count = 1 before allocating oa->data. If that allocation fails, -ENOMEM is returned with oa->count == 1 and oa->data == NULL. All other error paths jump to free_oa: which frees oa->data and NULLs it but also leaves oa->count == 1. The caller trusts the count: gssp_accept_sec_context_upcall() gssx_dec_accept_sec_context() gssx_dec_option_array() /* fails, count=1 data=NULL */ data = res.options.data[0].value /* NULL deref */ Independently, free_creds: releases the partially decoded svc_cred with a bare kfree(creds). gssx_dec_linux_creds() installs a groups_alloc() result into creds->cr_group_info; that object is kvmalloc-backed and refcounted, and only put_group_info() reaches kvfree(). A plain kfree(creds) drops the wrapper and leaks the group_info allocation. The natural fix for the leak is to call free_svc_cred(creds) before kfree(creds), but free_svc_cred() invokes put_group_info() on creds->cr_group_info unconditionally when non-NULL. The existing out_free_groups: path in gssx_dec_linux_creds() already called groups_free() on that pointer without clearing it, so once free_svc_cred() is wired in, the subsequent put_group_info() would touch freed memory. Fix all four together: - Move the oa->count = 1 assignment below the oa->data allocation so it is never set when oa->data is NULL. - Reset oa->count to 0 at free_oa: so count and data stay coherent and the caller sees an empty option array. - Call free_svc_cred(creds) before kfree(creds) at free_creds: so the refcounted cr_group_info is released. free_svc_cred() either NULL-guards each field explicitly (cr_group_info has an if() check) or delegates to a helper that is NULL-safe itself (kfree for the string fields, gss_mech_put() which guards with if(gm) at gss_mech_switch.c:342), so it is safe to call on a partially decoded svc_cred where only cr_uid/cr_gid/cr_group_info have been written and everything else is zero from kzalloc. - In gssx_dec_linux_creds()'s out_free_groups: path, release cr_group_info with put_group_info() rather than groups_free() so the teardown matches free_svc_cred()'s refcount-aware path, and clear the pointer so a later free_svc_cred() on the same creds does not release it a second time. Fixes: 3cfcfc102a5e ("SUNRPC: fix some memleaks in gssx_dec_option_array") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-2-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: Reject krb5 v2 wrap tokens with oversized ec fieldChuck Lever
gss_krb5_unwrap_v2() sets buf->len to a logical length, which can be much smaller than head[0].iov_len (the allocated receive-page capacity). It then calls xdr_buf_trim() with a trim length derived from the 16-bit "extra count" (ec) field in the Kerberos v2 token header. The ec field is authenticated by the post-decrypt memcmp() against the encrypted header copy, so a randomly-mutated value is rejected. However, any peer holding a valid GSS context can legitimately encrypt a token whose ec exceeds the plaintext length. Per RFC 4121, such a token is structurally malformed. Although xdr_buf_trim() now clamps the buf->len subtraction to avoid unsigned underflow, the buffer is still left in a semantically invalid state (zero length, inconsistent iov lengths) when ec is oversized. Reject these tokens before calling xdr_buf_trim(), giving callers a well-defined GSS_S_DEFECTIVE_TOKEN error and keeping the xdr_buf internally consistent. The wrapped blob begins at a nonzero offset -- both callers pass len as offset + opaque_len -- so buf->len still counts the offset bytes that precede the blob. Compare the trim length against the remaining wrapped segment, buf->len - offset, rather than the whole buffer; comparing against buf->len alone leaves an offset-wide window in which an oversized ec passes the test and xdr_buf_trim() cuts into the bytes ahead of the blob. Fixes: cf4c024b9083 ("sunrpc: trim off EC bytes in GSSAPI v2 unwrap") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260528-tier2-v1-1-d026a1415e0b@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Clear sc_cm_id when ADDR_CHANGE replacement failsChuck Lever
When svc_rdma_listen_handler() handles RDMA_CM_EVENT_ADDR_CHANGE, it creates a replacement listener cm_id and returns 1, telling the CM core to destroy the old one. If the replacement allocation fails, sc_cm_id still points at the old cm_id that the CM core is about to destroy. Any subsequent dereference of sc_cm_id -- such as svc_rdma_detach()'s rdma_disconnect() call -- is a use-after-free. NULL sc_cm_id on the failure path and guard svc_rdma_detach()'s rdma_disconnect() call against NULL so that the listener can be torn down safely when the server shuts down. Fixes: d1b586e75ec6 ("svcrdma: Handle ADDR_CHANGE CM event properly") Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260527-rdma-follow-on-v1-5-1b09bd87b6cd@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Reject connection when transport allocation failsChuck Lever
handle_connect_req() returns without action when svc_rdma_create_xprt() fails to allocate the new transport. The CM core returns 0 for CONNECT_REQUEST events, so it does not destroy the new rdma_cm_id. Each allocation failure under memory pressure leaks one rdma_cm_id, and a remote peer driving connection attempts can amplify this. Reject the connection by returning a non-zero status from the CM event handler, which tells the CM core to destroy the orphaned cm_id. Fixes: 377f9b2f4529 ("rdma: SVCRDMA Core Transport Services") Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260527-rdma-follow-on-v1-4-1b09bd87b6cd@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Use svc_xprt_put to free listener on create failureChuck Lever
svc_rdma_create() calls kfree(cma_xprt) when svc_rdma_create_listen_id() fails. svc_xprt_init() has already acquired a net namespace reference via get_net_track(); kfree bypasses svc_xprt_free() which releases it. Replace the kfree() with svc_xprt_put() so the kref_init birth reference drops to zero and svc_xprt_free() dispatches svc_rdma_free() to clean up properly. sc_cm_id is still NULL at that point; the preceding patch added the necessary NULL guard in svc_rdma_free(). svc_xprt_free() also drops the module reference via module_put(), but the caller _svc_xprt_create() does the same on xpo_create failure, double-putting the single try_module_get() it acquired. Take a compensating __module_get() before the svc_xprt_put() to keep the count balanced, matching the convention in svc_rdma_accept()'s error path. Fixes: 4fb8518bdac8 ("sunrpc: Tag svc_xprt with net") Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260527-rdma-follow-on-v1-3-1b09bd87b6cd@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Reorder rpcrdma_rn_unregister before rdma_destroy_idChuck Lever
svc_rdma_free() caches rdma->sc_cm_id->device before teardown, then calls rdma_destroy_id(sc_cm_id) which frees the cm_id. rpcrdma_rn_unregister() follows, but between those two calls the transport's sc_rn entry is still installed in the device's rd_xa. A concurrent ib_unregister_device walk can dispatch svc_rdma_xprt_done() against the now-freed sc_cm_id. Move rpcrdma_rn_unregister() before rdma_destroy_id() so the transport's notification entry is removed from the xarray before the cm_id it references is destroyed. Also guard the sc_cm_id dereference with a NULL check: the following patches introduce paths that reach svc_rdma_free() with sc_cm_id == NULL (listener create failure, ADDR_CHANGE replacement failure). Fixes: c4de97f7c454 ("svcrdma: Handle device removal outside of the CM event handler") Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260527-rdma-follow-on-v1-2-1b09bd87b6cd@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Fix unmatched rn_unregister on failed acceptChris Mason
When svc_rdma_accept() takes the errout path before rpcrdma_rn_register() has succeeded, the existing cleanup block calls rpcrdma_rn_unregister(dev, &newxprt->sc_rn) unconditionally. svcxprt_rdma is kzalloc'd, so on that path sc_rn.rn_index is 0 and sc_rn.rn_done is NULL; the unregister therefore xa_erase()s another caller's slot 0 and performs an unmatched kref_put() on the rpcrdma_device's rd_kref. The same errout also brackets the cleanup with svc_xprt_get()/ svc_xprt_put() around the kref_init() birth reference. The kref goes 1 -> 2 -> 1 and never reaches 0, so the svcxprt_rdma (and the net/ns_tracker it pinned) is leaked on every failed accept. rpcrdma_rn_register() writes rn->rn_done last, only after xa_alloc() and kref_get() have both succeeded, so rn_done == NULL is a natural "never registered" sentinel. Guard rpcrdma_rn_unregister() with an early return when rn_done is NULL, and clear rn_done before the matching xa_erase() so a repeated unregister is also a no-op. With that guard in place, the accept errout drops the kref_init() birth reference via svc_xprt_put(), which dispatches svc_rdma_free(). Teardown of sc_qp, sc_sq_cq, sc_rq_cq, and sc_pd runs under existing IS_ERR/NULL guards in svc_rdma_free(); sc_rn is covered by the new rn_done sentinel; sc_cm_id is non-NULL on every errout path because svc_rdma_accept() dereferences it above the first goto errout. svc_xprt_free() drops the module reference associated with the freed transport, and svc_handle_xprt() drops its pre-acquired reference when ->xpo_accept() returns NULL. Take a replacement module reference before svc_xprt_put() so the two module_put()s remain balanced. The rn_done guard also covers svc_rdma_free()'s non-listener call to rpcrdma_rn_unregister() for transports whose register attempt failed or never ran. Fixes: 8ac6fcae5dc0 ("svcrdma: Unregister the device if svc_rdma_accept() fails") Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260527-rdma-follow-on-v1-1-1b09bd87b6cd@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: always drain cache_cleaner before destroying a cache_detailJeff Layton
sunrpc_destroy_cache_detail() only cancels the global cache_cleaner delayed_work when cache_list is empty. During per-netns teardown cache_list is never empty because init_net's caches remain registered, so the cancel never fires. After unlink, the caller proceeds to cache_destroy_net() which kfrees the cache_detail while cache_clean() may still hold a dangling pointer to it. The result is a use-after-free: cache_dequeue() takes cd->queue_lock on freed memory, and cache_put() dereferences cd->cache_put as a function pointer from freed slab. Drop the list_empty guard so that cancel_delayed_work_sync() always runs, ensuring any in-flight cache_clean() completes before the cache_detail is freed. Re-arm the cleaner afterwards if other caches are still registered. Fixes: 820f9442e711 ("SUNRPC: split cache creation and PipeFS registration") Cc: stable@vger.kernel.org Assisted-by: Claude:claude-opus-4-6 Signed-off-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-cache_cleaner_vs_destroy_no_sync-v1-1-a707a6fcfd32@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Reject Read lists that exceed the page budgetChuck Lever
Individual Read segment lengths are validated at decode time, but nothing prevents a requester from sending multiple segments whose cumulative length exceeds the rq_pages array budget. When one segment fills the page array exactly, the runtime guard in svc_rdma_build_read_segment() is bypassed because len reaches zero. A subsequent segment then accesses the NULL sentinel slot at rq_pages[rq_maxpages], resulting in a NULL pointer dereference during DMA mapping. Accumulate pages across all Read segments and reject the message at decode time when the total would overflow the page budget. Fixes: 026d958b38c6 ("svcrdma: Add recvfrom helpers to svc_rdma_rw.c") Cc: stable@vger.kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Validate Read chunk positions at decode timeChuck Lever
Read chunk position and length validation is currently scattered across three consumer functions: svc_rdma_read_data_item(), svc_rdma_read_multiple_chunks(), and svc_rdma_read_call_chunk(). Each independently guards against the same class of unsigned arithmetic underflow from untrusted wire values. Any new consumer of the parsed Read chunk list must replicate these checks or risk re-introducing the defects fixed by earlier patches in this series. Add pcl_check_read_chunk_positions() to consolidate position and length validation into a single post-decode pass, called from svc_rdma_xdr_decode_req() after all three chunk lists have been parsed and the inline body length is known. The pass verifies three properties: - Each Read chunk's inline-body offset (its unreduced-stream position minus the cumulative length of preceding Read chunks) falls within the inline body length, or within the Call chunk length for interleaved reads. - Adjacent Read chunk positions do not overlap: cumulative read bytes at each transition do not exceed the next position. - Each chunk length does not exceed the receive context's page budget. Malformed frames are rejected before reaching any consumer. The existing consumer-side guards remain as defense in depth. Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-6-e251306ccca9@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Reject Write/Reply chunks with segcount 0Chris Mason
A peer can send a Write or Reply chunk whose segcount field is zero. xdr_check_write_chunk() only rejects segcount > rc_maxpages, so zero passes the range check, and xdr_inline_decode(stream, 0) returns the current (non-NULL) cursor without advancing. The function returns true and pcl_alloc_write() then links a struct svc_rdma_chunk with ch_segcount == 0 onto rc_write_pcl or rc_reply_pcl. An earlier patch in this series made pcl_for_each_segment() safe for ch_segcount == 0, so this no longer drives the memory walk it used to. Rejecting the malformed frame at the decode boundary is still worthwhile as defense in depth: it keeps degenerate zero-segment chunks off the parsed chunk lists entirely, so any future consumer that walks ch_segments directly cannot observe one, and it makes the zero-floor easy to backport to trees where the macro change is more intrusive. RFC 8166 has no meaning for a Write/Reply chunk that describes no remote buffer, so no legitimate client is affected. xdr_check_reply_chunk() funnels Reply chunks through xdr_check_write_chunk() and inherits the same rejection. pcl_alloc_write() also links each chunk onto the parsed chunk list before filling its segment array. If a future change weakens the segcount-0 rejection, an incomplete chunk is visible to consumers during the fill loop. Reorder so that list_add_tail() follows the segment fill loop, ensuring only fully-populated chunks appear on the list. Fixes: 78147ca8b4a9 ("svcrdma: Add a "parsed chunk list" data structure") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-5-e251306ccca9@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Reject oversized Read segments at decode timeChuck Lever
The RPC/RDMA Read list decoder stores wire-supplied segment lengths without validation. xdr_count_read_segments() checks 4-byte alignment for non-zero position values but does not cap the segment length. An oversized rs_length reaches svc_rdma_build_read_segment(), which derives nr_bvec from it and can drive a large dynamic bvec allocation before verifying that enough rq_pages remain. If the post-allocation page-overrun guard fires, the freshly acquired rw context is not returned, leaking the resource. Reject any segment whose length exceeds the receive context's page budget during Read list decoding, consistent with how xdr_check_write_chunk() bounds Write segment counts against rc_maxpages. Also return the rw context on the existing post-allocation overrun path in svc_rdma_build_read_segment(), keeping that defensive guard balanced. Fixes: 5ee62b4a9113 ("svcrdma: use bvec-based RDMA read/write API") Cc: stable@vger.kernel.org Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-3-e251306ccca9@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Fix offset arithmetic in read_chunk_rangeChris Mason
svc_rdma_read_chunk_range() walks a Read chunk's segment list to build a sub-range starting at byte offset and spanning length bytes for a Position-Zero or Call chunk. Two arithmetic defects in the per-segment loop produce wrong DMA lengths and a u32 underflow: pcl_for_each_segment(segment, chunk) { if (offset > segment->rs_length) { offset -= segment->rs_length; continue; } dummy.rs_handle = segment->rs_handle; dummy.rs_length = min_t(u32, length, segment->rs_length) - offset; dummy.rs_offset = segment->rs_offset + offset; First, the skip predicate uses '>' instead of '>='. When offset equals the segment's full rs_length, the segment is fully consumed and should be skipped, but the loop falls through into the body. The resulting dummy.rs_length is min_t(u32, length, rs_length) - rs_length, which underflows to a near-UINT_MAX u32 when length is smaller than rs_length, or is zero otherwise. Second, the length formula subtracts offset from the min_t() result rather than from segment->rs_length before the cap. For offset > 0 the segment's residual is rs_length - offset, not rs_length, so the cap must be applied to the residual. With the current bracketing, whenever length is smaller than rs_length - offset the per-segment length becomes length - offset instead of length, silently dropping offset bytes from the rebuilt chunk. Combined with the boundary case above it also enables the u32 underflow path, which propagates a huge nr_bvec into svc_rdma_build_read_segment() and a multi-MiB kmalloc_array_node() in svc_rdma_get_rw_ctxt(). Additionally, svc_rdma_read_call_chunk() can invoke this function with length == 0 when the last Read chunk ends exactly at the end of the Call chunk. With the corrected >= predicate, every segment is skipped and the function returns the initial -EINVAL, rejecting a valid request. Return success immediately when length is zero. Also break out of the loop once length is fully consumed to avoid passing zero-length segments to svc_rdma_build_read_segment(). Fix by using '>=' so a fully-consumed segment is skipped, by moving '- offset' inside min_t() so the cap is applied to the segment's residual length, by returning success for zero-length requests, and by stopping iteration when the requested range has been consumed. Fixes: d7cc73972661 ("svcrdma: support multiple Read chunks per RPC") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-2-e251306ccca9@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 dayssvcrdma: Validate Read chunk positions before reconstructionChuck Lever
The RPC/RDMA Read chunk position field is supplied by the remote client and stored verbatim in the parsed chunk list. xdr_count_read_segments() checks only 4-byte alignment; it never compares the position against the received inline body length. In the single-chunk path, svc_rdma_read_complete_one() splits the head and tail kvecs at ch_position. A position past the inline body underflows the tail length, exposing adjacent slab memory to the upper XDR decoder. In the multi-chunk path, svc_rdma_read_multiple_chunks() computes gap lengths between chunks as unsigned subtractions from ch_position. Overlapping Read chunks cause these subtractions to underflow. A final position past the inline body likewise underflows the trailing gap length. svc_rdma_copy_inline_range() then copies past the receive buffer into request pages that are returned to the client through the Reply channel. Bound inline-range copies in svc_rdma_copy_inline_range() against the decoded inline RPC body saved in rc_saved_arg. Reject a single Read chunk positioned beyond that body, and reject multi-chunk lists where accumulated read bytes exceed the next chunk's position. Apply the same position and overlap checks in the call-chunk interleaving path. Fixes: d96962e6d0e2 ("svcrdma: Use the new parsed chunk list when pulling Read chunks") Cc: stable@vger.kernel.org Acked-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260526-rpc-kernel-bugs-v1-1-e251306ccca9@oracle.com Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: harden gss_krb5_unwrap_v2 against short tokensChris Mason
gss_krb5_unwrap_v2() reads the EC and RRC header fields at ptr+4 and ptr+6 before validating that the token is at least GSS_KRB5_TOK_HDR_LEN (16) bytes long, and its rotate_left() helper passes buf->len - base to xdr_buf_subsegment() without verifying that base <= buf->len. When a caller hands in a sub-16-byte token, or a token whose declared len leaves base past the end of the buffer, three distinct failures follow: gss_krb5_unwrap_v2(offset, len, buf) ptr = buf->head[0].iov_base + offset ec = *(ptr + 4) /* OOB read on short head */ rrc = *(ptr + 6) /* OOB read on short head */ rotate_left(offset + 16, buf, rrc) xdr_buf_subsegment(buf, &subbuf, base, buf->len - base) /* u32 wrap when base > len */ _rotate_left(&subbuf, shift) shift %= buf->len /* divide-by-zero when base == len */ After decryption, the cleanup arithmetic has the same shape: movelen = min_t(unsigned int, buf->head[0].iov_len, len); movelen -= offset + GSS_KRB5_TOK_HDR_LEN + headskip; BUG_ON(offset + GSS_KRB5_TOK_HDR_LEN + headskip + movelen > buf->head[0].iov_len); The BUG_ON re-adds the value just subtracted, so it reduces to min(A, B) > A and is permanently false; it cannot catch the unsigned underflow of movelen, which then drives a ~UINT_MAX-byte memmove(). Add four defense-in-depth guards inside the unwrap core so it is safe regardless of what its callers validate: - reject tokens with len - offset < GSS_KRB5_TOK_HDR_LEN before touching ptr+4/ptr+6; - bail from rotate_left() when buf->len <= base, covering both the underflow and zero-length cases; - return early from _rotate_left() when buf->len is zero, so the shift %= buf->len modulo cannot fault; - replace the dead BUG_ON with a live check that returns GSS_S_DEFECTIVE_TOKEN before the movelen subtraction. Fixes: de9c17eb4a91 ("gss_krb5: add support for new token formats in rfc4121") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260524010213.557424-5-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: xdr_buf_trim: clamp buf->len to avoid underflowChris Mason
xdr_buf_trim() trims `len` bytes from the tail of an xdr_buf by walking the tail, pages, and head iovecs. Each per-section step uses min_t() so it never removes more bytes than that section holds, but the final accounting at the fix_len label subtracts the total bytes actually consumed from buf->len without any clamp: fix_len: buf->len -= (len - trim); When the caller has set buf->len to a value smaller than the sum of the iov_lens, (len - trim) can exceed buf->len and the unsigned subtraction wraps to near UINT_MAX. gss_krb5_unwrap_v2() reaches xdr_buf_trim() in exactly that state: buf->head[0].iov_len -= GSS_KRB5_TOK_HDR_LEN + headskip; buf->len = len - (GSS_KRB5_TOK_HDR_LEN + headskip); xdr_buf_trim(buf, ec + GSS_KRB5_TOK_HDR_LEN + tailskip); buf->len is a small wire-derived value while the iov_lens are at page scale, so the per-section loops legitimately consume far more bytes than buf->len records. The wrapped buf->len then propagates as the authoritative stream bound into every downstream XDR decoder. Fix by clamping the decrement so buf->len bottoms out at zero: buf->len -= min_t(unsigned int, buf->len, len - trim); On the normal path where the iov_lens sum to buf->len, (len - trim) is always <= buf->len and the result is identical to before. No callers change behavior outside the underflow case. Fixes: 4c190e2f913f ("sunrpc: trim off trailing checksum before returning decrypted or integrity authenticated buffer") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260524010213.557424-4-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: harden gss_unwrap_resp_priv length checksChris Mason
gss_unwrap_resp_priv() validates the RPCSEC_GSS opaque length with offset = (u8 *)(p) - (u8 *)head->iov_base; if (offset + opaque_len > rcv_buf->len) goto unwrap_failed; maj_stat = gss_unwrap(ctx->gc_gss_ctx, offset, offset + opaque_len, rcv_buf); Both operands are u32 and the sum is computed in u32. A reply with opaque_len near 0xffffffff makes offset + opaque_len wrap to a small value that is below rcv_buf->len, so the bound check passes and gss_unwrap() is called with end < begin. The check also lacks a lower bound, so any opaque_len in [0, GSS_KRB5_TOK_HDR_LEN) is accepted and forwarded to gss_krb5_unwrap_v2(), whose pre-decrypt header reads at ptr+4 and ptr+6 then run past the token. A krb5p NFS server returning a crafted RPCSEC_GSS reply can drive the client into out-of-bounds reads in gss_krb5_unwrap_v2() and the rotate_left() loop that follows. Fix by replacing the single combined check with three guards that are safe in u32 arithmetic and that enforce the RFC 4121 minimum outer token length: if (offset > rcv_buf->len) goto unwrap_failed; if (opaque_len > rcv_buf->len - offset) goto unwrap_failed; if (opaque_len < GSS_KRB5_TOK_HDR_LEN) goto unwrap_failed; The first guard makes the subtraction in the second guard unconditionally safe; offset is derived from a successful xdr_inline_decode() in the head kvec, so in practice it already satisfies the bound. The floor mirrors the server-side check added in commit 5b757c2e57a5 ("SUNRPC: svcauth_gss: enforce krb5 token minimum length"). Fixes: 2d2da60c63b6 ("RPCSEC_GSS: client-side privacy support") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260524010213.557424-3-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: svcauth_gss: enforce krb5 token minimum lengthChris Mason
svcauth_gss_unwrap_priv() validates only an upper bound on the wire-supplied opaque length before handing the buffer to gss_unwrap(): if (len > xdr_stream_remaining(xdr)) goto unwrap_failed; offset = xdr_stream_pos(xdr); ... maj_stat = gss_unwrap(ctx, offset, offset + len, buf); The wire value `len` flows unchanged as the upper bound into the krb5 unwrap path, so a len in [0, 16] passes this check and is handed to gss_unwrap(). For a krb5 v2 context that lands in gss_krb5_unwrap_v2(), which reads the 16-byte RFC 4121 token header fields at ptr+4 and ptr+6 and then calls rotate_left() before any integrity check. With a sub-header length the header reads run past the token, and _rotate_left()'s `shift %= buf->len` path can divide by zero when buf->len has been driven to zero by the truncated token. A header-only token (len == 16) is equally invalid: with a non-zero RRC field and the opaque blob ending at the XDR buffer boundary, rotate_left() builds a zero-length subbuffer, reaching the same division. Reject the token at the server entry point before it reaches the krb5 unwrap core. A valid sealed RFC 4121 token must contain the 16-byte header plus at least some encrypted payload. Fix by adding a minimum-length check immediately after the existing upper-bound check: if (len <= GSS_KRB5_TOK_HDR_LEN) goto unwrap_failed; Fixes: 7c9fdcfb1b64 ("[PATCH] knfsd: svcrpc: gss: server-side implementation of rpcsec_gss privacy") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260524010213.557424-2-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
7 daysSUNRPC: Reject short RFC 4121 MIC tokens in gss_krb5_verify_mic_v2Chuck Lever
gss_krb5_verify_mic_v2() reads the token ID at ptr[0..1], the flags byte at ptr[2], and padding at ptr[3..7], then passes ptr + GSS_KRB5_TOK_HDR_LEN and cksum_len to gss_krb5_mic_build_sg(). None of these accesses check read_token->len first. The minimum safe token size is GSS_KRB5_TOK_HDR_LEN (16) plus ctx->krb5e->cksum_len (12-24, depending on the enctype). All callers accept shorter tokens from the wire: - gss_unwrap_resp_integ() enforces only an upper bound (offset + len <= rcv_buf->len) before allocating mic.data = kmalloc(len) and passing it to gss_verify_mic(). A malicious NFS server can therefore supply a short checksum opaque, producing a small slab allocation that the Kerberos MIC verifier reads past. - gss_validate() enforces only len <= RPC_MAX_AUTH_SIZE (400) before passing the wire-supplied length to gss_validate_seqno_mic(), which constructs a mic xdr_netobj and calls gss_verify_mic(). - svcauth_gss_verify_header() enforces only checksum.len >= XDR_UNIT (4 bytes) before dispatching to gss_verify_mic(). - svcauth_gss_unwrap_integ() checks only that the checksum fits in gsd->gsd_scratch. Add a length guard at the top of gss_krb5_verify_mic_v2(), before any ptr[] access or scatterlist construction. Well-formed MIC tokens from gss_krb5_get_mic_v2() already have exactly GSS_KRB5_TOK_HDR_LEN + cksum_len bytes, so valid traffic is unaffected. Reported-by: Chris Mason <clm@meta.com> Fixes: de9c17eb4a91 ("gss_krb5: add support for new token formats in rfc4121") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Link: https://patch.msgid.link/20260523165237.510204-1-cel@kernel.org Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-23Merge tag 'nfs-for-7.2-1' of git://git.linux-nfs.org/projects/anna/linux-nfsLinus Torvalds
Pull NFS client updates from Anna Schumaker: "New features: - XPRTRDMA: Decouple req recycling from RPC completion - NFS: Expose FMODE_NOWAIT for read-only files Bugfixes: - SUNRPC: - Fix sunrpc sysfs error handling - Fix uninitialized xprt_create_args structure - XPRTRDMA: - Harden connect and reply handling - NFS: - Fix EOF updates after fallocate/zero-range - Keep PG_UPTODATE clear after read errors in page groups - Use nfsi->rwsem to protect traversal of the file lock list - Prevent resource leak in nfs_alloc_server() - NFSv4: - Clear exception state on successful mkdir retry - Don't skip revalidate when holding a dir delegation and attrs are stale - pNFS: - Fix use-after-free in pnfs_update_layout() - Defer return_range callbacks until after inode unlock - Fix LAYOUTCOMMIT retry loop on OLD_STATEID - Reject zero-length r_addr in nfs4_decode_mp_ds_addr - NFS/flexfiles: - Reject zero-length filehandle version arrays - Fix checking if a layout is striped - Fixes for honoring FF_FLAGS_NO_IO_THRU_MDS Other cleanups and improvements: - Remove the fileid field from struct nfs_inode - Move long-delayed xprtrdma work onto the system_dfl_long_wq - Convert xprtrdma send buffer free list to an llist - Show "<redacted>" for cert_serial and privkey_serial mount options" * tag 'nfs-for-7.2-1' of git://git.linux-nfs.org/projects/anna/linux-nfs: (42 commits) NFS: Use common error handling code in nfs_alloc_server() NFS: Prevent resource leak in nfs_alloc_server() NFSv4/pNFS: reject zero-length r_addr in nfs4_decode_mp_ds_addr nfs: don't skip revalidate on directory delegation when attrs flagged stale xprtrdma: Return sendctx slot after Send preparation failure xprtrdma: Repost Receive buffers for malformed replies xprtrdma: Sanitize the reply credit grant after parsing xprtrdma: Fix bcall rep leak and unbounded peek xprtrdma: Resize reply buffers before reposting receives xprtrdma: Check frwr_wp_create() during connect xprtrdma: Initialize re_id before removal registration xprtrdma: Fix ep kref imbalance on ADDR_CHANGE xprtrdma: Convert send buffer free list to llist NFS: correct CONFIG_NFS_V4 macro name in #endif comment nfs: use nfsi->rwsem to protect traversal of the file lock list NFSv4.1/pNFS: fix LAYOUTCOMMIT retry loop on OLD_STATEID nfs: expose FMODE_NOWAIT for read-only files nfs: add nowait version of nfs_start_io_direct NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS in pg_get_mirror_count_write NFSv4/flexfiles: honor FF_FLAGS_NO_IO_THRU_MDS on fatal DS connect errors ...
2026-06-10xprtrdma: Return sendctx slot after Send preparation failureChuck Lever
rpcrdma_prepare_send_sges() gets a sendctx before it maps the SGEs for the Send WR. If one of the mapping helpers fails, no Send WR is posted, so no Send completion is guaranteed to advance rb_sc_tail. Current cleanup clears sc_req so a later completion can sweep over that slot, but a consecutive run of preparation failures can still advance rb_sc_head until the ring appears full. At that point rpcrdma_sendctx_get_locked() returns NULL and no Send can be posted to produce the completion needed to recover the ring. The trigger requires CONFIG_SUNRPC_XPRT_RDMA and an NFS/RDMA mount. Mount setup and reliable DMA-map fault injection require local admin authority. Unprivileged I/O on an existing mount can exercise the send path, but a remote peer alone cannot force this local DMA-map failure. Add rpcrdma_sendctx_unget_locked() for the single-consumer send path to rewind rb_sc_head when the just-acquired sendctx is canceled before ib_post_send(). Wake waiters after making the slot available again. After the rewind, every slot the completion sweep visits belongs to a posted Send, so rpcrdma_sendctx_put_locked() no longer needs to test sc_req before unmapping. Fixes: ae72950abf99 ("xprtrdma: Add data structure to manage RDMA Send arguments") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Repost Receive buffers for malformed repliesChuck Lever
rpcrdma_wc_receive() decrements the transport's Receive count for every completion before it dispatches a successful Receive to rpcrdma_reply_handler(). The handler must post a replacement Receive WR before returning unless ownership of the rep has moved elsewhere, as on the backchannel path. Commit 2ae50ad68cd7 ("xprtrdma: Close window between waking RPC senders and posting Receives") moved the Receive refill out of rpcrdma_wc_receive(), where it had run ahead of every reply, into rpcrdma_reply_handler() so that the responder's credit grant could be parsed before reposting. The bad-version and short-reply exits never reach that refill: they recycle the rep and return without calling rpcrdma_post_recvs(). A remote peer can therefore drain the client's posted Receive queue by sending a sustained stream of replies that are shorter than the fixed transport header or that carry an unrecognized RPC/RDMA version. Each such reply consumes one posted Receive without replacing it. Once the queue empties, the peer's next Send finds no posted Receive and the transport stalls until reconnect. Route both malformed-reply exits through the shared repost tail after recycling the rep, refilling against buf->rb_credits, the most recent accepted credit grant. Neither exit updates the congestion window, so RPCs admitted under the previous grant remain in flight awaiting replies. A smaller refill target would let a stream of malformed replies ratchet the posted Receive count down to the batch floor while the congestion window still admits rb_credits RPCs; a burst of valid replies to those RPCs could then overrun the posted Receives, and because the client connects with rnr_retry_count of zero, a single RNR NAK terminates the connection. Refilling against rb_credits also restores the target that applied to malformed replies before commit 2ae50ad68cd7 ("xprtrdma: Close window between waking RPC senders and posting Receives") when rpcrdma_post_recvs() computed it from rb_credits internally. rb_credits is at least one from connection establishment onward, so the repost path always keeps Receives posted. Fixes: 2ae50ad68cd7 ("xprtrdma: Close window between waking RPC senders and posting Receives") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Sanitize the reply credit grant after parsingChuck Lever
The out_norqst exit in rpcrdma_reply_handler() branches away before the credit clamp, so a reply that matches no pending request reaches out_post carrying the raw credit value parsed from the wire. rpcrdma_post_recvs() does not bound its @needed argument: the refill loop allocates and chains Receive WRs until the count is satisfied or allocation fails. A peer that sends a well-formed reply carrying an unknown XID and an inflated credit grant therefore drives rep allocation and Receive posting past re_max_requests on every such reply. Move the clamp to immediately after the credit field is parsed, ahead of the first branch that can reach out_post, so every later consumer sees a sanitized value. The cwnd update stays on the matched-request path. Fixes: 704f3f640f72 ("xprtrdma: Post receive buffers after RPC completion") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Fix bcall rep leak and unbounded peekChris Mason
rpcrdma_is_bcall() decodes a reply's first words to decide whether the frame is a backchannel call. Two issues in that decode path let a short or malformed reply leak the receive buffer and drain the Receive queue. First, the speculative peek p = xdr_inline_decode(xdr, 0); /* five p++ reads follow */ asks xdr_inline_decode() for zero bytes, which returns xdr->p without consulting xdr->end. The five subsequent __be32 reads can then walk up to 20 bytes past the wire payload into stale regbuf contents and misclassify the reply as a backchannel call. Second, after the post-peek p = xdr_inline_decode(xdr, 3 * sizeof(*p)); if (unlikely(!p)) return true; the short-header arm returns true without calling rpcrdma_bc_receive_call(). The contract with the caller is that a true return transfers ownership of rep to the backchannel path: rpcrdma_reply_handler() if (rpcrdma_is_bcall(r_xprt, rep)) return; /* bare return, skips out_post */ ... out_post: rpcrdma_post_recvs(r_xprt, credits + ...); Because rpcrdma_bc_receive_call() never ran, no one took rep, but rpcrdma_reply_handler still bare-returns past rpcrdma_rep_put() and rpcrdma_post_recvs(). The rep, with its persistently DMA-mapped receive buffer, is orphaned on rb_all_reps and freed only at transport teardown. This completion reposts nothing, so its slot is reclaimed only when a later forward-channel reply reaches out_post and rpcrdma_post_recvs() allocates a fresh rep to backfill; absent that traffic the Receive queue drains and the peer's Sends draw RNR NAKs. Fix by consulting xdr->end after the zero-length peek so the five __be32 reads cannot run unless 20 bytes of wire payload remain. A byte-precise comparison against xdr->end is required because a non-4-aligned receive rounds the stream's word count up past the true payload. Also return false from the short-header arm so the reply falls through the normal out_norqst cleanup chain (rpcrdma_rep_put() plus rpcrdma_post_recvs()). Fixes: 41c8f70f5a3d ("xprtrdma: Harden backchannel call decoding") Assisted-by: kres:claude-opus-4-7 Signed-off-by: Chris Mason <clm@meta.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Resize reply buffers before reposting receivesChuck Lever
Commit 0e13dd9ea8be ("xprtrdma: Remove temp allocation of rpcrdma_rep objects") made rpcrdma_rep objects survive disconnects. That is normally fine, but it also means their receive regbufs keep the size they had when they were first allocated. Each rep's receive buffer is sized to ep->re_inline_recv when the rep is created. rpcrdma_ep_create() resets that threshold to the rdma_max_inline_read ceiling for every new endpoint, and the connect handshake then shrinks it to the peer's advertised inline send size. A rep allocated under a smaller negotiated threshold keeps that size: on disconnect, rpcrdma_xprt_disconnect() drains and DMA-unmaps the surviving reps but does not free or resize them. The threshold can come back larger on the next connection. The first peer may supply no RPC-over-RDMA CM private data, defaulting its send size to 1024, while the reconnect target is an ordinary server offering 4096; or, with rdma_max_inline_read raised above its default, the reconnect target may advertise a larger svcrdma_max_req_size than the first. rpcrdma_post_recvs() then reposts a surviving rep whose SGE length is still the old, smaller value, and a larger inline Reply hits a receive length error and forces another disconnect. The undersized rep returns to the free list when its failed Receive flushes, so the following reconnect reposts the same rep and fails the same way. The transport flaps without making forward progress for as long as the peer keeps advertising the larger inline size. This is local/admin-triggerable rather than remote-triggerable: a local administrator must create and maintain the NFS/RDMA mount, while the server or reconnect target has to advertise a larger inline send size and return a reply that uses it. Fix this by checking each rep before it is reposted. If the receive regbuf is smaller than the current endpoint's inline receive size, reallocate it on the current RDMA device's NUMA node and reinitialize the rep's xdr_buf before DMA-mapping and posting the Receive WR. Fixes: 0e13dd9ea8be ("xprtrdma: Remove temp allocation of rpcrdma_rep objects") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Check frwr_wp_create() during connectChuck Lever
frwr_wp_create() creates the singleton Memory Region used to encode padding for Write chunks whose payload length is not XDR-aligned. Its failure paths return a negative errno and leave ep->re_write_pad_mr set to NULL. rpcrdma_xprt_connect() currently ignores that return value. If frwr_wp_create() fails after the rest of the connection setup succeeds, xprt_rdma_connect_worker() treats the connection attempt as successful and sets XPRT_CONNECTED. A later NFS/RDMA read with a non-4-byte-aligned receive page length reaches rpcrdma_encode_write_list(), passes the NULL write-pad MR to encode_rdma_segment(), and dereferences it. This is locally triggerable on an NFS/RDMA client after a connect or reconnect hits a local MR allocation, DMA-map, MR-map, or post-send failure; a remote peer alone cannot force the local MR setup failure. Check the return value and fail the connect as -ENOTCONN, matching the adjacent setup failures. This keeps XPRT_CONNECTED clear and lets the normal reconnect path retry. Fixes: 21037b8c2258 ("xprtrdma: Provide a buffer to pad Write chunks of unaligned length") Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Initialize re_id before removal registrationChris Mason
rpcrdma_create_id() registers ep->re_rn with the rpcrdma ib_client before returning the new rdma_cm_id to rpcrdma_ep_create(). However rpcrdma_ep_create() currently stores that pointer in ep->re_id only after rpcrdma_create_id() returns. A local administrator can race an NFS/RDMA mount against RDMA device removal. If rpcrdma_remove_one() observes the just-registered notification before rpcrdma_ep_create() assigns ep->re_id, rpcrdma_ep_removal_done() calls trace_xprtrdma_device_removal(NULL). The tracepoint dereferences id->device->name and copies id->route.addr.dst_addr, so the callback can crash the kernel with a NULL pointer dereference. Store the rdma_cm_id in ep->re_id immediately before publishing ep->re_rn. The existing error path still destroys the id directly if registration fails; ep is then freed by the caller without using ep->re_id. Remove the later duplicate assignment in rpcrdma_ep_create(). Fixes: 3f4eb9ff9234 ("xprtrdma: Handle device removal outside of the CM event handler") Assisted-by: kres:openai-gpt-5 Signed-off-by: Chris Mason <clm@meta.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Fix ep kref imbalance on ADDR_CHANGEChris Mason
rpcrdma_cm_event_handler() falls through to the disconnected: label on RDMA_CM_EVENT_ADDR_CHANGE and calls rpcrdma_ep_put() with no matching get when the event arrives before RDMA_CM_EVENT_ESTABLISHED. The kref then underflows during connect teardown and rpcrdma_xprt_disconnect() operates on a freed ep. Reference counts across a normal connection lifecycle: rpcrdma_ep_create() kref_init ->1 rpcrdma_xprt_connect() ep_get ->2 (before post_recvs) RDMA_CM_EVENT_ESTABLISHED ep_get ->3 RDMA_CM_EVENT_DISCONNECTED ep_put ->2 rpcrdma_xprt_drain() ep_put ->1 rpcrdma_xprt_disconnect() tail ep_put ->0 (ep_destroy) The connect-time get in rpcrdma_xprt_connect(), taken just before rpcrdma_post_recvs() "while there are outstanding Receives," is balanced by rpcrdma_xprt_drain. ADDR_CHANGE before ESTABLISHED has no get to consume, so its put drops the count to 1 and the drain put then frees the ep while rpcrdma_xprt_disconnect() still holds a pointer to it. Fix by dispatching on the prior re_connect_status via xchg(): for prev == 0 (pre-ESTABLISHED) wake the connect waiter and return with no put; for prev == 1 call rpcrdma_force_disconnect() and return. The case-1 arm relies on the subsequent RDMA_CM_EVENT_DISCONNECTED event -- reliably delivered when rdma_disconnect() is called on a still-connected cm_id -- to balance the ESTABLISHED get; rpcrdma_xprt_drain() continues to balance only that connect-time get. Any other prior value means teardown is already in flight. Fixes: 2acc5cae2923 ("xprtrdma: Prevent dereferencing r_xprt->rx_ep after it is freed") Assisted-by: kres:claude-opus-4-7 Signed-off-by: Chris Mason <clm@meta.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-10xprtrdma: Convert send buffer free list to llistChuck Lever
rpcrdma_buffer_get() and rpcrdma_buffer_put() both take rb_lock to pop/push from the rb_send_bufs free list. Under high I/O concurrency (e.g., nconnect=N with small random writes), this spinlock is contended between the request submission path and the transport completion path. Replace the list_head with an llist_head. The put side uses lockless llist_add(), which is safe for concurrent producers. The get side retains the spinlock to satisfy the llist single-consumer contract portably; submitters continue to serialize there. Completion handlers returning buffers no longer contend on rb_lock, eliminating contention on the return path. rb_lock remains for the MR free list and the tracking lists used during setup and teardown. rb_free_reps already uses llist_head, so the llist idiom is established in this structure. The precedent is the data structure, not the locking: rb_free_reps serializes its single consumer through the re_receiving gate in rpcrdma_post_recvs, whereas rb_send_bufs serializes its consumer with rb_lock. Both satisfy the llist single-consumer contract. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
2026-06-09svcrdma: wake sq waiters when the transport closesChuck Lever
Threads parked in svc_rdma_sq_wait() on sc_sq_ticket_wait or sc_send_wait can hang indefinitely in TASK_UNINTERRUPTIBLE state across transport teardown, pinning svc_xprt references and blocking svc_rdma_free(). The close path sets XPT_CLOSE before invoking xpo_detach and both wait_event predicates include an XPT_CLOSE term, but the predicates are re-evaluated only on wakeup. sc_sq_ticket_wait has no completion-driven wake path; it is advanced solely by the chained ticket handoff inside svc_rdma_sq_wait() itself. Without an explicit wake at close, parked threads never observe XPT_CLOSE, hold their svc_xprt_get reference forever, and svc_rdma_free() blocks on xpt_ref dropping to zero. Two close entry points reach this transport. Local teardown runs svc_rdma_detach() from svc_handle_xprt() -> svc_delete_xprt() -> xpo_detach() on a worker thread. A remote disconnect arrives at svc_rdma_cma_handler(), which calls svc_xprt_deferred_close(): that sets XPT_CLOSE and enqueues the transport but does not access either RDMA waitqueue, so a worker already parked in svc_rdma_sq_wait() never re-evaluates its predicate. With every worker parked on this transport, no thread is available to run the local teardown either, and the wake site there is unreachable. Introduce svc_rdma_xprt_deferred_close(), a thin svcrdma wrapper that calls svc_xprt_deferred_close() and then wakes both sc_sq_ticket_wait and sc_send_wait. Convert the svcrdma producers that called svc_xprt_deferred_close() directly: svc_rdma_cma_handler(), qp_event_handler(), svc_rdma_post_send_err(), svc_rdma_wc_send(), the sendto drop path, the rw completion error paths, and the recvfrom flush and read-list error paths. Wake both waitqueues from svc_rdma_detach() as well. The synchronous svc_xprt_close() path (backchannel ENOTCONN, device removal via svc_rdma_xprt_done) reaches detach without flowing through svc_xprt_deferred_close() and therefore does not invoke the new helper. Fixes: ccc89b9d1ed2 ("svcrdma: Add fair queuing for Send Queue access") Cc: stable@vger.kernel.org Assisted-by: kres (claude-opus-4-7) Signed-off-by: Chris Mason <clm@meta.com> [ cel: add svc_rdma_xprt_deferred_close() to complete the fix ] Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09sunrpc: wait for in-flight TLS handshake callback when cancel loses raceChuck Lever
When wait_for_completion_interruptible_timeout() in svc_tcp_handshake() returns 0 (timeout) or -ERESTARTSYS (signal) and tls_handshake_cancel() then returns false, handshake_complete() has won the cancellation race: it has set HANDSHAKE_F_REQ_COMPLETED and is about to invoke svc_tcp_handshake_done(), but the callback's side effects on xpt_flags and on svsk->sk_handshake_done have not yet committed. The current code reads xpt_flags immediately to decide whether the session succeeded. Two races result. If the callback has executed set_bit(XPT_TLS_SESSION) but not yet clear_bit(XPT_HANDSHAKE), svc_tcp_handshake() sees a session, enqueues the transport, and returns. svc_xprt_received() then clears XPT_BUSY, a worker thread picks the transport up, the dispatcher in svc_handle_xprt() observes XPT_HANDSHAKE still set, and xpo_handshake is invoked a second time. That svc_tcp_handshake() calls init_completion(&svsk->sk_handshake_done) while the original callback concurrently calls complete_all() on it, corrupting the embedded swait_queue. If the callback has set HANDSHAKE_F_REQ_COMPLETED but not yet entered svc_tcp_handshake_done(), svc_tcp_handshake() reads XPT_TLS_SESSION as clear and tears the connection down even though the handshake is about to succeed. Wait for the callback to commit before inspecting xpt_flags. The completion is guaranteed to fire because handshake_complete() invokes svc_tcp_handshake_done() unconditionally once it has set HANDSHAKE_F_REQ_COMPLETED. Fixes: b3cbf98e2fdf ("SUNRPC: Support TLS handshake in the server-side TCP socket code") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09sunrpc: pin svc_xprt across the asynchronous TLS handshake callbackChris Mason
svc_tcp_handshake() stores the raw svc_xprt pointer in tls_handshake_args.ta_data and submits the request through tls_server_hello_x509(). The handshake core takes only sock_hold(req->hr_sk); nothing references the embedding struct svc_sock that svc_tcp_handshake_done() reaches via container_of(). Two close races leave the in-flight callback writing through a freed svc_sock. svc_sock_free() calls tls_handshake_cancel() and discards its return value: a false return means handshake_complete() has already set HANDSHAKE_F_REQ_COMPLETED but hp_done() may not have finished, yet svc_sock_free() proceeds to kfree(svsk). The cancel-loser fall-through inside svc_tcp_handshake() itself produces the same window: when wait_for_completion_interruptible_timeout() returns <= 0 (timeout or signal) and tls_handshake_cancel() returns false, the function does not drain, returns, and svc_handle_xprt() calls svc_xprt_received(), which clears XPT_BUSY and can drop the last reference. A concurrent close then runs svc_sock_free() while svc_tcp_handshake_done() is still updating xpt_flags and walking svsk->sk_handshake_done. The corruption surfaces as set_bit/clear_bit RMW into the freed xpt_flags slab slot and as complete_all() walking and writing the freed wait_queue_head_t list embedded in sk_handshake_done -- a slab-corruption primitive, not a benign read. The path is reachable on any TLS-enabled NFS server whenever a connection close overlaps the tlshd downcall delivery window; the interruptible wait means signal delivery suffices, not just SVC_HANDSHAKE_TO expiry. Take svc_xprt_get(xprt) immediately before tls_server_hello_x509() so the in-flight callback owns its own reference. Release it on the two edges where the callback is guaranteed not to fire -- submission failure from tls_server_hello_x509() and a successful tls_handshake_cancel() -- and at the tail of svc_tcp_handshake_done() after complete_all(). Fixes: b3cbf98e2fdf ("SUNRPC: Support TLS handshake in the server-side TCP socket code") Cc: stable@vger.kernel.org Signed-off-by: Chris Mason <clm@meta.com> Assisted-by: kres (claude-opus-4-7) [cel: rewrote commit message to describe the actual change] Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09sunrpc: harden rq_procinfo lifecycle to prevent double-freeLuxiao Xu
The svc_release_rqst() function executes the callback inside rqstp->rq_procinfo->pc_release. However, if a worker thread begins processing a new request and encounters an early error path (e.g., unsupported protocol, short frame, or bad auth) before a valid rq_procinfo is installed, a stale release hook can be re-triggered against reused state from the previous RPC, resulting in a double-free or use-after-free vulnerability. Harden the lifecycle of rq_procinfo by: 1. Ensuring svc_release_rqst() always clears rq_procinfo after the optional pc_release() call, regardless of whether the hook exists. 2. Explicitly clearing rq_procinfo at request entry in svc_process() before any early decode or drop paths. 3. Ensuring svc_process_bc() does the same at backchannel entry. This guarantees that error flows will not encounter a non-NULL stale rq_procinfo pointer when there is nothing to release. Fixes: d9adbb6e10bf ("sunrpc: delay pc_release callback until after the reply is sent") Cc: stable@vger.kernel.org Reported-by: Yuan Tan <yuantan098@gmail.com> Reported-by: Yifan Wu <yifanwucs@gmail.com> Reported-by: Juefei Pu <tomapufckgml@gmail.com> Reported-by: Xin Liu <bird@lzu.edu.cn> Suggested-by: Chuck Lever <cel@kernel.org> Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Luxiao Xu <rakukuip@gmail.com> Signed-off-by: Ren Wei <n05ec@lzu.edu.cn> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09SUNRPC: Return an error from xdr_buf_to_bvec() on overflowChuck Lever
xdr_buf_to_bvec() returns a slot count even when the caller's bvec budget is exhausted partway through the xdr_buf. Callers feed that count into iov_iter_bvec() and continue as if the conversion had succeeded, silently sending or writing fewer bytes than the data length declares. For an NFS WRITE the server reports the truncated transfer to the client as full success. The overflow represents an internal invariant violation: a higher layer reserved a bvec budget too small for the xdr_buf it then asked the encoder to convert. That is a server-side fault, not a media I/O failure and not a malformed client argument. Change xdr_buf_to_bvec() to return a signed int and have the overflow label return -ESERVERFAULT. Update the three callers to detect the negative return and fail the request: nfsd_vfs_write() folds the error into host_err, which nfserrno() translates to nfserr_serverfault for the WRITE reply; svc_udp_sendto() and svc_tcp_sendmsg() propagate the error out of the send path. Reported-by: Chris Mason <clm@meta.com> Fixes: 2eb2b9358181 ("SUNRPC: Convert svc_tcp_sendmsg to use bio_vecs directly") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09SUNRPC: Bound-check xdr_buf_to_bvec() stores before writingChuck Lever
xdr_buf_to_bvec() writes a bio_vec into the caller's array before testing whether that slot is in range, and the head branch performs the store with no check at all. When the caller's budget is exactly used up, the next store lands one element past the end of the array. The overflow label returns count - 1, which masks the surplus store but cannot undo it. rq_bvec, the array passed by nfsd_vfs_write(), is allocated to exactly rq_maxpages entries with no slack. The OOB store can land in adjacent slab memory; the bv_len and bv_offset fields written there are derived from client-supplied RPC payload sizes. Move the in-range check ahead of the store in the head, page-loop, and tail branches. With the check at the top of each sequence, count is incremented only after a successful store, so the overflow label can return count directly. Reported-by: Chris Mason <clm@meta.com> Fixes: 2eb2b9358181 ("SUNRPC: Convert svc_tcp_sendmsg to use bio_vecs directly") Cc: stable@vger.kernel.org Reviewed-by: Jeff Layton <jlayton@kernel.org> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09Revert "svcrdma: Use contiguous pages for RDMA Read sink buffers"Chuck Lever
Jonathan Flynn reports that commit 18755b8c2f24 ("svcrdma: Use contiguous pages for RDMA Read sink buffers") regresses NFS/RDMA WRITE throughput from 73.9 GiB/s to 30.3 GiB/s on a 128-core single-NUMA-node server driving dual 400Gb/s links with 640 nfsd threads. Server CPU utilization rises from 8.5% to 76%, with roughly three quarters of all cycles spent spinning on zone->lock. The sink buffers are allocated as high-order page blocks, split into single pages so each sub-page carries an independent refcount, and later released one page at a time through folio batches. The per-CPU page caches cannot satisfy an allocation stream whose alloc order differs from its free order, so every sink buffer page makes a round trip through the buddy allocator's free lists, serialized on the zone lock of the single NUMA node. The rq_pages entries that the split pages displace, bulk-allocated moments earlier by svc_alloc_arg(), are freed without ever being used, doubling the allocator traffic. The regression cannot be addressed trivially. Revert the commit now; a reworked approach can return in an upcoming merge window. Reported-by: Jonathan Flynn <jonathan.flynn@hammerspace.com> Reported-by: Mike Snitzer <snitzer@kernel.org> Closes: https://lore.kernel.org/linux-nfs/aiHlPmeZq3WgMwoJ@kernel.org/ Closes: https://lore.kernel.org/linux-nfs/3cb119b4b2a8aada30c0c60286778a54@mail.gmail.com/ Fixes: 18755b8c2f24 ("svcrdma: Use contiguous pages for RDMA Read sink buffers") Cc: stable@vger.kernel.org Tested-by: Jonathan Flynn <jonathan.flynn@hammerspace.com> Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
2026-06-09svcrdma: Defer send context release to xpo_release_ctxtChuck Lever
Send completion currently queues a work item to an unbound workqueue for each completed send context. Under load, the Send Completion handlers contend for the shared workqueue pool lock. Replace the workqueue with a per-transport lock-free list (llist). The Send completion handler appends the send_ctxt to sc_send_release_list and does no further teardown. The nfsd thread drains the list in xpo_release_ctxt between RPCs, performing DMA unmapping, chunk I/O resource release, and page release in a batch. This eliminates both the workqueue pool lock and the DMA unmap cost from the Send completion path. DMA unmapping can be expensive when an IOMMU is present in strict mode, as each unmap triggers a synchronous hardware IOTLB invalidation. Moving it to the nfsd thread, where that latency is harmless, avoids penalizing completion handler throughput. The nfsd threads absorb the release cost at a point where the client is no longer waiting on a reply, and natural batching amortizes the overhead when completions arrive faster than RPCs complete. A self-enqueue backstops drain on a quiescing transport. When svc_rdma_send_ctxt_put() observes that its llist_add() transitions sc_send_release_list from empty to non-empty, it sets XPT_DATA and calls svc_xprt_enqueue() so that svc_xprt_ready() schedules an nfsd thread. The thread enters svc_rdma_recvfrom(), finds no pending receive, clears XPT_DATA, and returns 0; svc_xprt_release() then runs xpo_release_ctxt and drains the list. Under steady load the foreground drain keeps the list non-empty between adds and no enqueue fires; only the trailing edge of a burst pays for a wakeup. Without this path, a Send completion arriving after the last xpo_release_ctxt on an idle connection would leave the send_ctxt's DMA mappings and reply pages pinned until the next RPC, send-context exhaustion, or transport close. Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
generated by cgit v1.2.3 (git 2.39.1) at 2026-07-13 06:13:21 +0000