NFSD: Bound on-demand DRC slot growth by the thread ceiling

When a client uses its highest session slot, nfsd4_sequence() grows
the session's slot table by 20%, up to NFSD_MAX_SLOTS_PER_SESSION, on
the theory that a client at its ceiling can put more requests in
flight. The heuristic keys only on the client's appetite, so its
incentive runs backwards: the client that keeps every slot busy --
already the largest consumer of the thread pool -- is the one the
server rewards with still more slots. A single session's table can
climb toward 2048 slots even on a server with far fewer threads to
run them.

A slot stays occupied for a full round trip -- request out, server
processing, reply back -- but ties up an nfsd thread only during the
processing. When the round trip is short, one slot per thread keeps the
pool busy and further slots add only backlog. Across a high-RTT link
more slots are in flight than the pool serves at any instant, so there
extra slots do raise throughput by masking link latency -- but that
is the client's call to make by sizing its session at CREATE_SESSION,
not a reason for the server to grow every busy session toward 2048. Cap
on-demand growth at the thread ceiling, the point past which added
slots stop buying concurrency on a short round trip, so a table stops
climbing once it can keep every thread busy.

Apply the cap per session rather than across the namespace. A session
cannot use another session's slots, so one client's table size has no
bearing on what a second client may grow to. A shared per-namespace
budget would also misbehave at the floor: every active session holds
one slot that cannot be reclaimed, so once the session count reaches
the thread ceiling those floors alone exhaust the budget, pinning the
one busy client small while most of the pool sits idle. NFSD sizes
its pool dynamically, so compare against svc_serv_maxthreads(), the
configured maximum, rather than the running thread count, which
tracks recent load and would deny a client resuming from idle the
slots it needs to ramp up.

This removes a perverse incentive without becoming slot admission
control. A client still sizes its sessions directly at CREATE_SESSION,
bounded by NFSD_MAX_SLOTS_PER_SESSION, and a client determined to
monopolize threads can do so through that path regardless of this
change. Enforcing per-client fairness against thread starvation
belongs in the dispatch layer, not in slot accounting.

Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Benjamin Coddington <bcodding@hammerspace.com>
Link: https://patch.msgid.link/20260610-nfsd-slot-growth-clamp-v1-5-7b966700df0b@kernel.org
Signed-off-by: Chuck Lever <cel@kernel.org>

1 files changed, 21 insertions, 9 deletions

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index b4c9ebec6c96..e59aec57e9e8 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -4665,15 +4665,26 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 	 * gently try to allocate another 20%.  This allows
 	 * fairly quick growth without grossly over-shooting what
 	 * the client might use.
+	 *
+	 * Bound that growth by the service's thread ceiling:
+	 * slots beyond the nfsd thread count cannot raise this
+	 * client's throughput, only deepen its backlog.  Cap each
+	 * session independently, since a session cannot use
+	 * another's slots; a shared budget would let idle sessions
+	 * pin an active client small.  Compare against the
+	 * configured maximum, not the running thread count, so a
+	 * client resuming from idle can grow back before the pool
+	 * scales up.
 	 */
 	if (seq->slotid == session->se_fchannel.maxreqs - 1 &&
-	    session->se_target_maxslots >= session->se_fchannel.maxreqs &&
-	    session->se_fchannel.maxreqs < NFSD_MAX_SLOTS_PER_SESSION) {
+	    session->se_target_maxslots >= session->se_fchannel.maxreqs) {
 		int s = session->se_fchannel.maxreqs;
-		int cnt = DIV_ROUND_UP(s, 5);
+		int ceiling = min_t(int, NFSD_MAX_SLOTS_PER_SESSION,
+				    svc_serv_maxthreads(rqstp->rq_server));
+		int cnt = min(DIV_ROUND_UP(s, 5), ceiling - s);
 		void *prev_slot;
 
-		do {
+		while (cnt-- > 0) {
 			/*
 			 * GFP_NOWAIT both allows allocation under a
 			 * spinlock, and only succeeds if there is
@@ -4681,13 +4692,14 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 			 */
 			slot = nfsd4_alloc_slot(&session->se_fchannel, s,
 						GFP_NOWAIT);
+			if (!slot)
+				break;
 			prev_slot = xa_load(&session->se_slots, s);
-			if (xa_is_value(prev_slot) && slot) {
+			if (xa_is_value(prev_slot)) {
 				slot->sl_seqid = xa_to_value(prev_slot);
 				slot->sl_flags |= NFSD4_SLOT_REUSED;
 			}
-			if (slot &&
-			    !xa_is_err(xa_store(&session->se_slots, s, slot,
+			if (!xa_is_err(xa_store(&session->se_slots, s, slot,
 						GFP_NOWAIT))) {
 				s += 1;
 				session->se_fchannel.maxreqs = s;
@@ -4696,9 +4708,9 @@ nfsd4_sequence(struct svc_rqst *rqstp, struct nfsd4_compound_state *cstate,
 				session->se_target_maxslots = s;
 			} else {
 				kfree(slot);
-				slot = NULL;
+				break;
 			}
-		} while (slot && --cnt > 0);
+		}
 	}
 
 out: