| Age | Commit message (Collapse) | Author |
|---|
|
scx_local_or_reject_dsq() authorizes a local-DSQ insert against the caps
of the scheduler doing the insert. On the consume/dispatch paths that is
the scheduler running balance_one(), passed down through
scx_consume_dispatch_q() and move_local_task_to_local_dsq(), so the check
is correct.
The remote-move path loses it. move_remote_task_to_local_dsq()
re-activates @p on the destination rq through enqueue_task_scx(), which
reconstructs the scheduler from the task, i.e. @p's owner. When an
ancestor places a descendant's task - e.g. draining a bypassed
sub-scheduler - the owner is a sub-scheduler of the placer, so
authorizing against the owner checks a narrower cap set and can
spuriously reject a task the placer is entitled to run.
Carry the placing scheduler across the activate_task() boundary the same
way enq_flags already are, via a per-rq field set only for the duration
of the re-activation, and have scx_local_or_reject_dsq() authorize
against it. The placer's caps are a superset of the owner's, so this
admits what the placer may run and keeps rejecting what it may not.
v2: Document @sch in move_remote_task_to_local_dsq()'s kerneldoc. (Andrea)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A kick forces a scheduling event on the target cpu, and a preemption also
evicts the running task. Gate both on caps. Any kick requires baseline
access on the cid, and preempting a task the sub-sched does not own -
whether by a SCX_ENQ_PREEMPT insert or a SCX_KICK_PREEMPT kick - requires
the new SCX_CAP_PREEMPT. Gating either alone would leave a hole - the
weakest cap authorizing preempting kicks, or plain kicks disturbing cpus the
kicker has no access to.
Preempting the sched's own subtree is always allowed, and the cap extends
the right to any task on the cid. PREEMPT implies ENQ, and so ENQ_IMMED.
A preempting insert tests the running task under the target rq lock and is
rejected and reenqueued unless the victim is in the inserter's subtree or it
holds PREEMPT. A migration-disabled task is admitted regardless, but with
SCX_ENQ_PREEMPT stripped.
Kicks are enforced on the delivery path, where the effective caps can be
read coherently under the target rq's lock. A kick from a sub-sched lacking
SCX_CAP_BASE on the cid is dropped, and a SCX_KICK_PREEMPT kick without
PREEMPT for a task outside the kicker's subtree degrades to a plain
reschedule.
Unlike the enqueue caps, PREEMPT is checked only at the instant of the
insert or kick, never as a standing property of a queued task.
v2: Clear SCX_ENQ_PREEMPT on the offline and migration_pending force-admits.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Add SCX_CAP_ENQ, which gates inserting tasks onto a cid's local DSQ. Unlike
IMMED enqueue, plain enqueues can pile up, so ENQ is the stronger cap and
implies ENQ_IMMED. Losing ENQ also triggers the reenq scan. The scan tests
each queued task and the running task against the cap each needs via
scx_caps_for_task(), so an ENQ-only loss reenqueues plain tasks, evicting a
running one, while IMMED tasks, which need only ENQ_IMMED, stay put.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A task's slice grants it cpu occupancy - how long it holds its cpu. In a
sub-scheduler hierarchy cpu access is delegated through revocable
capabilities, so a task's occupancy must follow them. Only its own scheduler
sets its slice, and extending the slice is allowed only while that scheduler
holds baseline cpu access (SCX_CAP_BASE) on the cpu. Otherwise a scheduler
could keep occupying a cpu it has been denied simply by handing out long
slices.
The cap check reads effective caps, which are coherent only under the task's
rq lock, and the kernel decrements the slice under that lock as the task
runs, so a running task's slice can be changed only there while a queued
task's can be set directly. Make scx_bpf_task_set_slice() apply the slice
under the rq lock. Synchronously when the caller already holds it, otherwise
by stashing it in the new p->scx.slice_oob, tagged with the scheduler's id
so a request that outlived a reassignment is dropped. Whether the caller
holds @p's current rq lock is tested with p->scx.runnable_cpu.
Revocation is enforced through the same grant. When a cpu's effective caps
lose SCX_CAP_BASE, the cap-revoke reenq scan also checks the running task
and zeroes its slice to evict it. The scan runs as a balance callback after
the pick, so this catches both the task that was running when the revoke
landed and a capless task the pick just promoted off the local DSQ. The
paths that keep a task on its cpu - holding on to the last runnable task in
balance, the ENQ_LAST reinsertion and the slice refill on pick - skip tasks
lacking baseline access. A migration-disabled task is exempt, mirroring its
capless admission on insert.
v4: Test rq ownership with p->scx.runnable_cpu, closing a remote-wakeup TOCTOU. (sashiko AI)
v3: Keep a pending out-of-band slice request across refill and preserve. (sashiko AI)
v2: Only write slice directly when @p is queued on the held rq. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Add p->scx.runnable_cpu, the cpu @p is runnable on, or -1 when it is not.
It is stamped as @p joins the runnable_list (set_task_runnable()) and
cleared as it leaves (clr_task_runnable()), both under the rq lock.
task_cpu() can't answer "is @p on this rq" reliably: a remote wakeup changes
it under @p's pi_lock alone, without the source rq lock, so it can read as
the locked rq while @p is really elsewhere. runnable_cpu changes only under
the rq lock, so a caller holding an rq lock can compare against it to know
whether that is @p's current rq.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A later change makes set_task_slice() also drop a pending out-of-band slice
request, so the BPF-triggered writes to p->scx.slice need to funnel through
one place. Introduce set_task_slice() and route those writes through it.
update_curr_scx() decrements curr->scx.slice directly for accounting and is
left alone. No functional change - the helper only assigns p->scx.slice.
v2: Reword the set_task_slice comment to "BPF-triggered writes". (Andrea)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Neither a scx_sched pointer nor its cgroup id uniquely identifies a
scheduler instance. A freed sched's memory can be reallocated, and a cgroup
can detach one sched and attach another. Add a monotonic, never-reused u64
id. A later patch compares it to drop a slice request that outlived a change
of a task's owning scheduler.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Replace the __SCX_CAP_DUMMY placeholder with SCX_CAP_ENQ_IMMED, which gates
inserting IMMED tasks onto a cid's local DSQ. An IMMED enqueue is guaranteed
to either get its task running on the cpu at once or hand it back to the
scheduler, so IMMED work can never pile up on the cpu's queue and a cpu can
be shared across sub-scheds through IMMED access without any of them
swamping it.
That makes ENQ_IMMED the natural baseline, the minimal cap to make any use
of a cpu. SCX_CAP_BASE aliases it so gates on basic cpu access can state the
intention instead of naming ENQ_IMMED.
Enforcement covers inserts and queued tasks. An insert without the cap is
diverted to the reject DSQ, and queued tasks are reenqueued when the cap is
lost. scx_bpf_sub_dispatch() skips a child that lacks the cap on the cpu, as
its inserts would only be rejected. Vacating the running task on cap loss
lands in a later patch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A SAVE/RESTORE requeue re-inserts a running task in place and is immediately
followed by set_next_task_scx(). It is not a real scheduling event: the task
is already admitted to its cid and must return to the local DSQ
unconditionally.
scx_caps_for_enq() maps an enqueue to the cap its local-DSQ insert requires.
Add SCX_ENQ_IGNORE_CAPS, set it on the RESTORE-in-place branch of
enqueue_task_scx(), and have scx_caps_for_enq() require no caps for it, so
the cid admission gate never diverts an in-place restore to the reject DSQ.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
When a sub-scheduler dispatches a task to a CPU it lacks the required
capability on, the task must be rejected rather than allowed to run.
Add the machinery for that. Each rq gets a reject DSQ, a kernel-internal
holding queue that is never run and that the BPF scheduler cannot reach. An
insert that must be refused is diverted there instead of the local DSQ, and
a deferred requeue then hands the parked tasks back to the BPF scheduler to
re-decide. A cap revoke extends this to already-queued tasks. When the
revoke reaches the cpu's effective caps, the cpu scans its local DSQ and
reenqueues the tasks that no longer qualify.
A migration-disabled task must run on its cpu, so a capless one is admitted
anyway and counted in the new SCX_EV_SUB_FORCED_ADMIT event.
This is preparation for the actual sub-sched cap enforcement. The divert is
wired but inert here.
v2: Admit offline-rq and migration_pending inserts to local, not reject. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The local DSQ is synchronized by the containing rq lock rather than its own
dsq->lock. A later patch adds a second such DSQ. In preparation, factor the
"rq owns the lock" test into dsq_is_rq_owned() and rename
local_dsq_post_enq() to rq_owned_post_enq(), taking @rq explicitly.
No behavior change.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A sub-scheduler that gains or loses effective caps on a cpu may want to act
on it right away - e.g. place or preempt on a newly usable cpu. The existing
ops.sub_caps_updated() doesn't fit as it is delivered asynchronously to
scheduling operations and can arrive before the per-cpu effective caps go
live.
Add ops.sub_ecaps_updated(cid, before, after), a cid-form callback fired
from scx_process_sync_ecaps() when a sub-sched's effective caps on a cid
change. It runs in dispatch context so the sched can insert, kick or preempt
on the cid directly. @before is the caps as of the last delivery.
Cpu hotplug rides the same machinery. Going down zeroes each sched's ecaps
on the cpu's cid, with queued syncs discarded at consumption while the cpu
is inactive. Coming back up queues a sync for every sched. reported_ecaps is
kept across the down/up cycle, so the resync fires the callback only if
ownership actually changed while the cpu was down.
v2: Compute cid below the active-cpu guard; discard queued syncs on !cpu_active(). (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Checking a sched's caps on a cid would need to test several cap bits against
caps[] to account for implied caps. Also, caps[] modifications aren't
synchronized against scheduling operations on each cpu, which can lead to
awkward race conditions.
Collect them per cpu instead. caps[] under pshard->lock stays the target
configuration. scx_sched_pcpu->ecaps is added, the transposed effective
copy: the set of cap bits the sched holds on that cpu which can be accessed
with a single read. It is stable under the rq lock. It can also be read
locklessly with READ_ONCE().
Grant and revoke only mutate caps[]. They queue a sync request on the target
cpu's rq->scx.ecaps_to_sync and kick it, and the cpu recomputes the queued
scheds' ecaps from caps[] in balance_one() under its own rq lock. A dying
sched runs the sync directly to retire its queued request before freeing. As
held references can defer the freeing past the enclosing root scheduler's
lifetime, root enable discards leftover sync requests before going live.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Wire up ops_cid.sub_caps_updated() to notify sub-scheds of cap changes.
Three constraints shape the design:
1. Static memory. Deliveries use a fixed-size buffer, both for runtime
efficiency and so notifications can't be lost under memory pressure.
2. High-frequency updates. Grant/revoke can mutate caps in bursts, and the
notifier path must absorb that without amplifying it.
3. Recursive grant/revoke from the callback. A child receiving a
notification can call grant/revoke on its own children, which can
cascade recursively down its subtree.
(1) and (2) lead to coalescing into a fixed payload. Each delivery carries a
single (cmask, caps) pair covering every change since the previous one.
Direction (set vs cleared) isn't encoded as it doesn't fit in the fixed-size
summary. The callback queries scx_bpf_sub_caps() for current state. Only one
delivery is in flight per shard. Further changes fold into the same buffer
and ship as the next callback, so a shard's callbacks fire in order.
(3) leads to deferred delivery. Events accumulate during grant/revoke and
are delivered after the shard lock is released.
v2:
- Request a private stack for ops.sub_caps_updated(). (sashiko AI)
- Build cmask_arena_out via scx_cmask_ref, not by re-reading its header.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Caps are per-cid permissions parents delegate to direct children via
scx_bpf_sub_grant() / scx_bpf_sub_revoke(). A child's cap set is always a
subset of its parent's. Sub-scheds check their caps locally, and cross-sched
communication is needed only when the delegation set itself changes.
Caps will be used to implement sub-sched scheduling on the enqueue path.
Picking a cid for a task at a leaf depends on which cids the leaf is allowed
to use, and resolving that programmatically on every enqueue would mean a
cross-sched round-trip call chain, possibly retrying if the request can't be
granted as-is. The dispatch path is different - it runs as top-down
recursion via scx_bpf_sub_dispatch().
Locking is per shard. cid space is split into shards, and each sub-sched has
its own pshard->lock for each shard. Operations are broken up on shard
boundaries. Different shards never contend. Shards are expected to be
topology-aligned and likely to serve as the locality unit when cids are
allocated to schedulers, so per-shard lock granularity scales naturally with
the allocation pattern.
This patch adds the framework with a single dummy cap. Real caps land in
later patches.
The enable path is reordered for pshards. scx_arena_pool_init() moves ahead
of scx_link_sched() so the pshards are allocated before the sched becomes
reachable - scx_alloc_pshards() skips allocation when the arena pool isn't
initialized.
- scx_bpf_sub_grant(): Per-cid all-or-nothing grant to direct child.
- scx_bpf_sub_revoke(): Clear caps on @cmask across @child and its subtree.
- scx_bpf_sub_caps(): Lockless snapshot of caps on a cid range.
/sys/kernel/sched_ext/SCHED/caps shows the caps each scheduler currently
holds.
v4: Move the pshard[] full build/publish and the err_disable scx_error() recording to earlier patches. (sashiko AI)
v3: Build pshard[] fully before publishing it, read it with READ_ONCE. (sashiko AI)
v2: Validate ops before scx_link_sched() publishes the sub. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A scheduler's BPF programs can outlive it. A timer it armed or a tracing
program it loaded can fire after ops.exit() has run, before the programs are
unloaded, and scx_prog_sched() still resolves the program to its scheduler
through ops->priv. Harmless while kfuncs touch only lifetime-stable state,
but a hazard once a kfunc reads global state a newly loaded scheduler can
change underneath it.
Add scx_sched->dead, set right after ops.exit() and drained with
synchronize_rcu(). It follows exit() rather than preceding it so exit()'s own
kfunc calls still resolve to @sch. scx_prog_sched() returns NULL for a dead
scheduler, so every kfunc's existing !sch bail rejects it at one choke
point.
v2: Check dead in the CONFIG_EXT_SUB_SCHED=n scx_prog_sched() too. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Factor the sibling/ancestor portion of scx_next_descendant_pre() out as
scx_skip_subtree_pre(), a pre-order walk primitive that skips @pos's
subtree, and call it from scx_next_descendant_pre(). Same locking rules as
the existing primitive.
Used in a follow-up to fast-skip subtrees that have nothing to do during a
descendant walk.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Future kfuncs need to walk descendants without scx_sched_lock. Make the
walker RCU-safe so that they can. A sub-sched's fields are initialized
before it is linked, so a walk that observes a linked node also observes its
setup. In-place changes after linking carry their own ordering.
Switch the children/sibling list ops to RCU and expand the descendant walker
to accept rcu_read_lock as a valid read-side context. Walkers that mutate
keep scx_sched_lock.
A sub-sched can be linked while an ancestor is bypassing, after the bypass
walk that propagates the depth has passed its parent. Bypass state is a
per-cpu flag plus a depth count and can't be established atomically at link
time, so refuse to link under a bypassing ancestor. Take scx_bypass_lock
across linking to check the parent's bypass state coherently.
v3: Reject linking under a bypassing ancestor instead of inheriting bypass_depth. (sashiko AI)
v2: Inherit bypass_depth before publishing @sch on the RCU sibling list.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_call_op_set_cpumask() builds a per-cpu cmask in the set_cmask scratch,
which lives in BPF-writable arena. A scheduler can corrupt the scratch's
inline header (base, nr_cids, alloc_words) from another cpu, so sizing and
indexing the write from it risks an out-of-bounds write.
Drive the build from kernel-known geometry instead.
scx_cmask_ref_init_kern() imposes base and nr_cids rather than reading them,
and scx_cmask_ref_from_cpumask() fills the scratch from the ref. Neither
reads the header back.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
kfuncs taking struct scx_cmask * from BPF arena memory have two problems.
The pointer can be any value the BPF prog hands in, and the header (@base,
@nr_cids, @alloc_words) can be mutated by the prog concurrently with kernel
access.
Add scx_cmask_ref, a validated handle. _init() normalizes the input pointer
into the arena's kern_vm range via scx_arena_to_kaddr() and snapshots the
header, rejecting a range outside the machine or a nr_cids whose words
exceed the declared @alloc_words. Downstream sizing uses the snapshot, not
the live header. _shard() reads slices while _or() and _copy() write back,
all bounded by the snapshot. No callers yet.
struct scx_cmask's bits[] carried __counted_by(alloc_words), so
UBSAN_BOUNDS and FORTIFY_SOURCE bound accesses to the array. That bound is
read from @alloc_words at the access. For an arena cmask @alloc_words is
BPF-writable. A prog that sets it larger than the real allocation makes the
check pass on a genuine overrun, so the annotation catches nothing, and it
only runs under those debug configs. Drop it - _init() validates
@alloc_words explicitly, and kernel-owned cmasks set it themselves.
v2: Validate @alloc_words in _init(), drop __counted_by. (Andrea, sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Add struct scx_pshard and sch->pshard[] indexed by shard_idx, each entry
allocated on its shard's NUMA node from scx_shard_node[si]. The struct
starts empty (one dummy field). Follow-up patches will grow it as
shard-local state lands. Only cid-type schedulers with an arena pool get
pshards.
Allocation happens after ops.init_cids() returns so any
scx_bpf_cid_override() it issues has finalized scx_nr_cid_shards and
scx_shard_node[]. sch->nr_pshards records the array size for the async RCU
free path, which may run after a later scheduler's scx_cid_init() has
rewritten the global.
v3: Build and publish pshard[] fully-formed here rather than a later patch.
v2: Free the partially-allocated pshard array on alloc failure. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Split kobject_init_and_add() in scx_alloc_and_add_sched(): only
kobject_init() runs there. A new scx_sched_sysfs_add() helper does
kobject_add() (and creates sub_kset when the scheduler implements
ops.sub_attach), called by both enable workfns once @sch is linked and its
sysfs-visible state is initialized. Prep so a future caps attribute can rely
on @sch being fully built by the time it's sysfs-visible. Add early enough
that a stall later in enable still leaves sysfs inspectable.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
An overridden cid mapping invalidates the auto-generated shard layout, so
the override call has to provide both. Extend scx_bpf_cid_override() with a
shard_start[] array that lists the first cid of each shard (starting at 0,
strictly increasing, last shard implicitly extends to num_possible_cpus()).
A scheduler that wants only custom shards with the auto-generated cid
mapping can read the current mapping and pass it back unchanged.
Overridden shards can span NUMA nodes, so scx_shard_node[] is rebuilt by
majority count: each shard is assigned to the node that owns the most cpus
in it.
v2: Snapshot the caller's cpu_to_cid/shard_start arrays before validating. (sashiko AI)
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Sub-sched operations need a scalable locking / work domain smaller than the
whole cid space. Carve the cid space into topology-respecting shards: each
shard is a contiguous cid range that stays within one LLC, and LLCs larger
than the per-shard cap (default 24 cids, configurable via
ops.cid_shard_size) split into enough shards to fit. A hard cap of
SCX_CID_SHARD_MAX_CPUS prevents pathological sizes under custom
configurations.
No-topo cids pack into their own shards so every cid has a shard assignment.
Also build scx_cid_shard_ranges[] for O(1) shard-to-cid-range lookup and
scx_shard_node[] so callers can size or place work by NUMA without walking
cids. Auto-built shards inherit their LLC's node. No-topo shards carry
NUMA_NO_NODE.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A cid-form scheduler that calls scx_bpf_cid_override() to install a custom
cid layout can only do so from ops.init(). Enable-path setup that depends on
the cid layout thus has to run after ops.init(), and ops.init() itself can't
use anything derived from the final layout, which turned out to be too
restrictive.
Add an ops.init_cids() callback dedicated to finalizing the cid layout. It
runs before the rest of the enable-path setup, so the final layout is in
effect for everything that follows including ops.init(), which now runs
after the arena pool and cmask scratch allocations.
scx_bpf_cid_override() is restricted to ops.init_cids() at load time. It
sits in a kfunc set gated by SCX_KF_ALLOW_INIT_CIDS, a flag set only on the
init_cids op, so the verifier rejects a call from any other context. The
runtime root-only check is dropped as ops.init_cids() only runs during root
enable.
The qmap demo moves its override into a dedicated qmap_init_cids() and,
while at it, introduces an enum for the cid override modes instead of
hard-coded integers.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The kick machinery kept its targets in rq->scx shared by every sched on the
cpu. A preempt kick carried no record of which scheduler requested it.
A later patch needs preempt kicks scoped to the requesting scheduler so a
sub-scheduler can preempt only tasks in its own subtree. Move the kick masks
into the per-sched per-cpu scx_sched_pcpu and have scx_kick_cpu() link the
sched onto a per-cpu list (rq->scx.sched_pcpus_to_kick). The cpu's single
kick irq_work walks that list and kicks each sched's targets on its behalf,
so a kick stays attributed to its scheduler.
The SCX_KICK_WAIT sync set (cpus_to_sync, the kick_sync snapshot and the
balance-callback trigger) stays in rq->scx: the waiter is the cpu, not the
scheduler, and its only writers, the kick irq_work and the wait balance
callback, are cpu-local.
On disable, free_kick_syncs() flushes each cpu's pending kick irq_work
before clearing @ksyncs, so a late kick unlinks its to_kick_node instead of
early-returning on a NULL @ksyncs and leaving the node linked at free.
v3: Flush the kick irq_work in free_kick_syncs() before clearing @ksyncs. (sashiko AI)
v2: Warn once per sched on scx_bpf_kick_cpu() from NMI.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A per-task op must be dispatched on the scheduler that owns the task.
SCX_CALL_OP_TASK() and its _RET twin take @sch explicitly, and a caller that
passes the wrong scheduler would silently run the op on it. Add a
WARN_ON_ONCE() that @sch matches the task's owner so such a mismatch is
caught rather than hidden.
Two sites legitimately target a scheduler other than the task's owner:
cgroup_move() runs on the root sched, and scx_sub_init_cancel_task() fires
exit_task() on a task not yet associated with @sch. Both switch to the inner
__SCX_CALL_OP_TASK(), which dispatches on the explicit @sch without the
assert.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Improve readability of the sg_dma_len assignment in the P2PDMA cases by
removing duplicated code, which was a direct result of the d0d08f4bd7f6
("dma-direct: Fix missing sg_dma_len assignment in P2PDMA bus mappings")
fix. No functional change.
Suggested-by: Leon Romanovsky <leon@kernel.org>
Link: https://lore.kernel.org/all/20260604071856.GA245424@unreal/
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Leon Romanovsky <leon@kernel.org>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
Link: https://lore.kernel.org/r/20260713083022.1110993-1-m.szyprowski@samsung.com
Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com>
|
|
panel_bind_callback() has been compiled out with #if 0 since
the commit 630231776da4916e ("Staging: panel: remove support
for smartcards") in v2.6.29 and now has no callers. Remove
the dead function to reduce maintenance burden.
No functional change.
Signed-off-by: Miles Krause <mileskrause5200@gmail.com>
Reviewed-by: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: Willy Tarreau <willy@haproxy.com>
Link: https://patch.msgid.link/20260712-panel-remove-dead-callback-v1-1-1091ce75b8d5@gmail.com
[andy: update commit message as suggested by Geert]
Signed-off-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
|
|
Qualcomm Maili SoC has exactly the same RPMh power domains as Qualcomm
Hawi SoC. Add "qcom,maili-rpmhpd" string as a compatible entry for
"qcom,hawi-rpmhpd".
Signed-off-by: Fenglin Wu <fenglin.wu@oss.qualcomm.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Ulf Hansson <ulfh@kernel.org>
|
|
The core always hands the driver's query_device() callback a zeroed
ib_device_attr. There are only two callers of the op and both clear the
structure before invoking it: setup_device() memsets &device->attrs, and
ib_uverbs_ex_query_device() passes an on-stack structure initialized to {}.
The open-coded memset(props, 0, sizeof(*props)) at the top of the driver
callbacks is therefore redundant. Remove it from all drivers.
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
|
|
When prefetch region is DRM_XE_CONSULT_MEM_ADVISE_PREF_LOC for a BO VMA,
the code used it as an index into region_to_mem_type[], causing an
out-of-bounds access since the value is -1.
Resolve the preferred location for BO VMAs directly: local VRAM on dGFX
(using the BO's tile placement) or system memory on iGPU.
Discovered using AI-assisted static analysis confirmed by Intel Product
Security.
v2:
-Fix null dereference
Reported-by: Martin Hodo <martin.hodo@intel.com>
Fixes: c1bb69a2e8e2 ("drm/xe/svm: Consult madvise preferred location in prefetch")
Cc: Matthew Brost <matthew.brost@intel.com>
Cc: stable@vger.kernel.org
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Link: https://patchwork.freedesktop.org/patch/msgid/20260624174943.2808767-2-himal.prasad.ghimiray@intel.com
Signed-off-by: Himal Prasad Ghimiray <himal.prasad.ghimiray@intel.com>
|
|
Add support for 0xefa4 devices.
Reviewed-by: Michael Margolin <mrgolin@amazon.com>
Signed-off-by: Anas Mousa <anasmous@amazon.com>
Link: https://patch.msgid.link/20260712134413.19226-3-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Update device interface adding one more bit from reserved to enable
>4GB page sizes that can be supported on 0xefa4 devices.
Reviewed-by: Yonatan Nachum <ynachum@amazon.com>
Signed-off-by: Michael Margolin <mrgolin@amazon.com>
Link: https://patch.msgid.link/20260712134413.19226-2-mrgolin@amazon.com
Signed-off-by: Leon Romanovsky <leon@kernel.org>
|
|
Add missing include to fix compile warning:
drivers/tty/serial/8250/8250_hub6.c:44:6: warning: no previous prototype for
'hub6_match_port' [-Wmissing-prototypes]
Fixes: 3d406299d882 ("serial: 8250_hub6: add hub6_match_port()")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202607110715.VGT2dVVz-lkp@intel.com/
Closes: https://lore.kernel.org/oe-kbuild-all/202607111219.QG9uOW8H-lkp@intel.com/
Signed-off-by: Hugo Villeneuve <hvilleneuve@dimonoff.com>
Link: https://patch.msgid.link/20260714012610.576746-1-hugo@hugovil.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Use the "var" keyword when describing data definitions to avoid
kernel-doc warnings:
Warning: sound/usb/usx2y/us144mkii_pcm.h:14 cannot understand function prototype: 'const struct snd_pcm_hardware tascam_pcm_hw;'
Warning: sound/usb/usx2y/us144mkii_pcm.h:21 cannot understand function prototype: 'const struct snd_pcm_ops tascam_playback_ops;'
Warning: sound/usb/usx2y/us144mkii_pcm.h:28 cannot understand function prototype: 'const struct snd_pcm_ops tascam_capture_ops;'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260713180303.526409-5-rdunlap@infradead.org
|
|
Add missing comment for struct member @messages.
Use the struct keyword for a struct's kernel-doc heading.
Add missing comments for nested aggregate structs.
Repair some typos.
Warning: include/uapi/sound/firewire.h:97 struct member 'messages' not described in 'snd_firewire_event_ff400_message'
Warning: ../include/uapi/sound/firewire.h:220 cannot understand function prototype: 'struct snd_firewire_motu_register_dsp_parameter'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260713180303.526409-4-rdunlap@infradead.org
|
|
- add missing function parameter descriptions
- drop some incorrect function parameter descriptions
- add missing function Return value sections
- use the correct function prototype names in the comments
These changes avoid many warnings (examples):
Warning: include/sound/hda_regmap.h:80 function parameter 'codec' not described in 'snd_hdac_regmap_write'
Warning: include/sound/hda_regmap.h:80 function parameter 'verb' not described in 'snd_hdac_regmap_write'
Warning: include/sound/hda_regmap.h:80 Excess function parameter 'reg' description in 'snd_hdac_regmap_write'
Warning: include/sound/hda_regmap.h:80 No description found for return value of 'snd_hdac_regmap_write'
Warning: include/sound/hda_regmap.h:99 function parameter 'codec' not described in 'snd_hdac_regmap_update'
Warning: include/sound/hda_regmap.h:99 expecting prototype for snd_hda_regmap_update(). Prototype was for snd_hdac_regmap_update() instead
Warning: include/sound/hda_regmap.h:116 expecting prototype for snd_hda_regmap_read(). Prototype was for snd_hdac_regmap_read() instead
Warning: include/sound/hda_regmap.h:116 function parameter 'codec' not described in 'snd_hdac_regmap_read'
Warning: include/sound/hda_regmap.h:137 function parameter 'dir' not described in 'snd_hdac_regmap_get_amp'
Warning: include/sound/hda_regmap.h:137 Excess function parameter 'direction' description in 'snd_hdac_regmap_get_amp'
Warning: include/sound/hda_regmap.h:137 No description found for return value of 'snd_hdac_regmap_get_amp'
Warning: include/sound/hda_regmap.h:161 function parameter 'dir' not described in 'snd_hdac_regmap_update_amp'
Warning: include/sound/hda_regmap.h:161 Excess function parameter 'direction' description in 'snd_hdac_regmap_update_amp'
Warning: include/sound/hda_regmap.h:161 No description found for return value of 'snd_hdac_regmap_update_amp'
Warning: include/sound/hda_regmap.h:182 function parameter 'dir' not described in 'snd_hdac_regmap_get_amp_stereo'
Warning: include/sound/hda_regmap.h:182 Excess function parameter 'ch' description in 'snd_hdac_regmap_get_amp_stereo'
Warning: include/sound/hda_regmap.h:182 No description found for return value of 'snd_hdac_regmap_get_amp_stereo'
Warning: include/sound/hda_regmap.h:206 function parameter 'dir' not described in 'snd_hdac_regmap_update_amp_stereo'
Warning: include/sound/hda_regmap.h:206 Excess function parameter 'direction' description in 'snd_hdac_regmap_update_amp_stereo'
Warning: include/sound/hda_regmap.h:206 No description found for return value of 'snd_hdac_regmap_update_amp_stereo'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260713180303.526409-3-rdunlap@infradead.org
|
|
Inform kernel-doc that the comment block is for structs to void
warnings:
Warning: include/sound/ac97/codec.h:46 cannot understand function prototype: 'struct ac97_codec_device'
Warning: include/sound/ac97/codec.h:62 cannot understand function prototype: 'struct ac97_codec_driver'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Takashi Iwai <tiwai@suse.de>
Link: https://patch.msgid.link/20260713180303.526409-2-rdunlap@infradead.org
|
|
btrfs_search_slot() returns a positive value when the search key does
not exactly match an item. This is expected here, since offset 0 is used
to find the first ROOT_BACKREF for the subvolume and the actual key has
the parent root ID as its offset.
Before the compat ioctl refactoring, the native handler still copied the
filled structure to userspace when the search returned 1. After the
lookup was moved to a shared helper, both native and compat callers
treat the positive return value as a failure and skip copy_to_user(),
leaving BTRFS_IOC_GET_SUBVOL_INFO unusable for non-top-level
subvolumes.
Reset ret after successfully validating and reading the ROOT_BACKREF so
the helper reports success and both callers copy the result to
userspace.
Fixes: 538e5bdbc899 ("btrfs: add 32-bit compat ioctl for BTRFS_IOC_GET_SUBVOL_INFO")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Daan De Meyer <daan@amutable.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
__add_reloc_root() allocates a mapping_node before inserting it into
rc->reloc_root_tree. If rb_simple_insert() finds an existing entry, it
returns the existing rb_node and leaves the newly allocated node unlinked.
The error path then returns -EEXIST without freeing the new node. Since
the node was never inserted into reloc_root_tree, the later cleanup in
put_reloc_control() cannot find it either.
Free the newly allocated node before returning -EEXIST.
The callers currently assert that -EEXIST should not happen, so this is a
defensive cleanup for an unexpected duplicate insert path. If the path is
ever reached, the local allocation should still be released.
Fixes: 57a304cfd43b ("btrfs: do not panic in __add_reloc_root")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Guanghui Yang <3497809730@qq.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
The following script (already submitted as generic/798) will report
incorrect dirty page numbers, with 64K page size systems and 4K fs block
size:
# mkfs.btrfs -s 4k -f $dev
# mount $dev $mnt
# xfs_io -f -c "pwrite 0 64K" -c fsync -c "cachestat 0 64K" $mnt/foobar
Cached: 1, Dirty: 1, Writeback: 0, Evicted: 0, Recently Evicted: 0
Note that the dirtied page number is still 1.
[CAUSE]
The cachestat() goes through the XArray of the page cache, but
instead of checking each folio's flag, it uses the
PAGECACHE_TAG_DIRTY tag to report dirty pages.
Since commit 095be159f3eb ("btrfs: unify folio dirty flag clearing"),
btrfs replaced a folio_clear_dirty_for_io() call inside
extent_write_cache_pages() with folio_test_dirty().
This will cause the following call sequence for the folio at file offset
0:
extent_write_cache_pages()
|- folio_test_dirty()
| The folio is still dirty, continue to writeback.
|
|- extent_writepage()
|- extent_writepage_io()
|- submit_one_sector() for range [0, 4K)
| |- btrfs_folio_clear_dirty()
| |- btrfs_folio_set_writeback()
| |- folio_start_writeback()
| It's the first writeback block, we set the writeback
| flag for the folio.
| But the folio is still dirty, PAGECACHE_TAG_DIRTY is
| kept
|
|- submit_one_sector() for range [4K, 8K)
| |- btrfs_folio_clear_dirty()
| |- btrfs_folio_set_writeback()
| The folio already has writeback flag, no need to call
| folio_start_writeback()
|
| ...
|- submit_one_sector() for range [60K, 64K)
|- btrfs_folio_clear_dirty()
|- btrfs_folio_set_writeback()
The folio already has writeback flag, no need to call
folio_start_writeback()
So the PAGECACHE_TAG_DIRTY is never cleared.
Meanwhile for the old code, before that commit, the sequence looks
like:
extent_write_cache_pages()
|- folio_clear_dirty_for_io()
| The folio is still dirty, so continue to writeback.
| But the folio dirty flag is cleared now.
|
|- extent_writepage()
|- extent_writepage_io()
|- submit_one_sector() for range [0, 4K)
| |- btrfs_folio_clear_dirty()
| |- btrfs_folio_set_writeback()
| |- folio_start_writeback()
| |- xas_clear(PAGECACHE_TAG)
|
| It's the first writeback block, we set the writeback
| flag for the folio.
| And the folio is not dirty, PAGECACHE_TAG_DIRTY is
| cleared
|
|- submit_one_sector() for range [4K, 8K)
| |- btrfs_folio_clear_dirty()
| |- btrfs_folio_set_writeback()
| The folio already has writeback flag, no need to call
| folio_start_writeback()
|
| ...
|- submit_one_sector() for range [60K, 64K)
|- btrfs_folio_clear_dirty()
|- btrfs_folio_set_writeback()
The folio already has writeback flag, no need to call
folio_start_writeback()
Unlike the new code, old code will clear PAGECACHE_TAG_DIRTY for the
first writeback block.
There is a deeper problem, dirty and writeback folio flags are updated
at very different timing.
The dirty flag is only cleared when the last sub-folio block has dirty
flag cleared.
But the writeback flag is set when the first block starts writeback, and
later blocks that go through writeback will not call
folio_start_writeback() again.
If we rely on folio_start_writeback() to update the
PAGECACHE_TAG_DIRTY and PAGECACHE_TAG_TOWRITE, it will always be
incorrect in one way or another.
[FIX]
Do not let folio_start_writeback() do any PAGECACHE_TAG_TOWRITE
handling.
Instead, manually clear both PAGECACHE_TAG_TOWRITE and
PAGECACHE_TAG_DIRTY flags when the folio is no longer dirty during
btrfs_subpage_set_writeback().
However this is only a hot-fix, for the long term solution we will
follow iomap, by calling folio_start_writeback() immediately for the
whole folio, and folio_end_writeback() after all writeback finished
for the folio.
Fixes: 095be159f3eb ("btrfs: unify folio dirty flag clearing")
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
When btrfs_drop_extent_map_range() splits an extent map, the new split
maps inherit the original map's flags through a local 'flags' variable.
Commit f86f7a75e2fb ("btrfs: use the flags of an extent map to identify
the compression type") changed the EXTENT_FLAG_LOGGING clearing to
operate on em->flags instead of that local 'flags' copy, so a split of
an extent map that is currently being logged wrongly inherits
EXTENT_FLAG_LOGGING.
The flag is then never cleared on the split, and when it is freed while
still on the inode's modified_extents list (for example by the extent
map shrinker) it trips the WARN_ON(!list_empty(&em->list)) in
btrfs_free_extent_map() and leads to a use-after-free.
Clear EXTENT_FLAG_LOGGING from the local 'flags' copy used for the
splits and only clear EXTENT_FLAG_PINNED from em->flags, restoring the
behaviour prior to f86f7a75e2fb.
CC: Jeff Layton <jlayton@kernel.org>
Link: https://lore.kernel.org/all/20260629-btrfs-skip-logging-v1-1-4e3a28c1acaf@kernel.org/
Fixes: f86f7a75e2fb ("btrfs: use the flags of an extent map to identify the compression type")
Reviewed-by: Jeff Layton <jlayton@kernel.org>
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Leo Martins <loemra.dev@gmail.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The percpu_counter dirty_metadata_bytes is updated by negating eb->len
and passing it to percpu_counter_add_batch(), whose amount parameter is
s64. Since commit 84cda1a6087d ("btrfs: cache folio size and shift in
extent_buffer"), eb->len is u32. The u32 result of -eb->len, when
widened to the s64 parameter, becomes a large positive value instead of
the intended negative value. For eb->len == 16384 the counter adds
+4294950912 instead of subtracting 16384.
The counter therefore grows on every metadata writeback instead of
shrinking by the extent buffer size, permanently exceeding
BTRFS_DIRTY_METADATA_THRESH and causing __btrfs_btree_balance_dirty()
to trigger balance_dirty_pages_ratelimited() unconditionally, adding
unnecessary writeback pressure.
Cast eb->len to s64 before negation at both call sites so the
subtraction is performed in signed 64-bit arithmetic.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Fixes: 84cda1a6087d ("btrfs: cache folio size and shift in extent_buffer")
Signed-off-by: Dave Chen <davechen@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
In btrfs_backref_free_node() we have the following assertion:
ASSERT(node->eb == NULL, "node->eb->start=%llu", node->eb->start);
and a user reported the following crash:
Oops: general protection fault, probably for non-canonical address 0xdffffc0000000000: 0000 [#1] SMP KASAN NOPTI
KASAN: null-ptr-deref in range [0x0000000000000000-0x0000000000000007]
CPU: 0 UID: 0 PID: 10422 Comm: syz.0.17 Not tainted 7.1.0-02765-g6b5a2b7d9bc1-dirty #44 PREEMPT(full)
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
RIP: 0010:btrfs_backref_free_node fs/btrfs/backref.c:3057 [inline]
RIP: 0010:btrfs_backref_free_node+0xb9/0x200 fs/btrfs/backref.c:3051
Code: 00 fc ff (...)
RSP: 0018:ffa0000006b0f3c0 EFLAGS: 00010246
RAX: dffffc0000000000 RBX: 0000000000000000 RCX: ffffffff840eb78b
RDX: 0000000000000000 RSI: ffffffff840eafa5 RDI: ff110000742ab768
RBP: ff110000742ab700 R08: 0000000000000000 R09: 0000000000000000
R10: ff110000742ab700 R11: 00000000000a81f9 R12: ff11000107a92020
R13: ff1100005c182ea8 R14: 0000000000000000 R15: dffffc0000000000
FS: 0000555575536500(0000) GS:ff11000183985000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007fa3d0e9d580 CR3: 000000002232a000 CR4: 0000000000753ef0
PKRU: 00000000
Call Trace:
<TASK>
btrfs_backref_cleanup_node+0x27/0x30 fs/btrfs/backref.c:3133
relocate_tree_block fs/btrfs/relocation.c:2604 [inline]
relocate_tree_blocks+0x11b0/0x1a20 fs/btrfs/relocation.c:2707
relocate_block_group+0x499/0xf30 fs/btrfs/relocation.c:3635
do_nonremap_reloc fs/btrfs/relocation.c:5323 [inline]
btrfs_relocate_block_group+0x1749/0x5fb0 fs/btrfs/relocation.c:5490
btrfs_relocate_chunk+0x12b/0x950 fs/btrfs/volumes.c:3647
__btrfs_balance fs/btrfs/volumes.c:4586 [inline]
btrfs_balance+0x1c7f/0x55c0 fs/btrfs/volumes.c:4973
btrfs_ioctl_balance fs/btrfs/ioctl.c:3474 [inline]
btrfs_ioctl+0x38a4/0x5d20 fs/btrfs/ioctl.c:5570
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:597 [inline]
__se_sys_ioctl fs/ioctl.c:583 [inline]
__x64_sys_ioctl+0x18f/0x210 fs/ioctl.c:583
do_syscall_x64 arch/x86/entry/syscall_64.c:63 [inline]
do_syscall_64+0x11f/0x860 arch/x86/entry/syscall_64.c:94
entry_SYSCALL_64_after_hwframe+0x77/0x7f
RIP: 0033:0x7fb38e3b56dd
Code: 02 b8 ff (...)
RSP: 002b:00007fff04115788 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
RAX: ffffffffffffffda RBX: 00007fb38f6b0020 RCX: 00007fb38e3b56dd
RDX: 00002000000003c0 RSI: 00000000c4009420 RDI: 0000000000000004
RBP: 00007fb38e451b48 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000000
R13: 0000000000000000 R14: 00007fb38f6b0020 R15: 00007fb38f6b002c
</TASK>
It seems that this happens on some systems for some reason, when the
ASSERT() macro calls the inline function verify_assert_printk_format()
to evaluate the format string and arguments, causing the NULL pointer
dereference on node->eb.
So change the assertion to check for a NULL node->eb before dereferencing
it. Also, while at it, make the assertion more useful by printing the
owner of the extent buffer as well as its level.
Reported-by: Yue Sun <samsun1006219@gmail.com>
Link: https://lore.kernel.org/linux-btrfs/20260626065542.38413-1-samsun1006219@gmail.com/
Fixes: c4e7778580d6 ("btrfs: use verbose assertions in backref.c")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
btrfs_getattr() unconditionally reads BTRFS_I(inode)->new_delalloc_bytes
and adds it (sector-aligned) to stat->blocks for every inode type.
However, new_delalloc_bytes lives in a union with last_dir_index_offset:
union {
u64 new_delalloc_bytes; /* files only */
u64 last_dir_index_offset; /* directories only */
};
For a directory inode this memory holds last_dir_index_offset, which is
set during directory logging (e.g. flush_dir_items_batch()) to the
offset of the last logged BTRFS_DIR_INDEX_KEY. That offset grows with
the number of entries ever created in the directory (dir indexes are
monotonic and never reused), so it can be arbitrarily large.
As a result, after a directory has been logged (e.g. via an fsync that
triggers directory logging), btrfs_getattr() reports inflated st_blocks
for that directory. The inflation is purely in-core and disappears
after the inode is evicted and reloaded (btrfs_alloc_inode() zeroes the
union), e.g. after a remount.
Reproducer (on a btrfs filesystem):
D=/mnt/btrfs/d
mkdir -p $D
for i in $(seq 1 20000); do touch $D/f$i; done
sync # commit, push dir index high
touch $D/trigger # dirty the dir in a new transaction
xfs_io -c fsync $D # log the directory -> sets last_dir_index_offset
stat -c '%b' $D # st_blocks is now inflated (e.g. 40)
# umount + mount -> st_blocks drops back to the correct value
The evict path already knows this union is type-dependent and guards the
corresponding WARN_ON with !S_ISDIR() in btrfs_destroy_inode(); only
btrfs_getattr() was missing the equivalent check.
Only read new_delalloc_bytes for regular files, which are the only
inodes that ever set it.
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Dave Chen <davechen@synology.com>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
Commit a6908f88c9da ("btrfs: validate data reloc tree file extent item
members") introduced extra checks on file extent items for data reloc
inodes, but it checked the file extent offset without checking if the file
extent is inlined.
This can lead to either false alerts (as the offset member is inside the
inlined data) or even reading beyond the item range.
This has already triggered a warning in a syzbot report.
Although the root fix is to avoid compression for data reloc inodes, for
the sake of consistency, reject inlined file extents first.
Fixes: a6908f88c9da ("btrfs: validate data reloc tree file extent item members")
CC: stable@vger.kernel.org
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
[BUG]
There is a syzbot report that the check inside get_new_location()
triggered:
BTRFS info (device loop0): found 31 extents, stage: move data extents
BTRFS info (device loop0): leaf 8908800 gen 16 total ptrs 28 free space 1676 owner 18446744073709551607
item 0 key (256 INODE_ITEM 0) itemoff 3835 itemsize 160
inode generation 5 transid 0 size 0 nbytes 0
block group 0 mode 40755 links 1 uid 0 gid 0
rdev 0 sequence 0 flags 0x0
atime 1669132761.0
ctime 1669132761.0
mtime 1669132761.0
otime 0.0
item 1 key (256 INODE_REF 256) itemoff 3823 itemsize 12
index 0 name_len 2
item 2 key (258 INODE_ITEM 0) itemoff 3663 itemsize 160
inode generation 1 transid 16 size 733184 nbytes 106496
block group 0 mode 100600 links 0 uid 0 gid 0
rdev 0 sequence 24 flags 0x18
item 3 key (258 EXTENT_DATA 0) itemoff 3595 itemsize 68
generation 16 type 0
inline extent data size 47 ram_bytes 4096 compression 1
[...]
item 27 key (18446744073709551611 ORPHAN_ITEM 258) itemoff 2376 itemsize 0
BTRFS error (device loop0): unexpected non-zero offset in file extent item for data reloc inode 258 key offset 0 offset 9277520992061368337
------------[ cut here ]------------
btrfs_abort_should_print_stack(__error)
[CAUSE]
The above dump tree shows the first file extent item is inlined, which
should make no sense for data reloc inodes, as such inodes just
represent where the data extents are in the relocation destination chunk.
However the relocation path preallocates space for each block,
then dirties them, cluster by cluster.
It's possible to have a single block at the beginning of the block
group, and no other block in the same cluster.
So relocation will preallocate a file extent for that block and dirty
the first block. Then memory pressure forces the data reloc inode to be
written back, before any other blocks are dirtied/allocated.
Finally commit 3eaf5f082c4c ("btrfs: extract inlined creation into a dedicated
delalloc helper") changed the sequence of delalloc. Before that commit we
always tried NOCOW first, so that dirtied block would be written back into
the preallocated space, and appear as a regular extent.
But with that commit, we always try inline first, and since compression
is forced, we try compressing the first block, and then inline the
compressed data, resulting in the above inlined file extent in the data
reloc tree.
Then the check in get_new_location() will check the file offset, without
checking if the file extent is inlined or not, resulting in the above
failure.
[FIX]
Do not allow compression for data reloc inodes.
Since data reloc inode sizes are always block aligned, as long as we do
not compress, @data_len will always be at least one block, and
that will cause can_cow_file_range_inline() to return false, thus no
inlined extent will be created.
Reported-by: syzbot+d950c6ba09b79f6e1864@syzkaller.appspotmail.com
Link: https://lore.kernel.org/linux-btrfs/6a373dc5.764cf64f.168fbe.0001.GAE@google.com/
Fixes: 3eaf5f082c4c ("btrfs: extract inlined creation into a dedicated delalloc helper")
CC: stable@vger.kernel.org
Reviewed-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
The variable-sized buffer buf in struct btrfs_ioctl_search_args_v2 is
declared as __u64[], but it holds a packed byte stream of search results,
where all offsets into the buffer are in bytes.
Declaring buf as __u64[] makes it easy for user space to write incorrect
pointer arithmetic: adding a byte offset directly to a __u64 pointer
scales the offset by 8, landing at byte position offset*8 instead of
offset.
This recently caused an infinite loop in btrfs-progs: the accessor read
all-zero data from misaddressed items, which fed zeroed search keys back
into the ioctl loop and spun forever. The issue was worked around at the
time by disabling TREE_SEARCH_V2 entirely in btrfs-progs (d73e69824854:
"btrfs-progs: temporarily disable usage of v2 of search tree ioctl").
The kernel side already treats buf as a byte buffer, so change the
declaration to __u8[] to match the actual semantics and prevent similar
misuse in user space. The change is ABI compatible: both the structure size
and alignment are unchanged.
Fixes: cc68a8a5a433 ("btrfs: new ioctl TREE_SEARCH_V2")
Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: You-Kai Zheng <ykzheng@synology.com>
Reviewed-by: David Sterba <dsterba@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
|
If the root we got has zero root refs in its root item, we are resetting
the root's ->reloc_root without using barriers like we do everywhere else.
Sashiko complained about this while reviewing another patch, and it's
correct (see the Link tag below).
Also, we should not clear BTRFS_ROOT_DEAD_RELOC_TREE from the root unless
the root points to the reloc root we have.
Fix this by using clear_reloc_root(), which issues the memory barrier
after setting the root's ->reloc_root to NULL and before clearing the bit
BTRFS_ROOT_DEAD_RELOC_TREE from the root.
Link: https://sashiko.dev/#/patchset/cf84f1a217c719e25b6b69e4298dd7afd36c9427.1781194426.git.fdmanana%40suse.com
Reviewed-by: Boris Burkov <boris@bur.io>
Signed-off-by: Filipe Manana <fdmanana@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
|
