| Age | Commit message (Collapse) | Author |
|---|
|
directories
A lot of places relies upon directories never having NORCU dentries;
currently that property holds, but the proof is not straightforward
and rather brittle.
It's better to have that verified in the sole caller of d_alloc_pseudo(),
so that any future bugs in that direction were caught early.
That way we can be sure that
* current directory of any process is not NORCU
* root directory of any process is not NORCU
* starting point of any LOOKUP_RCU pathwalk is not NORCU
* dget_parent() can rely upon ->d_parent not being NORCU
* d_walk() and is_subdir() can rely upon the same
* alloc_file_pseudo() won't create multiple aliases for a directory
without having to go through a convoluted audit.
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
|
|
Parallel lookup starts with a call of d_alloc_parallel(). That primitive
either returns a matching hashed dentry or allocates a new one in the
in-lookup state and returns it to the caller. Once the caller is done
with lookup, it indicates so either by call of d_{splice_alias,add}()
or by call of d_done_lookup(); at that point dentry leaves the in-lookup
state.
If d_alloc_parallel() finds a matching in-lookup dentry, it must wait for
that dentry to leave the in-lookup state, one way or another. Currently
by supplying wait_queue_head to d_alloc_parallel(). If d_alloc_parallel()
creates a new in-lookup dentry, the address of that wait_queue_head is stored
in ->d_wait of new dentry and stays there while it's in the in-lookup;
subsequent d_alloc_parallel() will wait on the queue found in the matching
in-lookup dentry. Transition out of in-lookup state wakes waiters on that
queue (if any).
That works, but the calling conventions are inconvenient - the caller must
supply wait_queue_head and make sure that it survives at least until the new
in-lookup dentry leaves the in-lookup state. That amounts to boilerplate
in the d_alloc_parallel() callers that are followed by a call of d_lookup_done()
in the same function; in cases like nfs asynchronous unlink it gets worse than
that.
This patch changes d_alloc_parallel() to use wake_up_var_locked() to
wake up waiters, and wait_var_event_spinlock() to wait. dentry->d_lock
is used for synchronisation as it is already held and the relevant
times.
That eliminates the need of caller-supplied wait_queue_head, simplifying
the calling conventions. Better yet, we only need one bit of information
stored in dentry itself: whether there are any waiters to be woken up,
and that can be easily stored in ->d_flags; ->d_wait goes away.
The reason we need that bit (DCACHE_LOOKUP_WAITERS) is that with wait_var
machinery the queues are shared with all kinds of stuff and there's
no way tell if any of the waiters have anything to do with our dentry;
most of the time none of them will be relevant, so we need to avoid the
pointless wakeups.
Another benefit of the new scheme comes from the fact that wakeups
have to be done outside of write-side critical areas of ->i_dir_seq;
with the old scheme we need to carry the value picked from ->d_wait from
__d_lookup_unhash() to the place where we actually wake the waiters up.
Now we can just leave DCACHE_LOOKUP_WAITERS in ->d_flags until we get
to doing wakeups - that's done within the same ->d_lock scope, so we
are fine; new bit is accessed only under ->d_lock and it's seen only
on dentries with DCACHE_PAR_LOOKUP in ->d_flags.
__d_lookup_unhash() no longer needs to re-init ->d_lru. That was
previously shared (in a union) with ->d_wait but ->d_wait is now gone
so it no longer corrupts ->d_lru.
Co-developed-by: Al Viro <viro@zeniv.linux.org.uk> # saner handling of flags
Signed-off-by: NeilBrown <neil@brown.name>
Signed-off-by: Al Viro <viro@zeniv.linux.org.uk>
|
|
[BUG]
Unlinking a refcounted file whose refcount tree has leaf blocks
triggers a fortify panic due to an out-of-bounds write.
[CAUSE]
When the last leaf block is removed from a refcount tree,
ocfs2_remove_refcount_extent() converts the root back to leaf mode
with a bulk memset on &rb->rf_records. rf_records sits in an anonymous
union with rf_list. rf_list.l_tree_depth aliases rf_records.rl_count,
and is 0 for a single-level tree. With rl_count equal to 0, the memset
writes past the 16-byte declared size of rf_records, which the fortify
checker catches.
[FIX]
Replace the bulk memset on &rb->rf_records with a correctly-bounded
memset on rl_recs[] alone, after setting rl_count to the correct value.
Link: https://lore.kernel.org/ah3TESOsEO9j_JLU@dev
Fixes: 2f26f58df041 ("ocfs2: annotate flexible array members with __counted_by_le()")
Signed-off-by: Ian Bridges <icb@fastmail.org>
Reported-by: syzbot+3ef989aae096b30f1663@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=3ef989aae096b30f1663
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
ocfs2_control_release()
Move atomic_inc(&ocfs2_control_opened) and the handshake state update
inside ocfs2_control_lock to close a race window where
ocfs2_control_release() can observe ocfs2_control_opened dropping to zero
(resetting ocfs2_control_this_node and running_proto) while
ocfs2_control_install_private() is about to bump the counter and mark the
connection valid.
Link: https://lore.kernel.org/20260601121618.1263346-1-joseph.qi@linux.alibaba.com
Fixes: 3cfd4ab6b6b4 ("ocfs2: Add the local node id to the handshake.")
Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reported-by: Ginger <ginger.jzllee@gmail.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
debug_lockres_open() copies inode->i_private into struct debug_lockres and
debug_lockres_release() later drops that pointer with dlm_put(). That
only works if open successfully pins the struct dlm_ctxt.
Today open calls dlm_grab(dlm) but ignores its return value. Once the
last domain unregister has removed the context from dlm_domains,
dlm_grab() returns NULL, yet open still stores the raw pointer and returns
success. The later release path is outside the debugfs removal barrier,
so it can call dlm_put() after dlm_free_ctxt_mem() has freed the context.
KASAN reports this as a slab-use-after-free in dlm_put() called from
debug_lockres_release().
Fail the open when dlm_grab() cannot acquire the reference and unwind the
seq_file private state before returning. That keeps locking_state from
handing out a file descriptor whose release path does not own the
dlm_ctxt.
The buggy scenario involves two paths, with each column showing the order
within that path:
locking_state debugfs open: last domain unregister:
1. debug_lockres_open() reads 1. dlm_unregister_domain() calls
inode->i_private. dlm_complete_dlm_shutdown().
2. debug_lockres_open() calls 2. shutdown removes the dlm_ctxt from
dlm_grab(dlm) and gets NULL. dlm_domains.
3. open still stores the raw dlm 3. final teardown reaches
pointer in dl->dl_ctxt and dlm_free_ctxt_mem() and frees it.
returns success.
4. debug_lockres_release() later
calls dlm_put(dl->dl_ctxt).
Validation reproduced this kernel report:
KASAN slab-use-after-free in dlm_put+0x82/0x200
RIP: 0033:0x7f4d349bc9e0
The buggy address belongs to the object at ffff888103a3c000 which belongs
to the cache kmalloc-2k of size 2048
The buggy address is located 816 bytes inside of freed 2048-byte region
[ffff888103a3c000, ffff888103a3c800)
Write of size 4
Call trace:
dump_stack_lvl+0x66/0xa0 (?:?)
print_report+0xd0/0x630 (?:?)
dlm_put+0x82/0x200 (?:?)
srso_alias_return_thunk+0x5/0xfbef5 (?:?)
__virt_addr_valid+0x188/0x2f0 (?:?)
kasan_report+0xe4/0x120 (?:?)
kasan_check_range+0x105/0x1b0 (?:?)
debug_lockres_release+0x53/0x80 (fs/ocfs2/dlm/dlmdebug.c:587)
dlm_put+0x9/0x200 (?:?)
debug_lockres_release+0x5c/0x80 (fs/ocfs2/dlm/dlmdebug.c:587)
full_proxy_release+0x67/0x90 (?:?)
__fput+0x1df/0x4b0 (?:?)
do_raw_spin_lock+0x10f/0x1b0 (?:?)
fput_close_sync+0xd2/0x170 (?:?)
__x64_sys_close+0x55/0x90 (?:?)
do_syscall_64+0x10c/0x640 (arch/x86/entry/syscall_64.c:87)
irqentry_exit+0xac/0x6e0 (?:?)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (?:?)
Freed by task stack:
kasan_save_stack+0x33/0x60 (?:?)
kasan_save_track+0x14/0x30 (?:?)
kasan_save_free_info+0x3b/0x60 (?:?)
__kasan_slab_free+0x5f/0x80 (?:?)
kfree+0x30f/0x580 (?:?)
dlm_put+0x1ce/0x200 (?:?)
dlm_unregister_domain+0xf6/0xb30 (?:?)
o2cb_cluster_disconnect+0x6b/0x90 (?:?)
ocfs2_cluster_disconnect+0x41/0x70 (?:?)
ocfs2_dlm_shutdown+0x1c4/0x220 (?:?)
ocfs2_dismount_volume+0x38a/0x550 (?:?)
generic_shutdown_super+0xc3/0x220 (?:?)
kill_block_super+0x29/0x60 (?:?)
deactivate_locked_super+0x66/0xe0 (?:?)
cleanup_mnt+0x13d/0x210 (?:?)
task_work_run+0xfa/0x170 (?:?)
exit_to_user_mode_loop+0xd6/0x430 (?:?)
do_syscall_64+0x3cb/0x640 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (?:?)
Link: https://lore.kernel.org/20260531044714.1640172-1-rollkingzzc@gmail.com
Fixes: 4e3d24ed1a12 ("ocfs2/dlm: Dumps the lockres' into a debugfs file")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
ocfs2_trim_mainbm() trims the global bitmap in cluster units, but its
too-short range validation only checks sb->s_blocksize.
On filesystems with a cluster size larger than the block size, a FITRIM
range that is at least one block but shorter than one cluster is accepted
and shifted down to len == 0. The later start + len - 1 and len -= ...
arithmetic then underflows and can drive trimming past the requested
range.
Reject ranges shorter than s_clustersize instead. That preserves the
existing -EINVAL behavior for requests that cannot discard even one
allocation unit and keeps zero-cluster trims out of the group walk.
Link: https://lore.kernel.org/20260528151247.361854-1-rollkingzzc@gmail.com
Fixes: aa89762c5480 ("ocfs2: return EINVAL if the given range to discard is less than block size")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
ocfs2_validate_inode_block() already rejects several inconsistent
self-contained dinodes before they are exposed to the rest of the
filesystem. Fast symlinks need the same treatment.
A zero-cluster symlink is treated as a fast symlink and later read through
page_get_link() and ocfs2_fast_symlink_read_folio(). That path uses
strnlen() on the inline payload and then copies len + 1 bytes into the
folio. If a corrupt dinode stores an i_size that does not fit the inline
area or omits the terminating NUL at i_size, that copy reads past the end
of the inode block buffer.
Reject zero-cluster symlink dinodes whose i_size exceeds the inline
fast-symlink capacity or whose inline payload is not NUL-terminated
exactly at i_size when the inode block is validated. This keeps malformed
fast symlinks from reaching the read path.
Validation reproduced this kernel report:
KASAN use-after-free in ocfs2_fast_symlink_read_folio+0x12c/0x1f0
RIP: 0033:0x7f5c6d859aa7
Read of size 3905
Call trace:
dump_stack_lvl+0x66/0xa0 (?:?)
print_report+0xce/0x630 (?:?)
ocfs2_fast_symlink_read_folio+0x12c/0x1f0 (fs/ocfs2/inode.c:?)
srso_alias_return_thunk+0x5/0xfbef5 (?:?)
__virt_addr_valid+0x19f/0x330 (?:?)
kasan_report+0xe0/0x110 (?:?)
kasan_check_range+0x105/0x1b0 (?:?)
__asan_memcpy+0x23/0x60 (?:?)
filemap_read_folio+0x27/0xe0 (?:?)
filemap_read_folio+0x35/0xe0 (?:?)
do_read_cache_folio+0x138/0x230 (?:?)
__page_get_link+0x26/0x110 (?:?)
page_get_link+0x2e/0x70 (?:?)
vfs_readlink+0x15e/0x250 (?:?)
touch_atime+0x4d/0x370 (?:?)
do_readlinkat+0x186/0x200 (?:?)
do_user_addr_fault+0x65a/0x890 (?:?)
__x64_sys_readlink+0x46/0x60 (?:?)
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (?:?)
Link: https://lore.kernel.org/20260528151230.361127-1-rollkingzzc@gmail.com
Fixes: ea022dfb3c2a ("ocfs: simplify symlink handling")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Gui-Dong Han <2045gemini@gmail.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
During unmount, ocfs2_journal_shutdown() frees the journal and sets
osb->journal to NULL. Later, when VFS evicts remaining cached inodes,
ocfs2_evict_inode() -> ocfs2_clear_inode() -> ocfs2_checkpoint_inode()
-> ocfs2_ci_fully_checkpointed() dereferences osb->journal, causing a
NULL pointer dereference.
Fix this by adding a NULL check for osb->journal in
ocfs2_checkpoint_inode(). If the journal is NULL, it has already been
fully flushed and destroyed during shutdown, so there is nothing to
checkpoint.
Link: https://lore.kernel.org/20260531131645.3650299-1-joseph.qi@linux.alibaba.com
Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Fixes: da5e7c87827e ("ocfs2: cleanup journal init and shutdown")
Signed-off-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Tested-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Reviewed-by: Heming Zhao <heming.zhao@suse.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
In ocfs2_read_blocks(), caller should't assume that buffer head returned
by 'sb_getblk()' is exclusively owned and so 'put_bh()' always drops
b_count from 1 to 0. If it is not so, buffer head remains on hold and
likely to be returned by the next call to 'sb_getblk()' unchanged - that
is, with BH_Uptodate bit set even if it has failed validation previously,
thus allowing to insert that buffer head into OCFS2 metadata cache and
submit it to upper layers. To avoid such a scenario, BH_Uptodate should
be cleared immediately after 'validate()' callback has detected some data
inconsistency.
Link: https://lore.kernel.org/20260529094128.494293-1-dmantipov@yandex.ru
Fixes: cf76c78595ca ("ocfs2: don't put and assigning null to bh allocated outside")
Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Reported-by: syzbot+caacd220635a9cc3bac9@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=caacd220635a9cc3bac9
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
ocfs2_validate_gd_parent() only bounds bg_bits against the parent
allocator's chain geometry. A malicious descriptor can still claim a
bg_size/bg_bits pair that exceeds the bitmap bytes that physically fit in
the group descriptor block, so later bitmap scans and bit updates can run
past bg_bitmap.
Add a physical-cap check based on ocfs2_group_bitmap_size() for the parent
allocator type and reject descriptors whose bg_size or bg_bits exceed that
capacity. Keep the existing chain geometry check so both the on-disk
bitmap layout and the allocator metadata must agree before the descriptor
is used.
Validation reproduced this kernel report:
KASAN use-after-free in _find_next_bit+0x7f/0xc0
Read of size 8
Call trace:
dump_stack_lvl+0x66/0xa0 (?:?)
print_report+0xd0/0x630 (?:?)
_find_next_bit+0x7f/0xc0 (?:?)
srso_alias_return_thunk+0x5/0xfbef5 (?:?)
__virt_addr_valid+0x188/0x2f0 (?:?)
kasan_report+0xe4/0x120 (?:?)
ocfs2_find_max_contig_free_bits+0x35/0x70 (fs/ocfs2/suballoc.c:1375)
ocfs2_block_group_set_bits+0x472/0x4b0 (fs/ocfs2/suballoc.c:1457)
ocfs2_cluster_group_search+0x16b/0x440 (fs/ocfs2/suballoc.c:86)
ocfs2_bg_discontig_fix_result+0x1ef/0x230 (fs/ocfs2/suballoc.c:1786)
ocfs2_search_chain+0x8f8/0x10a0 (fs/ocfs2/suballoc.c:1886)
get_page_from_freelist+0x70e/0x2370 (?:?)
lock_release+0xc6/0x290 (?:?)
do_raw_spin_unlock+0x9a/0x100 (?:?)
kasan_unpoison+0x27/0x60 (?:?)
__bfs+0x147/0x240 (?:?)
get_page_from_freelist+0x83d/0x2370 (?:?)
ocfs2_claim_suballoc_bits+0x38c/0xe70 (fs/ocfs2/suballoc.c:96)
sched_domains_numa_masks_clear+0x70/0xd0 (?:?)
check_irq_usage+0xe8/0xb70 (?:?)
__ocfs2_claim_clusters+0x18d/0x4c0 (fs/ocfs2/suballoc.c:2497)
check_path+0x24/0x50 (?:?)
rcu_is_watching+0x20/0x50 (?:?)
check_prev_add+0xfd/0xd00 (?:?)
ocfs2_add_clusters_in_btree+0x17d/0x810 (fs/ocfs2/suballoc.c:?)
__folio_batch_add_and_move+0x1f5/0x3d0 (?:?)
ocfs2_add_inode_data+0xd9/0x120 (fs/ocfs2/suballoc.c:?)
filemap_add_folio+0x105/0x1f0 (?:?)
ocfs2_write_begin_nolock+0x29f7/0x2f80 (fs/ocfs2/suballoc.c:3043)
ocfs2_read_inode_block+0xb5/0x110 (fs/ocfs2/suballoc.c:?)
down_write+0xf5/0x180 (?:?)
ocfs2_write_begin+0x180/0x240 (fs/ocfs2/suballoc.c:?)
__mark_inode_dirty+0x758/0x9a0 (?:?)
inode_to_bdi+0x41/0x90 (?:?)
balance_dirty_pages_ratelimited_flags+0xf8/0x1d0 (?:?)
generic_perform_write+0x252/0x440 (?:?)
mnt_put_write_access_file+0x16/0x70 (?:?)
file_update_time_flags+0xe4/0x200 (?:?)
ocfs2_file_write_iter+0x80a/0x1320 (fs/ocfs2/suballoc.c:?)
lock_acquire+0x184/0x2f0 (?:?)
ksys_write+0xd2/0x170 (?:?)
apparmor_file_permission+0xf5/0x310 (?:?)
read_zero+0x8d/0x140 (?:?)
lock_is_held_type+0x8f/0x100 (?:?)
Link: https://lore.kernel.org/20260524111248.1429884-1-rollkingzzc@gmail.com
Fixes: ccd979bdbce9 ("[PATCH] OCFS2: The Second Oracle Cluster Filesystem")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The locking_state debugfs iterator snapshots struct ocfs2_lock_res by
value under ocfs2_dlm_tracking_lock and later formats that copy in
ocfs2_dlm_seq_show(). That is fine for the inline fields, but the
userspace fsdlm stack stores the LVB through lksb_fsdlm.sb_lvbptr. Once
the iterator drops the tracking lock, a copied non-NULL sb_lvbptr still
points into the original lockres owner, so teardown can free that
container before the debugfs dump walks the raw LVB bytes.
Rebase the copied sb_lvbptr to the copied l_lksb before dumping the raw
LVB. The seq snapshot already carries the inline LVB storage reserved in
struct ocfs2_dlm_lksb, so the debugfs reader can dump the copied bytes
without borrowing the original lockres lifetime.
The buggy scenario involves two paths, with each column showing the order
within that path:
locking_state reader: lockres teardown:
1. ocfs2_dlm_seq_start()/next() 1. file release or another owner
copies struct ocfs2_lock_res teardown reaches
2. ocfs2_dlm_seq_show() formats ocfs2_lock_res_free()
the copied row 2. the lockres is removed from the
3. ocfs2_dlm_lvb() follows the tracking list
copied sb_lvbptr 3. the owner frees the original
lockres container
Validation reproduced this kernel report:
KASAN slab-use-after-free in ocfs2_dlm_seq_show+0x1bd/0x430
RIP: 0033:0x7f8ec4b1e29d
The buggy address belongs to the object at ffff88810a1e0800 which belongs
to the cache kmalloc-1k of size 1024
The buggy address is located 368 bytes inside of freed 1024-byte region
[ffff88810a1e0800, ffff88810a1e0c00)
Read of size 1
Call trace:
dump_stack_lvl+0x66/0xa0
print_report+0xce/0x630
ocfs2_dlm_seq_show+0x1bd/0x430 (fs/ocfs2/dlmglue.c:3137)
srso_alias_return_thunk+0x5/0xfbef5
__virt_addr_valid+0x19f/0x330
kasan_report+0xe0/0x110
seq_read_iter+0x29d/0x790
seq_read+0x20a/0x280
find_held_lock+0x2b/0x80
rcu_read_unlock+0x18/0x70
full_proxy_read+0x9e/0xd0
vfs_read+0x12c/0x590
ksys_read+0xd2/0x170
do_user_addr_fault+0x65a/0x890
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Allocated by task stack:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
__kasan_kmalloc+0xaa/0xb0
ocfs2_file_open+0x13e/0x300
do_dentry_open+0x233/0x7f0
vfs_open+0x5a/0x1b0
path_openat+0x66d/0x1540
do_file_open+0x186/0x2b0
do_sys_openat2+0xce/0x150
__x64_sys_openat+0xd0/0x140
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Freed by task stack:
kasan_save_stack+0x33/0x60
kasan_save_track+0x14/0x30
kasan_save_free_info+0x3b/0x60
__kasan_slab_free+0x5f/0x80
kfree+0x313/0x590
ocfs2_file_release+0x138/0x260
__fput+0x1df/0x4b0
fput_close_sync+0xd2/0x170
__x64_sys_close+0x55/0x90
do_syscall_64+0x115/0x6a0 (arch/x86/entry/syscall_64.c:87)
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Link: https://lore.kernel.org/20260525041726.4112882-1-rollkingzzc@gmail.com
Fixes: cf4d8d75d8ab ("ocfs2: add fsdlm to stackglue")
Assisted-by: Codex:gpt-5.5
Signed-off-by: Zhang Cen <rollkingzzc@gmail.com>
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Cc: Mark Fasheh <mark@fasheh.com>
Cc: Joel Becker <jlbec@evilplan.org>
Cc: Junxiao Bi <junxiao.bi@oracle.com>
Cc: Changwei Ge <gechangwei@live.cn>
Cc: Jun Piao <piaojun@huawei.com>
Cc: Heming Zhao <heming.zhao@suse.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "use vma locks for proc/pid/{smaps|numa_maps} reads", v2.
Use per-vma locks when reading /proc/pid/smaps and /proc/pid/numa_maps
similar to /proc/pid/maps to reduce contention on central mmap_lock. One
major difference between maps and smaps/numa_maps reading is that the
latter executes page table walk which can't be done under RCU due to a
possibility of sleeping. Therefore we drop RCU read lock before this walk
while keeping the VMA locked. After the walk we retake RCU read lock,
reset VMA iterator and proceed with the next VMA.
The last two patches extend /proc/pid/maps test to cover /proc/pid/smaps
reading during concurrent address space modification.
This patch (of 3):
proc/pid/{smaps|numa_maps} can be read using the combination of RCU and
VMA read locks, similar to proc/pid/maps. RCU is required to safely
traverse the VMA tree and VMA lock stabilizes the VMA being processed and
the pagetable walk.
Link: https://lore.kernel.org/20260426062718.1238437-1-surenb@google.com
Link: https://lore.kernel.org/20260426062718.1238437-2-surenb@google.com
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Liam R. Howlett <liam@infradead.org>
Reviewed-by: Lorenzo Stoakes <ljs@kernel.org>
Cc: Jann Horn <jannh@google.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: "Paul E . McKenney" <paulmck@kernel.org>
Cc: Pedro Falcato <pfalcato@suse.de>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Wei Yang <richard.weiyang@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "userfaultfd: merge fs/userfaultfd.c into mm/userfaultfd.c",
v3.
These patches merge fs/userfaultfd.c into mm/userfaultfd.c and make
functions used only inside mm/userfaultfd.c static.
This patch (of 2):
Historically userfaultfd implementation has been split between
fs/userfaultfd.c and mm/userfaultfd.c.
The mm/ part implemented memory management operations, while the fs/ part
implemented file descriptor handling and called into the mm/ part for the
actual memory management work.
This separation is quite artificial and fs/userfaultfd.c does not seem to
belong to fs/ because it's only a user if vfs APIs and like for other
users, for example, memfd and secretmem, the file descriptor handling
could live in mm/ as well.
"Append" fs/userfaultfd.c to mm/userfaultfd and update fs/Makefile and
MAINTAINERS accordingly.
No intended functional changes.
Link: https://lore.kernel.org/20260523173759.3964908-1-rppt@kernel.org
Link: https://lore.kernel.org/20260523173759.3964908-2-rppt@kernel.org
Assisted-by: Copilot:claude-opus-4-6
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: David Hildenbrand <david@kernel.org>
Cc: Jan Kara <jack@suse.cz>
Cc: "Kirill A. Shutemov" <kirill@shutemov.name>
Cc: Peter Xu <peterx@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Use kasprintf() to simplify ecryptfs_crypto_api_algify_cipher_name().
Use const char * for the read-only cipher name and chaining modifier
while at it.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Tyler Hicks <code@tyhicks.com>
|
|
Commit e09c77d94003 ("eventpoll: hoist CTL_ADD scratch state into
struct ep_ctl_ctx") moved tfile_check_list from a file-scope global into
the stack-allocated struct ep_ctl_ctx, and in doing so replaced the
EP_UNACTIVE_PTR sentinel with NULL on the grounds that "NULL is the
obvious 'empty' value and zero-init handles it for free", describing the
change as "No functional change". It is not.
epitems_head->next is overloaded with two roles:
1. the "next" pointer that threads a head onto ctx->tfile_check_list;
2. a membership flag: ep_remove_file() uses
!smp_load_acquire(&v->next) to mean "this head is not on any
pending ctx->tfile_check_list and is therefore safe to free".
Before that change the EP_UNACTIVE_PTR sentinel kept the two roles
disjoint: a head on the list always had a non-NULL ->next (another head,
or the sentinel at the tail), so ->next == NULL was equivalent to "never
listed". With the sentinel gone the list is NULL-terminated, so the tail
head's ->next is NULL as well. ep_remove_file()'s gate can no longer
distinguish "never listed" from "listed at the tail", and misfires on
the tail head.
The reader (reverse_path_check_proc) holds epnested_mutex +
rcu_read_lock; the freer (ep_remove_file) holds ep->mtx + file->f_lock.
The two sides share no mutex -- the sentinel was the invariant the gate
relied on to know it could skip the read side. With it gone,
ep_remove_file() frees the tail head while reverse_path_check_proc() is
still walking it, producing the slab-use-after-free read. The syzbot
reproducer hits this within seconds on a multi-CPU VM.
Restore the sentinel: initialize ctx.tfile_check_list to EP_UNACTIVE_PTR
in do_epoll_ctl_file(), and terminate the walk on "!= EP_UNACTIVE_PTR"
in reverse_path_check() and clear_tfile_check_list(). The tail head's
->next becomes the sentinel again rather than NULL, so
ep_remove_file()'s gate regains its exclusivity and stops misfiring on
the tail. ep_remove_file() itself is unchanged.
This restores the invariant the file-scope tfile_check_list relied on
before that change while preserving the ctx packaging it introduced.
Reported-by: syzbot+e70e1b6cba8714543f7c@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=e70e1b6cba8714543f7c
Fixes: e09c77d94003 ("eventpoll: hoist CTL_ADD scratch state into struct ep_ctl_ctx")
Suggested-by: Christian Brauner <brauner@kernel.org>
Link: https://lore.kernel.org/all/20260528-rotwild-summt-kuhhandel-7276ef4c33b7@brauner.io/
Signed-off-by: Zhan Wei <zhanwei919@gmail.com>
Link: https://patch.msgid.link/20260529142533.23696-1-zhanwei919@gmail.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
It has no callers left, so delete it. Inline __end_buffer_write_sync()
into bh_end_write().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-35-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Unify end_buffer_read_sync() and __end_buffer_read_notouch()
by requiring the caller put the refcount on the buffer. The only caller
is in the gfs2_meta_read() path, and there we can put the refcount
after locking the buffer.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-34-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
This shrinks buffer_head by 8 bytes, letting us pack more buffer heads
per slab. With a Debian config, it shrinks from 104 bytes to 96 bytes
which is 42 objects per 4KiB page rather than 39, a 7% reduction in the
amount of memory used.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-33-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
No users are left; remove this API. Also remove/fix comments mentioning
it, and end_bio_bh_io_sync() as it's now unused.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-32-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-30-willy@infradead.org
Acked-by: Ryusuke Konishi <konishi.ryusuke@gmail.com>
Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-nilfs@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-29-willy@infradead.org
Acked-by: Ryusuke Konishi <konishi.ryusuke@gmail.com>
Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-nilfs@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-28-willy@infradead.org
Acked-by: Ryusuke Konishi <konishi.ryusuke@gmail.com>
Reviewed-by: Viacheslav Dubeyko <slava@dubeyko.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-nilfs@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
There are no more callers of this function, so delete it.
end_buffer_async_write() then has only one caller left, so
inline it into bh_end_async_write().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-27-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using bh_submit() instead of
submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-26-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: gfs2@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
All buffer heads submitted by gfs2_submit_bhs() use
end_buffer_read_sync() so we can call it directly.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-25-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: gfs2@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using bh_submit() instead of
submit_bh(). Also simplify the control flow now that the buffer
refcount is not put by bh_end_read().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-24-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: gfs2@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using bh_submit() instead
of submit_bh(). Also simplify the control flow now that the buffer
refcount is not put by bh_end_read().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-23-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: gfs2@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-22-willy@infradead.org
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: ocfs2-devel@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-21-willy@infradead.org
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: ocfs2-devel@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-20-willy@infradead.org
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: ocfs2-devel@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer
refcount by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-19-willy@infradead.org
Reviewed-by: Joseph Qi <joseph.qi@linux.alibaba.com>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: ocfs2-devel@lists.linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-18-willy@infradead.org
Acked-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using bh_submit()
instead of submit_bh() in journal_submit_commit_record()
and jbd2_journal_commit_transaction(). These both use
journal_end_buffer_io_sync(), so it's more straightforward to do them
both at once.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-17-willy@infradead.org
Acked-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-16-willy@infradead.org
Acked-by: Theodore Ts'o <tytso@mit.edu>
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-15-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by converting
ext4_end_buffer_io_sync() from bh_end_io_t to bio_end_io_t and
calling bh_submit().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-14-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by converting ext4_end_bitmap_read() from bh_end_io_t to bio_end_io_t
and calling bh_submit().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-13-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Cc: linux-ext4@vger.kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using __bh_submit() instead
of submit_bh_wbc(). Since there is only one caller of submit_bh_wbc()
left, inline it into submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-12-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call by using bh_submit() instead of
submit_bh(). Since mark_buffer_async_read() would collapse to a single
function call, inline it into block_read_full_folio() along with its
extensive comment. Convert end_buffer_async_read_io() to
bh_end_async_read().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-11-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-10-willy@infradead.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-9-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-8-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-7-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Avoid an extra indirect function call and changing the buffer refcount
by using bh_submit() instead of submit_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-6-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
These are the bio_end_io_t versions of end_buffer_read_sync(),
end_buffer_write_sync() and end_buffer_async_write(). They do not
contain a put_bh() call as it is no longer necessary.
Also add the helper function bio_endio_bh().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-5-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
All callers of mark_buffer_async_write_endio() pass
end_buffer_async_write, so we can inline mark_buffer_async_write_endio()
into mark_buffer_async_write() and just call that instead.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-4-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
bh_submit() takes a bio_end_io allowing users to avoid the indirect
function call through bh->b_end_io, and eventually allowing us to remove
bh->b_end_io.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-3-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Rearrange functions to avoid this forward declaration.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Link: https://patch.msgid.link/20260528173150.1093780-2-willy@infradead.org
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
ep_events_available() checks for available events by looking at
ep->rdllist and ep_is_scanning(). However, this is done without a lock
and can report false negative if ep_start_scan() or ep_done_scan() are
executed by another task concurrently. For example:
_________________________________________________________________________
|ep_start_scan()
| list_splice_init(&ep->rdllist, ...)
ep_events_available() |
!list_empty_careful(&ep->rdllist)|
|| ep_is_scanning(ep) |
| ep_enter_scan(ep)
___________________________________|_____________________________________
Another example:
_________________________________________________________________________
ep_events_available() |
|ep_start_scan()
| list_splice_init(&ep->rdllist, ...)
| ep_enter_scan(ep)
!list_empty_careful(&ep->rdllist)|
|ep_done_scan()
| ep_exit_scan(ep)
| list_splice(..., &ep->rdllist)
|| ep_is_scanning(ep) |
___________________________________|_____________________________________
In the above examples, ep_events_available() sees no event despite
events being available. In case epoll_wait() is called with timeout=0,
epoll_wait() will wrongly return "no event" to user.
Introduce a sequence lock to resolve this issue.
Measuring the time consumption of 10 million loop iterations doing
epoll_wait(), the following performance drop is observed:
timeout #event before after diff
0ms 0 3727ms 3974ms +6.6%
0ms 1 8099ms 9134ms +13%
1ms 1 13525ms 13586ms +0.45%
Considering the use case of epoll_wait() (wait for events, do something
with the events, repeat), it should only contribute to a small portion of
user's CPU consumption. Therefore this performance drop is not alarming.
Fixes: c5a282e9635e ("fs/epoll: reduce the scope of wq lock in epoll_wait()")
Suggested-by: Mateusz Guzik <mjguzik@gmail.com>
Signed-off-by: Nam Cao <namcao@linutronix.de>
Link: https://patch.msgid.link/4363cd8e34a21d4f0d257be1b33e84dc25030fdf.1780422138.git.namcao@linutronix.de
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The IOCB_DONTCACHE writeback path in generic_write_sync() calls
filemap_flush_range() on every write, submitting writeback inline in
the writer's context. Perf lock contention profiling shows the
performance problem is not lock contention but the writeback submission
work itself — walking the page tree and submitting I/O blocks the writer
for milliseconds, inflating p99.9 latency from 23ms (buffered) to 93ms
(dontcache).
Replace the inline filemap_flush_range() call with a flusher kick that
drains dirty pages in the background. This moves writeback submission
completely off the writer's hot path.
To avoid flushing unrelated buffered dirty data, add a dedicated
WB_start_dontcache bit and wb_check_start_dontcache() handler that uses
the per-wb WB_DONTCACHE_DIRTY counter to determine how many pages to
write back. The flusher writes back that many pages from the oldest dirty
inodes (not restricted to dontcache-specific inodes). This helps
preserve I/O batching while limiting the scope of expedited writeback.
Like WB_start_all, the WB_start_dontcache bit coalesces multiple
DONTCACHE writes into a single flusher wakeup without per-write
allocations. Use test_and_clear_bit to atomically consume the kick
request before reading the dirty counter and starting writeback, so that
concurrent DONTCACHE writes during writeback can re-set the bit and
schedule a follow-up flusher run.
Read the dirty counter with wb_stat_sum() (aggregating per-CPU batches)
rather than wb_stat() (which reads only the global counter) to ensure
small writes below the percpu batch threshold are visible to the flusher.
In filemap_dontcache_kick_writeback(), set the WB_start_dontcache bit
inside the unlocked_inode_to_wb_begin/end section for correct cgroup
writeback domain targeting, but defer the wb_wakeup() call until after
the section ends, since wb_wakeup() uses spin_unlock_irq() which would
unconditionally re-enable interrupts while the i_pages xa_lock may still
be held under irqsave during a cgroup writeback switch. Pin the wb with
wb_get() inside the RCU critical section before calling wb_wakeup()
outside it, since cgroup bdi_writeback structures are RCU-freed and the
wb pointer could become invalid after unlocked_inode_to_wb_end() drops
the RCU read lock.
Also add WB_REASON_DONTCACHE as a new writeback reason for tracing
visibility.
dontcache-bench results (same host, T6F_SKL_1920GBF, 251 GiB RAM,
xfs on NVMe, fio io_uring):
Buffered and direct I/O paths are unaffected by this patchset. All
improvements are confined to the dontcache path:
Single-stream throughput (MB/s):
Before After Change
seq-write/dontcache 298 897 +201%
rand-write/dontcache 131 236 +80%
Tail latency improvements (seq-write/dontcache):
p99: 135,266 us -> 23,986 us (-82%)
p99.9: 8,925,479 us -> 28,443 us (-99.7%)
Multi-writer (4 jobs, sequential write):
Before After Change
dontcache aggregate (MB/s) 2,529 4,532 +79%
dontcache p99 (us) 8,553 1,002 -88%
dontcache p99.9 (us) 109,314 1,057 -99%
Dontcache multi-writer throughput now matches buffered (4,532 vs
4,616 MB/s).
32-file write (Axboe test):
Before After Change
dontcache aggregate (MB/s) 1,548 3,499 +126%
dontcache p99 (us) 10,170 602 -94%
Peak dirty pages (MB) 1,837 213 -88%
Dontcache now reaches 81% of buffered throughput (was 35%).
Competing writers (dontcache vs buffered, separate files):
Before After
buffered writer 868 433 MB/s
dontcache writer 415 433 MB/s
Aggregate 1,284 866 MB/s
Previously the buffered writer starved the dontcache writer 2:1.
With per-bdi_writeback tracking, both writers now receive equal
bandwidth. The aggregate matches the buffered-vs-buffered baseline
(863 MB/s), indicating fair sharing regardless of I/O mode.
The dontcache writer's p99.9 latency collapsed from 119 ms to
33 ms (-73%), eliminating the severe periodic stalls seen in the
baseline. Both writers now share identical latency profiles,
matching the buffered-vs-buffered pattern.
The per-bdi_writeback dirty tracking dramatically reduces peak dirty
pages in dontcache workloads, with the 32-file test dropping from
1.8 GB to 213 MB. Dontcache sequential write throughput triples and
multi-writer throughput reaches parity with buffered I/O, with tail
latencies collapsing by 1-2 orders of magnitude.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Jeff Layton <jlayton@kernel.org>
Link: https://patch.msgid.link/20260511-dontcache-v7-3-2848ddce8090@kernel.org
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Ritesh Harjani (IBM) <ritesh.list@gmail.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
