| Age | Commit message (Collapse) | Author |
|---|
|
Restart handling in the previous patch was incorrect, so: move btree
operations into a separate helper, and run it with a lockrestart_do().
Additionally, clarify whether pagecache or the btree takes precedence.
Right now, the btree takes precedence: this is incorrect, but it's
needed to pass fstests. Add a giant comment explaining why.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
bcachefs currently populates fiemap data from the extents btree.
This works correctly when the fiemap sync flag is provided, but if
not, it skips all delalloc extents that have not yet been flushed.
This is because delalloc extents from buffered writes are first
stored as reservation in the pagecache, and only become resident in
the extents btree after writeback completes.
Update the fiemap implementation to process holes between extents by
scanning pagecache for data, via seek data/hole. If a valid data
range is found over a hole in the extent btree, fake up an extent
key and flag the extent as delalloc for reporting to userspace.
Note that this does not necessarily change behavior for the case
where there is dirty pagecache over already written extents, where
when in COW mode, writeback will allocate new blocks for the
underlying ranges. The existing behavior is consistent with btrfs
and it is recommended to use the sync flag for the most up to date
extent state from fiemap.
Signed-off-by: Brian Foster <bfoster@redhat.com>
|
|
The bulk of the loop in bch2_fiemap() involves processing the
current extent key from the iter, including following indirections
and trimming the extent size and such. This patch makes a few
changes to reduce the size of the loop and facilitate future changes
to support delalloc extents.
Define a new bch_fiemap_extent structure to wrap the bkey buffer
that holds the extent key to report to userspace along with
associated fiemap flags. Update bch2_fill_extent() to take the
bch_fiemap_extent as a param instead of the individual fields.
Finally, lift the bulk of the extent processing into a
bch2_fiemap_extent() helper that takes the current key and formats
the bch_fiemap_extent appropriately for the fill function.
No functional changes intended by this patch.
Signed-off-by: Brian Foster <bfoster@redhat.com>
|
|
Signed-off-by: Brian Foster <bfoster@redhat.com>
|
|
FIEMAP_FLAG_SYNC handling was deliberately moved into core code in
commit 45dd052e67ad ("fs: handle FIEMAP_FLAG_SYNC in fiemap_prep"),
released in kernel v5.8. Update bcachefs accordingly.
Signed-off-by: Brian Foster <bfoster@redhat.com>
|
|
At the end of the inode, on an extents iterator, peek_slot() has to
advance to the next position to avoid returning a 0 size extent, which
is not allowed.
Changing iter->pos confuses peek_prev(), but we don't need to call
peek_slot() in this case.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
The issue this assert is guarding against is that in
BTREE_ITER_filter_snapshots mode we only want to be iterating within a
single inode number - if we iterate into another inode number with keys
for a different snapshot tree, we'll loop arbitrarily long before
finding a key we can return.
This comes up in the unit tests, where we're using inode 0 for our test
keys.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
The peek_end() tests expect an empty btree.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
If we aren't mounting with the correct degraded option, it's helpful to
know that before we fail to mount degraded.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
casefolding results in additional aliases on lookup for the
non-casefolded names - these need invalidating on unlink.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Add casefolding to bch2_lookup_trans:
During the delay between when casefolding was written and when it was
merged, the main filesystem lookup path grew self healing - which meant
it was no longer using bch2_dirent_lookup_trans(), where casefolding on
lookups happens.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
inode_operations.fileattr_(get|set) didn't exist when the various flag
ioctls where implemented - but they do now, which means we can delete a
bunch of ioctl code in favor of standard VFS level wrappers.
Closes: https://lore.kernel.org/linux-bcachefs/7ltgrgqgfummyrlvw7hnfhnu42rfiamoq3lpcvrjnlyytldmzp@yazbhusnztqn/
Cc: Petr Vorel <pvorel@suse.cz>
Cc: Andrea Cervesato <andrea.cervesato@suse.de>
Cc: Dave Chinner <david@fromorbit.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We had a buggy release of bcachefs-tools that wasn't properly aligning
bucket sizes.
We can't ask users to reformat - and it's easy to teach the allocator to
make sure writes are properly aligned.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Previously, copygc and rebalance weren't started until the very end of
mounting, after all recvoery passes have finished.
But copygc really should be started earlier, since it may be needed for
allocations to make forward progress. Additionally, we've been seeing
occasional bug reports where starting the kthread fails due to a pending
signal - i.e. we're getting timed out by systemd (during a version
upgrade), but we're not seeing the signal until mount is about to
complete.
Additionally, we now have copygc/rebalance explicitly wait for
check_snapshots to complete (if being run); they require that for
snapshot_is_ancestor() in the data move path.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Don't use a continue; this simplifies the next patch where
run_recovery_passes() will be responsible for waking up copygc and
rebalance at the appropriate time.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
found with the new enumerated_ref code
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We've got some reports of this happening in the wild, and need a bit
more info to debug it:
https://github.com/koverstreet/bcachefs/issues/854
https://www.reddit.com/r/bcachefs/comments/1k28kjm/surprise_soft_lockup/
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We don't require that bucket size is block size aligned (although it
should be!) - so we need to handle this in the journal code.
This fixes an assertion pop in jorunal_entry_close(), where the journal
entry overruns available space - after rounding it up to block size.
Fixes: https://github.com/koverstreet/bcachefs/issues/854
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Syzbot managed to come up with a filesystem where check/repair got
rather confused at finding a reflink pointer in the inodes btree.
Currently, the "key allowed in this btree" checks only apply at commit
time, not read time - for forwards compatibility. It seems this is too
loose.
Now, strict key type allowed checks apply:
- at commit time (no forward compatibility issues)
- for btree node pointers
- if it's a known btree, known key type, and the key type has the
"BKEY_TYPE_strict_btree_checks" flag.
This means we still have the option of using generic key types - e.g.
KEY_TYPE_error, KEY_TYPE_set - on more existing btrees in the future,
while most key types that are intended for only a specific btree get
stricter checks.
Reported-by: syzbot+baee8591f336cab0958b@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We now more often do repair automatically, without the user invoking
fsck - and sometimes that can involve fixing lots of errors, so let's
avoid flooding the dmesg log.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Reported-by: syzbot+baee8591f336cab0958b@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Don't set JOURNAL_running until we're also calling
journal_space_available() for the first time.
If JOURNAL_running is set, shutdown will write an empty journal entry -
but this will hit an assert in journal_entry_open() if we've never
called journal_space_available().
Reported-by: syzbot+53bb24d476ef8368a7f0@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Pull bcachefs fixes from Kent Overstreet:
"Usual set of small fixes/logging improvements.
One bigger user reported fix, for inode <-> dirent inconsistencies
reported in fsck, after moving a subvolume that had been snapshotted"
* tag 'bcachefs-2025-04-17' of git://evilpiepirate.org/bcachefs:
bcachefs: Fix snapshotting a subvolume, then renaming it
bcachefs: Add missing READ_ONCE() for metadata replicas
bcachefs: snapshot_node_missing is now autofix
bcachefs: Log message when incompat version requested but not enabled
bcachefs: Print version_incompat_allowed on startup
bcachefs: Silence extent_poisoned error messages
bcachefs: btree_root_unreadable_and_scan_found_nothing now AUTOFIX
bcachefs: fix bch2_dev_usage_full_read_fast()
bcachefs: Don't print data read retry success on non-errors
bcachefs: Add missing error handling
bcachefs: Prevent granting write refs when filesystem is read-only
|
|
Subvolume roots and the dirents that point to them are special; they
don't obey the normal snapshot versioning rules because they cross
snapshot boundaries.
We don't keep around older versions of subvolume dirents on rename - we
don't need to, because subvolume dirents are only visible in the parent
subvolume, and we wouldn't be able to match up the different dirent and
inode versions due to crossing the snapshot ID boundary.
That means that when we rename a subvolume, that's been snapshotted, the
older version of the subvolume root will become dangling - it won't have
a dirent that points to it.
That's expected, we just need to tell fsck that this is ok.
Fixes: https://github.com/koverstreet/bcachefs/issues/856
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
If we race with the user changing the metadata_replicas setting, this
could cause us to get an incorrectly sized disk reservation.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Let users know if incompatible features aren't enabled
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
extent poisoning is partly so that we don't keep spewing the dmesg log
when we've got unreadable data - we don't want to print these.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
This will likely mean that the btree had only one node - there was
nothing or almost nothing in it, and we should reconstruct and continue.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
One reference to bch_dev_usage wasn't updated, which meant we weren't
reading the full bch_dev_usage_full - oops.
Fixes: 955ba7b5ea03 ("bcachefs: bch_dev_usage_full")
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
We may end up in the data read retry path when reading cached data and
racing with invalidation, or on checksum error when we were reading into
a userspace buffer that might have been modified while the read was in
flight.
These aren't real errors, so we shouldn't print the 'retry success'
message.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Reported-by: syzbot+d10151bf01574a09a915@syzkaller.appspotmail.com
Signed-off-by: Alan Huang <mmpgouride@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Fix a shutdown WARNING in bch2_dev_free caused by active write I/O
references (ca->io_ref[WRITE]) on a device being freed.
The problem occurs when:
- The filesystem is marked read-only (BCH_FS_rw clear in c->flags).
- A subsequent operation (e.g., error handling for device removal)
incorrectly tries to grant write references back to a device.
- During final shutdown, the read-only flag causes the system to skip
stopping write I/O references (bch2_dev_io_ref_stop(ca, WRITE)).
- The leftover active write reference triggers the WARN_ON in
bch2_dev_free.
Prevent this by checking if the filesystem is read-only before
attempting to grant write references to a device in the problematic
code path. Ensure consistency between the filesystem state flag
and the device I/O reference state during shutdown.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Pull bcachefs fixes from Kent Overstreet:
"Mostly minor fixes.
Eric Biggers' crypto API conversion is included because of long
standing sporadic crashes - mostly, but not entirely syzbot - in the
crypto API code when calling poly1305, which have been nigh impossible
to reproduce and debug.
His rework deletes the code where we've seen the crashes, so either
it'll be a fix or we'll end up with backtraces we can debug. (Thanks
Eric!)"
* tag 'bcachefs-2025-04-10' of git://evilpiepirate.org/bcachefs:
bcachefs: Use sort_nonatomic() instead of sort()
bcachefs: Remove unnecessary softdep on xxhash
bcachefs: use library APIs for ChaCha20 and Poly1305
bcachefs: Fix duplicate "ro,read_only" in opts at startup
bcachefs: Fix UAF in bchfs_read()
bcachefs: Use cpu_to_le16 for dirent lengths
bcachefs: Fix type for parameter in journal_advance_devs_to_next_bucket
bcachefs: Fix escape sequence in prt_printf
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux
Pull CRC cleanups from Eric Biggers:
"Finish cleaning up the CRC kconfig options by removing the remaining
unnecessary prompts and an unnecessary 'default y', removing
CONFIG_LIBCRC32C, and documenting all the CRC library options"
* tag 'crc-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/linux:
lib/crc: remove CONFIG_LIBCRC32C
lib/crc: document all the CRC library kconfig options
lib/crc: remove unnecessary prompt for CONFIG_CRC_ITU_T
lib/crc: remove unnecessary prompt for CONFIG_CRC_T10DIF
lib/crc: remove unnecessary prompt for CONFIG_CRC16
lib/crc: remove unnecessary prompt for CONFIG_CRC_CCITT
lib/crc: remove unnecessary prompt for CONFIG_CRC32 and drop 'default y'
|
|
Fixes "task out to lunch" warnings during recovery on large machines
with lots of dirty data in the journal.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
As with the other algorithms, bcachefs does not access xxhash through
the crypto API. So there is no need to use a module softdep to ensure
that it is loaded.
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Just use the ChaCha20 and Poly1305 libraries instead of the clunky
crypto API. This is much simpler. It is also slightly faster, since
the libraries provide more direct access to the same
architecture-optimized ChaCha20 and Poly1305 code.
I've tested that existing encrypted bcachefs filesystems can be continue
to be accessed with this patch applied.
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Commit 3ba0240a8789 fixed a bug in the read retry path in __bch2_read(),
and changed bchfs_read() to match - to avoid a landmine if
bch2_read_extent() ever starts returning transaction restarts.
But that was incorrect, because bchfs_read() doesn't use a separate
stack allocated bvec_iter, it uses the one in the rbio being submitted.
Add a comment explaining the issue, and revert the buggy change.
Fixes: 3ba0240a8789 ("bcachefs: Fix silent short reads in data read retry path")
Reported-by: syzbot+2deb10b8dc9aae6fab67@syzkaller.appspotmail.com
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Prevent incorrect byte ordering for big-endian systems.
Signed-off-by: Gabriel Shahrouzi <gshahrouzi@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Replace u64 with __le64 to match the expected parameter type. Ensure consistency both in function calls and within the function itself.
Signed-off-by: Gabriel Shahrouzi <gshahrouzi@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
Remove backslash before format specifier. Ensure correct output.
Signed-off-by: Gabriel Shahrouzi <gshahrouzi@gmail.com>
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
|
timer_delete[_sync]() replaces del_timer[_sync](). Convert the whole tree
over and remove the historical wrapper inlines.
Conversion was done with coccinelle plus manual fixups where necessary.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
|
Now that LIBCRC32C does nothing besides select CRC32, make every option
that selects LIBCRC32C instead select CRC32 directly. Then remove
LIBCRC32C.
Reviewed-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: "Martin K. Petersen" <martin.petersen@oracle.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20250401221600.24878-8-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@google.com>
|
|
Pull more bcachefs updates from Kent Overstreet:
"More notable fixes:
- Fix for striping behaviour on tiering filesystems where replicas
exceeds durability on destination target
- Fix a race in device removal where deleting alloc info races with
the discard worker
- Some small stack usage improvements: this is just enough for KMSAN
builds to not blow the stack, more is queued up for 6.16"
* tag 'bcachefs-2025-04-03' of git://evilpiepirate.org/bcachefs:
bcachefs: Fix "journal stuck" during recovery
bcachefs: backpointer_get_key: check for null from peek_slot()
bcachefs: Fix null ptr deref in invalidate_one_bucket()
bcachefs: Fix check_snapshot_exists() restart handling
bcachefs: use nonblocking variant of print_string_as_lines in error path
bcachefs: Fix scheduling while atomic from logging changes
bcachefs: Add error handling for zlib_deflateInit2()
bcachefs: add missing selection of XARRAY_MULTI
bcachefs: bch_dev_usage_full
bcachefs: Kill btree_iter.trans
bcachefs: do_trace_key_cache_fill()
bcachefs: Split up bch_dev.io_ref
bcachefs: fix ref leak in btree_node_read_all_replicas
bcachefs: Fix null ptr deref in bch2_write_endio()
bcachefs: Fix field spanning write warning
bcachefs: Fix striping behaviour
|
|
If we crash when the journal pin fifo is completely full - i.e. we're
at the maximum number of dirty journal entries - that may put us in a
sticky situation in recovery, as journal replay will need to be able to
open new journal entries in order to get going.
bch2_fs_journal_start() already had provisions for resizing the journal
pin fifo if needed, but it needs a fudge factor to ensure there's room
for journal replay.
Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
|
