kernel/git/stable/linux.git/fs/btrfs/subpage.c, branch master

btrfs: trigger cow fixup via dirty_folio()

2026-07-30T17:28:36+00:00

The problem scenario: If we have a folio mmapped shared and then somebody does a dio read with that folio as the read destination, then it is possible that the dio will see a dirty destination page when it starts (and thus skip dirtying and just GUP pin it) but then while it is doing the read, btrfs finishes writing it back and by the endio, the folio is clean. In that case, the dio read must re-dirty the folio with aops->dirty_folio(): btrfs_check_read_bio() |- __iomap_dio_bio_end_io() from btrfs_bio_end_io() |- bio_check_pages_dirty() |- bio_dirty_fn() |- bio_release_pages(bio, true) |- __bio_release_pages(bio, mark_dirty == true) |- folio_lock() |- folio_mark_dirty() |- aops->dirty_folio() |- folio_unlock() A data block normally moves through writeback as follows: TASK folio_lock write clean -> dirty bit + delalloc folio_unlock WRITEBACK for-each-dirty-folio: folio_lock run_delalloc delalloc consumed -> dirty bit + OE submission dirty bit consumed -> writeback bit + OE folio_unlock ENDIO endio OE bytes accounted OE finish writeback -> clean; destroy OE Three critical invariants that this path maintains are: I1. Any dirty block is covered by delalloc xor an ordered extent I2. Any dirty block covered by an OE will be submitted into that OE I3. Any dirty block already submitted into an OE will not be submitted again into the same OE. These ensure that the block will be written exactly once. It is clear that not reserving delalloc for the re-dirty case violates I1. This situation, even without bs < folio_size, has long required btrfs to fixup such dirty pages during writeback with an asynchronous worker that is allowed to do this expensive work and writeback does not proceed for a folio while it is doing this work. Commit 247e743cbe6e ("Btrfs: Use async helpers to deal with pages that have been improperly dirtied") introduced the COW fixup to catch exactly this class at writeback, way back in 2008. Since then, there have been many advances to prevent most of the causes of such re-dirtying and we thought we could get away with removing the annoying cow-fixup in the hope of simplifying writeback for large folio support. Commit b2a9f217ad3f ("btrfs: remove the COW fixup mechanism") Commit 4927b141877c ("btrfs: remove folio ordered flag and subpage bitmap") Since it turns out this assumption was incorrect, as evidenced by the report and attendant reproducers, we must reintroduce the fixup concept. This is of course critically further complicated by bs < folio_size. In that case, rather than just a folio dirty bit, we have a bitmap for the dirty blocks in the folio. And the (also broken) invariant is: I4. folio dirty IFF at least one block bitmap dirty. The original report of a stall on a misinterpreted empty bitmap is exactly evidence of a violation of I4. It is exactly because of bs < folio_size we don't want to simply revert the removal patches. The original fixup was not properly bs < folio_size aware, which motivated removal in the first place. So we wish to build a bs < folio_size aware fixup. One other important detail from the old design, any normal write that happens after a re-dirty but before a fixup is racing with the cow fixup to do the delalloc reservation, therefore it must cancel the fixup state. If it arrives after the reservation exists, it will be a normal dirty overwrite. This critically informs the design in a pretty clear way. fixup requiring re-dirty has folio granularity, while cancellation has delalloc (block) granularity so while we only ever produce fixup in chunks of folios, we must be able to clear it in blocks. Therefore we must track the blocks needing fixup at block granularity. The obvious way to do this is with a new bitmap in btrfs_folio_state, but it is desirable to avoid that if possible. Unfortunately, I don't think it is possible and the reason is subtle and leans on a sort of extreme reproducer, but I think can be explained relatively succinctly. Consider a folio whose two halves will land in different ordered extents (can be accomplished with tricks using nodatasum) and a dio read is running with it as the shared mmap destination. 1. The front half: a. folio comes clean on a normal write b. dio read completes into the folio marking it fixup. c. a write comes for the previous folio for a range extending into this folio, this is a cancellation of the fixup which reserves space. d. writeback runs on the range *not* overlapping the folio. This half remains dirty but is now covered by an OE and is awaiting writeback running on its range to be submitted and finish the OE. 2. The back half: a. the folio is part of an OE that gets far enough along to clear writeback. b. dio read completes into the folio marking it fixup. After this, the folio's front half is dirty in the "normal" sense, it needs to be submitted to the OE waiting for it. It's a cancelled fixup. Meanwhile, the second half is a true fresh fixup. So at this point if we run writeback on this folio, we genuinely can't know what to do without block level information. If we submit it, we submit unreserved dirty from the back half. If we don't, we will never finish the OE waiting for it. So it's either a corruption or a deadlock. Thus, the full high level design picture: - btrfs_data_dirty_folio(): For out of band non-reserving dirties, mark still-clean blocks inside EOF dirty and set their fixup bits (the event carries no range, so every clean block is suspect). Already-dirty blocks are covered or pending and are left alone. - Writeback: skip fixup blocks and enqueue work for them - writepage_fixup(): for each fixup block do the fixup reservation in a worker, after which the blocks can be written back normally. - Typical reserving write paths cancel fixup state for the ranges they cover with btrfs_folio_cancel_fixup() Link: https://lore.kernel.org/linux-btrfs/20260721191152.101118-1-borntraeger@linux.ibm.com/ Assisted-by: LLM Reviewed-by: Qu Wenruo Signed-off-by: Boris Burkov Signed-off-by: David Sterba

btrfs: fix a regression where PAGECACHE_TAG_DIRTY is never cleared

2026-07-14T05:04:48+00:00

[BUG] The following script (already submitted as generic/798) will report incorrect dirty page numbers, with 64K page size systems and 4K fs block size: # mkfs.btrfs -s 4k -f $dev # mount $dev $mnt # xfs_io -f -c "pwrite 0 64K" -c fsync -c "cachestat 0 64K" $mnt/foobar Cached: 1, Dirty: 1, Writeback: 0, Evicted: 0, Recently Evicted: 0 Note that the dirtied page number is still 1. [CAUSE] The cachestat() goes through the XArray of the page cache, but instead of checking each folio's flag, it uses the PAGECACHE_TAG_DIRTY tag to report dirty pages. Since commit 095be159f3eb ("btrfs: unify folio dirty flag clearing"), btrfs replaced a folio_clear_dirty_for_io() call inside extent_write_cache_pages() with folio_test_dirty(). This will cause the following call sequence for the folio at file offset 0: extent_write_cache_pages() |- folio_test_dirty() | The folio is still dirty, continue to writeback. | |- extent_writepage() |- extent_writepage_io() |- submit_one_sector() for range [0, 4K) | |- btrfs_folio_clear_dirty() | |- btrfs_folio_set_writeback() | |- folio_start_writeback() | It's the first writeback block, we set the writeback | flag for the folio. | But the folio is still dirty, PAGECACHE_TAG_DIRTY is | kept | |- submit_one_sector() for range [4K, 8K) | |- btrfs_folio_clear_dirty() | |- btrfs_folio_set_writeback() | The folio already has writeback flag, no need to call | folio_start_writeback() | | ... |- submit_one_sector() for range [60K, 64K) |- btrfs_folio_clear_dirty() |- btrfs_folio_set_writeback() The folio already has writeback flag, no need to call folio_start_writeback() So the PAGECACHE_TAG_DIRTY is never cleared. Meanwhile for the old code, before that commit, the sequence looks like: extent_write_cache_pages() |- folio_clear_dirty_for_io() | The folio is still dirty, so continue to writeback. | But the folio dirty flag is cleared now. | |- extent_writepage() |- extent_writepage_io() |- submit_one_sector() for range [0, 4K) | |- btrfs_folio_clear_dirty() | |- btrfs_folio_set_writeback() | |- folio_start_writeback() | |- xas_clear(PAGECACHE_TAG) | | It's the first writeback block, we set the writeback | flag for the folio. | And the folio is not dirty, PAGECACHE_TAG_DIRTY is | cleared | |- submit_one_sector() for range [4K, 8K) | |- btrfs_folio_clear_dirty() | |- btrfs_folio_set_writeback() | The folio already has writeback flag, no need to call | folio_start_writeback() | | ... |- submit_one_sector() for range [60K, 64K) |- btrfs_folio_clear_dirty() |- btrfs_folio_set_writeback() The folio already has writeback flag, no need to call folio_start_writeback() Unlike the new code, old code will clear PAGECACHE_TAG_DIRTY for the first writeback block. There is a deeper problem, dirty and writeback folio flags are updated at very different timing. The dirty flag is only cleared when the last sub-folio block has dirty flag cleared. But the writeback flag is set when the first block starts writeback, and later blocks that go through writeback will not call folio_start_writeback() again. If we rely on folio_start_writeback() to update the PAGECACHE_TAG_DIRTY and PAGECACHE_TAG_TOWRITE, it will always be incorrect in one way or another. [FIX] Do not let folio_start_writeback() do any PAGECACHE_TAG_TOWRITE handling. Instead, manually clear both PAGECACHE_TAG_TOWRITE and PAGECACHE_TAG_DIRTY flags when the folio is no longer dirty during btrfs_subpage_set_writeback(). However this is only a hot-fix, for the long term solution we will follow iomap, by calling folio_start_writeback() immediately for the whole folio, and folio_end_writeback() after all writeback finished for the folio. Fixes: 095be159f3eb ("btrfs: unify folio dirty flag clearing") Reviewed-by: Boris Burkov Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: introduce support for huge folios

2026-06-09T10:49:26+00:00

With all the previous preparations, it's finally time to enable the huge folio support. - The max folio size Here we define BTRFS_MAX_FOLIO_SIZE, which is fixed at 2MiB. This will ensure we have a large enough but not too large folio for btrfs. This limit applies to all systems regardless of page size. Then we also define BTRFS_MAX_BLOCKS_PER_FOLIO, which depends on CONFIG_BTRFS_EXPERIMENTAL. If it's an experimental build, BTRFS_MAX_BLOCKS_PER_FOLIO is 512, otherwise it's BITS_PER_LONG. The filemap max order will be calculated using both BTRFS_MAX_FOLIO_SIZE and BTRFS_MAX_BLOCKS_PER_FOLIO. E.g. for 64K page size with 64K fs block size, the limit will be BTRFS_MAX_FOLIO_SIZE (2M), which limits the filemap max order to 5. This will be lower than the old order (6), but folios larger than 2M are rarely any better for IO performance. Meanwhile excessively large folios can cause other problems like stalling the IO pipeline for too long. For 4K page size and 4K fs block size, the limit will be increased to 2M from the old 256K. This new size is constrained by both BTRFS_MAX_FOLIO_SIZE (2M) and BTRFS_MAX_BLOCKS_PER_FOLIO (512 * 4K), allowing x86_64 to achieve huge folio support, and the filemap max order will be 9. - btrfs_bio_ctrl::submit_bitmap This will be enlarged to contain BTRFS_MAX_BLOCKS_PER_FOLIO bits, and this will be on-stack memory. This will increase on-stack memory usage by 56 bytes compared to the baseline (before the first patch in the series). - Local @delalloc_bitmap inside writepage_delalloc() Unfortunately we cannot afford to handle an allocation error here, thus again we use on-stack memory. Thus this will increase on-stack memory usage by 56 bytes again. So unfortunately this means during the delalloc window, the writeback path will have +112 bytes on-stack memory usage, and for other cases the writeback path will have +56 bytes on-stack memory usage. The +56 bytes (btrfs_bio_ctrl::submit_bitmap) can be removed after we have reworked the compression submission, so the current on-stack submit_bitmap is mostly a workaround until then. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: migrate btrfs_bio_ctrl::submit_bitmap to support larger bitmaps

2026-06-09T10:49:26+00:00

[CURRENT LIMIT] Btrfs currently only supports sub-bitmaps (e.g. dirty bitmap) no larger than BITS_PER_LONG. One call site that utilizes this limit is btrfs_bio_ctrl::submit_bitmap, which makes it very simple and straightforward to just grab an unsigned long value and assign it to submit_bitmap. Unfortunately that limit prevents us from supporting huge folios. For 4K page size and block size, a huge folio (order 9) means 512 blocks inside a 2M folio. [ENHANCEMENT] Instead of using a fixed unsigned long value, change btrfs_bio_ctrl::submit_bitmap to an unsigned long pointer. And for cases where an unsigned long can hold the whole bitmap, introduce @submit_bitmap_value, and just point that pointer to that unsigned long. Then update all direct users of bio_ctrl->submit_bitmap to use the pointer version. There are several call sites that get extra changes: - @range_bitmap inside extent_writepage_io() Which is only utilized to truncate the bitmap. Since we do not want to allocate new memory just for such temporary usage, change the original bitmap_set() and bitmap_and() into bitmap_clear() for the ranges outside of the target range. - Getting dirty subpage bitmap inside writepage_delalloc() Since we're passing an unsigned long pointer now, we need to go with different handling (bs == ps, blocks_per_folio <= BITS_PER_LONG, blocks_per_folio > BITS_PER_LONG). Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: prepare subpage operations to support more than BITS_PER_LONG sub-bitmaps

2026-06-09T10:49:26+00:00

[CURRENT LIMIT] Btrfs currently only supports sub-bitmaps (e.g. dirty bitmap) no larger than BITS_PER_LONG. That limit allows us to easily grab an unsigned long without the need to properly allocate memory for a larger bitmap. Unfortunately that limit prevents us from supporting huge folios. For 4K page size and block size, a huge folio (order 9) means 512 blocks inside a 2M folio. [ENHANCEMENT] To allow direct bitmap operations without allocating new memory, introduce two different ways to access the subpage bitmaps: - Return an unsigned long value This only happens if blocks_per_folio <= BITS_PER_LONG. We read out the sub-bitmap into an unsigned long, and return the value. This is the old existing method. This involves get_bitmap_value_##name() helper functions. And this time the helper functions are defined as inline functions instead of macros to provide better type checks. - Return a pointer where the sub-bitmap starts This only happens if blocks_per_folio >= BITS_PER_LONG. This is the new method for sub-bitmaps larger than BITS_PER_LONG. Since the sizes of sub-bitmaps are all aligned to BITS_PER_LONG, we can directly access the start word of the sub-bitmap. This involves get_bitmap_pointer_##name() helper functions. Then change the existing sub-bitmaps users to use the new helpers: - Bitmap dumping Switch between get_bitmap_value_##name() and get_bitmap_pointer_##name() depending on the sub-bitmap size. - btrfs_get_subpage_dirty_bitmap() Rename it to btrfs_get_subpage_dirty_bitmap_value() to follow the new value/pointer naming. Since we do not support huge folios yet, there is no pointer version for the dirty bitmap. Furthermore, add the support for block size == page size cases for btrfs_get_subpage_dirty_bitmap_value(), so that the caller no longer needs to check if the folio needs subpage handling. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: update the out-of-date comments on subpage

2026-06-09T10:49:25+00:00

The comments at the beginning of subpage.c are out-of-date, a lot of the limitations have been already resolved. Update them to reflect the latest status. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: remove folio ordered flag and subpage bitmap

2026-06-08T13:53:32+00:00

Btrfs has an internal flag/subpage bitmap called ordered, which is to indicate that a block has corresponding ordered extent covering it. However this requires extra synchronization between the inode ordered tree, and the folio flag/subpage bitmap, not to mention we need to maintain the extra folio flag with subpage bitmap. As a step to align btrfs_folio_state more closely to iomap_folio_state, remove the btrfs specific ordered flag/bitmap. This will also save us 64 bytes for the bitmap of a huge folio. Since we're here, also update the ASCII graph of the bitmap, as there are only 3 sub-bitmaps now, show all sub-bitmaps directly. Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: remove locked subpage bitmap

2026-06-08T13:53:32+00:00

Currently there are two members inside btrfs_folio_state that are related to locked bitmap: - locked sub-bitmap inside btrfs_folio_state::bitmaps[] The enum btrfs_bitmap_nr_locked determines the sub-bitmap. - btrfs_folio_state::nr_locked Which records how many blocks are locked inside the folio. The locked sub-bitmap is a btrfs specific per-block tracking mechanism, which is mostly for async-submission, utilized by compressed writes. The sub-bitmap itself is a super set of nr_locked, as it can provide a more reliable tracking. But the sub-bitmap itself can be pretty large for the incoming huge folio, 2M sized folio for 4K page size, meaning 512 bits for one sub-bitmap. Furthermore, in the long run compression will be reworked to get rid of async-submission completely, there is not much need for a full sub-bitmap to track the locked status. This patch removes the locked sub-bitmap and only relies on @nr_locked atomic to do the tracking. This can also save 64 bytes from btrfs_folio_state::bitmaps[] for a huge folio. This will reduce some safety checks, as previously if a block is not locked, btrfs_folio_end_lock()/btrfs_folio_end_lock_bitmap() will find out that, and skip reducing @nr_locked for that block, and avoid under-flow. But this safety net itself shouldn't be necessary in the first place. If we're unlocking a block that is not locked, it's a bug in the logic, and we should catch it, not silently ignoring it. Thus I believe the removal of the extra safety net should not be a problem. Reviewed-by: David Sterba Signed-off-by: Qu Wenruo Signed-off-by: David Sterba

btrfs: remove folio checked subpage bitmap tracking

2026-06-08T13:53:28+00:00

The folio checked flag is only utilized by the COW fixup mechanism inside btrfs. Since the COW fixup is already removed from non-experimental builds, there is no need to keep the checked subpage bitmap. This will saves us some space for large folios, for example for a single 256K sized large folio on 4K page sized systems: Old bitmap size = 6 * (256K / 4K / 8) = 48 bytes New bitmap size = 5 * (256K / 4K / 8) = 40 bytes This will be more obvious when we're going to support huge folios (order = 9). Signed-off-by: Qu Wenruo Reviewed-by: David Sterba Signed-off-by: David Sterba

Merge tag 'for-6.19-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux

2025-12-04T04:03:46+00:00

Pull btrfs updates from David Sterba: "Features: - shutdown ioctl support (needs CONFIG_BTRFS_EXPERIMENTAL for now): - set filesystem state as being shut down (also named going down in other filesystems), where all active operations return EIO and this cannot be changed until unmount - pending operations are attempted to be finished but error messages may still show up depending on where exactly the shutdown happened - scrub (and device replace) vs suspend/hibernate: - a running scrub will prevent suspend, which can be annoying as suspend is an immediate request and scrub is not critical - filesystem freezing before suspend was not sufficient as the problem was in process freezing - behaviour change: on suspend scrub and device replace are cancelled, where scrub can record the last state and continue from there; the device replace has to be restarted from the beginning - zone stats exported in sysfs, from the perspective of the filesystem this includes active, reclaimable, relocation etc zones Performance: - improvements when processing space reservation tickets by optimizing locking and shrinking critical sections, cumulative improvements in lockstat numbers show +15% Notable fixes: - use vmalloc fallback when allocating bios as high order allocations can happen with wide checksums (like sha256) - scrub will always track the last position of progress so it's not starting from zero after an error Core: - under experimental config, checksum calculations are offloaded to process context, simplifies locking and allows to remove compression write worker kthread(s): - speed improvement in direct IO throughput with buffered IO fallback is +15% when not offloaded but this is more related to internal crypto subsystem improvements - this will be probably default in the future removing the sysfs tunable - (experimental) block size > page size updates: - support more operations when not using large folios (encoded read/write and send) - raid56 - more preparations for fscrypt support Other: - more conversions to auto-cleaned variables - parameter cleanups and removals - extended warning fixes - improved printing of structured values like keys - lots of other cleanups and refactoring" * tag 'for-6.19-tag' of git://git.kernel.org/pub/scm/linux/kernel/git/kdave/linux: (147 commits) btrfs: remove unnecessary inode key in btrfs_log_all_parents() btrfs: remove redundant zero/NULL initializations in btrfs_alloc_root() btrfs: remaining BTRFS_PATH_AUTO_FREE conversions btrfs: send: do not allocate memory for xattr data when checking it exists btrfs: send: add unlikely to all unexpected overflow checks btrfs: reduce arguments to btrfs_del_inode_ref_in_log() btrfs: remove root argument from btrfs_del_dir_entries_in_log() btrfs: use test_and_set_bit() in btrfs_delayed_delete_inode_ref() btrfs: don't search back for dir inode item in INO_LOOKUP_USER btrfs: don't rewrite ret from inode_permission btrfs: add orig_logical to btrfs_bio for encryption btrfs: disable verity on encrypted inodes btrfs: disable various operations on encrypted inodes btrfs: remove redundant level reset in btrfs_del_items() btrfs: simplify leaf traversal after path release in btrfs_next_old_leaf() btrfs: optimize balance_level() path reference handling btrfs: factor out root promotion logic into promote_child_to_root() btrfs: raid56: remove the "_step" infix btrfs: raid56: enable bs > ps support btrfs: raid56: prepare finish_parity_scrub() to support bs > ps cases ...