diff options
| author | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-27 09:17:06 -0700 |
|---|---|---|
| committer | Linus Torvalds <torvalds@linux-foundation.org> | 2026-08-27 09:17:06 -0700 |
| commit | 18fbf5151d2c0bfe433c7428eef03cabf5fdb2fa (patch) | |
| tree | 6e151850bf46f900bbf9dcc682e0c75d91924865 /drivers | |
| parent | 5e6ff28676dd92a608eb00eeb8d1319ad34024dc (diff) | |
| parent | 0685630fdccb62dcb0e3f44525a40578da5f6dc8 (diff) | |
| download | linux-18fbf5151d2c0bfe433c7428eef03cabf5fdb2fa.tar.gz linux-18fbf5151d2c0bfe433c7428eef03cabf5fdb2fa.zip | |
Merge tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton:
- "mm/rmap: index MAP_PRIVATE file-backed folios by anonymous pgoff"
(Lorenzo Stoakes)
Index MAP_PRIVATE file-backed folios by their anonymous page offset
to resolve confusion around reverse mapping for zeroed and CoW'd
file-backed memory.
Use this new VMA anonymous page offset tracking to eliminate index
conflicts and lay the foundation for scalable CoW performance
improvements.
- "promote mapped executable folios after first usage for MGLRU"
(Baolin Wang)
Make MGLRU's protection of mapped executable file folios more
reliable. Follow the classical LRU's logic, promoting mapped
executable file folios after their first usage to give executable
code a better chance to stay in memory and improve workload
performance.
- "mm: vmscan: fix node reclaim ignoring swappiness parameter" (Ridong
Chen)
Fix per-node proactive reclaim interface's ignoring the swappiness
parameter when CONFIG_MEMCG is disabled by consolidating
sc_swappiness() into a single function that checks
proactive_swappiness regardless of kernel configuration.
- "mm/vmscan: reduce lru_lock contention via vmstat-derived
scan-balance cost" (Usama Arif)
Reduce lru_lock contention in the reclaim path by deriving
scan-balance costs from vmstat counters rather than lock-acquired
producer updates.
Read and decay these cost signals on the reclaim side under a
dedicated per-lruvec lock, reducing total LRU lock wait time by over
60% without impacting scan throughput.
- "zram: fix zram issues reported by sashiko" (Sergey Senozhatsky)
Fix two low-risk zram bugs which Sashiko spotted in drive-by review.
- "Honor XA_FLAGS_ACCOUNT in xas_split_alloc() and charge to folio's
memcg" (Zi Yan)
Fix xas_split_alloc() by enabling target folio memcg charging during
splits and adding the missing __GFP_ACCOUNT flag for proper XArray
node memory accounting.
- "selftests/mm: use pattern matching in .gitignore" (Pratyush Mallick)
Replace hardcoded binary names in selftests/mm/.gitignore with a
generic pattern-matching rule to automatically ignore generated test
files and avoid manual updates when adding new tests.
- "mm/page_ext: remove pgdat_page_ext_init()" (Sang-Heon Jeon)
Make the incompatibility between FLATMEM and NUMA explicit in
mm/Kconfig and remove the unused pgdat_page_ext_init() function.
- "zram: fix zstd error paths and add parameter validation" (Haoqin
Huang)
Clean up zram compression backends by removing redundant error
cleanup, adding parameter and dictionary validation, auto-prefixing
algorithm error logs, and resetting parameters prior to
reinitialization.
- "zram: fix stale scan bounds after reinitialization" (Longlong Xia)
Prevent out-of-bounds slot accesses during concurrent zram resets by
moving table scan bound calculations under dev_lock in
writeback_store() and read_block_state().
- "add anon mTHP collapse test cases" (Baolin Wang)
Extend selftests helper functions to support arbitrary page orders
and add new test cases and options for mTHP collapse in khugepaged.
- "selftests/mm: Handle unsupported and transient test conditions"
(Muhammad Usama Anjum)
Update MM selftests to report a SKIP status instead of a failure when
required kernel or filesystem features are unsupported, while adding
retry logic for transient page migration errors.
- "mm/zswap: Fixes and improves the zswap shrink" (Hao Jia)
Fix the missing zswap global shrinker when CONFIG_MEMCG is disabled
and extend shrink_memcg() to support batch writeback for improved
writeback efficiency.
- "alloc_tag: introduce IOCTL-based filtering for MAP" (Suren
Baghdasaryan)
Introduce an IOCTL-based binary interface for memory allocation
profiling that enables kernel-side filtering before per-CPU counter
aggregation.
This eliminates the text-parsing overhead of /proc/allocinfo and
provides up to a 20x speedup by transferring only filtered allocation
data to userspace.
- "better block swap batching and a different take on swap_ops v5"
(Christoph Hellwig)
Refactor block swap I/O to use swap_iocb for batching instead of
single-bio requests and rebase the swap_ops interface, achieving
faster swap throughput during kernel builds.
- "mm: kmemleak: reduce transient false positives by confirming leaks"
(Catalin Marinas)
Reduce false-positive kmemleak reports by combining two kmemleak
enhancements that add a second confirmation scan and a configurable
minimum unreferenced scan count module parameter.
- "mm: kmemleak: default min_unref_scans to 2 for verbose kernels"
(Breno Leitao)
Auto-scanning kernels can generate false-positive memory leak reports
on single scans, so this patch defaults min_unref_scans to 2 when
CONFIG_DEBUG_KMEMLEAK_VERBOSE is enabled to require a second
confirming scan.
- "swap_ops updates" (Christoph Hellwig)
Batching I/O for synchronous swap devices causes performance
regressions and filesystem-based swap suffers from double-indirection
overhead. This series resolves both issues by reintroducing per-folio
writes for synchronous swap and allowing filesystems to directly
export their own swap_ops.
- "mm/khugepaged: several cleanups" (Nico Pache)
khugepaged accumulated redundant state-checking patterns and outdated
comments following mTHP integration. Introduce dedicated helpers for
PTE validation and event counting while refreshing the internal
documentation.
- "maple_tree: lock checking and clean ups" (Liam Howlett)
Syzbot reports incorrectly blame memory management exit paths for
locking bugs, maple tree erase operations risk allocation failures
without gfp flags and internal documentation lacks clarity.
Improve lock error detection, update docs, fix race and allocation
edge cases and optimize erase allocations using a fallback to
GFP_KERNEL | GFP_NOFAIL.
* tag 'mm-stable-2026-08-26-15-22' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (172 commits)
selftests/proc: make proc-maps-race work with READ_IMPLIES_EXEC
memcg: move LRU size accounting on reparenting instead of copying it
mm/vmscan: fix comment logic in balance_pgdat
maple_tree: add helper mas_make_walkable()
maple_tree: avoid extra gap calculation
maple_tree: fix argument name in header
maple_tree: change two GFP flags in tests
maple_tree: document erase and allocations better
maple_tree: avoid mas_erase() and mtree_erase() failures
maple_tree: document that erase may use GFP_KERNEL for allocations
maple_tree: catch race in mas_alloc_cyclic()
maple_tree: add bulk parent set helper
maple_tree: micro optimisation of mas_wr_store_type()
maple_tree: optimise mas_wr_node_store() when not in rcu mode
maple_tree: use prefetched value in mas_wr_store_type()
maple_tree: clarify comments on mas_nomem()
maple_tree: drop MAPLE_ALLOC_SLOTS
maple_tree: drop dead code from mas_extend_spanning_null()
maple_tree: documentation fix
maple_tree: add write lock checking with lockdep sequence numbers
...
Diffstat (limited to 'drivers')
| -rw-r--r-- | drivers/base/Kconfig | 7 | ||||
| -rw-r--r-- | drivers/base/Makefile | 1 | ||||
| -rw-r--r-- | drivers/base/arch_numa.c | 371 | ||||
| -rw-r--r-- | drivers/block/zram/backend_842.c | 10 | ||||
| -rw-r--r-- | drivers/block/zram/backend_deflate.c | 25 | ||||
| -rw-r--r-- | drivers/block/zram/backend_lz4.c | 10 | ||||
| -rw-r--r-- | drivers/block/zram/backend_lz4hc.c | 16 | ||||
| -rw-r--r-- | drivers/block/zram/backend_lzo.c | 10 | ||||
| -rw-r--r-- | drivers/block/zram/backend_lzorle.c | 10 | ||||
| -rw-r--r-- | drivers/block/zram/backend_zstd.c | 11 | ||||
| -rw-r--r-- | drivers/block/zram/zram_drv.c | 188 | ||||
| -rw-r--r-- | drivers/block/zram/zram_drv.h | 14 | ||||
| -rw-r--r-- | drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 4 | ||||
| -rw-r--r-- | drivers/gpu/drm/drm_gem_shmem_helper.c | 2 | ||||
| -rw-r--r-- | drivers/gpu/drm/i915/gem/i915_gem_shmem.c | 2 | ||||
| -rw-r--r-- | drivers/gpu/drm/panthor/panthor_gem.c | 2 | ||||
| -rw-r--r-- | drivers/gpu/drm/ttm/ttm_backup.c | 2 | ||||
| -rw-r--r-- | drivers/gpu/drm/ttm/ttm_bo_vm.c | 2 | ||||
| -rw-r--r-- | drivers/gpu/drm/xe/xe_device.c | 2 |
19 files changed, 216 insertions, 473 deletions
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 43f20ca95a2a..a9acf4575f4f 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -230,13 +230,6 @@ config GENERIC_ARCH_TOPOLOGY appropriate scaling, sysfs interface for reading capacity values at runtime. -config GENERIC_ARCH_NUMA - bool - select NUMA_MEMBLKS - help - Enable support for generic NUMA implementation. Currently, RISC-V - and ARM64 use it. - config FW_DEVLINK_SYNC_STATE_TIMEOUT bool "sync_state() behavior defaults to timeout instead of strict" help diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 8074a10183dc..435710f643a5 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -25,7 +25,6 @@ obj-$(CONFIG_PINCTRL) += pinctrl.o obj-$(CONFIG_DEV_COREDUMP) += devcoredump.o obj-$(CONFIG_GENERIC_MSI_IRQ) += platform-msi.o obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o -obj-$(CONFIG_GENERIC_ARCH_NUMA) += arch_numa.o obj-$(CONFIG_ACPI) += physical_location.o obj-y += test/ diff --git a/drivers/base/arch_numa.c b/drivers/base/arch_numa.c deleted file mode 100644 index 6476227b772c..000000000000 --- a/drivers/base/arch_numa.c +++ /dev/null @@ -1,371 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * NUMA support, based on the x86 implementation. - * - * Copyright (C) 2015 Cavium Inc. - * Author: Ganapatrao Kulkarni <gkulkarni@cavium.com> - */ - -#define pr_fmt(fmt) "NUMA: " fmt - -#include <linux/acpi.h> -#include <linux/memblock.h> -#include <linux/module.h> -#include <linux/of.h> -#include <linux/numa_memblks.h> - -#include <asm/sections.h> - -static int cpu_to_node_map[NR_CPUS] = { [0 ... NR_CPUS-1] = NUMA_NO_NODE }; - -bool numa_off; - -static __init int numa_parse_early_param(char *opt) -{ - if (!opt) - return -EINVAL; - if (str_has_prefix(opt, "off")) - numa_off = true; - if (!strncmp(opt, "fake=", 5)) - return numa_emu_cmdline(opt + 5); - - return 0; -} -early_param("numa", numa_parse_early_param); - -cpumask_var_t node_to_cpumask_map[MAX_NUMNODES]; -EXPORT_SYMBOL(node_to_cpumask_map); - -#ifdef CONFIG_DEBUG_PER_CPU_MAPS - -/* - * Returns a pointer to the bitmask of CPUs on Node 'node'. - */ -const struct cpumask *cpumask_of_node(int node) -{ - - if (node == NUMA_NO_NODE) - return cpu_all_mask; - - if (WARN_ON(node < 0 || node >= nr_node_ids)) - return cpu_none_mask; - - if (WARN_ON(node_to_cpumask_map[node] == NULL)) - return cpu_online_mask; - - return node_to_cpumask_map[node]; -} -EXPORT_SYMBOL(cpumask_of_node); - -#endif - -#ifndef CONFIG_NUMA_EMU -static void numa_update_cpu(unsigned int cpu, bool remove) -{ - int nid = cpu_to_node(cpu); - - if (nid == NUMA_NO_NODE) - return; - - if (remove) - cpumask_clear_cpu(cpu, node_to_cpumask_map[nid]); - else - cpumask_set_cpu(cpu, node_to_cpumask_map[nid]); -} - -void numa_add_cpu(unsigned int cpu) -{ - numa_update_cpu(cpu, false); -} - -void numa_remove_cpu(unsigned int cpu) -{ - numa_update_cpu(cpu, true); -} -#endif - -void numa_clear_node(unsigned int cpu) -{ - numa_remove_cpu(cpu); - set_cpu_numa_node(cpu, NUMA_NO_NODE); -} - -/* - * Allocate node_to_cpumask_map based on number of available nodes - * Requires node_possible_map to be valid. - * - * Note: cpumask_of_node() is not valid until after this is done. - * (Use CONFIG_DEBUG_PER_CPU_MAPS to check this.) - */ -static void __init setup_node_to_cpumask_map(void) -{ - int node; - - /* setup nr_node_ids if not done yet */ - if (nr_node_ids == MAX_NUMNODES) - setup_nr_node_ids(); - - /* allocate and clear the mapping */ - for (node = 0; node < nr_node_ids; node++) { - alloc_bootmem_cpumask_var(&node_to_cpumask_map[node]); - cpumask_clear(node_to_cpumask_map[node]); - } - - /* cpumask_of_node() will now work */ - pr_debug("Node to cpumask map for %u nodes\n", nr_node_ids); -} - -/* - * Set the cpu to node and mem mapping - */ -void numa_store_cpu_info(unsigned int cpu) -{ - set_cpu_numa_node(cpu, cpu_to_node_map[cpu]); -} - -void __init early_map_cpu_to_node(unsigned int cpu, int nid) -{ - /* fallback to node 0 */ - if (nid < 0 || nid >= MAX_NUMNODES || numa_off) - nid = 0; - - cpu_to_node_map[cpu] = nid; - - /* - * We should set the numa node of cpu0 as soon as possible, because it - * has already been set up online before. cpu_to_node(0) will soon be - * called. - */ - if (!cpu) - set_cpu_numa_node(cpu, nid); -} - -#ifdef CONFIG_HAVE_SETUP_PER_CPU_AREA -unsigned long __per_cpu_offset[NR_CPUS] __read_mostly; -EXPORT_SYMBOL(__per_cpu_offset); - -int early_cpu_to_node(int cpu) -{ - return cpu_to_node_map[cpu]; -} - -static int __init pcpu_cpu_distance(unsigned int from, unsigned int to) -{ - return node_distance(early_cpu_to_node(from), early_cpu_to_node(to)); -} - -void __init setup_per_cpu_areas(void) -{ - unsigned long delta; - unsigned int cpu; - int rc = -EINVAL; - - if (pcpu_chosen_fc != PCPU_FC_PAGE) { - /* - * Always reserve area for module percpu variables. That's - * what the legacy allocator did. - */ - rc = pcpu_embed_first_chunk(PERCPU_MODULE_RESERVE, - PERCPU_DYNAMIC_RESERVE, PAGE_SIZE, - pcpu_cpu_distance, - early_cpu_to_node); -#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK - if (rc < 0) - pr_warn("PERCPU: %s allocator failed (%d), falling back to page size\n", - pcpu_fc_names[pcpu_chosen_fc], rc); -#endif - } - -#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK - if (rc < 0) - rc = pcpu_page_first_chunk(PERCPU_MODULE_RESERVE, early_cpu_to_node); -#endif - if (rc < 0) - panic("Failed to initialize percpu areas (err=%d).", rc); - - delta = (unsigned long)pcpu_base_addr - (unsigned long)__per_cpu_start; - for_each_possible_cpu(cpu) - __per_cpu_offset[cpu] = delta + pcpu_unit_offsets[cpu]; -} -#endif - -/* - * Initialize NODE_DATA for a node on the local memory - */ -static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) -{ - if (start_pfn >= end_pfn) - pr_info("Initmem setup node %d [<memory-less node>]\n", nid); - - alloc_node_data(nid); - - NODE_DATA(nid)->node_id = nid; - NODE_DATA(nid)->node_start_pfn = start_pfn; - NODE_DATA(nid)->node_spanned_pages = end_pfn - start_pfn; -} - -static int __init numa_register_nodes(void) -{ - int nid; - - /* Check the validity of the memblock/node mapping */ - if (!memblock_validate_numa_coverage(0)) - return -EINVAL; - - /* Finally register nodes. */ - for_each_node_mask(nid, numa_nodes_parsed) { - unsigned long start_pfn, end_pfn; - - get_pfn_range_for_nid(nid, &start_pfn, &end_pfn); - setup_node_data(nid, start_pfn, end_pfn); - node_set_online(nid); - } - - return 0; -} - -static int __init numa_init(int (*init_func)(void)) -{ - int ret; - - ret = numa_memblks_init(init_func, /* memblock_force_top_down */ false); - if (ret < 0) - goto out_free_distance; - - if (nodes_empty(numa_nodes_parsed)) { - pr_info("No NUMA configuration found\n"); - ret = -EINVAL; - goto out_free_distance; - } - - ret = numa_register_nodes(); - if (ret < 0) - goto out_free_distance; - - setup_node_to_cpumask_map(); - - return 0; -out_free_distance: - numa_reset_distance(); - return ret; -} - -/** - * dummy_numa_init() - Fallback dummy NUMA init - * - * Used if there's no underlying NUMA architecture, NUMA initialization - * fails, or NUMA is disabled on the command line. - * - * Must online at least one node (node 0) and add memory blocks that cover all - * allowed memory. It is unlikely that this function fails. - * - * Return: 0 on success, -errno on failure. - */ -static int __init dummy_numa_init(void) -{ - phys_addr_t start = memblock_start_of_DRAM(); - phys_addr_t end = memblock_end_of_DRAM() - 1; - int ret; - - if (numa_off) - pr_info("NUMA disabled\n"); /* Forced off on command line. */ - pr_info("Faking a node at [mem %pap-%pap]\n", &start, &end); - - ret = numa_add_memblk(0, start, end + 1); - if (ret) { - pr_err("NUMA init failed\n"); - return ret; - } - - numa_off = true; - return 0; -} - -#ifdef CONFIG_ACPI_NUMA -static int __init arch_acpi_numa_init(void) -{ - int ret; - - ret = acpi_numa_init(); - if (ret) { - pr_debug("Failed to initialise from firmware\n"); - return ret; - } - - return srat_disabled() ? -EINVAL : 0; -} -#else -static int __init arch_acpi_numa_init(void) -{ - return -EOPNOTSUPP; -} -#endif - -/** - * arch_numa_init() - Initialize NUMA - * - * Try each configured NUMA initialization method until one succeeds. The - * last fallback is dummy single node config encompassing whole memory. - */ -void __init arch_numa_init(void) -{ - if (!numa_off) { - if (!acpi_disabled && !numa_init(arch_acpi_numa_init)) - return; - if (acpi_disabled && !numa_init(of_numa_init)) - return; - } - - numa_init(dummy_numa_init); -} - -#ifdef CONFIG_NUMA_EMU -void __init numa_emu_update_cpu_to_node(int *emu_nid_to_phys, - unsigned int nr_emu_nids) -{ - int i, j; - - /* - * Transform cpu_to_node_map table to use emulated nids by - * reverse-mapping phys_nid. The maps should always exist but fall - * back to zero just in case. - */ - for (i = 0; i < ARRAY_SIZE(cpu_to_node_map); i++) { - if (cpu_to_node_map[i] == NUMA_NO_NODE) - continue; - for (j = 0; j < nr_emu_nids; j++) - if (cpu_to_node_map[i] == emu_nid_to_phys[j]) - break; - cpu_to_node_map[i] = j < nr_emu_nids ? j : 0; - } -} - -u64 __init numa_emu_dma_end(void) -{ - return memblock_start_of_DRAM() + SZ_4G; -} - -void debug_cpumask_set_cpu(unsigned int cpu, int node, bool enable) -{ - struct cpumask *mask; - - if (node == NUMA_NO_NODE) - return; - - mask = node_to_cpumask_map[node]; - if (!cpumask_available(mask)) { - pr_err("node_to_cpumask_map[%i] NULL\n", node); - dump_stack(); - return; - } - - if (enable) - cpumask_set_cpu(cpu, mask); - else - cpumask_clear_cpu(cpu, mask); - - pr_debug("%s cpu %d node %d: mask now %*pbl\n", - enable ? "numa_add_cpu" : "numa_remove_cpu", - cpu, node, cpumask_pr_args(mask)); -} -#endif /* CONFIG_NUMA_EMU */ diff --git a/drivers/block/zram/backend_842.c b/drivers/block/zram/backend_842.c index 10d9d5c60f53..3846a04c69d7 100644 --- a/drivers/block/zram/backend_842.c +++ b/drivers/block/zram/backend_842.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#define pr_fmt(fmt) "842: " fmt + #include <linux/kernel.h> #include <linux/slab.h> #include <linux/sw842.h> @@ -13,6 +15,14 @@ static void release_params_842(struct zcomp_params *params) static int setup_params_842(struct zcomp_params *params) { + if (params->dict_sz) { + pr_err("dictionary is not supported\n"); + return -EOPNOTSUPP; + } + if (params->level != ZCOMP_PARAM_NOT_SET) { + pr_err("compression level is not supported\n"); + return -EOPNOTSUPP; + } return 0; } diff --git a/drivers/block/zram/backend_deflate.c b/drivers/block/zram/backend_deflate.c index f92a52a720d1..f71b11bcac78 100644 --- a/drivers/block/zram/backend_deflate.c +++ b/drivers/block/zram/backend_deflate.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#define pr_fmt(fmt) "deflate: " fmt + #include <linux/kernel.h> #include <linux/slab.h> #include <linux/vmalloc.h> @@ -22,10 +24,29 @@ static void deflate_release_params(struct zcomp_params *params) static int deflate_setup_params(struct zcomp_params *params) { - if (params->level == ZCOMP_PARAM_NOT_SET) + if (params->dict_sz) { + pr_err("dictionary is not supported\n"); + return -EOPNOTSUPP; + } + + if (params->level == ZCOMP_PARAM_NOT_SET) { params->level = Z_DEFAULT_COMPRESSION; - if (params->deflate.winbits == ZCOMP_PARAM_NOT_SET) + } else if (params->level < Z_DEFAULT_COMPRESSION || + params->level > Z_BEST_COMPRESSION) { + pr_err("invalid compression level %d\n", params->level); + return -EINVAL; + } + + if (params->deflate.winbits == ZCOMP_PARAM_NOT_SET) { params->deflate.winbits = DEFLATE_DEF_WINBITS; + } else { + s32 wb = params->deflate.winbits; + + if ((wb < -15 || wb > -9) && (wb < 9 || wb > 15)) { + pr_err("invalid winbits %d\n", wb); + return -EINVAL; + } + } return 0; } diff --git a/drivers/block/zram/backend_lz4.c b/drivers/block/zram/backend_lz4.c index c449d511ba86..1e28104ad964 100644 --- a/drivers/block/zram/backend_lz4.c +++ b/drivers/block/zram/backend_lz4.c @@ -1,3 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#define pr_fmt(fmt) "lz4: " fmt + #include <linux/kernel.h> #include <linux/lz4.h> #include <linux/slab.h> @@ -28,8 +32,12 @@ static int lz4_setup_params(struct zcomp_params *params) LZ4_stream_t *dict_stream; int ret; - if (params->level == ZCOMP_PARAM_NOT_SET) + if (params->level == ZCOMP_PARAM_NOT_SET) { params->level = LZ4_ACCELERATION_DEFAULT; + } else if (params->level < LZ4_ACCELERATION_DEFAULT) { + pr_err("invalid compression level %d\n", params->level); + return -EINVAL; + } if (!params->dict || !params->dict_sz) return 0; diff --git a/drivers/block/zram/backend_lz4hc.c b/drivers/block/zram/backend_lz4hc.c index f6a336acfe20..d8aa01bb258f 100644 --- a/drivers/block/zram/backend_lz4hc.c +++ b/drivers/block/zram/backend_lz4hc.c @@ -1,3 +1,7 @@ +// SPDX-License-Identifier: GPL-2.0-or-later + +#define pr_fmt(fmt) "lz4hc: " fmt + #include <linux/kernel.h> #include <linux/lz4.h> #include <linux/slab.h> @@ -18,8 +22,18 @@ static void lz4hc_release_params(struct zcomp_params *params) static int lz4hc_setup_params(struct zcomp_params *params) { - if (params->level == ZCOMP_PARAM_NOT_SET) + if (params->level == ZCOMP_PARAM_NOT_SET) { params->level = LZ4HC_DEFAULT_CLEVEL; + } else if (params->level < 1 || params->level > LZ4HC_MAX_CLEVEL) { + /* + * Use < 1 rather than < LZ4HC_MIN_CLEVEL here because + * LZ4HC_compress_generic() only clamps levels below 1 + * (levels 1 and 2 are valid). LZ4HC_MIN_CLEVEL (3) is + * advisory and not enforced by the library. + */ + pr_err("invalid compression level %d\n", params->level); + return -EINVAL; + } return 0; } diff --git a/drivers/block/zram/backend_lzo.c b/drivers/block/zram/backend_lzo.c index 4c906beaae6b..d83f92cf757c 100644 --- a/drivers/block/zram/backend_lzo.c +++ b/drivers/block/zram/backend_lzo.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#define pr_fmt(fmt) "lzo: " fmt + #include <linux/kernel.h> #include <linux/slab.h> #include <linux/lzo.h> @@ -12,6 +14,14 @@ static void lzo_release_params(struct zcomp_params *params) static int lzo_setup_params(struct zcomp_params *params) { + if (params->dict_sz) { + pr_err("dictionary is not supported\n"); + return -EOPNOTSUPP; + } + if (params->level != ZCOMP_PARAM_NOT_SET) { + pr_err("compression level is not supported\n"); + return -EOPNOTSUPP; + } return 0; } diff --git a/drivers/block/zram/backend_lzorle.c b/drivers/block/zram/backend_lzorle.c index 10640c96cbfc..1b120d062c92 100644 --- a/drivers/block/zram/backend_lzorle.c +++ b/drivers/block/zram/backend_lzorle.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#define pr_fmt(fmt) "lzo-rle: " fmt + #include <linux/kernel.h> #include <linux/slab.h> #include <linux/lzo.h> @@ -12,6 +14,14 @@ static void lzorle_release_params(struct zcomp_params *params) static int lzorle_setup_params(struct zcomp_params *params) { + if (params->dict_sz) { + pr_err("dictionary is not supported\n"); + return -EOPNOTSUPP; + } + if (params->level != ZCOMP_PARAM_NOT_SET) { + pr_err("compression level is not supported\n"); + return -EOPNOTSUPP; + } return 0; } diff --git a/drivers/block/zram/backend_zstd.c b/drivers/block/zram/backend_zstd.c index d00b548056dc..08da3810cffd 100644 --- a/drivers/block/zram/backend_zstd.c +++ b/drivers/block/zram/backend_zstd.c @@ -1,5 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-or-later +#define pr_fmt(fmt) "zstd: " fmt + #include <linux/kernel.h> #include <linux/slab.h> #include <linux/vmalloc.h> @@ -58,8 +60,13 @@ static int zstd_setup_params(struct zcomp_params *params) return -ENOMEM; params->drv_data = zp; - if (params->level == ZCOMP_PARAM_NOT_SET) + if (params->level == ZCOMP_PARAM_NOT_SET) { params->level = zstd_default_clevel(); + } else if (params->level < zstd_min_clevel() || + params->level > zstd_max_clevel()) { + pr_err("invalid compression level %d\n", params->level); + goto error; + } zp->cprm = zstd_get_params(params->level, PAGE_SIZE); @@ -85,7 +92,6 @@ static int zstd_setup_params(struct zcomp_params *params) return 0; error: - zstd_release_params(params); return -EINVAL; } @@ -161,7 +167,6 @@ static int zstd_create(struct zcomp_params *params, struct zcomp_ctx *ctx) return 0; error: - zstd_release_params(params); zstd_destroy(ctx); return -EINVAL; } diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c index 4bfe63a5225d..a9b3bb1d3bef 100644 --- a/drivers/block/zram/zram_drv.c +++ b/drivers/block/zram/zram_drv.c @@ -56,7 +56,7 @@ static size_t huge_class_size; static const struct block_device_operations zram_devops; -static void slot_free(struct zram *zram, u32 index); +static void slot_free(struct zram *zram, unsigned long index); /* * entry locking rules: @@ -70,11 +70,11 @@ static void slot_free(struct zram *zram, u32 index); * 4) Use TRY lock variant when in atomic context * - must check return value and handle locking failers */ -static __must_check bool slot_trylock(struct zram *zram, u32 index) +static __must_check bool slot_trylock(struct zram *zram, unsigned long index) { unsigned long *lock = &zram->table[index].__lock; - if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK, lock)) { + if (!test_and_set_bit_lock(ZRAM_ENTRY_LOCK_BIT, lock)) { mutex_acquire(&zram->table_lock_map, 0, 1, _RET_IP_); lock_acquired(&zram->table_lock_map, _RET_IP_); return true; @@ -83,21 +83,21 @@ static __must_check bool slot_trylock(struct zram *zram, u32 index) return false; } -static void slot_lock(struct zram *zram, u32 index) +static void slot_lock(struct zram *zram, unsigned long index) { unsigned long *lock = &zram->table[index].__lock; mutex_acquire(&zram->table_lock_map, 0, 0, _RET_IP_); - wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK, TASK_UNINTERRUPTIBLE); + wait_on_bit_lock(lock, ZRAM_ENTRY_LOCK_BIT, TASK_UNINTERRUPTIBLE); lock_acquired(&zram->table_lock_map, _RET_IP_); } -static void slot_unlock(struct zram *zram, u32 index) +static void slot_unlock(struct zram *zram, unsigned long index) { unsigned long *lock = &zram->table[index].__lock; mutex_release(&zram->table_lock_map, _RET_IP_); - clear_and_wake_up_bit(ZRAM_ENTRY_LOCK, lock); + clear_and_wake_up_bit(ZRAM_ENTRY_LOCK_BIT, lock); } static inline bool init_done(struct zram *zram) @@ -110,55 +110,56 @@ static inline struct zram *dev_to_zram(struct device *dev) return (struct zram *)dev_to_disk(dev)->private_data; } -static unsigned long get_slot_handle(struct zram *zram, u32 index) +static unsigned long get_slot_handle(struct zram *zram, unsigned long index) { return zram->table[index].handle; } -static void set_slot_handle(struct zram *zram, u32 index, unsigned long handle) +static void set_slot_handle(struct zram *zram, unsigned long index, + unsigned long handle) { zram->table[index].handle = handle; } -static bool test_slot_flag(struct zram *zram, u32 index, +static bool test_slot_flag(struct zram *zram, unsigned long index, enum zram_pageflags flag) { return zram->table[index].attr.flags & BIT(flag); } -static void set_slot_flag(struct zram *zram, u32 index, +static void set_slot_flag(struct zram *zram, unsigned long index, enum zram_pageflags flag) { zram->table[index].attr.flags |= BIT(flag); } -static void clear_slot_flag(struct zram *zram, u32 index, +static void clear_slot_flag(struct zram *zram, unsigned long index, enum zram_pageflags flag) { zram->table[index].attr.flags &= ~BIT(flag); } -static size_t get_slot_size(struct zram *zram, u32 index) +static size_t get_slot_size(struct zram *zram, unsigned long index) { return zram->table[index].attr.flags & (BIT(ZRAM_FLAG_SHIFT) - 1); } -static void set_slot_size(struct zram *zram, u32 index, size_t size) +static void set_slot_size(struct zram *zram, unsigned long index, size_t size) { unsigned long flags = zram->table[index].attr.flags >> ZRAM_FLAG_SHIFT; zram->table[index].attr.flags = (flags << ZRAM_FLAG_SHIFT) | size; } -static inline bool slot_allocated(struct zram *zram, u32 index) +static inline bool slot_allocated(struct zram *zram, unsigned long index) { return get_slot_size(zram, index) || test_slot_flag(zram, index, ZRAM_SAME) || test_slot_flag(zram, index, ZRAM_WB); } -static inline void set_slot_comp_priority(struct zram *zram, u32 index, - u32 prio) +static inline void set_slot_comp_priority(struct zram *zram, + unsigned long index, u32 prio) { prio &= ZRAM_COMP_PRIORITY_MASK; /* @@ -170,14 +171,14 @@ static inline void set_slot_comp_priority(struct zram *zram, u32 index, zram->table[index].attr.flags |= (prio << ZRAM_COMP_PRIORITY_BIT1); } -static inline u32 get_slot_comp_priority(struct zram *zram, u32 index) +static inline u32 get_slot_comp_priority(struct zram *zram, unsigned long index) { u32 prio = zram->table[index].attr.flags >> ZRAM_COMP_PRIORITY_BIT1; return prio & ZRAM_COMP_PRIORITY_MASK; } -static void mark_slot_accessed(struct zram *zram, u32 index) +static void mark_slot_accessed(struct zram *zram, unsigned long index) { clear_slot_flag(zram, index, ZRAM_IDLE); clear_slot_flag(zram, index, ZRAM_PP_SLOT); @@ -284,7 +285,7 @@ static void release_pp_ctl(struct zram *zram, struct zram_pp_ctl *ctl) } static bool place_pp_slot(struct zram *zram, struct zram_pp_ctl *ctl, - u32 index) + unsigned long index) { struct zram_pp_slot *pps; u32 bid; @@ -418,7 +419,7 @@ static void mark_idle(struct zram *zram, ktime_t cutoff) { int is_idle = 1; unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; - int index; + unsigned long index; for (index = 0; index < nr_pages; index++) { /* @@ -485,8 +486,9 @@ static ssize_t idle_store(struct device *dev, struct device_attribute *attr, #define INVALID_BDEV_BLOCK (~0UL) static int read_from_zspool_raw(struct zram *zram, struct page *page, - u32 index); -static int read_from_zspool(struct zram *zram, struct page *page, u32 index); + unsigned long index); +static int read_from_zspool(struct zram *zram, struct page *page, + unsigned long index); struct zram_wb_ctl { /* idle list is accessed only by the writeback task, no concurency */ @@ -522,7 +524,7 @@ struct zram_rb_req { /* error status (sync read) */ int error; }; - u32 index; + unsigned long index; }; #define FOUR_K(x) ((x) * (1 << (PAGE_SHIFT - 12))) @@ -910,7 +912,7 @@ static void zram_account_writeback_submit(struct zram *zram) static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) { - u32 index = req->pps->index; + unsigned long index = req->pps->index; int err; err = blk_status_to_errno(req->bio.bi_status); @@ -1032,7 +1034,7 @@ static int zram_writeback_slots(struct zram *zram, struct zram_wb_req *req = NULL; struct zram_pp_slot *pps; int ret = 0, err = 0; - u32 index = 0; + unsigned long index = 0; while ((pps = select_pp_slot(ctl))) { if (zram->wb_limit_enable && !zram->bd_wb_limit) { @@ -1198,7 +1200,7 @@ static void scan_slots_for_writeback(struct zram *zram, u32 mode, unsigned long lo, unsigned long hi, struct zram_pp_ctl *ctl) { - u32 index = lo; + unsigned long index = lo; while (index < hi) { bool ok = true; @@ -1235,8 +1237,8 @@ static ssize_t writeback_store(struct device *dev, const char *buf, size_t len) { struct zram *zram = dev_to_zram(dev); - u64 nr_pages = zram->disksize >> PAGE_SHIFT; - unsigned long lo = 0, hi = nr_pages; + unsigned long nr_pages; + unsigned long lo = 0, hi; struct zram_pp_ctl *pp_ctl = NULL; struct zram_wb_ctl *wb_ctl = NULL; char *args, *param, *val; @@ -1250,6 +1252,9 @@ static ssize_t writeback_store(struct device *dev, if (!zram->backing_dev) return -ENODEV; + nr_pages = zram->disksize >> PAGE_SHIFT; + hi = nr_pages; + pp_ctl = init_pp_ctl(); if (!pp_ctl) return -ENOMEM; @@ -1333,7 +1338,8 @@ out: return ret; } -static int decompress_bdev_page(struct zram *zram, struct page *page, u32 index) +static int decompress_bdev_page(struct zram *zram, struct page *page, + unsigned long index) { struct zcomp_strm *zstrm; unsigned int size; @@ -1375,7 +1381,7 @@ static void zram_deferred_decompress(struct work_struct *w) struct zram_rb_req *req = container_of(w, struct zram_rb_req, work); struct page *page = bio_first_page_all(req->bio); struct zram *zram = req->zram; - u32 index = req->index; + unsigned long index = req->index; int ret; ret = decompress_bdev_page(zram, page, index); @@ -1426,7 +1432,7 @@ static void zram_async_read_endio(struct bio *bio) } static int read_from_bdev_async(struct zram *zram, struct page *page, - u32 index, unsigned long blk_idx, + unsigned long index, unsigned long blk_idx, struct bio *parent) { struct zram_rb_req *req; @@ -1476,8 +1482,8 @@ static void zram_sync_read(struct work_struct *w) * chained IO with parent IO in same context, it's a deadlock. To avoid that, * use a worker thread context. */ -static int read_from_bdev_sync(struct zram *zram, struct page *page, u32 index, - unsigned long blk_idx) +static int read_from_bdev_sync(struct zram *zram, struct page *page, + unsigned long index, unsigned long blk_idx) { struct zram_rb_req req; @@ -1496,8 +1502,9 @@ static int read_from_bdev_sync(struct zram *zram, struct page *page, u32 index, return decompress_bdev_page(zram, page, index); } -static int read_from_bdev(struct zram *zram, struct page *page, u32 index, - unsigned long blk_idx, struct bio *parent) +static int read_from_bdev(struct zram *zram, struct page *page, + unsigned long index, unsigned long blk_idx, + struct bio *parent) { atomic64_inc(&zram->stats.bd_reads); if (!parent) { @@ -1509,8 +1516,9 @@ static int read_from_bdev(struct zram *zram, struct page *page, u32 index, } #else static inline void reset_bdev(struct zram *zram) {}; -static int read_from_bdev(struct zram *zram, struct page *page, u32 index, - unsigned long blk_idx, struct bio *parent) +static int read_from_bdev(struct zram *zram, struct page *page, + unsigned long index, unsigned long blk_idx, + struct bio *parent) { return -EIO; } @@ -1538,9 +1546,10 @@ static ssize_t read_block_state(struct file *file, char __user *buf, size_t count, loff_t *ppos) { char *kbuf; - ssize_t index, written = 0; + unsigned long index; + ssize_t written = 0; struct zram *zram = file->private_data; - unsigned long nr_pages = zram->disksize >> PAGE_SHIFT; + unsigned long nr_pages; kbuf = kvmalloc(count, GFP_KERNEL); if (!kbuf) @@ -1552,6 +1561,8 @@ static ssize_t read_block_state(struct file *file, char __user *buf, return -EINVAL; } + nr_pages = zram->disksize >> PAGE_SHIFT; + for (index = *ppos; index < nr_pages; index++) { int copied; @@ -1560,7 +1571,7 @@ static ssize_t read_block_state(struct file *file, char __user *buf, goto next; copied = snprintf(kbuf + written, count, - "%12zd %12u.%06d %c%c%c%c%c%c\n", + "%12lu %12u.%06d %c%c%c%c%c%c\n", index, zram->table[index].attr.ac_time, 0, test_slot_flag(zram, index, ZRAM_SAME) ? 's' : '.', test_slot_flag(zram, index, ZRAM_WB) ? 'w' : '.', @@ -1652,6 +1663,17 @@ static void comp_algorithm_set(struct zram *zram, u32 prio, const char *alg) zram->comp_algs[prio] = alg; } +static void comp_params_reset(struct zram *zram, u32 prio) +{ + struct zcomp_params *params = &zram->params[prio]; + + vfree(params->dict); + params->level = ZCOMP_PARAM_NOT_SET; + params->deflate.winbits = ZCOMP_PARAM_NOT_SET; + params->dict_sz = 0; + params->dict = NULL; +} + static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) { const char *alg; @@ -1672,20 +1694,10 @@ static int __comp_algorithm_store(struct zram *zram, u32 prio, const char *buf) } comp_algorithm_set(zram, prio, alg); + comp_params_reset(zram, prio); return 0; } -static void comp_params_reset(struct zram *zram, u32 prio) -{ - struct zcomp_params *params = &zram->params[prio]; - - vfree(params->dict); - params->level = ZCOMP_PARAM_NOT_SET; - params->deflate.winbits = ZCOMP_PARAM_NOT_SET; - params->dict_sz = 0; - params->dict = NULL; -} - static int comp_params_store(struct zram *zram, u32 prio, s32 level, const char *dict_path, struct deflate_params *deflate_params) @@ -1700,8 +1712,16 @@ static int comp_params_store(struct zram *zram, u32 prio, s32 level, INT_MAX, NULL, READING_POLICY); - if (sz < 0) + if (sz < 0) { + pr_err("failed to load dictionary %s (err=%zd)\n", + dict_path, sz); + return sz; + } + if (sz == 0) { + pr_err("failed to load dictionary %s (empty file)\n", + dict_path); return -EINVAL; + } } zram->params[prio].dict_sz = sz; @@ -1958,8 +1978,8 @@ static ssize_t debug_stat_show(struct device *dev, static void zram_meta_free(struct zram *zram, u64 disksize) { - size_t num_pages = disksize >> PAGE_SHIFT; - size_t index; + unsigned long num_pages = disksize >> PAGE_SHIFT; + unsigned long index; if (!zram->table) return; @@ -1976,7 +1996,7 @@ static void zram_meta_free(struct zram *zram, u64 disksize) static bool zram_meta_alloc(struct zram *zram, u64 disksize) { - size_t num_pages; + unsigned long num_pages; num_pages = disksize >> PAGE_SHIFT; zram->table = vzalloc(array_size(num_pages, sizeof(*zram->table))); @@ -1999,7 +2019,7 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize) return true; } -static void slot_free(struct zram *zram, u32 index) +static void slot_free(struct zram *zram, unsigned long index) { unsigned long handle; @@ -2053,7 +2073,7 @@ out: } static int read_same_filled_page(struct zram *zram, struct page *page, - u32 index) + unsigned long index) { void *mem; @@ -2064,7 +2084,7 @@ static int read_same_filled_page(struct zram *zram, struct page *page, } static int read_incompressible_page(struct zram *zram, struct page *page, - u32 index) + unsigned long index) { unsigned long handle; void *src, *dst; @@ -2079,7 +2099,8 @@ static int read_incompressible_page(struct zram *zram, struct page *page, return 0; } -static int read_compressed_page(struct zram *zram, struct page *page, u32 index) +static int read_compressed_page(struct zram *zram, struct page *page, + unsigned long index) { struct zcomp_strm *zstrm; unsigned long handle; @@ -2104,7 +2125,8 @@ static int read_compressed_page(struct zram *zram, struct page *page, u32 index) } #if defined CONFIG_ZRAM_WRITEBACK -static int read_from_zspool_raw(struct zram *zram, struct page *page, u32 index) +static int read_from_zspool_raw(struct zram *zram, struct page *page, + unsigned long index) { struct zcomp_strm *zstrm; unsigned long handle; @@ -2136,7 +2158,8 @@ static int read_from_zspool_raw(struct zram *zram, struct page *page, u32 index) * Reads (decompresses if needed) a page from zspool (zsmalloc). * Corresponding ZRAM slot should be locked. */ -static int read_from_zspool(struct zram *zram, struct page *page, u32 index) +static int read_from_zspool(struct zram *zram, struct page *page, + unsigned long index) { if (test_slot_flag(zram, index, ZRAM_SAME) || !get_slot_handle(zram, index)) @@ -2148,8 +2171,8 @@ static int read_from_zspool(struct zram *zram, struct page *page, u32 index) return read_incompressible_page(zram, page, index); } -static int zram_read_page(struct zram *zram, struct page *page, u32 index, - struct bio *parent) +static int zram_read_page(struct zram *zram, struct page *page, + unsigned long index, struct bio *parent) { int ret; @@ -2171,7 +2194,7 @@ static int zram_read_page(struct zram *zram, struct page *page, u32 index, /* Should NEVER happen. Return bio error if it does. */ if (WARN_ON(ret < 0)) - pr_err("Decompression failed! err=%d, page=%u\n", ret, index); + pr_err("Decompression failed! err=%d, page=%lu\n", ret, index); return ret; } @@ -2181,7 +2204,7 @@ static int zram_read_page(struct zram *zram, struct page *page, u32 index, * always expects a full page for the output. */ static int zram_bvec_read_partial(struct zram *zram, struct bio_vec *bvec, - u32 index, int offset) + unsigned long index, int offset) { struct page *page = alloc_page(GFP_NOIO); int ret; @@ -2196,7 +2219,7 @@ static int zram_bvec_read_partial(struct zram *zram, struct bio_vec *bvec, } static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, - u32 index, int offset, struct bio *bio) + unsigned long index, int offset, struct bio *bio) { if (is_partial_io(bvec)) return zram_bvec_read_partial(zram, bvec, index, offset); @@ -2204,7 +2227,7 @@ static int zram_bvec_read(struct zram *zram, struct bio_vec *bvec, } static int write_same_filled_page(struct zram *zram, unsigned long fill, - u32 index) + unsigned long index) { slot_lock(zram, index); slot_free(zram, index); @@ -2219,7 +2242,7 @@ static int write_same_filled_page(struct zram *zram, unsigned long fill, } static int write_incompressible_page(struct zram *zram, struct page *page, - u32 index) + unsigned long index) { unsigned long handle; void *src; @@ -2259,7 +2282,8 @@ static int write_incompressible_page(struct zram *zram, struct page *page, return 0; } -static int zram_write_page(struct zram *zram, struct page *page, u32 index) +static int zram_write_page(struct zram *zram, struct page *page, + unsigned long index) { int ret = 0; unsigned long handle; @@ -2326,7 +2350,7 @@ static int zram_write_page(struct zram *zram, struct page *page, u32 index) * This is a partial IO. Read the full page before writing the changes. */ static int zram_bvec_write_partial(struct zram *zram, struct bio_vec *bvec, - u32 index, int offset) + unsigned long index, int offset) { struct page *page = alloc_page(GFP_NOIO); int ret; @@ -2344,7 +2368,7 @@ static int zram_bvec_write_partial(struct zram *zram, struct bio_vec *bvec, } static int zram_bvec_write(struct zram *zram, struct bio_vec *bvec, - u32 index, int offset) + unsigned long index, int offset) { if (is_partial_io(bvec)) return zram_bvec_write_partial(zram, bvec, index, offset); @@ -2412,8 +2436,9 @@ next: * * Corresponding ZRAM slot should be locked. */ -static int recompress_slot(struct zram *zram, u32 index, struct page *page, - u64 *num_recomp_pages, u32 threshold, u32 prio) +static int recompress_slot(struct zram *zram, unsigned long index, + struct page *page, u64 *num_recomp_pages, + u32 threshold, u32 prio) { struct zcomp_strm *zstrm = NULL; unsigned long handle_old; @@ -2665,7 +2690,7 @@ out: static void zram_bio_discard(struct zram *zram, struct bio *bio) { size_t n = bio->bi_iter.bi_size; - u32 index = bio->bi_iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; + unsigned long index = bio->bi_iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; u32 offset = (bio->bi_iter.bi_sector & (SECTORS_PER_PAGE - 1)) << SECTOR_SHIFT; @@ -2706,7 +2731,7 @@ static void zram_bio_read(struct zram *zram, struct bio *bio) struct bvec_iter iter = bio->bi_iter; do { - u32 index = iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; + unsigned long index = iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; u32 offset = (iter.bi_sector & (SECTORS_PER_PAGE - 1)) << SECTOR_SHIFT; struct bio_vec bv = bio_iter_iovec(bio, iter); @@ -2737,7 +2762,7 @@ static void zram_bio_write(struct zram *zram, struct bio *bio) struct bvec_iter iter = bio->bi_iter; do { - u32 index = iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; + unsigned long index = iter.bi_sector >> SECTORS_PER_PAGE_SHIFT; u32 offset = (iter.bi_sector & (SECTORS_PER_PAGE - 1)) << SECTOR_SHIFT; struct bio_vec bv = bio_iter_iovec(bio, iter); @@ -2828,6 +2853,7 @@ static void zram_destroy_comps(struct zram *zram) zram->comp_algs[prio] = NULL; zram_comp_params_reset(zram); + comp_algorithm_set(zram, ZRAM_PRIMARY_COMP, default_compressor); } static void zram_reset_device(struct zram *zram) @@ -2845,13 +2871,12 @@ static void zram_reset_device(struct zram *zram) zram_destroy_comps(zram); memset(&zram->stats, 0, sizeof(zram->stats)); reset_bdev(zram); - - comp_algorithm_set(zram, ZRAM_PRIMARY_COMP, default_compressor); } static ssize_t disksize_store(struct device *dev, struct device_attribute *attr, const char *buf, size_t len) { + unsigned long num_pages; u64 disksize; struct zcomp *comp; struct zram *zram = dev_to_zram(dev); @@ -2869,6 +2894,11 @@ static ssize_t disksize_store(struct device *dev, struct device_attribute *attr, } disksize = PAGE_ALIGN(disksize); + num_pages = disksize >> PAGE_SHIFT; + /* Slots are addressed by an unsigned long index */ + if (!num_pages || ((u64)num_pages << PAGE_SHIFT) != disksize) + return -EINVAL; + if (!zram_meta_alloc(zram, disksize)) return -ENOMEM; diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h index 4fddc582f3b8..7a55d751417e 100644 --- a/drivers/block/zram/zram_drv.h +++ b/drivers/block/zram/zram_drv.h @@ -15,6 +15,7 @@ #ifndef _ZRAM_DRV_H_ #define _ZRAM_DRV_H_ +#include <asm/byteorder.h> #include <linux/rwsem.h> #include <linux/zsmalloc.h> @@ -58,6 +59,19 @@ enum zram_pageflags { }; /* + * The slot lock is a bit-wait lock on the whole __lock word, while + * flags and ac_time alias that word as two u32s. The lock bit must + * land in the slot that ZRAM_ENTRY_LOCK reserves in attr.flags; on + * 64-bit big-endian the flags word maps to the upper half of __lock, + * so the bit position has to be shifted up. + */ +#if defined(CONFIG_64BIT) && defined(__BIG_ENDIAN) +#define ZRAM_ENTRY_LOCK_BIT (ZRAM_ENTRY_LOCK + 32) +#else +#define ZRAM_ENTRY_LOCK_BIT ZRAM_ENTRY_LOCK +#endif + +/* * Allocated for each disk page. We use bit-lock (ZRAM_ENTRY_LOCK bit * of flags) to save memory. There can be plenty of entries and standard * locking primitives (e.g. mutex) will significantly increase sizeof() diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index f754a4a3a1c2..165344f7197f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -377,9 +377,9 @@ static int amdgpu_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_str /* Workaround for Thunk bug creating PROT_NONE,MAP_PRIVATE mappings * for debugger access to invisible VRAM. Should have used MAP_SHARED * instead. Clearing VM_MAYWRITE prevents the mapping from ever - * becoming writable and makes is_cow_mapping(vm_flags) false. + * becoming writable and makes vma_is_cow_mapping(vma) false. */ - if (is_cow_mapping(vma->vm_flags) && + if (vma_is_cow_mapping(vma) && !(vma->vm_flags & VM_ACCESS_FLAGS)) vm_flags_clear(vma, VM_MAYWRITE); diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index 557369d77139..d654a22a30de 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -771,7 +771,7 @@ int drm_gem_shmem_mmap(struct drm_gem_shmem_object *shmem, struct vm_area_struct return ret; } - if (is_cow_mapping(vma->vm_flags)) + if (vma_is_cow_mapping(vma)) return -EINVAL; dma_resv_lock(shmem->base.resv, NULL); diff --git a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c index 06543ae60706..ef9440166295 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_shmem.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_shmem.c @@ -325,7 +325,7 @@ void __shmem_writeback(size_t size, struct address_space *mapping) if (folio_mapped(folio)) folio_redirty_for_writepage(&wbc, folio); else - error = shmem_writeout(folio, NULL, NULL); + error = shmem_write_folio(folio); } } diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c index b090b01211ad..72908be5e144 100644 --- a/drivers/gpu/drm/panthor/panthor_gem.c +++ b/drivers/gpu/drm/panthor/panthor_gem.c @@ -761,7 +761,7 @@ static int panthor_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *v return ret; } - if (is_cow_mapping(vma->vm_flags)) + if (vma_is_cow_mapping(vma)) return -EINVAL; if (!refcount_inc_not_zero(&bo->cmap.mmap_count)) { diff --git a/drivers/gpu/drm/ttm/ttm_backup.c b/drivers/gpu/drm/ttm/ttm_backup.c index 3c067aadc52d..0c2d53a13b2a 100644 --- a/drivers/gpu/drm/ttm/ttm_backup.c +++ b/drivers/gpu/drm/ttm/ttm_backup.c @@ -160,7 +160,7 @@ ttm_backup_backup_folio(struct file *backup, struct folio *folio, if (writeback && !folio_mapped(to_folio) && folio_clear_dirty_for_io(to_folio)) { folio_set_reclaim(to_folio); - ret = shmem_writeout(to_folio, NULL, NULL); + ret = shmem_write_folio(to_folio); if (!folio_test_writeback(to_folio)) folio_clear_reclaim(to_folio); if (ret == AOP_WRITEPAGE_ACTIVATE) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 88babf435ac2..872bf444b1f0 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -489,7 +489,7 @@ static const struct vm_operations_struct ttm_bo_vm_ops = { int ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo) { /* Enforce no COW since would have really strange behavior with it. */ - if (is_cow_mapping(vma->vm_flags)) + if (vma_is_cow_mapping(vma)) return -EINVAL; drm_gem_object_get(&bo->base); diff --git a/drivers/gpu/drm/xe/xe_device.c b/drivers/gpu/drm/xe/xe_device.c index f9791c850e52..ee732e5495f7 100644 --- a/drivers/gpu/drm/xe/xe_device.c +++ b/drivers/gpu/drm/xe/xe_device.c @@ -331,7 +331,7 @@ static int xe_pci_barrier_mmap(struct file *filp, if (vma->vm_end - vma->vm_start > SZ_4K) return -EINVAL; - if (is_cow_mapping(vma->vm_flags)) + if (vma_is_cow_mapping(vma)) return -EINVAL; if (vma->vm_flags & (VM_READ | VM_EXEC)) |
