diff options
| author | Chuyi Zhou <zhouchuyi@bytedance.com> | 2026-07-09 20:29:30 +0800 |
|---|---|---|
| committer | Thomas Gleixner <tglx@kernel.org> | 2026-07-23 12:30:31 +0200 |
| commit | c86f522bb57b257befe2ff5fbcb4b94065abb011 (patch) | |
| tree | 213cb9fadff0a769b5340cea2f208076759d3948 | |
| parent | 33c179b5020554722bcb4c5a3ff565192ac53f54 (diff) | |
| download | linux-next-c86f522bb57b257befe2ff5fbcb4b94065abb011.tar.gz linux-next-c86f522bb57b257befe2ff5fbcb4b94065abb011.zip | |
x86/mm: Cap flush_tlb_info alignment at 64 bytes
A stack allocated flush_tlb_info should keep cacheline alignment to avoid
the regression that motivated the per-CPU storage, but using
SMP_CACHE_BYTES directly can make the stack frame grow excessively on
configurations with large cache lines. This was addressed by
commit 780e0106d468 ("x86/mm/tlb: Revert "x86/mm: Align TLB
invalidation info""), where the stack consumption reached 320 bytes.
Add FLUSH_TLB_INFO_ALIGN and cap the type alignment at 64 bytes. The
existing per-CPU flush_tlb_info instance remains
DEFINE_PER_CPU_SHARED_ALIGNED(), so its per-CPU shared-cacheline
alignment is unchanged.
This prepares for moving flush_tlb_info back to stack storage without
reintroducing the old large-cacheline stack usage problem.
Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Reviewed-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Link: https://patch.msgid.link/20260709122933.4021501-12-zhouchuyi@bytedance.com
| -rw-r--r-- | arch/x86/include/asm/tlbflush.h | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 0545fe75c3fa..70098d448e99 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -4,6 +4,7 @@ #include <linux/mm_types.h> #include <linux/mmu_notifier.h> +#include <linux/minmax.h> #include <linux/sched.h> #include <asm/barrier.h> @@ -212,6 +213,12 @@ extern u16 invlpgb_count_max; extern void initialize_tlbstate_and_flush(void); /* + * Keep stack-allocated flush_tlb_info cacheline aligned, but cap the + * alignment to avoid excessive stack usage on large-cacheline systems. + */ +#define FLUSH_TLB_INFO_ALIGN MIN(SMP_CACHE_BYTES, 64) + +/* * TLB flushing: * * - flush_tlb_all() flushes all processes TLBs @@ -249,7 +256,7 @@ struct flush_tlb_info { u8 stride_shift; u8 freed_tables; u8 trim_cpumask; -}; +} __aligned(FLUSH_TLB_INFO_ALIGN); void flush_tlb_local(void); void flush_tlb_one_user(unsigned long addr); |