| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 fixes from Will Deacon:
"A mixture of ptdump, compat and MTE fixes that came in during the
merge window:
- Fix address handling of final memory region in ptdump
- Fix emulation of decrementing load/store multiple from 32-bit task
- Fix SCTLR context-switching for store-only MTE mode
- Fix numerous issues in MTE selftests"
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
selftests/arm64: Add MTE test config fragment
selftests/arm64: Fix MTE prctl TAP plan
selftests/arm64: Treat KSM merge_across_nodes as optional
selftests/arm64: Print missing MTE TAP headers
arm64: compat: Fix decrementing LDM/STM alignment emulation
arm64: process: Fix context switching MTE store-only tag check
KVM: arm64: ptdump: Flush the last region
arm64: ptdump: Make note_page_flush() range aware
|
|
Pull kvm updates from Paolo Bonzini:
"ARM64:
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when
FEAT_NV2p1 is present, guaranteeing that EL2-specific register bits
are stateful in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the
scope of those instructions remains 'in host' (i.e. L1
kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly
in the VGIC, our in-kernel generator of bugs (and sometimes
interrupts)
LoongArch:
- Advertise already-supported capabilities
- Some bug fixes about timer and MMIO
- Some hardening about interrupt injection
- Replace kvm_err() with kvm_pr_unimpl()
- Add FPU/LSX/LASX test cases for selftests
RISC-V:
- Svadu/Zicfiss/Zicfilp FWFT support for Guest
- Use try_cmpxchg for IMSIC MRIF RMW
- More arch-specific tracepoints in KVM RISC-V
- Eager page splitting when enabling dirty logging
- Optimize hfence request handling for SMP Guests
- Improve dirty log clearing by skipping zero bits in mask
- Guard HFENCE range loops against overflow
- CPU PM notifiers in KVM RISC-V for non-retentive idle states
- Fix kernel-mode vector context save/restore for Guest
s390:
- Fixes for vfio-ap
- Fixes for the gmap rework
- Fixes for vsie
- AI triggered fixes all over
- diag9c tracing
- code move preparation for the additional arm64 support
- enable CONTEXT_ANALYSIS
x86:
- Perform spring cleaning on x86.{c,h} and asm/kvm_host.h, by adding
regs.c (the kvm_cache_regs.h => regs.h is already applied) and
msrs.{c,h}, and moving relevant code out of x86.c
- Split kvm_mmu in three parts, respectively to describe the format
of page tables, walking the guest page tables and building the page
tables. Always use the same page table walker kvm->arch.gva_walk as
the entry point to convert a guest's virtual address, where the
previous code used two different kvm_mmu structs depending on
whether the walk included nested EPT/NPT or not. Make page fault
vmexits reuse the permission checking machinery that is used for
guest page faults. This is both a cleanup and a baby step towards
supporting XS/XU memory permissions
- Document some of the "fun" gotchas with the APIC base when creating
IRQCHIPs on x86
- Remove a defunct masterclock update from kvm_xen_shared_info_init().
It could result in incorrect kvmclock due to triggering an
unnecessary switch to/from masterclock mode
- Skip Xen runstate time updates if time has effectively gone
backwards, so that the guest doesn't report 100% steal time for
a very, very long time
- Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM
was updating the wrong sub-leaf, and upstream KVM will soon provide
all the information needed by userspace to populate the CPUID field
itself
- Fix a bug where KVM would walk a newly created rmap without holding
the rmap lock (or mmu_lock) during aging
- Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs
- Fix a variety of #DB priority bugs
- Fix a class of races related to enabling Hyper-V emulation on a
vCPU after the vCPU is visible to the rest of KVM
- Use static calls for nested virtualization ops
- Move more KVM-internal code out of x86's kvm_host.h
- Enumerate support for a variety of Zhaoxin instructions that don't
require explicit virtualization
- Fix missing EFER validation bugs, including in the KVM_SET_SREGS*
path
- Harden kvm_vcpu_map() against double-mapping and thus leaking
references
- Misc fixes and cleanups, e.g. for largely benign syzkaller splats
x86 (Intel):
- Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used
for IPI virtualization when the vCPU is freed, to fix a
use-after-free where hardware will write to a freed vCPU's PID
- Service local TLB flushes on a failed nested VM-Enter to fix a bug
where KVM could miss a TLB on a future, successful VM-Enter with
the same L2 VPID
- Cap the maximum value shoved into the VMX Preemption Timer to
workaround an erratum that affects all existing Intel CPUs that
support CPUID 0x15
- Fix VPID virtualization bugs where KVM would fail to flush hardware
TLBs
- Harden the TDX "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion
x86 (AMD):
- Forcefully invalidate SNP VMSA pages if their backing guest_memfd
page is zapped/invalidated, e.g. due to a PUNCH_HOLE in response to
a Page-State Change request
- Remove a dying VM from the GA Log notifier list before the VM is
actually destroyed, to fix a potential use-after-free
- While FOLL_WRITE was needed in the past to trigger CoW unsharing,
nowadays FOLL_LONGTERM does that already even without FOLL_WRITE,
and in fact, get_user_pages() actually disallows FOLL_WRITE
together with FOLL_LONGTERM. So don't pass FOLL_WRITE when
registering encrypted memory regions, i.e. when pinning SEV/SEV-ES
guest memory, to fix a regression with file-backed memory
introduced by KVM's (correct) usage of long-term pins
(This was reviewed by mm maintainers; for more information, see
commit ee1a586dd1fa "KVM: SEV: Drop FOLL_WRITE for encrypted region
registration")
- Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled
hosts to fix a data corruption issue due to the PSP driver
assigning to-be-written pages to firmware (as required by the SNP
specs)
- Unconditionally intercept ICBEP so that KVM generates the correct
guest RIP when handling an ICEBP-induced TASK_SWITCH #VMEXIT
- Harden the SNP "populate" ioctls against bad input, and to prepare
for supporting in-place private<=>shared conversion
Generic:
- Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's
debugfs
- Add a per-VM bitmap to track which vCPU IDs have been "claimed" but
for which the vCPU isn't yet online, and use the bitmap to reject
duplicate IDs before calling into arch code. This allows arch code
to consume vcpu_id without having to worry about cross-vCPU
clobbering (at least s390 and x86 have had related bugs)
- Rework the so called "prepare" and "invalidate" guest_memfd hooks
to prepare for in-place private<=>shared conversion, and clean up a
few warts along the way
Selftests:
- Automatically allocate a full page for L2 guest stacks on x86
instead of requiring test-specific L1 guest code to carve out a
portion of the L1 stack for L2 usage, and to ensure the L2 stack
also adheres to the x86-64 calling convention ABI
- Add a selftest to verify {Guest,Host}-Only behavior in x86's
mediated PMU
- Clean up nested SVM's handling of GPRs on L2<=>L1 transitions,
reuse the functionality for nested VMX, and drop the ucall hack
that was fudging around the lack of GPR switching on nVMX
- Add a stress test to verify KVM doesn't clobber/drop #PF state,
e.g. CR2, across save/restore, including when L2 is active
- Add a test to verify KVM_CREATE_VM accepts exactly what is reported
by KVM_CAP_VM_TYPES
- Misc selftests fixes and cleanups
- Fix several issues with seeding the pRNG, and rework the pRNG APIs
to that the pRNG can be sanely used in host code, not just guest
code
- Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up
via KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for
triggering IRQs via writes to an assigned VFIO device
- Add syscall wrappers to assert success on a variety of pthreads and
CPU affinity APIs
- Set vCPU pthread affinity as early as possible to reduce contention
issues that were surfaced by PREEMPT_LAZY, which result in runtimes
of over a minute on large hosts, versus the expected ~5 seconds
- Rework the PMU counters test to run each testcase using a single VM
with many vCPUs for each sub-testcase, instead of using a unique VM
for each sub-testcase. This cuts the runtime by ~20x
Miscellaneous:
- MAINTAINERS updates for vfio-ap, guest_memfd, kvm-x86. Mostly
representing the status quo more accurately, but also... welcome
David Hildenbrand as guest_memfd reviewer!"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (413 commits)
KVM: arm64: Validate GICv5 timer PPIs before claiming ownership
KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs
KVM: arm64: vgic: Prevent speculative SPI array underflow
KVM: arm64: vgic: Free gic_kvm_info on initialization failure
KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params'
s390/vfio-ap: Fix NULL deref in status_show() during queue probe
s390/vfio-ap: Fix hot-unplug skipped when last AP adapter or domain removed
s390/vfio-ap: fix potential use of uninitialized apm_filtered bitmap
s390/vfio-ap: Fix control domain removal in vfio_ap_mdev_cfg_remove
s390/vfio-ap: Fix required lock not held during update of ap_matrix_mdev object
s390/vfio-ap: Fix missing lock required to access list of ap_matrix_mdev objects
s390/vfio-ap: Fix dereference matrix_mdev->kvm without checking for NULL
s390/vfio-ap: Fix stale do_remove flag across iterations in vfio_ap_mdev_cfg_remove
RISC-V: KVM: fix vcpu vector context handling for kernel-mode vector
riscv: vector: allow non-preemptible kernel-mode vector with IRQs off
riscv: vector: refactor riscv_v_start_kernel_context
KVM: s390: gmap: Make prefix handling optional
KVM: s390: gmap: Make CMMA optional
KVM: s390: gmap: Make storage keys optional
KVM: s390: Prepare gmap for a second KVM implementation
...
|
|
https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 changes for 7.3
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
|
|
SCTLR_EL1.TCSO0 is set when user opt-in for MTE store-only tag check
mode. However, it is not part of SCTLR_USER_MASK which imply that on
context switch we never clear SCTLR_EL1.TCSO0, so we are leaking that
setting into another task.
Fix that by including SCTLR_EL1_TCSO0_MASK into SCTLR_USER_MASK
Fixes: 4d51ff5bba00 ("arm64/kernel: Support store-only mte tag check")
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Yeoreum Yun <yeoreum.yun@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
note_page_flush() calls note_page() with addr == 0 and level == -1 to
dump the last row of a ptdump. addr == 0 (1 << 64 wrapped around)
renders a huge region with enormous size for address spaces with
IA bits < 64. For example the stage-2 page tables and the EFI runtime
page table.
More importantly, the last region of the address space and everything
after the address space up to 1 << 64 are merged into one row of
output. If the last region within the address space is valid, it will
appear to remain valid up to 1 << 64 with the same attributes.
Currently only the EFI runtime ptdump is affected by this, but KVM will
soon fix its stage-2 ptdump by using note_page_flush(). Here is an
example of an EFI runtime ptdump (last row):
0x0000008000000000-0x0000000000000000 17179868672G PGD
With this patch:
0x0000008000000000-0x0001000000000000 261632G PGD
To fix this, cache the end address of a ptdump in ptdump_pg_state so
note_page_flush() can call the final note_page() with the correct end
address.
Fixes: 9d80448ac92b ("efi/arm64: Add debugfs node to dump UEFI runtime page tables")
Signed-off-by: Wei-Lin Chang <weilin.chang@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "mm: drop "sub" prefix from various places" (Dev Jain)
page->folio conversion and a naming cleanup
- "mm/kasan: remove redundant initialization for kasan_flag_write_only"
(Igor Putko)
KASAN cleanup work
- "mm/filemap: reduce unnecessary xarray lookups" (Chi Zhiling)
Small speedup in the pagecaache read code
- "mm/percpu: Fix possible NOFS/NOIO reclaim recursion" (Kaitao Cheng)
Improve the vmalloc code - mainly the avoidance of GFP_KERNEL
allocations when the caller asked for GFP_NOFS or GFP_NOIO
- "mm/kmemleak: avoid soft lockup when scanning task stacks" (Breno
Leitao)
Avoid a soft lockup watchdog trigger from the kmemleak scanning code
in extreme situations
- "mm/page_owner: misc cleanups" (Ye Liu)
Cleanups to the page_owner code. For some reason lots of people have
been working on the page_owner code this cycle.
- "mm: convert to walk_page_range_vma() to eliminate find_vma()"
(Kefeng Wang)
Simplify and accelerate the page walking library function
- "mm/migrate: preparatory cleanups for batch copy and offload"
(Shivank Garg)
Cleanups in the migration code
- "mm/page_owner: add per-fd filter infrastructure for print_mode and
NUMA filtering" (Zhen Ni)
Per-fd filtering to page_owner in order to reduce the sometimes vast
amount of output it can produce
- "mm: Refactor bootmem gigantic hugepage allocation" (Muchun Song)
Fixes and preparatory cleanups around bootmem HugeTLB handling,
sparse initialization ordering, and related vmemmap setup
- "mm/zsmalloc: reduce lock contention in zs_free()" (Wenchao Hao)
Reduce lock contention in zs_free(), which dominates the unmap path
under memory pressure on Android (LMK kills) and on x86 servers
running zswap-heavy workloads.
Up to 1.83x improvement in microbenchmarking.
- "move alloc_tag.c file under mm/" (Suren Baghdasaryan)
- "samples/damon: handle damon_{start,stop}() failures" (SJ Park)
Fix improper handling of damon_start(), damon_stop(), and
damon_call() failures across DAMON sample modules to prevent
potential memory leaks, operation disruptions and use-after-free
bugs
- "mm/damon/sysfs: kobject_del() directories that users can
create/remove" (SJ Park)
Fix delayed sysfs directory removal under DEBUG_KOBJECT_RELEASE
causeing creation failures due to duplicate directory names by adding
missing kobject_del() calls before creating new directories
- "mm: cleanup clear_not_present_full_ptes()" (David Hildenbrand)
Clean up the core pte handling code
- "selftests/damon: misc fixes for test bugs" (Kunwu Chan)
Fix several bugs in the DAMON selftests
- "selftests/damon: fix memcg_path staging handling" (Cheng Nie)
Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and
add a test case for it in sysfs.py.
- "selftests/damon: test kdamond refresh_ms" (Ruslan Valiyev)
Selftest coverage for DAMON's refresh_ms sysfs feature by updating
the test control module and verifying that scheme stats update
automatically without manual intervention
- "mm/damon: five misc fixups" (Akinobu Mita)
Miscellaneous DAMON fixups.
- "mm/damon/core: detect internal variation above max_nr_regions/2"
(Jiayuan Chen)
Fix DAMON's region splitting behavior when region counts exceed half
the maximum budget by dynamically scaling down the split fraction as
the limit approaches, preventing large regions from staying un-split,
and add corresponding KUnit test coverage
- "mm: preparatory patches for PMD level swap entries" (Usama Arif)
Refactor and clean up PMD softleaf helpers, call sites, and
architecture flags to lay the groundwork for a follow-up series that
introduces PMD page table swap entries
- "mm/damon: update, optimize, and clean up doc, tests, and code" (SJ
Park)
Update DAMON design and ABI documentation, expands unit and selftest
coverage, optimize damon_commit_target_regions(), and clean up
recently added sysfs interface code for better readability
- "mm/vmpressure: reduce CPU, memory and code overhead on cgroup v2"
(Usama Arif)
Optimize vmpressure() by skipping unnecessary work on cgroup v2 for
userspace event notifications and refactor v1-only eventfd handling
into mm/memcontrol-v1.c to reduce memory overhead and code complexity
- "selftests/mm: refactor pkey helpers and fix mmap error handling"
(Hongfu Li)
Refactor pkeys shared tracing and assertion helpers into a common
file, unify protection key selftests to use consistent diagnostic
logging and assertions, and enforce standardized MAP_FAILED return
checks for mmap() calls across the tests
- "mm/damon: optimize out nr_accesses_bp" (SJ Park)
Replace the error-prone, continuously updated nr_accesses_bp field in
damon_region with an on-demand moving sum function, reducing
structure memory overhead and avoiding state corruption bugs
- "Open HugeTLB allocation routine for more generic use" (Ackerley Tng)
Decouple HugeTLB folio allocation from VMA dependencies by
introducing hugetlb_alloc_folio(), enabling subsystems like
guest_memfd to allocate HugeTLB folios without standard VMA
reservations or pseudo-VMAs
- "mm/damon: provide pseudo moving sum probe_hits" (SJ Park)
Integrate DAMON's probe_hits attribute counter into the pseudo moving
sum infrastructure, enabling real-time, online monitoring without
waiting for full aggregation intervals
- "mm: Some cleanups for page allocator APIs" (Brendan Jackman)
Simplify and refactor the page allocator entry points and flags by
unifying allocation paths, adding internal alloc_flags arguments, and
eliminating redundant __ prefixed alloc_pages variants.
- "Fix incorrect access of hugetlb pte entries" (Dev Jain)
Enforce the consistent use of huge_ptep_get() instead of ptep_get()
for HugeTLB entries and fixes an unaligned address issue in arm64's
huge_ptep_get() implementation
- "mm/damon: validate all parameters in the core" (SJ Park)
Consolidate parameter validation into the DAMON core specifically
within damon_start() and damon_commit_ctx() to centralize error
checking, eliminate caller-side redundant checks and to improve
maintenance efficiency
- "tools/mm/page_owner_sort: fix filtering and cleanup issues" (Yichong
Chen)
Rename is_need() to filter_record() for clearer return semantics, fix
per-record allocation memory leaks and bound output copies in
search_pattern() to address an existing buffer issue
- "memcg: bail out reclaim when memcg is dying" (Jiayuan Chen)
Mitigate a system-wide stall which occurs when a cgroup is removed
while one of its memory control files is doing synchronous reclaim
- "mm/memory-failure: add panic option for unrecoverable pages" (Breno
Leitao)
Introduce an opt-in vm.panic_on_unrecoverable_memory_failure sysctl
that immediately panics the kernel on unrecoverable memory errors in
kernel-owned pages to preserve error context and prevent delayed,
silent data corruption
- "mm/damon: refactor damon_{start,stop,commit}() for simple error
handling" (SJ Park)
Refactor the DAMON core API functions to guarantee that all contexts
are fully stopped when damon_start(), damon_stop(), or damon_commit()
fail, eliminating the need for complex and error-prone caller-side
cleanup code
- "Keep tail page private zero at free and folio split" (Zi Yan)
Add checks to ensure tail_page->private is zero when freeing compound
or high-order pages and when promoting tail pages during large folio
splits. By validating these fields at free and split time, it allows
the removal of redundant private field clearing inside
prep_compound_tail()
- "mm: drop redundant lru_add_drain in anon folio reuse paths" (Barry
Song)
Eliminate redundant lru_add_drain() calls in
wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
contention and system overhead
By validating folio refcounts against the LRU cache before draining
and removing unnecessary drains in the swap path, it achieves up to a
30.5% reduction in drain calls during heavy swap workloads
- "mm: clean up folio LRU and swap declarations" (Jianyue Wu)
Reorganize folio LRU and swap code by relocating page-cluster state
to mm/swap_state.c, renaming mm/swap.c to mm/folio.c, and moving
MM-internal reclaim declarations into mm/internal.h.
- "userfaultfd: working set tracking for VM guest memory" (Kiryl
Shutsemau)
Add userfaultfd support for tracking the working set of VM guest
memory, so a VMM can identify hot pages and reclaim cold ones to
tiered or remote storage
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 2)" (David
Hildenbrand)
Remove the remaining pieces of CONFIG_HAVE_BOOTMEM_INFO_NODE,
performing some smaller cleanups around freeing of reserved vmemmap
pages on the way.
- "mm/damon: update probe hits for runtime parameter commits" (SJ Park)
Ensure that DAMON's probe_hits attribute counter is properly updated
when monitoring intervals are changed at runtime, matching the
behavior of nr_accesses. To achieve this, it refactors and renames
existing helper functions for shared use, applies the updates to
probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
maintain measurement accuracy.
- "KSM: performance optimizations for rmap_walk_ksm" (xu xin)
Resolve a severe KSM reverse-mapping performance bottleneck where
thousands of split VMAs sharing a single anon_vma cause extended lock
contention.
By adding an interval-filtering check during the rmap walk, it
reduces worst-case anon_vma lock hold times from over 500ms down to
under 2ms, preventing application freezes and latency spikes under
memory pressure.
- "mm: split a couple of headers from internal.h" (Mike Rapoport)
Split declarations related to mm_init, memblock, vmalloc and sparse
into new headers
- "KSM: use linear_page_index in collect_procs_ksm()" (xu xin)
Apply the interval tree optimization from rmap_walk_ksm() to
collect_procs_ksm() to avoid iterating over non-matching VMAs during
KSM memory error handling.
It hoists loop-invariant address initialization and restricts the
anon_vma_interval_tree_foreach walk to a targeted page offset range,
reducing redundant checks and improving lookup efficiency.
- "selftests/mm: avoid false failures in hugetlb and KSM tests" (Sayali
Patil)
Fix issues in the hugetlb and KSM MM selftest categories that can
report failures when the prerequisites for the tests are not
satisfied
- "mm/damon: introduce data attributes only monitoring" (SJ Park)
Introduce attribute-weighted region management in DAMON, allowing
users to prioritize specific data attributes (such as page sizes or
cgroups) over or instead of access monitoring.
By assigning weights to attribute probes, DAMON can completely
disable access tracking and adjust monitoring regions based on
weighted probe-hit counters to optimize monitoring quality for
attribute-focused workloads.
- "mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings"
(Stanislav Kinsburskii)
Extend hmm_range_fault() to support userfaultfd-backed regions by
allowing the mmap lock to be dropped during fault handling via a new
hmm_range_fault_locked() helper.
By accepting a locked pointer and signaling retry status when lock
release occurs, it enables page fault resolution in userfaultfd
regions while preserving backward compatibility for existing callers.
- "mm: make VMA page offset handling more consistent" (Lorenzo Stoakes)
Clean up and standardize how vma->vm_pgoff is accessed and
manipulated across file-backed and anonymous mappings in the kernel
It introduces dedicated helper functions such as vma_start_pgoff(),
vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while
renaming rmap interval tree helpers to better reflect their
functionality.
These changes establish a cleaner foundation for future work that
will unify virtual page offset indexing for all anonymous and CoW'd
folios.
- "mm: handle device-private PMDs in walk callbacks" (Usama Arif)
Address kernel panics and state corruption caused by MM walk
callbacks reaching non-present device-private PMD swap entries
created during HMM migrations
It ensures that functions which acquire pmd_trans_huge_lock()
properly recognize device-private PMDs instead of assuming a present
THP or a standard migration entry.
- "mm/rmap: Refactor try_to_unmap_one" (Dev Jain)
Refactor try_to_unmap_one by modularizing Hugetlb,
anonymous-lazyfree, and anonymous-swapbacked logic into dedicated
functions, laying the structural groundwork for batched anonymous
large folio unmapping.
- "Docs/ABI/damon: sysfs ABI document fixes and additions" (Song Hu)
Fix typos and fills in missing entries in the DAMON sysfs ABI
document
- "dax/kmem: atomic whole-device hotplug via sysfs" (Gregory Price)
Introduce an atomic sysfs state attribute and supporting DAX/MM
infrastructure to prevent userland races when offlining and removing
entire memory regions
By adding an unplugged state alongside standard online modes, it
enables whole-device atomic hotplug control while preserving backward
compatibility.
- "mm: convert more vm_flags_t users to vma_flags_t" (Lorenzo Stoakes)
Continue transitioning the kernel from the deprecated vm_flags_t type
to vma_flags_t across core memory management infrastructure.
It replaces legacy type usage in core functions such as do_mmap(),
unmapped area allocation, mm->def_vma_flags, and VMA operations like
mlock, mprotect, and mremap.
- "Two small patches to clean up mm/mm_slot.h" (xu xin)
Refactor mm_slot.h by introducing mm_slot_remove() to unify duplicate
slot deletion sequences in khugepaged and KSM. It also adds code
documentation explaining why mm_slot_lookup and mm_slot_insert must
remain as preprocessor macros rather than static inline functions.
- "mm/damon/core: hide core-private struct fields" (SJ Park)
Clean up DAMON core structures by consistently marking internal-only
fields with private: comment tags to prevent improper direct access
from outer layers.
It enforces encapsulation across core structures including
damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
interact through approved access APIs instead of exposing raw struct
members.
- "mm/damon: unurgent fixes for infinite loop, NULL de-ref and races"
(SJ Park)
Address potential infinite loops, NULL dereferences, and race
conditions identified in DAMON
It fixes an infinite loop triggered by extreme user configurations, a
NULL pointer dereference within unit tests and minor monitoring
accuracy degradation caused by subtle runtime races.
- "mm/page_alloc: fixes for free_pages_nolock() on RT/UP" (Brendan
Jackman)
Fix an NMI safety flaw in __free_frozen_pages() where freeing pages
on non-SMP or PREEMPT_RT kernels can bypass can_spin_trylock() checks
via non-PCP or isolated migration paths.
It also resolves potential kernel crashes and privilege escalation
risks triggered when BPF tracing runs in NMI context alongside memory
hotplug or large allocation frees.
- "mm/page_alloc: couple of followups for recent cleanups" (Brendan
Jackman)
Clean up and update page allocator nomenclature, documentation, and
debug assertions.
It aligns internal FPI_ flags with the public "nolock" naming
convention, removes outdated internal implementation details from
high-level page allocator comments, and eliminates obsolete
VM_BUG_ON() assertions in allocation paths.
- "mm/mseal: further cleanups" (Lorenzo Stoakes)
Refactor and simplify the mseal implementation by clarifying API
boundaries and removing unnecessary code complexity.
It replaces generic do_mseal() usage outside the syscall with a
dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO,
eliminates mm_struct parameters to enforce that sealing applies only
to current->mm, and streamlines overall logic and comments with no
functional changes intended.
- "mm/vmscan: fix swappiness=max and clean up per-node proactive
reclaim" (Ridong Chen)
Resolve reclaim behavior bugs and clean up function parameters across
memory reclaim paths
It fixes swappiness=max in both standard reclaim and MGLRU so
unswappable anonymous memory no longer falls back to evicting page
cache, ensures reclaim_store() returns accurate error codes instead
of collapsing all failures into -EAGAIN, and removes the obsolete
gfp_mask parameter from __node_reclaim().
- "mm: mincore: misc cleanups" (Kefeng Wang)
Clean up and simplifies the mincore code. Most importantly, it
removes the historical special behavior that always reports VM_PFNMAP
pages as non-resident.
- "mm/huge_memory: drop dead split helper variants" (Kiryl Shutsemau)
Two trivial cleanups in the folio split API
- "mm/damon: fix uninitialized DAMOS field and kunit exec expectation
bugs" (SJ Park)
Resolve minor operational and testing bugs in DAMON identified by
Sashiko. It initializes the damos->last_applied field to prevent
occasional efficiency degradation and fixes invalid memory accesses
in DAMON KUnit tests during test failure handling.
- "cleanup for stable_page_flags()" (Jinjiang Tu)
Clean up and refactor stable_page_flags() used by /proc/kpageflags
without altering functionality.
It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
bits, converts folio-specific flag checks to standard folio_test_*()
helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
- "Batch unmap of uffd-wp file folios" (Dev Jain)
Extend batched folio unmapping support to file folios within
userfaultfd write-protect (uffd-wp) VMAs by adding batching
capabilities to pte_install_uffd_wp_if_needed().
This removes special-case restrictions on uffd-wp VMAs in
try_to_unmap_one(), significantly simplifying the function's control
flow and complexity.
- "mm/early_ioremap: clarify and clean up early_ioremap_reset()"
(Sang-Heon Jeon)
Clarify and clean up the architecture-specific usage of
__late_set_fixmap() and __late_clear_fixmap() after
early_ioremap_reset()
It adds explicit documentation regarding when early_ioremap_reset()
must be called and removes redundant macro definitions and reset
calls in the RISC-V and ARM64 architectures.
- "mm: fix reclaim storms in defrag_mode" (Johannes Weiner)
Address severe performance regressions, swap storms, and spurious
OOMs caused by vm.defrag_mode=1 under high memory pressure in Meta
production
It updates the page allocator slowpath so non-movable allocation
requests actively trigger direct reclaim and direct compaction at
pageblock_order scale, allowing them to claim whole pageblocks rather
than spinning unproductively.
- "zram: lockmap tweaks" (Sebastian Siewior)
Optimize and fix lockdep tracking for zram devices by consolidating
per-entry lockmaps and isolate lock classes across multiple instances
This reduces memory overhead by replacing per-entry lockdep_map
instances with a single map per struct zram, and assigns a dynamic
lock_class_key to each instance to prevent false deadlock reports
when different zram devices are backed by distinct filesystems.
* tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (501 commits)
selftests/mm: thuge-gen: fix test_shmget() for PAGE_SIZE check
selftests/mm: unpoison pages in memory-failure teardown
mm/shmem: downgrade final i_blocks check in shmem_evict_inode() to pr_warn()
mm/khugepaged: replace mutex_lock/mutex_unlock usage with guard macro
mm/zsmalloc: fix release order of locks in zs_page_migrate()
Documentation: zram: remove sections numbering
ksm: stop iterating VMAs when ksm_test_exit returns true
mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE
mm/migrate: report RCU-tasks quiescent states in migrate_pages_batch()
zram: use a custom key for each zram object
zram: move lockmap to be per-zram instead per table
selftests/mm: fix gup_longterm EINVAL error message
mm: page_alloc: fix non-movable reclaim storm in defrag_mode
mm: page_alloc: move capture_control to the page allocator
mm: compaction: support non-movable compaction for pageblock requests
mm: page_alloc: __GFP_FS lockdep annotation for direct compaction
hugetlb: evaluate subpool free state while locked
mm/damon: remove trailing semicolons after function definitions
mm/damon/ops-common: prevent migration fallback to non-target nodes
mm/damon: update outdated comment about DAMOS filter handling
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Daniel Borkmann:
"Major changes:
- Redesign the verifier error reporting: failures now carry source
and instruction annotations along with the causal event history
that led to them, making program rejections far easier to debug and
repair (Kumar Kartikeya Dwivedi)
- Add arena argument support to kfuncs and struct_ops through the new
__arena and __arena__nullable suffixes (Tejun Heo, Puranjay Mohan,
Kumar Kartikeya Dwivedi, Ihor Solodrai)
- Signed BPF program loader rework to accommodate both BPF and
security community needs where the kernel runs the signature
verification at BPF_PROG_LOAD time before the LSM admission hook
(Daniel Borkmann)
- Add a set of ksock kfuncs which let BPF LSM and syscall programs
create, connect and send on UDP sockets in order to emit telemetry
data (Mahe Tardy)
- Unify helper and kfunc call argument verification and classify
kfunc arguments purely from BTF into a generated bpf_func_proto
which is computed once at add-call time (Amery Hung)
Other features and fixes:
- Enable EXECMEM_ROX_CACHE for BPF allocations on x86 (Mike Rapoport)
- Add bidirectional VLAN support to bpf_fib_lookup() through the new
BPF_FIB_LOOKUP_VLAN and BPF_FIB_LOOKUP_VLAN_INPUT flags (Avinash
Duduskar)
- Infer zext_dst from static register liveness analysis to fix 32-bit
zero-extension semantics, and remove the artificial limitations on
pointer types eligible for spilling (Eduard Zingerman)
- Inline the numeric open-coded iterator kfuncs so that bpf_for()
loops no longer pay a kfunc call on every iteration (Puranjay
Mohan)
- Add an arena-based bitmap data structure to libarena along with
serial and parallel selftests (Emil Tsalapatis)
- Teach resolve_btfids to discover kfuncs from the kernel's BTF ID
sets and to emit kfunc BTF decl tags, reducing the kernel build's
dependency on pahole features (Ihor Solodrai)
- Add BPF_F_ADJ_ROOM_DECAP_* flags to bpf_skb_adjust_room() so that
tunnel decapsulation can update the GSO and encapsulation state of
the skb (Nick Hudson)
- Fix the ring buffer pending_pos walk and the available-data
accounting on 32-bit position wrap (Israel Téllez García)
- Add memory usage accounting for arena maps and fix an mmap_lock
deadlock on arena lock failure (Jiayuan Chen)
- Add tracing_multi link info support to the kernel UAPI and bpftool,
and refactor the stack map code to run with preemption disabled
(Jiri Olsa)
- Support BPF_F_EGRESS in bpf_redirect_peer() to emit the skb in the
egress direction of the target's peer device (Jordan Rife)
- Add a KF_SPINLOCK_SAFE kfunc flag so that providers, in particular
modules, can declare kfuncs safe to call under bpf_spin_lock
instead of relying on the verifier's hard-coded allowlist (Kaitao
Cheng)
- Introduce global percpu data for BPF programs with libbpf probing
and bpftool skeleton support, and stop exposing uninitialized
kernel heap memory when copying per-CPU map values (Leon Hwang)
- Add s390 JIT support for load-acquire and store-release
instructions (Maxim Khmelevskii)
- Fix a CFI mismatch in the task work callback and an arm64 KASAN
false positive after bpf_throw() (Mykyta Yatsenko)
- Reject writes through untrusted BTF pointers and bound the
rdonly/rdwr_buf_size kfunc arguments (Nicholas Dudar)
- Invalidate RCU pointers only after the final spin unlock and
account for preempt and IRQ disabled regions as overlapping RCU
protection (Ning Ding)
- Support mixing bpf2bpf calls and tail calls on RV64, add signed
operations and 32-bit atomics to the RV32 JIT, and add timed
may_goto support (Pu Lehui, Kuan-Wei Chiu, Feng Jiang)
- Fix a use-after-free on mm_struct in bpf_find_vma() for foreign
tasks and an mmap_lock leak in the irq_work path (Sanghyun Park)
- Populate mmap-able BPF array map memory lazily which makes mmap()
O(1) instead of proportional to the map size (Song Liu)
- Introduce a jit_required flag and reject programs with inlined
helpers when no JIT is available, where the interpreter would
otherwise jump into an invalid address (Tiezhu Yang)
- Fix the x86 JIT per-CPU address resolution into an extended
register where the REX prefix dropped the high destination register
bit (Vineet Gupta)
- Reject MEM_ALLOC BTF accesses past object bounds, arena frees below
the arena base, and mixed arena and ordinary atomic paths (Yiyang
Chen)
- Fix the trampoline handling of 128-bit arguments and of return
values larger than 8 bytes (Yonghong Song)
- Ensure that any fault prone load is rewritten with exception table
handling, and fix the arena load-acquire and atomic fetch handling
in the x86, arm64, riscv and s390 JITs (Daniel Borkmann)
- Many more fixes and cleanups across the verifier, arena,
trampolines, sockmap, cgroup, ring buffer, x86/arm64/riscv/s390
JITs, libbpf, bpftool, resolve_btfids and selftests"
* tag 'bpf-next-7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (373 commits)
selftests/bpf: Add tests for a store on a fault prone qdisc pointer
selftests/bpf: Add tests for fault prone loads out of RCU pointers
selftests/bpf: Add tests for pointer type merge at a shared load
selftests/bpf: Remove duplicate copies of the arena spinlock qnodes
selftests/bpf: Retry stat generation in cgroup_iter_memcg
selftests/bpf: Test pseudo-function policy diagnostics
bpf: Distinguish function references in policy diagnostics
bpf: Preserve source attribution without source text
selftests/bpf: Test kfunc argument diagnostics
bpf: Correct kfunc argument diagnostics
bpf: Use canonical stack argument names in diagnostics
bpf: Preserve R0 lineage across helper calls
selftests/bpf: Exercise negative optlen in cgroup getsockopt hook
bpf: Reject negative optlen in cgroup getsockopt hook
selftests/bpf: tc_tunnel - validate decap GSO and encapsulation state
bpf: Clear decap state on skb_adjust_room shrink path
bpf: Allow new DECAP flags and add guard rails
bpf: Add BPF_F_ADJ_ROOM_DECAP_* flags for tunnel decapsulation
bpf: Refactor masks for ADJ_ROOM flags and encap validation
bpf: Name the enum for BPF_FUNC_skb_adjust_room flags
...
|
|
* kvm-arm64/misc-7.3:
: Miscellaneous fixes for KVM/arm64, 7.3
:
: - Fixes for saving invalid table entries as part of saving the ITS
: tables (Fuad Tabba)
:
: - Don't reallocate the SPI array for re-attempted vgic_init(), avoiding
: a memory leak (Fuad Tabba)
:
: - Hold a reference on an LPI when saving the pending state (Qihang)
:
: - Don't WARN for out-of-range, guest-supplied INTID (Karl)
:
: - Avoid corrupting GPRs for 32-bit CP64 reads (Karl)
:
: - Reset 'in kernel' VGIC state when private IRQ allocation fails (Fuad)
:
: - Avoid kallsyms lookup in nVHE panic unless the host stage-2 is also
: disabled (Vincent)
:
: - Disregard Pending+Active state when computing maintenance IRQ for
: ICH_MISR_EL2.NP (Kajetan)
:
: - Various Sashiko-identified issues dealing with GICv5 (Sascha)
:
: - Fix CPU onlining in pKVM due to mismatched accesses when the MMU is
: disabled (Will)
KVM: arm64: Validate GICv5 timer PPIs before claiming ownership
KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs
KVM: arm64: vgic: Prevent speculative SPI array underflow
KVM: arm64: vgic: Free gic_kvm_info on initialization failure
KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params'
KVM: arm64: vgic: Fix detection of MI on no pending LR
KVM: arm64: Drop %pB on nVHE panic when stage-2 is active
KVM: arm64: vgic: Reset in_kernel on private IRQ allocation failure
KVM: arm64: GICv2: Don't WARN on out-of-range GICV_DIR INTID
KVM: arm64: Preserve GPRs for AArch32 CP64 reads generating an UNDEF
KVM: arm64: vgic-v3: take an LPI reference in vgic_v3_save_pending_tables
KVM: arm64: vgic-its: Point saved ITEs at the next valid entry
KVM: arm64: vgic-its: Don't save collections the table cannot hold
KVM: arm64: vgic: Don't leak the SPI array when init is retried
KVM: arm64: vgic-its: Don't dereference a NULL collection on ITT save
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/vtr-patch:
: Inline patching of ICH_VTR_EL2 constant, courtesy of Marc Zyngier
:
: Unify readers of ICH_VTR_EL2 on an instruction-patched constant value,
: avoiding system register accesses known to trap under nested
: virtualization and sharing the implementation between pKVM and 'regular'
: KVM.
KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2
KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling
KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value
KVM: arm64: Add a helper providing an inlined literal value for ICH_VTR_EL2
KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata
KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field symbols
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/vncr-fixes:
: Fixes for handling L1 VNCR mappings, courtesy of Marc Zyngier
:
: - Recompute the VNCR software TLB when the MMU is toggled at stage-1
:
: - Improve TLB invalidation intersection to handle TLBIs affecting the
: end of the VA space
:
: - Fix race to invalidate the VNCR fixmap between TLBI emulation and
: vcpu_put()
:
: - Add missing sign extension for computing TLBI ranges
:
: - Make VNCR invalidation participate in the MMU notifier seqcount,
: preventing a concurrent VNCR TLB fill from consuming a stale
: translation
KVM: arm64: Correctly cap TLBI Range to the architural limit
KVM: arm64: Add VNCR TLB tracking again
KVM: arm64: Make VNCR invalidation participate in MMU invalidation retry
KVM: arm64: Sign-extend VA for range-based TLBI invalidation
KVM: arm64: Handle VNCR TLB invalidation race with vcpu_put() VNCR unmapping
KVM: arm64: Correctly handle end of VA space TLBI invalidation
KVM: arm64: Consider SCTLR_EL2.M when mapping the L1 VNCR page
KVM: arm64: Handle negative S1 walk levels in VNCR TLB size evaluation
KVM: arm64: Remove VM-wide VNCR mapping counter
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/feat-nv3: (29 commits)
: Support for FEAT_NV2p1 and FEAT_NV3, courtesy of Marc Zyngier
:
: Couple of accelerations for nested virtualization:
:
: - Elide traps for EL1 alias registers when hardware guarantees that
: EL2-only bits are stateful (NV2p1)
:
: - Use TGE-aware hardware to avoid unnecesary ERET/TLBI traps that
: remain 'in-context' (i.e. L1 kernel/userspace)
arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list
KVM: arm64: Expose FEAT_NV3 to guests
KVM: arm64: Add FEAT_NV3 detection
KVM: arm64: Engage NV3 TLBI trap elision
KVM: arm64: Engage NV3 ERET trap elision
KVM: arm64: Add NVHCR_EL2 context switching
KVM: arm64: Add routing for NVHCR_EL2 trap
KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
KVM: arm64: Add sanitisation for NVHCR_EL2
KVM: arm64: Make HCR_EL2 a non-VNCR register
KVM: arm64: Add kvm_has_nv{2,3}() predicates
KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
arm64: Add ARM64_HAS_NV3 capability
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
arm64: Add FEAT_NV2p1 detection
KVM: arm64: Expose FEAT_NV2p1 to NV guests
KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/pkvm-7.3: (26 commits)
: pKVM updates for 7.3
:
: - Avoid name collision on trace_clock() when CONFIG_NVHE_EL2_TRACING is
: disabled (Mostafa Saleh)
:
: - Clean up state tracking for whether the EL2 shadow VM has been
: created (Fuad Tabba)
:
: - Synchronize SCTLR_EL1 when injecting an exception to use current
: PAN/SSBS state (Fuad Tabba)
:
: - Avoid unnecessary cache maintenance when I/D-cache are known to be
: coherent in pKVM (Mostafa Saleh)
:
: - Lazy vCPU context save/restore for pKVM (Fuad Tabba)
:
: - Various fixes to the stage-2 MMU for pKVM (Fuad Tabba)
:
: - Allow counter offsetting of non-protected guests in protected mode
: (Mostafa Saleh)
:
: - Condition the 'broken CNTVOFF' mitigation on a VM actually having a
: nonzero offset, fixing boot failures of pVMs on affected hardware
: (Mostafa Saleh)
KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
KVM: arm64: Fix timer offsets for non-protected VMs
KVM: arm64: Make timer_get_offset() work in all contexts
KVM: arm64: selftests: Add stage-2 block transition test
KVM: arm64: Don't advertise eager page splitting under pKVM
KVM: arm64: Don't WARN on pKVM stage-2 map failures
KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
KVM: arm64: Top up stage-2 memcache for dirty logging faults
KVM: arm64: Top up the memcache for pKVM permission faults
KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: selftests: Add a userspace watchpoint test
KVM: arm64: Flush external_mdscr_el1 to the pKVM hyp vCPU
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/pmu-7.3:
: vPMU updates for 7.3
:
: - Support for slot-based PMU events, relying on new UAPI that makes
: selection of a vPMU implementation mandatory (Congkai Tan)
KVM: arm64: Add KVM_ARM_VCPU_PMU_V3_STRICT vCPU feature
KVM: arm64: Ignore writes to PMCR_EL0.N when using strict UAPI
KVM: arm64: Advertise STALL_SLOT* in PMCEID1 under strict PMUv3 UAPI
KVM: arm64: Expose PMMIR_EL1.SLOTS under strict PMUv3 UAPI
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
When running with hVHE enabled, ___kvm_hyp_init() calls
__kvm_init_el2_state() on the CPU initialisation path during onlining
and resume from suspend.
In order to avoid clobbering the link register across this call, it is
stashed away in the 'tmp' member of 'struct kvm_nvhe_init_params',
however this save/restore operation is performed with the stage-1 MMU
disabled at EL2 and therefore gives rise to coherency problems because
the field is not aligned or padded to the CWG. For example, a cacheable
write to a physically-adjacent structure sharing the same cacheline
could lead to an eviction and subsequent write-back, overwriting the
saved LR while the incoming CPU is executing __kvm_init_el2_state().
Save the lr in far_el2 and remove the 'tmp' member from
'struct kvm_nvhe_init_params' altogether.
Cc: Oliver Upton <oupton@kernel.org>
Cc: Marc Zyngier <maz@kernel.org>
Fixes: afa9b48f327c ("KVM: arm64: Shave a few bytes from the EL2 idmap code")
Signed-off-by: Will Deacon <will@kernel.org>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260813131717.5885-1-will@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking updates from Ingo Molnar:
"Futexes:
- Use runtime constants for futex_hash computation (K Prateek Nayak,
Peter Zijlstra)
- Optimise the size check get_futex_key() (Sebastian Andrzej Siewior)
- Avoid private hash use-after-free on final put (Felix Hoffmann)
- Tell kmemleak we're not leaking __futex_queues (Peter Zijlstra)
Rust integration updates:
- Implement refcounted interrupt disable and SpinLockIrq for Rust
(Boqun Feng, Heiko Carstens, Joel Fernandes, Lyude Paul)
- Rust sync: add helpers for mb, dma_mb and friends; add generic
memory barriers and use LKMM atomics instead of Rust atomics in the
revocable code (Gary Guo)
- Add abstraction and integrate synchronize_rcu() (Philipp Stanner)
Lock debugging:
- Add qspinlock contended_release tracepoint (Dmitry Ilvokhin, Peter
Zijlstra)
- Enable the printing of held locks of remote running tasks and print
task CPU (Ingo Molnar)
- percpu-rwsem: Annotate intentional data race in readers_active_check()
(Sun Shaojie)
Misc fixes and updates by Boqun Feng, Peter Zijlstra, Fangrui Song,
Naveen Kumar Chaudhary and Thomas Huth"
* tag 'locking-core-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (44 commits)
rust: sync: Introduce SpinLockIrq::lock_with() and friends
rust: sync: Add SpinLockIrq
rust: sync: Use super::* in spinlock.rs
rust: helper: Add spin_{un,}lock_irq_{enable,disable}() helpers
rust: Introduce interrupt module
s390/preempt: Enable HAS_SEPARATE_PREEMPT_RESCHED_BITS
arm64: sched/preempt: Enable HAS_SEPARATE_PREEMPT_RESCHED_BITS
preempt: Introduce HAS_SEPARATE_PREEMPT_RESCHED_BITS
sched: Avoid signed comparison of preempt_count() in __cant_migrate()
sched: Remove the unused preempt_offset parameter of __cant_sleep()
locking: Switch to _irq_{disable,enable}() variants in cleanup guards
irq: Add KUnit test for refcounted interrupt enable/disable
irq,spin_lock: Add counted interrupt disabling/enabling
openrisc: Include <linux/cpumask.h> in smp.h
preempt: Introduce __preempt_count_{sub,add}_return()
preempt: Introduce HARDIRQ_DISABLE_BITS
preempt: Track NMI nesting to separate per-CPU counter
futex: Tell kmemleak we're not leaking __futex_queues
x86/paravirt: Trace contended_release on unlock
tracing/lock: Use TRACE_EVENT_FN() for contended_release
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon:
"There's a reasonable amount of stuff here, including a bunch of
updates to the perf PMU drivers and some MPAM updates to expose the
memory bandwidth counters via resctrl.
On the architecture side, some highlights include support for BBML3
and steps towards support for an architectural NMI solution, all
wrapped up in a web of fixes for latent issues identified by Sashiko.
ACPI:
- Combine reads of AMU counters into a single FFH feedback counter op
Confidential computing:
- Fix smp_processor_id() in preemptible context when retrieving an
attestation token inside a realm
- Convert pKVM over to a "CC platform"
- Clean-up our SWIOTLB configuration in preparation for reworking the
handling of encrypted/decryped DMA buffers in the dma-mapping tree
CPU errata handling:
- Work around broken device memory ordering on NVIDIA Olympus cores
- Fix broken 'nospectre_bhb' command-line option
- Select the idle loop backend instruction on the command-line
CPU features:
- Replace our BBML2-noabort feature with the new architectural BBML3
feature
- Disable in-kernel BTI for recent versions of Clang due to issues
with livepatch that are still being investigated
- Clean-up documentation describing which ID register fields are
exposed to userspace
Interrupts:
- Preliminary work towards supporting FEAT_NMI, which cleans up our
IRQ entry code and fixes some latent issues with pseudo-NMI
- Support for an SDEI backend to trigger an NMI backtrace
Memory management:
- Treat all devices as coherent when CLIDR_EL1.LoC == 0
- Fix no-map handling of sub-page-sized regions
- Second attempt at unmapping the linear aliases of the kernel data
and bss sections
- Fix EFI runtime calls when software-PAN is enabled
Miscellaneous:
- Add Mark Rutland as a reviewer!
- Tidy-up our futex cmpxchg logic when using the new LSUI
instructions
- Drop the requirement on DYNAMIC_FTRACE_WITH_CALL_OPS when
selecting HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS
- Fix a false-positive KCSCAN splat in the delay loop
- Use a portable typedef for 128-bit scalar types in our UAPI headers
- Non-critical fixes for Sashiko reports all over
MPAM:
- Hook MPAM memory bandwidth counters into resctrl's counter
assignment interface
- Fix a quirk in the MPAM bandwidth counting on Nvidia T241 so that
it also applies to 63 bit counters
Perf:
- Workarounds for hardware issues in the CMN-S3 PMU (Graviton 5) and
CPU PMU (NVIDIA Olympus again!)
- Add support for the DDR PMU on Marvell CN20K SoCs
- Add support for Picoheart implementations of the DCW PCIe PMU
- Add support for Channel/Rank/Bank filtering in the CXL PMU driver
- Add support for 64-bit counters in the CSPMU device
- Add support for revision 2 of the CMN S3 PMU
Ptrace:
- Fix a decade-old bug in our handling of seccomp and tracing on
syscall entry
- Fix regset handling for inactive SVE and SSVE registers
Selftests
- Add some tests for the decade-old bug that we just tried to fix in
our syscall entry path
- Fix SVE test crash on SME-only CPUs"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (95 commits)
arm64/efi: Avoid voluntary preemption with efi_mm installed
arm64: bti: Disable in-kernel BTI with recent versions of Clang
arm64: entry: Avoid unnecessary local_irq_disable() on kernel exit
irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs
arm64: Disable KCSAN instrumentation in delay.o
arm_mpam: Disable driver unbind to avoid UAF
arm_mpam: Fix a NULL pointer dereference on unbinding after an error interrupt
perf: arm_pmuv3: Zero initialize hw_id branch stack field
arm64: mm: Unmap kernel data/bss entirely from the linear map
iommu/arm-smmu-v3-sva: Use system_supports_bbml3() to detect CPU feature
perf/arm-cmn: Support CMN S3 r2
perf/arm-cmn: Plumb in new filter types
perf/arm-cmn: Refactor event filter data
perf/arm-cmn: Refactor event filter programming
perf/arm-cmn: Rename filter variables for clarity
arm64: mm: fix accidental linear mapping of no-map reserved memory
tools: Ensure tools copy of linux/filter.h exports the UAPI
kselftest/arm64: Fix abi test compilation errors
arch: arm64: add early_param idle=<wfi|yield|nop>
arm64: entry: mask DAIF before returning from C EL1 handlers
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux
Pull Kbuild/Kconfig updates from Nicolas Schier:
"Kbuild updates:
- Use --force-group-allocation when linking modules
Have the linker resolve the COMDAT groups and place their members
as regular sections instead of possibly leaving multiple copies in
the resulting modules and unnecessary group metadata.
- UAPI header files: Canonicalize __ASSEMBLER__ / __ASSEMBLY__ mixed
use to __ASSEMBLER__
There is an ongoing effort to change __ASSEMBLY__ to __ASSEMBLER__
treewide. For consistency, UAPI headers are normalised to use
__ASSEMBLER__ only. Normalisation is done in two subsequent patches
to simplify a revert in the unexpected case of a regression report.
- link-vmlinux.sh: Improve detection of third pass requirement
- modpost: Canonicalize format of warnings and errors
- Minor changes:
- Remove srctree path from CHECK output
- Set the initial value of subdir-rustflags-y
- Remove broken and unused modules.builtin(.modinfo) targets from
the top-level Makefile
- Add symbol size for kallsyms symbols that can change size
- modpost: Prevent leak when early return no suffix .o in
read_symbols()
- scripts/config: Update usage of POSIX sed
- 'make tags': Add support for rust source files and prevent
binary files from being analysed
- Several spelling mistakes and rephrasing
Kconfig updates:
- Add Julian Braha as Kconfig reviewer
- Fix submenu rendering of negative dependencies
- Minor changes:
- merge_config.sh: Keep temp file in the output dir
- Abort rather than loop for ever on EOF"
* tag 'kbuild-7.3-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux: (23 commits)
modpost: use mod_warn() and mod_error(), clean up logging
modpost: add module as parameter to modpost_log()
kconfig: fix submenu rendering of negative dependencies
kbuild: link-vmlinux.sh: improve detection of third pass requirement
kallsyms: add symbol size for kallsyms symbols that can change size
kbuild: fix modules.builtin(.modinfo) targets in the top-level Makefile
kbuild: set the initial value of subdir-rustflags-y
scripts/config: Use in-place editing (-i) in sed portably
scripts/config: Use POSIX standard ERE (-E) in sed
modpost: prevent leak when early return no suffix .o in read_symbols()
usr: Correct a spelling by changing a letter
fixdep: make gendered language gender-neutral
kconfig: fix minor typos in comments
scripts: fix spelling mistakes
kconfig: abort rather than loop for ever on EOF
scripts/tags.sh: Add support for rust source files
scripts/tags.sh: Prevent binary files appearing in cscope.files
MAINTAINERS: add Julian Braha as Kconfig reviewer
scripts: headers_install.sh: Normalize __ASSEMBLY__ to __ASSEMBLER__
scripts: headers_install.sh: Normalize __ASSEMBLER__ to __ASSEMBLY__
...
|
|
* for-next/uapi:
arm64: uapi: Use __u128 instead of __uint128_t in UAPI headers
tools: linux/types.h: Add 128-bit integer types for arm64 UAPI structures
|
|
* for-next/sdei:
arm64: escalate smp_send_stop() to an SDEI NMI as a last resort
drivers/firmware: add SDEI cross-CPU NMI service for arm64
firmware: arm_sdei: add SDEI_EVENT_SIGNAL support
firmware: arm_sdei: add sdei_is_present()
|
|
* for-next/ptrace:
arm64: syscall: Pass 'orig_x0' as first argument to native system call
arm64: ptrace: Keep 'orig_x0' in-sync with x0 on syscall entry
arm64/fpsimd: ptrace: Fix inactive SVE and SSVE regsets
|
|
* for-next/nmi:
arm64: entry: Avoid unnecessary local_irq_disable() on kernel exit
irqchip/gic-v3: make the unmasking of pseudo-NMIs explicit when handling IRQs
arm64: entry: mask DAIF before returning from C EL1 handlers
arm64: suspend: Initialize PMR on resume
arm64: suspend: rely on daif helpers to handle PMR
arm64: hibernate: Restore DAIF state on error
arm64: hibernate: mask DAIF before restoring hibernated kernel
arm64: debug: don't mask DAIF for mdscr_write()
arm64: ptrace: Remove INIT_PSTATE_EL2
|
|
* for-next/mm:
arm64/efi: Avoid voluntary preemption with efi_mm installed
arm64: mm: Unmap kernel data/bss entirely from the linear map
arm64: mm: fix accidental linear mapping of no-map reserved memory
arm64: pgtable: convert pte_present() from macro to static inline
arm64: mm: Treat all devices as dma-coherent when CLIDR_EL1.LoC == 0
|
|
* for-next/misc:
arm64: Disable KCSAN instrumentation in delay.o
MAINTAINERS: arm64: Add Mark Rutland as an official Reviewer
arm64: smp: Fix IPI teardown for GICv5 flow
arm64: futex: Consolidate 'old == new' check in __lsui_cmpxchg32()
arm64: ftrace: allow DIRECT_CALLS without CALL_OPS
arm64: ftrace: prepare ftrace_modify_call() for use without CALL_OPS
|
|
* for-next/errata:
arch: arm64: add early_param idle=<wfi|yield|nop>
arm64: proton-pack: Restore the nospectre_bhb command-line option
arm64: errata: work around NVIDIA Olympus device store/load ordering
arm64: Clarify ARM64_WORKAROUND_REPEAT_TLBI semantics
|
|
* for-next/cpufeature:
arm64: bti: Disable in-kernel BTI with recent versions of Clang
iommu/arm-smmu-v3-sva: Use system_supports_bbml3() to detect CPU feature
arm64: cpufeature: Detect BBML3 based on ID_AA64MMFR2_EL1.BBM
arm64: cpufeature: Rename BBML2_NOABORT as BBML3
arm64: sysreg: Add BBM_3
arm64: cpufeature: Extend bbml2_noabort support list
arm64: cputype: Add C1-Nano definitions
arm64: cputype: Add Cortex-A520AE definitions
arm64: cpucaps: Remove stale comment about keeping capabilities sorted
arm64: fix cpu-feature-registers Malformed table
arm64: Remove hidden bitfields from cpu-feature-registers.rst
arm64: Sort registers in cpu-feature-registers.rst
arm64: Document missing bitfields in cpu-feature-registers.rst
arm64: Don't number registers in cpu-feature-registers.rst
|
|
The insn library encodes the immediate and shifted-register forms of
ADD/SUB but not the extended-register form. The BPF JIT wants it to
rebase a 32-bit arena offset onto the arena kernel base in a single
instruction, add xN, xBASE, wN, uxtw, instead of a separate zero-extend
followed by a plain add.
Add aarch64_insn_gen_add_sub_extended_reg(), modeled on the
shifted-register generator. The option and imm3 fields occupy the same
bits as the shifted form's shift amount, so they are encoded through the
existing IMM_6 field type. The opt field in bits 23:22 is part of the
opcode here rather than a shift type, and any value other than 00 is
unallocated, so the decode masks cover it.
Note that register 31 does not mean the same thing in the two forms: in
the extended-register encoding it is SP for Rn, and for Rd unless the
instruction sets the flags, while it stays XZR for Rm. Callers porting a
shifted-register site that passes A64_ZR need to be aware of that, so
say so above the function.
Signed-off-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Reviewed-by: Xu Kuohai <xukuohai@huawei.com>
Link: https://lore.kernel.org/bpf/20260813190356.335181-3-puranjay@kernel.org
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
TLB Invalidation by Range has a fairly powerful way of encoding pretty
large ranges in a small number of bits. This range can be based on an
arbitrary VA, which means it is pretty easy for a guest to generate an
overflow should the hypervisor be naive enough to add the range to the
base...
Make sure the range is capped to the limit dictated by the address bit
that determines the VA range. For an IPA invalidation, this is further
corrected down the line to ignore the upper range.
Fixes: 4ffa72ad8f37e ("KVM: arm64: nv: Add S1 TLB invalidation primitive for VNCR_EL2")
Reported-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://lore.kernel.org/r/yifz3wn5gk5sr6mapi32trgk5m5kp33bquctsjmkifebnsnndt@fix6u4rthx4g
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Reviewed-by: Wei-Lin Chang <weilin.chang@arm.com>
Link: https://patch.msgid.link/20260810170616.746100-1-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
`gic_arch_enable_irqs()` is only used when handling IRQs (which could
be pseudo-NMIs) and unmasking pseudo-NMIs.
The chain of `gic_pmr_mask_irqs()` and `gic_arch_enable_irqs()` for it
is slightly confusing without further explanation.
Remove `gic_arch_enable_irqs()` and instead do the whole pseudo-NMI
umasking in `gic_unmask_pnmis()`, making the operation explicit.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
In order to use preempt_count() to track the interrupt disable nesting
level, __preempt_count_{add,sub}_return() are introduced, as their names
suggest, these primitives return the new value of the preempt_count()
after changing it. The following example shows the usage of it in
local_interrupt_disable():
// increase the HARDIRQ_DISABLE bit
new_count = __preempt_count_add_return(HARDIRQ_DISABLE_OFFSET);
// if it's the first-time increment, then disable the interrupt
// at hardware level.
if ((new_count & HARDIRQ_DISABLE_MASK) == HARDIRQ_DISABLE_OFFSET) {
local_irq_save(flags);
raw_cpu_write(local_interrupt_disable_state, flags);
}
Having these primitives will avoid a read of preempt_count() after
changing preempt_count() on certain architectures.
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Heiko Carstens <hca@linux.ibm.com> # s390
Link: https://patch.msgid.link/20260804161447.84806-4-boqun@kernel.org
|
|
Having established that our VNCR TLB tracking was flawed and having
dropped it from KVM, it is time to replace it with something that
actually works in order to get some performance back, as the TLBI
overhead is otherwise pretty high.
The goal of such tracking is to hit the TLBI slow path if there are
any VNCR TLBs in the guest, irrespective of their mapping state.
For this purpose, we introduce an VM wide counter (vncr_tlb_count)
that tracks how many valid VNCR TLB are present. This means that
creating such TLB must increment the counter, and invalidation
decrement it, and both these operations must be done with the MMU lock
held for write.
On TLBI handling affecting EL2 S1, a non-zero counter forces the
handling to take the slow path to consider the VNCR TLBs. Note that
the bumping up is done "speculatively" in order to make sure that a
concurrent invalidation will be taken on the slow path.
Not exactly rocket science. Hopefully I got it right this time.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260806091026.620700-9-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
When the decode_range_tlbi() helper was moved to be used for S1 TLBIs,
the required sign extension was omitted. Add it.
As a result, special care must be taken to not overflow PA bits when
this is used for S2 invalidation.
Fixes: 85bba00425ae0 ("KVM: arm64: nv: Move TLBI range decoding to a helper")
Reported-by: sashiko-bot@kernel.org
Link: https://lore.kernel.org/r/20260801130337.EB2BA1F00AC4@smtp.kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260806091026.620700-7-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
We record a VNCR TLB even when SCTLR_EL2.M is 0 in order to make
our life easier. But this is not something that the architecture
anticipate.
As a consequence, a hypervisor is free to set VNCR_EL2 to
some PA when SCTLR_EL2.M==0, use it to run a guest which indirectly
accesses the VNCR page, then eventually set SCTLR_EL2.M==1 with
the same VA. Yes, this is odd, but apparently legal.
A common trick in HW is to invalidate the TLBs on SCTLR_ELx.M being
flipped. But doing this is a not a good idea for us (we'd need to
trap SCTLR accesses), and wouldn't scale as we nest deeper.
Instead, use the fact that the S1 MMU being off at the point of
translation is cached in our TLB, and if it doesn't match the current
MMU state, leave the VNCR unmapped.
Fixes: 2a359e072596f ("KVM: arm64: nv: Handle mapping of VNCR_EL2 at EL2")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260806091026.620700-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The global VNCR mapping counter is used to decide whether an L1
provided VNCR page is mapped in L0 on any CPU at the point of
dealing with a TLB invalidation. It is incremented when a mapping
is made in the fixmap, and decremented when unmapped.
As it turns out, this tracking has several flaws:
- we are trying to invalidate TLBs, and the mapping is only an
opportunistic consequence of the TLB. Checking this counter to
decide whether a TLB needs to be invalidated may result in missed
invalidations.
- an L1 vcpu invalidating its own TLB (a very likely case) will not
succeed in invalidating the VNCR pseudo TLB because that page is
not mapped in L0 at this stage.
Given that this tracking fails at delivering the minimum guarantees
that are required and is only a performance optimisation, remove it
completely.
Fixes: 4ffa72ad8f37e ("KVM: arm64: nv: Add S1 TLB invalidation primitive for VNCR_EL2")
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260806091026.620700-2-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM fixes from Andrew Morton:
"17 hotfixes. 15 are cc:stable. 16 are for MM.
There's a patch series from Lorenzo "mm: fix UAF caused by race
between ptdump and vmap pgtable freeing" which addresses a quite old
bug in the ptdump code.
And another series also from Lorenzo which fixes a four year old bug
in the huge_zero_folio handling.
A series from SJ fixes a few possible divide-by-zero issues which
Sashiko sniffed out. And a series which fixes handling of the
commit_inputs parameters.
The remainder are singletons, please see their changelogs for details"
* tag 'mm-hotfixes-stable-2026-08-06-18-44' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm:
mm/damon: adjust isolated pages stat for DAMOS_MIGRATE_{HOT,COLD}
mm/damon/ops-common: putback folios on invalid migrate nid
mm/huge_memory: initialise workingset state before folio split
mm/page_table_check: skip special zero mappings
mm/damon/lru_sort: skip damon_call() if ctx has not started
mm/damon/reclaim: skip damon_call() if ctx has not started
mm/damon/lru_sort: error out for >10000 active_mem_bp
samples/damon/mtier: error out for zero quota goal target values
mailmap: map old addresses to Danila Tikhonov
mm/huge_memory: separate out CONFIG_PERSISTENT_HUGE_ZERO_FOLIO logic
mm/huge_memory: fix huge_zero_pfn race
MAINTAINERS: update address for Brendan Jackman
mm/filemap: __filemap_add_folio() restore index before retrying
microblaze: restore the page alignment of swapper_pg_dir
arm64: remove redundant concurrent ptdump UAF mitigation
mm/ptdump: always stabilise against page table freeing using init_mm
mm/vmalloc: acquire init_mm lock on huge vmap to avoid ptdump UAF
|
|
On arm64, __early_set_fixmap(), __late_set_fixmap() and
__late_clear_fixmap() are all __set_fixmap(). Calling
early_ioremap_reset() changes nothing. So remove the call and the macros.
No functional change.
Link: https://lore.kernel.org/20260708170647.362562-4-ekffu200098@gmail.com
Signed-off-by: Sang-Heon Jeon <ekffu200098@gmail.com>
Acked-by: Will Deacon <will@kernel.org>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Most EL1 exceptions already call local_daif_mask() before returning,
with the exception of debug exception handlers which do not change
DAIF, and the IRQ/FIQ/Error handlers.
However, DAIF get masked in kernel_exit() in all cases when returning
from EL1 C handlers anyway.
Move this masking from assembly to C by calling local_daif_mask()
before irqentry_nmi_exit(). Unlike the raw DAIF masking helper,
local_daif_mask() invokes trace_hardirqs_off(), so it must execute
while RCU is still watching.
Remove the disable_daif assembly macro, as this was its only use.
Signed-off-by: Ada Couprie Diaz <ada.coupriediaz@arm.com>
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Last user of INIT_PSTATE_EL2 has gone with ae4b7e38e9a9 ("arm64: Allow
sticky E2H when entering EL1"), so remove it.
Signed-off-by: Vladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
- As bbml2_noabort is functionally equivalent to bbml3,
rename cpu/system_supports_bbml2_noabort to
cpu/system_supports_bbml3.
The ARM64 capability name is also renamed accordingly.
- As BBML2_NOABORT or the equivalent BBML3 is the
kernel requirement for setting up linear map with
block/contpte mappings and not BBML2, replace all
bbml2 references with bbml3.
FEAT_BBML3, is introduced as part of 2025 Architecture Extensions.
https://developer.arm.com/documentation/109697/2026_03/2025-Architecture-Extensions
No functional changes are introduced with this patch.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add cputype definitions for C1-Nano.
The definition can be found in C1-Nano TRM,
https://developer.arm.com/documentation/107753/0002
as part of MIDR_EL1 bit descriptions.
This is going to be used in the bbml3 support list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Add cputype definitions for Cortex-A520AE.
The definition can be found in Cortex-A520AE TRM,
https://developer.arm.com/documentation/107726/0001/
as part of MIDR_EL1 bit descriptions.
This is going to be used in the bbml3 support list.
Reviewed-by: Gavin Shan <gshan@redhat.com>
Reviewed-by: Anshuman Khandual <anshuman.khandual@arm.com>
Signed-off-by: Linu Cherian <linu.cherian@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
pte_present() is used as the basis for both pmd_present() and pud_present().
It is currently implemented as a macro composed of pte_val() and
pte_present_invalid().
When pte_present() or its higher-level variants are used directly with
ptep_get() or pXdp_get(), for example:
pte_present(ptep_get(pte));
pmd_present(pmdp_get(pmd));
pud_present(pudp_get(pud));
the macro expansion causes the compiler to evaluate the argument twice,
resulting in redundant loads. For example, pte_present() expands to:
!pte_val(READ_ONCE(*pte) || pte_present_invalid(READ_ONCE(*pte))
A typical example is pud_free_pmd_page(), where the expansion of
pmd_present() generates:
...
/* pmd_present() (x20 = pmdp) */
1b88: f9400288 ldr x8, [x20] // read pmdp.
1b8c: f9000fa8 str x8, [x29, #0x18]
1b90: 3707fec8 tbnz w8, #0x0, 0x1b68 <pud_free_pmd_page+0xd0>
1b94: f9400288 ldr x8, [x20] // redundant read of pmdp.
1b98: 8a170109 and x9, x8, x23
1b9c: f9000fa8 str x8, [x29, #0x18]
1ba0: f120013f cmp x9, #0x800
1ba4: 54fffe20 b.eq 0x1b68 <pud_free_pmd_page+0xd0>
1ba8: 17fffff4 b 0x1b78 <pud_free_pmd_page+0xe0>
...
Convert pte_present() to static inline function so that prevent the
generation of redundant code and move pte_valid() and
pte_present_invalid() further up so the inline function can use them.
After this change, the generated code becomes:
...
/* pmd_present() (x20 = pmdp) */
1a30: f9400288 ldr x8, [x20]
1a34: 8a170109 and x9, x8, x23
1a38: f9000fa8 str x8, [x29, #0x18]
1a3c: f120013f cmp x9, #0x800
1a40: 54fffe80 b.eq 0x1a10 <pud_free_pmd_page+0xd0>
1a44: 3607fee8 tbz w8, #0x0, 0x1a20 <pud_free_pmd_page+0xe0>
1a48: 17fffff2 b 0x1a10 <pud_free_pmd_page+0xd0>
...
This eliminates the redundant load and also reduces code size at
call sites using this pattern. For example, pud_free_pmd_page() shrinks
from 7,500 bytes to 7,148 bytes, a reduction of approximately 4.7%.
Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
This partially reverts commit fa93b45fd397 ("arm64: Enable vmalloc-huge
with ptdump"), retaining vmalloc-huge support but eliminating the now
redundant mitigation against a race between huge vmap page table freeing
and ptdump, as this issue has now been fixed at core.
We also simultaneously remove the arm64 if-deffery when acquiring the mmap
read lock upon vmap huge page table promotion as it is no longer required.
Note that this patch relies on the preceding vmalloc patch, and should not
be backported alone.
Link: https://lore.kernel.org/20260723-series-vmap-race-fix-v6-5-8cc77dcc0018@kernel.org
Fixes: fa93b45fd397 ("arm64: Enable vmalloc-huge with ptdump")
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Reviewed-by: Dev Jain <dev.jain@arm.com>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: "Borah, Chaitanya Kumar" <chaitanya.kumar.borah@intel.com>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Carlier <devnexen@gmail.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Toshi Kani <toshi.kani@hpe.com>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Userfaultfd RWP will reuse the uffd-wp PTE bit to mark access-tracking
PTEs, alongside the write-protected ones it already marks. The bit's
meaning now depends on the VMA flag (WP or RWP), not on its name.
Rename the kernel-internal names that describe the bit:
- pte/pmd/huge_pte accessors (and swap variants)
- pgtable_supports_uffd() capability query
- SCAN_PTE_UFFD khugepaged enum
The ftrace string emitted by mm_khugepaged_scan_pmd for this enum is
kept as "pte_uffd_wp" so existing trace-based tooling keeps matching.
Pure mechanical rename -- no behavior change.
Link: https://lore.kernel.org/20260708111417.173443-4-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: SeongJae Park <sj@kernel.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: James Houghton <jthoughton@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The uffd-wp PTE bit is about to gain a second consumer: userfaultfd RWP
will use the same bit to mark access-tracking PTEs, distinct from
mprotect(PROT_NONE) or NUMA-hinting PTEs. WP vs RWP semantics come from
the VMA flag; the bit is just "uffd has claimed this entry." Drop the
"_wp" suffix from the arch-private bit macros so they reflect that.
x86: _PAGE_BIT_UFFD_WP -> _PAGE_BIT_UFFD
_PAGE_UFFD_WP -> _PAGE_UFFD
_PAGE_SWP_UFFD_WP -> _PAGE_SWP_UFFD
arm64: PTE_UFFD_WP -> PTE_UFFD
PTE_SWP_UFFD_WP -> PTE_SWP_UFFD
riscv: _PAGE_UFFD_WP -> _PAGE_UFFD
_PAGE_SWP_UFFD_WP -> _PAGE_SWP_UFFD
Pure mechanical rename -- no behavior change.
Link: https://lore.kernel.org/20260708111417.173443-3-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: SeongJae Park <sj@kernel.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: James Houghton <jthoughton@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
Patch series "userfaultfd: working set tracking for VM guest memory", v10.
This series adds userfaultfd support for tracking the working set of VM
guest memory, so a VMM can identify hot pages and reclaim cold ones to
tiered or remote storage.
This patch (of 15):
pte_protnone() and pmd_protnone() detect present-but-inaccessible page
table entries. This capability is useful beyond NUMA balancing -- for
example, userfaultfd working set tracking uses protnone PTEs to track page
access without unmapping pages.
Introduce CONFIG_ARCH_HAS_PTE_PROTNONE to decouple the protnone PTE
infrastructure from CONFIG_NUMA_BALANCING. The six architectures that
support protnone PTEs (x86_64, arm64, powerpc, s390, riscv, loongarch) now
select this option, and CONFIG_NUMA_BALANCING depends on it.
No functional change -- the same set of architectures continues to have
working protnone support, but the infrastructure is now available
independently of NUMA balancing.
Link: https://lore.kernel.org/20260708111417.173443-1-kirill@shutemov.name
Link: https://lore.kernel.org/20260708111417.173443-2-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Assisted-by: Claude:claude-opus-4-6
Acked-by: SeongJae Park <sj@kernel.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Hildenbrand <david@kernel.org>
Cc: James Houghton <jthoughton@google.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
syscall_get_arguments() returns 'regs->orig_x0' for the first system
call argument so as to avoid aliasing with the syscall return value in
'regs->regs[0]' on the return path, however the actual syscall
invocation passes 'regs->regs[0]' as the first parameter.
Although the two registers should be kept in sync during syscall entry
for native tasks, pass 'regs->orig_x0' as the first syscall parameter
for consistency with the syscall argument APIs. Compat tasks continue to
use 'regs->regs[0]' for compatibility with the behaviour of the 32-bit
kernel.
Suggested-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
A CPU wedged with interrupts masked ignores the stop IPI, and without
pseudo-NMI there is no NMI IPI to escalate to: a reboot proceeds with
the CPU still running, and a kdump misses its registers.
Add a third rung to smp_send_stop(): once the IPI (and pseudo-NMI IPI,
if enabled) rungs have run, signal SDEI event 0 at whatever stayed
online. Firmware delivers it regardless of the target's DAIF, so it
reaches a CPU a plain IPI cannot; the target acks by going offline,
which the caller already polls for.
Fold the stop bookkeeping into one arm64_nmi_cpu_stop(regs,
die_on_crash), shared by the stop IPI handlers, panic_smp_self_stop()
and the SDEI handler, replacing the near-duplicate local_cpu_stop() and
ipi_cpu_crash_stop(). @die_on_crash is the only difference: the IPI
handlers pass true and PSCI CPU_OFF the CPU on a crash stop so a capture
kernel can reclaim it; the SDEI handler and self-stop pass false and
park. The SDEI park is required, not conservative -- its handler runs
inside an SDEI event that is never completed (completing it resumes the
wedged context), and a CPU_OFF from that unfinished-event context wedges
EL3 on some firmware (left as a follow-up). The dump is unaffected; only
re-onlining the CPU in an SMP capture kernel is lost.
Suggested-by: Douglas Anderson <dianders@chromium.org>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Deliver an NMI-like event to an interrupt-masked arm64 CPU via the
standard SDEI software-signalled event (event 0), without the pseudo-NMI
hot-path cost: register a handler for event 0 and poke a target with
sdei_event_signal(0, mpidr).
First user is arch_trigger_cpumask_backtrace() (sysrq-l, RCU stalls,
hung-task/soft-lockup dumps), which otherwise rides an IPI that can't
reach a masked CPU. Falls back to the IPI path when SDEI is absent; no
watchdog backend yet, so the stock detector is untouched.
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Tested-by: Yin Fengwei <fengwei_yin@linux.alibaba.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
In a very unsurprising turn of events, there is a large class of
firmware that is totally unable to deal with FEAT_NV3, and doesn't
set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception
or an unhandled trap to EL3, depending on the implementation.
Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac
and get a working system. Hopefully firmware will be fixed before
actually HW ships, but I have been there before... :-/
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-30-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs
that only affects the guest's S1 translation. Enable this, with the
express condition that the guest isn't NV2 aware, as we otherwise
need to trap these TLBIs to deal with VNCR mappings.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-26-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
