| Age | Commit message (Collapse) | Author |
|---|
|
Drop the local 10-byte buffer. The old code copied at most 9 bytes from
the user buffer, but only the first byte was used to select the RTAS
operation.
Use get_user() to read the command byte instead and compare it directly
with '0' and '1'. Drop the explicit user buffer check, since get_user()
will fail on a NULL pointer and correctly return -EFAULT instead of
-EINVAL. Remove the now-obsolete string constants as well as any
strncmp() and strlen() calls.
Return the original count instead of a potentially capped value, since
the full user write has been consumed once the command is accepted.
Use unsigned int op to better match the manage_flash() interface.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260528201226.1599977-3-thorsten.blum@linux.dev
|
|
On a machine without PCI memory-I/O (MIO) support or when running with
pci=nomio the s390 specific PCI MMIO write syscall checks if the MMIO
cookie is above ZPCI_IOMAP_ADDR_BASE as a sanity check before even
trying to perform the MMIO. If this check fails the return value was
left unchanged and thus 0 from prior operations falsely indicating
success. This could potentially confuse user-space into falsely
believing the MMIO, on a mapping not valid for MMIO was successful.
Fix this by setting the return value to -EFAULT prior to the check
following the same pattern as elsewhere in the same function.
Cc: stable@vger.kernel.org
Reviewed-by: Julian Ruess <julianr@linux.ibm.com>
Reviewed-by: Farhan Ali <alifm@linux.ibm.com>
Fixes: a67a88b0b8de ("s390/pci: remove races against pte updates")
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
|
|
Drop the redundant buffer assignment in serial_edit_cmdline() since the
cp pointer is immediately overwritten.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260528082357.1397611-3-thorsten.blum@linux.dev
|
|
Replace the manual string length accounting, memcpy(), and NUL
termination with a single snprintf() call to prepend the elfcorehdr=
address and to detect string truncation at the same time.
Use kmalloc() to avoid unnecessarily zeroing the memory. While at it,
also use "prepending" instead of "appending" in the error message.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260527000105.1081651-3-thorsten.blum@linux.dev
|
|
Now that all kvm entry to guest paths handle the task work
using the generic framework, enable HAVE_POSIX_CPU_TIMERS_TASK_WORK
which allows running posix cpu timers in task context instead of running
them in hardirq. This would is a necessary step towards enabling
PREEMPT_RT on powerNV systems.
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260709092140.1753715-5-vishalc@linux.ibm.com
|
|
Since commit 2cd571245b43 ("sched/fair: Add related data structure for
task based throttle") in v6.18, CFS bandwidth throttling no longer
dequeues a task directly; it queues task_work via TWA_RESUME and sets
TIF_NOTIFY_RESUME, relying on that work running before the task returns
to guest/user mode. The powerpc KVM run loops only checked for reschedule
and signals, never TIF_NOTIFY_RESUME, so the deferred throttle never ran
while a vCPU stayed in the run loop: a CPU-bound guest that rarely exits
to userspace ran far past its cpu.max quota and then appeared frozen for
minutes while the accrued throttle debt was repaid.
Use the generic infrastructure to check for and handle pending work
before transitioning into guest mode, replacing the open-coded
need_resched() and cond_resched() checks in the Book3S HV run loops and
in the common kvmppc_prepare_to_enter() used by the Book3S PR and BookE
run loops. The redundant signal_pending() recheck (and its sigpend label)
in kvmhv_run_single_vcpu() is also dropped, as
xfer_to_guest_mode_work_pending() is a superset of it.
This picks up handling for TIF_NOTIFY_RESUME, which was previously
ignored, meaning task work will now be correctly handled on every
guest re-entry.
Selecting VIRT_XFER_TO_GUEST_WORK disables RCU's last-resort self-IPI
fallback for vCPU tasks (see rcu_irq_work_resched()), which on
nohz_full CPUs was what forced a reschedule for deferred rcuog wakeups
queued right before guest entry. Take over that obligation the same way
x86 and s390 do: call xfer_to_guest_mode_prepare() with IRQs disabled
immediately before the final xfer_to_guest_mode_work_pending() check at
each guest-entry gate (kvmhv_run_single_vcpu(), kvmppc_run_core() and
kvmppc_prepare_to_enter()).
In kvmppc_prepare_to_enter(), IRQs are now disabled with
local_irq_disable() before hard_irq_disable(): on 32-bit,
hard_irq_disable() is a raw MSR[EE] clear that bypasses the
lockdep/irq-tracing state, and the strict xfer_to_guest_mode helpers
assert that IRQs are seen as disabled. This also allows upgrading the
racy __xfer_to_guest_mode_work_pending() check to the asserting
variant, as this loop is the terminal gate for the PR and BookE paths.
In kvmhv_run_single_vcpu(), the -EINTR exit and the pre-existing
kvmhv_setup_mmu() failure exit now leave via the done label instead of
returning directly, keeping the run_vcpu enter/exit tracepoints
balanced and vcpu->arch.ret consistent with the returned value.
In kvmppc_prepare_to_enter() the generic helper accounts the signal exit
(vcpu->stat.signal_exits and KVM_EXIT_INTR) but does not set the exit
type, so kvmppc_set_exit_type(SIGNAL_EXITS) is retained on the signal
path to preserve the E500 CONFIG_KVM_EXIT_TIMING histogram; it is a no-op
otherwise.
Signed-off-by: Vishal Chourasia <vishalc@linux.ibm.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260709092140.1753715-4-vishalc@linux.ibm.com
|
|
Enabling rust support for ppc64le.
Tested on pseries Power11:
╰─❯ dmesg | grep rust
[ 0.225728] Initialise system trusted keyrings
[ 0.270961] rust_minimal: Rust minimal sample (init)
[ 0.270968] rust_minimal: Am I built-in? true
[ 0.270974] rust_minimal: test_parameter: 1
[ 0.270983] rust_misc_device: Initialising Rust Misc Device Sample
[ 0.271012] rust_print: Rust printing macros sample (init)
[ 0.271019] rust_print: Emergency message (level 0) without args
[ 0.271023] rust_print: Alert message (level 1) without args
[ 0.271026] rust_print: Critical message (level 2) without args
[ 0.271030] rust_print: Error message (level 3) without args
[ 0.271033] rust_print: Warning message (level 4) without args
[ 0.271037] rust_print: Notice message (level 5) without args
[ 0.271040] rust_print: Info message (level 6) without args
[ 0.271043] rust_print: A line that is continued without args
[ 0.271054] rust_print: Emergency message (level 0) with args
[ 0.271064] rust_print: Alert message (level 1) with args
[ 0.271072] rust_print: Critical message (level 2) with args
[ 0.271077] rust_print: Error message (level 3) with args
[ 0.271083] rust_print: Warning message (level 4) with args
[ 0.271091] rust_print: Notice message (level 5) with args
[ 0.271097] rust_print: Info message (level 6) with args
[ 0.271102] rust_print: A line that is continued with args
[ 0.271110] rust_print: 1
[ 0.271113] rust_print: "hello, world"
[ 0.271121] rust_print: [samples/rust/rust_print_main.rs:35:5] c = "hello, world"
[ 0.271129] rust_print: Arc<dyn Display> says 42
[ 0.271130] rust_print: Arc<dyn Display> says hello, world
[ 0.271136] rust_print: "hello, world"
[ 0.271198] usbcore: registered new interface driver rust_driver_usb
[ 0.271207] rust_faux_driver: Initialising Rust Faux Device Sample
[ 0.271227] faux_driver rust-faux-sample-device: Hello from faux device!
[ 0.271297] rust_configfs: Rust configfs sample (init)
Reviewed-by: Link Mauve <linkmauve@linkmauve.fr>
Tested-by: Link Mauve <linkmauve@linkmauve.fr>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Tested-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Acked-by: Gary Guo <gary@garyguo.net>
Link: https://github.com/rust-lang/compiler-team/issues/987
Link: https://github.com/rust-lang/compiler-team/issues/988
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-8-mkchauras@gmail.com
|
|
For now only Big Endian 32-bit PowerPC is supported, as that is the only
hardware I have. This has been tested on the Nintendo Wii so far, but I
plan on also using it on the GameCube, Wii U and Apple G4.
These changes aren’t the only ones required to get the kernel to compile
and link on PowerPC, libcore will also have to be changed to not use
integer division to format u64, u128 and core::time::Duration, otherwise
__udivdi3() and __umoddi3() will have to be added. I have tested this
change by replacing the three implementations with unimplemented!() and
it linked just fine.
Signed-off-by: Link Mauve <linkmauve@linkmauve.fr>
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Acked-by: Gary Guo <gary@garyguo.net>
Link: https://github.com/rust-lang/compiler-team/issues/986
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-7-mkchauras@gmail.com
|
|
Added support for a new macro ARCH_STATIC_BRANCH_ASM in powerpc
to avoid duplication of inline asm between C and Rust. This is
inline with 'commit aecaf181651c ("jump_label: adjust inline asm to be consistent")'
Co-developed-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Reviewed-by: Alice Ryhl <aliceryhl@google.com>
Reviewed-by: Christophe Leroy (CS GROUP) <chleroy@kernel.org>
Reviewed-by: Gary Guo <gary@garyguo.net>
Link: https://github.com/Rust-for-Linux/linux/issues/105
Link: https://github.com/linuxppc/issues/issues/451
Signed-off-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
Signed-off-by: Madhavan Srinivasan <maddy@linux.ibm.com>
Link: https://patch.msgid.link/20260708082454.1254320-4-mkchauras@gmail.com
|
|
@lost and @topa_index are no longer part of struct pt_buffer so
delete their comment lines.
Warning: ../arch/x86/events/intel/pt.h:86 Excess struct member 'lost' description in 'pt_buffer'
Warning: ../arch/x86/events/intel/pt.h:86 Excess struct member 'topa_index' description in 'pt_buffer'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Link: https://patch.msgid.link/20260730233429.285788-2-rdunlap@infradead.org
|
|
Add a short description for imr_self_test() and drop the comments for
the non-existent imr_self_test_exit() to avoid kernel-doc warnings:
Warning: arch/x86/platform/intel-quark/imr_selftest.c:52 missing initial short description on line:
* imr_self_test
Warning: ../arch/x86/platform/intel-quark/imr_selftest.c:128 function parameter 'imr_self_test_init' not described in 'device_initcall'
Warning: ../arch/x86/platform/intel-quark/imr_selftest.c:128 expecting prototype for imr_self_test_exit(). Prototype was for device_initcall() instead
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260730233429.285788-6-rdunlap@infradead.org
|
|
These function parameters don't exist so drop them to avoid kernel-doc
warnings:
Warning: ../arch/x86/include/asm/spec-ctrl.h:28 Excess function parameter 'guest_spec_ctrl' description in 'x86_spec_ctrl_set_guest'
Warning: ../arch/x86/include/asm/spec-ctrl.h:42 Excess function parameter 'guest_spec_ctrl' description in 'x86_spec_ctrl_restore_host'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260730233429.285788-5-rdunlap@infradead.org
|
|
- correct missing function parameter names
- add missing function return value sections
- use the correct function name in comments
to avoid these warnings (samples):
Warning: arch/x86/include/asm/checksum_64.h:23 No description found for return value of 'csum_fold'
Warning: arch/x86/include/asm/checksum_64.h:46 function parameter 'iph' not described in 'ip_fast_csum'
Warning: arch/x86/include/asm/checksum_64.h:46 function parameter 'ihl' not described in 'ip_fast_csum'
Warning: arch/x86/include/asm/checksum_64.h:46 No description found for return value of 'ip_fast_csum'
Warning: arch/x86/include/asm/checksum_64.h:89 expecting prototype for csum_tcpup_nofold(). Prototype was for csum_tcpudp_nofold() instead
Warning: arch/x86/include/asm/checksum_64.h:115 expecting prototype for csum_tcpup_magic(). Prototype was for csum_tcpudp_magic() instead
Warning: arch/x86/include/asm/checksum_64.h:119 No description found for return value of 'csum_tcpudp_magic'
Warning: arch/x86/include/asm/checksum_64.h:129 No description found for return value of 'csum_partial'
Warning: ./arch/x86/include/asm/checksum_64.h:175 function parameter '_saddr' not described in 'csum_ipv6_magic'
Warning: ./arch/x86/include/asm/checksum_64.h:175 function parameter '_daddr' not described in 'csum_ipv6_magic'
Warning: ./arch/x86/include/asm/checksum_64.h:175 Excess function parameter 'saddr' description in 'csum_ipv6_magic'
Warning: ./arch/x86/include/asm/checksum_64.h:175 Excess function parameter 'daddr' description in 'csum_ipv6_magic'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260730233429.285788-4-rdunlap@infradead.org
|
|
Use the correct macro parameter names in the kernel-doc comments
to avoid kernel-doc warnings:
Warning: arch/x86/include/asm/barrier.h:35 function parameter 'idx' not described in 'array_index_mask_nospec'
Warning: arch/x86/include/asm/barrier.h:35 function parameter 'sz' not described in 'array_index_mask_nospec'
Warning: arch/x86/include/asm/barrier.h:35 Excess function parameter 'index' description in 'array_index_mask_nospec'
Warning: arch/x86/include/asm/barrier.h:35 Excess function parameter 'size' description in 'array_index_mask_nospec'
Signed-off-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://patch.msgid.link/20260730233429.285788-3-rdunlap@infradead.org
|
|
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
|
While the GCC and Clang compilers already define __ASSEMBLER__
automatically when compiling assembly code, __ASSEMBLY__ is a
macro that only gets defined by the Makefiles in the kernel.
This can be very confusing when switching between userspace
and kernelspace coding, or when dealing with uapi headers that
rather should use __ASSEMBLER__ instead. So let's standardize now
on the __ASSEMBLER__ macro that is provided by the compilers.
This is a completely mechanical patch (done with a simple "sed -i"
statement).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260619124936.208519-1-thuth@redhat.com
|
|
In a very unsurprising turn of events, there is a large class of
firmware that is totally unable to deal with FEAT_NV3, and doesn't
set the required SCR2_EL3.NV3En bit, leading to an UNDEF exception
or an unhandled trap to EL3, depending on the implementation.
Allow the unfortunate user to override ID_AA64MMFR4_EL1.NV_frac
and get a working system. Hopefully firmware will be fixed before
actually HW ships, but I have been there before... :-/
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-30-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Further enable FEAT_NV3 by making it visible to NV guests.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-28-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Now that everything is in place to engage the FEAT_NV3 fast-path,
add the detection code to cpufeature.c.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-27-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Similarly to the ERET elision mechanism, FEAT_NV3 can elide TLBIs
that only affects the guest's S1 translation. Enable this, with the
express condition that the guest isn't NV2 aware, as we otherwise
need to trap these TLBIs to deal with VNCR mappings.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-26-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
When running on NV3 HW, always engage ERET trap elision when running
the L1 context, as there is no benefit in not doing so.
An L1 can itself engage trap elision by setting its own view of
HCRX_EL2.NVTGE==1, which will subsequently be honored.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-25-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it
must be dealt with in some particular way:
- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly
reflects HCR_EL2 as read and written by the guest itself. It must
therefore be eagerly synced back with the emulation code which only
knows about HCR_EL2. This is unconditional if NV3 is available on
the host.
- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we
just context switch it like any other EL1 register. Yes, EL1, as
that's where this thing runs from the PoV of L1. This is conditioned
on the guest using NV3.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-24-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
NVHCR_EL2 accesses from EL1 are taken to EL2 when HCRX_EL2.NVTGE==0
and HCR_EL2.NV==1. Describe this in the exception routing tables.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-23-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Expose NVHCR_EL2 to userspace, and treat the direct access as UNDEF,
as that would only outline a bug in our exception routing.
The generic accessors are also updated to deal with the relatively
uncommon location of that register.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-22-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Just like any other VNCR-based register, NVHCR_EL2 requires some
level of sanitisation. Being specified as a live copy of HCR_EL2,
it adopts the exact same format, but depends on FEAT_NV3 instead.
A subtle aspect is that we only want to apply the sanitisation if
FEAT_NV3 is actually present, as the VNCR location is otherwise
used to back accesses to HCR_EL2.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-21-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
FEAT_NV3 makes a fundamental change to the architecture, by moving
guest-initiated HCR_EL2 accesses to the NVHCR_EL2 register. As the
names suggests, this is HCR_EL2 for a NV guest.
But where do NVHCR_EL2 accesses from a guest go? The are redirected
to the VNCR page, right where HCR_EL2 is stored in the NV2 case.
Does it hurt? Good. There's more coming.
The challenge here is to make KVM work seamlessly, without rewriting
everything. Which implies that things such as __vcpu_sys_reg(HCR_EL2)
must work, no matter the underlying NV implementation.
A simple way to deal with it is to move HCR_EL2's canonical storage
outside of VNCR for the vast majority of the KVM code, and only have
a copy at entry/exit times. Given that we don't really support NV3
yet, this is pretty simple.
In the process, advertise NVHCR_EL2 as the register that now holds
offset 0x78 in the VNCR page.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-20-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Add a new set of predicates indicating whether VM is capable of
NV2, NV3, and is in a nested NV3 context.
This is going to become useful as we start dealing with a mix of
behaviours (NV2, NV3, NV2 on NV3...).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-19-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Expose the FEAT_NV3 control bits to the sanitisation code so that
KVM stops moaning about the unattributed bits.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-18-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
In order to facilitate further changes, move the NV handling of
early fixups in its own helper. This also makes the code slightly
simpler to parse.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-17-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
As a bunch of KVM code is going to depend on FEAT_NV3 being detected
on the host, add a new capability that will describe it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-16-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
FEAT_NV3 introduces 4 new HCRX_EL2 control bits. Describe them
in the sysreg file.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-15-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
FEAT_NV3 introduces a new register that contains the HCR_EL2 value
exposed to a NV guest. As such, it has the exact same layout as
HCR_EL2.
Describe NVHCR_EL2 as a mapping to HCR_EL2.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-14-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Add the necessary NV2p1 probing to the cpufeature infrastructure.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-13-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Since NV2p1 is reducing the number of traps, it is valuable to expose
it to NV guests. Do so.
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-12-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
With NV2p1, it is no longer necessary to use the split approach
where bits of CNTHCTL_EL2 cannot be accessed via CNTKCTL_EL1,
and we can treat the CNTKCTL_EL1 accessor as if it was "normal".
Key the special casing on FEAT_NV2P1 not being implemented.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-11-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses
via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.
Prevent such trapping and context switch CPACTR_EL1 in NV contexts
when NV2P1 is present.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-10-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
As we're about to deal with FEAT_NV2P1, add a new capability that
will be used to key any support for it.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-9-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
We currently avoid using the ERET fast path if the guest has HCR_EL2.NV
set. This is an odd check, as NV doesn't mean much if HCR_EL2.TGE==1.
Similarly, evaluating HFGITR_EL2.ERET makes little sense, as
this only applies to the nested context, while the ERET fast-path
is purely for the benefit of L1.
Replace these bizarre checks with is_nested_ctxt() which makes a lot
more sense: if we are running an L2, the ERET trap must go to L1.
Fixes: dd0717a998f77 ("KVM: arm64: nv: Fast-track 'InHost' exception returns")
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-8-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
It may not be obvious unless you look at it closely, but CPTR_EL2
is treated very differently from other registers. It is one the
registers that, despite looking very similar between EL1 and EL2
when E2H==1, have RES0 bits that get in the way.
Make it clear that CPTR_EL2 is odd by classifying it as SR_LOC_SPECIAL,
just like CNTHCTL_EL2 (and for the same reasons). This makes it
possible to use vcpu_read_sys_reg() with it, and will be necessary
once we support FEAT_NV2P1.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-7-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
HCRX_EL2.SRMASKEn is a new bit enabling FEAT_SRMASK for a guest.
We don't plan to support it any time soon, but it doesn't hurt to
actively document it, specially as we are going to add more bits
we actually care about.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-6-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The __HCRX_EL2_* masks are a leftover from a time where we didn't
have much sanitisation for the system registers. Since we are now
in a better place, rely on the existing checks to detect unhandled
bits in HCRX_EL2.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-5-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The way we merge the guest-provided HCRX_EL2 value with the host's
is bonkers. We try to make it look like the FGT registers by using
positive and negative polarities for traps, but most of these bits
are not strictly about trapping, as they actively change the way
some architectural state is managed.
It would be far better to deal with these bits like we do for
HCR_EL2, by enumerating the list of bits we don't allow the guest
to override. This is simplified by the fact that HCRX_EL2 only
affects EL1, and not EL2.
Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
from the merge (TMEA, PTTWI, EnIDCP128).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
ID_AA64MMFR4_EL1 has gained a few fields and enum values in the past
few months, so resync its definition with the 2026-03 JSON release.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-3-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The sysreg file is using the Mapping or Fields qualifiers to indicate
that a given encoding is only a mapping to a particular register (or
an instance of a more generic register definition).
As a result, we don't output any definition, and instead expect
the canonical definitions to be used.
This works rather well for individual fields, but creates problems
for macros that refer to more generic classes of bits such as RESx.
Relax the above rule by emitting the RESx and UNKN values for Mapping
and Fields qualifiers as well.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-2-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Under pKVM the stage-2 walker resolves to pkvm_pgtable_stage2_split(), a
WARN_ON_ONCE(1) stub, yet KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE is still
enabled and reported for non-protected guests: the capability check keys
on the per-VM protected state while the walker dispatch keys on the
host-global mode. Enabling the cap and then dirty-logging the guest
reaches the stub, splatting a userspace-reachable WARN.
Reject the capability, and stop reporting a chunk size and the
supported block sizes, for every VM once pKVM is enabled, keyed on the
host-global mode like the split dispatch. Gating only protected VMs
would leave the non-protected guests that reach the stub still able to
enable it. Userspace decides whether eager splitting is available from
the block-size bitmap (QEMU falls back to no eager splitting when it
reads 0), so leaving it advertised steers an explicit request into the
enable failure instead of the fallback.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-7-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pkvm_pgtable_stage2_map() wraps the __pkvm_host_share_guest() and
__pkvm_host_donate_guest() return in WARN_ON(), but those hypercalls
fail for reasons that are not EL1 invariant violations: -EINVAL for a
pfn that is not memblock RAM (check_range_allowed_memory() rejects a
device page mapped into a non-protected guest) and -ENOMEM under
memcache pressure. Both are reachable from a guest fault, so the WARN
splats on host input.
Return the error without warning. The unshare and write-protect WARNs
stay, since a failure there does signal a broken EL1 invariant.
Fixes: 3669ddd8fa8b5 ("KVM: arm64: Add a range to pkvm_mappings")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-6-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
pkvm_pgtable_stage2_flush() cleans the D-cache for every mapping in the
range even on hardware with stage-2 Force Write-Back, where FWB keeps
guest memory coherent to the PoC and the maintenance is unnecessary. The
generic kvm_pgtable_stage2_flush() returns early in that case, but the
pKVM MMU does not, so it needlessly cleans the whole range on, e.g.,
every set/way trap.
Return early when FWB is enabled, matching the generic walker.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-5-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Dirty logging forces new stage-2 mappings to page size but does not
always split an existing block first (eager splitting is best effort
and off by default). A non-write permission fault on such a block, an
instruction fetch, still needs a page-table page to split it, but the
top-up is gated on write faults.
With the cache empty, kvm_mmu_memory_cache_alloc() hits its
guest-triggerable WARN_ON(!nobjs) and falls back to a GFP_ATOMIC
allocation under mmu_lock, with a BUG_ON() if that fails.
Top up the memcache for any permission fault while dirty logging is
active.
Fixes: 6f745f1bb5bf ("KVM: arm64: Convert user_mem_abort() to generic page-table API")
Link: https://lore.kernel.org/all/20260623165634.699011F000E9@smtp.kernel.org/
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: reword the commit message for the generic, non-pKVM failure mode]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
A permission fault normally only relaxes a leaf, so user_mem_abort()
skips the memcache top-up. Under pKVM such a fault can instead collapse
pages into a block. That needs a fresh pkvm_mapping object, and without
it cache->mapping is NULL, so pkvm_pgtable_stage2_map() dereferences NULL
and faults the host under mmu_lock. Staging only the object is not
enough: the hypervisor requires kvm_mmu_cache_min_pages in the memcache
even for the allocation-free install, so under memcache pressure the
collapse returns -ENOMEM and trips the WARN_ON(ret) in
pkvm_pgtable_stage2_map() where a non-pKVM guest succeeds.
Top up the full memcache for pKVM permission faults so both the mapping
object and the min-pages are staged before mmu_lock.
Fixes: db14091d8f75 ("KVM: arm64: Stage-2 huge mappings for np-guests")
Reported-by: Bradley Morgan <include@grrlz.net>
Link: https://lore.kernel.org/all/20260623161545.EA08E1F000E9@smtp.kernel.org/
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The pKVM flush path walks its own pkvm_mappings list and cleans the
data cache for every mapping, unlike the generic stage-2 walker it
shadows, which skips non-cacheable leaves. Cleaning the cacheable
alias of a non-cacheable mapping is pointless and can corrupt a
device endpoint. Record whether a mapping is non-cacheable in spare
bits of nr_pages and skip cache maintenance for it.
Fixes: e912efed485a ("KVM: arm64: Introduce the EL1 pKVM MMU")
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Bradley Morgan <include@grrlz.net>
[tabba: use Marc's anonymous bitfield in place of the open-coded mask and helpers]
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Bradley Morgan <include@grrlz.net> # On pixel 7, Android 17 CP2A.260705.006
Test: Bradley Morgan <include@grrlz.net> # On QEMU arm64 host
Link: https://patch.msgid.link/20260717130317.1953574-2-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
