| Age | Commit message (Collapse) | Author |
|---|
|
https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 changes for 7.3
- Add support for 'slot' based PMU events, paired with new UAPI that
compels the user to select a specific PMU implementation
- Lazy save/restore of vCPU state for pKVM, along with various fixes
and cleanups to the management of vCPU state between the untrusted
host and pKVM hypervisor
- Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1
is present, guaranteeing that EL2-specific register bits are stateful
in the EL1 counterpart
- Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope
of those instructions remains 'in host' (i.e. L1 kernel/userspace)
- Pile of fixes for the management of the VNCR pseudo-TLB, such as
under-invalidations and races with concurrent TLBIs on other vCPUs
- Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a
runtime-patched constant, allowing the same data to be shared with
pKVM prior to dropping host privileges
- Considerable pile of LLM-assisted fixes around the shop but mostly in
the VGIC, our in-kernel generator of bugs (and sometimes interrupts)
|
|
* kvm-arm64/misc-7.3:
: Miscellaneous fixes for KVM/arm64, 7.3
:
: - Fixes for saving invalid table entries as part of saving the ITS
: tables (Fuad Tabba)
:
: - Don't reallocate the SPI array for re-attempted vgic_init(), avoiding
: a memory leak (Fuad Tabba)
:
: - Hold a reference on an LPI when saving the pending state (Qihang)
:
: - Don't WARN for out-of-range, guest-supplied INTID (Karl)
:
: - Avoid corrupting GPRs for 32-bit CP64 reads (Karl)
:
: - Reset 'in kernel' VGIC state when private IRQ allocation fails (Fuad)
:
: - Avoid kallsyms lookup in nVHE panic unless the host stage-2 is also
: disabled (Vincent)
:
: - Disregard Pending+Active state when computing maintenance IRQ for
: ICH_MISR_EL2.NP (Kajetan)
:
: - Various Sashiko-identified issues dealing with GICv5 (Sascha)
:
: - Fix CPU onlining in pKVM due to mismatched accesses when the MMU is
: disabled (Will)
KVM: arm64: Validate GICv5 timer PPIs before claiming ownership
KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs
KVM: arm64: vgic: Prevent speculative SPI array underflow
KVM: arm64: vgic: Free gic_kvm_info on initialization failure
KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params'
KVM: arm64: vgic: Fix detection of MI on no pending LR
KVM: arm64: Drop %pB on nVHE panic when stage-2 is active
KVM: arm64: vgic: Reset in_kernel on private IRQ allocation failure
KVM: arm64: GICv2: Don't WARN on out-of-range GICV_DIR INTID
KVM: arm64: Preserve GPRs for AArch32 CP64 reads generating an UNDEF
KVM: arm64: vgic-v3: take an LPI reference in vgic_v3_save_pending_tables
KVM: arm64: vgic-its: Point saved ITEs at the next valid entry
KVM: arm64: vgic-its: Don't save collections the table cannot hold
KVM: arm64: vgic: Don't leak the SPI array when init is retried
KVM: arm64: vgic-its: Don't dereference a NULL collection on ITT save
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/vtr-patch:
: Inline patching of ICH_VTR_EL2 constant, courtesy of Marc Zyngier
:
: Unify readers of ICH_VTR_EL2 on an instruction-patched constant value,
: avoiding system register accesses known to trap under nested
: virtualization and sharing the implementation between pKVM and 'regular'
: KVM.
KVM: arm64: vgic-v3: Kill kvm_vgic_global_state.ich_vtr_el2
KVM: arm64: vgic-v3: Simplify initial GICv3 configuration sampling
KVM: arm64: Convert most ICH_VTR_EL2 accesses to inlined literal value
KVM: arm64: Add a helper providing an inlined literal value for ICH_VTR_EL2
KVM: arm64: Move GICv3 broken SEIS implementation detection to a CPU errrata
KVM: arm64: vgic-v3: Make vtr_to_* helpers use architectural field symbols
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
GICv5 supports up to 128 PPIs, but KVM currently implements only the
first 64, which contain the architected PPIs it supports.
An encoded PPI with an ID outside that range passes irq_is_ppi(),
which only checks the encoded interrupt type. vgic_get_vcpu_irq()
therefore looks it up in private_irqs[], where array_index_nospec()
clamps the out-of-range index to zero and aliases PPI 0.
Include the supported PPI range in irq_is_ppi() so that KVM interfaces
reject unsupported PPIs. Also reject an out-of-range PPI in the lookup
as a safeguard against callers bypassing the predicate.
Fixes: 4d591252bacb ("KVM: arm64: gic-v5: Implement PPI interrupt injection")
Fixes: eb8bce08ecb1 ("KVM: arm64: gic: Introduce interrupt type helpers")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=27
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-4-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
For a non-GICv5 VM, SPI interrupt IDs include the private-interrupt
offset, while KVM's SPI array is indexed from zero. The lookup applies
array_index_nospec() to the absolute interrupt ID and subtracts the
private-interrupt offset afterwards.
On a speculative bypass of the range check for an interrupt ID below
the private range, the clamp preserves the small absolute value and
the subtraction underflows to an out-of-bounds SPI array index.
Convert the interrupt ID to a zero-based index into the SPI array
before applying array_index_nospec(). This way, we ensure that we
clamp to a reachable SPI ID, rather than an out-of-range SPI index.
Fixes: 41b87599c743 ("KVM: arm/arm64: vgic: fix possible spectre-v1 in vgic_get_irq()")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=27
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-3-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
vgic_set_kvm_info() allocates gic_kvm_info for use by
kvm_vgic_hyp_init(). When a maintenance interrupt is mandatory but not
provided, kvm_vgic_hyp_init() returns -ENXIO before reaching the
common cleanup path, leaking said allocation.
Route this error through that cleanup path so that gic_kvm_info is
freed and the global pointer is cleared.
Fixes: 0e5cb7770684 ("irqchip/gic: Split vGIC probing information from the GIC code")
Link: https://sashiko.dev/#/patchset/20260724104819.1296803-1-sascha.bischoff@arm.com?part=1
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260811150941.941295-2-sascha.bischoff@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
As per the ARM GICv3 spec, the maintenance interrupt identified by
ICH_MISR_EL2.NP is asserted when it is enabled and no List register is
in pending state. This is further described in the document as "no List
registers with the State field set to 0b01 (pending)". By checking only
the pending bit of the LR (bit 62), KVM currently asserts the MI when
there are no LRs in "pending" or "pending and active" states.
Fix the detection logic to consider only the "pending" state.
Cc: stable@vger.kernel.org
Fixes: 96c2f03311de ("KVM: arm64: nv: Plumb handling of GICv3 EL2 accesses")
Signed-off-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260810102923.2426475-2-kajetan.puchalski@arm.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
kvm_vgic_create() sets vgic.in_kernel before allocating the per-vCPU
private IRQs, but the allocation-failure path resets only vgic_model and
leaves in_kernel set. As irqchip_in_kernel() is !!in_kernel, the VM is
left with an in-kernel irqchip but no model, and the -EEXIST guard at the
top of kvm_vgic_create() rejects every retry, so userspace cannot recover
from a transient -ENOMEM.
Reset in_kernel alongside vgic_model on the failure path.
Fixes: 9435c1e1431003 ("KVM: arm64: gic: Set vgic_model before initing private IRQs")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Acked-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260802150845.3485757-1-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
vgic_v2_deactivate() passes the INTID a guest wrote to GICV_DIR straight
to vgic_get_vcpu_irq(), and treats a failed lookup as a "can't happen"
condition with WARN_ON_ONCE().
The guest can make it happen at will, though: for any INTID outside of
the implemented SGI, PPI and SPI ranges the lookup returns NULL, since
GICv2 has no LPIs. A guest running with EOImode==1 writing such an INTID
to GICV_DIR triggers the WARN, and panics hosts running with
panic_on_warn.
Drop the WARN and ignore failed lookups.
Fixes: 255de897e7fb ("KVM: arm64: GICv2: Handle deactivation via GICV_DIR traps")
Cc: stable@vger.kernel.org
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Link: https://patch.msgid.link/20260726174803.5880-1-kmehltretter@gmail.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
vgic_v3_save_pending_tables() iterates dist->lpi_xa using xa_for_each()
and dereferences the returned struct vgic_irq in the loop body without
holding a reference on the LPI.
The xarray iterator only provides temporary RCU coverage while looking up
the current entry. That is not sufficient for this loop body, which reads
fields from struct vgic_irq and performs guest memory accesses before the
iteration completes.
A concurrent path can trigger this race: the irqfd cached injection path
(vgic_its_inject_cached_translation) obtains a transient LPI reference
via vgic_its_check_cache() without holding kvm->lock, vcpu->mutex,
config_lock, or its_lock. If guest ITS DISCARD then drops the cache and
ITE references under its_lock, the transient inject reference may become
the final one. When vgic_put_irq() drops it, the LPI is erased from
lpi_xa and freed via kfree_rcu(). Meanwhile, vgic_v3_save_pending_tables()
may still hold a stale pointer obtained from the xarray iterator and
dereference it after the RCU grace period completes.
Fix this by re-fetching each iterated LPI via vgic_get_irq(), which takes
a stable reference, and dropping it with vgic_put_irq() on all paths.
This matches the pattern already used by other lpi_xa iterators in the
vgic ITS code.
Cc: stable@vger.kernel.org
Signed-off-by: Qihang <q.h.hack.winter@gmail.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260807025534.34125-1-q.h.hack.winter@gmail.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
An ITE whose collection was dropped is saved as an invalid entry, and
vgic_its_restore_ite() has no offset to follow from one, so the scan
steps a single entry at a time until it reaches a valid entry or the
end of the ITT.
Compute the offset to the next ITE that is saved as valid instead.
Suggested-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260807104102.2410744-5-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
A guest that disables the ITS and rewrites GITS_BASER with fewer pages,
VALID still set, keeps every collection it mapped against the larger
table: KVM stores the new BASER unconditionally and frees the list only
when VALID is cleared. vgic_its_save_collection_table() then walks the
whole list, writing up to 448K past the end of the table, and saves
collection IDs that vgic_its_restore_cte() rejects, so the save succeeds
and the restore fails with -EINVAL on the destination. The overrun stays
in guest memory, as vgic_write_guest_lock() validates every gfn.
Validate each collection against the current table with
vgic_its_check_id() and return -EINVAL, as vgic_its_save_device_tables()
does for devices. Collection IDs are unique and the collection table is
never indirect, so the check also bounds the walk.
Fixes: ea1ad53e1e31a ("KVM: arm64: vgic-its: Collection table save/restore")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260807104102.2410744-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Nothing latches a failed vgic_init(), so userspace can retry
KVM_DEV_ARM_VGIC_CTRL_INIT after a failure past kvm_vgic_dist_init().
kvm_vgic_setup_default_irq_routing() is the reachable case, running on
every configuration. Each retry overwrites dist->spis and only the last
allocation is freed at teardown, leaking up to 960 struct vgic_irq,
about 90KB, per attempt.
Return early when the array is already allocated, as
vgic_allocate_private_irqs_locked() and vgic_v4_init() do.
Fixes: ad275b8bb1e65 ("KVM: arm/arm64: vgic-new: vgic_init: implement vgic_init")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260807104102.2410744-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
MAPC with V=0 drops ite->collection but leaves the ITE on the device's
ITT list, and vgic_its_save_ite() dereferences it unconditionally. A
guest that issues MAPD, MAPTI and then MAPC(V=0) therefore oopses the
host when the VMM issues KVM_DEV_ARM_ITS_SAVE_TABLES to migrate it.
That sequence is UNPREDICTABLE per the architecture, but KVM already
handles the resulting state in the translate, MOVI and DISCARD paths.
Save a zeroed entry, which vgic_its_restore_ite() reads back as
invalid. Skipping the ITE instead would leave the ITT slot holding
whatever is in guest memory, and restore rejects an entry naming a
collection the restored collection table does not have.
Fixes: eff484e0298da ("KVM: arm64: vgic-its: ITT save and restore")
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260807104102.2410744-2-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Mitigate a potential failure when inserting a new LPI into the VGIC LPI
xarray.
When vgic_add_lpi() is preparing to register a new LPI, it pre-allocates
an xarray entry using xa_reserve_irq(), so that it can later perform the
insertion under the xarray lock without allocating.
However, since xa_reserve_irq() is called before acquiring such lock,
there is a potential race where xa_reserve_irq() observes a populated
entry, thus not performing the allocation, and another CPU removes that
entry before the xarray lock is grabbed to perform the insertion.
CPU0 (Adding new LPI) CPU1 (Releasing LPI)
===================== ===================
vgic_add_lpi()
/* Entry populated, does not allocate */
xa_reserve_irq(.., intid, ..)
vgic_release_deleted_lpis()
xa_lock_irqsave()
vgic_release_lpi_locked()
xarray node freed --> __xa_erase(.., intid)
xa_unlock_irqrestore()
xa_lock_irqsave()
xa_load(.., intid) == NULL
vgic_try_get_irq_ref(NULL) == false
__xa_store(.., intid, irq, 0) <-- xarray node was freed, gfp=0
cannot allocate, returns -ENOMEM
This can happen e.g. if the guest issues a DISCARD while the LPI is
still referenced from a vCPU's active-pending list (ap_list), and the
same INTID is re-mapped via MAPTI.
Mitigate this by passing GFP_NOWAIT to __xa_store(), so that the
allocation can happen under the lock in the rare case that this
condition is hit. Add __GFP_ACCOUNT as well to match xa_reserve_irq()'s
flags.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Fixes: 1d6f83f60f79 ("KVM: arm64: vgic: Store LPIs in an xarray")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260715105137.3973823-5-clopez@suse.de
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Fix a potential race between decrementing an LPI's reference count and
evicting that structure from the LPI xarray.
LPI structures are maintained in the VGIC LPI xarray (dist->lpi_xa).
When the reference count of an LPI structure drops to zero,
vgic_release_lpi_locked() removes the structure from the xarray and
frees it under the xarray lock.
However, the release of an LPI can race with a concurrent LPI
re-registration with the same INTID via vgic_add_lpi() on another CPU,
since the reference count drop and the xarray eviction are not performed
in a single atomic step. This can happen e.g. if the guest issues a
DISCARD while the LPI is still referenced from a vCPU's active-pending
list (ap_list), and the same INTID is re-mapped via MAPTI.
Particularly, vgic_release_lpi_locked() is called from two distinct
paths: direct release via vgic_put_irq(), and deferred release via
vgic_release_deleted_lpis(). During direct release, the issue can result
in deleting a newly registered LPI from the xarray:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq()
__vgic_put_irq()
refcount_dec_and_test()
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(old_irq) == false
new IRQ inserted --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
xa_lock_irqsave();
vgic_release_lpi_locked()
__xa_erase(.., irq->intid) <-- BUG: new IRQ is erased
kfree_rcu(old_irq)
During the deferred release path, the old IRQ can be leaked:
CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq_norelease()
__vgic_put_irq()
refcount_dec_and_test()
irq->pending_release = true
vgic_add_lpi()
xa_lock_irqsave()
old_irq = xa_load(.., intid)
vgic_try_get_irq_ref(oldirq) == false
BUG: old IRQ overwritten --> __xa_store(.., intid, ..)
xa_unlock_irqrestore()
vgic_release_deleted_lpis()
xa_lock_irqsave()
xa_for_each() { .. } <-- old IRQ with pending_release = true
is gone, so it cannot be released
To fix the direct release path, move the reference count drop inside
the xarray lock, making sure that vgic_add_lpi() never encounters the
to-be-released LPI.
In the deferred release path, the refcount drop must happen under a raw
spinlock, so the xarray lock cannot be grabbed, and the same solution
does not work. Instead, update vgic_add_lpi(), so that if it evicts
an LPI from the xarray, it takes on the responsibility of freeing it.
Consequently, an LPI may now be freed concurrently after a deferred
release drops the refcount, so accessing the pending_release field is no
longer safe from use-after-free. Delete all uses of the flag, and update
vgic_release_deleted_lpis() to identify orphaned LPIs purely based on
their refcount.
Reported-by: Claude:claude-opus-4-6
Fixes: 3a08a6ca7c37 ("KVM: arm64: vgic-v3: Use bare refcount for VGIC LPIs")
Fixes: d54594accf73 ("KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260715105137.3973823-4-clopez@suse.de
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
kvm_vgic_global_state.ich_vtr_el2 is the last bit of caching that
we can get rid of. Not as bad as a sysreg access, but still worse
than a constant.
Move over to the inlined stuff and remove the cached value.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260721170754.3150521-7-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Now that we have our magic inline helper for ICH_VTR_EL2, we can
get rid of the hack that was reporting a combination of that
register and of the indication of the CPU interface supporting
GICv2 compatibility. We now only report the latter.
As a small benefit, GICv5 is not involved in this stuff anymore,
since it never has GICv2 compatibility..
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260721170754.3150521-6-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
We already have two ways to deal with ICH_VTR_EL2:
- directly read the system register
- read a cached copy in the vgic global state
Add a third way, in the form of an inlined literal value that will
eventually replace all of the above. This literal value is computed
at boot time, and patched in the relevant code locations.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260721170754.3150521-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Using is_midr_in_range_list() in a noinstr function isn't a good idea.
And yet kvm_compute_ich_hcr_trap_bits() is doing that by calling
vgic_v3_broken_seis().
Move all the broken SEIS detection logic to the errata detection
framework, and use a new ARM64_WORKAROUND_GICv3_BROKEN_SEIS cap
to indicate that we're running on broken CPUs.
This reuses the MIDR list used for IMPDEF PMU detection, which
has a 100% overlap with the SEIS stuff...
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260721170754.3150521-3-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
In the nested state, the physical interrupt has already been
deactivated through the HW bit in the LR. The extra deactivation
would be harmless but can hit an errata case on AmpereOne, so
avoid it here.
On AmpereOne, deactivating a physical interrupt through
ICC_DIR_EL1 or ICC_EOIR1_EL1 (depending on EOImode) which is not
active, but is the highest priority pending interrupt causes the
cpu to lose the interrupt pending state and also prevents the
delivery of future interrupts.
Fixes: 6dd333c8942b2 ("KVM: arm64: GICv3: nv: Plug L1 LR sync into deactivation primitive")
Signed-off-by: D Scott Phillips <scott@os.amperecomputing.com>
Cc: stable@vger.kernel.org
Link: https://lore.kernel.org/linux-arm-kernel/20260710222128.416581-1-scott@os.amperecomputing.com/
Link: https://patch.msgid.link/20260714231158.496808-1-scott@os.amperecomputing.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 fixes for 7.2, take #2
- Move locking for kvm_io_bus_get_dev() into the caller, ensuring
race-free checks that the returned object is of the correct type
- Fix initialisation of the page-table walk level when relaxing
permissions
- Correctly update the XN attribute when relaxing permissions
- Fix the sign extension of loads from emulated MMIO regions
- Assorted collection of fixes for pKVM's FFA proxy, together with a
couple of FFA driver adjustments
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 fixes for 7.2, take #1
- Fix an accounting buglet when reclaiming pages from a protected
guest
- Fix a bunch of architectural compliance issues when injecting a
synthesised exception, most of which were missing the PSTATE.IL bit
indicating a 32bit-wide instruction
- Another set of fixes addressing issues with translation of VNCR_EL2,
including corner cases where the guest point that register at a RO
page...
- Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as
that's not unexpected at all
- Address a bunch of races with LPI migration vs LPIs being disabled
- Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in
exception returning in the wrong place...
- Coerce Fuad Tabba into a reviewer role, and may his Inbox catch
fire!
|
|
kvm_io_bus_get_dev() returns a device that is only matched by the
address, and nothing else. This can cause a lifetime issue if
the matched device is not the expected type, as by the time
the caller can introspect the object, it might be gone (the srcu
lock having been dropped).
Given that there is only a single user of this helper, the simplest
option is to move the locking responsibility to the caller, which
can keep the srcu lock held for as long as it wants.
Note that this aligns with other kvm_io_bus*() helpers, which
already require the srcu lock to be held by the callers.
Reported-by: Will Deacon <will@kernel.org>
Fixes: 8a39d00670f07 ("KVM: kvm_io_bus: Add kvm_io_bus_get_dev() call")
Link: https://lore.kernel.org/all/20260626111344.802555-1-maz@kernel.org
Cc: stable@vger.kernel.org
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260627105105.1005990-1-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
vgic_prune_ap_list() drops both ap_list_lock and irq_lock while migrating
an interrupt to another vCPU. After reacquiring the locks it only checks
that the affinity is unchanged (target_vcpu == vgic_target_oracle(irq))
before moving the interrupt, which assumes that an interrupt whose affinity
is preserved is still queued on this vCPU's ap_list.
That assumption no longer holds if the interrupt is taken off the ap_list
while the locks are dropped. vgic_flush_pending_lpis() removes the
interrupt from the list and sets irq->vcpu to NULL, but leaves
enabled/pending/target_vcpu untouched. As the interrupt is still enabled
and pending, vgic_target_oracle() returns the same target_vcpu, so the
affinity check passes and list_del() is run a second time on an entry that
has already been removed.
Also check that the interrupt is still assigned to this vCPU
(irq->vcpu == vcpu) before moving it.
Fixes: 0919e84c0fc1 ("KVM: arm/arm64: vgic-new: Add IRQ sync/flush framework")
Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com>
Link: https://patch.msgid.link/aiHnI1mu6SGQrgnz@v4bel
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
|
|
disabling
Hyunwoo Kim reports some really bad races should the following
situation occur:
- LPI-I is pending in vcpu-B's AP list
- vcpu-A writes to vcpu-B's RD to disable its LPIs
- vcpu-C moves I from B to C
If the last two race nicely enough, vgic_prune_ap_list() can drop
the irq and AP list locks, reacquire them, and in the interval
the irq has been freed. UAF follows.
The fix is two-fold:
- Before dropping the irq and ap_list locks, take a reference on
the irq
- Do not try to handle migration of the pending bit: there is no
expectation that this state is retained, as per the architecture
With that, we're sure that the interrupt is still around, and we
safely remove it from the AP list as it has no target at this
stage (unless another interrupt fires, but that's another story).
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Tested-by: Hyunwoo Kim <imv4bel@gmail.com>
Link: https://lore.kernel.org/r/ailsCnyoS82r_QRz@v4bel
Link: https://patch.msgid.link/20260615181625.3029352-1-maz@kernel.org
Fixes: 5dd4b924e390a ("KVM: arm/arm64: vgic: Add refcounting for IRQs")
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.2
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
|
|
* kvm-arm64/misc-7.2:
: .
: - Check for a valid vcpu pointer upon deactivating traps when handling
: a HYP panic in VHE mode
:
: - Make the __deactivate_fgt() macro use its arguments instead of the
: surrounding context
:
: - Don't bother with initialising TPIDR_EL2 in the hyp stubs, as this
: is already taken care of in more obvious places
:
: - Drop the unused kvm_arch pointer passed to __load_stage2()
:
: - Return -EOPNOTSUPP when a hypercall fails for some reason, instead of
: returning whatever was in the result structure
:
: - Make the ITS ABI selection helpers return void, which avoids wondering
: about the nature of the return code (always 0)
: .
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: arm64: Set a Linux errno on SMCCC error in kvm_call_hyp_nvhe()
KVM: arm64: Remove @arch from __load_stage2()
KVM: arm64: Don't populate TPIDR_EL2 in finalise_el2()
KVM: arm64: Fix __deactivate_fgt macro parameter typo
KVM: arm64: Guard against NULL vcpu on VHE hyp panic path
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/vgic-v5-PPI-fixes:
: .
: Substantial cleanup of the vgic-v5 PPI support. From the original
: cover letter:
:
: "With the GICv5 PPi support merged in, it has become obvious that a few
: things could be improved, both from the correctness and maintainability
: angles."
: .
KVM: arm64: Fix arch timer interrupts for GICv3-on-GICv5 guests
irqchip/gic-v5: Immediately exec priority drop following activate
Documentation: KVM: Clarify that PMU_V3_IRQ IntID requirements for GICv5
Documentation: KVM: Fix typos in VGICv5 documentation
KVM: arm64: selftests: Improve error handling for GICv5 PPI selftest
KVM: arm64: selftests: Cleanup unused vars in GICv5 PPI selftest
KVM: arm64: selftests: Add missing GIC CDEN to no-vgic-v5 selftest
KVM: arm64: vgic-v5: Atomically assign bits to PPI DVI bitmap
KVM: arm64: vgic-v5: Add missing trap handing for NV triage
KVM: arm64: vgic-v5: Limit support to 64 PPIs
KVM: arm64: vgic: Rationalise per-CPU irq accessor
KVM: arm64: vgic-v5: Drop defensive checks from vgic_v5_ppi_queue_irq_unlock()
KVM: arm64: vgic: Consolidate vgic_allocate_private_irqs_locked()
KVM: arm64: vgic: Constify struct irq_ops usage
KVM: arm64: vgic-v5: Drop pointless ARM64_HAS_GICV5_CPUIF check
KVM: arm64: vgic-v5: Remove use of __assign_bit() with a constant
KVM: arm64: vgic-v5: Move PPI caps into kvm_vgic_global_state
KVM: arm64: vgic-v5: Add for_each_visible_v5_ppi() iterator
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
The return values of vgic_its_set_abi() and vgic_its_commit_v0() are always
0 and do not carry useful error information. Simplify by changing them to
void.
Suggested-by: Oliver Upton <oupton@kernel.org>
Signed-off-by: Jackie Liu <liuyun01@kylinos.cn>
Reviewed-by: Oliver Upton <oupton@kernel.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Link: https://patch.msgid.link/20260604075147.53299-1-liu.yun@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
erased entry
vgic_its_invalidate_cache() walks the per-ITS translation cache with
xa_for_each() and drops the cache's reference on each entry with
vgic_put_irq(). It puts the iterated pointer, though, rather than the
value returned by xa_erase().
The function is called from contexts that do not exclude one another: the
ITS command handlers hold its_lock, the GITS_CTLR write path holds
cmd_lock, and the path that clears EnableLPIs in a redistributor's
GICR_CTLR holds neither. Two or more of them can drain the same cache
concurrently, and if each one observes the same entry, erases it and then
puts it, the single reference the cache holds on that entry is dropped
more than once. The entry can then be freed while an ITE still maps it.
xa_erase() is atomic and returns the previous entry, so put only the entry
that this context actually removed. The cache reference is then dropped
exactly once per entry even when the invalidations run concurrently, and
the behavior is unchanged when only one context runs.
Fixes: 8201d1028caa ("KVM: arm64: vgic-its: Maintain a translation cache per ITS")
Signed-off-by: Hyunwoo Kim <imv4bel@gmail.com>
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/ah2c5lu4JbUg7dj-@v4bel
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
|
|
For GICv5 guests we make use of the DVI mechanism for PPIs where
possible. When mapping a virtual irq to a physical one for a GICv5
guest, the corresponding bit in the DVI bitmap is set. When unmapping,
said bit is cleared again. The key user of this mechanism is the arch
timer.
The existing code used the non-atomic __assign_bit() rather than doing
the update atomically. This could technically result in losing state
if a second PPI's DVI bit were being manipulated concurrently. Each
individual bit within the DVI bitmap is guarded using
vgic_irq->irq_lock, but there's no locking for the overall
bitmap. Therefore, switch to using the atomic assign_bit() function
instead.
Fixes: 5a98d0e17e59 ("KVM: arm64: gic-v5: Implement direct injection of PPIs")
Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-12-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Although we have some code supporting 128 PPIs, the only supported
configuration is 64 PPIs. There is no way to test the 128 PPI code,
so it is bound to bitrot very quickly.
Given that KVM/arm64's goal has always been to stick to non-IMPDEF
behaviours, drop the 128 PPI support. Someone motivated enough and
with very strong arguments can always bring it back -- it's all in
the git history.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-10-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Despite adding the necessary infrastructure to identify irq types,
vgic_get_vcpu_irq() treats GICv5 PPIs in a special way, which
impairs the readability of the code.
Use the existing irq classifiers to handle per-CPU irqs for all
vgic types, and let the normal control flow reach global interrupt
handling without any v5-specific path.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-9-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
vgic_v5_ppi_queue_irq_unlock() performs a bunch of sanity checks
that are pretty pointless as there is no code path that can
result in these invariants to be violated. And if they are, a nice
crash is just as instructive than a warning.
Drop what is evidently debug code and simplify the whole thing.
Link: https://lore.kernel.org/r/20260520091949.542365-8-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
vgic_allocate_private_irqs_locked() calls two helpers, oddly named
vgic_{,v5_}allocate_private_irq().
Not only these helpers don't allocate anything, but they also
contain duplicate init code that would be better placed in the
caller.
Consolidate the common init code in the caller, rename the helpers
to vgic_{,v5_}setup_private_irq(), and pass the irq pointer around
instead of the index of the interrupt.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-7-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
vgic-v5 has introduced much more prevalent usage of the struct
irq_ops mechanism.
In the process, it becomes evident that suffers from two related
problems:
- it contains flags, rather than only callbacks
- it is mutable, because we need to update the above flags
Swap the flags for a helper retrieving the flags, and make all
irq_ops const, something that is slightly satisfying.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-6-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
vgic_v5_get_implemented_ppis() can only be called when we have
a GICv5, by construction.
Remove the pointless check against ARM64_HAS_GICV5_CPUIF.
Link: https://lore.kernel.org/r/20260520091949.542365-5-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Using __assign_bit() is very useful when the value of the bit is
not known at compile time. In all other cases, __set_bit() and
__clear_bit() are the correct tool for the job.
This also fixes an odd case of using VGIC_V5_NR_PRIVATE_IRQS as
the bit value...
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://lore.kernel.org/r/20260520091949.542365-4-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Constant vgic properties are usually kept in kvm_vgic_global_state,
but the vgic-v5 code does its own thing.
Move the ppi_caps data into the global structure, which has the
modest additional advantage of making it ro_after_init.
Link: https://lore.kernel.org/r/20260520091949.542365-3-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
We have multiple instances of iterators walking the vgic_ppi_mask
mask, and the way it is written has a tendency to make one's eyes
bleed.
Factor it as a helper and use that across the code base.
Link: https://lore.kernel.org/r/20260520091949.542365-2-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
We now have the lazy init on three paths:
- on first run of a vcpu
- on first injection of an interrupt from userspace and irqfd
- on first injection of an interrupt from kernel space as
part of the device emulation (timers, PMU, vgic MI)
Given that we recompute the state of each in-kernel interrupt
every time we are about to enter the guest, we can drop the lazy
init from the kernel injection path.
This solves a bunch of issues related to vgic_lazy_init() being called
in non-preemptible context, such as vcpu reset.
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260520100200.543845-7-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Make sure that any attempt to inject an interrupt from userspace
or an irqfd results in the GICv2 lazy init to take place.
This is not currently necessary as the init is also performed on
*any* interrupt injection. But as we're about to remove that,
let's introduce it here.
Reviewed-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260520100200.543845-6-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Userspace can restore an ITS Device Table Entry whose Size field encodes
more EventID bits than the virtual ITS supports. The live MAPD path
rejects that state, but vgic_its_restore_dte() accepts it and stores the
out-of-range value in dev->num_eventid_bits.
Reject restored DTEs with num_eventid_bits > VITS_TYPER_IDBITS before
allocating the device. This mirrors the MAPD check and prevents the
restored state from reaching vgic_its_restore_itt(), where the unchecked
value can be converted into an oversized scan_its_table() range.
Fixes: 57a9a117154c ("KVM: arm64: vgic-its: Device table save/restore")
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Michael Bommarito <michael.bommarito@gmail.com>
Link: https://lore.kernel.org/r/20260519132519.2142458-1-michael.bommarito@gmail.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
|
|
The uaccess write handlers for GICD_IIDR in both GICv2 and GICv3
extract the revision field from 'reg' (the current IIDR value read back
from the emulated distributor) instead of 'val' (the value userspace is
trying to write). This means userspace can never actually change the
implementation revision — the extracted value is always the current one.
Fix the FIELD_GET to use 'val' so that userspace can select a different
revision for migration compatibility.
Fixes: 49a1a2c70a7f ("KVM: arm64: vgic-v3: Advertise GICR_CTLR.{IR, CES} as a new GICD_IIDR revision")
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260407210949.2076251-2-dwmw2@infradead.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
|
|
GICv5 supports up to 128 PPIs, which would introduce a large amount of
overhead if all of them were actively tracked. Rather than keeping
track of all 128 potential PPIs, we instead only consider the set of
architected PPIs (the first 64). Moreover, we further reduce that set
by only exposing a subset of the PPIs to a guest. In practice, this
means that only 4 PPIs are typically exposed to a guest - the SW_PPI,
PMUIRQ, and the timers.
When folding the PPI state, changed bits in the active or pending were
used to choose which state to sync back. However, this breaks badly
for Edge interrupts when exiting the guest before it has consumed the
edge. There is no change in pending state detected, and the edge is
lost forever.
Given the reduced set of PPIs exposed to the guest, and the issues
around tracking the edges, drop the tracking of changed state, and
instead iterate over the limited subset of PPIs exposed to the guest
directly.
This change drops the second copy of the PPI pending state used for
detecting edges in the pending state, and reworks
vgic_v5_fold_ppi_state() to iterate over the VM's PPI mask instead.
Signed-off-by: Sascha Bischoff <sascha.bischoff@arm.com>
Link: https://patch.msgid.link/20260401162152.932243-1-sascha.bischoff@arm.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Although the AArch32 ID regs are architecturally UNKNOWN when AArch32
isn't supported at any EL, KVM makes a point in making them RAZ.
Therefore, advertising GICv3 in ID_PFR1_EL1 must be gated on AArch32
being supported at least at EL0.
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Fixes: a258a383b9177 ("KVM: arm64: gic-v5: Sanitize ID_AA64PFR2_EL1.GCIE")
Reported-by: Mark Brown <broonie@kernel.org>
Tested-by: Mark Brown <broonie@kernel.org>
Link: https://patch.msgid.link/20260401103611.357092-16-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
The vgic-v5 code added some evaluations of the timers in a helper funtion
(kvm_cpu_has_pending_timer()) that is called to determine whether
the vcpu can wake-up.
But looking at the timer there is wrong:
- we want to see timers that are signalling an interrupt to the
vcpu, and not just that have a pending interrupt
- we already have kvm_arch_vcpu_runnable() that evaluates the
state of interrupts
- kvm_cpu_has_pending_timer() really is about WFIT, as the timeout
does not generate an interrupt, and is therefore distinct from
the point above
As a consequence, revert these changes and teach vgic_v5_has_pending_ppi()
about checking for pending HW interrupts instead.
Fixes: 9491c63b6cd7b ("KVM: arm64: gic-v5: Enlighten arch timer for GICv5")
Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com
Link: https://patch.msgid.link/20260401103611.357092-13-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
kvm_vgic_map_resources() targetting a v5 model results in vgic->dist_ready
never being set. This doesn't result in anything really bad, only
some more heavy locking as we go and re-init something for no good reason.
Rejig the code to correctly set the ready flag in all non-failing
cases.
Reviewed-by: Sascha Bischoff <sascha.bischoff@arm.com>
Fixes: f4d37c7c35769 ("KVM: arm64: gic-v5: Create and initialise vgic_v5")
Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com
Link: https://patch.msgid.link/20260401103611.357092-11-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
The way the effective priority mask is compared to the priority of
an interrupt to decide whether to wake-up or not, is slightly odd,
and breaks at the limits.
This could result in spurious wake-ups that are undesirable.
Make the computed priority mask comparison a strict inequality, so
that interrupts that have the same priority as the mask are not
signalled.
Fixes: 933e5288fa971 ("KVM: arm64: gic-v5: Check for pending PPIs")
Link: https://sashiko.dev/#/patchset/20260319154937.3619520-1-sascha.bischoff%40arm.com
Link: https://patch.msgid.link/20260401103611.357092-10-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
