| Age | Commit message (Collapse) | Author |
|---|
|
Move "struct kvm_vcpu_hv" and all of its child structures to hyperv.h,
guarded by CONFIG_KVM_HYPERV=y, as "struct kvm_vcpu_arch" holds a pointer
to the structure, i.e. only needs the structure to be declared, not fully
defined.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move the definition of "struct kvm_apic_map", a.k.a. the optimized local
APIC map, to lapic.h, as it is very nearly an implementation details that's
internal to KVM's local APIC emulation (KVM also uses the map to do quick
lookups when a vCPU is yielding to a different vCPU).
No functional change intended.
Suggested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Relocate KVM's TASK_SWITCH_<reason> enums from kvm_host.h to x86.h, as the
enums are arbitrary values, i.e. not architectural, and are intended to be
used only to translate vendor specific information to a common x86 reason
when invoking kvm_task_switch().
Opportunistically name the overall enum to help document the role of the
values.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Add static asserts to sanity check the I/O permission map and TSS size
macros against tss_segment_32. Alternatively, the macros could simply use
offsetof() and sizeof(), but having literal numbers makes it easier to
understand the bigger picture, and provides a good excuse for the sanity
checks.
Opportunistically add the necessary includes to make tss.h self sufficient.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move KVM_GUESTDBG_VALID_MASK into x86.c so that it's not globally visible.
As explained by commit 462474588b19 ("KVM: x86: Move misc "VALID MASK"
defines from kvm_host.h => x86.c"), which unintentionally missed GUESTDBG,
the set of valid flags/bits is very much a KVM-internal detail, as the
values from the hardcoded #defines are often captured and massaged by KVM's
setup code, i.e. *directly* using the macros outside of KVM x86 would be
actively dangerous.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Relocate a variety of Control/Debug Register macros that unintentionally
got left behind when the related helper function prototypes were moved to
regs.h.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Add an 'S' to msrs.h's macro guard so that both the file and guard names
are plural.
No functional change intended.
Fixes: 7a2683080158 ("KVM: x86: Move the bulk of MSR specific code from x86.c to msrs.{c,h}")
Reported-by: Binbin Wu <binbin.wu@linux.intel.com>
Closes: https://lore.kernel.org/all/ead7d7fd-aa4e-4c18-b399-90fb448e0af6@linux.intel.com
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Tag tdp_enabled with __read_mostly as the variable is only ever written
during vendor module load, same as all the other global MMU variables that
are handled by kvm_configure_mmu().
Opportunistically annotate the tdp_mmu_enabled and eager_page_split
declarations with __read_mostly, to match their definitions. The compiler
will warn if there are conflicting annotations, i.e. there's minimal risk
of the declaration annotation becoming stale.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move the macros that define the mostly-obsolete apic_attention bits into
lapic.c, as the gory details of PV EOIs and the pre-APICv TPR acceleration
are 100% internal to KVM's local APIC emulation.
No functional change intended.
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260625220450.3354415-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
kvm_init_debug() runs before several steps that can fail
(kvm_vfio_ops_init(), kvm_gmem_init(), kvm_init_virtualization() and
misc_register()), but none of the corresponding error labels remove the
"kvm" debugfs directory. Any failure after kvm_init_debug() therefore
leaks the directory and its stat files for the lifetime of the boot.
kvm_exit() already calls debugfs_remove_recursive(kvm_debugfs_dir); add
the same at the err_vfio label, whose fall-through covers every path
taken after kvm_init_debug().
Fixes: 2b0128127373 ("KVM: Register /dev/kvm as the _very_ last thing during initialization")
Signed-off-by: leixiang <leixiang@kylinos.cn>
Link: https://patch.msgid.link/20260706095910.39798-1-leixiang@kylinos.cn
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When activating Hyper-V's Synthetic Interrupt Controller (SynIC), mark it
active with WRITE_ONCE() and query it using READ_ONCE() in synic_get(),
the only known cross-task reader, to document that the flag is accessed
without holding the vCPU's mutex.
Note, there are no data dependencies on the SynIC being marked active,
e.g. the vector read by synic_set_irq() is set (usually in response to
guest activity) long after the SynIC is initially activated, and a false
negative on the SynIC being active would be benign (ignoring that such a
race is likely to be problematic for the guest irrespective of what KVM
does).
Link: https://patch.msgid.link/20260630225619.511632-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Assert that either vcpu->mutex is held or the VM is otherwise unreachable
when using the normal vCPU => HyperV accessor to help detect improper
cross-task usage of the HyperV structure. When accessing the structure
without holding the vCPU's mutex, e.g. to send interrupts or to queue TLB
flushes, KVM needs to use the more paranoid to_hv_vcpu_safe() to guarantee
that it can't see a half-baked structure.
To avoid false positives, open code accesses to vcpu->arch.hyperv in the
Synthetic Timer callbacks (can be reached if and only if HyperV state is
fully initialized).
Link: https://patch.msgid.link/20260630225619.511632-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
In the "vCPU locked or unreachable" lockdep assertion, treat a vCPU as
unreachable if its index is invalid, i.e. if the vCPU is in the process of
being created. Until the vCPU is inserted into the array of vCPUs, the
only way to get at the vCPU is via kvm_vm_ioctl_create_vcpu(). Note, the
actual index is set _before_ adding the vCPU to the array, i.e. there's no
risk of a false negative on the lockdep assertion.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Extract nVMX's lockdep assertion that a vCPU is locked or otherwise
unreachable into a common helper, as KVM x86 is about to gain another user,
but there is nothing x86-specific about the logic, i.e. the assertion may
be useful for other architectures.
No functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Invalidate a vCPU's index immediately after allocating storage for the vCPU
so that KVM doesn't incorrectly treat a vCPU that is the process of being
created as being vCPU0. This will also allow detecting that a vCPU is in
the process of being created and thus otherwise unreachable, which is
useful for avoiding false positives in lockdep assertions on vcpu->mutex.
Unwind the index back to -1 if inserting the vCPU into the array or adding
the vCPU to the fd table fails, so that kvm_arch_vcpu_destroy() sees the
vCPU as unreachable, i.e. so that teardown logic doesn't hit false positive
lockdep assertions. Opportunistically add a comment to call out that the
"real" index needs to be set before making the vCPU visible to other tasks.
Note, kvm_wait_for_vcpu_online() naturally does the right thing thanks to
vcpu->vcpu_idx and kvm->online_vcpus being signed values.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Explicitly invalidate KVM's internal Xen vCPU ID during vCPU creation
instead of *trying* to set the Xen ID to the vCPU index by default, and
forward singleshot timer hypercalls to userspace if the VMM hasn't set the
Xen ID via KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID. Using the vCPU's index as its
default Xen ID is reasonable in concept, but in practice is horribly flawed
as the index is left as '0' until after vCPU initialization completes, i.e.
every vCPU gets a Xen ID of '0' by default.
Forward hypercalls to userspace instead of trying to salvage any kind of
default behavior, as all userspace implementations that support multiple
vCPUs either don't enable the timer, are guaranteed to set Xen ID, or work
only because *all* guests also screw up the singleshot timer hypercalls.
The last scenarios is extremely unlikely given that Linux-as-a-guest uses
the actual Xen vCPU ID when making timer hypercalls. In other words, for
all intents and purposes, KVM's ABI is already that userspace must set the
Xen vCPU ID, so just commit to that ABI.
Note, KVM's handling of KVM_XEN_VCPU_ATTR_TYPE_VCPU_ID restricts the ID to
KVM_MAX_VCPUS, so there's no chance of a valid ID colliding with U32_MAX.
Add a compile-time assertion to ensure this holds true in the future (KVM
doesn't care what value is used for "invalid", only that there can't be a
collision).
Link: https://lore.kernel.org/all/20260612233017.1F9771F000E9@smtp.kernel.org
Suggested-by: David Woodhouse <dwmw2@infradead.org>
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260630225619.511632-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Hoist the checks on the Xen vCPU ID when handling set_singleshot_timer and
stop_singleshot_timer hypercalls out of their individual if-statements,
so that both checks on the ID are in common code. kvm_xen_hcall_vcpu_op()
is already doubly committed to handling only singleshot timer hypercalls,
and even if that were to change in the future, the function could simply
be renamed and turned into a helper specifically for timer hypercalls.
Opportunistically add a comment to explain why the check exists; the code
looks rather nonsensical without the knowledge that @vcpu_id is a common
param for all per-vCPU hypercalls.
No functional change intended.
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When handling Xen vCPU hypercalls, explicitly route non-singleshot-timer
commands to userspace, *before* checking if in-kernel emulation of the Xen
timer is enabled. Punting hypercalls that are never accelerated by KVM
because some other hypercall happens to be disabled is confusing and
actively dangerous, e.g. it's easy to miss that the only reason KVM can
bail early is because the timer-disabled case provides the same semantics
as the implicit "default" path in the switch-statement.
Opportunistically convert the switch-statement to an if-else-statement to
avoid having to carry code for an impossible "default" case.
For all intents and purposes, no functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
accesses
When initializing a vCPU's Hyper-V object, ensure the object is fully
initialized prior to exposing it through the vCPU, and ensure accesses from
other tasks (e.g. other vCPUs) see the fully initialized object if
vcpu->arch.hyperv is non-NULL.
Lack of ordering manifests as a lockdep splat due to attempting to lock a
TLB flush FIFO before the spinlock is initialized.
INFO: trying to register non-static key.
The code is fine but needs lockdep annotation, or maybe
you didn't initialize this object before use?
turning off the locking correctness validator.
CPU: 1 PID: 5005 Comm: syz-executor189 Not tainted 6.6.120-smp-DEV #1
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 03/18/2026
Call Trace:
<TASK>
[<ffffffff810dd10c>] dump_stack_lvl+0xcc/0x130 lib/dump_stack.c:106
[<ffffffff8192bddd>] assign_lock_key+0x1fd/0x230 kernel/locking/lockdep.c:977
[<ffffffff8191cb97>] register_lock_class+0x187/0x7a0 kernel/locking/lockdep.c:1291
[<ffffffff8191e7a9>] __lock_acquire+0x179/0x7650 kernel/locking/lockdep.c:5016
[<ffffffff8191e28f>] lock_acquire+0x13f/0x3d0 kernel/locking/lockdep.c:5756
[<ffffffff8101a65b>] __raw_spin_lock include/linux/spinlock_api_smp.h:133 [inline]
[<ffffffff8101a65b>] _raw_spin_lock+0x2b/0x40 kernel/locking/spinlock.c:154
[<ffffffff81319d44>] spin_lock include/linux/spinlock.h:351 [inline]
[<ffffffff81319d44>] hv_tlb_flush_enqueue+0xb4/0x270 arch/x86/kvm/hyperv.c:1946
[<ffffffff813160c6>] kvm_hv_flush_tlb+0xa96/0x1dc0 arch/x86/kvm/hyperv.c:2145
[<ffffffff8131438b>] kvm_hv_hypercall+0x103b/0x1fe0 arch/x86/kvm/hyperv.c:-1
[<ffffffff8133bff3>] __vmx_handle_exit arch/x86/kvm/vmx/vmx.c:6624 [inline]
[<ffffffff8133bff3>] vmx_handle_exit+0x12e3/0x21f0 arch/x86/kvm/vmx/vmx.c:6641
[<ffffffff81215d11>] vcpu_enter_guest arch/x86/kvm/x86.c:11649 [inline]
[<ffffffff81215d11>] vcpu_run+0x4d01/0x79c0 arch/x86/kvm/x86.c:11832
[<ffffffff8120fe39>] kvm_arch_vcpu_ioctl_run+0xb49/0x1c80 arch/x86/kvm/x86.c:12179
[<ffffffff8119cd60>] kvm_vcpu_ioctl+0xc80/0xff0 virt/kvm/kvm_main.c:6029
[<ffffffff8226fefd>] vfs_ioctl fs/ioctl.c:52 [inline]
[<ffffffff8226fefd>] __do_sys_ioctl fs/ioctl.c:872 [inline]
[<ffffffff8226fefd>] __se_sys_ioctl+0xfd/0x170 fs/ioctl.c:858
[<ffffffff85ac97d9>] do_syscall_x64 arch/x86/entry/common.c:52 [inline]
[<ffffffff85ac97d9>] do_syscall_64+0x69/0xb0 arch/x86/entry/common.c:93
[<ffffffff85c000d0>] entry_SYSCALL_64_after_hwframe+0x68/0xd2
</TASK>
Use the "safe" variant in all paths that are known to access the Hyper-V
object, as detected by an upcoming lockdep assertion, with an assist or two
from Sashiko.
Link: https://lore.kernel.org/all/20260612232258.0D9131F000E9@smtp.kernel.org
Fixes: 0823570f0198 ("KVM: x86: hyper-v: Introduce TLB flush fifo")
Fixes: fc08b628d7c9 ("KVM: x86: hyper-v: Allocate Hyper-V context lazily")
Reported-by: syzbot+5b32c49cd8f005e65654@syzkaller.appspotmail.com
Reported-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Closes: https://lore.kernel.org/all/6a396a66.52ae72c2.136ac7.0002.GAE@google.com
Tested-by: syzbot+5d2b94b77112148d1744@syzkaller.appspotmail.com
Link: https://patch.msgid.link/20260630225619.511632-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
kvm_hv_get_tlb_flush_fifo()
Check for a NULL Hyper-V object in kvm_hv_get_tlb_flush_fifo() instead of
relying on the caller to do so. This will allow fixing a cross-vCPU race
where KVM can access a vCPU's FIFO before it's fully initialized, without
having to jump through too many cognitive hoops to reason about the
correctness of the logic.
Ignoring changes in ordering that only affect the aforementioned race, no
functional change intended.
Reviewed-by: Philippe Mathieu-Daudé <philmd@oss.qualcomm.com>
Link: https://patch.msgid.link/20260630225619.511632-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When handling Hyper-V PV TLB flushes, retrieve the to-be-used FIFO in
hv_tlb_flush_enqueue() instead of having the caller pass in the FIFO. This
will make it easier to fix a cross-vCPU race where KVM can access a vCPU's
FIFO before it's fully initialized.
No functional change intended.
Link: https://patch.msgid.link/20260630225619.511632-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Micro-optimize emulation of MOV DR instructions by checking CR4.DE if and
only if DR4 or DR5 is being accessed.
No functional change intended.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
WARN if ->set_dr() => kvm_set_dr() fails when emulating a MOV DR write,
as the emulator _must_ pre-check for #GPs in order to get the event
priority right when emulating MOV DR for L2 on SVM (all exceptions have
higher priority than the instruction intercept).
Opportunistically update the comment as the blurb about "#UD" being
checked is incomplete and misleading.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Use kvm_dr{6,7}_valid() to validate the incoming DR{4,5,6,7} value in the
emulator instead of open coding an equivalent check. In the unlikely event
that the behavior of DR6/7 (and their aliases) changes in the future, using
common helpers will hopefully make it less likely the emulator logic will
be overlooked.
No functional change intended.
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When emulating a MOV DR on Intel with DR7.GD=1 at CPL>0, prioritize the #DB
due to DR7.GD over the #GP due to CPL>0, as empirical testing shows that
Intel CPUs (Skylake, Icelake and Emerald Rapids) prioritize the DR7.GD #DB
over all #GPs, whereas AMD CPUs prioritize the CPL>0 #GP (but not illegal
value #GPs) over the #DB.
Outside of the emulator, don't bother trying to provide the "correct"
priority based on the virtual CPU model, as it's simply impossible to do
so without intercepting *all* MOV DR accesses, which would result in a
massive, unacceptable performance hit. Note, getting the priority right
when advertising Intel on AMD would also require intercepting #GP, as SVM
prioritizes all exceptions over the instruction intercept.
Note, neither Intel's SDM nor AMD's APM says anything about the relative
priority, hence the empirical testing. Arguably Intel's description of
DR7.GD:
causes a debug exception to be generated prior to any MOV instruction
that accesses a debug register.
implies that DR7.GD has higher priority. But that's a fairly weak argument
as the statement would still hold true if the #GP due to CPL>0 had higher
priority, as the #GP would prevent any access to a DR.
Fixes: 3b88e41a4134 ("KVM: SVM: Add intercept check for accessing dr registers")
Link: https://patch.msgid.link/20260612230113.684301-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Manually handle the CPL check for MOV DR instructions instead of using the
Priv flag, *after* checking for #UD scenarios, as #GP due to CPL>0 has
lower priority than all #UDs.
Fixes: 1e470be5a108 ("KVM: x86 emulator: fix mov dr to inject #UD when needed.")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Manually (pre)check the values being written to DR4/5, i.e. the DR6/DR7
aliases, instead of relying on ->set_dr() => kvm_set_dr() to signal a #GP.
SVM unfortunately prioritizes all exceptions over an instruction intercept,
i.e. nSVM is relying on the emulator to perform *all* exception checks
prior to attempting to execute the instruction.
Fixes: 3b88e41a4134 ("KVM: SVM: Add intercept check for accessing dr registers")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When emulating a MOV DR, specifically a write to DR6 or DR7, treat a #DB
due to DR7.GD (General Detect) as higher priority than a #GP due to an
illegal value. While neither Intel's SDM nor AMD's APM says anything
about the relative priority, empirical testing on Intel and AMD shows that
the #DB has higher priority. And for VMX, where the instruction intercept
has priority over *all* exceptions, KVM already treats the #DB as having
higher priority.
Cc: Maciej W. Rozycki <macro@orcam.me.uk>
Fixes: 3b88e41a4134 ("KVM: SVM: Add intercept check for accessing dr registers")
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When emulating a MOV to a debug register, em_dr_write() calls
@ctxt->ops->set_dr(), which is forwarded to emulator_set_dr() and
then kvm_set_dr(). The latter checks that the written value is valid,
otherwise returning an error, in which case the emulator is supposed to
inject a #GP fault into the guest.
Commit 996ff5429e98 ("KVM: x86: move kvm_inject_gp up from kvm_set_dr
to callers") changed the contract of kvm_set_dr() (and thus
emulator_set_dr()), returning 1 as an error instead of -1, but the
caller in em_dr_write() was never updated, checking only if the returned
value is negative. The end result is that em_dr_write() does not detect
the error, so an invalid write does not generate a #GP, but at the same
time the register value is not updated.
The practical impact is limited, as check_dr_write() already checks DR6
and DR7 manually. However, it misses DR4/DR5, which alias DR6/DR7 when
CR4.DE=0.
Fix the bug by treating any non-zero return from set_dr() as a reason to
inject #GP.
Note, the manual checks on DR6 and DR7 are flawed, as they incorrectly
prioritize the #GP over a DR7.GD=1 #DB (the General Detect #DB has
priority on both Intel and AMD).
Note #2, relying on ->set_dr() to detect #GP is also flawed as all
exceptions have higher priority than the instruction intercept on SVM,
i.e. the manual checks need to be extended to DR4 and DR5 (after the
priority bug is fixed).
Fixes: 996ff5429e98 ("KVM: x86: move kvm_inject_gp up from kvm_set_dr to callers")
Signed-off-by: Carlos López <clopez@suse.de>
Link: https://patch.msgid.link/20260601133320.91479-2-clopez@suse.de
[sean: drop explicit "!= 0", massage changelog]
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260612230113.684301-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Pass the CPUID index into cpuid_func_emulated() and return no emulated
features for indexed CPUID leaves with a non-zero index.
KVM currently emulates CPUID features only for index 0, but
kvm_vcpu_after_set_cpuid() looks up emulated features by function alone.
As a result, reverse_cpuid[] entries that share a function but use a
non-zero index, e.g. CPUID.7.1:ECX, can inherit emulated features that
belong to index 0. For example, RDPID, which is CPUID.7.0:ECX[22], can
be incorrectly OR'd into CPUID.7.1:ECX.
This is benign today because the affected bits do not correspond to
features KVM cares about, but it can become a real bug as new CPUID
features are defined. Make the helper index-aware so emulated features
are applied only to the CPUID entry they actually describe.
Fixes: e592ec657d84 ("KVM: x86: Initialize guest cpu_caps based on KVM support")
Suggested-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Link: https://patch.msgid.link/20260609075748.612704-1-binbin.wu@linux.intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Return EINVAL instead of EOPNOTSUPP if userspace attempts to pass a NULL
pointer for the source page of INIT_MEM_REGION, so that KVM's ABI is
consistent between TDX and SNP (for LAUNCH_UPDATE). EOPNOTSUPP was chosen
to be a forward-looking error code for when guest_memfd supports in-place
conversion, but even when in-place conversion comes along, it's an awkward
error code as KVM is deliberately choosing to disallow virtual address '0',
which is technically a legal userspace address. I.e. it's not so much a
lack of support as it is that KVM reserves address '0' to simplify KVM's
internal implementation.
Opportunistically move the check so that it's co-located with the other
checks on the userspace address, and so that it's more obvious that a NULL
source address is explicitly disallowed.
Fixes: 2a62345b3052 ("KVM: guest_memfd: GUP source pages prior to populating guest memory")
Cc: Yan Zhao <yan.y.zhao@intel.com>
Cc: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Binbin Wu <binbin.wu@linxu.intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Tested-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260630213711.479692-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Explicitly reject a NULL userspace virtual address for the source page of
SNP_LAUNCH_UPDATE instead of relying on the post-populate callback to do
the check, and don't WARN on failure, as the scenario is blatantly user-
triggerable, as reported by Sashiko. Waiting until post-populate to check
the address "works", but makes it unnecessarily difficult to see that KVM's
ABI is to disallow a NULL source page for non-ZERO pages.
Note, several existing VMMs pass a valid userspace address for the ZERO
case, i.e. KVM can't *require* the userspace address to be NULL for ZERO
pages, at least not without breaking userspace.
Fixes: dee5a47cc7a4 ("KVM: SEV: Add KVM_SEV_SNP_LAUNCH_UPDATE command")
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260611125849.9ED631F00893@smtp.kernel.org
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Link: https://patch.msgid.link/20260630213711.479692-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When emulating INVVPID, KVM executes INVVPID on the physical CPU using
vpid02 (instead of the L1 assigned VPID), after doing some validations
on the operands. However, it is possible that the physical CPU KVM
executes INVVPID on is different from the CPU L2 is running on.
For example, in the following scenario:
- L2 runs on CPU #1 and exits to L1 (vmx->nested.vmcs02.cpu=1)
- L1 migrates to CPU #2 and executes INVVPID
- KVM executes INVVPID on CPU #2
- L1 migrates back to CPU #1 and runs L2 (vmx->nested.vmcs02.cpu=1)
The TLB entries on CPU #1 are never invalidated, because INVVPID was
executed on CPU #2, and vmcs02 never ran on a different pCPU (i.e.
vmx_vcpu_load_vmcs() will *not* request KVM_REQ_TLB_FLUSH).
Ensure that INVVPID is being executed on the same pCPU that L2 last ran
on, and if not, fallback to clearing last_vpid=0 to trigger a full VPID
flush on the next nested VM-Enter (as KVM will detect L1 using a
different VPID for L2). If L2 ends up running on a different pCPU, KVM
will flush the TLB anyway through vmx_vcpu_load_vmcs().
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260616214652.2157032-4-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Separate the INVVPID operand checks from the actual flushing of vpid02 so
the flushing can be adjusted to do the right thing when vmcs02 was last
loaded on a different pCPU, without having to duplicate the logic across
multiple case-statements.
Opportunistically let the VM-Fail paths poke out past 80 chars.
No functional change intended.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260616214652.2157032-3-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Make sure vpid02 is always flushed on first use by setting last_vpid=0
when allocating vpid02. nested_vmx_transition_tlb_flush() will always
detect a VPID change on first VM-Enter after VMXON, because VPID=0 in
vmcs12 is not allowed if L1 enables VPID.
This avoids using stale TLB entries from a previous lifetime of the
VPID, that might have been associated with a different vCPU (or a
completely different VM).
Note that last_vpid is already being initialized as 0 when the vCPU is
created, but it is not reset when vpid02 is freed on VMXOFF. Hence, the
problem can only occur if L1 does VMXOFF -> VMXON, runs an L2, and KVM
happens to reuse a VPID that has TLB entries on the physical CPU.
Cc: stable@vger.kernel.org
Signed-off-by: Yosry Ahmed <yosry@kernel.org>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Jim Mattson <jmattson@google.com>
Link: https://patch.msgid.link/20260616214652.2157032-2-yosry@kernel.org
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
The valid amdkfd event ids go from 0 to KFD_SIGNAL_EVENT_LIMIT - 1.
During CRIU restore, ensure that the provided event ids are
in that range.
v2: No need for lower bound check since idr_alloc rejects negative
inputs
v3: Also change error message to reflect new error condition
Reviewed-by: David Yat Sin <david.yatsin@amd.com>
Signed-off-by: David Francis <David.Francis@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 5c6c247992d4d9200e073b83f4ec6c703c096845)
|
|
This is needed to allocate occlusion counters across
both gfx pipes.
Fixes: b7a1a0ef12b8 ("drm/amd/amdgpu: add pipe1 hardware support")
Reviewed-by: Timur Kristóf <timur.kristof@gmail.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 6807352cbabb74b61ba42888769283af72191f66)
Cc: stable@vger.kernel.org
|
|
era_ctr() replaces the actual error codes returned by dm_get_device()
and dm_set_target_max_io_len() with hardcoded -EINVAL, discarding
the real reason for the failure (e.g. -ENODEV, -ENOMEM). This makes
it harder for users to diagnose problems and is inconsistent with
other dm targets (dm-thin, dm-verity, dm-flakey, dm-ebs) which
propagate the original error.
Fix all three sites to return 'r' instead of -EINVAL.
Signed-off-by: Cao Guanghui <caoguanghui@kylinos.cn>
Reviewed-by: Su Yue <glass.su@suse.com>
Reviewed-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
|
|
metadata_open() returns NULL when kzalloc_obj() fails, but the
caller era_ctr() only checks IS_ERR(md). Since IS_ERR(NULL)
returns false, the NULL pointer is treated as a valid result
and later assigned to era->md, leading to a NULL pointer
dereference when the metadata is accessed.
Fix this by returning ERR_PTR(-ENOMEM) on allocation failure,
consistent with dm-cache-metadata.c, dm-thin-metadata.c, and
dm-clone-metadata.c which all use ERR_PTR(-ENOMEM) for the
same pattern.
Fixes: eec40579d848 ("dm: add era target")
Signed-off-by: Cao Guanghui <caoguanghui@kylinos.cn>
Reviewed-by: Su Yue <glass.su@suse.com>
Reviewed-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
|
|
The refactoring in commit a28d893eb327 ("md: port block device access to file")
accidentally causes the caller's thread keyring to be kept alive long
beyond the caller's lifetime.
As a result, "cryptsetup luksSuspend" silently fails to wipe the
LUKS volume key from memory.
In detail: "cryptsetup luksOpen" uses its supposedly ephemeral thread
keyring to pass the volume key to the kernel. dm-crypt's
crypt_set_keyring_key() copies the key material into its own
crypt_config structure and then drops its own reference to the key in
the keyring with key_put().
With this fix, restoring pre-v6.9 behavior, the copy in the thread
keyring is then promptly garbage collected, such that exactly one copy
of the volume key remains. This single copy is correctly wiped from
memory on "cryptsetup luksSuspend".
Without this fix, the thread keyring and the volume key in it remains.
This second copy is only freed on "luksClose". "luksSuspend" neither
knows about this copy nor has any way to remove it, so the key remains
recoverable from RAM after a suspend that is documented to have wiped it.
This fix should not introduce new security problems, as the code is
anyway gated by CAP_SYS_ADMIN. The device-mapper core, not the calling
task, is the legitimate owner of this long-lived file.
Fixes: a28d893eb327 ("md: port block device access to file")
Closes: https://gitlab.com/cryptsetup/cryptsetup/-/work_items/993
Link: https://www.speicherleck.de/iblech/cryptsetup-luksSuspend-issue-reproduction/
Signed-off-by: Ingo Blechschmidt <iblech@speicherleck.de>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Tested-by: Ondrej Kozina <okozina@redhat.com>
|
|
All error handling paths, except but this one, branch to the 'bad' label in
the error handling path.
If not done, there is a memory leak and some sensitive data may be kept
around.
So, fix this error path and also do the needed clean-up.
Also, fix missing goto in the "Wrong alignment of iv_offset sector" path.
Fixes: e7f57d2c47e2 ("dm-inlinecrypt: add target for inline block device encryption")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Reviewed-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
|
|
The pcache target parses optional arguments as name/value pairs. A
table that advertises one optional argument and supplies only a
recognized option name, for example "cache_mode", reaches
parse_cache_opts() with argc == 1. The parser consumes the name,
decrements argc to zero, then calls dm_shift_arg() again for the value.
dm_shift_arg() returns NULL when no arguments remain, and the following
strcmp() dereferences that NULL pointer.
Check that each recognized option has a value before consuming it. This
keeps valid "cache_mode writeback" and "data_crc true/false" tables
unchanged while making malformed tables fail during target construction
with a precise missing-value error.
Assisted-by: Codex:gpt-5.5-cyber-preview
Signed-off-by: Samuel Moelius <sam.moelius@trailofbits.com>
Reviewed-by: Zheng Gu <cengku@gmail.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: 1d57628ff95b ("dm-pcache: add persistent cache target in device-mapper")
Cc: stable@vger.kernel.org
|
|
__reserve_metadata_snap() and __release_metadata_snap() modify the
superblock's held_root directly in the block_manager's buffer. If the
subsequent metadata commit fails, the held_root gets flushed to disk
through the abort_transaction path, resulting in inconsistent metadata.
Reproducer 1: __reserve_metadata_snap()
1. Create a 2 MiB metadata device and make the region after the 14th
block inaccessible, to trigger metadata commit failure in the
subsequent reserve_metadata_snap operation. The 14th block will be
the shadow destination for the index block.
dmsetup create tmeta --table "0 112 linear /dev/sdc 0
112 3984 error"
2. Create a 16 MiB thin-pool
dmsetup create tdata --table "0 32768 zero"
dd if=/dev/zero of=/dev/mapper/tmeta bs=4k count=1
dmsetup create tpool --table "0 32768 thin-pool /dev/mapper/tmeta \
/dev/mapper/tdata 128 0 1 skip_block_zeroing"
3. Take a metadata snapshot to trigger metadata commit failure and
transaction abort. However, the held_root is written to disk,
breaking metadata consistency.
dmsetup message tpool 0 "reserve_metadata_snap"
thin_check v1.2.2 result:
Bad reference count for metadata block 6. Expected 2, but space map contains 1.
Bad reference count for metadata block 7. Expected 2, but space map contains 1.
Bad reference count for metadata block 13. Expected 1, but space map contains 0.
Reproducer 2: __release_metadata_snap()
1. Create a 2 MiB metadata device and make the region after the 16th
block inaccessible, to trigger metadata commit failure in the
subsequent release_metadata_snap operation. The 16th block will be
the shadow destination for the index block.
dmsetup create tmeta --table "0 128 linear /dev/sdc 0
128 3968 error"
2. Create a 16 MiB thin-pool
dmsetup create tdata --table "0 32768 zero"
dd if=/dev/zero of=/dev/mapper/tmeta bs=4k count=1
dmsetup create tpool --table "0 32768 thin-pool /dev/mapper/tmeta \
/dev/mapper/tdata 128 0 1 skip_block_zeroing"
3. Reserve then release the metadata snapshot, to trigger metadata
commit failure and transaction abort. The held_root gets removed
from the on-disk superblock, causing inconsistent metadata.
dmsetup message tpool 0 "reserve_metadata_snap"
dmsetup message tpool 0 "release_metadata_snap"
thin_check v1.2.2 result:
Bad reference count for metadata block 6. Expected 1, but space map contains 2.
Bad reference count for metadata block 7. Expected 1, but space map contains 2.
1 metadata blocks have leaked.
Fix by deferring the held_root update to commit time.
Additionally, move the existing-snapshot check in __reserve_metadata_snap
before the shadow operation to avoid unnecessary work. In
__release_metadata_snap, clear pmd->held_root before btree deletion so
partial failure leaks blocks rather than leaving a stale reference, and
unlock the snapshot block before decrementing its refcount.
Fixes: 991d9fa02da0 ("dm: add thin provisioning target")
Cc: stable@vger.kernel.org
Signed-off-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
|
|
There's a buffer overflow in dm-verity-fec:
if (neras && *neras <= v->fec->roots)
fio->erasures[(*neras)++] = i;
This allows *neras to reach roots + 1 (the post-increment pushes it past
roots). This value is then passed as no_eras to decode_rs8(). Inside the
RS decoder (lib/reed_solomon/decode_rs.c:113-121), the erasure locator
polynomial loop writes lambda[j] where j can reach nroots + 1 — one
element past the end of lambda[] (which is sized nroots + 1, valid
indices 0..nroots). The out-of-bounds write lands on syn[0], corrupting
the syndrome buffer.
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Assisted-by: Claude:claude-opus-4-6
Cc: stable@vger.kernel.org
Fixes: a739ff3f543a ("dm verity: add support for forward error correction")
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
|
|
dm-era tracks writes in target-relative blocks, but era_map() calculates
the writeset block before applying the target offset. Tables with a
non-zero start sector can therefore pass an absolute mapped-device block
to metadata_current_marked().
If the absolute block is beyond the current writeset size,
writeset_marked() tests past the end of the in-core bitset. KASAN reports
this as a vmalloc-out-of-bounds access.
Apply the target offset before calculating the era block so writeset
lookups use the target-relative block number.
Assisted-by: Codex:gpt-5.5-cyber-preview
Signed-off-by: Samuel Moelius <sam.moelius@trailofbits.com>
Reviewed-by: Ming-Hung Tsai <mtsai@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Cc: stable@vger.kernel.org
Fixes: eec40579d848 ("dm: add era target")
|
|
Commit c20e36b7631d ("dm log: fix out-of-bounds write due to
region_count overflow") made sure that region_count could fit in an
unsigned int. But the bitmap memory isn't allocated based on
region_count. It uses bitset_size (a size_t variable). The first step of
calculating bitset_size is to set it to region_count, rounded up to a
multiple of BITS_PER_LONG. If region_size is less than BITS_PER_LONG
smaller than UINT_MAX, it will get rounded up to 2^32. On a 32bit
architecture, this will make bitset_size wrap around to 0 and fail,
despite region_count being valid.
Since bitset_size gets divided by 8, it can hold any valid region_count.
It just needs a special case to handle the rollover. If it is 0, the
value rolled over, and bitset size should be set to the number of bytes
needed to hold 2^32 bits.
Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Signed-off-by: Mikulas Patocka <mpatocka@redhat.com>
Fixes: c20e36b7631d ("dm log: fix out-of-bounds write due to region_count overflow")
Cc: stable@vger.kernel.org
|
|
The vm pointer returned from amdgpu_vm_get_vm_from_pasid() is only
valid while the lock is still being held. Once xa_unlock_irqrestore is
called and returned, the pointer is no longer under lock and is subject
to modification. Since, the caller still dereferences vm->task_info in
amdgpu_vm_get_task_info_vm() after the lock is removed, this causes a
use after unlock problem.
Remove the lifetime issue present in amdgpu_vm_get_task_info_pasid()
through removing the amdgpu_vm_get_vm_from_pasid() function from
amdgpu_vm.c and making the relevant code inline to hold the lock while
it is still in use.
Signed-off-by: Shahyan Soltani <shahyan.soltani@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 9d01579f3f868b333acc901815972685989092c7)
Cc: stable@vger.kernel.org
|
|
Destroying a hung user queue issues a MES REMOVE_QUEUE that times out,
The destroy path only logged the error and freed the queue, so the
next userq submission failed and forced a GPU reset attributed to an innocent workload.
Kick the userq reset work when unmap fails so the GPU is recovered at
destroy time.
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 8396b9de4198a54ec4760a94a179347540a9764d)
Cc: stable@vger.kernel.org
|
|
Disable ASPM on VI if PCIE dpm is disabled.
Fixes: bb00bf17328d ("drm/amd/amdgpu: decouple ASPM with pcie dpm")
Closes: https://gitlab.freedesktop.org/drm/amd/-/work_items/5370
Signed-off-by: Kenneth Feng <kenneth.feng@amd.com>
Reviewed-by: Yang Wang <kevinyang.wang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit 873a8d6b3c0a386408c891e4ff1c684fa11783e1)
Cc: stable@vger.kernel.org
|
|
JDPG does not support on VCN5
This patch will disable JDPG, because DPG is not correctly
copying the JRBC Read/Write Pointers (R/WPTR) from the PG
(Power Gating) block to JRBC.
Signed-off-by: Suresh Guttula <suresh.guttula@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit ea3fdd1eda088030d8925f023613728969f55955)
|
