| Age | Commit message (Collapse) | Author |
|---|
|
Commit c85cdc1cc1ea ("KVM: x86/pmu: Move handling PERF_GLOBAL_CTRL and
friends to common x86") moved the existence check for the following Intel
PMU MSRs to kvm_pmu_is_valid_msr():
- MSR_CORE_PERF_GLOBAL_STATUS
- MSR_CORE_PERF_GLOBAL_CTRL
- MSR_CORE_PERF_GLOBAL_OVF_CTRL
That commit deemed these MSRs valid whenever pmu->version > 1. It intended
to share the check with AMD PerfMonV2 because both vendor implementations
require version 2 or greater for global PMU controls. However, as noted in
the commit message, AMD uses different MSR indices for its global PMU
registers.
Commit 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")
subsequently added AMD PerfMonV2 support and set pmu->version = 2. Because
kvm_pmu_is_valid_msr() validated the Intel MSRs whenever pmu->version > 1,
KVM incorrectly permitted AMD guests with PerfMonV2 to access these Intel
MSRs without a #GP.
Move the validation of these Intel MSRs to intel_is_valid_msr() and remove
the common switch statement from kvm_pmu_is_valid_msr(). AMD already
validates its own global PMU MSRs in amd_is_valid_msr().
Fixes: 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")
Signed-off-by: Jim Mattson <jmattson@google.com>
Reviewed-by: Like Xu <likexu@tencent.com>
Reviewed-by: Sandipan Das <sandipan.das@amd.com>
Link: https://patch.msgid.link/20260902184711.138538-1-jmattson@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
For all intents and purposes, revert KVM's ability to walk rmaps outside of
mmu_lock when running on a realtime (PREEMPT_RT=y) kernel. I.e. don't use
a non-sleepable bit-spinlock to protect rmap entries, as realtime kernels
are highly unlikely to benefit from increased aging throughput and reduced
jitter for memory-overcommitted nested VMs, whereas using a non-sleepable
lock is currently buggy and goes against the spirit of realtime kernels.
Because KVM's rmap locks are hand-crafted bit-spinlocks, preemption must be
disabled before acquiring the lock, otherwise a preempted lock holder will
result in all other walkers of the locked rmap to spin and wait, with no
tracked owner for PI to boost. For non-RT kernels, acquiring mmu_lock
suffices, as mmu_lock is a non-sleepable rwlock. But on RT, where mmu_lock
becomes sleepable, preemption is left enabled for rmap writers:
WARNING: arch/x86/kvm/mmu/mmu.c:920 at __kvm_rmap_lock+0x1a7/0x1e0 [kvm], CPU#16: vmx_apic_update/3708
CPU: 16 UID: 0 PID: 3708 Comm: vmx_apic_update Not tainted 7.2.0-rc7 #52 PREEMPT_{RT,LAZY}
RIP: 0010:__kvm_rmap_lock+0x1a7/0x1e0 [kvm]
Call Trace:
pte_list_add+0x67/0x4d0 [kvm]
__link_shadow_page+0x249/0x480 [kvm]
ept_fetch+0x4d5/0x1220 [kvm]
ept_page_fault+0x60b/0x850 [kvm]
kvm_mmu_do_page_fault+0x252/0x690 [kvm]
Alternatively, KVM could manually disable preemption when grabbing an rmap
lock, but as above, that isn't what RT kernels generally want, and it's
actually more complex to implement (cleanly).
To not completely lose the scaling advantage of per-rmap locks, take
mmu_lock for read in the aging path, i.e. allow multiple concurrent aging
tasks, as the aging code needs to use atomic SPTE accesses no matter what,
i.e. no extra code/work is required to guard against concurrent aging of
SPTEs.
Reported-by: David Woodhouse <dwmw2@infradead.org>
Closes: https://lore.kernel.org/all/8d47b43e1829ac92703723e6a1a4afc7a2eaacb5.camel@infradead.org
Fixes: 4834eaded91e ("KVM: x86/mmu: Add infrastructure to allow walking rmaps outside of mmu_lock")
Reviewed-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: James Houghton <jthoughton@google.com>
Link: https://patch.msgid.link/20260902211759.2700289-1-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mm/slab
Pull slab fixes from Vlastimil Babka:
- Stable fix for an ABA issue causing slab list corruption introduced
in 7.2 (Harry Yoo, with big thanks to Hyunwoo Kim for the thorough
report and initial version of the fix)
- Fix for 7.3 regression of kvfree_rcu() on PREEMPT_RT which can cause
a deadlock from the set_cpus_allowed_force() caller (Vlastimil Babka)
* tag 'slab-for-7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/mm/slab:
mm/slab: take n->list_lock in __slab_try_return_freelist() to avoid race
mm/slab: disallow kfree_rcu_sheaf() on PREEMPT_RT again
|
|
Fix order of the nodes to follow convention.
Signed-off-by: Nora Schiffer <nora.schiffer@ew.tq-group.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
|
|
regulator-(min|max)-microvolt refer to the supported voltages according
to board design, not the possible range of the regulators themselves.
Update the configuration for the TQMa7x's PMIC and also add a comment
regarding the PMIC's VCC_SD switching feature.
Fixes: a80a1af6ec8a ("ARM: dts: tq imx7 common board support")
Signed-off-by: Nora Schiffer <nora.schiffer@ew.tq-group.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
|
|
LDO5 powers the USDHC2 I/O rail on DART-MX8M-PLUS.
Without a regulator consumer, the kernel disables LDO5 during late init,
which removes the SD card and causes filesystem I/O errors.
Reference LDO5 as the Sonata USDHC2 VQMMC supply, so the regulator
remains enabled while the SD interface is active.
Fixes: 41cd3d91ca859 ("arm64: dts: imx8mp-var-dart: Add support for Variscite Sonata board")
Co-developed-by: Luan Rafael Carneiro <luan.r@variscite.com>
Signed-off-by: Luan Rafael Carneiro <luan.r@variscite.com>
Signed-off-by: Stefano Radaelli <stefano.r@variscite.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
|
|
Harden the "map private PFN" flow against potentially-fatal bugs or future
KVM changes by checking for a stale "fault" prior to actually mapping the
PFN into the guest. While it should be impossible for the "page fault" to
become stale, the sanity check is cheap, whereas a broken assumption would
have a high probability of leading to a guest-expoitable use-after-free.
Snapshot the invalidation sequence after acquiring mmu_lock to avoid false
positives, even though doing so completely voids anys and all protection
against unexpected invalidations. Pretty much the entire point of
kvm_tdp_mmu_map_private_pfn() is that it allows mapping a PFN that was
gifted by the caller, i.e. the caller would have to mess up its one and
only responsibility.
Link: https://patch.msgid.link/20260826164214.756512-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When mapping a private PFN in TDX's post-populate callback, top-up the
memory caches on every attempt to map the PFN to harden against bugs in the
map flow that could consume cache entries even if mapping ultimately fails.
E.g. as pointed out by Sashiko, the in-progress Dynamic PAMT support could
consume PAMT cache entries on TDX-Module lock contention.
Harden KVM even though consuming an entry on failure is considered a KVM
bug. Retry should only be encountered if KVM is buggy (the locks held by
the sole call path will prevent retries from being needed due to
TDX-specific details, and memory can be faulted in only once the VM is
TD_STATE_RUNNABLE, and KVM_TDX_INIT_MEM_REGION is only usable if the VM is
*not* TD_STATE_RUNNABLE), top-up is "free" if there's no work to be done,
and populating a TDX guest's memory is a slow path, i.e. there's no
meaningful downside to the hardening.
Reported-by: Sashiko Bot <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260718061050.E17B01F000E9@smtp.kernel.org
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260826164214.756512-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move kvm_tdp_mmu_map_private_pfn()'s reload of the MMU into its tight loop
so that an unexpected root invalidation has a better chance of being
handled gracefully, even though it should be impossible for the vCPU's root
to be invalidated after the initial reload. As is, encountering an invalid
root is *guaranteed* to put the task into an infinite loop (albeit a
breakable loop that honors NEED_RESCHED).
Add a WARN to try and detect bugs that break KVM's expectations, along with
a comment to explain why it should be impossible for the root to be
invalidated.
Note, the loop in question doesn't actually check for a stale page fault,
i.e. likely won't detect an invalid loop in the first place. That bug will
be addressed shortly.
Cc: Kai Huang <kai.huang@intel.com>
Cc: Yan Zhao <yan.y.zhao@intel.com>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260826164214.756512-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Reload the MMU (which is a nop if the MMU doesn't need to be reloaded) on
every attempt to pre-fault a guest page, i.e. when the page fault path
signals that the caller should retry. If the synchronize_srcu_expedited()
in kvm_invalidate_memslot() completes before kvm_vcpu_pre_fault_memory()
grabs SRCU, but kvm_mmu_reload() in the pre-fault path completes before
kvm_invalidate_memslot() triggers x86's "fast zap all", then the pre-fault
task will reach kvm_tdp_page_prefault() with an invalid root.
Attempting to fault-in memory with an invalid root ultimately puts
kvm_tdp_page_prefault() into an infinite (breakable) retry loop, which
manifests most obviously as a hang in the pre_fault_memory_test selftest,
but also eventually causes RCU (SRCU?) to complain.
INFO: rcu_tasks detected stalls on tasks:
000000000cda47bd: .. nvcsw: 6/6 holdout: 1 idle_cpu: -1/25
task:pre_fault_memor state:R running task stack:12696
pid:95588 tgid:95588 ppid:95584 task_flags:0x400000 flags:0x00080801
Call Trace:
<TASK>
lock_release+0x4e/0x320
__get_user_pages+0x546/0xcd0
up_read+0x1b/0x30
get_user_pages_unlocked+0xee/0x350
hva_to_pfn+0xd3/0x3d0 [kvm]
lock_release+0x4e/0x320
xa_load+0x5c/0x170
xa_load+0x14c/0x170
__kvm_faultin_pfn+0xd9/0x130 [kvm]
lock_acquire+0x65/0x2b0
lock_release+0x4e/0x320
kvm_mmu_faultin_pfn+0x1e1/0x690 [kvm]
gup_fast_fallback+0x63e/0xdf0
kvm_tdp_page_fault+0xeb/0x140 [kvm]
kvm_mmu_do_page_fault+0x12e/0x200 [kvm]
kvm_arch_vcpu_pre_fault_memory+0x16e/0x200 [kvm]
kvm_vcpu_pre_fault_memory+0xc1/0x1f0 [kvm]
kvm_vcpu_pre_fault_memory+0x116/0x1f0 [kvm]
kvm_vcpu_ioctl+0x3a4/0x6b0 [kvm]
clockevents_program_event+0x5d/0x170
__se_sys_ioctl+0x6d/0xb0
entry_SYSCALL_64_after_hwframe+0x4b/0x53
do_syscall_64+0x10a/0x480
__irq_exit_rcu+0x8e/0x140
entry_SYSCALL_64_after_hwframe+0x4b/0x53
</TASK>
Fixes: 6e01b7601dfe ("KVM: x86: Implement kvm_arch_vcpu_pre_fault_memory()")
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260826164214.756512-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
kvm_zap_gfn_range() walks memslots and rmaps and may drop mmu_lock to
reschedule. Callers that do not already hold kvm->srcu (or slots_lock)
can race with memslot deletion: synchronize_srcu_expedited() does not
wait, kvm_free_memslot() frees the old slot and its rmap, and the zap
resumes on freed memory.
The VFIO noncoherent-DMA path hits this by zapping the entire GPA
space without SRCU. KASAN reported:
BUG: KASAN: vmalloc-out-of-bounds in slot_rmap_walk_next+0x82/0x1c0
Read of size 8 at addr ffffc900005c1008
Call Trace:
slot_rmap_walk_next+0x82/0x1c0
__kvm_rmap_zap_gfn_range+0x17a/0x280
kvm_zap_gfn_range+0x2a6/0x6a0
kvm_vfio_set_attr+0x576/0x770
kvm_device_ioctl+0x1ff/0x3b0
Take SRCU inside kvm_zap_gfn_range() so every caller is covered.
Nesting with an existing kvm->srcu critical section is fine; the
helper uses a local index. Drop the now-redundant SRCU pair from
__kvm_set_or_clear_apicv_inhibit().
Fixes: 362ff6dca541 ("KVM: x86/mmu: Zap KVM TDP when noncoherent DMA assignment starts/stops")
Cc: stable@vger.kernel.org
Signed-off-by: Chengfeng Ye <nicoyip.dev@gmail.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Link: https://patch.msgid.link/20260825183924.237727-1-nicoyip.dev@gmail.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Replace the KVM-private vgettsc()+/do_do_realtime() timekeeping
reimplementation with calls to the recently crafted, generic
ktime_get_snapshot_id() interface. As noted previously, the snapshot
provides both the system time and the raw_cycles (TSC), atomically paired
using a sequence counter.
With great pleasure, delete the now unused read_tsc() and vgettsc()
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
[sean: separate from other conversions, express joy at vgettsc()'s demise]
Link: https://patch.msgid.link/20260826213303.914988-22-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Replace the KVM-private vgettsc()+/do_monotonic() timekeeping
reimplementation with calls to the recently crafted, generic
ktime_get_snapshot_id() interface. As noted previously, the snapshot
provides both the system time and the raw_cycles (TSC), atomically paired
using a sequence counter.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
[sean: separate from other conversions]
Link: https://patch.msgid.link/20260826213303.914988-21-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Replace the KVM-private vgettsc()+do_kvmclock_base() timekeeping
reimplementation with calls to the recently crafted, generic
ktime_get_snapshot_id() interface. This is the first step towards dropping
KVM's homebrewed implementation entirely (do_monotonic() and do_realtime()
will be converted in the near future).
As with KVM's implementation, the snapshot provides both the system time
and the raw_cycles (TSC), atomically paired using a sequence counter. The
equivalents to vgettsc()'s TSC and HVCLOCK modes respectively are if the
clocksource itself is TSC (cs_id == CSID_X86_TSC) and if the underlying
hardware clocksource is TSC (hw_csid == CSID_X86_TSC). In the Hyper-V
case, i.e. hw_csid == CSID_X86_TSC, if the clocksource couldn't provide a
raw hardware counter value, treat the clock not being based on TSC, which
which is equivalent to vgettsc() returning VDSO_CLOCKMODE_NONE.
Unlike KVM's current implementation, don't include offs_boot in the
atomically-acquired tuple as there's simply no need to do so: the time
since boot only changes at boot (duh) and at suspend/resume boundaries.
Unless processes aren't being frozen/thawed before/after suspend/resume,
which would completely break suspend/resume, TK_OFFS_BOOT can't change
while kvm_get_time_and_clockread() is running. And if KVM does somehow try
to take a snapshot during suspend, timekeeping core will WARN and refuse to
provide the snapshot.
This is a step towards eliminating the pvclock_gtod_data private copy
of timekeeping state and the associated notifier callback.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
[sean: separate from other conversions, massage changelog accordingly]
Link: https://patch.msgid.link/20260826213303.914988-20-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Let the callers pass the host TSC value in as an explicit parameter.
This leaves some fairly obviously stupid code, which is using this
function to compare the guest TSC at some *other* time, with the
newly-minted TSC value from rdtsc(). Unless it's being used to measure
*elapsed* time, that isn't very sensible.
In this case, "obviously stupid" is an improvement over being
non-obviously so.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>
Link: https://patch.msgid.link/20260826213303.914988-19-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
The kvm_get_time_scale() function was entirely opaque. Add comments
explaining what it does: compute a fixed-point multiplier and shift for
converting TSC ticks to nanoseconds via pvclock_scale_delta().
Rename the local variables from the cryptic tps64/tps32/scaled64 to
base_hz_u64/base32/scaled_hz_u64 to make the code self-documenting.
The "tps32" name stood for "Ticks Per Second" but was misleading since
it held the shifted base frequency, not a tick count.
No functional change.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>
Link: https://patch.msgid.link/20260826213303.914988-18-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When doing TSC upscaling in software, e.g. when the guest TSC frequency is
configured to run faster than the host without hardware support, always
advance guest TSC to "now", not to the master clock's reference values,
which represents some time in the past.
In master clock mode, there are two points of time which need to be taken
into account. First there is the master clock reference point, stored in
kvm->arch.master_kernel_ns (and associated host TSC ->master_cycle_now).
Secondly, there is the time *now*, at the point kvm_update_guest_time()
is being called.
With software TSC upscaling, the guest TSC is getting further and further
ahead of the host TSC as time elapses. So at time "now", the guest TSC
should be further ahead of the host, than it was at master_kernel_ns.
Not taking that into account means KVM was advancing the guest TSC only by
the amount appropriate for the snapshot taken in the past, *not* to the
current time (which is what RDTSC is supposed to return).
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Co-developed-by: Sean Christopherson <seanjc@google.com>
Link: https://patch.msgid.link/20260826213303.914988-17-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Wrap kvm_guest_time_update()'s entire use_master_clock block in #ifdef
CONFIG_X86_64 and provide a hardcoded-to-false variable for the 32-bit
path, as use_master_clock is never true on 32-bit (host_tsc_clocksource is
only set under CONFIG_X86_64)
Keep the local "ka" variable outside of the 64-bit-only code as it's also
used for Xen code, but tag it as potentially unused since it's not used on
32-bit kernels without Xen emulation.
For all intents and purposes, no functional change intended.
Link: https://patch.msgid.link/20260826213303.914988-16-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Disable "just" preemption, not IRQs, when reading the TSC+frequency pair to
update guest time, as disabling IRQs to protect against task migration is
overkill (though it's *extremely* hard to see that it's overkill).
Disabling IRQs was added by commit 18068523d3a0 ("KVM: paravirtualized
clocksource: host part") before there was any coordination with timekeeping
(presumably disabling IRQs prevented the kernel from completing a software-
induced frequency change).
After the coordination and locking was added, commit c09664bb4418 ("KVM:
x86: fix deadlock in clock-in-progress request handling") moved the locking
and coordination out of IRQ protection, and thus made disabling IRQs
pointless, except for protecting get_cpu_tsc_khz().
And while cpu_tsc_khz is written only from IRQ context, and the *extremely*
confusing double IPIs sent by __kvmclock_cpufreq_notifier() to update the
per-CPU frequency make it seem like they would require readers to disable
IRQs, it is safe to read and consume cpu_tsc_khz (via get_cpu_tsc_khz())
with IRQs enabled. The per-CPU variable is specifically written only in
IRQ context to ensure hotplugging a CPU wouldn't write cpu_tsc_khz with a
stale value (because apparently disabling IRQs would be too simple?!?).
As for the double IPIs in the frequency notifier, both IPIs are red
herrings. The actual sequence that ensures KVM updates guest time with the
new frequency is that the first write is completed *before* the notifier
sets KVM_REQ_CLOCK_UPDATE for all vCPUs that last ran on the target pCPU.
The first write is done via IPI to adhere to the above rules, and the
second IPI is sent purely to kick any vCPU that happens to be running on
the target CPU out of the guest. I.e. the second IPI writes cpu_tsc_khz
out of pure KVM laziness: it saves having to define another IPI callback.
In fact prior to commit 8cfdc0008542 ("KVM: x86: Make cpu_tsc_khz updates
use local CPU"), KVM did indeed use an empty callback to ack the IPI. As
for why it was deemed cleaner to abuse tsc_khz_changed()...
Link: https://patch.msgid.link/20260826213303.914988-15-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
The compute_guest_tsc() function computes the guest TSC at a given
kernel_ns timestamp. When the master clock reference point
(master_kernel_ns) is earlier than vcpu->arch.this_tsc_nsec, the delta
is negative. Since pvclock_scale_delta() takes a u64, the negative
value wraps to a huge positive number, producing a wildly wrong result.
Handle negative deltas explicitly by scaling the absolute value of the
delta and applying it to this_tsc_write with the appropriate sign.
This is believed to be unreachable in practice; no path has been
identified which invokes compute_guest_tsc() with a timestamp from
before the vCPU's TSC generation was established. Fix it for
robustness, in the spirit of defence in depth.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260826213303.914988-14-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Now that get_kvmclock() correctly handles TSC scaling and captures both
wallclock and kvmclock from the same TSC reading,
kvm_get_wall_clock_epoch() can simply call it instead of duplicating
the pvclock computation.
This eliminates the last instance of the "definition C" kvmclock
calculation — as described in commit 633d7652f80f ("KVM: x86/xen: Do not
corrupt KVM clock in kvm_xen_shared_info_init()") — which computed
nanoseconds directly from the host TSC without accounting for guest TSC
scaling.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260826213303.914988-13-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When in master clock mode, the KVM clock is defined in terms of the
guest TSC. But get_kvmclock() was computing it from the host TSC
without applying TSC scaling, leading to a systemic drift from the
values the guest computes from its own TSC.
Store the VM's TSC scaling ratio in kvm_arch and precompute the
guest-TSC-based mul/shift in pvclock_update_vm_gtod_copy(). Use these
in get_kvmclock() to scale the host TSC delta to guest TSC before
converting to nanoseconds.
This avoids "definition C" of the KVM clock described in commit
633d7652f80f ("KVM: x86/xen: Do not corrupt KVM clock in
kvm_xen_shared_info_init()").
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260826213303.914988-12-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When computing kvmclock and it's currently in master-clock mode, fall back
to the non-master-clock path if the clock read fails, e.g. if the kernel's
clocksource is transitioning away from TSC but ka->use_master_clock hasn't
been updated yet.
The rdtsc() fallback was added (well, kept) in commit c68dc1b577ea ("KVM:
x86: Report host tsc and realtime values in KVM_GET_CLOCK") purely to avoid
uninitialized variables and compilation problems on 32-bit kernels (already
addressed). In hindsight, keeping the rdtsc() was a hack and a mistake.
Link: https://lore.kernel.org/all/CAOQ_QsgVqS_PuJo8F10Gg5Xw+tKt+5gDx+kJf1j3CiPO4MAOqg@mail.gmail.com
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
[sean: isolate from refactoring changes, write changelog]
Link: https://patch.msgid.link/20260826213303.914988-11-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Wrap the entire use_master_clock block in #ifdef CONFIG_X86_64, since
use_master_clock is never true on 32-bit (host_tsc_clocksource is only
set under CONFIG_X86_64), and declare hv_clock inside the block so it is
not left as an unused variable on 32-bit.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
[sean: keep only the CONFIG_X86_64 ifdef, update changelog accordingly]
Link: https://patch.msgid.link/20260826213303.914988-10-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
get_kvmclock()
Move the fallback logic for getting the current kvmclock when not in master
clock mode out of __get_kvmclock() and into its sole caller, get_kvmclock().
This will allow using early-return logic in the master clock code, without
having to resort to a do-while() loop and/or gotos.
No functional change intended.
Link: https://patch.msgid.link/20260826213303.914988-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When computing the current kvmclock value, don't pin the task to the
current CPU for the entire duration of the master clock path, as the
CPU pinning was never about ensuring rdtsc() and cpu_tsc_khz would
agree. As pointed out by David, ka->use_master_clock can only be true
when the host clocksource is TSC based, which in turn requires a stable,
constant and synchronised TSC across all CPUs.
The CPU pinning was added in commit e2c2206a1899 ("KVM: x86: Fix potential
preemption when get the current kvmclock timestamp") purely in response to
a CONFIG_DEBUG_PREEMPT=y bug due to accessing a per-CPU variable with
preemption enabled. Despite what the comment would suggest, including
rdtsc() in the {get,put}_cpu() section was opportunistic. In fact, Paolo
even said exactly that when suggesting that KVM guarantee the rdtsc() would
execute on the same CPU[*]:
: Also, rdtsc() should really be on the same CPU as __this_cpu_read. We
: know it's not really really necessary because the master clock is
: active, but since we need a get_cpu/put_cpu pair, better be clean.
Nothing has changed in the last ~9 years, i.e. the rdtsc() still *should*
be on the same CPU, but super strictly speaking, all will be fine if the
task is migrated between grabbing the frequency and doing rdtsc().
Dropping the CPU pinning will allow dropping the rdtsc() entirely without
having to resort to a large "rewrite get_kvmclock()" patch.
Opportunistically add a comment to explain why KVM needs to snapshot the
frequency, because that _is_ a hard requirement to avoid reintroducing the
bug fixed by commit e70b57a6ce4e ("KVM: X86: Fix softlockup when get the
current kvmclock")
Link: https://lore.kernel.org/all/ae8de642-8f14-a70a-1fab-57e2c4093cd5@redhat.com [*]
Suggested-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260826213303.914988-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Now that 64-bit KVM doesn't consume pvclock_gtod_data when getting the
kvmclock base, use the ktime_mono_to_any() version on 32-bit hosts so
that they too are unaffected by NPT sync.
Note, even if KVM still used pvclock_gtod_data for getting kvmclock base,
it would still be very doable and reasonable to avoid skew on 32-bit hosts,
it's just much simpler now that KVM uses ktime_mono_to_any().
Specifically, commit 53fafdbb8b21 ("KVM: x86: switch KVMCLOCK base to
monotonic raw clock") did so only for 64-bit hosts, by capturing the boot
offset from within the existing clocksource notifier update_pvclock_gtod().
That notifier was added in commit 16e8d74d2da9 ("KVM: x86: notifier for
clocksource changes") but only on x86_64, because its original purpose
was just to disable the "master clock" mode which is only supported on
x86_64.
Now that the notifier is used for more than disabling master clock mode,
enabling it would be trivial for 32-bit builds as well, but as above,
that's a moot point since get_kvmclock_base_ns() no longer relies on the
notifier.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>
[sean: rebase on top of ktime_mono_to_any() usage]
Link: https://patch.msgid.link/20260826213303.914988-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
get_kvmclock_base_ns() needs CLOCK_MONOTONIC_RAW + offs_boot. Compute
this directly rather than reading offs_boot from the pvclock_gtod_data
private copy. offs_boot only changes at suspend/resume so does not
need to be atomically paired with the raw clock read.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Assisted-by: Kiro:claude-opus-4.6-1m
Link: https://patch.msgid.link/20260826213303.914988-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Previously, the master clock was only activated when the first vCPU
processed KVM_REQ_MASTERCLOCK_UPDATE during KVM_RUN. This meant that
KVM_GET_CLOCK could not return the host_tsc field until after the
first KVM_RUN, making it impossible for userspace to follow the
documented TSC migration procedure without a dummy vCPU run.
Fix this by calling kvm_update_masterclock() directly from
kvm_arch_vcpu_postcreate(), after kvm_synchronize_tsc() has already
set all_vcpus_matched_freq. This ensures the master clock is active
immediately, and KVM_GET_CLOCK returns a valid {host_tsc, realtime}
pair as soon as a vCPU exists.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Link: https://patch.msgid.link/20260826213303.914988-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
KVM does make an attempt to cope with non-constant TSC, and has
notifiers to handle host TSC frequency changes. However, it *only*
adjusts the KVM clock, and doesn't adjust TSC frequency scaling when
the host changes.
This is presumably because non-constant TSCs were fixed in hardware
long before TSC scaling was implemented, so there should never be real
CPUs which have TSC scaling but *not* CONSTANT_TSC.
Such a combination could potentially happen in some odd L1 nesting
environment, but it isn't worth trying to support it. Just make the
dependency explicit.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Reviewed-by: Paul Durrant <paul@xen.org>
Link: https://patch.msgid.link/20260826213303.914988-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
The kvm_guest_time_update() function scales the host TSC frequency to
the guest's using kvm_scale_tsc() and the v->arch.l1_tsc_scaling_ratio
scaling ratio previously calculated for that vCPU. Then calculates the
scaling factors for the KVM clock itself based on that guest TSC
frequency.
However, it uses kHz as the unit when scaling, and then multiplies by
1000 only at the end.
With a host TSC frequency of 3000MHz and a guest set to 2500MHz, the
result of kvm_scale_tsc() will actually come out at 2,499,999kHz. So
the KVM clock advertised to the guest is based on a frequency of
2,499,999,000 Hz.
By using Hz as the unit from the beginning, the KVM clock would be based
on a more accurate frequency of 2,499,999,999 Hz in this example.
Use u64 for the hw_tsc_hz field since an unsigned int would overflow for
TSC frequencies above 4GHz.
Fixes: 78db6a503796 ("KVM: x86: rewrite handling of scaled TSC for kvmclock")
Reviewed-by: Paul Durrant <paul@xen.org>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Tested-by: Dongli Zhang <dongli.zhang@oracle.com>
Link: https://patch.msgid.link/20260826213303.914988-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
When refreshing the last observed guest TSC during a guest time update,
write the snapshot before enabling IRQs, i.e. before enabling preemption.
If the task is migrated between updating the local tsc_timestamp, e.g. to
account for catch-up mode, and setting last_guest_tsc, kvm_arch_vcpu_load()
would set the vCPU's TSC offset using the old last_guest_tsc.
In practice, the bug is largely benign as it's not even strictly necessary
for KVM to refresh last_guest_tsc when updating guest time, as KVM's goal
is purely to prevent the guest from observing time jump backwards, i.e.
super duper strictly speaking, KVM only *needs* to update last_guest_tsc in
the VM-Exit path.
In fact, the update kvm_guest_time_update() in wasn't even added to play
nice with kvm_arch_vcpu_load(), it was added by commit 28e4639adf0c ("KVM:
x86: Fix kvmclock bug") to fix code that no longer exists. As of commit
28e4639adf0c, kvm_guest_time_update() also consumed last_guest_tsc, to try
and prevent guest time from jumping backwards. That code was eventually
removed by commit f25e656d31ad ("KVM: x86: fix tsc catchup issue with tsc
scaling"), but the last_guest_tsc update hung around.
Keep the update even though it's technically ok to drop the update, e.g. so
that the tsc_catchup updates aren't lost, and so that the guest won't see a
PV clock timestamp that appears to be in the future.
Link: https://patch.msgid.link/20260826213303.914988-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Add the power supply to the pcal6408 gpio expander, it doesn't have the
dependency loop like the pcal6524.
Signed-off-by: Maud Spierings <maud_spierings@hotmail.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
|
|
pcf2131 has been assigned the rtc0 alias, but if the bbnsm_rtc gets
probed first it will take this name.
Properly assign bbnsm_rtc to rtc1 so that this doesn't happen
Signed-off-by: Maud Spierings <maud_spierings@hotmail.com>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
|
|
The a750 uses the GMU CX AO Counters instead of the GMU_ALWAYS_ON_COUNTER
register on A6xx and other A7xx GPUs, use it when running a A750 GPU.
The GMU_ALWAYS_ON_COUNTER at offset 0x1f888 doesn't seem to exist
on the SM8650 A750 GMU and returns 0, but the CX AO counter at offset
0x1f880 returns some proper timestamp data.
Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/752235/
Message-ID: <20260909-topic-sm8650-gmu-a750-timestamp-reg-v2-2-091d74958951@linaro.org>
Signed-off-by: Rob Clark <robin.clark@oss.qualcomm.com>
|
|
The a750 uses the CX AO Counters instead of the GMU_ALWAYS_ON_COUNTER
register on A6xx and other A7xx GPUs.
Signed-off-by: Neil Armstrong <neil.armstrong@linaro.org>
Patchwork: https://patchwork.freedesktop.org/patch/752234/
Message-ID: <20260909-topic-sm8650-gmu-a750-timestamp-reg-v2-1-091d74958951@linaro.org>
Signed-off-by: Rob Clark <robin.clark@oss.qualcomm.com>
|
|
The adreno_perfcntrs macro already takes the XML file stem as its second
argument, allowing each perf counter JSON file to select the appropriate
register XML. The a2xx and a5xx entries already pass their respective
XML file stems.
Remove the stale TODO comment.
Signed-off-by: George Emmanuel Thomas <georgeemmanuelthomas@gmail.com>
Patchwork: https://patchwork.freedesktop.org/patch/746745/
Message-ID: <20260815164335.158958-1-georgeemmanuelthomas@gmail.com>
Signed-off-by: Rob Clark <robin.clark@oss.qualcomm.com>
|
|
bio_set_dev() associates the bio with a blkg through bio_associate_blkg().
If the blkg lookup misses, blkg_tryget_closest() takes q->queue_lock with
spin_lock_irq() and releases it with spin_unlock_irq(), which
unconditionally enables local interrupts.
Callers may call bio_set_dev() with interrupts already disabled, e.g.
dm-thin's pool_map() does so while holding pool->lock taken with
spin_lock_irq(). The nested spin_unlock_irq() then enables interrupts
while pool->lock is still held, so an I/O completion softirq can run on
the same CPU, re-acquire pool->lock (thin_endio(), or overwrite_endio()
-> complete_mapping_preparation()) and deadlock. lockdep reports this
as inconsistent SOFTIRQ-ON-W to IN-SOFTIRQ-W usage.
Commit 3a762de55b4e ("block: save irq state in blkg_lookup_create()")
fixed the same problem while the lock lived in blkg_lookup_create(), but
commit 9327a865e395 ("blk-cgroup: don't nest queue_lock under rcu in
blkg_lookup_create()") moved the locking into blkg_tryget_closest() and
reverted it to spin_lock_irq().
Save and restore the caller's IRQ state instead.
Fixes: 9327a865e395 ("blk-cgroup: don't nest queue_lock under rcu in blkg_lookup_create()")
Cc: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Hao Zhang <zhanghao1@kylinos.cn>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://patch.msgid.link/aqQmqb1k57PXj8Ef@192.168.1.215
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Rename all common prefixes for the APIs that get exit information common to
VMX and TDX from "vmx" to "vt" to capture that they are used for both VMX
and TDX vCPUs.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-9-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move the "to" vCPU converters and common exit information getters from
vmx.h to common.h to capture that they are indeed common VT code. Defer
renaming their prefixes to isolate the code movement from the massive
churn that will result from said renaming.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-8-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Define a combined vcpu_vmx_tdx structure to represent the common layout of
vcpu_vmx and vcpu_tdx, instead of having vcpu_tdx piggyback vcpu_vmx. I.e.
require VMX and TDX to match the common definition, not for TDX to match
VMX. This will allow moving to_vt() and vt_to_vcpu() to common.h where
they belong.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-7-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Rename __vmx_handle_ept_violation() to __vt_handle_ept_violation to capture
that it is shared between VMX and TDX.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-6-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Rename all common posted interrupt APIs that use a "vmx" prefix to instead
use a "vt" prefix to capture that they are used for both VMX and TDX vCPUs.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-5-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move vmx_handle_nmi() to main.c to capture that it's a common handler and
to allow guarding against incorrectly using to_vmx().
Opportunistically use a "vt" prefix instead of "vmx" to communicate that
it's a shared handler.
No functional change intended.
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-4-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Move vmx_handle_exit_irqoff() and its helpers to common.h / main.c to
capture that it's a common handler and to allow guarding against incorrectly
using to_vmx().
Opportunistically use a "vt" prefix instead of "vmx" to communicate that
it's a shared handler.
No functional change intended.
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: Xiaoyao Li <xiaoyao.li@intel.com>
Cc: Binbin Wu <binbin.wu@linux.intel.com>
Cc: Kai Huang <kai.huang@intel.com>
Cc: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-3-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Poison to_vmx() in main.c and posted_intr.c so that attempting to interpret
the vCPU as a VMX vCPU will fail at compile time, as opposed to failing at
runtime, or worse corrupting state without outright failing.
Note, to_tdx() is buried in tdx.c, i.e. isn't broadly reachable, and so
doesn't need the same treatment as to_vmx().
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Tested-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Link: https://patch.msgid.link/20260904010353.3175819-2-seanjc@google.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Extend the BPF LSM selftest to exercise create, link, symlink, mkdir,
mknod, and permission through both the VFS identity idmap and a real
idmapped tmpfs mount.
Record the idmap observed by each hook and verify that every updated hook
receives the mapping used by the VFS operation. This provides regression
coverage for passing mount idmaps through inode security hooks.
Signed-off-by: Daan De Meyer <daan@amutable.com>
Reviewed-by: Matt Bobrowski <matt@bobrowski.net>
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
OverlayFS performs upper-layer operations through inode-based security
hooks. Those hooks receive the upper inode and dentry, but not the mount
idmap used by the VFS operation.
The security layer cannot distinguish an identity-mapped upper from an
idmapped one or make the same ownership decision as the VFS. The VFS
layer already passes the idmap down into all relevant inode operations
so this just brings the security hooks to parity.
So pass the mount idmap through the create, link, symlink, mkdir, mknod,
and permission hooks. Update the in-tree security implementations and
non-VFS callers accordingly.
Signed-off-by: Daan De Meyer <daan@amutable.com>
Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org>
[PM: updated the commit description as discussed on-list]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
rb_wake_up_waiters() is a irq_work callback which is initialized with
init_irq_work(). As such it will be invoked in thread context on
PREEMPT_RT. Invoking the callback in IRQ context on PREEMPT_RT is not an
option due its usage of wake_up_all(). Since this callback may run in
thread context, it needs to acquire ring_buffer_per_cpu::reader_lock with
disabling interrupts and may not assume that they are disabled.
Use raw_spinlock_irqsave() to acquire ring_buffer_per_cpu::reader_lock.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260911102152.YEtwkBj9@linutronix.de
Fixes: 68282dd930ea3 ("ring-buffer: Fix resetting of shortest_full")
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
With symlinkroot unset, an absolute target is copied without conversion
to an NT drive path. Later code still assumes an NT prefix is present
when modifying the target and calculating the print name length.
For "/ab", this causes two failures: sym[5] and path[5] are written
past their allocations, and plen -= 2 * poff subtracts an assumed
8-byte prefix from a 6-byte UTF-16 target, wrapping u16 plen to 65534.
That underflow causes another overflow: memcpy() copies 65534 bytes
into a 24-byte buffer. A user with write access to a mounted share
can trigger these bugs with default settings.
Validate the NT drive prefix, including an ASCII drive letter, before
accessing fixed offsets or subtracting the prefix length.
Fixes: 3363da82e02f ("smb: client: fix native SMB symlink traversal")
Assisted-by: Codex:gpt-5
Signed-off-by: Jérémy Jean <Jeremy.Jean@oss.cyber.gouv.fr>
Reviewed-by: Namjae Jeon <linkinjeon@kernel.org>
Signed-off-by: Paulo Alcantara <pc@manguebit.org>
|
