| Age | Commit message (Collapse) | Author |
|---|
|
Use global SMN addresses for MMHUB v4_2_0 register access
across all instances.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <Le.Ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Update the address mask to support 52-bits addresses when validating the
PDE addresses for GFX 12.1. Currently the address mask was incorrect and
only permitted 48-bits addresses. Additionally, drop the BUG_ON() and
replace it with code to reset the invalid address and valid bit if the
address verification fails. This should help generate a page fault instead
of crashing the kernel.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Philip Yang <philip.yang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Clean up gfx_v12_1_ring_emit_vm_flush function, since
PFP_SYNC_ME cmd not support on gc v12_1, and it does
not have gfx ring.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Work around this in both the amdgpu GMC and KFD SVM paths by explicitly
selecting MTYPE_NC for remote memory when amdgpu_mtype_local is 0. Remove
this once rev_id-based AID detection is reliable.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
This reverts commit bfeb7b28e9f8c5d16ab00b60b42559791b16a12c.
Revert the rev 0x1 (B0) special case for IP_VERSION(12, 1, 0) that
forced noretry_default off. Align GFX12.1.0 with the GC >= 10.3.0
noretry default.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The current combination of PTE flags to translate a retry
fault into a no-retry fault is not valid for GFX12. Use a
valid no-retry PTE flags combination that works for GFX 12.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
gmc v12_1 currently derives the number of mmhub vmhubs
from the aid_mask when populating vmhubs_mask. This
breaks when mmhub is represented by mid and mmhub
inst_mask does not match the aid_mask.
Iterate over adev->mmhub.inst_mask instead so vmhubs_mask
reflects the discovered mmhub instances.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Mangesh Gadre <Mangesh.Gadre@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
mmhub v4_2_0 used adev->aid_mask as the mid_mask
when programming MMHUB instances. That assumption
is not valid for products with more AIDs than MIDs.
Track discovered MMHUB instances in mmhub.inst_mask
and use that mask for mmhub v4_2_0 programming paths.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
For GMC v12.1.0, the IH cookie NodeID spacing is 8 (MID0=0x0, MID1=0x8),
not 4. The wrong divisor caused NPA page faults (node_id=8) to select
mmhub2 instead of the correct mmhub1.
Fix the divisor from 4 to 8, and add a comment documenting the NodeID
to DIE mapping for reference.
Warn once if page fault from the vmhub but vmhub print fault status
function not initialized because not all hubs are initialized.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
GFX12.1 doesn't follow the pair-wise CU mask requirement as
is the case for GFX 10,GFX 11 and GFX 12.
Currently, the ioctl call is returning error because of this
incorrect condition of checking for a pair of CUs to be enabled
in the CU mask.
Fix the issue by removing this check for GFX 12.1.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
On process teardown, kfd_process_notifier_release_internal() called
kfd_process_dequeue_from_all_devices() (which issues the MES
SET_SHADER_DEBUGGER process_ctx_flush=1 that retires the process context)
before kfd_dbg_trap_disable(). For a process under a debugger, the debug
teardown path (kfd_dbg_trap_disable -> kfd_dbg_trap_deactivate) then issues
several SET_SHADER_DEBUGGER process_ctx_flush=0 packets (reset wave launch
mode, clear debug flags, reset MES debug mode). Because those packets carry
the process_context_addr, MES re-adds the process to its scheduler list
after the flush had retired it.
Teardown then frees proc_ctx_bo, and a later MES scheduling pass dereferences
the freed process context, producing a GPUVM page fault (vmid 0, GC_UTCL2)
on the freed page. This is intermittent because it depends on an MES pass
landing on the process before its GART page is reused. It is specific to the
debugger path, where queues are added with skip_process_ctx_clear=1 so the
last REMOVE_QUEUE does not clear the context and delisting relies on the
flush.
Reorder teardown so debug is disabled first, and the process context is
flushed/retired last, so nothing re-adds the process after it is retired.
kfd_dbg_trap_disable() now runs while queues still exist, which matches the
normal live debug-disable ordering.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
status_ptr was declared as u64 * but the WB allocation is only one u32.
Writing through a u64 * pointer zeroed 8 bytes, corrupting the adjacent
WB slot. Fix by using u32 * to match the actual allocation size.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The heavy weight shootdown need special handling in MES sched pipe
Signed-off-by: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The optimized QUE_SLEEP was only programmed on the HWS packet-manager
path via a PM4 WRITE_DATA packet, which is skipped when MES is enabled.
Add gfx12.1 get_iq_wait_times/build_dequeue_wait_counts_packet_info
callbacks and program CP_IQ_WAIT_TIME2 through the MES WRITE_REG op at
start_cpsch, per XCC.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Unmap the KIQ pipe during driver unload so stale pointers do
not carry over to the next driver load.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Michael Chen <michael.chen@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Declare MEC_1 and RLC firmware in gfx_v12_1 with MODULE_FIRMWARE()
so modinfo and distro firmware packaging pick them up.
Signed-off-by: Candice Li <candice.li@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Remove retired cp command package for GC version 12.1.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
A workaround sequence which toggles MGCG override is required to enable
hardware MGCG. As this is not the final solution, don't change the CGCG
flags.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
RLC fw disables GFX CGCG and CGLS on GC 12.1.0 by default, so clear the
corresponding CG flags and keep GUI idle interrupts disabled during gfx init.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
MEC will have the specific FW binary for silicon rev 0.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
To avoid allocation failures in high-density queue scenarios, move user
mode compute MQD allocations for GFX12.1 from GTT sub-allocator to
dedicated kernel memory via amdgpu_amdkfd_alloc_kernel_mem.
This ensures that compute queue creation is not bottlenecked by the
fixed size of the sub-allocator pool.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
For GC info table v1.5, calculate the WGP totals
for SA and SA1 independently, then initialize
max_cu_per_sh with the greater value.
If both totals are equal, keep the SA value.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Shiwu Zhang <shiwu.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Initialize amdgpu_gfx_config using values from
gc_info_v1_5.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The GB_ADDR_CONFIG_READ is initialized to zero.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Both of them are not required for gfx_v12_1.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Use the default all zero settings.
Signed-off-by: Shiwu Zhang <shiwu.zhang@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The underlying functionality was removed
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Le Ma <le.ma@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
add_hw_queue, remove_hw_queue, set_hw_resources_1 and inv_tlbs_pasid
read master_xcc_ids[MES_PIPE_INST(xcc_id, SCHED)]. That only worked
for xcc_id == 0; with a non-zero target (e.g. partition-1 master in
DPX) it read an uninitialised slot. Index by xcc_id consistently, and
fix the related shared_cmd_buf_gpu_addr[] index in
set_hw_resources_1() to use MES_PIPE_INST(master_xcc_id, pipe).
All indexing stays in logical XCC space, so this remains correct with
harvested XCCs; the existing ip_map / GET_INST() translation handles
the logical-to-physical conversion at the HW register boundary.
Signed-off-by: Alex Sierra <alex.sierra@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
On multi-node configurations without XGMI interconnect (e.g. A+A),
num_physical_nodes can be greater than 1 even though XGMI is not supported.
Signed-off-by: Le Ma <le.ma@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Control register is access restricted. Use the read-only copy to read
the partition settings.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
DPX is only valid partition mode in NPS2.
Updated check which was making CPX as
available compute partition mode instead of DPX
Signed-off-by: Mangesh Gadre <Mangesh.Gadre@amd.com>
Reviewed-by: Asad Kamal <asad.kamal@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
NICs attached to remote GPUs are unable to access VRAM coherently.
Hence, use DEVICE_PRIVATE for GFX12.1
v2: Squash in Fix migrate_vma DEVICE_COHERENT setting
Signed-off-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Correct regGFX_IMU_PARTITION_SWITCH value to suitable for harvest
situation for gfx v12.1.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Jinzhou Su <jinzhou.su@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Set partition_mode and physical xcc mask fields in
GFX_IMU_PARTITION_SWITCH register
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Reviewed-by: Likun Gao <Likun.Gao@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Check LSDMA PIO error status bits after each transfer completes to
detect silent copy failures caused by issues such as IFoE (Infinity
Fabric over Ethernet) errors.
The following error conditions are checked:
- Write return NACK / general error
- Read return NACK / general error
- Invalid address
- Zero count
- DRAM ECC error
Log the error via dev_warn() and clear only the set error bits
(write-1-to-clear) so that subsequent transfers can proceed once the
underlying issue is resolved.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add mutex to serialize LSDMA PIO copy and fill operations for
concurrent access from multiple threads.
A mutex is used rather than a spinlock because the underlying
lsdma_v7_1_wait_pio_status() polls with udelay(), which can
busy-wait for up to usec_timeout microseconds.
It is optional to call mutex_destroy for clarity and debug
correctness, it is no-op for release build, leave it as lsdma
block no ip callback funcs to call mutex_destroy.
Signed-off-by: Philip Yang <Philip.Yang@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
On GFX 12.1, all SDMA engines are the same from HW perspective.
As a result, report all SDMA engines as regular/PCIe SDMA engines.
Signed-off-by: Mukul Joshi <mukul.joshi@amd.com>
Reviewed-by: Harish Kasiviswanathan <Harish.Kasiviswanathan@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
A temporary workaround to unblock usermode tests.
Will revert once issue resolved.
Signed-off-by: Feifei Xu <Feifei.Xu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Add parameter to switch sdma RB CMD for sdma v7.1.
Signed-off-by: Likun Gao <Likun.Gao@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
- Add JPEG v5.0.2 SRIOV code path
- Add JPEG MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
- Add VCN v5.0.2 SRIOV code path
- Add VCN MMSCH init table transfer
- Skip unsupported actions such as power gating
and per queue reset for SRIOV VF.
Signed-off-by: Bokun Zhang <Bokun.Zhang@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
jpeg_v5_0_2_is_idle() initializes ret to false and then accumulates ring
idle status using &=. Since false & condition always remains false, the
function can never report the JPEG block as idle.
Initialize ret to true so the function returns true only when all JPEG
rings report RB_JOB_DONE.
Signed-off-by: Boyuan Zhang <boyuan.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Replace external find calls with GNU Make’s $(rwildcard …)
to collect files. Depending on ‘find’ can fail if it’s
unavailable or returns no matches, which left CFLAGS_*
unset. In that case, floating-point code was built
with -mno-sse, leading to build failures.
Signed-off-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The fatal error interrupt is broadcast to every device of the hive and
they all race for amdgpu_ras_global_ras_isr(), which hands -EBUSY to
everyone but the winner. Treating that as a failure returns before the
device is marked, so seven devices out of eight are left without their
fatal and SRAM ECC state, the one that actually logged the error among
them. KFD then tells the processes on those devices that the reset was
a plain hang.
-EBUSY only means the reset has already been asked for. Record the
state anyway and leave the request to the winner.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
kfd_signal_reset_event() picks between KFD_HW_EXCEPTION_ECC and
KFD_HW_EXCEPTION_GPU_HANG from the SRAM ECC flag, and only delivers the
memory exception event for the former. Nothing raises that flag on the
RAS module paths, so a reset caused by an uncorrectable or a consumed
poison error is reported to every process on the device as a plain hang
and the runtime carries on instead of tearing the workload down.
Raise it the way the per IP callbacks used to.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
The failure paths free the RAS manager but keep the flags that advertise
it as the owner of RAS and of the RAS firmware load. Every entry point
gated on those hits the NULL manager, hw_init turns that into -EINVAL
and the whole probe fails.
Clear both once the manager is gone, and route the allocation failure
through the same place.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
ras_mp1_set_debug_mode() reports -EOPNOTSUPP where the control does not
exist, an SR-IOV guest among them. Treating that as fatal fails
ras_core_sw_init() and takes RAS down on every VF.
Nothing else in the MP1 block depends on it, so carry on.
Signed-off-by: Xiang Liu <xiang.liu@amd.com>
Reviewed-by: Tao Zhou <tao.zhou1@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
Different from detect_and_reset, reset_hw_queue performs per queue
reset. Per queue reset setting requires reset_queue_only=1 and hung
queue's doorbell_offset in RESET packet. This packet must be sent to MES
pipe 0(SCHED_PIPE).
We also need to set enable_compute_pipe_reset=1 in SET_HW_RESOURCES so
when queue reset fails, MES will proceed pipe reset.
This patch also fixes a bug in mes_v12_1_reset_hw_queue function. Instead
of using MESAPI__RESET offset for api_status, the existing code uses
MESAPI__REMOVE_QUEUE.
Signed-off-by: Amber Lin <amber.lin@amd.com>
Reviewed-by: Jesse Zhang <Jesse.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
|
removing MES oversubscription timer will affect both amdgpu/amdkfd, so
add the similar timer for amdgpu userq as well.
Signed-off-by: Eric Huang <jinhuieric.huang@amd.com>
Reviewed-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
|
