diff options
| author | Stefan Hajnoczi <stefanha@redhat.com> | 2026-07-02 10:44:27 +0200 |
|---|---|---|
| committer | Stefan Hajnoczi <stefanha@redhat.com> | 2026-07-02 10:44:27 +0200 |
| commit | 654b54fb372180924f8206d6dfd29cecdef1d8ac (patch) | |
| tree | e61c584a3b06631d1e8c4b228f922c9ddf5bd2d6 | |
| parent | 30e8a06b64aa58a3990ba39cb5d09531e7d265e0 (diff) | |
| parent | 94e3ad78004787b0dea9625544fb45d8cf037147 (diff) | |
| download | qemu-654b54fb372180924f8206d6dfd29cecdef1d8ac.tar.gz qemu-654b54fb372180924f8206d6dfd29cecdef1d8ac.zip | |
Merge tag 'pull-target-arm-20260629' of https://gitlab.com/pm215/qemu into staging
target-arm queue:
* hw/timer/imx_epit: Replace DPRINTF with trace events
target/arm: Enable SCTLR_EL1.EnFPM for user-only
target/arm: Implement FEAT_SME_F8F32
target/arm: Implement FEAT_SSVE_AES
target/arm: Implement FEAT_SME_F8F16
target/arm: GICv5: Fix some minor bugs
target/arm: Add GPC3 granule bypass windows
target/arm: Fix some minor timer related bugs
hw/arm/sabrelite: Add FlexCAN emulation
docs/system: add FEAT_ECV_POFF to the emulation list
docs/system/arm/virt: Document accelerated SMMUv3 and Tegra241 CMDQV
# -----BEGIN PGP SIGNATURE-----
#
# iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmpCXaQZHHBldGVyLm1h
# eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3nfkEACJkoMzHDyHcAiRdO8fK4o6
# zHP3f42UOnXTbX//Yga0PpxScWfvD8XbbOSeJRvpjuxl8nP8QF4/rF4b+atMy9Vl
# MH0r/CWl9fZwQOSmjOLzgRGzXx0j9RPPpB/7eYTnKYImfOaEEaGvW4JqoBRE2Nbo
# x5PaQjaqFQi76uGAJvALPgAPCgaK1DGbNDSRuH4RM7auLBWmSaoxdidiTDSBUqY0
# xsI/lU7t+/LLWirjP/QhM4mbxEc2DjENbguRHYlOqe5aHc6KdSmNj2B4/hTfyDON
# c6APaAAPfCy3duL3JsvmwRZ8YM7zoUFEHysLjRxLWyiFfXZUIXPSMZaGpz88iyDV
# Cbraw24K5tVVNvwQTKOpHYCnjNb4dZj1Zt/jdGIu16LQ8nsKgX2EJ6oh6lI85Q6n
# d3Jbq+iLOy2r2r4CRTMIJYKZ2Bikkmyr+wZGO18nttnTVpWNzWVZtq4cutygr5vb
# 0+5Lmr7YeYsdmIc1tpcJmlfmmo7dW987HyzK3/B65gPXV64w+a3eALRLPkMGevTT
# MhG48151NEovHxfKqzsOMIixnPUKGPtAUbeKy/Ywv2ezKUmER19h/7nJ0lsa32pl
# HYctGj4QeK4VjOO8E1q44ZIionhZFt+RHXBxxbiBzQBns/ryFBOQFEA3WzKi7rnd
# a0v1M+AAK/UxmCjV7Sl0WA==
# =OvGk
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 29 Jun 2026 13:57:24 CEST
# gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg: issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [full]
# gpg: aka "Peter Maydell <pmaydell@gmail.com>" [full]
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [full]
# gpg: aka "Peter Maydell <peter@archaic.org.uk>" [unknown]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE
* tag 'pull-target-arm-20260629' of https://gitlab.com/pm215/qemu: (54 commits)
docs/system/arm/virt: Document accelerated SMMUv3 and Tegra241 CMDQV
target/arm: Enable FEAT_SME_F8F16 for -cpu max
target/arm: Implement FVDOT (FP8 to FP16)
target/arm: Rename FVDOT pattern
target/arm: Implement FMOPA (widening, 2-way, FP8 to FP16)
target/arm: Implement FDOT (multiple and indexed, FP8 to FP16)
target/arm: Implement FDOT (multiple, multiple and single, FP8 to FP16)
target/arm: Implement FMLAL (multiple and indexed, FP8 to FP16)
target/arm: Implement FMLAL (multiple, multiple and single, FP8 to FP16)
target/arm: Rename SME FMLAL/FMLSL patterns
target/arm: Enable FADD/FSUB (half-precision) with FEAT_SME_F8F16
docs/system: add FEAT_ECV_POFF to the emulation list
target/arm: trigger timer recalc on HCR:(E2H|TGE) changes
target/arm: gate check on scr_el3 behind ARM_FEATURE_EL3 check
target/arm: trigger timer recalc on SCR:ECVEN change
target/arm: trigger timer recalculation when toggling CNTHCTL:ECV
target/arm: split evaluation of CNTHCTL timer IRQ masks
docs/arm/sabrelite: Mention FlexCAN support
tests: Add qtests for FlexCAN
hw/arm: Plug FlexCAN into FSL_IMX6 and Sabrelite
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
48 files changed, 3431 insertions, 177 deletions
diff --git a/MAINTAINERS b/MAINTAINERS index 2ecfd7159d..08e04b6cbf 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2123,6 +2123,15 @@ F: hw/net/can/xlnx-* F: include/hw/net/xlnx-* F: tests/qtest/xlnx-can*-test* +FlexCAN +M: Matyas Bobek <matyas.bobek@gmail.com> +M: Pavel Pisa <pisa@cmp.felk.cvut.cz> +S: Maintained +F: hw/net/can/flexcan.c +F: hw/net/can/flexcan_regs.h +F: include/hw/net/flexcan.h +F: tests/qtest/flexcan-test.c + EDU M: Jiri Slaby <jslaby@suse.cz> S: Maintained diff --git a/bsd-user/aarch64/target_arch_elf.h b/bsd-user/aarch64/target_arch_elf.h index 5ea5232a57..2fb7dcd778 100644 --- a/bsd-user/aarch64/target_arch_elf.h +++ b/bsd-user/aarch64/target_arch_elf.h @@ -126,7 +126,7 @@ static uint32_t get_elf_hwcap2(void) GET_FEATURE_ID(aa64_dcpodp, ARM_HWCAP2_A64_DCPODP); GET_FEATURE_ID(aa64_sve2, ARM_HWCAP2_A64_SVE2); - GET_FEATURE_ID(aa64_sve2_aes, ARM_HWCAP2_A64_SVEAES); + GET_FEATURE_ID(aa64_sve_aes, ARM_HWCAP2_A64_SVEAES); GET_FEATURE_ID(aa64_sve2_pmull128, ARM_HWCAP2_A64_SVEPMULL); GET_FEATURE_ID(aa64_sve2_bitperm, ARM_HWCAP2_A64_SVEBITPERM); GET_FEATURE_ID(aa64_sve2_sha3, ARM_HWCAP2_A64_SVESHA3); diff --git a/docs/system/arm/emulation.rst b/docs/system/arm/emulation.rst index a8072ddb67..191d1a8c93 100644 --- a/docs/system/arm/emulation.rst +++ b/docs/system/arm/emulation.rst @@ -60,6 +60,7 @@ the following architecture extensions: - FEAT_E2H0 (Programming of HCR_EL2.E2H) - FEAT_EBF16 (AArch64 Extended BFloat16 instructions) - FEAT_ECV (Enhanced Counter Virtualization) +- FEAT_ECV_POFF (Enhanced Counter Virtualization Physical Offset) - FEAT_EL0 (Support for execution at EL0) - FEAT_EL1 (Support for execution at EL1) - FEAT_EL2 (Support for execution at EL2) @@ -169,8 +170,11 @@ the following architecture extensions: - FEAT_SME_FA64 (Full A64 instruction set in Streaming SVE mode) - FEAT_SME_F16F16 (Non-widening half-precision FP16 arithmetic for SME2) - FEAT_SME_F64F64 (Double-precision floating-point outer product instructions) +- FEAT_SME_F8F16 (SME2 ZA-targeting FP8 multiply-accumulate, dot product, and outer product to half-precision instructions) +- FEAT_SME_F8F32 (SME2 ZA-targeting FP8 multiply-accumulate, dot product, and outer product to single-precision instructions) - FEAT_SME_I16I64 (16-bit to 64-bit integer widening outer product instructions) - FEAT_SME_LUTv2 (Lookup table instructions with 4-bit indices and 8-bit elements) +- FEAT_SSVE_AES (Streaming SVE Mode Advanced Encryption Standard and 128-bit polynomial multiply long instructions) - FEAT_SSVE_FP8DOT2 (SVE2 FP8 2-way dot product to half-precision instructions in Streaming SVE mode) - FEAT_SSVE_FP8DOT4 (SVE2 FP8 4-way dot product to single-precision instructions in Streaming SVE mode) - FEAT_SSVE_FP8FMA (SVE2 FP8 multiply-accumulate to half-precision and single-precision instructions in Streaming SVE mode) diff --git a/docs/system/arm/sabrelite.rst b/docs/system/arm/sabrelite.rst index 4ccb0560af..d3a3c01dd6 100644 --- a/docs/system/arm/sabrelite.rst +++ b/docs/system/arm/sabrelite.rst @@ -24,6 +24,7 @@ The SABRE Lite machine supports the following devices: * 4 SDHC storage controllers * 4 USB 2.0 host controllers * 5 ECSPI controllers + * 2 FlexCAN CAN controllers * 1 SST 25VF016B flash Please note above list is a complete superset the QEMU SABRE Lite machine can diff --git a/docs/system/arm/virt.rst b/docs/system/arm/virt.rst index f811e662d6..ae0be35d57 100644 --- a/docs/system/arm/virt.rst +++ b/docs/system/arm/virt.rst @@ -288,6 +288,32 @@ User-creatable SMMUv3 devices -device pxb-pcie,id=pcie.1,numa_node=1 -device arm-smmuv3,primary-bus=pcie.1,id=smmuv3.1 + *Accelerated SMMUv3 (nested translation)* + + The ``accel=on`` option enables hardware-accelerated nested translation + for vfio-pci passthrough devices. In this mode the guest SMMU driver + programs its own Stage-1 page tables, with the host SMMUv3 handling both + Stage-1 (guest) and Stage-2 (host) translations in hardware. The host + SMMUv3 must support nested translation. This mode requires the iommufd + backend and is only supported when booting with ACPI (not device tree). + + When ``accel=on``, QEMU automatically derives the values for the + ``ril``, ``ats``, ``oas``, ``ssidsize`` and ``cmdqv`` sub-options + from the host SMMUv3 capabilities unless they are set explicitly. + + Example:: + + -device arm-smmuv3,primary-bus=pcie.0,id=smmuv3.0,accel=on + + *Accelerated SMMUv3 command queues (Tegra241 CMDQV)* + + The ``cmdqv`` sub-option enables NVIDIA Tegra241 Command Queue + Virtualization (CMDQV) on supported hosts. With CMDQV, each accelerated + ``arm-smmuv3`` device gets dedicated hardware command queues and the + guest issues SMMU invalidation commands directly to real hardware, + bypassing QEMU and improving throughput for workloads that issue many + invalidations. Without it, every invalidation command traps into QEMU. + Linux guest kernel configuration """""""""""""""""""""""""""""""" diff --git a/docs/system/devices/can.rst b/docs/system/devices/can.rst index 09121836fd..622f898952 100644 --- a/docs/system/devices/can.rst +++ b/docs/system/devices/can.rst @@ -173,6 +173,30 @@ The test can also be run the other way around, generating messages in the guest system and capturing them in the host system. Other combinations are also possible. +Examples on how to use CAN emulation for FlexCAN on SabreLite board +------------------------------------------------------------------- +FlexCANs are connected to QEMU CAN buses by passing the bus IDs as machine +properties: + +* property ``canbus0`` for connecting ``flexcan1`` +* property ``canbus1`` for connecting ``flexcan2`` + +Note that upstream Linux SabreLite DTs have only a single FlexCAN (``flexcan1``) +enabled. + +An example command to run QEMU emulating a Sabrelite development board +with both FlexCANs connected to a single QEMU CAN bus (called ``qcan0``), +bridged to host system ``can0`` interface:: + + qemu-system-arm -M sabrelite -smp 4 -m 1G \ + -object can-bus,id=qcan0 \ + -machine canbus0=qcan0 -machine canbus1=qcan0 \ + -object can-host-socketcan,if=can0,canbus=qcan0,id=qcan0-socketcan \ + -kernel ... -dtb ... -initrd ... + +Note that in the Linux guest, bitrate for the FlexCAN device is ignored, +but needs to be set via the ``ip`` command. + Links to other resources ------------------------ diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig index 500bfdfe2a..5f5c4899ad 100644 --- a/hw/arm/Kconfig +++ b/hw/arm/Kconfig @@ -520,6 +520,7 @@ config FSL_IMX6 select IMX_FEC select IMX_I2C select IMX_USBPHY + select CAN_FLEXCAN select WDT_IMX2 select PL310 # cache controller select PCI_EXPRESS_DESIGNWARE diff --git a/hw/arm/fsl-imx6.c b/hw/arm/fsl-imx6.c index f663ddbf0a..bf106a9063 100644 --- a/hw/arm/fsl-imx6.c +++ b/hw/arm/fsl-imx6.c @@ -90,6 +90,10 @@ static void fsl_imx6_init(Object *obj) snprintf(name, NAME_SIZE, "spi%d", i + 1); object_initialize_child(obj, name, &s->spi[i], TYPE_IMX_SPI); } + for (i = 0; i < FSL_IMX6_NUM_CANS; i++) { + snprintf(name, NAME_SIZE, "flexcan%d", i + 1); + object_initialize_child(obj, name, &s->flexcan[i], TYPE_CAN_FLEXCAN); + } for (i = 0; i < FSL_IMX6_NUM_WDTS; i++) { snprintf(name, NAME_SIZE, "wdt%d", i); object_initialize_child(obj, name, &s->wdt[i], TYPE_IMX2_WDT); @@ -377,6 +381,27 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp) qdev_get_gpio_in(gic, spi_table[i].irq)); } + /* Initialize all FLEXCANs */ + for (i = 0; i < FSL_IMX6_NUM_CANS; i++) { + static const struct { + hwaddr addr; + unsigned int irq; + } flexcan_table[FSL_IMX6_NUM_CANS] = { + { FSL_IMX6_CAN1_ADDR, FSL_IMX6_FLEXCAN1_IRQ }, + { FSL_IMX6_CAN2_ADDR, FSL_IMX6_FLEXCAN2_IRQ }, + }; + + s->flexcan[i].ccm = IMX_CCM(&s->ccm); + object_property_set_link(OBJECT(&s->flexcan[i]), "canbus", + OBJECT(s->canbus[i]), &error_abort); + + sysbus_realize(SYS_BUS_DEVICE(&s->flexcan[i]), &error_abort); + + sysbus_mmio_map(SYS_BUS_DEVICE(&s->flexcan[i]), 0, flexcan_table[i].addr); + sysbus_connect_irq(SYS_BUS_DEVICE(&s->flexcan[i]), 0, + qdev_get_gpio_in(gic, flexcan_table[i].irq)); + } + object_property_set_uint(OBJECT(&s->eth), "phy-num", s->phy_num, &error_abort); qemu_configure_nic_device(DEVICE(&s->eth), true, NULL); @@ -480,6 +505,10 @@ static void fsl_imx6_realize(DeviceState *dev, Error **errp) static const Property fsl_imx6_properties[] = { DEFINE_PROP_UINT32("fec-phy-num", FslIMX6State, phy_num, 0), + DEFINE_PROP_LINK("canbus0", FslIMX6State, canbus[0], TYPE_CAN_BUS, + CanBusState *), + DEFINE_PROP_LINK("canbus1", FslIMX6State, canbus[1], TYPE_CAN_BUS, + CanBusState *), }; static void fsl_imx6_class_init(ObjectClass *oc, const void *data) diff --git a/hw/arm/sabrelite.c b/hw/arm/sabrelite.c index db5669c5c2..84bbc2f979 100644 --- a/hw/arm/sabrelite.c +++ b/hw/arm/sabrelite.c @@ -20,6 +20,16 @@ #include "qemu/error-report.h" #include "system/qtest.h" +struct SabreliteMachineState { + MachineState parent_obj; + + FslIMX6State soc; + CanBusState *canbus[FSL_IMX6_NUM_CANS]; +}; + +#define TYPE_SABRELITE_MACHINE MACHINE_TYPE_NAME("sabrelite") +OBJECT_DECLARE_SIMPLE_TYPE(SabreliteMachineState, SABRELITE_MACHINE) + static struct arm_boot_info sabrelite_binfo = { /* DDR memory start */ .loader_start = FSL_IMX6_MMDC_ADDR, @@ -41,7 +51,7 @@ static void sabrelite_reset_secondary(ARMCPU *cpu, static void sabrelite_init(MachineState *machine) { - FslIMX6State *s; + SabreliteMachineState *s = SABRELITE_MACHINE(machine); /* Check the amount of memory is compatible with the SOC */ if (machine->ram_size > FSL_IMX6_MMDC_SIZE) { @@ -50,13 +60,19 @@ static void sabrelite_init(MachineState *machine) exit(1); } - s = FSL_IMX6(object_new(TYPE_FSL_IMX6)); - object_property_add_child(OBJECT(machine), "soc", OBJECT(s)); + object_initialize_child(OBJECT(machine), "soc", &s->soc, TYPE_FSL_IMX6); /* Ethernet PHY address is 6 */ - object_property_set_int(OBJECT(s), "fec-phy-num", 6, &error_fatal); + object_property_set_int(OBJECT(&s->soc), "fec-phy-num", 6, &error_fatal); - qdev_realize(DEVICE(s), NULL, &error_fatal); + for (int i = 0; i < FSL_IMX6_NUM_CANS; i++) { + g_autofree char *bus_name = g_strdup_printf("canbus%d", i); + + object_property_set_link(OBJECT(&s->soc), bus_name, + OBJECT(s->canbus[i]), &error_fatal); + } + + qdev_realize(DEVICE(&s->soc), NULL, &error_fatal); memory_region_add_subregion(get_system_memory(), FSL_IMX6_MMDC_ADDR, machine->ram); @@ -70,7 +86,7 @@ static void sabrelite_init(MachineState *machine) /* Add the sst25vf016b NOR FLASH memory to first SPI */ Object *spi_dev; - spi_dev = object_resolve_path_component(OBJECT(s), "spi1"); + spi_dev = object_resolve_path_component(OBJECT(&s->soc), "spi1"); if (spi_dev) { SSIBus *spi_bus; @@ -89,7 +105,7 @@ static void sabrelite_init(MachineState *machine) qdev_realize_and_unref(flash_dev, BUS(spi_bus), &error_fatal); cs_line = qdev_get_gpio_in_named(flash_dev, SSI_GPIO_CS, 0); - qdev_connect_gpio_out(DEVICE(&s->gpio[2]), 19, cs_line); + qdev_connect_gpio_out(DEVICE(&s->soc.gpio[2]), 19, cs_line); } } } @@ -100,12 +116,29 @@ static void sabrelite_init(MachineState *machine) sabrelite_binfo.secondary_cpu_reset_hook = sabrelite_reset_secondary; if (!qtest_enabled()) { - arm_load_kernel(&s->cpu[0], machine, &sabrelite_binfo); + arm_load_kernel(&s->soc.cpu[0], machine, &sabrelite_binfo); } } -static void sabrelite_machine_init(MachineClass *mc) +static void sabrelite_machine_instance_init(Object *obj) +{ + SabreliteMachineState *s = SABRELITE_MACHINE(obj); + + object_property_add_link(obj, "canbus0", TYPE_CAN_BUS, + (Object **)&s->canbus[0], + object_property_allow_set_link, + 0); + + object_property_add_link(obj, "canbus1", TYPE_CAN_BUS, + (Object **)&s->canbus[1], + object_property_allow_set_link, + 0); +} + +static void sabrelite_machine_class_init(ObjectClass *oc, const void *data) { + MachineClass *mc = MACHINE_CLASS(oc); + mc->desc = "Freescale i.MX6 Quad SABRE Lite Board (Cortex-A9)"; mc->init = sabrelite_init; mc->max_cpus = FSL_IMX6_NUM_CPUS; @@ -114,4 +147,19 @@ static void sabrelite_machine_init(MachineClass *mc) mc->auto_create_sdcard = true; } -DEFINE_MACHINE_ARM("sabrelite", sabrelite_machine_init) +static const TypeInfo sabrelite_machine_init_typeinfo = { + .name = TYPE_SABRELITE_MACHINE, + .parent = TYPE_MACHINE, + .class_init = sabrelite_machine_class_init, + .instance_init = sabrelite_machine_instance_init, + .instance_size = sizeof(SabreliteMachineState), + .abstract = false, + .interfaces = arm_machine_interfaces, +}; + +static void sabrelite_machine_init_register_types(void) +{ + type_register_static(&sabrelite_machine_init_typeinfo); +} + +type_init(sabrelite_machine_init_register_types) diff --git a/hw/intc/arm_gicv5.c b/hw/intc/arm_gicv5.c index 6b1dd04991..08878193df 100644 --- a/hw/intc/arm_gicv5.c +++ b/hw/intc/arm_gicv5.c @@ -4,6 +4,11 @@ * Copyright (c) 2025 Linaro Limited * * SPDX-License-Identifier: GPL-2.0-or-later + * + * The IRS is defined in IHI 111701 + * (ARM Generic Interrupt Controller Architecture Specification, + * GIC architecture version 5): + * https://developer.arm.com/documentation/111701/latest */ #include "qemu/osdep.h" @@ -424,8 +429,7 @@ static void irs_recalc_hppi(GICv5 *s, GICv5Domain domain, uint32_t iaffid) ARMCPU *cpu = cpuidx >= 0 ? cs->cpus[cpuidx] : NULL; GICv5PendingIrq best; - best.intid = 0; - best.prio = PRIO_IDLE; + best = GICV5_PENDING_IRQ_NONE; if (!cpu) { /* Nothing happens for iaffids targeting nonexistent CPUs */ @@ -521,8 +525,7 @@ static void irs_recall_hppis(GICv5 *s, GICv5Domain domain) GICv5Common *cs = ARM_GICV5_COMMON(s); for (int i = 0; i < cs->num_cpus; i++) { - s->hppi[domain][i].intid = 0; - s->hppi[domain][i].prio = PRIO_IDLE; + s->hppi[domain][i] = GICV5_PENDING_IRQ_NONE; gicv5_forward_interrupt(cs->cpus[i], domain); } } diff --git a/hw/misc/imx6_ccm.c b/hw/misc/imx6_ccm.c index a10b67d396..45fdd0d5a8 100644 --- a/hw/misc/imx6_ccm.c +++ b/hw/misc/imx6_ccm.c @@ -257,6 +257,15 @@ static uint64_t imx6_analog_get_pll2_clk(IMX6CCMState *dev) return freq; } +static uint64_t imx6_analog_get_pll3_clk(IMX6CCMState *dev) +{ + uint64_t freq = 480000000; + + trace_imx6_analog_get_pll3_clk(freq); + + return freq; +} + static uint64_t imx6_analog_get_pll2_pfd0_clk(IMX6CCMState *dev) { uint64_t freq = 0; @@ -344,6 +353,18 @@ static uint64_t imx6_ccm_get_per_clk(IMX6CCMState *dev) return freq; } +static uint64_t imx6_ccm_get_can_clk(IMX6CCMState *dev) +{ + uint64_t freq = 0; + + freq = imx6_analog_get_pll3_clk(dev) / 8; + freq /= (1 + EXTRACT(dev->ccm[CCM_CSCMR2], CAN_CLK_PODF)); + + trace_imx6_ccm_get_can_clk(freq); + + return freq; +} + static uint32_t imx6_ccm_get_clock_frequency(IMXCCMState *dev, IMXClk clock) { uint32_t freq = 0; @@ -358,6 +379,9 @@ static uint32_t imx6_ccm_get_clock_frequency(IMXCCMState *dev, IMXClk clock) case CLK_IPG_HIGH: freq = imx6_ccm_get_per_clk(s); break; + case CLK_CAN: + freq = imx6_ccm_get_can_clk(s); + break; case CLK_32k: freq = CKIL_FREQ; break; diff --git a/hw/misc/trace-events b/hw/misc/trace-events index b88accc437..99910fc068 100644 --- a/hw/misc/trace-events +++ b/hw/misc/trace-events @@ -242,11 +242,13 @@ imx6_analog_get_periph_clk(uint32_t freq) "freq = %u Hz" imx6_analog_get_pll2_clk(uint32_t freq) "freq = %u Hz" imx6_analog_get_pll2_pfd0_clk(uint32_t freq) "freq = %u Hz" imx6_analog_get_pll2_pfd2_clk(uint32_t freq) "freq = %u Hz" +imx6_analog_get_pll3_clk(uint32_t freq) "freq = %u Hz" imx6_analog_read(const char *reg, uint32_t value) "reg[%s] => 0x%" PRIx32 imx6_analog_write(const char *reg, uint32_t value) "reg[%s] <= 0x%" PRIx32 imx6_ccm_get_ahb_clk(uint32_t freq) "freq = %u Hz" imx6_ccm_get_ipg_clk(uint32_t freq) "freq = %u Hz" imx6_ccm_get_per_clk(uint32_t freq) "freq = %u Hz" +imx6_ccm_get_can_clk(uint32_t freq) "freq = %u Hz" imx6_ccm_get_clock_frequency(unsigned clock, uint32_t freq) "(Clock = %d) = %u" imx6_ccm_read(const char *reg, uint32_t value) "reg[%s] => 0x%" PRIx32 imx6_ccm_reset(void) "" diff --git a/hw/net/Kconfig b/hw/net/Kconfig index f9a1dfb80d..b56a173eed 100644 --- a/hw/net/Kconfig +++ b/hw/net/Kconfig @@ -157,3 +157,8 @@ config CAN_CTUCANFD_PCI default y if PCI_DEVICES depends on PCI && CAN_CTUCANFD select CAN_BUS + +config CAN_FLEXCAN + bool + depends on IMX + select CAN_BUS diff --git a/hw/net/can/flexcan.c b/hw/net/can/flexcan.c new file mode 100644 index 0000000000..1ea459d9f6 --- /dev/null +++ b/hw/net/can/flexcan.c @@ -0,0 +1,1396 @@ +/* + * QEMU model of the NXP FLEXCAN device. + * + * This implementation is based on the following reference manual: + * i.MX 6Dual/6Quad Applications Processor Reference Manual + * Document Number: IMX6DQRM, Rev. 6, 05/2020 + * + * Copyright (c) 2025 Matyas Bobek <matyas.bobek@gmail.com> + * + * Based on CTU CAN FD emulation implemented by Jan Charvat. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "qemu/log.h" +#include "hw/core/sysbus.h" +#include "qapi/error.h" +#include "hw/core/irq.h" +#include "migration/vmstate.h" +#include "net/can_emu.h" +#include "hw/core/qdev-properties.h" +#include "trace.h" + +#include "hw/net/flexcan.h" +#include "flexcan_regs.h" +#include "qemu/timer.h" + +/* + * Indicates MB w/ received frame has not been serviced yet + * This is an emulator-only flag in position of unused (reserved) bit + * of message buffer control register + */ +#define FLEXCAN_MB_CNT_NOT_SRV BIT(23) +/** + * if no MB is locked, FlexcanState.locked_mb + * is set to FLEXCAN_NO_MB_LOCKED + */ +#define FLEXCAN_NO_MB_LOCKED -1 +/** + * if no frame is waiting in the SMB, FlexcanState.smb_target_mbid + * is set to FLEXCAN_SMB_EMPTY + */ +#define FLEXCAN_SMB_EMPTY -1 +/** + * When the module is disabled or in freeze mode, + * the timer is not running. That is indicated by setting + * FlexcanState.timer_start to FLEXCAN_TIMER_STOPPED. + */ +#define FLEXCAN_TIMER_STOPPED -1 + +/* These constants are returned by flexcan_fifo_rx() and flexcan_mb_rx(), */ +enum FlexcanRx { +/* Retry the other receiving mechanism (ie. message bufer or mailbox). */ + FLEXCAN_RX_SEARCH_RETRY, +/* The frame was received and stored. */ + FLEXCAN_RX_SEARCH_ACCEPT, +/* The frame was filtered out and dropped. */ + FLEXCAN_RX_SEARCH_DROPPED, +}; + +/* + * These constants are returned by flexcan_mb_rx_check_mb(). + * See flexcan_mb_rx_check_mb() kerneldoc for details. + */ +enum FlexcanCheck { + FLEXCAN_CHECK_MB_NIL = 0, + FLEXCAN_CHECK_MB_MATCH = 3, + FLEXCAN_CHECK_MB_MATCH_NON_FREE = 1, + FLEXCAN_CHECK_MB_MATCH_LOCKED = 5, +}; + +static const FlexcanRegs flexcan_regs_write_mask = { + .mcr = 0xF6EB337F, + .ctrl = 0xFFFFFFFF, + .timer = 0xFFFFFFFF, + .tcr = 0xFFFFFFFF, + .rxmgmask = 0xFFFFFFFF, + .rx14mask = 0xFFFFFFFF, + .rx15mask = 0xFFFFFFFF, + .ecr = 0xFFFFFFFF, + .esr = 0xFFFFFFFF, + .imask2 = 0xFFFFFFFF, + .imask1 = 0xFFFFFFFF, + .iflag2 = 0, + .iflag1 = 0, + .ctrl2 = 0xFFFFFFFF, + .esr2 = 0, + .imeur = 0, + .lrfr = 0, + .crcr = 0, + .rxfgmask = 0xFFFFFFFF, + .rxfir = 0, + .cbt = 0, + ._reserved2 = 0, + .dbg1 = 0, + .dbg2 = 0, + .mbs = { [0 ... 63] = { + .can_ctrl = 0xFFFFFFFF & ~FLEXCAN_MB_CNT_NOT_SRV, + .can_id = 0xFFFFFFFF, + .data = { 0xFFFFFFFF, 0xFFFFFFFF }, + } }, + ._reserved4 = {0}, + .rximr = { [0 ... 63] = 0xFFFFFFFF }, + ._reserved5 = {0}, + .gfwr_mx6 = 0xFFFFFFFF, + ._reserved6 = {0}, + ._reserved8 = {0}, + .rx_smb0_raw = {0, 0, 0, 0}, + .rx_smb1 = {0, 0, 0, 0}, +}; +static const FlexcanRegs flexcan_regs_reset_mask = { + .mcr = 0x80000000, + .ctrl = 0xFFFFFFFF, + .timer = 0, + .tcr = 0, + .rxmgmask = 0xFFFFFFFF, + .rx14mask = 0xFFFFFFFF, + .rx15mask = 0xFFFFFFFF, + .ecr = 0, + .esr = 0, + .imask2 = 0, + .imask1 = 0, + .iflag2 = 0, + .iflag1 = 0, + .ctrl2 = 0xFFFFFFFF, + .esr2 = 0, + .imeur = 0, + .lrfr = 0, + .crcr = 0, + .rxfgmask = 0xFFFFFFFF, + .rxfir = 0xFFFFFFFF, + .cbt = 0, + ._reserved2 = 0, + .dbg1 = 0, + .dbg2 = 0, + .mb = {0xFFFFFFFF}, + ._reserved4 = {0}, + .rximr = {0xFFFFFFFF}, + ._reserved5 = {0}, + .gfwr_mx6 = 0, + ._reserved6 = {0}, + ._reserved8 = {0}, + .rx_smb0_raw = {0, 0, 0, 0}, + .rx_smb1 = {0, 0, 0, 0}, +}; + +/* length of buffer used to format register names in trace output */ +#define FLEXCAN_DBG_BUF_LEN 16 + +/** + * flexcan_dbg_mb_code_strs - Readable names for CODE field codes + * + * Readable names for possible values of CODE field in message buffer + * control word. + */ +static const char *flexcan_dbg_mb_code_strs[16] = { + "INACTIVE_RX", + "FULL", + "EMPTY", + "OVERRUN", + "INACTIVE_TX", + "RANSWER", + "DATA", + "TANSWER" +}; + +/** + * flexcan_dbg_mb_code() - Get the string representation of a mailbox code + * @mb_ctrl: The mailbox control register value + * @buf: The buffer to store the string representation + * + * Return: Either constant string or string formatted into @buf + */ +static const char *flexcan_dbg_mb_code(uint32_t mb_ctrl, char *buf) +{ + uint32_t code = mb_ctrl & FLEXCAN_MB_CODE_MASK; + uint32_t code_idx = code >> 24; + if (code == FLEXCAN_MB_CODE_TX_ABORT) { + return "ABORT"; + } else { + const char *code_str = flexcan_dbg_mb_code_strs[code_idx >> 1]; + if (code_idx & 1) { + g_snprintf(buf, FLEXCAN_DBG_BUF_LEN, "%s+BUSY", code_str); + return buf; + } + + return code_str; + } +} + +static const char *flexcan_dbg_reg_name_fixed(hwaddr addr) +{ + switch (addr) { + case offsetof(FlexcanRegs, mcr): + return "MCR"; + case offsetof(FlexcanRegs, ctrl): + return "CTRL"; + case offsetof(FlexcanRegs, timer): + return "TIMER"; + case offsetof(FlexcanRegs, esr): + return "ESR"; + case offsetof(FlexcanRegs, rxmgmask): + return "RXMGMASK"; + case offsetof(FlexcanRegs, rx14mask): + return "RX14MASK"; + case offsetof(FlexcanRegs, rx15mask): + return "RX15MASK"; + case offsetof(FlexcanRegs, rxfgmask): + return "RXFGMASK"; + case offsetof(FlexcanRegs, ecr): + return "ECR"; + case offsetof(FlexcanRegs, ctrl2): + return "CTRL2"; + case offsetof(FlexcanRegs, imask2): + return "IMASK2"; + case offsetof(FlexcanRegs, imask1): + return "IMASK1"; + case offsetof(FlexcanRegs, iflag2): + return "IFLAG2"; + case offsetof(FlexcanRegs, iflag1): + return "IFLAG1"; + } + return NULL; +} + +static inline void flexcan_trace_mem_op(FlexcanState *s, hwaddr addr, + uint32_t value, int size, bool is_wr) +{ + if (trace_event_get_state_backends(TRACE_FLEXCAN_MEM_OP)) { + const char *reg_name = "unknown"; + char reg_name_buf[FLEXCAN_DBG_BUF_LEN] = { 0 }; + const char *reg_name_fixed = flexcan_dbg_reg_name_fixed(addr); + const char *op_string = is_wr ? "write" : "read"; + + if (reg_name_fixed) { + reg_name = reg_name_fixed; + } else if (addr >= 0x80 && addr < 0x480) { + int mbidx = (addr - 0x80) / 16; + g_snprintf(reg_name_buf, sizeof(reg_name_buf), "MB%i", mbidx); + reg_name = reg_name_buf; + } else if (addr >= 0x880 && addr < 0x9e0) { + int id = (addr - 0x880) / 4; + g_snprintf(reg_name_buf, sizeof(reg_name_buf), "RXIMR%i", id); + reg_name = reg_name_buf; + } + + trace_flexcan_mem_op(DEVICE(s)->canonical_path, op_string, value, addr, + reg_name, size); + } +} + +static enum FlexcanRx flexcan_mb_rx(FlexcanState *s, + const qemu_can_frame *frame); +static void flexcan_mb_unlock(FlexcanState *s); + +/* ========== Mailbox Utils ========== */ + +/** + * flexcan_mailbox_count() - Get number of enabled mailboxes + * @s: FlexCAN device pointer + * + * Count is based on MCR[MAXMB] field. Note that some of those mailboxes + * might be part of queue or queue ID filters or ordinary message buffers. + */ +static inline int flexcan_enabled_mailbox_count(const FlexcanState *s) +{ + return MIN((s->regs.mcr & FLEXCAN_MCR_MAXMB(UINT32_MAX)) + 1, + FLEXCAN_MAILBOX_COUNT); +} + +/** + * flexcan_get_first_message_buffer() - Get pointer to first message buffer + * @s: FlexCAN device pointer + * + * In context of this function, message buffer means a mailbox which is not + * a queue element nor a queue filter. Note this function does not take + * MCR[MAXMB] into account, meaning that the returned mailbox + * might be disabled. + */ +static FlexcanRegsMessageBuffer *flexcan_get_first_message_buffer( + FlexcanState *s) +{ + if (s->regs.mcr & FLEXCAN_MCR_FEN) { + int rffn = (s->regs.ctrl2 & FLEXCAN_CTRL2_RFFN(UINT32_MAX)) >> 24; + return s->regs.mbs + 8 + 2 * rffn; + } + + return s->regs.mbs; +} + +/** + * flexcan_get_last_enabled_mailbox() - Get pointer to last enabled mailbox. + * @s: FlexCAN device pointer + * + * When used with flexcan_get_first_message_buffer(), all mailboxes *ptr in + * range `first_message_buffer() <= ptr <= last_enabled_mailbox` are valid + * message buffer mailboxes. + * + * Return: Last enabled mailbox in MCR[MAXMB] sense. The mailbox might be + * of any type. + */ +static inline FlexcanRegsMessageBuffer *flexcan_get_last_enabled_mailbox( + FlexcanState *s) +{ + return s->regs.mbs + flexcan_enabled_mailbox_count(s); +} + +/* ========== Free-running Timer ========== */ +static inline int64_t flexcan_get_time(void) +{ + return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL); +} + +/** + * flexcan_get_bitrate() - Calculate CAN bitrate (in Hz) + * @s: FlexCAN device pointer + * + * The bitrate is determined by FlexCAN configuration in CTRL1 register, + * and CCM co + */ +static uint32_t flexcan_get_bitrate(FlexcanState *s) +{ + uint32_t conf_presdiv = (s->regs.ctrl & FLEXCAN_CTRL_PRESDIV_MASK) >> 24; + uint32_t conf_pseg1 = (s->regs.ctrl & FLEXCAN_CTRL_PSEG1_MASK) >> 19; + uint32_t conf_pseg2 = (s->regs.ctrl & FLEXCAN_CTRL_PSEG2_MASK) >> 16; + uint32_t conf_propseg = s->regs.ctrl & FLEXCAN_CTRL_PROPSEG_MASK; + + /* N of time quanta for segments */ + uint32_t tseg1 = 2 + conf_pseg1 + conf_propseg; + uint32_t tseg2 = 1 + conf_pseg2; + uint32_t total_qpb = 1 + tseg1 + tseg2; + + uint32_t pe_freq, s_freq, bitrate; + + assert(s->ccm); + + /* s_freq: CAN clock from CCM divided by the prescaler */ + pe_freq = imx_ccm_get_clock_frequency(s->ccm, CLK_CAN); + s_freq = pe_freq / (1 + conf_presdiv); + bitrate = s_freq / total_qpb; + + trace_flexcan_get_bitrate(DEVICE(s)->canonical_path, pe_freq, + 1 + conf_presdiv, s_freq, tseg1, tseg2, total_qpb, + bitrate); + return bitrate; +} + +/** + * int128_mul_6464() - Multiply two 64-bit integers into a 128-bit one + */ +static Int128 int128_muls_6464(int64_t ai, int64_t bi) +{ + uint64_t l, h; + + muls64(&l, &h, ai, bi); + return int128_make128(l, h); +} + +/** + * flexcan_get_timestamp() - Get current value of the 16-bit free-running timer + * @s: FlexCAN device pointer + * @mk_unique: if true, make the timestamp unique by incrementing it if needed + */ +static uint32_t flexcan_get_timestamp(FlexcanState *s, bool mk_unique) +{ + const Int128 nanoseconds_in_second = int128_makes64((int64_t)1e9); + Int128 ncycles, cycles128; + int64_t current_time, elapsed_time_ns; + uint64_t cycles; + uint32_t rv, shift = 0; + + if (s->timer_start == FLEXCAN_TIMER_STOPPED) { + /* timer is not running, return last value */ + trace_flexcan_get_timestamp(DEVICE(s)->canonical_path, -1, 0, 0, 0, + s->regs.timer); + return s->regs.timer; + } + + current_time = flexcan_get_time(); + elapsed_time_ns = current_time - s->timer_start; + if (elapsed_time_ns < 0) { + trace_flexcan_timer_overflow(DEVICE(s)->canonical_path, current_time, + s->timer_start, elapsed_time_ns); + return 0xFFFF; + } + + ncycles = int128_muls_6464(s->timer_freq, elapsed_time_ns); + cycles128 = int128_divs(ncycles, nanoseconds_in_second); + /* 64 bits hold for over 50k years at 10MHz */ + cycles = int128_getlo(cycles128); + + if (mk_unique && cycles <= s->last_rx_timer_cycles) { + shift = 1; + cycles = s->last_rx_timer_cycles + shift; + } + + s->last_rx_timer_cycles = cycles; + rv = (uint32_t)cycles & 0xFFFF; + + trace_flexcan_get_timestamp(DEVICE(s)->canonical_path, + elapsed_time_ns / (uint32_t)1e6, + s->timer_freq, cycles, shift, rv); + return rv; +} + +/** + * flexcan_timer_start() - Start the free-running timer + * @s: FlexCAN device pointer + * + * This should be called when the module leaves freeze mode. + */ +static void flexcan_timer_start(FlexcanState *s) +{ + s->timer_freq = flexcan_get_bitrate(s); + s->timer_start = flexcan_get_time(); + s->last_rx_timer_cycles = 0; + + trace_flexcan_timer_start(DEVICE(s)->canonical_path, s->timer_freq, + s->regs.timer); +} + +/** + * flexcan_timer_stop() - Stop the free-running timer + * @s: FlexCAN device pointer + * + * This should be called when the module enters freeze mode. + * Stores the current timestamp in the TIMER register. + */ +static void flexcan_timer_stop(FlexcanState *s) +{ + s->regs.timer = flexcan_get_timestamp(s, false); + s->timer_start = FLEXCAN_TIMER_STOPPED; + + trace_flexcan_timer_stop(DEVICE(s)->canonical_path, s->timer_freq, + s->regs.timer); +} + +/* ========== IRQ handling ========== */ +/** + * flexcan_irq_update() - Update qemu_irq line based on interrupt registers + * @s: FlexCAN device pointer + */ +static void flexcan_irq_update(FlexcanState *s) +{ + uint32_t mb_irqs[2]; + int irq_pending; + /* these are all interrupt sources from FlexCAN */ + /* mailbox interrupt sources */ + mb_irqs[0] = s->regs.iflag1 & s->regs.imask1; + mb_irqs[1] = s->regs.iflag2 & s->regs.imask2; + + /** + * these interrupts aren't currently used and they can never be raised + * + * bool irq_wake_up = (s->regs.mcr & FLEXCAN_MCR_WAK_MSK) && + * (s->regs.ecr & FLEXCAN_ESR_WAK_INT); + * bool irq_bus_off = (s->regs.ctrl & FLEXCAN_CTRL_BOFF_MSK) && + * (s->regs.ecr & FLEXCAN_ESR_BOFF_INT); + * bool irq_error = (s->regs.ctrl & FLEXCAN_CTRL_ERR_MSK) && + * (s->regs.ecr & FLEXCAN_ESR_ERR_INT); + * bool irq_tx_warn = (s->regs.ctrl & FLEXCAN_CTRL_TWRN_MSK) && + * (s->regs.ecr & FLEXCAN_ESR_TWRN_INT); + * bool irq_rx_warn = (s->regs.ctrl & FLEXCAN_CTRL_RWRN_MSK) && + * (s->regs.ecr & FLEXCAN_ESR_RWRN_INT); + */ + + irq_pending = (mb_irqs[0] || mb_irqs[1]) ? 1 : 0; + trace_flexcan_irq_update(DEVICE(s)->canonical_path, mb_irqs[0], mb_irqs[1], + irq_pending); + + qemu_set_irq(s->irq, irq_pending); +} + +/** + * flexcan_irq_iflag_set() - Set IFLAG bit corresponding to MB mbidx + * @s: FlexCAN device pointer + * @mbidx: mailbox index + */ +static void flexcan_irq_iflag_set(FlexcanState *s, int mbidx) +{ + if (mbidx < 32) { + s->regs.iflag1 |= BIT(mbidx); + } else { + s->regs.iflag2 |= BIT(mbidx - 32); + } +} + +/** + * flexcan_irq_iflag_clear() - Clear IFLAG bit corresponding to MB mbidx + * @s: FlexCAN device pointer + * @mbidx: mailbox index + */ +static void flexcan_irq_iflag_clear(FlexcanState *s, int mbidx) +{ + if (mbidx < 32) { + s->regs.iflag1 &= ~BIT(mbidx); + } else { + s->regs.iflag2 &= ~BIT(mbidx - 32); + } +} + +/* ========== RESET ========== */ +static void flexcan_reset_local_state(FlexcanState *s) +{ + uint32_t *reset_mask = (uint32_t *)&flexcan_regs_reset_mask; + for (int i = 0; i < (sizeof(FlexcanRegs) / 4); i++) { + s->regs_raw[i] &= reset_mask[i]; + } + + s->regs.mcr |= 0x5980000F; + s->locked_mbidx = FLEXCAN_NO_MB_LOCKED; + s->smb_target_mbidx = FLEXCAN_SMB_EMPTY; + s->timer_start = FLEXCAN_TIMER_STOPPED; + + trace_flexcan_reset(DEVICE(s)->canonical_path); +} + +static void flexcan_reset_enter(Object *obj, ResetType type) +{ + FlexcanState *s = CAN_FLEXCAN(obj); + + memset(&s->regs, 0, sizeof(s->regs)); + flexcan_reset_local_state(s); +} + +static void flexcan_reset_hold(Object *obj, ResetType type) +{ + FlexcanState *s = CAN_FLEXCAN(obj); + + flexcan_irq_update(s); +} + + +/* ========== Operation mode control ========== */ +/** + * flexcan_update_esr() - Update ESR based on mode and CAN bus connection state + * @s: FlexCAN device pointer + */ +static void flexcan_update_esr(FlexcanState *s) +{ + bool is_running = (s->regs.mcr & FLEXCAN_MCR_NOT_RDY) == 0; + /* potentially, there could be other influences on ESR[SYNCH] */ + + if (is_running && s->canbus) { + s->regs.esr |= FLEXCAN_ESR_SYNCH | FLEXCAN_ESR_IDLE; + } else { + s->regs.esr &= ~(FLEXCAN_ESR_SYNCH | FLEXCAN_ESR_IDLE); + } +} + +/** + * flexcan_update_esr() - Process MCR write + * @s: FlexCAN device pointer + * @pv: previously set MCR value + * + * This function expects the new MCR value to be already written in s->regs.mcr. + */ +static void flexcan_set_mcr(FlexcanState *s, const uint32_t pv) +{ + uint32_t cv = s->regs.mcr; + + /* -- module disable mode -- */ + if (!(pv & FLEXCAN_MCR_MDIS) && (cv & FLEXCAN_MCR_MDIS)) { + /* transition to Module Disable mode */ + cv |= FLEXCAN_MCR_LPM_ACK; + } else if ((pv & FLEXCAN_MCR_MDIS) && !(cv & FLEXCAN_MCR_MDIS)) { + /* transition from Module Disable mode */ + cv &= ~FLEXCAN_MCR_LPM_ACK; + } + + /* -- soft reset -- */ + if (!(cv & FLEXCAN_MCR_LPM_ACK) && (cv & FLEXCAN_MCR_SOFTRST)) { + if (s->regs.mcr & FLEXCAN_MCR_LPM_ACK) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: invalid soft reset request in low-power mode", + DEVICE(s)->canonical_path); + } + + flexcan_reset_local_state(s); + cv = s->regs.mcr; + } + + /* -- freeze mode -- */ + if (!(cv & FLEXCAN_MCR_LPM_ACK) && + (cv & FLEXCAN_MCR_FRZ) && + (cv & FLEXCAN_MCR_HALT)) { + cv |= FLEXCAN_MCR_FRZ_ACK; + } else { + cv &= ~FLEXCAN_MCR_FRZ_ACK; + } + + /* -- fifo mode -- */ + if ( + ((pv & FLEXCAN_MCR_FEN) && !(cv & FLEXCAN_MCR_FEN)) || + (!(pv & FLEXCAN_MCR_FEN) && (cv & FLEXCAN_MCR_FEN)) + ) { + /* clear iflags used by fifo */ + s->regs.iflag1 &= ~( + FLEXCAN_IFLAG_RX_FIFO_AVAILABLE | + FLEXCAN_IFLAG_RX_FIFO_OVERFLOW | + FLEXCAN_IFLAG_RX_FIFO_WARN + ); + } + if (!(pv & FLEXCAN_MCR_FEN) && (cv & FLEXCAN_MCR_FEN)) { + /* zero out fifo region, we rely on zeroed can_ctrl for empty slots */ + memset(s->regs.mbs, 0, + FLEXCAN_FIFO_DEPTH * sizeof(FlexcanRegsMessageBuffer)); + } + + /* + * assert NOT_RDY bit if in disable, + * stop (not implemented) or freeze mode + */ + if ((cv & FLEXCAN_MCR_LPM_ACK) || (cv & FLEXCAN_MCR_FRZ_ACK)) { + cv |= FLEXCAN_MCR_NOT_RDY; + } else { + cv &= ~FLEXCAN_MCR_NOT_RDY; + } + + if ((pv & FLEXCAN_MCR_NOT_RDY) && !(cv & FLEXCAN_MCR_NOT_RDY)) { + /* module went up, start the timer */ + flexcan_timer_start(s); + } else if (!(pv & FLEXCAN_MCR_NOT_RDY) && (cv & FLEXCAN_MCR_NOT_RDY)) { + /* module went down, store the current timer value */ + flexcan_timer_stop(s); + } + + s->regs.mcr = cv; + flexcan_update_esr(s); + trace_flexcan_set_mcr( + DEVICE(s)->canonical_path, + cv & FLEXCAN_MCR_LPM_ACK ? "DISABLED" : "ENABLED", + (cv & FLEXCAN_MCR_FRZ_ACK || cv & FLEXCAN_MCR_LPM_ACK) ? + "FROZEN" : "RUNNING", + cv & FLEXCAN_MCR_FEN ? "FIFO" : "MAILBOX", + cv & FLEXCAN_MCR_NOT_RDY ? "NOT_RDY" : "RDY", + s->regs.esr & FLEXCAN_ESR_SYNCH ? "SYNC" : "NOSYNC" + ); +} + +/* ========== TX ========== */ +static void flexcan_transmit(FlexcanState *s, int mbidx) +{ + FlexcanRegsMessageBuffer *mb = &s->regs.mbs[mbidx]; + qemu_can_frame frame = { + .flags = 0, + }; + uint32_t *frame_data = (uint32_t *)&frame.data; + uint32_t timestamp = flexcan_get_timestamp(s, true); + + if ((s->regs.ctrl & FLEXCAN_CTRL_LOM) || + (s->regs.mcr & FLEXCAN_MCR_NOT_RDY)) { + /* no transmiting in listen-only, freeze or low-power mode */ + return; + } + + if (mb->can_ctrl & FLEXCAN_MB_CNT_IDE) { + /* 29b ID stored in bits [0, 29) */ + uint32_t id = mb->can_id & 0x1FFFFFFF; + frame.can_id = id | QEMU_CAN_EFF_FLAG; + } else { + /* 11b ID stored in bits [18, 29) */ + uint32_t id = (mb->can_id & (0x7FF << 18)) >> 18; + frame.can_id = id; + } + + frame.can_dlc = (mb->can_ctrl & (0xF << 16)) >> 16; + + for (int i = 0; i < 2; i++) { + stl_be_p(&frame_data[i], mb->data[i]); + } + + if (!(s->regs.mcr & FLEXCAN_MCR_SRX_DIS)) { + /* self-reception */ + flexcan_mb_rx(s, &frame); + } + if (!(s->regs.ctrl & FLEXCAN_CTRL_LPB)) { + /* send to bus if not in loopback mode */ + if (s->canbus) { + can_bus_client_send(&s->bus_client, &frame, 1); + } else { + /* todo: raise error (no ack) */ + } + } + + mb->can_ctrl &= ~(FLEXCAN_MB_CODE_MASK | FLEXCAN_MB_CNT_TIMESTAMP_MASK); + mb->can_ctrl |= FLEXCAN_MB_CODE_TX_INACTIVE | + FLEXCAN_MB_CNT_TIMESTAMP(timestamp); + + /* todo: compute the CRC */ + s->regs.crcr = FLEXCAN_CRCR_TXCRC(0) | FLEXCAN_CRCR_MBCRC(mbidx); + + flexcan_irq_iflag_set(s, mbidx); +} + +static void flexcan_mb_write(FlexcanState *s, int mbid) +{ + FlexcanRegsMessageBuffer *mb = &s->regs.mbs[mbid]; + + bool is_mailbox = (mb <= flexcan_get_last_enabled_mailbox(s)) && + (mb >= flexcan_get_first_message_buffer(s)); + + if (trace_event_get_state_backends(TRACE_FLEXCAN_MB_WRITE)) { + char code_str_buf[FLEXCAN_DBG_BUF_LEN] = { 0 }; + const char *code_str = flexcan_dbg_mb_code(mb->can_ctrl, code_str_buf); + trace_flexcan_mb_write(DEVICE(s)->canonical_path, mbid, code_str, + is_mailbox, mb->can_ctrl, mb->can_id); + } + + if (!is_mailbox) { + /** + * Disabled mailbox or mailbox in region of queue filters + * was updated. Either way there is nothing to do. + */ + return; + } + + /* any write to message buffer clears the not_serviced flag */ + mb->can_ctrl &= ~FLEXCAN_MB_CNT_NOT_SRV; + + /** + * todo: search for active tx mbs on transition from freeze/disable mode + */ + switch (mb->can_ctrl & FLEXCAN_MB_CODE_MASK) { + case FLEXCAN_MB_CODE_TX_INACTIVE: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_INACTIVE: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_EMPTY: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_FULL: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_RANSWER: + break; + + case FLEXCAN_MB_CODE_TX_DATA: + flexcan_transmit(s, mbid); + break; + case FLEXCAN_MB_CODE_TX_ABORT: + /* + * as transmission is instant, it can never be aborted + * we need to set CODE in C/S back to the previous code + */ + mb->can_ctrl &= ~FLEXCAN_MB_CODE(1); + break; + case FLEXCAN_MB_CODE_TX_TANSWER: + break; + default: + /* prevent setting the busy bit */ + mb->can_ctrl &= ~FLEXCAN_MB_CODE_RX_BUSY_BIT; + break; + } + +} + +/* ========== RX ========== */ +static void flexcan_mb_move_in(FlexcanState *s, const qemu_can_frame *frame, + FlexcanRegsMessageBuffer *target_mb) +{ + uint32_t frame_len = frame->can_dlc; + uint32_t *frame_data = (uint32_t *)&frame->data; + int timestamp = flexcan_get_timestamp(s, true); + uint32_t new_code = 0; + + memset(target_mb, 0, sizeof(FlexcanRegsMessageBuffer)); + + if (frame_len > 8) { + frame_len = 8; + } + for (int i = 0; i < 2; i++) { + target_mb->data[i] = ldl_be_p(&frame_data[i]); + } + + switch (target_mb->can_ctrl & FLEXCAN_MB_CODE_MASK) { + case FLEXCAN_MB_CODE_RX_FULL: + case FLEXCAN_MB_CODE_RX_OVERRUN: + if (target_mb->can_ctrl & FLEXCAN_MB_CNT_NOT_SRV) { + new_code = FLEXCAN_MB_CODE_RX_OVERRUN; + } else { + new_code = FLEXCAN_MB_CODE_RX_FULL; + } + break; + case FLEXCAN_MB_CODE_RX_RANSWER: + assert(s->regs.ctrl2 & FLEXCAN_CTRL2_RRS); + new_code = FLEXCAN_MB_CODE_TX_TANSWER; + break; + default: + new_code = FLEXCAN_MB_CODE_RX_FULL; + } + + target_mb->can_ctrl = new_code + | FLEXCAN_MB_CNT_TIMESTAMP(timestamp) + | FLEXCAN_MB_CNT_LENGTH(frame_len) + | FLEXCAN_MB_CNT_NOT_SRV + | FLEXCAN_MB_CNT_SRR; /* always set for received frames */ + if (frame->can_id & QEMU_CAN_RTR_FLAG) { + target_mb->can_ctrl |= FLEXCAN_MB_CNT_RTR; + } + + if (frame->can_id & QEMU_CAN_EFF_FLAG) { + target_mb->can_ctrl |= FLEXCAN_MB_CNT_IDE; + target_mb->can_id |= frame->can_id & QEMU_CAN_EFF_MASK; + } else { + target_mb->can_id |= (frame->can_id & QEMU_CAN_SFF_MASK) << 18; + } +} +static void flexcan_mb_lock(FlexcanState *s, int mbidx) +{ + FlexcanRegsMessageBuffer *mb = &s->regs.mbs[mbidx]; + if ((mb > flexcan_get_last_enabled_mailbox(s)) || + (mb < flexcan_get_first_message_buffer(s))) { + return; + } + switch (mb->can_ctrl & FLEXCAN_MB_CODE_MASK) { + case FLEXCAN_MB_CODE_RX_FULL: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_OVERRUN: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_RANSWER: + /* continue */ + trace_flexcan_mb_lock(DEVICE(s)->canonical_path, mbidx, 1); + break; + default: + trace_flexcan_mb_lock(DEVICE(s)->canonical_path, mbidx, 0); + return; + } + + s->locked_mbidx = mbidx; +} + +static void flexcan_mb_unlock(FlexcanState *s) +{ + int locked_mbidx = s->locked_mbidx; + bool has_pending_frame = locked_mbidx == s->smb_target_mbidx; + + if (s->locked_mbidx == FLEXCAN_NO_MB_LOCKED) { + return; + } + + assert(locked_mbidx >= 0 && locked_mbidx < FLEXCAN_MAILBOX_COUNT); + FlexcanRegsMessageBuffer *locked_mb = &s->regs.mbs[locked_mbidx]; + s->locked_mbidx = FLEXCAN_NO_MB_LOCKED; + + if (locked_mb >= flexcan_get_first_message_buffer(s) && + locked_mb <= flexcan_get_last_enabled_mailbox(s) + ) { + /* mark the message buffer as serviced */ + locked_mb->can_ctrl &= ~FLEXCAN_MB_CNT_NOT_SRV; + } + + /* try move in from SMB */ + trace_flexcan_mb_unlock(DEVICE(s)->canonical_path, locked_mbidx, + has_pending_frame ? " PENDING FRAME IN SMB" : ""); + + /* todo: in low-power modes, this should be postponed until exit */ + if (has_pending_frame) { + FlexcanRegsMessageBuffer *target_mb = &s->regs.mbs[locked_mbidx]; + memcpy(target_mb, &s->regs.rx_smb0, sizeof(FlexcanRegsMessageBuffer)); + + memset(&s->regs.rx_smb0, 0, sizeof(FlexcanRegsMessageBuffer)); + s->locked_mbidx = FLEXCAN_SMB_EMPTY; + + flexcan_irq_iflag_set(s, locked_mbidx); + } +} + +static bool flexcan_can_receive(CanBusClientState *client) +{ + FlexcanState *s = container_of(client, FlexcanState, bus_client); + return !(s->regs.mcr & FLEXCAN_MCR_NOT_RDY); +} + +/* --------- RX FIFO ---------- */ + +/** + * flexcan_fifo_pop() - Pop message from FIFO and update IRQs + * @s: FlexCAN device pointer + * + * Does not require the queue to be non-empty. + */ +static void flexcan_fifo_pop(FlexcanState *s) +{ + if (s->regs.fifo.mb_back.can_ctrl != 0) { + /* move queue elements forward */ + memmove(&s->regs.fifo.mb_back, &s->regs.fifo.mbs_queue[0], + sizeof(s->regs.fifo.mbs_queue)); + + /* clear the first-in slot */ + memset(&s->regs.mbs[FLEXCAN_FIFO_DEPTH - 1], 0, + sizeof(FlexcanRegsMessageBuffer)); + + trace_flexcan_fifo_pop(DEVICE(s)->canonical_path, 1, + s->regs.fifo.mb_back.can_ctrl != 0); + } else { + trace_flexcan_fifo_pop(DEVICE(s)->canonical_path, 0, 0); + } + + if (s->regs.fifo.mb_back.can_ctrl != 0) { + flexcan_irq_iflag_set(s, I_FIFO_AVAILABLE); + } else { + flexcan_irq_iflag_clear(s, I_FIFO_AVAILABLE); + } +} + +/** + * flexcan_fifo_find_free_slot() - Find the first free slot in the FIFO + * @s: FlexCAN device pointer + * + * Return: Pointer to the first free slot in the FIFO, + * or NULL if the queue is full. + */ +static FlexcanRegsMessageBuffer *flexcan_fifo_find_free_slot(FlexcanState *s) +{ + for (int i = 0; i < FLEXCAN_FIFO_DEPTH; i++) { + FlexcanRegsMessageBuffer *mb = &s->regs.mbs[i]; + if (mb->can_ctrl == 0) { + return mb; + } + } + return NULL; +} + +/** + * flexcan_fifo_push() - Update FIFO IRQs after frame move-in + * @s: FlexCAN device pointer + * @slot: Target FIFO slot + * + * The usage is as follows: + * 1. Get free slot pointer using flexcan_fifo_find_free_slot() + * 2. Move the frame in if not NULL + * 3. Call flexcan_fifo_push() regardless of the NULL pointer + */ +static void flexcan_fifo_push(FlexcanState *s, FlexcanRegsMessageBuffer *slot) +{ + if (slot) { + int n_occupied = slot - s->regs.mbs; + if (n_occupied == 4) { /* 4 means the 5th slot was filled in */ + /* + * fifo occupancy increased from 4 to 5, + * raising FIFO_WARN interrupt + */ + flexcan_irq_iflag_set(s, I_FIFO_WARN); + } + flexcan_irq_iflag_set(s, I_FIFO_AVAILABLE); + + trace_flexcan_fifo_push(DEVICE(s)->canonical_path, n_occupied); + } else { + flexcan_irq_iflag_set(s, I_FIFO_OVERFLOW); + + trace_flexcan_fifo_push(DEVICE(s)->canonical_path, -1); + } +} + +static enum FlexcanRx flexcan_fifo_rx(FlexcanState *s, + const qemu_can_frame *buf) +{ + /* todo: filtering. return FLEXCAN_FIFO_RX_RETRY if filtered out */ + if ((s->regs.mcr & FLEXCAN_MCR_IDAM_MASK) == FLEXCAN_MCR_IDAM_D) { + /* all frames rejected */ + return FLEXCAN_RX_SEARCH_RETRY; + } else { + /* push message to queue if not full */ + FlexcanRegsMessageBuffer *slot = flexcan_fifo_find_free_slot(s); + if (slot) { + flexcan_mb_move_in(s, buf, slot); + } + flexcan_fifo_push(s, slot); + + return slot ? FLEXCAN_RX_SEARCH_ACCEPT : FLEXCAN_RX_SEARCH_DROPPED; + } +} + +/* --------- RX message buffer ---------- */ + +/** + * flexcan_mb_rx_check_mb() - Check if a mb matches a received frame + * @s: FlexCAN device pointer + * @buf: Frame to be received from CAN subsystem + * @mbid: Target mailbox index. The mailbox must be a valid message buffer. + * + * Return: FLEXCAN_CHECK_MB_NIL if the message buffer does not match. + * FLEXCAN_CHECK_MB_MATCH if the message buffer matches the received + * frame and is free-to-receive, + * FLEXCAN_CHECK_MB_MATCH_LOCKED if the message buffer matches, + * but is locked, + * FLEXCAN_CHECK_MB_MATCH_NON_FREE if the message buffer matches, + * but is not free-to-receive + * for some other reason. + */ +static enum FlexcanCheck flexcan_mb_rx_check_mb(FlexcanState *s, + const qemu_can_frame *buf, + int mbid) +{ + FlexcanRegsMessageBuffer *mb = &s->regs.mbs[mbid]; + const bool is_rtr = !!(buf->can_id & QEMU_CAN_RTR_FLAG); + const bool is_serviced = !(mb->can_ctrl & FLEXCAN_MB_CNT_NOT_SRV); + const bool is_locked = s->locked_mbidx == mbid; + + bool is_free_to_receive = false; + bool is_matched = false; + + switch (mb->can_ctrl & FLEXCAN_MB_CODE_MASK) { + case FLEXCAN_MB_CODE_RX_RANSWER: + if (is_rtr && !(s->regs.ctrl2 & FLEXCAN_CTRL2_RRS)) { + /* todo: do the actual matching/filtering and RTR answer */ + is_matched = true; + } + break; + case FLEXCAN_MB_CODE_RX_FULL: + QEMU_FALLTHROUGH; + case FLEXCAN_MB_CODE_RX_OVERRUN: + is_free_to_receive = is_serviced; + /* todo: do the actual matching/filtering */ + is_matched = true; + break; + case FLEXCAN_MB_CODE_RX_EMPTY: + is_free_to_receive = true; + /* todo: do the actual matching/filtering */ + is_matched = true; + break; + default: + break; + } + + if (trace_event_get_state_backends(TRACE_FLEXCAN_MB_RX_CHECK_MB)) { + char code_str_buf[FLEXCAN_DBG_BUF_LEN] = { 0 }; + const char *code_str = flexcan_dbg_mb_code(mb->can_ctrl, code_str_buf); + trace_flexcan_mb_rx_check_mb(DEVICE(s)->canonical_path, mbid, code_str, + is_matched, is_free_to_receive, + is_serviced, is_locked); + } + + if (!is_matched) { + return FLEXCAN_CHECK_MB_NIL; + } + + if (is_locked) { + return FLEXCAN_CHECK_MB_MATCH_LOCKED; + } + + if (is_free_to_receive) { + return FLEXCAN_CHECK_MB_MATCH; + } + + return FLEXCAN_CHECK_MB_MATCH_NON_FREE; +} + +static enum FlexcanRx flexcan_mb_rx(FlexcanState *s, const qemu_can_frame *buf) +{ + int last_not_free_to_receive_mbid = -1; + bool last_not_free_to_receive_locked = false; + + FlexcanRegsMessageBuffer *first_mb = flexcan_get_first_message_buffer(s); + FlexcanRegsMessageBuffer *last_mb = flexcan_get_last_enabled_mailbox(s); + + for (FlexcanRegsMessageBuffer *mb = first_mb; + mb <= last_mb; mb++) { + int mbid = mb - s->regs.mbs; + enum FlexcanCheck r = flexcan_mb_rx_check_mb(s, buf, mbid); + if (r == FLEXCAN_CHECK_MB_MATCH) { + flexcan_mb_move_in(s, buf, mb); + flexcan_irq_iflag_set(s, mbid); + return FLEXCAN_RX_SEARCH_ACCEPT; + } + + if (r == FLEXCAN_CHECK_MB_MATCH_NON_FREE) { + last_not_free_to_receive_mbid = mbid; + last_not_free_to_receive_locked = false; + } else if (r == FLEXCAN_CHECK_MB_MATCH_LOCKED) { + /* + * message buffer is locked, + * we can move in the message after it's unlocked + */ + last_not_free_to_receive_mbid = mbid; + last_not_free_to_receive_locked = true; + } + } + + if (last_not_free_to_receive_mbid >= -1) { + if (last_not_free_to_receive_locked) { + /* + * copy to temporary mailbox (SMB) + * it will be moved in when the mailbox is unlocked + */ + s->regs.rx_smb0.can_ctrl = + s->regs.mbs[last_not_free_to_receive_mbid].can_id; + flexcan_mb_move_in(s, buf, &s->regs.rx_smb0); + s->smb_target_mbidx = last_not_free_to_receive_mbid; + return FLEXCAN_RX_SEARCH_ACCEPT; + } + + if (s->regs.mcr & FLEXCAN_MCR_IRMQ) { + flexcan_mb_move_in(s, buf, + &s->regs.mbs[last_not_free_to_receive_mbid]); + flexcan_irq_iflag_set(s, last_not_free_to_receive_mbid); + return FLEXCAN_RX_SEARCH_ACCEPT; + } + } + + return FLEXCAN_RX_SEARCH_RETRY; +} + +static ssize_t flexcan_receive(CanBusClientState *client, + const qemu_can_frame *frames, size_t frames_cnt) +{ + FlexcanState *s = container_of(client, FlexcanState, bus_client); + trace_flexcan_receive(DEVICE(s)->canonical_path, frames_cnt); + + if (frames_cnt == 0) { + return 0; + } + + /* clear the SMB, as it would be overriden in hardware */ + memset(&s->regs.rx_smb0, 0, sizeof(FlexcanRegsMessageBuffer)); + s->smb_target_mbidx = FLEXCAN_SMB_EMPTY; + + for (size_t i = 0; i < frames_cnt; i++) { + int r; + const qemu_can_frame *frame = &frames[i]; + if (frame->can_id & QEMU_CAN_ERR_FLAG) { + /* todo: error frame handling */ + continue; + } + if (frame->flags & QEMU_CAN_FRMF_TYPE_FD) { + /* CAN FD supported only in later FlexCAN version */ + continue; + } + + /* todo: this order logic is not complete and needs further work */ + if (s->regs.mcr & FLEXCAN_MCR_FEN && + s->regs.ctrl2 & FLEXCAN_CTRL2_MRP) { + r = flexcan_mb_rx(s, frame); + if (r == FLEXCAN_RX_SEARCH_RETRY) { + flexcan_fifo_rx(s, frame); + } + } else if (s->regs.mcr & FLEXCAN_MCR_FEN) { + r = flexcan_fifo_rx(s, frame); + if (r == FLEXCAN_RX_SEARCH_RETRY) { + flexcan_mb_rx(s, frame); + } + } else { + flexcan_mb_rx(s, frame); + } + } + + flexcan_irq_update(s); + return 1; +} + +/* ========== I/O handling ========== */ +static void flexcan_mem_write(void *opaque, hwaddr addr, uint64_t val, + unsigned size) +{ + FlexcanState *s = opaque; + uint32_t write_mask = ((const uint32_t *) + &flexcan_regs_write_mask)[addr / 4]; + uint32_t old_value = s->regs_raw[addr / 4]; + + /* + * 0 for bits that can "only be written in Freeze mode as it is blocked + * by hardware in other modes" + */ + const uint32_t freeze_mask_mcr = 0xDF54CC80; + const uint32_t freeze_mask_ctrl1 = 0x0000E740; + + flexcan_trace_mem_op(s, addr, val, size, true); + switch (addr) { + case offsetof(FlexcanRegs, mcr): + if (!(s->regs.mcr & FLEXCAN_MCR_FRZ_ACK)) { + write_mask &= freeze_mask_mcr; + } + s->regs.mcr = (val & write_mask) | (old_value & ~write_mask); + flexcan_set_mcr(s, old_value); + break; + case offsetof(FlexcanRegs, ctrl): + if (!(s->regs.mcr & FLEXCAN_MCR_FRZ_ACK)) { + write_mask &= freeze_mask_ctrl1; + } + s->regs.ctrl = (val & write_mask) | (old_value & ~write_mask); + break; + case offsetof(FlexcanRegs, iflag1): + s->regs.iflag1 &= ~val; + if ((s->regs.mcr & FLEXCAN_MCR_FEN) && + (val & FLEXCAN_IFLAG_RX_FIFO_AVAILABLE)) { + flexcan_fifo_pop(s); + } + break; + case offsetof(FlexcanRegs, iflag2): + s->regs.iflag2 &= ~val; + break; + case offsetof(FlexcanRegs, ctrl2): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, ecr): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, rxmgmask): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, rx14mask): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, rx15mask): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, rxfgmask): + QEMU_FALLTHROUGH; + case offsetof(FlexcanRegs, rximr[0]) ... offsetof(FlexcanRegs, rximr[63]): + /* these registers can only be written in freeze mode */ + if (!(s->regs.mcr & FLEXCAN_MCR_FRZ_ACK)) { + break; + } + QEMU_FALLTHROUGH; + default: + s->regs_raw[addr / 4] = (val & write_mask) | (old_value & ~write_mask); + + if (addr >= offsetof(FlexcanRegs, mb) && + addr < offsetof(FlexcanRegs, _reserved4)) { + /* access to mailbox */ + int mbid = (addr - offsetof(FlexcanRegs, mb)) / + sizeof(FlexcanRegsMessageBuffer); + + if (s->locked_mbidx == mbid) { + flexcan_mb_unlock(s); + } + + /* check for invalid writes into FIFO region */ + if (s->regs.mcr & FLEXCAN_MCR_FEN && mbid < FLEXCAN_FIFO_DEPTH) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Invalid write to Rx-FIFO structure", + DEVICE(s)->canonical_path); + return; + } + + /* run mailbox processing function on write to control word */ + if ((addr & 0xF) == 0) { + flexcan_mb_write(s, mbid); + } + } + break; + } + + flexcan_irq_update(s); +} + +static uint64_t flexcan_mem_read(void *opqaue, hwaddr addr, unsigned size) +{ + FlexcanState *s = opqaue; + uint32_t rv = s->regs_raw[addr >> 2]; + + if (addr >= offsetof(FlexcanRegs, mb) && + addr < offsetof(FlexcanRegs, _reserved4)) { + /* reading from mailbox */ + hwaddr offset = addr - offsetof(FlexcanRegs, mb); + int mbid = offset / sizeof(FlexcanRegsMessageBuffer); + + if (addr % 16 == 0 && s->locked_mbidx != mbid) { + /* reading control word locks the mailbox */ + flexcan_mb_unlock(s); + flexcan_mb_lock(s, mbid); + flexcan_irq_update(s); + rv = s->regs.mbs[mbid].can_ctrl & ~FLEXCAN_MB_CNT_NOT_SRV; + } + } else if (addr == offsetof(FlexcanRegs, timer)) { + flexcan_mb_unlock(s); + flexcan_irq_update(s); + rv = flexcan_get_timestamp(s, false); + } + + flexcan_trace_mem_op(s, addr, rv, size, false); + return rv; +} + +static bool flexcan_mem_accepts(void *opaque, hwaddr addr, + unsigned size, bool is_write, + MemTxAttrs attrs) +{ + FlexcanState *s = opaque; + + if ((s->regs.ctrl2 & FLEXCAN_CTRL2_WRMFRZ) && + (s->regs.mcr & FLEXCAN_MCR_FRZ_ACK)) { + /* unrestricted access to FlexCAN memory in freeze mode */ + return true; + } else if (attrs.user && (s->regs.mcr & FLEXCAN_MCR_SUPV)) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Invalid user-mode access to restricted register", + DEVICE(s)->canonical_path); + return false; + } else if (attrs.user && is_write && addr < 4) { + qemu_log_mask(LOG_GUEST_ERROR, + "%s: Invalid user-mode access to MCR", + DEVICE(s)->canonical_path); + return false; + } + + return true; +} + +static const struct MemoryRegionOps flexcan_ops = { + .read = flexcan_mem_read, + .write = flexcan_mem_write, + .endianness = DEVICE_LITTLE_ENDIAN, + .valid = { + .min_access_size = 1, + .max_access_size = 4, + .unaligned = true, + .accepts = flexcan_mem_accepts + }, + .impl = { + .min_access_size = 4, + .max_access_size = 4, + .unaligned = false + }, +}; + +static CanBusClientInfo flexcan_bus_client_info = { + .can_receive = flexcan_can_receive, + .receive = flexcan_receive, +}; + +static int flexcan_connect_to_bus(FlexcanState *s, CanBusState *bus) +{ + s->bus_client.info = &flexcan_bus_client_info; + + if (can_bus_insert_client(bus, &s->bus_client) < 0) { + return -1; + } + return 0; +} + +static void flexcan_init(Object *obj) +{ + FlexcanState *s = CAN_FLEXCAN(obj); + + memory_region_init_io( + &s->iomem, obj, &flexcan_ops, s, TYPE_CAN_FLEXCAN, + offsetof(FlexcanRegs, _reserved6) + ); +} + +static void flexcan_realize(DeviceState *dev, Error **errp) +{ + FlexcanState *s = CAN_FLEXCAN(dev); + + if (s->canbus) { + if (flexcan_connect_to_bus(s, s->canbus) < 0) { + error_setg(errp, "%s: flexcan_connect_to_bus failed", + dev->canonical_path); + return; + } + } + + sysbus_init_mmio(SYS_BUS_DEVICE(dev), &s->iomem); + sysbus_init_irq(SYS_BUS_DEVICE(SYS_BUS_DEVICE(dev)), &s->irq); +} + +static const VMStateDescription vmstate_can = { + .name = TYPE_CAN_FLEXCAN, + .version_id = 1, + .minimum_version_id = 1, + .fields = (const VMStateField[]) { + VMSTATE_INT64(timer_start, FlexcanState), + VMSTATE_UINT32_ARRAY(regs_raw, FlexcanState, sizeof(FlexcanRegs) / 4), + VMSTATE_INT32(locked_mbidx, FlexcanState), + VMSTATE_INT32(smb_target_mbidx, FlexcanState), + VMSTATE_END_OF_LIST(), + }, +}; + +static const Property flexcan_properties[] = { + DEFINE_PROP_LINK("canbus", FlexcanState, canbus, TYPE_CAN_BUS, + CanBusState *), +}; + +static void flexcan_class_init(ObjectClass *klass, const void *data) +{ + DeviceClass *dc = DEVICE_CLASS(klass); + ResettableClass *rc = RESETTABLE_CLASS(klass); + + rc->phases.enter = flexcan_reset_enter; + rc->phases.hold = flexcan_reset_hold; + dc->realize = flexcan_realize; + device_class_set_props(dc, flexcan_properties); + dc->vmsd = &vmstate_can; + dc->desc = "i.MX FLEXCAN Controller"; +} + +static const TypeInfo flexcan_info = { + .name = TYPE_CAN_FLEXCAN, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(FlexcanState), + .class_init = flexcan_class_init, + .instance_init = flexcan_init, +}; + +static void can_register_types(void) +{ + type_register_static(&flexcan_info); +} +type_init(can_register_types) diff --git a/hw/net/can/flexcan_regs.h b/hw/net/can/flexcan_regs.h new file mode 100644 index 0000000000..c18bc4736e --- /dev/null +++ b/hw/net/can/flexcan_regs.h @@ -0,0 +1,197 @@ +/* + * Field bitmasks and register structs definitions for FlexCAN + * + * This implementation is based on the following datasheet: + * i.MX 6Dual/6Quad Applications Processor Reference Manual + * Document Number: IMX6DQRM, Rev. 6, 05/2020 + * + * Copyright (c) 2025 Matyas Bobek <matyas.bobek@gmail.com> + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ +#include "qemu/bitops.h" + +#ifndef HW_CAN_FLEXCAN_REGS_H +#define HW_CAN_FLEXCAN_REGS_H + +#define FLEXCAN_GENMASK(h, l) (((~(uint32_t)0) >> (31 - (h) + (l))) << (l)) + +/** + * The following macros were originally written for the Linux kernel by: + * Andrey Volkov <andrey@volkov.fr> + * Sascha Hauer <s.hauer@pengutronix.de> + * Marc Kleine-Budde <mkl@pengutronix.de> + * David Jander <david@protonic.nl> + * and they have agreed to license them under GPL-2.0-or-later. + */ + +/* FLEXCAN module configuration register (CANMCR) bits */ +#define FLEXCAN_MCR_MDIS BIT(31) +#define FLEXCAN_MCR_FRZ BIT(30) +#define FLEXCAN_MCR_FEN BIT(29) +#define FLEXCAN_MCR_HALT BIT(28) +#define FLEXCAN_MCR_NOT_RDY BIT(27) +#define FLEXCAN_MCR_WAK_MSK BIT(26) +#define FLEXCAN_MCR_SOFTRST BIT(25) +#define FLEXCAN_MCR_FRZ_ACK BIT(24) +#define FLEXCAN_MCR_SUPV BIT(23) +#define FLEXCAN_MCR_SLF_WAK BIT(22) +#define FLEXCAN_MCR_WRN_EN BIT(21) +#define FLEXCAN_MCR_LPM_ACK BIT(20) +#define FLEXCAN_MCR_WAK_SRC BIT(19) +#define FLEXCAN_MCR_DOZE BIT(18) +#define FLEXCAN_MCR_SRX_DIS BIT(17) +#define FLEXCAN_MCR_IRMQ BIT(16) +#define FLEXCAN_MCR_LPRIO_EN BIT(13) +#define FLEXCAN_MCR_AEN BIT(12) +#define FLEXCAN_MCR_FDEN BIT(11) +#define FLEXCAN_MCR_MAXMB(x) ((x) & 0x7f) +#define FLEXCAN_MCR_IDAM_A (0x0 << 8) +#define FLEXCAN_MCR_IDAM_B (0x1 << 8) +#define FLEXCAN_MCR_IDAM_C (0x2 << 8) +#define FLEXCAN_MCR_IDAM_D (0x3 << 8) +#define FLEXCAN_MCR_IDAM_MASK (0x3 << 8) + +/* FLEXCAN control register (CANCTRL) bits */ +#define FLEXCAN_CTRL_PRESDIV(x) (((x) & 0xFF) << 24) +#define FLEXCAN_CTRL_PRESDIV_MASK FLEXCAN_CTRL_PRESDIV(UINT32_MAX) +#define FLEXCAN_CTRL_RJW(x) (((x) & 0x03) << 22) +#define FLEXCAN_CTRL_RJW_MASK FLEXCAN_CTRL_RJW(UINT32_MAX) +#define FLEXCAN_CTRL_PSEG1(x) (((x) & 0x07) << 19) +#define FLEXCAN_CTRL_PSEG1_MASK FLEXCAN_CTRL_PSEG1(UINT32_MAX) +#define FLEXCAN_CTRL_PSEG2(x) (((x) & 0x07) << 16) +#define FLEXCAN_CTRL_PSEG2_MASK FLEXCAN_CTRL_PSEG2(UINT32_MAX) +#define FLEXCAN_CTRL_BOFF_MSK BIT(15) +#define FLEXCAN_CTRL_ERR_MSK BIT(14) +#define FLEXCAN_CTRL_CLK_SRC BIT(13) +#define FLEXCAN_CTRL_LPB BIT(12) +#define FLEXCAN_CTRL_TWRN_MSK BIT(11) +#define FLEXCAN_CTRL_RWRN_MSK BIT(10) +#define FLEXCAN_CTRL_SMP BIT(7) +#define FLEXCAN_CTRL_BOFF_REC BIT(6) +#define FLEXCAN_CTRL_TSYN BIT(5) +#define FLEXCAN_CTRL_LBUF BIT(4) +#define FLEXCAN_CTRL_LOM BIT(3) +#define FLEXCAN_CTRL_PROPSEG(x) ((x) & 0x07) +#define FLEXCAN_CTRL_PROPSEG_MASK FLEXCAN_CTRL_PROPSEG(UINT32_MAX) +#define FLEXCAN_CTRL_ERR_BUS (FLEXCAN_CTRL_ERR_MSK) +#define FLEXCAN_CTRL_ERR_STATE \ + (FLEXCAN_CTRL_TWRN_MSK | FLEXCAN_CTRL_RWRN_MSK | \ + FLEXCAN_CTRL_BOFF_MSK) +#define FLEXCAN_CTRL_ERR_ALL \ + (FLEXCAN_CTRL_ERR_BUS | FLEXCAN_CTRL_ERR_STATE) + +/* FLEXCAN control register 2 (CTRL2) bits */ +#define FLEXCAN_CTRL2_ECRWRE BIT(29) +#define FLEXCAN_CTRL2_WRMFRZ BIT(28) +#define FLEXCAN_CTRL2_RFFN(x) (((x) & 0x0f) << 24) +#define FLEXCAN_CTRL2_TASD(x) (((x) & 0x1f) << 19) +#define FLEXCAN_CTRL2_MRP BIT(18) +#define FLEXCAN_CTRL2_RRS BIT(17) +#define FLEXCAN_CTRL2_EACEN BIT(16) +#define FLEXCAN_CTRL2_ISOCANFDEN BIT(12) + +/* FLEXCAN memory error control register (MECR) bits */ +#define FLEXCAN_MECR_ECRWRDIS BIT(31) +#define FLEXCAN_MECR_HANCEI_MSK BIT(19) +#define FLEXCAN_MECR_FANCEI_MSK BIT(18) +#define FLEXCAN_MECR_CEI_MSK BIT(16) +#define FLEXCAN_MECR_HAERRIE BIT(15) +#define FLEXCAN_MECR_FAERRIE BIT(14) +#define FLEXCAN_MECR_EXTERRIE BIT(13) +#define FLEXCAN_MECR_RERRDIS BIT(9) +#define FLEXCAN_MECR_ECCDIS BIT(8) +#define FLEXCAN_MECR_NCEFAFRZ BIT(7) + +/* FLEXCAN error and status register (ESR) bits */ +#define FLEXCAN_ESR_SYNCH BIT(18) +#define FLEXCAN_ESR_TWRN_INT BIT(17) +#define FLEXCAN_ESR_RWRN_INT BIT(16) +#define FLEXCAN_ESR_BIT1_ERR BIT(15) +#define FLEXCAN_ESR_BIT0_ERR BIT(14) +#define FLEXCAN_ESR_ACK_ERR BIT(13) +#define FLEXCAN_ESR_CRC_ERR BIT(12) +#define FLEXCAN_ESR_FRM_ERR BIT(11) +#define FLEXCAN_ESR_STF_ERR BIT(10) +#define FLEXCAN_ESR_TX_WRN BIT(9) +#define FLEXCAN_ESR_RX_WRN BIT(8) +#define FLEXCAN_ESR_IDLE BIT(7) +#define FLEXCAN_ESR_BOFF_INT BIT(2) +#define FLEXCAN_ESR_ERR_INT BIT(1) +#define FLEXCAN_ESR_WAK_INT BIT(0) + +/* FLEXCAN Bit Timing register (CBT) bits */ +#define FLEXCAN_CBT_BTF BIT(31) +#define FLEXCAN_CBT_EPRESDIV_MASK FLEXCAN_GENMASK(30, 21) +#define FLEXCAN_CBT_ERJW_MASK FLEXCAN_GENMASK(20, 16) +#define FLEXCAN_CBT_EPROPSEG_MASK FLEXCAN_GENMASK(15, 10) +#define FLEXCAN_CBT_EPSEG1_MASK FLEXCAN_GENMASK(9, 5) +#define FLEXCAN_CBT_EPSEG2_MASK FLEXCAN_GENMASK(4, 0) + +/* FLEXCAN FD control register (FDCTRL) bits */ +#define FLEXCAN_FDCTRL_FDRATE BIT(31) +#define FLEXCAN_FDCTRL_MBDSR1 FLEXCAN_GENMASK(20, 19) +#define FLEXCAN_FDCTRL_MBDSR0 FLEXCAN_GENMASK(17, 16) +#define FLEXCAN_FDCTRL_MBDSR_8 0x0 +#define FLEXCAN_FDCTRL_MBDSR_12 0x1 +#define FLEXCAN_FDCTRL_MBDSR_32 0x2 +#define FLEXCAN_FDCTRL_MBDSR_64 0x3 +#define FLEXCAN_FDCTRL_TDCEN BIT(15) +#define FLEXCAN_FDCTRL_TDCFAIL BIT(14) +#define FLEXCAN_FDCTRL_TDCOFF FLEXCAN_GENMASK(12, 8) +#define FLEXCAN_FDCTRL_TDCVAL FLEXCAN_GENMASK(5, 0) + +/* FLEXCAN FD Bit Timing register (FDCBT) bits */ +#define FLEXCAN_FDCBT_FPRESDIV_MASK FLEXCAN_GENMASK(29, 20) +#define FLEXCAN_FDCBT_FRJW_MASK FLEXCAN_GENMASK(18, 16) +#define FLEXCAN_FDCBT_FPROPSEG_MASK FLEXCAN_GENMASK(14, 10) +#define FLEXCAN_FDCBT_FPSEG1_MASK FLEXCAN_GENMASK(7, 5) +#define FLEXCAN_FDCBT_FPSEG2_MASK FLEXCAN_GENMASK(2, 0) + +/* FLEXCAN CRC Register (CRCR) bits */ +#define FLEXCAN_CRCR_MBCRC_MASK FLEXCAN_GENMASK(22, 16) +#define FLEXCAN_CRCR_MBCRC(x) (((x) & FLEXCAN_CRCR_MBCRC_MASK) << 16) +#define FLEXCAN_CRCR_TXCRC_MASK FLEXCAN_GENMASK(14, 0) +#define FLEXCAN_CRCR_TXCRC(x) ((x) & FLEXCAN_CRCR_TXCRC_MASK) + +/* FLEXCAN interrupt flag register (IFLAG) bits */ +/* Errata ERR005829 step7: Reserve first valid MB */ +#define I_FIFO_OVERFLOW 7 +#define I_FIFO_WARN 6 +#define I_FIFO_AVAILABLE 5 + +#define FLEXCAN_TX_MB_RESERVED_RX_FIFO 8 +#define FLEXCAN_TX_MB_RESERVED_RX_MAILBOX 0 +#define FLEXCAN_RX_MB_RX_MAILBOX_FIRST (FLEXCAN_TX_MB_RESERVED_RX_MAILBOX + 1) +#define FLEXCAN_IFLAG_MB(x) BIT_ULL(x) +#define FLEXCAN_IFLAG_RX_FIFO_OVERFLOW BIT(I_FIFO_OVERFLOW) +#define FLEXCAN_IFLAG_RX_FIFO_WARN BIT(I_FIFO_WARN) +#define FLEXCAN_IFLAG_RX_FIFO_AVAILABLE BIT(I_FIFO_AVAILABLE) + +/* FLEXCAN message buffers */ +#define FLEXCAN_MB_CODE_RX_BUSY_BIT (0x1 << 24) +#define FLEXCAN_MB_CODE_RX_INACTIVE (0x0 << 24) +#define FLEXCAN_MB_CODE_RX_EMPTY (0x4 << 24) +#define FLEXCAN_MB_CODE_RX_FULL (0x2 << 24) +#define FLEXCAN_MB_CODE_RX_OVERRUN (0x6 << 24) +#define FLEXCAN_MB_CODE_RX_RANSWER (0xa << 24) + +#define FLEXCAN_MB_CODE_TX_INACTIVE (0x8 << 24) +#define FLEXCAN_MB_CODE_TX_ABORT (0x9 << 24) +#define FLEXCAN_MB_CODE_TX_DATA (0xc << 24) +#define FLEXCAN_MB_CODE_TX_TANSWER (0xe << 24) + +#define FLEXCAN_MB_CODE(x) (((x) & 0xF) << 24) +#define FLEXCAN_MB_CODE_MASK FLEXCAN_MB_CODE(UINT32_MAX) + +#define FLEXCAN_MB_CNT_EDL BIT(31) +#define FLEXCAN_MB_CNT_BRS BIT(30) +#define FLEXCAN_MB_CNT_ESI BIT(29) +#define FLEXCAN_MB_CNT_SRR BIT(22) +#define FLEXCAN_MB_CNT_IDE BIT(21) +#define FLEXCAN_MB_CNT_RTR BIT(20) +#define FLEXCAN_MB_CNT_LENGTH(x) (((x) & 0xF) << 16) +#define FLEXCAN_MB_CNT_TIMESTAMP(x) ((x) & 0xFFFF) +#define FLEXCAN_MB_CNT_TIMESTAMP_MASK FLEXCAN_MB_CNT_TIMESTAMP(UINT32_MAX) + +#endif diff --git a/hw/net/can/meson.build b/hw/net/can/meson.build index 7382344628..401afde2e4 100644 --- a/hw/net/can/meson.build +++ b/hw/net/can/meson.build @@ -6,3 +6,4 @@ system_ss.add(when: 'CONFIG_CAN_CTUCANFD', if_true: files('ctucan_core.c')) system_ss.add(when: 'CONFIG_CAN_CTUCANFD_PCI', if_true: files('ctucan_pci.c')) system_ss.add(when: 'CONFIG_XLNX_ZYNQMP', if_true: files('xlnx-zynqmp-can.c')) system_ss.add(when: 'CONFIG_XLNX_VERSAL', if_true: files('xlnx-versal-canfd.c')) +system_ss.add(when: 'CONFIG_CAN_FLEXCAN', if_true: files('flexcan.c')) diff --git a/hw/net/can/trace-events b/hw/net/can/trace-events index de64ac1b31..7500f10d7a 100644 --- a/hw/net/can/trace-events +++ b/hw/net/can/trace-events @@ -1,3 +1,21 @@ +# flexcan.c +flexcan_irq_update(const char *inst, uint32_t mb_irqs1, uint32_t mb_irqs2, int setting) "%s: irqs1 0x%08x irqs2 0x%08x request %i" +flexcan_set_mcr(const char *inst, const char *enabled, const char *freeze, const char *fifo, const char *rdy, const char *sync) "%s: %s %s %s %s %s" +flexcan_mb_write(const char *inst, int mbidx, const char *code, int is_mailbox, uint32_t ctrl, uint32_t id) "%s: mbidx %i code %s is_mailbox %i ctrl 0x%08x id 0x%08x" +flexcan_mb_lock(const char *inst, int mbidx, int had_rx_code) "%s: mbidx %i had_rx_code %i" +flexcan_mb_unlock(const char *inst, int mbidx, const char *pending_frame) "%s: mbidx %i%s" +flexcan_fifo_pop(const char *inst, int non_empty_before, int non_empty_after) "%s: non_empty before %i non_empty_after %i" +flexcan_fifo_push(const char *inst, int n_occupied) "%s: n_slots_occupied %i" +flexcan_reset(const char *inst) "%s: resetting" +flexcan_mem_op(const char *inst, const char *op, uint32_t v, int offset, const char *reg_name, int size) "%s: %s 0x%08x at offset %i register %s size %i" +flexcan_get_timestamp(const char *inst, int64_t time_elapsed_ms, uint32_t bitrate, uint64_t cycles, uint32_t shift, uint32_t timestamp) "%s: time_elapsed %" PRIi64 "ms bitrate %ub/s cycles %" PRIu64 " shift %u timestamp 0x%04x" +flexcan_get_bitrate(const char *inst, uint32_t pe_freq, uint32_t prediv, uint32_t s_freq, uint32_t tseg1, uint32_t tseg2, uint32_t quata_per_bit, uint32_t bitrate) "%s: pe_freq %uHz prescaler %u s_freq %uHz tseg1 %uq tseg2 %uq total %uq/b bitrate %ub/s" +flexcan_timer_start(const char *inst, uint32_t bitrate, uint32_t value) "%s: bitrate %ub/s value 0x%04x" +flexcan_timer_stop(const char *inst, uint32_t bitrate, uint32_t value) "%s: bitrate %ub/s value 0x%04x" +flexcan_timer_overflow(const char *inst, int64_t current_time, int64_t timer_start, int64_t elapsed_ns) "%s: current_time %" PRIi64 "timer_start %" PRIi64 "elapsed_ns %" PRIi64 +flexcan_mb_rx_check_mb(const char *inst, int mbidx, const char *code, int is_matched, int is_ftr, int is_serviced, int is_locked) "%s: checking mb %i code %s is_matched %i is_free_to_receive %i is_serviced %i is_locked %i" +flexcan_receive(const char *inst, size_t n_frames) "%s: received %zu frames" + # xlnx-zynqmp-can.c xlnx_can_update_irq(uint32_t isr, uint32_t ier, uint32_t irq) "ISR: 0x%08x IER: 0x%08x IRQ: 0x%08x" xlnx_can_reset(uint32_t val) "Resetting controller with value = 0x%08x" diff --git a/hw/timer/imx_epit.c b/hw/timer/imx_epit.c index c67a39f10c..374ec7d324 100644 --- a/hw/timer/imx_epit.c +++ b/hw/timer/imx_epit.c @@ -20,18 +20,7 @@ #include "hw/misc/imx_ccm.h" #include "qemu/module.h" #include "qemu/log.h" - -#ifndef DEBUG_IMX_EPIT -#define DEBUG_IMX_EPIT 0 -#endif - -#define DPRINTF(fmt, args...) \ - do { \ - if (DEBUG_IMX_EPIT) { \ - fprintf(stderr, "[%s]%s: " fmt , TYPE_IMX_EPIT, \ - __func__, ##args); \ - } \ - } while (0) +#include "trace.h" static const char *imx_epit_reg_name(uint32_t reg) { @@ -80,7 +69,7 @@ static uint32_t imx_epit_get_freq(IMXEPITState *s) uint32_t prescaler = 1 + extract32(s->cr, CR_PRESCALE_SHIFT, CR_PRESCALE_BITS); uint32_t f_in = imx_ccm_get_clock_frequency(s->ccm, imx_epit_clocks[clksrc]); uint32_t freq = f_in / prescaler; - DPRINTF("ptimer frequency is %u\n", freq); + trace_imx_epit_get_freq(freq); return freq; } @@ -146,8 +135,7 @@ static uint64_t imx_epit_read(void *opaque, hwaddr offset, unsigned size) HWADDR_PRIx "\n", TYPE_IMX_EPIT, __func__, offset); break; } - - DPRINTF("(%s) = 0x%08x\n", imx_epit_reg_name(offset >> 2), reg_value); + trace_imx_epit_read(imx_epit_reg_name(offset >> 2), reg_value); return reg_value; } @@ -328,8 +316,7 @@ static void imx_epit_write(void *opaque, hwaddr offset, uint64_t value, { IMXEPITState *s = IMX_EPIT(opaque); - DPRINTF("(%s, value = 0x%08x)\n", imx_epit_reg_name(offset >> 2), - (uint32_t)value); + trace_imx_epit_write(imx_epit_reg_name(offset >> 2), value); switch (offset >> 2) { case 0: /* CR */ @@ -362,7 +349,7 @@ static void imx_epit_cmp(void *opaque) /* The cmp ptimer can't be running when the peripheral is disabled */ assert(s->cr & CR_EN); - DPRINTF("sr was %d\n", s->sr); + trace_imx_epit_cmp(s->sr); /* Set interrupt status bit SR.OCIF and update the interrupt state */ s->sr |= SR_OCIF; imx_epit_update_int(s); @@ -399,8 +386,6 @@ static void imx_epit_realize(DeviceState *dev, Error **errp) IMXEPITState *s = IMX_EPIT(dev); SysBusDevice *sbd = SYS_BUS_DEVICE(dev); - DPRINTF("\n"); - sysbus_init_irq(sbd, &s->irq); memory_region_init_io(&s->iomem, OBJECT(s), &imx_epit_ops, s, TYPE_IMX_EPIT, 0x00001000); diff --git a/hw/timer/trace-events b/hw/timer/trace-events index ac5afe84e8..634ba1da27 100644 --- a/hw/timer/trace-events +++ b/hw/timer/trace-events @@ -122,3 +122,9 @@ hpet_ram_write_tn_cmp(uint8_t reg_off) "hpet_ram_writel HPET_TN_CMP + %" PRIu8 hpet_ram_write_invalid_tn_cmp(void) "invalid HPET_TN_CMP + 4 write" hpet_ram_write_invalid(void) "invalid hpet_ram_writel" hpet_ram_write_counter_write_while_enabled(void) "Writing counter while HPET enabled!" + +# imx_epit.c +imx_epit_get_freq(uint32_t freq) "ptimer frequency is %u" +imx_epit_read(const char *name, uint32_t value) "(%s) = 0x%08x" +imx_epit_write(const char *name, uint64_t value) "(%s, value = 0x%08" PRIx64 ")" +imx_epit_cmp(uint32_t sr) "sr was %d" diff --git a/include/hw/arm/fsl-imx6.h b/include/hw/arm/fsl-imx6.h index cddd100dd6..376deea3a4 100644 --- a/include/hw/arm/fsl-imx6.h +++ b/include/hw/arm/fsl-imx6.h @@ -30,12 +30,14 @@ #include "hw/sd/sdhci.h" #include "hw/ssi/imx_spi.h" #include "hw/net/imx_fec.h" +#include "hw/net/flexcan.h" #include "hw/usb/chipidea.h" #include "hw/usb/imx-usb-phy.h" #include "hw/pci-host/designware.h" #include "hw/core/or-irq.h" #include "system/memory.h" #include "target/arm/cpu.h" +#include "net/can_emu.h" #include "qom/object.h" #define TYPE_FSL_IMX6 "fsl-imx6" @@ -51,6 +53,7 @@ OBJECT_DECLARE_SIMPLE_TYPE(FslIMX6State, FSL_IMX6) #define FSL_IMX6_NUM_WDTS 2 #define FSL_IMX6_NUM_USB_PHYS 2 #define FSL_IMX6_NUM_USBS 4 +#define FSL_IMX6_NUM_CANS 2 struct FslIMX6State { /*< private >*/ @@ -73,6 +76,7 @@ struct FslIMX6State { IMXUSBPHYState usbphy[FSL_IMX6_NUM_USB_PHYS]; ChipideaState usb[FSL_IMX6_NUM_USBS]; IMXFECState eth; + FlexcanState flexcan[FSL_IMX6_NUM_CANS]; DesignwarePCIEHost pcie; OrIRQState pcie4_msi_irq; MemoryRegion rom; @@ -80,6 +84,8 @@ struct FslIMX6State { MemoryRegion ocram; MemoryRegion ocram_alias; uint32_t phy_num; + + CanBusState *canbus[FSL_IMX6_NUM_CANS]; }; diff --git a/include/hw/intc/arm_gicv5_types.h b/include/hw/intc/arm_gicv5_types.h index de4f78a149..851286b1b5 100644 --- a/include/hw/intc/arm_gicv5_types.h +++ b/include/hw/intc/arm_gicv5_types.h @@ -105,6 +105,10 @@ typedef struct GICv5PendingIrq { uint8_t prio; } GICv5PendingIrq; +/* A GICv5PendingIrq struct initializer for "no pending interrupt" */ +#define GICV5_PENDING_IRQ_NONE \ + ((GICv5PendingIrq) { .intid = 0, .prio = PRIO_IDLE }) + /* Fields in a generic 32-bit INTID, per R_TJPHS */ FIELD(INTID, ID, 0, 24) FIELD(INTID, TYPE, 29, 3) diff --git a/include/hw/misc/imx6_ccm.h b/include/hw/misc/imx6_ccm.h index ccf46d7353..a54b940686 100644 --- a/include/hw/misc/imx6_ccm.h +++ b/include/hw/misc/imx6_ccm.h @@ -164,6 +164,10 @@ #define PERCLK_PODF_SHIFT (0) #define PERCLK_PODF_LENGTH (6) +/* CCM_CSCMR2 */ +#define CAN_CLK_PODF_SHIFT (2) +#define CAN_CLK_PODF_LENGTH (6) + /* CCM_ANALOG_PFD_528 */ #define PFD0_FRAC_SHIFT (0) #define PFD0_FRAC_LENGTH (6) diff --git a/include/hw/misc/imx_ccm.h b/include/hw/misc/imx_ccm.h index c4212d04ea..6839716ea3 100644 --- a/include/hw/misc/imx_ccm.h +++ b/include/hw/misc/imx_ccm.h @@ -46,6 +46,7 @@ typedef enum { CLK_EXT, CLK_HIGH_DIV, CLK_HIGH, + CLK_CAN, } IMXClk; struct IMXCCMClass { diff --git a/include/hw/net/flexcan.h b/include/hw/net/flexcan.h new file mode 100644 index 0000000000..153afc1e03 --- /dev/null +++ b/include/hw/net/flexcan.h @@ -0,0 +1,145 @@ +/* + * QEMU model of the NXP FLEXCAN device. + * + * Copyright (c) 2025 Matyas Bobek <matyas.bobek@gmail.com> + * + * Based on CTU CAN FD emulation implemented by Jan Charvat. + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef HW_CAN_FLEXCAN_H +#define HW_CAN_FLEXCAN_H + +#include "net/can_emu.h" +#include "qom/object.h" +#include "hw/misc/imx_ccm.h" + +#define FLEXCAN_FIFO_DEPTH 6 +#define FLEXCAN_MAILBOX_COUNT 64 + +/** + * Definitions of structs FlexcanRegs and FlexcanRegsMessageBuffer were + * originally written for the Linux kernel by: + * Andrey Volkov <andrey@volkov.fr> + * Sascha Hauer <s.hauer@pengutronix.de> + * Marc Kleine-Budde <mkl@pengutronix.de> + * David Jander <david@protonic.nl> + * and they have agreed to license them under GPL-2.0-or-later. + */ + +/* view of single message buffer registers */ +typedef struct FlexcanRegsMessageBuffer { + uint32_t can_ctrl; + uint32_t can_id; + uint32_t data[2]; +} FlexcanRegsMessageBuffer; + +/* RX FIFO view of message buffer registers */ +typedef struct FlexcanRegsRXFifo { + /* 6 message buffer deep queue, queue back first */ + FlexcanRegsMessageBuffer mb_back; + FlexcanRegsMessageBuffer mbs_queue[FLEXCAN_FIFO_DEPTH - 1]; + + /* number of filter elements active depends on ctrl2 | FLEXCAN_CTRL2_RFFN */ + uint32_t filter_table_els[128]; +} FlexcanRegsRXFifo; + +/* FlexCAN register in hw layout */ +typedef struct FlexcanRegs { + uint32_t mcr; /* 0x00 */ + uint32_t ctrl; /* 0x04 - not affected by soft reset */ + uint32_t timer; /* 0x08 */ + uint32_t tcr; /* 0x0C */ + uint32_t rxmgmask; /* 0x10 - not affected by soft reset */ + uint32_t rx14mask; /* 0x14 - not affected by soft reset */ + uint32_t rx15mask; /* 0x18 - not affected by soft reset */ + uint32_t ecr; /* 0x1C */ + uint32_t esr; /* 0x20 */ + uint32_t imask2; /* 0x24 */ + uint32_t imask1; /* 0x28 */ + uint32_t iflag2; /* 0x2C */ + uint32_t iflag1; /* 0x30 */ + union { /* 0x34 */ + uint32_t gfwr_mx28; /* MX28, MX53 */ + uint32_t ctrl2; /* MX6, VF610 - not affected by soft reset */ + }; + uint32_t esr2; /* 0x38 */ + uint32_t imeur; /* 0x3C, unused */ + uint32_t lrfr; /* 0x40, unused */ + uint32_t crcr; /* 0x44 */ + uint32_t rxfgmask; /* 0x48 */ + uint32_t rxfir; /* 0x4C - not affected by soft reset */ + uint32_t cbt; /* 0x50, unused - not affected by soft reset */ + uint32_t _reserved2; /* 0x54 */ + uint32_t dbg1; /* 0x58, unused */ + uint32_t dbg2; /* 0x5C, unused */ + uint32_t _reserved3[8]; /* 0x60 */ + union { /* 0x80 - not affected by soft reset */ + uint32_t mb[sizeof(FlexcanRegsMessageBuffer) * FLEXCAN_MAILBOX_COUNT]; + FlexcanRegsMessageBuffer mbs[FLEXCAN_MAILBOX_COUNT]; + FlexcanRegsRXFifo fifo; + }; + uint32_t _reserved4[256]; /* 0x480 */ + uint32_t rximr[64]; /* 0x880 - not affected by soft reset */ + uint32_t _reserved5[24]; /* 0x980 */ + uint32_t gfwr_mx6; /* 0x9E0 - MX6 */ + + /* the rest is unused except for SMB */ + uint32_t _reserved6[39]; /* 0x9E4 */ + uint32_t _rxfir[6]; /* 0xA80 */ + uint32_t _reserved8[2]; /* 0xA98 */ + uint32_t _rxmgmask; /* 0xAA0 */ + uint32_t _rxfgmask; /* 0xAA4 */ + uint32_t _rx14mask; /* 0xAA8 */ + uint32_t _rx15mask; /* 0xAAC */ + uint32_t tx_smb[4]; /* 0xAB0 */ + union { /* 0xAC0, used for SMB emulation */ + uint32_t rx_smb0_raw[4]; + FlexcanRegsMessageBuffer rx_smb0; + }; + uint32_t rx_smb1[4]; /* 0xAD0 */ + uint32_t mecr; /* 0xAE0 */ + uint32_t erriar; /* 0xAE4 */ + uint32_t erridpr; /* 0xAE8 */ + uint32_t errippr; /* 0xAEC */ + uint32_t rerrar; /* 0xAF0 */ + uint32_t rerrdr; /* 0xAF4 */ + uint32_t rerrsynr; /* 0xAF8 */ + uint32_t errsr; /* 0xAFC */ + uint32_t _reserved7[64]; /* 0xB00 */ + uint32_t fdctrl; /* 0xC00 - not affected by soft reset */ + uint32_t fdcbt; /* 0xC04 - not affected by soft reset */ + uint32_t fdcrc; /* 0xC08 */ + uint32_t _reserved9[199]; /* 0xC0C */ + uint32_t tx_smb_fd[18]; /* 0xF28 */ + uint32_t rx_smb0_fd[18]; /* 0xF70 */ + uint32_t rx_smb1_fd[18]; /* 0xFB8 */ +} FlexcanRegs; + +typedef struct FlexcanState { + SysBusDevice parent_obj; + + MemoryRegion iomem; + IMXCCMState *ccm; + qemu_irq irq; + + CanBusState *canbus; + CanBusClientState bus_client; + + union { + FlexcanRegs regs; + uint32_t regs_raw[sizeof(FlexcanRegs) / 4]; + }; + int64_t timer_start; + uint64_t last_rx_timer_cycles; + int32_t locked_mbidx; + int32_t smb_target_mbidx; + uint32_t timer_freq; +} FlexcanState; + +#define TYPE_CAN_FLEXCAN "flexcan" + +OBJECT_DECLARE_SIMPLE_TYPE(FlexcanState, CAN_FLEXCAN); + +#endif diff --git a/linux-user/aarch64/elfload.c b/linux-user/aarch64/elfload.c index 4887a3a7b7..42aeb29306 100644 --- a/linux-user/aarch64/elfload.c +++ b/linux-user/aarch64/elfload.c @@ -173,6 +173,7 @@ abi_ulong get_elf_hwcap(CPUState *cs) GET_FEATURE_ID(aa64_cmpbr, ARM_HWCAP_A64_CMPBR); GET_FEATURE_ID(aa64_f8mm8, ARM_HWCAP_A64_F8MM8); GET_FEATURE_ID(aa64_f8mm4, ARM_HWCAP_A64_F8MM4); + GET_FEATURE_ID(aa64_ssve_aes, ARM_HWCAP_A64_SME_AES); return hwcaps; } @@ -184,7 +185,7 @@ abi_ulong get_elf_hwcap2(CPUState *cs) GET_FEATURE_ID(aa64_dcpodp, ARM_HWCAP2_A64_DCPODP); GET_FEATURE_ID(aa64_sve2, ARM_HWCAP2_A64_SVE2); - GET_FEATURE_ID(aa64_sve2_aes, ARM_HWCAP2_A64_SVEAES); + GET_FEATURE_ID(aa64_sve_aes, ARM_HWCAP2_A64_SVEAES); GET_FEATURE_ID(aa64_sve2_pmull128, ARM_HWCAP2_A64_SVEPMULL); GET_FEATURE_ID(aa64_sve2_bitperm, ARM_HWCAP2_A64_SVEBITPERM); GET_FEATURE_ID(aa64_sve2_sha3, ARM_HWCAP2_A64_SVESHA3); @@ -234,6 +235,8 @@ abi_ulong get_elf_hwcap2(CPUState *cs) GET_FEATURE_ID(aa64_ssve_f8fma, ARM_HWCAP2_A64_SME_SF8FMA); GET_FEATURE_ID(aa64_ssve_f8dp4, ARM_HWCAP2_A64_SME_SF8DP4); GET_FEATURE_ID(aa64_ssve_f8dp2, ARM_HWCAP2_A64_SME_SF8DP2); + GET_FEATURE_ID(aa64_sme_f8f32, ARM_HWCAP2_A64_SME_F8F32); + GET_FEATURE_ID(aa64_sme_f8f16, ARM_HWCAP2_A64_SME_F8F16); return hwcaps; } diff --git a/target/arm/cpu-features.h b/target/arm/cpu-features.h index a80b251589..98102d75bf 100644 --- a/target/arm/cpu-features.h +++ b/target/arm/cpu-features.h @@ -1159,6 +1159,11 @@ static inline bool isar_feature_aa64_rme_gpc2(const ARMISARegisters *id) return FIELD_EX64_IDREG(id, ID_AA64PFR0, RME) >= 2; } +static inline bool isar_feature_aa64_rme_gpc3(const ARMISARegisters *id) +{ + return FIELD_EX64_IDREG(id, ID_AA64PFR0, RME) >= 3; +} + static inline bool isar_feature_aa64_dit(const ARMISARegisters *id) { return FIELD_EX64_IDREG(id, ID_AA64PFR0, DIT) != 0; @@ -1524,7 +1529,7 @@ static inline bool isar_feature_aa64_sve2p2(const ARMISARegisters *id) return FIELD_EX64_IDREG(id, ID_AA64ZFR0, SVEVER) >= 3; } -static inline bool isar_feature_aa64_sve2_aes(const ARMISARegisters *id) +static inline bool isar_feature_aa64_sve_aes(const ARMISARegisters *id) { return FIELD_EX64_IDREG(id, ID_AA64ZFR0, AES) != 0; } @@ -1575,6 +1580,11 @@ static inline bool isar_feature_aa64_sve_b16b16(const ARMISARegisters *id) return FIELD_EX64_IDREG(id, ID_AA64ZFR0, B16B16); } +static inline bool isar_feature_aa64_ssve_aes(const ARMISARegisters *id) +{ + return FIELD_EX64_IDREG(id, ID_AA64SMFR0, AES); +} + static inline bool isar_feature_aa64_ssve_f8fma(const ARMISARegisters *id) { return FIELD_EX64_IDREG(id, ID_AA64SMFR0, SF8FMA); @@ -1595,6 +1605,16 @@ static inline bool isar_feature_aa64_sme_b16b16(const ARMISARegisters *id) return FIELD_EX64_IDREG(id, ID_AA64SMFR0, B16B16); } +static inline bool isar_feature_aa64_sme_f8f32(const ARMISARegisters *id) +{ + return FIELD_EX64_IDREG(id, ID_AA64SMFR0, F8F32); +} + +static inline bool isar_feature_aa64_sme_f8f16(const ARMISARegisters *id) +{ + return FIELD_EX64_IDREG(id, ID_AA64SMFR0, F8F16); +} + static inline bool isar_feature_aa64_sme_f16f16(const ARMISARegisters *id) { return FIELD_EX64_IDREG(id, ID_AA64SMFR0, F16F16); @@ -1766,6 +1786,12 @@ isar_feature_aa64_sme2_or_sve2_lut(const ARMISARegisters *id) return isar_feature_aa64_sme2_or_sve2(id) && isar_feature_aa64_lut(id); } +static inline bool +isar_feature_aa64_sme_f16f16_or_f8f16(const ARMISARegisters *id) +{ + return isar_feature_aa64_sme_f16f16(id) || isar_feature_aa64_sme_f8f16(id); +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 86aae36ae5..128c78f6cf 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -421,6 +421,10 @@ static void arm_cpu_reset_hold(Object *obj, ResetType type) env->cp15.mdscr_el1 |= 1 << 12; /* Enable FEAT_MOPS */ env->cp15.sctlr_el[1] |= SCTLR_MSCEN; + /* Enable FEAT_FPMR */ + if (cpu_isar_feature(aa64_fpmr, cpu)) { + env->cp15.sctlr_el[1] |= SCTLR_EnFPM; + } /* For Linux, GCSPR_EL0 is always readable. */ if (cpu_isar_feature(aa64_gcs, cpu)) { env->cp15.gcscr_el[0] = GCSCRE0_NTR; diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 31a5567c95..84d33e87dc 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -559,6 +559,7 @@ typedef struct CPUArchState { /* RME registers */ uint64_t gpccr_el3; uint64_t gptbr_el3; + uint64_t gpcbw_el3; uint64_t mfar_el3; /* NV2 register */ @@ -1244,6 +1245,8 @@ void arm_v7m_cpu_do_interrupt(CPUState *cpu); typedef struct ARMGranuleProtectionConfig { /* GPCCR_EL3 */ uint64_t gpccr; + /* GPCBW_EL3 */ + uint64_t gpcbw; /* GPTBR_EL3 */ uint64_t gptbr; /* ID_AA64MMFR0_EL1.PARange */ @@ -2116,6 +2119,10 @@ FIELD(GPCCR, NA6, 27, 1) FIELD(GPCCR, NA7, 28, 1) FIELD(GPCCR, GPCBW, 29, 1) +FIELD(GPCBW, BWSIZE, 37, 2) +FIELD(GPCBW, BWSTRIDE, 32, 5) +FIELD(GPCBW, BWADDR, 0, 25) + FIELD(MFAR, FPA, 12, 40) FIELD(MFAR, NSE, 62, 1) FIELD(MFAR, NS, 63, 1) diff --git a/target/arm/helper.c b/target/arm/helper.c index a234aa031c..93e3d8b575 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -37,6 +37,9 @@ #include "qemu/plugin.h" static void switch_mode(CPUARMState *env, int mode); +#ifndef CONFIG_USER_ONLY +static void gt_recalc_timer(ARMCPU *cpu, int timeridx); +#endif int compare_u64(const void *a, const void *b) { @@ -821,6 +824,12 @@ static void scr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) changed = env->cp15.scr_el3 ^ value; env->cp15.scr_el3 = value; +#ifndef CONFIG_USER_ONLY + if (changed & SCR_ECVEN) { + gt_recalc_timer(cpu, GTIMER_PHYS); + } +#endif + /* * If SCR_EL3.{NS,NSE} changes, i.e. change of security state, * we must invalidate all TLBs below EL3. @@ -1404,10 +1413,10 @@ void gt_rme_post_el_change(ARMCPU *cpu, void *ignored) static uint64_t gt_phys_raw_cnt_offset(CPUARMState *env) { - if ((env->cp15.scr_el3 & SCR_ECVEN) && - FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, ECV) && - arm_is_el2_enabled(env) && - (arm_hcr_el2_eff(env) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) { + if ((!arm_feature(env, ARM_FEATURE_EL3) || (env->cp15.scr_el3 & SCR_ECVEN)) + && FIELD_EX64(env->cp15.cnthctl_el2, CNTHCTL, ECV) + && arm_is_el2_enabled(env) + && (arm_hcr_el2_eff(env) & (HCR_E2H | HCR_TGE)) != (HCR_E2H | HCR_TGE)) { return env->cp15.cntpoff_el2; } return 0; @@ -1802,9 +1811,13 @@ static void gt_cnthctl_write(CPUARMState *env, const ARMCPRegInfo *ri, if ((oldval ^ value) & R_CNTHCTL_CNTVMASK_MASK) { gt_update_irq(cpu, GTIMER_VIRT); - } else if ((oldval ^ value) & R_CNTHCTL_CNTPMASK_MASK) { + } + if ((oldval ^ value) & R_CNTHCTL_CNTPMASK_MASK) { gt_update_irq(cpu, GTIMER_PHYS); } + if ((oldval ^ value) & R_CNTHCTL_ECV_MASK) { + gt_recalc_timer(cpu, GTIMER_PHYS); + } } static void gt_cntvoff_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3703,6 +3716,7 @@ static const ARMCPRegInfo v8_aa32_el1_reginfo[] = { static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask) { ARMCPU *cpu = env_archcpu(env); + bool hcr_change_timer; if (arm_feature(env, ARM_FEATURE_V8)) { valid_mask |= MAKE_64BIT_MASK(0, 34); /* ARMv8.0 */ @@ -3794,6 +3808,10 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask) (HCR_VM | HCR_PTW | HCR_DC | HCR_DCT | HCR_FWB | HCR_NV | HCR_NV1)) { tlb_flush(CPU(cpu)); } + hcr_change_timer = (env->cp15.hcr_el2 ^ value) & + (HCR_E2H | HCR_TGE); + + /* update */ env->cp15.hcr_el2 = value; /* @@ -3815,6 +3833,11 @@ static void do_hcr_write(CPUARMState *env, uint64_t value, uint64_t valid_mask) arm_cpu_update_vinmi(cpu); arm_cpu_update_vfnmi(cpu); } + if (hcr_change_timer) { +#ifndef CONFIG_USER_ONLY + gt_recalc_timer(cpu, GTIMER_PHYS); +#endif + } } static void hcr_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) @@ -5001,6 +5024,10 @@ static void gpccr_write(CPUARMState *env, const ARMCPRegInfo *ri, R_GPCCR_SPAD_MASK | R_GPCCR_NSPAD_MASK | R_GPCCR_RLPAD_MASK; } + if (cpu_isar_feature(aa64_rme_gpc3, env_archcpu(env))) { + rw_mask |= R_GPCCR_GPCBW_MASK; + } + env->cp15.gpccr_el3 = (value & rw_mask) | (env->cp15.gpccr_el3 & ~rw_mask); } @@ -5010,11 +5037,26 @@ static void gpccr_reset(CPUARMState *env, const ARMCPRegInfo *ri) env_archcpu(env)->reset_l0gptsz); } +static void gpcbw_write(CPUARMState *env, const ARMCPRegInfo *ri, + uint64_t value) +{ + uint64_t rw_mask = R_GPCBW_BWADDR_MASK | R_GPCBW_BWSTRIDE_MASK | + R_GPCBW_BWSIZE_MASK; + ARMCPU *cpu = env_archcpu(env); + + tlb_flush(CPU(cpu)); + env->cp15.gpcbw_el3 = (value & rw_mask); +} + static const ARMCPRegInfo rme_reginfo[] = { { .name = "GPCCR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 2, .crm = 1, .opc2 = 6, .access = PL3_RW, .writefn = gpccr_write, .resetfn = gpccr_reset, .fieldoffset = offsetof(CPUARMState, cp15.gpccr_el3) }, + { .name = "GPCBW_EL3", .state = ARM_CP_STATE_AA64, + .opc0 = 3, .opc1 = 6, .crn = 2, .crm = 1, .opc2 = 5, + .access = PL3_RW, .writefn = gpcbw_write, + .fieldoffset = offsetof(CPUARMState, cp15.gpcbw_el3) }, { .name = "GPTBR_EL3", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 6, .crn = 2, .crm = 1, .opc2 = 4, .access = PL3_RW, .fieldoffset = offsetof(CPUARMState, cp15.gptbr_el3) }, diff --git a/target/arm/ptw.c b/target/arm/ptw.c index 1470de3010..99295954a1 100644 --- a/target/arm/ptw.c +++ b/target/arm/ptw.c @@ -343,11 +343,21 @@ bool arm_granule_protection_check(ARMGranuleProtectionConfig config, .space = ARMSS_Root, }; const uint64_t gpccr = config.gpccr; + const uint64_t gpcbw = config.gpcbw; unsigned pps, pgs, l0gptsz, level = 0; uint64_t tableaddr, pps_mask, align, entry, index; MemTxResult result; int gpi; + const uint64_t BW_ADDR_SHIFT = 30; + const uint64_t BW_SIZE_SHIFT = 30; + const uint64_t BW_STRIDE_SHIFT = 40; + + uint64_t bw_size_field = FIELD_EX64(gpcbw, GPCBW, BWSIZE); + uint64_t bw_stride_field = FIELD_EX64(gpcbw, GPCBW, BWSTRIDE); + uint64_t bw_addr = FIELD_EX64(gpcbw, GPCBW, BWADDR) << BW_ADDR_SHIFT; + uint64_t bw_mask = 0; + /* * We assume Granule Protection Check is enabled when * calling this function (GPCCR.GPC == 1). @@ -399,6 +409,58 @@ bool arm_granule_protection_check(ARMGranuleProtectionConfig config, goto fault_walk; } + /* At this point, GPCCR_EL3 is valid */ + + /* + * GPC Priority 1 (R_GMGRR): + * If GPCCR_EL3.GPCBW is 1 and the configuration GPCBW + * is invalid, the access fails as GPT walk fault at level 0. + */ + if (FIELD_EX64(gpccr, GPCCR, GPCBW)) { + uint64_t bw_size = 0; + uint64_t bw_stride = 0; + + /* BWSIZE, BWSTRIDE have a limited number of acceptable values. */ + switch (bw_size_field) { + case 0b000: + case 0b001: + case 0b010: + case 0b100: + case 0b110: + bw_size = 1ULL << (bw_size_field + BW_SIZE_SHIFT); + break; + default: /* Reserved value */ + goto fault_walk; + } + switch (bw_stride_field) { + case 0b00000: + case 0b00010: + case 0b00100: + case 0b00110: + case 0b00111: + case 0b01000: + case 0b01001: + case 0b01010: + case 0b10000: + bw_stride = 1ULL << (bw_stride_field + BW_STRIDE_SHIFT); + break; + default: /* Reserved value */ + goto fault_walk; + } + /* + * GPCBW is invalid if the base address is: + * not aligned to the size programmed in BWSIZE, or + * greater than or equal to the stride value configured by BWSTRIDE. + * We can make bw_mask which marks exactly which bits in bw_addr may + * be set (gpcbwu:gpcbwl). + */ + bw_mask = bw_stride - bw_size; + + if (bw_addr & ~bw_mask) { + goto fault_walk; + } + } + /* Note this field is read-only and fixed at reset. */ l0gptsz = 30 + FIELD_EX64(gpccr, GPCCR, L0GPTSZ); @@ -433,6 +495,20 @@ bool arm_granule_protection_check(ARMGranuleProtectionConfig config, goto fault_fail; } + /* + * Bypass window check. + * I_JJLRM: Granule Protection Table (GPT) lookups can be skipped + * in portions of the memory map by using GPC bypass windows. + * I_XNHTX: The GPC bypass window check (...) is performed + * immediately after priority 3. + * bw_mask from earlier makes this check for us. + */ + if (FIELD_EX64(gpccr, GPCCR, GPCBW)) { + if ((paddress & bw_mask) == bw_addr) { + return true; + } + } + /* GPC Priority 4: the base address of GPTBR_EL3 exceeds PPS. */ tableaddr = config.gptbr << 12; if (tableaddr & ~pps_mask) { @@ -3879,6 +3955,7 @@ static bool get_phys_addr_gpc(CPUARMState *env, S1Translate *ptw, }; struct ARMGranuleProtectionConfig config = { .gpccr = env->cp15.gpccr_el3, + .gpcbw = env->cp15.gpcbw_el3, .gptbr = env->cp15.gptbr_el3, .parange = FIELD_EX64_IDREG(&cpu->isar, ID_AA64MMFR0, PARANGE), .support_sel2 = cpu_isar_feature(aa64_sel2, cpu), diff --git a/target/arm/tcg/cpu64.c b/target/arm/tcg/cpu64.c index f7a920a202..ae3952a1ed 100644 --- a/target/arm/tcg/cpu64.c +++ b/target/arm/tcg/cpu64.c @@ -159,8 +159,8 @@ static void cpu_arm_set_rme(Object *obj, bool value, Error **errp) { ARMCPU *cpu = ARM_CPU(obj); - /* Enable FEAT_RME_GPC2 */ - FIELD_DP64_IDREG(&cpu->isar, ID_AA64PFR0, RME, value ? 2 : 0); + /* Enable FEAT_RME_GPC3 */ + FIELD_DP64_IDREG(&cpu->isar, ID_AA64PFR0, RME, value ? 3 : 0); } static void cpu_max_set_l0gptsz(Object *obj, Visitor *v, const char *name, @@ -1383,6 +1383,7 @@ void aarch64_max_tcg_initfn(Object *obj) SET_IDREG(isar, ID_AA64DFR0, t); t = GET_IDREG(isar, ID_AA64SMFR0); + t = FIELD_DP64(t, ID_AA64SMFR0, AES, 1); /* FEAT_SSVE_AES */ t = FIELD_DP64(t, ID_AA64SMFR0, SF8DP2, 1); /* FEAT_SSVE_FP8DOT2 */ t = FIELD_DP64(t, ID_AA64SMFR0, SF8DP4, 1); /* FEAT_SSVE_FP8DOT4 */ t = FIELD_DP64(t, ID_AA64SMFR0, SF8FMA, 1); /* FEAT_SSVE_FP8FMA */ @@ -1391,6 +1392,8 @@ void aarch64_max_tcg_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64SMFR0, B16F32, 1); /* FEAT_SME */ t = FIELD_DP64(t, ID_AA64SMFR0, F16F32, 1); /* FEAT_SME */ t = FIELD_DP64(t, ID_AA64SMFR0, I8I32, 0xf); /* FEAT_SME */ + t = FIELD_DP64(t, ID_AA64SMFR0, F8F32, 1); /* FEAT_SME_F8F32 */ + t = FIELD_DP64(t, ID_AA64SMFR0, F8F16, 1); /* FEAT_SME_F8F16 */ t = FIELD_DP64(t, ID_AA64SMFR0, F16F16, 1); /* FEAT_SME_F16F16 */ t = FIELD_DP64(t, ID_AA64SMFR0, B16B16, 1); /* FEAT_SME_B16B16 */ t = FIELD_DP64(t, ID_AA64SMFR0, I16I32, 5); /* FEAT_SME2 */ diff --git a/target/arm/tcg/fp8_helper.c b/target/arm/tcg/fp8_helper.c index bd0877071b..902e4b0ae9 100644 --- a/target/arm/tcg/fp8_helper.c +++ b/target/arm/tcg/fp8_helper.c @@ -857,3 +857,119 @@ void HELPER(gvec_fmmla_hb)(void *vd, void *vn, void *vm, clear_tail(vd, oprsz, simd_maxsz(desc)); } + +void HELPER(sme_fmopa_sb)(void *vza, void *vzn, void *vzm, void *vpn, + void *vpm, CPUARMState *env, uint32_t desc) +{ + FP8MulContext ctx = fp8_mul_start(env, -1); + intptr_t oprsz = simd_maxsz(desc); + uint16_t *pn = vpn, *pm = vpm; + + for (intptr_t row = 0; row < oprsz; ) { + uint16_t prow = pn[H2(row >> 4)]; + do { + void *vza_row = vza + tile_vslice_offset(row); + uint32_t n = *(uint32_t *)(vzn + H1_4(row)); + + n &= expand_pred_b(prow & 0xf); + + for (intptr_t col = 0; col < oprsz; ) { + uint16_t pcol = pm[H2(col >> 4)]; + do { + if (prow & pcol & 0xf) { + uint32_t *a = vza_row + H1_4(col); + uint32_t m = *(uint32_t *)(vzm + H1_4(col)); + + m &= expand_pred_b(pcol & 0xf); + *a = f8dotadd_s(n, m, 4, *a, &ctx); + } + col += 4; + pcol >>= 4; + } while (col & 15); + } + row += 4; + prow >>= 4; + } while (row & 15); + } +} + +void HELPER(sme_fmopa_hb)(void *vza, void *vzn, void *vzm, void *vpn, + void *vpm, CPUARMState *env, uint32_t desc) +{ + FP8MulContext ctx = fp8_mul_start(env, 0xf); + intptr_t oprsz = simd_maxsz(desc); + uint16_t *pn = vpn, *pm = vpm; + + for (intptr_t row = 0; row < oprsz; ) { + uint16_t prow = pn[H2(row >> 4)]; + do { + void *vza_row = vza + tile_vslice_offset(row); + uint16_t n = *(uint16_t *)(vzn + H1_2(row)); + + n &= expand_pred_b(prow & 3); + + for (intptr_t col = 0; col < oprsz; ) { + uint16_t pcol = pm[H2(col >> 4)]; + do { + if (prow & pcol & 0x3) { + uint16_t *a = vza_row + H1_2(col); + uint16_t m = *(uint16_t *)(vzm + H1_2(col)); + + m &= expand_pred_b(pcol & 3); + *a = f8dotadd_h(n, m, 2, *a, &ctx); + } + col += 2; + pcol >>= 2; + } while (col & 15); + } + row += 2; + prow >>= 2; + } while (row & 15); + } +} + +void HELPER(sme_fvdot_idx_sb)(void *vd, void *vn, void *vm, + CPUARMState *env, uint32_t desc) +{ + FP8MulContext ctx = fp8_mul_start(env, -1); + intptr_t oprsz = simd_maxsz(desc); + intptr_t elements = oprsz / sizeof(float32); + int idx_n = extract32(desc, SIMD_DATA_SHIFT, 2); + int idx_m = extract32(desc, SIMD_DATA_SHIFT + 2, 3); + float32 *d = vd; + uint8_t *n0 = vn; + uint8_t *n1 = vn + sizeof(ARMVectorReg); + uint16_t *m = vm; + intptr_t i = 0; + + do { + uint16_t mm = m[H2(2 * i + idx_m)]; + do { + uint16_t nn = n0[H1(4 * i + idx_n)] | (n1[H1(4 * i + idx_n)] << 8); + d[H4(i)] = f8dotadd_s(nn, mm, 2, d[H4(i)], &ctx); + } while (++i & 3); + } while (i < elements); +} + +void HELPER(sme_fvdot_idx_hb)(void *vd, void *vn, void *vm, + CPUARMState *env, uint32_t desc) +{ + FP8MulContext ctx = fp8_mul_start(env, 0xf); + intptr_t oprsz = simd_maxsz(desc); + intptr_t elements = oprsz / sizeof(float16); + int idx_n = extract32(desc, SIMD_DATA_SHIFT, 1); + int idx_m = extract32(desc, SIMD_DATA_SHIFT + 1, 3); + float16 *d = vd; + uint8_t *n0 = vn; + uint8_t *n1 = vn + sizeof(ARMVectorReg); + uint16_t *m = vm; + intptr_t i = 0; + + do { + uint16_t mm = m[H2(2 * i + idx_m)]; + do { + uint16_t nn = n0[H1(4 * i + idx_n)] | (n1[H1(4 * i + idx_n)] << 8); + d[H2(i)] = f8dotadd_h(nn, mm, 2, d[H2(i)], &ctx); + } while (++i & 7); + } while (i < elements); +} diff --git a/target/arm/tcg/gicv5-cpuif.c b/target/arm/tcg/gicv5-cpuif.c index dd2f696511..1cdd4103d0 100644 --- a/target/arm/tcg/gicv5-cpuif.c +++ b/target/arm/tcg/gicv5-cpuif.c @@ -4,6 +4,11 @@ * Copyright (c) 2025 Linaro Limited * * SPDX-License-Identifier: GPL-2.0-or-later + * + * The cpu interface is defined in IHI 111701 + * (ARM Generic Interrupt Controller Architecture Specification, + * GIC architecture version 5): + * https://developer.arm.com/documentation/111701/latest */ #include "qemu/osdep.h" @@ -139,7 +144,7 @@ static GICv5PendingIrq gic_hppi(CPUARMState *env, GICv5Domain domain) if (!(env->gicv5_cpuif.icc_cr0[domain] & R_ICC_CR0_EN_MASK)) { /* If cpuif is disabled there is no HPPI */ - return (GICv5PendingIrq) { .intid = 0, .prio = PRIO_IDLE }; + return GICV5_PENDING_IRQ_NONE; } irs_hppi = gicv5_get_hppi(gic, domain, env->gicv5_iaffid); @@ -163,7 +168,7 @@ static GICv5PendingIrq gic_hppi(CPUARMState *env, GICv5Domain domain) if (best.prio == PRIO_IDLE || best.prio > env->gicv5_cpuif.icc_pcr[domain] || best.prio >= gic_running_prio(env, domain)) { - return (GICv5PendingIrq) { .intid = 0, .prio = PRIO_IDLE }; + return GICV5_PENDING_IRQ_NONE; } return best; } @@ -253,8 +258,7 @@ static void gic_recalc_ppi_hppi(CPUARMState *env) * enabled, pending and not active. */ for (int i = 0; i < ARRAY_SIZE(env->gicv5_cpuif.ppi_hppi); i++) { - env->gicv5_cpuif.ppi_hppi[i].intid = 0; - env->gicv5_cpuif.ppi_hppi[i].prio = PRIO_IDLE; + env->gicv5_cpuif.ppi_hppi[i] = GICV5_PENDING_IRQ_NONE; }; for (int i = 0; i < ARRAY_SIZE(env->gicv5_cpuif.ppi_active); i++) { @@ -506,15 +510,6 @@ static void gic_icc_apr_el1_reset(CPUARMState *env, const ARMCPRegInfo *ri) } } -static uint64_t gic_icc_hapr_el1_read(CPUARMState *env, const ARMCPRegInfo *ri) -{ - /* - * ICC_HAPR_EL1 reports the current running priority, which can be - * calculated from the APR register. - */ - return gic_running_prio(env, gicv5_current_phys_domain(env)); -} - /* ICC_CR0_EL1 is also banked */ static uint64_t gic_icc_cr0_el1_read(CPUARMState *env, const ARMCPRegInfo *ri) { @@ -560,6 +555,7 @@ static void gic_icc_pcr_el1_write(CPUARMState *env, const ARMCPRegInfo *ri, value &= R_ICC_PCR_PRIORITY_MASK; env->gicv5_cpuif.icc_pcr[domain] = value; + gicv5_update_irq_fiq(env); } static void gic_icc_pcr_el1_reset(CPUARMState *env, const ARMCPRegInfo *ri) @@ -922,11 +918,6 @@ static const ARMCPRegInfo gicv5_cpuif_reginfo[] = { .writefn = gic_icc_pcr_el1_write, .resetfn = gic_icc_pcr_el1_reset, }, - { .name = "ICC_HAPR_EL1", .state = ARM_CP_STATE_AA64, - .opc0 = 3, .opc1 = 1, .crn = 12, .crm = 0, .opc2 = 3, - .access = PL1_R, .type = ARM_CP_IO | ARM_CP_NO_RAW, - .readfn = gic_icc_hapr_el1_read, .raw_writefn = arm_cp_write_ignore, - }, }; void define_gicv5_cpuif_regs(ARMCPU *cpu) @@ -944,7 +935,7 @@ void define_gicv5_cpuif_regs(ARMCPU *cpu) .name = name, .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 14 + (i >> 3), .opc2 = i & 7, - .access = PL1_RW, .type = ARM_CP_IO, + .access = PL1_RW, .type = ARM_CP_IO | ARM_CP_NO_RAW, .fieldoffset = offsetof(CPUARMState, gicv5_cpuif.ppi_priority[i]), .writefn = gic_ppi_priority_write, .raw_writefn = raw_write, }; diff --git a/target/arm/tcg/helper-fp8-defs.h b/target/arm/tcg/helper-fp8-defs.h index e942308af4..126dcadf77 100644 --- a/target/arm/tcg/helper-fp8-defs.h +++ b/target/arm/tcg/helper-fp8-defs.h @@ -38,3 +38,9 @@ DEF_HELPER_FLAGS_5(gvec_fdot_idx_hb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, DEF_HELPER_FLAGS_5(gvec_fmmla_sb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) DEF_HELPER_FLAGS_5(gvec_fmmla_hb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) + +DEF_HELPER_FLAGS_7(sme_fmopa_sb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_7(sme_fmopa_hb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, env, i32) + +DEF_HELPER_FLAGS_5(sme_fvdot_idx_sb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) +DEF_HELPER_FLAGS_5(sme_fvdot_idx_hb, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, env, i32) diff --git a/target/arm/tcg/helper-sve-defs.h b/target/arm/tcg/helper-sve-defs.h index de2254bb19..19bd936f6e 100644 --- a/target/arm/tcg/helper-sve-defs.h +++ b/target/arm/tcg/helper-sve-defs.h @@ -733,6 +733,8 @@ DEF_HELPER_FLAGS_4(sve_trn_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_trn_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_compact_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_compact_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) @@ -959,6 +961,9 @@ DEF_HELPER_FLAGS_4(sve_brkns, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_cntp, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) DEF_HELPER_FLAGS_2(sve2p1_cntp_c, TCG_CALL_NO_RWG_SE, i64, i32, i32) +DEF_HELPER_FLAGS_3(sve_firstp, TCG_CALL_NO_RWG_SE, i64, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve_lastp, TCG_CALL_NO_RWG_SE, i64, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_whilel, TCG_CALL_NO_RWG, i32, ptr, i32, i32) DEF_HELPER_FLAGS_3(sve_whileg, TCG_CALL_NO_RWG, i32, ptr, i32, i32) @@ -3203,3 +3208,8 @@ DEF_HELPER_FLAGS_6(sve2_famin_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, fpst, i32) DEF_HELPER_FLAGS_6(sve2_famin_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, fpst, i32) + +DEF_HELPER_FLAGS_4(sve_expand_b, TCG_CALL_NO_WG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_expand_h, TCG_CALL_NO_WG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_expand_s, TCG_CALL_NO_WG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve_expand_d, TCG_CALL_NO_WG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/tcg/sme.decode b/target/arm/tcg/sme.decode index 495330aed7..3a65e1ad4b 100644 --- a/target/arm/tcg/sme.decode +++ b/target/arm/tcg/sme.decode @@ -198,6 +198,9 @@ BFMOPA 10000001 101 ..... ... ... ..... . 100 . @op_16 BFMOPA_w 10000001 100 ..... ... ... ..... . 00 .. @op_32 FMOPA_w_h 10000001 101 ..... ... ... ..... . 00 .. @op_32 +FMOPA_sb 10000000 101 zm:5 pm:3 pn:3 zn:5 0 00 zad:2 &op sub=0 +FMOPA_hb 10000000 101 zm:5 pm:3 pn:3 zn:5 0100 zad:1 &op sub=0 + SMOPA_s 1010000 0 10 0 ..... ... ... ..... . 00 .. @op_32 SUMOPA_s 1010000 0 10 1 ..... ... ... ..... . 00 .. @op_32 USMOPA_s 1010000 1 10 0 ..... ... ... ..... . 00 .. @op_32 @@ -322,13 +325,13 @@ SUB_azz_n1_d 11000001 0111 .... 0 .. 110 ..... 11 ... @azz_nx1_o3 n=4 @azz_nx1_o2x2 ........ ... . zm:4 . .. ... zn:5 ... .. \ &azz_n off=%off2_x2 rv=%mova_rv -FMLAL_n1 11000001 001 0 .... 0 .. 011 ..... 00 ... @azz_nx1_o3x2 n=1 -FMLAL_n1 11000001 001 0 .... 0 .. 010 ..... 000 .. @azz_nx1_o2x2 n=2 -FMLAL_n1 11000001 001 1 .... 0 .. 010 ..... 000 .. @azz_nx1_o2x2 n=4 +FMLAL_n1_sh 11000001 001 0 .... 0 .. 011 ..... 00 ... @azz_nx1_o3x2 n=1 +FMLAL_n1_sh 11000001 001 0 .... 0 .. 010 ..... 000 .. @azz_nx1_o2x2 n=2 +FMLAL_n1_sh 11000001 001 1 .... 0 .. 010 ..... 000 .. @azz_nx1_o2x2 n=4 -FMLSL_n1 11000001 001 0 .... 0 .. 011 ..... 01 ... @azz_nx1_o3x2 n=1 -FMLSL_n1 11000001 001 0 .... 0 .. 010 ..... 010 .. @azz_nx1_o2x2 n=2 -FMLSL_n1 11000001 001 1 .... 0 .. 010 ..... 010 .. @azz_nx1_o2x2 n=4 +FMLSL_n1_sh 11000001 001 0 .... 0 .. 011 ..... 01 ... @azz_nx1_o3x2 n=1 +FMLSL_n1_sh 11000001 001 0 .... 0 .. 010 ..... 010 .. @azz_nx1_o2x2 n=2 +FMLSL_n1_sh 11000001 001 1 .... 0 .. 010 ..... 010 .. @azz_nx1_o2x2 n=4 BFMLAL_n1 11000001 001 0 .... 0 .. 011 ..... 10 ... @azz_nx1_o3x2 n=1 BFMLAL_n1 11000001 001 0 .... 0 .. 010 ..... 100 .. @azz_nx1_o2x2 n=2 @@ -443,6 +446,20 @@ FMLS_n1_h 11000001 001 1 .... 0 .. 111 ..... 01 ... @azz_nx1_o3 n=4 FMLS_n1_s 11000001 001 1 .... 0 .. 110 ..... 01 ... @azz_nx1_o3 n=4 FMLS_n1_d 11000001 011 1 .... 0 .. 110 ..... 01 ... @azz_nx1_o3 n=4 +FMLALL_n1_b 11000001 001 1 .... 0 .. 001 ..... 000 .. @azz_nx1_o2x4 n=1 +FMLALL_n1_b 11000001 001 0 .... 0 .. 000 ..... 0001 . @azz_nx1_o1x4 n=2 +FMLALL_n1_b 11000001 001 1 .... 0 .. 000 ..... 0001 . @azz_nx1_o1x4 n=4 + +FDOT_n1_sb 11000001 001 0 .... 0 .. 100 ..... 11 ... @azz_nx1_o3 n=2 +FDOT_n1_sb 11000001 001 1 .... 0 .. 100 ..... 11 ... @azz_nx1_o3 n=4 + +FMLAL_n1_hb 11000001 001 1 .... 0 .. 011 ..... 00 ... @azz_nx1_o3x2 n=1 +FMLAL_n1_hb 11000001 001 0 .... 0 .. 010 ..... 001 .. @azz_nx1_o2x2 n=2 +FMLAL_n1_hb 11000001 001 1 .... 0 .. 010 ..... 001 .. @azz_nx1_o2x2 n=4 + +FDOT_n1_hb 11000001 001 0 .... 0 .. 100 ..... 01 ... @azz_nx1_o3 n=2 +FDOT_n1_hb 11000001 001 1 .... 0 .. 100 ..... 01 ... @azz_nx1_o3 n=4 + ### SME2 Multi-vector Multiple Array Vectors %zn_ax2 6:4 !function=times_2 @@ -468,11 +485,11 @@ SUB_azz_nn_d 11000001 111 ...01 0 .. 110 ...00 11 ... @azz_4x4_o3 @azz_4x4_o2x2 ........ ... ..... . .. ... ..... ... .. \ &azz_n n=4 rv=%mova_rv zn=%zn_ax4 zm=%zm_ax4 off=%off2_x2 -FMLAL_nn 11000001 101 ....0 0 .. 010 ....0 000 .. @azz_2x2_o2x2 -FMLAL_nn 11000001 101 ...01 0 .. 010 ...00 000 .. @azz_4x4_o2x2 +FMLAL_nn_sh 11000001 101 ....0 0 .. 010 ....0 000 .. @azz_2x2_o2x2 +FMLAL_nn_sh 11000001 101 ...01 0 .. 010 ...00 000 .. @azz_4x4_o2x2 -FMLSL_nn 11000001 101 ....0 0 .. 010 ....0 010 .. @azz_2x2_o2x2 -FMLSL_nn 11000001 101 ...01 0 .. 010 ...00 010 .. @azz_4x4_o2x2 +FMLSL_nn_sh 11000001 101 ....0 0 .. 010 ....0 010 .. @azz_2x2_o2x2 +FMLSL_nn_sh 11000001 101 ...01 0 .. 010 ...00 010 .. @azz_4x4_o2x2 BFMLAL_nn 11000001 101 ....0 0 .. 010 ....0 100 .. @azz_2x2_o2x2 BFMLAL_nn 11000001 101 ...01 0 .. 010 ...00 100 .. @azz_4x4_o2x2 @@ -563,6 +580,18 @@ FMLS_nn_h 11000001 101 ...01 0 .. 100 ...00 11 ... @azz_4x4_o3 FMLS_nn_s 11000001 101 ...01 0 .. 110 ...00 01 ... @azz_4x4_o3 FMLS_nn_d 11000001 111 ...01 0 .. 110 ...00 01 ... @azz_4x4_o3 +FMLALL_nn_b 11000001 101 ....0 0 .. 000 ....1 0000 . @azz_2x2_o1x4 +FMLALL_nn_b 11000001 101 ...01 0 .. 000 ...01 0000 . @azz_4x4_o1x4 + +FDOT_nn_sb 11000001 101 ....0 0 .. 100 ....1 10 ... @azz_2x2_o3 +FDOT_nn_sb 11000001 101 ...01 0 .. 100 ...01 10 ... @azz_4x4_o3 + +FMLAL_nn_hb 11000001 101 ....0 0 .. 010 ....1 000 .. @azz_2x2_o2x2 +FMLAL_nn_hb 11000001 101 ...01 0 .. 010 ...01 000 .. @azz_4x4_o2x2 + +FDOT_nn_hb 11000001 101 ....0 0 .. 100 ....1 00 ... @azz_2x2_o3 +FDOT_nn_hb 11000001 101 ...01 0 .. 100 ...01 00 ... @azz_4x4_o3 + &az_n n off rv zm @az_2x2_o3 ........ ... ..... . .. ... ..... .. off:3 \ &az_n n=2 rv=%mova_rv zm=%zn_ax2 @@ -602,13 +631,13 @@ BFSUB_nn 11000001 111 00101 0 .. 111 ...00 01 ... @az_4x4_o3 @azx_4x1_o2x2 ........ .... zm:4 . .. . .. ..... .. ... \ &azx_n n=4 rv=%mova_rv off=%off2_x2 zn=%zn_ax4 idx=%idx2_10_2 -FMLAL_nx 11000001 1000 .... . .. 1 .. ..... 00 ... @azx_1x1_o3x2 -FMLAL_nx 11000001 1001 .... 0 .. 1 .. ....0 00 ... @azx_2x1_o2x2 -FMLAL_nx 11000001 1001 .... 1 .. 1 .. ...00 00 ... @azx_4x1_o2x2 +FMLAL_nx_sh 11000001 1000 .... . .. 1 .. ..... 00 ... @azx_1x1_o3x2 +FMLAL_nx_sh 11000001 1001 .... 0 .. 1 .. ....0 00 ... @azx_2x1_o2x2 +FMLAL_nx_sh 11000001 1001 .... 1 .. 1 .. ...00 00 ... @azx_4x1_o2x2 -FMLSL_nx 11000001 1000 .... . .. 1 .. ..... 01 ... @azx_1x1_o3x2 -FMLSL_nx 11000001 1001 .... 0 .. 1 .. ....0 01 ... @azx_2x1_o2x2 -FMLSL_nx 11000001 1001 .... 1 .. 1 .. ...00 01 ... @azx_4x1_o2x2 +FMLSL_nx_sh 11000001 1000 .... . .. 1 .. ..... 01 ... @azx_1x1_o3x2 +FMLSL_nx_sh 11000001 1001 .... 0 .. 1 .. ....0 01 ... @azx_2x1_o2x2 +FMLSL_nx_sh 11000001 1001 .... 1 .. 1 .. ...00 01 ... @azx_4x1_o2x2 BFMLAL_nx 11000001 1000 .... . .. 1 .. ..... 10 ... @azx_1x1_o3x2 BFMLAL_nx 11000001 1001 .... 0 .. 1 .. ....0 10 ... @azx_2x1_o2x2 @@ -633,7 +662,7 @@ FDOT_nx 11000001 0101 .... 1 .. 1 .. ...00 01 ... @azx_4x1_i2_o3 BFDOT_nx 11000001 0101 .... 0 .. 1 .. ....0 11 ... @azx_2x1_i2_o3 BFDOT_nx 11000001 0101 .... 1 .. 1 .. ...00 11 ... @azx_4x1_i2_o3 -FVDOT 11000001 0101 .... 0 .. 0 .. ....0 01 ... @azx_2x1_i2_o3 +FVDOT_sh 11000001 0101 .... 0 .. 0 .. ....0 01 ... @azx_2x1_i2_o3 BFVDOT 11000001 0101 .... 0 .. 0 .. ....0 11 ... @azx_2x1_i2_o3 SDOT_nx_2h 11000001 0101 .... 0 .. 1 .. ....0 00 ... @azx_2x1_i2_o3 @@ -762,6 +791,38 @@ FMLS_nx_h 11000001 0001 .... 1 .. 1.. ...00 1 .... @azx_4x1_i3_o3 FMLS_nx_s 11000001 0101 .... 1 .. 0.. ...00 10 ... @azx_4x1_i2_o3 FMLS_nx_d 11000001 1101 .... 1 .. 00. ...00 10 ... @azx_4x1_i1_o3 +FMLALL_nx_b 11000001 0100 .... . .. ... ..... 000 .. @azx_1x1_i4_o2 +FMLALL_nx_b 11000001 1001 .... 0 .. 0.. ....1 00.. . @azx_2x1_i4_o1 +FMLALL_nx_b 11000001 0001 .... 1 .. 0.. ...10 00.. . @azx_4x1_i4_o1 + +FDOT_nx_b 11000001 0101 .... 0 .. 0.. ....1 11 ... @azx_2x1_i2_o3 +FDOT_nx_b 11000001 0101 .... 1 .. 0.. ...00 01 ... @azx_4x1_i2_o3 + +%idx4_15_10_3 15:1 10:2 3:1 +%idx4_10_2 10:2 2:2 +@azx_1x1_i4_o3x2 ........ .... zm:4 . .. . .. zn:5 .. ... \ + &azx_n n=1 rv=%mova_rv off=%off3_x2 idx=%idx4_15_10_3 +@azx_2x2_i4_o3x2 ........ .... zm:4 . .. . .. .... .. .. .. \ + &azx_n n=2 rv=%mova_rv zn=%zn_ax2 off=%off2_x2 idx=%idx4_10_2 +@azx_4x4_i4_o3x2 ........ .... zm:4 . .. . .. ... ... .. .. \ + &azx_n n=4 rv=%mova_rv zn=%zn_ax4 off=%off2_x2 idx=%idx4_10_2 + +FMLAL_nx_hb 11000001 1100 .... . .. 0.. ..... 0. ... @azx_1x1_i4_o3x2 +FMLAL_nx_hb 11000001 1001 .... 0 .. 1.. ....1 1.. .. @azx_2x2_i4_o3x2 +FMLAL_nx_hb 11000001 1001 .... 1 .. 1.. ...01 0.. .. @azx_4x4_i4_o3x2 + +FDOT_nx_hb 11000001 1101 .... 0 .. 0.. ....1 0. ... @azx_2x1_i3_o3 +FDOT_nx_hb 11000001 0001 .... 1 .. 1.. ...10 0. ... @azx_4x1_i3_o3 + +%idx2_10_3 10:1 3:1 +@azx_4x2_i2_o3 ........ .... zm:4 . .. ... .... ... off:3 \ + &azx_n n=4 rv=%mova_rv zn=%zn_ax2 idx=%idx2_10_3 + +FVDOTB_sb 11000001 1101 .... 0 .. 01. ....0 0. ... @azx_4x2_i2_o3 +FVDOTT_sb 11000001 1101 .... 0 .. 01. ....0 1. ... @azx_4x2_i2_o3 + +FVDOT_hb 11000001 1101 .... 0 .. 1.. ....1 0. ... @azx_2x1_i3_o3 + ### SME2 Add / Sub array accumulators ADD_aaz_s 11000001 101 000000 .. 111 ....0 10 ... @az_2x2_o3 diff --git a/target/arm/tcg/sme_helper.c b/target/arm/tcg/sme_helper.c index 8e8e6c00ca..685b6b46be 100644 --- a/target/arm/tcg/sme_helper.c +++ b/target/arm/tcg/sme_helper.c @@ -64,39 +64,6 @@ void helper_sme_zero(CPUARMState *env, uint32_t imm, uint32_t svl) } } - -/* - * When considering the ZA storage as an array of elements of - * type T, the index within that array of the Nth element of - * a vertical slice of a tile can be calculated like this, - * regardless of the size of type T. This is because the tiles - * are interleaved, so if type T is size N bytes then row 1 of - * the tile is N rows away from row 0. The division by N to - * convert a byte offset into an array index and the multiplication - * by N to convert from vslice-index-within-the-tile to - * the index within the ZA storage cancel out. - */ -#define tile_vslice_index(i) ((i) * sizeof(ARMVectorReg)) - -/* - * When doing byte arithmetic on the ZA storage, the element - * byteoff bytes away in a tile vertical slice is always this - * many bytes away in the ZA storage, regardless of the - * size of the tile element, assuming that byteoff is a multiple - * of the element size. Again this is because of the interleaving - * of the tiles. For instance if we have 1 byte per element then - * each row of the ZA storage has one byte of the vslice data, - * and (counting from 0) byte 8 goes in row 8 of the storage - * at offset (8 * row-size-in-bytes). - * If we have 8 bytes per element then each row of the ZA storage - * has 8 bytes of the data, but there are 8 interleaved tiles and - * so byte 8 of the data goes into row 1 of the tile, - * which is again row 8 of the storage, so the offset is still - * (8 * row-size-in-bytes). Similarly for other element sizes. - */ -#define tile_vslice_offset(byteoff) ((byteoff) * sizeof(ARMVectorReg)) - - /* * Move Zreg vector to ZArray column. */ diff --git a/target/arm/tcg/sve.decode b/target/arm/tcg/sve.decode index 2795c2ec7f..bdc2b7f0c0 100644 --- a/target/arm/tcg/sve.decode +++ b/target/arm/tcg/sve.decode @@ -693,7 +693,6 @@ TBXQ 00000101 .. 1 ..... 001 101 ..... ..... @rd_rn_rm ### SVE Permute - Predicated Group # SVE compress active elements -# Note esz >= 2 COMPACT 00000101 .. 100001 100 ... ..... ..... @rd_pg_rn # SVE conditionally broadcast element to vector @@ -742,6 +741,8 @@ SPLICE 00000101 .. 101 100 100 ... ..... ..... @rdn_pg_rm # SVE2 vector splice (predicated, constructive) SPLICE_sve2 00000101 .. 101 101 100 ... ..... ..... @rd_pg_rn +EXPAND 00000101 .. 110 001 100 ... ..... ..... @rd_pg_rn + ### SVE Select Vectors Group # SVE select vector elements (predicated) @@ -851,6 +852,9 @@ BRKN 00100101 0. 01100001 .... 0 .... 0 .... @pd_pg_pn_s CNTP 00100101 .. 100 000 10 .... 0 .... ..... @rd_pg4_pn CNTP_c 00100101 esz:2 100 000 10 000 vl:1 1 rn:4 rd:5 +FIRSTP 00100101 .. 100 001 10 .... 0 .... ..... @rd_pg4_pn +LASTP 00100101 .. 100 010 10 .... 0 .... ..... @rd_pg4_pn + # SVE inc/dec register by predicate count INCDECP_r 00100101 .. 10110 d:1 10001 00 .... ..... @incdec_pred u=1 diff --git a/target/arm/tcg/sve_helper.c b/target/arm/tcg/sve_helper.c index a0a189eb1e..ac96b18784 100644 --- a/target/arm/tcg/sve_helper.c +++ b/target/arm/tcg/sve_helper.c @@ -917,12 +917,45 @@ DO_ZPZ(sve_ah_fneg_h, uint16_t, H1_2, DO_AH_FNEG_H) DO_ZPZ(sve_ah_fneg_s, uint32_t, H1_4, DO_AH_FNEG_S) DO_ZPZ_D(sve_ah_fneg_d, uint64_t, DO_AH_FNEG_D) -#define DO_NOT(N) (~N) +static inline void +sve_not_zpz(uint64_t *d, uint64_t *n, uint8_t *pg, uint32_t desc, + uint64_t (*expand)(uint8_t)) +{ + intptr_t opr_sz = simd_oprsz(desc) / 8; + bool zeroing = simd_data(desc) & 1; + + if (zeroing) { + for (intptr_t i = 0; i < opr_sz; ++i) { + uint64_t p = expand(pg[H1(i)]); + d[i] = ~n[i] & p; + } + } else { + for (intptr_t i = 0; i < opr_sz; ++i) { + uint64_t p = expand(pg[H1(i)]); + d[i] = (~n[i] & p) | (d[i] & ~p); + } + } +} + +void HELPER(sve_not_zpz_b)(void *vd, void *vn, void *pg, uint32_t desc) +{ + sve_not_zpz(vd, vn, pg, desc, expand_pred_b); +} + +void HELPER(sve_not_zpz_h)(void *vd, void *vn, void *pg, uint32_t desc) +{ + sve_not_zpz(vd, vn, pg, desc, expand_pred_h); +} + +void HELPER(sve_not_zpz_s)(void *vd, void *vn, void *pg, uint32_t desc) +{ + sve_not_zpz(vd, vn, pg, desc, expand_pred_s); +} -DO_ZPZ(sve_not_zpz_b, uint8_t, H1, DO_NOT) -DO_ZPZ(sve_not_zpz_h, uint16_t, H1_2, DO_NOT) -DO_ZPZ(sve_not_zpz_s, uint32_t, H1_4, DO_NOT) -DO_ZPZ_D(sve_not_zpz_d, uint64_t, DO_NOT) +void HELPER(sve_not_zpz_d)(void *vd, void *vn, void *pg, uint32_t desc) +{ + sve_not_zpz(vd, vn, pg, desc, expand_pred_d); +} #define DO_SXTB(N) ((int8_t)N) #define DO_SXTH(N) ((int16_t)N) @@ -3637,40 +3670,59 @@ DO_TRN(sve2_trn_q, Int128, ) #undef DO_UZP #undef DO_TRN -void HELPER(sve_compact_s)(void *vd, void *vn, void *vg, uint32_t desc) -{ - intptr_t i, j, opr_sz = simd_oprsz(desc) / 4; - uint32_t *d = vd, *n = vn; - uint8_t *pg = vg; - - for (i = j = 0; i < opr_sz; i++) { - if (pg[H1(i / 2)] & (i & 1 ? 0x10 : 0x01)) { - d[H4(j)] = n[H4(i)]; - j++; - } - } - for (; j < opr_sz; j++) { - d[H4(j)] = 0; - } +#define DO_COMPACT(NAME, TYPE, H) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t j = 0, oprsz = simd_oprsz(desc); \ + for (intptr_t i = 0; i < oprsz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + if (pg & 1) { \ + *(TYPE *)(vd + H(j)) = *(TYPE *)(vn + H(i)); \ + j += sizeof(TYPE); \ + } \ + i += sizeof(TYPE); \ + pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ + for (; j < oprsz; j += sizeof(TYPE)) { \ + *(TYPE *)(vd + H(j)) = 0; \ + } \ } -void HELPER(sve_compact_d)(void *vd, void *vn, void *vg, uint32_t desc) -{ - intptr_t i, j, opr_sz = simd_oprsz(desc) / 8; - uint64_t *d = vd, *n = vn; - uint8_t *pg = vg; +DO_COMPACT(sve_compact_b, uint8_t, H1) +DO_COMPACT(sve_compact_h, uint16_t, H1_2) +DO_COMPACT(sve_compact_s, uint32_t, H1_4) +DO_COMPACT(sve_compact_d, uint64_t, H1_8) - for (i = j = 0; i < opr_sz; i++) { - if (pg[H1(i)] & 1) { - d[j] = n[i]; - j++; - } - } - for (; j < opr_sz; j++) { - d[j] = 0; - } +#undef DO_COMPACT + +#define DO_EXPAND(NAME, TYPE, H) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ +{ \ + intptr_t oprsz = simd_oprsz(desc); \ + ARMVectorReg tmp_n = *(ARMVectorReg *)vn; \ + for (intptr_t i = 0, j = 0; i < oprsz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE nn = 0; \ + if (pg & 1) { \ + nn = *(TYPE *)((void *)&tmp_n + H(j)); \ + j += sizeof(TYPE); \ + } \ + *(TYPE *)(vd + H(i)) = nn; \ + i += sizeof(TYPE); \ + pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ } +DO_EXPAND(sve_expand_b, uint8_t, H1) +DO_EXPAND(sve_expand_h, uint16_t, H1_2) +DO_EXPAND(sve_expand_s, uint32_t, H1_4) +DO_EXPAND(sve_expand_d, uint64_t, H1_8) + +#undef DO_EXPAND /* Similar to the ARM LastActiveElement pseudocode function, except the * result is multiplied by the element size. This includes the not found * indication; e.g. not found for esz=3 is -8. @@ -4309,6 +4361,36 @@ uint64_t HELPER(sve2p1_cntp_c)(uint32_t png, uint32_t desc) return count >> p.lg2_stride; } +uint64_t HELPER(sve_firstp)(void *vn, void *vg, uint32_t pred_desc) +{ + intptr_t words = DIV_ROUND_UP(FIELD_EX32(pred_desc, PREDDESC, OPRSZ), 8); + intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); + uint64_t *n = vn, *g = vg, mask = pred_esz_masks[esz]; + + for (intptr_t i = 0; i < words; ++i) { + uint64_t t = n[i] & g[i] & mask; + if (t) { + return (i * 64 + ctz64(t)) >> esz; + } + } + return -1; +} + +uint64_t HELPER(sve_lastp)(void *vn, void *vg, uint32_t pred_desc) +{ + intptr_t words = DIV_ROUND_UP(FIELD_EX32(pred_desc, PREDDESC, OPRSZ), 8); + intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); + uint64_t *n = vn, *g = vg, mask = pred_esz_masks[esz]; + + for (intptr_t i = words - 1; i >= 0; --i) { + uint64_t t = n[i] & g[i] & mask; + if (t) { + return (i * 64 + (63 - clz64(t))) >> esz; + } + } + return -1; +} + /* C.f. Arm pseudocode EncodePredCount */ static uint64_t encode_pred_count(uint32_t elements, uint32_t count, uint32_t esz, bool invert) diff --git a/target/arm/tcg/translate-sme.c b/target/arm/tcg/translate-sme.c index 98d3d18791..ff5554eefb 100644 --- a/target/arm/tcg/translate-sme.c +++ b/target/arm/tcg/translate-sme.c @@ -616,6 +616,29 @@ TRANS_FEAT(BFMOPA_w, aa64_sme, do_outprod_env, a, MO_32, : !s->fpcr_ah ? gen_helper_sme_bfmops_w : gen_helper_sme_ah_bfmops_w) +static bool do_outprod_fp8(DisasContext *s, arg_op *a, MemOp esz, + gen_helper_gvec_5_ptr *fn) +{ + if (fpmr_access_check(s) && sme_smza_enabled_check(s)) { + int svl = streaming_vec_reg_size(s); + uint32_t desc = simd_desc(svl, svl, 0); + + TCGv_ptr za = get_tile(s, esz, a->zad); + TCGv_ptr zn = vec_full_reg_ptr(s, a->zn); + TCGv_ptr zm = vec_full_reg_ptr(s, a->zm); + TCGv_ptr pn = pred_full_reg_ptr(s, a->pn); + TCGv_ptr pm = pred_full_reg_ptr(s, a->pm); + + fn(za, zn, zm, pn, pm, tcg_env, tcg_constant_i32(desc)); + } + return true; +} + +TRANS_FEAT(FMOPA_sb, aa64_sme_f8f32, do_outprod_fp8, + a, MO_32, gen_helper_sme_fmopa_sb) +TRANS_FEAT(FMOPA_hb, aa64_sme_f8f16, do_outprod_fp8, + a, MO_16, gen_helper_sme_fmopa_hb) + TRANS_FEAT(SMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_smopa_s) TRANS_FEAT(UMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_umopa_s) TRANS_FEAT(SUMOPA_s, aa64_sme, do_outprod, a, MO_32, gen_helper_sme_sumopa_s) @@ -1040,6 +1063,47 @@ static bool do_azz_acc_fp(DisasContext *s, int nreg, int nsel, return true; } +static bool do_azz_acc_fp8(DisasContext *s, int nreg, int nsel, + int rv, int off, int zn, int zm, + int data, int shsel, bool multi, + gen_helper_gvec_3_ptr *fn) +{ + /* + * TODO: Could plausibly reuse do_azz_acc_fp, after the fpmr check, + * but the fp8 helpers were written without a separate addend operand. + */ + if (fpmr_access_check(s) && sme_smza_enabled_check(s)) { + int svl = streaming_vec_reg_size(s); + int vstride = svl / nreg; + TCGv_ptr t_za = get_zarray(s, rv, off, nreg, nsel); + TCGv_ptr t; + + t = tcg_temp_new_ptr(); + + for (int r = 0; r < nreg; ++r) { + TCGv_ptr t_zn = vec_full_reg_ptr(s, zn); + TCGv_ptr t_zm = vec_full_reg_ptr(s, zm); + + for (int i = 0; i < nsel; ++i) { + int o_za = (r * vstride + i) * sizeof(ARMVectorReg); + int desc = simd_desc(svl, svl, data | (i << shsel)); + + tcg_gen_addi_ptr(t, t_za, o_za); + fn(t, t_zn, t_zm, tcg_env, tcg_constant_i32(desc)); + } + + /* + * For multiple-and-single vectors, Zn may wrap. + * For multiple vectors, both Zn and Zm are aligned. + */ + zn = (zn + 1) % 32; + zm += multi; + } + } + return true; +} + + static bool do_fmlal(DisasContext *s, arg_azz_n *a, bool sub, bool multi) { return do_azz_acc_fp(s, a->n, 2, a->rv, a->off, a->zn, a->zm, @@ -1047,10 +1111,28 @@ static bool do_fmlal(DisasContext *s, arg_azz_n *a, bool sub, bool multi) multi, FPST_ENV, gen_helper_sve2_fmlal_zzzw_s); } -TRANS_FEAT(FMLAL_n1, aa64_sme2, do_fmlal, a, false, false) -TRANS_FEAT(FMLSL_n1, aa64_sme2, do_fmlal, a, true, false) -TRANS_FEAT(FMLAL_nn, aa64_sme2, do_fmlal, a, false, true) -TRANS_FEAT(FMLSL_nn, aa64_sme2, do_fmlal, a, true, true) +TRANS_FEAT(FMLAL_n1_sh, aa64_sme2, do_fmlal, a, false, false) +TRANS_FEAT(FMLSL_n1_sh, aa64_sme2, do_fmlal, a, true, false) +TRANS_FEAT(FMLAL_nn_sh, aa64_sme2, do_fmlal, a, false, true) +TRANS_FEAT(FMLSL_nn_sh, aa64_sme2, do_fmlal, a, true, true) + +static bool do_fmlall_fp8(DisasContext *s, arg_azz_n *a, bool multi) +{ + return do_azz_acc_fp8(s, a->n, 4, a->rv, a->off, a->zn, a->zm, + 0, 0, multi, gen_helper_gvec_fmla_sb); +} + +TRANS_FEAT(FMLALL_n1_b, aa64_sme_f8f32, do_fmlall_fp8, a, false) +TRANS_FEAT(FMLALL_nn_b, aa64_sme_f8f32, do_fmlall_fp8, a, true) + +static bool do_fmlal_fp8(DisasContext *s, arg_azz_n *a, bool multi) +{ + return do_azz_acc_fp8(s, a->n, 2, a->rv, a->off, a->zn, a->zm, + 0, 0, multi, gen_helper_gvec_fmla_hb); +} + +TRANS_FEAT(FMLAL_n1_hb, aa64_sme_f8f16, do_fmlal_fp8, a, false) +TRANS_FEAT(FMLAL_nn_hb, aa64_sme_f8f16, do_fmlal_fp8, a, true) static bool do_fmlal_nx(DisasContext *s, arg_azx_n *a, bool sub) { @@ -1059,8 +1141,12 @@ static bool do_fmlal_nx(DisasContext *s, arg_azx_n *a, bool sub) false, FPST_ENV, gen_helper_sve2_fmlal_zzxw_s); } -TRANS_FEAT(FMLAL_nx, aa64_sme2, do_fmlal_nx, a, false) -TRANS_FEAT(FMLSL_nx, aa64_sme2, do_fmlal_nx, a, true) +TRANS_FEAT(FMLAL_nx_sh, aa64_sme2, do_fmlal_nx, a, false) +TRANS_FEAT(FMLSL_nx_sh, aa64_sme2, do_fmlal_nx, a, true) + +TRANS_FEAT(FMLAL_nx_hb, aa64_sme_f8f16, do_azz_acc_fp8, + a->n, 2, a->rv, a->off, a->zn, a->zm, + a->idx << 2, 0, false, gen_helper_gvec_fmla_idx_hb) static bool do_bfmlal(DisasContext *s, arg_azz_n *a, bool sub, bool multi) { @@ -1088,6 +1174,18 @@ static bool do_bfmlal_nx(DisasContext *s, arg_azx_n *a, bool sub) TRANS_FEAT(BFMLAL_nx, aa64_sme2, do_bfmlal_nx, a, false) TRANS_FEAT(BFMLSL_nx, aa64_sme2, do_bfmlal_nx, a, true) +TRANS_FEAT(FMLALL_nx_b, aa64_sme_f8f32, do_azz_acc_fp8, + a->n, 4, a->rv, a->off, a->zn, a->zm, + a->idx << 2, 0, false, gen_helper_gvec_fmla_idx_sb) + +TRANS_FEAT(FDOT_nx_b, aa64_sme_f8f32, do_azz_acc_fp8, + a->n, 1, a->rv, a->off, a->zn, a->zm, + a->idx, 0, false, gen_helper_gvec_fdot_idx_sb) + +TRANS_FEAT(FDOT_nx_hb, aa64_sme_f8f16, do_azz_acc_fp8, + a->n, 1, a->rv, a->off, a->zn, a->zm, + a->idx, 0, false, gen_helper_gvec_fdot_idx_hb) + static bool do_fdot(DisasContext *s, arg_azz_n *a, bool multi) { return do_azz_acc_fp(s, a->n, 1, a->rv, a->off, a->zn, a->zm, 1, 0, @@ -1097,6 +1195,24 @@ static bool do_fdot(DisasContext *s, arg_azz_n *a, bool multi) TRANS_FEAT(FDOT_n1, aa64_sme2, do_fdot, a, false) TRANS_FEAT(FDOT_nn, aa64_sme2, do_fdot, a, true) +static bool do_fdot_fp8(DisasContext *s, arg_azz_n *a, bool multi) +{ + return do_azz_acc_fp8(s, a->n, 1, a->rv, a->off, a->zn, a->zm, + 0, 0, multi, gen_helper_gvec_fdot_sb); +} + +TRANS_FEAT(FDOT_n1_sb, aa64_sme_f8f32, do_fdot_fp8, a, false) +TRANS_FEAT(FDOT_nn_sb, aa64_sme_f8f32, do_fdot_fp8, a, true) + +static bool do_fdot_hb(DisasContext *s, arg_azz_n *a, bool multi) +{ + return do_azz_acc_fp8(s, a->n, 1, a->rv, a->off, a->zn, a->zm, + 0, 0, multi, gen_helper_gvec_fdot_hb); +} + +TRANS_FEAT(FDOT_n1_hb, aa64_sme_f8f16, do_fdot_hb, a, false) +TRANS_FEAT(FDOT_nn_hb, aa64_sme_f8f16, do_fdot_hb, a, true) + static bool do_fdot_nx(DisasContext *s, arg_azx_n *a) { return do_azz_acc_fp(s, a->n, 1, a->rv, a->off, a->zn, a->zm, @@ -1144,9 +1260,23 @@ static bool do_vdot(DisasContext *s, arg_azx_n *a, gen_helper_gvec_4_ptr *fn) return true; } -TRANS_FEAT(FVDOT, aa64_sme, do_vdot, a, gen_helper_sme2_fvdot_idx_h) +TRANS_FEAT(FVDOT_sh, aa64_sme, do_vdot, a, gen_helper_sme2_fvdot_idx_h) TRANS_FEAT(BFVDOT, aa64_sme, do_vdot, a, gen_helper_sme2_bfvdot_idx) +static bool do_fvdot_sb(DisasContext *s, arg_azx_n *a, bool top) +{ + return do_azz_acc_fp8(s, a->n, 1, a->rv, a->off, a->zn, a->zm, + (2 * a->idx + top) << 2, 0, false, + gen_helper_sme_fvdot_idx_sb); +} + +TRANS_FEAT(FVDOTB_sb, aa64_sme_f8f32, do_fvdot_sb, a, false) +TRANS_FEAT(FVDOTT_sb, aa64_sme_f8f32, do_fvdot_sb, a, true) + +TRANS_FEAT(FVDOT_hb, aa64_sme_f8f16, do_azz_acc_fp8, + a->n, 2, a->rv, a->off, a->zn, a->zm, + (a->idx << 1), 0, false, gen_helper_sme_fvdot_idx_hb) + static bool do_fmla(DisasContext *s, arg_azz_n *a, bool multi, ARMFPStatusFlavour fpst, gen_helper_gvec_3_ptr *fn) { @@ -1239,9 +1369,9 @@ static bool do_faddsub(DisasContext *s, arg_az_n *a, ARMFPStatusFlavour fpst, return true; } -TRANS_FEAT(FADD_nn_h, aa64_sme_f16f16, do_faddsub, a, +TRANS_FEAT(FADD_nn_h, aa64_sme_f16f16_or_f8f16, do_faddsub, a, FPST_ZA_F16, gen_helper_gvec_fadd_h) -TRANS_FEAT(FSUB_nn_h, aa64_sme_f16f16, do_faddsub, a, +TRANS_FEAT(FSUB_nn_h, aa64_sme_f16f16_or_f8f16, do_faddsub, a, FPST_ZA_F16, gen_helper_gvec_fsub_h) TRANS_FEAT(FADD_nn_s, aa64_sme2, do_faddsub, a, diff --git a/target/arm/tcg/translate-sve.c b/target/arm/tcg/translate-sve.c index 9a72e03513..79e8e16b24 100644 --- a/target/arm/tcg/translate-sve.c +++ b/target/arm/tcg/translate-sve.c @@ -2769,11 +2769,23 @@ TRANS_FEAT_NONSTREAMING(TRN2_q, aa64_sve_f64mm, do_interleave_q, *** SVE Permute Vector - Predicated Group */ -static gen_helper_gvec_3 * const compact_fns[4] = { - NULL, NULL, gen_helper_sve_compact_s, gen_helper_sve_compact_d -}; -TRANS_FEAT_NONSTREAMING(COMPACT, aa64_sve, gen_gvec_ool_arg_zpz, - compact_fns[a->esz], a, 0) +static bool trans_COMPACT(DisasContext *s, arg_COMPACT *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve_compact_b, gen_helper_sve_compact_h, + gen_helper_sve_compact_s, gen_helper_sve_compact_d + }; + + if (!dc_isar_feature(aa64_sme2p2, s)) { + if (!(a->esz >= MO_32 + ? dc_isar_feature(aa64_sve, s) + : dc_isar_feature(aa64_sve2p2, s))) { + return false; + } + s->is_nonstreaming = true; + } + return gen_gvec_ool_arg_zpz(s, fns[a->esz], a, 0); +} /* Call the helper that computes the ARM LastActiveElement pseudocode * function, scaled by the element size. This includes the not found @@ -3469,6 +3481,62 @@ static bool trans_SINCDECP_z(DisasContext *s, arg_incdec2_pred *a) return true; } +static bool do_firstp_lastp(DisasContext *s, arg_rpr_esz *a, bool firstp) +{ + if (sve_access_check(s)) { + unsigned psz = pred_full_reg_size(s); + TCGv_i64 v = cpu_reg(s, a->rd); + + if (psz <= 8) { + uint64_t psz_mask; + + tcg_gen_ld_i64(v, tcg_env, pred_full_reg_offset(s, a->rn)); + if (a->rn != a->pg) { + TCGv_i64 g = tcg_temp_new_i64(); + tcg_gen_ld_i64(g, tcg_env, pred_full_reg_offset(s, a->pg)); + tcg_gen_and_i64(v, v, g); + } + + /* + * Reduce the pred_esz_masks value simply to reduce the + * size of the code generated here. + */ + psz_mask = MAKE_64BIT_MASK(0, psz * 8); + tcg_gen_andi_i64(v, v, pred_esz_masks[a->esz] & psz_mask); + + if (firstp) { + tcg_gen_ctzi_i64(v, v, -1); + } else { + tcg_gen_clzi_i64(v, v, 64); + tcg_gen_subfi_i64(v, 63, v); + } + tcg_gen_sari_i64(v, v, a->esz); + } else { + TCGv_ptr t_pn = tcg_temp_new_ptr(); + TCGv_ptr t_pg = tcg_temp_new_ptr(); + unsigned desc = 0; + TCGv_i32 t_desc; + + desc = FIELD_DP32(desc, PREDDESC, OPRSZ, psz); + desc = FIELD_DP32(desc, PREDDESC, ESZ, a->esz); + + tcg_gen_addi_ptr(t_pn, tcg_env, pred_full_reg_offset(s, a->rn)); + tcg_gen_addi_ptr(t_pg, tcg_env, pred_full_reg_offset(s, a->pg)); + t_desc = tcg_constant_i32(desc); + + if (firstp) { + gen_helper_sve_firstp(v, t_pn, t_pg, t_desc); + } else { + gen_helper_sve_lastp(v, t_pn, t_pg, t_desc); + } + } + } + return true; +} + +TRANS_FEAT(FIRSTP, aa64_sme2p2_or_sve2p2, do_firstp_lastp, a, true) +TRANS_FEAT(LASTP, aa64_sme2p2_or_sve2p2, do_firstp_lastp, a, false) + /* *** SVE Integer Compare Scalars Group */ @@ -6956,6 +7024,13 @@ static gen_helper_gvec_3 * const sqneg_fns[4] = { TRANS_FEAT(SQNEG_m, aa64_sme_or_sve2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 0) TRANS_FEAT(SQNEG_z, aa64_sme2p2_or_sve2p2, gen_gvec_ool_arg_zpz, sqneg_fns[a->esz], a, 1) +static gen_helper_gvec_3 * const expand_fns[4] = { + gen_helper_sve_expand_b, gen_helper_sve_expand_h, + gen_helper_sve_expand_s, gen_helper_sve_expand_d, +}; +TRANS_FEAT_STREAMING_IF(EXPAND, aa64_sme2p2_or_sve2p2, aa64_sme2p2, + gen_gvec_ool_arg_zpz, expand_fns[a->esz], a, 0) + DO_ZPZZ(SQSHL, aa64_sme_or_sve2, sve2_sqshl) DO_ZPZZ(SQRSHL, aa64_sme_or_sve2, sve2_sqrshl) DO_ZPZZ(SRSHL, aa64_sme_or_sve2, sve2_srshl) @@ -7094,7 +7169,7 @@ static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) if (!dc_isar_feature(aa64_sve2_pmull128, s)) { return false; } - s->is_nonstreaming = true; + s->is_nonstreaming = !dc_isar_feature(aa64_ssve_aes, s); } return gen_gvec_ool_arg_zzz(s, fns[a->esz], a, sel); } @@ -8099,15 +8174,17 @@ TRANS_FEAT(SDOT_zzzz_2s, aa64_sme2_or_sve2p1, gen_gvec_ool_arg_zzzz, TRANS_FEAT(UDOT_zzzz_2s, aa64_sme2_or_sve2p1, gen_gvec_ool_arg_zzzz, gen_helper_gvec_udot_2h, a, 0) -TRANS_FEAT_NONSTREAMING(AESMC, aa64_sve2_aes, gen_gvec_ool_zz, - gen_helper_crypto_aesmc, a->rd, a->rd, 0) -TRANS_FEAT_NONSTREAMING(AESIMC, aa64_sve2_aes, gen_gvec_ool_zz, - gen_helper_crypto_aesimc, a->rd, a->rd, 0) - -TRANS_FEAT_NONSTREAMING(AESE, aa64_sve2_aes, gen_gvec_ool_arg_zzz, - gen_helper_crypto_aese, a, 0) -TRANS_FEAT_NONSTREAMING(AESD, aa64_sve2_aes, gen_gvec_ool_arg_zzz, - gen_helper_crypto_aesd, a, 0) +TRANS_FEAT_STREAMING_IF(AESMC, aa64_sve_aes, aa64_ssve_aes, + gen_gvec_ool_zz, gen_helper_crypto_aesmc, + a->rd, a->rd, 0) +TRANS_FEAT_STREAMING_IF(AESIMC, aa64_sve_aes, aa64_ssve_aes, + gen_gvec_ool_zz, gen_helper_crypto_aesimc, + a->rd, a->rd, 0) + +TRANS_FEAT_STREAMING_IF(AESE, aa64_sve_aes, aa64_ssve_aes, + gen_gvec_ool_arg_zzz, gen_helper_crypto_aese, a, 0) +TRANS_FEAT_STREAMING_IF(AESD, aa64_sve_aes, aa64_ssve_aes, + gen_gvec_ool_arg_zzz, gen_helper_crypto_aesd, a, 0) TRANS_FEAT_NONSTREAMING(SM4E, aa64_sve2_sm4, gen_gvec_ool_arg_zzz, gen_helper_crypto_sm4e, a, 0) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index 06fef2a209..77e5b01550 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -56,6 +56,37 @@ typedef struct CPUArchState CPUARMState; #define H1_8(x) (x) /* + * When considering the ZA storage as an array of elements of + * type T, the index within that array of the Nth element of + * a vertical slice of a tile can be calculated like this, + * regardless of the size of type T. This is because the tiles + * are interleaved, so if type T is size N bytes then row 1 of + * the tile is N rows away from row 0. The division by N to + * convert a byte offset into an array index and the multiplication + * by N to convert from vslice-index-within-the-tile to + * the index within the ZA storage cancel out. + */ +#define tile_vslice_index(i) ((i) * sizeof(ARMVectorReg)) + +/* + * When doing byte arithmetic on the ZA storage, the element + * byteoff bytes away in a tile vertical slice is always this + * many bytes away in the ZA storage, regardless of the + * size of the tile element, assuming that byteoff is a multiple + * of the element size. Again this is because of the interleaving + * of the tiles. For instance if we have 1 byte per element then + * each row of the ZA storage has one byte of the vslice data, + * and (counting from 0) byte 8 goes in row 8 of the storage + * at offset (8 * row-size-in-bytes). + * If we have 8 bytes per element then each row of the ZA storage + * has 8 bytes of the data, but there are 8 interleaved tiles and + * so byte 8 of the data goes into row 1 of the tile, + * which is again row 8 of the storage, so the offset is still + * (8 * row-size-in-bytes). Similarly for other element sizes. + */ +#define tile_vslice_offset(byteoff) ((byteoff) * sizeof(ARMVectorReg)) + +/* * Expand active predicate bits to bytes, for byte elements. */ extern const uint64_t expand_pred_b_data[256]; diff --git a/tests/qtest/flexcan-test.c b/tests/qtest/flexcan-test.c new file mode 100644 index 0000000000..87d947a806 --- /dev/null +++ b/tests/qtest/flexcan-test.c @@ -0,0 +1,421 @@ +/* + * QTests for FlexCAN CAN controller device model + * + * Copyright (c) 2025 Matyas Bobek <matyas.bobek@gmail.com> + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "qemu/osdep.h" +#include "libqtest-single.h" + +#include "hw/net/flexcan.h" +#include "hw/net/can/flexcan_regs.h" + +#define FSL_IMX6_CAN2_ADDR 0x02094000 +#define FSL_IMX6_CAN2_SIZE 0x4000 +#define FSL_IMX6_CAN1_ADDR 0x02090000 +#define FSL_IMX6_CAN1_SIZE 0x4000 + +#define FC_QEMU_ARGS "-nographic -M sabrelite " \ + "-object can-bus,id=qcan0 " \ + "-machine canbus0=qcan0 -machine canbus1=qcan0" + +/* used for masking out unused/reserved bits */ +#define FC_MB_CNT_USED_MASK (~0xF080FFFFu) + +#define FCREG(BASE_ADDR, REG) ((BASE_ADDR) + offsetof(FlexcanRegs, REG)) +#define FCMB(BASE_ADDR, MB_IDX, WORD_IDX) (FCREG(BASE_ADDR, mbs) + \ + 0x10 * (MB_IDX) + 4 * (WORD_IDX)) + +typedef struct FcTestFrame { + uint32_t id; + uint32_t data[2]; + uint8_t len; + bool ide; + bool rtr; + + /* rx only */ + bool expect_overrun; +} FcTestFrame; + +static const FcTestFrame fc_test_frame_1 = { + .id = 0x5AF, + .len = 8, + .data = { + 0x01020304, + 0x0A0B0C0D + }, + .ide = false +}; + +static const FcTestFrame fc_test_frame_1_ide = { + .id = 0x105AF5AF, + .len = 8, + .data = { + 0x01020304, + 0x0A0B0C0D + }, + .ide = true +}; + +static void fc_reset(hwaddr ba, uint32_t mcr_flags) +{ + /* disable */ + writel(FCREG(ba, mcr), 0xD890000F); + + /* enable in freeze mode */ + writel(FCREG(ba, mcr), 0x5980000F); + + /* soft reset */ + writel(FCREG(ba, mcr), 0x5980000F | FLEXCAN_MCR_SOFTRST); + + g_assert_cmpuint(readl(FCREG(ba, mcr)), ==, 0x5980000F); + g_assert_cmpuint(readl(FCREG(ba, ctrl)), ==, 0); + g_assert_cmpuint(readl(FCREG(ba, ctrl2)), ==, 0); + + writel(FCREG(ba, mcr), (0x5980000F & ~FLEXCAN_MCR_HALT) | mcr_flags); + writel(FCREG(ba, ctrl2), FLEXCAN_CTRL2_RRS); + + /* initialize all mailboxes as rx inactive */ + for (int i = 0; i < FLEXCAN_MAILBOX_COUNT; i++) { + writel(FCMB(ba, i, 0), FLEXCAN_MB_CODE_RX_INACTIVE); + writel(FCMB(ba, i, 1), 0); + writel(FCMB(ba, i, 2), 0); + writel(FCMB(ba, i, 3), 0); + } +} + +static uint64_t fc_get_irqs(hwaddr ba) +{ + return (uint64_t)readl(FCREG(ba, iflag1)) | + ((uint64_t)readl(FCREG(ba, iflag2)) << 32); +} + +static void fc_clear_irq(hwaddr ba, int idx) +{ + if (idx >= 32) { + writel(FCREG(ba, iflag2), (uint32_t)1 << (idx - 32)); + } else { + writel(FCREG(ba, iflag1), (uint32_t)1 << idx); + } + + g_assert_cmpuint(fc_get_irqs(ba) & ((uint64_t)1 << idx), ==, 0); +} + +static void fc_setup_rx_mb(hwaddr ba, int mbidx) +{ + writel(FCMB(ba, mbidx, 0), FLEXCAN_MB_CODE_RX_EMPTY); + writel(FCMB(ba, mbidx, 1), 0); + /* the data value should be ignored for RX mb */ + writel(FCMB(ba, mbidx, 2), 0); + writel(FCMB(ba, mbidx, 3), 0); + + g_assert_cmpuint(readl(FCMB(ba, mbidx, 0)), ==, FLEXCAN_MB_CODE_RX_EMPTY); +} + +static void fc_tx(hwaddr ba, int mbidx, const FcTestFrame *frame) +{ + g_assert_cmpuint(frame->len, <=, 8); + + writel(FCMB(ba, mbidx, 0), FLEXCAN_MB_CODE_TX_INACTIVE); + uint32_t id = frame->ide ? frame->id : frame->id << 18; + writel(FCMB(ba, mbidx, 1), id); + writel(FCMB(ba, mbidx, 2), frame->data[0]); + writel(FCMB(ba, mbidx, 3), frame->data[1]); + + uint32_t ctrl = FLEXCAN_MB_CODE_TX_DATA | FLEXCAN_MB_CNT_LENGTH(frame->len); + if (frame->ide) { + ctrl |= FLEXCAN_MB_CNT_IDE | FLEXCAN_MB_CNT_SRR; + } + if (frame->rtr) { + ctrl |= FLEXCAN_MB_CNT_RTR; + } + writel(FCMB(ba, mbidx, 0), ctrl); + + /* check frame was transmitted */ + g_assert_cmpuint(fc_get_irqs(ba) & ((uint64_t)1 << mbidx), + !=, 0); + + uint32_t xpectd_ctrl = (ctrl & ~FLEXCAN_MB_CODE_MASK) | + FLEXCAN_MB_CODE_TX_INACTIVE; + g_assert_cmpuint(readl(FCMB(ba, mbidx, 0)) & FC_MB_CNT_USED_MASK, ==, + xpectd_ctrl); + /* other fields should stay unchanged */ + g_assert_cmpuint(readl(FCMB(ba, mbidx, 1)), ==, id); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 2)), ==, frame->data[0]); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 3)), ==, frame->data[1]); +} + +static void fc_rx_check(hwaddr ba, int mbidx, const FcTestFrame *frame) +{ + uint32_t xpectd_ctrl = frame->expect_overrun ? FLEXCAN_MB_CODE_RX_OVERRUN + : FLEXCAN_MB_CODE_RX_FULL; + xpectd_ctrl |= FLEXCAN_MB_CNT_LENGTH(frame->len) | FLEXCAN_MB_CNT_SRR; + if (frame->ide) { + xpectd_ctrl |= FLEXCAN_MB_CNT_IDE; + } + if (frame->rtr) { + xpectd_ctrl |= FLEXCAN_MB_CNT_RTR; + } + + uint32_t xpectd_id = frame->ide ? frame->id : frame->id << 18; + + uint32_t ctrl = readl(FCMB(ba, mbidx, 0)) & FC_MB_CNT_USED_MASK; + if ((ctrl & FLEXCAN_MB_CODE_MASK) == FLEXCAN_MB_CODE_RX_EMPTY) { + fprintf(stderr, "expected frame (id=0x%X) not received\n", frame->id); + } + + g_assert_cmpuint(ctrl, ==, xpectd_ctrl); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 1)), ==, xpectd_id); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 2)), ==, frame->data[0]); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 3)), ==, frame->data[1]); +} + +static void fc_check_empty_multi(hwaddr ba, int idx_count, int mbidxs[]) +{ + for (int i = 0; i < FLEXCAN_MAILBOX_COUNT; i++) { + bool check_empty = true; + int ctrl = readl(FCMB(ba, i, 0)); + + for (int j = 0; j < idx_count; j++) { + if (i == mbidxs[j]) { + check_empty = false; + } + } + + if (check_empty) { + if (!(ctrl == FLEXCAN_MB_CODE_RX_EMPTY || + ctrl == FLEXCAN_MB_CODE_RX_INACTIVE)) { + g_assert_cmpuint(ctrl, ==, FLEXCAN_MB_CODE_RX_INACTIVE); + } + g_assert_cmpuint(readl(FCMB(ba, i, 1)), ==, 0); + g_assert_cmpuint(readl(FCMB(ba, i, 2)), ==, 0); + g_assert_cmpuint(readl(FCMB(ba, i, 3)), ==, 0); + } else { + g_assert_cmpuint( + ctrl & FLEXCAN_MB_CODE_MASK, !=, + FLEXCAN_MB_CODE_RX_INACTIVE + ); + } + } +} + +static void fc_check_empty(hwaddr ba, int mbidx) +{ + fc_check_empty_multi(ba, 1, &mbidx); +} + +static void flexcan_test_linux_probe_impl(hwaddr fba) +{ + /* -- test a Linux driver-like probe sequence -- */ + /* disable */ + writel(FCREG(fba, mcr), 0xD890000F); + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xD890000F); + g_assert_cmpuint(readl(FCREG(fba, ctrl)), ==, 0); + + /* set bit in reserved field we do not implement (CTRL_CLK_SRC) */ + writel(FCREG(fba, ctrl), 0x00002000); + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xD890000F); + + /* enable in freeze mode */ + writel(FCREG(fba, mcr), 0x5980000F); + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x5980000F); + + /* enable Rx-FIFO */ + writel(FCREG(fba, mcr), 0x7980000F); + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x7980000F); + g_assert_cmpuint(readl(FCREG(fba, ecr)), ==, 0); + + /* disable */ + writel(FCREG(fba, mcr), 0xF890000F); + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xF890000F); +} + +static void flexcan_test_freeze_disable_interaction_impl(hwaddr fba) +{ + /* -- test normal <=> freeze <=> disable transitions -- */ + + /* leave freeze in disabled, FRZ_ACK should stay cleared */ + writel(FCREG(fba, mcr), 0xF890000F); /* disable */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xF890000F); + writel(FCREG(fba, mcr), 0xB890000F); /* by clearing FRZ */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xB890000F); + + writel(FCREG(fba, mcr), 0xF890000F); /* disable */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xF890000F); + writel(FCREG(fba, mcr), 0xE890000F); /* by clearing HALT */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xE890000F); + + writel(FCREG(fba, mcr), 0xF890000F); /* disable */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xF890000F); + writel(FCREG(fba, mcr), 0xA890000F); /* by clearing both */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0xA890000F); + + /* enter and leave freeze */ + writel(FCREG(fba, mcr), 0x7980000F); /* enable in freeze mode */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x7980000F); + writel(FCREG(fba, mcr), 0x3980000F); /* leave by clearing FRZ */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x3080000F); + + writel(FCREG(fba, mcr), 0x7980000F); /* enable in freeze mode */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x7980000F); + writel(FCREG(fba, mcr), 0x6980000F); /* leave by clearing HALT */ + g_assert_cmpuint(readl(FCREG(fba, mcr)), ==, 0x6080000F); +} + +static void flexcan_test_mailbox_io_impl(hwaddr ba_tx, hwaddr ba_rx) +{ + /* -- test correct handling of mailbox IO -- */ + const int test_1_mbidx = 0; + fc_reset(ba_tx, + FLEXCAN_MCR_SRX_DIS | FLEXCAN_MCR_MAXMB(FLEXCAN_MAILBOX_COUNT)); + fc_reset(ba_rx, + FLEXCAN_MCR_SRX_DIS | FLEXCAN_MCR_MAXMB(FLEXCAN_MAILBOX_COUNT)); + + fc_setup_rx_mb(ba_rx, test_1_mbidx); + fc_tx(ba_tx, test_1_mbidx, &fc_test_frame_1_ide); + g_assert_cmpuint(fc_get_irqs(ba_rx), ==, 1 << test_1_mbidx); + fc_rx_check(ba_rx, test_1_mbidx, &fc_test_frame_1_ide); + readl(FCREG(ba_rx, timer)); /* reset lock */ + + writel(FCMB(ba_rx, test_1_mbidx, 0), 0); + g_assert_cmpuint(readl(FCMB(ba_rx, test_1_mbidx, 0)), ==, 0); + writel(FCMB(ba_rx, test_1_mbidx, 1), 0x99AABBCC); + g_assert_cmpuint(readl(FCMB(ba_rx, test_1_mbidx, 1)), ==, 0x99AABBCC); +} + +static void flexcan_test_dual_transmit_receive_impl(hwaddr ba_tx, hwaddr ba_rx) +{ + /* -- test TX and RX between the two FlexCAN instances -- */ + const int test_1_mbidx = 50; + const int test_rounds = 50; + + /* self-receive enabled on tx FC */ + fc_reset(ba_tx, + FLEXCAN_MCR_MAXMB(FLEXCAN_MAILBOX_COUNT)); + fc_reset(ba_rx, + FLEXCAN_MCR_SRX_DIS | FLEXCAN_MCR_MAXMB(FLEXCAN_MAILBOX_COUNT)); + + /* tests self-receive on tx and reception on rx */ + fc_setup_rx_mb(ba_rx, test_1_mbidx); + fc_check_empty(ba_rx, test_1_mbidx); + fc_setup_rx_mb(ba_tx, test_1_mbidx + 1); + fc_check_empty(ba_tx, test_1_mbidx + 1); + g_assert_cmpuint(fc_get_irqs(ba_rx), ==, 0); + g_assert_cmpuint(fc_get_irqs(ba_tx), ==, 0); + + fc_tx(ba_tx, test_1_mbidx, &fc_test_frame_1); + fc_clear_irq(ba_tx, test_1_mbidx); + + fc_rx_check(ba_rx, test_1_mbidx, &fc_test_frame_1); + fc_check_empty(ba_rx, test_1_mbidx); + fc_rx_check(ba_tx, test_1_mbidx + 1, &fc_test_frame_1); + int tx_non_empty_mbidxs[] = {test_1_mbidx, test_1_mbidx + 1}; + + fc_check_empty_multi(ba_tx, 2, tx_non_empty_mbidxs); + fc_clear_irq(ba_rx, test_1_mbidx); + fc_clear_irq(ba_tx, test_1_mbidx + 1); + readl(FCREG(ba_rx, timer)); /* reset lock */ + + for (int ridx = 0; ridx < test_rounds; ridx++) { + /* test extended IDs sent to all mailboxes */ + for (int i = 0; i < FLEXCAN_MAILBOX_COUNT; i++) { + fc_setup_rx_mb(ba_rx, i); + } + fc_check_empty_multi(ba_rx, 0, NULL); + g_assert_cmpuint(fc_get_irqs(ba_rx), ==, 0); + g_assert_cmpuint(fc_get_irqs(ba_tx), ==, 0); + + for (int i = 0; i < FLEXCAN_MAILBOX_COUNT; i++) { + fc_tx(ba_tx, i, &fc_test_frame_1_ide); + } + g_assert_cmpuint(fc_get_irqs(ba_rx), ==, UINT64_MAX); + g_assert_cmpuint(fc_get_irqs(ba_tx), ==, UINT64_MAX); + for (int i = 0; i < FLEXCAN_MAILBOX_COUNT; i++) { + fc_rx_check(ba_rx, i, &fc_test_frame_1_ide); + } + + /* reset interrupts */ + writel(FCREG(ba_rx, iflag1), UINT32_MAX); + writel(FCREG(ba_rx, iflag2), UINT32_MAX); + writel(FCREG(ba_tx, iflag1), UINT32_MAX); + writel(FCREG(ba_tx, iflag2), UINT32_MAX); + g_assert_cmpuint(fc_get_irqs(ba_rx), ==, 0); + g_assert_cmpuint(fc_get_irqs(ba_tx), ==, 0); + } +} + +static void flexcan_test_tx_abort_impl(hwaddr ba) +{ + /* -- test the TX abort feature -- */ + fc_reset(ba, + FLEXCAN_MCR_SRX_DIS | FLEXCAN_MCR_MAXMB(FLEXCAN_MAILBOX_COUNT)); + + + for (int mbidx = 0; mbidx < FLEXCAN_MAILBOX_COUNT; mbidx++) { + fc_tx(ba, mbidx, &fc_test_frame_1); + + writel(FCMB(ba, mbidx, 0), FLEXCAN_MB_CODE_TX_ABORT); + g_assert_cmpuint(readl(FCMB(ba, mbidx, 0)), ==, + FLEXCAN_MB_CODE_TX_INACTIVE); + } +} + +static void flexcan_test_freeze_disable_interaction(void) +{ + qtest_start(FC_QEMU_ARGS); + flexcan_test_freeze_disable_interaction_impl(FSL_IMX6_CAN1_ADDR); + flexcan_test_freeze_disable_interaction_impl(FSL_IMX6_CAN2_ADDR); + qtest_end(); +} + +static void flexcan_test_linux_probe(void) +{ + qtest_start(FC_QEMU_ARGS); + flexcan_test_linux_probe_impl(FSL_IMX6_CAN1_ADDR); + flexcan_test_linux_probe_impl(FSL_IMX6_CAN2_ADDR); + qtest_end(); +} + +static void flexcan_test_dual_transmit_receive(void) +{ + qtest_start(FC_QEMU_ARGS); + flexcan_test_dual_transmit_receive_impl(FSL_IMX6_CAN1_ADDR, + FSL_IMX6_CAN2_ADDR); + flexcan_test_dual_transmit_receive_impl(FSL_IMX6_CAN2_ADDR, + FSL_IMX6_CAN1_ADDR); + qtest_end(); +} + +static void flexcan_test_tx_abort(void) +{ + qtest_start(FC_QEMU_ARGS); + flexcan_test_tx_abort_impl(FSL_IMX6_CAN1_ADDR); + flexcan_test_tx_abort_impl(FSL_IMX6_CAN2_ADDR); + qtest_end(); +} + +static void flexcan_test_mailbox_io(void) +{ + qtest_start(FC_QEMU_ARGS); + flexcan_test_mailbox_io_impl(FSL_IMX6_CAN1_ADDR, FSL_IMX6_CAN2_ADDR); + flexcan_test_mailbox_io_impl(FSL_IMX6_CAN2_ADDR, FSL_IMX6_CAN1_ADDR); + qtest_end(); +} + +int main(int argc, char **argv) +{ + g_test_init(&argc, &argv, NULL); + + qtest_add_func("/net/flexcan/linux_probe", flexcan_test_linux_probe); + qtest_add_func("/net/flexcan/freeze_disable_interaction", + flexcan_test_freeze_disable_interaction); + qtest_add_func("/net/flexcan/dual_transmit_receive", + flexcan_test_dual_transmit_receive); + qtest_add_func("/net/flexcan/tx_abort", + flexcan_test_tx_abort); + qtest_add_func("/net/flexcan/mailbox_io", flexcan_test_mailbox_io); + + return g_test_run(); +} diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build index 822e0bd286..088e64aca5 100644 --- a/tests/qtest/meson.build +++ b/tests/qtest/meson.build @@ -251,6 +251,7 @@ qtests_arm = \ (config_all_devices.has_key('CONFIG_MICROBIT') ? ['microbit-test'] : []) + \ (config_all_devices.has_key('CONFIG_STM32L4X5_SOC') ? qtests_stm32l4x5 : []) + \ (config_all_devices.has_key('CONFIG_FSI_APB2OPB_ASPEED') ? ['aspeed_fsi-test'] : []) + \ + (config_all_devices.has_key('CONFIG_CAN_FLEXCAN') ? ['flexcan-test'] : []) + \ (config_all_devices.has_key('CONFIG_STM32L4X5_SOC') and config_all_devices.has_key('CONFIG_DM163')? ['dm163-test'] : []) + \ ['arm-cpu-features', diff --git a/tests/tcg/aarch64/Makefile.softmmu-target b/tests/tcg/aarch64/Makefile.softmmu-target index f7a7d2b800..3f30be6c5d 100644 --- a/tests/tcg/aarch64/Makefile.softmmu-target +++ b/tests/tcg/aarch64/Makefile.softmmu-target @@ -25,7 +25,7 @@ LDFLAGS=-Wl,-T$(LINK_SCRIPT) TESTS+=$(AARCH64_TESTS) $(MULTIARCH_TESTS) EXTRA_RUNS+=$(MULTIARCH_RUNS) CFLAGS+=-nostdlib -ggdb -O0 $(MINILIB_INC) -LDFLAGS+=-static -nostdlib $(CRT_OBJS) $(MINILIB_OBJS) -lgcc +LDFLAGS+=-static -nostdlib $(MINILIB_OBJS) -lgcc config-cc.mak: Makefile $(quiet-@)( \ @@ -36,21 +36,27 @@ config-cc.mak: Makefile # building head blobs .PRECIOUS: $(CRT_OBJS) +vector_log_boot.o: $(CRT_PATH)/boot.S + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -DLOGGING_VECTOR_TABLE -x assembler-with-cpp -Wa,--noexecstack -c $< -o $@ + %.o: $(CRT_PATH)/%.S $(CC) $(CFLAGS) $(EXTRA_CFLAGS) -x assembler-with-cpp -Wa,--noexecstack -c $< -o $@ # Build and link the tests %: %.c $(LINK_SCRIPT) $(CRT_OBJS) $(MINILIB_OBJS) - $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) boot.o memory: CFLAGS+=-DCHECK_UNALIGNED=1 memory-sve: memory.c $(LINK_SCRIPT) $(CRT_OBJS) $(MINILIB_OBJS) - $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) boot.o memory-sve: CFLAGS+=-DCHECK_UNALIGNED=1 -march=armv8.1-a+sve -O3 -TESTS+=memory-sve +gpc-test: gpc-test.c $(LINK_SCRIPT) vector_log_boot.o $(MINILIB_OBJS) + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) vector_log_boot.o + +TESTS+=memory-sve gpc-test # Running QEMU_BASE_MACHINE=-M virt -cpu max -display none @@ -71,6 +77,12 @@ QEMU_EL2_MACHINE=-machine virt,virtualization=on,gic-version=2 -cpu cortex-a57 - QEMU_EL2_BASE_ARGS=-semihosting-config enable=on,target=native,chardev=output,arg="2" run-vtimer: QEMU_OPTS=$(QEMU_EL2_MACHINE) $(QEMU_EL2_BASE_ARGS) -kernel +# gpc tests need EL3 and RME +QEMU_EL3_MACHINE=-machine virt,virtualization=on,secure=on,gic-version=3 -cpu max,x-rme=on +QEMU_EL3_BASE_ARGS=-semihosting-config enable=on,target=native,chardev=output,arg="3" +run-gpc-test: QEMU_OPTS=$(QEMU_EL3_MACHINE) $(QEMU_EL3_BASE_ARGS) -kernel +run-gpc3-test: QEMU_OPTS=$(QEMU_EL3_MACHINE) $(QEMU_EL3_BASE_ARGS) -kernel + # Simple Record/Replay Test .PHONY: memory-record run-memory-record: memory-record memory @@ -107,7 +119,7 @@ QEMU_MTE_ENABLED_MACHINE=-M virt,mte=on -cpu max -display none QEMU_OPTS_WITH_MTE_ON = $(QEMU_MTE_ENABLED_MACHINE) $(QEMU_BASE_ARGS) -kernel mte: CFLAGS+=-march=armv8.5-a+memtag mte: mte.S $(LINK_SCRIPT) $(CRT_OBJS) $(MINILIB_OBJS) - $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) + $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $< -o $@ $(LDFLAGS) boot.o run-mte: QEMU_OPTS=$(QEMU_OPTS_WITH_MTE_ON) run-mte: mte diff --git a/tests/tcg/aarch64/system/boot.S b/tests/tcg/aarch64/system/boot.S index 8bfa4e4efc..c6e5ad889b 100644 --- a/tests/tcg/aarch64/system/boot.S +++ b/tests/tcg/aarch64/system/boot.S @@ -60,6 +60,40 @@ curr_sp0_irq: curr_sp0_fiq: curr_sp0_serror: curr_spx_sync: +#ifdef LOGGING_VECTOR_TABLE + sub sp, sp, #16 + stp x0, x1, [sp, #0] + mrs x0, ESR_EL3 + lsr x0, x0, #26 + and x0, x0, #0x3f + cmp x0, #37 + beq data_fault + cmp x0, #30 + beq gpc_fault + b generic_exception + +data_fault: + mrs x0, FAR_EL3 + adrp x1, exception_log + str x0, [x1] + ldr x0, =0x1001 + str x0, [x1, #8] + b skip_return +gpc_fault: + mrs x0, FAR_EL3 + adrp x1, exception_log + str x0, [x1] + ldr x0, =0x1002 + str x0, [x1, #8] + /* Fall through */ +skip_return: + mrs x0, ELR_EL3 + add x0, x0, #4 /* Skip faulting instruction */ + msr ELR_EL3, x0 + ldp x0, x1, [sp, #0] + add sp, sp, #16 + eret +#endif curr_spx_irq: curr_spx_fiq: curr_spx_serror: @@ -71,6 +105,7 @@ lower_a32_sync: lower_a32_irq: lower_a32_fiq: lower_a32_serror: +generic_exception: adr x1, .unexp_excp exit_msg: mov x0, SYS_WRITE0 @@ -404,6 +439,36 @@ ttb_stage2: .space 4096, 0 .align 12 + .global realms_gpt0 + /* GPT stage 0 table */ +realms_gpt0: + .space 4096, 0 + .align 17 + .global realms_gpt1 + /* GPT stage 1 table, initialised to all 0xFF (full access) */ +realms_gpt1: + .space 524288, 0xFF +#ifdef LOGGING_VECTOR_TABLE + .align 12 + .global exception_fault_address + .type exception_fault_address, @object + .size exception_fault_address, 8 + .global exception_type_code + .type exception_type_code, @object + .size exception_type_code, 8 + /* + * These fields record details of the last exception, if + * LOGGING_VECTOR_TABLE is defined. + */ +exception_log: +exception_fault_address: + /* The contents of FAR_EL3 when an exception is taken. */ + .space 8, 0 +exception_type_code: + /* A generic code indicating what type of exception occurred. */ + .space 8, 0 +#endif + .align 12 system_stack: .space 4096, 0 system_stack_end: diff --git a/tests/tcg/aarch64/system/boot.h b/tests/tcg/aarch64/system/boot.h new file mode 100644 index 0000000000..cb9ab6c1b9 --- /dev/null +++ b/tests/tcg/aarch64/system/boot.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * + * + * Copyright (c) 2026 Linaro Ltd + * + */ + + +/* Global variables exported in boot.S */ +extern volatile uint64_t exception_fault_address; /* Updated by ISR */ +extern volatile uint64_t exception_type_code; /* Updated by ISR */ +extern uint64_t realms_gpt0[]; +extern uint64_t realms_gpt1[]; diff --git a/tests/tcg/aarch64/system/gpc-test.c b/tests/tcg/aarch64/system/gpc-test.c new file mode 100644 index 0000000000..f26b23efaf --- /dev/null +++ b/tests/tcg/aarch64/system/gpc-test.c @@ -0,0 +1,165 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * + * + * Copyright (c) 2026 Linaro Ltd + * + */ + +#include <stdbool.h> +#include <stdint.h> +#include <minilib.h> +#include "boot.h" + +#define ID_AA64PFR0_EL1 "S3_0_C0_C4_0" + +#define GPTBR_EL3 "S3_6_C2_C1_4" +#define GPCBW_EL3 "S3_6_C2_C1_5" +#define GPCCR_EL3 "S3_6_C2_C1_6" +#define VBAR_EL3 "S3_6_C12_C0_0" + +#define get_sys_reg(register_name, dest) \ + asm("mrs %[reg], " register_name "\n\t" : [reg] "=r" (dest)) +#define set_sys_reg(register_name, value) \ + asm("msr " register_name ", %[reg]\n\r" : : [reg] "r" (value)) + +const uint32_t gpc_granule_size = 4096; +const uint32_t gpis_per_64_bits = 16; + +int main(uint64_t sp) +{ + uint64_t out; + uint64_t pfr0; + uint64_t gpt_base; + uint64_t rme_status; + uint64_t currentel_raw; + uint64_t currentel; + uint64_t gpcbw; + uint64_t gpt_table0_addr = (uint64_t) realms_gpt0; + uint64_t gpt_table1_addr = (uint64_t) realms_gpt1; + + /* Mask is FNG1, FNG0, and A2 */ + const uint64_t feature_mask = (1ULL << 18 | 1ULL << 17 | 1ULL << 16); + const uint64_t in = feature_mask; + + get_sys_reg("CurrentEL", currentel_raw); + currentel = (currentel_raw >> 2) & 0x3; + + if (currentel < 3) { + ml_printf("FAIL: Test must be run at EL3 (it is %d)\n", currentel); + return 1; + } + + get_sys_reg(ID_AA64PFR0_EL1, pfr0); + + /* rme_status is 1 for RME, 2 for RME + GPC2, 3 for RME+GPC3 */ + rme_status = (pfr0 >> 52) & 0xF; + if (rme_status < 2) { + ml_printf("SKIP: System does not support RME (RME=%ld)\n", rme_status); + return 0; + } + + /* Configure the level 0 table for the first 4GB of memory */ + realms_gpt0[0] = gpt_table1_addr | 0x3; /* Covers GB 0; table descriptor */ + realms_gpt0[1] = 0xf1; /* Covers GB 1; full access */ + realms_gpt0[2] = 0xf1; /* Covers GB 2; full access */ + realms_gpt0[3] = 0xf1; /* Covers GB 3; full access */ + + /* Pick an artibtrary location to read inside the first 1GB. */ + uint64_t fault_location = 0x10202008; + uint32_t gpi_index = fault_location / gpc_granule_size; + realms_gpt1[gpi_index / gpis_per_64_bits] = 0; + + gpt_base = gpt_table0_addr >> 12; + set_sys_reg(GPTBR_EL3, gpt_base); + + /* + * Default values: + * PPS=0: GPC table 0 protects 4GB. + * RLPAD=0: Realm physical address spaces are normal + * NSPAD=0: Non-secure physical address spaces are normal + * SPAD=0: Secure physical address spaces are normal + * IRGN=0: Inner non-cacheable + * ORGN=0: Outer non-cacheable + * PGS=0: Physical granule size is 4KB. + * GPCP=0: All GPC faults reported + * TBGPCP=0: Trace buffer rejects trace + * L0GPTSZ=0: Each entry in table 0 protects 1GB. + * APPSAA=0: Accesses above 4GB must be to Non-secure PAs + * GPCBW=0: Bypass windows disabled. + * NA6, NA7, NSP, SA, NSO are all reserved values for GPI. + */ + uint64_t gpccr = 0; + + /* Switch on granule protection check */ + gpccr |= 1 << 16; /* GPC enabled. */ + gpccr |= 0b10 << 12; /* SH = Outer shareable */ + set_sys_reg(GPCCR_EL3, gpccr); + + /* Access some memory outside the GPC forbidden region */ + uint64_t x = *(unsigned int *) (fault_location + 4096 * 16); + ml_printf("Fault address: %lx\n", exception_fault_address); + if (exception_fault_address != 0) { + ml_printf("FAIL: Memory access was blocked by GPC, " + "and should not have been\n"); + return 1; + } + + /* Access the GPC forbidden region */ + x = *(unsigned int *) fault_location; + + ml_printf("Fault address: %lx\n", exception_fault_address); + if (exception_fault_address != fault_location) { + ml_printf("FAIL: Memory access was not blocked by GPC, " + "and should have been\n"); + return 1; + } + + rme_status = (pfr0 >> 52) & 0xF; + if (rme_status < 3) { + ml_printf("SKIP: System does not support GPC3 (RME=%ld)\n", rme_status); + return 0; + } + + /* Clear the exception record */ + exception_fault_address = 0; + + /* Enable bypass windows */ + gpccr |= 1 << 29; /* GPC Bypass windows enabled */ + set_sys_reg(GPCCR_EL3, gpccr); + + gpcbw = 0; /* Base 0GB, Size 1GB, Stride 1TB */ + set_sys_reg(GPCBW_EL3, gpcbw); + ml_printf("GPCBW configured\n"); + + /* Access the GPC forbidden region again */ + x = *(unsigned int *) fault_location; + + ml_printf("Fault address: %lx\n", exception_fault_address); + if (exception_fault_address != 0) { + ml_printf("FAIL: Memory access was blocked by GPC, " + "and should have been allowed by bypass window. code=%lx\n", + exception_type_code); + return 1; + } + + /* Clear the exception record */ + exception_fault_address = 0; + /* Reconfigure GPCBW to 1GB start */ + gpcbw = 1; /* Base 1GB, Size 1GB, Stride 1TB */ + set_sys_reg(GPCBW_EL3, gpcbw); + ml_printf("GPCBW reconfigured for 1GB start\n"); + + /* Access the GPC forbidden region again */ + x = *(unsigned int *) fault_location; + + ml_printf("Fault address: %lx\n", exception_fault_address); + if (exception_fault_address != fault_location) { + ml_printf("FAIL: Memory access was allowed by GPC, " + "and should not have been allowed by bypass window. code=%lx\n", + exception_type_code); + return 1; + } + + return 0; +} |
