Background ========== On ARM, TLB invalidation instructions (TLBI) traditionally broadcast to all PEs sharing a translation regime. In large KVM guests this becomes expensive: each invalidation reaches every vCPU even when the affected translations are only used by a subset of PEs, e.g. a sub-cluster running one workload group. The bigger the guest, the more of the invalidation traffic is wasted on unaffected PEs. The ARM architecture addresses this with TLBI domains: each PE can be programmed to belong to several (virtual) domains, and domain-targeted TLBI variants then invalidate only within the given domain instead of broadcasting. When the host KVM exposes virtual TLBI domains (vdomains), the guest can group its vCPUs and use domain-scoped TLBI to cut invalidation traffic to the relevant subset of PEs. Using this from a guest requires the VMM to do three things: let the user describe the guest's domain topology, program each vCPU's domain membership into KVM, and describe the topology to the guest so its kernel can discover and use it. The guest kernel detects the TLBID field in ID_AA64MMFR4_EL1 and parses an ACPI "TLBI" table to learn the per-PE domain membership. Design Overview =============== The series adds the following to the ARM virt machine: 1. KVM capability query helpers (kvm_arm_vtlbid_supported(), kvm_arm_get_max_vdomains()), together with the corresponding KVM uapi definitions (KVM_CAP_ARM_TLBIDOMAIN, KVM_ARM_GET_VDOMAIN_NUM, KVM_ARM_VCPU_SET_VDOMAIN) in linux-headers. 2. A -tlbidomain command-line option, parsed like the NUMA options into per-domain CPU bitmaps held in VirtMachineState. A vCPU's membership is computed on demand as a uint32 bitmask of domain IDs (virt_cpu_vdomain_bitmap()): -tlbidomain domain-id=0,cpus=0-7 -tlbidomain domain-id=1,cpus=0-3,cpus=5-7 -tlbidomain domain-id=2,cpus=4 Domain 0 is the broadcast domain and must cover all vCPUs; domain IDs must be contiguous from 0; the total number must not exceed what KVM supports. With TCG the topology is only reflected in the ACPI table (a warning is printed). 3. Per-vCPU KVM configuration at vCPU init (kvm_arm_vtlbid_init()): the vdomain bitmap is pushed to KVM via KVM_ARM_VCPU_SET_VDOMAIN, and the TLBID field of ID_AA64MMFR4_EL1 is registered in the sysreg properties table so the writable-idreg machinery exposes it to the guest when KVM reports the bits writable. All possible vCPUs are configured before the VM starts, so vCPU hot-plug needs no extra handling, and MMFR4 is read back from KVM so the TLBID bit survives hot-plug. 4. ACPI table generation: build_tlbi() emits the "TLBI" table with one PE Component subtable per vCPU listing the domain IDs it belongs to; the guest kernel's acpi_tlbi_init() parses it to build the TLBI domain topology. Example topology for a 8-vCPU guest: ---------------------------------------------------------------------- vCPU | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 ---------------------------------------------------------------------- domain 0 (broadcast) | x | x | x | x | x | x | x | x domain 1 | x | x | x | x | | x | x | x domain 2 | | | | | x | | | ---------------------------------------------------------------------- A TLBI issued by vCPU 5 with domain 1 only invalidates translations shared by the vCPUs of domain 1, instead of all 8 vCPUs. This series depends on the companion kernel series that adds KVM_CAP_ARM_TLBIDOMAIN support and on a guest kernel that parses the TLBI ACPI table. Tian Zheng (4): target/arm: add KVM TLBI domain capability query helpers hw/arm/virt: add -tlbidomain command-line option and parsing target/arm: configure KVM TLBI vdomain at vCPU init hw/arm/virt-acpi: generate TLBI ACPI table hw/arm/virt-acpi-build.c | 73 ++++++++++++- hw/arm/virt.c | 161 +++++++++++++++++++++++++++++ include/hw/arm/virt.h | 13 +++ linux-headers/linux/kvm.h | 15 +++ qapi/machine.json | 16 +++++ qemu-options.hx | 17 +++++ system/vl.c | 16 +++++ target/arm/cpu-sysreg-properties.c | 1 + target/arm/kvm64.c | 82 +++++++++++++++ target/arm/kvm_arm.h | 25 +++++ 10 files changed, 418 insertions(+), 1 deletion(-) -- 2.33.0