-------- Forwarded Message -------- Subject: S390 testing for IOMMUFD Date: Mon, 7 Nov 2022 21:09:02 -0400 From: Jason Gunthorpe jgg@nvidia.com To: Cornelia Huck cohuck@redhat.com, Eric Farman farman@linux.ibm.com, Matthew Rosato mjrosato@linux.ibm.com, Niklas Schnelle schnelle@linux.ibm.com, Tony Krowiak akrowiak@linux.ibm.com, Halil Pasic pasic@linux.ibm.com, Jason Herne jjherne@linux.ibm.com, linux-s390@vger.kernel.org CC: iommu@lists.linux.dev, Kevin Tian kevin.tian@intel.com, Alex Williamson alex.williamson@redhat.com, kvm@vger.kernel.org, Lu Baolu baolu.lu@linux.intel.com, Nicolin Chen nicolinc@nvidia.com
On Mon, Nov 07, 2022 at 08:48:53PM -0400, Jason Gunthorpe wrote:
[ This has been in linux-next for a little while now, and we've completed the syzkaller run. 1300 hours of CPU time have been invested since the last report with no improvement in coverage or new detections. syzkaller coverage reached 69%(75%), and review of the misses show substantial amounts are WARN_ON's and other debugging which are not expected to be covered. ]
iommufd is the user API to control the IOMMU subsystem as it relates to managing IO page tables that point at user space memory.
[chop cc list]
s390 mdev maintainers,
Can I ask your help to test this with the two S390 mdev drivers? Now that gvt is passing and we've covered alot of the QA ground it is a good time to run it.
Take the branch from here:
https://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd.git/log/?h=for-n...
And build the kernel with CONFIG_VFIO_CONTAINER=n CONFIG_IOMMUFD=y CONFIG_IOMMUFD_VFIO_CONTAINER=y
And your existing stuff should work with iommufd providing the iommu support to vfio. There will be a dmesg confirming this.
Let me know if there are any problems!
If I recall there was some desire from the S390 platform team to start building on iommufd to create some vIOMMU acceleration for S390 guests, this is a necessary first step.
Thanks, Jason
On 2022/11/9 10:01, Zhangfei Gao wrote:
test this with the two S390 mdev drivers
Hi Zhangfei I test iommufd with SEC, get the following error.
start qemu： echo 1 > /sys/bus/pci/devices/0000:76:00.0/sriov_numvfs echo 0000:76:00.1 > /sys/bus/pci/devices/0000:76:00.1/driver/unbind echo vfio-pci > /sys/bus/pci/devices/0000:76:00.1/driver_override echo 0000:76:00.1 > /sys/bus/pci/drivers_probe
./qemu-system-aarch64 -machine virt,gic-version=3,iommu=nested-smmuv3,iommufd=iommufd0 \ -enable-kvm -cpu host -m 4G -smp cpus=8,maxcpus=8 \ -object iommufd,id=iommufd0 \ -kernel /home/kernel/Image \ -initrd /home/mini-rootfs/rootfs.cpio.gz \ -device vfio-pci,host=0000:76:00.1,iommufd=iommufd0 \ -bios QEMU_EFI.fd \ -append "rdinit=init console=ttyAMA0 root=/dev/vda rw kpti=off" \ -net none \ -nographic
the error log： [root@localhost ~]# [ 141.122110] pci 0000:76:00.1: [19e5:a256] type 00 class 0x100000 [ 141.128377] pci 0000:76:00.1: enabling Extended Tags [ 141.134001] hisi_sec2 0000:76:00.1: Adding to iommu group 72 [ 141.140616] hisi_sec2 0000:76:00.1: enabling device (0000 -> 0002) [ 141.164904] hisi_sec2 0000:76:00.1: SMMU Opened, the iommu type = 3 [ 141.171538] hisi_sec2 0000:76:00.1: HW V2 not both use uacce sva mode and hardware crypto algs. [ 141.180765] hisi_sec2 0000:76:00.0: VF enabled, vfs_num(=1)! [ 149.623052] vfio-pci 0000:76:00.1: enabling device (0000 -> 0002) [ 149.734168] vfio_ecap_init, failed to add special caps for VF 0000:76:00.1 [ 182.349340] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 182.358575] Mem abort info: [ 182.361456] ESR = 0x0000000096000004 [ 182.365348] EC = 0x25: DABT (current EL), IL = 32 bits [ 182.370791] SET = 0, FnV = 0 [ 182.373856] EA = 0, S1PTW = 0 [ 182.377109] FSC = 0x04: level 0 translation fault [ 182.382173] Data abort info: [ 182.385146] ISV = 0, ISS = 0x00000004 [ 182.389117] CM = 0, WnR = 0 [ 182.392139] user pgtable: 4k pages, 48-bit VAs, pgdp=00002020b5ae9000 [ 182.398586] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000 [ 182.405496] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP [ 182.411770] Modules linked in: ip6table_nat ip6table_mangle iptable_mangle vfio_iommu_type1 vfio_pci vfio_pci_core vfio ip6table_filter ip6_tables p2pdma_provider(O) arm_spe_pmu hns_roce_hw_v2 hibmc_drm drm_vram_helper drm_ttm_helper ttm tst_bbml(O) drm_kms_helper hisi_uncore_hha_pmu hisi_uncore_ddrc_pmu hisi_uncore_l3c_pmu hisi_uncore_pmu spi_dw_mmio fuse hclge crct10dif_ce sbsa_gwdt hns3 hisi_sas_v3_hw hnae3 hisi_zip hisi_sec2 hisi_hpre hisi_sas_main xhci_pci hisi_dma xhci_pci_renesas hisi_qm libsas uacce hinic dm_mirror dm_region_hash dm_log [ 182.461471] CPU: 81 PID: 7961 Comm: qemu-system-aar Kdump: loaded Tainted: G O 6.2.0-rc4+ #2 [ 182.471756] Hardware name: Huawei TaiShan 200 (Model 2280)/BC82AMDD, BIOS 2280-V2 CS V5.B221.01 12/09/2021 [ 182.481886] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 182.489084] pc : iommufd_hw_pagetable_destroy+0x30/0x130 [ 182.494648] lr : iommufd_object_destroy_user+0xb4/0x114 [ 182.500122] sp : ffff800173bebbc0 [ 182.503679] x29: ffff800173bebbc0 x28: ffff204006046300 x27: 0000000000000000 [ 182.511042] x26: 0000000000000000 x25: 0000000000000000 x24: ffff204006046900 [ 182.518400] x23: ffff00400b59e578 x22: ffff2020ad3b7f08 x21: ffff2020ad3b7f00 [ 182.525764] x20: ffff004006d78188 x19: ffff004006d78100 x18: 0000000000010000 [ 182.533129] x17: 0000000000000001 x16: 0000000000000040 x15: ffffffffffffffff [ 182.540497] x14: 00000000000007ff x13: 000000000000ffff x12: 00000000000001ca [ 182.547865] x11: 000000000001ffff x10: 0000000000000002 x9 : ffffc3f62d434714 [ 182.555233] x8 : 0000000000000238 x7 : 0000000000000000 x6 : 0000000000000001 [ 182.562601] x5 : ffffc3f62fe33000 x4 : ffffc3f62fe33b78 x3 : 0000000000000000 [ 182.569969] x2 : 0000000000000002 x1 : ffff004006d78178 x0 : 0000000000000000 [ 182.577340] Call trace: [ 182.580038] iommufd_hw_pagetable_destroy+0x30/0x130 [ 182.585264] iommufd_object_destroy_user+0xb4/0x114 [ 182.590402] iommufd_device_detach+0x13c/0x1fc [ 182.595106] vfio_iommufd_physical_unbind+0x48/0x70 [vfio] [ 182.600856] vfio_iommufd_unbind+0x24/0x3c [vfio] [ 182.605818] __vfio_device_close+0x78/0xd0 [vfio] [ 182.610771] vfio_device_fops_release+0x68/0x80 [vfio] [ 182.616154] __fput+0x78/0x22c [ 182.619452] ____fput+0x18/0x24 [ 182.622825] task_work_run+0x88/0xc0 [ 182.626622] do_exit+0x31c/0x9bc [ 182.630067] do_group_exit+0x3c/0xa0 [ 182.633848] __wake_up_parent+0x0/0x40 [ 182.637794] invoke_syscall+0x50/0x120 [ 182.641734] el0_svc_common.constprop.0+0x17c/0x1b0 [ 182.646792] do_el0_svc+0x40/0xc4 [ 182.650288] el0_svc+0x30/0xd0 [ 182.653523] el0t_64_sync_handler+0xb8/0xc0 [ 182.657878] el0t_64_sync+0x1a0/0x1a4 [ 182.661710] Code: eb00003f 54000681 f9405e60 91022274 (b9400001) [ 182.667965] ---[ end trace 0000000000000000 ]--- [ 182.677034] Fixing recursive fault but reboot is needed!
Message from syslogd@localhost at Feb 2 09:53:07 ... kernel:[ 182.405496] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP
Message from syslogd@localhost at Feb 2 09:53:07 ... kernel:[ 182.661710] Code: eb00003f 54000681 f9405e60 91022274 (b9400001)
https://github.com/Linaro/linux-kernel-uadk/commit/f3a739ec0bbae75f6e96c7447...
------------------ Original ------------------ From: "Weili Qian" <qianweili@huawei.com>; Date: Thu, Feb 2, 2023 10:04 AM To: "Zhangfei Gao"<zhangfei.gao@linaro.org>;"acc@openeuler.org"<acc@openeuler.org>;"acc"<acc@lists.linaro.org>;
Subject: [Acc] Re: Fwd: S390 testing for IOMMUFD
On 2022/11/9 10:01, Zhangfei Gao wrote: > test this with the two S390 mdev drivers
Hi Zhangfei I test iommufd with SEC, get the following error.
start qemu： echo 1 > /sys/bus/pci/devices/0000:76:00.0/sriov_numvfs echo 0000:76:00.1 > /sys/bus/pci/devices/0000:76:00.1/driver/unbind echo vfio-pci > /sys/bus/pci/devices/0000:76:00.1/driver_override echo 0000:76:00.1 > /sys/bus/pci/drivers_probe
./qemu-system-aarch64 -machine virt,gic-version=3,iommu=nested-smmuv3,iommufd=iommufd0 \ -enable-kvm -cpu host -m 4G -smp cpus=8,maxcpus=8 \ -object iommufd,id=iommufd0 \ -kernel /home/kernel/Image \ -initrd /home/mini-rootfs/rootfs.cpio.gz \ -device vfio-pci,host=0000:76:00.1,iommufd=iommufd0 \ -bios QEMU_EFI.fd \ -append "rdinit=init console=ttyAMA0 root=/dev/vda rw kpti=off" \ -net none \ -nographic
the error log： [root@localhost ~]# [ 141.122110] pci 0000:76:00.1: [19e5:a256] type 00 class 0x100000 [ 141.128377] pci 0000:76:00.1: enabling Extended Tags [ 141.134001] hisi_sec2 0000:76:00.1: Adding to iommu group 72 [ 141.140616] hisi_sec2 0000:76:00.1: enabling device (0000 -> 0002) [ 141.164904] hisi_sec2 0000:76:00.1: SMMU Opened, the iommu type = 3 [ 141.171538] hisi_sec2 0000:76:00.1: HW V2 not both use uacce sva mode and hardware crypto algs. [ 141.180765] hisi_sec2 0000:76:00.0: VF enabled, vfs_num(=1)! [ 149.623052] vfio-pci 0000:76:00.1: enabling device (0000 -> 0002) [ 149.734168] vfio_ecap_init, failed to add special caps for VF 0000:76:00.1 [ 182.349340] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 [ 182.358575] Mem abort info: [ 182.361456] ESR = 0x0000000096000004 [ 182.365348] EC = 0x25: DABT (current EL), IL = 32 bits [ 182.370791] SET = 0, FnV = 0 [ 182.373856] EA = 0, S1PTW = 0 [ 182.377109] FSC = 0x04: level 0 translation fault [ 182.382173] Data abort info: [ 182.385146] ISV = 0, ISS = 0x00000004 [ 182.389117] CM = 0, WnR = 0 [ 182.392139] user pgtable: 4k pages, 48-bit VAs, pgdp=00002020b5ae9000 [ 182.398586] [0000000000000000] pgd=0000000000000000, p4d=0000000000000000 [ 182.405496] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP [ 182.411770] Modules linked in: ip6table_nat ip6table_mangle iptable_mangle vfio_iommu_type1 vfio_pci vfio_pci_core vfio ip6table_filter ip6_tables p2pdma_provider(O) arm_spe_pmu hns_roce_hw_v2 hibmc_drm drm_vram_helper drm_ttm_helper ttm tst_bbml(O) drm_kms_helper hisi_uncore_hha_pmu hisi_uncore_ddrc_pmu hisi_uncore_l3c_pmu hisi_uncore_pmu spi_dw_mmio fuse hclge crct10dif_ce sbsa_gwdt hns3 hisi_sas_v3_hw hnae3 hisi_zip hisi_sec2 hisi_hpre hisi_sas_main xhci_pci hisi_dma xhci_pci_renesas hisi_qm libsas uacce hinic dm_mirror dm_region_hash dm_log [ 182.461471] CPU: 81 PID: 7961 Comm: qemu-system-aar Kdump: loaded Tainted: G O 6.2.0-rc4+ #2 [ 182.471756] Hardware name: Huawei TaiShan 200 (Model 2280)/BC82AMDD, BIOS 2280-V2 CS V5.B221.01 12/09/2021 [ 182.481886] pstate: 60400009 (nZCv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) [ 182.489084] pc : iommufd_hw_pagetable_destroy+0x30/0x130 [ 182.494648] lr : iommufd_object_destroy_user+0xb4/0x114 [ 182.500122] sp : ffff800173bebbc0 [ 182.503679] x29: ffff800173bebbc0 x28: ffff204006046300 x27: 0000000000000000 [ 182.511042] x26: 0000000000000000 x25: 0000000000000000 x24: ffff204006046900 [ 182.518400] x23: ffff00400b59e578 x22: ffff2020ad3b7f08 x21: ffff2020ad3b7f00 [ 182.525764] x20: ffff004006d78188 x19: ffff004006d78100 x18: 0000000000010000 [ 182.533129] x17: 0000000000000001 x16: 0000000000000040 x15: ffffffffffffffff [ 182.540497] x14: 00000000000007ff x13: 000000000000ffff x12: 00000000000001ca [ 182.547865] x11: 000000000001ffff x10: 0000000000000002 x9 : ffffc3f62d434714 [ 182.555233] x8 : 0000000000000238 x7 : 0000000000000000 x6 : 0000000000000001 [ 182.562601] x5 : ffffc3f62fe33000 x4 : ffffc3f62fe33b78 x3 : 0000000000000000 [ 182.569969] x2 : 0000000000000002 x1 : ffff004006d78178 x0 : 0000000000000000 [ 182.577340] Call trace: [ 182.580038] iommufd_hw_pagetable_destroy+0x30/0x130 [ 182.585264] iommufd_object_destroy_user+0xb4/0x114 [ 182.590402] iommufd_device_detach+0x13c/0x1fc [ 182.595106] vfio_iommufd_physical_unbind+0x48/0x70 [vfio] [ 182.600856] vfio_iommufd_unbind+0x24/0x3c [vfio] [ 182.605818] __vfio_device_close+0x78/0xd0 [vfio] [ 182.610771] vfio_device_fops_release+0x68/0x80 [vfio] [ 182.616154] __fput+0x78/0x22c [ 182.619452] ____fput+0x18/0x24 [ 182.622825] task_work_run+0x88/0xc0 [ 182.626622] do_exit+0x31c/0x9bc [ 182.630067] do_group_exit+0x3c/0xa0 [ 182.633848] __wake_up_parent+0x0/0x40 [ 182.637794] invoke_syscall+0x50/0x120 [ 182.641734] el0_svc_common.constprop.0+0x17c/0x1b0 [ 182.646792] do_el0_svc+0x40/0xc4 [ 182.650288] el0_svc+0x30/0xd0 [ 182.653523] el0t_64_sync_handler+0xb8/0xc0 [ 182.657878] el0t_64_sync+0x1a0/0x1a4 [ 182.661710] Code: eb00003f 54000681 f9405e60 91022274 (b9400001) [ 182.667965] ---[ end trace 0000000000000000 ]--- [ 182.677034] Fixing recursive fault but reboot is needed!
Message from syslogd@localhost at Feb 2 09:53:07 ... kernel:[ 182.405496] Internal error: Oops: 0000000096000004 [#1] PREEMPT SMP
Message from syslogd@localhost at Feb 2 09:53:07 ... kernel:[ 182.661710] Code: eb00003f 54000681 f9405e60 91022274 (b9400001) _______________________________________________ Acc mailing list -- acc@lists.linaro.org To unsubscribe send an email to acc-leave@lists.linaro.org
-
Weili Qian -
Zhangfei Gao -
Zhangfei Gao