Fw:[PATCH RFC 00/12] IOMMUFD Generic interface - Acc

19 Mar 2022


      ------------------&nbsp;Original&nbsp;------------------
From:                                                                                                                        "Jason Gunthorpe"                                                                                    <iommu@lists.linux-foundation.org&gt;;
Date:&nbsp;Sat, Mar 19, 2022 01:27 AM
To:&nbsp;
Cc:&nbsp;"Jean-Philippe Brucker"<jean-philippe@linaro.org&gt;;"Chaitanya Kulkarni"<chaitanyak@nvidia.com&gt;;"kvm"<kvm@vger.kernel.org&gt;;"Michael S. Tsirkin"<mst@redhat.com&gt;;"Jason Wang"<jasowang@redhat.com&gt;;"Cornelia Huck"<cohuck@redhat.com&gt;;"Niklas Schnelle"<schnelle@linux.ibm.com&gt;;"iommu"<iommu@lists.linux-foundation.org&gt;;"Daniel Jordan"<daniel.m.jordan@oracle.com&gt;;"Kevin Tian"<kevin.tian@intel.com&gt;;"Alex Williamson"<alex.williamson@redhat.com&gt;;"Joao Martins"<joao.m.martins@oracle.com&gt;;"David Gibson"<david@gibson.dropbear.id.au&gt;;
Subject:&nbsp;[PATCH RFC 00/12] IOMMUFD Generic interface
iommufd is the user API to control the IOMMU subsystem as it relates to
managing IO page tables that point at user space memory.
It takes over from drivers/vfio/vfio_iommu_type1.c (aka the VFIO
container) which is the VFIO specific interface for a similar idea.
We see a broad need for extended features, some being highly IOMMU device
specific:
&nbsp;- Binding iommu_domain's to PASID/SSID
&nbsp;- Userspace page tables, for ARM, x86 and S390
&nbsp;- Kernel bypass'd invalidation of user page tables
&nbsp;- Re-use of the KVM page table in the IOMMU
&nbsp;- Dirty page tracking in the IOMMU
&nbsp;- Runtime Increase/Decrease of IOPTE size
&nbsp;- PRI support with faults resolved in userspace
As well as a need to access these features beyond just VFIO, VDPA for
instance, but other classes of accelerator HW are touching on these areas
now too.
The v1 series proposed re-using the VFIO type 1 data structure, however it
was suggested that if we are doing this big update then we should also
come with a data structure that solves the limitations that VFIO type1
has. Notably this addresses:
&nbsp;- Multiple IOAS/'containers' and multiple domains inside a single FD
&nbsp;- Single-pin operation no matter how many domains and containers use
&nbsp;&nbsp; a page
&nbsp;- A fine grained locking scheme supporting user managed concurrency for
&nbsp;&nbsp; multi-threaded map/unmap
&nbsp;- A pre-registration mechanism to optimize vIOMMU use cases by
&nbsp;&nbsp; pre-pinning pages
&nbsp;- Extended ioctl API that can manage these new objects and exposes
&nbsp;&nbsp; domains directly to user space
&nbsp;- domains are sharable between subsystems, eg VFIO and VDPA
The bulk of this code is a new data structure design to track how the
IOVAs are mapped to PFNs.
iommufd intends to be general and consumable by any driver that wants to
DMA to userspace. From a driver perspective it can largely be dropped in
in-place of iommu_attach_device() and provides a uniform full feature set
to all consumers.
As this is a larger project this series is the first step. This series
provides the iommfd "generic interface" which is designed to be suitable
for applications like DPDK and VMM flows that are not optimized to
specific HW scenarios. It is close to being a drop in replacement for the
existing VFIO type 1.
This is part two of three for an initial sequence:
&nbsp;- Move IOMMU Group security into the iommu layer
&nbsp;&nbsp; https://lore.kernel.org/linux-iommu/20220218005521.172832-1-baolu.lu@linux.i...
&nbsp;* Generic IOMMUFD implementation
&nbsp;- VFIO ability to consume IOMMUFD
&nbsp;&nbsp; An early exploration of this is available here:
&nbsp;&nbsp;&nbsp; https://github.com/luxis1999/iommufd/commits/iommufd-v5.17-rc6
Various parts of the above extended features are in WIP stages currently
to define how their IOCTL interface should work.
At this point, using the draft VFIO series, unmodified qemu has been
tested to operate using iommufd on x86 and ARM systems.
Several people have contributed directly to this work: Eric Auger, Kevin
Tian, Lu Baolu, Nicolin Chen, Yi L Liu. Many more have participated in the
discussions that lead here, and provided ideas. Thanks to all!
This is on github: https://github.com/jgunthorpe/linux/commits/iommufd
# S390 in-kernel page table walker
Cc: Niklas Schnelle <schnelle@linux.ibm.com&gt;
Cc: Matthew Rosato <mjrosato@linux.ibm.com&gt;
# AMD Dirty page tracking
Cc: Joao Martins <joao.m.martins@oracle.com&gt;
# ARM SMMU Dirty page tracking
Cc: Keqian Zhu <zhukeqian1@huawei.com&gt;
Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com&gt;
# ARM SMMU nesting
Cc: Eric Auger <eric.auger@redhat.com&gt;
Cc: Jean-Philippe Brucker <jean-philippe@linaro.org&gt;
# Map/unmap performance
Cc: Daniel Jordan <daniel.m.jordan@oracle.com&gt;
# VDPA
Cc: "Michael S. Tsirkin" <mst@redhat.com&gt;
Cc: Jason Wang <jasowang@redhat.com&gt;
# Power
Cc: David Gibson <david@gibson.dropbear.id.au&gt;
# vfio
Cc: Alex Williamson <alex.williamson@redhat.com&gt;
Cc: Cornelia Huck <cohuck@redhat.com&gt;
Cc: kvm@vger.kernel.org
# iommu
Cc: iommu@lists.linux-foundation.org
# Collaborators
Cc: "Chaitanya Kulkarni" <chaitanyak@nvidia.com&gt;
Cc: Nicolin Chen <nicolinc@nvidia.com&gt;
Cc: Lu Baolu <baolu.lu@linux.intel.com&gt;
Cc: Kevin Tian <kevin.tian@intel.com&gt;
Cc: Yi Liu <yi.l.liu@intel.com&gt;
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com&gt;
Jason Gunthorpe (11):
&nbsp; interval-tree: Add a utility to iterate over spans in an interval tree
&nbsp; iommufd: File descriptor, context, kconfig and makefiles
&nbsp; kernel/user: Allow user::locked_vm to be usable for iommufd
&nbsp; iommufd: PFN handling for iopt_pages
&nbsp; iommufd: Algorithms for PFN storage
&nbsp; iommufd: Data structure to provide IOVA to PFN mapping
&nbsp; iommufd: IOCTLs for the io_pagetable
&nbsp; iommufd: Add a HW pagetable object
&nbsp; iommufd: Add kAPI toward external drivers
&nbsp; iommufd: vfio container FD ioctl compatibility
&nbsp; iommufd: Add a selftest
Kevin Tian (1):
&nbsp; iommufd: Overview documentation
&nbsp;Documentation/userspace-api/index.rst&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 1 +
&nbsp;.../userspace-api/ioctl/ioctl-number.rst&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 1 +
&nbsp;Documentation/userspace-api/iommufd.rst&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 224 +++
&nbsp;MAINTAINERS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 10 +
&nbsp;drivers/iommu/Kconfig&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 1 +
&nbsp;drivers/iommu/Makefile&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 2 +-
&nbsp;drivers/iommu/iommufd/Kconfig&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 22 +
&nbsp;drivers/iommu/iommufd/Makefile&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 13 +
&nbsp;drivers/iommu/iommufd/device.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 274 ++++
&nbsp;drivers/iommu/iommufd/hw_pagetable.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 142 ++
&nbsp;drivers/iommu/iommufd/io_pagetable.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 890 +++++++++++
&nbsp;drivers/iommu/iommufd/io_pagetable.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 170 +++
&nbsp;drivers/iommu/iommufd/ioas.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 252 ++++
&nbsp;drivers/iommu/iommufd/iommufd_private.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 231 +++
&nbsp;drivers/iommu/iommufd/iommufd_test.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 65 +
&nbsp;drivers/iommu/iommufd/main.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 346 +++++
&nbsp;drivers/iommu/iommufd/pages.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 1321 +++++++++++++++++
&nbsp;drivers/iommu/iommufd/selftest.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 495 ++++++
&nbsp;drivers/iommu/iommufd/vfio_compat.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 401 +++++
&nbsp;include/linux/interval_tree.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 41 +
&nbsp;include/linux/iommufd.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 50 +
&nbsp;include/linux/sched/user.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 2 +-
&nbsp;include/uapi/linux/iommufd.h&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp; 223 +++
&nbsp;kernel/user.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 1 +
&nbsp;lib/interval_tree.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 98 ++
&nbsp;tools/testing/selftests/Makefile&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 1 +
&nbsp;tools/testing/selftests/iommu/.gitignore&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 2 +
&nbsp;tools/testing/selftests/iommu/Makefile&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp; 11 +
&nbsp;tools/testing/selftests/iommu/config&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; |&nbsp;&nbsp;&nbsp; 2 +
&nbsp;tools/testing/selftests/iommu/iommufd.c&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; | 1225 +++++++++++++++
&nbsp;30 files changed, 6515 insertions(+), 2 deletions(-)
&nbsp;create mode 100644 Documentation/userspace-api/iommufd.rst
&nbsp;create mode 100644 drivers/iommu/iommufd/Kconfig
&nbsp;create mode 100644 drivers/iommu/iommufd/Makefile
&nbsp;create mode 100644 drivers/iommu/iommufd/device.c
&nbsp;create mode 100644 drivers/iommu/iommufd/hw_pagetable.c
&nbsp;create mode 100644 drivers/iommu/iommufd/io_pagetable.c
&nbsp;create mode 100644 drivers/iommu/iommufd/io_pagetable.h
&nbsp;create mode 100644 drivers/iommu/iommufd/ioas.c
&nbsp;create mode 100644 drivers/iommu/iommufd/iommufd_private.h
&nbsp;create mode 100644 drivers/iommu/iommufd/iommufd_test.h
&nbsp;create mode 100644 drivers/iommu/iommufd/main.c
&nbsp;create mode 100644 drivers/iommu/iommufd/pages.c
&nbsp;create mode 100644 drivers/iommu/iommufd/selftest.c
&nbsp;create mode 100644 drivers/iommu/iommufd/vfio_compat.c
&nbsp;create mode 100644 include/linux/iommufd.h
&nbsp;create mode 100644 include/uapi/linux/iommufd.h
&nbsp;create mode 100644 tools/testing/selftests/iommu/.gitignore
&nbsp;create mode 100644 tools/testing/selftests/iommu/Makefile
&nbsp;create mode 100644 tools/testing/selftests/iommu/config
&nbsp;create mode 100644 tools/testing/selftests/iommu/iommufd.c
base-commit: d1c716ed82a6bf4c35ba7be3741b9362e84cd722
-- 
2.35.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu