[Description] The ROH module driver consists of the ROH Core and ROH DRV modules, which work with hardware to implement communication between nodes through HCCS packets. ROH Core is a protocol stack of the ROH architecture. It provides related services for upper layers by invoking operation interfaces provided by the ROH DRV. The ROH DRV implements the lower layer functions of the ROH featureand provides a series of interfaces for operating hardware for the ROH Core.
This patch supports basic ROH functions, such as: sysfs file node query, abnormal interrupt handling, reset capability and dfx information query.
RDMA/hns supports ROH mode, mainly adapted to the device id of ROH, and the different capabilities and features of RDMA/hns in ROH mode.
[Testing] kernel options: CONFIG_ROH=m CONFIG_ROH_HNS=m
Test passed with below step: 1. Using a hardware environment that supports ROH, insmod net/hns, RDMA/hns and ROH related drivers: insmod hnae3.ko insmod hclge.ko insmod hns3.ko insmod roh_core.ko insmod hns-roh-v1.ko insmod hns-roce-hw-v2.ko 2. Check whether ROH generates the corresponding sysfs node: ls /sys/class/roh/hns3_0/ 3. Check whether the abnormal interrupt information of roh is correct. The down or up of the network device link corresponding to roh will cause the roh abnormal interrupt count to increase. cat /proc/interrupts | grep roh 4. Configure the network port ip and check whether the ip/mac has the correct mapping relationship. 5. Query eid to check whether it complies with ip conversion rules: cat /sys/class/roh/hns3_0/node_eid 6. Use ping to check Ethernet communication is normal. 7. Use perftest to check RDMA communication is normal. 8. Use the "ethtool --reset eth1 all" command to trigger a reset.
Ke Chen (9): roh/core: Add roh device sysfs node roh/hns3: Add support for roh abnormal interruption roh/core: Add support for inetaddr notifier roh/hns3: Add support for roh reset roh/hns3: Add support for roh dfx(debugfs) RDMA/hns: Add new device ID RDMA/hns: Add ROH basic configuration and check RDMA/hns: Support RDMA_CM in ROH mode RDMA/hns: Pass mac type to user driver for ROH mode
drivers/infiniband/hw/hns/hns_roce_device.h | 1 + drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 35 ++ drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 1 + drivers/infiniband/hw/hns/hns_roce_main.c | 12 +- drivers/roh/core/Makefile | 2 +- drivers/roh/core/core.c | 205 ++++++++++ drivers/roh/core/core.h | 24 ++ drivers/roh/core/core_priv.h | 9 + drivers/roh/core/sysfs.c | 270 +++++++++++++ drivers/roh/hw/hns3/Makefile | 2 + drivers/roh/hw/hns3/hns3_cmdq.c | 135 +++++++ drivers/roh/hw/hns3/hns3_cmdq.h | 46 +++ drivers/roh/hw/hns3/hns3_common.h | 32 ++ drivers/roh/hw/hns3/hns3_device.h | 40 ++ drivers/roh/hw/hns3/hns3_intr.c | 136 +++++++ drivers/roh/hw/hns3/hns3_intr.h | 13 + drivers/roh/hw/hns3/hns3_main.c | 406 +++++++++++++++++++- drivers/roh/hw/hns3/hns3_verbs.c | 270 +++++++++++++ drivers/roh/hw/hns3/hns3_verbs.h | 16 + include/uapi/rdma/hns-abi.h | 2 + 20 files changed, 1651 insertions(+), 6 deletions(-) create mode 100644 drivers/roh/core/sysfs.c create mode 100644 drivers/roh/hw/hns3/hns3_device.h create mode 100644 drivers/roh/hw/hns3/hns3_intr.c create mode 100644 drivers/roh/hw/hns3/hns3_intr.h create mode 100644 drivers/roh/hw/hns3/hns3_verbs.c create mode 100644 drivers/roh/hw/hns3/hns3_verbs.h