- High-performance-network - mailweb.openeuler.org

[PATCH] libhns: Fixes several bugs for libhns
by Chengchang Tang 22 Nov '24

22 Nov '24

From: Xinghai Cen <cenxinghai(a)h-partners.com> Fixes several bugs for libhns: libhns: Fix memory leakage when DCA is enabled libhns: Fix the exception branch of wr_start() is not locked Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> --- ...xception-branch-of-wr_start-is-not-l.patch | 42 +++++++++++++++++++ ...x-memory-leakage-when-DCA-is-enabled.patch | 41 ++++++++++++++++++ rdma-core.spec | 10 ++++- 3 files changed, 92 insertions(+), 1 deletion(-) create mode 100644 0039-libhns-Fix-the-exception-branch-of-wr_start-is-not-l.patch create mode 100644 0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch diff --git a/0039-libhns-Fix-the-exception-branch-of-wr_start-is-not-l.patch b/0039-libhns-Fix-the-exception-branch-of-wr_start-is-not-l.patch new file mode 100644 index 0000000..a5fc588 --- /dev/null +++ b/0039-libhns-Fix-the-exception-branch-of-wr_start-is-not-l.patch @@ -0,0 +1,42 @@ +From 7acbe6f79bbfe32f207800173e5b2b0c13ef23d9 Mon Sep 17 00:00:00 2001 +From: wenglianfa <wenglianfa(a)huawei.com> +Date: Wed, 12 Jun 2024 17:11:13 +0800 +Subject: [PATCH 39/40] libhns: Fix the exception branch of wr_start() is not + locked + +driver inclusion +category: feature +bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IB66RT + +------------------------------------------------------------------ + +The provider should provide locking to ensure that ibv_wr_start() +and ibv_wr_complete()/abort() form a per-QP critical section +where no other threads can enter. + +The exception branch of wr_start() is not locked, fix it here. +Because check_qp_send () does not require lock protection, +hns_roce_spin_lock () is placed after check_qp_send (). + +Fixes: 36446a56eea5 ("libhns: Extended QP supports the new post send mechanism") +Signed-off-by: wenglianfa <wenglianfa(a)huawei.com> +Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> +--- + providers/hns/hns_roce_u_hw_v2.c | 1 + + 1 file changed, 1 insertion(+) + +diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c +index 465ef1e..e4232ea 100644 +--- a/providers/hns/hns_roce_u_hw_v2.c ++++ b/providers/hns/hns_roce_u_hw_v2.c +@@ -2930,6 +2930,7 @@ static void wr_start(struct ibv_qp_ex *ibv_qp) + + ret = check_qp_send(qp, ctx); + if (ret) { ++ hns_roce_spin_lock(&qp->sq.hr_lock); + qp->err = ret; + return; + } +-- +2.33.0 + diff --git a/0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch b/0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch new file mode 100644 index 0000000..6f5a9c6 --- /dev/null +++ b/0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch @@ -0,0 +1,41 @@ +From 1f5569fa17fc65391b504ca8118c3089e598dbc1 Mon Sep 17 00:00:00 2001 +From: wenglianfa <wenglianfa(a)huawei.com> +Date: Thu, 25 Jul 2024 11:06:01 +0800 +Subject: [PATCH 40/40] libhns: Fix memory leakage when DCA is enabled + +driver inclusion +category: feature +bugzilla: https://gitee.com/src-openeuler/rdma-core/issues/IB66RT + +------------------------------------------------------------------ + +After DCA is enabled and a QP is created, the memory block +applied for DCA is not free when the QP is destroyed. Here +fix it. + +Fixes: 41e39ab792c8 ("[BigDipperV3R9,NeZha][ROCE] libhns: Add support for attaching QP's WQE buffer") +Signed-off-by: wenglianfa <wenglianfa(a)huawei.com> +Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> +--- + providers/hns/hns_roce_u_verbs.c | 5 ++++- + 1 file changed, 4 insertions(+), 1 deletion(-) + +diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c +index e30880c..154e800 100644 +--- a/providers/hns/hns_roce_u_verbs.c ++++ b/providers/hns/hns_roce_u_verbs.c +@@ -1357,7 +1357,10 @@ static void qp_free_wqe(struct hns_roce_qp *qp) + + if (qp->rq.wqe_cnt) + free(qp->rq.wrid); +- hns_roce_free_buf(&qp->buf); ++ if (qp->dca_wqe.bufs) ++ free(qp->dca_wqe.bufs); ++ else ++ hns_roce_free_buf(&qp->buf); + } + + static int qp_alloc_wqe(struct ibv_qp_init_attr_ex *attr, +-- +2.33.0 + diff --git a/rdma-core.spec b/rdma-core.spec index fd956d5..2f4ebe5 100644 --- a/rdma-core.spec +++ b/rdma-core.spec @@ -1,6 +1,6 @@ Name: rdma-core Version: 50.0 -Release: 15 +Release: 16 Summary: RDMA core userspace libraries and daemons License: GPL-2.0-only OR BSD-2-Clause AND BSD-3-Clause Url: https://github.com/linux-rdma/rdma-core @@ -44,6 +44,8 @@ patch35: 0035-Fix-the-stride-calculation-for-MSN-PSN-area.patch patch36: 0036-add-ZTE-Dinghai-rdma-driver.patch patch37: 0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch patch38: 0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch +patch39: 0039-libhns-Fix-the-exception-branch-of-wr_start-is-not-l.patch +patch40: 0040-libhns-Fix-memory-leakage-when-DCA-is-enabled.patch BuildRequires: binutils cmake >= 2.8.11 gcc libudev-devel pkgconfig pkgconfig(libnl-3.0) BuildRequires: pkgconfig(libnl-route-3.0) systemd systemd-devel @@ -623,6 +625,12 @@ fi %doc %{_docdir}/%{name}-%{version}/70-persistent-ipoib.rules %changelog +* Thu Nov 21 2024 Xinghai Cen <cenxinghai(a)h-partners.com> - 50.0-16 +- Type: bugfix +- ID: NA +- SUG: NA +- DESC: Fixes several bugs for libhns + * Fri Nov 15 2024 Xinghai Cen <cenxinghai(a)h-partners.com> - 50.0-15 - Type: bugfix - ID: NA -- 2.33.0

1 0

[PATCH OLK-6.6 00/13] Some patches for RDMA/hns
by Chengchang Tang 20 Nov '24

20 Nov '24

From: Xinghai Cen <cenxinghai(a)h-partners.com> Some patches for RDMA/hns Chengchang Tang (6): RDMA/hns: Fix HW UAF when destroy context timeout RDMA/hns: Fix integer overflow in calc_loading_percent() RDMA/hns: Fix possible RAS when DCA is not attached RDMA/hns: Fix a meaningless loop in active_dca_pages_proc() RDMA/hns: Fix list_*_careful() not being used in pairs RDMA/hns: Use one CQ bank per context Junxian Huang (5): RDMA/hns: Fix wrong output of sysfs scc pram when configuration failed RDMA/hns: Fix concurrency between sysfs store and FW configuration of scc params RDMA/hns: Fix mixed use of u32 and __le32 in sysfs RDMA/hns: Fix dereference of noderef expression RDMA/hns: Fix "Should it be static?" warnings wenglianfa (2): RDMA/hns: Fix the modification of max_send_sge RDMA/hns: Fix RoCEE hang when multiple QP banks use EXT_SGE EXT_SGE drivers/infiniband/hw/hns/hns_roce_bond.c | 8 ++- drivers/infiniband/hw/hns/hns_roce_cq.c | 73 +++++++++++++++++-- drivers/infiniband/hw/hns/hns_roce_dca.c | 60 ++++++++++++++-- drivers/infiniband/hw/hns/hns_roce_dca.h | 2 + drivers/infiniband/hw/hns/hns_roce_debugfs.c | 10 ++- drivers/infiniband/hw/hns/hns_roce_device.h | 17 ++++- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 26 +++++-- drivers/infiniband/hw/hns/hns_roce_main.c | 20 ++++++ drivers/infiniband/hw/hns/hns_roce_mr.c | 6 +- drivers/infiniband/hw/hns/hns_roce_qp.c | 74 ++++++++++++++++---- drivers/infiniband/hw/hns/hns_roce_srq.c | 6 +- drivers/infiniband/hw/hns/hns_roce_sysfs.c | 41 +++++++---- 12 files changed, 288 insertions(+), 55 deletions(-) -- 2.33.0

1 13

[PATCH OLK-6.6] RDMA/hns: Fix different dgids mapping to the same dip_idx
by Chengchang Tang 20 Nov '24

20 Nov '24

From: Feng Fang <fangfeng4(a)huawei.com> maillist inclusion commit faa62440a5772b40bb7d78bf9e29556a82ecf153 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IB4OOG Reference: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git/commit/?id=fa… --------------------------------------------------------------------- DIP algorithm requires a one-to-one mapping between dgid and dip_idx. Currently a queue 'spare_idx' is used to store QPN of QPs that use DIP algorithm. For a new dgid, use a QPN from spare_idx as dip_idx. This method lacks a mechanism for deduplicating QPN, which may result in different dgids sharing the same dip_idx and break the one-to-one mapping requirement. This patch replaces spare_idx with xarray and introduces a refcnt of a dip_idx to indicate the number of QPs that using this dip_idx. The state machine for dip_idx management is implemented as: * The entry at an index in xarray is empty -- This indicates that the corresponding dip_idx hasn't been created. * The entry at an index in xarray is not empty but with 0 refcnt -- This indicates that the corresponding dip_idx has been created but not used as dip_idx yet. * The entry at an index in xarray is not empty and with non-0 refcnt -- This indicates that the corresponding dip_idx is being used by refcnt number of DIP QPs. Fixes: eb653eda1e91 ("RDMA/hns: Bugfix for incorrect association between dip_idx and dgid") Fixes: f91696f2f053 ("RDMA/hns: Support congestion control type selection according to the FW") Signed-off-by: Feng Fang <fangfeng4(a)huawei.com> Signed-off-by: Junxian Huang <huangjunxian6(a)hisilicon.com> Link: https://patch.msgid.link/20241112055553.3681129-1-huangjunxian6@hisilicon.c… Signed-off-by: Leon Romanovsky <leon(a)kernel.org> Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> --- drivers/infiniband/hw/hns/hns_roce_device.h | 11 +-- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 96 +++++++++++++++------ drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 2 +- drivers/infiniband/hw/hns/hns_roce_main.c | 2 - drivers/infiniband/hw/hns/hns_roce_qp.c | 8 +- 5 files changed, 75 insertions(+), 44 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index fdc1fe5e6a81..e3303cc3584a 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -570,12 +570,6 @@ struct hns_roce_bank { u32 next; /* Next ID to allocate. */ }; -struct hns_roce_idx_table { - u32 *spare_idx; - u32 head; - u32 tail; -}; - struct hns_roce_qp_table { struct hns_roce_hem_table qp_table; struct hns_roce_hem_table irrl_table; @@ -584,7 +578,7 @@ struct hns_roce_qp_table { struct mutex scc_mutex; struct hns_roce_bank bank[HNS_ROCE_QP_BANK_NUM]; struct mutex bank_mutex; - struct hns_roce_idx_table idx_table; + struct xarray dip_xa; }; struct hns_roce_cq_table { @@ -742,6 +736,7 @@ struct hns_roce_qp { bool delayed_destroy_flag; struct hns_roce_mtr_node *mtr_node; spinlock_t flush_lock; + struct hns_roce_dip *dip; }; struct hns_roce_ib_iboe { @@ -1102,8 +1097,6 @@ struct hns_roce_dev { enum hns_roce_device_state state; struct list_head qp_list; /* list of all qps on this dev */ spinlock_t qp_list_lock; /* protect qp_list */ - struct list_head dip_list; /* list of all dest ips on this dev */ - spinlock_t dip_list_lock; /* protect dip_list */ struct list_head pgdir_list; struct mutex pgdir_mutex; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 3f254ac48b42..88bd75c5743e 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -2720,20 +2720,19 @@ static void hns_roce_free_link_table(struct hns_roce_dev *hr_dev) free_link_table_buf(hr_dev, &priv->ext_llm); } -static void free_dip_list(struct hns_roce_dev *hr_dev) +static void free_dip_entry(struct hns_roce_dev *hr_dev) { struct hns_roce_dip *hr_dip; - struct hns_roce_dip *tmp; - unsigned long flags; + unsigned long idx; - spin_lock_irqsave(&hr_dev->dip_list_lock, flags); + xa_lock(&hr_dev->qp_table.dip_xa); - list_for_each_entry_safe(hr_dip, tmp, &hr_dev->dip_list, node) { - list_del(&hr_dip->node); + xa_for_each(&hr_dev->qp_table.dip_xa, idx, hr_dip) { + __xa_erase(&hr_dev->qp_table.dip_xa, hr_dip->dip_idx); kfree(hr_dip); } - spin_unlock_irqrestore(&hr_dev->dip_list_lock, flags); + xa_unlock(&hr_dev->qp_table.dip_xa); } static int hns_roce_v2_get_reset_page(struct hns_roce_dev *hr_dev) @@ -3182,7 +3181,7 @@ static void hns_roce_v2_exit(struct hns_roce_dev *hr_dev) hns_roce_v2_put_reset_page(hr_dev); if (hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP09) - free_dip_list(hr_dev); + free_dip_entry(hr_dev); } static inline void mbox_desc_init(struct hns_roce_post_mbox *mb, @@ -5042,26 +5041,49 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp, int attr_mask, return 0; } +static int alloc_dip_entry(struct xarray *dip_xa, u32 qpn) +{ + struct hns_roce_dip *hr_dip; + int ret; + + hr_dip = xa_load(dip_xa, qpn); + if (hr_dip) + return 0; + + hr_dip = kzalloc(sizeof(*hr_dip), GFP_KERNEL); + if (!hr_dip) + return -ENOMEM; + + ret = xa_err(xa_store(dip_xa, qpn, hr_dip, GFP_KERNEL)); + if (ret) + kfree(hr_dip); + + return ret; +} + static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, u32 *dip_idx) { const struct ib_global_route *grh = rdma_ah_read_grh(&attr->ah_attr); struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device); - u32 *spare_idx = hr_dev->qp_table.idx_table.spare_idx; - u32 *head = &hr_dev->qp_table.idx_table.head; - u32 *tail = &hr_dev->qp_table.idx_table.tail; + struct xarray *dip_xa = &hr_dev->qp_table.dip_xa; + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp); struct hns_roce_dip *hr_dip; - unsigned long flags; + unsigned long idx; int ret = 0; - spin_lock_irqsave(&hr_dev->dip_list_lock, flags); + ret = alloc_dip_entry(dip_xa, ibqp->qp_num); + if (ret) + return ret; - spare_idx[*tail] = ibqp->qp_num; - *tail = (*tail == hr_dev->caps.num_qps - 1) ? 0 : (*tail + 1); + xa_lock(dip_xa); - list_for_each_entry(hr_dip, &hr_dev->dip_list, node) { - if (!memcmp(grh->dgid.raw, hr_dip->dgid, GID_LEN_V2)) { + xa_for_each(dip_xa, idx, hr_dip) { + if (hr_dip->qp_cnt && + !memcmp(grh->dgid.raw, hr_dip->dgid, GID_LEN_V2)) { *dip_idx = hr_dip->dip_idx; + hr_dip->qp_cnt++; + hr_qp->dip = hr_dip; goto out; } } @@ -5069,19 +5091,24 @@ static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, /* If no dgid is found, a new dip and a mapping between dgid and * dip_idx will be created. */ - hr_dip = kzalloc(sizeof(*hr_dip), GFP_ATOMIC); - if (!hr_dip) { - ret = -ENOMEM; - goto out; + xa_for_each(dip_xa, idx, hr_dip) { + if (hr_dip->qp_cnt) + continue; + + *dip_idx = idx; + memcpy(hr_dip->dgid, grh->dgid.raw, sizeof(grh->dgid.raw)); + hr_dip->dip_idx = idx; + hr_dip->qp_cnt++; + hr_qp->dip = hr_dip; + break; } - memcpy(hr_dip->dgid, grh->dgid.raw, sizeof(grh->dgid.raw)); - hr_dip->dip_idx = *dip_idx = spare_idx[*head]; - *head = (*head == hr_dev->caps.num_qps - 1) ? 0 : (*head + 1); - list_add_tail(&hr_dip->node, &hr_dev->dip_list); + /* This should never happen. */ + if (WARN_ON_ONCE(!hr_qp->dip)) + ret = -ENOSPC; out: - spin_unlock_irqrestore(&hr_dev->dip_list_lock, flags); + xa_unlock(dip_xa); return ret; } @@ -6005,6 +6032,20 @@ static int hns_roce_v2_destroy_qp_common(struct hns_roce_dev *hr_dev, return ret; } +static void put_dip_ctx_idx(struct hns_roce_dev *hr_dev, + struct hns_roce_qp *hr_qp) +{ + struct hns_roce_dip *hr_dip = hr_qp->dip; + + xa_lock(&hr_dev->qp_table.dip_xa); + + hr_dip->qp_cnt--; + if (!hr_dip->qp_cnt) + memset(hr_dip->dgid, 0, GID_LEN_V2); + + xa_unlock(&hr_dev->qp_table.dip_xa); +} + int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) { struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device); @@ -6018,6 +6059,9 @@ int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) spin_unlock_irqrestore(&hr_qp->flush_lock, flags); flush_work(&hr_qp->flush_work.work); + if (hr_qp->cong_type == CONG_TYPE_DIP) + put_dip_ctx_idx(hr_dev, hr_qp); + ret = hns_roce_v2_destroy_qp_common(hr_dev, hr_qp, udata); if (ret) ibdev_err_ratelimited(&hr_dev->ib_dev, diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h index 201902dff611..f43444be0308 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h @@ -1366,7 +1366,7 @@ struct hns_roce_v2_priv { struct hns_roce_dip { u8 dgid[GID_LEN_V2]; u32 dip_idx; - struct list_head node; /* all dips are on a list */ + u32 qp_cnt; }; struct fmea_ram_ecc { diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c index d488c3d5986f..520112bd43d5 100644 --- a/drivers/infiniband/hw/hns/hns_roce_main.c +++ b/drivers/infiniband/hw/hns/hns_roce_main.c @@ -1250,8 +1250,6 @@ static int hns_roce_setup_hca(struct hns_roce_dev *hr_dev) INIT_LIST_HEAD(&hr_dev->qp_list); spin_lock_init(&hr_dev->qp_list_lock); - INIT_LIST_HEAD(&hr_dev->dip_list); - spin_lock_init(&hr_dev->dip_list_lock); INIT_LIST_HEAD(&hr_dev->uctx_list); mutex_init(&hr_dev->uctx_list_mutex); diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c index 5ed2647567aa..77ec0c8678d3 100644 --- a/drivers/infiniband/hw/hns/hns_roce_qp.c +++ b/drivers/infiniband/hw/hns/hns_roce_qp.c @@ -1701,14 +1701,10 @@ int hns_roce_init_qp_table(struct hns_roce_dev *hr_dev) unsigned int reserved_from_bot; unsigned int i; - qp_table->idx_table.spare_idx = kcalloc(hr_dev->caps.num_qps, - sizeof(u32), GFP_KERNEL); - if (!qp_table->idx_table.spare_idx) - return -ENOMEM; - mutex_init(&qp_table->scc_mutex); mutex_init(&qp_table->bank_mutex); xa_init(&hr_dev->qp_table_xa); + xa_init(&qp_table->dip_xa); reserved_from_bot = hr_dev->caps.reserved_qps; @@ -1733,7 +1729,7 @@ void hns_roce_cleanup_qp_table(struct hns_roce_dev *hr_dev) for (i = 0; i < HNS_ROCE_QP_BANK_NUM; i++) ida_destroy(&hr_dev->qp_table.bank[i].ida); + xa_destroy(&hr_dev->qp_table.dip_xa); mutex_destroy(&hr_dev->qp_table.bank_mutex); mutex_destroy(&hr_dev->qp_table.scc_mutex); - kfree(hr_dev->qp_table.idx_table.spare_idx); } -- 2.33.0

1 0

[PATCH OLK-6.6] RDMA/hns: Fix different dgids mapping to the same dip_idx
by Chengchang Tang 19 Nov '24

19 Nov '24

From: Feng Fang <fangfeng4(a)huawei.com> mainline inclusion from mainline-v6.12-rc2 commit faa62440a5772b40bb7d78bf9e29556a82ecf153 category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IB4OOG CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git/commit/?id=fa… --------------------------------------------------------------------- DIP algorithm requires a one-to-one mapping between dgid and dip_idx. Currently a queue 'spare_idx' is used to store QPN of QPs that use DIP algorithm. For a new dgid, use a QPN from spare_idx as dip_idx. This method lacks a mechanism for deduplicating QPN, which may result in different dgids sharing the same dip_idx and break the one-to-one mapping requirement. This patch replaces spare_idx with xarray and introduces a refcnt of a dip_idx to indicate the number of QPs that using this dip_idx. The state machine for dip_idx management is implemented as: * The entry at an index in xarray is empty -- This indicates that the corresponding dip_idx hasn't been created. * The entry at an index in xarray is not empty but with 0 refcnt -- This indicates that the corresponding dip_idx has been created but not used as dip_idx yet. * The entry at an index in xarray is not empty and with non-0 refcnt -- This indicates that the corresponding dip_idx is being used by refcnt number of DIP QPs. Fixes: eb653eda1e91 ("RDMA/hns: Bugfix for incorrect association between dip_idx and dgid") Fixes: f91696f2f053 ("RDMA/hns: Support congestion control type selection according to the FW") Signed-off-by: Feng Fang <fangfeng4(a)huawei.com> Signed-off-by: Junxian Huang <huangjunxian6(a)hisilicon.com> Link: https://patch.msgid.link/20241112055553.3681129-1-huangjunxian6@hisilicon.c… Signed-off-by: Leon Romanovsky <leon(a)kernel.org> --- drivers/infiniband/hw/hns/hns_roce_device.h | 11 +-- drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 96 +++++++++++++++------ drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 2 +- drivers/infiniband/hw/hns/hns_roce_main.c | 2 - drivers/infiniband/hw/hns/hns_roce_qp.c | 8 +- 5 files changed, 75 insertions(+), 44 deletions(-) diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h index fdc1fe5e6a81..e3303cc3584a 100644 --- a/drivers/infiniband/hw/hns/hns_roce_device.h +++ b/drivers/infiniband/hw/hns/hns_roce_device.h @@ -570,12 +570,6 @@ struct hns_roce_bank { u32 next; /* Next ID to allocate. */ }; -struct hns_roce_idx_table { - u32 *spare_idx; - u32 head; - u32 tail; -}; - struct hns_roce_qp_table { struct hns_roce_hem_table qp_table; struct hns_roce_hem_table irrl_table; @@ -584,7 +578,7 @@ struct hns_roce_qp_table { struct mutex scc_mutex; struct hns_roce_bank bank[HNS_ROCE_QP_BANK_NUM]; struct mutex bank_mutex; - struct hns_roce_idx_table idx_table; + struct xarray dip_xa; }; struct hns_roce_cq_table { @@ -742,6 +736,7 @@ struct hns_roce_qp { bool delayed_destroy_flag; struct hns_roce_mtr_node *mtr_node; spinlock_t flush_lock; + struct hns_roce_dip *dip; }; struct hns_roce_ib_iboe { @@ -1102,8 +1097,6 @@ struct hns_roce_dev { enum hns_roce_device_state state; struct list_head qp_list; /* list of all qps on this dev */ spinlock_t qp_list_lock; /* protect qp_list */ - struct list_head dip_list; /* list of all dest ips on this dev */ - spinlock_t dip_list_lock; /* protect dip_list */ struct list_head pgdir_list; struct mutex pgdir_mutex; diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c index 3f254ac48b42..88bd75c5743e 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c @@ -2720,20 +2720,19 @@ static void hns_roce_free_link_table(struct hns_roce_dev *hr_dev) free_link_table_buf(hr_dev, &priv->ext_llm); } -static void free_dip_list(struct hns_roce_dev *hr_dev) +static void free_dip_entry(struct hns_roce_dev *hr_dev) { struct hns_roce_dip *hr_dip; - struct hns_roce_dip *tmp; - unsigned long flags; + unsigned long idx; - spin_lock_irqsave(&hr_dev->dip_list_lock, flags); + xa_lock(&hr_dev->qp_table.dip_xa); - list_for_each_entry_safe(hr_dip, tmp, &hr_dev->dip_list, node) { - list_del(&hr_dip->node); + xa_for_each(&hr_dev->qp_table.dip_xa, idx, hr_dip) { + __xa_erase(&hr_dev->qp_table.dip_xa, hr_dip->dip_idx); kfree(hr_dip); } - spin_unlock_irqrestore(&hr_dev->dip_list_lock, flags); + xa_unlock(&hr_dev->qp_table.dip_xa); } static int hns_roce_v2_get_reset_page(struct hns_roce_dev *hr_dev) @@ -3182,7 +3181,7 @@ static void hns_roce_v2_exit(struct hns_roce_dev *hr_dev) hns_roce_v2_put_reset_page(hr_dev); if (hr_dev->pci_dev->revision == PCI_REVISION_ID_HIP09) - free_dip_list(hr_dev); + free_dip_entry(hr_dev); } static inline void mbox_desc_init(struct hns_roce_post_mbox *mb, @@ -5042,26 +5041,49 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp, int attr_mask, return 0; } +static int alloc_dip_entry(struct xarray *dip_xa, u32 qpn) +{ + struct hns_roce_dip *hr_dip; + int ret; + + hr_dip = xa_load(dip_xa, qpn); + if (hr_dip) + return 0; + + hr_dip = kzalloc(sizeof(*hr_dip), GFP_KERNEL); + if (!hr_dip) + return -ENOMEM; + + ret = xa_err(xa_store(dip_xa, qpn, hr_dip, GFP_KERNEL)); + if (ret) + kfree(hr_dip); + + return ret; +} + static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, u32 *dip_idx) { const struct ib_global_route *grh = rdma_ah_read_grh(&attr->ah_attr); struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device); - u32 *spare_idx = hr_dev->qp_table.idx_table.spare_idx; - u32 *head = &hr_dev->qp_table.idx_table.head; - u32 *tail = &hr_dev->qp_table.idx_table.tail; + struct xarray *dip_xa = &hr_dev->qp_table.dip_xa; + struct hns_roce_qp *hr_qp = to_hr_qp(ibqp); struct hns_roce_dip *hr_dip; - unsigned long flags; + unsigned long idx; int ret = 0; - spin_lock_irqsave(&hr_dev->dip_list_lock, flags); + ret = alloc_dip_entry(dip_xa, ibqp->qp_num); + if (ret) + return ret; - spare_idx[*tail] = ibqp->qp_num; - *tail = (*tail == hr_dev->caps.num_qps - 1) ? 0 : (*tail + 1); + xa_lock(dip_xa); - list_for_each_entry(hr_dip, &hr_dev->dip_list, node) { - if (!memcmp(grh->dgid.raw, hr_dip->dgid, GID_LEN_V2)) { + xa_for_each(dip_xa, idx, hr_dip) { + if (hr_dip->qp_cnt && + !memcmp(grh->dgid.raw, hr_dip->dgid, GID_LEN_V2)) { *dip_idx = hr_dip->dip_idx; + hr_dip->qp_cnt++; + hr_qp->dip = hr_dip; goto out; } } @@ -5069,19 +5091,24 @@ static int get_dip_ctx_idx(struct ib_qp *ibqp, const struct ib_qp_attr *attr, /* If no dgid is found, a new dip and a mapping between dgid and * dip_idx will be created. */ - hr_dip = kzalloc(sizeof(*hr_dip), GFP_ATOMIC); - if (!hr_dip) { - ret = -ENOMEM; - goto out; + xa_for_each(dip_xa, idx, hr_dip) { + if (hr_dip->qp_cnt) + continue; + + *dip_idx = idx; + memcpy(hr_dip->dgid, grh->dgid.raw, sizeof(grh->dgid.raw)); + hr_dip->dip_idx = idx; + hr_dip->qp_cnt++; + hr_qp->dip = hr_dip; + break; } - memcpy(hr_dip->dgid, grh->dgid.raw, sizeof(grh->dgid.raw)); - hr_dip->dip_idx = *dip_idx = spare_idx[*head]; - *head = (*head == hr_dev->caps.num_qps - 1) ? 0 : (*head + 1); - list_add_tail(&hr_dip->node, &hr_dev->dip_list); + /* This should never happen. */ + if (WARN_ON_ONCE(!hr_qp->dip)) + ret = -ENOSPC; out: - spin_unlock_irqrestore(&hr_dev->dip_list_lock, flags); + xa_unlock(dip_xa); return ret; } @@ -6005,6 +6032,20 @@ static int hns_roce_v2_destroy_qp_common(struct hns_roce_dev *hr_dev, return ret; } +static void put_dip_ctx_idx(struct hns_roce_dev *hr_dev, + struct hns_roce_qp *hr_qp) +{ + struct hns_roce_dip *hr_dip = hr_qp->dip; + + xa_lock(&hr_dev->qp_table.dip_xa); + + hr_dip->qp_cnt--; + if (!hr_dip->qp_cnt) + memset(hr_dip->dgid, 0, GID_LEN_V2); + + xa_unlock(&hr_dev->qp_table.dip_xa); +} + int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) { struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device); @@ -6018,6 +6059,9 @@ int hns_roce_v2_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata) spin_unlock_irqrestore(&hr_qp->flush_lock, flags); flush_work(&hr_qp->flush_work.work); + if (hr_qp->cong_type == CONG_TYPE_DIP) + put_dip_ctx_idx(hr_dev, hr_qp); + ret = hns_roce_v2_destroy_qp_common(hr_dev, hr_qp, udata); if (ret) ibdev_err_ratelimited(&hr_dev->ib_dev, diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h index 201902dff611..f43444be0308 100644 --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h @@ -1366,7 +1366,7 @@ struct hns_roce_v2_priv { struct hns_roce_dip { u8 dgid[GID_LEN_V2]; u32 dip_idx; - struct list_head node; /* all dips are on a list */ + u32 qp_cnt; }; struct fmea_ram_ecc { diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c index d488c3d5986f..520112bd43d5 100644 --- a/drivers/infiniband/hw/hns/hns_roce_main.c +++ b/drivers/infiniband/hw/hns/hns_roce_main.c @@ -1250,8 +1250,6 @@ static int hns_roce_setup_hca(struct hns_roce_dev *hr_dev) INIT_LIST_HEAD(&hr_dev->qp_list); spin_lock_init(&hr_dev->qp_list_lock); - INIT_LIST_HEAD(&hr_dev->dip_list); - spin_lock_init(&hr_dev->dip_list_lock); INIT_LIST_HEAD(&hr_dev->uctx_list); mutex_init(&hr_dev->uctx_list_mutex); diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c index 5ed2647567aa..77ec0c8678d3 100644 --- a/drivers/infiniband/hw/hns/hns_roce_qp.c +++ b/drivers/infiniband/hw/hns/hns_roce_qp.c @@ -1701,14 +1701,10 @@ int hns_roce_init_qp_table(struct hns_roce_dev *hr_dev) unsigned int reserved_from_bot; unsigned int i; - qp_table->idx_table.spare_idx = kcalloc(hr_dev->caps.num_qps, - sizeof(u32), GFP_KERNEL); - if (!qp_table->idx_table.spare_idx) - return -ENOMEM; - mutex_init(&qp_table->scc_mutex); mutex_init(&qp_table->bank_mutex); xa_init(&hr_dev->qp_table_xa); + xa_init(&qp_table->dip_xa); reserved_from_bot = hr_dev->caps.reserved_qps; @@ -1733,7 +1729,7 @@ void hns_roce_cleanup_qp_table(struct hns_roce_dev *hr_dev) for (i = 0; i < HNS_ROCE_QP_BANK_NUM; i++) ida_destroy(&hr_dev->qp_table.bank[i].ida); + xa_destroy(&hr_dev->qp_table.dip_xa); mutex_destroy(&hr_dev->qp_table.bank_mutex); mutex_destroy(&hr_dev->qp_table.scc_mutex); - kfree(hr_dev->qp_table.idx_table.spare_idx); } -- 2.33.0

1 0

高性能网络sig会议
by openEuler conference 18 Nov '24

18 Nov '24

您好！ sig-high-performance-network 邀请您参加 2024-11-18 14:30 召开的WeLink会议(自动录制) 会议主题：高性能网络sig会议会议链接：https://meeting.huaweicloud.com:36443/#/j/982559975 会议纪要：https://etherpad.openeuler.org/p/sig-high-performance-network-meetings 更多资讯尽在：https://www.openeuler.org/zh/ Hello! sig-high-performance-network invites you to attend the WeLink conference(auto recording) will be held at 2024-11-18 14:30, The subject of the conference is 高性能网络sig会议, You can join the meeting at https://meeting.huaweicloud.com:36443/#/j/982559975. Add topics at https://etherpad.openeuler.org/p/sig-high-performance-network-meetings. More information: https://www.openeuler.org/en/

1 0

[PATCH] libhns: Two bugfixes in post_send flow
by Chengchang Tang 15 Nov '24

15 Nov '24

From: Xinghai Cen <cenxinghai(a)h-partners.com> Two bugfixes in post_send flow: libhns: Fix out-of-order issue of requester when setting FENCE libhns: Fix reference to uninitialized cq pointer Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> --- ...f-order-issue-of-requester-when-sett.patch | 50 ++++++++++++++ ...eference-to-uninitialized-cq-pointer.patch | 68 +++++++++++++++++++ rdma-core.spec | 10 ++- 3 files changed, 127 insertions(+), 1 deletion(-) create mode 100644 0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch create mode 100644 0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch diff --git a/0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch b/0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch new file mode 100644 index 0000000..0996c7f --- /dev/null +++ b/0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch @@ -0,0 +1,50 @@ +From 23c00575e9dc2b61f2ac429f9d039cb7e2e6deb2 Mon Sep 17 00:00:00 2001 +From: Junxian Huang <huangjunxian6(a)hisilicon.com> +Date: Fri, 8 Nov 2024 17:04:09 +0800 +Subject: [PATCH] libhns: Fix out-of-order issue of requester when setting + FENCE + +mainline inclusion +from mainline-master +commit c4119911c212aaa552c9cb928fba0a696640c9b5 +category: bugfix +bugzilla: https://gitee.com/openeuler/kernel/issues/IB3ZHQ +CVE: NA +Reference: https://github.com/linux-rdma/rdma-core/pull/1513/commits/c4119911c212aaa55… + +---------------------------------------------------------------------- + +The FENCE indicator in hns WQE doesn't ensure that response data from +a previous Read/Atomic operation has been written to the requester's +memory before the subsequent Send/Write operation is processed. This +may result in the subsequent Send/Write operation accessing the original +data in memory instead of the expected response data. + +Unlike FENCE, the SO (Strong Order) indicator blocks the subsequent +operation until the previous response data is written to memory and a +bresp is returned. Set the SO indicator instead of FENCE to maintain +strict order. + +Fixes: cbdf5e32a855 ("libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE") +Signed-off-by: Junxian Huang <huangjunxian6(a)hisilicon.com> +Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> +--- + providers/hns/hns_roce_u_hw_v2.c | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c +index 9371150..2debcb3 100644 +--- a/providers/hns/hns_roce_u_hw_v2.c ++++ b/providers/hns/hns_roce_u_hw_v2.c +@@ -1527,7 +1527,7 @@ static int set_rc_wqe(void *wqe, struct hns_roce_qp *qp, struct ibv_send_wr *wr, + + hr_reg_write_bool(wqe, RCWQE_CQE, + !!(wr->send_flags & IBV_SEND_SIGNALED)); +- hr_reg_write_bool(wqe, RCWQE_FENCE, ++ hr_reg_write_bool(wqe, RCWQE_SO, + !!(wr->send_flags & IBV_SEND_FENCE)); + hr_reg_write_bool(wqe, RCWQE_SE, + !!(wr->send_flags & IBV_SEND_SOLICITED)); +-- +2.33.0 + diff --git a/0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch b/0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch new file mode 100644 index 0000000..e45d61a --- /dev/null +++ b/0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch @@ -0,0 +1,68 @@ +From e941cc407d89595d1affd5211e1daf34786c7641 Mon Sep 17 00:00:00 2001 +From: Chengchang Tang <tangchengchang(a)huawei.com> +Date: Fri, 8 Nov 2024 17:04:08 +0800 +Subject: [PATCH] libhns: Fix reference to uninitialized cq pointer +MIME-Version: 1.0 +Content-Type: text/plain; charset=UTF-8 +Content-Transfer-Encoding: 8bit + +mainline inclusion +from mainline-master +commit 18e3117cdd161a3f40b8a917f24cfb5227a1d75a +category: bugfix +bugzilla: https://gitee.com/openeuler/kernel/issues/IB3ZHQ +CVE: NA +Reference: https://github.com/linux-rdma/rdma-core/pull/1513/commits/18e3117cdd161a3f4… + +---------------------------------------------------------------------- + +For QPs which do not have an SQ, such as XRC TGT，the send_cq +pointer will not be initailized. Since the supported max_gs +will be 0 in this case, check it and return before referencing +the send_cq pointer. + +Fixes: cbdf5e32a855 ("libhns: Reimplement verbs of post_send and post_recv for hip08 RoCE") +Signed-off-by: Chengchang Tang <tangchengchang(a)huawei.com> +Signed-off-by: Junxian Huang <huangjunxian6(a)hisilicon.com> +Signed-off-by: Xinghai Cen <cenxinghai(a)h-partners.com> +--- + providers/hns/hns_roce_u_hw_v2.c | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c +index 2debcb3..465ef1e 100644 +--- a/providers/hns/hns_roce_u_hw_v2.c ++++ b/providers/hns/hns_roce_u_hw_v2.c +@@ -1579,7 +1579,7 @@ int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr, + struct hns_roce_context *ctx = to_hr_ctx(ibvqp->context); + struct hns_roce_qp *qp = to_hr_qp(ibvqp); + struct hns_roce_sge_info sge_info = {}; +- struct hns_roce_rc_sq_wqe *wqe; ++ struct hns_roce_rc_sq_wqe *wqe = NULL; + struct ibv_qp_attr attr = {}; + unsigned int wqe_idx, nreq; + int ret; +@@ -1595,15 +1595,15 @@ int hns_roce_u_v2_post_send(struct ibv_qp *ibvqp, struct ibv_send_wr *wr, + sge_info.start_idx = qp->next_sge; /* start index of extend sge */ + + for (nreq = 0; wr; ++nreq, wr = wr->next) { +- if (hns_roce_v2_wq_overflow(&qp->sq, nreq, +- to_hr_cq(qp->verbs_qp.qp.send_cq))) { +- ret = ENOMEM; ++ if (wr->num_sge > (int)qp->sq.max_gs) { ++ ret = qp->sq.max_gs > 0 ? EINVAL : EOPNOTSUPP; + *bad_wr = wr; + goto out; + } + +- if (wr->num_sge > qp->sq.max_gs) { +- ret = EINVAL; ++ if (hns_roce_v2_wq_overflow(&qp->sq, nreq, ++ to_hr_cq(qp->verbs_qp.qp.send_cq))) { ++ ret = ENOMEM; + *bad_wr = wr; + goto out; + } +-- +2.33.0 + diff --git a/rdma-core.spec b/rdma-core.spec index 21a690e..4135ada 100644 --- a/rdma-core.spec +++ b/rdma-core.spec @@ -1,6 +1,6 @@ Name: rdma-core Version: 50.0 -Release: 13 +Release: 14 Summary: RDMA core userspace libraries and daemons License: GPLv2 or BSD Url: https://github.com/linux-rdma/rdma-core @@ -42,6 +42,8 @@ patch33: 0033-libhns-Fix-missing-flag-when-creating-qp-by-hnsdv_cr.patch patch34: 0034-librdmacm-Fix-an-overflow-bug-in-qsort-comparison-function.patch patch35: 0035-Fix-the-stride-calculation-for-MSN-PSN-area.patch patch36: 0036-add-ZTE-Dinghai-rdma-driver.patch +patch37: 0037-libhns-Fix-out-of-order-issue-of-requester-when-sett.patch +patch38: 0038-libhns-Fix-reference-to-uninitialized-cq-pointer.patch BuildRequires: binutils cmake >= 2.8.11 gcc libudev-devel pkgconfig pkgconfig(libnl-3.0) BuildRequires: pkgconfig(libnl-route-3.0) systemd systemd-devel @@ -627,6 +629,12 @@ fi %doc %{_docdir}/%{name}-%{version}/tag_matching.md %doc %{_docdir}/%{name}-%{version}/70-persistent-ipoib.rules +* Fri Nov 15 2024 Xinghai Cen <cenxinghai(a)h-partners.com> - 50.0-14 +- Type: bugfix +- ID: NA +- SUG: NA +- DESC: Two bugfixes in post_send flow + %changelog * Sat Aug 31 2024 Li Fuyan <li.fuyan(a)zte.com.cn> - 50.0-13 - Type: requirement -- 2.33.0

1 0

高性能网络sig会议
by openEuler conference 15 Nov '24

15 Nov '24

您好！ sig-high-performance-network 邀请您参加 2024-11-15 14:00 召开的WeLink会议(自动录制) 会议主题：高性能网络sig会议会议链接：https://meeting.huaweicloud.com:36443/#/j/965651261 会议纪要：https://etherpad.openeuler.org/p/sig-high-performance-network-meetings 更多资讯尽在：https://www.openeuler.org/zh/ Hello! sig-high-performance-network invites you to attend the WeLink conference(auto recording) will be held at 2024-11-15 14:00, The subject of the conference is 高性能网络sig会议, You can join the meeting at https://meeting.huaweicloud.com:36443/#/j/965651261. Add topics at https://etherpad.openeuler.org/p/sig-high-performance-network-meetings. More information: https://www.openeuler.org/en/

1 0

[PATCH rdma-core 0/2] libhns: Two bugfixes in post_send flow
by Junxian Huang 08 Nov '24

08 Nov '24

Two hns bugfixes in post_send flow Chengchang Tang (1): libhns: Fix reference to uninitialized cq pointer Junxian Huang (1): libhns: Fix out-of-order issue of requester when setting FENCE providers/hns/hns_roce_u_hw_v2.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) -- 2.33.0

1 2

高性能网络sig例会
by openEuler conference 06 Nov '24

06 Nov '24

您好！ sig-high-performance-network 邀请您参加 2024-11-08 11:00 召开的WeLink会议(自动录制) 会议主题：高性能网络sig例会会议链接：https://meeting.huaweicloud.com:36443/#/j/960933891 会议纪要：https://etherpad.openeuler.org/p/sig-high-performance-network-meetings 更多资讯尽在：https://www.openeuler.org/zh/ Hello! sig-high-performance-network invites you to attend the WeLink conference(auto recording) will be held at 2024-11-08 11:00, The subject of the conference is 高性能网络sig例会, You can join the meeting at https://meeting.huaweicloud.com:36443/#/j/960933891. Add topics at https://etherpad.openeuler.org/p/sig-high-performance-network-meetings. More information: https://www.openeuler.org/en/

1 0

高性能网络sig会议
by openEuler conference 23 Oct '24

23 Oct '24

您好！ sig-high-performance-network 邀请您参加 2024-10-25 11:00 召开的WeLink会议会议主题：高性能网络sig会议会议链接：https://meeting.huaweicloud.com:36443/#/j/982220702 会议纪要：https://etherpad.openeuler.org/p/sig-high-performance-network-meetings 更多资讯尽在：https://www.openeuler.org/zh/ Hello! sig-high-performance-network invites you to attend the WeLink conference will be held at 2024-10-25 11:00, The subject of the conference is 高性能网络sig会议, You can join the meeting at https://meeting.huaweicloud.com:36443/#/j/982220702. Add topics at https://etherpad.openeuler.org/p/sig-high-performance-network-meetings. More information: https://www.openeuler.org/en/

1 0