Two performance improvements. Chengchang Tang (1): libhns: Optimize memory allocation to reduce page faults Junxian Huang (1): libhns: Inline set_extend_atomic_seg() to improve performance providers/hns/hns_roce_u_buf.c | 2 +- providers/hns/hns_roce_u_hw_v2.c | 6 ++++-- providers/hns/hns_roce_u_verbs.c | 5 ++++- 3 files changed, 9 insertions(+), 4 deletions(-) -- 2.33.0
From: Chengchang Tang <tangchengchang@huawei.com> We observed page faults during post_send() in flame graph, which increased the latency of post_send() and degraded performance. Add MAP_POPULATE flag to mmap() calls in hns_roce_alloc_buf() to pre-fault pages at allocation time, avoiding page faults during data path operations. Replace calloc() with malloc()+memset() for wrid buffers in SRQ and QP allocation, as explicit memset after malloc allows the kernel to handle page faults in bulk rather than on first write access, improving performance for RDMA workloads. Signed-off-by: Chengchang Tang <tangchengchang@huawei.com> Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> --- providers/hns/hns_roce_u_buf.c | 2 +- providers/hns/hns_roce_u_verbs.c | 5 ++++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/providers/hns/hns_roce_u_buf.c b/providers/hns/hns_roce_u_buf.c index 471dd9ca5..b0992c1a0 100644 --- a/providers/hns/hns_roce_u_buf.c +++ b/providers/hns/hns_roce_u_buf.c @@ -43,7 +43,7 @@ int hns_roce_alloc_buf(struct hns_roce_buf *buf, unsigned int size, buf->length = align(size, page_size); buf->buf = mmap(NULL, buf->length, PROT_READ | PROT_WRITE, - MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + MAP_PRIVATE | MAP_ANONYMOUS | MAP_POPULATE, -1, 0); if (buf->buf == MAP_FAILED) return errno; diff --git a/providers/hns/hns_roce_u_verbs.c b/providers/hns/hns_roce_u_verbs.c index 7a861ba32..0cf7dc033 100644 --- a/providers/hns/hns_roce_u_verbs.c +++ b/providers/hns/hns_roce_u_verbs.c @@ -720,11 +720,12 @@ static int alloc_srq_buf(struct hns_roce_srq *srq) if (ret) goto err_idx_que; - srq->wrid = calloc(srq->wqe_cnt, sizeof(*srq->wrid)); + srq->wrid = malloc(srq->wqe_cnt * sizeof(*srq->wrid)); if (!srq->wrid) { ret = -ENOMEM; goto err_wqe_buf; } + memset(srq->wrid, 0, srq->wqe_cnt * sizeof(*srq->wrid)); return 0; @@ -1180,11 +1181,13 @@ static int qp_alloc_wqe(struct ibv_qp_cap *cap, struct hns_roce_qp *qp, qp->sq.wrid = malloc(qp->sq.wqe_cnt * sizeof(uint64_t)); if (!qp->sq.wrid) return -ENOMEM; + memset(qp->sq.wrid, 0, qp->sq.wqe_cnt * sizeof(uint64_t)); if (qp->rq.wqe_cnt) { qp->rq.wrid = malloc(qp->rq.wqe_cnt * sizeof(uint64_t)); if (!qp->rq.wrid) goto err_alloc; + memset(qp->rq.wrid, 0, qp->rq.wqe_cnt * sizeof(uint64_t)); } if (qp->rq_rinl_buf.wqe_cnt) { -- 2.33.0
Commit ca944a322f9b ("libhns: Fix immediately error sign type in data path.") changed return values from negative errno to positive errno in the data path. While semantically correct, this changed GCC's inline cost estimate for set_extend_atomic_seg(), causing it to no longer be inlined into hns_roce_u_v2_post_send() and resulting in performance loss. The GCC version we used is 10.3.1. Add inline explicitly to restores GCC's inlining decision. Signed-off-by: Junxian Huang <huangjunxian6@hisilicon.com> --- providers/hns/hns_roce_u_hw_v2.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/providers/hns/hns_roce_u_hw_v2.c b/providers/hns/hns_roce_u_hw_v2.c index 9f34fb4b3..20b95b8e5 100644 --- a/providers/hns/hns_roce_u_hw_v2.c +++ b/providers/hns/hns_roce_u_hw_v2.c @@ -85,8 +85,10 @@ static inline void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg, dseg->len = htole32(sg->length); } -static void set_extend_atomic_seg(struct hns_roce_qp *qp, unsigned int sge_cnt, - struct hns_roce_sge_info *sge_info, void *buf) +static inline void set_extend_atomic_seg(struct hns_roce_qp *qp, + unsigned int sge_cnt, + struct hns_roce_sge_info *sge_info, + void *buf) { unsigned int sge_mask = qp->ex_sge.sge_cnt - 1; unsigned int i; -- 2.33.0
participants (1)
-
Junxian Huang