From: JiangShui Yang yangjiangshui@h-partners.com
Add debugfs 'dev_state' to query the status of the stop queue. And the root user can set 'dev_timeout', if task flow fails to be stopped, the driver waits dev_timeout * 20ms before releasing the queue.
Weili Qian (2): crypto: hisilicon/qm - add debugfs to query the status of the stop queue crypto: hisilicon/qm - support dumping stop queue status
Documentation/ABI/testing/debugfs-hisi-hpre | 15 ++++ Documentation/ABI/testing/debugfs-hisi-sec | 15 ++++ Documentation/ABI/testing/debugfs-hisi-zip | 15 ++++ drivers/crypto/hisilicon/debugfs.c | 5 ++ drivers/crypto/hisilicon/qm.c | 98 +++++++++++++++------ include/linux/hisi_acc_qm.h | 14 +++ 6 files changed, 135 insertions(+), 27 deletions(-)
Debugfs for query the status of stopping queue, the root user can set the waiting time after the task flow fails to be stopped.
Signed-off-by: Weili Qian qianweili@huawei.com --- Documentation/ABI/testing/debugfs-hisi-hpre | 15 +++++++++++++++ Documentation/ABI/testing/debugfs-hisi-sec | 15 +++++++++++++++ Documentation/ABI/testing/debugfs-hisi-zip | 15 +++++++++++++++ 3 files changed, 45 insertions(+)
diff --git a/Documentation/ABI/testing/debugfs-hisi-hpre b/Documentation/ABI/testing/debugfs-hisi-hpre index b4be5f1db4b7..f50964977b12 100644 --- a/Documentation/ABI/testing/debugfs-hisi-hpre +++ b/Documentation/ABI/testing/debugfs-hisi-hpre @@ -94,6 +94,21 @@ Description: Dump the status of the QM. Four states: initiated, started, stopped and closed. Available for both PF and VF, and take no other effect on HPRE.
+What: /sys/kernel/debug/hisi_hpre/<bdf>/qm/dev_timeout +Date: May 2023 +Contact: qianweili@huawei.com +Description: Set the wait time when stop queue fails. Available for both PF + and VF, and take no other effect on HPRE. + 0: not wait(default), others value: wait dev_timeout * 20 microsecond. + +What: /sys/kernel/debug/hisi_hpre/<bdf>/qm/dev_state +Date: May 2023 +Contact: qianweili@huawei.com +Description: Dump the stop queue status of the QM. The default value is 0, + if dev_timeout is set, when stop queue fails, the dev_state + will return non-zero value. Available for both PF and VF, + and take no other effect on HPRE. + What: /sys/kernel/debug/hisi_hpre/<bdf>/hpre_dfx/send_cnt Date: Apr 2020 Contact: linux-crypto@vger.kernel.org diff --git a/Documentation/ABI/testing/debugfs-hisi-sec b/Documentation/ABI/testing/debugfs-hisi-sec index 85feb4408e0f..b3bf2011e669 100644 --- a/Documentation/ABI/testing/debugfs-hisi-sec +++ b/Documentation/ABI/testing/debugfs-hisi-sec @@ -74,6 +74,21 @@ Description: Dump the status of the QM. Four states: initiated, started, stopped and closed. Available for both PF and VF, and take no other effect on SEC.
+What: /sys/kernel/debug/hisi_sec2/<bdf>/qm/dev_timeout +Date: May 2023 +Contact: qianweili@huawei.com +Description: Set the wait time when stop queue fails. Available for both PF + and VF, and take no other effect on SEC. + 0: not wait(default), others value: wait dev_timeout * 20 microsecond. + +What: /sys/kernel/debug/hisi_sec2/<bdf>/qm/dev_state +Date: May 2023 +Contact: qianweili@huawei.com +Description: Dump the stop queue status of the QM. The default value is 0, + if dev_timeout is set, when stop queue fails, the dev_state + will return non-zero value. Available for both PF and VF, + and take no other effect on SEC. + What: /sys/kernel/debug/hisi_sec2/<bdf>/sec_dfx/send_cnt Date: Apr 2020 Contact: linux-crypto@vger.kernel.org diff --git a/Documentation/ABI/testing/debugfs-hisi-zip b/Documentation/ABI/testing/debugfs-hisi-zip index 3034a2bf99ca..62ef009ffd68 100644 --- a/Documentation/ABI/testing/debugfs-hisi-zip +++ b/Documentation/ABI/testing/debugfs-hisi-zip @@ -87,6 +87,21 @@ Description: Dump the status of the QM. Four states: initiated, started, stopped and closed. Available for both PF and VF, and take no other effect on ZIP.
+What: /sys/kernel/debug/hisi_zip/<bdf>/qm/dev_timeout +Date: May 2023 +Contact: qianweili@huawei.com +Description: Set the wait time when stop queue fails. Available for both PF + and VF, and take no other effect on ZIP. + 0: not wait(default), others value: wait dev_timeout * 20 microsecond. + +What: /sys/kernel/debug/hisi_zip/<bdf>/qm/dev_state +Date: May 2023 +Contact: qianweili@huawei.com +Description: Dump the stop queue status of the QM. The default value is 0, + if dev_timeout is set, when stop queue fails, the dev_state + will return non-zero value. Available for both PF and VF, + and take no other effect on ZIP. + What: /sys/kernel/debug/hisi_zip/<bdf>/zip_dfx/send_cnt Date: Apr 2020 Contact: linux-crypto@vger.kernel.org
The debugfs files 'dev_state' and 'dev_timeout' are added.
dev_state: if dev_timeout is set, dev_state indicates the status of stopping the queue. 0 indicates that the queue is stopped successfully. Other values indicate that the queue stops fail. if dev_timeout is not set, the value of dev_state is 0;
dev_timeout: If the queue fails to stop, the queue is released after waiting dev_timeout * 20ms.
Signed-off-by: Weili Qian qianweili@huawei.com --- drivers/crypto/hisilicon/debugfs.c | 5 ++ drivers/crypto/hisilicon/qm.c | 98 ++++++++++++++++++++++-------- include/linux/hisi_acc_qm.h | 14 +++++ 3 files changed, 90 insertions(+), 27 deletions(-)
diff --git a/drivers/crypto/hisilicon/debugfs.c b/drivers/crypto/hisilicon/debugfs.c index 2cc1591949db..318c8ac1ba5e 100644 --- a/drivers/crypto/hisilicon/debugfs.c +++ b/drivers/crypto/hisilicon/debugfs.c @@ -1073,6 +1073,7 @@ DEFINE_DEBUGFS_ATTRIBUTE(qm_atomic64_ops, qm_debugfs_atomic64_get, void hisi_qm_debug_init(struct hisi_qm *qm) { struct dfx_diff_registers *qm_regs = qm->debug.qm_diff_regs; + struct qm_dev_dfx *dev_dfx = &qm->debug.dev_dfx; struct qm_dfx *dfx = &qm->debug.dfx; struct dentry *qm_d; void *data; @@ -1098,6 +1099,10 @@ void hisi_qm_debug_init(struct hisi_qm *qm)
debugfs_create_file("status", 0444, qm->debug.qm_d, qm, &qm_status_fops); + + debugfs_create_u32("dev_state", 0444, qm->debug.qm_d, &dev_dfx->dev_state); + debugfs_create_u32("dev_timeout", 0644, qm->debug.qm_d, &dev_dfx->dev_timeout); + for (i = 0; i < ARRAY_SIZE(qm_dfx_files); i++) { data = (atomic64_t *)((uintptr_t)dfx + qm_dfx_files[i].offset); debugfs_create_file(qm_dfx_files[i].name, diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c index 063b448f4d48..e0ce837f342c 100644 --- a/drivers/crypto/hisilicon/qm.c +++ b/drivers/crypto/hisilicon/qm.c @@ -2099,55 +2099,38 @@ static void qp_stop_fail_cb(struct hisi_qp *qp) } }
-/** - * qm_drain_qp() - Drain a qp. - * @qp: The qp we want to drain. - * - * Determine whether the queue is cleared by judging the tail pointers of - * sq and cq. - */ -static int qm_drain_qp(struct hisi_qp *qp) +static int qm_wait_qp_empty(struct hisi_qm *qm, u32 *state, u32 qp_id) { size_t size = sizeof(struct qm_sqc) + sizeof(struct qm_cqc); - struct hisi_qm *qm = qp->qm; - struct hisi_qm *pf_qm = pci_get_drvdata(pci_physfn(qm->pdev)); struct device *dev = &qm->pdev->dev; struct qm_sqc *sqc; struct qm_cqc *cqc; dma_addr_t dma_addr; - int ret = 0, i = 0; void *addr;
- /* No need to judge if master OOO is blocked. */ - if (qm_check_dev_error(pf_qm)) - return 0; - - /* Kunpeng930 supports drain qp by device */ - if (test_bit(QM_SUPPORT_STOP_QP, &qm->caps)) { - ret = qm_stop_qp(qp); - if (ret) - dev_err(dev, "Failed to stop qp(%u)!\n", qp->qp_id); - return ret; - } + int ret = 0; + int i = 0;
addr = hisi_qm_ctx_alloc(qm, size, &dma_addr); if (IS_ERR(addr)) { dev_err(dev, "Failed to alloc ctx for sqc and cqc!\n"); + *state = ALLOC_CTX_FAIL; return -ENOMEM; }
while (++i) { - ret = qm_dump_sqc_raw(qm, dma_addr, qp->qp_id); + ret = qm_dump_sqc_raw(qm, dma_addr, qp_id); if (ret) { dev_err_ratelimited(dev, "Failed to dump sqc!\n"); + *state = DUMP_SQC_FAIL; break; } sqc = addr;
- ret = qm_dump_cqc_raw(qm, (dma_addr + sizeof(struct qm_sqc)), - qp->qp_id); + ret = qm_dump_cqc_raw(qm, (dma_addr + sizeof(struct qm_sqc)), qp_id); if (ret) { dev_err_ratelimited(dev, "Failed to dump cqc!\n"); + *state = DUMP_CQC_FAIL; break; } cqc = addr + sizeof(struct qm_sqc); @@ -2157,7 +2140,8 @@ static int qm_drain_qp(struct hisi_qp *qp) break;
if (i == MAX_WAIT_COUNTS) { - dev_err(dev, "Fail to empty queue %u!\n", qp->qp_id); + dev_err(dev, "Fail to empty queue %u!\n", qp_id); + *state = STOP_QUEUE_FAIL; ret = -EBUSY; break; } @@ -2170,6 +2154,47 @@ static int qm_drain_qp(struct hisi_qp *qp) return ret; }
+/** + * qm_drain_qp() - Drain a qp. + * @qp: The qp we want to drain. + * + * Determine whether the queue is cleared by judging the tail pointers of + * sq and cq. + */ +static int qm_drain_qp(struct hisi_qp *qp) +{ + struct hisi_qm *qm = qp->qm; + struct hisi_qm *pf_qm = pci_get_drvdata(pci_physfn(qm->pdev)); + u32 state = 0; + int ret = 0; + + /* No need to judge if master OOO is blocked. */ + if (qm_check_dev_error(pf_qm)) + return 0; + + /* HW V3 supports drain qp by device */ + if (test_bit(QM_SUPPORT_STOP_QP, &qm->caps)) { + ret = qm_stop_qp(qp); + if (ret) { + dev_err(&qm->pdev->dev, "Failed to stop qp!\n"); + state = STOP_QUEUE_FAIL; + goto set_dev_state; + } + return ret; + } + + ret = qm_wait_qp_empty(qm, &state, qp->qp_id); + if (ret) + goto set_dev_state; + + return 0; + +set_dev_state: + if (qp->qm->debug.dev_dfx.dev_timeout) + qp->qm->debug.dev_dfx.dev_state = state; + + return ret; +} static int qm_stop_qp_nolock(struct hisi_qp *qp) { struct device *dev = &qp->qm->pdev->dev; @@ -2397,7 +2422,26 @@ static int hisi_qm_uacce_start_queue(struct uacce_queue *q)
static void hisi_qm_uacce_stop_queue(struct uacce_queue *q) { - hisi_qm_stop_qp(q->priv); + struct hisi_qp *qp = q->priv; + struct hisi_qm *qm = qp->qm; + struct qm_dev_dfx *dev_dfx = &qm->debug.dev_dfx; + u32 i = 0; + + hisi_qm_stop_qp(qp); + + if (!dev_dfx->dev_timeout || !dev_dfx->dev_state) + return; + + while (++i) { + if (i > dev_dfx->dev_timeout) { + dev_err(&qm->pdev->dev, "Stop q %u timeout, state %u\n", + qp->qp_id, dev_dfx->dev_state); + dev_dfx->dev_state = FINISH_WAIT; + break; + } + + msleep(WAIT_PERIOD); + } }
static int hisi_qm_is_q_updated(struct uacce_queue *q) diff --git a/include/linux/hisi_acc_qm.h b/include/linux/hisi_acc_qm.h index c0987492f39f..11da059c5e2a 100644 --- a/include/linux/hisi_acc_qm.h +++ b/include/linux/hisi_acc_qm.h @@ -154,6 +154,19 @@ enum qm_cap_bits { QM_SUPPORT_RPM, };
+enum qm_dev_fail_state { + STOP_QUEUE_FAIL = 1, + ALLOC_CTX_FAIL, + DUMP_SQC_FAIL, + DUMP_CQC_FAIL, + FINISH_WAIT, +}; + +struct qm_dev_dfx { + u32 dev_state; + u32 dev_timeout; +}; + struct dfx_diff_registers { u32 *regs; u32 reg_offset; @@ -182,6 +195,7 @@ struct qm_debug { struct dentry *debug_root; struct dentry *qm_d; struct debugfs_file files[DEBUG_FILE_NUM]; + struct qm_dev_dfx dev_dfx; unsigned int *qm_last_words; /* ACC engines recoreding last regs */ unsigned int *last_words;
-
Weili Qian