- Linuxarm - mailweb.openeuler.org

[PATCH v7] ACPI / APEI: fix the regression of synchronous external aborts occur in user-mode
by Xiaofei Tan 17 Jun '21

17 Jun '21

Before commit 8fcc4ae6faf8 ("arm64: acpi: Make apei_claim_sea() synchronise with APEI's irq work"), do_sea() would unconditionally signal the affected task from the arch code. Since that change, the GHES driver sends the signals. This exposes a problem as errors the GHES driver doesn't understand or doesn't handle effectively are silently ignored. It will cause the errors get taken again, and circulate endlessly. User-space task get stuck in this loop. Existing firmware on Kunpeng9xx systems reports cache errors with the 'ARM Processor Error' CPER records. Do memory failure handling for ARM Processor Error Section just like for Memory Error Section. Fixes: 8fcc4ae6faf8 ("arm64: acpi: Make apei_claim_sea() synchronise with APEI's irq work") Signed-off-by: Xiaofei Tan <tanxiaofei(a)huawei.com> Reviewed-by: James Morse <james.morse(a)arm.com> --- Changes since v5: - Do some changes following James's suggestions: 1) optimize commit log 2) use err_info->length instead of err_info++' 3) some coding style advice. Changes since v4: - 1. Change the patch name from " ACPI / APEI: do memory failure on the physical address reported by ARM processor error section" to this more proper one. - 2. Add a comment in the code to tell why not filter out corrected error in an uncorrected section. Changes since v3: - Print unhandled error following James Morse's advice. Changes since v2: - Updated commit log --- drivers/acpi/apei/ghes.c | 81 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index fce7ade..0c8330e 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -441,28 +441,35 @@ static void ghes_kick_task_work(struct callback_head *head) gen_pool_free(ghes_estatus_pool, (unsigned long)estatus_node, node_len); } -static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, - int sev) +static bool ghes_do_memory_failure(u64 physical_addr, int flags) { unsigned long pfn; - int flags = -1; - int sec_sev = ghes_severity(gdata->error_severity); - struct cper_sec_mem_err *mem_err = acpi_hest_get_payload(gdata); if (!IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE)) return false; - if (!(mem_err->validation_bits & CPER_MEM_VALID_PA)) - return false; - - pfn = mem_err->physical_addr >> PAGE_SHIFT; + pfn = PHYS_PFN(physical_addr); if (!pfn_valid(pfn)) { pr_warn_ratelimited(FW_WARN GHES_PFX "Invalid address in generic error data: %#llx\n", - mem_err->physical_addr); + physical_addr); return false; } + memory_failure_queue(pfn, flags); + return true; +} + +static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, + int sev) +{ + int flags = -1; + int sec_sev = ghes_severity(gdata->error_severity); + struct cper_sec_mem_err *mem_err = acpi_hest_get_payload(gdata); + + if (!(mem_err->validation_bits & CPER_MEM_VALID_PA)) + return false; + /* iff following two events can be handled properly by now */ if (sec_sev == GHES_SEV_CORRECTED && (gdata->flags & CPER_SEC_ERROR_THRESHOLD_EXCEEDED)) @@ -470,14 +477,56 @@ static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, if (sev == GHES_SEV_RECOVERABLE && sec_sev == GHES_SEV_RECOVERABLE) flags = 0; - if (flags != -1) { - memory_failure_queue(pfn, flags); - return true; - } + if (flags != -1) + return ghes_do_memory_failure(mem_err->physical_addr, flags); return false; } +static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata, int sev) +{ + struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata); + bool queued = false; + int sec_sev, i; + char *p; + + log_arm_hw_error(err); + + sec_sev = ghes_severity(gdata->error_severity); + if (sev != GHES_SEV_RECOVERABLE || sec_sev != GHES_SEV_RECOVERABLE) + return false; + + p = (char *)(err + 1); + for (i = 0; i < err->err_info_num; i++) { + struct cper_arm_err_info *err_info = (struct cper_arm_err_info *)p; + bool is_cache = (err_info->type == CPER_ARM_CACHE_ERROR); + bool has_pa = (err_info->validation_bits & CPER_ARM_INFO_VALID_PHYSICAL_ADDR); + const char *error_type = "unknown error"; + + /* + * The field (err_info->error_info & BIT(26)) is fixed to set to + * 1 in some old firmware of HiSilicon Kunpeng920. We assume that + * firmware won't mix corrected errors in an uncorrected section, + * and don't filter out 'corrected' error here. + */ + if (is_cache && has_pa) { + queued = ghes_do_memory_failure(err_info->physical_fault_addr, 0); + p += err_info->length; + continue; + } + + if (err_info->type < ARRAY_SIZE(cper_proc_error_type_strs)) + error_type = cper_proc_error_type_strs[err_info->type]; + + pr_warn_ratelimited(FW_WARN GHES_PFX + "Unhandled processor error type: %s\n", + error_type); + p += err_info->length; + } + + return queued; +} + /* * PCIe AER errors need to be sent to the AER driver for reporting and * recovery. The GHES severities map to the following AER severities and @@ -605,9 +654,7 @@ static bool ghes_do_proc(struct ghes *ghes, ghes_handle_aer(gdata); } else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) { - struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata); - - log_arm_hw_error(err); + queued = ghes_handle_arm_hw_error(gdata, sev); } else { void *err = acpi_hest_get_payload(gdata); -- 2.8.1

2 3

questions about netdevice->features
by shenjian (K) 17 Jun '21

17 Jun '21

Hi, I noticed that the enum NETIF_F_XXX_BIT has already used 64 bits since NETIF_F_HW_HSR_DUP_BIT being added, while the prototype of netdev_features_t is u64. So there is no useable bit for new feature if I understand correct. Is there any solution or plan for it ? Thanks, shen jian

1 0

[PATCH net] net: sched: add barrier to ensure correct ordering for lockless qdisc
by Yunsheng Lin 16 Jun '21

16 Jun '21

The spin_trylock() is assumed to contain the implicit barrier needed to ensure the correct ordering between STATE_MISSED setting/clearing and STATE_MISSED checking in commit a90c57f2cedd ("net: sched: fix packet stuck problem for lockless qdisc"). But it turns out that spin_trylock() only has load-acquire semantic, for strongly-ordered system(like x86), the compiler barrier implicitly contained in spin_trylock() seems enough to ensure the correct ordering. But for weakly-orderly system (like arm64), the store-release semantic is needed to ensure the correct ordering as clear_bit() and test_bit() is store operation, see queued_spin_lock(). So add the explicit barrier to ensure the correct ordering for the above case. Signed-off-by: Yunsheng Lin <linyunsheng(a)huawei.com> --- The above ordering issue can easily cause out of order packet problem when testing lockless qdisc bypass patchset [1] with two iperf threads and one netdev queue in arm64 system. 1. https://lkml.org/lkml/2021/6/2/1417 --- include/net/sch_generic.h | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 1e62551..5771030 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -163,6 +163,12 @@ static inline bool qdisc_run_begin(struct Qdisc *qdisc) if (spin_trylock(&qdisc->seqlock)) goto nolock_empty; + /* Paired with smp_mb__after_atomic() to make sure + * STATE_MISSED checking is synchronized with clearing + * in pfifo_fast_dequeue(). + */ + smp_mb__before_atomic(); + /* If the MISSED flag is set, it means other thread has * set the MISSED flag before second spin_trylock(), so * we can return false here to avoid multi cpus doing @@ -180,6 +186,12 @@ static inline bool qdisc_run_begin(struct Qdisc *qdisc) */ set_bit(__QDISC_STATE_MISSED, &qdisc->state); + /* spin_trylock() only has load-acquire semantic, so use + * smp_mb__after_atomic() to ensure STATE_MISSED is set + * before doing the second spin_trylock(). + */ + smp_mb__after_atomic(); + /* Retry again in case other CPU may not see the new flag * after it releases the lock at the end of qdisc_run_end(). */ -- 2.7.4

1 0

[PATCH net-next v2 0/3] Some optimization for lockless qdisc
by Yunsheng Lin 16 Jun '21

16 Jun '21

Patch 1: remove unnecessary seqcount operation. Patch 2: implement TCQ_F_CAN_BYPASS. Patch 3: remove qdisc->empty. Performance data for pktgen in queue_xmit mode + dummy netdev with pfifo_fast: threads unpatched patched delta 1 2.60Mpps 3.21Mpps +23% 2 3.84Mpps 5.56Mpps +44% 4 5.52Mpps 5.58Mpps +1% 8 2.77Mpps 2.76Mpps -0.3% 16 2.24Mpps 2.23Mpps +0.4% Performance for IP forward testing: 1.05Mpps increases to 1.16Mpps, about 10% improvement. V2: Adjust the comment and commit log according to discussion in V1. V1: Drop RFC tag, Add nolock_qdisc_is_empty() and do the qdisc empty checking without the protection of qdisc->seqlock to aviod doing unnecessary spin_trylock() for contention case. RFC v4: Use STATE_MISSED and STATE_DRAINING to indicate non-empty qdisc, and add patch 1 and 3. Yunsheng Lin (3): net: sched: avoid unnecessary seqcount operation for lockless qdisc net: sched: implement TCQ_F_CAN_BYPASS for lockless qdisc net: sched: remove qdisc->empty for lockless qdisc include/net/sch_generic.h | 31 ++++++++++++++++++------------- net/core/dev.c | 27 +++++++++++++++++++++++++-- net/sched/sch_generic.c | 23 ++++++++++++++++------- 3 files changed, 59 insertions(+), 22 deletions(-) -- 2.7.4

3 8

[PATCH V2 net-next 0/7] net: hns3: updates for -next
by Yufeng Mo 15 Jun '21

15 Jun '21

This series includes some optimization in IO path for the HNS3 ethernet driver. --- Change log: V1 -> V2: 1. use sgt->orig_nents instead of sgt->nents in patch #5 2. fix a type mismatch issue in patch #5 Huazhong Tan (1): net: hns3: add support to query tx spare buffer size for pf Yunsheng Lin (6): net: hns3: minor refactor related to desc_cb handling net: hns3: refactor for hns3_fill_desc() function net: hns3: use tx bounce buffer for small packets net: hns3: support dma_map_sg() for multi frags skb net: hns3: optimize the rx page reuse handling process net: hns3: use bounce buffer when rx page can not be reused drivers/net/ethernet/hisilicon/hns3/hnae3.h | 8 +- drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c | 54 ++ drivers/net/ethernet/hisilicon/hns3/hns3_enet.c | 575 ++++++++++++++++++--- drivers/net/ethernet/hisilicon/hns3/hns3_enet.h | 58 ++- drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c | 66 +++ .../net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h | 2 + .../ethernet/hisilicon/hns3/hns3pf/hclge_main.c | 14 + .../ethernet/hisilicon/hns3/hns3pf/hclge_main.h | 2 + 8 files changed, 680 insertions(+), 99 deletions(-) -- 2.8.1

1 7

Re: [PATCH net-next 7/7] {topost} net: hns3: use bounce buffer when rx page can not be reused
by Salil Mehta 15 Jun '21

15 Jun '21

> From: moyufeng > Sent: Wednesday, June 2, 2021 1:13 PM > To: tanhuazhong <tanhuazhong(a)huawei.com>; shenjian (K) <shenjian15(a)huawei.com>; > lipeng (Y) <lipeng321(a)huawei.com>; Zhuangyuzeng (Yisen) > <yisen.zhuang(a)huawei.com>; linyunsheng <linyunsheng(a)huawei.com>; zhangjiaran > <zhangjiaran(a)huawei.com>; huangguangbin (A) <huangguangbin2(a)huawei.com>; > chenhao (DY) <chenhao288(a)hisilicon.com>; moyufeng <moyufeng(a)huawei.com>; Salil > Mehta <salil.mehta(a)huawei.com> > Subject: [PATCH net-next 7/7] {topost} net: hns3: use bounce buffer when rx page > can not be reused > > From: Yunsheng Lin <linyunsheng(a)huawei.com> > > Currently rx page will be reused to receive future packet when > the stack releases the previous skb quickly. If the old page > can not be reused, a new page will be allocated and mapped, > which comsumes a lot of cpu when IOMMU is in the strict mode, > especially when the application and irq/NAPI happens to run on > the same cpu. > > So allocate a new frag to memcpy the data to avoid the costly > IOMMU unmapping/mapping operation, and add "frag_alloc_err" > and "frag_alloc" stats in "ethtool -S ethX" cmd. > > The throughput improves above 50% when running single thread of > iperf using TCP when IOMMU is in strict mode and iperf shares the > same cpu with irq/NAPI(rx_copybreak = 2048 and mtu = 1500). Performance gains are quite good! Few questions: How we have ensured this will work efficiently in real world workloads and that there are no repercussions? Also, is there any impact on end-to-end *latency* or *jitter* because of this vis-a-vis without this approach? Also, have you checked why MLX5 driver has removed this copybreak concept for small packets but MLX4 did had this or why other latest drivers don't have this? Hope I have not missed this anywhere but what are the default values for both {rx,tx}_copybreak? Thanks

3 2

Re: [RFC net-next 0/8] Introducing subdev bus and devlink extension
by Yunsheng Lin 10 Jun '21

10 Jun '21

On 2021/6/8 3:46, Jakub Kicinski wrote: > On Mon, 7 Jun 2021 09:36:38 +0800 Yunsheng Lin wrote: >> On 2021/6/5 2:41, Jakub Kicinski wrote: >>> On Fri, 4 Jun 2021 09:18:04 +0800 Yunsheng Lin wrote: >>>> My initial thinking is a id from a global IDA pool, which indeed may >>>> change on every boot. >>>> >>>> I am not really thinking much deeper about the controller id, just >>>> mirroring the bus identifiers for pcie device and ifindex for netdev, >>> >>> devlink instance id seems fine, but there's already a controller >>> concept in devlink so please steer clear of that naming. >> I am not sure if controller concept already existed is reusable for >> the devlink instance representing problem for multi-function which >> shares common resource in the same ASIC. If not, we do need to pick >> up other name. >> >> Another thing I am not really think throught is how is the VF represented >> by the devlink instance when VF is passed through to a VM. >> I was thinking about VF is represented as devlink port, just like PF(with >> different port flavour), and VF devlink port only exist on the same host >> as PF(which assumes PF is never passed through to a VM), so it may means >> the PF is responsible for creating the devlink port for VF when VF is passed >> through to a VM? >> >> Or do we need to create a devlink instance for VF in the VM too when the >> VF is passed through to a VM? Or more specificly, does user need to query >> or configure devlink info or configuration in a VM? If not, then devlink >> instance in VM seems unnecessary? > > I believe the current best practice is to create a devlink instance for > the VF with a devlink port of type "virtual". Such instance represents > a "virtualized" view of the device. Afer discussion with Parav in other thread, I undersood it was the current practice, but I am not sure I understand why it is current *best* practice. If we allow all PF of a ASCI to register to the same devlink instance, does it not make sense that all VF under one PF also register to the same devlink instance that it's PF is registering to when they are in the same host? For eswitch legacy mode, whether VF and PF are the same host or not, the VF can also provide the serial number of a ASIC to register to the devlink instance, if that devlink instance does not exist yet, just create that devlink instance according to the serial number, just like PF does. For eswitch DEVLINK_ESWITCH_MODE_SWITCHDEV mode, the flavour type for devlink port instance representing the netdev of VF function is FLAVOUR_VIRTUAL, the flavour type for devlink port instance representing the representor netdev of VF is FLAVOUR_PCI_VF, which are different type, so they can register to the same devlink instance even when both of the devlink port instance is in the same host? Is there any reason why VF use its own devlink instance? > >>>> which may change too if the device is pluged into different pci slot >>>> on every boot? >>> >>> Heh. What is someone reflashes the part to change it's serial number? :) >>> pci slot is reasonably stable, as proven by years of experience trying >>> to find stable naming for netdevs. >> >> I suppose that requires a booting to take effect and a vendor tool >> to reflash the serial number, it seems reasonable the vendor/user will >> try their best to not mess the serial number, otherwise service and >> maintenance based on serial number will not work? >> I was thinking about adding the vendor name besides the serial number >> to indicate a devlink instance, to avoid that case that two hw from >> different vendor having the same serial number accidentally. > > I'm not opposed to the use of attributes such as serial number for > selecting instance, in principle. I was just trying to prove that PCI > slot/PCI device name is as stable as any other attribute. > > In fact for mass-produced machines using PCI slot is far more > convenient than globally unique identifiers because it can be used > to talk to a specific device in a server for all machines of a given > model, hence easing automation. Make sense. > >>>> We could still allow devlink instances to have multiple names, >>>> which seems to be more like devlink tool problem? >>>> >>>> For example, devlink tool could use the id or the vendor_info/ >>>> serial_number to indicate a devlink instance according to user's >>>> request. >>> >>> Typing serial numbers seems pretty painful. >>> >>>> Aliase could be allowed too as long as devlink core provide a >>>> field and ops to set/get the field mirroring the ifalias for >>>> netdevice? >>> >>> I don't understand. >> >> I meant we could still allow the user to provide a more meaningful >> name to indicate a devlink instance besides the id. > > To clarify/summarize my statement above serial number may be a useful > addition but PCI device names should IMHO remain the primary > identifiers, even if it means devlink instances with multiple names. I am not sure I understand what does it mean by "devlink instances with multiple names"? Does that mean whenever a devlink port instance is registered to a devlink instance, that devlink instance get a new name according to the PCI device which the just registered devlink port instance corresponds to? > > In addition I don't think that user-controlled names/aliases are > necessarily a great idea for devlink. > > . >

3 14

[PATCH v6] ACPI / APEI: fix the regression of synchronous external aborts occur in user-mode
by Xiaofei Tan 10 Jun '21

10 Jun '21

Before commit 8fcc4ae6faf8 ("arm64: acpi: Make apei_claim_sea() synchronise with APEI's irq work"), do_sea() would unconditionally signal the affected task from the arch code. Since that change, the GHES driver sends the signals. This exposes a problem as errors the GHES driver doesn't understand or doesn't handle effectively are silently ignored. It will cause the errors get taken again, and circulate endlessly. User-space task get stuck in this loop. Existing firmware on Kunpeng9xx systems reports cache errors with the 'ARM Processor Error' CPER records. Do memory failure handling for ARM Processor Error Section just like for Memory Error Section. Signed-off-by: Xiaofei Tan <tanxiaofei(a)huawei.com> Reviewed-by: James Morse <james.morse(a)arm.com> --- Changes since v5: - Do some changes following James's suggestions: 1) optimize commit log 2) use err_info->length instead of err_info++' 3) some coding style advice. Changes since v4: - 1. Change the patch name from " ACPI / APEI: do memory failure on the physical address reported by ARM processor error section" to this more proper one. - 2. Add a comment in the code to tell why not filter out corrected error in an uncorrected section. Changes since v3: - Print unhandled error following James Morse's advice. Changes since v2: - Updated commit log --- drivers/acpi/apei/ghes.c | 81 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index fce7ade..0c8330e 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -441,28 +441,35 @@ static void ghes_kick_task_work(struct callback_head *head) gen_pool_free(ghes_estatus_pool, (unsigned long)estatus_node, node_len); } -static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, - int sev) +static bool ghes_do_memory_failure(u64 physical_addr, int flags) { unsigned long pfn; - int flags = -1; - int sec_sev = ghes_severity(gdata->error_severity); - struct cper_sec_mem_err *mem_err = acpi_hest_get_payload(gdata); if (!IS_ENABLED(CONFIG_ACPI_APEI_MEMORY_FAILURE)) return false; - if (!(mem_err->validation_bits & CPER_MEM_VALID_PA)) - return false; - - pfn = mem_err->physical_addr >> PAGE_SHIFT; + pfn = PHYS_PFN(physical_addr); if (!pfn_valid(pfn)) { pr_warn_ratelimited(FW_WARN GHES_PFX "Invalid address in generic error data: %#llx\n", - mem_err->physical_addr); + physical_addr); return false; } + memory_failure_queue(pfn, flags); + return true; +} + +static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, + int sev) +{ + int flags = -1; + int sec_sev = ghes_severity(gdata->error_severity); + struct cper_sec_mem_err *mem_err = acpi_hest_get_payload(gdata); + + if (!(mem_err->validation_bits & CPER_MEM_VALID_PA)) + return false; + /* iff following two events can be handled properly by now */ if (sec_sev == GHES_SEV_CORRECTED && (gdata->flags & CPER_SEC_ERROR_THRESHOLD_EXCEEDED)) @@ -470,14 +477,56 @@ static bool ghes_handle_memory_failure(struct acpi_hest_generic_data *gdata, if (sev == GHES_SEV_RECOVERABLE && sec_sev == GHES_SEV_RECOVERABLE) flags = 0; - if (flags != -1) { - memory_failure_queue(pfn, flags); - return true; - } + if (flags != -1) + return ghes_do_memory_failure(mem_err->physical_addr, flags); return false; } +static bool ghes_handle_arm_hw_error(struct acpi_hest_generic_data *gdata, int sev) +{ + struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata); + bool queued = false; + int sec_sev, i; + char *p; + + log_arm_hw_error(err); + + sec_sev = ghes_severity(gdata->error_severity); + if (sev != GHES_SEV_RECOVERABLE || sec_sev != GHES_SEV_RECOVERABLE) + return false; + + p = (char *)(err + 1); + for (i = 0; i < err->err_info_num; i++) { + struct cper_arm_err_info *err_info = (struct cper_arm_err_info *)p; + bool is_cache = (err_info->type == CPER_ARM_CACHE_ERROR); + bool has_pa = (err_info->validation_bits & CPER_ARM_INFO_VALID_PHYSICAL_ADDR); + const char *error_type = "unknown error"; + + /* + * The field (err_info->error_info & BIT(26)) is fixed to set to + * 1 in some old firmware of HiSilicon Kunpeng920. We assume that + * firmware won't mix corrected errors in an uncorrected section, + * and don't filter out 'corrected' error here. + */ + if (is_cache && has_pa) { + queued = ghes_do_memory_failure(err_info->physical_fault_addr, 0); + p += err_info->length; + continue; + } + + if (err_info->type < ARRAY_SIZE(cper_proc_error_type_strs)) + error_type = cper_proc_error_type_strs[err_info->type]; + + pr_warn_ratelimited(FW_WARN GHES_PFX + "Unhandled processor error type: %s\n", + error_type); + p += err_info->length; + } + + return queued; +} + /* * PCIe AER errors need to be sent to the AER driver for reporting and * recovery. The GHES severities map to the following AER severities and @@ -605,9 +654,7 @@ static bool ghes_do_proc(struct ghes *ghes, ghes_handle_aer(gdata); } else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) { - struct cper_sec_proc_arm *err = acpi_hest_get_payload(gdata); - - log_arm_hw_error(err); + queued = ghes_handle_arm_hw_error(gdata, sev); } else { void *err = acpi_hest_get_payload(gdata); -- 2.8.1

2 2

Re: [PATCH RESEND iproute2-next] devlink: Add optional controller user input
by Yunsheng Lin 09 Jun '21

09 Jun '21

On 2021/6/7 14:10, Parav Pandit wrote: >> From: Yunsheng Lin <linyunsheng(a)huawei.com> >> Sent: Monday, June 7, 2021 9:01 AM >> >> On 2021/6/6 15:10, Parav Pandit wrote: >>> Hi Yunsheng, >>> >>>> From: Yunsheng Lin <linyunsheng(a)huawei.com> >>>> Sent: Friday, June 4, 2021 7:05 AM >>>> >>>> On 2021/6/3 19:19, Parav Pandit wrote: >>>>> A user optionally provides the external controller number when user >>>>> wants to create devlink port for the external controller. >>>> >>>> Hi, Parav >>>> I was planing to use controller id to solve the devlink instance >>>> representing problem for multi-function which shares common resource >>>> in the same ASIC, see [1]. >>>> >>>> It seems the controller id used here is to differentiate the function >>>> used in different host? >>>> >>> That’s correct. Controller holds one or more PCI functions (PF,VF,SF). >> >> I am not sure I understand the exact usage of controller and why controller id >> is in "devlink_port_*_attrs". >> >> Let's consider a simplified case where there is two PF(supposing both have >> VF enabled), and each PF has different controller and each PF corresponds to >> a different physical port(Or it is about multi-host case multi PF may sharing >> the same physical port?): > Typically single host with two PFs have their own physical ports. > Multi-host with two PFs, on each host they share respective physical ports. > > Single host: > Pf0.physical_port = p0 > Pf1.physical_port = p1. > > Multi-host (two) host setup > H1.pf0.phyical_port = p0. > H1.pf1.phyical_port = p1. > H2.pf0.phyical_port = p0. > H2.pf1.phyical_port = p1. Multi-host (two) host setup with separate physical port for each host: H1.pf0.phyical_port = p0 H2.pf0.phyical_port = p1 Does above use case make sense for mlx, it seems a common case for our internal use. > >> 1. I suppose each PF has it's devlink instance for mlx case(I suppose each >> VF can not have it's own devlink instance for VF shares the same physical >> port with PF, right?). > VF and SF ports are of flavour VIRTUAL. Which devlink instance does the flavour VIRTUAL port instance for VF and SF is registered to? Does it mean VF has it's own devlink instance in VM when it is passed a VM, and flavour VIRTUAL port instance for that VF is registered to that devlink instance in the VM too？ Even in the same host as PF, the VF also has it's own devlink instance? > >> 2. each PF's devlink instance has three types of port, which is >> FLAVOUR_PHYSICAL, FLAVOUR_PCI_PF and FLAVOUR_PCI_VF(supposing I >> understand >> port flavour correctly). >> > FLAVOUR_PCI_{PF,VF,SF} belongs to eswitch (representor) side on switchdev device. If devlink instance or eswitch is in DEVLINK_ESWITCH_MODE_LEGACY mode, the FLAVOUR_PCI_{PF,VF,SF} port instance does not need to created? > >> If I understand above correctly, all ports in the same devlink instance should >> have the same controller id, right? If yes, why not put the controller id in the >> devlink instance? > Need not be. All PCI_{PF,VF,SF} can have controller id different for different controllers. The point is that two VF from different PF may be in the different host, all VF of a specific PF need to be in the same host, right? otherwise it may break PCI enumeration process? If yes, as PCI_{PF,VF,SF} belongs to eswitch (representor) side on switchdev device(which means PCI_{PF,VF,SF} port instance is in the same host, as the host corresponding to "controller_num=0" in diagram [1]), so it seems all the PCI_{PF,VF,SF} of a specific PF should have the same controller id, and using a controller id of the devlink instance in "controller_num=0" in diagram [1] seems enough? > Usually each multi-host is a different controller. > Refer to this diagram [1] and detailed description. devlink instance does not exist in the host corresponding to "controller_num=1" in diagram [1]? Or devlink instance does exist in the host corresponding to "controller_num=1", but the mode of that devlink instance is DEVLINK_ESWITCH_MODE_LEGACY in diagram [1]? Also, eswitch mode can only be set on the devlink instance corresponding to PF, but not for VF/SF(supposing that VF/SF could have it's own devlink instance too), right? > >> >>> In your case if there is single devlink instance representing ASIC, it is better >> to have health reporters under this single instance. >>> >>> Devlink parameters do not span multiple devlink instance. >> >> Yes, that is what I try to do: shared status/parameters in devlink instance, >> physical port specific status/parameters in devlink port instance. >> >>> So if you need to control devlink instance parameters of each function >> byitself, you likely need devlink instance for each. >>> And still continue to have ASIC wide health reporters under single instance >> that represents whole ASIC. >> >> I do not think each function need a devlink instance if there is a devlink >> instance representing a whole ASIC, using the devlink port instance to >> represent the function seems enough? > 'devlink port function's is equivalent of hypervisor/nicvisor entity controlled by the network/sysadmin. > While devlink instance of a given PF,VF,SF is managed by the user of such function itself. 'devlink port function' means "struct devlink_port", right? It seems 'devlink port function' in the host is representing a VF when devlink instance of that VF is in the VM? > For example when a VF is mapped to a VM, devlink instance of this VF resides in the VM managed by the guest VM. Does the user in VM really care about devlink info or configuration when network/sysadmin has configured the VF through 'devlink port function' in the host? which devlink info or configuration does user need to query or configure in a VM? > > Before this VF is given to VM, usually hypervisor/nicvisor admin programs this function (such as mac address) explained in [3]. > So that a given VM always gets the same mac address regardless of which VF {a or b). > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Doc… > [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Doc… > [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Doc… > >> >>> >>>> 1. https://lkml.org/lkml/2021/5/31/296 >

2 9

[PATCH 0/4] Free cached iovas when rmmod the driver of the last device in group
by chenxiang 08 Jun '21

08 Jun '21

From: Xiang Chen <chenxiang66(a)hisilicon.com> When rmmod the driver of the last device in the group, cached iovas are not used, and it is better to free them to save memories. And also export function free_rcache_cached_iovas() and iommu_domain_to_iova(). Xiang Chen (4): iommu/iova: add a function to free all rcached iovas and export it iommu/iova: use function free_rcache_cached_iovas() to free all rcached iovas dma-iommu: add a interface to get iova_domain from iommu domain iommu: free cached iovas when rmmod the driver of the last device in the group drivers/iommu/dma-iommu.c | 7 +++++++ drivers/iommu/iommu.c | 7 +++++++ drivers/iommu/iova.c | 17 ++++++++++++----- include/linux/dma-iommu.h | 6 ++++++ include/linux/iova.h | 5 +++++ 5 files changed, 37 insertions(+), 5 deletions(-) -- 2.8.1

3 7