- Linuxarm - mailweb.openeuler.org

[PATCH net v3] net: sched: fix packet stuck problem for lockless qdisc
by Yunsheng Lin 20 Apr '21

20 Apr '21

Lockless qdisc has below concurrent problem: cpu0 cpu1 . . q->enqueue . . . qdisc_run_begin() . . . dequeue_skb() . . . sch_direct_xmit() . . . . q->enqueue . qdisc_run_begin() . return and do nothing . . qdisc_run_end() . cpu1 enqueue a skb without calling __qdisc_run() because cpu0 has not released the lock yet and spin_trylock() return false for cpu1 in qdisc_run_begin(), and cpu0 do not see the skb enqueued by cpu1 when calling dequeue_skb() because cpu1 may enqueue the skb after cpu0 calling dequeue_skb() and before cpu0 calling qdisc_run_end(). Lockless qdisc has below another concurrent problem when tx_action is involved: cpu0(serving tx_action) cpu1 cpu2 . . . . q->enqueue . . qdisc_run_begin() . . dequeue_skb() . . . q->enqueue . . . . sch_direct_xmit() . . . qdisc_run_begin() . . return and do nothing . . . clear __QDISC_STATE_SCHED . . qdisc_run_begin() . . return and do nothing . . . . . . qdisc_run_end() . This patch fixes the above data race by: 1. Get the flag before doing spin_trylock(). 2. If the first spin_trylock() return false and the flag is not set before the first spin_trylock(), Set the flag and retry another spin_trylock() in case other CPU may not see the new flag after it releases the lock. 3. reschedule if the flags is set after the lock is released at the end of qdisc_run_end(). For tx_action case, the flags is also set when cpu1 is at the end if qdisc_run_end(), so tx_action will be rescheduled again to dequeue the skb enqueued by cpu2. Only clear the flag before retrying a dequeuing when dequeuing returns NULL in order to reduce the overhead of the above double spin_trylock() and __netif_schedule() calling. The performance impact of this patch, tested using pktgen and dummy netdev with pfifo_fast qdisc attached: threads without+this_patch with+this_patch delta 1 2.61Mpps 2.60Mpps -0.3% 2 3.97Mpps 3.82Mpps -3.7% 4 5.62Mpps 5.59Mpps -0.5% 8 2.78Mpps 2.77Mpps -0.3% 16 2.22Mpps 2.22Mpps -0.0% Fixes: 6b3ba9146fe6 ("net: sched: allow qdiscs to handle locking") Signed-off-by: Yunsheng Lin <linyunsheng(a)huawei.com> --- V3: fix a compile error and a few comment typo, remove the __QDISC_STATE_DEACTIVATED checking, and update the performance data. V2: Avoid the overhead of fixing the data race as much as possible. --- include/net/sch_generic.h | 38 +++++++++++++++++++++++++++++++++++++- net/sched/sch_generic.c | 12 ++++++++++++ 2 files changed, 49 insertions(+), 1 deletion(-) diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index f7a6e14..e3f46eb 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -36,6 +36,7 @@ struct qdisc_rate_table { enum qdisc_state_t { __QDISC_STATE_SCHED, __QDISC_STATE_DEACTIVATED, + __QDISC_STATE_NEED_RESCHEDULE, }; struct qdisc_size_table { @@ -159,8 +160,38 @@ static inline bool qdisc_is_empty(const struct Qdisc *qdisc) static inline bool qdisc_run_begin(struct Qdisc *qdisc) { if (qdisc->flags & TCQ_F_NOLOCK) { + bool dont_retry = test_bit(__QDISC_STATE_NEED_RESCHEDULE, + &qdisc->state); + + if (spin_trylock(&qdisc->seqlock)) + goto nolock_empty; + + /* If the flag is set before doing the spin_trylock() and + * the above spin_trylock() return false, it means other cpu + * holding the lock will do dequeuing for us, or it wil see + * the flag set after releasing lock and reschedule the + * net_tx_action() to do the dequeuing. + */ + if (dont_retry) + return false; + + /* We could do set_bit() before the first spin_trylock(), + * and avoid doing second spin_trylock() completely, then + * we could have multi cpus doing the set_bit(). Here use + * dont_retry to avoid doing the set_bit() and the second + * spin_trylock(), which has 5% performance improvement than + * doing the set_bit() before the first spin_trylock(). + */ + set_bit(__QDISC_STATE_NEED_RESCHEDULE, + &qdisc->state); + + /* Retry again in case other CPU may not see the new flag + * after it releases the lock at the end of qdisc_run_end(). + */ if (!spin_trylock(&qdisc->seqlock)) return false; + +nolock_empty: WRITE_ONCE(qdisc->empty, false); } else if (qdisc_is_running(qdisc)) { return false; @@ -176,8 +207,13 @@ static inline bool qdisc_run_begin(struct Qdisc *qdisc) static inline void qdisc_run_end(struct Qdisc *qdisc) { write_seqcount_end(&qdisc->running); - if (qdisc->flags & TCQ_F_NOLOCK) + if (qdisc->flags & TCQ_F_NOLOCK) { spin_unlock(&qdisc->seqlock); + + if (unlikely(test_bit(__QDISC_STATE_NEED_RESCHEDULE, + &qdisc->state))) + __netif_schedule(qdisc); + } } static inline bool qdisc_may_bulk(const struct Qdisc *qdisc) diff --git a/net/sched/sch_generic.c b/net/sched/sch_generic.c index 44991ea..4953430 100644 --- a/net/sched/sch_generic.c +++ b/net/sched/sch_generic.c @@ -640,8 +640,10 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc) { struct pfifo_fast_priv *priv = qdisc_priv(qdisc); struct sk_buff *skb = NULL; + bool need_retry = true; int band; +retry: for (band = 0; band < PFIFO_FAST_BANDS && !skb; band++) { struct skb_array *q = band2list(priv, band); @@ -652,6 +654,16 @@ static struct sk_buff *pfifo_fast_dequeue(struct Qdisc *qdisc) } if (likely(skb)) { qdisc_update_stats_at_dequeue(qdisc, skb); + } else if (need_retry && + test_and_clear_bit(__QDISC_STATE_NEED_RESCHEDULE, + &qdisc->state)) { + /* do another enqueuing after clearing the flag to + * avoid calling __netif_schedule(). + */ + smp_mb__after_atomic(); + need_retry = false; + + goto retry; } else { WRITE_ONCE(qdisc->empty, true); } -- 2.7.4

2 6

[RFC PATCH v6 4/4] scheduler: Add cluster scheduler level for x86
by Barry Song 20 Apr '21

20 Apr '21

From: Tim Chen <tim.c.chen(a)linux.intel.com> There are x86 CPU architectures (e.g. Jacobsville) where L2 cahce is shared among a cluster of cores instead of being exclusive to one single core. To prevent oversubscription of L2 cache, load should be balanced between such L2 clusters, especially for tasks with no shared data. Also with cluster scheduling policy where tasks are woken up in the same L2 cluster, we will benefit from keeping tasks related to each other and likely sharing data in the same L2 cluster. Add CPU masks of CPUs sharing the L2 cache so we can build such L2 cluster scheduler domain. Signed-off-by: Tim Chen <tim.c.chen(a)linux.intel.com> Signed-off-by: Barry Song <song.bao.hua(a)hisilicon.com> --- -v6: * added topology_cluster_cpumask() for x86, code provided by Tim. arch/x86/Kconfig | 8 ++++++++ arch/x86/include/asm/smp.h | 7 +++++++ arch/x86/include/asm/topology.h | 2 ++ arch/x86/kernel/cpu/cacheinfo.c | 1 + arch/x86/kernel/cpu/common.c | 3 +++ arch/x86/kernel/smpboot.c | 43 ++++++++++++++++++++++++++++++++++++++++- 6 files changed, 63 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 2792879..d597de2 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -1002,6 +1002,14 @@ config NR_CPUS This is purely to save memory: each supported CPU adds about 8KB to the kernel image. +config SCHED_CLUSTER + bool "Cluster scheduler support" + default n + help + Cluster scheduler support improves the CPU scheduler's decision + making when dealing with machines that have clusters of CPUs + sharing L2 cache. If unsure say N here. + config SCHED_SMT def_bool y if SMP diff --git a/arch/x86/include/asm/smp.h b/arch/x86/include/asm/smp.h index c0538f8..9cbc4ae 100644 --- a/arch/x86/include/asm/smp.h +++ b/arch/x86/include/asm/smp.h @@ -16,7 +16,9 @@ DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_die_map); /* cpus sharing the last level cache: */ DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); +DECLARE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map); DECLARE_PER_CPU_READ_MOSTLY(u16, cpu_llc_id); +DECLARE_PER_CPU_READ_MOSTLY(u16, cpu_l2c_id); DECLARE_PER_CPU_READ_MOSTLY(int, cpu_number); static inline struct cpumask *cpu_llc_shared_mask(int cpu) @@ -24,6 +26,11 @@ static inline struct cpumask *cpu_llc_shared_mask(int cpu) return per_cpu(cpu_llc_shared_map, cpu); } +static inline struct cpumask *cpu_l2c_shared_mask(int cpu) +{ + return per_cpu(cpu_l2c_shared_map, cpu); +} + DECLARE_EARLY_PER_CPU_READ_MOSTLY(u16, x86_cpu_to_apicid); DECLARE_EARLY_PER_CPU_READ_MOSTLY(u32, x86_cpu_to_acpiid); DECLARE_EARLY_PER_CPU_READ_MOSTLY(u16, x86_bios_cpu_apicid); diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index 9239399..800fa48 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -103,6 +103,7 @@ static inline void setup_node_to_cpumask_map(void) { } #include <asm-generic/topology.h> extern const struct cpumask *cpu_coregroup_mask(int cpu); +extern const struct cpumask *cpu_clustergroup_mask(int cpu); #define topology_logical_package_id(cpu) (cpu_data(cpu).logical_proc_id) #define topology_physical_package_id(cpu) (cpu_data(cpu).phys_proc_id) @@ -114,6 +115,7 @@ static inline void setup_node_to_cpumask_map(void) { } #ifdef CONFIG_SMP #define topology_die_cpumask(cpu) (per_cpu(cpu_die_map, cpu)) +#define topology_cluster_cpumask(cpu) (cpu_clustergroup_mask(cpu)) #define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu)) #define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu)) diff --git a/arch/x86/kernel/cpu/cacheinfo.c b/arch/x86/kernel/cpu/cacheinfo.c index 3ca9be4..0d03a71 100644 --- a/arch/x86/kernel/cpu/cacheinfo.c +++ b/arch/x86/kernel/cpu/cacheinfo.c @@ -846,6 +846,7 @@ void init_intel_cacheinfo(struct cpuinfo_x86 *c) l2 = new_l2; #ifdef CONFIG_SMP per_cpu(cpu_llc_id, cpu) = l2_id; + per_cpu(cpu_l2c_id, cpu) = l2_id; #endif } diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index ab640ab..0ba282d 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -78,6 +78,9 @@ /* Last level cache ID of each logical CPU */ DEFINE_PER_CPU_READ_MOSTLY(u16, cpu_llc_id) = BAD_APICID; +/* L2 cache ID of each logical CPU */ +DEFINE_PER_CPU_READ_MOSTLY(u16, cpu_l2c_id) = BAD_APICID; + /* correctly size the local cpu masks */ void __init setup_cpu_local_masks(void) { diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 02813a7..c85ffa8 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -101,6 +101,8 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_llc_shared_map); +DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map); + /* Per CPU bogomips and other parameters */ DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info); EXPORT_PER_CPU_SYMBOL(cpu_info); @@ -501,6 +503,21 @@ static bool match_llc(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) return topology_sane(c, o, "llc"); } +static bool match_l2c(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) +{ + int cpu1 = c->cpu_index, cpu2 = o->cpu_index; + + /* Do not match if we do not have a valid APICID for cpu: */ + if (per_cpu(cpu_l2c_id, cpu1) == BAD_APICID) + return false; + + /* Do not match if L2 cache id does not match: */ + if (per_cpu(cpu_l2c_id, cpu1) != per_cpu(cpu_l2c_id, cpu2)) + return false; + + return topology_sane(c, o, "l2c"); +} + /* * Unlike the other levels, we do not enforce keeping a * multicore group inside a NUMA node. If this happens, we will @@ -522,7 +539,7 @@ static bool match_die(struct cpuinfo_x86 *c, struct cpuinfo_x86 *o) } -#if defined(CONFIG_SCHED_SMT) || defined(CONFIG_SCHED_MC) +#if defined(CONFIG_SCHED_SMT) || defined(CONFIG_SCHED_CLUSTER) || defined(CONFIG_SCHED_MC) static inline int x86_sched_itmt_flags(void) { return sysctl_sched_itmt_enabled ? SD_ASYM_PACKING : 0; @@ -540,12 +557,21 @@ static int x86_smt_flags(void) return cpu_smt_flags() | x86_sched_itmt_flags(); } #endif +#ifdef CONFIG_SCHED_CLUSTER +static int x86_cluster_flags(void) +{ + return cpu_cluster_flags() | x86_sched_itmt_flags(); +} +#endif #endif static struct sched_domain_topology_level x86_numa_in_package_topology[] = { #ifdef CONFIG_SCHED_SMT { cpu_smt_mask, x86_smt_flags, SD_INIT_NAME(SMT) }, #endif +#ifdef CONFIG_SCHED_CLUSTER + { cpu_clustergroup_mask, x86_cluster_flags, SD_INIT_NAME(CLS) }, +#endif #ifdef CONFIG_SCHED_MC { cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC) }, #endif @@ -556,6 +582,9 @@ static int x86_smt_flags(void) #ifdef CONFIG_SCHED_SMT { cpu_smt_mask, x86_smt_flags, SD_INIT_NAME(SMT) }, #endif +#ifdef CONFIG_SCHED_CLUSTER + { cpu_clustergroup_mask, x86_cluster_flags, SD_INIT_NAME(CLS) }, +#endif #ifdef CONFIG_SCHED_MC { cpu_coregroup_mask, x86_core_flags, SD_INIT_NAME(MC) }, #endif @@ -583,6 +612,7 @@ void set_cpu_sibling_map(int cpu) if (!has_mp) { cpumask_set_cpu(cpu, topology_sibling_cpumask(cpu)); cpumask_set_cpu(cpu, cpu_llc_shared_mask(cpu)); + cpumask_set_cpu(cpu, cpu_l2c_shared_mask(cpu)); cpumask_set_cpu(cpu, topology_core_cpumask(cpu)); cpumask_set_cpu(cpu, topology_die_cpumask(cpu)); c->booted_cores = 1; @@ -598,6 +628,8 @@ void set_cpu_sibling_map(int cpu) if ((i == cpu) || (has_mp && match_llc(c, o))) link_mask(cpu_llc_shared_mask, cpu, i); + if ((i == cpu) || (has_mp && match_l2c(c, o))) + link_mask(cpu_l2c_shared_mask, cpu, i); } /* @@ -649,6 +681,11 @@ const struct cpumask *cpu_coregroup_mask(int cpu) return cpu_llc_shared_mask(cpu); } +const struct cpumask *cpu_clustergroup_mask(int cpu) +{ + return cpu_l2c_shared_mask(cpu); +} + static void impress_friends(void) { int cpu; @@ -1332,6 +1369,7 @@ void __init native_smp_prepare_cpus(unsigned int max_cpus) zalloc_cpumask_var(&per_cpu(cpu_core_map, i), GFP_KERNEL); zalloc_cpumask_var(&per_cpu(cpu_die_map, i), GFP_KERNEL); zalloc_cpumask_var(&per_cpu(cpu_llc_shared_map, i), GFP_KERNEL); + zalloc_cpumask_var(&per_cpu(cpu_l2c_shared_map, i), GFP_KERNEL); } /* @@ -1556,7 +1594,10 @@ static void remove_siblinginfo(int cpu) cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling)); for_each_cpu(sibling, cpu_llc_shared_mask(cpu)) cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling)); + for_each_cpu(sibling, cpu_l2c_shared_mask(cpu)) + cpumask_clear_cpu(cpu, cpu_l2c_shared_mask(sibling)); cpumask_clear(cpu_llc_shared_mask(cpu)); + cpumask_clear(cpu_l2c_shared_mask(cpu)); cpumask_clear(topology_sibling_cpumask(cpu)); cpumask_clear(topology_core_cpumask(cpu)); cpumask_clear(topology_die_cpumask(cpu)); -- 1.8.3.1

1 0

[PATCH net-next 0/3] net: hns3: misc updates for -next
by Huazhong Tan 20 Apr '21

20 Apr '21

This series includes some misc updates for the HNS3 ethernet driver. Huazhong Tan (3): net: hns3: remove a duplicate pf reset counting net: hns3: cleanup inappropriate spaces in struct hlcgevf_tqp_stats net: hns3: change the value of the SEPARATOR_VALUE macro in hclgevf_main.c drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c | 2 -- drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c | 2 +- drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h | 4 ++-- 3 files changed, 3 insertions(+), 5 deletions(-) -- 2.7.4

2 4

Re: [PATCH V4] app/testpmd: support Tx mbuf free on demand cmd
by Ferruh Yigit 19 Apr '21

19 Apr '21

On 4/19/2021 1:36 PM, Lijun Ou wrote: > From: Chengwen Feng <fengchengwen(a)huawei.com> > > This patch support tx_done_cleanup command: > tx_done_cleanup port (port_id) (queue_id) (free_cnt) > Instead of creating a new root level, 'tx_done_cleanup' command, what do you think to use existing port command, something like: "port cleanup <port_id> txq <queue_id> <free_cnt>" > Users must make sure there are no concurrent access to the same Tx > queue (like rte_eth_tx_burst, rte_eth_dev_tx_queue_stop and so on) > this command executed. > > Signed-off-by: Chengwen Feng <fengchengwen(a)huawei.com> > Signed-off-by: Lijun Ou <oulijun(a)huawei.com> > --- > V3->V4: > - revert the V3 scheme. > > V2->V3: > - The command implementation is changed so that the queuestate does > not depend on the command execution. > > V1->V2: > - use Tx instead of TX > - add note in doc > --- > app/test-pmd/cmdline.c | 84 +++++++++++++++++++++++++++++ > doc/guides/rel_notes/release_21_05.rst | 2 + > doc/guides/testpmd_app_ug/testpmd_funcs.rst | 11 ++++ > 3 files changed, 97 insertions(+) > > diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c > index 5bf1497..b43964d 100644 > --- a/app/test-pmd/cmdline.c > +++ b/app/test-pmd/cmdline.c > @@ -675,6 +675,9 @@ static void cmd_help_long_parsed(void *parsed_result, > "set port (port_id) ptype_mask (ptype_mask)\n" > " set packet types classification for a specific port\n\n" > > + "tx_done_cleanup (port_id) (queue_id) (free_cnt)\n" > + " Cleanup a Tx queue's mbuf on a port\n\n" > + If 'free_cnt' is '0', driver should free all possible mbufs, can you add this detail to help? > "set port (port_id) queue-region region_id (value) " > "queue_start_index (value) queue_num (value)\n" > " Set a queue region on a port\n\n" > @@ -16915,6 +16918,86 @@ cmdline_parse_inst_t cmd_showport_macs = { > }, > }; > > +/* *** tx_done_cleanup *** */ > +struct cmd_tx_done_cleanup_result { > + cmdline_fixed_string_t clean; > + cmdline_fixed_string_t port; > + uint16_t port_id; > + uint16_t queue_id; > + uint32_t free_cnt; > +}; > + > +static void > +cmd_tx_done_cleanup_parsed(void *parsed_result, > + __rte_unused struct cmdline *cl, > + __rte_unused void *data) > +{ > + struct cmd_tx_done_cleanup_result *res = parsed_result; > + uint16_t port_id = res->port_id; > + uint16_t queue_id = res->queue_id; > + uint32_t free_cnt = res->free_cnt; > + struct rte_eth_txq_info qinfo; > + int ret; > + > + if (rte_eth_tx_queue_info_get(port_id, queue_id, &qinfo)) { > + printf("Failed to get port %u Tx queue %u info!\n", > + port_id, queue_id); > + return; > + } > + > + if (qinfo.queue_state != RTE_ETH_QUEUE_STATE_STARTED) { > + printf("TX queue %u not started!\n", queue_id); > + return; > + } > + > + /* > + * rte_eth_tx_done_cleanup is a dataplane API, user must make sure > + * there are no concurrent access to the same Tx queue (like > + * rte_eth_tx_burst, rte_eth_dev_tx_queue_stop and so on) when this API > + * called. > + */ Should there be a check to see if testpmd forwarding is stared or not? This API shouldn't be allowed when forwarding is started. > + ret = rte_eth_tx_done_cleanup(port_id, queue_id, free_cnt); > + if (ret < 0) { > + printf("Failed to cleanup mbuf for port %u Tx queue %u " > + "error desc: %s(%d)\n", > + port_id, queue_id, strerror(-ret), ret); > + return; > + } > + > + printf("Cleanup port %u Tx queue %u mbuf nums: %u\n", > + port_id, queue_id, ret); > +} > + > +cmdline_parse_token_string_t cmd_tx_done_cleanup_clean = > + TOKEN_STRING_INITIALIZER(struct cmd_tx_done_cleanup_result, clean, > + "tx_done_cleanup"); > +cmdline_parse_token_string_t cmd_tx_done_cleanup_port = > + TOKEN_STRING_INITIALIZER(struct cmd_tx_done_cleanup_result, port, > + "port"); > +cmdline_parse_token_num_t cmd_tx_done_cleanup_port_id = > + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, port_id, > + UINT16); > +cmdline_parse_token_num_t cmd_tx_done_cleanup_queue_id = > + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, queue_id, > + UINT16); > +cmdline_parse_token_num_t cmd_tx_done_cleanup_free_cnt = > + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, free_cnt, > + UINT32); > + > +cmdline_parse_inst_t cmd_tx_done_cleanup = { > + .f = cmd_tx_done_cleanup_parsed, > + .data = NULL, > + .help_str = "tx_done_cleanup port <port_id> <queue_id> <free_cnt>", > + .tokens = { > + (void *)&cmd_tx_done_cleanup_clean, > + (void *)&cmd_tx_done_cleanup_port, > + (void *)&cmd_tx_done_cleanup_port_id, > + (void *)&cmd_tx_done_cleanup_queue_id, > + (void *)&cmd_tx_done_cleanup_free_cnt, > + NULL, > + }, > +}; > + > /* ******************************************************************************** */ > > /* list of instructions */ > @@ -17040,6 +17123,7 @@ cmdline_parse_ctx_t main_ctx[] = { > (cmdline_parse_inst_t *)&cmd_config_rss_reta, > (cmdline_parse_inst_t *)&cmd_showport_reta, > (cmdline_parse_inst_t *)&cmd_showport_macs, > + (cmdline_parse_inst_t *)&cmd_tx_done_cleanup, > (cmdline_parse_inst_t *)&cmd_config_burst, > (cmdline_parse_inst_t *)&cmd_config_thresh, > (cmdline_parse_inst_t *)&cmd_config_threshold, > diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst > index 4538bb9..554499f 100644 > --- a/doc/guides/rel_notes/release_21_05.rst > +++ b/doc/guides/rel_notes/release_21_05.rst > @@ -217,6 +217,8 @@ New Features > ``show port (port_id) rxq (queue_id) desc used count`` > * Added command to dump internal representation information of single flow. > ``flow dump (port_id) rule (rule_id)`` > + * Added command to cleanup a Tx queue's mbuf on a port. > + ``tx_done_cleanup port <port_id> <queue_id> <free_cnt>`` > > * **Updated ipsec-secgw sample application.** > > diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > index a736e7d..6472647 100644 > --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst > +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst > @@ -270,6 +270,17 @@ and ready to be processed by the driver on a given RX queue:: > > testpmd> show port (port_id) rxq (queue_id) desc used count > > +cleanup txq mbufs > +~~~~~~~~~~~~~~~~~~~~~~~~ > + > +Request the driver to free mbufs currently cached by the driver for a given port's > +Tx queue:: > + testpmd> tx_done_cleanup port (port_id) (queue_id) (free_cnt) > + > +.. note:: > + This command is dangerous, users must make sure there are no cucurrent access to > + the same Tx queue (link rte_eth_tx_burst, rte_eth_dev_tx_queue_stop and so on). > + > show config > ~~~~~~~~~~~ > >

1 0

[PATCH] app/testpmd: support Tx mbuf free on demand cmd
by Lijun Ou 19 Apr '21

19 Apr '21

From: Chengwen Feng <fengchengwen(a)huawei.com> This patch support tx_done_cleanup command: tx_done_cleanup port (port_id) (queue_id) (free_cnt) User must make sure there are no concurrent access to the same Tx queue (like rte_eth_tx_burst, rte_eth_dev_tx_queue_stop and so on) when this command executed. Signed-off-by: Chengwen Feng <fengchengwen(a)huawei.com> Signed-off-by: Lijun Ou <oulijun(a)huawei.com> --- app/test-pmd/cmdline.c | 91 +++++++++++++++++++++++++++++ doc/guides/rel_notes/release_21_05.rst | 2 + doc/guides/testpmd_app_ug/testpmd_funcs.rst | 7 +++ 3 files changed, 100 insertions(+) diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c index 14110eb..832ae70 100644 --- a/app/test-pmd/cmdline.c +++ b/app/test-pmd/cmdline.c @@ -36,6 +36,7 @@ #include <rte_pci.h> #include <rte_ether.h> #include <rte_ethdev.h> +#include <rte_ethdev_driver.h> #include <rte_string_fns.h> #include <rte_devargs.h> #include <rte_flow.h> @@ -675,6 +676,9 @@ static void cmd_help_long_parsed(void *parsed_result, "set port (port_id) ptype_mask (ptype_mask)\n" " set packet types classification for a specific port\n\n" + "tx_done_cleanup (port_id) (queue_id) (free_cnt)\n" + " Cleanup a tx queue's mbuf on a port\n\n" + "set port (port_id) queue-region region_id (value) " "queue_start_index (value) queue_num (value)\n" " Set a queue region on a port\n\n" @@ -16910,6 +16914,92 @@ cmdline_parse_inst_t cmd_showport_macs = { }, }; +/* *** tx_done_cleanup *** */ +struct cmd_tx_done_cleanup_result { + cmdline_fixed_string_t clean; + cmdline_fixed_string_t port; + uint16_t port_id; + uint16_t queue_id; + uint32_t free_cnt; +}; + +static void +cmd_tx_done_cleanup_parsed(void *parsed_result, + __rte_unused struct cmdline *cl, + __rte_unused void *data) +{ + struct cmd_tx_done_cleanup_result *res = parsed_result; + struct rte_eth_dev *dev; + uint16_t port_id = res->port_id; + uint16_t queue_id = res->queue_id; + uint32_t free_cnt = res->free_cnt; + int ret; + + if (!rte_eth_dev_is_valid_port(port_id)) { + printf("Invalid port_id %u\n", port_id); + return; + } + + dev = &rte_eth_devices[port_id]; + if (queue_id >= dev->data->nb_tx_queues) { + printf("Invalid TX queue_id %u\n", queue_id); + return; + } + + if (dev->data->tx_queue_state[queue_id] != + RTE_ETH_QUEUE_STATE_STARTED) { + printf("TX queue_id %u not started!\n", queue_id); + return; + } + + /* + * rte_eth_tx_done_cleanup is a dataplane API, user must make sure + * there are no concurrent access to the same Tx queue (like + * rte_eth_tx_burst, rte_eth_dev_tx_queue_stop and so on) when this API + * called. + */ + ret = rte_eth_tx_done_cleanup(port_id, queue_id, free_cnt); + if (ret < 0) { + printf("Failed to cleanup mbuf for port %u TX queue %u " + "error desc: %s(%d)\n", + port_id, queue_id, strerror(-ret), ret); + return; + } + + printf("Cleanup port %u TX queue %u mbuf nums: %u\n", + port_id, queue_id, ret); +} + +cmdline_parse_token_string_t cmd_tx_done_cleanup_clean = + TOKEN_STRING_INITIALIZER(struct cmd_tx_done_cleanup_result, clean, + "tx_done_cleanup"); +cmdline_parse_token_string_t cmd_tx_done_cleanup_port = + TOKEN_STRING_INITIALIZER(struct cmd_tx_done_cleanup_result, port, + "port"); +cmdline_parse_token_num_t cmd_tx_done_cleanup_port_id = + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, port_id, + UINT16); +cmdline_parse_token_num_t cmd_tx_done_cleanup_queue_id = + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, queue_id, + UINT16); +cmdline_parse_token_num_t cmd_tx_done_cleanup_free_cnt = + TOKEN_NUM_INITIALIZER(struct cmd_tx_done_cleanup_result, free_cnt, + UINT32); + +cmdline_parse_inst_t cmd_tx_done_cleanup = { + .f = cmd_tx_done_cleanup_parsed, + .data = NULL, + .help_str = "tx_done_cleanup port <port_id> <queue_id> <free_cnt>", + .tokens = { + (void *)&cmd_tx_done_cleanup_clean, + (void *)&cmd_tx_done_cleanup_port, + (void *)&cmd_tx_done_cleanup_port_id, + (void *)&cmd_tx_done_cleanup_queue_id, + (void *)&cmd_tx_done_cleanup_free_cnt, + NULL, + }, +}; + /* ******************************************************************************** */ /* list of instructions */ @@ -17035,6 +17125,7 @@ cmdline_parse_ctx_t main_ctx[] = { (cmdline_parse_inst_t *)&cmd_config_rss_reta, (cmdline_parse_inst_t *)&cmd_showport_reta, (cmdline_parse_inst_t *)&cmd_showport_macs, + (cmdline_parse_inst_t *)&cmd_tx_done_cleanup, (cmdline_parse_inst_t *)&cmd_config_burst, (cmdline_parse_inst_t *)&cmd_config_thresh, (cmdline_parse_inst_t *)&cmd_config_threshold, diff --git a/doc/guides/rel_notes/release_21_05.rst b/doc/guides/rel_notes/release_21_05.rst index 23f7f0b..8077573 100644 --- a/doc/guides/rel_notes/release_21_05.rst +++ b/doc/guides/rel_notes/release_21_05.rst @@ -69,6 +69,8 @@ New Features * Added command to display Rx queue used descriptor count. ``show port (port_id) rxq (queue_id) desc used count`` + * Added command to cleanup a Tx queue's mbuf on a port. + ``tx_done_cleanup port <port_id> <queue_id> <free_cnt>`` Removed Items diff --git a/doc/guides/testpmd_app_ug/testpmd_funcs.rst b/doc/guides/testpmd_app_ug/testpmd_funcs.rst index f59eb8a..39281f5 100644 --- a/doc/guides/testpmd_app_ug/testpmd_funcs.rst +++ b/doc/guides/testpmd_app_ug/testpmd_funcs.rst @@ -272,6 +272,13 @@ and ready to be processed by the driver on a given RX queue:: testpmd> show port (port_id) rxq (queue_id) desc used count +cleanup txq mbufs +~~~~~~~~~~~~~~~~~~~~~~~~ + +Request the driver to free mbufs currently cached by the driver for a given port's +Tx queue:: + testpmd> tx_done_cleanup port (port_id) (queue_id) (free_cnt) + show config ~~~~~~~~~~~ -- 2.7.4

7 22

Re: [RFC PATCH 2/3] vfio/hisilicon: register the driver to vfio
by Jason Gunthorpe 19 Apr '21

19 Apr '21

On Mon, Apr 19, 2021 at 08:24:40PM +0800, liulongfang wrote: > > I'm also confused how this works securely at all, as a general rule a > > VFIO PCI driver cannot access the MMIO memory of the function it is > > planning to assign to the guest. There is a lot of danger that the > > guest could access that MMIO space one way or another. > > VF's MMIO memory is divided into two parts, one is the guest part, > and the other is the live migration part. They do not affect each other, > so there is no security problem. AFAIK there are several scenarios where a guest can access this MMIO memory using DMA even if it is not mapped into the guest for CPU access. > If pci_release_mem_regions() is not added here, > The guests using pci_request_mem_regions() will return an error. > Then, the guest will not be able to obtain the MMIO address of the VF. Which is why VFIO has this protection to prevent sharing MMIO regions on the VF assigned to the guest Jason

1 0

Re: [PATCH v7] ethdev: add queue state when retrieve queue information
by Thomas Monjalon 19 Apr '21

19 Apr '21

19/04/2021 10:57, Lijun Ou: > Currently, upper-layer application could get queue state only > through pointers such as dev->data->tx_queue_state[queue_id], > this is not the recommended way to access it. So this patch > add get queue state when call rte_eth_rx_queue_info_get and > rte_eth_tx_queue_info_get API. > > Note: After add queue_state field, the 'struct rte_eth_rxq_info' size > remains 128B, and the 'struct rte_eth_txq_info' size remains 64B, so > it could be ABI compatible. > > Signed-off-by: Chengwen Feng <fengchengwen(a)huawei.com> > Signed-off-by: Lijun Ou <oulijun(a)huawei.com> > Acked-by: Konstantin Ananyev <konstantin.ananyev(a)intel.com> Acked-by: Thomas Monjalon <thomas(a)monjalon.net>

2 1

Re: [dpdk-dev] [PATCH V6] ethdev: add queue state when retrieve queue information
by Thomas Monjalon 19 Apr '21

19 Apr '21

19/04/2021 04:03, Lijun Ou: > --- a/doc/guides/rel_notes/release_21_05.rst > +++ b/doc/guides/rel_notes/release_21_05.rst > +* **Enhanced function for getting rxq/txq info ABI.** Reword: Added queue state in queried Rx/Tx queue info > + * Added new field ``queue_state`` to ``rte_eth_rxq_info`` structure to > + provide indicated rxq queue state. s/rxq queue/Rx queue/ > + * Added new field ``queue_state`` to ``rte_eth_txq_info`` structure to > + provide indicated txq queue state. s/txq queue/Tx queue/ Thanks

2 1

Re: [dpdk-dev] [PATCH V4 0/3] add Hisilicon kunpeng CPU support
by Thomas Monjalon 19 Apr '21

19 Apr '21

16/04/2021 11:57, Lijun Ou: > Chengchang Tang (3): > config/arm: add Hisilicon kunpeng implementer > config/arm: add kunpeng920 cross target > config/arm: add kunpeng930 cross target Squashed, sorted SoCs alphabetically with others, and applied.

1 0

Re: [PATCH v3] iommu/iova: put free_iova_mem() outside of spinlock iova_rbtree_lock
by John Garry 19 Apr '21

19 Apr '21

On 19/04/2021 08:13, chenxiang wrote: > From: Xiang Chen<chenxiang66(a)hisilicon.com> > > It is not necessary to put free_iova_mem() inside of spinlock/unlock > iova_rbtree_lock which only leads to more completion for the spinlock. > It has a small promote on the performance after the change. And also > rename private_free_iova() as remove_iova() because the function will not > free iova after that change. > > Signed-off-by: Xiang Chen<chenxiang66(a)hisilicon.com> I think that you will need to resend after v5.13-rc1 comes out, however: Reviewed-by: John Garry <john.garry(a)huawei.com> > --- > drivers/iommu/

1 0