[PATCH OLK-6.6 0/5] backport cpufreq patches from linux mainline on August 31st
From: Hongye Lin <linhongye@h-partners.com> driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/9163 -------------------------------- backport cpufreq patches from linux mainline on August 31st Jie Zhan (1): cppc_cpufreq: Use desired perf if feedback ctrs are 0 or unchanged Pengjie Zhang (4): Revert "cpufreq: CPPC: Keep the target core awake when reading its cpufreq rate" ACPI: CPPC: add paired FFH feedback-counter read hook arm64: topology: read CPPC FFH feedback counters in one operation cpufreq: CPPC: Optimize feedback counter sampling based on BIOS FFH capability arch/arm64/kernel/topology.c | 92 +++++++++++++++++++++++++++--- drivers/acpi/cppc_acpi.c | 85 ++++++++++++++++++++++----- drivers/cpufreq/cppc_cpufreq.c | 101 +++++++++++++++++++++++++++------ include/acpi/cppc_acpi.h | 12 ++++ 4 files changed, 253 insertions(+), 37 deletions(-) -- 2.33.0
Upstream: YES Feature or Bugfix: Bugfix AR: AR20230713054251 DTS:DTS2026072919573 This reverts commit 12f136b2134d4ded731c3ef23ac08c85b9c0b1fa. Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- drivers/cpufreq/cppc_cpufreq.c | 51 ++++++---------------------------- 1 file changed, 8 insertions(+), 43 deletions(-) diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index 66b07e15b564..bfe1f56e855b 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -74,12 +74,6 @@ static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data, struct cppc_perf_fb_ctrs *fb_ctrs_t0, struct cppc_perf_fb_ctrs *fb_ctrs_t1); -struct fb_ctr_pair { - u32 cpu; - struct cppc_perf_fb_ctrs fb_ctrs_t0; - struct cppc_perf_fb_ctrs fb_ctrs_t1; -}; - /** * __cppc_scale_freq_tick - CPPC arch_freq_scale updater for frequency invariance * @cppc_fi: per-cpu CPPC FIE data. @@ -786,37 +780,9 @@ static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data, return (reference_perf * delta_delivered) / delta_reference; } -static int cppc_get_perf_ctrs_pair(void *val) -{ - struct fb_ctr_pair *fb_ctrs = val; - int cpu = fb_ctrs->cpu; - int ret; - ktime_t timeout; - - ret = cppc_get_perf_ctrs(cpu, &fb_ctrs->fb_ctrs_t0); - if (ret) - return ret; - - if (likely(!in_atomic() && !irqs_disabled())) { - /* - * Set 1ms as sampling interval, but never schedule - * to the idle task to prevent the AMU counters from - * stopping working. - */ - timeout = ktime_add_ms(ktime_get(), 1); - while (ktime_before(ktime_get(), timeout)) - cond_resched(); - } else { - pr_warn_once("CPU%d: Get rate in atomic context", cpu); - udelay(2); /* 2usec delay between sampling */ - } - - return cppc_get_perf_ctrs(cpu, &fb_ctrs->fb_ctrs_t1); -} - static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) { - struct fb_ctr_pair fb_ctrs = { .cpu = cpu, }; + struct cppc_perf_fb_ctrs fb_ctrs_t0 = {0}, fb_ctrs_t1 = {0}; struct cpufreq_policy *policy = cpufreq_cpu_get(cpu); struct cppc_cpudata *cpu_data; u64 delivered_perf; @@ -829,18 +795,17 @@ static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) cpufreq_cpu_put(policy); - if (cpu_has_amu_feat(cpu)) - ret = smp_call_on_cpu(cpu, cppc_get_perf_ctrs_pair, - &fb_ctrs, false); - else - ret = cppc_get_perf_ctrs_pair(&fb_ctrs); + ret = cppc_get_perf_ctrs(cpu, &fb_ctrs_t0); + if (ret) + return 0; + udelay(2); /* 2usec delay between sampling */ + ret = cppc_get_perf_ctrs(cpu, &fb_ctrs_t1); if (ret) return 0; - delivered_perf = cppc_perf_from_fbctrs(cpu_data, - &fb_ctrs.fb_ctrs_t0, - &fb_ctrs.fb_ctrs_t1); + delivered_perf = cppc_perf_from_fbctrs(cpu_data, &fb_ctrs_t0, + &fb_ctrs_t1); return cppc_perf_to_khz(&cpu_data->perf_caps, delivered_perf); } -- 2.33.0
From: Jie Zhan <zhanjie9@hisilicon.com> mainline inclusion from mainline-v6.13-rc1 commit c47195631960b626058c335aec31f186fa854f97 category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/9163 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- The CPPC performance feedback counters could be 0 or unchanged when the target cpu is in a low-power idle state, e.g. power-gated or clock-gated. When the counters are 0, cppc_cpufreq_get_rate() returns 0 KHz, which makes cpufreq_online() get a false error and fail to generate a cpufreq policy. When the counters are unchanged, the existing cppc_perf_from_fbctrs() returns a cached desired perf, but some platforms may update the real frequency back to the desired perf reg. For the above cases in cppc_cpufreq_get_rate(), get the latest desired perf from the CPPC reg to reflect the frequency because some platforms may update the actual frequency back there; if failed, use the cached desired perf. Fixes: 6a4fec4f6d30 ("cpufreq: cppc: cppc_cpufreq_get_rate() returns zero in all error cases.") Signed-off-by: Jie Zhan <zhanjie9@hisilicon.com> Reviewed-by: Zeng Heng <zengheng4@huawei.com> Reviewed-by: Ionela Voinescu <ionela.voinescu@arm.com> Reviewed-by: Huisong Li <lihuisong@huawei.com> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org> Conflicts: drivers/cpufreq/cppc_cpufreq.c [drivers/cpufreq/cppc_cpufreq.c: The OLK-6.6 baseline already refactored the feedback-counter sampling into cppc_get_perf_ctrs_pair() (sampling on the target CPU with smp_call_on_cpu and a ktime based sampling interval, from an earlier backport series), so the patch's mechanical extraction into cppc_get_perf_ctrs_sample() was not introduced to avoid a redundant helper. The semantic changes were transplanted onto the baseline mechanism: cppc_perf_from_fbctrs() now returns 0 on zero or unchanged deltas (instead of the cached desired perf) and leaves it to callers, and cppc_cpufreq_get_rate() falls back to the latest desired perf register value (or the cached desired perf) via the out_invalid_counters path when the counters are invalid (-EFAULT) or the computed delivered perf is 0.] Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- drivers/cpufreq/cppc_cpufreq.c | 50 +++++++++++++++++++++++++++++++--- 1 file changed, 46 insertions(+), 4 deletions(-) diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index bfe1f56e855b..9efaea2dbe83 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -113,6 +113,9 @@ static void __cppc_scale_freq_tick(struct cppc_freq_invariance *cppc_fi) perf = cppc_perf_from_fbctrs(cpu_data, &cppc_fi->prev_perf_fb_ctrs, &fb_ctrs); + if (!perf) + return; + cppc_fi->prev_perf_fb_ctrs = fb_ctrs; perf <<= SCHED_CAPACITY_SHIFT; @@ -773,13 +776,31 @@ static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data, delta_delivered = get_delta(fb_ctrs_t1->delivered, fb_ctrs_t0->delivered); - /* Check to avoid divide-by zero and invalid delivered_perf */ + /* + * Avoid divide-by zero and unchanged feedback counters. + * Leave it for callers to handle. + */ if (!delta_reference || !delta_delivered) - return cpu_data->perf_ctrls.desired_perf; + return 0; return (reference_perf * delta_delivered) / delta_reference; } +static int cppc_get_perf_ctrs_sample(int cpu, + struct cppc_perf_fb_ctrs *fb_ctrs_t0, + struct cppc_perf_fb_ctrs *fb_ctrs_t1) +{ + int ret; + + ret = cppc_get_perf_ctrs(cpu, fb_ctrs_t0); + if (ret) + return ret; + + udelay(2); /* 2usec delay between sampling */ + + return cppc_get_perf_ctrs(cpu, fb_ctrs_t1); +} + static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) { struct cppc_perf_fb_ctrs fb_ctrs_t0 = {0}, fb_ctrs_t1 = {0}; @@ -801,12 +822,33 @@ static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) udelay(2); /* 2usec delay between sampling */ ret = cppc_get_perf_ctrs(cpu, &fb_ctrs_t1); - if (ret) - return 0; + if (ret) { + if (ret == -EFAULT) + /* Any of the associated CPPC regs is 0. */ + goto out_invalid_counters; + else + return 0; + } delivered_perf = cppc_perf_from_fbctrs(cpu_data, &fb_ctrs_t0, &fb_ctrs_t1); + if (!delivered_perf) + goto out_invalid_counters; + + return cppc_perf_to_khz(&cpu_data->perf_caps, delivered_perf); + +out_invalid_counters: + /* + * Feedback counters could be unchanged or 0 when a cpu enters a + * low-power idle state, e.g. clock-gated or power-gated. + * Use desired perf for reflecting frequency. Get the latest register + * value first as some platforms may update the actual delivered perf + * there; if failed, resort to the cached desired perf. + */ + if (cppc_get_desired_perf(cpu, &delivered_perf)) + delivered_perf = cpu_data->perf_ctrls.desired_perf; + return cppc_perf_to_khz(&cpu_data->perf_caps, delivered_perf); } -- 2.33.0
mainline inclusion from mainline-v7.3-rc1 commit 9aa7df52052399f7759b70f64c6ee561f901a28a category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9163 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- cppc_get_perf_ctrs() reads the delivered and reference performance counters one at a time. Allow architectures to provide both FFH feedback counters in one operation when that either narrows the sampling window or avoids extra cross-CPU reads. Add a small FFH-specific hook for that case and fall back to the existing per-register reads when unsupported. Tested-by: Sumit Gupta <sumitg@nvidia.com> Reviewed-by: Sumit Gupta <sumitg@nvidia.com> Tested-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com> Reviewed-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com> Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Acked-by: Rafael J. Wysocki (Intel) <rafael@kernel.org> Tested-by: Jeremy Linton <jeremy.linton@arm.com> Reviewed-by: Jeremy Linton <jeremy.linton@arm.com> Signed-off-by: Will Deacon <will@kernel.org> Conflicts: drivers/acpi/cppc_acpi.c include/acpi/cppc_acpi.h [drivers/acpi/cppc_acpi.c: In cppc_get_perf_ctrs() the OLK-6.6 baseline reads the delivered and reference counters with two unchecked cpc_read() calls, so the rejected hunk was adapted by replacing those calls with cppc_read_fb_ctrs() and propagating its error via the existing out_err path.] [include/acpi/cppc_acpi.h: The !CONFIG_ACPI_CPPC_LIB inline stubs in this baseline return -ENOTSUPP instead of the -EOPNOTSUPP used in the patch context, so the new cpc_read_ffh_fb_ctrs() stub was inserted following the baseline -ENOTSUPP convention.] Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- drivers/acpi/cppc_acpi.c | 49 ++++++++++++++++++++++++++++++++++++++-- include/acpi/cppc_acpi.h | 7 ++++++ 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index f78a468bfde4..33390f7e77ee 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -982,6 +982,22 @@ int __weak cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val) return -ENOTSUPP; } +/** + * cpc_read_ffh_fb_ctrs() - Read FFH feedback counters together + * @cpunum: Target CPU + * @reg1: first CPPC register information + * @val1: place holder for first return value + * @reg2: second CPPC register information + * @val2: place holder for second return value + * + * Return: 0 on success, error code otherwise + */ +int __weak cpc_read_ffh_fb_ctrs(int cpunum, struct cpc_reg *reg1, + u64 *val1, struct cpc_reg *reg2, u64 *val2) +{ + return -EOPNOTSUPP; +} + /** * cpc_write_ffh() - Write FFH register * @cpunum: CPU number to write @@ -1398,6 +1414,33 @@ bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu) } EXPORT_SYMBOL_GPL(cppc_perf_ctrs_in_pcc_cpu); +static int cppc_read_fb_ctrs(int cpunum, + struct cpc_register_resource *delivered_reg, + struct cpc_register_resource *reference_reg, + u64 *delivered, u64 *reference) +{ + int ret; + + /* + * For FFH feedback counters, try a paired read first to reduce + * sampling skew between delivered and reference counters. Fall + * back to the existing per-register reads if unsupported. + */ + if (CPC_IN_FFH(delivered_reg) && CPC_IN_FFH(reference_reg)) { + ret = cpc_read_ffh_fb_ctrs(cpunum, + &delivered_reg->cpc_entry.reg, delivered, + &reference_reg->cpc_entry.reg, reference); + if (ret != -EOPNOTSUPP) + return ret; + } + + ret = cpc_read(cpunum, delivered_reg, delivered); + if (ret) + return ret; + + return cpc_read(cpunum, reference_reg, reference); +} + /** * cppc_perf_ctrs_in_pcc - Check if any perf counters are in a PCC region. * @@ -1463,8 +1506,10 @@ int cppc_get_perf_ctrs(int cpunum, struct cppc_perf_fb_ctrs *perf_fb_ctrs) } } - cpc_read(cpunum, delivered_reg, &delivered); - cpc_read(cpunum, reference_reg, &reference); + ret = cppc_read_fb_ctrs(cpunum, delivered_reg, reference_reg, + &delivered, &reference); + if (ret) + goto out_err; /* * Per spec, if ctr_wrap_time optional register is unsupported, then the diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h index a8e1723c2317..cdd5b3756fec 100644 --- a/include/acpi/cppc_acpi.h +++ b/include/acpi/cppc_acpi.h @@ -156,6 +156,8 @@ extern unsigned int cppc_get_transition_latency(int cpu); extern bool cpc_ffh_supported(void); extern bool cpc_supported_by_cpu(void); extern int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val); +extern int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1, + struct cpc_reg *reg2, u64 *val2); extern int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val); extern int cppc_get_epp_perf(int cpunum, u64 *epp_perf); extern int cppc_set_epp_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls, bool enable); @@ -224,6 +226,11 @@ static inline int cpc_read_ffh(int cpunum, struct cpc_reg *reg, u64 *val) { return -ENOTSUPP; } +static inline int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1, + struct cpc_reg *reg2, u64 *val2) +{ + return -ENOTSUPP; +} static inline int cpc_write_ffh(int cpunum, struct cpc_reg *reg, u64 val) { return -ENOTSUPP; -- 2.33.0
mainline inclusion from mainline-v7.3-rc1 commit 42971d5329d9304eb115faebdb71d91650ada8dc category: feature bugzilla: https://atomgit.com/openeuler/kernel/issues/9163 CVE: NA Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?i... -------------------------------- arm64 implements CPPC FFH feedback-counter reads using AMU counters. Because those counters must be sampled on the target CPU, reading the delivered and reference counters separately widens the observation window between them. Implement the paired FFH feedback-counter read hook on arm64 and sample both AMU counters together before decoding the requested CPC register values. Also factor the FFH bitfield extraction logic into a helper and reuse it from the existing single-counter FFH read path. Tested-by: Sumit Gupta <sumitg@nvidia.com> Reviewed-by: Sumit Gupta <sumitg@nvidia.com> Tested-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com> Reviewed-by: Vanshidhar Konda <vanshikonda@os.amperecomputing.com> Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Tested-by: Jeremy Linton <jeremy.linton@arm.com> Reviewed-by: Jeremy Linton <jeremy.linton@arm.com> Signed-off-by: Will Deacon <will@kernel.org> Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- arch/arm64/kernel/topology.c | 92 ++++++++++++++++++++++++++++++++---- 1 file changed, 84 insertions(+), 8 deletions(-) diff --git a/arch/arm64/kernel/topology.c b/arch/arm64/kernel/topology.c index d78c917b3f9c..c17b7ad4a997 100644 --- a/arch/arm64/kernel/topology.c +++ b/arch/arm64/kernel/topology.c @@ -287,6 +287,16 @@ core_initcall(init_amu_fie); #ifdef CONFIG_ACPI_CPPC_LIB #include <acpi/cppc_acpi.h> +struct amu_ffh_ctrs { + u64 corecnt; + u64 constcnt; +}; + +enum cpc_ffh_ctr_id { + CPC_FFH_CTR_CORE = 0x0, + CPC_FFH_CTR_CONST = 0x1, +}; + static void cpu_read_corecnt(void *val) { /* @@ -311,7 +321,7 @@ static void cpu_read_constcnt(void *val) } static inline -int counters_read_on_cpu(int cpu, smp_call_func_t func, u64 *val) +int counters_read_on_cpu(int cpu, smp_call_func_t func, void *val) { /* * Abort call on counterless CPU. @@ -361,24 +371,90 @@ bool cpc_ffh_supported(void) return true; } +static void amu_read_core_const_ctrs(void *val) +{ + struct amu_ffh_ctrs *ctrs = val; + + /* + * cpu_read_constcnt() incurs slight latency due to the + * ARM64_WORKAROUND_2457168 check. Read it first to minimize + * the sampling skew between the const and core counters. + */ + cpu_read_constcnt(&ctrs->constcnt); + cpu_read_corecnt(&ctrs->corecnt); +} + +static u64 cpc_ffh_extract_bits(const struct cpc_reg *reg, u64 val) +{ + val &= GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, + reg->bit_offset); + val >>= reg->bit_offset; + + return val; +} + +static void cpc_ffh_ctr_value(const struct cpc_reg *reg, + const struct amu_ffh_ctrs *ctrs, u64 *val) +{ + switch ((u64)reg->address) { + case CPC_FFH_CTR_CORE: + *val = ctrs->corecnt; + break; + case CPC_FFH_CTR_CONST: + *val = ctrs->constcnt; + break; + } + + *val = cpc_ffh_extract_bits(reg, *val); +} + +static bool is_amu_ctr_reg(const struct cpc_reg *reg) +{ + return reg->address == CPC_FFH_CTR_CORE || + reg->address == CPC_FFH_CTR_CONST; +} + +int cpc_read_ffh_fb_ctrs(int cpu, struct cpc_reg *reg1, u64 *val1, + struct cpc_reg *reg2, u64 *val2) +{ + struct amu_ffh_ctrs ctrs; + int ret; + + if (!is_amu_ctr_reg(reg1) || !is_amu_ctr_reg(reg2)) + return -EINVAL; + + ret = counters_read_on_cpu(cpu, amu_read_core_const_ctrs, &ctrs); + if (ret) { + /* + * If AMU is unsupported (-EOPNOTSUPP), translate the error + * to -ENODEV. This explicitly tells the generic CPPC layer + * to abort immediately and avoid falling back to pointless + * single-counter reads. + */ + return ret == -EOPNOTSUPP ? -ENODEV : ret; + } + + cpc_ffh_ctr_value(reg1, &ctrs, val1); + cpc_ffh_ctr_value(reg2, &ctrs, val2); + + return 0; +} + int cpc_read_ffh(int cpu, struct cpc_reg *reg, u64 *val) { int ret = -EOPNOTSUPP; switch ((u64)reg->address) { - case 0x0: + case CPC_FFH_CTR_CORE: ret = counters_read_on_cpu(cpu, cpu_read_corecnt, val); break; - case 0x1: + case CPC_FFH_CTR_CONST: ret = counters_read_on_cpu(cpu, cpu_read_constcnt, val); break; } - if (!ret) { - *val &= GENMASK_ULL(reg->bit_offset + reg->bit_width - 1, - reg->bit_offset); - *val >>= reg->bit_offset; - } + if (!ret) + *val = cpc_ffh_extract_bits(reg, *val); return ret; } -- 2.33.0
driver inclusion category: bugfix bugzilla: https://atomgit.com/openeuler/kernel/issues/9163 -------------------------------- This patch adopts differentiated performance counter sampling strategies, depending on how the BIOS exposes AMU counter values. - FFH-supported BIOS: If the BIOS exposes the counters in the FFH address space, they can be accessed highly efficiently. In this case, cppc_get_perf_ctrs() reads both counters in a single IPI on the target core (e.g., via cpc_read_ffh_fb_ctrs on ARM64). Since they are sampled together quickly, a short udelay(2) window is sufficient, completely avoiding unnecessary cross-CPU scheduling overhead. - Legacy / Non-FFH BIOS: If it is a legacy BIOS (counters exposed via PCC or system memory), we preserve the existing frequency reading behavior. The driver routes the sampling through smp_call_on_cpu() and explicitly waits 1ms using cond_resched() to amortize the cpc_read() latency and jitter under memory pressure. Fixes: 12f136b2134d ("cpufreq: CPPC: Keep the target core awake when reading its cpufreq rate") Signed-off-by: Pengjie Zhang <zhangpengjie2@huawei.com> Signed-off-by: Hongye Lin <linhongye@h-partners.com> --- drivers/acpi/cppc_acpi.c | 38 ++++++++++++------ drivers/cpufreq/cppc_cpufreq.c | 72 +++++++++++++++++++++++++++++++--- include/acpi/cppc_acpi.h | 5 +++ 3 files changed, 98 insertions(+), 17 deletions(-) diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c index 33390f7e77ee..0ec2ea6a7682 100644 --- a/drivers/acpi/cppc_acpi.c +++ b/drivers/acpi/cppc_acpi.c @@ -1397,20 +1397,9 @@ EXPORT_SYMBOL_GPL(cppc_get_perf_caps); bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu) { struct cpc_desc *cpc_desc = per_cpu(cpc_desc_ptr, cpu); - struct cpc_register_resource *ref_perf_reg; - - /* - * If reference perf register is not supported then we should use the - * nominal perf value - */ - ref_perf_reg = &cpc_desc->cpc_regs[REFERENCE_PERF]; - if (!CPC_SUPPORTED(ref_perf_reg)) - ref_perf_reg = &cpc_desc->cpc_regs[NOMINAL_PERF]; return CPC_IN_PCC(&cpc_desc->cpc_regs[DELIVERED_CTR]) || - CPC_IN_PCC(&cpc_desc->cpc_regs[REFERENCE_CTR]) || - CPC_IN_PCC(&cpc_desc->cpc_regs[CTR_WRAP_TIME]) || - CPC_IN_PCC(ref_perf_reg); + CPC_IN_PCC(&cpc_desc->cpc_regs[REFERENCE_CTR]); } EXPORT_SYMBOL_GPL(cppc_perf_ctrs_in_pcc_cpu); @@ -1463,6 +1452,31 @@ bool cppc_perf_ctrs_in_pcc(void) } EXPORT_SYMBOL_GPL(cppc_perf_ctrs_in_pcc); +/** + * cppc_fb_ctrs_in_ffh - Check if any feedback counters are in a FFH region. + * + * Return: true if any of the counters are in FFH regions, false otherwise + */ +bool cppc_fb_ctrs_in_ffh(void) +{ + int cpu; + + for_each_present_cpu(cpu) { + struct cpc_desc *cpc_desc; + + cpc_desc = per_cpu(cpc_desc_ptr, cpu); + if (!cpc_desc) + continue; + + if (CPC_IN_FFH(&cpc_desc->cpc_regs[DELIVERED_CTR]) || + CPC_IN_FFH(&cpc_desc->cpc_regs[REFERENCE_CTR])) + return true; + } + + return false; +} +EXPORT_SYMBOL_GPL(cppc_fb_ctrs_in_ffh); + /** * cppc_get_perf_ctrs - Read a CPU's performance feedback counters. * @cpunum: CPU from which to read counters. diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c index 9efaea2dbe83..f8b69ce42d74 100644 --- a/drivers/cpufreq/cppc_cpufreq.c +++ b/drivers/cpufreq/cppc_cpufreq.c @@ -27,6 +27,7 @@ #include <acpi/cppc_acpi.h> static bool boost_supported; +static bool ffh_supported; struct cppc_workaround_oem_info { char oem_id[ACPI_OEM_ID_SIZE + 1]; @@ -74,6 +75,12 @@ static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data, struct cppc_perf_fb_ctrs *fb_ctrs_t0, struct cppc_perf_fb_ctrs *fb_ctrs_t1); +struct fb_ctr_pair { + u32 cpu; + struct cppc_perf_fb_ctrs *fb_ctrs_t0; + struct cppc_perf_fb_ctrs *fb_ctrs_t1; +}; + /** * __cppc_scale_freq_tick - CPPC arch_freq_scale updater for frequency invariance * @cppc_fi: per-cpu CPPC FIE data. @@ -801,6 +808,47 @@ static int cppc_get_perf_ctrs_sample(int cpu, return cppc_get_perf_ctrs(cpu, fb_ctrs_t1); } +static int cppc_get_perf_ctrs_pair(void *val) +{ + struct fb_ctr_pair *fb_ctrs = val; + int cpu = fb_ctrs->cpu; + int ret; + ktime_t timeout; + + ret = cppc_get_perf_ctrs(cpu, fb_ctrs->fb_ctrs_t0); + if (ret) + return ret; + + if (likely(!in_atomic() && !irqs_disabled())) { + /* + * Set 1ms as sampling interval, but never schedule + * to the idle task to prevent the AMU counters from + * stopping working. + */ + timeout = ktime_add_ms(ktime_get(), 1); + while (ktime_before(ktime_get(), timeout)) + cond_resched(); + } else { + pr_warn_once("CPU%d: Get rate in atomic context", cpu); + udelay(2); /* 2usec delay between sampling */ + } + + return cppc_get_perf_ctrs(cpu, fb_ctrs->fb_ctrs_t1); +} + +static int cppc_get_perf_ctrs_on_cpu(unsigned int cpu, + struct cppc_perf_fb_ctrs *fb_ctrs_t0, + struct cppc_perf_fb_ctrs *fb_ctrs_t1) +{ + struct fb_ctr_pair fb_ctrs = { + .cpu = cpu, + .fb_ctrs_t0 = fb_ctrs_t0, + .fb_ctrs_t1 = fb_ctrs_t1, + }; + + return smp_call_on_cpu(cpu, cppc_get_perf_ctrs_pair, &fb_ctrs, false); +} + static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) { struct cppc_perf_fb_ctrs fb_ctrs_t0 = {0}, fb_ctrs_t1 = {0}; @@ -816,12 +864,24 @@ static unsigned int cppc_cpufreq_get_rate(unsigned int cpu) cpufreq_cpu_put(policy); - ret = cppc_get_perf_ctrs(cpu, &fb_ctrs_t0); - if (ret) - return 0; - udelay(2); /* 2usec delay between sampling */ + /* + * Pick the feedback-counter sampling strategy from how BIOS exposes + * the counters (ffh_supported is cached once in cppc_cpufreq_init): + * + * - FFH : cppc_get_perf_ctrs() reads both counters in a single IPI on + * the target core, so they are sampled together and a short udelay(2) + * window suffices. + * + * - non-FFH (PCC / system memory): run t0/window/t1 on the target core + * via smp_call_on_cpu() and wait 1ms with cond_resched() to amortize + * cpc_read() latency jitter under memory pressure (udelay(2) in atomic + * context). + */ + if (ffh_supported) + ret = cppc_get_perf_ctrs_sample(cpu, &fb_ctrs_t0, &fb_ctrs_t1); + else + ret = cppc_get_perf_ctrs_on_cpu(cpu, &fb_ctrs_t0, &fb_ctrs_t1); - ret = cppc_get_perf_ctrs(cpu, &fb_ctrs_t1); if (ret) { if (ret == -EFAULT) /* Any of the associated CPPC regs is 0. */ @@ -1072,6 +1132,8 @@ static int __init cppc_cpufreq_init(void) cppc_freq_invariance_init(); populate_efficiency_class(); + ffh_supported = cppc_fb_ctrs_in_ffh(); + ret = cpufreq_register_driver(&cppc_cpufreq_driver); if (ret) cppc_freq_invariance_exit(); diff --git a/include/acpi/cppc_acpi.h b/include/acpi/cppc_acpi.h index cdd5b3756fec..4d189d5c0abf 100644 --- a/include/acpi/cppc_acpi.h +++ b/include/acpi/cppc_acpi.h @@ -147,6 +147,7 @@ extern int cppc_set_enable(int cpu, bool enable); extern int cppc_get_perf_caps(int cpu, struct cppc_perf_caps *caps); extern bool cppc_perf_ctrs_in_pcc_cpu(unsigned int cpu); extern bool cppc_perf_ctrs_in_pcc(void); +extern bool cppc_fb_ctrs_in_ffh(void); extern unsigned int cppc_perf_to_khz(struct cppc_perf_caps *caps, unsigned int perf); extern unsigned int cppc_khz_to_perf(struct cppc_perf_caps *caps, unsigned int freq); extern bool acpi_cpc_valid(void); @@ -206,6 +207,10 @@ static inline bool cppc_perf_ctrs_in_pcc(void) { return false; } +static inline bool cppc_fb_ctrs_in_ffh(void) +{ + return false; +} static inline bool acpi_cpc_valid(void) { return false; -- 2.33.0
反馈： 您发送到kernel@openeuler.org的补丁/补丁集，已成功转换为PR！ PR链接地址： https://atomgit.com/openeuler/kernel/merge_requests/27028 邮件列表地址：https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/CDZ... FeedBack: The patch(es) which you have sent to kernel@openeuler.org mailing list has been converted to a pull request successfully! Pull request link: https://atomgit.com/openeuler/kernel/merge_requests/27028 Mailing list address: https://mailweb.openeuler.org/archives/list/kernel@openeuler.org/message/CDZ...
participants (2)
-
patchwork bot -
Pengjie Zhang