October 2021 - Kernel - mailweb.openeuler.org

perf spe-c2c record失败的问题
by haoxin 15 Dec '21

15 Dec '21

您好，我采用最细的openEuler-1.0测试perf spe-c2c的功能，但是发现/sys/devices下没有arm_spe_0这个目录执行perf spe-c2c record时显示： event syntax error: 'arm_spe_0/ts_enable=1,pct_enable=1,pa_enable=1,load_filter=1,jitter=1,store_filter=1,min_latency=0/' \___ Cannot find PMU `arm_spe_0'. Missing kernel support? 想问下您这是为什么？需要哪里开启spe功能吗？

3 2

[PATCH openEuler-1.0-LTS 01/19] nbd: don't handle response without a corresponding request message
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Yu Kuai <yukuai3(a)huawei.com> mainline inclusion from mainline-next-20211018 commit b5644a3a79bf3be5f1238db1b2f241374b27b0f0 category: bugfix bugzilla: 49890 CVE: NA --------------------------- While handling a response message from server, nbd_read_stat() will try to get request by tag, and then complete the request. However, this is problematic if nbd haven't sent a corresponding request message: t1 t2 submit_bio nbd_queue_rq blk_mq_start_request recv_work nbd_read_stat blk_mq_tag_to_rq blk_mq_complete_request nbd_send_cmd Thus add a new cmd flag 'NBD_CMD_INFLIGHT', it will be set in nbd_send_cmd() and checked in nbd_read_stat(). Noted that this patch can't fix that blk_mq_tag_to_rq() might return a freed request, and this will be fixed in following patches. Signed-off-by: Yu Kuai <yukuai3(a)huawei.com> Reviewed-by: Ming Lei <ming.lei(a)redhat.com> Reviewed-by: Josef Bacik <josef(a)toxicpanda.com> Link: https://lore.kernel.org/r/20210916093350.1410403-2-yukuai3@huawei.com Signed-off-by: Jens Axboe <axboe(a)kernel.dk> Reviewed-by: Jason Yan <yanaijie(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- drivers/block/nbd.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 6a72c07ce3cba..05153b84d5400 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -120,6 +120,12 @@ struct nbd_device { }; #define NBD_CMD_REQUEUED 1 +/* + * This flag will be set if nbd_queue_rq() succeed, and will be checked and + * cleared in completion. Both setting and clearing of the flag are protected + * by cmd->lock. + */ +#define NBD_CMD_INFLIGHT 2 struct nbd_cmd { struct nbd_device *nbd; @@ -369,6 +375,7 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req, if (!mutex_trylock(&cmd->lock)) return BLK_EH_RESET_TIMER; + __clear_bit(NBD_CMD_INFLIGHT, &cmd->flags); if (!refcount_inc_not_zero(&nbd->config_refs)) { cmd->status = BLK_STS_TIMEOUT; mutex_unlock(&cmd->lock); @@ -674,6 +681,12 @@ static struct nbd_cmd *nbd_read_stat(struct nbd_device *nbd, int index) cmd = blk_mq_rq_to_pdu(req); mutex_lock(&cmd->lock); + if (!__test_and_clear_bit(NBD_CMD_INFLIGHT, &cmd->flags)) { + dev_err(disk_to_dev(nbd->disk), "Suspicious reply %d (status %u flags %lu)", + tag, cmd->status, cmd->flags); + ret = -ENOENT; + goto out; + } if (cmd->cmd_cookie != nbd_handle_to_cookie(handle)) { dev_err(disk_to_dev(nbd->disk), "Double reply on req %p, cmd_cookie %u, handle cookie %u\n", req, cmd->cmd_cookie, nbd_handle_to_cookie(handle)); @@ -768,6 +781,7 @@ static void nbd_clear_req(struct request *req, void *data, bool reserved) struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); mutex_lock(&cmd->lock); + __clear_bit(NBD_CMD_INFLIGHT, &cmd->flags); cmd->status = BLK_STS_IOERR; mutex_unlock(&cmd->lock); @@ -903,7 +917,13 @@ static int nbd_handle_cmd(struct nbd_cmd *cmd, int index) * returns EAGAIN can be retried on a different socket. */ ret = nbd_send_cmd(nbd, cmd, index); - if (ret == -EAGAIN) { + /* + * Access to this flag is protected by cmd->lock, thus it's safe to set + * the flag after nbd_send_cmd() succeed to send request to server. + */ + if (!ret) + __set_bit(NBD_CMD_INFLIGHT, &cmd->flags); + else if (ret == -EAGAIN) { dev_err_ratelimited(disk_to_dev(nbd->disk), "Request send failed, requeueing\n"); nbd_mark_nsock_dead(nbd, nsock, 1); -- 2.25.1

1 18

[PATCH openEuler-1.0-LTS 1/5] numa: Move the management structures for cdm nodes to ddr
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Wang Wensheng <wangwensheng4(a)huawei.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4D63I CVE: NA ------------------------------------------------- The cdm nodes are easiler to raise an ECC error and it may cause the kernel crash if the essential structures went wrong. So move the management structures for hbm nodes to the ddr nodes of the same partion to reduce the probability of kernel crashes. Signed-off-by: Wang Wensheng <wangwensheng4(a)huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- arch/arm64/Kconfig | 10 ++++++++ arch/arm64/mm/numa.c | 54 +++++++++++++++++++++++++++++++++++++++- include/linux/nodemask.h | 7 ++++++ mm/sparse.c | 8 +++--- 4 files changed, 75 insertions(+), 4 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 9d49b9524e1d4..2f34aef79179e 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -1470,6 +1470,16 @@ config ASCEND_SHARE_POOL help This feature allows multiple processes to share virtual memory both in kernel and user level, which is only enabled for ascend platform. + +config ASCEND_CLEAN_CDM + bool "move the management structure for HBM to DDR" + def_bool n + depends on COHERENT_DEVICE + help + The cdm nodes sometimes are easiler to raise an ECC error and it may + cause the kernel crash if the essential structures went wrong. So move + the management structures for hbm nodes to the ddr nodes of the same + partion to reduce the probability of kernel crashes. endif endmenu diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c index a9d3ad5ee0cc3..a194bad6fdfcf 100644 --- a/arch/arm64/mm/numa.c +++ b/arch/arm64/mm/numa.c @@ -45,6 +45,57 @@ inline int arch_check_node_cdm(int nid) return node_isset(nid, cdmmask); } +#ifdef CONFIG_ASCEND_CLEAN_CDM +/** + * cdm_node_to_ddr_node - Convert the cdm node to the ddr node of the + * same partion. + * @nid: input node ID + * + * Here is a typical memory topology in usage. + * There are some DDR and HBM in each partion and DDRs present at first, then + * come all the HBMs of the first partion, then HBMs of the second partion, etc. + * + * ------------------------- + * | P0 | P1 | + * ----------- | ----------- + * |node0 DDR| | |node1 DDR| + * |---------- | ----------| + * |node2 HBM| | |node4 HBM| + * |---------- | ----------| + * |node3 HBM| | |node5 HBM| + * ----------- | ----------- + * + * Return: + * This function returns a ddr node which is of the same partion with the input + * node if the input node is a HBM node. + * The input nid is returned if it is a DDR node or if the memory topology of + * the system doesn't apply to the above model. + */ +int __init cdm_node_to_ddr_node(int nid) +{ + nodemask_t ddr_mask; + int nr_ddr, cdm_per_part, fake_nid; + int nr_cdm = nodes_weight(cdmmask); + + if (!nr_cdm || nodes_empty(numa_nodes_parsed)) + return nid; + + if (!node_isset(nid, cdmmask)) + return nid; + + nodes_xor(ddr_mask, cdmmask, numa_nodes_parsed); + nr_ddr = nodes_weight(ddr_mask); + cdm_per_part = nr_cdm / nr_ddr ? : 1; + + fake_nid = (nid - nr_ddr) / cdm_per_part; + fake_nid = !node_isset(fake_nid, cdmmask) ? fake_nid : nid; + + pr_info("nid: %d, fake_nid: %d\n", nid, fake_nid); + + return fake_nid; +} +#endif + static int __init cdm_nodes_setup(char *s) { int nid; @@ -264,11 +315,12 @@ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) u64 nd_pa; void *nd; int tnid; + int fake_nid = cdm_node_to_ddr_node(nid); if (start_pfn >= end_pfn) pr_info("Initmem setup node %d [<memory-less node>]\n", nid); - nd_pa = memblock_alloc_try_nid(nd_size, SMP_CACHE_BYTES, nid); + nd_pa = memblock_alloc_try_nid(nd_size, SMP_CACHE_BYTES, fake_nid); nd = __va(nd_pa); /* report and initialize */ diff --git a/include/linux/nodemask.h b/include/linux/nodemask.h index 41fb047bdba80..7c0571b95ce4d 100644 --- a/include/linux/nodemask.h +++ b/include/linux/nodemask.h @@ -508,6 +508,12 @@ static inline int node_random(const nodemask_t *mask) #ifdef CONFIG_COHERENT_DEVICE extern int arch_check_node_cdm(int nid); +#ifdef CONFIG_ASCEND_CLEAN_CDM +extern int cdm_node_to_ddr_node(int nid); +#else +static inline int cdm_node_to_ddr_node(int nid) { return nid; } +#endif + static inline nodemask_t system_mem_nodemask(void) { nodemask_t system_mem; @@ -551,6 +557,7 @@ static inline void node_clear_state_cdm(int node) #else static inline int arch_check_node_cdm(int nid) { return 0; } +static inline int cdm_node_to_ddr_node(int nid) { return nid; } static inline nodemask_t system_mem_nodemask(void) { diff --git a/mm/sparse.c b/mm/sparse.c index 62ae3880a9add..f19d2ca250cee 100644 --- a/mm/sparse.c +++ b/mm/sparse.c @@ -458,21 +458,23 @@ static void __init sparse_init_nid(int nid, unsigned long pnum_begin, { unsigned long pnum, usemap_longs, *usemap; struct page *map; + int fake_nid = cdm_node_to_ddr_node(nid); usemap_longs = BITS_TO_LONGS(SECTION_BLOCKFLAGS_BITS); - usemap = sparse_early_usemaps_alloc_pgdat_section(NODE_DATA(nid), + usemap = sparse_early_usemaps_alloc_pgdat_section(NODE_DATA(fake_nid), usemap_size() * map_count); if (!usemap) { pr_err("%s: node[%d] usemap allocation failed", __func__, nid); goto failed; } - sparse_buffer_init(map_count * section_map_size(), nid); + + sparse_buffer_init(map_count * section_map_size(), fake_nid); for_each_present_section_nr(pnum_begin, pnum) { if (pnum >= pnum_end) break; - map = sparse_mem_map_populate(pnum, nid, NULL); + map = sparse_mem_map_populate(pnum, fake_nid, NULL); if (!map) { pr_err("%s: node[%d] memory map backing failed. Some memory will not be available.", __func__, nid); -- 2.25.1

1 4

[PATCH openEuler-1.0-LTS 1/4] perf: hisi: Add support for HiSilicon SoC PMU driver dt probe
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Fang Lijun <fanglijun3(a)huawei.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4D4WR CVE: NA --------------------------- Add support for hisi PMU driver dt probe, Fix its compile error when disable CONFIG_ACPI. Signed-off-by: Fang Lijun <fanglijun3(a)huawei.com> Reviewed-by: Hanjun Guo <guohanjun(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- drivers/perf/Kconfig | 2 +- drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c | 1 + drivers/perf/hisilicon/hisi_uncore_hha_pmu.c | 23 +++++++++++++++---- drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c | 20 +++++++++++++--- 4 files changed, 38 insertions(+), 8 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 92be6a36a128f..d4b9681418f88 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -72,7 +72,7 @@ config ARM_DSU_PMU config HISI_PMU bool "HiSilicon SoC PMU" - depends on ARM64 && ACPI + depends on ARM64 help Support for HiSilicon SoC uncore performance monitoring unit (PMU), such as: L3C, HHA and DDRC. diff --git a/drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c b/drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c index 090667d487504..3f3f4ab3aacce 100644 --- a/drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c +++ b/drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c @@ -17,6 +17,7 @@ #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/list.h> +#include <linux/of.h> #include <linux/platform_device.h> #include <linux/smp.h> diff --git a/drivers/perf/hisilicon/hisi_uncore_hha_pmu.c b/drivers/perf/hisilicon/hisi_uncore_hha_pmu.c index c35bc248db7e7..4dd4d6b650aed 100644 --- a/drivers/perf/hisilicon/hisi_uncore_hha_pmu.c +++ b/drivers/perf/hisilicon/hisi_uncore_hha_pmu.c @@ -17,6 +17,7 @@ #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/list.h> +#include <linux/of.h> #include <linux/platform_device.h> #include <linux/smp.h> @@ -235,20 +236,34 @@ static const struct acpi_device_id hisi_hha_pmu_acpi_match[] = { }; MODULE_DEVICE_TABLE(acpi, hisi_hha_pmu_acpi_match); -static int hisi_hha_pmu_init_data(struct platform_device *pdev, +#ifdef CONFIG_ACPI +static int hisi_hha_pmu_init_index(struct platform_device *pdev, struct hisi_pmu *hha_pmu) { - unsigned long long id; - struct resource *res; acpi_status status; + unsigned long long id; status = acpi_evaluate_integer(ACPI_HANDLE(&pdev->dev), - "_UID", NULL, &id); + "_UID", NULL, &id); if (ACPI_FAILURE(status)) return -EINVAL; hha_pmu->index_id = id; + return 0; +} +#endif + +static int hisi_hha_pmu_init_data(struct platform_device *pdev, + struct hisi_pmu *hha_pmu) +{ + struct resource *res; + +#ifdef CONFIG_ACPI + if (hisi_hha_pmu_init_index(pdev, hha_pmu)) + dev_info(&pdev->dev, "Can not init index id by acpi!\n"); +#endif + /* * Use SCCL_ID and UID to identify the HHA PMU, while * SCCL_ID is in MPIDR[aff2]. diff --git a/drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c b/drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c index 6ce1d69c63198..4a42926800e50 100644 --- a/drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c +++ b/drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c @@ -17,6 +17,7 @@ #include <linux/interrupt.h> #include <linux/irq.h> #include <linux/list.h> +#include <linux/of.h> #include <linux/platform_device.h> #include <linux/smp.h> @@ -234,20 +235,33 @@ static const struct acpi_device_id hisi_l3c_pmu_acpi_match[] = { }; MODULE_DEVICE_TABLE(acpi, hisi_l3c_pmu_acpi_match); -static int hisi_l3c_pmu_init_data(struct platform_device *pdev, +#ifdef CONFIG_ACPI +static int hisi_l3c_pmu_init_index(struct platform_device *pdev, struct hisi_pmu *l3c_pmu) { unsigned long long id; - struct resource *res; acpi_status status; status = acpi_evaluate_integer(ACPI_HANDLE(&pdev->dev), - "_UID", NULL, &id); + "_UID", NULL, &id); if (ACPI_FAILURE(status)) return -EINVAL; l3c_pmu->index_id = id; + return 0; +} +#endif + +static int hisi_l3c_pmu_init_data(struct platform_device *pdev, + struct hisi_pmu *l3c_pmu) +{ + struct resource *res; + +#ifdef CONFIG_ACPI + if (hisi_l3c_pmu_init_index(pdev, l3c_pmu)) + dev_info(&pdev->dev, "Can not init index id by acpi!"); +#endif /* * Use the SCCL_ID and CCL_ID to identify the L3C PMU, while * SCCL_ID is in MPIDR[aff2] and CCL_ID is in MPIDR[aff1]. -- 2.25.1

1 3

[PATCH openEuler-1.0-LTS 1/7] corelockup: Add support of cpu core hang check
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Dong Kai <dongkai11(a)huawei.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4F3V1 CVE: NA -------------------------------- The softlockup and hardlockup detector only check the status of the cpu which it resides. If certain cpu core suspends, they are both not works. There is no any valid log but the cpu already abnormal and brings a lot of problems of system. To detect this case, we add the corelockup detector. First we use whether cpu core can responds to nmi as a sectence to determine if it is suspended. Then things is simple. Per cpu core maintains it's nmi interrupt counts and detector the nmi_counts of next cpu core. If the nmi interrupt counts not changed any more which means it can't respond nmi normally, we regard it as suspend. To ensure robustness, only consecutive lost nmi more than two times then trigger the warn. The detection chain is as following: cpu0->cpu1->...->cpuN->cpu0 Signed-off-by: Dong Kai <dongkai11(a)huawei.com> Reviewed-by: Kuohai Xu <xukuohai(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> Reviewed-by: Ding Tianhong <dingtianhong(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- include/linux/nmi.h | 6 ++ kernel/watchdog.c | 15 +++- kernel/watchdog_hld.c | 165 ++++++++++++++++++++++++++++++++++++++++++ lib/Kconfig.debug | 8 ++ 4 files changed, 192 insertions(+), 2 deletions(-) diff --git a/include/linux/nmi.h b/include/linux/nmi.h index 84f324d65068b..745d66c36e244 100644 --- a/include/linux/nmi.h +++ b/include/linux/nmi.h @@ -124,6 +124,12 @@ static inline int hardlockup_detector_perf_init(void) { return 0; } # endif #endif +#ifdef CONFIG_CORELOCKUP_DETECTOR +extern void corelockup_detector_init(void); +extern void corelockup_detector_online_cpu(unsigned int cpu); +extern void corelockup_detector_offline_cpu(unsigned int cpu); +#endif + void watchdog_nmi_stop(void); void watchdog_nmi_start(void); int watchdog_nmi_probe(void); diff --git a/kernel/watchdog.c b/kernel/watchdog.c index 0dd17265dcbd4..8b54fd30a597f 100644 --- a/kernel/watchdog.c +++ b/kernel/watchdog.c @@ -551,15 +551,23 @@ static void softlockup_start_all(void) int lockup_detector_online_cpu(unsigned int cpu) { - if (cpumask_test_cpu(cpu, &watchdog_allowed_mask)) + if (cpumask_test_cpu(cpu, &watchdog_allowed_mask)) { watchdog_enable(cpu); +#ifdef CONFIG_CORELOCKUP_DETECTOR + corelockup_detector_online_cpu(cpu); +#endif + } return 0; } int lockup_detector_offline_cpu(unsigned int cpu) { - if (cpumask_test_cpu(cpu, &watchdog_allowed_mask)) + if (cpumask_test_cpu(cpu, &watchdog_allowed_mask)) { watchdog_disable(cpu); +#ifdef CONFIG_CORELOCKUP_DETECTOR + corelockup_detector_offline_cpu(cpu); +#endif + } return 0; } @@ -783,4 +791,7 @@ void __init lockup_detector_init(void) if (!watchdog_nmi_probe()) nmi_watchdog_available = true; lockup_detector_setup(); +#ifdef CONFIG_CORELOCKUP_DETECTOR + corelockup_detector_init(); +#endif } diff --git a/kernel/watchdog_hld.c b/kernel/watchdog_hld.c index 904a95262fcf6..e965c31958203 100644 --- a/kernel/watchdog_hld.c +++ b/kernel/watchdog_hld.c @@ -39,6 +39,163 @@ notrace void __weak arch_touch_nmi_watchdog(void) } EXPORT_SYMBOL(arch_touch_nmi_watchdog); +#ifdef CONFIG_CORELOCKUP_DETECTOR +/* + * The softlockup and hardlockup detector only check the status + * of the cpu which it resides. If certain cpu core suspends, + * they are both not works. There is no any valid log but the + * cpu already abnormal and brings a lot of problems of system. + * To detect this case, we add the corelockup detector. + * + * First we use whether cpu core can responds to nmi as a sectence + * to determine if it is suspended. Then things is simple. Per cpu + * core maintains it's nmi interrupt counts and detector the + * nmi_counts of next cpu core. If the nmi interrupt counts not + * changed any more which means it can't respond nmi normally, we + * regard it as suspend. + * + * To ensure robustness, only consecutive lost nmi more than two + * times then trigger the warn. + * + * The detection chain is as following: + * cpu0->cpu1->...->cpuN->cpu0 + * + * detector_cpu: the target cpu to detector of current cpu + * nmi_interrupts: the nmi counts of current cpu + * nmi_cnt_saved: saved nmi counts of detector_cpu + * nmi_cnt_missed: the nmi consecutive miss counts of detector_cpu + */ +static DEFINE_PER_CPU(unsigned int, detector_cpu); +static DEFINE_PER_CPU(unsigned long, nmi_interrupts); +static DEFINE_PER_CPU(unsigned long, nmi_cnt_saved); +static DEFINE_PER_CPU(unsigned long, nmi_cnt_missed); +static DEFINE_PER_CPU(bool, core_watchdog_warn); + +static void watchdog_nmi_interrupts(void) +{ + __this_cpu_inc(nmi_interrupts); +} + +static void corelockup_status_copy(unsigned int from, unsigned int to) +{ + per_cpu(nmi_cnt_saved, to) = per_cpu(nmi_cnt_saved, from); + per_cpu(nmi_cnt_missed, to) = per_cpu(nmi_cnt_missed, from); + + /* always update detector cpu at the end */ + per_cpu(detector_cpu, to) = per_cpu(detector_cpu, from); +} + +static void corelockup_status_init(unsigned int cpu, unsigned int target) +{ + /* + * initialize saved count to max to avoid unnecessary misjudge + * caused by delay running of nmi on target cpu + */ + per_cpu(nmi_cnt_saved, cpu) = ULONG_MAX; + per_cpu(nmi_cnt_missed, cpu) = 0; + + /* always update detector cpu at the end */ + per_cpu(detector_cpu, cpu) = target; +} + +void __init corelockup_detector_init(void) +{ + unsigned int cpu, next; + + /* detector cpu is set to the next valid logically one */ + for_each_cpu_and(cpu, &watchdog_cpumask, cpu_online_mask) { + next = cpumask_next_and(cpu, &watchdog_cpumask, + cpu_online_mask); + if (next >= nr_cpu_ids) + next = cpumask_first_and(&watchdog_cpumask, + cpu_online_mask); + corelockup_status_init(cpu, next); + } +} + +/* + * Before: first->next + * After: first->[new]->next + */ +void corelockup_detector_online_cpu(unsigned int cpu) +{ + unsigned int first = cpumask_first_and(&watchdog_cpumask, + cpu_online_mask); + + if (WARN_ON(first >= nr_cpu_ids)) + return; + + /* cpu->next */ + corelockup_status_copy(first, cpu); + + /* first->cpu */ + corelockup_status_init(first, cpu); +} + +/* + * Before: prev->cpu->next + * After: prev->next + */ +void corelockup_detector_offline_cpu(unsigned int cpu) +{ + unsigned int prev = nr_cpu_ids; + unsigned int i; + + /* found prev cpu */ + for_each_cpu_and(i, &watchdog_cpumask, cpu_online_mask) { + if (per_cpu(detector_cpu, i) == cpu) { + prev = i; + break; + } + } + + if (WARN_ON(prev == nr_cpu_ids)) + return; + + /* prev->next */ + corelockup_status_copy(cpu, prev); +} + +static bool is_corelockup(unsigned int cpu) +{ + unsigned long nmi_int = per_cpu(nmi_interrupts, cpu); + + /* skip check if only one cpu online */ + if (cpu == smp_processor_id()) + return false; + + if (__this_cpu_read(nmi_cnt_saved) != nmi_int) { + __this_cpu_write(nmi_cnt_saved, nmi_int); + __this_cpu_write(nmi_cnt_missed, 0); + per_cpu(core_watchdog_warn, cpu) = false; + return false; + } + + __this_cpu_inc(nmi_cnt_missed); + if (__this_cpu_read(nmi_cnt_missed) > 2) + return true; + + return false; +} +NOKPROBE_SYMBOL(is_corelockup); + +static void watchdog_corelockup_check(struct pt_regs *regs) +{ + unsigned int cpu = __this_cpu_read(detector_cpu); + + if (is_corelockup(cpu)) { + if (per_cpu(core_watchdog_warn, cpu) == true) + return; + pr_emerg("Watchdog detected core LOCKUP on cpu %d\n", cpu); + + if (hardlockup_panic) + nmi_panic(regs, "Core LOCKUP"); + + per_cpu(core_watchdog_warn, cpu) = true; + } +} +#endif + #ifdef CONFIG_HARDLOCKUP_CHECK_TIMESTAMP static DEFINE_PER_CPU(ktime_t, last_timestamp); static DEFINE_PER_CPU(unsigned int, nmi_rearmed); @@ -106,6 +263,14 @@ static inline bool watchdog_check_timestamp(void) void watchdog_hardlockup_check(struct pt_regs *regs) { +#ifdef CONFIG_CORELOCKUP_DETECTOR + /* Kick nmi interrupts */ + watchdog_nmi_interrupts(); + + /* corelockup check */ + watchdog_corelockup_check(regs); +#endif + if (__this_cpu_read(watchdog_nmi_touch) == true) { __this_cpu_write(watchdog_nmi_touch, false); return; diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 0ee305de7d0ec..4a78bacd405bd 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -881,6 +881,14 @@ config HARDLOCKUP_DETECTOR chance to run. The current stack trace is displayed upon detection and the system will stay locked up. +config CORELOCKUP_DETECTOR + bool "Detect Core Lockups" + depends on HARDLOCKUP_DETECTOR && SOFTLOCKUP_DETECTOR + depends on ARM64 + default n + help + Corelockups is used to check whether cpu core hungup or not. + config BOOTPARAM_HARDLOCKUP_PANIC bool "Panic (Reboot) On Hard Lockups" depends on HARDLOCKUP_DETECTOR -- 2.25.1

1 6

[PATCH openEuler-1.0-LTS 01/41] share_pool: Rename sp_stat_idr to sp_proc_stat_idr
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Tang Yizhou <tangyizhou(a)huawei.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4EUVI CVE: NA ------------------------------------------------- We are going to redesign the accounting subsystem of share pool. We need to disambiguate the meaning of sp_stat_idr, as we will introduce an struct which representing per-spg statistics. Signed-off-by: Tang Yizhou <tangyizhou(a)huawei.com> Reviewed-by: Ding Tianhong <dingtianhong(a)huawei.com> Signed-off-by: Zhou Guanghui <zhouguanghui1(a)huawei.com> Reviewed-by: Weilong Chen <chenweilong(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- mm/share_pool.c | 44 ++++++++++++++++++++++---------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index cd6e137fe6698..c4b8daa47fcfd 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -95,9 +95,9 @@ static DEFINE_IDA(sp_group_id_ida); /*** Statistical and maintenance tools ***/ /* idr of all sp_proc_stats */ -static DEFINE_IDR(sp_stat_idr); -/* rw semaphore for sp_stat_idr and mm->sp_stat_id */ -static DECLARE_RWSEM(sp_stat_sem); +static DEFINE_IDR(sp_proc_stat_idr); +/* rw semaphore for sp_proc_stat_idr */ +static DECLARE_RWSEM(sp_proc_stat_sem); /* for kthread buff_module_guard_work */ static struct sp_proc_stat kthread_stat; @@ -107,7 +107,7 @@ static struct sp_proc_stat *sp_get_proc_stat_locked(int tgid) { struct sp_proc_stat *stat; - stat = idr_find(&sp_stat_idr, tgid); + stat = idr_find(&sp_proc_stat_idr, tgid); /* maybe NULL or not, we always return it */ return stat; @@ -118,7 +118,7 @@ static struct sp_proc_stat *sp_get_proc_stat_ref_locked(int tgid) { struct sp_proc_stat *stat; - stat = idr_find(&sp_stat_idr, tgid); + stat = idr_find(&sp_proc_stat_idr, tgid); if (!stat || !atomic_inc_not_zero(&stat->use_count)) stat = NULL; @@ -137,16 +137,16 @@ static struct sp_proc_stat *sp_init_proc_stat(struct task_struct *tsk, int id, tgid = tsk->tgid; int ret; - down_write(&sp_stat_sem); + down_write(&sp_proc_stat_sem); id = mm->sp_group_master->sp_stat_id; if (id) { /* other threads in the same process may have initialized it */ stat = sp_get_proc_stat_locked(tgid); if (stat) { - up_write(&sp_stat_sem); + up_write(&sp_proc_stat_sem); return stat; } else { - up_write(&sp_stat_sem); + up_write(&sp_proc_stat_sem); /* if enter this branch, that's our mistake */ pr_err_ratelimited("share pool: proc stat invalid id %d\n", id); return ERR_PTR(-EBUSY); @@ -155,7 +155,7 @@ static struct sp_proc_stat *sp_init_proc_stat(struct task_struct *tsk, stat = kzalloc(sizeof(*stat), GFP_KERNEL); if (stat == NULL) { - up_write(&sp_stat_sem); + up_write(&sp_proc_stat_sem); pr_err_ratelimited("share pool: alloc proc stat failed due to lack of memory\n"); return ERR_PTR(-ENOMEM); } @@ -167,16 +167,16 @@ static struct sp_proc_stat *sp_init_proc_stat(struct task_struct *tsk, stat->mm = mm; get_task_comm(stat->comm, tsk); - ret = idr_alloc(&sp_stat_idr, stat, tgid, tgid + 1, GFP_KERNEL); + ret = idr_alloc(&sp_proc_stat_idr, stat, tgid, tgid + 1, GFP_KERNEL); if (ret < 0) { - up_write(&sp_stat_sem); + up_write(&sp_proc_stat_sem); pr_err_ratelimited("share pool: proc stat idr alloc failed %d\n", ret); kfree(stat); return ERR_PTR(ret); } mm->sp_group_master->sp_stat_id = ret; - up_write(&sp_stat_sem); + up_write(&sp_proc_stat_sem); return stat; } @@ -184,9 +184,9 @@ static struct sp_proc_stat *sp_get_proc_stat(int tgid) { struct sp_proc_stat *stat; - down_read(&sp_stat_sem); + down_read(&sp_proc_stat_sem); stat = sp_get_proc_stat_locked(tgid); - up_read(&sp_stat_sem); + up_read(&sp_proc_stat_sem); return stat; } @@ -195,9 +195,9 @@ struct sp_proc_stat *sp_get_proc_stat_ref(int tgid) { struct sp_proc_stat *stat; - down_read(&sp_stat_sem); + down_read(&sp_proc_stat_sem); stat = sp_get_proc_stat_ref_locked(tgid); - up_read(&sp_stat_sem); + up_read(&sp_proc_stat_sem); return stat; } @@ -2850,10 +2850,10 @@ __setup("enable_sp_multi_group_mode", enable_sp_multi_group_mode); static void free_sp_proc_stat(struct sp_proc_stat *stat) { - down_write(&sp_stat_sem); + down_write(&sp_proc_stat_sem); stat->mm->sp_group_master->sp_stat_id = 0; - idr_remove(&sp_stat_idr, stat->tgid); - up_write(&sp_stat_sem); + idr_remove(&sp_proc_stat_idr, stat->tgid); + up_write(&sp_proc_stat_sem); kfree(stat); } @@ -3163,9 +3163,9 @@ static int proc_stat_show(struct seq_file *seq, void *offset) byte2kb(atomic64_read(&kthread_stat.k2u_size))); /* pay attention to potential ABBA deadlock */ - down_read(&sp_stat_sem); - idr_for_each(&sp_stat_idr, idr_proc_stat_cb, seq); - up_read(&sp_stat_sem); + down_read(&sp_proc_stat_sem); + idr_for_each(&sp_proc_stat_idr, idr_proc_stat_cb, seq); + up_read(&sp_proc_stat_sem); return 0; } -- 2.25.1

1 40

[PATCH openEuler-1.0-LTS 1/7] sharepool: Fix ASLR broken
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Wang Wensheng <wangwensheng4(a)huawei.com> ascend inclusion category: bugfix bugzilla: NA CVE: NA --------------------------- To avoid mmap vspace reserved for sharepool, we currently change the high_limit to MMAP_SHARE_POOL_START in arch_get_unmapped_area() and arch_get_unmapped_area_topdown(). In mmap-topdown scene, this make the start address of mmap being always MMAP_SHARE_POOL_START. ASLR got broken. To fix this, this patch set the mm->mmap_base based on MMAP_SHARE_POOL_START instead of STACK_TOP in topdown scene. Fixes: 4bdd5c21793e ("ascend: memory: introduce do_mm_populate and hugetlb_insert_hugepage") Signed-off-by: Wang Wensheng <wangwensheng4(a)huawei.com> Reviewed-by: Weilong Chen <chenweilong(a)huawei.com> Reviewed-by: Ding Tianhong <dingtianhong(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- arch/arm64/mm/mmap.c | 6 +++++- include/linux/share_pool.h | 4 ++-- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/arm64/mm/mmap.c b/arch/arm64/mm/mmap.c index ac89686c4af89..87f29df8126ba 100644 --- a/arch/arm64/mm/mmap.c +++ b/arch/arm64/mm/mmap.c @@ -28,6 +28,7 @@ #include <linux/io.h> #include <linux/personality.h> #include <linux/random.h> +#include <linux/share_pool.h> #include <asm/cputype.h> @@ -80,7 +81,10 @@ static unsigned long mmap_base(unsigned long rnd, struct rlimit *rlim_stack) else if (gap > MAX_GAP) gap = MAX_GAP; - return PAGE_ALIGN(STACK_TOP - gap - rnd); + if (sp_is_enabled()) + return ALIGN_DOWN(MMAP_SHARE_POOL_START - rnd, PAGE_SIZE); + else + return PAGE_ALIGN(STACK_TOP - gap - rnd); } /* diff --git a/include/linux/share_pool.h b/include/linux/share_pool.h index 9650f257b3ad7..9557a8be46677 100644 --- a/include/linux/share_pool.h +++ b/include/linux/share_pool.h @@ -130,8 +130,6 @@ struct sp_proc_stat { atomic64_t k2u_size; }; -#ifdef CONFIG_ASCEND_SHARE_POOL - #define MAP_SHARE_POOL 0x100000 #define MMAP_TOP_4G_SIZE 0x100000000UL @@ -148,6 +146,8 @@ struct sp_proc_stat { #define MMAP_SHARE_POOL_START (MMAP_SHARE_POOL_END - MMAP_SHARE_POOL_SIZE) #define MMAP_SHARE_POOL_16G_START (MMAP_SHARE_POOL_END - MMAP_SHARE_POOL_DVPP_SIZE) +#ifdef CONFIG_ASCEND_SHARE_POOL + static inline void sp_init_mm(struct mm_struct *mm) { mm->sp_group = NULL; -- 2.25.1

1 6

[PATCH openEuler-5.10 01/20] perf stat: Add --quiet option
by Zheng Zengkai 30 Oct '21

30 Oct '21

From: Andi Kleen <andi(a)firstfloor.org> mainline inclusion from mainline-5.11 commit 55a4de94c64bacffbcd802c954764e0de2ab217f category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4CMQA CVE: NA -------------------------------- Add a new --quiet option to 'perf stat'. This is useful with 'perf stat record' to write the data only to the perf.data file, which can lower measurement overhead because the data doesn't need to be formatted. On my 4C desktop: % time ./perf stat record -e $(python -c 'print ",\ ".join(["cycles"]*1000)') -a -I 1000 sleep 5 ... real 0m5.377s user 0m0.238s sys 0m0.452s % time ./perf stat record --quiet -e $(python -c 'print ",\ ".join(["cycles"]*1000)') -a -I 1000 sleep 5 real 0m5.452s user 0m0.183s sys 0m0.423s In this example it cuts the user time by 20%. On systems with more cores the savings are higher. Signed-off-by: Andi Kleen <andi(a)firstfloor.org> Acked-by: Jiri Olsa <jolsa(a)kernel.org> Cc: Alexey Budankov <alexey.budankov(a)linux.intel.com> Link: http://lore.kernel.org/lkml/20201027002737.30942-1-andi@firstfloor.org Signed-off-by: Arnaldo Carvalho de Melo <acme(a)redhat.com> Signed-off-by: yin-xiujiang <yinxiujiang(a)kylinos.cn> Reviewed-by: Wang ShaoBo <bobo.shaobowang(a)huawei.com> Reviewed-by: Yang Jihong <yangjihong1(a)huawei.com> Signed-off-by: Zheng Zengkai <zhengzengkai(a)huawei.com> --- tools/perf/Documentation/perf-stat.txt | 4 ++++ tools/perf/builtin-stat.c | 6 +++++- tools/perf/util/stat.h | 1 + 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 9f9f29025e49..f9bcd95bf352 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -320,6 +320,10 @@ STAT RECORD ----------- Stores stat data into perf data file. +--quiet:: +Don't print output. This is useful with perf stat record below to only +write data to the perf.data file. + -o file:: --output file:: Output file name. diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c index b01af171d94f..89e80a3bc9c3 100644 --- a/tools/perf/builtin-stat.c +++ b/tools/perf/builtin-stat.c @@ -973,6 +973,8 @@ static void print_counters(struct timespec *ts, int argc, const char **argv) if (STAT_RECORD && perf_stat.data.is_pipe) return; + if (stat_config.quiet) + return; perf_evlist__print_counters(evsel_list, &stat_config, &target, ts, argc, argv); } @@ -1171,6 +1173,8 @@ static struct option stat_options[] = { "threads of same physical core"), OPT_BOOLEAN(0, "summary", &stat_config.summary, "print summary for interval mode"), + OPT_BOOLEAN(0, "quiet", &stat_config.quiet, + "don't print output (useful with record)"), #ifdef HAVE_LIBPFM OPT_CALLBACK(0, "pfm-events", &evsel_list, "event", "libpfm4 event selector. use 'perf list' to list available events", @@ -2132,7 +2136,7 @@ int cmd_stat(int argc, const char **argv) goto out; } - if (!output) { + if (!output && !stat_config.quiet) { struct timespec tm; mode = append_file ? "a" : "w"; diff --git a/tools/perf/util/stat.h b/tools/perf/util/stat.h index 487010c624be..05adf8165025 100644 --- a/tools/perf/util/stat.h +++ b/tools/perf/util/stat.h @@ -122,6 +122,7 @@ struct perf_stat_config { bool metric_no_group; bool metric_no_merge; bool stop_read_counter; + bool quiet; FILE *output; unsigned int interval; unsigned int timeout; -- 2.20.1

1 19

[PATCH openEuler-1.0-LTS 001/103] mm/vmalloc: Hugepage vmalloc mappings
by Yang Yingliang 30 Oct '21

30 Oct '21

From: Nicholas Piggin <npiggin(a)gmail.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4EUVI CVE: NA https://lwn.net/ml/linux-kernel/20200825145753.529284-12-npiggin@gmail.com/ Don't distinction between vmalloc and hugepage vmalloc, because there is no size print in alloc_large_system_hash in v4.19. And this patch add page_order in vm_struct, it will break kabi. -------------- Support huge page vmalloc mappings. Config option HAVE_ARCH_HUGE_VMALLOC enables support on architectures that define HAVE_ARCH_HUGE_VMAP and supports PMD sized vmap mappings. vmalloc will attempt to allocate PMD-sized pages if allocating PMD size or larger, and fall back to small pages if that was unsuccessful. Allocations that do not use PAGE_KERNEL prot are not permitted to use huge pages, because not all callers expect this (e.g., module allocations vs strict module rwx). This reduces TLB misses by nearly 30x on a `git diff` workload on a 2-node POWER9 (59,800 -> 2,100) and reduces CPU cycles by 0.54%. This can result in more internal fragmentation and memory overhead for a given allocation, an option nohugevmalloc is added to disable at boot. Signed-off-by: Nicholas Piggin <npiggin(a)gmail.com> Signed-off-by: Rui Xiang <rui.xiang(a)huawei.com> Reviewed-by: Ding Tianhong <dingtianhong(a)huawei.com> Reviewed-by: Zefan Li <lizefan(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> Reviewed-by: Weilong Chen <chenweilong(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- arch/Kconfig | 4 + include/linux/vmalloc.h | 1 + mm/vmalloc.c | 160 +++++++++++++++++++++++++++++++--------- 3 files changed, 130 insertions(+), 35 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index e906cbb213444..00f55932ba781 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -559,6 +559,10 @@ config HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD config HAVE_ARCH_HUGE_VMAP bool +config HAVE_ARCH_HUGE_VMALLOC + depends on HAVE_ARCH_HUGE_VMAP + bool + config HAVE_ARCH_SOFT_DIRTY bool diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 496ac80046c01..07b4b1141ed8a 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -39,6 +39,7 @@ struct vm_struct { unsigned long size; unsigned long flags; struct page **pages; + unsigned int page_order; unsigned int nr_pages; phys_addr_t phys_addr; const void *caller; diff --git a/mm/vmalloc.c b/mm/vmalloc.c index fc6394184a1ba..e76b806a6c003 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -41,6 +41,19 @@ #include "internal.h" +#ifdef CONFIG_HAVE_ARCH_HUGE_VMALLOC +static bool __ro_after_init vmap_allow_huge = true; + +static int __init set_nohugevmalloc(char *str) +{ + vmap_allow_huge = false; + return 0; +} +early_param("nohugevmalloc", set_nohugevmalloc); +#else /* CONFIG_HAVE_ARCH_HUGE_VMALLOC */ +static const bool vmap_allow_huge = false; +#endif /* CONFIG_HAVE_ARCH_HUGE_VMALLOC */ + struct vfree_deferred { struct llist_head list; struct work_struct wq; @@ -410,6 +423,61 @@ static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr, return 0; } +static int vmap_small_pages_range_noflush(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages) +{ + pgd_t *pgd; + unsigned long next; + int err = 0; + int nr = 0; + + BUG_ON(addr >= end); + pgd = pgd_offset_k(addr); + do { + next = pgd_addr_end(addr, end); + err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr); + if (err) + return err; + } while (pgd++, addr = next, addr != end); + + return 0; +} + +static int vmap_pages_range_noflush(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int page_shift) +{ + unsigned int i, nr = (end - addr) >> PAGE_SHIFT; + + WARN_ON(page_shift < PAGE_SHIFT); + + if (page_shift == PAGE_SHIFT) + return vmap_small_pages_range_noflush(addr, end, prot, pages); + + for (i = 0; i < nr; i += 1U << (page_shift - PAGE_SHIFT)) { + int err; + + err = vmap_range_noflush(addr, addr + (1UL << page_shift), + __pa(page_address(pages[i])), prot, + page_shift); + if (err) + return err; + + addr += 1UL << page_shift; + } + + return 0; +} + +static int vmap_pages_range(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages, unsigned int page_shift) +{ + int err; + + err = vmap_pages_range_noflush(addr, end, prot, pages, page_shift); + flush_cache_vmap(addr, end); + return err; +} + /** * map_kernel_range_noflush - map kernel VM area with the specified pages * @addr: start of the VM area to map @@ -431,22 +499,7 @@ static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr, int map_kernel_range_noflush(unsigned long addr, unsigned long size, pgprot_t prot, struct page **pages) { - unsigned long end = addr + size; - unsigned long next; - pgd_t *pgd; - int err = 0; - int nr = 0; - - BUG_ON(addr >= end); - pgd = pgd_offset_k(addr); - do { - next = pgd_addr_end(addr, end); - err = vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr); - if (err) - return err; - } while (pgd++, addr = next, addr != end); - - return 0; + return vmap_pages_range_noflush(addr, addr + size, prot, pages, PAGE_SHIFT); } int map_kernel_range(unsigned long start, unsigned long size, pgprot_t prot, @@ -2270,11 +2323,11 @@ static void __vunmap(const void *addr, int deallocate_pages) if (deallocate_pages) { int i; - for (i = 0; i < area->nr_pages; i++) { + for (i = 0; i < area->nr_pages; i += 1U << area->page_order) { struct page *page = area->pages[i]; BUG_ON(!page); - __free_pages(page, 0); + __free_pages(page, area->page_order); } kvfree(area->pages); @@ -2403,9 +2456,12 @@ static void *__vmalloc_node(unsigned long size, unsigned long align, gfp_t gfp_mask, pgprot_t prot, int node, const void *caller); static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, - pgprot_t prot, int node) + pgprot_t prot, unsigned int page_shift, int node) { struct page **pages; + unsigned long addr = (unsigned long)area->addr; + unsigned long size = get_vm_area_size(area); + unsigned int page_order = page_shift - PAGE_SHIFT; unsigned int nr_pages; unsigned long array_size; unsigned int i; @@ -2415,7 +2471,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, 0 : __GFP_HIGHMEM; - nr_pages = get_vm_area_size(area) >> PAGE_SHIFT; + nr_pages = size >> PAGE_SHIFT; array_size = (unsigned long)nr_pages * sizeof(struct page *); /* Please note that the recursion is strictly bounded. */ @@ -2434,27 +2490,27 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, area->pages = pages; area->nr_pages = nr_pages; + area->page_order = page_order; - for (i = 0; i < area->nr_pages; i++) { + for (i = 0; i < area->nr_pages; i += 1U << page_order) { struct page *page; + int p; - if (node == NUMA_NO_NODE) - page = alloc_page(alloc_mask|highmem_mask); - else - page = alloc_pages_node(node, alloc_mask|highmem_mask, 0); - + page = alloc_pages_node(node, alloc_mask|highmem_mask, page_order); if (unlikely(!page)) { /* Successfully allocated i pages, free them in __vunmap() */ area->nr_pages = i; goto fail; } - area->pages[i] = page; + + for (p = 0; p < (1U << page_order); p++) + area->pages[i + p] = page + p; + if (gfpflags_allow_blocking(gfp_mask|highmem_mask)) cond_resched(); } - if (map_kernel_range((unsigned long)area->addr, get_vm_area_size(area), - prot, pages) < 0) + if (vmap_pages_range(addr, addr + size, prot, pages, page_shift) < 0) goto fail; return area->addr; @@ -2462,7 +2518,7 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask, fail: warn_alloc(gfp_mask, NULL, "vmalloc: allocation failure, allocated %ld of %ld bytes", - (area->nr_pages*PAGE_SIZE), area->size); + (area->nr_pages*PAGE_SIZE), size); vfree(area->addr); return NULL; } @@ -2491,19 +2547,42 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, struct vm_struct *area; void *addr; unsigned long real_size = size; + unsigned long real_align = align; + unsigned int shift = PAGE_SHIFT; - size = PAGE_ALIGN(size); if (!size || (size >> PAGE_SHIFT) > totalram_pages) goto fail; + if (vmap_allow_huge && (pgprot_val(prot) == pgprot_val(PAGE_KERNEL))) { + unsigned long size_per_node; + + /* + * Try huge pages. Only try for PAGE_KERNEL allocations, + * others like modules don't yet expect huge pages in + * their allocations due to apply_to_page_range not + * supporting them. + */ + + size_per_node = size; + if (node == NUMA_NO_NODE) + size_per_node /= num_online_nodes(); + if (size_per_node >= PMD_SIZE) { + shift = PMD_SHIFT; + align = max(real_align, 1UL << shift); + size = ALIGN(real_size, 1UL << shift); + } + } + +again: + size = PAGE_ALIGN(size); area = __get_vm_area_node(size, align, VM_ALLOC | VM_UNINITIALIZED | vm_flags, start, end, node, gfp_mask, caller); if (!area) goto fail; - addr = __vmalloc_area_node(area, gfp_mask, prot, node); + addr = __vmalloc_area_node(area, gfp_mask, prot, shift, node); if (!addr) - return NULL; + goto fail; /* * First make sure the mappings are removed from all page-tables @@ -2523,8 +2602,19 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, return addr; fail: - warn_alloc(gfp_mask, NULL, + if (shift > PAGE_SHIFT) { + free_vm_area(area); + shift = PAGE_SHIFT; + align = real_align; + size = real_size; + goto again; + } + + if (!area) { + /* Warn for area allocation, page allocations already warn */ + warn_alloc(gfp_mask, NULL, "vmalloc: allocation failure: %lu bytes", real_size); + } return NULL; } @@ -3503,7 +3593,7 @@ static int s_show(struct seq_file *m, void *p) seq_printf(m, " %pS", v->caller); if (v->nr_pages) - seq_printf(m, " pages=%d", v->nr_pages); + seq_printf(m, " pages=%d order=%d", v->nr_pages, v->page_order); if (v->phys_addr) seq_printf(m, " phys=%pa", &v->phys_addr); -- 2.25.1

1 102

[PATCH openEuler-1.0-LTS 01/26] mm: add do_vm_mmap
by Yang Yingliang 29 Oct '21

29 Oct '21

From: Fang Lijun <fanglijun3(a)huawei.com> ascend inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I4D63I CVE: NA ------------------------------------------------- An interface do_vm_mmap is added to support the allocation in the address spaces of other processes. Signed-off-by: Fang Lijun <fanglijun3(a)huawei.com> Signed-off-by: Zhou Guanghui <zhouguanghui1(a)huawei.com> Signed-off-by: Yang Yingliang <yangyingliang(a)huawei.com> --- include/linux/mm.h | 3 +++ mm/mmap.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 75d94ea5d1c20..58fe28dd959b9 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2436,6 +2436,9 @@ static inline void mm_populate(unsigned long addr, unsigned long len) {} extern int __must_check vm_brk(unsigned long, unsigned long); extern int __must_check vm_brk_flags(unsigned long, unsigned long, unsigned long); extern int vm_munmap(unsigned long, size_t); +extern unsigned long do_vm_mmap(struct mm_struct *mm, unsigned long addr, + unsigned long len, unsigned long prot, + unsigned long flag, unsigned long pgoff); extern unsigned long __must_check vm_mmap(struct file *, unsigned long, unsigned long, unsigned long, unsigned long, unsigned long); diff --git a/mm/mmap.c b/mm/mmap.c index 80779bbb1c048..f7f1fd3b5fa39 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -3094,6 +3094,40 @@ int vm_munmap(unsigned long start, size_t len) } EXPORT_SYMBOL(vm_munmap); +/* + * Must acquire an additional reference to the mm struct to prevent the + * mm struct of other process from being released. + * + * This interface is applicable only to kernel thread scenarios. + */ +unsigned long do_vm_mmap(struct mm_struct *mm, unsigned long addr, + unsigned long len, unsigned long prot, + unsigned long flag, unsigned long pgoff) +{ + unsigned long ret; + unsigned long populate; + LIST_HEAD(uf); + + if (mm == NULL || current->mm) + return -EINVAL; + + if (down_write_killable(&mm->mmap_sem)) + return -EINTR; + + current->mm = mm; + ret = do_mmap_pgoff(0, addr, len, prot, flag, pgoff, + &populate, &uf); + + current->mm = NULL; + up_write(&mm->mmap_sem); + userfaultfd_unmap_complete(mm, &uf); + if (populate) + mm_populate(ret, populate); + + return ret; +} +EXPORT_SYMBOL(do_vm_mmap); + SYSCALL_DEFINE2(munmap, unsigned long, addr, size_t, len) { profile_munmap(addr); -- 2.25.1

1 25

2024

2023

2022

2021

2020

2019

Kernel October 2021