[PATCH v2 OLK-5.10 00/37] backport patches of share_pool and
v2: - remove patch "mm/sharepool: Add mg_sp_alloc_nodemask" Chen Jun (3): mm/sharepool: Fix a double free problem caused by init_local_group mm/sharepool: extract group_add_task mm/sharepool: use delete_spg_node to replace some repetitive code Guo Mengqi (1): mm: sharepool: add static modifier to find_spg_node_by_spg() Wang Wensheng (8): mm/sharepool: Fix NULL pointer dereference in mg_sp_group_del_task mm/sharepool: Reorganize create_spg() mm/sharepool: Simplify sp_make_share_k2u() mm/sharepool: Rename sp_group operations mm/sharepool: Simplify sp_unshare_uva() mm/sharepool: Fix null-pointer-deference in sp_free_area mm/sharepool: Modify error message in mg_sp_group_del_task mm/sharepool: Fix double delete list in sp_group_exit Xu Qiang (17): mm/sharepool: Refactoring proc file interface similar code mm/sharepool: Add helper for master_list mm/sharepool: Delete unused spg_id and hugepage_failures. mm/sharepool: Delete unused mm in sp_proc_stat. mm/sharepool: Move spa_num field to sp_group. mm/sharepool: Rename sp_spg_stat to sp_meminfo. mm/sharepool: Split meminfo_update into meminfo_inc_usage and meminfo_dec_usage. mm/sharepool: split meminfo_update_k2u into meminfo_inc_k2u and meminfo_dec_k2u. mm/sharepool: Delete redundant tgid in sp_proc_stat. mm/sharepool: Move comm from sp_proc_stat to sp_group_master. mm/sharepool: replace sp_proc_stat with sp_meminfo. mm/sharepool: Delete unused tgid and spg_id in spg_proc_stat. mm/sharepool: Replace spg_proc_stat with sp_meminfo. mm/sharepool: Add meminfo_alloc_sum_byKB and meminfo_alloc_sum. mm/sharepool: Add meminfo_k2u_size. mm/sharepool: Delete unused kthread_stat. mm/sharepool: Delete redundant size and alloc_size in sp_meminfo. Zhang Zekun (6): perf: hisi: Add configs for PMU isolation driver: Add CONFIG_ACPI_APEI_GHES_TS_CORE for code isolation ACPI / APEI: Add config to isolate Notify all ras err vmalloc: Add config for Extend for hugepages mapping iommu/arm-smmu-v3: Add config to Add support for suspend and resume hugetlbfs: Add config to isolate the code of share_pool Zhou Guanghui (2): mm/sharepool: Don't display sharepool statistics in the container mm: sharepool: Charge Buddy hugepage to memcg arch/arm64/Kconfig | 2 +- arch/arm64/configs/openeuler_defconfig | 6 + drivers/acpi/apei/Kconfig | 14 + drivers/acpi/apei/ghes.c | 10 +- drivers/iommu/Kconfig | 7 + drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 +- drivers/perf/hisilicon/Kconfig | 19 + drivers/perf/hisilicon/Makefile | 6 +- fs/hugetlbfs/inode.c | 21 +- include/acpi/ghes.h | 2 + include/linux/cper.h | 3 + include/linux/mm.h | 2 + include/linux/vmalloc.h | 7 + mm/Kconfig | 6 + mm/hugetlb.c | 2 + mm/share_pool.c | 1027 ++++++++----------- mm/share_pool_internal.h | 22 + mm/vmalloc.c | 16 + 18 files changed, 569 insertions(+), 612 deletions(-) create mode 100644 mm/share_pool_internal.h -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ------------------------------------------ Add CONFIG_HISI_L3T_PMU and CONFIG_HISI_LPDDRC_PMU to isolate features of hisi pmu driver. This patch isolates commit 0edc58409e30 and 6bf896bea639. Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 2 ++ drivers/perf/hisilicon/Kconfig | 19 +++++++++++++++++++ drivers/perf/hisilicon/Makefile | 6 +++--- 3 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 67ee1ac76c7d..a2e93ba92d14 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -6110,6 +6110,8 @@ CONFIG_THUNDERX2_PMU=m CONFIG_XGENE_PMU=y CONFIG_ARM_SPE_PMU=y CONFIG_HISI_PMU=m +CONFIG_HISI_L3T_PMU=m +CONFIG_HISI_LPDDRC_PMU=m CONFIG_HISI_PCIE_PMU=m CONFIG_HNS3_PMU=m # end of Performance monitor support diff --git a/drivers/perf/hisilicon/Kconfig b/drivers/perf/hisilicon/Kconfig index 171bfc1b6bc2..e19feba8b5d3 100644 --- a/drivers/perf/hisilicon/Kconfig +++ b/drivers/perf/hisilicon/Kconfig @@ -24,3 +24,22 @@ config HNS3_PMU devices. Adds the HNS3 PMU into perf events system for monitoring latency, bandwidth etc. + +config HISI_L3T_PMU + tristate "HiSilicon SoC L3T PMU drivers" + depends on HISI_PMU + default n + help + Support for HiSilicon SoC L3 Cache performance monitor, Hydra Home + Agent performance monitor and DDR Controller performance monitor. + L3T is a specialized PMU driver. + +config HISI_LPDDRC_PMU + tristate "HiSilicon SoC LDPPRC PMU drivers" + depends on HISI_PMU + default n + help + Support for HiSilicon SoC L3 Cache performance monitor, Hydra Home + Agent performance monitor and DDR Controller performance monitor. + LPDDRC is a specialize PMU driver. + diff --git a/drivers/perf/hisilicon/Makefile b/drivers/perf/hisilicon/Makefile index a35705795dfc..fc7ed8c6cc0f 100644 --- a/drivers/perf/hisilicon/Makefile +++ b/drivers/perf/hisilicon/Makefile @@ -1,9 +1,9 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_HISI_PMU) += hisi_uncore_pmu.o hisi_uncore_l3c_pmu.o \ hisi_uncore_hha_pmu.o hisi_uncore_ddrc_pmu.o hisi_uncore_sllc_pmu.o \ - hisi_uncore_pa_pmu.o hisi_uncore_cpa_pmu.o \ - hisi_uncore_l3t_pmu.o \ - hisi_uncore_lpddrc_pmu.o + hisi_uncore_pa_pmu.o hisi_uncore_cpa_pmu.o obj-$(CONFIG_HISI_PCIE_PMU) += hisi_pcie_pmu.o obj-$(CONFIG_HNS3_PMU) += hns3_pmu.o +obj-$(CONFIG_HISI_L3T_PMU) += hisi_uncore_l3t_pmu.o +obj-$(CONFIG_HISI_LPDDRC_PMU) += hisi_uncore_lpddrc_pmu.o -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ------------------------------------------------- Add CONFIG_ACPI_APEI_GHES_TS_CORE to isolate code introduced in 01dbadfe7fcbfcca5ce5a974daaef4e9ff5eedfe. Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + drivers/acpi/apei/Kconfig | 7 +++++++ drivers/acpi/apei/ghes.c | 4 ++++ include/acpi/ghes.h | 2 ++ include/linux/cper.h | 3 +++ 5 files changed, 17 insertions(+) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index a2e93ba92d14..218c77399da2 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -721,6 +721,7 @@ CONFIG_ACPI_HMAT=y CONFIG_HAVE_ACPI_APEI=y CONFIG_ACPI_APEI=y CONFIG_ACPI_APEI_GHES=y +CONFIG_ACPI_APEI_GHES_TS_CORE=y CONFIG_ACPI_APEI_PCIEAER=y CONFIG_ACPI_APEI_SEA=y CONFIG_ACPI_APEI_MEMORY_FAILURE=y diff --git a/drivers/acpi/apei/Kconfig b/drivers/acpi/apei/Kconfig index 6b18f8bc7be3..f33c23062471 100644 --- a/drivers/acpi/apei/Kconfig +++ b/drivers/acpi/apei/Kconfig @@ -33,6 +33,13 @@ config ACPI_APEI_GHES by firmware to produce more valuable hardware error information for Linux. +config ACPI_APEI_GHES_TS_CORE + bool "Support ts core ras process for ascend" + depends on ARM64 && ACPI_APEI_GHES + default n + help + Enable the support for ts core ras process for ascend. + config ACPI_APEI_PCIEAER bool "APEI PCIe AER logging/recovering support" depends on ACPI_APEI && PCIEAER diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 9c38c2cdd2fd..316648e09571 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -118,8 +118,10 @@ module_param_named(disable, ghes_disable, bool, 0); static LIST_HEAD(ghes_hed); static DEFINE_MUTEX(ghes_list_mutex); +#ifdef CONFIG_ACPI_APEI_GHES_TS_CORE BLOCKING_NOTIFIER_HEAD(ghes_ts_err_chain); EXPORT_SYMBOL(ghes_ts_err_chain); +#endif /* * Because the memory area used to transfer hardware error information @@ -657,10 +659,12 @@ static bool ghes_do_proc(struct ghes *ghes, } else if (guid_equal(sec_type, &CPER_SEC_PROC_ARM)) { queued = ghes_handle_arm_hw_error(gdata, sev); +#ifdef CONFIG_ACPI_APEI_GHES_TS_CORE } else if (guid_equal(sec_type, &CPER_SEC_TS_CORE)) { blocking_notifier_call_chain(&ghes_ts_err_chain, 0, acpi_hest_get_payload(gdata)); +#endif } else { void *err = acpi_hest_get_payload(gdata); diff --git a/include/acpi/ghes.h b/include/acpi/ghes.h index 89330e4872c0..ca054bf08ee9 100644 --- a/include/acpi/ghes.h +++ b/include/acpi/ghes.h @@ -145,6 +145,8 @@ int ghes_notify_sea(void); static inline int ghes_notify_sea(void) { return -ENOENT; } #endif +#ifdef CONFIG_ACPI_APEI_GHES_TS_CORE extern struct blocking_notifier_head ghes_ts_err_chain; +#endif #endif /* GHES_H */ diff --git a/include/linux/cper.h b/include/linux/cper.h index 78cf8a0b05a5..4dbf03678cc6 100644 --- a/include/linux/cper.h +++ b/include/linux/cper.h @@ -197,10 +197,13 @@ enum { #define CPER_SEC_DMAR_IOMMU \ GUID_INIT(0x036F84E1, 0x7F37, 0x428c, 0xA7, 0x9E, 0x57, 0x5F, \ 0xDF, 0xAA, 0x84, 0xEC) + +#ifdef CONFIG_ACPI_APEI_GHES_TS_CORE /* HISI ts core */ #define CPER_SEC_TS_CORE \ GUID_INIT(0xeb4c71f8, 0xbc76, 0x4c46, 0xbd, 0x9, 0xd0, 0xd3, \ 0x45, 0x0, 0x5a, 0x92) +#endif #define CPER_PROC_VALID_TYPE 0x0001 #define CPER_PROC_VALID_ISA 0x0002 -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK -------------------------------------------------- Add CONFIG_ACPI_APEI_NOTIFY_ALL_RAS_ERR to isolate Notify all ras err to driver, which was introduced in 924ceaed70a63e5eaaf39b78dc756e4a7ac3c250. Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + drivers/acpi/apei/Kconfig | 7 +++++++ drivers/acpi/apei/ghes.c | 6 +++++- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 218c77399da2..afe1060947e2 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -722,6 +722,7 @@ CONFIG_HAVE_ACPI_APEI=y CONFIG_ACPI_APEI=y CONFIG_ACPI_APEI_GHES=y CONFIG_ACPI_APEI_GHES_TS_CORE=y +CONFIG_ACPI_APEI_GHES_NOTIFY_ALL_RAS_ERR=y CONFIG_ACPI_APEI_PCIEAER=y CONFIG_ACPI_APEI_SEA=y CONFIG_ACPI_APEI_MEMORY_FAILURE=y diff --git a/drivers/acpi/apei/Kconfig b/drivers/acpi/apei/Kconfig index f33c23062471..bf65cce12f61 100644 --- a/drivers/acpi/apei/Kconfig +++ b/drivers/acpi/apei/Kconfig @@ -40,6 +40,13 @@ config ACPI_APEI_GHES_TS_CORE help Enable the support for ts core ras process for ascend. +config ACPI_APEI_GHES_NOTIFY_ALL_RAS_ERR + bool "Notify all ras err to driver" + depends on ARM64 && ACPI_APEI_GHES + default n + help + Deliver all types of error to driver. + config ACPI_APEI_PCIEAER bool "APEI PCIe AER logging/recovering support" depends on ACPI_APEI && PCIEAER diff --git a/drivers/acpi/apei/ghes.c b/drivers/acpi/apei/ghes.c index 316648e09571..ec2c887bccb7 100644 --- a/drivers/acpi/apei/ghes.c +++ b/drivers/acpi/apei/ghes.c @@ -667,14 +667,18 @@ static bool ghes_do_proc(struct ghes *ghes, #endif } else { void *err = acpi_hest_get_payload(gdata); - +#ifndef CONFIG_ACPI_APEI_GHES_NOTIFY_ALL_RAS_ERR + ghes_defer_non_standard_event(gdata, sev); +#endif log_non_standard_event(sec_type, fru_id, fru_text, sec_sev, err, gdata->error_data_length); } +#ifdef CONFIG_ACPI_APEI_GHES_NOTIFY_ALL_RAS_ERR /* Customization deliver all types error to driver. */ ghes_defer_non_standard_event(gdata, sev); +#endif } return queued; -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ---------------------------------------- Use CONFIG_EXTEND_HUGEPAGE_MAPPING to isolate code introduced in a3425d4173784e41644abc3cc70d41aa2aef16fd. Besides, use tab instead of space to match the format of Kconfig Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- arch/arm64/Kconfig | 2 +- arch/arm64/configs/openeuler_defconfig | 1 + include/linux/mm.h | 2 ++ include/linux/vmalloc.h | 7 +++++++ mm/Kconfig | 6 ++++++ mm/vmalloc.c | 16 ++++++++++++++++ 6 files changed, 33 insertions(+), 1 deletion(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index a0bba8e5426a..e543b74f1fab 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -2078,7 +2078,7 @@ config ASCEND_CHARGE_MIGRATE_HUGEPAGES config ASCEND_SHARE_POOL bool "Enable support for the Share Pool Memory" default n - depends on HAVE_ARCH_HUGE_VMALLOC + depends on HAVE_ARCH_HUGE_VMALLOC && EXTEND_HUGEPAGE_MAPPING select ARCH_USES_HIGH_VMA_FLAGS help This feature allows multiple processes to share virtual memory both diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index afe1060947e2..3f1532646775 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -1131,6 +1131,7 @@ CONFIG_PIN_MEMORY=y CONFIG_PID_RESERVE=y CONFIG_MEMORY_RELIABLE=y # CONFIG_CLEAR_FREELIST_PAGE is not set +CONFIG_EXTEND_HUGEPAGE_MAPPING=y # # Data Access Monitoring diff --git a/include/linux/mm.h b/include/linux/mm.h index 430eb04ca390..0b5ce84212d7 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -235,11 +235,13 @@ int overcommit_policy_handler(struct ctl_table *, int, void *, size_t *, /* test whether an address (unsigned long or pointer) is aligned to PAGE_SIZE */ #define PAGE_ALIGNED(addr) IS_ALIGNED((unsigned long)(addr), PAGE_SIZE) +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING /* to align the pointer to the (next) PMD hugepage boundary */ #define PMD_ALIGN(addr) ALIGN(addr, PMD_SIZE) /* test whether an address (unsigned long or pointer) is aligned to PMD_SIZE */ #define PMD_ALIGNED(addr) IS_ALIGNED((unsigned long)(addr), PMD_SIZE) +#endif #define lru_to_page(head) (list_entry((head)->prev, struct page, lru)) diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h index 49c94afce25b..1ebe364ed29a 100644 --- a/include/linux/vmalloc.h +++ b/include/linux/vmalloc.h @@ -27,7 +27,9 @@ struct notifier_block; /* in notifier.h */ #define VM_FLUSH_RESET_PERMS 0x00000100 /* reset direct map and flush TLB on unmap, can't be freed in atomic context */ #define VM_MAP_PUT_PAGES 0x00000200 /* put pages and free array in vfree */ #define VM_NO_HUGE_VMAP 0x00000400 /* force PAGE_SIZE pte mapping */ +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING #define VM_HUGE_PAGES 0x00001000 /* used for vmalloc hugepages */ +#endif #ifdef CONFIG_ASCEND_SHARE_POOL #define VM_SHAREPOOL 0x00002000 /* remapped to sharepool */ #else @@ -142,8 +144,11 @@ extern void *__vmalloc_node_range(unsigned long size, unsigned long align, void *__vmalloc_node(unsigned long size, unsigned long align, gfp_t gfp_mask, int node, const void *caller); void *vmalloc_no_huge(unsigned long size); + +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING extern void *vmalloc_hugepage(unsigned long size); extern void *vmalloc_hugepage_user(unsigned long size); +#endif extern void vfree(const void *addr); extern void vfree_atomic(const void *addr); @@ -160,6 +165,7 @@ extern int remap_vmalloc_range_partial(struct vm_area_struct *vma, extern int remap_vmalloc_range(struct vm_area_struct *vma, void *addr, unsigned long pgoff); +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING extern void *vmap_hugepage(struct page **pages, unsigned int count, unsigned long flags, pgprot_t prot); extern int remap_vmalloc_hugepage_range_partial(struct vm_area_struct *vma, @@ -167,6 +173,7 @@ extern int remap_vmalloc_hugepage_range_partial(struct vm_area_struct *vma, unsigned long pgoff, unsigned long size); extern int remap_vmalloc_hugepage_range(struct vm_area_struct *vma, void *addr, unsigned long pgoff); +#endif /* * Architectures can set this mask to a combination of PGTBL_P?D_MODIFIED values diff --git a/mm/Kconfig b/mm/Kconfig index be7fd4ed2c4f..f66457168de9 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -992,6 +992,12 @@ config CLEAR_FREELIST_PAGE To enable this feature, kernel parameter "clear_freelist" also needs to be added. +config EXTEND_HUGEPAGE_MAPPING + bool "Extend for hugepages mapping" + depends on ARM64 + default n + help + Introduce vmalloc/vmap/remap interfaces that handle only hugepages. source "mm/damon/Kconfig" diff --git a/mm/vmalloc.c b/mm/vmalloc.c index d7a68eb0db42..e27cd716ca95 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -578,6 +578,7 @@ static int vmap_pages_range(unsigned long addr, unsigned long end, return err; } +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING static int vmap_hugepages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { @@ -609,6 +610,7 @@ static int vmap_hugepages_range(unsigned long addr, unsigned long end, return err; } +#endif /** * map_kernel_range_noflush - map kernel VM area with the specified pages @@ -2792,6 +2794,7 @@ void *vmap(struct page **pages, unsigned int count, } EXPORT_SYMBOL(vmap); +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING /** * vmap_hugepage - map an array of huge pages into virtually contiguous space * @pages: array of huge page pointers (only the header) @@ -2830,6 +2833,7 @@ void *vmap_hugepage(struct page **pages, unsigned int count, return area->addr; } EXPORT_SYMBOL(vmap_hugepage); +#endif #ifdef CONFIG_VMAP_PFN struct vmap_pfn_data { @@ -3015,7 +3019,11 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, size_per_node = size; if (node == NUMA_NO_NODE) size_per_node /= num_online_nodes(); +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING if (size_per_node >= PMD_SIZE || vm_flags & VM_HUGE_PAGES) { +#else + if (size_per_node >= PMD_SIZE) { +#endif shift = PMD_SHIFT; align = max(real_align, 1UL << shift); size = ALIGN(real_size, 1UL << shift); @@ -3050,8 +3058,12 @@ void *__vmalloc_node_range(unsigned long size, unsigned long align, return addr; fail: +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING /* User could specify VM_HUGE_PAGES to alloc only hugepages. */ if (shift > PAGE_SHIFT && !(vm_flags & VM_HUGE_PAGES)) { +#else + if (shift > PAGE_SHIFT) { +#endif shift = PAGE_SHIFT; align = real_align; size = real_size; @@ -3261,6 +3273,7 @@ void *vmalloc_32_user(unsigned long size) } EXPORT_SYMBOL(vmalloc_32_user); +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING /** * vmalloc_hugepage - allocate virtually contiguous hugetlb memory * @size: allocation size @@ -3298,6 +3311,7 @@ void *vmalloc_hugepage_user(unsigned long size) __builtin_return_address(0)); } EXPORT_SYMBOL(vmalloc_hugepage_user); +#endif /* * small helper routine , copy contents to buf from addr. @@ -3620,6 +3634,7 @@ int remap_vmalloc_range(struct vm_area_struct *vma, void *addr, } EXPORT_SYMBOL(remap_vmalloc_range); +#ifdef CONFIG_EXTEND_HUGEPAGE_MAPPING /** * remap_vmalloc_hugepage_range_partial - map vmalloc hugepages * to userspace @@ -3706,6 +3721,7 @@ int remap_vmalloc_hugepage_range(struct vm_area_struct *vma, void *addr, vma->vm_end - vma->vm_start); } EXPORT_SYMBOL(remap_vmalloc_hugepage_range); +#endif void free_vm_area(struct vm_struct *area) { -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ---------------------------------------------------- Add CONFIG_ARM_SMMU_V3_PM to control the support for arm-smmu-v3 suspend and resume which was introduced in 4b009f708c37ce215483ac68c6a0b6b9f0d29c6d. Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- arch/arm64/configs/openeuler_defconfig | 1 + drivers/iommu/Kconfig | 7 +++++++ drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 9 ++++----- 3 files changed, 12 insertions(+), 5 deletions(-) diff --git a/arch/arm64/configs/openeuler_defconfig b/arch/arm64/configs/openeuler_defconfig index 3f1532646775..d5d28adedd5e 100644 --- a/arch/arm64/configs/openeuler_defconfig +++ b/arch/arm64/configs/openeuler_defconfig @@ -5930,6 +5930,7 @@ CONFIG_ARM_SMMU=y CONFIG_ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT=y CONFIG_ARM_SMMU_V3=y CONFIG_ARM_SMMU_V3_SVA=y +CONFIG_ARM_SMMU_V3_PM=y # CONFIG_AGENT_SMMU_ATOS is not set # CONFIG_QCOM_IOMMU is not set # CONFIG_VIRTIO_IOMMU is not set diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index d632d456ed3c..f04a2bde0018 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -336,6 +336,13 @@ config AGENT_SMMU_ATOS Say Y here if your system will be used in Ascend Advanced Accelerator with HCCS bus. Or want use the ATOS of SMMU. +config ARM_SMMU_V3_PM + bool "Add arm_smmu_v3 suspend and resume support" + depends on ARM_SMMU_V3 && PM_SLEEP + default n + help + Add support for suspend and resume support for arm smmu v3. + config S390_IOMMU def_bool y if S390 && PCI depends on S390 && PCI diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index 3f6d25bf0587..be8cc53659f8 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -4349,7 +4349,7 @@ static void arm_smmu_write_msi_msg(struct msi_desc *desc, struct msi_msg *msg) doorbell = (((u64)msg->address_hi) << 32) | msg->address_lo; doorbell &= MSI_CFG0_ADDR_MASK; -#ifdef CONFIG_PM_SLEEP +#ifdef CONFIG_ARM_SMMU_V3_PM /* Saves the msg (base addr of msi irq) and restores it during resume */ desc->msg.address_lo = msg->address_lo; desc->msg.address_hi = msg->address_hi; @@ -4411,7 +4411,7 @@ static void arm_smmu_setup_msis(struct arm_smmu_device *smmu) devm_add_action(dev, arm_smmu_free_msis, dev); } -#ifdef CONFIG_PM_SLEEP +#ifdef CONFIG_ARM_SMMU_V3_PM static void arm_smmu_resume_msis(struct arm_smmu_device *smmu) { struct msi_desc *desc; @@ -5313,8 +5313,7 @@ static void __iomem *arm_smmu_ioremap(struct device *dev, resource_size_t start, return devm_ioremap_resource(dev, &res); } -#ifdef CONFIG_PM_SLEEP - +#ifdef CONFIG_ARM_SMMU_V3_PM static int arm_smmu_ecmdq_disable(struct device *dev) { int i, j; @@ -5521,7 +5520,7 @@ static const struct of_device_id arm_smmu_of_match[] = { }; MODULE_DEVICE_TABLE(of, arm_smmu_of_match); -#ifdef CONFIG_PM_SLEEP +#ifdef CONFIG_ARM_SMMU_V3_PM static const struct dev_pm_ops arm_smmu_pm_ops = { .suspend = arm_smmu_suspend, .resume = arm_smmu_resume, -- 2.17.1
Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ------------------------------------------------ Add config to encapsulate the code introduced in 2fb141bf9c23 ("[Huawei] mm: sharepool: fix hugepage_rsvd count increase error"). Signed-off-by: Zhang Zekun <zhangzekun11@huawei.com> --- fs/hugetlbfs/inode.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index cfdd8cffe6d7..58d1dd32a223 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -602,13 +602,20 @@ static void remove_inode_hugepages(struct inode *inode, loff_t lstart, * if the page is from buddy system, do not add to freed. * because freed is used for hugetlbfs reservation accounting. */ - if (!HPageTemporary(page)) { - freed++; - if (!truncate_op) { - if (unlikely(hugetlb_unreserve_pages(inode, + +#ifdef CONFIG_ASCEND_SHARE_POOL + if (HPageTemporary(page) != 0) { + unlock_page(page); + if (!truncate_op) + mutex_unlock(&hugetlb_fault_mutex_table[hash]); + continue; + } +#endif + freed++; + if (!truncate_op) { + if (unlikely(hugetlb_unreserve_pages(inode, index, index + 1, 1))) hugetlb_fix_reserve_counts(inode); - } } unlock_page(page); @@ -1061,8 +1068,12 @@ static int hugetlbfs_error_remove_page(struct address_space *mapping, pgoff_t index = page->index; remove_huge_page(page); +#ifdef CONFIG_ASCEND_SHARE_POOL if (!HPageTemporary(page) && unlikely(hugetlb_unreserve_pages(inode, index, index + 1, 1))) +#else + if (unlikely(hugetlb_unreserve_pages(inode, index, index + 1, 1))) +#endif hugetlb_fix_reserve_counts(inode); return 0; -- 2.17.1
From: Chen Jun <chenjun102@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I64Y5Y ------------------------------- If local_group_add_task fails in init_local_group. ida free the same id twice. init_local_group local_group_add_task // failed goto free_spg free_spg: free_sp_group_locked free_sp_group_id // free spg->id free_spg_id: free_new_spg_id // double free spg->id To fix it, return before calling free_new_spg_id. Signed-off-by: Chen Jun <chenjun102@huawei.com> --- mm/share_pool.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index e9d2ae0a5352..ababe279c44d 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -487,8 +487,8 @@ static int init_local_group(struct mm_struct *mm) spg = create_spg(spg_id, 0); if (IS_ERR(spg)) { - ret = PTR_ERR(spg); - goto free_spg_id; + free_new_spg_id(true, spg_id); + return PTR_ERR(spg); } master->local = spg; @@ -509,11 +509,9 @@ static int init_local_group(struct mm_struct *mm) return 0; free_spg: + /* spg_id is freed in free_sp_group_locked */ free_sp_group_locked(spg); master->local = NULL; -free_spg_id: - free_new_spg_id(true, spg_id); - return ret; } -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- If we delete a task that has not been added to any group from a specified group, NULL pointer dereference would occur. [ 162.566615] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000008 [ 162.567699] Mem abort info: [ 162.567971] ESR = 0x96000006 [ 162.568187] EC = 0x25: DABT (current EL), IL = 32 bits [ 162.568508] SET = 0, FnV = 0 [ 162.568670] EA = 0, S1PTW = 0 [ 162.568794] Data abort info: [ 162.568906] ISV = 0, ISS = 0x00000006 [ 162.569032] CM = 0, WnR = 0 [ 162.569314] user pgtable: 4k pages, 48-bit VAs, pgdp=00000001029e0000 [ 162.569516] [0000000000000008] pgd=00000001026da003, p4d=00000001026da003, pud=0000000102a90003, pmd=0000000000000000 [ 162.570346] Internal error: Oops: 96000006 [#1] SMP [ 162.570524] CPU: 0 PID: 880 Comm: test_sp_group_d Tainted: G W O 5.10.0+ #1 [ 162.570868] Hardware name: linux,dummy-virt (DT) [ 162.571053] pstate: 00000005 (nzcv daif -PAN -UAO -TCO BTYPE=--) [ 162.571370] pc : mg_sp_group_del_task+0x164/0x488 [ 162.571511] lr : mg_sp_group_del_task+0x158/0x488 [ 162.571644] sp : ffff8000127d3ca0 [ 162.571749] x29: ffff8000127d3ca0 x28: ffff372281b8c140 [ 162.571922] x27: 0000000000000000 x26: ffff372280b261c0 [ 162.572090] x25: ffffd075db9a9000 x24: ffffd075db9a90f8 [ 162.572259] x23: ffffd075db9a90e0 x22: 0000000000000371 [ 162.572425] x21: ffff372280826b00 x20: 0000000000000000 [ 162.572592] x19: ffffd075db12b000 x18: 0000000000000000 [ 162.572756] x17: 0000000000000000 x16: ffffd075da51e60c [ 162.572923] x15: 0000ffffdcf1a540 x14: 0000000000000000 [ 162.573087] x13: 0000000000000000 x12: 0000000000000000 [ 162.573250] x11: 0000000000000040 x10: ffffd075db5f1908 [ 162.573415] x9 : ffffd075db5f1900 x8 : ffff3722816f54b0 [ 162.573579] x7 : 0000000000000000 x6 : 0000000000000000 [ 162.573741] x5 : ffff3722816f5488 x4 : 0000000000000000 [ 162.573906] x3 : ffff372280b2620c x2 : ffff37228036b4a0 [ 162.574069] x1 : 0000000000000000 x0 : ffff372280b261c0 [ 162.574239] Call trace: [ 162.574336] mg_sp_group_del_task+0x164/0x488 [ 162.575262] dev_ioctl+0x10cc/0x2478 [sharepool_dev] [ 162.575443] __arm64_sys_ioctl+0xb4/0xf0 [ 162.575585] el0_svc_common.constprop.0+0xe4/0x2d4 [ 162.575726] do_el0_svc+0x34/0xa8 [ 162.575838] el0_svc+0x1c/0x28 [ 162.575941] el0_sync_handler+0x90/0xf0 [ 162.576060] el0_sync+0x168/0x180 [ 162.576391] Code: 97f4d4bf aa0003fa b4001580 f9420c01 (f8408c20) Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/mm/share_pool.c b/mm/share_pool.c index ababe279c44d..ee2d5fec7099 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -1661,6 +1661,13 @@ int mg_sp_group_del_task(int tgid, int spg_id) goto out_put_task; } + if (!mm->sp_group_master) { + up_write(&sp_group_sem); + pr_err("task(%d) is not in any group(%d)\n", tgid, spg_id); + ret = -EINVAL; + goto out_put_mm; + } + spg_node = find_spg_node_by_spg(mm, spg); if (!spg_node) { up_write(&sp_group_sem); -- 2.17.1
From: Zhou Guanghui <zhouguanghui1@huawei.com> Offering: HULK ascend inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK -------------------------------------------- The sharepool statistics record the sharepool memory information used by all containers in the system. We do not expect to query the sharepool memory information applied by processes in other containers in the container. Therefore, the sharepool statistics cannot be queried in the container to solve this problem. Signed-off-by: Zhou Guanghui <zhouguanghui1@huawei.com> --- mm/share_pool.c | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/mm/share_pool.c b/mm/share_pool.c index ee2d5fec7099..b3f5942dd7a4 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -4063,8 +4063,22 @@ void spg_overview_show(struct seq_file *seq) pr_info("\n"); } +static bool should_show_statistics(void) +{ + if (!capable(CAP_SYS_ADMIN)) + return false; + + if (task_active_pid_ns(current) != &init_pid_ns) + return false; + + return true; +} + static int spa_stat_show(struct seq_file *seq, void *offset) { + if (!should_show_statistics()) + return -EPERM; + spg_overview_show(seq); spa_overview_show(seq); /* print the file header */ @@ -4113,6 +4127,9 @@ static int proc_usage_by_group(int id, void *p, void *data) static int proc_group_usage_show(struct seq_file *seq, void *offset) { + if (!should_show_statistics()) + return -EPERM; + spg_overview_show(seq); spa_overview_show(seq); @@ -4140,6 +4157,9 @@ static int proc_usage_show(struct seq_file *seq, void *offset) long sp_res, sp_res_nsize, non_sp_res, non_sp_shm; struct sp_proc_stat *proc_stat; + if (!should_show_statistics()) + return -EPERM; + seq_printf(seq, "%-8s %-16s %-9s %-9s %-9s %-10s %-10s %-8s\n", "PID", "COMM", "SP_ALLOC", "SP_K2U", "SP_RES", "Non-SP_RES", "Non-SP_Shm", "VIRT"); -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ---------------------------------------------- In spa_overview_show, spg_info_show and spg_overview_show, there is similar code. The solution is to extract the difference into the function macro. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 89 +++++++++++++++++-------------------------------- 1 file changed, 31 insertions(+), 58 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index b3f5942dd7a4..ffba9ee1d2eb 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -99,6 +99,14 @@ static DEFINE_MUTEX(spm_list_lock); /* for kthread buff_module_guard_work */ static struct sp_proc_stat kthread_stat; +#define SEQ_printf(m, x...) \ +do { \ + if (m) \ + seq_printf(m, x); \ + else \ + pr_info(x); \ +} while (0) + #ifndef __GENKSYMS__ struct sp_spg_stat { int spg_id; @@ -3978,29 +3986,16 @@ void spa_overview_show(struct seq_file *seq) dvpp_va_size = spa_stat.dvpp_va_size; spin_unlock(&sp_area_lock); - if (seq != NULL) { - seq_printf(seq, "Spa total num %u.\n", total_num); - seq_printf(seq, "Spa alloc num %u, k2u(task) num %u, k2u(spg) num %u.\n", - alloc_num, k2u_task_num, k2u_spg_num); - seq_printf(seq, "Spa total size: %13lu KB\n", byte2kb(total_size)); - seq_printf(seq, "Spa alloc size: %13lu KB\n", byte2kb(alloc_size)); - seq_printf(seq, "Spa k2u(task) size: %13lu KB\n", byte2kb(k2u_task_size)); - seq_printf(seq, "Spa k2u(spg) size: %13lu KB\n", byte2kb(k2u_spg_size)); - seq_printf(seq, "Spa dvpp size: %13lu KB\n", byte2kb(dvpp_size)); - seq_printf(seq, "Spa dvpp va size: %13lu MB\n", byte2mb(dvpp_va_size)); - seq_puts(seq, "\n"); - } else { - pr_info("Spa total num %u.\n", total_num); - pr_info("Spa alloc num %u, k2u(task) num %u, k2u(spg) num %u.\n", - alloc_num, k2u_task_num, k2u_spg_num); - pr_info("Spa total size: %13lu KB\n", byte2kb(total_size)); - pr_info("Spa alloc size: %13lu KB\n", byte2kb(alloc_size)); - pr_info("Spa k2u(task) size: %13lu KB\n", byte2kb(k2u_task_size)); - pr_info("Spa k2u(spg) size: %13lu KB\n", byte2kb(k2u_spg_size)); - pr_info("Spa dvpp size: %13lu KB\n", byte2kb(dvpp_size)); - pr_info("Spa dvpp va size: %13lu MB\n", byte2mb(dvpp_va_size)); - pr_info("\n"); - } + SEQ_printf(seq, "Spa total num %u.\n", total_num); + SEQ_printf(seq, "Spa alloc num %u, k2u(task) num %u, k2u(spg) num %u.\n", + alloc_num, k2u_task_num, k2u_spg_num); + SEQ_printf(seq, "Spa total size: %13lu KB\n", byte2kb(total_size)); + SEQ_printf(seq, "Spa alloc size: %13lu KB\n", byte2kb(alloc_size)); + SEQ_printf(seq, "Spa k2u(task) size: %13lu KB\n", byte2kb(k2u_task_size)); + SEQ_printf(seq, "Spa k2u(spg) size: %13lu KB\n", byte2kb(k2u_spg_size)); + SEQ_printf(seq, "Spa dvpp size: %13lu KB\n", byte2kb(dvpp_size)); + SEQ_printf(seq, "Spa dvpp va size: %13lu MB\n", byte2mb(dvpp_va_size)); + SEQ_printf(seq, "\n"); } static int spg_info_show(int id, void *p, void *data) @@ -4011,29 +4006,16 @@ static int spg_info_show(int id, void *p, void *data) if (id >= SPG_ID_LOCAL_MIN && id <= SPG_ID_LOCAL_MAX) return 0; - if (seq != NULL) { - seq_printf(seq, "Group %6d ", id); - - down_read(&spg->rw_lock); - seq_printf(seq, "size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", - byte2kb(atomic64_read(&spg->instat.size)), - atomic_read(&spg->instat.spa_num), - byte2kb(atomic64_read(&spg->instat.alloc_size)), - byte2kb(atomic64_read(&spg->instat.alloc_nsize)), - byte2kb(atomic64_read(&spg->instat.alloc_hsize))); - up_read(&spg->rw_lock); - } else { - pr_info("Group %6d ", id); + SEQ_printf(seq, "Group %6d ", id); - down_read(&spg->rw_lock); - pr_info("size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", - byte2kb(atomic64_read(&spg->instat.size)), - atomic_read(&spg->instat.spa_num), - byte2kb(atomic64_read(&spg->instat.alloc_size)), - byte2kb(atomic64_read(&spg->instat.alloc_nsize)), - byte2kb(atomic64_read(&spg->instat.alloc_hsize))); - up_read(&spg->rw_lock); - } + down_read(&spg->rw_lock); + SEQ_printf(seq, "size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", + byte2kb(atomic64_read(&spg->instat.size)), + atomic_read(&spg->instat.spa_num), + byte2kb(atomic64_read(&spg->instat.alloc_size)), + byte2kb(atomic64_read(&spg->instat.alloc_nsize)), + byte2kb(atomic64_read(&spg->instat.alloc_hsize))); + up_read(&spg->rw_lock); return 0; } @@ -4043,24 +4025,15 @@ void spg_overview_show(struct seq_file *seq) if (!sp_is_enabled()) return; - if (seq != NULL) { - seq_printf(seq, "Share pool total size: %lld KB, spa total num: %d.\n", - byte2kb(atomic64_read(&sp_overall_stat.spa_total_size)), - atomic_read(&sp_overall_stat.spa_total_num)); - } else { - pr_info("Share pool total size: %lld KB, spa total num: %d.\n", - byte2kb(atomic64_read(&sp_overall_stat.spa_total_size)), - atomic_read(&sp_overall_stat.spa_total_num)); - } + SEQ_printf(seq, "Share pool total size: %lld KB, spa total num: %d.\n", + byte2kb(atomic64_read(&sp_overall_stat.spa_total_size)), + atomic_read(&sp_overall_stat.spa_total_num)); down_read(&sp_group_sem); idr_for_each(&sp_group_idr, spg_info_show, seq); up_read(&sp_group_sem); - if (seq != NULL) - seq_puts(seq, "\n"); - else - pr_info("\n"); + SEQ_printf(seq, "\n"); } static bool should_show_statistics(void) -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6HRGK ---------------------------------------------- Add two Helper functions sp_add_group_master and sp_del_group_master to manipulate master_list. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index ffba9ee1d2eb..b8ba31314adf 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -264,6 +264,20 @@ struct sp_group_node { }; #endif +static inline void sp_add_group_master(struct sp_group_master *master) +{ + mutex_lock(&master_list_lock); + list_add_tail(&master->list_node, &master_list); + mutex_unlock(&master_list_lock); +} + +static inline void sp_del_group_master(struct sp_group_master *master) +{ + mutex_lock(&master_list_lock); + list_del(&master->list_node); + mutex_unlock(&master_list_lock); +} + /* The caller should hold mmap_sem to protect master (TBD) */ static void sp_init_group_master_stat(int tgid, struct mm_struct *mm, struct sp_proc_stat *stat) @@ -541,10 +555,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct master->mm = mm; sp_init_group_master_stat(tsk->tgid, mm, &master->instat); mm->sp_group_master = master; - - mutex_lock(&master_list_lock); - list_add_tail(&master->list_node, &master_list); - mutex_unlock(&master_list_lock); + sp_add_group_master(master); ret = init_local_group(mm); if (ret) @@ -553,9 +564,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct return 0; free_master: - mutex_lock(&master_list_lock); - list_del(&master->list_node); - mutex_unlock(&master_list_lock); + sp_del_group_master(master); mm->sp_group_master = NULL; kfree(master); @@ -4387,9 +4396,7 @@ void sp_group_post_exit(struct mm_struct *mm) } up_write(&sp_group_sem); - mutex_lock(&master_list_lock); - list_del(&master->list_node); - mutex_unlock(&master_list_lock); + sp_del_group_master(master); kfree(master); } -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- 1. Extract a function that initialize all the members for a newly allocated sp_group. Just to decrease the function size. 2. Move the idr_alloc to the end of the function, since we should not add an uninitialized sp_group to the global idr. 3. Rename the file for hugetlb map. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 60 ++++++++++++++++++++++++++----------------------- 1 file changed, 32 insertions(+), 28 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index b8ba31314adf..9a7923eab134 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -1143,17 +1143,31 @@ static bool is_online_node_id(int node_id) return node_id >= 0 && node_id < MAX_NUMNODES && node_online(node_id); } +static void sp_group_init(struct sp_group *spg, int spg_id, unsigned long flag) +{ + spg->id = spg_id; + spg->flag = flag; + spg->is_alive = true; + spg->proc_num = 0; + atomic_set(&spg->use_count, 1); + INIT_LIST_HEAD(&spg->procs); + INIT_LIST_HEAD(&spg->spa_list); + INIT_LIST_HEAD(&spg->mnode); + init_rwsem(&spg->rw_lock); + sp_init_group_stat(&spg->instat); +} + static struct sp_group *create_spg(int spg_id, unsigned long flag) { int ret; struct sp_group *spg; - char name[20]; + char name[DNAME_INLINE_LEN]; struct user_struct *user = NULL; int hsize_log = MAP_HUGE_2MB >> MAP_HUGE_SHIFT; if (unlikely(system_group_count + 1 == MAX_GROUP_FOR_SYSTEM && !is_local_group(spg_id))) { - pr_err_ratelimited("reach system max group num\n"); + pr_err("reach system max group num\n"); return ERR_PTR(-ENOSPC); } @@ -1161,50 +1175,40 @@ static struct sp_group *create_spg(int spg_id, unsigned long flag) if (spg == NULL) return ERR_PTR(-ENOMEM); - ret = idr_alloc(&sp_group_idr, spg, spg_id, spg_id + 1, GFP_KERNEL); - if (ret < 0) { - pr_err_ratelimited("group %d idr alloc failed %d\n", - spg_id, ret); - goto out_kfree; - } - - spg->id = spg_id; - spg->flag = flag; - spg->is_alive = true; - spg->proc_num = 0; - atomic_set(&spg->use_count, 1); - INIT_LIST_HEAD(&spg->procs); - INIT_LIST_HEAD(&spg->spa_list); - INIT_LIST_HEAD(&spg->mnode); - init_rwsem(&spg->rw_lock); - sp_init_group_stat(&spg->instat); - sprintf(name, "sp_group_%d", spg_id); - spg->file = shmem_kernel_file_setup(name, MAX_LFS_FILESIZE, - VM_NORESERVE); + spg->file = shmem_kernel_file_setup(name, MAX_LFS_FILESIZE, VM_NORESERVE); if (IS_ERR(spg->file)) { pr_err("spg file setup failed %ld\n", PTR_ERR(spg->file)); ret = PTR_ERR(spg->file); - goto out_idr; + goto out_kfree; } + sprintf(name, "sp_group_%d_huge", spg_id); spg->file_hugetlb = hugetlb_file_setup(name, MAX_LFS_FILESIZE, - VM_NORESERVE, &user, HUGETLB_ANONHUGE_INODE, hsize_log); + VM_NORESERVE, &user, HUGETLB_ANONHUGE_INODE, hsize_log); if (IS_ERR(spg->file_hugetlb)) { - pr_err("spg file_hugetlb setup failed %ld\n", - PTR_ERR(spg->file_hugetlb)); + pr_err("spg file_hugetlb setup failed %ld\n", PTR_ERR(spg->file_hugetlb)); ret = PTR_ERR(spg->file_hugetlb); goto out_fput; } + sp_group_init(spg, spg_id, flag); + + ret = idr_alloc(&sp_group_idr, spg, spg_id, spg_id + 1, GFP_KERNEL); + if (ret < 0) { + pr_err("group %d idr alloc failed %d\n", spg_id, ret); + goto out_fput_huge; + } + if (!is_local_group(spg_id)) system_group_count++; + return spg; +out_fput_huge: + fput(spg->file_hugetlb); out_fput: fput(spg->file); -out_idr: - idr_remove(&sp_group_idr, spg_id); out_kfree: kfree(spg); return ERR_PTR(ret); -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- The process for k2task can be normalized with k2spg since there exist a local sp group for k2task. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 152 +++++++++++++++--------------------------------- 1 file changed, 48 insertions(+), 104 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 9a7923eab134..4ffc384ea160 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -928,12 +928,11 @@ struct sp_k2u_context { unsigned long size_aligned; unsigned long sp_flags; int state; - int spg_id; - bool to_task; + enum spa_type type; }; -static unsigned long sp_remap_kva_to_vma(unsigned long kva, struct sp_area *spa, - struct mm_struct *mm, unsigned long prot, struct sp_k2u_context *kc); +static unsigned long sp_remap_kva_to_vma(struct sp_area *spa, struct mm_struct *mm, + unsigned long prot, struct sp_k2u_context *kc); static void free_sp_group_id(int spg_id) { @@ -1539,7 +1538,7 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) spin_unlock(&sp_area_lock); if (spa->type == SPA_TYPE_K2SPG && spa->kva) { - addr = sp_remap_kva_to_vma(spa->kva, spa, mm, prot_spa, NULL); + addr = sp_remap_kva_to_vma(spa, mm, prot_spa, NULL); if (IS_ERR_VALUE(addr)) pr_warn("add group remap k2u failed %ld\n", addr); @@ -2699,14 +2698,15 @@ static unsigned long __sp_remap_get_pfn(unsigned long kva) } /* when called by k2u to group, always make sure rw_lock of spg is down */ -static unsigned long sp_remap_kva_to_vma(unsigned long kva, struct sp_area *spa, - struct mm_struct *mm, unsigned long prot, struct sp_k2u_context *kc) +static unsigned long sp_remap_kva_to_vma(struct sp_area *spa, struct mm_struct *mm, + unsigned long prot, struct sp_k2u_context *kc) { struct vm_area_struct *vma; unsigned long ret_addr; unsigned long populate = 0; int ret = 0; unsigned long addr, buf, offset; + unsigned long kva = spa->kva; down_write(&mm->mmap_lock); if (unlikely(mm->core_state)) { @@ -2767,111 +2767,50 @@ static unsigned long sp_remap_kva_to_vma(unsigned long kva, struct sp_area *spa, return ret_addr; } -/** - * sp_make_share_kva_to_task() - Share kernel memory to current task. - * @kva: the VA of shared kernel memory - * @size: the size of area to share, should be aligned properly - * @sp_flags: the flags for the opreation - * - * Return: - * * if succeed, return the shared user address to start at. - * * if fail, return the pointer of -errno. - */ -static void *sp_make_share_kva_to_task(unsigned long kva, unsigned long size, unsigned long sp_flags) -{ - int ret; - void *uva; - struct sp_area *spa; - struct sp_group_node *spg_node; - unsigned long prot = PROT_READ | PROT_WRITE; - struct sp_k2u_context kc; - struct sp_group *spg; - - down_write(&sp_group_sem); - ret = sp_init_group_master_locked(current, current->mm); - if (ret) { - up_write(&sp_group_sem); - pr_err_ratelimited("k2u_task init local mapping failed %d\n", ret); - return ERR_PTR(ret); - } - - spg = current->mm->sp_group_master->local; - up_write(&sp_group_sem); - - spa = sp_alloc_area(size, sp_flags, spg, SPA_TYPE_K2TASK, current->tgid); - if (IS_ERR(spa)) { - pr_err_ratelimited("alloc spa failed in k2u_task (potential no enough virtual memory when -75): %ld\n", - PTR_ERR(spa)); - return spa; - } - - spa->kva = kva; - kc.sp_flags = sp_flags; - uva = (void *)sp_remap_kva_to_vma(kva, spa, current->mm, prot, &kc); - if (IS_ERR(uva)) - pr_err("remap k2u to task failed %ld\n", PTR_ERR(uva)); - else { - spg_node = find_spg_node_by_spg(current->mm, spa->spg); - update_mem_usage(size, true, spa->is_hugepage, spg_node, SPA_TYPE_K2TASK); - spa->mm = current->mm; - } - __sp_area_drop(spa); - - return uva; -} - /** * Share kernel memory to a spg, the current process must be in that group - * @kva: the VA of shared kernel memory - * @size: the size of area to share, should be aligned properly - * @sp_flags: the flags for the opreation + * @kc: the context for k2u, including kva, size, flags... * @spg: the sp group to be shared with * * Return: the shared user address to start at */ -static void *sp_make_share_kva_to_spg(unsigned long kva, unsigned long size, - unsigned long sp_flags, struct sp_group *spg) +static void *sp_make_share_kva_to_spg(struct sp_k2u_context *kc, struct sp_group *spg) { struct sp_area *spa; struct mm_struct *mm; struct sp_group_node *spg_node; - void *uva = ERR_PTR(-ENODEV); - struct sp_k2u_context kc; unsigned long ret_addr = -ENODEV; down_read(&spg->rw_lock); - spa = sp_alloc_area(size, sp_flags, spg, SPA_TYPE_K2SPG, current->tgid); + spa = sp_alloc_area(kc->size_aligned, kc->sp_flags, spg, kc->type, current->tgid); if (IS_ERR(spa)) { up_read(&spg->rw_lock); - pr_err_ratelimited("alloc spa failed in k2u_spg (potential no enough virtual memory when -75): %ld\n", + pr_err("alloc spa failed in k2u_spg (potential no enough virtual memory when -75): %ld\n", PTR_ERR(spa)); return spa; } - spa->kva = kva; - kc.sp_flags = sp_flags; + spa->kva = kc->kva_aligned; list_for_each_entry(spg_node, &spg->procs, proc_node) { mm = spg_node->master->mm; - kc.state = K2U_NORMAL; - ret_addr = sp_remap_kva_to_vma(kva, spa, mm, spg_node->prot, &kc); + kc->state = K2U_NORMAL; + ret_addr = sp_remap_kva_to_vma(spa, mm, spg_node->prot, kc); if (IS_ERR_VALUE(ret_addr)) { - if (kc.state == K2U_COREDUMP) + if (kc->state == K2U_COREDUMP) continue; - uva = (void *)ret_addr; - pr_err("remap k2u to spg failed %ld\n", PTR_ERR(uva)); + pr_err("remap k2u to spg failed %ld\n", ret_addr); __sp_free(spg, spa->va_start, spa_size(spa), mm); goto out; } - uva = (void *)ret_addr; } out: up_read(&spg->rw_lock); - if (!IS_ERR(uva)) + if (!IS_ERR_VALUE(ret_addr)) sp_update_process_stat(current, true, spa); __sp_area_drop(spa); - return uva; + return (void *)ret_addr; } static bool vmalloc_area_set_flag(unsigned long kva, unsigned long flags) @@ -2930,16 +2869,13 @@ static int sp_k2u_prepare(unsigned long kva, unsigned long size, return -EINVAL; } - kc->kva = kva; - kc->kva_aligned = kva_aligned; - kc->size = size; + kc->kva = kva; + kc->kva_aligned = kva_aligned; + kc->size = size; kc->size_aligned = size_aligned; - kc->sp_flags = sp_flags; - kc->spg_id = spg_id; - if (spg_id == SPG_ID_DEFAULT || spg_id == SPG_ID_NONE) - kc->to_task = true; - else - kc->to_task = false; + kc->sp_flags = sp_flags; + kc->type = (spg_id == SPG_ID_DEFAULT || spg_id == SPG_ID_NONE) + ? SPA_TYPE_K2TASK : SPA_TYPE_K2SPG; return 0; } @@ -2977,6 +2913,7 @@ void *mg_sp_make_share_k2u(unsigned long kva, unsigned long size, void *uva; int ret; struct sp_k2u_context kc; + struct sp_group *spg; if (!sp_is_enabled()) return ERR_PTR(-EOPNOTSUPP); @@ -2987,24 +2924,31 @@ void *mg_sp_make_share_k2u(unsigned long kva, unsigned long size, if (ret) return ERR_PTR(ret); - if (kc.to_task) { - uva = sp_make_share_kva_to_task(kc.kva_aligned, kc.size_aligned, kc.sp_flags); - } else { - struct sp_group *spg; - - spg = __sp_find_spg(current->tgid, kc.spg_id); - if (spg) { - ret = sp_check_caller_permission(spg, current->mm); - if (ret < 0) { - sp_group_drop(spg); - uva = ERR_PTR(ret); - goto out; - } - uva = sp_make_share_kva_to_spg(kc.kva_aligned, kc.size_aligned, kc.sp_flags, spg); + if (kc.type == SPA_TYPE_K2TASK) { + down_write(&sp_group_sem); + ret = sp_init_group_master_locked(current, current->mm); + up_write(&sp_group_sem); + if (ret) { + pr_err("k2u_task init local mapping failed %d\n", ret); + uva = ERR_PTR(ret); + goto out; + } + /* the caller could use SPG_ID_NONE */ + spg_id = SPG_ID_DEFAULT; + } + + spg = __sp_find_spg(current->tgid, spg_id); + if (spg) { + ret = sp_check_caller_permission(spg, current->mm); + if (ret < 0) { sp_group_drop(spg); - } else { - uva = ERR_PTR(-ENODEV); + uva = ERR_PTR(ret); + goto out; } + uva = sp_make_share_kva_to_spg(&kc, spg); + sp_group_drop(spg); + } else { + uva = ERR_PTR(-ENODEV); } out: -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- Rename sp_group_drop[_locked] to sp_group_put[_locked]. Rename __sp_find_spg[_locked] to sp_group_get[_locked]. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 56 ++++++++++++++++++++++++------------------------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 4ffc384ea160..6c4ab8d397af 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -974,7 +974,7 @@ static void free_sp_group(struct sp_group *spg) up_write(&sp_group_sem); } -static void sp_group_drop_locked(struct sp_group *spg) +static void sp_group_put_locked(struct sp_group *spg) { lockdep_assert_held_write(&sp_group_sem); @@ -982,7 +982,7 @@ static void sp_group_drop_locked(struct sp_group *spg) free_sp_group_locked(spg); } -static void sp_group_drop(struct sp_group *spg) +static void sp_group_put(struct sp_group *spg) { if (atomic_dec_and_test(&spg->use_count)) free_sp_group(spg); @@ -1025,8 +1025,8 @@ static bool is_process_in_group(struct sp_group *spg, return false; } -/* user must call sp_group_drop() after use */ -static struct sp_group *__sp_find_spg_locked(int tgid, int spg_id) +/* user must call sp_group_put() after use */ +static struct sp_group *sp_group_get_locked(int tgid, int spg_id) { struct sp_group *spg = NULL; struct task_struct *tsk = NULL; @@ -1055,12 +1055,12 @@ static struct sp_group *__sp_find_spg_locked(int tgid, int spg_id) return spg; } -static struct sp_group *__sp_find_spg(int tgid, int spg_id) +static struct sp_group *sp_group_get(int tgid, int spg_id) { struct sp_group *spg; down_read(&sp_group_sem); - spg = __sp_find_spg_locked(tgid, spg_id); + spg = sp_group_get_locked(tgid, spg_id); up_read(&sp_group_sem); return spg; } @@ -1218,7 +1218,7 @@ static struct sp_group *find_or_alloc_sp_group(int spg_id, unsigned long flag) { struct sp_group *spg; - spg = __sp_find_spg_locked(current->tgid, spg_id); + spg = sp_group_get_locked(current->tgid, spg_id); if (!spg) { spg = create_spg(spg_id, flag); @@ -1226,11 +1226,11 @@ static struct sp_group *find_or_alloc_sp_group(int spg_id, unsigned long flag) down_read(&spg->rw_lock); if (!spg_valid(spg)) { up_read(&spg->rw_lock); - sp_group_drop_locked(spg); + sp_group_put_locked(spg); return ERR_PTR(-ENODEV); } up_read(&spg->rw_lock); - /* spg->use_count has increased due to __sp_find_spg() */ + /* spg->use_count has increased due to sp_group_get() */ } return spg; @@ -1413,7 +1413,7 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) } if (spg_id >= SPG_ID_AUTO_MIN && spg_id <= SPG_ID_AUTO_MAX) { - spg = __sp_find_spg(tgid, spg_id); + spg = sp_group_get(tgid, spg_id); if (!spg) { pr_err_ratelimited("spg %d hasn't been created\n", spg_id); @@ -1424,12 +1424,12 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) if (!spg_valid(spg)) { up_read(&spg->rw_lock); pr_err_ratelimited("add group failed, group id %d is dead\n", spg_id); - sp_group_drop(spg); + sp_group_put(spg); return -EINVAL; } up_read(&spg->rw_lock); - sp_group_drop(spg); + sp_group_put(spg); } if (spg_id == SPG_ID_AUTO) { @@ -1603,7 +1603,7 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) out_drop_group: if (unlikely(ret)) { up_write(&sp_group_sem); - sp_group_drop(spg); + sp_group_put(spg); } else up_write(&sp_group_sem); out_put_mm: @@ -1646,7 +1646,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) return -EINVAL; } - spg = __sp_find_spg(tgid, spg_id); + spg = sp_group_get(tgid, spg_id); if (!spg) { pr_err_ratelimited("spg not found or get task failed."); return -EINVAL; @@ -1701,7 +1701,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) is_alive = spg->is_alive = false; spg->proc_num--; list_del(&spg_node->proc_node); - sp_group_drop(spg); + sp_group_put(spg); up_write(&spg->rw_lock); if (!is_alive) blocking_notifier_call_chain(&sp_notifier_chain, 0, spg); @@ -1718,7 +1718,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) out_put_task: put_task_struct(tsk); out: - sp_group_drop(spg); /* if spg dead, freed here */ + sp_group_put(spg); /* if spg dead, freed here */ return ret; } EXPORT_SYMBOL_GPL(mg_sp_group_del_task); @@ -2163,7 +2163,7 @@ static int sp_free_get_spa(struct sp_free_context *fc) struct sp_area *spa; struct sp_group *spg; - spg = __sp_find_spg(current->tgid, fc->spg_id); + spg = sp_group_get(current->tgid, fc->spg_id); if (!spg) { pr_debug("sp free get group failed %d\n", fc->spg_id); return -EINVAL; @@ -2172,7 +2172,7 @@ static int sp_free_get_spa(struct sp_free_context *fc) fc->state = FREE_CONT; spa = get_sp_area(spg, addr); - sp_group_drop(spg); + sp_group_put(spg); if (!spa) { pr_debug("sp free invalid input addr %lx\n", addr); return -EINVAL; @@ -2344,7 +2344,7 @@ static int sp_alloc_prepare(unsigned long size, unsigned long sp_flags, sp_flags |= SP_HUGEPAGE; if (spg_id != SPG_ID_DEFAULT) { - spg = __sp_find_spg(current->tgid, spg_id); + spg = sp_group_get(current->tgid, spg_id); if (!spg) { pr_err_ratelimited("allocation failed, can't find group\n"); return -ENODEV; @@ -2354,14 +2354,14 @@ static int sp_alloc_prepare(unsigned long size, unsigned long sp_flags, down_read(&spg->rw_lock); if (!spg_valid(spg)) { up_read(&spg->rw_lock); - sp_group_drop(spg); + sp_group_put(spg); pr_err_ratelimited("allocation failed, spg is dead\n"); return -ENODEV; } if (!is_process_in_group(spg, current->mm)) { up_read(&spg->rw_lock); - sp_group_drop(spg); + sp_group_put(spg); pr_err_ratelimited("allocation failed, task not in group\n"); return -ENODEV; } @@ -2598,7 +2598,7 @@ static void sp_alloc_finish(int result, struct sp_area *spa, if (spa && !IS_ERR(spa)) __sp_area_drop(spa); - sp_group_drop(spg); + sp_group_put(spg); } /** @@ -2937,16 +2937,16 @@ void *mg_sp_make_share_k2u(unsigned long kva, unsigned long size, spg_id = SPG_ID_DEFAULT; } - spg = __sp_find_spg(current->tgid, spg_id); + spg = sp_group_get(current->tgid, spg_id); if (spg) { ret = sp_check_caller_permission(spg, current->mm); if (ret < 0) { - sp_group_drop(spg); + sp_group_put(spg); uva = ERR_PTR(ret); goto out; } uva = sp_make_share_kva_to_spg(&kc, spg); - sp_group_drop(spg); + sp_group_put(spg); } else { uva = ERR_PTR(-ENODEV); } @@ -3290,7 +3290,7 @@ static int sp_unshare_uva(unsigned long uva, unsigned long size, int group_id) unsigned int page_size; struct sp_group *spg; - spg = __sp_find_spg(current->tgid, group_id); + spg = sp_group_get(current->tgid, group_id); if (!spg) { pr_debug("sp unshare find group failed %d\n", group_id); return -EINVAL; @@ -3418,7 +3418,7 @@ static int sp_unshare_uva(unsigned long uva, unsigned long size, int group_id) out_drop_area: __sp_area_drop(spa); out: - sp_group_drop(spg); + sp_group_put(spg); return ret; } @@ -3643,7 +3643,7 @@ bool mg_sp_config_dvpp_range(size_t start, size_t size, int device_id, int tgid) err = true; put_spg: - sp_group_drop(spg); + sp_group_put(spg); put_mm: mmput(mm); put_task: -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- The unshare process for k2task can be normalized with k2spg since there exist a local sp group for k2task. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 147 +++++++++++++++--------------------------------- 1 file changed, 45 insertions(+), 102 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 6c4ab8d397af..5ffcb40e174e 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -3264,54 +3264,42 @@ void *mg_sp_make_share_u2k(unsigned long uva, unsigned long size, int tgid) EXPORT_SYMBOL_GPL(mg_sp_make_share_u2k); /* - * Input parameters uva, tgid and spg_id are now useless. spg_id will be useful - * when supporting a process in multiple sp groups. + * sp_unshare_uva - unshare a uva from sp_make_share_k2u + * @uva: the uva to be unshared + * @size: not used actually and we just check it + * @group_id: specify the spg of the uva; for local group, it can be SPG_ID_DEFAULT + * unless current process is exiting. * * Procedure of unshare uva must be compatible with: * * 1. DVPP channel destroy procedure: * do_exit() -> exit_mm() (mm no longer in spg and current->mm == NULL) -> * exit_task_work() -> task_work_run() -> __fput() -> ... -> vdec_close() -> - * sp_unshare(uva, SPG_ID_DEFAULT) - * - * 2. Process A once was the target of k2u(to group), then it exits. - * Guard worker kthread tries to free this uva and it must succeed, otherwise - * spa of this uva leaks. - * - * This also means we must trust DVPP channel destroy and guard worker code. + * sp_unshare(uva, local_spg_id) */ static int sp_unshare_uva(unsigned long uva, unsigned long size, int group_id) { int ret = 0; - struct mm_struct *mm; struct sp_area *spa; - unsigned long uva_aligned; - unsigned long size_aligned; unsigned int page_size; struct sp_group *spg; spg = sp_group_get(current->tgid, group_id); if (!spg) { - pr_debug("sp unshare find group failed %d\n", group_id); + pr_err("sp unshare find group failed %d\n", group_id); return -EINVAL; } - /* - * at first we guess it's a hugepage addr - * we can tolerate at most PMD_SIZE or PAGE_SIZE which is matched in k2u - */ + /* All the spa are aligned to 2M. */ spa = get_sp_area(spg, ALIGN_DOWN(uva, PMD_SIZE)); if (!spa) { - spa = get_sp_area(spg, ALIGN_DOWN(uva, PAGE_SIZE)); - if (!spa) { - ret = -EINVAL; - pr_debug("invalid input uva %lx in unshare uva\n", (unsigned long)uva); - goto out; - } + ret = -EINVAL; + pr_err("invalid input uva %lx in unshare uva\n", (unsigned long)uva); + goto out; } if (spa->type != SPA_TYPE_K2TASK && spa->type != SPA_TYPE_K2SPG) { - pr_err_ratelimited("unshare wrong type spa\n"); + pr_err("unshare wrong type spa\n"); ret = -EINVAL; goto out_drop_area; } @@ -3322,97 +3310,52 @@ static int sp_unshare_uva(unsigned long uva, unsigned long size, int group_id) * Thus input parameter size is not necessarily needed. */ page_size = (spa->is_hugepage ? PMD_SIZE : PAGE_SIZE); - uva_aligned = spa->va_start; - size_aligned = spa->real_size; - if (size_aligned < ALIGN(size, page_size)) { + if (spa->real_size < ALIGN(size, page_size)) { ret = -EINVAL; - pr_err_ratelimited("unshare uva failed, invalid parameter size %lu\n", size); + pr_err("unshare uva failed, invalid parameter size %lu\n", size); goto out_drop_area; } - if (spa->type == SPA_TYPE_K2TASK) { - if (spa->applier != current->tgid) { - pr_err_ratelimited("unshare uva(to task) no permission\n"); - ret = -EPERM; - goto out_drop_area; - } - - /* - * current thread may be exiting in a multithread process - * - * 1. never need a kthread to make unshare when process has exited - * 2. in dvpp channel destroy procedure, exit_mm() has been called - * and don't need to make unshare - */ - mm = get_task_mm(current->group_leader); - if (!mm) { - pr_info_ratelimited("no need to unshare uva(to task), target process mm is exiting\n"); - goto out_clr_flag; - } - - down_write(&mm->mmap_lock); - if (unlikely(mm->core_state)) { - ret = 0; - up_write(&mm->mmap_lock); - mmput(mm); - goto out_drop_area; - } - - ret = do_munmap(mm, uva_aligned, size_aligned, NULL); - up_write(&mm->mmap_lock); - mmput(mm); - /* we are not supposed to fail */ - if (ret) - pr_err("failed to unmap VA %pK when munmap in unshare uva\n", - (void *)uva_aligned); - sp_update_process_stat(current, false, spa); - - } else if (spa->type == SPA_TYPE_K2SPG) { - down_read(&spa->spg->rw_lock); - /* always allow kthread and dvpp channel destroy procedure */ - if (current->mm) { - if (!is_process_in_group(spa->spg, current->mm)) { - up_read(&spa->spg->rw_lock); - pr_err_ratelimited("unshare uva(to group) failed, caller process doesn't belong to target group\n"); - ret = -EPERM; - goto out_drop_area; - } - } + down_read(&spa->spg->rw_lock); + /* always allow dvpp channel destroy procedure */ + if (current->mm && !is_process_in_group(spa->spg, current->mm)) { up_read(&spa->spg->rw_lock); + pr_err("unshare uva failed, caller process doesn't belong to target group\n"); + ret = -EPERM; + goto out_drop_area; + } + up_read(&spa->spg->rw_lock); - down_write(&spa->spg->rw_lock); - if (!spg_valid(spa->spg)) { - up_write(&spa->spg->rw_lock); - pr_info_ratelimited("share pool: no need to unshare uva(to group), sp group of spa is dead\n"); - goto out_clr_flag; - } - /* the life cycle of spa has a direct relation with sp group */ - if (unlikely(spa->is_dead)) { - up_write(&spa->spg->rw_lock); - pr_err_ratelimited("unexpected double sp unshare\n"); - dump_stack(); - ret = -EINVAL; - goto out_drop_area; - } - spa->is_dead = true; + down_write(&spa->spg->rw_lock); + if (!spg_valid(spa->spg)) { up_write(&spa->spg->rw_lock); + pr_info("no need to unshare uva, sp group of spa is dead\n"); + goto out_clr_flag; + } + /* the life cycle of spa has a direct relation with sp group */ + if (unlikely(spa->is_dead)) { + up_write(&spa->spg->rw_lock); + pr_err("unexpected double sp unshare\n"); + dump_stack(); + ret = -EINVAL; + goto out_drop_area; + } + spa->is_dead = true; + up_write(&spa->spg->rw_lock); - down_read(&spa->spg->rw_lock); - __sp_free(spa->spg, uva_aligned, size_aligned, NULL); - up_read(&spa->spg->rw_lock); + down_read(&spa->spg->rw_lock); + __sp_free(spa->spg, spa->va_start, spa->real_size, NULL); + up_read(&spa->spg->rw_lock); - if (current->mm == NULL) - atomic64_sub(spa->real_size, &kthread_stat.k2u_size); - else - sp_update_process_stat(current, false, spa); - } else { - WARN(1, "unshare uva invalid spa type"); - } + if (current->mm == NULL) + atomic64_sub(spa->real_size, &kthread_stat.k2u_size); + else + sp_update_process_stat(current, false, spa); out_clr_flag: if (!vmalloc_area_clr_flag(spa->kva, VM_SHAREPOOL)) - pr_debug("clear spa->kva %ld is not valid\n", spa->kva); + pr_info("clear spa->kva %ld is not valid\n", spa->kva); spa->kva = 0; out_drop_area: -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6G76L ---------------------------------------------- When a process is deleted from a group, the process does not apply for memory from the shared group. Otherwise, the UAF problem occurs. We checked this, but it didn't do a good job of preventing sp_alloc and del_task concurrency. The process applies for memory after passing the check, which violates our requirements and causes problems. The solution is to place the checked code in the critical area to ensure that no memory can be allocated after the check is passed. [ T7596] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000098 [ T7596] Mem abort info: [ T7596] ESR = 0x96000004 [ T7596] EC = 0x25: DABT (current EL), IL = 32 bits [ T7596] SET = 0, FnV = 0 [ T7596] EA = 0, S1PTW = 0 [ T7596] Data abort info: [ T7596] ISV = 0, ISS = 0x00000004 [ T7596] CM = 0, WnR = 0 [ T7596] user pgtable: 4k pages, 48-bit VAs, pgdp=00000001040a3000 [ T7596] [0000000000000098] pgd=0000000000000000, p4d=0000000000000000 [ T7596] Internal error: Oops: 96000004 [#1] SMP [ T7596] Modules linked in: sharepool_dev(OE) [last unloaded: demo] [ T7596] CPU: 1 PID: 7596 Comm: test_sp_group_d Tainted: G OE 5.10.0+ #8 [ T7596] Hardware name: linux,dummy-virt (DT) [ T7596] pstate: 20000005 (nzCv daif -PAN -UAO -TCO BTYPE=--) [ T7596] pc : sp_free_area+0x34/0x120 [ T7596] lr : sp_free_area+0x30/0x120 [ T7596] sp : ffff80001c6a3b20 [ T7596] x29: ffff80001c6a3b20 x28: 0000000000000009 [ T7596] x27: 0000000000000000 x26: ffff800011c49d20 [ T7596] x25: ffff0000c227f6c0 x24: 0000000000000008 [ T7596] x23: ffff0000c0cf0ce8 x22: 0000000000000001 [ T7596] x21: ffff0000c4082b30 x20: 0000000000000000 [ T7596] x19: ffff0000c4082b00 x18: 0000000000000000 [ T7596] x17: 0000000000000000 x16: 0000000000000000 [ T7596] x15: 0000000000000000 x14: 0000000000000000 [ T7596] x13: 0000000000000000 x12: ffff0005fffe12c0 [ T7596] x11: 0000000000000008 x10: ffff0005fffe12c0 [ T7596] x9 : ffff8000103eb690 x8 : 0000000000000001 [ T7596] x7 : 0000000000210d00 x6 : 0000000000000000 [ T7596] x5 : ffff8000123edea0 x4 : 0000000000000030 [ T7596] x3 : ffffeff000000000 x2 : 0000eff000000000 [ T7596] x1 : 0000e80000000000 x0 : 0000000000000000 [ T7596] Call trace: [ T7596] sp_free_area+0x34/0x120 [ T7596] __sp_area_drop_locked+0x3c/0x60 [ T7596] sp_area_drop+0x80/0xbc [ T7596] remove_vma+0x54/0x70 [ T7596] exit_mmap+0x114/0x1d0 [ T7596] mmput+0x90/0x1ec [ T7596] exit_mm+0x1d0/0x2f0 [ T7596] do_exit+0x180/0x400 [ T7596] do_group_exit+0x40/0x114 [ T7596] get_signal+0x1e8/0x720 [ T7596] do_signal+0x11c/0x1e4 [ T7596] do_notify_resume+0x15c/0x250 [ T7596] work_pending+0xc/0x6d8 [ T7596] Code: f9400001 f9402c00 97fff0e5 aa0003f4 (f9404c00) [ T7596] ---[ end trace 3c8368d77e758ebd ]--- Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 5ffcb40e174e..3f0b10696e9d 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -1660,13 +1660,6 @@ int mg_sp_group_del_task(int tgid, int spg_id) goto out; } - if (!list_empty(&spg->spa_list)) { - up_write(&sp_group_sem); - pr_err_ratelimited("spa is not empty"); - ret = -EINVAL; - goto out; - } - ret = get_task(tgid, &tsk); if (ret) { up_write(&sp_group_sem); @@ -1697,6 +1690,15 @@ int mg_sp_group_del_task(int tgid, int spg_id) } down_write(&spg->rw_lock); + + if (!list_empty(&spg->spa_list)) { + up_write(&spg->rw_lock); + up_write(&sp_group_sem); + pr_err_ratelimited("spa is not empty"); + ret = -EINVAL; + goto out_put_mm; + } + if (list_is_singular(&spg->procs)) is_alive = spg->is_alive = false; spg->proc_num--; -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6G76L ---------------------------------------------- 1. Give more informaton in the error log. 2. No need to limit thre rate. 3. Add a '\n' at the end of the format string. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 3f0b10696e9d..8ae0aa75222c 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -1642,20 +1642,21 @@ int mg_sp_group_del_task(int tgid, int spg_id) return -EOPNOTSUPP; if (spg_id < SPG_ID_MIN || spg_id > SPG_ID_AUTO) { - pr_err_ratelimited("del from group failed, invalid group id %d\n", spg_id); + pr_err("del from group failed, invalid group id %d\n", spg_id); return -EINVAL; } spg = sp_group_get(tgid, spg_id); if (!spg) { - pr_err_ratelimited("spg not found or get task failed."); + pr_err("spg not found or get task failed, tgid:%d, spg_id:%d\n", + tgid, spg_id); return -EINVAL; } down_write(&sp_group_sem); if (!spg_valid(spg)) { up_write(&sp_group_sem); - pr_err_ratelimited("spg dead."); + pr_err("spg dead, spg_id:%d\n", spg_id); ret = -EINVAL; goto out; } @@ -1663,13 +1664,13 @@ int mg_sp_group_del_task(int tgid, int spg_id) ret = get_task(tgid, &tsk); if (ret) { up_write(&sp_group_sem); - pr_err_ratelimited("task is not found"); + pr_err("task is not found, tgid:%d\n", tgid); goto out; } mm = get_task_mm(tsk->group_leader); if (!mm) { up_write(&sp_group_sem); - pr_err_ratelimited("mm is not found"); + pr_err("mm is not found, tgid:%d\n", tgid); ret = -ESRCH; goto out_put_task; } @@ -1684,7 +1685,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) spg_node = find_spg_node_by_spg(mm, spg); if (!spg_node) { up_write(&sp_group_sem); - pr_err_ratelimited("process not in group"); + pr_err("task(%d) not in group(%d)\n", tgid, spg_id); ret = -ESRCH; goto out_put_mm; } @@ -1694,7 +1695,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) if (!list_empty(&spg->spa_list)) { up_write(&spg->rw_lock); up_write(&sp_group_sem); - pr_err_ratelimited("spa is not empty"); + pr_err("spa is not empty, task:%d, spg_id:%d\n", tgid, spg_id); ret = -EINVAL; goto out_put_mm; } -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 8ae0aa75222c..014c95d6c95d 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -109,9 +109,6 @@ do { \ #ifndef __GENKSYMS__ struct sp_spg_stat { - int spg_id; - /* record the number of hugepage allocation failures */ - atomic_t hugepage_failures; /* number of sp_area */ atomic_t spa_num; /* total size of all sp_area from sp_alloc and k2u */ @@ -690,7 +687,6 @@ static void sp_init_spg_proc_stat(struct spg_proc_stat *stat, int spg_id) static void sp_init_group_stat(struct sp_spg_stat *stat) { - atomic_set(&stat->hugepage_failures, 0); atomic_set(&stat->spa_num, 0); atomic64_set(&stat->size, 0); atomic64_set(&stat->alloc_nsize, 0); @@ -2462,7 +2458,6 @@ static void sp_alloc_fallback(struct sp_area *spa, struct sp_alloc_context *ac) return; } - atomic_inc(&ac->spg->instat.hugepage_failures); if (!(ac->sp_flags & SP_HUGEPAGE_ONLY)) { ac->file = ac->spg->file; ac->size_aligned = ALIGN(ac->size, PAGE_SIZE); -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 014c95d6c95d..3dfd8d4b7c45 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -126,7 +126,6 @@ struct sp_spg_stat { /* per process memory usage statistics indexed by tgid */ struct sp_proc_stat { int tgid; - struct mm_struct *mm; char comm[TASK_COMM_LEN]; /* * alloc amount minus free amount, may be negative when freed by @@ -282,7 +281,6 @@ static void sp_init_group_master_stat(int tgid, struct mm_struct *mm, atomic64_set(&stat->alloc_nsize, 0); atomic64_set(&stat->alloc_hsize, 0); atomic64_set(&stat->k2u_size, 0); - stat->mm = mm; stat->tgid = tgid; get_task_comm(stat->comm, current); } -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- spa_num is not general information, but differentiated information. It should not be placed in sp_spg_stat. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 3dfd8d4b7c45..b72aed1258f9 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -109,8 +109,6 @@ do { \ #ifndef __GENKSYMS__ struct sp_spg_stat { - /* number of sp_area */ - atomic_t spa_num; /* total size of all sp_area from sp_alloc and k2u */ atomic64_t size; /* total size of all sp_area from sp_alloc 0-order page */ @@ -216,6 +214,7 @@ struct sp_group { /* is_alive == false means it's being destroyed */ bool is_alive; atomic_t use_count; + atomic_t spa_num; /* protect the group internal elements, except spa_list */ struct rw_semaphore rw_lock; /* list node for dvpp mapping */ @@ -602,7 +601,6 @@ static void update_spg_stat_alloc(unsigned long size, bool inc, bool huge, struct sp_spg_stat *stat) { if (inc) { - atomic_inc(&stat->spa_num); atomic64_add(size, &stat->size); atomic64_add(size, &stat->alloc_size); if (huge) @@ -610,7 +608,6 @@ static void update_spg_stat_alloc(unsigned long size, bool inc, else atomic64_add(size, &stat->alloc_nsize); } else { - atomic_dec(&stat->spa_num); atomic64_sub(size, &stat->size); atomic64_sub(size, &stat->alloc_size); if (huge) @@ -624,11 +621,9 @@ static void update_spg_stat_k2u(unsigned long size, bool inc, struct sp_spg_stat *stat) { if (inc) { - atomic_inc(&stat->spa_num); atomic64_add(size, &stat->size); atomic64_add(size, &stat->k2u_size); } else { - atomic_dec(&stat->spa_num); atomic64_sub(size, &stat->size); atomic64_sub(size, &stat->k2u_size); } @@ -685,7 +680,6 @@ static void sp_init_spg_proc_stat(struct spg_proc_stat *stat, int spg_id) static void sp_init_group_stat(struct sp_spg_stat *stat) { - atomic_set(&stat->spa_num, 0); atomic64_set(&stat->size, 0); atomic64_set(&stat->alloc_nsize, 0); atomic64_set(&stat->alloc_hsize, 0); @@ -799,6 +793,7 @@ static void spa_inc_usage(struct sp_area *spa) spa_stat.dvpp_va_size += ALIGN(size, PMD_SIZE); } + atomic_inc(&spa->spg->spa_num); /* * all the calculations won't overflow due to system limitation and * parameter checking in sp_alloc_area() @@ -845,6 +840,7 @@ static void spa_dec_usage(struct sp_area *spa) spa_stat.dvpp_va_size -= ALIGN(size, PMD_SIZE); } + atomic_dec(&spa->spg->spa_num); spa_stat.total_num -= 1; spa_stat.total_size -= size; @@ -1143,6 +1139,7 @@ static void sp_group_init(struct sp_group *spg, int spg_id, unsigned long flag) spg->is_alive = true; spg->proc_num = 0; atomic_set(&spg->use_count, 1); + atomic_set(&spg->spa_num, 0); INIT_LIST_HEAD(&spg->procs); INIT_LIST_HEAD(&spg->spa_list); INIT_LIST_HEAD(&spg->mnode); @@ -3907,7 +3904,7 @@ static int spg_info_show(int id, void *p, void *data) down_read(&spg->rw_lock); SEQ_printf(seq, "size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", byte2kb(atomic64_read(&spg->instat.size)), - atomic_read(&spg->instat.spa_num), + atomic_read(&spg->spa_num), byte2kb(atomic64_read(&spg->instat.alloc_size)), byte2kb(atomic64_read(&spg->instat.alloc_nsize)), byte2kb(atomic64_read(&spg->instat.alloc_hsize))); -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 136 +++++++++++++++++++++++------------------------- 1 file changed, 66 insertions(+), 70 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index b72aed1258f9..1732a615ea32 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -107,20 +107,20 @@ do { \ pr_info(x); \ } while (0) -#ifndef __GENKSYMS__ -struct sp_spg_stat { - /* total size of all sp_area from sp_alloc and k2u */ - atomic64_t size; - /* total size of all sp_area from sp_alloc 0-order page */ - atomic64_t alloc_nsize; - /* total size of all sp_area from sp_alloc hugepage */ - atomic64_t alloc_hsize; - /* total size of all sp_area from ap_alloc */ - atomic64_t alloc_size; - /* total size of all sp_area from sp_k2u */ - atomic64_t k2u_size; +struct sp_meminfo { + /* total size from sp_alloc and k2u */ + atomic64_t size; + /* not huge page size from sp_alloc */ + atomic64_t alloc_nsize; + /* huge page size from sp_alloc */ + atomic64_t alloc_hsize; + /* total size from sp_alloc */ + atomic64_t alloc_size; + /* total size from sp_k2u */ + atomic64_t k2u_size; }; +#ifndef __GENKSYMS__ /* per process memory usage statistics indexed by tgid */ struct sp_proc_stat { int tgid; @@ -210,7 +210,7 @@ struct sp_group { /* list head of sp_area. it is protected by spin_lock sp_area_lock */ struct list_head spa_list; /* group statistics */ - struct sp_spg_stat instat; + struct sp_meminfo meminfo; /* is_alive == false means it's being destroyed */ bool is_alive; atomic_t use_count; @@ -273,6 +273,43 @@ static inline void sp_del_group_master(struct sp_group_master *master) mutex_unlock(&master_list_lock); } +static void meminfo_init(struct sp_meminfo *meminfo) +{ + memset(meminfo, 0, sizeof(struct sp_meminfo)); +} + +static void meminfo_update(unsigned long size, bool inc, + bool huge, struct sp_meminfo *meminfo) +{ + if (inc) { + atomic64_add(size, &meminfo->size); + atomic64_add(size, &meminfo->alloc_size); + if (huge) + atomic64_add(size, &meminfo->alloc_hsize); + else + atomic64_add(size, &meminfo->alloc_nsize); + } else { + atomic64_sub(size, &meminfo->size); + atomic64_sub(size, &meminfo->alloc_size); + if (huge) + atomic64_sub(size, &meminfo->alloc_hsize); + else + atomic64_sub(size, &meminfo->alloc_nsize); + } +} + +static void meminfo_update_k2u(unsigned long size, bool inc, + struct sp_meminfo *meminfo) +{ + if (inc) { + atomic64_add(size, &meminfo->size); + atomic64_add(size, &meminfo->k2u_size); + } else { + atomic64_sub(size, &meminfo->size); + atomic64_sub(size, &meminfo->k2u_size); + } +} + /* The caller should hold mmap_sem to protect master (TBD) */ static void sp_init_group_master_stat(int tgid, struct mm_struct *mm, struct sp_proc_stat *stat) @@ -597,38 +634,6 @@ static struct sp_group *sp_get_local_group(struct task_struct *tsk, struct mm_st return master->local; } -static void update_spg_stat_alloc(unsigned long size, bool inc, - bool huge, struct sp_spg_stat *stat) -{ - if (inc) { - atomic64_add(size, &stat->size); - atomic64_add(size, &stat->alloc_size); - if (huge) - atomic64_add(size, &stat->alloc_hsize); - else - atomic64_add(size, &stat->alloc_nsize); - } else { - atomic64_sub(size, &stat->size); - atomic64_sub(size, &stat->alloc_size); - if (huge) - atomic64_sub(size, &stat->alloc_hsize); - else - atomic64_sub(size, &stat->alloc_nsize); - } -} - -static void update_spg_stat_k2u(unsigned long size, bool inc, - struct sp_spg_stat *stat) -{ - if (inc) { - atomic64_add(size, &stat->size); - atomic64_add(size, &stat->k2u_size); - } else { - atomic64_sub(size, &stat->size); - atomic64_sub(size, &stat->k2u_size); - } -} - static void update_mem_usage_alloc(unsigned long size, bool inc, bool is_hugepage, struct sp_group_node *spg_node) { @@ -678,15 +683,6 @@ static void sp_init_spg_proc_stat(struct spg_proc_stat *stat, int spg_id) atomic64_set(&stat->k2u_size, 0); } -static void sp_init_group_stat(struct sp_spg_stat *stat) -{ - atomic64_set(&stat->size, 0); - atomic64_set(&stat->alloc_nsize, 0); - atomic64_set(&stat->alloc_hsize, 0); - atomic64_set(&stat->alloc_size, 0); - atomic64_set(&stat->k2u_size, 0); -} - /* statistics of all sp area, protected by sp_area_lock */ struct sp_spa_stat { unsigned int total_num; @@ -772,17 +768,17 @@ static void spa_inc_usage(struct sp_area *spa) case SPA_TYPE_ALLOC: spa_stat.alloc_num += 1; spa_stat.alloc_size += size; - update_spg_stat_alloc(size, true, is_huge, &spa->spg->instat); + meminfo_update(size, true, is_huge, &spa->spg->meminfo); break; case SPA_TYPE_K2TASK: spa_stat.k2u_task_num += 1; spa_stat.k2u_task_size += size; - update_spg_stat_k2u(size, true, &spa->spg->instat); + meminfo_update_k2u(size, true, &spa->spg->meminfo); break; case SPA_TYPE_K2SPG: spa_stat.k2u_spg_num += 1; spa_stat.k2u_spg_size += size; - update_spg_stat_k2u(size, true, &spa->spg->instat); + meminfo_update_k2u(size, true, &spa->spg->meminfo); break; default: WARN(1, "invalid spa type"); @@ -819,17 +815,17 @@ static void spa_dec_usage(struct sp_area *spa) case SPA_TYPE_ALLOC: spa_stat.alloc_num -= 1; spa_stat.alloc_size -= size; - update_spg_stat_alloc(size, false, is_huge, &spa->spg->instat); + meminfo_update(size, false, is_huge, &spa->spg->meminfo); break; case SPA_TYPE_K2TASK: spa_stat.k2u_task_num -= 1; spa_stat.k2u_task_size -= size; - update_spg_stat_k2u(size, false, &spa->spg->instat); + meminfo_update_k2u(size, false, &spa->spg->meminfo); break; case SPA_TYPE_K2SPG: spa_stat.k2u_spg_num -= 1; spa_stat.k2u_spg_size -= size; - update_spg_stat_k2u(size, false, &spa->spg->instat); + meminfo_update_k2u(size, false, &spa->spg->meminfo); break; default: WARN(1, "invalid spa type"); @@ -1144,7 +1140,7 @@ static void sp_group_init(struct sp_group *spg, int spg_id, unsigned long flag) INIT_LIST_HEAD(&spg->spa_list); INIT_LIST_HEAD(&spg->mnode); init_rwsem(&spg->rw_lock); - sp_init_group_stat(&spg->instat); + meminfo_init(&spg->meminfo); } static struct sp_group *create_spg(int spg_id, unsigned long flag) @@ -3671,16 +3667,16 @@ static void get_process_sp_res(struct sp_group_master *master, list_for_each_entry(spg_node, &master->node_list, group_node) { spg = spg_node->spg; - *sp_res_out += byte2kb(atomic64_read(&spg->instat.alloc_nsize)); - *sp_res_out += byte2kb(atomic64_read(&spg->instat.alloc_hsize)); - *sp_res_nsize_out += byte2kb(atomic64_read(&spg->instat.alloc_nsize)); + *sp_res_out += byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)); + *sp_res_out += byte2kb(atomic64_read(&spg->meminfo.alloc_hsize)); + *sp_res_nsize_out += byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)); } } static long get_sp_res_by_spg_proc(struct sp_group_node *spg_node) { - return byte2kb(atomic64_read(&spg_node->spg->instat.alloc_nsize) + - atomic64_read(&spg_node->spg->instat.alloc_hsize)); + return byte2kb(atomic64_read(&spg_node->spg->meminfo.alloc_nsize) + + atomic64_read(&spg_node->spg->meminfo.alloc_hsize)); } /* @@ -3903,11 +3899,11 @@ static int spg_info_show(int id, void *p, void *data) down_read(&spg->rw_lock); SEQ_printf(seq, "size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", - byte2kb(atomic64_read(&spg->instat.size)), + byte2kb(atomic64_read(&spg->meminfo.size)), atomic_read(&spg->spa_num), - byte2kb(atomic64_read(&spg->instat.alloc_size)), - byte2kb(atomic64_read(&spg->instat.alloc_nsize)), - byte2kb(atomic64_read(&spg->instat.alloc_hsize))); + byte2kb(atomic64_read(&spg->meminfo.alloc_size)), + byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)), + byte2kb(atomic64_read(&spg->meminfo.alloc_hsize))); up_read(&spg->rw_lock); return 0; -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 1732a615ea32..9dd6692d75d1 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -278,24 +278,24 @@ static void meminfo_init(struct sp_meminfo *meminfo) memset(meminfo, 0, sizeof(struct sp_meminfo)); } -static void meminfo_update(unsigned long size, bool inc, - bool huge, struct sp_meminfo *meminfo) +static void meminfo_inc_usage(unsigned long size, bool huge, struct sp_meminfo *meminfo) { - if (inc) { - atomic64_add(size, &meminfo->size); - atomic64_add(size, &meminfo->alloc_size); - if (huge) - atomic64_add(size, &meminfo->alloc_hsize); - else - atomic64_add(size, &meminfo->alloc_nsize); - } else { - atomic64_sub(size, &meminfo->size); - atomic64_sub(size, &meminfo->alloc_size); - if (huge) - atomic64_sub(size, &meminfo->alloc_hsize); - else - atomic64_sub(size, &meminfo->alloc_nsize); - } + atomic64_add(size, &meminfo->size); + atomic64_add(size, &meminfo->alloc_size); + if (huge) + atomic64_add(size, &meminfo->alloc_hsize); + else + atomic64_add(size, &meminfo->alloc_nsize); +} + +static void meminfo_dec_usage(unsigned long size, bool huge, struct sp_meminfo *meminfo) +{ + atomic64_sub(size, &meminfo->size); + atomic64_sub(size, &meminfo->alloc_size); + if (huge) + atomic64_sub(size, &meminfo->alloc_hsize); + else + atomic64_sub(size, &meminfo->alloc_nsize); } static void meminfo_update_k2u(unsigned long size, bool inc, @@ -768,7 +768,7 @@ static void spa_inc_usage(struct sp_area *spa) case SPA_TYPE_ALLOC: spa_stat.alloc_num += 1; spa_stat.alloc_size += size; - meminfo_update(size, true, is_huge, &spa->spg->meminfo); + meminfo_inc_usage(size, is_huge, &spa->spg->meminfo); break; case SPA_TYPE_K2TASK: spa_stat.k2u_task_num += 1; @@ -815,7 +815,7 @@ static void spa_dec_usage(struct sp_area *spa) case SPA_TYPE_ALLOC: spa_stat.alloc_num -= 1; spa_stat.alloc_size -= size; - meminfo_update(size, false, is_huge, &spa->spg->meminfo); + meminfo_dec_usage(size, is_huge, &spa->spg->meminfo); break; case SPA_TYPE_K2TASK: spa_stat.k2u_task_num -= 1; -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 9dd6692d75d1..1dd699371879 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -298,16 +298,16 @@ static void meminfo_dec_usage(unsigned long size, bool huge, struct sp_meminfo * atomic64_sub(size, &meminfo->alloc_nsize); } -static void meminfo_update_k2u(unsigned long size, bool inc, - struct sp_meminfo *meminfo) +static void meminfo_inc_k2u(unsigned long size, struct sp_meminfo *meminfo) { - if (inc) { - atomic64_add(size, &meminfo->size); - atomic64_add(size, &meminfo->k2u_size); - } else { - atomic64_sub(size, &meminfo->size); - atomic64_sub(size, &meminfo->k2u_size); - } + atomic64_add(size, &meminfo->size); + atomic64_add(size, &meminfo->k2u_size); +} + +static void meminfo_dec_k2u(unsigned long size, struct sp_meminfo *meminfo) +{ + atomic64_sub(size, &meminfo->size); + atomic64_sub(size, &meminfo->k2u_size); } /* The caller should hold mmap_sem to protect master (TBD) */ @@ -773,12 +773,12 @@ static void spa_inc_usage(struct sp_area *spa) case SPA_TYPE_K2TASK: spa_stat.k2u_task_num += 1; spa_stat.k2u_task_size += size; - meminfo_update_k2u(size, true, &spa->spg->meminfo); + meminfo_inc_k2u(size, &spa->spg->meminfo); break; case SPA_TYPE_K2SPG: spa_stat.k2u_spg_num += 1; spa_stat.k2u_spg_size += size; - meminfo_update_k2u(size, true, &spa->spg->meminfo); + meminfo_inc_k2u(size, &spa->spg->meminfo); break; default: WARN(1, "invalid spa type"); @@ -820,12 +820,12 @@ static void spa_dec_usage(struct sp_area *spa) case SPA_TYPE_K2TASK: spa_stat.k2u_task_num -= 1; spa_stat.k2u_task_size -= size; - meminfo_update_k2u(size, false, &spa->spg->meminfo); + meminfo_dec_k2u(size, &spa->spg->meminfo); break; case SPA_TYPE_K2SPG: spa_stat.k2u_spg_num -= 1; spa_stat.k2u_spg_size -= size; - meminfo_update_k2u(size, false, &spa->spg->meminfo); + meminfo_dec_k2u(size, &spa->spg->meminfo); break; default: WARN(1, "invalid spa type"); -- 2.17.1
From: Wang Wensheng <wangwensheng4@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- There is a double delete list problem in sp_group_exit Unable to handle kernel paging request at virtual address dead000000000108 Call trace: sp_group_exit+0x104/0x238 do_exit+0x188/0xb88 __arm64_sys_exit+0x24/0x28 Calls to sp_group_exit depends on the value of group_dead, which is controlled by CLONE_THREAD. If process A clone B with CLONE_VM and *NO* CLONE_THREAD. A and B will have group_dead = 1 and have the same mm_struct on exit. So sp_group_exit processes an mm_struct more than once. To sovle the problem, we check the tgid in sp_group_exit and allow only the parent process to continue. Similar check should be added in mg_sp_group_add/del_task. Signed-off-by: Wang Wensheng <wangwensheng4@huawei.com> --- mm/share_pool.c | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/mm/share_pool.c b/mm/share_pool.c index 1dd699371879..2ba05eb7d1ca 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -224,6 +224,7 @@ struct sp_group { /* a per-process(per mm) struct which manages a sp_group_node list */ struct sp_group_master { + pid_t tgid; /* * number of sp groups the process belongs to, * a.k.a the number of sp_node in node_list @@ -584,6 +585,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct INIT_LIST_HEAD(&master->node_list); master->count = 0; master->mm = mm; + master->tgid = tsk->tgid; sp_init_group_master_stat(tsk->tgid, mm, &master->instat); mm->sp_group_master = master; sp_add_group_master(master); @@ -1465,6 +1467,15 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) goto out_put_task; } + if (mm->sp_group_master && mm->sp_group_master->tgid != tgid) { + up_write(&sp_group_sem); + pr_err("add: task(%d) is a vfork child of the original task(%d)\n", + tgid, mm->sp_group_master->tgid); + ret = -EINVAL; + free_new_spg_id(id_newly_generated, spg_id); + goto out_put_mm; + } + spg = find_or_alloc_sp_group(spg_id, flag); if (IS_ERR(spg)) { up_write(&sp_group_sem); @@ -1669,6 +1680,14 @@ int mg_sp_group_del_task(int tgid, int spg_id) goto out_put_mm; } + if (mm->sp_group_master->tgid != tgid) { + up_write(&sp_group_sem); + pr_err("del: task(%d) is a vfork child of the original task(%d)\n", + tgid, mm->sp_group_master->tgid); + ret = -EINVAL; + goto out_put_mm; + } + spg_node = find_spg_node_by_spg(mm, spg); if (!spg_node) { up_write(&sp_group_sem); @@ -4200,6 +4219,11 @@ int sp_group_exit(void) return 0; } + if (master->tgid != current->tgid) { + up_write(&sp_group_sem); + return 0; + } + list_for_each_entry_safe(spg_node, tmp, &master->node_list, group_node) { spg = spg_node->spg; -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- The tgid information is also stored in sp_group_master. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 2ba05eb7d1ca..e11dc2769de5 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -123,7 +123,6 @@ struct sp_meminfo { #ifndef __GENKSYMS__ /* per process memory usage statistics indexed by tgid */ struct sp_proc_stat { - int tgid; char comm[TASK_COMM_LEN]; /* * alloc amount minus free amount, may be negative when freed by @@ -312,13 +311,12 @@ static void meminfo_dec_k2u(unsigned long size, struct sp_meminfo *meminfo) } /* The caller should hold mmap_sem to protect master (TBD) */ -static void sp_init_group_master_stat(int tgid, struct mm_struct *mm, +static void sp_init_group_master_stat(struct mm_struct *mm, struct sp_proc_stat *stat) { atomic64_set(&stat->alloc_nsize, 0); atomic64_set(&stat->alloc_hsize, 0); atomic64_set(&stat->k2u_size, 0); - stat->tgid = tgid; get_task_comm(stat->comm, current); } @@ -586,7 +584,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct master->count = 0; master->mm = mm; master->tgid = tsk->tgid; - sp_init_group_master_stat(tsk->tgid, mm, &master->instat); + sp_init_group_master_stat(mm, &master->instat); mm->sp_group_master = master; sp_add_group_master(master); @@ -3771,7 +3769,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, "PID", "COMM", "SP_ALLOC", "SP_K2U", "SP_RES", "Non-SP_RES", "Non-SP_Shm", "VIRT"); seq_printf(m, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", - proc_stat->tgid, proc_stat->comm, + master->tgid, proc_stat->comm, get_proc_alloc(proc_stat), get_proc_k2u(proc_stat), sp_res, non_sp_res, non_sp_shm, @@ -3985,7 +3983,7 @@ static int proc_usage_by_group(int id, void *p, void *data) list_for_each_entry(spg_node, &spg->procs, proc_node) { master = spg_node->master; mm = master->mm; - tgid = master->instat.tgid; + tgid = master->tgid; get_mm_rss_info(mm, &anon, &file, &shmem, &total_rss); @@ -4054,7 +4052,7 @@ static int proc_usage_show(struct seq_file *seq, void *offset) get_process_non_sp_res(total_rss, shmem, sp_res_nsize, &non_sp_res, &non_sp_shm); seq_printf(seq, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", - proc_stat->tgid, proc_stat->comm, + master->tgid, proc_stat->comm, get_proc_alloc(proc_stat), get_proc_k2u(proc_stat), sp_res, non_sp_res, non_sp_shm, @@ -4285,7 +4283,7 @@ void sp_group_post_exit(struct mm_struct *mm) if (alloc_size != 0 || k2u_size != 0) pr_info("process %s(%d) exits. It applied %ld aligned KB, k2u shared %ld aligned KB\n", - stat->comm, stat->tgid, + stat->comm, master->tgid, byte2kb(alloc_size), byte2kb(k2u_size)); } -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- comm is not memory usage information and should not be stored in sp_proc_stat. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index e11dc2769de5..701a7811c012 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -123,7 +123,6 @@ struct sp_meminfo { #ifndef __GENKSYMS__ /* per process memory usage statistics indexed by tgid */ struct sp_proc_stat { - char comm[TASK_COMM_LEN]; /* * alloc amount minus free amount, may be negative when freed by * another task in the same sp group. @@ -239,6 +238,7 @@ struct sp_group_master { struct sp_group *local; struct sp_proc_stat instat; struct list_head list_node; + char comm[TASK_COMM_LEN]; }; /* @@ -317,7 +317,6 @@ static void sp_init_group_master_stat(struct mm_struct *mm, atomic64_set(&stat->alloc_nsize, 0); atomic64_set(&stat->alloc_hsize, 0); atomic64_set(&stat->k2u_size, 0); - get_task_comm(stat->comm, current); } static unsigned long sp_mapping_type(struct sp_mapping *spm) @@ -584,6 +583,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct master->count = 0; master->mm = mm; master->tgid = tsk->tgid; + get_task_comm(master->comm, current); sp_init_group_master_stat(mm, &master->instat); mm->sp_group_master = master; sp_add_group_master(master); @@ -3769,7 +3769,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, "PID", "COMM", "SP_ALLOC", "SP_K2U", "SP_RES", "Non-SP_RES", "Non-SP_Shm", "VIRT"); seq_printf(m, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", - master->tgid, proc_stat->comm, + master->tgid, master->comm, get_proc_alloc(proc_stat), get_proc_k2u(proc_stat), sp_res, non_sp_res, non_sp_shm, @@ -4052,7 +4052,7 @@ static int proc_usage_show(struct seq_file *seq, void *offset) get_process_non_sp_res(total_rss, shmem, sp_res_nsize, &non_sp_res, &non_sp_shm); seq_printf(seq, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", - master->tgid, proc_stat->comm, + master->tgid, master->comm, get_proc_alloc(proc_stat), get_proc_k2u(proc_stat), sp_res, non_sp_res, non_sp_shm, @@ -4283,7 +4283,7 @@ void sp_group_post_exit(struct mm_struct *mm) if (alloc_size != 0 || k2u_size != 0) pr_info("process %s(%d) exits. It applied %ld aligned KB, k2u shared %ld aligned KB\n", - stat->comm, master->tgid, + master->comm, master->tgid, byte2kb(alloc_size), byte2kb(k2u_size)); } -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 106 ++++++++++++++++-------------------------------- 1 file changed, 36 insertions(+), 70 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 701a7811c012..fd109b452257 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -97,7 +97,7 @@ static LIST_HEAD(spm_dvpp_list); static DEFINE_MUTEX(spm_list_lock); /* for kthread buff_module_guard_work */ -static struct sp_proc_stat kthread_stat; +static struct sp_meminfo kthread_stat; #define SEQ_printf(m, x...) \ do { \ @@ -121,18 +121,6 @@ struct sp_meminfo { }; #ifndef __GENKSYMS__ -/* per process memory usage statistics indexed by tgid */ -struct sp_proc_stat { - /* - * alloc amount minus free amount, may be negative when freed by - * another task in the same sp group. - */ - atomic64_t alloc_size; - atomic64_t alloc_nsize; - atomic64_t alloc_hsize; - atomic64_t k2u_size; -}; - /* per process/sp-group memory usage statistics */ struct spg_proc_stat { int tgid; @@ -236,7 +224,7 @@ struct sp_group_master { * For example, sp_alloc non-share memory or k2task. */ struct sp_group *local; - struct sp_proc_stat instat; + struct sp_meminfo meminfo; struct list_head list_node; char comm[TASK_COMM_LEN]; }; @@ -310,15 +298,6 @@ static void meminfo_dec_k2u(unsigned long size, struct sp_meminfo *meminfo) atomic64_sub(size, &meminfo->k2u_size); } -/* The caller should hold mmap_sem to protect master (TBD) */ -static void sp_init_group_master_stat(struct mm_struct *mm, - struct sp_proc_stat *stat) -{ - atomic64_set(&stat->alloc_nsize, 0); - atomic64_set(&stat->alloc_hsize, 0); - atomic64_set(&stat->k2u_size, 0); -} - static unsigned long sp_mapping_type(struct sp_mapping *spm) { return spm->type; @@ -584,7 +563,7 @@ static int sp_init_group_master_locked(struct task_struct *tsk, struct mm_struct master->mm = mm; master->tgid = tsk->tgid; get_task_comm(master->comm, current); - sp_init_group_master_stat(mm, &master->instat); + meminfo_init(&master->meminfo); mm->sp_group_master = master; sp_add_group_master(master); @@ -637,40 +616,30 @@ static struct sp_group *sp_get_local_group(struct task_struct *tsk, struct mm_st static void update_mem_usage_alloc(unsigned long size, bool inc, bool is_hugepage, struct sp_group_node *spg_node) { - struct sp_proc_stat *proc_stat = &spg_node->master->instat; - if (inc) { - if (is_hugepage) { + if (is_hugepage) atomic64_add(size, &spg_node->instat.alloc_hsize); - atomic64_add(size, &proc_stat->alloc_hsize); - return; - } - atomic64_add(size, &spg_node->instat.alloc_nsize); - atomic64_add(size, &proc_stat->alloc_nsize); - return; - } - - if (is_hugepage) { - atomic64_sub(size, &spg_node->instat.alloc_hsize); - atomic64_sub(size, &proc_stat->alloc_hsize); - return; + else + atomic64_add(size, &spg_node->instat.alloc_nsize); + meminfo_inc_usage(size, is_hugepage, &spg_node->master->meminfo); + } else { + if (is_hugepage) + atomic64_sub(size, &spg_node->instat.alloc_hsize); + else + atomic64_sub(size, &spg_node->instat.alloc_nsize); + meminfo_dec_usage(size, is_hugepage, &spg_node->master->meminfo); } - atomic64_sub(size, &spg_node->instat.alloc_nsize); - atomic64_sub(size, &proc_stat->alloc_nsize); - return; } static void update_mem_usage_k2u(unsigned long size, bool inc, struct sp_group_node *spg_node) { - struct sp_proc_stat *proc_stat = &spg_node->master->instat; - if (inc) { atomic64_add(size, &spg_node->instat.k2u_size); - atomic64_add(size, &proc_stat->k2u_size); + meminfo_inc_k2u(size, &spg_node->master->meminfo); } else { atomic64_sub(size, &spg_node->instat.k2u_size); - atomic64_sub(size, &proc_stat->k2u_size); + meminfo_dec_k2u(size, &spg_node->master->meminfo); } } @@ -3662,15 +3631,15 @@ static void get_mm_rss_info(struct mm_struct *mm, unsigned long *anon, *total_rss = *anon + *file + *shmem; } -static long get_proc_k2u(struct sp_proc_stat *stat) +static long get_proc_k2u(struct sp_meminfo *meminfo) { - return byte2kb(atomic64_read(&stat->k2u_size)); + return byte2kb(atomic64_read(&meminfo->k2u_size)); } -static long get_proc_alloc(struct sp_proc_stat *stat) +static long get_proc_alloc(struct sp_meminfo *meminfo) { - return byte2kb(atomic64_read(&stat->alloc_nsize) + - atomic64_read(&stat->alloc_hsize)); + return byte2kb(atomic64_read(&meminfo->alloc_nsize) + + atomic64_read(&meminfo->alloc_hsize)); } static void get_process_sp_res(struct sp_group_master *master, @@ -3740,7 +3709,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, { struct mm_struct *mm; struct sp_group_master *master; - struct sp_proc_stat *proc_stat; + struct sp_meminfo *meminfo; struct sp_group_node *spg_node; unsigned long anon, file, shmem, total_rss; long sp_res, sp_res_nsize, non_sp_res, non_sp_shm; @@ -3759,7 +3728,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, goto out; get_mm_rss_info(mm, &anon, &file, &shmem, &total_rss); - proc_stat = &master->instat; + meminfo = &master->meminfo; get_process_sp_res(master, &sp_res, &sp_res_nsize); get_process_non_sp_res(total_rss, shmem, sp_res_nsize, &non_sp_res, &non_sp_shm); @@ -3770,8 +3739,8 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, "Non-SP_Shm", "VIRT"); seq_printf(m, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, - get_proc_alloc(proc_stat), - get_proc_k2u(proc_stat), + get_proc_alloc(meminfo), + get_proc_k2u(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(mm->total_vm)); @@ -4034,7 +4003,7 @@ static int proc_usage_show(struct seq_file *seq, void *offset) struct sp_group_master *master = NULL; unsigned long anon, file, shmem, total_rss; long sp_res, sp_res_nsize, non_sp_res, non_sp_shm; - struct sp_proc_stat *proc_stat; + struct sp_meminfo *meminfo; if (!should_show_statistics()) return -EPERM; @@ -4046,15 +4015,15 @@ static int proc_usage_show(struct seq_file *seq, void *offset) down_read(&sp_group_sem); mutex_lock(&master_list_lock); list_for_each_entry(master, &master_list, list_node) { - proc_stat = &master->instat; + meminfo = &master->meminfo; get_mm_rss_info(master->mm, &anon, &file, &shmem, &total_rss); get_process_sp_res(master, &sp_res, &sp_res_nsize); get_process_non_sp_res(total_rss, shmem, sp_res_nsize, &non_sp_res, &non_sp_shm); seq_printf(seq, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, - get_proc_alloc(proc_stat), - get_proc_k2u(proc_stat), + get_proc_alloc(meminfo), + get_proc_k2u(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(master->mm->total_vm)); } @@ -4251,7 +4220,7 @@ int sp_group_exit(void) void sp_group_post_exit(struct mm_struct *mm) { - struct sp_proc_stat *stat; + struct sp_meminfo *meminfo; long alloc_size, k2u_size; /* lockless visit */ struct sp_group_master *master = mm->sp_group_master; @@ -4276,16 +4245,13 @@ void sp_group_post_exit(struct mm_struct *mm) * A process not in an sp group doesn't need to print because there * wont't be any memory which is not freed. */ - stat = &master->instat; - if (stat) { - alloc_size = atomic64_read(&stat->alloc_nsize) + atomic64_read(&stat->alloc_hsize); - k2u_size = atomic64_read(&stat->k2u_size); - - if (alloc_size != 0 || k2u_size != 0) - pr_info("process %s(%d) exits. It applied %ld aligned KB, k2u shared %ld aligned KB\n", - master->comm, master->tgid, - byte2kb(alloc_size), byte2kb(k2u_size)); - } + meminfo = &master->meminfo; + alloc_size = atomic64_read(&meminfo->alloc_nsize) + atomic64_read(&meminfo->alloc_hsize); + k2u_size = atomic64_read(&meminfo->k2u_size); + if (alloc_size != 0 || k2u_size != 0) + pr_info("process %s(%d) exits. It applied %ld aligned KB, k2u shared %ld aligned KB\n", + master->comm, master->tgid, + byte2kb(alloc_size), byte2kb(k2u_size)); down_write(&sp_group_sem); list_for_each_entry_safe(spg_node, tmp, &master->node_list, group_node) { -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index fd109b452257..54994d38ff5b 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -123,8 +123,6 @@ struct sp_meminfo { #ifndef __GENKSYMS__ /* per process/sp-group memory usage statistics */ struct spg_proc_stat { - int tgid; - int spg_id; /* 0 for non-group data, such as k2u_task */ /* * alloc amount minus free amount, may be negative when freed by * another task in the same sp group. @@ -643,10 +641,8 @@ static void update_mem_usage_k2u(unsigned long size, bool inc, } } -static void sp_init_spg_proc_stat(struct spg_proc_stat *stat, int spg_id) +static void sp_init_spg_proc_stat(struct spg_proc_stat *stat) { - stat->tgid = current->tgid; - stat->spg_id = spg_id; atomic64_set(&stat->alloc_nsize, 0); atomic64_set(&stat->alloc_hsize, 0); atomic64_set(&stat->k2u_size, 0); @@ -1269,7 +1265,7 @@ static struct sp_group_node *create_spg_node(struct mm_struct *mm, spg_node->spg = spg; spg_node->master = master; spg_node->prot = prot; - sp_init_spg_proc_stat(&spg_node->instat, spg->id); + sp_init_spg_proc_stat(&spg_node->instat); list_add_tail(&spg_node->group_node, &master->node_list); master->count++; -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 47 ++++++++++++++--------------------------------- 1 file changed, 14 insertions(+), 33 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 54994d38ff5b..f823486d5343 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -121,17 +121,6 @@ struct sp_meminfo { }; #ifndef __GENKSYMS__ -/* per process/sp-group memory usage statistics */ -struct spg_proc_stat { - /* - * alloc amount minus free amount, may be negative when freed by - * another task in the same sp group. - */ - atomic64_t alloc_size; - atomic64_t alloc_nsize; - atomic64_t alloc_hsize; - atomic64_t k2u_size; -}; enum sp_mapping_type { SP_MAPPING_START, @@ -241,7 +230,12 @@ struct sp_group_node { struct sp_group_master *master; struct sp_group *spg; unsigned long prot; - struct spg_proc_stat instat; + + /* + * alloc amount minus free amount, may be negative when freed by + * another task in the same sp group. + */ + struct sp_meminfo meminfo; }; #endif @@ -615,16 +609,10 @@ static void update_mem_usage_alloc(unsigned long size, bool inc, bool is_hugepage, struct sp_group_node *spg_node) { if (inc) { - if (is_hugepage) - atomic64_add(size, &spg_node->instat.alloc_hsize); - else - atomic64_add(size, &spg_node->instat.alloc_nsize); + meminfo_inc_usage(size, is_hugepage, &spg_node->meminfo); meminfo_inc_usage(size, is_hugepage, &spg_node->master->meminfo); } else { - if (is_hugepage) - atomic64_sub(size, &spg_node->instat.alloc_hsize); - else - atomic64_sub(size, &spg_node->instat.alloc_nsize); + meminfo_dec_usage(size, is_hugepage, &spg_node->meminfo); meminfo_dec_usage(size, is_hugepage, &spg_node->master->meminfo); } } @@ -633,21 +621,14 @@ static void update_mem_usage_k2u(unsigned long size, bool inc, struct sp_group_node *spg_node) { if (inc) { - atomic64_add(size, &spg_node->instat.k2u_size); + meminfo_inc_k2u(size, &spg_node->meminfo); meminfo_inc_k2u(size, &spg_node->master->meminfo); } else { - atomic64_sub(size, &spg_node->instat.k2u_size); + meminfo_dec_k2u(size, &spg_node->meminfo); meminfo_dec_k2u(size, &spg_node->master->meminfo); } } -static void sp_init_spg_proc_stat(struct spg_proc_stat *stat) -{ - atomic64_set(&stat->alloc_nsize, 0); - atomic64_set(&stat->alloc_hsize, 0); - atomic64_set(&stat->k2u_size, 0); -} - /* statistics of all sp area, protected by sp_area_lock */ struct sp_spa_stat { unsigned int total_num; @@ -1265,7 +1246,7 @@ static struct sp_group_node *create_spg_node(struct mm_struct *mm, spg_node->spg = spg; spg_node->master = master; spg_node->prot = prot; - sp_init_spg_proc_stat(&spg_node->instat); + meminfo_init(&spg_node->meminfo); list_add_tail(&spg_node->group_node, &master->node_list); master->count++; @@ -3681,13 +3662,13 @@ static void get_process_non_sp_res(unsigned long total_rss, unsigned long shmem, static long get_spg_proc_alloc(struct sp_group_node *spg_node) { - return byte2kb(atomic64_read(&spg_node->instat.alloc_nsize) + - atomic64_read(&spg_node->instat.alloc_hsize)); + return byte2kb(atomic64_read(&spg_node->meminfo.alloc_nsize) + + atomic64_read(&spg_node->meminfo.alloc_hsize)); } static long get_spg_proc_k2u(struct sp_group_node *spg_node) { - return byte2kb(atomic64_read(&spg_node->instat.k2u_size)); + return byte2kb(atomic64_read(&spg_node->meminfo.k2u_size)); } static void print_process_prot(struct seq_file *seq, unsigned long prot) -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Add meminfo_alloc_sum_byKB and meminfo_alloc_sum, and delete duplicate or similar code. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 46 +++++++++++++++++++--------------------------- 1 file changed, 19 insertions(+), 27 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index f823486d5343..beff18790354 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -290,6 +290,17 @@ static void meminfo_dec_k2u(unsigned long size, struct sp_meminfo *meminfo) atomic64_sub(size, &meminfo->k2u_size); } +static inline long meminfo_alloc_sum(struct sp_meminfo *meminfo) +{ + return atomic64_read(&meminfo->alloc_nsize) + + atomic64_read(&meminfo->alloc_hsize); +} + +static inline long meminfo_alloc_sum_byKB(struct sp_meminfo *meminfo) +{ + return byte2kb(meminfo_alloc_sum(meminfo)); +} + static unsigned long sp_mapping_type(struct sp_mapping *spm) { return spm->type; @@ -3613,12 +3624,6 @@ static long get_proc_k2u(struct sp_meminfo *meminfo) return byte2kb(atomic64_read(&meminfo->k2u_size)); } -static long get_proc_alloc(struct sp_meminfo *meminfo) -{ - return byte2kb(atomic64_read(&meminfo->alloc_nsize) + - atomic64_read(&meminfo->alloc_hsize)); -} - static void get_process_sp_res(struct sp_group_master *master, long *sp_res_out, long *sp_res_nsize_out) { @@ -3630,18 +3635,11 @@ static void get_process_sp_res(struct sp_group_master *master, list_for_each_entry(spg_node, &master->node_list, group_node) { spg = spg_node->spg; - *sp_res_out += byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)); - *sp_res_out += byte2kb(atomic64_read(&spg->meminfo.alloc_hsize)); + *sp_res_out += meminfo_alloc_sum_byKB(&spg->meminfo); *sp_res_nsize_out += byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)); } } -static long get_sp_res_by_spg_proc(struct sp_group_node *spg_node) -{ - return byte2kb(atomic64_read(&spg_node->spg->meminfo.alloc_nsize) + - atomic64_read(&spg_node->spg->meminfo.alloc_hsize)); -} - /* * Statistics of RSS has a maximum 64 pages deviation (256KB). * Please check_sync_rss_stat(). @@ -3660,12 +3658,6 @@ static void get_process_non_sp_res(unsigned long total_rss, unsigned long shmem, *non_sp_shm_out = non_sp_shm; } -static long get_spg_proc_alloc(struct sp_group_node *spg_node) -{ - return byte2kb(atomic64_read(&spg_node->meminfo.alloc_nsize) + - atomic64_read(&spg_node->meminfo.alloc_hsize)); -} - static long get_spg_proc_k2u(struct sp_group_node *spg_node) { return byte2kb(atomic64_read(&spg_node->meminfo.k2u_size)); @@ -3716,7 +3708,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, "Non-SP_Shm", "VIRT"); seq_printf(m, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, - get_proc_alloc(meminfo), + meminfo_alloc_sum_byKB(meminfo), get_proc_k2u(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(mm->total_vm)); @@ -3728,9 +3720,9 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, list_for_each_entry(spg_node, &master->node_list, group_node) { seq_printf(m, "%-8d %-9ld %-9ld %-9ld ", spg_node->spg->id, - get_spg_proc_alloc(spg_node), + meminfo_alloc_sum_byKB(&spg_node->meminfo), get_spg_proc_k2u(spg_node), - get_sp_res_by_spg_proc(spg_node)); + meminfo_alloc_sum_byKB(&spg_node->spg->meminfo)); print_process_prot(m, spg_node->prot); seq_putc(m, '\n'); } @@ -3936,9 +3928,9 @@ static int proc_usage_by_group(int id, void *p, void *data) seq_printf(seq, "%-8d ", tgid); seq_printf(seq, "%-8d ", id); seq_printf(seq, "%-9ld %-9ld %-9ld %-8ld %-7ld %-7ld ", - get_spg_proc_alloc(spg_node), + meminfo_alloc_sum_byKB(&spg_node->meminfo), get_spg_proc_k2u(spg_node), - get_sp_res_by_spg_proc(spg_node), + meminfo_alloc_sum_byKB(&spg_node->spg->meminfo), page2kb(mm->total_vm), page2kb(total_rss), page2kb(shmem)); print_process_prot(seq, spg_node->prot); @@ -3999,7 +3991,7 @@ static int proc_usage_show(struct seq_file *seq, void *offset) &non_sp_res, &non_sp_shm); seq_printf(seq, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, - get_proc_alloc(meminfo), + meminfo_alloc_sum_byKB(meminfo), get_proc_k2u(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(master->mm->total_vm)); @@ -4223,7 +4215,7 @@ void sp_group_post_exit(struct mm_struct *mm) * wont't be any memory which is not freed. */ meminfo = &master->meminfo; - alloc_size = atomic64_read(&meminfo->alloc_nsize) + atomic64_read(&meminfo->alloc_hsize); + alloc_size = meminfo_alloc_sum(meminfo); k2u_size = atomic64_read(&meminfo->k2u_size); if (alloc_size != 0 || k2u_size != 0) pr_info("process %s(%d) exits. It applied %ld aligned KB, k2u shared %ld aligned KB\n", -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Add meminfo_k2u_size and delete duplicate or similar code. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 23 +++++++++-------------- 1 file changed, 9 insertions(+), 14 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index beff18790354..5337fcda8ba3 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -301,6 +301,11 @@ static inline long meminfo_alloc_sum_byKB(struct sp_meminfo *meminfo) return byte2kb(meminfo_alloc_sum(meminfo)); } +static inline long meminfo_k2u_size(struct sp_meminfo *meminfo) +{ + return byte2kb(atomic64_read(&meminfo->k2u_size)); +} + static unsigned long sp_mapping_type(struct sp_mapping *spm) { return spm->type; @@ -3619,11 +3624,6 @@ static void get_mm_rss_info(struct mm_struct *mm, unsigned long *anon, *total_rss = *anon + *file + *shmem; } -static long get_proc_k2u(struct sp_meminfo *meminfo) -{ - return byte2kb(atomic64_read(&meminfo->k2u_size)); -} - static void get_process_sp_res(struct sp_group_master *master, long *sp_res_out, long *sp_res_nsize_out) { @@ -3658,11 +3658,6 @@ static void get_process_non_sp_res(unsigned long total_rss, unsigned long shmem, *non_sp_shm_out = non_sp_shm; } -static long get_spg_proc_k2u(struct sp_group_node *spg_node) -{ - return byte2kb(atomic64_read(&spg_node->meminfo.k2u_size)); -} - static void print_process_prot(struct seq_file *seq, unsigned long prot) { if (prot == PROT_READ) @@ -3709,7 +3704,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, seq_printf(m, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, meminfo_alloc_sum_byKB(meminfo), - get_proc_k2u(meminfo), + meminfo_k2u_size(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(mm->total_vm)); @@ -3721,7 +3716,7 @@ int proc_sp_group_state(struct seq_file *m, struct pid_namespace *ns, seq_printf(m, "%-8d %-9ld %-9ld %-9ld ", spg_node->spg->id, meminfo_alloc_sum_byKB(&spg_node->meminfo), - get_spg_proc_k2u(spg_node), + meminfo_k2u_size(&spg_node->meminfo), meminfo_alloc_sum_byKB(&spg_node->spg->meminfo)); print_process_prot(m, spg_node->prot); seq_putc(m, '\n'); @@ -3929,7 +3924,7 @@ static int proc_usage_by_group(int id, void *p, void *data) seq_printf(seq, "%-8d ", id); seq_printf(seq, "%-9ld %-9ld %-9ld %-8ld %-7ld %-7ld ", meminfo_alloc_sum_byKB(&spg_node->meminfo), - get_spg_proc_k2u(spg_node), + meminfo_k2u_size(&spg_node->meminfo), meminfo_alloc_sum_byKB(&spg_node->spg->meminfo), page2kb(mm->total_vm), page2kb(total_rss), page2kb(shmem)); @@ -3992,7 +3987,7 @@ static int proc_usage_show(struct seq_file *seq, void *offset) seq_printf(seq, "%-8d %-16s %-9ld %-9ld %-9ld %-10ld %-10ld %-8ld\n", master->tgid, master->comm, meminfo_alloc_sum_byKB(meminfo), - get_proc_k2u(meminfo), + meminfo_k2u_size(meminfo), sp_res, non_sp_res, non_sp_shm, page2kb(master->mm->total_vm)); } -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 5337fcda8ba3..51625bf11b87 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -96,9 +96,6 @@ static LIST_HEAD(spm_dvpp_list); /* mutex to protect insert/delete ops from master_list */ static DEFINE_MUTEX(spm_list_lock); -/* for kthread buff_module_guard_work */ -static struct sp_meminfo kthread_stat; - #define SEQ_printf(m, x...) \ do { \ if (m) \ @@ -2224,9 +2221,7 @@ int mg_sp_free(unsigned long addr, int id) sp_free_unmap_fallocate(fc.spa); - if (current->mm == NULL) - atomic64_sub(fc.spa->real_size, &kthread_stat.alloc_size); - else + if (current->mm != NULL) sp_update_process_stat(current, false, fc.spa); __sp_area_drop(fc.spa); /* match get_sp_area in sp_free_get_spa */ @@ -3316,9 +3311,7 @@ static int sp_unshare_uva(unsigned long uva, unsigned long size, int group_id) __sp_free(spa->spg, spa->va_start, spa->real_size, NULL); up_read(&spa->spg->rw_lock); - if (current->mm == NULL) - atomic64_sub(spa->real_size, &kthread_stat.k2u_size); - else + if (current->mm != NULL) sp_update_process_stat(current, false, spa); out_clr_flag: @@ -3949,11 +3942,6 @@ static int proc_group_usage_show(struct seq_file *seq, void *offset) seq_printf(seq, "%-8s %-8s %-9s %-9s %-9s %-8s %-7s %-7s %-4s\n", "PID", "Group_ID", "SP_ALLOC", "SP_K2U", "SP_RES", "VIRT", "RES", "Shm", "PROT"); - /* print kthread buff_module_guard_work */ - seq_printf(seq, "%-8s %-8s %-9lld %-9lld\n", - "guard", "-", - byte2kb(atomic64_read(&kthread_stat.alloc_size)), - byte2kb(atomic64_read(&kthread_stat.k2u_size))); down_read(&sp_group_sem); idr_for_each(&sp_group_idr, proc_usage_by_group, seq); -- 2.17.1
From: Xu Qiang <xuqiang36@huawei.com> Offering: HULK hulk inclusion category: other bugzilla: https://gitee.com/openeuler/kernel/issues/I6ET9W ---------------------------------------------- alloc_size can be obtained by adding alloc_nsize to alloc_hsize, and size can be obtained by adding alloc_nsize to alloc_hsize and k2u_size. Therefore, alloc_size and size are redundant and can be deleted. Signed-off-by: Xu Qiang <xuqiang36@huawei.com> --- mm/share_pool.c | 23 ++++++++++------------- 1 file changed, 10 insertions(+), 13 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index 51625bf11b87..eed1a454f3bd 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -105,14 +105,10 @@ do { \ } while (0) struct sp_meminfo { - /* total size from sp_alloc and k2u */ - atomic64_t size; /* not huge page size from sp_alloc */ atomic64_t alloc_nsize; /* huge page size from sp_alloc */ atomic64_t alloc_hsize; - /* total size from sp_alloc */ - atomic64_t alloc_size; /* total size from sp_k2u */ atomic64_t k2u_size; }; @@ -257,8 +253,6 @@ static void meminfo_init(struct sp_meminfo *meminfo) static void meminfo_inc_usage(unsigned long size, bool huge, struct sp_meminfo *meminfo) { - atomic64_add(size, &meminfo->size); - atomic64_add(size, &meminfo->alloc_size); if (huge) atomic64_add(size, &meminfo->alloc_hsize); else @@ -267,8 +261,6 @@ static void meminfo_inc_usage(unsigned long size, bool huge, struct sp_meminfo * static void meminfo_dec_usage(unsigned long size, bool huge, struct sp_meminfo *meminfo) { - atomic64_sub(size, &meminfo->size); - atomic64_sub(size, &meminfo->alloc_size); if (huge) atomic64_sub(size, &meminfo->alloc_hsize); else @@ -277,13 +269,11 @@ static void meminfo_dec_usage(unsigned long size, bool huge, struct sp_meminfo * static void meminfo_inc_k2u(unsigned long size, struct sp_meminfo *meminfo) { - atomic64_add(size, &meminfo->size); atomic64_add(size, &meminfo->k2u_size); } static void meminfo_dec_k2u(unsigned long size, struct sp_meminfo *meminfo) { - atomic64_sub(size, &meminfo->size); atomic64_sub(size, &meminfo->k2u_size); } @@ -303,6 +293,13 @@ static inline long meminfo_k2u_size(struct sp_meminfo *meminfo) return byte2kb(atomic64_read(&meminfo->k2u_size)); } +static inline long long meminfo_total_size(struct sp_meminfo *meminfo) +{ + return atomic64_read(&meminfo->alloc_nsize) + + atomic64_read(&meminfo->alloc_hsize) + + atomic64_read(&meminfo->k2u_size); +} + static unsigned long sp_mapping_type(struct sp_mapping *spm) { return spm->type; @@ -3841,10 +3838,10 @@ static int spg_info_show(int id, void *p, void *data) SEQ_printf(seq, "Group %6d ", id); down_read(&spg->rw_lock); - SEQ_printf(seq, "size: %lld KB, spa num: %d, total alloc: %lld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", - byte2kb(atomic64_read(&spg->meminfo.size)), + SEQ_printf(seq, "size: %lld KB, spa num: %d, total alloc: %ld KB, normal alloc: %lld KB, huge alloc: %lld KB\n", + byte2kb(meminfo_total_size(&spg->meminfo)), atomic_read(&spg->spa_num), - byte2kb(atomic64_read(&spg->meminfo.alloc_size)), + meminfo_alloc_sum_byKB(&spg->meminfo), byte2kb(atomic64_read(&spg->meminfo.alloc_nsize)), byte2kb(atomic64_read(&spg->meminfo.alloc_hsize))); up_read(&spg->rw_lock); -- 2.17.1
From: Chen Jun <chenjun102@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- Extract group_add_task from duplicate code. Signed-off-by: Chen Jun <chenjun102@huawei.com> --- mm/share_pool.c | 38 +++++++++++++++++++++----------------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index eed1a454f3bd..f0e3cd26076e 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -502,12 +502,14 @@ static struct sp_mapping *sp_mapping_find(struct sp_group *spg, static struct sp_group *create_spg(int spg_id, unsigned long flag); static void free_new_spg_id(bool new, int spg_id); static void free_sp_group_locked(struct sp_group *spg); -static int local_group_add_task(struct mm_struct *mm, struct sp_group *spg); +static struct sp_group_node *group_add_task(struct mm_struct *mm, struct sp_group *spg, + unsigned long prot); static int init_local_group(struct mm_struct *mm) { int spg_id, ret; struct sp_group *spg; struct sp_mapping *spm; + struct sp_group_node *spg_node; struct sp_group_master *master = mm->sp_group_master; spg_id = ida_alloc_range(&sp_group_id_ida, SPG_ID_LOCAL_MIN, @@ -533,10 +535,13 @@ static int init_local_group(struct mm_struct *mm) sp_mapping_attach(master->local, sp_mapping_normal); sp_mapping_attach(master->local, sp_mapping_ro); - ret = local_group_add_task(mm, spg); - if (ret < 0) + spg_node = group_add_task(mm, spg, PROT_READ | PROT_WRITE); + if (IS_ERR(spg_node)) { /* The spm would be released while destroying the spg */ + ret = PTR_ERR(spg_node); goto free_spg; + } + mmget(mm); return 0; @@ -1297,18 +1302,24 @@ static void free_spg_node(struct mm_struct *mm, struct sp_group *spg, kfree(spg_node); } -static int local_group_add_task(struct mm_struct *mm, struct sp_group *spg) +/* the caller must hold sp_group_sem and down_write(&spg->rw_lock) in order */ +static struct sp_group_node *group_add_task(struct mm_struct *mm, struct sp_group *spg, + unsigned long prot) { struct sp_group_node *node; + int ret; - node = create_spg_node(mm, PROT_READ | PROT_WRITE, spg); + node = create_spg_node(mm, prot, spg); if (IS_ERR(node)) - return PTR_ERR(node); + return node; - insert_spg_node(spg, node); - mmget(mm); + ret = insert_spg_node(spg, node); + if (unlikely(ret)) { + free_spg_node(mm, spg, node); + return ERR_PTR(ret); + } - return 0; + return node; } /** @@ -1451,19 +1462,13 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) goto out_drop_group; } - node = create_spg_node(mm, prot, spg); + node = group_add_task(mm, spg, prot); if (unlikely(IS_ERR(node))) { up_write(&spg->rw_lock); ret = PTR_ERR(node); goto out_drop_group; } - ret = insert_spg_node(spg, node); - if (unlikely(ret)) { - up_write(&spg->rw_lock); - goto out_drop_spg_node; - } - /* * create mappings of existing shared memory segments into this * new process' page table. @@ -1544,7 +1549,6 @@ int mg_sp_group_add_task(int tgid, unsigned long prot, int spg_id) delete_spg_node(spg, node); up_write(&spg->rw_lock); -out_drop_spg_node: if (unlikely(ret)) free_spg_node(mm, spg, node); /* -- 2.17.1
From: Chen Jun <chenjun102@huawei.com> Offering: HULK hulk inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I650K6 -------------------------------- delete_spg_node can be use to replace the code snippets as follows: list_del(&node->proc_node); spg->proc_num--; Signed-off-by: Chen Jun <chenjun102@huawei.com> --- mm/share_pool.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index f0e3cd26076e..55f3a00971a5 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -1666,8 +1666,7 @@ int mg_sp_group_del_task(int tgid, int spg_id) if (list_is_singular(&spg->procs)) is_alive = spg->is_alive = false; - spg->proc_num--; - list_del(&spg_node->proc_node); + delete_spg_node(spg, spg_node); sp_group_put(spg); up_write(&spg->rw_lock); if (!is_alive) @@ -4151,8 +4150,7 @@ int sp_group_exit(void) /* a dead group should NOT be reactive again */ if (spg_valid(spg) && list_is_singular(&spg->procs)) is_alive = spg->is_alive = false; - spg->proc_num--; - list_del(&spg_node->proc_node); + delete_spg_node(spg, spg_node); up_write(&spg->rw_lock); if (!is_alive) -- 2.17.1
From: Zhou Guanghui <zhouguanghui1@huawei.com> Offering: HULK ascend inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/I6A3YT ------------------------------------------------- Charge buddy hugepage to memcg when kmemcg is disabled. If kmemcg is enabled, we can also use kmemcg to charge buddy hugepages. Signed-off-by: Zhou Guanghui <zhouguanghui1@huawei.com> --- mm/hugetlb.c | 2 ++ mm/share_pool.c | 14 ++++++++++++-- mm/share_pool_internal.h | 22 ++++++++++++++++++++++ 3 files changed, 36 insertions(+), 2 deletions(-) create mode 100644 mm/share_pool_internal.h diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 9bfb781fafd3..acf69b883c12 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -46,6 +46,7 @@ #include <linux/dynamic_hugetlb.h> #include "internal.h" #include "hugetlb_vmemmap.h" +#include "share_pool_internal.h" int hugetlb_max_hstate __read_mostly; unsigned int default_hstate_idx; @@ -1625,6 +1626,7 @@ void free_huge_page(struct page *page) h->resv_huge_pages++; if (HPageTemporary(page)) { + sp_memcg_uncharge_hpage(page); remove_hugetlb_page(h, page, false); spin_unlock_irqrestore(&hugetlb_lock, flags); update_and_free_page(h, page, true); diff --git a/mm/share_pool.c b/mm/share_pool.c index 55f3a00971a5..bf5e75e36c87 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -4031,6 +4031,7 @@ vm_fault_t sharepool_no_page(struct mm_struct *mm, int err; int node_id; struct sp_area *spa; + bool charge_hpage; spa = vma->vm_private_data; if (!spa) { @@ -4046,12 +4047,15 @@ vm_fault_t sharepool_no_page(struct mm_struct *mm, if (idx >= size) goto out; + charge_hpage = false; page = alloc_huge_page(vma, haddr, 0); if (IS_ERR(page)) { page = hugetlb_alloc_hugepage(node_id, HUGETLB_ALLOC_BUDDY | HUGETLB_ALLOC_NORECLAIM); if (!page) page = ERR_PTR(-ENOMEM); + else if (!PageMemcgKmem(page)) + charge_hpage = true; } if (IS_ERR(page)) { ptl = huge_pte_lock(h, mm, ptep); @@ -4064,6 +4068,13 @@ vm_fault_t sharepool_no_page(struct mm_struct *mm, ret = vmf_error(PTR_ERR(page)); goto out; } + + if (charge_hpage && mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL)) { + put_page(page); + ret = vmf_error(-ENOMEM); + goto out; + } + __SetPageUptodate(page); new_page = true; @@ -4096,9 +4107,8 @@ vm_fault_t sharepool_no_page(struct mm_struct *mm, spin_unlock(ptl); - if (new_page) { + if (new_page) SetPagePrivate(&page[1]); - } unlock_page(page); out: diff --git a/mm/share_pool_internal.h b/mm/share_pool_internal.h new file mode 100644 index 000000000000..222993618dca --- /dev/null +++ b/mm/share_pool_internal.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef SHARE_POOL_INTERNAL_H +#define SHARE_POOL_INTERNAL_H + +#include <linux/hugetlb.h> +#include <linux/memcontrol.h> + +#ifdef CONFIG_ASCEND_SHARE_POOL +static inline void sp_memcg_uncharge_hpage(struct page *page) +{ + if (!sp_is_enabled()) + return; + + mem_cgroup_uncharge(page); +} +#else +static inline void sp_memcg_uncharge_hpage(struct page *page) +{ +} +#endif + +#endif -- 2.17.1
From: Guo Mengqi <guomengqi3@huawei.com> Offering: HULK hulk inclusion category: cleanup bugzilla: https://gitee.com/openeuler/kernel/issues/I6D0N5 ------------------------------- Add static modifier to find_spg_node_by_spg(), for this function is designed as an internal helper function, should only be used in share_pool.c scope. Also add static modifier to spa_overview_show and spg_overview_show. Signed-off-by: Guo Mengqi <guomengqi3@huawei.com> --- mm/share_pool.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/mm/share_pool.c b/mm/share_pool.c index bf5e75e36c87..ce4837da8a9a 100644 --- a/mm/share_pool.c +++ b/mm/share_pool.c @@ -114,7 +114,6 @@ struct sp_meminfo { }; #ifndef __GENKSYMS__ - enum sp_mapping_type { SP_MAPPING_START, SP_MAPPING_DVPP = SP_MAPPING_START, @@ -465,11 +464,13 @@ static int sp_mapping_group_setup(struct mm_struct *mm, struct sp_group *spg) if (is_mapping_empty(local_dvpp_mapping)) { sp_mapping_merge(spg_dvpp_mapping, local_dvpp_mapping); if (is_conflict) - pr_warn_ratelimited("task address space conflict, spg_id=%d\n", spg->id); + pr_warn_ratelimited("task address space conflict, spg_id=%d\n", + spg->id); } else if (is_mapping_empty(spg_dvpp_mapping)) { sp_mapping_merge(local_dvpp_mapping, spg_dvpp_mapping); if (is_conflict) - pr_warn_ratelimited("group address space conflict, spg_id=%d\n", spg->id); + pr_warn_ratelimited("group address space conflict, spg_id=%d\n", + spg->id); } else { pr_info_ratelimited("Duplicate address space, id=%d\n", spg->id); return -EINVAL; @@ -823,7 +824,7 @@ static void update_mem_usage(unsigned long size, bool inc, bool is_hugepage, } } -struct sp_group_node *find_spg_node_by_spg(struct mm_struct *mm, +static struct sp_group_node *find_spg_node_by_spg(struct mm_struct *mm, struct sp_group *spg) { struct sp_group_node *spg_node; @@ -2443,7 +2444,6 @@ static int sp_alloc_populate(struct mm_struct *mm, struct sp_area *spa, * page fault later on, and more importantly sp_make_share_u2k() * depends on this feature (and MAP_LOCKED) to work correctly. */ - return do_mm_populate(mm, spa->va_start, ac->populate, 0); } @@ -2464,7 +2464,6 @@ static int __sp_alloc_mmap_populate(struct mm_struct *mm, struct sp_area *spa, int ret; ret = sp_alloc_mmap(mm, spa, spg_node, ac); - if (ret < 0) return ret; @@ -2486,6 +2485,7 @@ static int __sp_alloc_mmap_populate(struct mm_struct *mm, struct sp_area *spa, pr_warn_ratelimited("allocation failed due to mm populate failed(potential no enough memory when -12): %d\n", ret); } + return ret; } @@ -2538,8 +2538,7 @@ static int sp_alloc_mmap_populate(struct sp_area *spa, sp_fallocate(spa); /* if hugepage allocation fails, this will transfer to normal page - * and try again. (only if SP_HUGEPAGE_ONLY is not flagged - */ + * and try again. (only if SP_HUGEPAGE_ONLY is not flagged */ sp_alloc_fallback(spa, ac); return mmap_ret; @@ -3796,7 +3795,7 @@ static void spa_dvpp_stat_show(struct seq_file *seq) } -void spa_overview_show(struct seq_file *seq) +static void spa_overview_show(struct seq_file *seq) { unsigned int total_num, alloc_num, k2u_task_num, k2u_spg_num; unsigned long total_size, alloc_size, k2u_task_size, k2u_spg_size; @@ -3852,7 +3851,7 @@ static int spg_info_show(int id, void *p, void *data) return 0; } -void spg_overview_show(struct seq_file *seq) +static void spg_overview_show(struct seq_file *seq) { if (!sp_is_enabled()) return; @@ -4121,7 +4120,7 @@ vm_fault_t sharepool_no_page(struct mm_struct *mm, goto out; } -/* +/** * The caller must ensure that this function is called * when the last thread in the thread group exits. */ -- 2.17.1
-
Zhang Zekun