From: "Rafael J. Wysocki" rafael.j.wysocki@intel.com

mainline inclusion from mainline-v5.19-rc commit cf6ba0750a22a54f5101986401271429995cc4a0 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I67QNJ CVE: NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...

--------------------------------------------

Introduce a wrapper around device_for_each_child() to iterate over the children of a given ACPI device object.

This function will be used in subsequent change sets.

Signed-off-by: Rafael J. Wysocki rafael.j.wysocki@intel.com Reviewed-by: Mika Westerberg mika.westerberg@linux.intel.com Signed-off-by: Zhang Zekun zhangzekun11@huawei.com --- drivers/acpi/bus.c | 6 ++++++ include/acpi/acpi_bus.h | 2 ++ 2 files changed, 8 insertions(+)

diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c index 4ee50f4f6f01..d8a6b32a17ad 100644 --- a/drivers/acpi/bus.c +++ b/drivers/acpi/bus.c @@ -963,6 +963,12 @@ struct bus_type acpi_bus_type = { .uevent = acpi_device_uevent, };

+int acpi_dev_for_each_child(struct acpi_device *adev, + int (*fn)(struct device *, void *), void *data) +{ + return device_for_each_child(&adev->dev, data, fn); +} + /* -------------------------------------------------------------------------- Initialization/Cleanup -------------------------------------------------------------------------- */ diff --git a/include/acpi/acpi_bus.h b/include/acpi/acpi_bus.h index b5ff52052db3..d16a181d8a0e 100644 --- a/include/acpi/acpi_bus.h +++ b/include/acpi/acpi_bus.h @@ -494,6 +494,8 @@ void acpi_bus_detach_private_data(acpi_handle); extern int acpi_notifier_call_chain(struct acpi_device *, u32, u32); extern int register_acpi_notifier(struct notifier_block *); extern int unregister_acpi_notifier(struct notifier_block *); +int acpi_dev_for_each_child(struct acpi_device *adev, + int (*fn)(struct device *, void *), void *data);

/* * External Functions

From: "Rafael J. Wysocki" rafael.j.wysocki@intel.com

mainline inclusion from mainline-v6.0-rc1 commit f8128c390e58928b16f197416d417cfa4c65f610 category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I67QNJ CVE: NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...

----------------------------------------------

Some pieces of modular code can benefit from using acpi_dev_for_each_child(), so export it to modules.

Signed-off-by: Rafael J. Wysocki rafael.j.wysocki@intel.com Reviewed-by: Andy Shevchenko andriy.shevchenko@linux.intel.com Signed-off-by: Zhang Zekun zhangzekun11@huawei.com --- drivers/acpi/bus.c | 1 + 1 file changed, 1 insertion(+)

diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c index de8185585f46..6ad227006283 100644 --- a/drivers/acpi/bus.c +++ b/drivers/acpi/bus.c @@ -988,6 +988,7 @@ int acpi_dev_for_each_child(struct acpi_device *adev,

return device_for_each_child(&adev->dev, &adwc, acpi_dev_for_one_check); } +EXPORT_SYMBOL_GPL(acpi_dev_for_each_child);

/* -------------------------------------------------------------------------- Initialization/Cleanup

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I67QNJ CVE: NA

------------------------------------------------------------------

Platform devices which supports power control are often required to be power off/on together with the devices in the same power domain. However, there isn't a generic driver that support the power control logic of these devices.

ACPI container seems to be a good place to hold these control logic. Add platform devices in the same power domain in a ACPI container, we can easily get the locality information about these devices and can moniter the power of these devices in the same power domain together.

This patch provide three userspace control interface to control the power of devices together in the container: - state: Echo online to state to power up the devices in the container and then online these devices which will be triggered by BIOS. Echo offline to the state to offline and eject the child devices in the container which are ejectable. - pxms: show the pxms of devices which are present in the container.

In our scenario, we need to control the power of HBM memory devices which can be power consuming and will only be used in some specialized scenarios, such as HPC. HBM memory devices in a socket are in the same power domain, and should be power off/on together. We have come up with an idea that put these power control logic in a specialized driver, but ACPI container seems to be a more generic place to hold these control logic.

Signed-off-by: Zhang Zekun zhangzekun11@huawei.com --- v7: - check there is a container has PNP0C80 before create attributes under container_subsys

v6: - get container dev form container_subsys

v5: - check containers have PNP0C80 before create attibute files

v4: - prettify the code

v3: - move the common code to hisi_internal.h

drivers/base/container.c | 1 + drivers/soc/Kconfig | 1 + drivers/soc/Makefile | 1 + drivers/soc/hisilicon/Kconfig | 19 ++ drivers/soc/hisilicon/Makefile | 3 + drivers/soc/hisilicon/hisi_hbmdev.c | 240 ++++++++++++++++++++++++++ drivers/soc/hisilicon/hisi_internal.h | 31 ++++ 7 files changed, 296 insertions(+) create mode 100644 drivers/soc/hisilicon/Kconfig create mode 100644 drivers/soc/hisilicon/Makefile create mode 100644 drivers/soc/hisilicon/hisi_hbmdev.c create mode 100644 drivers/soc/hisilicon/hisi_internal.h

diff --git a/drivers/base/container.c b/drivers/base/container.c index 1ba42d2d3532..12e572d0c69b 100644 --- a/drivers/base/container.c +++ b/drivers/base/container.c @@ -30,6 +30,7 @@ struct bus_type container_subsys = { .online = trivial_online, .offline = container_offline, }; +EXPORT_SYMBOL_GPL(container_subsys);

void __init container_dev_init(void) { diff --git a/drivers/soc/Kconfig b/drivers/soc/Kconfig index 425ab6f7e375..f7c59b063321 100644 --- a/drivers/soc/Kconfig +++ b/drivers/soc/Kconfig @@ -23,5 +23,6 @@ source "drivers/soc/versatile/Kconfig" source "drivers/soc/xilinx/Kconfig" source "drivers/soc/zte/Kconfig" source "drivers/soc/kendryte/Kconfig" +source "drivers/soc/hisilicon/Kconfig"

endmenu diff --git a/drivers/soc/Makefile b/drivers/soc/Makefile index 36452bed86ef..68f186e00e44 100644 --- a/drivers/soc/Makefile +++ b/drivers/soc/Makefile @@ -29,3 +29,4 @@ obj-$(CONFIG_PLAT_VERSATILE) += versatile/ obj-y += xilinx/ obj-$(CONFIG_ARCH_ZX) += zte/ obj-$(CONFIG_SOC_KENDRYTE) += kendryte/ +obj-y += hisilicon/ diff --git a/drivers/soc/hisilicon/Kconfig b/drivers/soc/hisilicon/Kconfig new file mode 100644 index 000000000000..497787af004e --- /dev/null +++ b/drivers/soc/hisilicon/Kconfig @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Hisilicon SoC drivers +# +menu "Hisilicon SoC driver support" + +config HISI_HBMDEV + tristate "add extra support for hbm memory device" + depends on ACPI_HOTPLUG_MEMORY + select ACPI_CONTAINER + help + This driver add extra supports for memory devices. The driver + provides methods for userpace to control the power of memory + devices in a container. + + To compile this driver as a module, choose M here: + the module will be called hisi_hbmdev. + +endmenu diff --git a/drivers/soc/hisilicon/Makefile b/drivers/soc/hisilicon/Makefile new file mode 100644 index 000000000000..22e87acb1ab3 --- /dev/null +++ b/drivers/soc/hisilicon/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_HISI_HBMDEV) += hisi_hbmdev.o diff --git a/drivers/soc/hisilicon/hisi_hbmdev.c b/drivers/soc/hisilicon/hisi_hbmdev.c new file mode 100644 index 000000000000..03f2c3632dd0 --- /dev/null +++ b/drivers/soc/hisilicon/hisi_hbmdev.c @@ -0,0 +1,240 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) Huawei Technologies Co., Ltd. 2023. All rights reserved. + */ + +#include <linux/kobject.h> +#include <linux/module.h> +#include <linux/nodemask.h> +#include <linux/acpi.h> +#include <linux/container.h> + +#include "hisi_internal.h" + +#define ACPI_MEMORY_DEVICE_HID "PNP0C80" +#define ACPI_GENERIC_CONTAINER_DEVICE_HID "PNP0A06" + +struct cdev_node { + struct device *dev; + struct list_head clist; +}; + +struct memory_dev { + struct kobject *memdev_kobj; + struct cdev_node cdev_list; +}; + +static struct memory_dev *mdev; + +static int get_pxm(struct acpi_device *acpi_device, void *arg) +{ + acpi_handle handle = acpi_device->handle; + nodemask_t *mask = arg; + unsigned long long sta; + acpi_status status; + int nid; + + status = acpi_evaluate_integer(handle, "_STA", NULL, &sta); + if (ACPI_SUCCESS(status) && (sta & ACPI_STA_DEVICE_ENABLED)) { + nid = acpi_get_node(handle); + if (nid >= 0) + node_set(nid, *mask); + } + + return 0; +} + +static ssize_t pxms_show(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct acpi_device *adev = ACPI_COMPANION(dev); + nodemask_t mask; + + nodes_clear(mask); + acpi_dev_for_each_child(adev, get_pxm, &mask); + + return sysfs_emit(buf, "%*pbl\n", + nodemask_pr_args(&mask)); +} +static DEVICE_ATTR_RO(pxms); + +static int memdev_power_on(struct acpi_device *adev) +{ + acpi_handle handle = adev->handle; + acpi_status status; + + status = acpi_evaluate_object(handle, "_ON", NULL, NULL); + if (ACPI_FAILURE(status)) { + acpi_handle_warn(handle, "Power on failed (0x%x)\n", status); + return -ENODEV; + } + + return 0; +} + +static int eject_device(struct acpi_device *acpi_device, void *not_used) +{ + acpi_object_type unused; + acpi_status status; + + status = acpi_get_type(acpi_device->handle, &unused); + if (ACPI_FAILURE(status) || !acpi_device->flags.ejectable) + return -ENODEV; + + get_device(&acpi_device->dev); + status = acpi_hotplug_schedule(acpi_device, ACPI_OST_EC_OSPM_EJECT); + if (ACPI_SUCCESS(status)) + return 0; + + put_device(&acpi_device->dev); + acpi_evaluate_ost(acpi_device->handle, ACPI_OST_EC_OSPM_EJECT, + ACPI_OST_SC_NON_SPECIFIC_FAILURE, NULL); + + return status == AE_NO_MEMORY ? -ENOMEM : -EAGAIN; +} + +static int memdev_power_off(struct acpi_device *adev) +{ + return acpi_dev_for_each_child(adev, eject_device, NULL); +} + +static ssize_t state_store(struct device *dev, struct device_attribute *attr, + const char *buf, size_t count) +{ + struct acpi_device *adev = ACPI_COMPANION(dev); + const int type = online_type_from_str(buf); + int ret = -EINVAL; + + switch (type) { + case STATE_ONLINE: + ret = memdev_power_on(adev); + break; + case STATE_OFFLINE: + ret = memdev_power_off(adev); + break; + default: + break; + } + + if (ret) + return ret; + + return count; +} +static DEVICE_ATTR_WO(state); + +static int hbmdev_find(struct acpi_device *adev, void *arg) +{ + const char *hid = acpi_device_hid(adev); + bool *found = arg; + + if (!strcmp(hid, ACPI_MEMORY_DEVICE_HID)) { + *found = true; + return -1; + } + + return 0; +} + +static bool has_hbmdev(struct device *dev) +{ + struct acpi_device *adev = ACPI_COMPANION(dev); + const char *hid = acpi_device_hid(adev); + bool found = false; + + if (strcmp(hid, ACPI_GENERIC_CONTAINER_DEVICE_HID)) + return found; + + acpi_dev_for_each_child(adev, hbmdev_find, &found); + + return found; +} + +static int container_add(struct device *dev, void *data) +{ + struct cdev_node *cnode; + + if (!has_hbmdev(dev)) + return 0; + + cnode = kmalloc(sizeof(struct cdev_node), GFP_KERNEL); + if (!cnode) + return -ENOMEM; + + cnode->dev = dev; + list_add_tail(&cnode->clist, &mdev->cdev_list.clist); + + return 0; +} + +static void container_remove(void) +{ + struct cdev_node *cnode, *tmp; + + list_for_each_entry_safe(cnode, tmp, &mdev->cdev_list.clist, clist) { + device_remove_file(cnode->dev, &dev_attr_state); + device_remove_file(cnode->dev, &dev_attr_pxms); + list_del(&cnode->clist); + kfree(cnode); + } +} + +static int container_init(void) +{ + struct cdev_node *cnode; + + INIT_LIST_HEAD(&mdev->cdev_list.clist); + + if (bus_for_each_dev(&container_subsys, NULL, NULL, container_add)) { + container_remove(); + return -ENOMEM; + } + + if (list_empty(&mdev->cdev_list.clist)) + return -ENODEV; + + list_for_each_entry(cnode, &mdev->cdev_list.clist, clist) { + device_create_file(cnode->dev, &dev_attr_state); + device_create_file(cnode->dev, &dev_attr_pxms); + } + + return 0; +} + + +static int __init mdev_init(void) +{ + int ret; + + mdev = kzalloc(sizeof(struct memory_dev), GFP_KERNEL); + if (!mdev) + return -ENOMEM; + + ret = container_init(); + if (ret) { + kfree(mdev); + return ret; + } + + mdev->memdev_kobj = kobject_create_and_add("hbm_memory", kernel_kobj); + if (!mdev->memdev_kobj) { + container_remove(); + kfree(mdev); + return -ENOMEM; + } + + return ret; +} +module_init(mdev_init); + +static void __exit mdev_exit(void) +{ + container_remove(); + kobject_put(mdev->memdev_kobj); + kfree(mdev); +} +module_exit(mdev_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_AUTHOR("Zhang Zekun zhangzekun11@huawei.com"); diff --git a/drivers/soc/hisilicon/hisi_internal.h b/drivers/soc/hisilicon/hisi_internal.h new file mode 100644 index 000000000000..5345174f6b84 --- /dev/null +++ b/drivers/soc/hisilicon/hisi_internal.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) Huawei Technologies Co., Ltd. 2023. All rights reserved. + */ + +#ifndef _HISI_INTERNAL_H +#define _HISI_INTERNAL_H + +enum { + STATE_ONLINE, + STATE_OFFLINE, +}; + +static const char *const online_type_to_str[] = { + [STATE_ONLINE] = "online", + [STATE_OFFLINE] = "offline", +}; + +static inline int online_type_from_str(const char *str) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(online_type_to_str); i++) { + if (sysfs_streq(str, online_type_to_str[i])) + return i; + } + + return -EINVAL; +} + +#endif

Offering: HULK hulk inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/I67QNJ CVE: NA

--------------------------------------------------

Provide additional memory device topology information in hisi_hbmdev. This memory topology informaton can provide useful information for userspace to select the closest memory device to a certain cpu.

Signed-off-by: Zhang Zekun zhangzekun11@huawei.com --- drivers/soc/hisilicon/Kconfig | 9 +++-- drivers/soc/hisilicon/hisi_hbmdev.c | 61 +++++++++++++++++++++++++++++ 2 files changed, 67 insertions(+), 3 deletions(-)

diff --git a/drivers/soc/hisilicon/Kconfig b/drivers/soc/hisilicon/Kconfig index 497787af004e..781e97b1b742 100644 --- a/drivers/soc/hisilicon/Kconfig +++ b/drivers/soc/hisilicon/Kconfig @@ -9,9 +9,12 @@ config HISI_HBMDEV depends on ACPI_HOTPLUG_MEMORY select ACPI_CONTAINER help - This driver add extra supports for memory devices. The driver - provides methods for userpace to control the power of memory - devices in a container. + This driver add two extra supports for memory devices. The driver + provides methods for userpace to control the power of memory devices + in a container. Besides, it provides extra locality information + between cpus and memory devices for userspace, which can take + advantage of this functionality to select the closet memory device + to a certain cpu.

To compile this driver as a module, choose M here: the module will be called hisi_hbmdev. diff --git a/drivers/soc/hisilicon/hisi_hbmdev.c b/drivers/soc/hisilicon/hisi_hbmdev.c index 03f2c3632dd0..5b6b1618148c 100644 --- a/drivers/soc/hisilicon/hisi_hbmdev.c +++ b/drivers/soc/hisilicon/hisi_hbmdev.c @@ -8,6 +8,9 @@ #include <linux/nodemask.h> #include <linux/acpi.h> #include <linux/container.h> +#include <linux/node.h> +#include <linux/arch_topology.h> +#include <linux/memory_hotplug.h>

#include "hisi_internal.h"

@@ -21,11 +24,67 @@ struct cdev_node {

struct memory_dev { struct kobject *memdev_kobj; + struct kobject *topo_kobj; struct cdev_node cdev_list; + nodemask_t cluster_cpumask[MAX_NUMNODES]; };

static struct memory_dev *mdev;

+static ssize_t memory_locality_show(struct kobject *kobj, + struct kobj_attribute *attr, + char *buf) +{ + int i, count = 0; + + for (i = 0; i < MAX_NUMNODES; i++) { + if (hotplug_mdev[i] != NULL && !nodes_empty(mdev->cluster_cpumask[i])) { + count += sysfs_emit_at(buf, count, "%d %*pbl\n", i, + nodemask_pr_args(&mdev->cluster_cpumask[i])); + } + } + + return count; +} + +static struct kobj_attribute memory_locality_attribute = + __ATTR(memory_locality, 0444, memory_locality_show, NULL); + +static void memory_topo_init(void) +{ + int ret, nid, cluster_id, cpu; + struct acpi_device *adev; + nodemask_t mask; + + for (nid = 0; nid < MAX_NUMNODES; nid++) { + if (!hotplug_mdev[nid]) + continue; + + adev = hotplug_mdev[nid]; + ret = fwnode_property_read_u32(acpi_fwnode_handle(adev), + "cluster-id", &cluster_id); + if (ret < 0) { + pr_debug("Failed to read cluster id\n"); + return; + } + + nodes_clear(mask); + for_each_possible_cpu(cpu) { + if (topology_cluster_id(cpu) == cluster_id) + node_set(cpu, mask); + } + mdev->cluster_cpumask[nid] = mask; + } + + mdev->topo_kobj = kobject_create_and_add("memory_topo", mdev->memdev_kobj); + if (!mdev->topo_kobj) + return; + + ret = sysfs_create_file(mdev->topo_kobj, &memory_locality_attribute.attr); + if (ret) + kobject_put(mdev->topo_kobj); +} + static int get_pxm(struct acpi_device *acpi_device, void *arg) { acpi_handle handle = acpi_device->handle; @@ -224,6 +283,7 @@ static int __init mdev_init(void) return -ENOMEM; }

+ memory_topo_init(); return ret; } module_init(mdev_init); @@ -232,6 +292,7 @@ static void __exit mdev_exit(void) { container_remove(); kobject_put(mdev->memdev_kobj); + kobject_put(mdev->topo_kobj); kfree(mdev); } module_exit(mdev_exit);