October 2024 - Kernel - mailweb.openeuler.org

[PATCH OLK-5.10] drm/nouveau: prime: fix refcount underflow
by Chen Jun 23 Oct '24

23 Oct '24

From: Danilo Krummrich <dakr(a)kernel.org> stable inclusion from stable-v5.10.224 commit 906372e753c5027a1dc88743843b6aa2ad1aaecf category: bugfix bugzilla: https://gitee.com/src-openeuler/kernel/issues/IALCQI CVE: CVE-2024-43867 Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id… -------------------------------- [ Upstream commit a9bf3efc33f1fbf88787a277f7349459283c9b95 ] Calling nouveau_bo_ref() on a nouveau_bo without initializing it (and hence the backing ttm_bo) leads to a refcount underflow. Instead of calling nouveau_bo_ref() in the unwind path of drm_gem_object_init(), clean things up manually. Fixes: ab9ccb96a6e6 ("drm/nouveau: use prime helpers") Reviewed-by: Ben Skeggs <bskeggs(a)nvidia.com> Reviewed-by: Christian König <christian.koenig(a)amd.com> Signed-off-by: Danilo Krummrich <dakr(a)kernel.org> Link: https://patchwork.freedesktop.org/patch/msgid/20240718165959.3983-2-dakr@ke… (cherry picked from commit 1b93f3e89d03cfc576636e195466a0d728ad8de5) Signed-off-by: Danilo Krummrich <dakr(a)kernel.org> Signed-off-by: Sasha Levin <sashal(a)kernel.org> Signed-off-by: Chen Jun <chenjun102(a)huawei.com> --- drivers/gpu/drm/nouveau/nouveau_prime.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_prime.c b/drivers/gpu/drm/nouveau/nouveau_prime.c index f08bda533bd9..65874a48dc80 100644 --- a/drivers/gpu/drm/nouveau/nouveau_prime.c +++ b/drivers/gpu/drm/nouveau/nouveau_prime.c @@ -81,7 +81,8 @@ struct drm_gem_object *nouveau_gem_prime_import_sg_table(struct drm_device *dev, * to the caller, instead of a normal nouveau_bo ttm reference. */ ret = drm_gem_object_init(dev, &nvbo->bo.base, size); if (ret) { - nouveau_bo_ref(NULL, &nvbo); + drm_gem_object_release(&nvbo->bo.base); + kfree(nvbo); obj = ERR_PTR(-ENOMEM); goto unlock; } -- 2.17.1

2 1

[PATCH OLK-6.6] LoongArch: prevent LS7A Bus Master clearing on kexec
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Ming Wang <wangming01(a)loongson.cn> LoongArch inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3M1 -------------------------------- This is similar to commit 62b6dee1b44a ("PCI/portdrv: Prevent LS7A Bus Master clearing on shutdown"), which prevents LS7A Bus Master clearing on kexec. The key point of this is to work around the LS7A defect that clearing PCI_COMMAND_MASTER prevents MMIO requests from going downstream, and we may need to do that even after .shutdown(), e.g., to print console messages. And in this case we rely on .shutdown() for the downstream devices to disable interrupts and DMA. Only skip Bus Master clearing on bridges because endpoint devices still need it. Signed-off-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Ming Wang <wangming01(a)loongson.cn> --- drivers/pci/pci-driver.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 9c59bf03d657..b699839a7d4f 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -526,7 +526,11 @@ static void pci_device_shutdown(struct device *dev) * If it is not a kexec reboot, firmware will hit the PCI * devices with big hammer and stop their DMA any way. */ +#ifdef CONFIG_LOONGARCH + if (kexec_in_progress && !pci_is_bridge(pci_dev) && (pci_dev->current_state <= PCI_D3hot)) +#else if (kexec_in_progress && (pci_dev->current_state <= PCI_D3hot)) +#endif pci_clear_master(pci_dev); } -- 2.33.0

2 1

[PATCH OLK-6.6] LoongArch: Add AVEC irqchip support
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Tianyang Zhang <zhangtianyang(a)loongson.cn> LoongArch inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3D3 -------------------------------- Introduce the advanced extended interrupt controllers (AVECINTC). This feature will allow each core to have 256 independent interrupt vectors and MSI interrupts can be independently routed to any vector on any CPU. Co-developed-by: Jianmin Lv <lvjianmin(a)loongson.cn> Signed-off-by: Jianmin Lv <lvjianmin(a)loongson.cn> Co-developed-by: Liupu Wang <wangliupu(a)loongson.cn> Signed-off-by: Liupu Wang <wangliupu(a)loongson.cn> Co-developed-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Tianyang Zhang <zhangtianyang(a)loongson.cn> --- .../arch/loongarch/irq-chip-model.rst | 32 ++ .../zh_CN/arch/loongarch/irq-chip-model.rst | 32 ++ arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/cpu-features.h | 1 + arch/loongarch/include/asm/cpu.h | 2 + arch/loongarch/include/asm/hardirq.h | 3 +- arch/loongarch/include/asm/irq.h | 29 +- arch/loongarch/include/asm/loongarch.h | 18 +- arch/loongarch/include/asm/smp.h | 2 + arch/loongarch/kernel/cpu-probe.c | 3 +- arch/loongarch/kernel/irq.c | 13 + arch/loongarch/kernel/legacy_boot.h | 10 + arch/loongarch/kernel/smp.c | 6 + drivers/irqchip/Makefile | 2 +- drivers/irqchip/irq-loongarch-avec.c | 452 ++++++++++++++++++ drivers/irqchip/irq-loongarch-cpu.c | 7 +- drivers/irqchip/irq-loongson-eiointc.c | 9 +- drivers/irqchip/irq-loongson-htvec.c | 2 + drivers/irqchip/irq-loongson-liointc.c | 2 + drivers/irqchip/irq-loongson-pch-lpc.c | 2 + drivers/irqchip/irq-loongson-pch-msi.c | 44 +- drivers/irqchip/irq-loongson-pch-pic.c | 2 + drivers/irqchip/irq-loongson.h | 27 ++ include/linux/cpuhotplug.h | 4 + 24 files changed, 675 insertions(+), 30 deletions(-) create mode 100644 drivers/irqchip/irq-loongarch-avec.c create mode 100644 drivers/irqchip/irq-loongson.h diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst index 7988f4192363..6dd48256e39f 100644 --- a/Documentation/arch/loongarch/irq-chip-model.rst +++ b/Documentation/arch/loongarch/irq-chip-model.rst @@ -85,6 +85,38 @@ to CPUINTC directly:: | Devices | +---------+ +Advanced Extended IRQ model +=========================== + +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go +to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, PCH-MSI interrupts go +to AVECINTC, and then go to CPUINTC directly, while all other devices interrupts +go to PCH-PIC/PCH-LPC and gathered by EIOINTC, and then go to CPUINTC directly:: + + +-----+ +-----------------------+ +-------+ + | IPI | --> | CPUINTC | <-- | Timer | + +-----+ +-----------------------+ +-------+ + ^ ^ ^ + | | | + +---------+ +----------+ +---------+ +-------+ + | EIOINTC | | AVECINTC | | LIOINTC | <-- | UARTs | + +---------+ +----------+ +---------+ +-------+ + ^ ^ + | | + +---------+ +---------+ + | PCH-PIC | | PCH-MSI | + +---------+ +---------+ + ^ ^ ^ + | | | + +---------+ +---------+ +---------+ + | Devices | | PCH-LPC | | Devices | + +---------+ +---------+ +---------+ + ^ + | + +---------+ + | Devices | + +---------+ + ACPI-related definitions ======================== diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst index f1e9ab18206c..472761938682 100644 --- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst +++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst @@ -87,6 +87,38 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC:: | Devices | +---------+ +高级扩展IRQ模型 +=============== + +在这种模型里面，IPI（Inter-Processor Interrupt）和CPU本地时钟中断直接发送到CPUINTC， +CPU串口（UARTs）中断发送到LIOINTC，PCH-MSI中断发送到AVECINTC，而后通过AVECINTC直接 +送达CPUINTC，而其他所有设备的中断则分别发送到所连接的PCH-PIC/PCH-LPC，然后由EIOINTC +统一收集，再直接到达CPUINTC:: + + +-----+ +-----------------------+ +-------+ + | IPI | --> | CPUINTC | <-- | Timer | + +-----+ +-----------------------+ +-------+ + ^ ^ ^ + | | | + +---------+ +----------+ +---------+ +-------+ + | EIOINTC | | AVECINTC | | LIOINTC | <-- | UARTs | + +---------+ +----------+ +---------+ +-------+ + ^ ^ + | | + +---------+ +---------+ + | PCH-PIC | | PCH-MSI | + +---------+ +---------+ + ^ ^ ^ + | | | + +---------+ +---------+ +---------+ + | Devices | | PCH-LPC | | Devices | + +---------+ +---------+ +---------+ + ^ + | + +---------+ + | Devices | + +---------+ + ACPI相关的定义 ============== diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 58da6e03ecb5..710ec4831887 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -77,6 +77,7 @@ config LOONGARCH select GENERIC_ENTRY select GENERIC_GETTIMEOFDAY select GENERIC_IOREMAP if !ARCH_IOREMAP + select GENERIC_IRQ_MATRIX_ALLOCATOR select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE select GENERIC_IRQ_SHOW diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h index 2eafe6a6aca8..16a716f88a5c 100644 --- a/arch/loongarch/include/asm/cpu-features.h +++ b/arch/loongarch/include/asm/cpu-features.h @@ -65,5 +65,6 @@ #define cpu_has_guestid cpu_opt(LOONGARCH_CPU_GUESTID) #define cpu_has_hypervisor cpu_opt(LOONGARCH_CPU_HYPERVISOR) #define cpu_has_ptw cpu_opt(LOONGARCH_CPU_PTW) +#define cpu_has_avecint cpu_opt(LOONGARCH_CPU_AVECINT) #endif /* __ASM_CPU_FEATURES_H */ diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h index 48b9f7168bcc..843f9c4ec980 100644 --- a/arch/loongarch/include/asm/cpu.h +++ b/arch/loongarch/include/asm/cpu.h @@ -99,6 +99,7 @@ enum cpu_type_enum { #define CPU_FEATURE_GUESTID 24 /* CPU has GuestID feature */ #define CPU_FEATURE_HYPERVISOR 25 /* CPU has hypervisor (running in VM) */ #define CPU_FEATURE_PTW 26 /* CPU has hardware page table walker */ +#define CPU_FEATURE_AVECINT 27 /* CPU has avec interrupt */ #define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG) #define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM) @@ -127,5 +128,6 @@ enum cpu_type_enum { #define LOONGARCH_CPU_GUESTID BIT_ULL(CPU_FEATURE_GUESTID) #define LOONGARCH_CPU_HYPERVISOR BIT_ULL(CPU_FEATURE_HYPERVISOR) #define LOONGARCH_CPU_PTW BIT_ULL(CPU_FEATURE_PTW) +#define LOONGARCH_CPU_AVECINT BIT_ULL(CPU_FEATURE_AVECINT) #endif /* _ASM_CPU_H */ diff --git a/arch/loongarch/include/asm/hardirq.h b/arch/loongarch/include/asm/hardirq.h index b26d596a73aa..5f70cb77b54d 100644 --- a/arch/loongarch/include/asm/hardirq.h +++ b/arch/loongarch/include/asm/hardirq.h @@ -15,8 +15,9 @@ extern void ack_bad_irq(unsigned int irq); enum ipi_msg_type { IPI_RESCHEDULE, IPI_CALL_FUNCTION, + IPI_CLEAR_VECTOR, }; -#define NR_IPI 2 +#define NR_IPI 3 typedef struct { unsigned int ipi_irqs[NR_IPI]; diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h index 79f4c834ffe8..fc600aed50c8 100644 --- a/arch/loongarch/include/asm/irq.h +++ b/arch/loongarch/include/asm/irq.h @@ -39,6 +39,17 @@ void spurious_interrupt(void); #define NR_IRQS_LEGACY 16 +/* + * 256 Vectors Mapping for AVECINTC: + * + * 0 - 15: Mapping classic IPs, e.g. IP0-12. + * 16 - 255: Mapping vectors for external IRQ. + * + */ +#define NR_VECTORS 256 +#define NR_LEGACY_VECTORS 16 +#define IRQ_MATRIX_BITS NR_VECTORS + #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace extern bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); @@ -67,7 +78,7 @@ extern struct acpi_vector_group msi_group[MAX_IO_PICS]; #define LOONGSON_LPC_LAST_IRQ (LOONGSON_LPC_IRQ_BASE + 15) #define LOONGSON_CPU_IRQ_BASE 16 -#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 14) +#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 15) #define LOONGSON_PCH_IRQ_BASE 64 #define LOONGSON_PCH_ACPI_IRQ (LOONGSON_PCH_IRQ_BASE + 47) @@ -90,20 +101,8 @@ struct acpi_madt_bio_pic; struct acpi_madt_msi_pic; struct acpi_madt_lpc_pic; -int liointc_acpi_init(struct irq_domain *parent, - struct acpi_madt_lio_pic *acpi_liointc); -int eiointc_acpi_init(struct irq_domain *parent, - struct acpi_madt_eio_pic *acpi_eiointc); - -int htvec_acpi_init(struct irq_domain *parent, - struct acpi_madt_ht_pic *acpi_htvec); -int pch_lpc_acpi_init(struct irq_domain *parent, - struct acpi_madt_lpc_pic *acpi_pchlpc); -int pch_msi_acpi_init(struct irq_domain *parent, - struct acpi_madt_msi_pic *acpi_pchmsi); -int pch_pic_acpi_init(struct irq_domain *parent, - struct acpi_madt_bio_pic *acpi_pchpic); -int find_pch_pic(u32 gsi); +void complete_irq_moving(void); + struct fwnode_handle *get_pch_msi_handle(int pci_segment); extern struct acpi_madt_lio_pic *acpi_liointc; diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h index e852c0f62eb7..964bcfe08d8a 100644 --- a/arch/loongarch/include/asm/loongarch.h +++ b/arch/loongarch/include/asm/loongarch.h @@ -254,8 +254,8 @@ #define CSR_ESTAT_EXC_WIDTH 6 #define CSR_ESTAT_EXC (_ULCAST_(0x3f) << CSR_ESTAT_EXC_SHIFT) #define CSR_ESTAT_IS_SHIFT 0 -#define CSR_ESTAT_IS_WIDTH 14 -#define CSR_ESTAT_IS (_ULCAST_(0x3fff) << CSR_ESTAT_IS_SHIFT) +#define CSR_ESTAT_IS_WIDTH 15 +#define CSR_ESTAT_IS (_ULCAST_(0x7fff) << CSR_ESTAT_IS_SHIFT) #define LOONGARCH_CSR_ERA 0x6 /* ERA */ @@ -650,6 +650,13 @@ #define LOONGARCH_CSR_CTAG 0x98 /* TagLo + TagHi */ +#define LOONGARCH_CSR_ISR0 0xa0 +#define LOONGARCH_CSR_ISR1 0xa1 +#define LOONGARCH_CSR_ISR2 0xa2 +#define LOONGARCH_CSR_ISR3 0xa3 + +#define LOONGARCH_CSR_IRR 0xa4 + #define LOONGARCH_CSR_PRID 0xc0 /* Shadow MCSR : 0xc0 ~ 0xff */ @@ -1004,7 +1011,7 @@ /* * CSR_ECFG IM */ -#define ECFG0_IM 0x00001fff +#define ECFG0_IM 0x00005fff #define ECFGB_SIP0 0 #define ECFGF_SIP0 (_ULCAST_(1) << ECFGB_SIP0) #define ECFGB_SIP1 1 @@ -1047,6 +1054,7 @@ #define IOCSRF_EIODECODE BIT_ULL(9) #define IOCSRF_FLATMODE BIT_ULL(10) #define IOCSRF_VM BIT_ULL(11) +#define IOCSRF_AVEC BIT_ULL(15) #define LOONGARCH_IOCSR_VENDOR 0x10 @@ -1057,6 +1065,7 @@ #define LOONGARCH_IOCSR_MISC_FUNC 0x420 #define IOCSR_MISC_FUNC_TIMER_RESET BIT_ULL(21) #define IOCSR_MISC_FUNC_EXT_IOI_EN BIT_ULL(48) +#define IOCSR_MISC_FUNC_AVEC_EN BIT_ULL(51) #define LOONGARCH_IOCSR_CPUTEMP 0x428 @@ -1378,9 +1387,10 @@ __BUILD_CSR_OP(tlbidx) #define INT_TI 11 /* Timer */ #define INT_IPI 12 #define INT_NMI 13 +#define INT_AVEC 14 /* ExcCodes corresponding to interrupts */ -#define EXCCODE_INT_NUM (INT_NMI + 1) +#define EXCCODE_INT_NUM (INT_AVEC + 1) #define EXCCODE_INT_START 64 #define EXCCODE_INT_END (EXCCODE_INT_START + EXCCODE_INT_NUM - 1) diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h index 75d30529748c..630e5ebec21c 100644 --- a/arch/loongarch/include/asm/smp.h +++ b/arch/loongarch/include/asm/smp.h @@ -67,9 +67,11 @@ extern int __cpu_logical_map[NR_CPUS]; #define ACTION_BOOT_CPU 0 #define ACTION_RESCHEDULE 1 #define ACTION_CALL_FUNCTION 2 +#define ACTION_CLEAR_VECTOR 3 #define SMP_BOOT_CPU BIT(ACTION_BOOT_CPU) #define SMP_RESCHEDULE BIT(ACTION_RESCHEDULE) #define SMP_CALL_FUNCTION BIT(ACTION_CALL_FUNCTION) +#define SMP_CLEAR_VECTOR BIT(ACTION_CLEAR_VECTOR) struct secondary_data { unsigned long stack; diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c index 55320813ee08..14f0449f5452 100644 --- a/arch/loongarch/kernel/cpu-probe.c +++ b/arch/loongarch/kernel/cpu-probe.c @@ -106,7 +106,6 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) elf_hwcap |= HWCAP_LOONGARCH_CRC32; } - config = read_cpucfg(LOONGARCH_CPUCFG2); if (config & CPUCFG2_LAM) { c->options |= LOONGARCH_CPU_LAM; @@ -174,6 +173,8 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) c->options |= LOONGARCH_CPU_FLATMODE; if (config & IOCSRF_EIODECODE) c->options |= LOONGARCH_CPU_EIODECODE; + if (config & IOCSRF_AVEC) + c->options |= LOONGARCH_CPU_AVECINT; if (config & IOCSRF_VM) c->options |= LOONGARCH_CPU_HYPERVISOR; diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c index a2f6ce3061f3..50089a7a1a9d 100644 --- a/arch/loongarch/kernel/irq.c +++ b/arch/loongarch/kernel/irq.c @@ -122,6 +122,19 @@ void fixup_irqs(void) } #endif +int __init arch_probe_nr_irqs(void) +{ + int nr_io_pics = bitmap_weight(&loongson_sysconf.cores_io_master, NR_CPUS); + + if (!cpu_has_avecint) + nr_irqs = (64 + NR_VECTORS * nr_io_pics); + else + nr_irqs = (64 + NR_VECTORS * (nr_cpu_ids + nr_io_pics)); + + return NR_IRQS_LEGACY; +} + + void __init init_IRQ(void) { int i, ret; diff --git a/arch/loongarch/kernel/legacy_boot.h b/arch/loongarch/kernel/legacy_boot.h index 104d8c53bd2d..943229137f51 100644 --- a/arch/loongarch/kernel/legacy_boot.h +++ b/arch/loongarch/kernel/legacy_boot.h @@ -90,4 +90,14 @@ pch_msi_parse_madt(union acpi_subtable_headers *header, extern struct irq_domain *get_pchpic_irq_domain(void); extern __init void fw_init_cmdline(unsigned long argc, unsigned long cmdp); + +extern int liointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lio_pic *acpi_liointc); +extern int eiointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_eio_pic *acpi_eiointc); +extern int htvec_acpi_init(struct irq_domain *parent, + struct acpi_madt_ht_pic *acpi_htvec); +extern int pch_lpc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lpc_pic *acpi_pchlpc); + #endif diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index ea69606fd9f5..bd0bd3decd32 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -71,6 +71,7 @@ static DEFINE_PER_CPU(int, cpu_state); static const char *ipi_types[NR_IPI] __tracepoint_string = { [IPI_RESCHEDULE] = "Rescheduling interrupts", [IPI_CALL_FUNCTION] = "Function call interrupts", + [IPI_CLEAR_VECTOR] = "Clear vector interrupts", }; void show_ipi_list(struct seq_file *p, int prec) @@ -245,6 +246,11 @@ static irqreturn_t loongson_ipi_interrupt(int irq, void *dev) per_cpu(irq_stat, cpu).ipi_irqs[IPI_CALL_FUNCTION]++; } + if (action & SMP_CLEAR_VECTOR) { + complete_irq_moving(); + per_cpu(irq_stat, cpu).ipi_irqs[IPI_CLEAR_VECTOR]++; + } + return IRQ_HANDLED; } diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 246aa0603d6e..787206e166fc 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -116,7 +116,7 @@ obj-$(CONFIG_LS1X_IRQ) += irq-ls1x.o obj-$(CONFIG_TI_SCI_INTR_IRQCHIP) += irq-ti-sci-intr.o obj-$(CONFIG_TI_SCI_INTA_IRQCHIP) += irq-ti-sci-inta.o obj-$(CONFIG_TI_PRUSS_INTC) += irq-pruss-intc.o -obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o +obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o irq-loongarch-avec.o obj-$(CONFIG_LOONGSON_LIOINTC) += irq-loongson-liointc.o obj-$(CONFIG_LOONGSON_EIOINTC) += irq-loongson-eiointc.o obj-$(CONFIG_LOONGSON_HTPIC) += irq-loongson-htpic.o diff --git a/drivers/irqchip/irq-loongarch-avec.c b/drivers/irqchip/irq-loongarch-avec.c new file mode 100644 index 000000000000..e8c43b3fd826 --- /dev/null +++ b/drivers/irqchip/irq-loongarch-avec.c @@ -0,0 +1,452 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2020-2024 Loongson Technologies, Inc. + */ + +#include <linux/cpuhotplug.h> +#include <linux/init.h> +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/irqchip.h> +#include <linux/irqchip/chained_irq.h> +#include <linux/irqdomain.h> +#include <linux/kernel.h> +#include <linux/msi.h> +#include <linux/radix-tree.h> +#include <linux/spinlock.h> + +#include <asm/loongarch.h> +#include <asm/setup.h> + +#include "irq-loongson.h" + +#define VECTORS_PER_REG 64 +#define IRR_VECTOR_MASK 0xffUL +#define IRR_INVALID_MASK 0x80000000UL +#define AVEC_MSG_OFFSET 0x100000 + +#ifdef CONFIG_SMP +struct pending_list { + struct list_head head; +}; + +static struct cpumask intersect_mask; +static DEFINE_PER_CPU(struct pending_list, pending_list); +#endif + +static DEFINE_PER_CPU(struct irq_desc * [NR_VECTORS], irq_map); + +struct avecintc_chip { + raw_spinlock_t lock; + struct fwnode_handle *fwnode; + struct irq_domain *domain; + struct irq_matrix *vector_matrix; + phys_addr_t msi_base_addr; +}; + +static struct avecintc_chip loongarch_avec; + +struct avecintc_data { + struct list_head entry; + unsigned int cpu; + unsigned int vec; + unsigned int prev_cpu; + unsigned int prev_vec; + unsigned int moving; +}; + +static inline void avecintc_ack_irq(struct irq_data *d) +{ +} + +static inline void avecintc_mask_irq(struct irq_data *d) +{ +} + +static inline void avecintc_unmask_irq(struct irq_data *d) +{ +} + +#ifdef CONFIG_SMP +static inline void pending_list_init(int cpu) +{ + struct pending_list *plist = per_cpu_ptr(&pending_list, cpu); + + INIT_LIST_HEAD(&plist->head); +} + +static void avecintc_sync(struct avecintc_data *adata) +{ + struct pending_list *plist; + + if (cpu_online(adata->prev_cpu)) { + plist = per_cpu_ptr(&pending_list, adata->prev_cpu); + list_add_tail(&adata->entry, &plist->head); + adata->moving = 1; + smp_ops.send_ipi_single(adata->prev_cpu, SMP_CLEAR_VECTOR); + } +} + +static int avecintc_set_affinity(struct irq_data *data, const struct cpumask *dest, bool force) +{ + int cpu, ret, vector; + struct avecintc_data *adata; + + raw_spin_lock(&loongarch_avec.lock); + adata = irq_data_get_irq_chip_data(data); + + if (adata->moving) { + raw_spin_unlock(&loongarch_avec.lock); + return -EBUSY; + } + + if (adata->vec == UINT_MAX) { + raw_spin_unlock(&loongarch_avec.lock); + return -EINVAL; + } + + if (cpu_online(adata->cpu) && cpumask_test_cpu(adata->cpu, dest)) { + raw_spin_unlock(&loongarch_avec.lock); + return 0; + } + + cpumask_and(&intersect_mask, dest, cpu_online_mask); + + ret = irq_matrix_alloc(loongarch_avec.vector_matrix, &intersect_mask, false, &cpu); + if (ret < 0) { + raw_spin_unlock(&loongarch_avec.lock); + return ret; + } + + vector = ret; + adata->cpu = cpu; + adata->vec = vector; + per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(data); + avecintc_sync(adata); + + raw_spin_unlock(&loongarch_avec.lock); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + return IRQ_SET_MASK_OK; +} + +static int avecintc_cpu_online(unsigned int cpu) +{ + if (!loongarch_avec.vector_matrix) + return 0; + + raw_spin_lock(&loongarch_avec.lock); + + irq_matrix_online(loongarch_avec.vector_matrix); + + pending_list_init(cpu); + + raw_spin_unlock(&loongarch_avec.lock); + + return 0; +} + +static int avecintc_cpu_offline(unsigned int cpu) +{ + struct pending_list *plist = per_cpu_ptr(&pending_list, cpu); + + if (!loongarch_avec.vector_matrix) + return 0; + + raw_spin_lock(&loongarch_avec.lock); + + if (!list_empty(&plist->head)) + pr_warn("CPU#%d vector is busy\n", cpu); + irq_matrix_offline(loongarch_avec.vector_matrix); + + raw_spin_unlock(&loongarch_avec.lock); + + return 0; +} + +void complete_irq_moving(void) +{ + struct pending_list *plist = this_cpu_ptr(&pending_list); + struct avecintc_data *adata, *tdata; + int cpu, vector, bias; + uint64_t isr; + + raw_spin_lock(&loongarch_avec.lock); + + list_for_each_entry_safe(adata, tdata, &plist->head, entry) { + cpu = adata->prev_cpu; + vector = adata->prev_vec; + bias = vector / VECTORS_PER_REG; + switch (bias) { + case 0: + isr = csr_read64(LOONGARCH_CSR_ISR0); + break; + case 1: + isr = csr_read64(LOONGARCH_CSR_ISR1); + break; + case 2: + isr = csr_read64(LOONGARCH_CSR_ISR2); + break; + case 3: + isr = csr_read64(LOONGARCH_CSR_ISR3); + break; + } + + if (isr & (1UL << (vector % VECTORS_PER_REG))) { + smp_ops.send_ipi_single(cpu, SMP_CLEAR_VECTOR); + continue; + } + list_del(&adata->entry); + irq_matrix_free(loongarch_avec.vector_matrix, cpu, vector, false); + this_cpu_write(irq_map[vector], NULL); + adata->moving = 0; + adata->prev_cpu = adata->cpu; + adata->prev_vec = adata->vec; + } + + raw_spin_unlock(&loongarch_avec.lock); +} +#endif + +static void avecintc_compose_msi_msg(struct irq_data *d, struct msi_msg *msg) +{ + struct avecintc_data *adata = irq_data_get_irq_chip_data(d); + + msg->address_hi = 0x0; + msg->address_lo = (loongarch_avec.msi_base_addr | (adata->vec & 0xff) << 4) + | ((cpu_logical_map(adata->cpu & 0xffff)) << 12); + msg->data = 0x0; +} + +static struct irq_chip avec_irq_controller = { + .name = "AVECINTC", + .irq_ack = avecintc_ack_irq, + .irq_mask = avecintc_mask_irq, + .irq_unmask = avecintc_unmask_irq, +#ifdef CONFIG_SMP + .irq_set_affinity = avecintc_set_affinity, +#endif + .irq_compose_msi_msg = avecintc_compose_msi_msg, +}; + +static void avecintc_irq_dispatch(struct irq_desc *desc) +{ + struct irq_chip *chip = irq_desc_get_chip(desc); + struct irq_desc *d; + + chained_irq_enter(chip, desc); + + while (true) { + unsigned long vector = csr_read64(LOONGARCH_CSR_IRR); + + if (vector & IRR_INVALID_MASK) + break; + + vector &= IRR_VECTOR_MASK; + + d = this_cpu_read(irq_map[vector]); + if (d) { + generic_handle_irq_desc(d); + } else { + spurious_interrupt(); + pr_warn("Unexpected IRQ occurs on CPU#%d [vector %ld]\n", + smp_processor_id(), vector); + } + } + + chained_irq_exit(chip, desc); +} + +static int avecintc_alloc_vector(struct irq_data *irqd, struct avecintc_data *adata) +{ + int cpu, ret; + unsigned long flags; + + raw_spin_lock_irqsave(&loongarch_avec.lock, flags); + + ret = irq_matrix_alloc(loongarch_avec.vector_matrix, cpu_online_mask, false, &cpu); + if (ret < 0) { + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + return ret; + } + + adata->prev_cpu = adata->cpu = cpu; + adata->prev_vec = adata->vec = ret; + per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(irqd); + + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + + return 0; +} + +static int avecintc_domain_alloc(struct irq_domain *domain, + unsigned int virq, unsigned int nr_irqs, void *arg) +{ + for (unsigned int i = 0; i < nr_irqs; i++) { + struct irq_data *irqd = irq_domain_get_irq_data(domain, virq + i); + struct avecintc_data *adata = kzalloc(sizeof(*adata), GFP_KERNEL); + int ret; + + if (!adata) + return -ENOMEM; + + ret = avecintc_alloc_vector(irqd, adata); + if (ret < 0) { + kfree(adata); + return ret; + } + + irq_domain_set_info(domain, virq + i, virq + i, &avec_irq_controller, + adata, handle_edge_irq, NULL, NULL); + irqd_set_single_target(irqd); + irqd_set_affinity_on_activate(irqd); + } + + return 0; +} + +static void avecintc_free_vector(struct irq_data *irqd, struct avecintc_data *adata) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&loongarch_avec.lock, flags); + + per_cpu(irq_map, adata->cpu)[adata->vec] = NULL; + irq_matrix_free(loongarch_avec.vector_matrix, adata->cpu, adata->vec, false); + +#ifdef CONFIG_SMP + if (!adata->moving) { + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + return; + } + + per_cpu(irq_map, adata->prev_cpu)[adata->prev_vec] = NULL; + irq_matrix_free(loongarch_avec.vector_matrix, adata->prev_cpu, adata->prev_vec, false); + list_del_init(&adata->entry); +#endif + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); +} + +static void avecintc_domain_free(struct irq_domain *domain, + unsigned int virq, unsigned int nr_irqs) +{ + for (unsigned int i = 0; i < nr_irqs; i++) { + struct irq_data *d = irq_domain_get_irq_data(domain, virq + i); + + if (d) { + struct avecintc_data *adata = irq_data_get_irq_chip_data(d); + + avecintc_free_vector(d, adata); + irq_domain_reset_irq_data(d); + kfree(adata); + } + } +} + +static const struct irq_domain_ops avecintc_domain_ops = { + .alloc = avecintc_domain_alloc, + .free = avecintc_domain_free, +}; + +static int __init irq_matrix_init(void) +{ + loongarch_avec.vector_matrix = irq_alloc_matrix(NR_VECTORS, 0, NR_VECTORS); + if (!loongarch_avec.vector_matrix) + return -ENOMEM; + + for (int i = 0; i < NR_LEGACY_VECTORS; i++) + irq_matrix_assign_system(loongarch_avec.vector_matrix, i, false); + + irq_matrix_online(loongarch_avec.vector_matrix); + + return 0; +} + +static int __init avecintc_init(struct irq_domain *parent) +{ + int ret, parent_irq; + unsigned long value; + + raw_spin_lock_init(&loongarch_avec.lock); + + loongarch_avec.fwnode = irq_domain_alloc_named_fwnode("AVECINTC"); + if (!loongarch_avec.fwnode) { + pr_err("Unable to allocate domain handle\n"); + ret = -ENOMEM; + goto out; + } + + loongarch_avec.domain = irq_domain_create_tree(loongarch_avec.fwnode, + &avecintc_domain_ops, NULL); + if (!loongarch_avec.domain) { + pr_err("Unable to create IRQ domain\n"); + ret = -ENOMEM; + goto out_free_handle; + } + + parent_irq = irq_create_mapping(parent, INT_AVEC); + if (!parent_irq) { + pr_err("Failed to mapping hwirq\n"); + ret = -EINVAL; + goto out_remove_domain; + } + + ret = irq_matrix_init(); + if (ret < 0) { + pr_err("Failed to init irq matrix\n"); + goto out_remove_domain; + } + irq_set_chained_handler_and_data(parent_irq, avecintc_irq_dispatch, NULL); + +#ifdef CONFIG_SMP + pending_list_init(0); + cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_AVECINTC_STARTING, + "irqchip/loongarch/avecintc:starting", + avecintc_cpu_online, avecintc_cpu_offline); +#endif + value = iocsr_read64(LOONGARCH_IOCSR_MISC_FUNC); + value |= IOCSR_MISC_FUNC_AVEC_EN; + iocsr_write64(value, LOONGARCH_IOCSR_MISC_FUNC); + + return ret; + +out_remove_domain: + irq_domain_remove(loongarch_avec.domain); +out_free_handle: + irq_domain_free_fwnode(loongarch_avec.fwnode); +out: + return ret; +} + +static int __init pch_msi_parse_madt(union acpi_subtable_headers *header, + const unsigned long end) +{ + struct acpi_madt_msi_pic *pchmsi_entry = (struct acpi_madt_msi_pic *)header; + + loongarch_avec.msi_base_addr = pchmsi_entry->msg_address - AVEC_MSG_OFFSET; + + return pch_msi_acpi_init_avec(loongarch_avec.domain); +} + +static inline int __init acpi_cascade_irqdomain_init(void) +{ + return acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1); +} + +int __init avecintc_acpi_init(struct irq_domain *parent) +{ + int ret = avecintc_init(parent); + + if (ret < 0) { + pr_err("Failed to init IRQ domain\n"); + return ret; + } + + ret = acpi_cascade_irqdomain_init(); + if (ret < 0) { + pr_err("Failed to init cascade IRQ domain\n"); + return ret; + } + + return ret; +} diff --git a/drivers/irqchip/irq-loongarch-cpu.c b/drivers/irqchip/irq-loongarch-cpu.c index 4380b4d8dd20..c6e0c9849ba9 100644 --- a/drivers/irqchip/irq-loongarch-cpu.c +++ b/drivers/irqchip/irq-loongarch-cpu.c @@ -13,6 +13,8 @@ #include <asm/loongarch.h> #include <asm/setup.h> +#include "irq-loongson.h" + static struct irq_domain *irq_domain; struct fwnode_handle *cpuintc_handle; @@ -140,7 +142,10 @@ static int __init acpi_cascade_irqdomain_init(void) if (r < 0) return r; - return 0; + if (cpu_has_avecint) + r = avecintc_acpi_init(irq_domain); + + return r; } struct irq_domain *get_cpudomain(void) diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c index 488951a740cd..1c7f97dac83b 100644 --- a/drivers/irqchip/irq-loongson-eiointc.c +++ b/drivers/irqchip/irq-loongson-eiointc.c @@ -16,6 +16,8 @@ #include <linux/kernel.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + #define EIOINTC_REG_NODEMAP 0x14a0 #define EIOINTC_REG_IPMAP 0x14c0 #define EIOINTC_REG_ENABLE 0x1600 @@ -395,6 +397,9 @@ static int __init acpi_cascade_irqdomain_init(void) if (r < 0) return r; + if (cpu_has_avecint) + return 0; + r = acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1); if (r < 0) return r; @@ -442,8 +447,8 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq, if (nr_pics == 1) { register_syscore_ops(&eiointc_syscore_ops); - cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_LOONGARCH_STARTING, - "irqchip/loongarch/intc:starting", + cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_EIOINTC_STARTING, + "irqchip/loongarch/eiointc:starting", eiointc_router_init, NULL); } diff --git a/drivers/irqchip/irq-loongson-htvec.c b/drivers/irqchip/irq-loongson-htvec.c index 0bff728b25e3..5da02c7ad0b3 100644 --- a/drivers/irqchip/irq-loongson-htvec.c +++ b/drivers/irqchip/irq-loongson-htvec.c @@ -17,6 +17,8 @@ #include <linux/of_irq.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define HTVEC_EN_OFF 0x20 #define HTVEC_MAX_PARENT_IRQ 8 diff --git a/drivers/irqchip/irq-loongson-liointc.c b/drivers/irqchip/irq-loongson-liointc.c index 0262cbefe9dd..257aa699ec98 100644 --- a/drivers/irqchip/irq-loongson-liointc.c +++ b/drivers/irqchip/irq-loongson-liointc.c @@ -22,6 +22,8 @@ #include <asm/loongson.h> #endif +#include "irq-loongson.h" + #define LIOINTC_CHIP_IRQ 32 #define LIOINTC_NUM_PARENT 4 #define LIOINTC_NUM_CORES 4 diff --git a/drivers/irqchip/irq-loongson-pch-lpc.c b/drivers/irqchip/irq-loongson-pch-lpc.c index 9b35492fb6be..2d4c3ec128b8 100644 --- a/drivers/irqchip/irq-loongson-pch-lpc.c +++ b/drivers/irqchip/irq-loongson-pch-lpc.c @@ -15,6 +15,8 @@ #include <linux/kernel.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define LPC_INT_CTL 0x00 #define LPC_INT_ENA 0x04 diff --git a/drivers/irqchip/irq-loongson-pch-msi.c b/drivers/irqchip/irq-loongson-pch-msi.c index dd4d699170f4..2c9f58536fce 100644 --- a/drivers/irqchip/irq-loongson-pch-msi.c +++ b/drivers/irqchip/irq-loongson-pch-msi.c @@ -15,6 +15,8 @@ #include <linux/pci.h> #include <linux/slab.h> +#include "irq-loongson.h" + static int nr_pics; struct pch_msi_data { @@ -266,17 +268,17 @@ IRQCHIP_DECLARE(pch_msi, "loongson,pch-msi-1.0", pch_msi_of_init); #ifdef CONFIG_ACPI struct fwnode_handle *get_pch_msi_handle(int pci_segment) { - int i; + if (cpu_has_avecint) + return pch_msi_handle[0]; - for (i = 0; i < MAX_IO_PICS; i++) { + for (int i = 0; i < MAX_IO_PICS; i++) { if (msi_group[i].pci_segment == pci_segment) return pch_msi_handle[i]; } - return NULL; + return pch_msi_handle[0]; } -int __init pch_msi_acpi_init(struct irq_domain *parent, - struct acpi_madt_msi_pic *acpi_pchmsi) +int __init pch_msi_acpi_init(struct irq_domain *parent, struct acpi_madt_msi_pic *acpi_pchmsi) { int ret; struct fwnode_handle *domain_handle; @@ -289,4 +291,36 @@ int __init pch_msi_acpi_init(struct irq_domain *parent, return ret; } + +static struct irq_chip pch_msi_irq_chip_avec = { + .name = "PCH PCI MSI", + .irq_ack = irq_chip_ack_parent, +}; + +static struct msi_domain_info pch_msi_domain_info_avec = { + .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | + MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + .chip = &pch_msi_irq_chip_avec, +}; + +int __init pch_msi_acpi_init_avec(struct irq_domain *parent) +{ + struct irq_domain *msi_domain; + + if (pch_msi_handle[0]) + return 0; + + pch_msi_handle[0] = parent->fwnode; + irq_domain_update_bus_token(parent, DOMAIN_BUS_NEXUS); + + msi_domain = pci_msi_create_irq_domain(pch_msi_handle[0], + &pch_msi_domain_info_avec, parent); + if (!msi_domain) { + pr_err("Failed to create PCI MSI domain\n"); + kfree(pch_msi_handle[0]); + return -ENOMEM; + } + + return 0; +} #endif diff --git a/drivers/irqchip/irq-loongson-pch-pic.c b/drivers/irqchip/irq-loongson-pch-pic.c index 6e4dde64687e..075d08aba75e 100644 --- a/drivers/irqchip/irq-loongson-pch-pic.c +++ b/drivers/irqchip/irq-loongson-pch-pic.c @@ -17,6 +17,8 @@ #include <linux/of_irq.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define PCH_PIC_MASK 0x20 #define PCH_PIC_HTMSI_EN 0x40 diff --git a/drivers/irqchip/irq-loongson.h b/drivers/irqchip/irq-loongson.h new file mode 100644 index 000000000000..11fa138d1f44 --- /dev/null +++ b/drivers/irqchip/irq-loongson.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Loongson Technology Corporation Limited + */ + +#ifndef _DRIVERS_IRQCHIP_IRQ_LOONGSON_H +#define _DRIVERS_IRQCHIP_IRQ_LOONGSON_H + +int find_pch_pic(u32 gsi); + +int liointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lio_pic *acpi_liointc); +int eiointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_eio_pic *acpi_eiointc); +int avecintc_acpi_init(struct irq_domain *parent); + +int htvec_acpi_init(struct irq_domain *parent, + struct acpi_madt_ht_pic *acpi_htvec); +int pch_lpc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lpc_pic *acpi_pchlpc); +int pch_pic_acpi_init(struct irq_domain *parent, + struct acpi_madt_bio_pic *acpi_pchpic); +int pch_msi_acpi_init(struct irq_domain *parent, + struct acpi_madt_msi_pic *acpi_pchmsi); +int pch_msi_acpi_init_avec(struct irq_domain *parent); + +#endif /* _DRIVERS_IRQCHIP_IRQ_LOONGSON_H */ diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index a33500a53d25..8f3b474f3a70 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -153,6 +153,10 @@ enum cpuhp_state { CPUHP_AP_IRQ_MIPS_GIC_STARTING, CPUHP_AP_IRQ_RISCV_STARTING, CPUHP_AP_IRQ_LOONGARCH_STARTING, +#ifdef CONFIG_LOONGARCH + CPUHP_AP_IRQ_EIOINTC_STARTING, + CPUHP_AP_IRQ_AVECINTC_STARTING, +#endif CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_MICROCODE_LOADER, -- 2.33.0

2 1

[PATCH OLK-6.6] LoongArch: add loongson SE SDF support
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Zhao Qunqin <zhaoqunqin(a)loongson.cn> LoongArch inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3GN -------------------------------- Signed-off-by: Zhao Qunqin <zhaoqunqin(a)loongson.cn> --- arch/loongarch/include/asm/se.h | 147 ++++++++ drivers/char/Kconfig | 20 ++ drivers/char/Makefile | 2 + drivers/char/loongson_se.c | 601 ++++++++++++++++++++++++++++++++ drivers/char/lsse_sdf_cdev.c | 380 ++++++++++++++++++++ 5 files changed, 1150 insertions(+) create mode 100644 arch/loongarch/include/asm/se.h create mode 100644 drivers/char/loongson_se.c create mode 100644 drivers/char/lsse_sdf_cdev.c diff --git a/arch/loongarch/include/asm/se.h b/arch/loongarch/include/asm/se.h new file mode 100644 index 000000000000..d0f9d43d60d2 --- /dev/null +++ b/arch/loongarch/include/asm/se.h @@ -0,0 +1,147 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2012 IBM Corporation + * + * Copyright 2023 Loongson Technology, Inc. + * Yinggang Gu <guyinggang(a)loongson.cn> + * + * Device driver for Loongson SE module. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation, version 2 of the + * License. + * + */ +#ifndef __LOONGSON_SE_H__ +#define __LOONGSON_SE_H__ + +#define SE_MAILBOX_S 0x0 +#define SE_MAILBOX_L 0x20 +#define SE_S2LINT_STAT 0x88 +#define SE_S2LINT_EN 0x8c +#define SE_S2LINT_SET 0x90 +#define SE_S2LINT_CL 0x94 +#define SE_L2SINT_STAT 0x98 +#define SE_L2SINT_EN 0x9c +#define SE_L2SINT_SET 0xa0 +#define SE_L2SINT_CL 0xa4 + +/* INT bit definition */ +#define SE_INT_SETUP BIT(0) +#define SE_INT_SM2 BIT(0) +#define SE_INT_SM3 BIT(0) +#define SE_INT_SM4 BIT(0) +#define SE_INT_RNG BIT(0) +#define SE_INT_TPM BIT(5) +#define SE_INT_ALL 0xffffffff + +#define SE_CMD_START 0x0 +#define SE_CMD_STOP 0x1 +#define SE_CMD_GETVER 0x2 +#define SE_CMD_SETBUF 0x3 +#define SE_CMD_SETMSG 0x4 + +#define SE_CMD_RNG 0x100 + +#define SE_CMD_SM2_SIGN 0x200 +#define SE_CMD_SM2_VSIGN 0x201 + +#define SE_CMD_SM3_DIGEST 0x300 +#define SE_CMD_SM3_UPDATE 0x301 +#define SE_CMD_SM3_FINISH 0x302 + +#define SE_CMD_SM4_ECB_ENCRY 0x400 +#define SE_CMD_SM4_ECB_DECRY 0x401 +#define SE_CMD_SM4_CBC_ENCRY 0x402 +#define SE_CMD_SM4_CBC_DECRY 0x403 +#define SE_CMD_SM4_CTR 0x404 + +#define SE_CMD_TPM 0x500 +#define SE_CMD_ZUC_INIT_READ 0x600 +#define SE_CMD_ZUC_READ 0x601 + +#define SE_CMD_SDF 0x700 + +#define SE_CH_MAX 32 + +#define SE_CH_RNG 1 +#define SE_CH_SM2 2 +#define SE_CH_SM3 3 +#define SE_CH_SM4 4 +#define SE_CH_TPM 5 +#define SE_CH_ZUC 6 +#define SE_CH_SDF 7 + +struct se_msg { + u32 cmd; + u32 data_off; + u32 data_len; + u32 info[5]; +}; + +struct se_cmd { + u32 cmd; + u32 info[7]; +}; + +struct se_res { + u32 cmd; + u32 cmd_ret; + u32 info[6]; +}; + +struct se_mailbox_data { + u32 int_bit; + union { + u32 mailbox[8]; + struct se_cmd gcmd; + struct se_res res; + } u; +}; + +struct lsse_ch { + u32 id; + u32 int_bit; + struct loongson_se *se; + void *priv; + spinlock_t ch_lock; + void *smsg; + void *rmsg; + int msg_size; + void *data_buffer; + dma_addr_t data_addr; + int data_size; + + void (*complete)(struct lsse_ch *se_ch); +}; + +struct loongson_se { + struct device *dev; + void __iomem *base; + u32 version; + u32 ch_status; + spinlock_t cmd_lock; + spinlock_t dev_lock; + + /* Interaction memory */ + void *mem_base; + dma_addr_t mem_addr; + unsigned long *mem_map; + int mem_map_size; + void *smsg; + void *rmsg; + + /* Synchronous CMD */ + struct completion cmd_completion; + + /* Virtual Channel */ + struct lsse_ch chs[SE_CH_MAX]; +}; + +struct lsse_ch *se_init_ch(int id, int data_size, int msg_size, void *priv, + void (*complete)(struct lsse_ch *se_ch)); +void se_deinit_ch(struct lsse_ch *ch); +int se_send_ch_requeset(struct lsse_ch *ch); + +#endif diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig index 625af75833fc..ea7ace87a9df 100644 --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -391,6 +391,26 @@ config UV_MMTIMER The uv_mmtimer device allows direct userspace access to the UV system timer. +config LOONGSON_SE + tristate "LOONGSON SECURITY MODULE Interface" + depends on LOONGARCH + default m + help + If you have LOONGSON security module (SE) support say Yes and it + will be accessible from within Linux. + To compile this driver as a module, choose M here,the module will + be called loongson-se. + +config LOONGSON_SE_SDF + tristate "LOONGSON SECURITY MODULE SDF Interface" + depends on LOONGARCH && LOONGSON_SE + default m + help + If you want to use LOONGSON security module (SE) as SDF say Yes + and it will be accessible from within Linux. + To compile this driver as a module, choose M here,the module will + be called loongson-se. + source "drivers/char/tpm/Kconfig" config TELCLOCK diff --git a/drivers/char/Makefile b/drivers/char/Makefile index c5f532e412f1..109af71c5416 100644 --- a/drivers/char/Makefile +++ b/drivers/char/Makefile @@ -32,6 +32,8 @@ obj-$(CONFIG_SCx200_GPIO) += scx200_gpio.o obj-$(CONFIG_PC8736x_GPIO) += pc8736x_gpio.o obj-$(CONFIG_NSC_GPIO) += nsc_gpio.o obj-$(CONFIG_TELCLOCK) += tlclk.o +obj-$(CONFIG_LOONGSON_SE) += loongson_se.o +obj-$(CONFIG_LOONGSON_SE_SDF) += lsse_sdf_cdev.o obj-$(CONFIG_MWAVE) += mwave/ obj-y += agp/ diff --git a/drivers/char/loongson_se.c b/drivers/char/loongson_se.c new file mode 100644 index 000000000000..40d2d6518265 --- /dev/null +++ b/drivers/char/loongson_se.c @@ -0,0 +1,601 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/kernel.h> +#include <linux/delay.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/errno.h> +#include <linux/cdev.h> +#include <linux/device.h> +#include <linux/of.h> +#include <linux/of_address.h> +#include <linux/platform_device.h> +#include <linux/miscdevice.h> +#include <linux/iopoll.h> +#include <linux/dma-mapping.h> +#include <linux/interrupt.h> +#include <asm/se.h> + +static int se_mem_size = 0x800000; +module_param(se_mem_size, int, 0444); +MODULE_PARM_DESC(se_mem_size, "LOONGSON SE shared memory size"); + +static int se_mem_page = PAGE_SIZE; +module_param(se_mem_page, int, 0444); +MODULE_PARM_DESC(se_mem_page, "LOONGSON SE shared memory page size"); + +static struct loongson_se se_dev; + +static int lsse_open(struct inode *inode, struct file *filp) +{ + return 0; +} + +static ssize_t lsse_write(struct file *filp, const char __user *buf, + size_t cnt, loff_t *offt) +{ + return 0; +} + +static const struct file_operations lsse_fops = { + .owner = THIS_MODULE, + .open = lsse_open, + .write = lsse_write, +}; + +static const struct miscdevice lsse_miscdev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "loongson-se", + .fops = &lsse_fops, +}; + +static inline u32 se_readl(u64 addr) +{ + return readl(se_dev.base + addr); +} + +static inline void se_writel(u32 val, u64 addr) +{ + writel(val, se_dev.base + addr); +} + +static inline bool se_ch_status(struct loongson_se *se, u32 int_bit) +{ + return !!(se->ch_status & int_bit) == 1; +} + +static void se_enable_int(struct loongson_se *se, u32 int_bit) +{ + unsigned long flag; + u32 tmp; + + if (!int_bit) + return; + + spin_lock_irqsave(&se->dev_lock, flag); + + tmp = se_readl(SE_S2LINT_EN); + tmp |= int_bit; + se_writel(tmp, SE_S2LINT_EN); + + spin_unlock_irqrestore(&se->dev_lock, flag); +} + +static void se_disable_int(struct loongson_se *se, u32 int_bit) +{ + unsigned long flag; + u32 tmp; + + if (!int_bit) + return; + + spin_lock_irqsave(&se->dev_lock, flag); + + tmp = se_readl(SE_S2LINT_EN); + tmp &= ~(int_bit); + se_writel(tmp, SE_S2LINT_EN); + + spin_unlock_irqrestore(&se->dev_lock, flag); +} + +static int se_send_requeset(struct loongson_se *se, + struct se_mailbox_data *req) +{ + unsigned long flag; + u32 status; + int err = 0; + int i; + + if (!se || !req) + return -EINVAL; + + if (se_readl(SE_L2SINT_STAT) || + !(se_readl(SE_L2SINT_EN) & req->int_bit)) + return -EBUSY; + + spin_lock_irqsave(&se->cmd_lock, flag); + + for (i = 0; i < ARRAY_SIZE(req->u.mailbox); i++) + se_writel(req->u.mailbox[i], SE_MAILBOX_S + i * 4); + + se_writel(req->int_bit, SE_L2SINT_SET); + + err = readl_relaxed_poll_timeout_atomic(se->base + SE_L2SINT_STAT, status, + !(status & req->int_bit), 10, 10000); + + spin_unlock_irqrestore(&se->cmd_lock, flag); + + return err; +} + +static int se_get_response(struct loongson_se *se, + struct se_mailbox_data *res) +{ + unsigned long flag; + int i; + + if (!se || !res) + return -EINVAL; + + if ((se_readl(SE_S2LINT_STAT) & res->int_bit) == 0) + return -EBUSY; + + spin_lock_irqsave(&se->cmd_lock, flag); + + for (i = 0; i < ARRAY_SIZE(res->u.mailbox); i++) + res->u.mailbox[i] = se_readl(SE_MAILBOX_L + i * 4); + + se_writel(res->int_bit, SE_S2LINT_CL); + + spin_unlock_irqrestore(&se->cmd_lock, flag); + + return 0; +} + +static int loongson_se_get_res(struct loongson_se *se, u32 int_bit, u32 cmd, + struct se_mailbox_data *res) +{ + int err = 0; + + res->int_bit = int_bit; + + if (se_get_response(se, res)) { + dev_err(se->dev, "Int 0x%x get response fail.\n", int_bit); + return -EFAULT; + } + + /* Check response */ + if (res->u.res.cmd == cmd) + err = 0; + else { + dev_err(se->dev, "Response cmd is 0x%x, not expect cmd 0x%x.\n", + res->u.res.cmd, cmd); + err = -EFAULT; + } + + return err; +} + +static int se_send_genl_cmd(struct loongson_se *se, struct se_mailbox_data *req, + struct se_mailbox_data *res, int retry) +{ + int err = 0, cnt = 0; + +try_again: + if (cnt++ >= retry) { + err = -ETIMEDOUT; + goto out; + } + + dev_dbg(se->dev, "%d time send cmd 0x%x\n", cnt, req->u.gcmd.cmd); + + err = se_send_requeset(se, req); + if (err) + goto try_again; + + if (!wait_for_completion_timeout(&se->cmd_completion, + msecs_to_jiffies(0x1000))) { + se_enable_int(se, req->int_bit); + goto try_again; + } + + err = loongson_se_get_res(se, req->int_bit, req->u.gcmd.cmd, res); + if (err || res->u.res.cmd_ret) { + se_enable_int(se, req->int_bit); + goto try_again; + } + +out: + se_enable_int(se, req->int_bit); + + return err; +} + +static int loongson_se_set_msg(struct lsse_ch *ch) +{ + struct loongson_se *se = ch->se; + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + int err; + + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_SETMSG; + /* MSG off */ + req.u.gcmd.info[0] = ch->id; + req.u.gcmd.info[1] = ch->smsg - se->mem_base; + req.u.gcmd.info[2] = ch->msg_size; + + dev_dbg(se->dev, "Set Channel %d msg off 0x%x, msg size %d\n", ch->id, + req.u.gcmd.info[1], req.u.gcmd.info[2]); + + err = se_send_genl_cmd(se, &req, &res, 5); + if (res.u.res.cmd_ret) + return res.u.res.cmd_ret; + + return err; +} + +static irqreturn_t loongson_se_irq(int irq, void *dev_id) +{ + struct loongson_se *se = (struct loongson_se *)dev_id; + struct lsse_ch *ch; + u32 int_status; + + int_status = se_readl(SE_S2LINT_STAT); + + dev_dbg(se->dev, "%s int status is 0x%x\n", __func__, int_status); + + se_disable_int(se, int_status); + + if (int_status & SE_INT_SETUP) { + complete(&se->cmd_completion); + int_status &= ~SE_INT_SETUP; + } + + while (int_status) { + int id = __ffs(int_status); + + ch = &se->chs[id]; + if (ch->complete) + ch->complete(ch); + int_status &= ~BIT(id); + se_writel(BIT(id), SE_S2LINT_CL); + } + + return IRQ_HANDLED; +} + +static int se_init_hw(struct loongson_se *se) +{ + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + struct device *dev = se->dev; + int err, retry = 5; + u64 size; + + size = se_mem_size; + + if (size & (size - 1)) { + size = roundup_pow_of_two(size); + se_mem_size = size; + } + + se_enable_int(se, SE_INT_SETUP); + + /* Start engine */ + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_START; + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + /* Get Version */ + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_GETVER; + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + se->version = res.u.res.info[0]; + + /* Setup data buffer */ + se->mem_base = dmam_alloc_coherent(dev, size, + &se->mem_addr, GFP_KERNEL); + if (!se->mem_base) + return -ENOMEM; + + memset(se->mem_base, 0, size); + + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_SETBUF; + /* MMAP */ + req.u.gcmd.info[0] = (se->mem_addr & 0xffffffff) | 0x80; + req.u.gcmd.info[1] = se->mem_addr >> 32; + /* MASK */ + req.u.gcmd.info[2] = ~(size - 1); + req.u.gcmd.info[3] = 0xffffffff; + + pr_debug("Set win mmap 0x%llx, mask 0x%llx\n", + ((u64)req.u.gcmd.info[1] << 32) | req.u.gcmd.info[0], + ((u64)req.u.gcmd.info[3] << 32) | req.u.gcmd.info[2]); + + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + se->mem_map_size = size / se_mem_page; + se->mem_map = bitmap_zalloc(se->mem_map_size, GFP_KERNEL); + if (!se->mem_map) + return -ENOMEM; + + dev_info(se->dev, "SE module setup down, shared memory size is 0x%x bytes, memory page size is 0x%x bytes\n", + se_mem_size, se_mem_page); + + return err; +} + +static void loongson_se_disable_hw(struct loongson_se *se) +{ + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + int retry = 5; + + /* Stop engine */ + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_STOP; + se_send_genl_cmd(se, &req, &res, retry); + + se_disable_int(se, SE_INT_ALL); + kfree(se->mem_map); +} + +int se_send_ch_requeset(struct lsse_ch *ch) +{ + struct loongson_se *se; + u32 status, int_bit; + int err = 0; + + if (!ch) + return -EINVAL; + + se = ch->se; + int_bit = ch->int_bit; + + if ((se_readl(SE_L2SINT_STAT) & int_bit) || + !(se_readl(SE_L2SINT_EN) & int_bit)) + return -EBUSY; + + se_enable_int(se, int_bit); + se_writel(int_bit, SE_L2SINT_SET); + + err = readl_relaxed_poll_timeout_atomic(se->base + SE_L2SINT_STAT, status, + !(status & int_bit), 10, 10000); + + return err; +} +EXPORT_SYMBOL_GPL(se_send_ch_requeset); + +struct lsse_ch *se_init_ch(int id, int data_size, int msg_size, void *priv, + void (*complete)(struct lsse_ch *se_ch)) +{ + struct loongson_se *se = &se_dev; + struct lsse_ch *ch; + unsigned long flag; + int data_first, data_nr; + int msg_first, msg_nr; + + if (!se) { + pr_err("SE has bot been initialized\n"); + return NULL; + } + + if (id == 0 || id > SE_CH_MAX) { + dev_err(se->dev, "Channel number %d is invalid\n", id); + return NULL; + } + + if (se_ch_status(se, BIT(id))) { + dev_err(se->dev, "Channel number %d has been initialized\n", id); + return NULL; + } + + spin_lock_irqsave(&se->dev_lock, flag); + + ch = &se_dev.chs[id]; + ch->se = se; + ch->id = id; + ch->int_bit = BIT(id); + se->ch_status |= BIT(id); + + data_nr = round_up(data_size, se_mem_page) / se_mem_page; + data_first = bitmap_find_next_zero_area(se->mem_map, se->mem_map_size, + 0, data_nr, 0); + if (data_first >= se->mem_map_size) { + dev_err(se->dev, "Insufficient memory space\n"); + spin_unlock_irqrestore(&se->dev_lock, flag); + return NULL; + } + + bitmap_set(se->mem_map, data_first, data_nr); + ch->data_buffer = se->mem_base + data_first * se_mem_page; + ch->data_addr = se->mem_addr + data_first * se_mem_page; + ch->data_size = data_size; + + msg_nr = round_up(msg_size, se_mem_page) / se_mem_page; + msg_first = bitmap_find_next_zero_area(se->mem_map, se->mem_map_size, + 0, msg_nr, 0); + if (msg_first >= se->mem_map_size) { + dev_err(se->dev, "Insufficient memory space\n"); + bitmap_clear(se->mem_map, data_first, data_nr); + spin_unlock_irqrestore(&se->dev_lock, flag); + return NULL; + } + + bitmap_set(se->mem_map, msg_first, msg_nr); + ch->smsg = se->mem_base + msg_first * se_mem_page; + ch->rmsg = ch->smsg + msg_size / 2; + ch->msg_size = msg_size; + + ch->complete = complete; + ch->priv = priv; + + spin_lock_init(&ch->ch_lock); + + spin_unlock_irqrestore(&se->dev_lock, flag); + + if (loongson_se_set_msg(ch)) { + dev_err(se->dev, "Channel %d setup message address failed\n", id); + return NULL; + } + + se_enable_int(se, ch->int_bit); + + return ch; +} +EXPORT_SYMBOL_GPL(se_init_ch); + +void se_deinit_ch(struct lsse_ch *ch) +{ + struct loongson_se *se = &se_dev; + unsigned long flag; + int first, nr; + int id = ch->id; + + if (!se) { + pr_err("SE has bot been initialized\n"); + return; + } + + if (id == 0 || id > SE_CH_MAX) { + dev_err(se->dev, "Channel number %d is invalid\n", id); + return; + } + + if (!se_ch_status(se, BIT(id))) { + dev_err(se->dev, "Channel number %d has not been initialized\n", id); + return; + } + + spin_lock_irqsave(&se->dev_lock, flag); + + se->ch_status &= ~BIT(ch->id); + + first = (ch->data_buffer - se->mem_base) / se_mem_page; + nr = round_up(ch->data_size, se_mem_page) / se_mem_page; + bitmap_clear(se->mem_map, first, nr); + + first = (ch->smsg - se->mem_base) / se_mem_page; + nr = round_up(ch->msg_size, se_mem_page) / se_mem_page; + bitmap_clear(se->mem_map, first, nr); + + spin_unlock_irqrestore(&se->dev_lock, flag); + + se_disable_int(se, ch->int_bit); +} +EXPORT_SYMBOL_GPL(se_deinit_ch); + +static struct platform_device lsse_sdf_pdev = { + .name = "loongson-sdf", + .id = -1, +}; + +static const struct of_device_id loongson_se_of_match[] = { + { .compatible = "loongson,ls3c6000se", }, + {} +}; +MODULE_DEVICE_TABLE(of, loongson_se_of_match); + +static int loongson_se_probe(struct platform_device *pdev) +{ + struct loongson_se *se = &se_dev; + struct resource *res; + struct device *dev = &pdev->dev; + int nr_irq, err, i; + int irq[8]; + + nr_irq = platform_irq_count(pdev); + if (nr_irq < 0) + return -ENODEV; + + for (i = 0; i < nr_irq; i++) { + irq[i] = platform_get_irq(pdev, i); + if (irq[i] < 0) + return -ENODEV; + } + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -ENODEV; + + se->base = devm_ioremap_resource(dev, res); + if (IS_ERR(se->base)) + return PTR_ERR(se->base); + + se->dev = &pdev->dev; + platform_set_drvdata(pdev, se); + dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); + init_completion(&se->cmd_completion); + spin_lock_init(&se->cmd_lock); + spin_lock_init(&se->dev_lock); + + for (i = 0; i < nr_irq; i++) { + err = devm_request_irq(dev, irq[i], loongson_se_irq, 0, + "loongson-se", se); + if (err) + goto out; + } + + err = se_init_hw(se); + if (err) + goto disable_hw; + + err = misc_register(&lsse_miscdev); + if (err) + goto disable_hw; + + err = platform_device_register(&lsse_sdf_pdev); + if (err) + pr_err("register sdf device failed\n"); + + return 0; + +disable_hw: + loongson_se_disable_hw(se); +out: + for ( ; i >= 0; i--) + devm_free_irq(dev, irq[i], se); + + return err; +} + +static int loongson_se_remove(struct platform_device *pdev) +{ + struct loongson_se *se = platform_get_drvdata(pdev); + + misc_deregister(&lsse_miscdev); + loongson_se_disable_hw(se); + platform_device_unregister(&lsse_sdf_pdev); + + return 0; +} + +static struct platform_driver loongson_se_driver = { + .probe = loongson_se_probe, + .remove = loongson_se_remove, + .driver = { + .name = "loongson-se", + .of_match_table = loongson_se_of_match, + }, +}; + +module_platform_driver(loongson_se_driver); + +MODULE_AUTHOR("Yinggang Gu"); +MODULE_DESCRIPTION("Loongson SE driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/char/lsse_sdf_cdev.c b/drivers/char/lsse_sdf_cdev.c new file mode 100644 index 000000000000..a4806fbf08d1 --- /dev/null +++ b/drivers/char/lsse_sdf_cdev.c @@ -0,0 +1,380 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/kernel.h> +#include <linux/slab.h> +#include <linux/init.h> +#include <linux/miscdevice.h> +#include <linux/module.h> +#include <linux/cdev.h> +#include <linux/module.h> +#include <linux/wait.h> +#include <linux/of.h> +#include <linux/iopoll.h> +#include <linux/platform_device.h> +#include <linux/uaccess.h> +#include <asm/se.h> +#include <linux/list.h> + +#define SE_SDF_BUFSIZE (PAGE_SIZE * 2) +#define SDF_OPENSESSION 0x204 +#define SDF_CLOSESESSION 0x205 + +struct lsse_sdf_dev { + struct lsse_ch *se_ch; + struct mutex data_lock; + bool processing_cmd; + + /* Synchronous CMD */ + wait_queue_head_t wq; +}; + +struct se_sdf_msg { + u32 cmd; + u32 data_off; + u32 data_len; + u32 info[5]; +}; + +struct sdf_command_header { + int command; + union { + int param_cnt; + int ret; + } u; + int param_len[14]; +}; + +struct sdf_kernel_command { + struct sdf_command_header header; + void *handle; +}; + +#define KERNEL_COMMAND_SIZE (sizeof(struct sdf_kernel_command)) + +struct sdf_handle { + struct list_head handle_list; + void *handle; +}; + +struct sdf_file_pvt_data { + struct lsse_sdf_dev *se; + struct list_head handle_list; + struct sdf_kernel_command skc; + struct sdf_handle *ph; +}; + +static struct lsse_sdf_dev *se_sdf_dev; + +static void lsse_sdf_complete(struct lsse_ch *ch) +{ + struct lsse_sdf_dev *se = (struct lsse_sdf_dev *)ch->priv; + + se->processing_cmd = false; + wake_up(&se->wq); +} + +static int se_send_sdf_cmd(struct lsse_sdf_dev *se, int len, int retry) +{ + struct se_sdf_msg *smsg = (struct se_sdf_msg *)se->se_ch->smsg; + unsigned long flag; + int err; + + spin_lock_irqsave(&se->se_ch->ch_lock, flag); + + smsg->cmd = SE_CMD_SDF; + /* One time one cmd */ + smsg->data_off = se->se_ch->data_buffer - se->se_ch->se->mem_base; + smsg->data_len = len; + +try_again: + if (!retry--) + goto out; + + pr_debug("Send sdf cmd, last retry %d times\n", retry); + + err = se_send_ch_requeset(se->se_ch); + if (err) { + udelay(5); + goto try_again; + } + +out: + spin_unlock_irqrestore(&se->se_ch->ch_lock, flag); + + return err; +} + +static int lsse_sdf_recv(struct sdf_file_pvt_data *pvt, char *buf, + size_t size, int user, int *se_ret) +{ + int len, time, ret = 0; + struct se_sdf_msg *rmsg; + struct sdf_kernel_command *skc; + struct sdf_handle *ph; + struct lsse_sdf_dev *se = pvt->se; + + if (!se->se_ch->rmsg) { + pr_err("se device is not ready\n"); + return -EBUSY; + } + + time = wait_event_timeout(se->wq, !se->processing_cmd, HZ*30); + if (!time) + return -ETIME; + + rmsg = (struct se_sdf_msg *)se->se_ch->rmsg; + if (rmsg->cmd != SE_CMD_SDF) { + pr_err("se get wrong response\n"); + return -EIO; + } + len = rmsg->data_len; + + if ((!user && len > KERNEL_COMMAND_SIZE) || len > SE_SDF_BUFSIZE + || (size && len > size)) + return -E2BIG; + + if (user) { + ret = copy_to_user((char __user *)buf, + se->se_ch->data_buffer + rmsg->data_off, len); + if (!se_ret) + return ret; + + skc = (struct sdf_kernel_command *) + (se->se_ch->data_buffer + rmsg->data_off); + *se_ret = skc->header.u.ret; + if (skc->header.command == SDF_OPENSESSION && !*se_ret) { + ph = kmalloc(sizeof(*ph), GFP_KERNEL); + if (!ph) + return -ENOMEM; + ph->handle = skc->handle; + list_add(&ph->handle_list, &pvt->handle_list); + } + } else + memcpy(buf, se->se_ch->data_buffer + rmsg->data_off, len); + + return ret; +} + +static struct sdf_handle *find_sdf_handle(void *handle, + struct sdf_file_pvt_data *pvt) +{ + struct sdf_handle *ph; + + list_for_each_entry(ph, &pvt->handle_list, handle_list) { + if (ph->handle == handle) + return ph; + } + + return NULL; +} + +static int lsse_sdf_send(struct sdf_file_pvt_data *pvt, const char *buf, + size_t count, int user) +{ + int ret, se_ret; + struct sdf_handle *ph = NULL; + struct sdf_kernel_command *skc; + struct lsse_sdf_dev *se = pvt->se; + + if (!se->se_ch->smsg) { + pr_err("se device is not ready\n"); + return 0; + } + + if (count > se->se_ch->data_size) { + pr_err("Invalid size in send: count=%zd, size=%d\n", + count, se->se_ch->data_size); + return -EIO; + } + + if (user) { + ret = mutex_lock_interruptible(&se->data_lock); + if (ret) + goto out; + } else + mutex_lock(&se->data_lock); + + if (user) { + ret = copy_from_user(se->se_ch->data_buffer, buf, count); + if (ret) { + ret = -EFAULT; + goto out_unlock; + } + skc = (struct sdf_kernel_command *)se->se_ch->data_buffer; + if (skc->header.command == SDF_CLOSESESSION) + ph = find_sdf_handle(skc->handle, pvt); + } else + memcpy(se->se_ch->data_buffer, buf, count); + + se->processing_cmd = true; + ret = se_send_sdf_cmd(se, count, 5); + if (ret) { + pr_err("se_send_sdf_cmd failed\n"); + goto out_unlock; + } + + ret = lsse_sdf_recv(pvt, (char *)buf, 0, user, &se_ret); + if (ret) { + pr_err("recv failed ret: %x\n", ret); + goto out_unlock; + } + if (ph && !se_ret) { + list_del(&ph->handle_list); + kfree(ph); + } +out_unlock: + mutex_unlock(&se->data_lock); +out: + return ret; +} + +static ssize_t lsse_sdf_write(struct file *filp, const char __user *buf, + size_t cnt, loff_t *offt) +{ + struct sdf_file_pvt_data *pvt = filp->private_data; + + if (cnt > SE_SDF_BUFSIZE) + return -E2BIG; + + if (lsse_sdf_send(pvt, buf, cnt, 1)) + return -EFAULT; + + return cnt; +} + +static ssize_t lsse_sdf_read(struct file *filp, char __user *buf, + size_t size, loff_t *off) +{ + return lsse_sdf_recv(filp->private_data, buf, size, 1, NULL); +} + +static int close_one_handle(struct sdf_file_pvt_data *pvt, struct sdf_handle *ph) +{ + struct sdf_kernel_command *skc = &pvt->skc; + + skc->header.command = 0x205; + skc->header.u.param_cnt = 1; + skc->header.param_len[0] = 8; + skc->handle = ph->handle; + /* close one session */ + lsse_sdf_send(pvt, (char *)&pvt->skc, KERNEL_COMMAND_SIZE, 0); + if (skc->header.u.ret) { + pr_err("Auto Close Session failed, session handle: %llx, ret: %d\n", + (u64)ph->handle, skc->header.u.ret); + return skc->header.u.ret; + } + kfree(ph); + + return 0; +} + +static int close_all_handle(struct sdf_file_pvt_data *pvt) +{ + int ret = 0; + struct sdf_handle *ph, *tmp; + + list_for_each_entry_safe(ph, tmp, &pvt->handle_list, handle_list) { + list_del(&ph->handle_list); + ret = close_one_handle(pvt, ph); + if (ret) + return ret; + } + + return 0; +} + +static int lsse_sdf_release(struct inode *inode, struct file *filp) +{ + int ret; + struct sdf_file_pvt_data *pvt = filp->private_data; + + ret = close_all_handle(pvt); + filp->private_data = NULL; + kfree(pvt); + + if (ret) + ret = -EFAULT; + return ret; +} + +static int lsse_sdf_open(struct inode *inode, struct file *filp) +{ + struct sdf_file_pvt_data *pvt = kmalloc(sizeof(*pvt), GFP_KERNEL); + + if (!pvt) + return -ENOMEM; + + INIT_LIST_HEAD(&pvt->handle_list); + pvt->se = se_sdf_dev; + filp->private_data = pvt; + + return 0; +} + +static const struct file_operations lsse_sdf_fops = { + .owner = THIS_MODULE, + .open = lsse_sdf_open, + .write = lsse_sdf_write, + .read = lsse_sdf_read, + .release = lsse_sdf_release, +}; + +static const struct miscdevice lsse_sdf_miscdev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "lsse_sdf", + .fops = &lsse_sdf_fops, +}; + +static int lsse_sdf_probe(struct platform_device *pdev) +{ + int msg_size; + int ret; + + se_sdf_dev = kzalloc(sizeof(*se_sdf_dev), GFP_KERNEL); + if (IS_ERR_OR_NULL(se_sdf_dev)) + return PTR_ERR(se_sdf_dev); + + mutex_init(&se_sdf_dev->data_lock); + init_waitqueue_head(&se_sdf_dev->wq); + se_sdf_dev->processing_cmd = false; + + msg_size = 2 * sizeof(struct se_sdf_msg); + se_sdf_dev->se_ch = se_init_ch(SE_CH_SDF, SE_SDF_BUFSIZE, msg_size, + se_sdf_dev, lsse_sdf_complete); + + ret = misc_register(&lsse_sdf_miscdev); + if (ret < 0) { + pr_err("register sdf dev failed!\n"); + goto out; + } + + return 0; + +out: + kfree(se_sdf_dev); + + return ret; +} + +static int lsse_sdf_remove(struct platform_device *pdev) +{ + misc_deregister(&lsse_sdf_miscdev); + se_deinit_ch(se_sdf_dev->se_ch); + kfree(se_sdf_dev); + + return 0; +} + +static struct platform_driver loongson_sdf_driver = { + .probe = lsse_sdf_probe, + .remove = lsse_sdf_remove, + .driver = { + .name = "loongson-sdf", + }, +}; +module_platform_driver(loongson_sdf_driver); + +MODULE_ALIAS("platform:loongson-sdf"); +MODULE_AUTHOR("Yinggang Gu"); +MODULE_DESCRIPTION("Loongson SE sdf driver"); +MODULE_LICENSE("GPL"); -- 2.33.0

2 1

[PATCH OLK-6.6 0/4] LoongArch: fix some 3C6000&LS7A related problems
by Hongchen Zhang 23 Oct '24

23 Oct '24

gaojuxin (1): LoongArch: Add workaround for 3C6000 about io wr/rd wusheng (1): pci/quirks: LS7A2000 enble msi zhangtianyang (1): Loongarch: Dynamic enable writecombine zhenghaowei (1): acpi: mcfg quirk: Increased multi-chip support for the 3C6000 arch/loongarch/include/asm/irq.h | 4 +- arch/loongarch/kernel/setup.c | 30 ++++++++++- arch/loongarch/kernel/smp.c | 20 ++++++- drivers/acpi/pci_mcfg.c | 13 +++++ drivers/pci/quirks.c | 13 +++++ include/asm-generic/io.h | 90 ++++++++++++++++++++++++++++++-- include/drm/drm_cache.h | 2 +- 7 files changed, 162 insertions(+), 10 deletions(-) -- 2.33.0

2 5

[PATCH OLK-6.6] LoongArch: prevent LS7A Bus Master clearing on kexec
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Ming Wang <wangming01(a)loongson.cn> LoongArch inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3M1 -------------------------------- This is similar to commit 62b6dee1b44a ("PCI/portdrv: Prevent LS7A Bus Master clearing on shutdown"), which prevents LS7A Bus Master clearing on kexec. The key point of this is to work around the LS7A defect that clearing PCI_COMMAND_MASTER prevents MMIO requests from going downstream, and we may need to do that even after .shutdown(), e.g., to print console messages. And in this case we rely on .shutdown() for the downstream devices to disable interrupts and DMA. Only skip Bus Master clearing on bridges because endpoint devices still need it. Signed-off-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Ming Wang <wangming01(a)loongson.cn> --- drivers/pci/pci-driver.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 9c59bf03d657..b699839a7d4f 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -526,7 +526,11 @@ static void pci_device_shutdown(struct device *dev) * If it is not a kexec reboot, firmware will hit the PCI * devices with big hammer and stop their DMA any way. */ +#ifdef CONFIG_LOONGARCH + if (kexec_in_progress && !pci_is_bridge(pci_dev) && (pci_dev->current_state <= PCI_D3hot)) +#else if (kexec_in_progress && (pci_dev->current_state <= PCI_D3hot)) +#endif pci_clear_master(pci_dev); } -- 2.33.0

2 1

[PATCH OLK-6.6] LoongArch: add loongson SE SDF support
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Zhao Qunqin <zhaoqunqin(a)loongson.cn> LoongArch inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3GN -------------------------------- Signed-off-by: Zhao Qunqin <zhaoqunqin(a)loongson.cn> --- arch/loongarch/include/asm/se.h | 146 ++++++++ drivers/char/Kconfig | 18 + drivers/char/Makefile | 2 + drivers/char/loongson_se.c | 598 ++++++++++++++++++++++++++++++++ drivers/char/lsse_sdf_cdev.c | 381 ++++++++++++++++++++ 5 files changed, 1145 insertions(+) create mode 100644 arch/loongarch/include/asm/se.h create mode 100644 drivers/char/loongson_se.c create mode 100644 drivers/char/lsse_sdf_cdev.c diff --git a/arch/loongarch/include/asm/se.h b/arch/loongarch/include/asm/se.h new file mode 100644 index 000000000000..a6b968d2d545 --- /dev/null +++ b/arch/loongarch/include/asm/se.h @@ -0,0 +1,146 @@ +/* + * Copyright (C) 2012 IBM Corporation + * + * Copyright 2023 Loongson Technology, Inc. + * Yinggang Gu <guyinggang(a)loongson.cn> + * + * Device driver for Loongson SE module. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation, version 2 of the + * License. + * + */ +#ifndef __LOONGSON_SE_H__ +#define __LOONGSON_SE_H__ + +#define SE_MAILBOX_S 0x0 +#define SE_MAILBOX_L 0x20 +#define SE_S2LINT_STAT 0x88 +#define SE_S2LINT_EN 0x8c +#define SE_S2LINT_SET 0x90 +#define SE_S2LINT_CL 0x94 +#define SE_L2SINT_STAT 0x98 +#define SE_L2SINT_EN 0x9c +#define SE_L2SINT_SET 0xa0 +#define SE_L2SINT_CL 0xa4 + +/* INT bit definition */ +#define SE_INT_SETUP BIT(0) +#define SE_INT_SM2 BIT(0) +#define SE_INT_SM3 BIT(0) +#define SE_INT_SM4 BIT(0) +#define SE_INT_RNG BIT(0) +#define SE_INT_TPM BIT(5) +#define SE_INT_ALL 0xffffffff + +#define SE_CMD_START 0x0 +#define SE_CMD_STOP 0x1 +#define SE_CMD_GETVER 0x2 +#define SE_CMD_SETBUF 0x3 +#define SE_CMD_SETMSG 0x4 + +#define SE_CMD_RNG 0x100 + +#define SE_CMD_SM2_SIGN 0x200 +#define SE_CMD_SM2_VSIGN 0x201 + +#define SE_CMD_SM3_DIGEST 0x300 +#define SE_CMD_SM3_UPDATE 0x301 +#define SE_CMD_SM3_FINISH 0x302 + +#define SE_CMD_SM4_ECB_ENCRY 0x400 +#define SE_CMD_SM4_ECB_DECRY 0x401 +#define SE_CMD_SM4_CBC_ENCRY 0x402 +#define SE_CMD_SM4_CBC_DECRY 0x403 +#define SE_CMD_SM4_CTR 0x404 + +#define SE_CMD_TPM 0x500 +#define SE_CMD_ZUC_INIT_READ 0x600 +#define SE_CMD_ZUC_READ 0x601 + +#define SE_CMD_SDF 0x700 + +#define SE_CH_MAX 32 + +#define SE_CH_RNG 1 +#define SE_CH_SM2 2 +#define SE_CH_SM3 3 +#define SE_CH_SM4 4 +#define SE_CH_TPM 5 +#define SE_CH_ZUC 6 +#define SE_CH_SDF 7 + +struct se_msg { + u32 cmd; + u32 data_off; + u32 data_len; + u32 info[5]; +}; + +struct se_cmd { + u32 cmd; + u32 info[7]; +}; + +struct se_res { + u32 cmd; + u32 cmd_ret; + u32 info[6]; +}; + +struct se_mailbox_data { + u32 int_bit; + union { + u32 mailbox[8]; + struct se_cmd gcmd; + struct se_res res; + } u; +}; + +struct lsse_ch { + u32 id; + u32 int_bit; + struct loongson_se *se; + void *priv; + spinlock_t ch_lock; + void *smsg; + void *rmsg; + int msg_size; + void *data_buffer; + dma_addr_t data_addr; + int data_size; + + void (*complete)(struct lsse_ch *se_ch); +}; + +struct loongson_se { + struct device *dev; + void __iomem *base; + u32 version; + u32 ch_status; + spinlock_t cmd_lock; + spinlock_t dev_lock; + + /* Interaction memory */ + void *mem_base; + dma_addr_t mem_addr; + unsigned long *mem_map; + int mem_map_size; + void *smsg; + void *rmsg; + + /* Synchronous CMD */ + struct completion cmd_completion; + + /* Virtual Channel */ + struct lsse_ch chs[SE_CH_MAX]; +}; + +struct lsse_ch *se_init_ch(int id, int data_size, int msg_size, void *priv, + void (*complete)(struct lsse_ch *se_ch)); +void se_deinit_ch(struct lsse_ch *ch); +int se_send_ch_requeset(struct lsse_ch *ch); + +#endif diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig index 625af75833fc..8a5e272bdc78 100644 --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -391,6 +391,24 @@ config UV_MMTIMER The uv_mmtimer device allows direct userspace access to the UV system timer. +config LOONGSON_SE + tristate "LOONGSON SECURITY MODULE Interface" + depends on LOONGARCH + default m + help + If you have LOONGSON security module (SE) support say Yes and it + will be accessible from within Linux. To compile this driver + as a module, choose M here; the module will be called loongson-se. + +config LOONGSON_SE_SDF + tristate "LOONGSON SECURITY MODULE SDF Interface" + depends on LOONGARCH && LOONGSON_SE + default m + help + If you want to use LOONGSON security module (SE) as SDF say Yes + and it will be accessible from within Linux. To compile this driver + as a module, choose M here; + source "drivers/char/tpm/Kconfig" config TELCLOCK diff --git a/drivers/char/Makefile b/drivers/char/Makefile index c5f532e412f1..109af71c5416 100644 --- a/drivers/char/Makefile +++ b/drivers/char/Makefile @@ -32,6 +32,8 @@ obj-$(CONFIG_SCx200_GPIO) += scx200_gpio.o obj-$(CONFIG_PC8736x_GPIO) += pc8736x_gpio.o obj-$(CONFIG_NSC_GPIO) += nsc_gpio.o obj-$(CONFIG_TELCLOCK) += tlclk.o +obj-$(CONFIG_LOONGSON_SE) += loongson_se.o +obj-$(CONFIG_LOONGSON_SE_SDF) += lsse_sdf_cdev.o obj-$(CONFIG_MWAVE) += mwave/ obj-y += agp/ diff --git a/drivers/char/loongson_se.c b/drivers/char/loongson_se.c new file mode 100644 index 000000000000..5c010b2e8d8c --- /dev/null +++ b/drivers/char/loongson_se.c @@ -0,0 +1,598 @@ +#include <linux/kernel.h> +#include <linux/delay.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/errno.h> +#include <linux/cdev.h> +#include <linux/device.h> +#include <linux/of.h> +#include <linux/of_address.h> +#include <linux/platform_device.h> +#include <linux/miscdevice.h> +#include <linux/iopoll.h> +#include <linux/dma-mapping.h> +#include <linux/interrupt.h> +#include <asm/se.h> + +static int se_mem_size = 0x800000; +module_param(se_mem_size, int, S_IRUGO); +MODULE_PARM_DESC(se_mem_size, "LOONGSON SE shared memory size"); + +static int se_mem_page = PAGE_SIZE; +module_param(se_mem_page, int, S_IRUGO); +MODULE_PARM_DESC(se_mem_page, "LOONGSON SE shared memory page size"); + +static struct loongson_se se_dev; + +static int lsse_open(struct inode *inode, struct file *filp) +{ + return 0; +} + +static ssize_t lsse_write(struct file *filp, const char __user *buf, + size_t cnt, loff_t *offt) +{ + return 0; +} + +static struct file_operations lsse_fops = { + .owner = THIS_MODULE, + .open = lsse_open, + .write = lsse_write, +}; + +static struct miscdevice lsse_miscdev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "loongson-se", + .fops = &lsse_fops, +}; + +static inline u32 se_readl(u64 addr) +{ + return readl(se_dev.base + addr); +} + +static inline void se_writel(u32 val, u64 addr) +{ + writel(val, se_dev.base + addr); +} + +static inline bool se_ch_status(struct loongson_se *se, u32 int_bit) +{ + return !!(se->ch_status & int_bit) == 1; +} + +static void se_enable_int(struct loongson_se *se, u32 int_bit) +{ + unsigned long flag; + u32 tmp; + + if (!int_bit) + return; + + spin_lock_irqsave(&se->dev_lock, flag); + + tmp = se_readl(SE_S2LINT_EN); + tmp |= int_bit; + se_writel(tmp, SE_S2LINT_EN); + + spin_unlock_irqrestore(&se->dev_lock, flag); +} + +static void se_disable_int(struct loongson_se *se, u32 int_bit) +{ + unsigned long flag; + u32 tmp; + + if (!int_bit) + return; + + spin_lock_irqsave(&se->dev_lock, flag); + + tmp = se_readl(SE_S2LINT_EN); + tmp &= ~(int_bit); + se_writel(tmp, SE_S2LINT_EN); + + spin_unlock_irqrestore(&se->dev_lock, flag); +} + +static int se_send_requeset(struct loongson_se *se, + struct se_mailbox_data *req) +{ + unsigned long flag; + u32 status; + int err = 0; + int i; + + if (!se || !req) + return -EINVAL; + + if (se_readl(SE_L2SINT_STAT) || + !(se_readl(SE_L2SINT_EN) & req->int_bit)) + return -EBUSY; + + spin_lock_irqsave(&se->cmd_lock, flag); + + for (i = 0; i < ARRAY_SIZE(req->u.mailbox); i++) + se_writel(req->u.mailbox[i], SE_MAILBOX_S + i * 4); + + se_writel(req->int_bit, SE_L2SINT_SET); + + err = readl_relaxed_poll_timeout_atomic(se->base + SE_L2SINT_STAT, status, + !(status & req->int_bit), 10, 10000); + + spin_unlock_irqrestore(&se->cmd_lock, flag); + + return err; +} + +static int se_get_response(struct loongson_se *se, + struct se_mailbox_data *res) +{ + unsigned long flag; + int i; + + if (!se || !res) + return -EINVAL; + + if ((se_readl(SE_S2LINT_STAT) & res->int_bit) == 0) + return -EBUSY; + + spin_lock_irqsave(&se->cmd_lock, flag); + + for (i = 0; i < ARRAY_SIZE(res->u.mailbox); i++) + res->u.mailbox[i] = se_readl(SE_MAILBOX_L + i * 4); + + se_writel(res->int_bit, SE_S2LINT_CL); + + spin_unlock_irqrestore(&se->cmd_lock, flag); + + return 0; +} + +static int loongson_se_get_res(struct loongson_se *se, u32 int_bit, u32 cmd, + struct se_mailbox_data *res) +{ + int err = 0; + + res->int_bit = int_bit; + + if (se_get_response(se, res)) { + dev_err(se->dev, "Int 0x%x get response fail.\n", int_bit); + return -EFAULT; + } + + /* Check response */ + if (res->u.res.cmd == cmd) + err = 0; + else { + dev_err(se->dev, "Response cmd is 0x%x, not expect cmd 0x%x.\n", + res->u.res.cmd, cmd); + err = -EFAULT; + } + + return err; +} + +static int se_send_genl_cmd(struct loongson_se *se, struct se_mailbox_data *req, + struct se_mailbox_data *res, int retry) +{ + int err = 0, cnt = 0; + +try_again: + if (cnt++ >= retry) { + err = -ETIMEDOUT; + goto out; + } + + dev_dbg(se->dev, "%d time send cmd 0x%x\n", cnt, req->u.gcmd.cmd); + + err = se_send_requeset(se, req); + if (err) + goto try_again; + + if (!wait_for_completion_timeout(&se->cmd_completion, + msecs_to_jiffies(0x1000))) { + se_enable_int(se, req->int_bit); + goto try_again; + } + + err = loongson_se_get_res(se, req->int_bit, req->u.gcmd.cmd, res); + if (err || res->u.res.cmd_ret) { + se_enable_int(se, req->int_bit); + goto try_again; + } + +out: + se_enable_int(se, req->int_bit); + + return err; +} + +static int loongson_se_set_msg(struct lsse_ch *ch) +{ + struct loongson_se *se = ch->se; + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + int err; + + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_SETMSG; + /* MSG off */ + req.u.gcmd.info[0] = ch->id; + req.u.gcmd.info[1] = ch->smsg - se->mem_base; + req.u.gcmd.info[2] = ch->msg_size; + + dev_dbg(se->dev, "Set Channel %d msg off 0x%x, msg size %d\n", ch->id, + req.u.gcmd.info[1], req.u.gcmd.info[2]); + + err = se_send_genl_cmd(se, &req, &res, 5); + if (res.u.res.cmd_ret) + return res.u.res.cmd_ret; + + return err; +} + +static irqreturn_t loongson_se_irq(int irq, void *dev_id) +{ + struct loongson_se *se = (struct loongson_se *)dev_id; + struct lsse_ch *ch; + u32 int_status; + + int_status = se_readl(SE_S2LINT_STAT); + + dev_dbg(se->dev, "%s int status is 0x%x\n", __func__, int_status); + + se_disable_int(se, int_status); + + if (int_status & SE_INT_SETUP) { + complete(&se->cmd_completion); + int_status &= ~SE_INT_SETUP; + } + + while (int_status) { + int id = __ffs(int_status); + ch = &se->chs[id]; + if (ch->complete) + ch->complete(ch); + int_status &= ~BIT(id); + se_writel(BIT(id), SE_S2LINT_CL); + } + + return IRQ_HANDLED; +} + +static int se_init_hw(struct loongson_se *se) +{ + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + struct device *dev = se->dev; + int err, retry = 5; + u64 size; + + size = se_mem_size; + + if (size & (size - 1)) { + size = roundup_pow_of_two(size); + se_mem_size = size; + } + + se_enable_int(se, SE_INT_SETUP); + + /* Start engine */ + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_START; + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + /* Get Version */ + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_GETVER; + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + se->version = res.u.res.info[0]; + + /* Setup data buffer */ + se->mem_base = dmam_alloc_coherent(dev, size, + &se->mem_addr, GFP_KERNEL); + if (!se->mem_base) + return -ENOMEM; + + memset(se->mem_base, 0, size); + + memset(&req, 0, sizeof(struct se_mailbox_data)); + memset(&res, 0, sizeof(struct se_mailbox_data)); + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_SETBUF; + /* MMAP */ + req.u.gcmd.info[0] = (se->mem_addr & 0xffffffff) | 0x80; + req.u.gcmd.info[1] = se->mem_addr >> 32; + /* MASK */ + req.u.gcmd.info[2] = ~(size - 1); + req.u.gcmd.info[3] = 0xffffffff; + + pr_debug("Set win mmap 0x%llx, mask 0x%llx\n", + ((u64)req.u.gcmd.info[1] << 32) | req.u.gcmd.info[0], + ((u64)req.u.gcmd.info[3] << 32) | req.u.gcmd.info[2]); + + err = se_send_genl_cmd(se, &req, &res, retry); + if (err) + return err; + + se->mem_map_size = size / se_mem_page; + se->mem_map = bitmap_zalloc(se->mem_map_size, GFP_KERNEL); + if (!se->mem_map) + return -ENOMEM; + + dev_info(se->dev, "SE module setup down, shared memory size is 0x%x bytes, " + "memory page size is 0x%x bytes\n", + se_mem_size, se_mem_page); + + return err; +} + +static void loongson_se_disable_hw(struct loongson_se *se) +{ + struct se_mailbox_data req = {0}; + struct se_mailbox_data res = {0}; + int retry = 5; + + /* Stop engine */ + req.int_bit = SE_INT_SETUP; + req.u.gcmd.cmd = SE_CMD_STOP; + se_send_genl_cmd(se, &req, &res, retry); + + se_disable_int(se, SE_INT_ALL); + kfree(se->mem_map); +} + +int se_send_ch_requeset(struct lsse_ch *ch) +{ + struct loongson_se *se; + u32 status, int_bit; + int err = 0; + + if (!ch) + return -EINVAL; + + se = ch->se; + int_bit = ch->int_bit; + + if ((se_readl(SE_L2SINT_STAT) & int_bit) || + !(se_readl(SE_L2SINT_EN) & int_bit)) + return -EBUSY; + + se_enable_int(se, int_bit); + se_writel(int_bit, SE_L2SINT_SET); + + err = readl_relaxed_poll_timeout_atomic(se->base + SE_L2SINT_STAT, status, + !(status & int_bit), 10, 10000); + + return err; +} +EXPORT_SYMBOL_GPL(se_send_ch_requeset); + +struct lsse_ch *se_init_ch(int id, int data_size, int msg_size, void *priv, + void (*complete)(struct lsse_ch *se_ch)) +{ + struct loongson_se *se = &se_dev; + struct lsse_ch *ch; + unsigned long flag; + int data_first, data_nr; + int msg_first, msg_nr; + + if (!se) { + pr_err("SE has bot been initialized\n"); + return NULL; + } + + if (id == 0 || id > SE_CH_MAX) { + dev_err(se->dev, "Channel number %d is invalid\n", id); + return NULL; + } + + if (se_ch_status(se, BIT(id))) { + dev_err(se->dev, "Channel number %d has been initialized\n", id); + return NULL; + } + + spin_lock_irqsave(&se->dev_lock, flag); + + ch = &se_dev.chs[id]; + ch->se = se; + ch->id = id; + ch->int_bit = BIT(id); + se->ch_status |= BIT(id); + + data_nr = round_up(data_size, se_mem_page) / se_mem_page; + data_first = bitmap_find_next_zero_area(se->mem_map, se->mem_map_size, + 0, data_nr, 0); + if (data_first >= se->mem_map_size) { + dev_err(se->dev, "Insufficient memory space\n"); + spin_unlock_irqrestore(&se->dev_lock, flag); + return NULL; + } + + bitmap_set(se->mem_map, data_first, data_nr); + ch->data_buffer = se->mem_base + data_first * se_mem_page; + ch->data_addr = se->mem_addr + data_first * se_mem_page; + ch->data_size = data_size; + + msg_nr = round_up(msg_size, se_mem_page) / se_mem_page; + msg_first = bitmap_find_next_zero_area(se->mem_map, se->mem_map_size, + 0, msg_nr, 0); + if (msg_first >= se->mem_map_size) { + dev_err(se->dev, "Insufficient memory space\n"); + bitmap_clear(se->mem_map, data_first, data_nr); + spin_unlock_irqrestore(&se->dev_lock, flag); + return NULL; + } + + bitmap_set(se->mem_map, msg_first, msg_nr); + ch->smsg = se->mem_base + msg_first * se_mem_page; + ch->rmsg = ch->smsg + msg_size / 2; + ch->msg_size = msg_size; + + ch->complete = complete; + ch->priv = priv; + + spin_lock_init(&ch->ch_lock); + + spin_unlock_irqrestore(&se->dev_lock, flag); + + if (loongson_se_set_msg(ch)) { + dev_err(se->dev, "Channel %d setup message address failed\n", id); + return NULL; + } + + se_enable_int(se, ch->int_bit); + + return ch; +} +EXPORT_SYMBOL_GPL(se_init_ch); + +void se_deinit_ch(struct lsse_ch *ch) +{ + struct loongson_se *se = &se_dev; + unsigned long flag; + int first, nr; + int id = ch->id; + + if (!se) { + pr_err("SE has bot been initialized\n"); + return; + } + + if (id == 0 || id > SE_CH_MAX) { + dev_err(se->dev, "Channel number %d is invalid\n", id); + return; + } + + if (!se_ch_status(se, BIT(id))) { + dev_err(se->dev, "Channel number %d has not been initialized\n", id); + return; + } + + spin_lock_irqsave(&se->dev_lock, flag); + + se->ch_status &= ~BIT(ch->id); + + first = (ch->data_buffer - se->mem_base) / se_mem_page; + nr = round_up(ch->data_size, se_mem_page) / se_mem_page; + bitmap_clear(se->mem_map, first, nr); + + first = (ch->smsg - se->mem_base) / se_mem_page; + nr = round_up(ch->msg_size, se_mem_page) / se_mem_page; + bitmap_clear(se->mem_map, first, nr); + + spin_unlock_irqrestore(&se->dev_lock, flag); + + se_disable_int(se, ch->int_bit); +} +EXPORT_SYMBOL_GPL(se_deinit_ch); + +static struct platform_device lsse_sdf_pdev = { + .name = "loongson-sdf", + .id = -1, +}; + +static const struct of_device_id loongson_se_of_match[] = { + { .compatible = "loongson,ls3c6000se", }, + {} +}; +MODULE_DEVICE_TABLE(of, loongson_se_of_match); + +static int loongson_se_probe(struct platform_device *pdev) +{ + struct loongson_se *se = &se_dev; + struct resource *res; + struct device *dev = &pdev->dev; + int nr_irq, err, i; + int irq[8]; + + nr_irq = platform_irq_count(pdev); + if (nr_irq < 0) + return -ENODEV; + + for (i = 0; i < nr_irq; i++) { + irq[i] = platform_get_irq(pdev, i); + if (irq[i] < 0) + return -ENODEV; + } + + res = platform_get_resource(pdev, IORESOURCE_MEM, 0); + if (!res) + return -ENODEV; + + se->base = devm_ioremap_resource(dev, res); + if (IS_ERR(se->base)) + return PTR_ERR(se->base); + + se->dev = &pdev->dev; + platform_set_drvdata(pdev, se); + dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64)); + init_completion(&se->cmd_completion); + spin_lock_init(&se->cmd_lock); + spin_lock_init(&se->dev_lock); + + for (i = 0; i < nr_irq; i++) { + err = devm_request_irq(dev, irq[i], loongson_se_irq, 0, + "loongson-se", se); + if (err) + goto out; + } + + err = se_init_hw(se); + if (err) + goto disable_hw; + + err = misc_register(&lsse_miscdev); + if (err) + goto disable_hw; + + err = platform_device_register(&lsse_sdf_pdev); + if (err) + pr_err("register sdf device failed\n"); + + return 0; + +disable_hw: + loongson_se_disable_hw(se); +out: + for (; i>=0; i--) + devm_free_irq(dev, irq[i], se); + return err; +} + +static int loongson_se_remove(struct platform_device *pdev) +{ + struct loongson_se *se = platform_get_drvdata(pdev); + + misc_deregister(&lsse_miscdev); + loongson_se_disable_hw(se); + platform_device_unregister(&lsse_sdf_pdev); + + return 0; +} + +static struct platform_driver loongson_se_driver = { + .probe = loongson_se_probe, + .remove = loongson_se_remove, + .driver = { + .name = "loongson-se", + .of_match_table = loongson_se_of_match, + }, +}; + +module_platform_driver(loongson_se_driver); + +MODULE_AUTHOR("Yinggang Gu"); +MODULE_DESCRIPTION("Loongson SE driver"); +MODULE_LICENSE("GPL"); diff --git a/drivers/char/lsse_sdf_cdev.c b/drivers/char/lsse_sdf_cdev.c new file mode 100644 index 000000000000..edadb56f2273 --- /dev/null +++ b/drivers/char/lsse_sdf_cdev.c @@ -0,0 +1,381 @@ +#include <linux/kernel.h> +#include <linux/slab.h> +#include <linux/init.h> +#include <linux/miscdevice.h> +#include <linux/module.h> +#include <linux/cdev.h> +#include <linux/module.h> +#include <linux/wait.h> +#include <linux/of.h> +#include <linux/iopoll.h> +#include <linux/platform_device.h> +#include <linux/uaccess.h> +#include <asm/se.h> +#include <linux/list.h> + +#define SE_SDF_BUFSIZE (PAGE_SIZE * 2) +#define SDF_OPENSESSION 0x204 +#define SDF_CLOSESESSION 0x205 + +struct lsse_sdf_dev { + struct lsse_ch *se_ch; + struct mutex data_lock; + bool processing_cmd; + + /* Synchronous CMD */ + wait_queue_head_t wq; +}; + +struct se_sdf_msg { + u32 cmd; + u32 data_off; + u32 data_len; + u32 info[5]; +}; + +struct sdf_command_header { + int command; + union { + int param_cnt; + int ret; + } u; + int param_len[14]; +}; + +struct sdf_kernel_command { + struct sdf_command_header header; + void *handle; +}; + +#define KERNEL_COMMAND_SIZE (sizeof(struct sdf_kernel_command)) + +struct sdf_handle { + struct list_head handle_list; + void *handle; +}; + +struct sdf_file_pvt_data { + struct lsse_sdf_dev *se; + struct list_head handle_list; + struct sdf_kernel_command skc; + struct sdf_handle *ph; +}; + +static struct lsse_sdf_dev *se_sdf_dev; + +static void lsse_sdf_complete(struct lsse_ch *ch) +{ + struct lsse_sdf_dev *se = (struct lsse_sdf_dev *)ch->priv; + + se->processing_cmd = false; + wake_up(&se->wq); +} + +static int se_send_sdf_cmd(struct lsse_sdf_dev *se, int len, int retry) +{ + struct se_sdf_msg *smsg = (struct se_sdf_msg *)se->se_ch->smsg; + unsigned long flag; + int err; + + spin_lock_irqsave(&se->se_ch->ch_lock, flag); + + smsg->cmd = SE_CMD_SDF; + /* One time one cmd */ + smsg->data_off = se->se_ch->data_buffer - se->se_ch->se->mem_base; + smsg->data_len = len; + +try_again: + if (!retry--) + goto out; + + pr_debug("Send sdf cmd, last retry %d times\n", retry); + + err = se_send_ch_requeset(se->se_ch); + if (err) { + udelay(5); + goto try_again; + } + +out: + spin_unlock_irqrestore(&se->se_ch->ch_lock, flag); + + return err; +} + +static int lsse_sdf_recv(struct sdf_file_pvt_data *pvt, char *buf, + size_t size, int user, int *se_ret) +{ + int len, time, ret = 0; + struct se_sdf_msg *rmsg; + struct sdf_kernel_command *skc; + struct sdf_handle *ph; + struct lsse_sdf_dev *se = pvt->se; + + if (!se->se_ch->rmsg) { + pr_err("se device is not ready\n"); + return -EBUSY; + } + + time = wait_event_timeout(se->wq, !se->processing_cmd, HZ*30); + if (!time) + return -ETIME; + + rmsg = (struct se_sdf_msg *)se->se_ch->rmsg; + if (rmsg->cmd != SE_CMD_SDF) { + pr_err("se get wrong response\n"); + return -EIO; + } + len = rmsg->data_len; + + if ((!user && len > KERNEL_COMMAND_SIZE) || len > SE_SDF_BUFSIZE + || (size && len > size)) + return -E2BIG; + + if (user) { + ret = copy_to_user((char __user *)buf, + se->se_ch->data_buffer + rmsg->data_off, len); + if (!se_ret) + return ret; + + skc = (struct sdf_kernel_command *) + (se->se_ch->data_buffer + rmsg->data_off); + *se_ret = skc->header.u.ret; + if (skc->header.command == SDF_OPENSESSION && !*se_ret) { + ph = kmalloc(sizeof(*ph), GFP_KERNEL); + if (!ph) + return -ENOMEM; + ph->handle = skc->handle; + list_add(&ph->handle_list, &pvt->handle_list); + } + } + else + memcpy(buf, se->se_ch->data_buffer + rmsg->data_off, len); + + return ret; +} + +static struct sdf_handle *find_sdf_handle(void *handle, + struct sdf_file_pvt_data *pvt) +{ + struct sdf_handle *ph; + + list_for_each_entry(ph, &pvt->handle_list, handle_list) { + if (ph->handle == handle) + return ph; + } + + return NULL; +} + +static int lsse_sdf_send(struct sdf_file_pvt_data *pvt, const char *buf, + size_t count, int user) +{ + int ret, se_ret; + struct sdf_handle *ph = NULL; + struct sdf_kernel_command *skc; + struct lsse_sdf_dev *se = pvt->se; + + if (!se->se_ch->smsg) { + pr_err("se device is not ready\n"); + return 0; + } + + if (count > se->se_ch->data_size) { + pr_err("Invalid size in send: count=%zd, size=%d\n", + count, se->se_ch->data_size); + return -EIO; + } + + if (user) { + ret = mutex_lock_interruptible(&se->data_lock); + if (ret) + goto out; + } else + mutex_lock(&se->data_lock); + + if (user) { + ret = copy_from_user(se->se_ch->data_buffer, buf, count); + if (ret) { + ret = -EFAULT; + goto out_unlock; + } + skc = (struct sdf_kernel_command *)se->se_ch->data_buffer; + if (skc->header.command == SDF_CLOSESESSION) { + ph = find_sdf_handle(skc->handle, pvt); + } + } + else + memcpy(se->se_ch->data_buffer, buf, count); + + se->processing_cmd = true; + ret = se_send_sdf_cmd(se, count, 5); + if (ret) { + pr_err("se_send_sdf_cmd failed\n"); + goto out_unlock; + } + + ret = lsse_sdf_recv(pvt, (char *)buf, 0, user, &se_ret); + if (ret) { + pr_err("recv failed ret: %x\n", ret); + goto out_unlock; + } + if (ph && !se_ret) { + list_del(&ph->handle_list); + kfree(ph); + } +out_unlock: + mutex_unlock(&se->data_lock); +out: + return ret; +} + +static ssize_t lsse_sdf_write(struct file *filp, const char __user *buf, + size_t cnt, loff_t *offt) +{ + struct sdf_file_pvt_data *pvt = filp->private_data; + + if (cnt > SE_SDF_BUFSIZE) + return -E2BIG; + + if (lsse_sdf_send(pvt, buf, cnt, 1)) + return -EFAULT; + + return cnt; +} + +static ssize_t lsse_sdf_read(struct file *filp, char __user *buf, + size_t size, loff_t *off) +{ + return lsse_sdf_recv(filp->private_data, buf, size, 1, NULL); +} + +static int close_one_handle(struct sdf_file_pvt_data *pvt, struct sdf_handle *ph) +{ + struct sdf_kernel_command *skc = &pvt->skc; + + skc->header.command = 0x205; + skc->header.u.param_cnt = 1; + skc->header.param_len[0] = 8; + skc->handle = ph->handle; + /* close one session */ + lsse_sdf_send(pvt, (char *)&pvt->skc, KERNEL_COMMAND_SIZE, 0); + if (skc->header.u.ret) { + pr_err("Auto Close Session failed, session handle: %llx, ret: %d\n", + (u64)ph->handle, skc->header.u.ret); + return skc->header.u.ret; + } + kfree(ph); + + return 0; +} + +static int close_all_handle(struct sdf_file_pvt_data *pvt) +{ + int ret = 0; + struct sdf_handle *ph, *tmp; + + list_for_each_entry_safe(ph, tmp, &pvt->handle_list, handle_list) { + list_del(&ph->handle_list); + ret = close_one_handle(pvt, ph); + if (ret) + return ret; + } + + return 0; +} + +static int lsse_sdf_release(struct inode *inode, struct file *filp) +{ + int ret; + struct sdf_file_pvt_data *pvt = filp->private_data; + + ret = close_all_handle(pvt); + filp->private_data = NULL; + kfree(pvt); + + if (ret) + ret = -EFAULT; + return ret; +} + +static int lsse_sdf_open(struct inode *inode, struct file *filp) +{ + struct sdf_file_pvt_data *pvt = kmalloc(sizeof(*pvt), GFP_KERNEL); + + if (!pvt) + return -ENOMEM; + + INIT_LIST_HEAD(&pvt->handle_list); + pvt->se = se_sdf_dev; + filp->private_data = pvt; + + return 0; +} + +static struct file_operations lsse_sdf_fops = { + .owner = THIS_MODULE, + .open = lsse_sdf_open, + .write = lsse_sdf_write, + .read = lsse_sdf_read, + .release = lsse_sdf_release, +}; + +static struct miscdevice lsse_sdf_miscdev = { + .minor = MISC_DYNAMIC_MINOR, + .name = "lsse_sdf", + .fops = &lsse_sdf_fops, +}; + +static int lsse_sdf_probe(struct platform_device *pdev) +{ + int msg_size; + int ret; + + se_sdf_dev = kzalloc(sizeof(*se_sdf_dev), GFP_KERNEL); + if (IS_ERR_OR_NULL(se_sdf_dev)) + return PTR_ERR(se_sdf_dev); + + mutex_init(&se_sdf_dev->data_lock); + init_waitqueue_head(&se_sdf_dev->wq); + se_sdf_dev->processing_cmd = false; + + msg_size = 2 * sizeof(struct se_sdf_msg); + se_sdf_dev->se_ch = se_init_ch(SE_CH_SDF, SE_SDF_BUFSIZE, msg_size, + se_sdf_dev, lsse_sdf_complete); + + ret = misc_register(&lsse_sdf_miscdev); + if (ret < 0) { + pr_err("register sdf dev failed!\n"); + goto out; + } + + return 0; + +out: + kfree(se_sdf_dev); + + return ret; +} + +static int lsse_sdf_remove(struct platform_device *pdev) +{ + misc_deregister(&lsse_sdf_miscdev); + se_deinit_ch(se_sdf_dev->se_ch); + kfree(se_sdf_dev); + + return 0; +} + +static struct platform_driver loongson_sdf_driver = { + .probe = lsse_sdf_probe, + .remove = lsse_sdf_remove, + .driver = { + .name = "loongson-sdf", + }, +}; +module_platform_driver(loongson_sdf_driver); + +MODULE_ALIAS("platform:loongson-sdf"); +MODULE_AUTHOR("Yinggang Gu"); +MODULE_DESCRIPTION("Loongson SE sdf driver"); +MODULE_LICENSE("GPL"); -- 2.33.0

2 1

[PATCH OLK-6.6] LoongArch: Add AVEC irqchip support
by Hongchen Zhang 23 Oct '24

23 Oct '24

From: Tianyang Zhang <zhangtianyang(a)loongson.cn> LoongArch inclusion category: feature bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ3D3 -------------------------------- Introduce the advanced extended interrupt controllers (AVECINTC). This feature will allow each core to have 256 independent interrupt vectors and MSI interrupts can be independently routed to any vector on any CPU. Co-developed-by: Jianmin Lv <lvjianmin(a)loongson.cn> Signed-off-by: Jianmin Lv <lvjianmin(a)loongson.cn> Co-developed-by: Liupu Wang <wangliupu(a)loongson.cn> Signed-off-by: Liupu Wang <wangliupu(a)loongson.cn> Co-developed-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Huacai Chen <chenhuacai(a)loongson.cn> Signed-off-by: Tianyang Zhang <zhangtianyang(a)loongson.cn> --- .../arch/loongarch/irq-chip-model.rst | 32 ++ .../zh_CN/arch/loongarch/irq-chip-model.rst | 32 ++ arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/cpu-features.h | 1 + arch/loongarch/include/asm/cpu.h | 2 + arch/loongarch/include/asm/hardirq.h | 3 +- arch/loongarch/include/asm/irq.h | 29 +- arch/loongarch/include/asm/loongarch.h | 18 +- arch/loongarch/include/asm/smp.h | 2 + arch/loongarch/kernel/cpu-probe.c | 3 +- arch/loongarch/kernel/irq.c | 13 + arch/loongarch/kernel/legacy_boot.c | 9 + arch/loongarch/kernel/smp.c | 6 + drivers/irqchip/Makefile | 2 +- drivers/irqchip/irq-loongarch-avec.c | 449 ++++++++++++++++++ drivers/irqchip/irq-loongarch-cpu.c | 7 +- drivers/irqchip/irq-loongson-eiointc.c | 9 +- drivers/irqchip/irq-loongson-htvec.c | 2 + drivers/irqchip/irq-loongson-liointc.c | 2 + drivers/irqchip/irq-loongson-pch-lpc.c | 2 + drivers/irqchip/irq-loongson-pch-msi.c | 44 +- drivers/irqchip/irq-loongson-pch-pic.c | 2 + drivers/irqchip/irq-loongson.h | 27 ++ include/linux/cpuhotplug.h | 3 +- 24 files changed, 669 insertions(+), 31 deletions(-) create mode 100644 drivers/irqchip/irq-loongarch-avec.c create mode 100644 drivers/irqchip/irq-loongson.h diff --git a/Documentation/arch/loongarch/irq-chip-model.rst b/Documentation/arch/loongarch/irq-chip-model.rst index 7988f4192363..6dd48256e39f 100644 --- a/Documentation/arch/loongarch/irq-chip-model.rst +++ b/Documentation/arch/loongarch/irq-chip-model.rst @@ -85,6 +85,38 @@ to CPUINTC directly:: | Devices | +---------+ +Advanced Extended IRQ model +=========================== + +In this model, IPI (Inter-Processor Interrupt) and CPU Local Timer interrupt go +to CPUINTC directly, CPU UARTS interrupts go to LIOINTC, PCH-MSI interrupts go +to AVECINTC, and then go to CPUINTC directly, while all other devices interrupts +go to PCH-PIC/PCH-LPC and gathered by EIOINTC, and then go to CPUINTC directly:: + + +-----+ +-----------------------+ +-------+ + | IPI | --> | CPUINTC | <-- | Timer | + +-----+ +-----------------------+ +-------+ + ^ ^ ^ + | | | + +---------+ +----------+ +---------+ +-------+ + | EIOINTC | | AVECINTC | | LIOINTC | <-- | UARTs | + +---------+ +----------+ +---------+ +-------+ + ^ ^ + | | + +---------+ +---------+ + | PCH-PIC | | PCH-MSI | + +---------+ +---------+ + ^ ^ ^ + | | | + +---------+ +---------+ +---------+ + | Devices | | PCH-LPC | | Devices | + +---------+ +---------+ +---------+ + ^ + | + +---------+ + | Devices | + +---------+ + ACPI-related definitions ======================== diff --git a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst index f1e9ab18206c..472761938682 100644 --- a/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst +++ b/Documentation/translations/zh_CN/arch/loongarch/irq-chip-model.rst @@ -87,6 +87,38 @@ PCH-LPC/PCH-MSI，然后被EIOINTC统一收集，再直接到达CPUINTC:: | Devices | +---------+ +高级扩展IRQ模型 +=============== + +在这种模型里面，IPI（Inter-Processor Interrupt）和CPU本地时钟中断直接发送到CPUINTC， +CPU串口（UARTs）中断发送到LIOINTC，PCH-MSI中断发送到AVECINTC，而后通过AVECINTC直接 +送达CPUINTC，而其他所有设备的中断则分别发送到所连接的PCH-PIC/PCH-LPC，然后由EIOINTC +统一收集，再直接到达CPUINTC:: + + +-----+ +-----------------------+ +-------+ + | IPI | --> | CPUINTC | <-- | Timer | + +-----+ +-----------------------+ +-------+ + ^ ^ ^ + | | | + +---------+ +----------+ +---------+ +-------+ + | EIOINTC | | AVECINTC | | LIOINTC | <-- | UARTs | + +---------+ +----------+ +---------+ +-------+ + ^ ^ + | | + +---------+ +---------+ + | PCH-PIC | | PCH-MSI | + +---------+ +---------+ + ^ ^ ^ + | | | + +---------+ +---------+ +---------+ + | Devices | | PCH-LPC | | Devices | + +---------+ +---------+ +---------+ + ^ + | + +---------+ + | Devices | + +---------+ + ACPI相关的定义 ============== diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index 58da6e03ecb5..710ec4831887 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -77,6 +77,7 @@ config LOONGARCH select GENERIC_ENTRY select GENERIC_GETTIMEOFDAY select GENERIC_IOREMAP if !ARCH_IOREMAP + select GENERIC_IRQ_MATRIX_ALLOCATOR select GENERIC_IRQ_MULTI_HANDLER select GENERIC_IRQ_PROBE select GENERIC_IRQ_SHOW diff --git a/arch/loongarch/include/asm/cpu-features.h b/arch/loongarch/include/asm/cpu-features.h index 2eafe6a6aca8..16a716f88a5c 100644 --- a/arch/loongarch/include/asm/cpu-features.h +++ b/arch/loongarch/include/asm/cpu-features.h @@ -65,5 +65,6 @@ #define cpu_has_guestid cpu_opt(LOONGARCH_CPU_GUESTID) #define cpu_has_hypervisor cpu_opt(LOONGARCH_CPU_HYPERVISOR) #define cpu_has_ptw cpu_opt(LOONGARCH_CPU_PTW) +#define cpu_has_avecint cpu_opt(LOONGARCH_CPU_AVECINT) #endif /* __ASM_CPU_FEATURES_H */ diff --git a/arch/loongarch/include/asm/cpu.h b/arch/loongarch/include/asm/cpu.h index 48b9f7168bcc..843f9c4ec980 100644 --- a/arch/loongarch/include/asm/cpu.h +++ b/arch/loongarch/include/asm/cpu.h @@ -99,6 +99,7 @@ enum cpu_type_enum { #define CPU_FEATURE_GUESTID 24 /* CPU has GuestID feature */ #define CPU_FEATURE_HYPERVISOR 25 /* CPU has hypervisor (running in VM) */ #define CPU_FEATURE_PTW 26 /* CPU has hardware page table walker */ +#define CPU_FEATURE_AVECINT 27 /* CPU has avec interrupt */ #define LOONGARCH_CPU_CPUCFG BIT_ULL(CPU_FEATURE_CPUCFG) #define LOONGARCH_CPU_LAM BIT_ULL(CPU_FEATURE_LAM) @@ -127,5 +128,6 @@ enum cpu_type_enum { #define LOONGARCH_CPU_GUESTID BIT_ULL(CPU_FEATURE_GUESTID) #define LOONGARCH_CPU_HYPERVISOR BIT_ULL(CPU_FEATURE_HYPERVISOR) #define LOONGARCH_CPU_PTW BIT_ULL(CPU_FEATURE_PTW) +#define LOONGARCH_CPU_AVECINT BIT_ULL(CPU_FEATURE_AVECINT) #endif /* _ASM_CPU_H */ diff --git a/arch/loongarch/include/asm/hardirq.h b/arch/loongarch/include/asm/hardirq.h index b26d596a73aa..5f70cb77b54d 100644 --- a/arch/loongarch/include/asm/hardirq.h +++ b/arch/loongarch/include/asm/hardirq.h @@ -15,8 +15,9 @@ extern void ack_bad_irq(unsigned int irq); enum ipi_msg_type { IPI_RESCHEDULE, IPI_CALL_FUNCTION, + IPI_CLEAR_VECTOR, }; -#define NR_IPI 2 +#define NR_IPI 3 typedef struct { unsigned int ipi_irqs[NR_IPI]; diff --git a/arch/loongarch/include/asm/irq.h b/arch/loongarch/include/asm/irq.h index 79f4c834ffe8..fc600aed50c8 100644 --- a/arch/loongarch/include/asm/irq.h +++ b/arch/loongarch/include/asm/irq.h @@ -39,6 +39,17 @@ void spurious_interrupt(void); #define NR_IRQS_LEGACY 16 +/* + * 256 Vectors Mapping for AVECINTC: + * + * 0 - 15: Mapping classic IPs, e.g. IP0-12. + * 16 - 255: Mapping vectors for external IRQ. + * + */ +#define NR_VECTORS 256 +#define NR_LEGACY_VECTORS 16 +#define IRQ_MATRIX_BITS NR_VECTORS + #define arch_trigger_cpumask_backtrace arch_trigger_cpumask_backtrace extern bool arch_trigger_cpumask_backtrace(const cpumask_t *mask, int exclude_cpu); @@ -67,7 +78,7 @@ extern struct acpi_vector_group msi_group[MAX_IO_PICS]; #define LOONGSON_LPC_LAST_IRQ (LOONGSON_LPC_IRQ_BASE + 15) #define LOONGSON_CPU_IRQ_BASE 16 -#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 14) +#define LOONGSON_CPU_LAST_IRQ (LOONGSON_CPU_IRQ_BASE + 15) #define LOONGSON_PCH_IRQ_BASE 64 #define LOONGSON_PCH_ACPI_IRQ (LOONGSON_PCH_IRQ_BASE + 47) @@ -90,20 +101,8 @@ struct acpi_madt_bio_pic; struct acpi_madt_msi_pic; struct acpi_madt_lpc_pic; -int liointc_acpi_init(struct irq_domain *parent, - struct acpi_madt_lio_pic *acpi_liointc); -int eiointc_acpi_init(struct irq_domain *parent, - struct acpi_madt_eio_pic *acpi_eiointc); - -int htvec_acpi_init(struct irq_domain *parent, - struct acpi_madt_ht_pic *acpi_htvec); -int pch_lpc_acpi_init(struct irq_domain *parent, - struct acpi_madt_lpc_pic *acpi_pchlpc); -int pch_msi_acpi_init(struct irq_domain *parent, - struct acpi_madt_msi_pic *acpi_pchmsi); -int pch_pic_acpi_init(struct irq_domain *parent, - struct acpi_madt_bio_pic *acpi_pchpic); -int find_pch_pic(u32 gsi); +void complete_irq_moving(void); + struct fwnode_handle *get_pch_msi_handle(int pci_segment); extern struct acpi_madt_lio_pic *acpi_liointc; diff --git a/arch/loongarch/include/asm/loongarch.h b/arch/loongarch/include/asm/loongarch.h index e852c0f62eb7..964bcfe08d8a 100644 --- a/arch/loongarch/include/asm/loongarch.h +++ b/arch/loongarch/include/asm/loongarch.h @@ -254,8 +254,8 @@ #define CSR_ESTAT_EXC_WIDTH 6 #define CSR_ESTAT_EXC (_ULCAST_(0x3f) << CSR_ESTAT_EXC_SHIFT) #define CSR_ESTAT_IS_SHIFT 0 -#define CSR_ESTAT_IS_WIDTH 14 -#define CSR_ESTAT_IS (_ULCAST_(0x3fff) << CSR_ESTAT_IS_SHIFT) +#define CSR_ESTAT_IS_WIDTH 15 +#define CSR_ESTAT_IS (_ULCAST_(0x7fff) << CSR_ESTAT_IS_SHIFT) #define LOONGARCH_CSR_ERA 0x6 /* ERA */ @@ -650,6 +650,13 @@ #define LOONGARCH_CSR_CTAG 0x98 /* TagLo + TagHi */ +#define LOONGARCH_CSR_ISR0 0xa0 +#define LOONGARCH_CSR_ISR1 0xa1 +#define LOONGARCH_CSR_ISR2 0xa2 +#define LOONGARCH_CSR_ISR3 0xa3 + +#define LOONGARCH_CSR_IRR 0xa4 + #define LOONGARCH_CSR_PRID 0xc0 /* Shadow MCSR : 0xc0 ~ 0xff */ @@ -1004,7 +1011,7 @@ /* * CSR_ECFG IM */ -#define ECFG0_IM 0x00001fff +#define ECFG0_IM 0x00005fff #define ECFGB_SIP0 0 #define ECFGF_SIP0 (_ULCAST_(1) << ECFGB_SIP0) #define ECFGB_SIP1 1 @@ -1047,6 +1054,7 @@ #define IOCSRF_EIODECODE BIT_ULL(9) #define IOCSRF_FLATMODE BIT_ULL(10) #define IOCSRF_VM BIT_ULL(11) +#define IOCSRF_AVEC BIT_ULL(15) #define LOONGARCH_IOCSR_VENDOR 0x10 @@ -1057,6 +1065,7 @@ #define LOONGARCH_IOCSR_MISC_FUNC 0x420 #define IOCSR_MISC_FUNC_TIMER_RESET BIT_ULL(21) #define IOCSR_MISC_FUNC_EXT_IOI_EN BIT_ULL(48) +#define IOCSR_MISC_FUNC_AVEC_EN BIT_ULL(51) #define LOONGARCH_IOCSR_CPUTEMP 0x428 @@ -1378,9 +1387,10 @@ __BUILD_CSR_OP(tlbidx) #define INT_TI 11 /* Timer */ #define INT_IPI 12 #define INT_NMI 13 +#define INT_AVEC 14 /* ExcCodes corresponding to interrupts */ -#define EXCCODE_INT_NUM (INT_NMI + 1) +#define EXCCODE_INT_NUM (INT_AVEC + 1) #define EXCCODE_INT_START 64 #define EXCCODE_INT_END (EXCCODE_INT_START + EXCCODE_INT_NUM - 1) diff --git a/arch/loongarch/include/asm/smp.h b/arch/loongarch/include/asm/smp.h index 75d30529748c..630e5ebec21c 100644 --- a/arch/loongarch/include/asm/smp.h +++ b/arch/loongarch/include/asm/smp.h @@ -67,9 +67,11 @@ extern int __cpu_logical_map[NR_CPUS]; #define ACTION_BOOT_CPU 0 #define ACTION_RESCHEDULE 1 #define ACTION_CALL_FUNCTION 2 +#define ACTION_CLEAR_VECTOR 3 #define SMP_BOOT_CPU BIT(ACTION_BOOT_CPU) #define SMP_RESCHEDULE BIT(ACTION_RESCHEDULE) #define SMP_CALL_FUNCTION BIT(ACTION_CALL_FUNCTION) +#define SMP_CLEAR_VECTOR BIT(ACTION_CLEAR_VECTOR) struct secondary_data { unsigned long stack; diff --git a/arch/loongarch/kernel/cpu-probe.c b/arch/loongarch/kernel/cpu-probe.c index 55320813ee08..14f0449f5452 100644 --- a/arch/loongarch/kernel/cpu-probe.c +++ b/arch/loongarch/kernel/cpu-probe.c @@ -106,7 +106,6 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) elf_hwcap |= HWCAP_LOONGARCH_CRC32; } - config = read_cpucfg(LOONGARCH_CPUCFG2); if (config & CPUCFG2_LAM) { c->options |= LOONGARCH_CPU_LAM; @@ -174,6 +173,8 @@ static void cpu_probe_common(struct cpuinfo_loongarch *c) c->options |= LOONGARCH_CPU_FLATMODE; if (config & IOCSRF_EIODECODE) c->options |= LOONGARCH_CPU_EIODECODE; + if (config & IOCSRF_AVEC) + c->options |= LOONGARCH_CPU_AVECINT; if (config & IOCSRF_VM) c->options |= LOONGARCH_CPU_HYPERVISOR; diff --git a/arch/loongarch/kernel/irq.c b/arch/loongarch/kernel/irq.c index a2f6ce3061f3..50089a7a1a9d 100644 --- a/arch/loongarch/kernel/irq.c +++ b/arch/loongarch/kernel/irq.c @@ -122,6 +122,19 @@ void fixup_irqs(void) } #endif +int __init arch_probe_nr_irqs(void) +{ + int nr_io_pics = bitmap_weight(&loongson_sysconf.cores_io_master, NR_CPUS); + + if (!cpu_has_avecint) + nr_irqs = (64 + NR_VECTORS * nr_io_pics); + else + nr_irqs = (64 + NR_VECTORS * (nr_cpu_ids + nr_io_pics)); + + return NR_IRQS_LEGACY; +} + + void __init init_IRQ(void) { int i, ret; diff --git a/arch/loongarch/kernel/legacy_boot.c b/arch/loongarch/kernel/legacy_boot.c index 29fdeb93dfd5..2665344d1d05 100644 --- a/arch/loongarch/kernel/legacy_boot.c +++ b/arch/loongarch/kernel/legacy_boot.c @@ -25,6 +25,15 @@ #define MSI_MSG_ADDRESS 0x2FF00000 #define MSI_MSG_DEFAULT_COUNT 0xC0 +extern int liointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lio_pic *acpi_liointc); +extern int eiointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_eio_pic *acpi_eiointc); +extern int htvec_acpi_init(struct irq_domain *parent, + struct acpi_madt_ht_pic *acpi_htvec); +extern int pch_lpc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lpc_pic *acpi_pchlpc); + struct boot_params *efi_bp; struct loongsonlist_mem_map *g_mmap; struct acpi_madt_lio_pic *acpi_liointc; diff --git a/arch/loongarch/kernel/smp.c b/arch/loongarch/kernel/smp.c index ea69606fd9f5..bd0bd3decd32 100644 --- a/arch/loongarch/kernel/smp.c +++ b/arch/loongarch/kernel/smp.c @@ -71,6 +71,7 @@ static DEFINE_PER_CPU(int, cpu_state); static const char *ipi_types[NR_IPI] __tracepoint_string = { [IPI_RESCHEDULE] = "Rescheduling interrupts", [IPI_CALL_FUNCTION] = "Function call interrupts", + [IPI_CLEAR_VECTOR] = "Clear vector interrupts", }; void show_ipi_list(struct seq_file *p, int prec) @@ -245,6 +246,11 @@ static irqreturn_t loongson_ipi_interrupt(int irq, void *dev) per_cpu(irq_stat, cpu).ipi_irqs[IPI_CALL_FUNCTION]++; } + if (action & SMP_CLEAR_VECTOR) { + complete_irq_moving(); + per_cpu(irq_stat, cpu).ipi_irqs[IPI_CLEAR_VECTOR]++; + } + return IRQ_HANDLED; } diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile index 246aa0603d6e..787206e166fc 100644 --- a/drivers/irqchip/Makefile +++ b/drivers/irqchip/Makefile @@ -116,7 +116,7 @@ obj-$(CONFIG_LS1X_IRQ) += irq-ls1x.o obj-$(CONFIG_TI_SCI_INTR_IRQCHIP) += irq-ti-sci-intr.o obj-$(CONFIG_TI_SCI_INTA_IRQCHIP) += irq-ti-sci-inta.o obj-$(CONFIG_TI_PRUSS_INTC) += irq-pruss-intc.o -obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o +obj-$(CONFIG_IRQ_LOONGARCH_CPU) += irq-loongarch-cpu.o irq-loongarch-avec.o obj-$(CONFIG_LOONGSON_LIOINTC) += irq-loongson-liointc.o obj-$(CONFIG_LOONGSON_EIOINTC) += irq-loongson-eiointc.o obj-$(CONFIG_LOONGSON_HTPIC) += irq-loongson-htpic.o diff --git a/drivers/irqchip/irq-loongarch-avec.c b/drivers/irqchip/irq-loongarch-avec.c new file mode 100644 index 000000000000..5c9dcc488e21 --- /dev/null +++ b/drivers/irqchip/irq-loongarch-avec.c @@ -0,0 +1,449 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) 2020-2024 Loongson Technologies, Inc. + */ + +#include <linux/cpuhotplug.h> +#include <linux/init.h> +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/irqchip.h> +#include <linux/irqchip/chained_irq.h> +#include <linux/irqdomain.h> +#include <linux/kernel.h> +#include <linux/msi.h> +#include <linux/radix-tree.h> +#include <linux/spinlock.h> + +#include <asm/loongarch.h> +#include <asm/setup.h> + +#include "irq-loongson.h" + +#define VECTORS_PER_REG 64 +#define IRR_VECTOR_MASK 0xffUL +#define IRR_INVALID_MASK 0x80000000UL +#define AVEC_MSG_OFFSET 0x100000 + +#ifdef CONFIG_SMP +struct pending_list { + struct list_head head; +}; + +static struct cpumask intersect_mask; +static DEFINE_PER_CPU(struct pending_list, pending_list); +#endif + +static DEFINE_PER_CPU(struct irq_desc * [NR_VECTORS], irq_map); + +struct avecintc_chip { + raw_spinlock_t lock; + struct fwnode_handle *fwnode; + struct irq_domain *domain; + struct irq_matrix *vector_matrix; + phys_addr_t msi_base_addr; +}; + +static struct avecintc_chip loongarch_avec; + +struct avecintc_data { + struct list_head entry; + unsigned int cpu; + unsigned int vec; + unsigned int prev_cpu; + unsigned int prev_vec; + unsigned int moving; +}; + +static inline void avecintc_ack_irq(struct irq_data *d) +{ +} + +static inline void avecintc_mask_irq(struct irq_data *d) +{ +} + +static inline void avecintc_unmask_irq(struct irq_data *d) +{ +} + +#ifdef CONFIG_SMP +static inline void pending_list_init(int cpu) +{ + struct pending_list *plist = per_cpu_ptr(&pending_list, cpu); + + INIT_LIST_HEAD(&plist->head); +} + +static void avecintc_sync(struct avecintc_data *adata) +{ + struct pending_list *plist; + + if (cpu_online(adata->prev_cpu)) { + plist = per_cpu_ptr(&pending_list, adata->prev_cpu); + list_add_tail(&adata->entry, &plist->head); + adata->moving = 1; + smp_ops.send_ipi_single(adata->prev_cpu, SMP_CLEAR_VECTOR); + } +} + +static int avecintc_set_affinity(struct irq_data *data, const struct cpumask *dest, bool force) +{ + int cpu, ret, vector; + struct avecintc_data *adata; + + raw_spin_lock(&loongarch_avec.lock); + adata = irq_data_get_irq_chip_data(data); + + if (adata->moving) { + raw_spin_unlock(&loongarch_avec.lock); + return -EBUSY; + } + + if (adata->vec == UINT_MAX) { + raw_spin_unlock(&loongarch_avec.lock); + return -EINVAL; + } + + if (cpu_online(adata->cpu) && cpumask_test_cpu(adata->cpu, dest)) { + raw_spin_unlock(&loongarch_avec.lock); + return 0; + } + + cpumask_and(&intersect_mask, dest, cpu_online_mask); + + ret = irq_matrix_alloc(loongarch_avec.vector_matrix, &intersect_mask, false, &cpu); + if (ret < 0) { + raw_spin_unlock(&loongarch_avec.lock); + return ret; + } + + vector = ret; + adata->cpu = cpu; + adata->vec = vector; + per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(data); + avecintc_sync(adata); + + raw_spin_unlock(&loongarch_avec.lock); + irq_data_update_effective_affinity(data, cpumask_of(cpu)); + + return IRQ_SET_MASK_OK; +} + +static int avecintc_cpu_online(unsigned int cpu) +{ + if (!loongarch_avec.vector_matrix) + return 0; + + raw_spin_lock(&loongarch_avec.lock); + + irq_matrix_online(loongarch_avec.vector_matrix); + + pending_list_init(cpu); + + raw_spin_unlock(&loongarch_avec.lock); + + return 0; +} + +static int avecintc_cpu_offline(unsigned int cpu) +{ + struct pending_list *plist = per_cpu_ptr(&pending_list, cpu); + + if (!loongarch_avec.vector_matrix) + return 0; + + raw_spin_lock(&loongarch_avec.lock); + + if (!list_empty(&plist->head)) + pr_warn("CPU#%d vector is busy\n", cpu); + irq_matrix_offline(loongarch_avec.vector_matrix); + + raw_spin_unlock(&loongarch_avec.lock); + + return 0; +} + +void complete_irq_moving(void) +{ + struct pending_list *plist = this_cpu_ptr(&pending_list); + struct avecintc_data *adata, *tdata; + int cpu, vector, bias; + uint64_t isr; + + raw_spin_lock(&loongarch_avec.lock); + + list_for_each_entry_safe(adata, tdata, &plist->head, entry) { + cpu = adata->prev_cpu; + vector = adata->prev_vec; + bias = vector / VECTORS_PER_REG; + switch (bias) { + case 0: + isr = csr_read64(LOONGARCH_CSR_ISR0); + break; + case 1: + isr = csr_read64(LOONGARCH_CSR_ISR1); + break; + case 2: + isr = csr_read64(LOONGARCH_CSR_ISR2); + break; + case 3: + isr = csr_read64(LOONGARCH_CSR_ISR3); + break; + } + + if (isr & (1UL << (vector % VECTORS_PER_REG))) { + smp_ops.send_ipi_single(cpu, SMP_CLEAR_VECTOR); + continue; + } + list_del(&adata->entry); + irq_matrix_free(loongarch_avec.vector_matrix, cpu, vector, false); + this_cpu_write(irq_map[vector], NULL); + adata->moving = 0; + adata->prev_cpu = adata->cpu; + adata->prev_vec = adata->vec; + } + + raw_spin_unlock(&loongarch_avec.lock); +} +#endif + +static void avecintc_compose_msi_msg(struct irq_data *d, struct msi_msg *msg) +{ + struct avecintc_data *adata = irq_data_get_irq_chip_data(d); + + msg->address_hi = 0x0; + msg->address_lo = (loongarch_avec.msi_base_addr | (adata->vec & 0xff) << 4) + | ((cpu_logical_map(adata->cpu & 0xffff)) << 12); + msg->data = 0x0; +} + +static struct irq_chip avec_irq_controller = { + .name = "AVECINTC", + .irq_ack = avecintc_ack_irq, + .irq_mask = avecintc_mask_irq, + .irq_unmask = avecintc_unmask_irq, +#ifdef CONFIG_SMP + .irq_set_affinity = avecintc_set_affinity, +#endif + .irq_compose_msi_msg = avecintc_compose_msi_msg, +}; + +static void avecintc_irq_dispatch(struct irq_desc *desc) +{ + struct irq_chip *chip = irq_desc_get_chip(desc); + struct irq_desc *d; + + chained_irq_enter(chip, desc); + + while (true) { + unsigned long vector = csr_read64(LOONGARCH_CSR_IRR); + if (vector & IRR_INVALID_MASK) + break; + + vector &= IRR_VECTOR_MASK; + + d = this_cpu_read(irq_map[vector]); + if (d) { + generic_handle_irq_desc(d); + } else { + spurious_interrupt(); + pr_warn("Unexpected IRQ occurs on CPU#%d [vector %ld]\n", smp_processor_id(), vector); + } + } + + chained_irq_exit(chip, desc); +} + +static int avecintc_alloc_vector(struct irq_data *irqd, struct avecintc_data *adata) +{ + int cpu, ret; + unsigned long flags; + + raw_spin_lock_irqsave(&loongarch_avec.lock, flags); + + ret = irq_matrix_alloc(loongarch_avec.vector_matrix, cpu_online_mask, false, &cpu); + if (ret < 0) { + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + return ret; + } + + adata->prev_cpu = adata->cpu = cpu; + adata->prev_vec = adata->vec = ret; + per_cpu_ptr(irq_map, adata->cpu)[adata->vec] = irq_data_to_desc(irqd); + + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + + return 0; +} + +static int avecintc_domain_alloc(struct irq_domain *domain, + unsigned int virq, unsigned int nr_irqs, void *arg) +{ + for (unsigned int i = 0; i < nr_irqs; i++) { + struct irq_data *irqd = irq_domain_get_irq_data(domain, virq + i); + struct avecintc_data *adata = kzalloc(sizeof(*adata), GFP_KERNEL); + int ret; + + if (!adata) + return -ENOMEM; + + ret = avecintc_alloc_vector(irqd, adata); + if (ret < 0) { + kfree(adata); + return ret; + } + + irq_domain_set_info(domain, virq + i, virq + i, &avec_irq_controller, + adata, handle_edge_irq, NULL, NULL); + irqd_set_single_target(irqd); + irqd_set_affinity_on_activate(irqd); + } + + return 0; +} + +static void avecintc_free_vector(struct irq_data *irqd, struct avecintc_data *adata) +{ + unsigned long flags; + + raw_spin_lock_irqsave(&loongarch_avec.lock, flags); + + per_cpu(irq_map, adata->cpu)[adata->vec] = NULL; + irq_matrix_free(loongarch_avec.vector_matrix, adata->cpu, adata->vec, false); + +#ifdef CONFIG_SMP + if (!adata->moving) { + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); + return; + } + + per_cpu(irq_map, adata->prev_cpu)[adata->prev_vec] = NULL; + irq_matrix_free(loongarch_avec.vector_matrix, adata->prev_cpu, adata->prev_vec, false); + list_del_init(&adata->entry); +#endif + raw_spin_unlock_irqrestore(&loongarch_avec.lock, flags); +} + +static void avecintc_domain_free(struct irq_domain *domain, + unsigned int virq, unsigned int nr_irqs) +{ + for (unsigned int i = 0; i < nr_irqs; i++) { + struct irq_data *d = irq_domain_get_irq_data(domain, virq + i); + + if (d) { + struct avecintc_data *adata = irq_data_get_irq_chip_data(d); + + avecintc_free_vector(d, adata); + irq_domain_reset_irq_data(d); + kfree(adata); + } + } +} + +static const struct irq_domain_ops avecintc_domain_ops = { + .alloc = avecintc_domain_alloc, + .free = avecintc_domain_free, +}; + +static int __init irq_matrix_init(void) +{ + loongarch_avec.vector_matrix = irq_alloc_matrix(NR_VECTORS, 0, NR_VECTORS); + if (!loongarch_avec.vector_matrix) + return -ENOMEM; + + for (int i = 0; i < NR_LEGACY_VECTORS; i++) + irq_matrix_assign_system(loongarch_avec.vector_matrix, i, false); + + irq_matrix_online(loongarch_avec.vector_matrix); + + return 0; +} + +static int __init avecintc_init(struct irq_domain *parent) +{ + int ret, parent_irq; + unsigned long value; + + raw_spin_lock_init(&loongarch_avec.lock); + + loongarch_avec.fwnode = irq_domain_alloc_named_fwnode("AVECINTC"); + if (!loongarch_avec.fwnode) { + pr_err("Unable to allocate domain handle\n"); + ret = -ENOMEM; + goto out; + } + + loongarch_avec.domain = irq_domain_create_tree(loongarch_avec.fwnode, + &avecintc_domain_ops, NULL); + if (!loongarch_avec.domain) { + pr_err("Unable to create IRQ domain\n"); + ret = -ENOMEM; + goto out_free_handle; + } + + parent_irq = irq_create_mapping(parent, INT_AVEC); + if (!parent_irq) { + pr_err("Failed to mapping hwirq\n"); + ret = -EINVAL; + goto out_remove_domain; + } + + ret = irq_matrix_init(); + if (ret < 0) { + pr_err("Failed to init irq matrix\n"); + goto out_remove_domain; + } + irq_set_chained_handler_and_data(parent_irq, avecintc_irq_dispatch, NULL); + +#ifdef CONFIG_SMP + pending_list_init(0); + cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_AVECINTC_STARTING, + "irqchip/loongarch/avecintc:starting", + avecintc_cpu_online, avecintc_cpu_offline); +#endif + value = iocsr_read64(LOONGARCH_IOCSR_MISC_FUNC); + value |= IOCSR_MISC_FUNC_AVEC_EN; + iocsr_write64(value, LOONGARCH_IOCSR_MISC_FUNC); + + return ret; + +out_remove_domain: + irq_domain_remove(loongarch_avec.domain); +out_free_handle: + irq_domain_free_fwnode(loongarch_avec.fwnode); +out: + return ret; +} + +static int __init pch_msi_parse_madt(union acpi_subtable_headers *header, + const unsigned long end) +{ + struct acpi_madt_msi_pic *pchmsi_entry = (struct acpi_madt_msi_pic *)header; + + loongarch_avec.msi_base_addr = pchmsi_entry->msg_address - AVEC_MSG_OFFSET; + + return pch_msi_acpi_init_avec(loongarch_avec.domain); +} + +static inline int __init acpi_cascade_irqdomain_init(void) +{ + return acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1); +} + +int __init avecintc_acpi_init(struct irq_domain *parent) +{ + int ret = avecintc_init(parent); + if (ret < 0) { + pr_err("Failed to init IRQ domain\n"); + return ret; + } + + ret = acpi_cascade_irqdomain_init(); + if (ret < 0) { + pr_err("Failed to init cascade IRQ domain\n"); + return ret; + } + + return ret; +} diff --git a/drivers/irqchip/irq-loongarch-cpu.c b/drivers/irqchip/irq-loongarch-cpu.c index 4380b4d8dd20..c6e0c9849ba9 100644 --- a/drivers/irqchip/irq-loongarch-cpu.c +++ b/drivers/irqchip/irq-loongarch-cpu.c @@ -13,6 +13,8 @@ #include <asm/loongarch.h> #include <asm/setup.h> +#include "irq-loongson.h" + static struct irq_domain *irq_domain; struct fwnode_handle *cpuintc_handle; @@ -140,7 +142,10 @@ static int __init acpi_cascade_irqdomain_init(void) if (r < 0) return r; - return 0; + if (cpu_has_avecint) + r = avecintc_acpi_init(irq_domain); + + return r; } struct irq_domain *get_cpudomain(void) diff --git a/drivers/irqchip/irq-loongson-eiointc.c b/drivers/irqchip/irq-loongson-eiointc.c index 488951a740cd..1c7f97dac83b 100644 --- a/drivers/irqchip/irq-loongson-eiointc.c +++ b/drivers/irqchip/irq-loongson-eiointc.c @@ -16,6 +16,8 @@ #include <linux/kernel.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + #define EIOINTC_REG_NODEMAP 0x14a0 #define EIOINTC_REG_IPMAP 0x14c0 #define EIOINTC_REG_ENABLE 0x1600 @@ -395,6 +397,9 @@ static int __init acpi_cascade_irqdomain_init(void) if (r < 0) return r; + if (cpu_has_avecint) + return 0; + r = acpi_table_parse_madt(ACPI_MADT_TYPE_MSI_PIC, pch_msi_parse_madt, 1); if (r < 0) return r; @@ -442,8 +447,8 @@ static int __init eiointc_init(struct eiointc_priv *priv, int parent_irq, if (nr_pics == 1) { register_syscore_ops(&eiointc_syscore_ops); - cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_LOONGARCH_STARTING, - "irqchip/loongarch/intc:starting", + cpuhp_setup_state_nocalls(CPUHP_AP_IRQ_EIOINTC_STARTING, + "irqchip/loongarch/eiointc:starting", eiointc_router_init, NULL); } diff --git a/drivers/irqchip/irq-loongson-htvec.c b/drivers/irqchip/irq-loongson-htvec.c index 0bff728b25e3..5da02c7ad0b3 100644 --- a/drivers/irqchip/irq-loongson-htvec.c +++ b/drivers/irqchip/irq-loongson-htvec.c @@ -17,6 +17,8 @@ #include <linux/of_irq.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define HTVEC_EN_OFF 0x20 #define HTVEC_MAX_PARENT_IRQ 8 diff --git a/drivers/irqchip/irq-loongson-liointc.c b/drivers/irqchip/irq-loongson-liointc.c index 0262cbefe9dd..257aa699ec98 100644 --- a/drivers/irqchip/irq-loongson-liointc.c +++ b/drivers/irqchip/irq-loongson-liointc.c @@ -22,6 +22,8 @@ #include <asm/loongson.h> #endif +#include "irq-loongson.h" + #define LIOINTC_CHIP_IRQ 32 #define LIOINTC_NUM_PARENT 4 #define LIOINTC_NUM_CORES 4 diff --git a/drivers/irqchip/irq-loongson-pch-lpc.c b/drivers/irqchip/irq-loongson-pch-lpc.c index 9b35492fb6be..2d4c3ec128b8 100644 --- a/drivers/irqchip/irq-loongson-pch-lpc.c +++ b/drivers/irqchip/irq-loongson-pch-lpc.c @@ -15,6 +15,8 @@ #include <linux/kernel.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define LPC_INT_CTL 0x00 #define LPC_INT_ENA 0x04 diff --git a/drivers/irqchip/irq-loongson-pch-msi.c b/drivers/irqchip/irq-loongson-pch-msi.c index dd4d699170f4..2c9f58536fce 100644 --- a/drivers/irqchip/irq-loongson-pch-msi.c +++ b/drivers/irqchip/irq-loongson-pch-msi.c @@ -15,6 +15,8 @@ #include <linux/pci.h> #include <linux/slab.h> +#include "irq-loongson.h" + static int nr_pics; struct pch_msi_data { @@ -266,17 +268,17 @@ IRQCHIP_DECLARE(pch_msi, "loongson,pch-msi-1.0", pch_msi_of_init); #ifdef CONFIG_ACPI struct fwnode_handle *get_pch_msi_handle(int pci_segment) { - int i; + if (cpu_has_avecint) + return pch_msi_handle[0]; - for (i = 0; i < MAX_IO_PICS; i++) { + for (int i = 0; i < MAX_IO_PICS; i++) { if (msi_group[i].pci_segment == pci_segment) return pch_msi_handle[i]; } - return NULL; + return pch_msi_handle[0]; } -int __init pch_msi_acpi_init(struct irq_domain *parent, - struct acpi_madt_msi_pic *acpi_pchmsi) +int __init pch_msi_acpi_init(struct irq_domain *parent, struct acpi_madt_msi_pic *acpi_pchmsi) { int ret; struct fwnode_handle *domain_handle; @@ -289,4 +291,36 @@ int __init pch_msi_acpi_init(struct irq_domain *parent, return ret; } + +static struct irq_chip pch_msi_irq_chip_avec = { + .name = "PCH PCI MSI", + .irq_ack = irq_chip_ack_parent, +}; + +static struct msi_domain_info pch_msi_domain_info_avec = { + .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | + MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + .chip = &pch_msi_irq_chip_avec, +}; + +int __init pch_msi_acpi_init_avec(struct irq_domain *parent) +{ + struct irq_domain *msi_domain; + + if (pch_msi_handle[0]) + return 0; + + pch_msi_handle[0] = parent->fwnode; + irq_domain_update_bus_token(parent, DOMAIN_BUS_NEXUS); + + msi_domain = pci_msi_create_irq_domain(pch_msi_handle[0], + &pch_msi_domain_info_avec, parent); + if (!msi_domain) { + pr_err("Failed to create PCI MSI domain\n"); + kfree(pch_msi_handle[0]); + return -ENOMEM; + } + + return 0; +} #endif diff --git a/drivers/irqchip/irq-loongson-pch-pic.c b/drivers/irqchip/irq-loongson-pch-pic.c index 6e4dde64687e..075d08aba75e 100644 --- a/drivers/irqchip/irq-loongson-pch-pic.c +++ b/drivers/irqchip/irq-loongson-pch-pic.c @@ -17,6 +17,8 @@ #include <linux/of_irq.h> #include <linux/syscore_ops.h> +#include "irq-loongson.h" + /* Registers */ #define PCH_PIC_MASK 0x20 #define PCH_PIC_HTMSI_EN 0x40 diff --git a/drivers/irqchip/irq-loongson.h b/drivers/irqchip/irq-loongson.h new file mode 100644 index 000000000000..11fa138d1f44 --- /dev/null +++ b/drivers/irqchip/irq-loongson.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (C) 2024 Loongson Technology Corporation Limited + */ + +#ifndef _DRIVERS_IRQCHIP_IRQ_LOONGSON_H +#define _DRIVERS_IRQCHIP_IRQ_LOONGSON_H + +int find_pch_pic(u32 gsi); + +int liointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lio_pic *acpi_liointc); +int eiointc_acpi_init(struct irq_domain *parent, + struct acpi_madt_eio_pic *acpi_eiointc); +int avecintc_acpi_init(struct irq_domain *parent); + +int htvec_acpi_init(struct irq_domain *parent, + struct acpi_madt_ht_pic *acpi_htvec); +int pch_lpc_acpi_init(struct irq_domain *parent, + struct acpi_madt_lpc_pic *acpi_pchlpc); +int pch_pic_acpi_init(struct irq_domain *parent, + struct acpi_madt_bio_pic *acpi_pchpic); +int pch_msi_acpi_init(struct irq_domain *parent, + struct acpi_madt_msi_pic *acpi_pchmsi); +int pch_msi_acpi_init_avec(struct irq_domain *parent); + +#endif /* _DRIVERS_IRQCHIP_IRQ_LOONGSON_H */ diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h index a33500a53d25..0a7c8725b0ca 100644 --- a/include/linux/cpuhotplug.h +++ b/include/linux/cpuhotplug.h @@ -152,7 +152,8 @@ enum cpuhp_state { CPUHP_AP_IRQ_BCM2836_STARTING, CPUHP_AP_IRQ_MIPS_GIC_STARTING, CPUHP_AP_IRQ_RISCV_STARTING, - CPUHP_AP_IRQ_LOONGARCH_STARTING, + CPUHP_AP_IRQ_EIOINTC_STARTING, + CPUHP_AP_IRQ_AVECINTC_STARTING, CPUHP_AP_IRQ_SIFIVE_PLIC_STARTING, CPUHP_AP_ARM_MVEBU_COHERENCY, CPUHP_AP_MICROCODE_LOADER, -- 2.33.0

2 1

[PATCH OLK-6.6 0/4] LoongArch: fix some 3C6000&LS7A related problems
by Hongchen Zhang 23 Oct '24

23 Oct '24

gaojuxin (1): LoongArch: Add workaround for 3C6000 about io wr/rd wusheng (1): pci/quirks: LS7A2000 enble msi zhangtianyang (1): Loongarch: Dynamic enable writecombine zhenghaowei (1): acpi: mcfg quirk: Increased multi-chip support for the 3C6000 arch/loongarch/include/asm/irq.h | 4 +- arch/loongarch/kernel/setup.c | 30 ++++++++++- arch/loongarch/kernel/smp.c | 20 ++++++- drivers/acpi/pci_mcfg.c | 13 +++++ drivers/pci/quirks.c | 13 +++++ include/asm-generic/io.h | 90 ++++++++++++++++++++++++++++++-- include/drm/drm_cache.h | 2 +- 7 files changed, 162 insertions(+), 10 deletions(-) -- 2.33.0

2 5

[PATCH OLK-6.6] LoongArch: fix dma-direct.h not found
by Hongchen Zhang 23 Oct '24

23 Oct '24

LoongArch inclusion category: bugfix bugzilla: https://gitee.com/openeuler/kernel/issues/IAZ2YN CVE: NA -------------------------------- This patch fix the compile error: dma-direct.h not found Fixes: 90d2bbc1ade2 ("LoongArch: use arch specific phys_to_dma") Signed-off-by: Hongchen Zhang <zhanghongchen(a)loongson.cn> --- arch/loongarch/include/asm/dma-direct.h | 21 +++++++++++++++++++++ arch/loongarch/kernel/dma.c | 17 ++--------------- 2 files changed, 23 insertions(+), 15 deletions(-) create mode 100644 arch/loongarch/include/asm/dma-direct.h diff --git a/arch/loongarch/include/asm/dma-direct.h b/arch/loongarch/include/asm/dma-direct.h new file mode 100644 index 000000000000..a8fbae3bd5da --- /dev/null +++ b/arch/loongarch/include/asm/dma-direct.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_LOONGARCH_DMA_DIRECT_H +#define _ASM_LOONGARCH_DMA_DIRECT_H + +extern int node_id_offset; + +static inline dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) +{ + long nid = (paddr >> 44) & 0xf; + + return ((nid << 44) ^ paddr) | (nid << node_id_offset); +} + +static inline phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) +{ + long nid = (daddr >> node_id_offset) & 0xf; + + return ((nid << node_id_offset) ^ daddr) | (nid << 44); +} + +#endif /* _ASM_LOONGARCH_DMA_DIRECT_H */ diff --git a/arch/loongarch/kernel/dma.c b/arch/loongarch/kernel/dma.c index cc0ccde58db8..34836408b15a 100644 --- a/arch/loongarch/kernel/dma.c +++ b/arch/loongarch/kernel/dma.c @@ -11,21 +11,8 @@ * 48bit physical address space and embed it into 40bit. */ -static int node_id_offset; - -dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr) -{ - long nid = (paddr >> 44) & 0xf; - - return ((nid << 44) ^ paddr) | (nid << node_id_offset); -} - -phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr) -{ - long nid = (daddr >> node_id_offset) & 0xf; - - return ((nid << node_id_offset) ^ daddr) | (nid << 44); -} +int node_id_offset; +EXPORT_SYMBOL_GPL(node_id_offset); void acpi_arch_dma_setup(struct device *dev) { -- 2.33.0

2 1