On 9/3/2020 9:33 PM, Haibin Zhang wrote:
Hi, YingFang.
I configure 4K PAGE_SIZE in order to use 1G hugepage in host. But VM cannot bring up secondary CPU
Start VM using follow command: qemu-system-aarch64 -name guest=12345,debug-threads=on \ -machine virt,accel=kvm,usb=off,dump-guest-core=off,gic-version=3 \ -cpu host \ -m 122880 \ -object iothread,id=iothread1 \ -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages,share=yes,size=64424509440,host-nodes=0,policy=preferred \ -numa node,nodeid=0,cpus=0-29,memdev=ram-node0 \ -object memory-backend-file,id=ram-node1,prealloc=yes,mem-path=/dev/hugepages,share=yes,size=64424509440,host-nodes=1,policy=preferred \ -numa node,nodeid=1,cpus=30-59,memdev=ram-node1 \ -smp sockets=2,cores=30,threads=1 \ -bios /usr/share/AAVMF/AAVMF_CODE.fd \ -drive file=/data/haibin/centos8.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none \ -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \ -net none -serial telnet::9001,server,nowait -monitor stdio
Thanks for your reporting, I will CC Zeyu Jin to lookup into it.
VM kernel logs: [ 0.010708] Mountpoint-cache hash table entries: 65536 (order: 3, 524288 bytes) [ 0.012517] ASID allocator initialised with 32768 entries [ 0.013105] rcu: Hierarchical SRCU implementation. [ 0.013843] Platform MSI: ITS@0x8080000 domain created [ 0.014375] PCI/MSI: ITS@0x8080000 domain created [ 0.014850] Remapping and enabling EFI services. [ 0.015946] smp: Bringing up secondary CPUs ... [ 1.039238] CPU1: failed to come online [ 1.039698] CPU1: failed in unknown state : 0x0 [ 1.051285] Detected VIPT I-cache on CPU2 [ 1.051303] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 [ 1.051520] GICv3: CPU2: using allocated LPI pending table @0x00000003c06a0000 [ 1.051545] CPU2: Booted secondary processor 0x0000000001 [0x481fd010] [ 2.079879] CPU3: failed to come online [ 2.082736] CPU3: failed in unknown state : 0x0 [ 2.085620] Detected VIPT I-cache on CPU4 [ 2.085638] GICv3: CPU4: found redistributor 4 region 0:0x0000000008120000 [ 2.085785] GICv3: CPU4: using allocated LPI pending table @0x00000003c06c0000 [ 2.085808] CPU4: Booted secondary processor 0x0000000002 [0x481fd010] [ 3.120358] Detected VIPT I-cache on CPU5 [ 3.120379] GICv3: CPU5: found redistributor 5 region 0:0x0000000008140000 [ 3.120607] GICv3: CPU5: using allocated LPI pending table @0x00000003c06d0000 [ 3.120619] CPU5: failed to come online [ 3.120632] CPU5: Booted secondary processor 0x0000000003 [0x481fd010] [ 3.121069] ------------[ cut here ]------------ [ 3.126345] kernel BUG at kernel/irq_work.c:147! [ 3.126797] Internal error: Oops - BUG: 0 [#1] SMP [ 3.127268] Modules linked in: [ 3.127568] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.18.0-193.14.2.el8_2.aarch64 #1 [ 3.128345] pstate: a0000005 (NzCv daif -PAN -UAO) [ 3.128818] pc : irq_work_run_list+0xa8/0xb8 [ 3.129239] lr : irq_work_run+0x24/0x48 [ 3.129615] sp : ffff000012aefc40 [ 3.129940] x29: ffff000012aefc40 x28: 0000000000000000 [ 3.130458] x27: 0000000000000000 x26: 0000000000000000 [ 3.131040] x25: 00008003ac000000 x24: ffff8003bd202130 [ 3.131592] x23: 0000000000000000 x22: 0000000000000000 [ 3.132119] x21: 0000000000000000 x20: ffff00001128e4c0 [ 3.132677] x19: ffff00001128a400 x18: 0000000000000010 [ 3.133247] x17: 0000000000000000 x16: ffff80038237ba00 [ 3.133816] x15: ffffffffffffffff x14: ffff0000115d3708 [ 3.134391] x13: ffff000092aef9a7 x12: ffff000012aef9b0 [ 3.134991] x11: ffff00001160f000 x10: ffff000012aef930 [ 3.135567] x9 : 00000000ffffffd0 x8 : ffff0000105dab10 [ 3.136128] x7 : 000000000000000d x6 : ffff000011fc696c [ 3.136685] x5 : 0000000000000015 x4 : 0000000000000000 [ 3.137258] x3 : ffff8003bcf6e4c0 x2 : 0000000000000000 [ 3.137825] x1 : 0000000000000000 x0 : ffff8003bcf6a400 [ 3.138405] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____)) [ 3.139096] Call trace: [ 3.139364] irq_work_run_list+0xa8/0xb8 [ 3.139773] irq_work_run+0x24/0x48 [ 3.140180] flush_smp_call_function_queue+0xa4/0x150 [ 3.140714] smpcfd_dying_cpu+0x18/0x28 [ 3.141134] cpuhp_invoke_callback+0xa8/0x618 [ 3.141621] _cpu_up+0x1c0/0x1d0 [ 3.142013] do_cpu_up+0x7c/0xb8 [ 3.142377] cpu_up+0x24/0x30 [ 3.142717] smp_init+0xac/0x120 [ 3.143089] kernel_init_freeable+0x168/0x314 [ 3.143559] kernel_init+0x18/0x10c [ 3.143942] ret_from_fork+0x10/0x18 [ 3.144336] Code: a8c37bfd d65f03c0 a90153f3 a9025bf5 (d4210000) [ 3.145028] ---[ end trace 392329bbeab599c5 ]--- [ 3.145548] Kernel panic - not syncing: Fatal exception [ 3.146149] SMP: stopping secondary CPUs [ 4.183106] SMP: failed to stop secondary CPUs 0,2,5 [ 4.183677] ---[ end Kernel panic - not syncing: Fatal exception ]---.