On 9/3/2020 9:33 PM, Haibin Zhang wrote:
> Hi, YingFang.
>
> I configure 4K PAGE_SIZE in order to use 1G hugepage in host. But VM cannot bring up secondary CPU
>
> Start VM using follow command:
> qemu-system-aarch64 -name guest=12345,debug-threads=on \
> -machine virt,accel=kvm,usb=off,dump-guest-core=off,gic-version=3 \
> -cpu host \
> -m 122880 \
> -object iothread,id=iothread1 \
> -object memory-backend-file,id=ram-node0,prealloc=yes,mem-path=/dev/hugepages,share=yes,size=64424509440,host-nodes=0,policy=preferred \
> -numa node,nodeid=0,cpus=0-29,memdev=ram-node0 \
> -object memory-backend-file,id=ram-node1,prealloc=yes,mem-path=/dev/hugepages,share=yes,size=64424509440,host-nodes=1,policy=preferred \
> -numa node,nodeid=1,cpus=30-59,memdev=ram-node1 \
> -smp sockets=2,cores=30,threads=1 \
> -bios /usr/share/AAVMF/AAVMF_CODE.fd \
> -drive file=/data/haibin/centos8.qcow2,format=qcow2,if=none,id=drive-virtio-disk0,cache=none \
> -device virtio-blk-pci,scsi=off,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
> -net none -serial telnet::9001,server,nowait -monitor stdio
>
Thanks for your reporting, I will CC Zeyu Jin to lookup into it.
> VM kernel logs:
> [ 0.010708] Mountpoint-cache hash table entries: 65536 (order: 3, 524288 bytes)
> [ 0.012517] ASID allocator initialised with 32768 entries
> [ 0.013105] rcu: Hierarchical SRCU implementation.
> [ 0.013843] Platform MSI: ITS@0x8080000 domain created
> [ 0.014375] PCI/MSI: ITS@0x8080000 domain created
> [ 0.014850] Remapping and enabling EFI services.
> [ 0.015946] smp: Bringing up secondary CPUs ...
> [ 1.039238] CPU1: failed to come online
> [ 1.039698] CPU1: failed in unknown state : 0x0
> [ 1.051285] Detected VIPT I-cache on CPU2
> [ 1.051303] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000
> [ 1.051520] GICv3: CPU2: using allocated LPI pending table @0x00000003c06a0000
> [ 1.051545] CPU2: Booted secondary processor 0x0000000001 [0x481fd010]
> [ 2.079879] CPU3: failed to come online
> [ 2.082736] CPU3: failed in unknown state : 0x0
> [ 2.085620] Detected VIPT I-cache on CPU4
> [ 2.085638] GICv3: CPU4: found redistributor 4 region 0:0x0000000008120000
> [ 2.085785] GICv3: CPU4: using allocated LPI pending table @0x00000003c06c0000
> [ 2.085808] CPU4: Booted secondary processor 0x0000000002 [0x481fd010]
> [ 3.120358] Detected VIPT I-cache on CPU5
> [ 3.120379] GICv3: CPU5: found redistributor 5 region 0:0x0000000008140000
> [ 3.120607] GICv3: CPU5: using allocated LPI pending table @0x00000003c06d0000
> [ 3.120619] CPU5: failed to come online
> [ 3.120632] CPU5: Booted secondary processor 0x0000000003 [0x481fd010]
> [ 3.121069] ------------[ cut here ]------------
> [ 3.126345] kernel BUG at kernel/irq_work.c:147!
> [ 3.126797] Internal error: Oops - BUG: 0 [#1] SMP
> [ 3.127268] Modules linked in:
> [ 3.127568] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.18.0-193.14.2.el8_2.aarch64 #1
> [ 3.128345] pstate: a0000005 (NzCv daif -PAN -UAO)
> [ 3.128818] pc : irq_work_run_list+0xa8/0xb8
> [ 3.129239] lr : irq_work_run+0x24/0x48
> [ 3.129615] sp : ffff000012aefc40
> [ 3.129940] x29: ffff000012aefc40 x28: 0000000000000000
> [ 3.130458] x27: 0000000000000000 x26: 0000000000000000
> [ 3.131040] x25: 00008003ac000000 x24: ffff8003bd202130
> [ 3.131592] x23: 0000000000000000 x22: 0000000000000000
> [ 3.132119] x21: 0000000000000000 x20: ffff00001128e4c0
> [ 3.132677] x19: ffff00001128a400 x18: 0000000000000010
> [ 3.133247] x17: 0000000000000000 x16: ffff80038237ba00
> [ 3.133816] x15: ffffffffffffffff x14: ffff0000115d3708
> [ 3.134391] x13: ffff000092aef9a7 x12: ffff000012aef9b0
> [ 3.134991] x11: ffff00001160f000 x10: ffff000012aef930
> [ 3.135567] x9 : 00000000ffffffd0 x8 : ffff0000105dab10
> [ 3.136128] x7 : 000000000000000d x6 : ffff000011fc696c
> [ 3.136685] x5 : 0000000000000015 x4 : 0000000000000000
> [ 3.137258] x3 : ffff8003bcf6e4c0 x2 : 0000000000000000
> [ 3.137825] x1 : 0000000000000000 x0 : ffff8003bcf6a400
> [ 3.138405] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
> [ 3.139096] Call trace:
> [ 3.139364] irq_work_run_list+0xa8/0xb8
> [ 3.139773] irq_work_run+0x24/0x48
> [ 3.140180] flush_smp_call_function_queue+0xa4/0x150
> [ 3.140714] smpcfd_dying_cpu+0x18/0x28
> [ 3.141134] cpuhp_invoke_callback+0xa8/0x618
> [ 3.141621] _cpu_up+0x1c0/0x1d0
> [ 3.142013] do_cpu_up+0x7c/0xb8
> [ 3.142377] cpu_up+0x24/0x30
> [ 3.142717] smp_init+0xac/0x120
> [ 3.143089] kernel_init_freeable+0x168/0x314
> [ 3.143559] kernel_init+0x18/0x10c
> [ 3.143942] ret_from_fork+0x10/0x18
> [ 3.144336] Code: a8c37bfd d65f03c0 a90153f3 a9025bf5 (d4210000)
> [ 3.145028] ---[ end trace 392329bbeab599c5 ]---
> [ 3.145548] Kernel panic - not syncing: Fatal exception
> [ 3.146149] SMP: stopping secondary CPUs
> [ 4.183106] SMP: failed to stop secondary CPUs 0,2,5
> [ 4.183677] ---[ end Kernel panic - not syncing: Fatal exception ]---.
>
