Greeting, FYI, we noticed a -25.2% regression of stress-ng.sock.ops_per_sec due to commit: commit: 5f94d1b650d14e01f383e3d02e40c13a9ccecaf4 ("[PATCH] sched/fair: don't use waker's cpu if the waker of sync wake-up is interrupt") url: https://github.com/0day-ci/linux/commits/Barry-Song/sched-fair-don-t-use-wak... base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 2ea46c6fc9452ac100ad907b051d797225847e33 in testcase: stress-ng on test machine: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory with following parameters: nr_threads: 10% disk: 1HDD testtime: 60s fs: ext4 class: os test: sock cpufreq_governor: performance ucode: 0x5003006 In addition to that, the commit also has significant impact on the following tests: +------------------+-------------------------------------------------------------------------------------+ | testcase: change | apachebench: apachebench.requests_per_second -5.6% regression | | test machine | 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory | | test parameters | cluster=cs-localhost | | | concurrency=8000 | | | cpufreq_governor=performance | | | runtime=300s | | | ucode=0x5003006 | +------------------+-------------------------------------------------------------------------------------+ | testcase: change | netperf: netperf.Throughput_Mbps -52.4% regression | | test machine | 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory | | test parameters | cluster=cs-localhost | | | cpufreq_governor=performance | | | ip=ipv4 | | | nr_threads=1 | | | runtime=300s | | | test=UDP_STREAM | | | ucode=0x5003006 | +------------------+-------------------------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang@intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run bin/lkp run generated-yaml-file ========================================================================================= class/compiler/cpufreq_governor/disk/fs/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime/ucode: os/gcc-9/performance/1HDD/ext4/x86_64-rhel-8.3/10%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2sp5/sock/stress-ng/60s/0x5003006 commit: 2ea46c6fc9 ("cpumask/hotplug: Fix cpu_dying() state tracking") 5f94d1b650 ("sched/fair: don't use waker's cpu if the waker of sync wake-up is interrupt") 2ea46c6fc9452ac1 5f94d1b650d14e01f383e3d02e4 ---------------- --------------------------- %stddev %change %stddev \ | \ 668165 -25.2% 499709 ± 7% stress-ng.sock.ops 11135 -25.2% 8328 ± 7% stress-ng.sock.ops_per_sec 2633 ± 5% -18.0% 2159 ± 8% stress-ng.time.involuntary_context_switches 11225 ± 2% -6.1% 10545 stress-ng.time.minor_page_faults 1283 -0.9% 1272 stress-ng.time.percent_of_cpu_this_job_got 32.76 -21.3% 25.78 ± 6% stress-ng.time.user_time 39827551 -9.3% 36139210 ± 4% stress-ng.time.voluntary_context_switches 0.53 -0.1 0.44 ± 4% mpstat.cpu.all.usr% 1218481 -8.8% 1111783 ± 4% vmstat.system.cs 36137621 +30.6% 47196861 ± 12% cpuidle.C1.time 9079420 +19.3% 10827569 ± 8% cpuidle.C1.usage 73682855 -15.0% 62612480 ± 6% cpuidle.POLL.time 30399709 -17.5% 25089524 ± 9% cpuidle.POLL.usage 20602586 ± 4% -23.7% 15709836 ± 12% numa-numastat.node0.local_node 20665440 ± 4% -23.8% 15744289 ± 11% numa-numastat.node0.numa_hit 21017060 ± 3% -25.6% 15638989 ± 12% numa-numastat.node1.local_node 21040859 ± 3% -25.4% 15691162 ± 12% numa-numastat.node1.numa_hit 11110566 -22.4% 8621534 ± 10% numa-vmstat.node0.numa_hit 11047694 -22.3% 8586688 ± 10% numa-vmstat.node0.numa_local 10652347 -23.7% 8131841 ± 8% numa-vmstat.node1.numa_hit 10462586 -24.4% 7914174 ± 9% numa-vmstat.node1.numa_local 6200 ± 2% -11.7% 5472 ± 4% slabinfo.dmaengine-unmap-16.active_objs 6204 ± 2% -11.7% 5478 ± 4% slabinfo.dmaengine-unmap-16.num_objs 8326 ± 2% -24.8% 6257 ± 8% slabinfo.sock_inode_cache.active_objs 8326 ± 2% -24.8% 6257 ± 8% slabinfo.sock_inode_cache.num_objs 9070645 +19.3% 10820329 ± 8% turbostat.C1 0.58 +0.2 0.77 ± 12% turbostat.C1% 54.00 +4.8% 56.57 turbostat.PkgTmp 202.21 +1.4% 205.06 turbostat.PkgWatt 80.33 +17.3% 94.21 ± 3% turbostat.RAMWatt 23149 -2.7% 22523 proc-vmstat.nr_slab_reclaimable 1054 ± 25% -59.0% 432.43 ± 20% proc-vmstat.numa_hint_faults 41537249 -24.9% 31186035 ± 7% proc-vmstat.numa_hit 41450577 -25.0% 31099388 ± 7% proc-vmstat.numa_local 3.312e+08 -24.8% 2.491e+08 ± 7% proc-vmstat.pgalloc_normal 3.311e+08 -24.8% 2.489e+08 ± 7% proc-vmstat.pgfree 273665 ± 3% -38.2% 169120 ± 17% interrupts.CAL:Function_call_interrupts 3560 ± 33% -44.7% 1968 ± 35% interrupts.CPU1.CAL:Function_call_interrupts 2900 ± 37% -50.5% 1436 ± 58% interrupts.CPU13.CAL:Function_call_interrupts 488.67 ± 29% -62.2% 184.71 ± 63% interrupts.CPU24.RES:Rescheduling_interrupts 3783 ± 19% -59.9% 1518 ± 40% interrupts.CPU25.CAL:Function_call_interrupts 282.67 ± 29% -56.6% 122.57 ± 54% interrupts.CPU25.RES:Rescheduling_interrupts 3736 ± 38% -66.6% 1246 ± 34% interrupts.CPU27.CAL:Function_call_interrupts 268.50 ± 39% -71.3% 77.00 ± 49% interrupts.CPU27.RES:Rescheduling_interrupts 3493 ± 41% -52.3% 1667 ± 40% interrupts.CPU36.CAL:Function_call_interrupts 3883 ± 54% -55.7% 1719 ± 46% interrupts.CPU5.CAL:Function_call_interrupts 4026 ± 20% -55.6% 1788 ± 37% interrupts.CPU50.CAL:Function_call_interrupts 4144 ± 23% -51.2% 2022 ± 70% interrupts.CPU50.NMI:Non-maskable_interrupts 4144 ± 23% -51.2% 2022 ± 70% interrupts.CPU50.PMI:Performance_monitoring_interrupts 2631 ± 38% -46.5% 1408 ± 35% interrupts.CPU52.CAL:Function_call_interrupts 4379 ± 54% -59.0% 1796 ± 78% interrupts.CPU55.CAL:Function_call_interrupts 2710 ± 35% -58.4% 1127 ± 63% interrupts.CPU56.CAL:Function_call_interrupts 183.67 ± 21% -65.4% 63.57 ± 58% interrupts.CPU56.RES:Rescheduling_interrupts 2969 ± 37% -52.6% 1407 ± 48% interrupts.CPU57.CAL:Function_call_interrupts 3712 ± 18% -73.9% 968.57 ± 96% interrupts.CPU60.NMI:Non-maskable_interrupts 3712 ± 18% -73.9% 968.57 ± 96% interrupts.CPU60.PMI:Performance_monitoring_interrupts 2946 ± 27% -56.2% 1289 ± 52% interrupts.CPU61.CAL:Function_call_interrupts 2221 ± 35% -59.7% 895.29 ± 89% interrupts.CPU61.NMI:Non-maskable_interrupts 2221 ± 35% -59.7% 895.29 ± 89% interrupts.CPU61.PMI:Performance_monitoring_interrupts 186.50 ± 29% -52.0% 89.43 ± 76% interrupts.CPU65.RES:Rescheduling_interrupts 4913 ± 42% -73.8% 1286 ± 51% interrupts.CPU67.CAL:Function_call_interrupts 3640 ± 42% -63.3% 1336 ± 37% interrupts.CPU75.CAL:Function_call_interrupts 4197 ± 41% -74.5% 1071 ± 51% interrupts.CPU76.CAL:Function_call_interrupts 274.00 ± 46% -66.9% 90.57 ± 70% interrupts.CPU76.RES:Rescheduling_interrupts 4458 ± 33% -48.2% 2307 ± 41% interrupts.CPU78.CAL:Function_call_interrupts 174.00 ± 21% -42.9% 99.43 ± 42% interrupts.CPU91.RES:Rescheduling_interrupts 2615 ± 43% -52.8% 1235 ± 41% interrupts.CPU93.CAL:Function_call_interrupts 17598 ± 4% -23.0% 13556 ± 14% softirqs.CPU0.RCU 13220 ± 6% -20.5% 10514 ± 10% softirqs.CPU1.RCU 12111 ± 13% -19.9% 9699 ± 9% softirqs.CPU16.RCU 12269 ± 4% -18.3% 10030 ± 13% softirqs.CPU2.RCU 15935 ± 10% -15.7% 13430 ± 20% softirqs.CPU24.RCU 14418 ± 14% -26.6% 10583 ± 14% softirqs.CPU25.RCU 788851 ± 32% -55.0% 354883 ± 28% softirqs.CPU27.NET_RX 13652 ± 13% -28.7% 9734 ± 5% softirqs.CPU27.RCU 13432 ± 10% -15.7% 11328 ± 4% softirqs.CPU27.SCHED 11413 ± 12% -17.5% 9415 ± 15% softirqs.CPU4.RCU 13077 ± 13% -13.7% 11288 ± 21% softirqs.CPU49.SCHED 14547 ± 12% -22.2% 11314 ± 8% softirqs.CPU50.RCU 13654 ± 3% -11.6% 12076 ± 11% softirqs.CPU52.RCU 830025 ± 36% -63.2% 305684 ± 58% softirqs.CPU56.NET_RX 13872 ± 16% -38.1% 8591 ± 10% softirqs.CPU56.RCU 13666 ± 10% -29.7% 9607 ± 17% softirqs.CPU57.RCU 13660 ± 13% -26.0% 10105 ± 13% softirqs.CPU60.RCU 13861 ± 11% -12.2% 12172 ± 10% softirqs.CPU60.SCHED 12378 ± 9% -25.7% 9199 ± 13% softirqs.CPU61.RCU 13237 ± 12% -17.8% 10879 ± 10% softirqs.CPU63.RCU 14837 ± 20% -26.4% 10923 ± 12% softirqs.CPU65.RCU 13078 ± 10% -20.0% 10462 ± 9% softirqs.CPU68.RCU 1014501 ± 29% -51.1% 496322 ± 35% softirqs.CPU76.NET_RX 14929 ± 14% -33.3% 9964 ± 8% softirqs.CPU76.RCU 14824 ± 9% -20.2% 11834 ± 9% softirqs.CPU76.SCHED 14956 ± 15% -29.7% 10517 ± 16% softirqs.CPU79.RCU 14781 ± 18% -30.9% 10210 ± 20% softirqs.CPU80.RCU 665014 ± 50% -46.2% 357487 ± 55% softirqs.CPU83.NET_RX 12417 ± 14% -25.5% 9246 ± 9% softirqs.CPU83.RCU 12715 ± 10% -19.8% 10191 ± 10% softirqs.CPU86.RCU 54301315 -10.1% 48817727 ± 3% softirqs.NET_RX 1165787 ± 2% -14.0% 1002845 ± 2% softirqs.RCU 21.24 -3.7% 20.44 ± 2% perf-stat.i.MPKI 7.91e+09 -18.5% 6.447e+09 ± 5% perf-stat.i.branch-instructions 1.218e+08 -17.6% 1.003e+08 ± 5% perf-stat.i.branch-misses 0.96 ± 31% +24.3 25.26 ± 28% perf-stat.i.cache-miss-rate% 4298146 ± 3% +3705.0% 1.635e+08 ± 22% perf-stat.i.cache-misses 8.4e+08 -20.9% 6.648e+08 ± 7% perf-stat.i.cache-references 1262713 -9.0% 1149476 ± 4% perf-stat.i.context-switches 1.30 +20.3% 1.57 ± 5% perf-stat.i.cpi 133.25 ± 2% -5.4% 126.01 ± 2% perf-stat.i.cpu-migrations 11560 ± 3% -96.1% 449.24 ± 21% perf-stat.i.cycles-between-cache-misses 1.12e+10 -18.1% 9.179e+09 ± 5% perf-stat.i.dTLB-loads 41181 ± 15% -31.8% 28074 ± 15% perf-stat.i.dTLB-store-misses 6.345e+09 -18.1% 5.195e+09 ± 5% perf-stat.i.dTLB-stores 94303727 -16.4% 78848232 ± 5% perf-stat.i.iTLB-load-misses 3.898e+10 -18.5% 3.178e+10 ± 5% perf-stat.i.instructions 0.78 -17.1% 0.64 ± 5% perf-stat.i.ipc 0.85 ± 8% -23.3% 0.65 ± 4% perf-stat.i.metric.K/sec 274.05 -18.0% 224.70 ± 5% perf-stat.i.metric.M/sec 81.79 +15.2 96.96 perf-stat.i.node-load-miss-rate% 874824 ± 3% +7178.9% 63677834 ± 23% perf-stat.i.node-load-misses 192930 ± 4% +534.0% 1223150 ± 21% perf-stat.i.node-loads 87.79 ± 2% +9.5 97.27 perf-stat.i.node-store-miss-rate% 657426 ± 3% +1728.3% 12019579 ± 22% perf-stat.i.node-store-misses 83513 ± 17% -51.2% 40778 ± 6% perf-stat.i.node-stores 21.55 -3.0% 20.90 perf-stat.overall.MPKI 1.54 +0.0 1.56 perf-stat.overall.branch-miss-rate% 0.51 ± 3% +24.6 25.11 ± 28% perf-stat.overall.cache-miss-rate% 1.28 +22.6% 1.57 ± 5% perf-stat.overall.cpi 11588 ± 3% -97.2% 322.82 ± 26% perf-stat.overall.cycles-between-cache-misses 0.78 -18.2% 0.64 ± 5% perf-stat.overall.ipc 81.93 +16.2 98.10 perf-stat.overall.node-load-miss-rate% 88.73 ± 2% +10.9 99.64 perf-stat.overall.node-store-miss-rate% 7.782e+09 -18.5% 6.343e+09 ± 5% perf-stat.ps.branch-instructions 1.198e+08 -17.6% 98698837 ± 5% perf-stat.ps.branch-misses 4230041 ± 3% +3703.2% 1.609e+08 ± 22% perf-stat.ps.cache-misses 8.265e+08 -20.9% 6.541e+08 ± 7% perf-stat.ps.cache-references 1242259 -9.0% 1130898 ± 4% perf-stat.ps.context-switches 131.16 ± 2% -5.4% 124.02 ± 2% perf-stat.ps.cpu-migrations 1.102e+10 -18.1% 9.031e+09 ± 5% perf-stat.ps.dTLB-loads 40640 ± 15% -31.9% 27660 ± 15% perf-stat.ps.dTLB-store-misses 6.243e+09 -18.1% 5.111e+09 ± 5% perf-stat.ps.dTLB-stores 92776465 -16.4% 77575353 ± 5% perf-stat.ps.iTLB-load-misses 3.835e+10 -18.5% 3.127e+10 ± 5% perf-stat.ps.instructions 860793 ± 3% +7176.9% 62639147 ± 23% perf-stat.ps.node-load-misses 189892 ± 4% +533.7% 1203290 ± 21% perf-stat.ps.node-loads 646849 ± 3% +1727.9% 11823793 ± 22% perf-stat.ps.node-store-misses 82269 ± 17% -51.1% 40208 ± 6% perf-stat.ps.node-stores 2.442e+12 -18.9% 1.98e+12 ± 5% perf-stat.total.instructions 5.42 ± 7% -1.0 4.38 ± 9% perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto 5.29 ± 8% -1.0 4.25 ± 9% perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 2.44 ± 8% -0.8 1.64 ± 14% perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked.tcp_sendmsg 2.33 ± 8% -0.8 1.56 ± 15% perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_sendmsg_locked 0.66 ± 8% -0.3 0.39 ± 63% perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 1.28 ± 7% -0.3 1.01 ± 10% perf-profile.calltrace.cycles-pp.sock_do_ioctl.sock_ioctl.do_vfs_ioctl.__x64_sys_ioctl.do_syscall_64 1.37 ± 8% -0.3 1.11 ± 10% perf-profile.calltrace.cycles-pp.lock_sock_nested.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 0.89 ± 7% -0.2 0.68 ± 11% perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe 0.81 ± 7% -0.2 0.66 ± 7% perf-profile.calltrace.cycles-pp.tcp_rcv_space_adjust.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.__sys_recvfrom 0.61 ± 8% +0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg.__sys_recvfrom 0.45 ± 44% +0.3 0.71 ± 13% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg 0.34 ± 70% +0.3 0.65 ± 12% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg 1.77 ± 7% -0.4 1.40 ± 9% perf-profile.children.cycles-pp.sock_ioctl 1.74 ± 5% -0.3 1.39 ± 9% perf-profile.children.cycles-pp.__dev_queue_xmit 1.15 ± 7% -0.3 0.85 ± 10% perf-profile.children.cycles-pp.syscall_return_via_sysret 0.94 ± 8% -0.2 0.71 ± 10% perf-profile.children.cycles-pp.syscall_exit_to_user_mode 0.93 ± 4% -0.2 0.76 ± 9% perf-profile.children.cycles-pp.dev_hard_start_xmit 0.61 ± 9% -0.2 0.45 ± 20% perf-profile.children.cycles-pp.update_rq_clock 0.71 ± 8% -0.2 0.55 ± 11% perf-profile.children.cycles-pp.exit_to_user_mode_prepare 0.64 ± 5% -0.1 0.50 ± 8% perf-profile.children.cycles-pp.__entry_text_start 0.66 ± 5% -0.1 0.52 ± 14% perf-profile.children.cycles-pp.aa_sk_perm 0.84 ± 5% -0.1 0.71 ± 9% perf-profile.children.cycles-pp.loopback_xmit 0.57 ± 7% -0.1 0.45 ± 10% perf-profile.children.cycles-pp.tcp_mstamp_refresh 0.45 ± 6% -0.1 0.33 ± 13% perf-profile.children.cycles-pp.security_socket_sendmsg 0.45 ± 8% -0.1 0.34 ± 12% perf-profile.children.cycles-pp.sockfd_lookup_light 0.35 ± 12% -0.1 0.24 ± 11% perf-profile.children.cycles-pp.__virt_addr_valid 0.55 ± 8% -0.1 0.45 ± 11% perf-profile.children.cycles-pp.___might_sleep 0.40 ± 8% -0.1 0.30 ± 11% perf-profile.children.cycles-pp.__fput 0.41 ± 8% -0.1 0.31 ± 11% perf-profile.children.cycles-pp.task_work_run 0.41 ± 7% -0.1 0.34 ± 12% perf-profile.children.cycles-pp.__might_fault 0.26 ± 8% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.sock_close 0.24 ± 9% -0.1 0.18 ± 9% perf-profile.children.cycles-pp.inet_release 0.25 ± 8% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.__sock_release 0.20 ± 7% -0.1 0.14 ± 12% perf-profile.children.cycles-pp.__tcp_close 0.22 ± 8% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.tcp_close 0.19 ± 11% -0.1 0.14 ± 18% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.28 ± 7% -0.1 0.22 ± 13% perf-profile.children.cycles-pp.__cond_resched 0.15 ± 9% -0.1 0.09 ± 21% perf-profile.children.cycles-pp.resched_curr 0.18 ± 9% -0.1 0.13 ± 19% perf-profile.children.cycles-pp.check_preempt_curr 0.18 ± 6% -0.0 0.13 ± 8% perf-profile.children.cycles-pp.__put_user_nocheck_4 0.14 ± 10% -0.0 0.10 ± 16% perf-profile.children.cycles-pp.__sk_mem_schedule 0.19 ± 8% -0.0 0.15 ± 10% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.16 ± 9% -0.0 0.12 ± 19% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.13 ± 9% -0.0 0.09 ± 19% perf-profile.children.cycles-pp.__sk_mem_raise_allocated 0.18 ± 8% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.do_tcp_setsockopt 0.21 ± 7% -0.0 0.17 ± 10% perf-profile.children.cycles-pp.tcp_rcv_synsent_state_process 0.11 ± 4% -0.0 0.08 ± 13% perf-profile.children.cycles-pp.syscall_exit_to_user_mode_prepare 0.14 ± 9% -0.0 0.11 ± 9% perf-profile.children.cycles-pp.open_related_ns 0.13 ± 6% -0.0 0.10 ± 16% perf-profile.children.cycles-pp.__sys_accept4 0.12 ± 5% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.tcp_fin 0.07 ± 7% -0.0 0.04 ± 64% perf-profile.children.cycles-pp.__x64_sys_accept4 0.13 ± 9% -0.0 0.10 ± 14% perf-profile.children.cycles-pp.import_single_range 0.09 ± 9% -0.0 0.07 ± 13% perf-profile.children.cycles-pp.__ip_finish_output 0.08 -0.0 0.06 ± 13% perf-profile.children.cycles-pp.__sock_wfree 0.07 ± 6% -0.0 0.05 ± 42% perf-profile.children.cycles-pp.ksys_read 0.07 ± 6% -0.0 0.05 ± 41% perf-profile.children.cycles-pp.vfs_read 0.09 ± 4% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.tcp_update_recv_tstamps 0.09 ± 14% +0.0 0.13 ± 12% perf-profile.children.cycles-pp.perf_tp_event 0.07 ± 11% +0.0 0.11 ± 13% perf-profile.children.cycles-pp.ip_copy_addrs 0.15 ± 9% +0.1 0.20 ± 10% perf-profile.children.cycles-pp.nr_iowait_cpu 0.08 ± 12% +0.1 0.17 ± 13% perf-profile.children.cycles-pp.available_idle_cpu 0.00 +0.2 0.22 ± 38% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.00 +0.2 0.23 ± 44% perf-profile.children.cycles-pp.sched_ttwu_pending 0.00 +0.4 0.37 ± 46% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 1.15 ± 7% -0.3 0.85 ± 10% perf-profile.self.cycles-pp.syscall_return_via_sysret 0.47 ± 9% -0.2 0.32 ± 19% perf-profile.self.cycles-pp.update_rq_clock 0.62 ± 5% -0.2 0.47 ± 12% perf-profile.self.cycles-pp.__dev_queue_xmit 0.64 ± 5% -0.2 0.49 ± 8% perf-profile.self.cycles-pp.__entry_text_start 0.50 ± 8% -0.1 0.39 ± 11% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.33 ± 12% -0.1 0.22 ± 12% perf-profile.self.cycles-pp.__virt_addr_valid 0.45 ± 4% -0.1 0.35 ± 14% perf-profile.self.cycles-pp.aa_sk_perm 0.33 ± 5% -0.1 0.26 ± 14% perf-profile.self.cycles-pp.set_next_entity 0.09 ± 8% -0.1 0.04 ± 63% perf-profile.self.cycles-pp.dev_hard_start_xmit 0.15 ± 9% -0.1 0.09 ± 21% perf-profile.self.cycles-pp.resched_curr 0.27 ± 7% -0.1 0.22 ± 12% perf-profile.self.cycles-pp.__sys_sendto 0.23 ± 7% -0.1 0.18 ± 9% perf-profile.self.cycles-pp.sock_ioctl 0.21 ± 6% -0.1 0.16 ± 10% perf-profile.self.cycles-pp._copy_to_iter 0.18 ± 5% -0.0 0.13 ± 8% perf-profile.self.cycles-pp.__put_user_nocheck_4 0.15 ± 10% -0.0 0.10 ± 11% perf-profile.self.cycles-pp.syscall_exit_to_user_mode 0.19 ± 11% -0.0 0.14 ± 14% perf-profile.self.cycles-pp.inet_ioctl 0.16 ± 9% -0.0 0.12 ± 20% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.19 ± 9% -0.0 0.15 ± 12% perf-profile.self.cycles-pp.__sys_recvfrom 0.14 ± 7% -0.0 0.09 ± 12% perf-profile.self.cycles-pp.lock_sock_nested 0.13 ± 9% -0.0 0.09 ± 16% perf-profile.self.cycles-pp.__sk_mem_raise_allocated 0.19 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.process_backlog 0.07 ± 8% -0.0 0.04 ± 63% perf-profile.self.cycles-pp.tcp_mstamp_refresh 0.12 ± 7% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.sockfd_lookup_light 0.10 ± 4% -0.0 0.07 ± 12% perf-profile.self.cycles-pp.syscall_exit_to_user_mode_prepare 0.08 -0.0 0.05 ± 6% perf-profile.self.cycles-pp.__sock_wfree 0.14 ± 6% -0.0 0.11 ± 10% perf-profile.self.cycles-pp.__cond_resched 0.08 ± 10% -0.0 0.06 ± 10% perf-profile.self.cycles-pp.sock_do_ioctl 0.08 ± 16% +0.0 0.12 ± 13% perf-profile.self.cycles-pp.perf_tp_event 0.07 ± 11% +0.0 0.11 ± 15% perf-profile.self.cycles-pp.ip_copy_addrs 0.15 ± 9% +0.1 0.20 ± 10% perf-profile.self.cycles-pp.nr_iowait_cpu 0.03 ± 99% +0.1 0.09 ± 26% perf-profile.self.cycles-pp.__wake_up_common 0.00 +0.1 0.08 ± 28% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.08 ± 12% +0.1 0.17 ± 14% perf-profile.self.cycles-pp.available_idle_cpu 0.13 ± 11% +0.1 0.25 ± 18% perf-profile.self.cycles-pp.tcp_sendmsg stress-ng.sock.ops_per_sec 11500 +-------------------------------------------------------------------+ |++.+++ +.+++++. + +.+++++.+++++.++++.++++ +++.++++.+++++.++| 11000 |-+ +.++++ + + +.++ | 10500 |-+ | | | 10000 |-+ | 9500 |-+ O | | O O O O O O O | 9000 |-+ | 8500 |O+ O | | O O O O O O | 8000 |-+ O O O OO OO O OO O | 7500 |-+ O O | | O | 7000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-csl-2sp8: 96 threads 2 sockets Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 192G memory ========================================================================================= cluster/compiler/concurrency/cpufreq_governor/kconfig/rootfs/runtime/tbox_group/testcase/ucode: cs-localhost/gcc-9/8000/performance/x86_64-rhel-8.3/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2sp8/apachebench/0x5003006 commit: 2ea46c6fc9 ("cpumask/hotplug: Fix cpu_dying() state tracking") 5f94d1b650 ("sched/fair: don't use waker's cpu if the waker of sync wake-up is interrupt") 2ea46c6fc9452ac1 5f94d1b650d14e01f383e3d02e4 ---------------- --------------------------- %stddev %change %stddev \ | \ 35891 -5.6% 33871 apachebench.requests_per_second 28.96 +5.8% 30.64 apachebench.time.elapsed_time 28.96 +5.8% 30.64 apachebench.time.elapsed_time.max 23.00 +8.7% 25.00 ± 2% apachebench.time.percent_of_cpu_this_job_got 222.89 +6.0% 236.22 apachebench.time_per_request 385478 -5.6% 363787 apachebench.transfer_rate 1.67 ± 22% -0.5 1.17 mpstat.cpu.all.irq% 1.02 +0.1 1.12 mpstat.cpu.all.sys% 0.07 ± 50% -59.4% 0.03 ± 43% perf-sched.wait_and_delay.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.06 ± 55% -50.8% 0.03 ± 42% perf-sched.wait_time.avg.ms.pipe_read.new_sync_read.vfs_read.ksys_read 3037 ± 76% +195.7% 8981 ± 51% softirqs.CPU15.NET_RX 822.17 ±127% +1070.4% 9622 ±144% softirqs.CPU37.NET_RX 2388 ±102% +443.8% 12988 ± 58% softirqs.CPU67.NET_RX 11797 ± 4% +15.8% 13658 ± 4% softirqs.TIMER 12572413 ± 6% -16.9% 10443087 ± 5% cpuidle.C1.time 511677 ± 8% -12.7% 446730 ± 7% cpuidle.C1.usage 1.375e+09 ± 74% +110.9% 2.899e+09 cpuidle.C1E.time 4565230 ± 26% +46.7% 6695295 cpuidle.C1E.usage 97938 ± 27% -58.4% 40694 ± 39% cpuidle.POLL.usage 606.17 ±153% -98.5% 9.29 ±148% interrupts.CPU12.TLB:TLB_shootdowns 571.00 ±143% -96.7% 18.57 ±110% interrupts.CPU29.TLB:TLB_shootdowns 530.00 ±146% -91.1% 47.43 ±184% interrupts.CPU30.TLB:TLB_shootdowns 1657 ±190% -99.6% 6.86 ± 74% interrupts.CPU5.TLB:TLB_shootdowns 57.50 ± 67% -62.0% 21.86 ± 32% interrupts.CPU95.TLB:TLB_shootdowns 3736 ± 7% -32.5% 2521 ± 10% interrupts.RES:Rescheduling_interrupts 1792 ± 6% +11.4% 1996 turbostat.Bzy_MHz 501037 ± 8% -12.7% 437439 ± 7% turbostat.C1 0.41 ± 6% -0.1 0.32 ± 5% turbostat.C1% 4561770 ± 26% +46.7% 6690622 turbostat.C1E 1857351 ± 68% -98.2% 32984 ±139% turbostat.C6 105.14 +20.6% 126.78 turbostat.PkgWatt 1.397e+09 -4.4% 1.335e+09 perf-stat.i.branch-instructions 40257529 ± 15% -20.1% 32145859 ± 2% perf-stat.i.branch-misses 9.06 ± 57% +9.2 18.23 ± 5% perf-stat.i.cache-miss-rate% 3159646 ± 13% +223.7% 10228041 ± 11% perf-stat.i.cache-misses 4.25 ± 24% -31.2% 2.93 perf-stat.i.cpi 925.45 ± 40% -80.1% 184.12 ± 3% perf-stat.i.cpu-migrations 4096 ± 10% -68.6% 1284 ± 9% perf-stat.i.cycles-between-cache-misses 0.11 ± 68% -0.1 0.01 ± 3% perf-stat.i.dTLB-store-miss-rate% 9.43e+08 -3.9% 9.059e+08 perf-stat.i.dTLB-stores 6.644e+09 -4.3% 6.357e+09 perf-stat.i.instructions 12.04 ± 5% +22.6% 14.76 ± 9% perf-stat.i.major-faults 44.87 ± 2% -5.1% 42.58 perf-stat.i.metric.M/sec 42764 -4.5% 40848 perf-stat.i.minor-faults 70.15 ± 4% +15.5 85.61 perf-stat.i.node-load-miss-rate% 330592 ± 22% +926.0% 3391986 ± 14% perf-stat.i.node-load-misses 157887 ± 4% +36.3% 215209 ± 3% perf-stat.i.node-loads 56.69 ± 10% +24.5 81.14 perf-stat.i.node-store-miss-rate% 110581 ± 16% +786.3% 980040 ± 15% perf-stat.i.node-store-misses 120278 ± 7% -19.9% 96300 ± 5% perf-stat.i.node-stores 42776 -4.5% 40862 perf-stat.i.page-faults 3.14 ± 16% +8.9 12.04 ± 11% perf-stat.overall.cache-miss-rate% 3656 ± 8% -71.4% 1045 ± 14% perf-stat.overall.cycles-between-cache-misses 67.09 ± 6% +26.8 93.93 perf-stat.overall.node-load-miss-rate% 47.64 ± 10% +43.2 90.83 perf-stat.overall.node-store-miss-rate% 1.355e+09 -4.3% 1.296e+09 perf-stat.ps.branch-instructions 39024819 ± 15% -20.0% 31208731 ± 2% perf-stat.ps.branch-misses 3059351 ± 13% +224.2% 9917945 ± 11% perf-stat.ps.cache-misses 893.60 ± 40% -80.0% 178.29 ± 3% perf-stat.ps.cpu-migrations 9.144e+08 -3.9% 8.791e+08 perf-stat.ps.dTLB-stores 6.446e+09 -4.3% 6.17e+09 perf-stat.ps.instructions 11.65 ± 5% +22.9% 14.31 ± 9% perf-stat.ps.major-faults 41450 -4.4% 39628 perf-stat.ps.minor-faults 320695 ± 22% +925.8% 3289591 ± 14% perf-stat.ps.node-load-misses 152926 ± 4% +36.5% 208688 ± 4% perf-stat.ps.node-loads 107301 ± 16% +786.0% 950672 ± 15% perf-stat.ps.node-store-misses 116716 ± 7% -19.9% 93516 ± 5% perf-stat.ps.node-stores 41461 -4.4% 39642 perf-stat.ps.page-faults 2.80 ± 9% -0.6 2.19 ± 14% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 2.69 ± 10% -0.6 2.12 ± 14% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.68 ± 9% -0.4 0.33 ± 87% perf-profile.calltrace.cycles-pp.ret_from_fork 0.68 ± 9% -0.4 0.33 ± 87% perf-profile.calltrace.cycles-pp.kthread.ret_from_fork 0.83 ± 9% -0.1 0.69 ± 6% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 0.81 ± 9% -0.1 0.67 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 0.49 ± 45% +0.2 0.68 ± 7% perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet6_recvmsg.sock_read_iter.new_sync_read 0.73 ± 10% +0.3 1.05 ± 6% perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg 0.73 ± 10% +0.3 1.06 ± 6% perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_read_iter 1.69 ± 7% +0.3 2.02 ± 6% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg 2.45 ± 6% +0.4 2.80 ± 4% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 1.98 ± 6% +0.4 2.34 ± 6% perf-profile.calltrace.cycles-pp.copyin._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg 2.46 ± 6% +0.4 2.82 ± 4% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 2.00 ± 6% +0.4 2.37 ± 6% perf-profile.calltrace.cycles-pp._copy_from_iter_full.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.sock_write_iter 0.52 ± 45% +0.4 0.88 ± 7% perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg 0.49 ± 45% +0.4 0.86 ± 7% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked 0.48 ± 45% +0.4 0.85 ± 8% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter 1.51 ± 9% +0.4 1.90 ± 7% perf-profile.calltrace.cycles-pp.read 1.43 ± 9% +0.4 1.82 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 1.40 ± 10% +0.4 1.79 ± 7% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 1.41 ± 9% +0.4 1.80 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 1.00 ± 10% +0.4 1.40 ± 6% perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_read_iter.new_sync_read 1.34 ± 10% +0.4 1.74 ± 8% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read 1.11 ± 10% +0.4 1.51 ± 7% perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.sock_read_iter.new_sync_read.vfs_read 1.14 ± 10% +0.4 1.54 ± 7% perf-profile.calltrace.cycles-pp.inet_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read 3.02 ± 7% +0.4 3.46 ± 4% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog 2.95 ± 7% +0.4 3.40 ± 4% perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv 3.00 ± 7% +0.4 3.44 ± 4% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core 0.20 ±141% +0.4 0.65 ± 12% perf-profile.calltrace.cycles-pp.ktime_get.clockevents_program_event.hrtimer_interrupt.__sysvec_apic_timer_interrupt.sysvec_apic_timer_interrupt 3.34 ± 6% +0.4 3.79 ± 4% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq 3.35 ± 6% +0.4 3.80 ± 4% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip 3.03 ± 7% +0.4 3.48 ± 4% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll 3.70 ± 6% +0.5 4.15 ± 4% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb 3.66 ± 6% +0.5 4.12 ± 4% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit 3.13 ± 6% +0.5 3.59 ± 4% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action 3.23 ± 6% +0.5 3.69 ± 4% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start 2.20 ± 7% +0.5 2.70 ± 6% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.13 ± 7% +0.5 2.63 ± 5% perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.79 ± 6% -0.2 1.55 ± 5% perf-profile.children.cycles-pp.__schedule 0.76 ± 10% -0.2 0.53 ± 15% perf-profile.children.cycles-pp.console_unlock 0.76 ± 10% -0.2 0.54 ± 14% perf-profile.children.cycles-pp.vprintk_emit 0.64 ± 9% -0.2 0.44 ± 14% perf-profile.children.cycles-pp.serial8250_console_write 0.61 ± 9% -0.2 0.41 ± 15% perf-profile.children.cycles-pp.uart_console_write 0.59 ± 9% -0.2 0.40 ± 14% perf-profile.children.cycles-pp.wait_for_xmitr 0.57 ± 10% -0.2 0.38 ± 15% perf-profile.children.cycles-pp.serial8250_console_putchar 0.50 ± 11% -0.2 0.34 ± 13% perf-profile.children.cycles-pp.io_serial_in 0.53 ± 16% -0.2 0.36 ± 23% perf-profile.children.cycles-pp.devkmsg_write.cold 0.53 ± 16% -0.2 0.36 ± 23% perf-profile.children.cycles-pp.devkmsg_emit 0.68 ± 9% -0.2 0.52 ± 14% perf-profile.children.cycles-pp.ret_from_fork 0.68 ± 9% -0.2 0.52 ± 14% perf-profile.children.cycles-pp.kthread 0.61 ± 11% -0.2 0.46 ± 17% perf-profile.children.cycles-pp.worker_thread 0.61 ± 11% -0.2 0.46 ± 18% perf-profile.children.cycles-pp.process_one_work 0.60 ± 11% -0.2 0.45 ± 19% perf-profile.children.cycles-pp.drm_fb_helper_damage_work 0.60 ± 11% -0.1 0.45 ± 19% perf-profile.children.cycles-pp.memcpy_toio 0.84 ± 8% -0.1 0.70 ± 6% perf-profile.children.cycles-pp.schedule_idle 1.25 ± 6% -0.1 1.12 ± 5% perf-profile.children.cycles-pp.schedule_hrtimeout_range_clock 0.18 ± 31% -0.1 0.09 ± 9% perf-profile.children.cycles-pp.rcu_idle_exit 0.59 ± 5% -0.1 0.52 ± 5% perf-profile.children.cycles-pp.unmap_page_range 0.35 ± 5% -0.1 0.28 ± 7% perf-profile.children.cycles-pp._raw_spin_lock 0.41 ± 5% -0.1 0.34 ± 7% perf-profile.children.cycles-pp.dequeue_task_fair 0.21 ± 11% -0.1 0.15 ± 13% perf-profile.children.cycles-pp.set_next_entity 0.65 ± 2% -0.1 0.58 ± 5% perf-profile.children.cycles-pp.unmap_vmas 0.13 ± 29% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.rcu_eqs_exit 0.30 ± 7% -0.1 0.25 ± 6% perf-profile.children.cycles-pp.asm_sysvec_call_function 0.21 ± 14% -0.1 0.16 ± 8% perf-profile.children.cycles-pp.__libc_fork 0.20 ± 16% -0.1 0.15 ± 10% perf-profile.children.cycles-pp.__do_sys_clone 0.20 ± 16% -0.1 0.15 ± 10% perf-profile.children.cycles-pp.kernel_clone 0.18 ± 17% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.dup_mmap 0.35 ± 9% -0.1 0.29 ± 8% perf-profile.children.cycles-pp.pick_next_task_fair 0.18 ± 16% -0.1 0.13 ± 14% perf-profile.children.cycles-pp.dup_mm 0.19 ± 15% -0.1 0.14 ± 9% perf-profile.children.cycles-pp.copy_process 0.12 ± 21% -0.0 0.07 ± 12% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.34 ± 7% -0.0 0.29 ± 7% perf-profile.children.cycles-pp.dequeue_entity 0.11 ± 20% -0.0 0.07 ± 13% perf-profile.children.cycles-pp.hrtimer_next_event_without 0.16 ± 5% -0.0 0.12 ± 8% perf-profile.children.cycles-pp.__sysvec_call_function 0.23 ± 4% -0.0 0.19 ± 5% perf-profile.children.cycles-pp.sysvec_call_function 0.10 ± 12% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.sk_filter_trim_cap 0.08 ± 27% -0.0 0.06 ± 13% perf-profile.children.cycles-pp._find_next_bit 0.07 ± 19% +0.0 0.10 ± 13% perf-profile.children.cycles-pp.skb_entail 0.16 ± 5% +0.0 0.19 ± 12% perf-profile.children.cycles-pp.flush_smp_call_function_queue 0.19 ± 9% +0.0 0.23 ± 7% perf-profile.children.cycles-pp.common_file_perm 0.07 ± 18% +0.0 0.11 ± 11% perf-profile.children.cycles-pp.sk_page_frag_refill 0.07 ± 14% +0.0 0.11 ± 11% perf-profile.children.cycles-pp.skb_page_frag_refill 0.10 ± 14% +0.1 0.15 ± 16% perf-profile.children.cycles-pp.__skb_clone 0.21 ± 13% +0.1 0.26 ± 7% perf-profile.children.cycles-pp.__inet_lookup_established 0.02 ±142% +0.1 0.07 ± 26% perf-profile.children.cycles-pp.lockref_get_not_dead 0.02 ±141% +0.1 0.07 ± 9% perf-profile.children.cycles-pp.tcp_ack_update_rtt 0.03 ±102% +0.1 0.09 ± 20% perf-profile.children.cycles-pp.cpuacct_charge 0.12 ± 18% +0.1 0.18 ± 5% perf-profile.children.cycles-pp.kfree 0.13 ± 12% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.__slab_free 0.00 +0.1 0.07 ± 20% perf-profile.children.cycles-pp.llist_reverse_order 0.37 ± 8% +0.1 0.45 ± 10% perf-profile.children.cycles-pp.kmem_cache_free 0.01 ±223% +0.1 0.09 ± 16% perf-profile.children.cycles-pp.llist_add_batch 0.40 ± 8% +0.1 0.48 ± 9% perf-profile.children.cycles-pp.sk_stream_alloc_skb 0.15 ± 8% +0.1 0.23 ± 14% perf-profile.children.cycles-pp.rcu_do_batch 0.38 ± 8% +0.1 0.47 ± 8% perf-profile.children.cycles-pp.__alloc_skb 0.20 ± 6% +0.1 0.30 ± 12% perf-profile.children.cycles-pp.rcu_core 0.00 +0.1 0.12 ± 20% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.55 ± 5% +0.1 0.68 ± 9% perf-profile.children.cycles-pp.__kfree_skb 0.69 ± 4% +0.1 0.83 ± 6% perf-profile.children.cycles-pp.tcp_clean_rtx_queue 0.98 ± 5% +0.2 1.13 ± 5% perf-profile.children.cycles-pp.tcp_ack 0.77 ± 15% +0.2 0.97 ± 9% perf-profile.children.cycles-pp.clockevents_program_event 0.00 +0.2 0.23 ± 18% perf-profile.children.cycles-pp.sched_ttwu_pending 1.30 ± 14% +0.3 1.59 ± 6% perf-profile.children.cycles-pp.ktime_get 2.39 ± 7% +0.3 2.69 ± 6% perf-profile.children.cycles-pp.copyin 0.69 ± 10% +0.3 1.00 ± 6% perf-profile.children.cycles-pp._copy_to_iter 0.65 ± 9% +0.3 0.96 ± 6% perf-profile.children.cycles-pp.copyout 0.00 +0.3 0.31 ± 20% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 2.40 ± 7% +0.3 2.71 ± 5% perf-profile.children.cycles-pp._copy_from_iter_full 0.98 ± 9% +0.4 1.34 ± 6% perf-profile.children.cycles-pp.__skb_datagram_iter 0.98 ± 9% +0.4 1.35 ± 6% perf-profile.children.cycles-pp.skb_copy_datagram_iter 1.54 ± 10% +0.4 1.92 ± 8% perf-profile.children.cycles-pp.read 1.14 ± 10% +0.4 1.54 ± 7% perf-profile.children.cycles-pp.inet_recvmsg 3.20 ± 7% +0.4 3.61 ± 3% perf-profile.children.cycles-pp.tcp_v4_rcv 3.25 ± 7% +0.4 3.66 ± 3% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu 3.26 ± 7% +0.4 3.68 ± 3% perf-profile.children.cycles-pp.ip_local_deliver_finish 3.28 ± 7% +0.4 3.70 ± 3% perf-profile.children.cycles-pp.ip_local_deliver 3.51 ± 7% +0.4 3.93 ± 4% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 3.39 ± 7% +0.4 3.82 ± 4% perf-profile.children.cycles-pp.ip_rcv 2.35 ± 7% +0.5 2.83 ± 6% perf-profile.children.cycles-pp.new_sync_read 2.69 ± 6% +0.5 3.17 ± 5% perf-profile.children.cycles-pp.vfs_read 2.79 ± 6% +0.5 3.27 ± 5% perf-profile.children.cycles-pp.ksys_read 2.26 ± 7% +0.5 2.75 ± 6% perf-profile.children.cycles-pp.sock_read_iter 1.98 ± 7% +0.5 2.47 ± 5% perf-profile.children.cycles-pp.tcp_recvmsg 1.67 ± 7% +0.5 2.17 ± 6% perf-profile.children.cycles-pp.tcp_recvmsg_locked 3.10 ± 7% +0.6 3.72 ± 6% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 9.78 ± 7% +1.0 10.78 ± 4% perf-profile.children.cycles-pp.sock_write_iter 9.71 ± 7% +1.0 10.72 ± 4% perf-profile.children.cycles-pp.sock_sendmsg 9.56 ± 7% +1.0 10.58 ± 4% perf-profile.children.cycles-pp.tcp_sendmsg 9.41 ± 7% +1.0 10.43 ± 4% perf-profile.children.cycles-pp.tcp_sendmsg_locked 0.50 ± 11% -0.2 0.34 ± 13% perf-profile.self.cycles-pp.io_serial_in 0.59 ± 11% -0.1 0.45 ± 19% perf-profile.self.cycles-pp.memcpy_toio 0.16 ± 15% -0.1 0.11 ± 16% perf-profile.self.cycles-pp.update_rq_clock 0.11 ± 14% -0.0 0.07 ± 15% perf-profile.self.cycles-pp.set_next_entity 0.32 ± 8% -0.0 0.28 ± 10% perf-profile.self.cycles-pp.__schedule 0.14 ± 11% -0.0 0.09 ± 12% perf-profile.self.cycles-pp.update_load_avg 0.29 ± 14% -0.0 0.25 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.17 ± 4% -0.0 0.15 ± 5% perf-profile.self.cycles-pp.apr_brigade_cleanup 0.10 ± 11% -0.0 0.08 ± 17% perf-profile.self.cycles-pp.arch_scale_freq_tick 0.17 ± 8% -0.0 0.15 ± 7% perf-profile.self.cycles-pp.ap_process_request_internal 0.07 ± 9% -0.0 0.05 ± 9% perf-profile.self.cycles-pp._find_next_bit 0.16 ± 6% +0.0 0.19 ± 8% perf-profile.self.cycles-pp.common_file_perm 0.07 ± 19% +0.0 0.10 ± 10% perf-profile.self.cycles-pp.sock_def_readable 0.04 ± 71% +0.0 0.07 ± 10% perf-profile.self.cycles-pp.tcp_rcv_space_adjust 0.07 ± 11% +0.0 0.11 ± 13% perf-profile.self.cycles-pp.poll_schedule_timeout 0.16 ± 8% +0.0 0.20 ± 8% perf-profile.self.cycles-pp.__check_object_size 0.05 ± 46% +0.0 0.09 ± 13% perf-profile.self.cycles-pp.skb_entail 0.07 ± 14% +0.0 0.11 ± 13% perf-profile.self.cycles-pp.skb_page_frag_refill 0.02 ±141% +0.0 0.06 ± 11% perf-profile.self.cycles-pp.__alloc_skb 0.07 ± 46% +0.1 0.12 ± 13% perf-profile.self.cycles-pp.__skb_clone 0.11 ± 14% +0.1 0.17 ± 14% perf-profile.self.cycles-pp.__ksize 0.12 ± 18% +0.1 0.17 ± 8% perf-profile.self.cycles-pp.kfree 0.02 ±142% +0.1 0.07 ± 26% perf-profile.self.cycles-pp.lockref_get_not_dead 0.18 ± 17% +0.1 0.24 ± 8% perf-profile.self.cycles-pp.__inet_lookup_established 0.00 +0.1 0.06 ± 14% perf-profile.self.cycles-pp.try_to_wake_up 0.03 ±102% +0.1 0.09 ± 20% perf-profile.self.cycles-pp.cpuacct_charge 0.01 ±223% +0.1 0.07 ± 15% perf-profile.self.cycles-pp.tcp_ack_update_rtt 0.20 ± 7% +0.1 0.26 ± 12% perf-profile.self.cycles-pp.kmem_cache_free 0.13 ± 12% +0.1 0.20 ± 9% perf-profile.self.cycles-pp.__slab_free 0.00 +0.1 0.07 ± 20% perf-profile.self.cycles-pp.llist_reverse_order 0.12 ± 11% +0.1 0.20 ± 14% perf-profile.self.cycles-pp.tcp_rcv_established 0.01 ±223% +0.1 0.09 ± 16% perf-profile.self.cycles-pp.llist_add_batch 0.26 ± 14% +0.1 0.34 ± 3% perf-profile.self.cycles-pp.tcp_recvmsg_locked 0.30 ± 10% +0.1 0.41 ± 8% perf-profile.self.cycles-pp.tcp_sendmsg_locked 0.98 ± 18% +0.3 1.31 ± 8% perf-profile.self.cycles-pp.ktime_get 1.08 ± 18% +0.4 1.48 ± 9% perf-profile.self.cycles-pp.cpuidle_enter_state 1.23 ± 22% +0.5 1.77 ± 12% perf-profile.self.cycles-pp.menu_select 1.86 ± 7% +0.6 2.46 ± 7% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string *************************************************************************************************** lkp-csl-2ap3: 192 threads 4 sockets Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory ========================================================================================= cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase/ucode: cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/1/debian-10.4-x86_64-20200603.cgz/300s/lkp-csl-2ap3/UDP_STREAM/netperf/0x5003006 commit: 2ea46c6fc9 ("cpumask/hotplug: Fix cpu_dying() state tracking") 5f94d1b650 ("sched/fair: don't use waker's cpu if the waker of sync wake-up is interrupt") 2ea46c6fc9452ac1 5f94d1b650d14e01f383e3d02e4 ---------------- --------------------------- %stddev %change %stddev \ | \ 62544 -52.4% 29784 ± 15% netperf.Throughput_Mbps 62544 -52.4% 29784 ± 15% netperf.Throughput_total_Mbps 998.17 ± 5% -16.2% 836.00 ± 10% netperf.time.minor_page_faults 81.17 +9.1% 88.57 netperf.time.percent_of_cpu_this_job_got 242.61 +9.5% 265.75 netperf.time.system_time 35803969 -52.4% 17050380 ± 15% netperf.workload 0.19 ± 6% -0.0 0.14 ± 5% mpstat.cpu.all.soft% 0.04 -0.0 0.04 ± 2% mpstat.cpu.all.usr% 1568380 ±162% -88.6% 178452 ± 11% numa-numastat.node3.local_node 1640558 ±153% -84.3% 257114 numa-numastat.node3.numa_hit 1572364 ± 2% -14.6% 1342693 vmstat.memory.cache 237206 -51.8% 114337 ± 15% vmstat.system.cs 8917350 ± 19% +840.2% 83837786 ± 13% cpuidle.C1.time 1687718 ± 18% +734.3% 14080429 ± 11% cpuidle.C1.usage 72623238 -89.0% 8015009 ±119% cpuidle.POLL.time 33405975 -90.8% 3070283 ±139% cpuidle.POLL.usage 427986 ± 7% -49.9% 214579 ± 2% numa-meminfo.node3.Active 427986 ± 7% -49.9% 214579 ± 2% numa-meminfo.node3.Active(anon) 702896 ± 5% -32.6% 473640 ± 3% numa-meminfo.node3.FilePages 463902 ± 7% -49.1% 235917 ± 2% numa-meminfo.node3.Shmem 106952 ± 7% -49.8% 53679 ± 2% numa-vmstat.node3.nr_active_anon 175755 ± 5% -32.6% 118442 ± 3% numa-vmstat.node3.nr_file_pages 116007 ± 7% -49.1% 59011 ± 2% numa-vmstat.node3.nr_shmem 106952 ± 7% -49.8% 53679 ± 2% numa-vmstat.node3.nr_zone_active_anon 433337 ± 7% -49.0% 220837 ± 2% meminfo.Active 433337 ± 7% -49.0% 220837 ± 2% meminfo.Active(anon) 1452704 ± 2% -15.7% 1223905 meminfo.Cached 841896 ± 3% -26.8% 616582 meminfo.Committed_AS 45250 -9.6% 40886 ± 2% meminfo.Mapped 483539 ± 7% -47.3% 254634 ± 2% meminfo.Shmem 8213 ± 16% -27.5% 5953 ± 10% softirqs.CPU12.RCU 6993 ± 23% -30.1% 4890 ± 7% softirqs.CPU124.RCU 8490 ± 15% -33.8% 5623 ± 15% softirqs.CPU190.RCU 8545 ± 14% -29.9% 5987 ± 12% softirqs.CPU191.RCU 8207 ± 14% -31.6% 5614 ± 11% softirqs.CPU45.RCU 8880 ± 16% -32.9% 5962 ± 12% softirqs.CPU51.RCU 8895 ± 11% -34.1% 5859 ± 14% softirqs.CPU83.RCU 8708 ± 20% -35.2% 5639 ± 12% softirqs.CPU95.RCU 17915040 -52.4% 8533057 ± 15% softirqs.NET_RX 1447713 ± 7% -26.6% 1062814 ± 5% softirqs.RCU 107830 ± 7% -48.8% 55213 ± 2% proc-vmstat.nr_active_anon 362869 ± 2% -15.7% 305817 proc-vmstat.nr_file_pages 79042 -5.3% 74879 proc-vmstat.nr_inactive_anon 11592 -11.7% 10240 ± 2% proc-vmstat.nr_mapped 120578 ± 7% -47.3% 63500 ± 2% proc-vmstat.nr_shmem 107830 ± 7% -48.8% 55213 ± 2% proc-vmstat.nr_zone_active_anon 79042 -5.3% 74879 proc-vmstat.nr_zone_inactive_anon 37297024 -50.6% 18413448 ± 14% proc-vmstat.numa_hit 37037152 -51.0% 18153607 ± 14% proc-vmstat.numa_local 291363 ± 7% -50.3% 144772 ± 2% proc-vmstat.pgactivate 1.147e+09 -52.3% 5.468e+08 ± 15% proc-vmstat.pgalloc_normal 1.147e+09 -52.3% 5.468e+08 ± 15% proc-vmstat.pgfree 0.02 ± 27% -71.7% 0.01 ± 6% perf-sched.sch_delay.avg.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.02 ± 21% -72.3% 0.00 ± 14% perf-sched.sch_delay.avg.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 0.01 ± 24% -54.2% 0.00 ± 13% perf-sched.sch_delay.avg.ms.do_task_dead.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 0.02 ± 26% -53.5% 0.01 ± 37% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.ep_poll.do_epoll_wait.__x64_sys_epoll_wait 0.02 ± 47% -63.7% 0.01 ± 23% perf-sched.sch_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.00 ±223% +2300.0% 0.00 perf-sched.sch_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 0.01 ± 11% -31.2% 0.01 ± 11% perf-sched.sch_delay.avg.ms.worker_thread.kthread.ret_from_fork 0.03 ± 44% -70.5% 0.01 ± 13% perf-sched.sch_delay.max.ms.__x64_sys_pause.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown] 0.03 ± 27% -67.1% 0.01 ± 53% perf-sched.sch_delay.max.ms.do_nanosleep.hrtimer_nanosleep.__x64_sys_nanosleep.do_syscall_64 0.04 ± 89% -75.9% 0.01 ± 10% perf-sched.sch_delay.max.ms.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe.[unknown] 0.10 ± 57% -59.0% 0.04 ±104% perf-sched.sch_delay.max.ms.pipe_read.new_sync_read.vfs_read.ksys_read 0.04 ± 62% -74.1% 0.01 ± 35% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_select 0.03 ± 52% -55.0% 0.02 ± 33% perf-sched.sch_delay.max.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.04 ± 65% +9892.1% 4.00 ± 93% perf-sched.sch_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 0.00 ±223% +2300.0% 0.00 perf-sched.total_sch_delay.average.ms 2.53 ± 8% +97.1% 5.00 ± 2% perf-sched.total_wait_and_delay.average.ms 950222 ± 6% -50.3% 472509 perf-sched.total_wait_and_delay.count.ms 2.53 ± 8% +97.0% 4.99 ± 2% perf-sched.total_wait_time.average.ms 4.28 -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr 32.30 ± 8% +96.7% 63.54 ± 3% perf-sched.wait_and_delay.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.00 ± 17% +394.5% 0.01 ± 4% perf-sched.wait_and_delay.avg.ms.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 2112 -100.0% 0.00 perf-sched.wait_and_delay.count.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr 556.83 ± 6% -47.8% 290.43 ± 2% perf-sched.wait_and_delay.count.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 938377 ± 6% -50.9% 460959 perf-sched.wait_and_delay.count.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 1000 -100.0% 0.00 perf-sched.wait_and_delay.max.ms.preempt_schedule_common.__cond_resched.wait_for_completion.affine_move_task.__set_cpus_allowed_ptr 0.07 ± 16% +5743.6% 4.10 ± 90% perf-sched.wait_and_delay.max.ms.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 129.86 ± 73% -63.7% 47.15 ±135% perf-sched.wait_and_delay.max.ms.schedule_timeout.rcu_gp_kthread.kthread.ret_from_fork 0.75 ± 2% -9.7% 0.68 perf-sched.wait_time.avg.ms.do_wait.kernel_wait4.__do_sys_wait4.do_syscall_64 32.30 ± 8% +96.7% 63.54 ± 3% perf-sched.wait_time.avg.ms.schedule_hrtimeout_range_clock.poll_schedule_timeout.constprop.0.do_sys_poll 0.00 ± 17% +209.9% 0.01 ± 6% perf-sched.wait_time.avg.ms.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 8.343e+08 -32.4% 5.641e+08 ± 7% perf-stat.i.branch-instructions 1.84 ± 24% +58.5 60.31 ± 27% perf-stat.i.cache-miss-rate% 4879389 ± 5% +1618.4% 83848142 ± 13% perf-stat.i.cache-misses 2.849e+08 ± 15% -44.8% 1.573e+08 ± 25% perf-stat.i.cache-references 238858 -51.8% 115114 ± 15% perf-stat.i.context-switches 3.79 ± 3% +67.9% 6.37 ± 5% perf-stat.i.cpi 1.499e+10 ± 3% +10.2% 1.652e+10 ± 2% perf-stat.i.cpu-cycles 3190 ± 8% -86.5% 430.75 ± 96% perf-stat.i.cycles-between-cache-misses 1.106e+09 -31.9% 7.533e+08 ± 7% perf-stat.i.dTLB-loads 6.313e+08 -33.1% 4.222e+08 ± 8% perf-stat.i.dTLB-stores 67.75 ± 6% -13.5 54.27 ± 7% perf-stat.i.iTLB-load-miss-rate% 6844433 -34.7% 4466144 ± 7% perf-stat.i.iTLB-load-misses 4.07e+09 -32.3% 2.757e+09 ± 7% perf-stat.i.instructions 0.27 ± 3% -38.6% 0.17 ± 8% perf-stat.i.ipc 0.08 ± 3% +10.2% 0.09 ± 2% perf-stat.i.metric.GHz 14.90 -31.8% 10.16 ± 7% perf-stat.i.metric.M/sec 88.72 ± 3% +9.7 98.42 ± 2% perf-stat.i.node-load-miss-rate% 390842 ± 54% +7043.8% 27921170 ± 13% perf-stat.i.node-load-misses 53149 ± 20% +140.2% 127659 ± 20% perf-stat.i.node-loads 84.06 ± 10% +13.9 97.91 ± 3% perf-stat.i.node-store-miss-rate% 108685 ± 5% +20162.1% 22022000 ± 13% perf-stat.i.node-store-misses 1.78 ± 23% +55.8 57.59 ± 30% perf-stat.overall.cache-miss-rate% 3.68 ± 3% +63.4% 6.02 ± 6% perf-stat.overall.cpi 3086 ± 8% -93.5% 201.38 ± 16% perf-stat.overall.cycles-between-cache-misses 67.82 ± 6% -13.3 54.47 ± 8% perf-stat.overall.iTLB-load-miss-rate% 0.27 ± 3% -38.6% 0.17 ± 7% perf-stat.overall.ipc 86.32 ± 4% +13.2 99.55 perf-stat.overall.node-load-miss-rate% 79.67 ± 11% +20.2 99.91 perf-stat.overall.node-store-miss-rate% 34248 +43.8% 49254 ± 8% perf-stat.overall.path-length 8.317e+08 -32.4% 5.625e+08 ± 7% perf-stat.ps.branch-instructions 4863939 ± 5% +1617.8% 83552434 ± 13% perf-stat.ps.cache-misses 2.839e+08 ± 15% -44.8% 1.568e+08 ± 25% perf-stat.ps.cache-references 238042 -51.8% 114736 ± 15% perf-stat.ps.context-switches 1.494e+10 ± 3% +10.2% 1.646e+10 ± 2% perf-stat.ps.cpu-cycles 1.102e+09 -31.9% 7.51e+08 ± 7% perf-stat.ps.dTLB-loads 6.293e+08 -33.1% 4.209e+08 ± 8% perf-stat.ps.dTLB-stores 6821496 -34.7% 4451472 ± 7% perf-stat.ps.iTLB-load-misses 4.057e+09 -32.3% 2.749e+09 ± 7% perf-stat.ps.instructions 389775 ± 54% +7038.0% 27822357 ± 13% perf-stat.ps.node-load-misses 53006 ± 20% +140.1% 127264 ± 20% perf-stat.ps.node-loads 108429 ± 5% +20137.9% 21943845 ± 13% perf-stat.ps.node-store-misses 1.226e+12 -32.3% 8.298e+11 ± 7% perf-stat.total.instructions 838643 ± 8% -67.5% 272512 ± 11% interrupts.CAL:Function_call_interrupts 67.33 ± 32% -60.3% 26.71 ± 36% interrupts.CPU0.RES:Rescheduling_interrupts 108.33 ± 36% +93.3% 209.43 ± 15% interrupts.CPU1.NMI:Non-maskable_interrupts 108.33 ± 36% +93.3% 209.43 ± 15% interrupts.CPU1.PMI:Performance_monitoring_interrupts 114.33 ± 46% +84.8% 211.29 ± 15% interrupts.CPU10.NMI:Non-maskable_interrupts 114.33 ± 46% +84.8% 211.29 ± 15% interrupts.CPU10.PMI:Performance_monitoring_interrupts 102.00 ± 18% +200.4% 306.43 ± 83% interrupts.CPU100.NMI:Non-maskable_interrupts 102.00 ± 18% +200.4% 306.43 ± 83% interrupts.CPU100.PMI:Performance_monitoring_interrupts 117.50 ± 22% +648.2% 879.14 ±122% interrupts.CPU101.NMI:Non-maskable_interrupts 117.50 ± 22% +648.2% 879.14 ±122% interrupts.CPU101.PMI:Performance_monitoring_interrupts 117.00 ± 24% +67.2% 195.57 ± 20% interrupts.CPU103.NMI:Non-maskable_interrupts 117.00 ± 24% +67.2% 195.57 ± 20% interrupts.CPU103.PMI:Performance_monitoring_interrupts 98.83 ± 32% +799.3% 888.86 ±184% interrupts.CPU104.NMI:Non-maskable_interrupts 98.83 ± 32% +799.3% 888.86 ±184% interrupts.CPU104.PMI:Performance_monitoring_interrupts 111.50 ± 23% +87.7% 209.29 ± 16% interrupts.CPU105.NMI:Non-maskable_interrupts 111.50 ± 23% +87.7% 209.29 ± 16% interrupts.CPU105.PMI:Performance_monitoring_interrupts 104.83 ± 34% +565.5% 697.71 ±133% interrupts.CPU112.NMI:Non-maskable_interrupts 104.83 ± 34% +565.5% 697.71 ±133% interrupts.CPU112.PMI:Performance_monitoring_interrupts 104.50 ± 36% +64.6% 172.00 ± 24% interrupts.CPU129.NMI:Non-maskable_interrupts 104.50 ± 36% +64.6% 172.00 ± 24% interrupts.CPU129.PMI:Performance_monitoring_interrupts 113.17 ± 25% +142.8% 274.71 ± 91% interrupts.CPU142.NMI:Non-maskable_interrupts 113.17 ± 25% +142.8% 274.71 ± 91% interrupts.CPU142.PMI:Performance_monitoring_interrupts 222.33 ±214% -98.2% 4.00 ± 61% interrupts.CPU145.RES:Rescheduling_interrupts 119.33 ± 32% +1189.9% 1539 ±187% interrupts.CPU16.NMI:Non-maskable_interrupts 119.33 ± 32% +1189.9% 1539 ±187% interrupts.CPU16.PMI:Performance_monitoring_interrupts 624.33 ±171% -82.9% 106.71 ± 24% interrupts.CPU163.NMI:Non-maskable_interrupts 624.33 ±171% -82.9% 106.71 ± 24% interrupts.CPU163.PMI:Performance_monitoring_interrupts 128.50 ± 18% +40.0% 179.86 ± 24% interrupts.CPU2.NMI:Non-maskable_interrupts 128.50 ± 18% +40.0% 179.86 ± 24% interrupts.CPU2.PMI:Performance_monitoring_interrupts 388.33 ±210% -97.8% 8.71 ± 62% interrupts.CPU3.RES:Rescheduling_interrupts 94.50 ± 37% +102.0% 190.86 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts 94.50 ± 37% +102.0% 190.86 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts 105.00 ± 32% +1495.6% 1675 ±173% interrupts.CPU5.NMI:Non-maskable_interrupts 105.00 ± 32% +1495.6% 1675 ±173% interrupts.CPU5.PMI:Performance_monitoring_interrupts 119.83 ± 35% +60.6% 192.43 ± 21% interrupts.CPU6.NMI:Non-maskable_interrupts 119.83 ± 35% +60.6% 192.43 ± 21% interrupts.CPU6.PMI:Performance_monitoring_interrupts 95.00 ± 41% +968.4% 1015 ±200% interrupts.CPU8.NMI:Non-maskable_interrupts 95.00 ± 41% +968.4% 1015 ±200% interrupts.CPU8.PMI:Performance_monitoring_interrupts 97.50 ± 48% +96.9% 192.00 ± 22% interrupts.CPU9.NMI:Non-maskable_interrupts 97.50 ± 48% +96.9% 192.00 ± 22% interrupts.CPU9.PMI:Performance_monitoring_interrupts 123.17 ± 29% +71.1% 210.71 ± 12% interrupts.CPU96.NMI:Non-maskable_interrupts 123.17 ± 29% +71.1% 210.71 ± 12% interrupts.CPU96.PMI:Performance_monitoring_interrupts 102.17 ± 17% +111.0% 215.57 ± 16% interrupts.CPU98.NMI:Non-maskable_interrupts 102.17 ± 17% +111.0% 215.57 ± 16% interrupts.CPU98.PMI:Performance_monitoring_interrupts 105.67 ± 24% +285.6% 407.43 ±118% interrupts.CPU99.NMI:Non-maskable_interrupts 105.67 ± 24% +285.6% 407.43 ±118% interrupts.CPU99.PMI:Performance_monitoring_interrupts 576.33 ± 7% -50.7% 284.14 ± 2% interrupts.IWI:IRQ_work_interrupts 30.77 -5.4 25.32 ± 8% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 30.57 -5.3 25.23 ± 8% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.27 ± 3% -2.8 8.43 ± 21% perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.24 ± 3% -2.8 8.42 ± 21% perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe 19.14 -2.4 16.71 ± 7% perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 19.10 -2.4 16.70 ± 7% perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.14 ± 11% -2.3 1.81 ± 36% perf-profile.calltrace.cycles-pp.udp_send_skb.udp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto 4.06 ± 11% -2.3 1.79 ± 36% perf-profile.calltrace.cycles-pp.ip_send_skb.udp_send_skb.udp_sendmsg.sock_sendmsg.__sys_sendto 3.99 ± 11% -2.2 1.76 ± 36% perf-profile.calltrace.cycles-pp.ip_output.ip_send_skb.udp_send_skb.udp_sendmsg.sock_sendmsg 3.85 ± 12% -2.1 1.70 ± 36% perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.ip_send_skb.udp_send_skb.udp_sendmsg 3.46 ± 13% -1.9 1.55 ± 36% perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.ip_send_skb.udp_send_skb 3.42 ± 13% -1.9 1.54 ± 36% perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.ip_send_skb 3.38 ± 13% -1.9 1.52 ± 36% perf-profile.calltrace.cycles-pp.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output 3.23 ± 13% -1.8 1.48 ± 36% perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip.ip_finish_output2 3.14 ± 13% -1.7 1.44 ± 36% perf-profile.calltrace.cycles-pp.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq.__local_bh_enable_ip 3.11 ± 13% -1.7 1.44 ± 36% perf-profile.calltrace.cycles-pp.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start.do_softirq 3.01 ± 14% -1.6 1.39 ± 36% perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action.__softirqentry_text_start 2.93 ± 14% -1.6 1.37 ± 36% perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll.net_rx_action 2.87 ± 14% -1.5 1.35 ± 36% perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.__napi_poll 2.86 ± 14% -1.5 1.34 ± 36% perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog 2.85 ± 14% -1.5 1.34 ± 36% perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core 2.82 ± 14% -1.5 1.33 ± 36% perf-profile.calltrace.cycles-pp.__udp4_lib_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv 2.00 ± 18% -1.4 0.58 ± 87% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable 2.68 ± 15% -1.4 1.29 ± 36% perf-profile.calltrace.cycles-pp.udp_unicast_rcv_skb.__udp4_lib_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver 2.65 ± 15% -1.4 1.27 ± 36% perf-profile.calltrace.cycles-pp.udp_queue_rcv_one_skb.udp_unicast_rcv_skb.__udp4_lib_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish 2.01 ± 18% -1.4 0.65 ± 69% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.sock_def_readable.__udp_enqueue_schedule_skb 2.26 ± 17% -1.3 0.93 ± 36% perf-profile.calltrace.cycles-pp.sock_def_readable.__udp_enqueue_schedule_skb.udp_queue_rcv_one_skb.udp_unicast_rcv_skb.__udp4_lib_rcv 2.06 ± 18% -1.3 0.76 ± 69% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.__udp_enqueue_schedule_skb.udp_queue_rcv_one_skb 2.21 ± 17% -1.3 0.92 ± 36% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.__udp_enqueue_schedule_skb.udp_queue_rcv_one_skb.udp_unicast_rcv_skb 2.40 ± 16% -1.3 1.13 ± 38% perf-profile.calltrace.cycles-pp.__udp_enqueue_schedule_skb.udp_queue_rcv_one_skb.udp_unicast_rcv_skb.__udp4_lib_rcv.ip_protocol_deliver_rcu 1.31 ± 5% -1.0 0.33 ± 87% perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg 1.29 ± 5% -1.0 0.33 ± 87% perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp 2.14 ± 4% -1.0 1.18 ± 19% perf-profile.calltrace.cycles-pp.__skb_recv_udp.udp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 1.35 ± 5% -0.9 0.48 ± 64% perf-profile.calltrace.cycles-pp.schedule_timeout.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg.inet_recvmsg 1.71 ± 4% -0.8 0.89 ± 19% perf-profile.calltrace.cycles-pp.__skb_wait_for_more_packets.__skb_recv_udp.udp_recvmsg.inet_recvmsg.__sys_recvfrom 0.84 ± 7% -0.4 0.40 ± 88% perf-profile.calltrace.cycles-pp.__consume_stateless_skb.udp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom 0.51 ± 44% +0.7 1.19 ± 36% perf-profile.calltrace.cycles-pp.ktime_get.tick_irq_enter.irq_enter_rcu.sysvec_apic_timer_interrupt.asm_sysvec_apic_timer_interrupt 16.97 ± 10% +3.7 20.65 ± 13% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 67.79 +5.8 73.59 ± 2% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 67.79 +5.8 73.60 ± 2% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64_no_verify 67.73 +5.8 73.55 ± 2% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 68.07 +5.9 74.00 ± 2% perf-profile.calltrace.cycles-pp.secondary_startup_64_no_verify 7.05 ± 46% +6.8 13.84 ± 14% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify 31.00 -5.5 25.53 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 30.80 -5.4 25.44 ± 7% perf-profile.children.cycles-pp.do_syscall_64 4.59 ± 4% -4.5 0.07 ± 21% perf-profile.children.cycles-pp.poll_idle 11.28 ± 3% -2.8 8.43 ± 21% perf-profile.children.cycles-pp.__x64_sys_recvfrom 11.24 ± 3% -2.8 8.42 ± 21% perf-profile.children.cycles-pp.__sys_recvfrom 19.14 -2.4 16.71 ± 7% perf-profile.children.cycles-pp.__x64_sys_sendto 19.11 -2.4 16.70 ± 7% perf-profile.children.cycles-pp.__sys_sendto 4.14 ± 11% -2.3 1.81 ± 36% perf-profile.children.cycles-pp.udp_send_skb 4.06 ± 11% -2.3 1.79 ± 36% perf-profile.children.cycles-pp.ip_send_skb 4.00 ± 11% -2.2 1.76 ± 36% perf-profile.children.cycles-pp.ip_output 3.85 ± 12% -2.2 1.70 ± 36% perf-profile.children.cycles-pp.ip_finish_output2 3.48 ± 13% -1.9 1.55 ± 36% perf-profile.children.cycles-pp.__local_bh_enable_ip 3.42 ± 13% -1.9 1.54 ± 36% perf-profile.children.cycles-pp.do_softirq 3.24 ± 13% -1.8 1.48 ± 36% perf-profile.children.cycles-pp.net_rx_action 2.46 ± 2% -1.7 0.73 ± 22% perf-profile.children.cycles-pp.__schedule 4.89 ± 9% -1.7 3.18 ± 19% perf-profile.children.cycles-pp.__softirqentry_text_start 3.14 ± 13% -1.7 1.45 ± 36% perf-profile.children.cycles-pp.__napi_poll 3.11 ± 13% -1.7 1.44 ± 36% perf-profile.children.cycles-pp.process_backlog 3.01 ± 14% -1.6 1.39 ± 36% perf-profile.children.cycles-pp.__netif_receive_skb_one_core 2.93 ± 14% -1.6 1.37 ± 36% perf-profile.children.cycles-pp.ip_rcv 2.87 ± 14% -1.5 1.35 ± 36% perf-profile.children.cycles-pp.ip_local_deliver 2.86 ± 14% -1.5 1.34 ± 36% perf-profile.children.cycles-pp.ip_local_deliver_finish 2.85 ± 14% -1.5 1.34 ± 36% perf-profile.children.cycles-pp.ip_protocol_deliver_rcu 2.82 ± 14% -1.5 1.33 ± 36% perf-profile.children.cycles-pp.__udp4_lib_rcv 2.68 ± 15% -1.4 1.29 ± 36% perf-profile.children.cycles-pp.udp_unicast_rcv_skb 2.65 ± 15% -1.4 1.27 ± 36% perf-profile.children.cycles-pp.udp_queue_rcv_one_skb 2.26 ± 17% -1.3 0.93 ± 36% perf-profile.children.cycles-pp.sock_def_readable 2.22 ± 17% -1.3 0.93 ± 36% perf-profile.children.cycles-pp.__wake_up_common_lock 2.40 ± 15% -1.3 1.13 ± 38% perf-profile.children.cycles-pp.__udp_enqueue_schedule_skb 2.02 ± 17% -1.2 0.78 ± 38% perf-profile.children.cycles-pp.autoremove_wake_function 2.02 ± 18% -1.2 0.78 ± 38% perf-profile.children.cycles-pp.try_to_wake_up 2.07 ± 18% -1.2 0.91 ± 37% perf-profile.children.cycles-pp.__wake_up_common 2.14 ± 4% -1.0 1.18 ± 19% perf-profile.children.cycles-pp.__skb_recv_udp 1.12 ± 3% -0.9 0.20 ± 36% perf-profile.children.cycles-pp.schedule_idle 1.37 ± 5% -0.8 0.54 ± 20% perf-profile.children.cycles-pp.schedule 1.71 ± 4% -0.8 0.89 ± 19% perf-profile.children.cycles-pp.__skb_wait_for_more_packets 1.36 ± 4% -0.7 0.61 ± 19% perf-profile.children.cycles-pp.schedule_timeout 0.83 ± 18% -0.6 0.19 ± 34% perf-profile.children.cycles-pp.ttwu_do_activate 0.82 ± 19% -0.6 0.18 ± 36% perf-profile.children.cycles-pp.enqueue_task_fair 0.60 ± 3% -0.5 0.07 ± 9% perf-profile.children.cycles-pp.pick_next_task_fair 0.66 ± 19% -0.5 0.17 ± 39% perf-profile.children.cycles-pp.enqueue_entity 0.80 ± 17% -0.5 0.32 ± 20% perf-profile.children.cycles-pp.update_rq_clock 0.72 ± 6% -0.4 0.32 ± 21% perf-profile.children.cycles-pp.dequeue_task_fair 0.39 ± 5% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.update_load_avg 0.82 ± 17% -0.3 0.52 ± 18% perf-profile.children.cycles-pp.sched_clock_cpu 0.59 ± 5% -0.3 0.29 ± 22% perf-profile.children.cycles-pp.dequeue_entity 0.72 ± 15% -0.3 0.45 ± 16% perf-profile.children.cycles-pp.sched_clock 0.69 ± 15% -0.3 0.42 ± 17% perf-profile.children.cycles-pp.native_sched_clock 0.84 ± 7% -0.3 0.57 ± 27% perf-profile.children.cycles-pp.__consume_stateless_skb 0.29 ± 6% -0.3 0.03 ± 88% perf-profile.children.cycles-pp.kmem_cache_free 0.61 ± 7% -0.2 0.40 ± 29% perf-profile.children.cycles-pp.__free_pages_ok 0.30 ± 24% -0.2 0.10 ± 37% perf-profile.children.cycles-pp.ip_route_output_flow 0.73 ± 7% -0.2 0.53 ± 9% perf-profile.children.cycles-pp._raw_spin_lock 0.29 ± 23% -0.2 0.10 ± 37% perf-profile.children.cycles-pp.ip_route_output_key_hash 0.30 ± 16% -0.2 0.12 ± 36% perf-profile.children.cycles-pp.__dev_queue_xmit 0.27 ± 23% -0.2 0.09 ± 37% perf-profile.children.cycles-pp.ip_route_output_key_hash_rcu 0.21 ± 21% -0.1 0.07 ± 51% perf-profile.children.cycles-pp.fib_table_lookup 0.64 ± 7% -0.1 0.50 ± 10% perf-profile.children.cycles-pp.read_tsc 0.21 ± 13% -0.1 0.07 ± 48% perf-profile.children.cycles-pp.dev_hard_start_xmit 0.19 ± 11% -0.1 0.06 ± 49% perf-profile.children.cycles-pp.__switch_to 0.19 ± 11% -0.1 0.07 ± 49% perf-profile.children.cycles-pp.loopback_xmit 0.40 ± 18% -0.1 0.28 ± 17% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.17 ± 10% -0.1 0.05 ± 42% perf-profile.children.cycles-pp.__switch_to_asm 0.19 ± 6% -0.1 0.08 ± 29% perf-profile.children.cycles-pp.move_addr_to_user 0.16 ± 7% -0.1 0.07 ± 12% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.13 ± 24% -0.1 0.06 ± 72% perf-profile.children.cycles-pp.__free_one_page 0.13 ± 20% -0.1 0.06 ± 46% perf-profile.children.cycles-pp.sockfd_lookup_light 0.12 ± 19% -0.1 0.05 ± 69% perf-profile.children.cycles-pp.__fget_light 0.18 ± 9% -0.1 0.11 ± 18% perf-profile.children.cycles-pp.call_cpuidle 0.18 ± 9% -0.1 0.13 ± 12% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.09 ± 11% -0.0 0.04 ± 67% perf-profile.children.cycles-pp.udp_rmem_release 0.09 ± 10% -0.0 0.05 ± 46% perf-profile.children.cycles-pp.trigger_load_balance 0.05 ± 47% +0.1 0.16 ± 21% perf-profile.children.cycles-pp.cpuacct_charge 0.00 +0.1 0.11 ± 26% perf-profile.children.cycles-pp.llist_reverse_order 0.00 +0.2 0.16 ± 13% perf-profile.children.cycles-pp.flush_smp_call_function_queue 0.00 +0.3 0.27 ± 59% perf-profile.children.cycles-pp.__smp_call_single_queue 0.00 +0.3 0.27 ± 59% perf-profile.children.cycles-pp.llist_add_batch 0.00 +0.4 0.36 ± 20% perf-profile.children.cycles-pp.sched_ttwu_pending 0.00 +0.4 0.38 ± 42% perf-profile.children.cycles-pp.ttwu_queue_wakelist 0.00 +0.5 0.55 ± 14% perf-profile.children.cycles-pp.flush_smp_call_function_from_idle 2.35 ± 25% +1.1 3.48 ± 19% perf-profile.children.cycles-pp.ktime_get 14.90 ± 4% +2.3 17.23 ± 10% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt 67.79 +5.8 73.60 ± 2% perf-profile.children.cycles-pp.start_secondary 68.07 +5.9 74.00 ± 2% perf-profile.children.cycles-pp.secondary_startup_64_no_verify 68.07 +5.9 74.00 ± 2% perf-profile.children.cycles-pp.cpu_startup_entry 68.06 +5.9 73.99 ± 2% perf-profile.children.cycles-pp.do_idle 7.12 ± 45% +6.8 13.97 ± 14% perf-profile.children.cycles-pp.menu_select 4.54 ± 4% -4.5 0.05 ± 65% perf-profile.self.cycles-pp.poll_idle 0.46 ± 20% -0.4 0.05 ± 66% perf-profile.self.cycles-pp.update_rq_clock 0.47 ± 9% -0.4 0.09 ± 18% perf-profile.self.cycles-pp.__schedule 0.65 ± 15% -0.3 0.39 ± 15% perf-profile.self.cycles-pp.native_sched_clock 0.55 ± 18% -0.2 0.32 ± 20% perf-profile.self.cycles-pp.do_idle 0.32 ± 25% -0.2 0.08 ± 30% perf-profile.self.cycles-pp.enqueue_entity 0.71 ± 7% -0.2 0.52 ± 9% perf-profile.self.cycles-pp._raw_spin_lock 0.35 ± 7% -0.2 0.18 ± 32% perf-profile.self.cycles-pp.__free_pages_ok 0.62 ± 7% -0.1 0.49 ± 10% perf-profile.self.cycles-pp.read_tsc 0.17 ± 9% -0.1 0.05 ± 70% perf-profile.self.cycles-pp.__switch_to 0.16 ± 22% -0.1 0.04 ± 89% perf-profile.self.cycles-pp.fib_table_lookup 0.16 ± 9% -0.1 0.05 ± 43% perf-profile.self.cycles-pp.__switch_to_asm 0.15 ± 8% -0.1 0.06 ± 18% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.12 ± 19% -0.1 0.05 ± 69% perf-profile.self.cycles-pp.__fget_light 0.16 ± 10% -0.1 0.11 ± 15% perf-profile.self.cycles-pp.call_cpuidle 0.12 ± 9% -0.1 0.06 ± 23% perf-profile.self.cycles-pp.__softirqentry_text_start 0.06 ± 52% +0.0 0.11 ± 30% perf-profile.self.cycles-pp.udp_queue_rcv_one_skb 0.00 +0.1 0.08 ± 24% perf-profile.self.cycles-pp.ttwu_queue_wakelist 0.02 ±142% +0.1 0.12 ± 30% perf-profile.self.cycles-pp.__wake_up_common 0.00 +0.1 0.11 ± 28% perf-profile.self.cycles-pp.sched_ttwu_pending 0.05 ± 47% +0.1 0.16 ± 21% perf-profile.self.cycles-pp.cpuacct_charge 0.00 +0.1 0.11 ± 26% perf-profile.self.cycles-pp.llist_reverse_order 0.00 +0.3 0.27 ± 59% perf-profile.self.cycles-pp.llist_add_batch 1.78 ± 35% +1.3 3.05 ± 23% perf-profile.self.cycles-pp.ktime_get 5.18 ± 60% +6.1 11.25 ± 19% perf-profile.self.cycles-pp.cpuidle_enter_state 4.81 ± 62% +6.7 11.54 ± 19% perf-profile.self.cycles-pp.menu_select Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. --- 0DAY/LKP+ Test Infrastructure Open Source Technology Center https://lists.01.org/hyperkitty/list/lkp@lists.01.org Intel Corporation Thanks, Oliver Sang