November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] sbin/submit: submit separate_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will add more spec cases, so it should give each type of cases for different input and output directories Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/separate_yaml/ftq.yaml | 22 +++++++++++++++ .../ftq/ftq-1000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-10000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-20-2-6000ss-cache.yaml | 28 +++++++++++++++++++ spec/submit_spec.rb | 4 +++ 5 files changed, 110 insertions(+) create mode 100644 spec/submit/separate_yaml/ftq.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml diff --git a/spec/submit/separate_yaml/ftq.yaml b/spec/submit/separate_yaml/ftq.yaml new file mode 100644 index 00000000..8564d443 --- /dev/null +++ b/spec/submit/separate_yaml/ftq.yaml @@ -0,0 +1,22 @@ +suite: ftq +category: noise-benchmark + +nr_task: 100% + +samples: 100000ss + +ftq: + test: add + freq: 10000 + +--- +ftq: + test: add + freq: 1000 + +--- + +samples: 6000ss +ftq: + test: cache + freq: 20 diff --git a/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml new file mode 100644 index 00000000..855e7210 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 1000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 1000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml new file mode 100644 index 00000000..78725424 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 10000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 10000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml new file mode 100644 index 00000000..40edfcb2 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 6000ss +ftq: + test: cache + freq: 20 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 6000ss + freq: 20 + test: cache + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ce6fed25 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'separate yaml spec' do + submit_job('separate_yaml') + end end -- 2.23.0

2 1

[PATCH v3 compass-ci 2/2] job.cr: give kernel_version a default value
by Xu Xijian 25 Nov '20

25 Nov '20

If the kernel_version is not set by users, give it a default value as same as vmlinuz version. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index b533665..b87b55f 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -358,11 +358,11 @@ class Job private def set_kernel_version boot_dir = "#{SRV_OS}/#{os_dir}/boot" - suffix = "-#{kernel_version}" if self["kernel_version"]? - self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz#{suffix}") + self["kernel_version"] ||= File.basename(File.real_path "#{boot_dir}/vmlinuz").lchop("vmlinuz-") + self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz-#{kernel_version}") if "#{os_mount}" == "initramfs" - self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules#{suffix}.cgz") - self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers#{suffix}.cgz") + self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules-#{kernel_version}.cgz") + self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers-#{kernel_version}.cgz") end end -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] block_helper.cr: change if block into one line
by Xu Xijian 25 Nov '20

25 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/block_helper.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/lib/block_helper.cr b/src/lib/block_helper.cr index 37ad367..596ef7e 100644 --- a/src/lib/block_helper.cr +++ b/src/lib/block_helper.cr @@ -28,9 +28,7 @@ class BlockHelper @block_helper[uuid] = fiber end - if yield == true - spawn fiber.run - end + spawn fiber.run if yield == true until fiber.dead? Fiber.yield -- 2.23.0

1 0

[PATCH v2 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will add more spec cases, so it should give each type of cases for different input and output directories Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 3 + .../template/template-defaults.yaml | 71 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 71 +++++++++++++++++++ spec/submit_spec.rb | 4 ++ 5 files changed, 150 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/template/template-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..60e3df16 --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,3 @@ +suite: test +category: benchmark +test: diff --git a/spec/submit/merge_yaml/template/template-defaults.yaml b/spec/submit/merge_yaml/template/template-defaults.yaml new file mode 100644 index 00000000..f1d13d04 --- /dev/null +++ b/spec/submit/merge_yaml/template/template-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +test: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..f1d13d04 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +test: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..b56a860c 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

3 2

[PATCH compass-ci 2/2] sparrow/0-package: fix docker.service restart failed
by Liu Yinsi 25 Nov '20

25 Nov '20

[why] when execute systemctl restart docker, error message: localhost.localdomain systemd[1]:docker.service: Failed with result 'timeout'. localhost.localdomain systemd[1]:Failed to start Docker Application Container Engine. because docker restart default timeout is 90s, when run too much containers, restart docker may need several minutes, then restart failed. [how] set TimeoutSec=0 to avoid timeout. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/common | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 5eb9f76..2753b9d 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -40,4 +40,7 @@ cat > /etc/docker/daemon.json <<EOF } EOF +sed '/^Environment=/a\TimeoutSec=0' /usr/lib/systemd/system/docker.service + +systemctl daemon-reload systemctl restart docker -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow/0-package: fix failed to start ssh-r
by Liu Yinsi 25 Nov '20

25 Nov '20

[why] when execute ssh-r/start, error message: docker: Error response from daemon: driver failed programming external connectivity on endpoint ssh_r (55638ba16aa1264dff0fdc1badc93176eec07d2b848d4a8d418d07011d84313d): Error starting userland proxy: listen tcp 0.0.0.0:51998: bind: address already in use. [how] because ssh-r will bind ports 50000-51999, set ip_local_port_range to prevent occupation. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/common | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 523438c..5eb9f76 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -12,6 +12,7 @@ cat >> /etc/sysctl.conf <<EOF net.ipv4.ip_forward=1 net.ipv6.bindv6only=1 vm.max_map_count=262144 +net.ipv4.ip_local_port_range = 1024 48999 EOF sysctl -p -- 2.23.0

1 0

[PATCH compass-ci] job.cr: give kernel_version a default value
by Xu Xijian 25 Nov '20

25 Nov '20

If the kernel_version is not set by users, give it a default value as same as vmlinuz version. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index b533665..fa804e5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -358,11 +358,11 @@ class Job private def set_kernel_version boot_dir = "#{SRV_OS}/#{os_dir}/boot" - suffix = "-#{kernel_version}" if self["kernel_version"]? - self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz#{suffix}") + self["kernel_version"] ||= File.basename(File.real_path "#{boot_dir}/vmlinuz").gsub("vmlinuz-", "") + self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz-#{kernel_version}") if "#{os_mount}" == "initramfs" - self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules#{suffix}.cgz") - self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers#{suffix}.cgz") + self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules-#{kernel_version}.cgz") + self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers-#{kernel_version}.cgz") end end -- 2.23.0

1 1

[PATCH v2 compass-ci] job.cr: give kernel_version a default value
by Xu Xijian 25 Nov '20

25 Nov '20

If the kernel_version is not set by users, give it a default value as same as vmlinuz version. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index b533665..a9923cc 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -358,11 +358,11 @@ class Job private def set_kernel_version boot_dir = "#{SRV_OS}/#{os_dir}/boot" - suffix = "-#{kernel_version}" if self["kernel_version"]? - self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz#{suffix}") + self["kernel_version"] ||= File.basename(File.real_path "#{boot_dir}/vmlinuz").gsub("vmlinuz-", "") + self["linux_vmlinuz_path"] = File.real_path("#{boot_dir}/vmlinuz-#{kernel_version}") if "#{os_mount}" == "initramfs" - self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules#{suffix}.cgz") - self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers#{suffix}.cgz") + self["linux_modules_initrd"] = File.real_path("#{boot_dir}/modules-#{kernel_version}.cgz") + self["linux_headers_initrd"] = File.real_path("#{boot_dir}/headers-#{kernel_version}.cgz") end end -- 2.23.0

1 0

[PATCH v2 lkp-tests] daemon/sshd: fix pub_key to my_ssh_pubkey
by Zhang Yale 25 Nov '20

25 Nov '20

[Why] - "my_ssh_pubkey" field is a set of all pub_keys that had been submitted by users. - We can save all the pub_keys submitted by users in the ES database and set the login mode to password-free. - Therefore, users can sometimes borrow a machine to test without providing a pub_key. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- daemon/sshd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index a3fdf954..c7d7b60c 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,5 +1,5 @@ #!/bin/sh -# pub_key +# my_ssh_pubkey # my_email # sshr_port # sshr_port_len @@ -14,10 +14,10 @@ sshr_ip=$LKP_SERVER run_ssh() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh - echo "$pub_key" > /root/.ssh/authorized_keys + echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys systemctl start sshd } @@ -73,11 +73,11 @@ compose_email() set_port() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%${sshr_port_len}+${sshr_port_base})) + port=$(($(date +%s%N)%$sshr_port_len+$sshr_port_base)) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return -- 2.23.0

1 1

[PATCH lkp-tests] setup/sysctl: add parameters
by Zhang Yu 25 Nov '20

25 Nov '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- setup/sysctl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/setup/sysctl b/setup/sysctl index 81374939..913b83d3 100755 --- a/setup/sysctl +++ b/setup/sysctl @@ -202,6 +202,9 @@ # - sc_netdev_max_backlog # - sc_rmem_max # - sc_wmem_max +# - sc_tcp_rmem +# - sc_tcp_wmem +# - sc_sched_features . $LKP_SRC/lib/reproduce-log.sh @@ -408,5 +411,8 @@ [ -n "$sc_netdev_max_backlog" ] && log_eval "echo '$sc_netdev_max_backlog' > '/proc/sys/net/core/netdev_max_backlog'" [ -n "$sc_rmem_max" ] && log_eval "echo '$sc_rmem_max' > '/proc/sys/net/core/rmem_max'" [ -n "$sc_wmem_max" ] && log_eval "echo '$sc_wmem_max' > '/proc/sys/net/core/wmem_max'" +[ -n "$sc_tcp_rmem" ] && log_eval "echo '$sc_tcp_rmem' > '/proc/sys/net/ipv4/tcp_rmem'" +[ -n "$sc_tcp_wmem" ] && log_eval "echo '$sc_tcp_wmem' > '/proc/sys/net/ipv4/tcp_wmem'" +[ -n "$sc_sched_features" ] && log_eval "echo '$sc_sched_features' > '/sys/kernel/debug/sched_features'" exit 0 -- 2.23.0

1 0