- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 1/2] daemon/multi-qemu: fix default parameter format
by Xiao Shenwei 25 Nov '20

25 Nov '20

before: : ${queues:="$tbox_group.$HOSTNAME, $tbox_group.$(arch)"} now: : ${queues:="$tbox_group.$HOSTNAME,$tbox_group.$(arch)"} this parameter will be used by ruby script for ruby command line parameter, if use space like: multi-qemu -q vm-2p8g.taishan200-2280-2s48p-256g--a1, vm-2p8g.aarch64 the vm-2p8g.aarch64 doesn't work correct usage: multi-qemu -q vm-2p8g.taishan200-2280-2s48p-256g--a1,vm-2p8g.aarch64 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- daemon/multi-qemu | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/daemon/multi-qemu b/daemon/multi-qemu index 7cc625f4..aa2348b4 100755 --- a/daemon/multi-qemu +++ b/daemon/multi-qemu @@ -6,7 +6,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${queues:="$tbox_group.$HOSTNAME, $tbox_group.$(arch)"} +: ${queues:="$tbox_group.$HOSTNAME,$tbox_group.$(arch)"} + multi_qemu() { export CCI_SRC=/c/compass-ci -- 2.23.0

2 1

[PATCH compass-ci] job.cr: delete 2 variables from METHOD_KEYS
by Xu Xijian 25 Nov '20

25 Nov '20

Since linux_modules_initrd and linux_headers_initrd are only used in initramfs, they don't need to be initialized in all jobs, so delete them from METHOD_KEYS. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index a9923cc..378fd00 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -97,8 +97,6 @@ class Job docker_image kernel_version linux_vmlinuz_path - linux_modules_initrd - linux_headers_initrd ) macro method_missing(call) @@ -390,9 +388,9 @@ class Job temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{linux_modules_initrd}")}" + "#{JobHelper.service_path(self["linux_modules_initrd"])}" temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{linux_headers_initrd}")}" + "#{JobHelper.service_path(self["linux_headers_initrd"])}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

1 0

[PATCH lkp-tests 2/2] lib/bootstrap.sh: fix redirect_stdout_stderr for docker
by Xiao Shenwei 25 Nov '20

25 Nov '20

refer-to: compass-ci: c7c4ba94b3ef00a3f5c91200f85ea2d924726132 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/bootstrap.sh | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index ce34a632..46e77e72 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -270,11 +270,7 @@ announce_bootup() redirect_stdout_stderr() { - if is_docker; then - exec > /tmp/stdout - exec 2> /tmp/stderr - return - fi + is_docker && return [ -c /dev/kmsg ] || return has_cmd tail || return -- 2.23.0

1 0

[PATCH compass-ci] provider/docker: forward correct and error logs to docker log collector
by Xiao Shenwei 25 Nov '20

25 Nov '20

[why] $LKP_SRC/lib/bootstrap.sh:271 redirect_stdout_stderr() result file: output stderr stdout these file will based on /tmp/stdout and /tmp/stderr the function will execute: exec > /tmp/stdout exec 2> /tmp/stderr docker log based on /dev/stdout and /dev/stderr, so docker log can't collect the job's output. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 63cb605..3d04406 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -2,4 +2,11 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap +# forward correct and error logs to docker log collector +touch /tmp/stderr /tmp/strout +tail -f /tmp/stdout > /dev/stdout & +echo $! >> /tmp/pid-tail-global +tail -f /tmp/stderr > /dev/stderr & +echo $! >> /tmp/pid-tail-global + +sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap 1> /tmp/stdout 2> /tmp/stderr -- 2.23.0

1 0

[PATCH v4 lkp-tests] daemon/sshd: change pub_key to my_ssh_pubkey
by Zhang Yale 25 Nov '20

25 Nov '20

[why] 1. "my_ssh_pubkey" field is a collection of all pub_keys submitted by users. 2. We can save all the pub_keys submitted by users in the ES database and password-free login. 3. Therefore, users can sometimes borrow a machine to test without providing a pub_key. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- daemon/sshd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index a3fdf954..c7d7b60c 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,5 +1,5 @@ #!/bin/sh -# pub_key +# my_ssh_pubkey # my_email # sshr_port # sshr_port_len @@ -14,10 +14,10 @@ sshr_ip=$LKP_SERVER run_ssh() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh - echo "$pub_key" > /root/.ssh/authorized_keys + echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys systemctl start sshd } @@ -73,11 +73,11 @@ compose_email() set_port() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%${sshr_port_len}+${sshr_port_base})) + port=$(($(date +%s%N)%$sshr_port_len+$sshr_port_base)) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return -- 2.23.0

1 0

[PATCH compass-ci] block_helper.cr: refactor if block into one line
by Xu Xijian 25 Nov '20

25 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/block_helper.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/lib/block_helper.cr b/src/lib/block_helper.cr index 37ad367..596ef7e 100644 --- a/src/lib/block_helper.cr +++ b/src/lib/block_helper.cr @@ -28,9 +28,7 @@ class BlockHelper @block_helper[uuid] = fiber end - if yield == true - spawn fiber.run - end + spawn fiber.run if yield == true until fiber.dead? Fiber.yield -- 2.23.0

1 0

[PATCH lkp-tests] submit: fix the processes of option: --my-queue
by Ren Wen 25 Nov '20

25 Nov '20

If use options '--my-queue' and '-o' at the same time, the output job file will not contain the value of the covered field 'queue'. So pass the 'job' not job_hash. Signed-off-by: Ren Wen <15991987063(a)163.com> --- sbin/submit | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index be588bed..74815202 100755 --- a/sbin/submit +++ b/sbin/submit @@ -139,12 +139,13 @@ ARGV.each do |jobfile| sh_hash['job2sh']['on_fail'] = sh_on_fail end + add_my_queue(job) if opt_my_queue + # merge job info job_hash = job.to_hash job_hash_list << job_hash job_hash = job_hash.merge(sh_hash) - add_my_queue(job_hash) if opt_my_queue # save job to yaml if opt_output_dir prefix = File.join(opt_output_dir, File.basename(jobfile, '.yaml')) -- 2.23.0

2 1

[PATCH v2 lkp-tests] tests/build-pkg: update mount option
by Lin Jiaxin 25 Nov '20

25 Nov '20

[why] refer-to: compass-ci: commit 96584b45e528a4bad1f9 [how] add port=446 option to mount.cifs Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 65c83e22..29d35997 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -37,7 +37,7 @@ mount_dest() mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } -- 2.23.0

1 0

[PATCH lkp-tests] lkp-tests/build-pkg: update mount option
by Lin Jiaxin 25 Nov '20

25 Nov '20

[why] refer-to: compass-ci: commit 96584b45e528a4bad1f9 [how] add port=446 option to mount.cifs Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 65c83e22..29d35997 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -37,7 +37,7 @@ mount_dest() mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } -- 2.23.0

1 0

[PATCH v2 lkp-tests] sbin/submit: submit separate_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will use 'rspec spec/submit_spec.rb' to separate job.yaml, it contains several yaml documents separated by "---" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/separate_yaml/ftq.yaml | 20 +++++++++++++ .../ftq/ftq-1000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-10000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-20-2-6000ss-cache.yaml | 28 +++++++++++++++++++ spec/submit_spec.rb | 4 +++ 5 files changed, 108 insertions(+) create mode 100644 spec/submit/separate_yaml/ftq.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml diff --git a/spec/submit/separate_yaml/ftq.yaml b/spec/submit/separate_yaml/ftq.yaml new file mode 100644 index 00000000..e54adc8b --- /dev/null +++ b/spec/submit/separate_yaml/ftq.yaml @@ -0,0 +1,20 @@ +suite: ftq +category: noise-benchmark + +nr_task: 100% +samples: 100000ss + +ftq: + test: add + freq: 10000 + +--- +ftq: + test: add + freq: 1000 + +--- +samples: 6000ss +ftq: + test: cache + freq: 20 diff --git a/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml new file mode 100644 index 00000000..855e7210 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 1000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 1000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml new file mode 100644 index 00000000..78725424 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 10000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 10000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml new file mode 100644 index 00000000..40edfcb2 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 6000ss +ftq: + test: cache + freq: 20 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 6000ss + freq: 20 + test: cache + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ce6fed25 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'separate yaml spec' do + submit_job('separate_yaml') + end end -- 2.23.0

1 0

[PATCH v3 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/atomic.yaml | 4 ++ .../merge_yaml/atomic/atomic-defaults.yaml | 72 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 71 ++++++++++++++++++ spec/submit_spec.rb | 4 ++ 5 files changed, 152 insertions(+) create mode 100644 spec/submit/merge_yaml/atomic.yaml create mode 100644 spec/submit/merge_yaml/atomic/atomic-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/atomic.yaml b/spec/submit/merge_yaml/atomic.yaml new file mode 100644 index 00000000..d0927923 --- /dev/null +++ b/spec/submit/merge_yaml/atomic.yaml @@ -0,0 +1,4 @@ +suite: atomic +category: benchmark + +atomic: diff --git a/spec/submit/merge_yaml/atomic/atomic-defaults.yaml b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml new file mode 100644 index 00000000..920ebec2 --- /dev/null +++ b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml @@ -0,0 +1,72 @@ +--- +suite: atomic +category: benchmark +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: {} + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..30019c6c --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: atomic.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..fc9e073a --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..b56a860c 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH compass-ci] service/scheduler-taskqueue: use before_all
by Wu Zhende 25 Nov '20

25 Nov '20

use before_all to replace the same content in each API Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/scheduler/scheduler.cr | 18 ++++-------------- src/taskqueue/taskqueue.cr | 10 ++++------ 2 files changed, 8 insertions(+), 20 deletions(-) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index a946ea0..5f97567 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -32,6 +32,10 @@ module Scheduler sched = Sched.new + before_all do |env| + env.response.headers["Connection"] = "close" + end + # for debug (maybe kemal debug|logger does better) def self.debug_message(env, response) puts %({"from": "#{env.request.remote_address}", "response": #{response.to_json}}) @@ -39,7 +43,6 @@ module Scheduler # echo alive get "/" do |env| - env.response.headers["Connection"] = "close" debug_message(env, "Env= {\n#{`export`}}") "LKP Alive! The time is #{Time.local}, version = #{VERSION}" end @@ -50,7 +53,6 @@ module Scheduler # /boot.xxx/host/${hostname} # /boot.yyy/mac/${mac} get "/boot.:boot_type/:parameter/:value" do |env| - env.response.headers["Connection"] = "close" response = sched.find_job_boot(env) job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? @@ -61,7 +63,6 @@ module Scheduler # /~lkp/cgi-bin/gpxelinux.cgi?hostname=:hostname&mac=:mac&last_kernel=:last_kernel get "/~lkp/cgi-bin/gpxelinux.cgi" do |env| - env.response.headers["Connection"] = "close" response = sched.find_next_job_boot(env) job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? @@ -74,7 +75,6 @@ module Scheduler # - echo job_id to caller # -- job_id = "0" ? means failed post "/submit_job" do |env| - env.response.headers["Connection"] = "close" job_messages = sched.submit_job(env) job_messages.each do |job_message| @@ -86,7 +86,6 @@ module Scheduler # file download server get "/job_initrd_tmpfs/:job_id/:job_package" do |env| - env.response.headers["Connection"] = "close" job_id = env.params.url["job_id"] job_package = env.params.url["job_package"] file_path = ::File.join [Kemal.config.public_folder, job_id, job_package] @@ -103,7 +102,6 @@ module Scheduler # # curl -X PUT "http://localhost:3000/set_host_mac?hostname=wfg&mac=00-01-02-03-04-05" put "/set_host_mac" do |env| - env.response.headers["Connection"] = "close" if (client_hostname = env.params.query["hostname"]?) && (client_mac = env.params.query["mac"]?) sched.set_host_mac(client_mac, client_hostname) @@ -115,7 +113,6 @@ module Scheduler # curl -X PUT "http://localhost:3000/set_host2queues?queues=vm-2p8g.aarch64&host=vm-2p8g.a…" put "/set_host2queues" do |env| - env.response.headers["Connection"] = "close" if (client_queues = env.params.query["queues"]?) && (client_host = env.params.query["host"]?) sched.set_host2queues(client_host, client_queues) @@ -127,7 +124,6 @@ module Scheduler # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| - env.response.headers["Connection"] = "close" if client_mac = env.params.query["mac"]? sched.del_host_mac(client_mac) @@ -139,7 +135,6 @@ module Scheduler # curl -X PUT "http://localhost:3000/del_host2queues?host=vm-2p8g.aarch64" put "/del_host2queues" do |env| - env.response.headers["Connection"] = "close" if client_host = env.params.query["host"]? sched.del_host2queues(client_host) @@ -155,7 +150,6 @@ module Scheduler # ?job_file=/lkp/scheduled/job.yaml&job_state=post_run&job_id=10 # ?job_file=/lkp/scheduled/job.yaml&loadavg=0.28 0.82 0.49 1/105 3389&start_time=1587725398&end_time=1587725698&job_id=10 get "/~lkp/cgi-bin/lkp-jobfile-append-var" do |env| - env.response.headers["Connection"] = "close" puts sched.update_job_parameter(env) "Done" @@ -179,7 +173,6 @@ module Scheduler # response: get "server ip" from cluster state, # return "server=<server ip>". get "/~lkp/cgi-bin/lkp-cluster-sync" do |env| - env.response.headers["Connection"] = "close" response = sched.request_cluster_state(env) debug_message(env, response) @@ -191,7 +184,6 @@ module Scheduler # /~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job.yaml&job_id=40 # curl "http://localhost:3000/~lkp/cgi-bin/lkp-post-run?job_file=/lkp/scheduled/job…" get "/~lkp/cgi-bin/lkp-post-run" do |env| - env.response.headers["Connection"] = "close" # get job_id from request job_id = env.params.query["job_id"]? if job_id @@ -202,14 +194,12 @@ module Scheduler end get "/~lkp/cgi-bin/lkp-wtmp" do |env| - env.response.headers["Connection"] = "close" puts sched.update_tbox_wtmp(env) "Done" end get "/~lkp/cgi-bin/report_ssh_port" do |env| - env.response.headers["Connection"] = "close" testbox = env.params.query["tbox_name"] ssh_port = env.params.query["ssh_port"].to_s job_id = env.params.query["job_id"].to_s diff --git a/src/taskqueue/taskqueue.cr b/src/taskqueue/taskqueue.cr index f41cfa8..0c82333 100644 --- a/src/taskqueue/taskqueue.cr +++ b/src/taskqueue/taskqueue.cr @@ -10,6 +10,10 @@ require "./queue" class TaskQueue VERSION = "0.0.2" + before_all do |env| + env.response.headers["Connection"] = "close" + end + # l2pH: limit 2 / hours # l100pms: limit 100 / ms @@rate_limiter = RateLimiter(String).new @@ -42,7 +46,6 @@ class TaskQueue # # response: TaskQueue(a)v0.0.1 is alive. get "/" do |env| - env.response.headers["Connection"] = "close" response = "TaskQueue@v#{VERSION} is alive." debug_message(env, response, Time.utc) @@ -61,7 +64,6 @@ class TaskQueue # 400 "Missing parameter <queue>" # 400 "Missing http body" post "/add" do |env| - env.response.headers["Connection"] = "close" response = queue_respond_add(env) debug_message(env, response, Time.utc) response if env.response.status_code == 200 @@ -75,7 +77,6 @@ class TaskQueue # 201 ## when there has no task in queue (scheduler/$tbox_group) # 400 "Missing parameter <queue>" put "/consume" do |env| - env.response.headers["Connection"] = "close" response = queue_respond_consume(env) debug_message(env, response, Time.utc) response if env.response.status_code == 200 @@ -89,7 +90,6 @@ class TaskQueue # 400 "Missing parameter <from|to|id>" # 409 "Can not find id <$id> in queue <scheduler/$tbox_group>" put "/hand_over" do |env| - env.response.headers["Connection"] = "close" response = queue_respond_hand_over(env) debug_message(env, response, Time.utc) nil @@ -103,7 +103,6 @@ class TaskQueue # 400 "Missing parameter <queue|id>" # 409 "Can not find id <$id> in queue <scheduler/$tbox_group>" put "/delete" do |env| - env.response.headers["Connection"] = "close" response = queue_respond_delete(env) debug_message(env, response, Time.utc) nil @@ -119,7 +118,6 @@ class TaskQueue # 400 "Missing parameter <queue>" # 413 "Query results too large keys" get "/keys" do |env| - env.response.headers["Connection"] = "close" response = queue_respond_keys(env) # debug_message(env, response, Time.utc) # maybe too large response.to_json unless env.response.status_code == 201 -- 2.23.0

1 0

[PATCH v4 lkp-tests 2/2] tests/iozone: add invalid test option identification
by Lu Kaiyi 25 Nov '20

25 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/iozone b/tests/iozone index bafc0087..5186766f 100755 --- a/tests/iozone +++ b/tests/iozone @@ -47,6 +47,9 @@ if [ -n "$block_size" ]; then ;; "preadv") args+=" -i 12" ;; + *) echo "invalid test option, please check again!" + exit 1 + ;; esac done else -- 2.23.0

4 3

[PATCH lkp-tests] tests/iozone: refactor format optimization
by Lu Kaiyi 25 Nov '20

25 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/iozone b/tests/iozone index 5186766f..923d8331 100755 --- a/tests/iozone +++ b/tests/iozone @@ -47,8 +47,7 @@ if [ -n "$block_size" ]; then ;; "preadv") args+=" -i 12" ;; - *) echo "invalid test option, please check again!" - exit 1 + *) echo "invalid test option, please check again!" && exit 1 ;; esac done -- 2.23.0

2 1

[PATCH v2 lkp-tests] setup/sysctl: add system parameters
by Zhang Yu 25 Nov '20

25 Nov '20

[why] When testing need to setup system parameters. [how] Add parameters to setup/sysctl. Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- setup/sysctl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/setup/sysctl b/setup/sysctl index 81374939..1085df49 100755 --- a/setup/sysctl +++ b/setup/sysctl @@ -202,6 +202,9 @@ # - sc_netdev_max_backlog # - sc_rmem_max # - sc_wmem_max +# - sc_tcp_rmem +# - sc_tcp_wmem +# - sc_sched_features . $LKP_SRC/lib/reproduce-log.sh @@ -405,8 +408,11 @@ [ -n "$sc_ipv4_tcp_max_tw_buckets" ] && log_eval "echo '$sc_ipv4_tcp_max_tw_buckets' > '/proc/sys/net/ipv4/tcp_max_tw_buckets'" [ -n "$sc_ipv4_tcp_max_syn_backlog" ] && log_eval "echo '$sc_ipv4_tcp_max_syn_backlog' > '/proc/sys/net/ipv4/tcp_max_syn_backlog'" [ -n "$sc_ipv4_ip_local_port_range" ] && log_eval "echo '$sc_ipv4_ip_local_port_range' > '/proc/sys/net/ipv4/ip_local_port_range'" +[ -n "$sc_tcp_rmem" ] && log_eval "echo '$sc_tcp_rmem' > '/proc/sys/net/ipv4/tcp_rmem'" +[ -n "$sc_tcp_wmem" ] && log_eval "echo '$sc_tcp_wmem' > '/proc/sys/net/ipv4/tcp_wmem'" [ -n "$sc_netdev_max_backlog" ] && log_eval "echo '$sc_netdev_max_backlog' > '/proc/sys/net/core/netdev_max_backlog'" [ -n "$sc_rmem_max" ] && log_eval "echo '$sc_rmem_max' > '/proc/sys/net/core/rmem_max'" [ -n "$sc_wmem_max" ] && log_eval "echo '$sc_wmem_max' > '/proc/sys/net/core/wmem_max'" +[ -n "$sc_sched_features" ] && log_eval "echo '$sc_sched_features' > '/sys/kernel/debug/sched_features'" exit 0 -- 2.23.0

1 0

[PATCH lkp-tests] daemon/sshd: fix pub_key to my_ssh_pubkey
by Zhang Yale 25 Nov '20

25 Nov '20

[why] - "my_ssh_pubkey" field is a set of all pub_keys that had been submitted by users. - We can save all the pub_keys submitted by users in the ES database and set the login mode to password-free. - Therefore, users can sometimes borrow a machine to test without providing a pub_key. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- daemon/sshd | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index a3fdf954..7919f81d 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,5 +1,5 @@ #!/bin/sh -# pub_key +# my_ssh_pubkey # my_email # sshr_port # sshr_port_len @@ -14,10 +14,10 @@ sshr_ip=$LKP_SERVER run_ssh() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh - echo "$pub_key" > /root/.ssh/authorized_keys + echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys systemctl start sshd } @@ -54,7 +54,7 @@ Subject: [NOTIFY Compass-ci] Applying $testbox failed Dear $my_username: Sorry to inform you that your application failed. - You may need to wait a moment, or check whether your pub_key exists. + You may need to wait a moment, or check whether your my_ssh_pubkey exists. Regards Compass-Ci @@ -73,11 +73,11 @@ compose_email() set_port() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%${sshr_port_len}+${sshr_port_base})) + port=$(($(date +%s%N)%$sshr_port_len+$sshr_port_base)) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return -- 2.23.0

3 4

[PATCH v3 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will give each type of cases for different input and output directories, and support merging from external file. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/atomic.yaml | 4 ++ .../merge_yaml/atomic/atomic-defaults.yaml | 72 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 71 ++++++++++++++++++ spec/submit_spec.rb | 4 ++ 5 files changed, 152 insertions(+) create mode 100644 spec/submit/merge_yaml/atomic.yaml create mode 100644 spec/submit/merge_yaml/atomic/atomic-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/atomic.yaml b/spec/submit/merge_yaml/atomic.yaml new file mode 100644 index 00000000..d0927923 --- /dev/null +++ b/spec/submit/merge_yaml/atomic.yaml @@ -0,0 +1,4 @@ +suite: atomic +category: benchmark + +atomic: diff --git a/spec/submit/merge_yaml/atomic/atomic-defaults.yaml b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml new file mode 100644 index 00000000..920ebec2 --- /dev/null +++ b/spec/submit/merge_yaml/atomic/atomic-defaults.yaml @@ -0,0 +1,72 @@ +--- +suite: atomic +category: benchmark +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: {} + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..30019c6c --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: atomic.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..fc9e073a --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..b56a860c 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH compass-ci] mail-robot: optimize check url for gitee.com
by Luan Shengde 25 Nov '20

25 Nov '20

[why] use 'git clone repo_url' will clone the whole repo it will need more time and network bandwidth [how] use 'git clone --bare repo_url' instead Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/gitee-commit-url-check.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/mail-robot/lib/gitee-commit-url-check.rb b/container/mail-robot/lib/gitee-commit-url-check.rb index 72c948f..7cac0d5 100755 --- a/container/mail-robot/lib/gitee-commit-url-check.rb +++ b/container/mail-robot/lib/gitee-commit-url-check.rb @@ -21,12 +21,12 @@ class GiteeCommitUrlCheck end def gitee_commit_check - repo_dir = @url.split('/')[-3] repo_url = [@base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] commit_id = @url.split('/')[-1] Dir.chdir '/tmp' - %x(/usr/bin/git clone #{repo_url} #{repo_dir}) + %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) email_index = %x(/usr/bin/git -C #{repo_dir} show #{commit_id}).index @my_info['my_email'] -- 2.23.0

1 0

[PATCH compass-ci] doc/job/: add os_fields.md
by Zhang Yu 25 Nov '20

25 Nov '20

Add a document for introducing the os, os_version, os_arch fields Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- doc/job/os_fields.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/job/os_fields.md diff --git a/doc/job/os_fields.md b/doc/job/os_fields.md new file mode 100644 index 0000000..1e6baae --- /dev/null +++ b/doc/job/os_fields.md @@ -0,0 +1,15 @@ +# os, os_version, os_arch + +Meaning: +- Each test tells which os it needs, and the os_version, os_arch is also need to be specified, then the machine will run an os according to those three fields. +- Os, os_version, os_arch is the key for users to specify the os-related parameters. +- If os, os_version, os_arch is not given by users, it will use the default openeuler os-related parameters. +- Here are the os, os_version, os_arch that we support: + os os_version os_arch + openeuler 20.03, 1.0 aarch64, x86_64 + debian sid, 10 aarch64, x86_64 + centos 7.6, 7.8 aarch64, x86_64 + ... + +Usage example: +- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 -- 2.23.0

4 6

[PATCH compass-ci] block_helper.cr: change if block into one line
by Xu Xijian 25 Nov '20

25 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/block_helper.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/lib/block_helper.cr b/src/lib/block_helper.cr index 37ad367..596ef7e 100644 --- a/src/lib/block_helper.cr +++ b/src/lib/block_helper.cr @@ -28,9 +28,7 @@ class BlockHelper @block_helper[uuid] = fiber end - if yield == true - spawn fiber.run - end + spawn fiber.run if yield == true until fiber.dead? Fiber.yield -- 2.23.0

2 2

[PATCH compass-ci] lib/compare_matrixes.rb: fix rubocop offenses
by Zhang Yu 25 Nov '20

25 Nov '20

Inspecting 1 file Offenses: compare_matrixes.rb:469:5: C: Layout/EmptyLineAfterGuardClause: Add empty line after guard clause. next if values['average'].size < 2 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/compare_matrixes.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 0b49e59..1550435 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -467,6 +467,7 @@ end def assign_metric_change(metrics_values) metrics_values.each do |metric, values| next if values['average'].size < 2 + metrics_values[metric]['change'] = {} dimension_list = values['average'].keys -- 2.23.0

1 0

[PATCH v3 lkp-tests] sbin/submit: submit merge_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will add more spec cases, so it should give each type of cases for different input and output directories Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 4 ++ .../template/template-defaults.yaml | 71 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 71 +++++++++++++++++++ spec/submit_spec.rb | 4 ++ 5 files changed, 151 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/template/template-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..6d6fb37c --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,4 @@ +suite: test +category: benchmark + +test: diff --git a/spec/submit/merge_yaml/template/template-defaults.yaml b/spec/submit/merge_yaml/template/template-defaults.yaml new file mode 100644 index 00000000..f1d13d04 --- /dev/null +++ b/spec/submit/merge_yaml/template/template-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +test: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..f1d13d04 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,71 @@ +--- +suite: test +category: benchmark +test: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kmsg: +boot-time: +uptime: +iostat: +heartbeat: +vmstat: +numa-numastat: +numa-vmstat: +numa-meminfo: +proc-vmstat: +proc-stat: +meminfo: +slabinfo: +interrupts: +lock_stat: +latency_stats: +softirqs: +bdi_dev_mapping: +diskstats: +nfsstat: +cpuidle: +cpufreq-stats: +sched_debug: +perf-stat: +mpstat: +perf-profile: +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + cpufreq_governor: {} +monitors: + kmsg: + boot-time: + uptime: + iostat: + heartbeat: + vmstat: + numa-numastat: + numa-vmstat: + numa-meminfo: + proc-vmstat: + proc-stat: + meminfo: + slabinfo: + interrupts: + lock_stat: + latency_stats: + softirqs: + bdi_dev_mapping: + diskstats: + nfsstat: + cpuidle: + cpufreq-stats: + sched_debug: + perf-stat: + mpstat: + perf-profile: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..b56a860c 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH v3 lkp-tests] daemon/sshd: fix pub_key to my_ssh_pubkey
by Zhang Yale 25 Nov '20

25 Nov '20

[why] 1. "my_ssh_pubkey" field is a collection of all pub_keys submitted by users. 2. We can save all the pub_keys submitted by users in the ES database and password-free login. 3. Therefore, users can sometimes borrow a machine to test without providing a pub_key. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- daemon/sshd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index a3fdf954..c7d7b60c 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,5 +1,5 @@ #!/bin/sh -# pub_key +# my_ssh_pubkey # my_email # sshr_port # sshr_port_len @@ -14,10 +14,10 @@ sshr_ip=$LKP_SERVER run_ssh() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh - echo "$pub_key" > /root/.ssh/authorized_keys + echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys systemctl start sshd } @@ -73,11 +73,11 @@ compose_email() set_port() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%${sshr_port_len}+${sshr_port_base})) + port=$(($(date +%s%N)%$sshr_port_len+$sshr_port_base)) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return -- 2.23.0

2 2

[PATCH v2 lkp-tests] lkp-tests/*: update AUR Helper for archlinux
by Wang Yong 25 Nov '20

25 Nov '20

yaourt is unmaintained and yay also support native pacman operation, so only minor changes are needed Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 15 +++++++++++++++ distro/installer/archlinux | 15 +++++++++------ lib/install.sh | 2 +- 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/distro/archlinux b/distro/archlinux index 408c5fa23..3f8d62ebc 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -109,3 +109,18 @@ pack_benchmark() ) done } + +add_aur_helper() +{ + echo "lkp ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers + + pacman -Q yay 2>/dev/null && return + + local user=lkp + local dir=/home/${user} + + mkdir -p ${dir} && chown ${user}:${user} ${dir} + pacman -S --needed --noconfirm git base-devel 2>/dev/null + su - ${user} -c "git clone https://aur.archlinux.org/yay.git" + su - ${user} -c "cd yay && makepkg -si --needed --noconfirm" +} diff --git a/distro/installer/archlinux b/distro/installer/archlinux index 27cdc63cf..af98e466f 100755 --- a/distro/installer/archlinux +++ b/distro/installer/archlinux @@ -2,12 +2,15 @@ # Archlinux package installation -# yaourt is a wrapper for pacman that adds automated access -# to the AUR. -# See: https://wiki.archlinux.org/index.php/Yaourt +# Yet Another Yogurt - An AUR Helper Written in Go +# Yay is based on the design of yaourt, apacman and pacaur +# See: https://github.com/Jguer/yay -# yaourt not support root to build package +# Yay not support root to build package # Use regular user to do this -regular_user=`grep 1000 /etc/passwd | awk -F':' '{ print $1 }'` +. $LKP_SRC/distro/${DISTRO} +add_aur_helper -sudo -u $regular_user yaourt -S --noconfirm $* +regular_user=`grep 1090 /etc/passwd | awk -F':' '{ print $1 }'` + +sudo -u $regular_user yay -S --needed --noconfirm $* diff --git a/lib/install.sh b/lib/install.sh index 881ade9f4..bf7bf21d7 100755 --- a/lib/install.sh +++ b/lib/install.sh @@ -17,7 +17,7 @@ sync_distro_sources() yum update fi ;; redhat) yum update ;; - archlinux) yaourt -Sy ;; + archlinux) yay -Sy ;; opensuse) zypper update ;; oracle) yum update ;; -- 2.23.0

1 0

[PATCH lkp-tests] sbin/submit: submit separate_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will add more spec cases, so it should give each type of cases for different input and output directories Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/separate_yaml/ftq.yaml | 22 +++++++++++++++ .../ftq/ftq-1000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-10000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-20-2-6000ss-cache.yaml | 28 +++++++++++++++++++ spec/submit_spec.rb | 4 +++ 5 files changed, 110 insertions(+) create mode 100644 spec/submit/separate_yaml/ftq.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml diff --git a/spec/submit/separate_yaml/ftq.yaml b/spec/submit/separate_yaml/ftq.yaml new file mode 100644 index 00000000..8564d443 --- /dev/null +++ b/spec/submit/separate_yaml/ftq.yaml @@ -0,0 +1,22 @@ +suite: ftq +category: noise-benchmark + +nr_task: 100% + +samples: 100000ss + +ftq: + test: add + freq: 10000 + +--- +ftq: + test: add + freq: 1000 + +--- + +samples: 6000ss +ftq: + test: cache + freq: 20 diff --git a/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml new file mode 100644 index 00000000..855e7210 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 1000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 1000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml new file mode 100644 index 00000000..78725424 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 10000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 10000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml new file mode 100644 index 00000000..40edfcb2 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 6000ss +ftq: + test: cache + freq: 20 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 6000ss + freq: 20 + test: cache + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ce6fed25 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'separate yaml spec' do + submit_job('separate_yaml') + end end -- 2.23.0

2 1