- Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] sbin/submit: submit separate_yaml/*.yaml
by Hu Xuejiao 25 Nov '20

25 Nov '20

[why] We will user 'rspec spec/submit_spec.rb' to separate job.yaml, it contains several yaml documents separated by "---" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/separate_yaml/ftq.yaml | 20 +++++++++++++ .../ftq/ftq-1000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-10000-2-100000ss-add.yaml | 28 +++++++++++++++++++ .../ftq/ftq-20-2-6000ss-cache.yaml | 28 +++++++++++++++++++ spec/submit_spec.rb | 4 +++ 5 files changed, 108 insertions(+) create mode 100644 spec/submit/separate_yaml/ftq.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml create mode 100644 spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml diff --git a/spec/submit/separate_yaml/ftq.yaml b/spec/submit/separate_yaml/ftq.yaml new file mode 100644 index 00000000..e54adc8b --- /dev/null +++ b/spec/submit/separate_yaml/ftq.yaml @@ -0,0 +1,20 @@ +suite: ftq +category: noise-benchmark + +nr_task: 100% +samples: 100000ss + +ftq: + test: add + freq: 10000 + +--- +ftq: + test: add + freq: 1000 + +--- +samples: 6000ss +ftq: + test: cache + freq: 20 diff --git a/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml new file mode 100644 index 00000000..855e7210 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-1000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 1000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 1000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml new file mode 100644 index 00000000..78725424 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-10000-2-100000ss-add.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 100000ss +ftq: + test: add + freq: 10000 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 100000ss + freq: 10000 + test: add + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml new file mode 100644 index 00000000..40edfcb2 --- /dev/null +++ b/spec/submit/separate_yaml/ftq/ftq-20-2-6000ss-cache.yaml @@ -0,0 +1,28 @@ +--- +suite: ftq +category: noise-benchmark +nr_task: 2 +samples: 6000ss +ftq: + test: cache + freq: 20 +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + ftq: + nr_task: 2 + samples: 6000ss + freq: 20 + test: cache + cpufreq_governor: {} +monitors: {} diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ce6fed25 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'separate yaml spec' do + submit_job('separate_yaml') + end end -- 2.23.0

2 1

[PATCH lkp-tests 1/2] daemon/multi-qemu: fix default parameter format
by Xiao Shenwei 25 Nov '20

25 Nov '20

before: : ${queues:="$tbox_group.$HOSTNAME, $tbox_group.$(arch)"} now: : ${queues:="$tbox_group.$HOSTNAME,$tbox_group.$(arch)"} this parameter will be used by ruby script for ruby command line parameter, if use space like: multi-qemu -q vm-2p8g.taishan200-2280-2s48p-256g--a1, vm-2p8g.aarch64 the vm-2p8g.aarch64 doesn't work correct usage: multi-qemu -q vm-2p8g.taishan200-2280-2s48p-256g--a1,vm-2p8g.aarch64 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- daemon/multi-qemu | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/daemon/multi-qemu b/daemon/multi-qemu index 7cc625f4..aa2348b4 100755 --- a/daemon/multi-qemu +++ b/daemon/multi-qemu @@ -6,7 +6,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${queues:="$tbox_group.$HOSTNAME, $tbox_group.$(arch)"} +: ${queues:="$tbox_group.$HOSTNAME,$tbox_group.$(arch)"} + multi_qemu() { export CCI_SRC=/c/compass-ci -- 2.23.0

2 1

[PATCH compass-ci] job.cr: delete 2 variables from METHOD_KEYS
by Xu Xijian 25 Nov '20

25 Nov '20

Since linux_modules_initrd and linux_headers_initrd are only used in initramfs, they don't need to be initialized in all jobs, so delete them from METHOD_KEYS. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index a9923cc..378fd00 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -97,8 +97,6 @@ class Job docker_image kernel_version linux_vmlinuz_path - linux_modules_initrd - linux_headers_initrd ) macro method_missing(call) @@ -390,9 +388,9 @@ class Job temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{linux_modules_initrd}")}" + "#{JobHelper.service_path(self["linux_modules_initrd"])}" temp_initrds << "#{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{linux_headers_initrd}")}" + "#{JobHelper.service_path(self["linux_headers_initrd"])}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

1 0

[PATCH lkp-tests 2/2] lib/bootstrap.sh: fix redirect_stdout_stderr for docker
by Xiao Shenwei 25 Nov '20

25 Nov '20

refer-to: compass-ci: c7c4ba94b3ef00a3f5c91200f85ea2d924726132 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/bootstrap.sh | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index ce34a632..46e77e72 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -270,11 +270,7 @@ announce_bootup() redirect_stdout_stderr() { - if is_docker; then - exec > /tmp/stdout - exec 2> /tmp/stderr - return - fi + is_docker && return [ -c /dev/kmsg ] || return has_cmd tail || return -- 2.23.0

1 0

[PATCH compass-ci] provider/docker: forward correct and error logs to docker log collector
by Xiao Shenwei 25 Nov '20

25 Nov '20

[why] $LKP_SRC/lib/bootstrap.sh:271 redirect_stdout_stderr() result file: output stderr stdout these file will based on /tmp/stdout and /tmp/stderr the function will execute: exec > /tmp/stdout exec 2> /tmp/stderr docker log based on /dev/stdout and /dev/stderr, so docker log can't collect the job's output. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 63cb605..3d04406 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -2,4 +2,11 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap +# forward correct and error logs to docker log collector +touch /tmp/stderr /tmp/strout +tail -f /tmp/stdout > /dev/stdout & +echo $! >> /tmp/pid-tail-global +tail -f /tmp/stderr > /dev/stderr & +echo $! >> /tmp/pid-tail-global + +sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap 1> /tmp/stdout 2> /tmp/stderr -- 2.23.0

1 0

[PATCH v4 lkp-tests] daemon/sshd: change pub_key to my_ssh_pubkey
by Zhang Yale 25 Nov '20

25 Nov '20

[why] 1. "my_ssh_pubkey" field is a collection of all pub_keys submitted by users. 2. We can save all the pub_keys submitted by users in the ES database and password-free login. 3. Therefore, users can sometimes borrow a machine to test without providing a pub_key. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- daemon/sshd | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index a3fdf954..c7d7b60c 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,5 +1,5 @@ #!/bin/sh -# pub_key +# my_ssh_pubkey # my_email # sshr_port # sshr_port_len @@ -14,10 +14,10 @@ sshr_ip=$LKP_SERVER run_ssh() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return umask 0077 mkdir -p /root/.ssh - echo "$pub_key" > /root/.ssh/authorized_keys + echo "$my_ssh_pubkey" > /root/.ssh/authorized_keys systemctl start sshd } @@ -73,11 +73,11 @@ compose_email() set_port() { - [ -n "$pub_key" ] || return + [ -n "$my_ssh_pubkey" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%${sshr_port_len}+${sshr_port_base})) + port=$(($(date +%s%N)%$sshr_port_len+$sshr_port_base)) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return -- 2.23.0

1 0

[PATCH compass-ci] block_helper.cr: refactor if block into one line
by Xu Xijian 25 Nov '20

25 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/block_helper.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/lib/block_helper.cr b/src/lib/block_helper.cr index 37ad367..596ef7e 100644 --- a/src/lib/block_helper.cr +++ b/src/lib/block_helper.cr @@ -28,9 +28,7 @@ class BlockHelper @block_helper[uuid] = fiber end - if yield == true - spawn fiber.run - end + spawn fiber.run if yield == true until fiber.dead? Fiber.yield -- 2.23.0

1 0

[PATCH lkp-tests] submit: fix the processes of option: --my-queue
by Ren Wen 25 Nov '20

25 Nov '20

If use options '--my-queue' and '-o' at the same time, the output job file will not contain the value of the covered field 'queue'. So pass the 'job' not job_hash. Signed-off-by: Ren Wen <15991987063(a)163.com> --- sbin/submit | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index be588bed..74815202 100755 --- a/sbin/submit +++ b/sbin/submit @@ -139,12 +139,13 @@ ARGV.each do |jobfile| sh_hash['job2sh']['on_fail'] = sh_on_fail end + add_my_queue(job) if opt_my_queue + # merge job info job_hash = job.to_hash job_hash_list << job_hash job_hash = job_hash.merge(sh_hash) - add_my_queue(job_hash) if opt_my_queue # save job to yaml if opt_output_dir prefix = File.join(opt_output_dir, File.basename(jobfile, '.yaml')) -- 2.23.0

2 1

[PATCH v2 lkp-tests] tests/build-pkg: update mount option
by Lin Jiaxin 25 Nov '20

25 Nov '20

[why] refer-to: compass-ci: commit 96584b45e528a4bad1f9 [how] add port=446 option to mount.cifs Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 65c83e22..29d35997 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -37,7 +37,7 @@ mount_dest() mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } -- 2.23.0

1 0

[PATCH lkp-tests] lkp-tests/build-pkg: update mount option
by Lin Jiaxin 25 Nov '20

25 Nov '20

[why] refer-to: compass-ci: commit 96584b45e528a4bad1f9 [how] add port=446 option to mount.cifs Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 65c83e22..29d35997 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -37,7 +37,7 @@ mount_dest() mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } -- 2.23.0

1 0