- Compass-ci - mailweb.openeuler.org

[PATCH v3 lkp-tests] submit: fix the processes of option: --my-queue
by Ren Wen 26 Nov '20

26 Nov '20

If use options '--my-queue' and '-o' at the same time, the output job file will not contain the value of the covered field 'queue'. So pass the 'jobs' not 'job_hash'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index be588bed..23518eb0 100755 --- a/sbin/submit +++ b/sbin/submit @@ -112,6 +112,7 @@ ARGV.each do |jobfile| jobs[:expand_params] = true jobs['testbox'] = opt_set_key_value['testbox'] if opt_set_key_value['testbox'] jobs['tbox_group'] = tbox_group(jobs['testbox']) if jobs.include?('testbox') + add_my_queue(jobs) if opt_my_queue jobs['node_roles'] ||= 'server client' if jobs['cluster'] jobs.each_jobs do |job| raise 'Please configure SCHED_HOST' unless job['SCHED_HOST'] @@ -144,7 +145,6 @@ ARGV.each do |jobfile| job_hash_list << job_hash job_hash = job_hash.merge(sh_hash) - add_my_queue(job_hash) if opt_my_queue # save job to yaml if opt_output_dir prefix = File.join(opt_output_dir, File.basename(jobfile, '.yaml')) -- 2.23.0

1 0

[PATCH v3 compass-ci] doc/job: add category.md
by Zhang Yu 26 Nov '20

26 Nov '20

Add a document for introducing the category field Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- doc/job/category.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 doc/job/category.md diff --git a/doc/job/category.md b/doc/job/category.md new file mode 100644 index 0000000..6425318 --- /dev/null +++ b/doc/job/category.md @@ -0,0 +1,21 @@ +# category + +Meaning: +- Each test has its category, the category field is a test type of test cases. +- When you write a job yaml, you can choose one of benchmark, functional, noise-benchmark + to assign the value of category. +- Here are some differences about different category values: + 1. If category is benchmark, it will monitor some system information: + kmsg: print the kernel startup information + boot-time: start time diagnosis item + uptime: display how long the system has been running + iostat: print the time when the data is collected + heartbeat: provide the heartbeat monitors service + ... + other monitors you can see them in $LKP_SRC/include/category/benchmark + 2. If category is functional, it will monitor some system information: + kmsg: print the kernel startup information + heartbeat: provide the heartbeat monitors service + meminfo: solid memory usage status or vm usage status + ... + you can see the monitors in $LKP_SRC/include/category/functional -- 2.23.0

1 0

[PATCH v2 lkp-tests] submit: fix the processes of option: --my-queue
by Ren Wen 26 Nov '20

26 Nov '20

If use options '--my-queue' and '-o' at the same time, the output job file will not contain the value of the covered field 'queue'. So pass the 'jobs' not 'job_hash'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index be588bed..9510d53d 100755 --- a/sbin/submit +++ b/sbin/submit @@ -108,6 +108,7 @@ ARGV.each do |jobfile| jobs = Job2sh.new jobs.cmdline_defaults = opt_cmdline_defaults jobs.overrides = opt_set_key_value + add_my_queue(jobs) if opt_my_queue jobs.load(jobfile, true) || next jobs[:expand_params] = true jobs['testbox'] = opt_set_key_value['testbox'] if opt_set_key_value['testbox'] @@ -144,7 +145,6 @@ ARGV.each do |jobfile| job_hash_list << job_hash job_hash = job_hash.merge(sh_hash) - add_my_queue(job_hash) if opt_my_queue # save job to yaml if opt_output_dir prefix = File.join(opt_output_dir, File.basename(jobfile, '.yaml')) -- 2.23.0

2 2

[PATCH compass-ci] doc/job/: add lkp_initrd_user.md
by Zhang Yale 26 Nov '20

26 Nov '20

Add a document for introducing the lkp_initrd_user field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/job/lkp_initrd_user.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/job/lkp_initrd_user.md diff --git a/doc/job/lkp_initrd_user.md b/doc/job/lkp_initrd_user.md new file mode 100644 index 0000000..b4a0982 --- /dev/null +++ b/doc/job/lkp_initrd_user.md @@ -0,0 +1,10 @@ +# lkp_inird_user + +Meaning: +- "lkp_initrd_user" is one of the init_field, the default value is "latest". +- User can set lkp_initrd_user to $USER or the default value in "/home/$USER/.config/compass-ci/default/$USER.yaml. +- If the user's lkp_tests code is changed, build & run the lkp_initrd container which in "${CCI_SRC}/container/lkp-initrd" + to pack a new lkp-${os_arch}.cgz. +- The new lkp-${os_arch}.cgz file is generated in the "${SRV_INITRD}/lkp/${lkp_initrd_user}/lkp-${os_arch}.cgz". +- When user submit one job, qemu-pxe or other testbox start mount the <"lkp"> initrd to consume the user submited <"job"> + "initrd http://${INITRD_HTTP_HOST}:${INITRD_HTTP_PORT}/initrd/lkp/${lkp_initrd_user}/lkp-${os_arch}.cgz". -- 2.23.0

3 5

[PATCH compass-ci 1/5] openeuler: configure files about build image with one comman
by Wang Chenglong 26 Nov '20

26 Nov '20

[Contains] openeule-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

3 4

[PATCH v1 lkp-tests] spec/submit: add job yaml for multi-args test
by Bai Jing 26 Nov '20

26 Nov '20

[test cmd] respec $LKP_SRC/spec/submit_spec.rb [output] lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml Signed-off-by: Bai Jing <799286817(a)qq.com> --- spec/submit/matrices-args/unixbench-spec.yaml | 8 ++++++ ...unixbench-spec-1-300-whetstone-double.yaml | 25 +++++++++++++++++++ .../unixbench-spec-3-300-shell1.yaml | 25 +++++++++++++++++++ spec/submit_spec.rb | 4 +++ 4 files changed, 62 insertions(+) create mode 100644 spec/submit/matrices-args/unixbench-spec.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml diff --git a/spec/submit/matrices-args/unixbench-spec.yaml b/spec/submit/matrices-args/unixbench-spec.yaml new file mode 100644 index 00000000..a763927b --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec.yaml @@ -0,0 +1,8 @@ +suite: unixbench + +runtime: 300s + +nr_task test: + - 1 whetstone-double + - 3 shell1 +unixbench: diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml new file mode 100644 index 00000000..16af43d2 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml @@ -0,0 +1,25 @@ +--- +suite: unixbench +runtime: 300 +unixbench: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '1' +test: whetstone-double +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: whetstone-double + nr_task: '1' + cpufreq_governor: +monitors: {} diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml new file mode 100644 index 00000000..5c0526a1 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml @@ -0,0 +1,25 @@ +--- +suite: unixbench +runtime: 300 +unixbench: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '3' +test: shell1 +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: shell1 + nr_task: '3' + cpufreq_governor: +monitors: {} diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..3a9b0865 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'link matrices-args' do + submit_job('matrices-args') + end end -- 2.23.0

2 2

[PATCH lkp-tests] tats/crystal: add hackbench-git.cr
by Zhang Yale 26 Nov '20

26 Nov '20

Make ruby2crystal convert Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/crystal/hackbench-git.cr | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 stats/crystal/hackbench-git.cr diff --git a/stats/crystal/hackbench-git.cr b/stats/crystal/hackbench-git.cr new file mode 100644 index 00000000..214dcd02 --- /dev/null +++ b/stats/crystal/hackbench-git.cr @@ -0,0 +1,23 @@ +#!/usr/bin/env crystal + +# --------socket thread num=20-------- +# 2020-11-10 19:51:31 /lkp/benchmarks/hackbench-git/hackbench/hackbench 20 thread 1000 +# Running with 20*40 (== 800) tasks. +# Time: 0.495 +# ... + +time = [] of Float64 + +while (line = gets) + case line + when /^-+(\w+ \w+ \w+)/ + args = $1.tr(" ", "_") + when /^Running with .* $== (\d+)$ tasks/ + tasks = $1.to_i + when /^Time:/ + seconds = line.split[1] + time << seconds.to_f + end +end + +puts "#{args}_#{tasks}: #{time.sum / time.size}" -- 2.23.0

2 2

[PATCH compass-ci 4/5] openeuler: build openeuler initramfs image with one command
by Wang Chenglong 26 Nov '20

26 Nov '20

[Why] Use the tool to create a new openeuler-$version initramfs image with one command. and we also can customize the pre-installed software by configuring ./packages-to-install. [How] We create a link button for the user to use. $CCI_SRC/rootfs/initramfs/openeuler/aarch64/${os_version}/build Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/run | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100755 container/osimage/openeuler/run diff --git a/container/osimage/openeuler/run b/container/osimage/openeuler/run new file mode 100755 index 0000000..c92c59f --- /dev/null +++ b/container/osimage/openeuler/run @@ -0,0 +1,27 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ${CCI_SRC}/container/osimage/centos/lib +. ${CCI_SRC}/container/defconfig.sh + +export_root_passwd +DIR=$(dirname $(realpath $0)) +cmd=( + docker run + --name init_docker + -v $DIR/create-image:/root/bin/create-image + -v $DIR/packages-to-install:/tmp/packages-to-install + -v $DIR/files-to-exclude:/tmp/files-to-exclude + -v $DIR/${image}.repo:/etc/yum.repos.d/openEuler.repo + -e ROOT_PASSWD=$ROOT_PASSWD + -e IMAGE_NAME=$IMAGE_NAME + $image + /root/bin/create-image +) + +pull_docker_image +"${cmd[@]}" +docker_cp_image +docker_rm init_docker &> /dev/null +echo "build finished" -- 2.23.0

3 6

[PATCH compass-ci 2/2] mail-robot: add sched info
by Luan Shengde 26 Nov '20

26 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/mail-robot/start b/container/mail-robot/start index 5827e18..1c55dd3 100755 --- a/container/mail-robot/start +++ b/container/mail-robot/start @@ -12,6 +12,8 @@ names = Set.new %w[ JUMPER_HOST JUMPER_PORT SEND_MAIL_PORT + SCHED_HTTPS_HOST + SCHED_HTTPS_PORT ] defaults = relevant_defaults(names) -- 2.23.0

1 0

[PATCH compass-ci 1/2] mail-robot: add sched info in email message
by Luan Shengde 26 Nov '20

26 Nov '20

for user to add the sched info in the default yaml file Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index b793232..4754c4c 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -25,6 +25,8 @@ def build_apply_account_email(my_info) my_email: #{my_info['my_email']} my_name: #{my_info['my_name']} my_uuid: #{my_info['my_uuid']} + SCHED_HTTPS_HOST: #{ENV['SCHED_HTTPS_HOST']} + SCHED_HTTPS_PORT: #{ENV['SCHED_HTTPS_PORT']} EOF 2) download lkp-tests and dependencies -- 2.23.0

1 0