- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] sched: fix cast from Nil to Hash(k, v) failed
by Cao Xueliang 26 Nov '20

26 Nov '20

The program {} => Nil in the commit 4301d1ece960dd0eff1f2e10c2d967e0e6150643 of lkp-tests, sched need to adapt to this case. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 378fd00..d88aa40 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -460,11 +460,11 @@ class Job private def get_program_params program_params = Hash(String, JSON::Any).new - if @hash["monitors"]? + if @hash["monitors"]? != nil program_params.merge!(@hash["monitors"].as_h) end - if @hash["pp"]? + if @hash["pp"]? != nil program_params.merge!(@hash["pp"].as_h) end -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: fix when to regard the job as a cluster job
by Ren Wen 26 Nov '20

26 Nov '20

If a job has a cluster field and its' value is not "cs-localhost" or empty, will regard it as a cluster job. When running a cluster job with cluster value "localhost", its' actions depend on field "node_roles", default is "server client". Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 8838e40..2836275 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -9,10 +9,8 @@ class Sched job = Job.new(job_content, job_content["id"]?) job["commit_date"] = get_commit_date(job) - # it is not a cluster job if cluster field is empty or - # field's prefix is 'cs-localhost' cluster_file = job["cluster"] - if cluster_file.empty? || cluster_file.starts_with?("cs-localhost") + if cluster_file.empty? || cluster_file == "cs-localhost" return submit_single_job(job) else cluster_config = get_cluster_config(cluster_file, -- 2.23.0

1 0

[PATCH v5 lkp-tests] spec/submit_spec.rb: submit merge_yaml/*.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to merge job.yaml, it support merging from external file by "<<: job.yaml" Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/merge_yaml/template.yaml | 3 +++ .../template/template-defaults.yaml | 19 +++++++++++++++++++ .../merge_yaml/yaml_merge_included_files.yaml | 1 + .../yaml_merge_included_files-defaults.yaml | 19 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 5 files changed, 46 insertions(+) create mode 100644 spec/submit/merge_yaml/template.yaml create mode 100644 spec/submit/merge_yaml/template/template-defaults.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files.yaml create mode 100644 spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml diff --git a/spec/submit/merge_yaml/template.yaml b/spec/submit/merge_yaml/template.yaml new file mode 100644 index 00000000..331cb470 --- /dev/null +++ b/spec/submit/merge_yaml/template.yaml @@ -0,0 +1,3 @@ +suite: atomic + +atomic: diff --git a/spec/submit/merge_yaml/template/template-defaults.yaml b/spec/submit/merge_yaml/template/template-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/template/template-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit/merge_yaml/yaml_merge_included_files.yaml b/spec/submit/merge_yaml/yaml_merge_included_files.yaml new file mode 100644 index 00000000..c291508d --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files.yaml @@ -0,0 +1 @@ +<<: template.yaml diff --git a/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml new file mode 100644 index 00000000..42b53c13 --- /dev/null +++ b/spec/submit/merge_yaml/yaml_merge_included_files/yaml_merge_included_files-defaults.yaml @@ -0,0 +1,19 @@ +--- +suite: atomic +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d71ed891 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'merge yaml spec' do + submit_job('merge_yaml') + end end -- 2.23.0

1 0

[PATCH compass-ci] update document: apply-ssh-account.md
by Luan Shengde 26 Nov '20

26 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 44 ++++++++-------------------------------- 1 file changed, 8 insertions(+), 36 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index a4902f5..bb9ec76 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -1,35 +1,6 @@ +# apply account -# apply for jumper account - -## purpose - -The jumper account is used to login the jumper VM, where you can -- submit a job -- ssh into a testbox -- read output of the job - -## steps overview -### manually assign account -for internal user -1. run assign account tool - usage: - answerback-mail.rb [-e|--email email] [-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey] - - login command: - ssh -p port account@server_ip - -2. login authentication - - use pub key - - use password (if no pub key) - -3. submit jobs - - prepare job.yaml - - submit job - - command: - submit job.yaml - -### apply account with email -for external user +## steps 1. send apply account email - send apply account email to: compass-ci(a)qq.com attention: @@ -52,17 +23,18 @@ for external user my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… # attach your ssh pub key to the email as attachment -2. successfully applied account email - it will return an email with following information: +2. receive email from compass-ci(a)qq.com + the email contains following information: - my_email - my_name - my_uuid - SCHED_HOST - SCHED_PORT -3. environment configuration +3. local environment configuration follow steps in the email to finish the following configuration - setup default yaml ~/.config/compass-ci/default/account.yaml - - download and install lkp-tests - - prepare to submit jobs + +for more information: how to use submit + https://gitee.com/openeuler/compass-ci/blob/master/doc/compass-ci测试平台使用教程--… -- 2.23.0

1 0

[PATCH v2 lkp-tests] spec/submit_spec.rb: submit debug_yaml/job_fail.yaml
by Hu Xuejiao 26 Nov '20

26 Nov '20

We will use 'rspec spec/submit_spec.rb' to debug job.yaml, and after the task is failed, it is be waiting state. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit/debug_yaml/job_fail.yaml | 5 +++++ .../debug_yaml/job_fail/job_fail-3000.yaml | 22 +++++++++++++++++++ spec/submit_spec.rb | 4 ++++ 3 files changed, 31 insertions(+) create mode 100644 spec/submit/debug_yaml/job_fail.yaml create mode 100644 spec/submit/debug_yaml/job_fail/job_fail-3000.yaml diff --git a/spec/submit/debug_yaml/job_fail.yaml b/spec/submit/debug_yaml/job_fail.yaml new file mode 100644 index 00000000..16dcae3f --- /dev/null +++ b/spec/submit/debug_yaml/job_fail.yaml @@ -0,0 +1,5 @@ +suite: atomic + +on_fail: + sleep: 3000 +atomic: diff --git a/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml new file mode 100644 index 00000000..3d7ad9b5 --- /dev/null +++ b/spec/submit/debug_yaml/job_fail/job_fail-3000.yaml @@ -0,0 +1,22 @@ +--- +suite: atomic +on_fail: + sleep: 3000 +atomic: +lab: spec_lab +testbox: vm-2p8g--spec_submit +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +cpufreq_governor: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + sleep: 3000 + atomic: + cpufreq_governor: +monitors: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 169d7812..d9b931e2 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -48,4 +48,8 @@ describe 'submit job spec' do it 'separate yaml spec' do submit_job('separate_yaml') end + + it 'debug yaml spec' do + submit_job('debug_yaml') + end end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/5] openeuler: configure files about build image with one command
by Wang Chenglong 26 Nov '20

26 Nov '20

[Contains] openeule-${os_version}.repo: yum configuration files files-to-exclude: list of files that need to be excluded packages-to-install: list of packages that need to install README.md: descriptive documentation Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/README.md | 15 +++++++ container/osimage/openeuler/files-to-exclude | 11 +++++ .../openeuler/openeuler-20.03-lts.repo | 43 +++++++++++++++++++ .../osimage/openeuler/openeuler-20.09.repo | 43 +++++++++++++++++++ .../osimage/openeuler/packages-to-install | 10 +++++ 5 files changed, 122 insertions(+) create mode 100644 container/osimage/openeuler/README.md create mode 100644 container/osimage/openeuler/files-to-exclude create mode 100644 container/osimage/openeuler/openeuler-20.03-lts.repo create mode 100644 container/osimage/openeuler/openeuler-20.09.repo create mode 100644 container/osimage/openeuler/packages-to-install diff --git a/container/osimage/openeuler/README.md b/container/osimage/openeuler/README.md new file mode 100644 index 0000000..cea68f9 --- /dev/null +++ b/container/osimage/openeuler/README.md @@ -0,0 +1,15 @@ +# Use the tool to create a new openeuler-${os_version} initramfs image. + +Usage: + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/${os_version}/ + ./build + +Some configuration items: +./packages-to-install + If you want to pre-install the software, you can write the package names in ./packages-to-install. + +./files-to-exclude + If you want remove some unnecessary files, you can write the names in ./files-to-exclude + +$HOME/.config/compass-ci/rootfs-passwd + Set the password for the image into this file. diff --git a/container/osimage/openeuler/files-to-exclude b/container/osimage/openeuler/files-to-exclude new file mode 100644 index 0000000..3854947 --- /dev/null +++ b/container/osimage/openeuler/files-to-exclude @@ -0,0 +1,11 @@ +/.dockerenv +/lib/modules +/usr/lib/modules +/usr/share/doc +/usr/share/man +/usr/share/info +/usr/share/i18n +/usr/share/locale +/usr/share/terminfo +/tmp/files-to-exclude +/tmp/packages-to-install diff --git a/container/osimage/openeuler/openeuler-20.03-lts.repo b/container/osimage/openeuler/openeuler-20.03-lts.repo new file mode 100644 index 0000000..691f438 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.03-lts.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.03-LTS/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.03-LTS/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/openeuler-20.09.repo b/container/osimage/openeuler/openeuler-20.09.repo new file mode 100644 index 0000000..1da7571 --- /dev/null +++ b/container/osimage/openeuler/openeuler-20.09.repo @@ -0,0 +1,43 @@ +#generic-repos is licensed under the Mulan PSL v2. +#You can use this software according to the terms and conditions of the Mulan PSL v2. +#You may obtain a copy of Mulan PSL v2 at: +# http://license.coscl.org.cn/MulanPSL2 +#THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR +#IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY OR FIT FOR A PARTICULAR +#PURPOSE. +#See the Mulan PSL v2 for more details. + +[OS] +name=OS +baseurl=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/OS/$basearch/RPM-GPG-KEY-openEuler + +[everything] +name=everything +baseurl=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/everything/$basearch/RPM-GPG-KEY-openEuler + +[EPOL] +name=EPOL +baseurl=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/EPOL/$basearch/RPM-GPG-KEY-openEuler + +[debuginfo] +name=debuginfo +baseurl=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/debuginfo/$basearch/RPM-GPG-KEY-openEuler + +[source] +name=source +baseurl=http://repo.openeuler.org/openEuler-20.09/source/ +enabled=1 +gpgcheck=1 +gpgkey=http://repo.openeuler.org/openEuler-20.09/source/RPM-GPG-KEY-openEuler diff --git a/container/osimage/openeuler/packages-to-install b/container/osimage/openeuler/packages-to-install new file mode 100644 index 0000000..f17eee1 --- /dev/null +++ b/container/osimage/openeuler/packages-to-install @@ -0,0 +1,10 @@ +cpio +systemd-udev.aarch64 +cifs-utils +nfs-utils +hostname +curl +openssh-server +openssl +iputils +wget -- 2.23.0

2 2

[PATCH v4 lkp-tests 2/2] stats/crystal: mysql-server to mysql-server.cr
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/crystal/mysql-server.cr | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 stats/crystal/mysql-server.cr diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr new file mode 100644 index 00000000..99b24b67 --- /dev/null +++ b/stats/crystal/mysql-server.cr @@ -0,0 +1,20 @@ +#!/usr/bin/env crystal + +results = [] of String + +while (line = STDIN.gets) + case line + # The output is as below: + # 200706 14:35:48 [ 99%] main.subquery_sj_innodb_all w8 [ pass ] 140 + # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 + # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 + when /\[\s*\d+%\]/ + results << line.split(']')[1].split[0] + ".pass: 1" if line.includes?("[ pass ]") + results << line.split(']')[1].split[0] + ".fail: 1" if line.includes?("[ fail ]") + results << line.split(']')[1].split[0] + ".skip: 1" if line.includes?("[ skipped ]") + end +end + +results.each do |item| + puts item +end -- 2.23.0

1 0

[PATCH v4 lkp-tests 1/2] stats/mysql-server: remove redundant whitespace
by Lu Kaiyi 26 Nov '20

26 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- stats/mysql-server | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/stats/mysql-server b/stats/mysql-server index 037d1f0f..ca31f9ea 100755 --- a/stats/mysql-server +++ b/stats/mysql-server @@ -1,5 +1,4 @@ #!/usr/bin/env ruby -# frozen_string_literal: true LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) @@ -16,9 +15,9 @@ while (line = STDIN.gets) # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 when /\[\s*\d+%\]/ - results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') - results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') - results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') + results << line.split(']')[1].split[0] + '.pass: 1' if line.include?('[ pass ]') + results << line.split(']')[1].split[0] + '.fail: 1' if line.include?('[ fail ]') + results << line.split(']')[1].split[0] + '.skip: 1' if line.include?('[ skipped ]') end end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/5] openeuler: the link for one command to build openeuler initramfs image
by Wang Chenglong 26 Nov '20

26 Nov '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- rootfs/initramfs/openeuler/aarch64/20.03-LTS/build | 1 + rootfs/initramfs/openeuler/aarch64/20.09/build | 1 + 2 files changed, 2 insertions(+) create mode 120000 rootfs/initramfs/openeuler/aarch64/20.03-LTS/build create mode 120000 rootfs/initramfs/openeuler/aarch64/20.09/build diff --git a/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build b/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build new file mode 120000 index 0000000..f04acac --- /dev/null +++ b/rootfs/initramfs/openeuler/aarch64/20.03-LTS/build @@ -0,0 +1 @@ +../../../../../container/osimage/openeuler/run \ No newline at end of file diff --git a/rootfs/initramfs/openeuler/aarch64/20.09/build b/rootfs/initramfs/openeuler/aarch64/20.09/build new file mode 120000 index 0000000..f04acac --- /dev/null +++ b/rootfs/initramfs/openeuler/aarch64/20.09/build @@ -0,0 +1 @@ +../../../../../container/osimage/openeuler/run \ No newline at end of file -- 2.23.0

2 2

[PATCH v2 compass-ci] web_backend.rb: fix : compare_groups_matrices needs arg suites_list
by Li Yuanchao 26 Nov '20

26 Nov '20

compare_groups_matrices needs three args, but now only get two, lost suites_list. divide method to pass rubocop limit Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 1f7224e..b5b06b4 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -147,6 +147,20 @@ def get_es_must(params) must end +def _get_group_matrices(query, dimension) + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + while matrices.empty? + from += size + break if from > total + + query[:from] = from + result = es_query(query) + matrices, suites_list = combine_group_query_data(result, dimension) + end + [matrices, suites_list] +end + def get_groups_matrices(conditions, dimension, must, size, from) must += build_mutli_field_subquery_body(conditions) count_query = { query: { bool: { must: must } } } @@ -166,27 +180,18 @@ def get_groups_matrices(conditions, dimension, must, size, from) }] } - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - while matrices.empty? - from += size - break if from > total - - query[:from] = from - result = es_query(query) - matrices = combine_group_query_data(result, dimension) - end - matrices + matrices, suites_list = _get_groups_matrices(query, dimension) + [matrices, suites_list] end def get_compare_body(params) dimension, conditions = get_dimension_conditions(params) must = get_es_must(params) - groups_matrices = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) + groups_matrices, suites_list = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) if !groups_matrices || groups_matrices.empty? body = 'No Data.' else - body = compare_group_matrices(groups_matrices, { no_print: true }) + body = compare_group_matrices(groups_matrices, suites_list, { no_print: true }) body = 'No Difference.' if !body || body.empty? end return body -- 2.23.0

2 1