March 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v5 compass-ci] doc/stats-rule.md
by Wang Chenglong 01 Mar '21

01 Mar '21

[Why] The rules to follow when writing the stats script. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/stats-rule.md | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 doc/stats-rule.md diff --git a/doc/stats-rule.md b/doc/stats-rule.md new file mode 100644 index 0000000..9e7af07 --- /dev/null +++ b/doc/stats-rule.md @@ -0,0 +1,46 @@ +${LKP_SERVER}/stats/${script} stat�� +========================================== + +## ��es�� + +1.��error_id�� + - original: {"changed": false, "failures": ["No package yum-plugin-priorities available."], "msg": "Failed to + install some of the specified packages", "rc": 1, "results": []} + + - error_id: error.No-package-yum-plugin-priorities-available: 1 + +2.�� + - ��error_id�� + # �� + # {"changed": false, "cmd": "csf -v", "msg": "[Errno 2] No such file or directory: b'csf': b'csf'", "rc": 2} + when /(No such file or directory: b'.+')/ + output_error $1 + end + + - ��jobs��'��'��es��error_id.��stats/${script}�� + +3.��error message�� + # original error message + # {"changed": false, "msg": "Failed to download packages: Cannot download 7.10.2/filebeat-7.10.2-aarch64.rpm: + # All mirrors were tried", "results": []} + when /(Failed to download packages: Cannot download .+):/ + output_error $1 + end + +4.��error_id�� + - ��'-'�� + - ��'-',��. + - ��'#'. + - �� : , ! ��. + + def common_error_id(line) + line = line.chomp + line.gsub!(/ /, '-') + line.gsub!(/-+/, '-') # ��'-', �� + line.gsub!(/\b-[a-z][0-9]\.[0-9]\.[0-9]+/, "#") # dgraph-linux-amd64-v0.8.2.tar.gz + line.gsub!(/:/, '') # error.:-'__mysql_packages'-is-undefined: 1 + line.gsub!(/,/, '') + line.gsub!(/'/, '') # Failed-to-download-metadata-for-repo-'dockerrepo: 1 + line.gsub!(/!/, '') # Distribution-openEuler-is-not-supported-by-this-role!: 1 + line + end -- 2.23.0

1 0

[PATCH compass-ci] src/lib/job.cr: delete the field submit_token
by Li Ping 01 Mar '21

01 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- src/lib/job.cr | 1 + 1 file changed, 1 insertion(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index c3eeaf4..33456ca 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -403,6 +403,7 @@ class Job @hash.delete("my_uuid") @hash.delete("my_token") + @hash.delete("submit_token") end private def is_valid_account?(account_info) -- 2.23.0

1 0

[PATCH lkp-tests] jobs/iso2rootfs.yaml: change some field to level-1 parameter.
by Li Ping 01 Mar '21

01 Mar '21

[why] the related fields when submit test.yaml to be level-1 parameter is easy for scheduler to delete, like submit_name: submit_email: submit_token: and the required fields when submit test.yaml not have much to do with the iso2rootfs. Signed-off-by: Li Ping <1477412247(a)qq.com> --- jobs/iso2rootfs.yaml | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 55c4345fa..22651d25e 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -51,23 +51,23 @@ iso2rootfs: # http://1.1.1.1//dailybuilds/1970-01-01-00-00-00/openEuler-20.03-LTS-aarch64… dailybuild_iso_url_file: - ################# - # submit test yaml related fields - ################# +################# +# submit test yaml related fields +################# - # three required fields when submit test yaml - submit_name: - submit_email: - submit_token: +# three required fields when submit test yaml +submit_name: +submit_email: +submit_token: - # submit target tests yaml - ## 1. You can add as many jobs as you like. - ## 2. The following three fields is required for every test job. - test1_yaml: - test1_os_mount: - test1_testbox: - ## the following three fields is required when your job.yaml and - ## script for this test are from the internet. - test1_git_url: - test1_git_yaml: - test1_git_script: +# submit target tests yaml +## 1. You can add as many jobs as you like. +## 2. The following three fields is required for every test job. +test1_yaml: +test1_os_mount: +test1_testbox: +## the following three fields is required when your job.yaml and +## script for this test are from the internet. +test1_git_url: +test1_git_yaml: +test1_git_script: -- 2.23.0

1 0

[PATCH compass-ci] container: add get_job_stats function in assist-result
by Cao Xueliang 01 Mar '21

01 Mar '21

We can expose assist-result host/port to the SUT, then the SUT can get the job stats. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/assist-result/routes.rb | 11 +++++++++++ container/assist-result/views/get_job_stats.rb | 10 ++++++++++ 2 files changed, 21 insertions(+) create mode 100644 container/assist-result/views/get_job_stats.rb diff --git a/container/assist-result/routes.rb b/container/assist-result/routes.rb index 077a427..02956aa 100755 --- a/container/assist-result/routes.rb +++ b/container/assist-result/routes.rb @@ -7,6 +7,7 @@ require 'json' require 'sinatra' require_relative './views/get_job_yaml' +require_relative './views/get_job_stats' require_relative './views/check_job_credible' configure do @@ -34,3 +35,13 @@ post '/check_job_credible' do return [200, {'credible' => result}.to_json] end + +get '/get_job_stats/:job_id' do + begin + result = get_job_stats(params[:job_id]) + rescue StandardError => e + return [400, e.backtrace.inspect] + end + + return [200, result.to_json] +end diff --git a/container/assist-result/views/get_job_stats.rb b/container/assist-result/views/get_job_stats.rb new file mode 100644 index 0000000..d294df3 --- /dev/null +++ b/container/assist-result/views/get_job_stats.rb @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative "#{ENV['CCI_SRC']}/lib/es_query" + +def get_job_stats(job_id) + content = ESQuery.new.query_by_id(job_id) + return content['stats'] +end -- 2.23.0

2 2

[PATCH v4 compass-ci] doc/stats-rule.md
by Wang Chenglong 01 Mar '21

01 Mar '21

[Why] The rules to follow when writing the stats script. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/stats-rule.md | 48 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100644 doc/stats-rule.md diff --git a/doc/stats-rule.md b/doc/stats-rule.md new file mode 100644 index 0000000..ccddbf0 --- /dev/null +++ b/doc/stats-rule.md @@ -0,0 +1,48 @@ +${LKP_SERVER}/stats/${script} stat�� +========================================== + +## ��es�� + +1.��error_id�� + - original: {"changed": false, "failures": ["No package yum-plugin-priorities available."], "msg": "Failed to + install some of the specified packages", "rc": 1, "results": []} + + - error_id: error.No-package-yum-plugin-priorities-available: 1 + +2.�� + - ��error_id�� + # �� + # {"changed": false, "cmd": "csf -v", "msg": "[Errno 2] No such file or directory: b'csf': b'csf'", "rc": 2} + when /(No such file or directory: b'.+')/ + output_error $1 + end + + - ��jobs��'��'��es��error_id.��stats/${script}�� + +3.��error message�� + # original error message + # {"changed": false, "msg": "Failed to download packages: Cannot download 7.10.2/filebeat-7.10.2-aarch64.rpm: + # All mirrors were tried", "results": []} + when /(Failed to download packages: Cannot download .+):/ + output_error $1 + end + +4.��error_id�� + - ��'-'�� + - ��'-',��. + - ��'#'. + - ��: , ! {%%} ��. + + def common_error_id(line) + line = line.chomp + line.gsub!(/ /, '-') + line.gsub!(/-+/, '-') # ��'-', �� + line.gsub!(/\b-[a-z][0-9]\.[0-9]\.[0-9]+/, "#") # dgraph-linux-amd64-v0.8.2.tar.gz + line.gsub!(/:/, '') # error.:-'__mysql_packages'-is-undefined: 1 + line.gsub!(/,/, '') + line.gsub!(/'/, '') # Failed-to-download-metadata-for-repo-'dockerrepo: 1 + line.gsub!(/!/, '') # Distribution-openEuler-is-not-supported-by-this-role!: 1 + line.gsub!(/{%/, ' ') + line.gsub!(/%}/, ' ') + line + end -- 2.23.0

2 2

[PATCH compass-ci 1/2] sparrow/4-docker/buildall: check whether skip ssh-r
by Liu Yinsi 01 Mar '21

01 Mar '21

[why] for user locally deploy compass-ci, ssh-r is not exists, so check when ssh-r already exists, skip ssh-r. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index bb77a1f..c9edbe9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -56,7 +56,8 @@ do_one_run() mkdir $tmpdir/start_$container_name 2>/dev/null && ( cd "$container" - [ "$container_name" == 'ssh-r' ] && exit + docker ps | grep -wq "ssh_r" + [ "$?" == "0" ] && [ "$container_name" == 'ssh-r' ] && exit [ -x first-run ] && ./first-run [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run -- 2.23.0

2 2

[PATCH v3 compass-ci] container/assign-account: optimize config login permission
by Luan Shengde 01 Mar '21

01 Mar '21

optimize account login permission setting: before: set enable login: has key: enable_login key value: true set disable login has key: enable_login key value: false or has no key: enable_login now: set enable login: has key: enable_login key value: true set disable login: has key: enable_login key value: false keep the default login permission: has no key: enable_login Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 7e0a5a7..2922f81 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -136,7 +136,7 @@ class AccountStorage def permit_login_config(login_name) if @data.key?('enable_login') && @data['enable_login'] %x(usermod -s /usr/bin/zsh #{login_name}) - else + elsif @data.key?('enable_login') && !@data['enable_login'] %x(usermod -s /sbin/nologin #{login_name}) end end -- 2.23.0

1 0

[PATCH compass-ci] add check runtime upper limit
by Luan Shengde 01 Mar '21

01 Mar '21

add time limit for borrowing machine. the max runtime is limited to no more than 10 days. case runtime/sleep beyond the limit or both of them was set empty value. it will throw error message and prevent the submit for borrowing machine. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- src/lib/job.cr | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 52965bc..63ec8fb 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -145,6 +145,7 @@ class Job check_required_keys() check_account_info() + check_run_time() set_defaults() end @@ -409,6 +410,29 @@ class Job @hash.delete("my_token") end + private def check_run_time + # only job.yaml for borrowing machine has the key: ssh_pub_key + return unless @hash.has_key?("ssh_pub_key") + + # the maxmum borrowing time is limited no more than 10 days. + # case the runtime/sleep value count beyond the limit, + # it will throw error message and prevent the submit for borrowing machine. + # runtime value is converted to second. + max_run_time = 10 * 24 * 3600 + error_msg = "\nMachine borrow time(runtime/sleep) cannot exceed 10 days. Consider re-borrow.\n" + + if @hash["pp"]["sleep"].as_i? + sleep_run_time = @hash["pp"]["sleep"] + elsif @hash["pp"]["sleep"].as_h? + sleep_run_time = @hash["pp"]["sleep"]["runtime"] + else + notice_msg = "\nPlease set runtime/sleep first for the job yaml and retry." + raise notice_msg + end + + raise error_msg if sleep_run_time.as_i > max_run_time + end + private def is_valid_account?(account_info) return false unless account_info.is_a?(JSON::Any) -- 2.23.0

1 0