- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] tests/cci-makepkg: change to logic condition
by Liu Shaofei 22 Dec '20

22 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/cci-makepkg | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a50a40c20..d48e8495e 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -41,6 +41,8 @@ curl_pkgfile() download_pkgfile() { + [ -n "$PKGBUILD_TAG" ] || return + local pkg_tag_file="PKGBUILD-$PKGBUILD_TAG" curl_pkgfile "pkg/$benchmark" "$pkg_tag_file" @@ -93,7 +95,7 @@ centos_update_gcc() source /opt/rh/devtoolset-8/enable } -[ -n "$PKGBUILD_TAG" ] && download_pkgfile +download_pkgfile distro_install_depends lkp-dev -- 2.23.0

1 0

[PATCH lkp-tests 2/2] lib/env.sh: refactor set-env
by Wei Jihui 22 Dec '20

22 Dec '20

move sbin/set-env.sh functions to lib/env.sh delete sbin/set-env.sh do set-env at sbin/install-dependencies Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/env.sh | 30 ++++++++++++++++++++++++++ sbin/install-dependencies.sh | 10 ++++++++- sbin/set-env.sh | 42 ------------------------------------ 3 files changed, 39 insertions(+), 43 deletions(-) delete mode 100755 sbin/set-env.sh diff --git a/lib/env.sh b/lib/env.sh index 8ac42a83..9f37d8cc 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -125,3 +125,33 @@ shell_profile() [ -f "$profile" ] || touch $profile echo $profile } + +write_shell_profile() +{ + shell_profile_file=$(shell_profile) + if [ $# -gt 0 ]; then + echo "$@" >> $shell_profile_file + fi + + source $shell_profile_file +} + +write_host() +{ + if is_system "Linux"; then + nr_cpu=$(nproc) + memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') + else + nr_cpu=$(sysctl -n hw.logicalcpu) + memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') + fi + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') + memory=$(echo $memory_new | awk '{print int($0)+1}')G + + cat > hosts/$(hostname) <<-EOF + nr_cpu: $nr_cpu + memory: $memory + hdd_partitions: + ssd_partitions: + EOF +} diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 6263dc91..8a4b29cd 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -4,7 +4,6 @@ SCRIPT_DIR=$(dirname $(realpath $0)) PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh -. $PROJECT_DIR/sbin/set-env.sh # choose install function base on DISTRIBUTION linux_dep() @@ -48,4 +47,13 @@ run() fi } +set_env() +{ + write_host + write_shell_profile "export LKP_SRC=$PWD" + write_shell_profile "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" + source $(shell_profile) +} + +set_env run diff --git a/sbin/set-env.sh b/sbin/set-env.sh deleted file mode 100755 index 8234eaf8..00000000 --- a/sbin/set-env.sh +++ /dev/null @@ -1,42 +0,0 @@ -#!/bin/sh - -SCRIPT_DIR=$(dirname $(realpath $0)) -PROJECT_DIR=$(dirname $SCRIPT_DIR) - -. $PROJECT_DIR/lib/env.sh - -write_shell_profile() -{ - shell_profile_file=$(shell_profile) - if [ $# -gt 0 ]; then - echo $@ >> $shell_profile_file - else - echo "export LKP_SRC=$PWD" >> $shell_profile_file - echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $shell_profile_file - fi - - source $shell_profile_file -} - -write_host() -{ - if is_system "Linux"; then - nr_cpu=$(nproc) - memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - else - nr_cpu=$(sysctl -n hw.logicalcpu) - memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') - fi - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G - - cat > hosts/$(hostname) <<-EOF - nr_cpu: $nr_cpu - memory: $memory - hdd_partitions: - ssd_partitions: - EOF -} - -write_shell_profile -write_host -- 2.23.0

1 0

[PATCH lkp-tests 1/2] sbin/install-dependencies: adjust file format
by Wei Jihui 22 Dec '20

22 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- sbin/install-dependencies.sh | 39 ++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 3a206134..6263dc91 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -6,28 +6,27 @@ PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh . $PROJECT_DIR/sbin/set-env.sh -# choose install function base on -# DISTRIBUTION +# choose install function base on DISTRIBUTION linux_dep() { -source /etc/os-release -case $ID in -ubuntu|debian) - export DEBIAN_FRONTEND=noninteractive - apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client - ;; -openEuler|fedora|rhel|centos) - if type dnf > /dev/null 2>&1; then - dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - else - yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - fi - sudo gem install -f git activesupport rest-client - ;; -*) - echo "$ID not support! please install dependencies manually." - ;; -esac + source /etc/os-release + case $ID in + ubuntu|debian) + export DEBIAN_FRONTEND=noninteractive + apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client + ;; + openEuler|fedora|rhel|centos) + if type dnf > /dev/null 2>&1; then + dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + else + yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + fi + sudo gem install -f git activesupport rest-client + ;; + *) + echo "$ID not support! please install dependencies manually." + ;; + esac } -- 2.23.0

1 0

[PATCH compass-ci] compare_matrixes.rb : reduce compare time
by Li Yuanchao 22 Dec '20

22 Dec '20

When there is multi object group, don't use one object group to compare. Cause one object groups are too many, they spend too much time. And they are not what we want. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/compare_matrixes.rb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 6e4c2aa..21bd5c9 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -390,7 +390,10 @@ end def compare_group_matrices(group_matrices, suites_hash, options) result_str = '' group_matrices_array = sort_by_matrix_size(group_matrices) + have_multi_member = group_matrices_array[0][1].size > 1 group_matrices_array.each do |matrice_kv| + next if have_multi_member && matrice_kv[1].size < 2 + result_str += get_matrix_str(matrice_kv[0], matrice_kv[1], suites_hash[matrice_kv[0]], options) end result_str -- 2.23.0

1 0

[PATCH lkp-tests] jobs/bisect.yaml: dc-1g--delimiter => dc-1g
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 5fab30bfc..11ffc7c05 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -1,4 +1,4 @@ suite: bisect docker_image: delimiter:latest -testbox: dc-1g--delimiter +testbox: dc-1g bisect: -- 2.23.0

1 0

[PATCH compass-ci] doc/tutorial.md: fix abnormal display in Markdown
by Lu Kaiyi 22 Dec '20

22 Dec '20

the middle part in * and * is italics for markdown document, if want show '*' in this condition, need use \* escape '*' character. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- doc/tutorial.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..19d5741 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -258,7 +258,7 @@ job yaml ��[YAML](http://yaml.org/YAM 4) �� archlinux �� PKGBUILD �� PKGBUILD �� - PKGBUILD �� /srv/git/archlinux/*/*/PKGBUILD�� + PKGBUILD �� /srv/git/archlinux/\*/\*/PKGBUILD�� ### depends -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result: auto_tool => assign_account_client
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 5e296a6..1d4ed69 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -4,7 +4,7 @@ require 'json' require 'base64' require_relative 'git' require_relative 'mail_client' -require_relative 'auto_tool' +require_relative 'assign_account_client' # compose and send email for bisect result class MailBisectResult -- 2.23.0

1 0

[PATCH compass-ci] web_backend.rb: optimize var name
by Li Yuanchao 22 Dec '20

22 Dec '20

As the implementation of method combine_group_query_data changes, var name suit_list is ok, cause the method return a hash in fact. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 2405107..0ca71e1 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -186,15 +186,15 @@ end def do_get_groups_matrices(must, dimension, total, size, from) job_list = get_dimension_job_list(dimension, must, size, from) - matrices, suites_list = combine_group_query_data(job_list, dimension) + matrices, suites_hash = combine_group_query_data(job_list, dimension) while matrices.empty? from += size break if from > total job_list = get_dimension_job_list(dimension, must, size, from) - matrices, suites_list = combine_group_query_data(job_list, dimension) + matrices, suites_hash = combine_group_query_data(job_list, dimension) end - [matrices, suites_list] + [matrices, suites_hash] end def get_groups_matrices(conditions, dimension, must, size, from) @@ -209,11 +209,11 @@ end def get_compare_body(params) dimension, conditions = get_dimension_conditions(params) must = get_es_must(params) - groups_matrices, suites_list = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) + groups_matrices, suites_hash = get_groups_matrices(conditions, dimension, must, COMPARE_RECORDS_NUMBER, 0) if !groups_matrices || groups_matrices.empty? body = 'No Data.' else - body = compare_group_matrices(groups_matrices, suites_list, { no_print: true }) + body = compare_group_matrices(groups_matrices, suites_hash, { no_print: true }) body = 'No Difference.' if !body || body.empty? end return body -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies: adjuest file format
by Wei Jihui 22 Dec '20

22 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- sbin/install-dependencies.sh | 39 ++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 3a206134..6263dc91 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -6,28 +6,27 @@ PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh . $PROJECT_DIR/sbin/set-env.sh -# choose install function base on -# DISTRIBUTION +# choose install function base on DISTRIBUTION linux_dep() { -source /etc/os-release -case $ID in -ubuntu|debian) - export DEBIAN_FRONTEND=noninteractive - apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client - ;; -openEuler|fedora|rhel|centos) - if type dnf > /dev/null 2>&1; then - dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - else - yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - fi - sudo gem install -f git activesupport rest-client - ;; -*) - echo "$ID not support! please install dependencies manually." - ;; -esac + source /etc/os-release + case $ID in + ubuntu|debian) + export DEBIAN_FRONTEND=noninteractive + apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client + ;; + openEuler|fedora|rhel|centos) + if type dnf > /dev/null 2>&1; then + dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + else + yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + fi + sudo gem install -f git activesupport rest-client + ;; + *) + echo "$ID not support! please install dependencies manually." + ;; + esac } -- 2.23.0

1 0

[PATCH lkp-tests] lib/env.sh: fix the check result of is_virt
by Zhang Yu 22 Dec '20

22 Dec '20

Hypervisor is a software that simulates hardware resources and scheduling on a physical machine. so if grep success, return 1 means it's a physical machine Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/env.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/env.sh b/lib/env.sh index 8ac42a836..a1d5e445b 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -69,7 +69,8 @@ is_virt() # run as root [ -n "$(virt-what)" ] else - grep -q -w hypervisor /proc/cpuinfo + grep -q -w hypervisor /proc/cpuinfo && return 1 + return 0 fi } -- 2.23.0

1 0

[PATCH compass-ci] 3-code/git: add repository version pointing
by Hu Xuejiao 22 Dec '20

22 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/3-code/git | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/git b/sparrow/3-code/git index 494107e..2159ddc 100755 --- a/sparrow/3-code/git +++ b/sparrow/3-code/git @@ -13,11 +13,18 @@ cd /c || exit # modify and manual run for now: [ -d "compass-ci/.git" ] || { git clone https://gitee.com/wu_fengguang/compass-ci.git - ln -s compass-ci cci + + cd /c/compass-ci || exit + [ -n "$compass_commitid" ] && git reset --hard "$compass_commitid" + + ln -s compass-ci /c/cci } [ -d "lkp-tests/.git" ] || { git clone https://gitee.com/wu_fengguang/lkp-tests.git + + cd /c/lkp-tests || exit + [ -n "$lkp_commitid" ] && git reset --hard "$lkp_commitid" } [ -d "upstream-repos/.git" ] || { -- 2.23.0

1 0

[PATCH lkp-tests] tests/deploy-cci: specifying the repository version
by Hu Xuejiao 22 Dec '20

22 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- tests/deploy-cci | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/deploy-cci b/tests/deploy-cci index e296568f..55835544 100755 --- a/tests/deploy-cci +++ b/tests/deploy-cci @@ -19,6 +19,9 @@ mkdir -p /c cd /c || exit git clone https://gitee.com/wu_fengguang/compass-ci.git +cd /c/compass-ci || exit +[ -n "$compass_commitid" ] && git reset --hard "$compass_commitid" + # Need mount tmpfs to fix compass-ci/container/os-nfs/start error # "exportfs: /exports/result does not support NFS export" mount -t tmpfs none /srv -- 2.23.0

1 0

[PATCH compass-ci] lib/git_bisect: return all errors to bisect mail.
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index bc1e035..f789a71 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -82,7 +82,6 @@ class GitBisect result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` end Utils.create_bisect_log(@work_dir) - FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) puts "\nbisect result: #{result}" analyse_result(result) end @@ -92,10 +91,16 @@ class GitBisect return nil unless temp[0].include?('is the first bad commit') || temp[-1].include?('bisect run success') first_bad_commit = Utils.parse_first_bad_commit(result) - error_messages = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_errorid(@error_id) + bisect_error = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_errorid(@error_id) + all_errors = Utils.obt_errors(@work_dir, first_bad_commit) + first_bad_commit_result_root = Utils.obt_result_root_by_commit(first_bad_commit) + FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) - return Hash['repo' => @upstream_repo, 'commit' => first_bad_commit, - 'error_messages' => error_messages] + return Hash['repo' => @upstream_repo, + 'commit' => first_bad_commit, + 'bisect_error' => bisect_error, + 'all_errors' => all_errors, + 'first_bad_commit_result_root' => first_bad_commit_result_root] end # first search the good commit in db -- 2.23.0

1 0

[PATCH compass-ci] delimiter/utils: add the get all errors function
by Cao Xueliang 22 Dec '20

22 Dec '20

1. fix some rubocop warnings. 2. add get errors by commits - get the pre_commit by find_parent_commit - get cur_id and pre_id by obt_id_by_commit - get errors by get_compare_result 3. add get result root by commit Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 48 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41803f3..372e821 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -4,10 +4,12 @@ require 'set' require 'json' +require 'yaml' require 'fileutils' require_relative './constants' require_relative '../../lib/sched_client' +require_relative '../../lib/compare_error_messages' require_relative "#{ENV['LKP_SRC']}/lib/monitor" # a utils module for delimiter service @@ -83,9 +85,20 @@ module Utils es = ESQuery.new new_job = es.query_by_id(new_job_id) - return 'bad' if new_job['stats'].key?(error_id) - return 'good' + status = new_job['stats'].key?(error_id) ? 'bad' : 'good' + puts "new_job_id: #{new_job_id}" + puts "upstream_commit: #{job['upstream_commit']}" + record_jobs(new_job_id, job['upstream_commit']) + + return status + end + + def record_jobs(job_id, job_commit) + FileUtils.mkdir_p TMP_RESULT_ROOT unless File.exist? TMP_RESULT_ROOT + commit_jobs = File.join(TMP_RESULT_ROOT, 'commit_jobs') + content = "#{job_commit}: #{job_id}" + File.open(commit_jobs, 'a+') { |f| f.puts content } end def get_account_info @@ -100,6 +113,8 @@ module Utils account_info = get_account_info raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info + record_jobs(job['id'], job['upstream_commit']) + job['suite'] = 'bisect' job['my_name'] = account_info['my_name'] job['my_email'] = account_info['my_email'] @@ -133,5 +148,34 @@ module Utils log_content.each { |line| f.puts(line) } end end + + def find_parent_commit(git_dir, commit) + response = %x(git -C #{git_dir} rev-parse #{commit}~1) + return response.chomp + end + + def obt_id_by_commit(commit) + puts "#{TMP_RESULT_ROOT}/commit_jobs" + content = YAML.load(File.open("#{TMP_RESULT_ROOT}/commit_jobs")) + puts content + content[commit] + end + + def obt_errors(git_dir, commit) + pre_commit = find_parent_commit(git_dir, commit) + obt_errors_by_commits(commit, pre_commit) + end + + def obt_errors_by_commits(cur_commit, pre_commit) + cur_id = obt_id_by_commit(cur_commit) + pre_id = obt_id_by_commit(pre_commit) + _, errors = get_compare_result(pre_id, cur_id) + return errors + end + + def obt_result_root_by_commit(commit) + id = obt_id_by_commit(commit) + ESQuery.new.query_by_id(id)['result_root'] + end end end -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_client: add send_mail_encode function
by Cao Xueliang 22 Dec '20

22 Dec '20

If some special characters the payload of post, we use base64 encode to pass the payload. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_client.rb | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/lib/mail_client.rb b/lib/mail_client.rb index 1d0938a..b13492f 100644 --- a/lib/mail_client.rb +++ b/lib/mail_client.rb @@ -2,6 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'base64' require 'rest-client' require_relative 'constants.rb' @@ -18,4 +19,9 @@ class MailClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_yaml") resource.post(mail_json) end + + def send_mail_encode(mail_data) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_encode") + resource.post(Base64.encode64(mail_data)) + end end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: delete input job key which is path
by Zhang Yu 22 Dec '20

22 Dec '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index 2ed06edc6..7553312ed 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -254,6 +254,7 @@ class Job @overrides.merge!(hash['override']){ |_key, a, _b| a} hash.delete('override') end + hash.delete_if { |key, _| key.start_with?('#!') } hash.merge!(@overrides) @jobs.concat(multi_args(hash)) # return [hash] or [h1,h2] end -- 2.23.0

1 0

[PATCH compass-ci] fix 404 error
by Li Ping 22 Dec '20

22 Dec '20

Signed-off-by: Li Ping <1477412247(a)qq.com> --- README.en.md | 4 ++-- README.zh.md | 4 ++-- ...34\272\350\260\203\346\265\213\344\273\273\345\212\241.md" | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.en.md b/README.en.md index 5e9386e..c082167 100644 --- a/README.en.md +++ b/README.en.md @@ -34,13 +34,13 @@ If a new error ID is generated, the bisect is automatically triggered to locate 2. Run the **git push** command to update the repository and automatically trigger the test. -3. You can [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/brow… and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/c… test results on the web page: https://compass-ci.openeuler.org/jobs. +3. You can [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/brow… and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/c… test results on the web page: https://compass-ci.openeuler.org/jobs **Manually Submitting a Test Task** 1. [Install the Compass-CI client](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E…. 2. [Compile test cases and manually submit test tasks](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testcas…. -3. You can [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/brow… and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/c… test results on the web page: https://compass-ci.openeuler.org/jobs. +3. You can [view](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/brow… and [compare](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/c… test results on the web page: https://compass-ci.openeuler.org/jobs **Logging in to the Test Environment** diff --git a/README.zh.md b/README.zh.md index 0616bec..c2140b0 100644 --- a/README.zh.md +++ b/README.zh.md @@ -37,13 +37,13 @@ Compass-CI 监控很多开源软件 git repos，一旦检测到代码更新， 2. 执行 git push 命令更新仓库，自动触发测试。 -3. 在网页中[查看](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… web: https://compass-ci.openeuler.org/jobs。 +3. 在网页中[查看](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… web: https://compass-ci.openeuler.org/jobs **手动提交测试任务** 1. [安装 Compass-CI 客户端](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/%E6%9… 2. [编写测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add-testca…, [手动提交测试任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… -3. 在网页中[查看](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… web: https://compass-ci.openeuler.org/jobs。 +3. 在网页中[查看](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… web: https://compass-ci.openeuler.org/jobs **登录测试环境** diff --git "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" "b/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" index b59431e..7a54505 100644 --- "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" +++ "b/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" @@ -3,8 +3,8 @@ # 1. 前提条件： ## 请先学习: * [配置个人邮箱](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/ap… -* [如何申请测试机](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/如… -* [submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +* [如何申请测试机](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/b… +* [submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… # 2. 操作方法 -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result: optimize mail bisect content
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 63 ++++++++++++++++++++++++++------------- 1 file changed, 42 insertions(+), 21 deletions(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 090855e..5e296a6 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -1,48 +1,69 @@ # frozen_string_literal: true -require_relative 'mail_client' -require_relative 'git' require 'json' +require 'base64' +require_relative 'git' +require_relative 'mail_client' +require_relative 'auto_tool' # compose and send email for bisect result class MailBisectResult def initialize(bisect_info) - @error_messages = bisect_info['error_messages'] + @bisect_error = bisect_info['bisect_error'] + @all_errors = bisect_info['all_errors'] @repo = bisect_info['repo'] @commit_id = bisect_info['commit'] + @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] @git_commit = GitCommit.new(@repo, @commit_id) + # now send mail to review + @to = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com' end def create_send_email - compose_mail - send_mail + send_report_mail(compose_mail) + send_account_mail end def compose_mail - subject = "[Compass-CI][#{(a)repo.split('/')[1]}]: #{(a)error_messages[0].split("\n")[0]}" - job_url = ENV['result_root'] ? "job url: http://#{ENV['SRV_HTTP_HOST']}:#{ENV['SRV_HTTP_PORT']}#{ENV['result_root']}\n" : '' - body = <<~BODY - Hi #{(a)git_commit.author_name}, + subject = "[Compass-CI][#{(a)repo.split('/')[1]}] #{@commit_id[0..9]} #{(a)bisect_error[0].split("\n")[0]}" + prefix_srv = "http://#{ENV['SRV_HTTP_HOST']}:#{ENV['SRV_HTTP_PORT']}" + bisect_job_url = ENV['result_root'] ? "bisect job info: #{prefix_srv}#{ENV['result_root']}\n" : '' + first_bad_commit_job_url = "first bad commit job info: #{prefix_srv}#{@first_bad_commit_result_root}\n" - git url: #{(a)git_commit.url} - git commit: #{@commit_id[0..11]} ("#{(a)git_commit.subject}") + data = <<~BODY + To: #{@to} + Subject: #{subject} - gcc version: 7.3.0 + Hi #{(a)git_commit.author_name}, - error_messages: - #{(a)error_messages.join("\n")} + url: #{(a)git_commit.url} + commit: #{@commit_id} ("#{(a)git_commit.subject}") + compiler: gcc (GCC) 7.3.0 - #{job_url} + all errors/warnings: + #{@all_errors} + + #{bisect_job_url} + #{first_bad_commit_job_url} Regards, Compass CI team BODY - to = 'caoxl(a)crystal.ci' - @hash = { 'to' => to, 'body' => body, 'subject' => subject } - # @hash = { 'to' => @git_commit.author_email, 'body' => body, 'subject' => subject } + + return data end - def send_mail - json = @hash.to_json - MailClient.new.send_mail(json) + def send_report_mail(mail_data) + MailClient.new.send_mail_encode(mail_data) + end + + def send_account_mail + user_info = { + 'my_email' => @to, + 'my_name' => @git_commit.author_name, + 'my_commit_url' => "#{@git_commit.url}/commit/#{@commit_id}" + } + + apply_account = AutoAssignAccount.new(user_info) + apply_account.send_account end end -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: temporarily cancel account verification
by Wu Zhende 22 Dec '20

22 Dec '20

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e6f7e6a..2554db1 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -347,13 +347,14 @@ class Job error_msg = "Failed to verify the account.\n" error_msg += "Please refer to https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco…" account_info = @es.get_account(self["my_email"]) - raise account_info unless account_info.is_a?(JSON::Any) + # raise account_info unless account_info.is_a?(JSON::Any) - @account_info = account_info.as_h + # @account_info = account_info.as_h + @account_info = account_info.as_h if account_info.is_a?(JSON::Any) - raise error_msg if @account_info["found"]? == false - raise error_msg unless self["my_name"] == @account_info["my_name"].to_s - raise error_msg unless self["my_token"] == @account_info["my_token"] + # raise error_msg if @account_info["found"]? == false + # raise error_msg unless self["my_name"] == @account_info["my_name"].to_s + # raise error_msg unless self["my_token"] == @account_info["my_token"] @hash.delete("my_uuid") @hash.delete("my_token") -- 2.23.0

1 0

[PATCH lkp-tests] adaptation/archlinux: add archlinux mapping
by Wang Yong 22 Dec '20

22 Dec '20

add mapping in order to run cci-depends job for build-pkg Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/archlinux | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/distro/adaptation/archlinux b/distro/adaptation/archlinux index 89163cf41..c52212e08 100644 --- a/distro/adaptation/archlinux +++ b/distro/adaptation/archlinux @@ -72,3 +72,11 @@ libtirpc: libtirpc1 initramfs-tools: openssh-server: openssh openssh-client: openssh +autopoint: gettext +g++: gcc +libarchive13: libarchive +libbz2-1.0: bzip2 +libssl: openssl +libtool-bin: libtool +linux-perf: perf +ruby-dev: ruby -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests: fix return value
by Zhang Yu 22 Dec '20

22 Dec '20

[why] if /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0 both not exit, the value of return should be 0 not 1 Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/host-info b/tests/host-info index 3984727b4..fe55d1ff2 100755 --- a/tests/host-info +++ b/tests/host-info @@ -102,7 +102,7 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX - [ -e /dev/ipmi0 ] || [ -e /dev/ipmi/0 ] || [ -e /dev/ipmidev/0 ] || return + [ -e /dev/ipmi0 ] || [ -e /dev/ipmi/0 ] || [ -e /dev/ipmidev/0 ] || return 0 local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip: $ip" } -- 2.23.0

1 0

[PATCH v4 compass-ci 1/6] fetch-mail: config file for fetchmail
by Luan Shengde 22 Dec '20

22 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/fetchmailrc.template | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/fetch-mail/fetchmailrc.template diff --git a/container/fetch-mail/fetchmailrc.template b/container/fetch-mail/fetchmailrc.template new file mode 100644 index 0000000..8c7e82b --- /dev/null +++ b/container/fetch-mail/fetchmailrc.template @@ -0,0 +1,5 @@ +poll imap.qq.com with proto IMAP +username "FETCH_MAIL_ADDRESS" +password "FETCH_MAIL_AUTH_CODE" +mda "/usr/bin/procmail -f %F -d %T" +nokeep -- 2.23.0

2 2

[PATCH v4 compass-ci 2/6] fetch-mail: config file for procmail
by Luan Shengde 22 Dec '20

22 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/procmailrc.template | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 container/fetch-mail/procmailrc.template diff --git a/container/fetch-mail/procmailrc.template b/container/fetch-mail/procmailrc.template new file mode 100644 index 0000000..9cea4bc --- /dev/null +++ b/container/fetch-mail/procmailrc.template @@ -0,0 +1,25 @@ +DROPPRIVS=yes +PATH=$HOME/bin:/usr/local/bin:/usr/bin:/bin +MAILDIR=FETCH_MAIL_DIR +MAILLIST=${MAILDIR} +DEFAULT=${MAILDIR}/FETCH_MAIL_BOX/ +ORGMAIL=${DEFAULT}/FETCH_MAIL_BOX/ +LOGFILE=${MAILDIR}/log/procmail # recommended + +UMASK=002 + +DISPLAY=:0 +XAUTHORITY=$HOME/.Xauthority + +# One often receives duplicate messages for cross-postings to mailing +# lists, or explicit replies to a post from you in addition to the +# mailing list message. +# +# The following recipe eliminates all duplicate messages. It tells +# formail(1) to keep a cache file of the Message-IDs of the most recent +# messages you've received. Since Message-IDs are guaranteed unique for +# distinct messages, they are ideally suited to weed out duplicates. + +# LOCKFILE=$MAILDIR/.msgid.lock +:0 Wh: $MAILDIR/.msgid.lock +| formail -D 655360 $MAILDIR/log/msgid -- 2.23.0

2 2

[PATCH v2 compass-ci] lib/matrix2.rb: fix rubocop warning
by Lu Weitao 22 Dec '20

22 Dec '20

[Why] lib/matrix2.rb:81:1: C: Metrics/AbcSize: Assignment Branch Condition size for create_matrix is too high. [<5, 16, 5> 17.49/15] def create_matrix(job_list) ... ^^^^^^^^^^^^^^^^^^^^^^^^^^^ lib/matrix2.rb:81:1: C: Metrics/CyclomaticComplexity: Cyclomatic complexity for create_matrix is too high. [9/7] def create_matrix(job_list) ... ^^^^^^^^^^^^^^^^^^^^^^^^^^^ lib/matrix2.rb:81:1: C: Metrics/MethodLength: Method has too many lines. [18/15] def create_matrix(job_list) ... ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/matrix2.rb | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index a7b83c6..19eb8f1 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -83,13 +83,7 @@ def create_matrix(job_list) suites = [] job_list.each do |job| stats = job['stats'] - next unless stats - - if job['suite'] - next unless stats.keys.any? { |stat| stat.start_with?(job['suite']) } - - suites << job['suite'] - end + next unless stats && assign_suites(suites, job, stats) stats.each do |key, value| next if key.include?('timestamp') @@ -103,6 +97,12 @@ def create_matrix(job_list) return matrix, suites end +def assign_suites(suites, job, stats) + return unless job['suite'] && stats.keys.any? { |stat| stat.start_with?(job['suite']) } + + suites << job['suite'] +end + # input: query results from es_query # return: matrix def combine_query_data(query_data) -- 2.23.0

1 0

[PATCH compass-ci] lib/matrix2.rb: fix rubocop offenses
by Lu Kaiyi 22 Dec '20

22 Dec '20

[error] Inspecting 1 file C Offenses: matrix2.rb:160:35: C: Style/TrailingUnderscoreVariable: Do not use trailing _s in parallel assignment. Prefer groups[group_key][dim_key], = create_matrix(jobs). groups[group_key][dim_key], _ = create_matrix(jobs) ^^ Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index dfc606e..5f538bd 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -157,7 +157,7 @@ def combine_group_jobs_list(query_data, groups_params, dimensions, metrics) groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) groups.each do |group_key, dims| dims.each do |dim_key, jobs| - groups[group_key][dim_key], _ = create_matrix(jobs) + groups[group_key][dim_key], = create_matrix(jobs) end end -- 2.23.0

1 0