- Compass-ci - mailweb.openeuler.org

[PATCH v3 lkp-tests] add email feedback for rpmbuild-pkg
by Li Ping 29 Apr '21

29 Apr '21

tests/rpmbuild-pkg: add a new input parameter whether_email [usage]: rpmbuild-pkg: whether_email: 1 when job failed, send an rpmbuild failed email. when job run success, send an rpmbuild success email. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/email.sh | 37 +++++++++++++++++++++++++++++++++++++ tests/rpmbuild-pkg | 15 +++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/lib/email.sh b/lib/email.sh index b5e56df79..0082e852d 100755 --- a/lib/email.sh +++ b/lib/email.sh @@ -96,6 +96,43 @@ Compass-CI " } +rpmbuild_report() +{ + local author_email=$(git_pretty_format "%ae") + local author_name=$(git_pretty_format "%an") + email_content="To: $author_email +Subject: [NOTIFY Compass-CI] rpmbuild report + +Dear $author_name: + + $rpmbuild_result + +Regards +Compass-CI +" +} + +rpmbuild_success_content() +{ + rpmbuild_result="Your RPM Package is successfully built. + +You can click the follow link to obtain your RPM Package: +http://api.compass-ci.openeuler.org:20012/rpm/pub/${os}-${os_version}/${compat_os}/${os_arch}/Packages + +And you can click the following link to view RPM Package build details: +http://api.compass-ci.openeuler.org:${SRV_HTTP_RESULT_PORT:-20007}$result_root" + + rpmbuild_report +} + +rpmbuild_failed_content() +{ + rpmbuild_result="Sorry to inform you that RPM Package built failed, you can click the following link to view details. +http://api.compass-ci.openeuler.org:${SRV_HTTP_RESULT_PORT:-20007}$result_root" + + rpmbuild_report +} + send_email() { local subject=$1 diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 3dd75b896..970418848 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -5,6 +5,7 @@ # - repo_addr # - custom_repo_name # - custom_repo_addr +# - whether_email # Git repo jobs only use the first two parameters, SRPMs jobs use all # user can add custom_repo_name and custom_repo_addr to setup # local repo, value can be an array @@ -12,6 +13,7 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh . $LKP_SRC/lib/rpm.sh +. $LKP_SRC/lib/email.sh : "${compat_os:=budding-openeuler}" @@ -19,6 +21,13 @@ dest_dir="/rpm/upload/${os}-${os_version}/${compat_os}" +die() +{ + [ -n "$whether_email" ] && send_email "rpmbuild_failed" + echo "$@" 1>&2 + exit 99 +} + from_git() { package_name=${upstream_repo##*/} @@ -29,6 +38,11 @@ from_git() download_upstream_repo } +git_pretty_format() +{ + curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' -d '{"git_repo": "'${upstream_dir}/${upstream_repo}.git'", "git_command": ["git-log","-n1","--pretty=format:'$1'", "'${upstream_commit}'"]}' +} + from_srpm() { [ -n "$repo_name" ] || die "repo_name is empty" @@ -143,3 +157,4 @@ add_repo build_rpm upload_rpm_pkg update_repo_mq +[ -n "$whether_email" ] && send_email "rpmbuild_success" -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: change borrow runtime 10 days to 30 days
by Cao Xueliang 29 Apr '21

29 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index bb4ef96..98b2c3d 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -439,12 +439,12 @@ class Job # only job.yaml for borrowing machine has the key: ssh_pub_key return unless @hash.has_key?("ssh_pub_key") - # the maxmum borrowing time is limited no more than 10 days. + # the maxmum borrowing time is limited no more than 30 days. # case the runtime/sleep value count beyond the limit, # it will throw error message and prevent the submit for borrowing machine. # runtime value is converted to second. - max_run_time = 10 * 24 * 3600 - error_msg = "\nMachine borrow time(runtime/sleep) cannot exceed 10 days. Consider re-borrow.\n" + max_run_time = 30 * 24 * 3600 + error_msg = "\nMachine borrow time(runtime/sleep) cannot exceed 30 days. Consider re-borrow.\n" if @hash["pp"]["sleep"].as_i? sleep_run_time = @hash["pp"]["sleep"] -- 2.23.0

1 0

[PATCH compass-ci] sparrow/8-testbox: add multi-qemu
by Liu Yinsi 29 Apr '21

29 Apr '21

run qemu testboxes for user locally deploy ci env, if the host is virtual, not run qemu. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/8-testbox/multi-qemu | 7 +++++++ sparrow/install-tiny | 1 + 2 files changed, 8 insertions(+) create mode 100755 sparrow/8-testbox/multi-qemu diff --git a/sparrow/8-testbox/multi-qemu b/sparrow/8-testbox/multi-qemu new file mode 100755 index 00000000..8a995cdb --- /dev/null +++ b/sparrow/8-testbox/multi-qemu @@ -0,0 +1,7 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +dmidecode -s system-product-name | grep -iq "virtual" && exit + +nohup $CCI_SRC/providers/multi-qemu -n vm-2p8g -c 10 -q vm-2p8g.$(arch) & diff --git a/sparrow/install-tiny b/sparrow/install-tiny index 40358a4c..a81f2d5c 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -24,3 +24,4 @@ cd $CCI_SRC/sparrow || exit 5-build/register-account 7-systemd/systemd-setup 8-testbox/multi-docker +8-testbox/multi-qemu -- 2.23.0

2 2

[PATCH lkp-tests] jobs: add borrow-30d.yaml
by Liu Yinsi 29 Apr '21

29 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- jobs/borrow-30d.yaml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 jobs/borrow-30d.yaml diff --git a/jobs/borrow-30d.yaml b/jobs/borrow-30d.yaml new file mode 100644 index 000000000..e24fd6480 --- /dev/null +++ b/jobs/borrow-30d.yaml @@ -0,0 +1,14 @@ +suite: borrow +testcase: borrow + +runtime: 30d +ssh_pub_key: <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> +sshd: +# sleep at the bottom +sleep: -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: add gem dep nokogiri for libvirt
by Xu Xijian 29 Apr '21

29 Apr '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/0-package/common | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 82c09ff..a827a65 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -8,7 +8,8 @@ public_network_ok && { gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ - gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch bunny + gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch bunny \ + nokogiri } grep -q '^lkp:' /etc/passwd || useradd -u 1090 lkp -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/0-package: add deps libxml2-devel libxslt-devel
by Xu Xijian 29 Apr '21

29 Apr '21

These 2 deps are for nokogiri, and nokogiri is dep of libvirt. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/0-package/os/centos | 2 ++ sparrow/0-package/os/openEuler | 2 ++ 2 files changed, 4 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index d4d1045..3f7dcac 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -26,6 +26,8 @@ pkgs=( curl sudo iputils + libxml2-devel + libxslt-devel ) yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5829a12..5153239 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -29,6 +29,8 @@ pkgs=( curl sudo iputils + libxml2-devel + libxslt-devel ) public_network_ok && { -- 2.23.0

1 0

[PATCH compass-ci 4/4] lib/mail_job_result.rb: fix incorrect logic
by Lu Weitao 29 Apr '21

29 Apr '21

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/mail_job_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_job_result.rb b/lib/mail_job_result.rb index 389ad26a..744b7d5c 100644 --- a/lib/mail_job_result.rb +++ b/lib/mail_job_result.rb @@ -77,7 +77,7 @@ def get_compare_result(job) commit_id = job['upstream_commit'] return nil unless base_commit && commit_id - condition_list = [{'base_commit' => base_commit}, {'upstream_commit' => commit_id}] + condition_list = [{'upstream_commit' => base_commit}, {'upstream_commit' => commit_id}] options = { :min_samples => min_samples, :no_print => true} matrices_list, suite_list = create_matrices_list(condition_list, options[:min_samples]) -- 2.23.0

1 0

[PATCH compass-ci 3/4] lib/es_query.rb: support unmatch fields
by Lu Weitao 29 Apr '21

29 Apr '21

Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/es_query.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 8c5a28fe..9d596bb2 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -15,17 +15,20 @@ class ESQuery end # Example @items: { key1 => value1, key2 => [value2, value3, ..], ...} + # Example @unmatch_items: { key1 => value1, key2 => [value2, value3, ..], ...} # means to query: key1 == value1 && (key2 in [value2, value3, ..]) - def multi_field_query(items, size: 10_000, desc_keyword: nil) + def multi_field_query(items, unmatch_items: {}, size: 10_000, desc_keyword: nil) unless items warn 'empty filter!' exit end query_fields = build_multi_field_subquery_body items + unmatch_fields = build_multi_field_subquery_body unmatch_items query = { query: { bool: { - must: query_fields + must: query_fields, + must_not: unmatch_fields } }, size: size } -- 2.23.0

1 0

[PATCH compass-ci 2/4] lib/compare_matrixes.rb cancel remove 0 changed metric
by Lu Weitao 29 Apr '21

29 Apr '21

[Why] some user are still need check 0 changed metric Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index da2c2b92..cff73cbe 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -191,7 +191,7 @@ def remove_unchanged_field(matrixes_values, suite_list) # matrixes_values.each_key do |success| matrixes_values[success].delete_if do |field| - if suite_list.any? { |suite| field.start_with?(suite) } + if success matrixes_values[success][field][:changed] = true end -- 2.23.0

1 0

[PATCH compass-ci 1/4] sbin/compare: cancel fill 0 to matrix
by Lu Weitao 29 Apr '21

29 Apr '21

[Why] matrix => { iperf.xxx => [100, 80, 0], ... } like above, if a stat_metric fill 0, the avg and stddev will be incorrect Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 13 +++++++++---- lib/matrix2.rb | 13 +------------ 2 files changed, 10 insertions(+), 16 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index fef5714a..da2c2b92 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -129,10 +129,15 @@ def get_values_by_field(matrixes_list, field, matrixes_size, success, options) # values = {} matrixes_list.length.times do |index| - value_list = fill_missing_with_zeros( - matrixes_list[index][field], - matrixes_size[index] - ) + value_list = matrixes_list[index][field] + unless value_list + if success + value_list = [0] unless value_list + else + value_list = [1] unless value_list + end + end + values[index] = get_values(value_list, success) next if index.zero? diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 2fd63243..82ed9a7a 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -62,16 +62,6 @@ def create_stats(result_root) # stats end -def samples_fill_missing_zeros(value, size) - value.concat([0] * (size - value.size)) -end - -def matrix_fill_miss_zeros(matrix, col_size) - matrix.each_value do |value| - samples_fill_missing_zeros(value, col_size) - end -end - # input: job_list # return: matrix of Hash(String, Array(Number)) # Eg: matrix: { @@ -94,8 +84,7 @@ def create_matrix(job_list) matrix[key] << value end end - col_size = job_list.size - matrix_fill_miss_zeros(matrix, col_size) + return matrix, suites end -- 2.23.0

1 0