- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] jobs: fix to report errors when submit jobs
by Liu Shaofei 22 Dec '20

22 Dec '20

[why] /home/liushaofei/lkp-tests/lib/job.rb:257:in `block (2 levels) in load': undefined method `start_with?' for :"# pkgbuild_repo examples":Symbol (NoMethodError) ERROR -- /home/liushaofei/lkp-tests/jobs/build-pkg.yaml: undefined method `start_with?' for :"# pkgbuild_repo examples":Symbol Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- jobs/build-pkg.yaml | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/jobs/build-pkg.yaml b/jobs/build-pkg.yaml index b17fd2ade..7a47bd724 100644 --- a/jobs/build-pkg.yaml +++ b/jobs/build-pkg.yaml @@ -1,9 +1,8 @@ -suite: build-pkg -category: functional - -# pkgbuild_repo examples +# the field of "pkgbuild_repo" refers to the path of PKGBUILD, the examples are as follow: # pkgbuild_repo: archlinux/aur-j/java8-openjdk # pkgbuild_repo: archlinux/community//keepalived/trunk # pkgbuild_repo: archlinux/packages//prometheus-node-exporter/trunk +suite: build-pkg +category: functional build-pkg: -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker/buildall: skip ssh-r
by Wu Zhende 22 Dec '20

22 Dec '20

Do not restart the ssh-r container randomly. Otherwise, the established connection will be disconnected, and the requested hosts can't be logged in. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/4-docker/buildall | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 98d3ce4..32801d1 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -31,6 +31,7 @@ do_one() mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container + [ "$container_name" == 'ssh-r' ] && exit if [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install -- 2.23.0

1 0

[PATCH compass-ci] lib/constants: MAIL_HOST=>SEND_MAIL_HOST, MAIL_PORT=>SEND_MAIL_PORT
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/constants.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/constants.rb b/lib/constants.rb index 3d1ef1d..f0938f2 100644 --- a/lib/constants.rb +++ b/lib/constants.rb @@ -8,8 +8,8 @@ config = cci_defaults ES_HOST = config['ES_HOST'] || '172.17.0.1' ES_PORT = config['ES_PORT'] || 9200 -MAIL_HOST = config['MAIL_HOST'] || '172.17.0.1' -MAIL_PORT = config['MAIL_PORT'] || 11311 +MAIL_HOST = config['SEND_MAIL_HOST'] || '172.17.0.1' +MAIL_PORT = config['SEND_MAIL_PORT'] || 49000 SRV_HTTP_HOST = config['SRV_HTTP_HOST'] || ENV['SRV_HTTP_HOST'] || '172.17.0.1' SRV_HTTP_PORT = config['SRV_HTTP_PORT'] || ENV['SRV_HTTP_PORT'] || 11300 -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib: add assign_account_client
by Luan Shengde 22 Dec '20

22 Dec '20

add assign_account_client for code calling to assign account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/assign_account_client.rb | 83 ++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100755 lib/assign_account_client.rb diff --git a/lib/assign_account_client.rb b/lib/assign_account_client.rb new file mode 100755 index 0000000..d7e0623 --- /dev/null +++ b/lib/assign_account_client.rb @@ -0,0 +1,83 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require 'set' +require 'optparse' +require_relative '../container/defconfig' +require_relative 'es_client' +require_relative '../container/mail-robot/lib/assign-account-email' + +names = Set.new %w[ + JUMPER_HOST + JUMPER_PORT + SEND_MAIL_HOST + SEND_MAIL_PORT +] + +defaults = relevant_defaults(names) +JUMPER_HOST = defaults['JUMPER_HOST'] +JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 +SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 +LAB = ENV['lab'] + +# used for other codes calling to assign account for user +class AutoAssignAccount + def initialize(user_info) + @my_info = user_info + + @my_info_es = {} + end + + def update_from_es + account_infos = ESQuery.new(index: 'accounts') + my_account_info = account_infos.query_by_id(@my_info['my_email']) || {} + + my_account_info.update @my_info + @my_info.update my_account_info + @my_info['my_ssh_pubkey'] = [] unless my_account_info['my_ssh_pubkey'] + + @my_info['my_token'] = %x(uuidgen).chomp if @my_info['my_login_name'].nil? + end + + def apply_account + apply_info = {} + + apply_info.update @my_info + apply_info['enable_login'] = true + apply_info['is_update_account'] = true unless apply_info['my_login_name'].nil? + apply_info['lab'] = LAB + + account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') + JSON.parse account_info_str + end + + def update_my_info_from_account_info + account_info = apply_account + + @my_info['my_login_name'] = account_info['my_login_name'] + @my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'] unless account_info['my_jumper_pubkey'].nil? + end + + def store_account_info + es = ESClient.new(index: 'accounts') + es.put_source_by_id(@my_info['my_email'], @my_info) + end + + def send_mail + message = build_apply_account_email(@my_info) + %x(curl -XPOST "#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text" -d "#{message}") + end + + def send_account + update_from_es + update_my_info_from_account_info + + store_account_info + send_mail + end +end -- 2.23.0

1 1

[PATCH compass-ci 2/2] lib/mail-post: add send_mail_encode function
by Luan Shengde 22 Dec '20

22 Dec '20

[why] the http protocol will parse in fail if the request data has special characters. or it's hard to analysis the data if encoded or not on the send mail service. add stand alone interface for it Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail-post.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/lib/mail-post.rb b/lib/mail-post.rb index df4d255..ac455fa 100644 --- a/lib/mail-post.rb +++ b/lib/mail-post.rb @@ -8,6 +8,7 @@ require 'json' require 'yaml' require 'open3' require 'mail' +require 'base64' set :bind, '0.0.0.0' set :port, ENV['SEND_MAIL_PORT'] @@ -37,6 +38,19 @@ post '/send_mail_text' do send_mail(mail_info) end +post '/send_mail_encode' do + data_decode = Base64.decode64(request.body.read) + data = Mail.read_from_string(data_decode) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + def check_send_mail(mail_info) raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? -- 2.23.0

1 0

[PATCH lkp-tests] tests/cci-makepkg: change to logic condition
by Liu Shaofei 22 Dec '20

22 Dec '20

Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/cci-makepkg | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a50a40c20..d48e8495e 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -41,6 +41,8 @@ curl_pkgfile() download_pkgfile() { + [ -n "$PKGBUILD_TAG" ] || return + local pkg_tag_file="PKGBUILD-$PKGBUILD_TAG" curl_pkgfile "pkg/$benchmark" "$pkg_tag_file" @@ -93,7 +95,7 @@ centos_update_gcc() source /opt/rh/devtoolset-8/enable } -[ -n "$PKGBUILD_TAG" ] && download_pkgfile +download_pkgfile distro_install_depends lkp-dev -- 2.23.0

1 0

[PATCH lkp-tests 2/2] lib/env.sh: refactor set-env
by Wei Jihui 22 Dec '20

22 Dec '20

move sbin/set-env.sh functions to lib/env.sh delete sbin/set-env.sh do set-env at sbin/install-dependencies Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/env.sh | 30 ++++++++++++++++++++++++++ sbin/install-dependencies.sh | 10 ++++++++- sbin/set-env.sh | 42 ------------------------------------ 3 files changed, 39 insertions(+), 43 deletions(-) delete mode 100755 sbin/set-env.sh diff --git a/lib/env.sh b/lib/env.sh index 8ac42a83..9f37d8cc 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -125,3 +125,33 @@ shell_profile() [ -f "$profile" ] || touch $profile echo $profile } + +write_shell_profile() +{ + shell_profile_file=$(shell_profile) + if [ $# -gt 0 ]; then + echo "$@" >> $shell_profile_file + fi + + source $shell_profile_file +} + +write_host() +{ + if is_system "Linux"; then + nr_cpu=$(nproc) + memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') + else + nr_cpu=$(sysctl -n hw.logicalcpu) + memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') + fi + memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') + memory=$(echo $memory_new | awk '{print int($0)+1}')G + + cat > hosts/$(hostname) <<-EOF + nr_cpu: $nr_cpu + memory: $memory + hdd_partitions: + ssd_partitions: + EOF +} diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 6263dc91..8a4b29cd 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -4,7 +4,6 @@ SCRIPT_DIR=$(dirname $(realpath $0)) PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh -. $PROJECT_DIR/sbin/set-env.sh # choose install function base on DISTRIBUTION linux_dep() @@ -48,4 +47,13 @@ run() fi } +set_env() +{ + write_host + write_shell_profile "export LKP_SRC=$PWD" + write_shell_profile "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" + source $(shell_profile) +} + +set_env run diff --git a/sbin/set-env.sh b/sbin/set-env.sh deleted file mode 100755 index 8234eaf8..00000000 --- a/sbin/set-env.sh +++ /dev/null @@ -1,42 +0,0 @@ -#!/bin/sh - -SCRIPT_DIR=$(dirname $(realpath $0)) -PROJECT_DIR=$(dirname $SCRIPT_DIR) - -. $PROJECT_DIR/lib/env.sh - -write_shell_profile() -{ - shell_profile_file=$(shell_profile) - if [ $# -gt 0 ]; then - echo $@ >> $shell_profile_file - else - echo "export LKP_SRC=$PWD" >> $shell_profile_file - echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $shell_profile_file - fi - - source $shell_profile_file -} - -write_host() -{ - if is_system "Linux"; then - nr_cpu=$(nproc) - memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') - else - nr_cpu=$(sysctl -n hw.logicalcpu) - memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') - fi - memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') - memory=$(echo $memory_new | awk '{print int($0)+1}')G - - cat > hosts/$(hostname) <<-EOF - nr_cpu: $nr_cpu - memory: $memory - hdd_partitions: - ssd_partitions: - EOF -} - -write_shell_profile -write_host -- 2.23.0

1 0

[PATCH lkp-tests 1/2] sbin/install-dependencies: adjust file format
by Wei Jihui 22 Dec '20

22 Dec '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- sbin/install-dependencies.sh | 39 ++++++++++++++++++------------------ 1 file changed, 19 insertions(+), 20 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 3a206134..6263dc91 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -6,28 +6,27 @@ PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh . $PROJECT_DIR/sbin/set-env.sh -# choose install function base on -# DISTRIBUTION +# choose install function base on DISTRIBUTION linux_dep() { -source /etc/os-release -case $ID in -ubuntu|debian) - export DEBIAN_FRONTEND=noninteractive - apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client - ;; -openEuler|fedora|rhel|centos) - if type dnf > /dev/null 2>&1; then - dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - else - yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git - fi - sudo gem install -f git activesupport rest-client - ;; -*) - echo "$ID not support! please install dependencies manually." - ;; -esac + source /etc/os-release + case $ID in + ubuntu|debian) + export DEBIAN_FRONTEND=noninteractive + apt-get install -yqm ruby-git ruby-activesupport ruby-rest-client + ;; + openEuler|fedora|rhel|centos) + if type dnf > /dev/null 2>&1; then + dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + else + yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git + fi + sudo gem install -f git activesupport rest-client + ;; + *) + echo "$ID not support! please install dependencies manually." + ;; + esac } -- 2.23.0

1 0

[PATCH compass-ci] compare_matrixes.rb : reduce compare time
by Li Yuanchao 22 Dec '20

22 Dec '20

When there is multi object group, don't use one object group to compare. Cause one object groups are too many, they spend too much time. And they are not what we want. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/compare_matrixes.rb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 6e4c2aa..21bd5c9 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -390,7 +390,10 @@ end def compare_group_matrices(group_matrices, suites_hash, options) result_str = '' group_matrices_array = sort_by_matrix_size(group_matrices) + have_multi_member = group_matrices_array[0][1].size > 1 group_matrices_array.each do |matrice_kv| + next if have_multi_member && matrice_kv[1].size < 2 + result_str += get_matrix_str(matrice_kv[0], matrice_kv[1], suites_hash[matrice_kv[0]], options) end result_str -- 2.23.0

1 0

[PATCH lkp-tests] jobs/bisect.yaml: dc-1g--delimiter => dc-1g
by Cao Xueliang 22 Dec '20

22 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 5fab30bfc..11ffc7c05 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -1,4 +1,4 @@ suite: bisect docker_image: delimiter:latest -testbox: dc-1g--delimiter +testbox: dc-1g bisect: -- 2.23.0

1 0