December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 16 Dec '20

16 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100755 index 0000000..c79b054 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +mail_conf () +{ + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf +fetchmail -d 100 -- 2.23.0

1 0

[PATCH compass-ci] web_backend.rb: fix error message
by Li Yuanchao 16 Dec '20

16 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 94883a3..07e2e9e 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -457,7 +457,7 @@ def query_testboxes body = testboxes_body rescue StandardError => e warn e.message - return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get repos error'] + return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get testboxes error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] end -- 2.23.0

1 0

[PATCH compass-ci] result-webdav/start: add REDIS_PORT and REDIS_HOST env
by Cui Yili 16 Dec '20

16 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/start | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/container/result-webdav/start b/container/result-webdav/start index d7b1097..bbbbb24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -5,6 +5,9 @@ . $CCI_SRC/container/defconfig.sh +: ${REDIS_HOST:=172.17.0.1} +: ${REDIS_PORT:=6379} + docker_rm result-webdav cmd=( @@ -12,6 +15,8 @@ cmd=( --name result-webdav --restart=always -p 3080:3080 + -e REDIS_HOST=$REDIS_HOST + -e REDIS_PORT=$REDIS_PORT -v $PWD/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -- 2.23.0

1 0

[PATCH compass-ci] assign-account: generate ssh keys for user by default
by Luan Shengde 16 Dec '20

16 Dec '20

[why] request is: when user submit jobs, the ssh keys are already there. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 10 ++-------- container/assign-account/get_account_info.rb | 6 ++---- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 766e1b3..21285e7 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -61,7 +61,6 @@ stdin_info = {} email_info = {} my_info_es = {} conf_info = { - 'gen_sshkey' => false, 'enable_login' => true, 'is_update_account' => false } @@ -83,13 +82,12 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' - opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" opts.banner += " -n is required when assigning account with -e\n" opts.banner += " -s is optional when use -e\n" - opts.banner += " -g is optional, used to generate sshkey for user\n" opts.banner += " -u is required when updating an account\n" opts.banner += ' -l is optional, used to enable/disable login permission' @@ -141,10 +139,6 @@ options = OptionParser.new do |opts| init_info(mail_content, email_info, my_info, my_info_es) end - opts.on('-g', '--gen-sshkey', 'generate jumper rsa public/private key and return pubkey') do - conf_info['gen_sshkey'] = true - end - opts.on('-u', '--update', 'updata configurations') do conf_info['is_update_account'] = true end diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 9a7acda..5096734 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -64,7 +64,7 @@ class AccountStorage def read_account_info available_dir = File.join(@account_dir, 'available-users') - files = Dir.entries(available_dir) + files = Dir.open(available_dir).to_a files -= ['.', '..'] message = 'no more available users' @@ -112,9 +112,7 @@ class AccountStorage config_authorized_key(login_name) config_yaml(login_name) permit_login_config(login_name) - if @data.key?('gen_sshkey') && @data['gen_sshkey'] - my_jumper_pubkey = generate_ssh_key(login_name) - end + my_jumper_pubkey = generate_ssh_key(login_name) jumper_account_info = { 'my_login_name' => login_name, -- 2.23.0

1 0

[PATCH compass-ci] src/lib: add common deps packages
by Zhang Yu 16 Dec '20

16 Dec '20

[why] when run job with initramfs, some command like time and perf are not found [how] add depends package of initramfs-common to install the common package Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- src/lib/job.cr | 1 + 1 file changed, 1 insertion(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..1c30f38 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -497,6 +497,7 @@ class Job private def get_depends_initrd(program_params, initrd_deps_arr, initrd_pkg_arr) initrd_http_prefix = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + program_params["initramfs-common"] = JSON::Any.new("") if os_mount == "initramfs" program_params.keys.each do |program| if program =~ /^(.*)-\d+$/ -- 2.23.0

2 2

[PATCH v3 compass-ci 6/6] fetch-mail: start file
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/start | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 container/fetch-mail/start diff --git a/container/fetch-mail/start b/container/fetch-mail/start new file mode 100755 index 0000000..45f3910 --- /dev/null +++ b/container/fetch-mail/start @@ -0,0 +1,36 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'fetch-mail' + +names = Set.new %w[ + FETCH_MAIL_ADDRESS + FETCH_MAIL_AUTH_CODE + FETCH_MAIL_DIR + FETCH_MAIL_BOX +] + +defaults = relevant_defaults(names) + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + -u mailer + --name=fetch-mail + -d +] + env + %W[ + -v #{defaults['FETCH_MAIL_DIR']}:#{defaults['FETCH_MAIL_DIR']}:rw + -w /home/mailer/ + fetch-mail +] + +cmd += ['sh', '-c', 'sh run.sh'] + +system(*cmd) -- 2.23.0

2 2

[PATCH compass-ci] scheduler: move and refactor function 'auto_submit_idle_job'
by Ren Wen 16 Dec '20

16 Dec '20

Idle job will be submitted when there's no job to consume, so move this function to 'find_job_boot.cr'. Refactor this function for subqueue: for idle job, subqueue is 'idle'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 14 -------------- src/scheduler/find_job_boot.cr | 7 +++++++ 2 files changed, 7 insertions(+), 14 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 10a51ab..a1ef28e 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -67,20 +67,6 @@ class Sched @redis.hash_del("sched/host2queues", hostname) end - def auto_submit_idle_job(tbox_group) - full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" - extra_job_fields = [ - "idle_job=true", - "MASTER_FLUENTD_HOST=#{ENV["MASTER_FLUENTD_HOST"]}", - "MASTER_FLUENTD_PORT=#{ENV["MASTER_FLUENTD_PORT"]}", - ] - - Jobfile::Operate.auto_submit_job( - full_path_patterns, - "testbox: #{tbox_group}", - extra_job_fields) if Dir.glob(full_path_patterns).size > 0 - end - def update_tbox_wtmp testbox = "" hash = Hash(String, String).new diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index fcc7ea9..d5ef505 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -230,4 +230,11 @@ class Sched return job end + + private def auto_submit_idle_job(testbox) + full_path_patterns = "#{CCI_REPOS}/#{LAB_REPO}/allot/idle/#{testbox}/*.yaml" + fields = ["testbox=#{testbox}", "subqueue=idle"] + + Jobfile::Operate.auto_submit_job(full_path_patterns, fields) if Dir.glob(full_path_patterns).size > 0 + end end -- 2.23.0

1 0

[PATCH lkp-tests 3/3] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 16 Dec '20

16 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index df518727..e3bf31d9 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -126,7 +126,7 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH lkp-tests 2/3] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 16 Dec '20

16 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dbde3e54..df518727 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -125,8 +125,8 @@ build_source_pkg() [ -n "$PKGBUILD_TAG" ] && replace_source "PKGBUILD-$PKGBUILD_TAG" cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: improve how to get filenames
by Ren Wen 16 Dec '20

16 Dec '20

Dir.children returns filenames except for “.” and “..”. Signed-off-by: Ren Wen <15991987063(a)163.com> --- lib/git_mirror.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9f254f5..dfcc016 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,7 +130,7 @@ class MirrorMain def traverse_repodir(repodir) if File.directory? repodir load_defaults(repodir) - entry_list = Dir.entries(repodir) - Array['.', '..', 'DEFAULTS', '.ignore', '.git'] + entry_list = Dir.children(repodir) - Array['DEFAULTS', '.ignore', '.git'] entry_list.each do |entry| traverse_repodir("#{repodir}/#{entry}") end -- 2.23.0

1 0