- Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..5bd1a33 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:latest +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 2002 mailer +RUN yum update -y +RUN yum install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer +RUN chmod +x /home/mailer/run.sh -- 2.23.0

1 0

[PATCH v4 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 16 Dec '20

16 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100755 index 0000000..c79b054 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +mail_conf () +{ + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf +fetchmail -d 100 -- 2.23.0

1 0

[PATCH compass-ci] web_backend.rb: fix error message
by Li Yuanchao 16 Dec '20

16 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 94883a3..07e2e9e 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -457,7 +457,7 @@ def query_testboxes body = testboxes_body rescue StandardError => e warn e.message - return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get repos error'] + return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get testboxes error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] end -- 2.23.0

1 0

[PATCH compass-ci] result-webdav/start: add REDIS_PORT and REDIS_HOST env
by Cui Yili 16 Dec '20

16 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/start | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/container/result-webdav/start b/container/result-webdav/start index d7b1097..bbbbb24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -5,6 +5,9 @@ . $CCI_SRC/container/defconfig.sh +: ${REDIS_HOST:=172.17.0.1} +: ${REDIS_PORT:=6379} + docker_rm result-webdav cmd=( @@ -12,6 +15,8 @@ cmd=( --name result-webdav --restart=always -p 3080:3080 + -e REDIS_HOST=$REDIS_HOST + -e REDIS_PORT=$REDIS_PORT -v $PWD/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -- 2.23.0

1 0

[PATCH compass-ci] assign-account: generate ssh keys for user by default
by Luan Shengde 16 Dec '20

16 Dec '20

[why] request is: when user submit jobs, the ssh keys are already there. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 10 ++-------- container/assign-account/get_account_info.rb | 6 ++---- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 766e1b3..21285e7 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -61,7 +61,6 @@ stdin_info = {} email_info = {} my_info_es = {} conf_info = { - 'gen_sshkey' => false, 'enable_login' => true, 'is_update_account' => false } @@ -83,13 +82,12 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' - opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" opts.banner += " -n is required when assigning account with -e\n" opts.banner += " -s is optional when use -e\n" - opts.banner += " -g is optional, used to generate sshkey for user\n" opts.banner += " -u is required when updating an account\n" opts.banner += ' -l is optional, used to enable/disable login permission' @@ -141,10 +139,6 @@ options = OptionParser.new do |opts| init_info(mail_content, email_info, my_info, my_info_es) end - opts.on('-g', '--gen-sshkey', 'generate jumper rsa public/private key and return pubkey') do - conf_info['gen_sshkey'] = true - end - opts.on('-u', '--update', 'updata configurations') do conf_info['is_update_account'] = true end diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 9a7acda..5096734 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -64,7 +64,7 @@ class AccountStorage def read_account_info available_dir = File.join(@account_dir, 'available-users') - files = Dir.entries(available_dir) + files = Dir.open(available_dir).to_a files -= ['.', '..'] message = 'no more available users' @@ -112,9 +112,7 @@ class AccountStorage config_authorized_key(login_name) config_yaml(login_name) permit_login_config(login_name) - if @data.key?('gen_sshkey') && @data['gen_sshkey'] - my_jumper_pubkey = generate_ssh_key(login_name) - end + my_jumper_pubkey = generate_ssh_key(login_name) jumper_account_info = { 'my_login_name' => login_name, -- 2.23.0

1 0

[PATCH compass-ci] src/lib: add common deps packages
by Zhang Yu 16 Dec '20

16 Dec '20

[why] when run job with initramfs, some command like time and perf are not found [how] add depends package of initramfs-common to install the common package Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- src/lib/job.cr | 1 + 1 file changed, 1 insertion(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..1c30f38 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -497,6 +497,7 @@ class Job private def get_depends_initrd(program_params, initrd_deps_arr, initrd_pkg_arr) initrd_http_prefix = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + program_params["initramfs-common"] = JSON::Any.new("") if os_mount == "initramfs" program_params.keys.each do |program| if program =~ /^(.*)-\d+$/ -- 2.23.0

2 2

[PATCH v3 compass-ci 6/6] fetch-mail: start file
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/start | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 container/fetch-mail/start diff --git a/container/fetch-mail/start b/container/fetch-mail/start new file mode 100755 index 0000000..45f3910 --- /dev/null +++ b/container/fetch-mail/start @@ -0,0 +1,36 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'fetch-mail' + +names = Set.new %w[ + FETCH_MAIL_ADDRESS + FETCH_MAIL_AUTH_CODE + FETCH_MAIL_DIR + FETCH_MAIL_BOX +] + +defaults = relevant_defaults(names) + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + -u mailer + --name=fetch-mail + -d +] + env + %W[ + -v #{defaults['FETCH_MAIL_DIR']}:#{defaults['FETCH_MAIL_DIR']}:rw + -w /home/mailer/ + fetch-mail +] + +cmd += ['sh', '-c', 'sh run.sh'] + +system(*cmd) -- 2.23.0

2 2

[PATCH compass-ci] scheduler: move and refactor function 'auto_submit_idle_job'
by Ren Wen 16 Dec '20

16 Dec '20

Idle job will be submitted when there's no job to consume, so move this function to 'find_job_boot.cr'. Refactor this function for subqueue: for idle job, subqueue is 'idle'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 14 -------------- src/scheduler/find_job_boot.cr | 7 +++++++ 2 files changed, 7 insertions(+), 14 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 10a51ab..a1ef28e 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -67,20 +67,6 @@ class Sched @redis.hash_del("sched/host2queues", hostname) end - def auto_submit_idle_job(tbox_group) - full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" - extra_job_fields = [ - "idle_job=true", - "MASTER_FLUENTD_HOST=#{ENV["MASTER_FLUENTD_HOST"]}", - "MASTER_FLUENTD_PORT=#{ENV["MASTER_FLUENTD_PORT"]}", - ] - - Jobfile::Operate.auto_submit_job( - full_path_patterns, - "testbox: #{tbox_group}", - extra_job_fields) if Dir.glob(full_path_patterns).size > 0 - end - def update_tbox_wtmp testbox = "" hash = Hash(String, String).new diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index fcc7ea9..d5ef505 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -230,4 +230,11 @@ class Sched return job end + + private def auto_submit_idle_job(testbox) + full_path_patterns = "#{CCI_REPOS}/#{LAB_REPO}/allot/idle/#{testbox}/*.yaml" + fields = ["testbox=#{testbox}", "subqueue=idle"] + + Jobfile::Operate.auto_submit_job(full_path_patterns, fields) if Dir.glob(full_path_patterns).size > 0 + end end -- 2.23.0

1 0

[PATCH lkp-tests 3/3] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 16 Dec '20

16 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index df518727..e3bf31d9 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -126,7 +126,7 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH lkp-tests 2/3] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 16 Dec '20

16 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dbde3e54..df518727 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -125,8 +125,8 @@ build_source_pkg() [ -n "$PKGBUILD_TAG" ] && replace_source "PKGBUILD-$PKGBUILD_TAG" cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: improve how to get filenames
by Ren Wen 16 Dec '20

16 Dec '20

Dir.children returns filenames except for “.” and “..”. Signed-off-by: Ren Wen <15991987063(a)163.com> --- lib/git_mirror.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9f254f5..dfcc016 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,7 +130,7 @@ class MirrorMain def traverse_repodir(repodir) if File.directory? repodir load_defaults(repodir) - entry_list = Dir.entries(repodir) - Array['.', '..', 'DEFAULTS', '.ignore', '.git'] + entry_list = Dir.children(repodir) - Array['DEFAULTS', '.ignore', '.git'] entry_list.each do |entry| traverse_repodir("#{repodir}/#{entry}") end -- 2.23.0

1 0

[PATCH lkp-tests] tests: fix error when running host-info in vm
by Zhang Yu 16 Dec '20

16 Dec '20

Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 0ab8e6c97..7513d6073 100755 --- a/tests/host-info +++ b/tests/host-info @@ -102,8 +102,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX - is_virt && return - is_docker && return + [[ -e /dev/ipmi0 ]] || return + [[ -e /dev/ipmi/0 ]] || return + [[ -e /dev/ipmidev/0 ]] || return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip: $ip" -- 2.23.0

3 2

[PATCH v2 compass-ci] doc/job: add SCHED_HOST, SCHED_PORT description
by Hu Xuejiao 16 Dec '20

16 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- doc/job/SCHED_HOST-SCHED_PORT.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/job/SCHED_HOST-SCHED_PORT.md diff --git a/doc/job/SCHED_HOST-SCHED_PORT.md b/doc/job/SCHED_HOST-SCHED_PORT.md new file mode 100644 index 0000000..74c628a --- /dev/null +++ b/doc/job/SCHED_HOST-SCHED_PORT.md @@ -0,0 +1,15 @@ +# SCHED_HOST, SCHED_PORT +Meaning: +- The job task depends on scheduler service, which will distribute tasks to test + machine. All tasks will run in test machine. +- `Scheduler` is one of our service, `SCHED` is the abbreviation of `Scheduler` + these variables mean: + SCHED_HOST: the scheduler service ip. + SCHED_PORT: the scheduler service port. + +Usage example: + +```$LKP_SRC/include/lab/z9 +SCHED_HOST: api.compass-ci.openeuler.org +SCHED_PORT: 11306 +``` -- 2.23.0

4 3

[PATCH v3 compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..7be6b21 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 2002 mailer +RUN yum update -y +RUN yum install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer -- 2.23.0

2 2

[PATCH v3 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 16 Dec '20

16 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100755 index 0000000..d7f9f38 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +function mail_conf { + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf + +fetchmail -d 100 -- 2.23.0

2 2

[PATCH compass-ci] git_mirror.rb: ignore all hidden files when load upstream-repos
by Li Yuanchao 16 Dec '20

16 Dec '20

When vim file, there would be a hidden file, which is not yaml like. Such files should be ignored when load upstream-repos. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index dee0285..389b185 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,6 +130,8 @@ class MirrorMain load_defaults(repodir) entry_list = Dir.entries(repodir) - Array['.', '..', 'DEFAULTS', '.ignore', '.git'] entry_list.each do |entry| + next if entry.start_with? '.' + traverse_repodir("#{repodir}/#{entry}") end else -- 2.23.0

1 0

[PATCH compass-ci] scheduler: refactor function 'auto_submit_job'
by Ren Wen 16 Dec '20

16 Dec '20

Now can use an array contains all fields you want to add. Example: auto_submit_job("job.yaml", ["field01=value01", "field02=value02", ...]) Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 2 +- src/scheduler/jobfile_operate.cr | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 5a11523..fcc7ea9 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -33,7 +33,7 @@ class Sched Jobfile::Operate.auto_submit_job( "#{ENV["LKP_SRC"]}/jobs/host-info.yaml", - "testbox: #{host}") + ["testbox=#{host}"]) end def rand_queues(queues) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index df79d3b..65d9b17 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -61,10 +61,11 @@ module Jobfile::Operate return target_path end - def self.auto_submit_job(job_file, override_parameter, other_parameters = nil) + # *fields* should be: ["field01=value01", "field02=value02", ...] + def self.auto_submit_job(job_file, fields : Array(String) = ["testbox=vm-2p8g"]) cmd = "#{ENV["LKP_SRC"]}/sbin/submit " - cmd += other_parameters.join(" ") if other_parameters - cmd += " -s '#{override_parameter}' #{job_file}" + cmd += "#{job_file} " + cmd += fields.join(" ") puts `#{cmd}` end end -- 2.23.0

1 0

[PATCH lkp-tests] lib/upload: change the key of upload results
by Wei Jihui 16 Dec '20

16 Dec '20

the key of upload results is changed from access_key to job id Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job-init.sh | 2 +- lib/upload.sh | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/lib/job-init.sh b/lib/job-init.sh index 40534af3..dc4a65d9 100755 --- a/lib/job-init.sh +++ b/lib/job-init.sh @@ -55,7 +55,7 @@ setup_result_service() supports_raw_upload && result_service=raw_upload && return - is_docker || [ -n "$access_key" ] || { + is_docker || [ -n "$id" ] || { if [ "$os_mount" = 'nfs' ] || [ -z "$os_mount" ]; then supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return elif [ "$os_mount" = 'cifs' ] || [ -z "$os_mount" ]; then diff --git a/lib/upload.sh b/lib/upload.sh index a5daceda..0115e350 100755 --- a/lib/upload.sh +++ b/lib/upload.sh @@ -78,9 +78,9 @@ upload_one_curl() ( cd $(dirname "$1") dir=$(basename "$1") - if [ -n "$access_key" ]; then - find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; - find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; + if [ -n "$id" ]; then + find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "JOBID=$id" \; + find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "JOBID=$id" \; else find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" \; @@ -88,8 +88,8 @@ upload_one_curl() ) else [ -s "$src" ] || return - if [ -n "$$access_key" ]; then - curl -sSf -T "$src" $http_url/ --cookie "ACCESSKEY=$access_key" + if [ -n "$id" ]; then + curl -sSf -T "$src" $http_url/ --cookie "JOBID=$id" else curl -sSf -T "$src" $http_url/ fi @@ -111,8 +111,8 @@ upload_files_curl() for dir in $(echo $target_directory | tr '/' ' ') do job_result_root=$job_result_root/$dir/ - if [ -n "$$access_key" ]; then - curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "ACCESSKEY=$access_key" >/dev/null + if [ -n "$id" ]; then + curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "JOBID=$id" >/dev/null else curl -sSf -X MKCOL "$http_host$job_result_root" >/dev/null fi @@ -178,12 +178,12 @@ upload_files() upload_files_copy "$@" return } - if has_cmd rsync && has_rsync_server && [ -z "$access_key" ]; then + if has_cmd rsync && has_rsync_server && [ -z "$id" ]; then upload_files_rsync "$@" return fi - if has_cmd lftp && [ -z "$access_key" ]; then + if has_cmd lftp && [ -z "$id" ]; then upload_files_lftp "$@" return fi -- 2.23.0

1 0

为什么你一定要参加 openEuler Summit 的 TC、Maintainer、SIG 组工作会议
by public openEuler 16 Dec '20

16 Dec '20

嗨，各位 openEuler 社区的开发者：很高兴能够向你们汇报 openEuler Summit 的会议进度。对于各位来说，本周最重要的事情是 TC、Maintainer和SIG 组开放工作会议的议程已经上线了。 *为什么你一定要参与 TC、Maintainer、SIG组开放工作会议？* 1. 你可以清楚的了解 openEuler 社区的运作模式 2. 你可以和 TC、Maintainer、SIG 的成员共同探讨openEuler 社区的下一个版本的计划和技术路线 3. 可以跟社区里的技术大佬进行面对面交流。 4. 更好的规划自己在社区中的成长路线 *会议时间和房间号：* TC 开放工作会议 Maintainer 开放工作会议 SIG 组开放工作会议时间 12/24 13:00-14:30 12/24 15:00-17:00 12/25 13:00-15:30 房间号悦府6和悦府7 悦府6和悦府7 大宴会厅B *TC **开放工作会议议程：* 1. 讨论 openEuler 支持 App Stream 机制 2. 申请成立智能运维 SIG 3. 讨论从 Gitee issue 整体切换到bugzilla 4. 讨论2021年 TC 合作的开展方式 5. …… *Maintainer **开放工作会议议程* 1. 开场 & 颁发 Maintainer 纪念徽章 2. openEuler 社区运作分析 3. 我在 openEuler 社区担任 Maintainer 的那些事儿 4. 自由问答 5. 总结 *参与本次 SIG 组开放工作会议的 SIG 组：* 1. sig-DDE 2. sig-UKUI 3. sig-Ha 4. sig-arrch32 5. A-Tune 6. iSulad & Container 7. sig-ai-bitdata 8. sig-security-facility 9. sig-confidential-computing 10. Compiler 11. Doc 12. Infrastructure 13. security-committee 14. sig-Compatibility-Infra 15. sig-QA 16. sig-release-management 17. Kernel 18. Virt *openEuler Summit **专属伴手礼：* [image: image.png] *openEuler **卫衣* [image: image.png] *openEuler Maintainer **纪念徽章* *欢迎通过openEuler 社区开发者专属的报名渠道报名：* https://etherpad.openeuler.org/p/openEuler-Summit-2020 openEuler Summit 官网： https://openeuler.org/zh/interaction/summit-list/

1 0

[PATCH v2 lkp-tests] tests: fix error when running host-info in vm or docker
by Zhang Yu 16 Dec '20

16 Dec '20

[why] Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory [how] Run ipmi command only on physical machines Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/host-info b/tests/host-info index 84f796dcb..6ac20c120 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,5 +1,7 @@ #!/bin/bash +. $LKP_SRC/lib/env.sh + get_memory() { memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') @@ -101,8 +103,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX + is_virt || is_docker && return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') - [ -n "$ip" ] && echo "ipmi_ip:" $ip + [ -n "$ip" ] && echo "ipmi_ip: $ip" } -- 2.23.0

3 5

[PATCH v2 lkp-tests] lib/env.sh: fix the check result of is_virt
by Zhang Yu 16 Dec '20

16 Dec '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/env.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/env.sh b/lib/env.sh index 78cfd6f57..fc34aaa8d 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -69,7 +69,8 @@ is_virt() # run as root [ -n "$(virt-what)" ] else - grep -q -w hypervisor /proc/cpuinfo + grep -q -w hypervisor /proc/cpuinfo && return 1 + return 0 fi } -- 2.23.0

2 3

[PATCH compass-ci] git_mirror.rb: gitclone.com does not works well
by Li Yuanchao 16 Dec '20

16 Dec '20

First, repos clone from gitclone.com don't have all history commits, which would make some pkgbuild fail. Second, gitclone.com seems will refuse too frequent download, when use it in git_mirror, often failed. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9f254f5..dee0285 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -34,8 +34,6 @@ class GitMirror def get_url(url) if url.include?('gitee.com/') && File.exist?("/srv/git/#{url.delete_prefix('https://')}") url = "/srv/git/#{url.delete_prefix('https://')}" - elsif url.include?('://github.com') - url = "https://gitclone.com/#{url.split('://')[1]}" end return url end -- 2.23.0

1 0

Re: [PATCH lkp-tests] pkg/: add PKGBUILD for AvxToNeon
by Zhang Yale 16 Dec '20

16 Dec '20

On Tue, Dec 15, 2020 at 05:17:21PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > pkg/api-avx2neon/PKGBUILD | 19 +++++++++++++++++++ > 1 file changed, 19 insertions(+) > create mode 100644 pkg/api-avx2neon/PKGBUILD > >diff --git a/pkg/api-avx2neon/PKGBUILD b/pkg/api-avx2neon/PKGBUILD >new file mode 100644 >index 00000000..1451c01b >--- /dev/null >+++ b/pkg/api-avx2neon/PKGBUILD >@@ -0,0 +1,19 @@ >+pkgname=api-avx2neon >+pkgver=1 ^ May be pkgver is "git". Thanks, Yale >+pkgrel=0 >+arch=('i386' 'x86_64') >+url="https://github.com/kunpengcompute/AvxToNeon.git" >+license=('GPL') >+source=("https://github.com/kunpengcompute/AvxToNeon.git") >+md5sums=('SKIP') >+ >+build() { >+ cd "$srcdir/AvxToNeon/tests" >+ make >+} >+ >+package() { >+ benchmark_path="${pkgdir}/lkp/benchmarks/${pkgname}" >+ mkdir -p "${benchmark_path}" >+ cp -af "${srcdir}/AvxToNeon/"* "${benchmark_path}" >+} >-- >2.23.0 >

1 0

[PATCH v2 compass-ci 1/2] sbin/build-my-info.rb: building my_info from configuration files
by Liu Yinsi 16 Dec '20

16 Dec '20

before: users build info by input my_name, my_email, then write into ES and yaml file. after: no need users to input, get defaults admin account info from /etc/compass-ci/defaults/admin-account.yaml, then write into ES. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/build-my-info.rb | 54 +++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 73f7390..d0cc7ec 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -3,48 +3,46 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'set' require 'json' require 'fileutils' require 'io/console' require_relative '../lib/es_client' - -print 'email: ' -my_email = $stdin.echo = gets.chomp -print 'name: ' -my_name = $stdin.echo = gets.chomp -my_uuid = %x(uuidgen).chomp - +require_relative '../container/defconfig' + +names = Set.new %w[ + my_email + my_name + my_token +] + +defaults = relevant_defaults(names) +my_email = defaults['my_email'] +my_name = defaults['my_name'] +my_token = defaults['my_token'] my_info = { 'my_email' => my_email, 'my_name' => my_name, - 'my_uuid' => my_uuid + 'my_token' => my_token, + 'my_login_name' => '', + 'my_commit_url' => '', + 'my_ssh_pubkey' => [] } -def store_account_info(my_info) - es = ESClient.new(index: 'accounts') - es.put_source_by_id(my_info['my_email'], my_info) +def es + ESClient.new(index: 'accounts') end -def config_default_yaml(my_info) - yaml_dir = "#{ENV['HOME']}/.config/compass-ci/defaults" - FileUtils.mkdir_p yaml_dir unless File.directory? yaml_dir - yaml_file = "#{yaml_dir}/account.yaml" - FileUtils.touch(yaml_file) unless File.exist? yaml_file - - yaml_info = YAML.load_file(yaml_file) || {} - yaml_info.update my_info +def store_account_info(my_info) + return if check_account_exist(my_info['my_email']) - File.open(yaml_file, 'w') do |f| - f.puts yaml_info.to_yaml - end + es.put_source_by_id(my_info['my_email'], my_info) end -def complete_my_info(my_info) - my_info['my_login_name'] = '' - my_info['my_commit_url'] = '' - my_info['my_ssh_pubkey'] = [] +def check_account_exist(email) + return true if es.query_by_id(email) + + return false end -config_default_yaml(my_info) -complete_my_info(my_info) store_account_info(my_info) -- 2.23.0

1 0

[PATCH v3 compass-ci] container/defconfig.rb: fix rubocop warning
by Liu Yinsi 16 Dec '20

16 Dec '20

W: Lint/UselessAssignment: Useless assignment to variable - kb. Use _ or _kb as a variable name to indicate that it won't be used. File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } ^^ C: Style/Semicolon: Do not use semicolons to terminate expressions. File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } ^ Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index 482dbc8..d1e2af0 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -35,11 +35,11 @@ def docker_rm(container) end def meminfo_hash - File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } + YAML.load_file('/proc/meminfo') end def get_available_memory - memtotal = meminfo_hash[0]['MemTotal'].to_f / 1048576 + memtotal = meminfo_hash['MemTotal'].to_f / 1048576 # set container available memory size, minimum size is 1024m, maximum size is 30720m, # take the middle value according to the system memory size. -- 2.23.0

1 0

[PATCH lkp-tests 1/3] lib/job.rb: load configuration files
by Liu Yinsi 16 Dec '20

16 Dec '20

scheduler need to read administrator account configuration info, such as account autosubmit. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index b41d55a7c..e6a96fd1f 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -427,6 +427,7 @@ class Job end Dir.glob(["/etc/compass-ci/defaults/*.yaml", + "/etc/compass-ci/account/*.yaml", "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| load_one_defaults(file, @job) end -- 2.23.0

2 4

[PATCH v2 compass-ci] docker-rootfs/packages-to-install: add ntpdate server
by Wang Chenglong 16 Dec '20

16 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/packages-to-install | 1 + 1 file changed, 1 insertion(+) diff --git a/container/docker-rootfs/packages-to-install b/container/docker-rootfs/packages-to-install index d7e890f..9098c27 100644 --- a/container/docker-rootfs/packages-to-install +++ b/container/docker-rootfs/packages-to-install @@ -1,3 +1,4 @@ kernel kernel-devel openssh-server +ntpdate -- 2.23.0

1 0

[PATCH compass-ci] container: add automatically start containers upon startup
by Bai Jing 16 Dec '20

16 Dec '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- container/conserver/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/conserver/start b/container/conserver/start index ae44291..e28ba4a 100755 --- a/container/conserver/start +++ b/container/conserver/start @@ -20,6 +20,7 @@ fi cmd=( docker run + --restart=always -d -e "IPMI_USER=${user}" -e "IPMI_PASSWORD=${passwd}" -- 2.23.0

1 0

[PATCH compass-ci] sparrow: add log for build/start every container
by Yu Chuan 16 Dec '20

16 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/4-docker/buildall | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index a9269e03a550..8a3a23af3e4e 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -5,6 +5,8 @@ CONTAINER_PATH="$CCI_SRC/container" export action=$1 +[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" + build_depends() { local container=$1 @@ -23,6 +25,7 @@ do_one() local container=$1 local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock + [ -n "$LKP_SRC" ] && log_info "start build $container" mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container @@ -33,6 +36,7 @@ do_one() [ -x first-run ] && ./first-run [ -x start ] && ./start ) + [ -n "$LKP_SRC" ] && log_info "finish build $container." lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

3 3

Re: [PATCH v2 lkp-tests] pkg: add PKGBUILD for AvxToNeon
by Zhang Yu 16 Dec '20

16 Dec '20

On Wed, Dec 16, 2020 at 10:34:50AM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > pkg/api-avx2neon/PKGBUILD | 19 +++++++++++++++++++ > 1 file changed, 19 insertions(+) > create mode 100644 pkg/api-avx2neon/PKGBUILD > >diff --git a/pkg/api-avx2neon/PKGBUILD b/pkg/api-avx2neon/PKGBUILD >new file mode 100644 >index 000000000..9706b3c5e >--- /dev/null >+++ b/pkg/api-avx2neon/PKGBUILD >@@ -0,0 +1,19 @@ >+pkgname=api-avx2neon >+pkgver=1 >+pkgrel=1 >+arch=('i386' 'x86_64') Can this PKGBUILD run in aarch64?? Thanks, Zhangyu >+url="https://github.com/kunpengcompute/AvxToNeon.git" >+license=('GPL') >+source=("https://github.com/kunpengcompute/AvxToNeon.git") >+md5sums=('SKIP') >+ >+build() { >+ cd "$srcdir/AvxToNeon/tests" >+ make >+} >+ >+package() { >+ benchmark_path="${pkgdir}/lkp/benchmarks/${pkgname}" >+ mkdir -p "${benchmark_path}" >+ cp -af "${srcdir}/AvxToNeon/"* "${benchmark_path}" >+} >-- >2.23.0 >

1 0

[PATCH lkp-tests] lkp-setup-rootfs: all use of env http_proxy and delete specific proxy settings
by Cui Yili 16 Dec '20

16 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- bin/lkp-setup-rootfs | 9 --------- 1 file changed, 9 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index cca1b53be..a7f1a7ffb 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -105,7 +105,6 @@ install_proxy_yum() { [ -d "/etc/yum/" ] || return - echo "proxy=http://$SQUID_HOST:$SQUID_PORT" >> /etc/yum.conf if ls /etc/yum.repos.d/*.repo >/dev/null 2>&1; then sed -i 's/https:/http:/g' /etc/yum.repos.d/*.repo fi @@ -117,11 +116,6 @@ install_proxy_apt() { [ -d "/etc/apt/" ] || return - cat << EOF > /etc/apt/apt.conf.d/proxy -Acquire::http::proxy "http://$SQUID_HOST:$SQUID_PORT/"; -Acquire::ftp::proxy "ftp://$SQUID_HOST:$SQUID_PORT/"; -EOF - for cfg in /etc/apt/sources.list /etc/apt/sources.list.d/* do [ -f "$cfg" ] || continue @@ -176,9 +170,6 @@ install_proxy() install_goproxy - [ -z "$SQUID_HOST" ] && return - [ -z "$SQUID_PORT" ] && return - install_proxy_apt || install_proxy_yum } -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] sparrow: refactor the code for docker build all image concurrently
by Yu Chuan 16 Dec '20

16 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/4-docker/buildall | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index eb245008ad45..a9269e03a550 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -8,18 +8,14 @@ export action=$1 build_depends() { local container=$1 - local should_wait=$2 for dep in $(cat $container/*-depends 2> /dev/null) do - build_depends $CONTAINER_PATH/$dep block_wait + build_depends $CONTAINER_PATH/$dep & done + wait - if [ -n "$should_wait" ]; then - do_one $container - else - do_one $container & - fi + do_one $container } do_one() @@ -44,8 +40,8 @@ tmpdir=$(mktemp -d) for dir in $CONTAINER_PATH/*/ do - build_depends $dir + build_depends $dir & done - wait + rm -fr $tmpdir -- 2.23.0

2 1

[PATCH v2 compass-ci 2/2] sparrow: fix that 0-package/os/openEuler can only be executed once
by Yu Chuan 16 Dec '20

16 Dec '20

[Error Msg] ++ rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm warning: lib64lockfile1-1.16-1-omv4002.aarch64.rpm: Header V4 RSA/SHA256 Signature, key ID bf81de15: NOKEY package lib64lockfile1-1.16-1.aarch64 is already installed file /usr/lib64/liblockfile.so.1.0 from install of lib64lockfile1-1.16-1.aarch64 conflicts with file from package lib64lockfile1-1.16-1.aarch64 package lockfile-progs-0.1.18-5.aarch64 is already installed file /usr/bin/lockfile-check from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-create from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-remove from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-touch from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-lock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-touchlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-unlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 [How] yuchuan@localhost ~% rpm --help | grep replacepkgs --force short hand for --replacepkgs --replacefiles --replacepkgs reinstall if the package is already present Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/0-package/os/openEuler | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 43e96ab8ab2e..813b0ad3c40a 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -27,5 +27,5 @@ yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm +rpm -i --replacepkgs lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

2 1

[PATCH compass-ci] sparrow/5-build: adapted for x86_64 machine
by Liu Yinsi 16 Dec '20

16 Dec '20

before: can build ipxe files on aarch64 machine after: can build ipxe files on aarch64 and x86_64 machine Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/ipxe | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/sparrow/5-build/ipxe b/sparrow/5-build/ipxe index a66e298..0f99fa0 100755 --- a/sparrow/5-build/ipxe +++ b/sparrow/5-build/ipxe @@ -15,11 +15,25 @@ cd ipxe/src || exit cpus=$(nproc) -make ARCH=arm64 bin-arm64-efi/ipxe.efi -make ARCH=arm64 bin-arm64-efi/snponly.efi -make ARCH=arm64 bin-arm64-efi/snp.efi -make CONFIG=rpi bin-arm64-efi/rpi.efi - -mkdir -p /tftpboot/ipxe/bin-arm64-efi/ -cp -a bin-arm64-efi/*.efi /tftpboot/ipxe/bin-arm64-efi/ -chmod 664 /tftpboot/ipxe/bin-arm64-efi/*.efi +build_ipxe() +{ + make ARCH=$1 bin-$1-efi/ipxe.efi + make ARCH=$1 bin-$1-efi/snponly.efi + make ARCH=$1 bin-$1-efi/snp.efi + make CONFIG=rpi bin-$1-efi/rpi.efi + + mkdir -p /tftpboot/ipxe/bin-$1-efi/ + cp -a bin-$1-efi/*.efi /tftpboot/ipxe/bin-$1-efi/ + chmod 664 /tftpboot/ipxe/bin-$1-efi/*.efi +} + +ARCH=$(arch) + +case "$ARCH" in + "aarch64") + build_ipxe arm64 + ;; + "x86_64") + build_ipxe x86_64 + ;; +esac -- 2.23.0

2 2

[PATCH v3 compass-ci 5/6] fetch-mail: build file
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/fetch-mail/build diff --git a/container/fetch-mail/build b/container/fetch-mail/build new file mode 100755 index 0000000..e72bed8 --- /dev/null +++ b/container/fetch-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t fetch-mail:latest . -- 2.23.0

1 0

[PATCH v3 compass-ci 2/6] fetch-mail: config file for procmail
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/procmailrc.template | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 container/fetch-mail/procmailrc.template diff --git a/container/fetch-mail/procmailrc.template b/container/fetch-mail/procmailrc.template new file mode 100644 index 0000000..9cea4bc --- /dev/null +++ b/container/fetch-mail/procmailrc.template @@ -0,0 +1,25 @@ +DROPPRIVS=yes +PATH=$HOME/bin:/usr/local/bin:/usr/bin:/bin +MAILDIR=FETCH_MAIL_DIR +MAILLIST=${MAILDIR} +DEFAULT=${MAILDIR}/FETCH_MAIL_BOX/ +ORGMAIL=${DEFAULT}/FETCH_MAIL_BOX/ +LOGFILE=${MAILDIR}/log/procmail # recommended + +UMASK=002 + +DISPLAY=:0 +XAUTHORITY=$HOME/.Xauthority + +# One often receives duplicate messages for cross-postings to mailing +# lists, or explicit replies to a post from you in addition to the +# mailing list message. +# +# The following recipe eliminates all duplicate messages. It tells +# formail(1) to keep a cache file of the Message-IDs of the most recent +# messages you've received. Since Message-IDs are guaranteed unique for +# distinct messages, they are ideally suited to weed out duplicates. + +# LOCKFILE=$MAILDIR/.msgid.lock +:0 Wh: $MAILDIR/.msgid.lock +| formail -D 655360 $MAILDIR/log/msgid -- 2.23.0

1 0

[PATCH v3 compass-ci 1/6] fetch-mail: config file for fetchmail
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/fetchmailrc.template | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/fetch-mail/fetchmailrc.template diff --git a/container/fetch-mail/fetchmailrc.template b/container/fetch-mail/fetchmailrc.template new file mode 100644 index 0000000..8c7e82b --- /dev/null +++ b/container/fetch-mail/fetchmailrc.template @@ -0,0 +1,5 @@ +poll imap.qq.com with proto IMAP +username "FETCH_MAIL_ADDRESS" +password "FETCH_MAIL_AUTH_CODE" +mda "/usr/bin/procmail -f %F -d %T" +nokeep -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] lib: friendly show compare template result in command line
by Lu Kaiyi 16 Dec '20

16 Dec '20

[why] friendly show compare template result in command line. [how] rehandle the results of "compare -t <compare_template.yaml>" with terminal-table in Ruby. [example] compare -t compare_template.yaml in z9 [input] a compare_template.yaml like below: compare_metrics: - fio.write_iops - fio.read_iops filter: suite: - fio-basic os_arch: - aarch64 - x86 compare_dimensions: - os: centos os_version: 7.6 - os: openeuler os_version: 20.03 x_params: - bs - test_size title: Hackbench Performance Testing unit: KB/s [output] +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | Hackbench Performance Testing (unit: KB/s, x_name: bs|test_size) | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | fio.read_iops | 4k|1G | 4k|80G | 16k|1G | 32k|1G | 64k|1G | 128k|1G | 256k|1G | 512k|1G | 1024k|1G | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | average openeuler 20.03 | 144076.29 | 11601.10 | 37865.30 | 21145.10 | 14010.34 | 6701.24 | 3205.08 | 1367.48 | 673.33 | | standard_deviation openeuler 20.03 | 195.00 | 0.00 | 214.00 | 205.00 | 188.00 | 183.00 | 180.00 | 191.00 | 191.00 | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ +--------------------------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | Hackbench Performance Testing (unit: KB/s, x_name: bs|test_size) | +--------------------------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | fio.write_iops | 4k|1G | 16k|1G | 32k|1G | 64k|1G | 128k|1G | 256k|1G | 512k|1G | 1024k|1G | +--------------------------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | average centos 7.6 | 345243.03 | 142698.79 | 62108.35 | 34747.73 | 26330.19 | 10317.85 | 7471.71 | 3558.30 | | average openeuler 20.03 | 122003.54 | 33528.53 | 31469.06 | 13870.14 | 8249.71 | 4329.45 | 1976.54 | 1141.00 | | standard_deviation centos 7.6 | 97.00 | 95.00 | 122.00 | 125.00 | 100.00 | 130.00 | 101.00 | 103.00 | | standard_deviation openeuler 20.03 | 174.00 | 188.00 | 171.00 | 197.00 | 181.00 | 175.00 | 170.00 | 176.00 | | change centos 7.6 vs openeuler 20.03 | 183.0% | 325.6% | 97.4% | 150.5% | 219.2% | 138.3% | 278.0% | 211.9% | +--------------------------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare_data_format.rb | 73 ++++++++++++++++++++++++++++++++++++++ lib/compare_matrixes.rb | 3 +- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/lib/compare_data_format.rb b/lib/compare_data_format.rb index bf4013b..c1f7023 100644 --- a/lib/compare_data_format.rb +++ b/lib/compare_data_format.rb @@ -2,6 +2,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'terminal-table' + # ---------------------------------------------------------------------------------------------------- # format compare results for a specific format # @@ -66,6 +68,77 @@ class FormatEchartData end end +# ---------------------------------------------------------------------------------------------------- +# format compare template results into a table format +# +class FormatTableData + def initialize(result_hash) + @title = result_hash['title'] + @tables = result_hash['tables'] + @unit = result_hash['unit'] + @x_name = result_hash['x_name'] + end + + def show_table + @tables.each do |table_title, table| + @tb = Terminal::Table.new + set_table_title + set_field_names(table_title, table) + add_rows(table) + set_align_column + print_table + end + end + + def set_table_title + @tb.title = "#{@title} (unit: #{@unit}, x_name: #{@x_name})" + end + + def set_field_names(table_title, table) + field_names = [table_title] + field_names.concat(table['average']['source'][0]) + @tb.add_row(field_names) + @tb.add_separator + end + + def add_rows(table) + row_names = %w[average standard_deviation change] + max_size = row_names.map(&:size).max + row_names.each do |row_name| + next unless table[row_name] + + dimensions_size = table[row_name]['dimensions'].size + (1...dimensions_size).each do |index| + add_row(table, row_name, index, max_size) + end + end + end + + def add_row(table, row_name, index, max_size) + row = table[row_name]['source'][index] + row_title = [row_name + ' ' * (max_size - row_name.size), row[0]].join(' ') + format_data_row = row[1..-1] + if row_name == 'change' + format_data_row.map! { |data| format('%.1f%%', data) } + else + format_data_row.map! { |data| format('%.2f', data) } + end + @tb.add_row([row_title, *format_data_row]) + end + + def set_align_column + @tb.number_of_columns.times do |index| + @tb.align_column(index + 1, :right) + end + @tb.align_column(0, :left) + end + + def print_table + puts @tb + puts + end +end + # input: x_params_list # eg: ["1G|4K", "1G|1024k", "1G|128k", 2G|4k] # output: diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 5069939..9bf676f 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -511,7 +511,8 @@ end def show_compare_result(metrics_compare_results, template_params) formatter = FormatEchartData.new(metrics_compare_results, template_params) echart_results = formatter.format_for_echart - print JSON.pretty_generate(echart_results) + table_results = FormatTableData.new(echart_results) + table_results.show_table end # Format Fields -- 2.23.0

1 0

[PATCH v3 compass-ci] docker-rootfs/setup-image: set the system time zone
by Wang Chenglong 16 Dec '20

16 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/setup-image | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/container/docker-rootfs/setup-image b/container/docker-rootfs/setup-image index 8e2e88a..787a42c 100755 --- a/container/docker-rootfs/setup-image +++ b/container/docker-rootfs/setup-image @@ -13,6 +13,11 @@ remove_file() rm -rf /.dockerenv } +set_localtime() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime +} + setup_login() { [ -n "$ROOT_PASSWD" ] || return @@ -32,5 +37,6 @@ pack_cgz() install_packages remove_file +set_localtime setup_login pack_cgz -- 2.23.0

1 0

[PATCH v5 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 16 Dec '20

16 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 87 ++++++++++++++++++++++++++----------------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..3110685 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -152,29 +152,20 @@ job yaml ��[YAML](http://yaml.org/YAM 1. �� os �� rootfs�� openEuler �� - �� docker �� rootfs - + 1) �� openEuler �� docker �� - + ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` + ``` + 2) {compass-ci}/container/docker-rootfs �� '.tar.xz' �� docker �� rootfs + ```bash + ./{compass-ci}/container/docker-rootfs/run openEuler-docker.aarch64.tar.xz /tmp/openeuler-rootfs-20.03 + ``` - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� + + 1) �� openEuler �� qcow2 �� ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… ``` @@ -183,45 +174,59 @@ job yaml ��[YAML](http://yaml.org/YAM cd {compass-ci}/container/qcow2rootfs ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) + + 2. ��rootfs + 1. �� chroot �� rootfs (�� root ��) ```bash chroot openEuler-rootfs ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 2. �� + a. �� /sbin/init + > ��systemd�� + ```bash + ls -l /sbin/init + /sbin/init -> ../lib/systemd/systemd + ``` + b. �� /etc/localtime + > �� + ```bash + ls -l /etc/localtime + /etc/localtime -> ../usr/share/zoneinfo/Asia/Shanghai + ``` + c. �� docker �� osimage �� + > docker �� /.dockerenv �� + + 3. ��rootfs + ```bash + cd $rootfs + find . | coip -o -Hnewc |gzip -9 > $os_name.cgz + ``` + + 4. ��/�� + �� {compass-ci} �� rootfs ��/�� + ```bash + find /lib/modules/* | cpio -o -Hnewc | gzip -9 > /modules-$os_name.cgz + find /usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > /headers-$os_name.cgz + ``` + #### FAQ -1. �� Unable to mount root fs on unknown-block�� +1. �� Unable to mount root fs on unknown-block�� - �� ```bash [ 0.390437] List of all partitions: - [ 0.390806] No filesystem could mount root, tried: + [ 0.390806] No filesystem could mount root, tried: [ 0.391489] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ... [ 0.399404] Memory Limit: none [ 0.399749] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]--- ``` - �� - + 1�� 2�� 644 �� - + 2. �� - + - �� -- 2.23.0

1 0

[PATCH compass-ci] sparrow: fix that 0-package/os/openEuler can only be executed once
by Yu Chuan 16 Dec '20

16 Dec '20

[Error Msg] ++ rpm -i --replacepkgs lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm warning: lib64lockfile1-1.16-1-omv4002.aarch64.rpm: Header V4 RSA/SHA256 Signature, key ID bf81de15: NOKEY file /usr/lib64/liblockfile.so.1.0 from install of lib64lockfile1-1.16-1.aarch64 conflicts with file from package lib64lockfile1-1.16-1.aarch64 file /usr/bin/lockfile-check from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-create from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-remove from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/lockfile-touch from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-lock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-touchlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 file /usr/bin/mail-unlock from install of lockfile-progs-0.1.18-5.aarch64 conflicts with file from package lockfile-progs-0.1.18-5.aarch64 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/0-package/os/openEuler | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 813b0ad3c40a..13e087599218 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -27,5 +27,5 @@ yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -rpm -i --replacepkgs lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm +rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

1 0

[PATCH compass-ci] buildall: avoid repeated building containers in sparrow
by Xu Xijian 15 Dec '20

15 Dec '20

After building a container in a sparrow task, record it. When there is another requirement of building it, just return and avoid building again to save time. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/4-docker/buildall | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 872b4c0..98d3ce4 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -25,6 +25,8 @@ do_one() local container=$1 local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock + echo "$started_containers" | grep -w -q "$container_name" && return 0 + [ -n "$LKP_SRC" ] && log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( @@ -37,6 +39,7 @@ do_one() [ -x start ] && ./start ) [ -n "$LKP_SRC" ] && log_info "finish build $container." + started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow: add log for build/start every container
by Yu Chuan 15 Dec '20

15 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- sparrow/4-docker/buildall | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index a9269e03a550..872b4c023a01 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -5,6 +5,8 @@ CONTAINER_PATH="$CCI_SRC/container" export action=$1 +[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" + build_depends() { local container=$1 @@ -23,6 +25,7 @@ do_one() local container=$1 local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock + [ -n "$LKP_SRC" ] && log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container @@ -33,6 +36,7 @@ do_one() [ -x first-run ] && ./first-run [ -x start ] && ./start ) + [ -n "$LKP_SRC" ] && log_info "finish build $container." lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/host-info: fix host-info script running error
by Zhang Yu 15 Dec '20

15 Dec '20

[why] The ipmi command just run in physical machine, the vm not support use ipmi command [how] Add a judgment only physical machine run ipmi operation Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/host-info b/tests/host-info index 84f796dcb..0ab8e6c97 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,4 +1,5 @@ #!/bin/bash +. $LKP_SRC/lib/env.sh get_memory() { @@ -101,8 +102,10 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX + is_virt && return + is_docker && return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') - [ -n "$ip" ] && echo "ipmi_ip:" $ip + [ -n "$ip" ] && echo "ipmi_ip: $ip" } -- 2.23.0

1 0

[PATCH v2 compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..6e553b9 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 1000 mailer +RUN apt-get update && \ + apt-get install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer -- 2.23.0

1 1

[PATCH v2 compass-ci 6/6] fetch-mail: start file
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/start | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 container/fetch-mail/start diff --git a/container/fetch-mail/start b/container/fetch-mail/start new file mode 100755 index 0000000..98cbd8f --- /dev/null +++ b/container/fetch-mail/start @@ -0,0 +1,36 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'fetch-mail' + +names = Set.new %w[ + FETCH_MAIL_ADDRESS + FETCH_MAIL_AUTH_CODE + FETCH_MAIL_DIR + FETCH_MAIL_BOX +] + +defaults = relevant_defaults(names) + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + -u mailer + --name=fetch-mail + -d +] + env + %W[ + -v #{defaults['FETCH_MAIL_DIR']}:#{defaults['FETCH_MAIL_DIR']}:rw + -w /home/mailer/ + fetch-mail +] + +cmd += ['sh', '-c', './run.sh'] + +system(*cmd) -- 2.23.0

1 0

[PATCH v2 compass-ci 5/6] fetch-mail: build file
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/fetch-mail/build diff --git a/container/fetch-mail/build b/container/fetch-mail/build new file mode 100755 index 0000000..e72bed8 --- /dev/null +++ b/container/fetch-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t fetch-mail:latest . -- 2.23.0

1 0

[PATCH v2 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 15 Dec '20

15 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100644 index 0000000..d7f9f38 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +function mail_conf { + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf + +fetchmail -d 100 -- 2.23.0

1 0

[PATCH v2 compass-ci 2/6] fetch-mail: config file for procmail
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/procmailrc.template | 25 ++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 container/fetch-mail/procmailrc.template diff --git a/container/fetch-mail/procmailrc.template b/container/fetch-mail/procmailrc.template new file mode 100644 index 0000000..9cea4bc --- /dev/null +++ b/container/fetch-mail/procmailrc.template @@ -0,0 +1,25 @@ +DROPPRIVS=yes +PATH=$HOME/bin:/usr/local/bin:/usr/bin:/bin +MAILDIR=FETCH_MAIL_DIR +MAILLIST=${MAILDIR} +DEFAULT=${MAILDIR}/FETCH_MAIL_BOX/ +ORGMAIL=${DEFAULT}/FETCH_MAIL_BOX/ +LOGFILE=${MAILDIR}/log/procmail # recommended + +UMASK=002 + +DISPLAY=:0 +XAUTHORITY=$HOME/.Xauthority + +# One often receives duplicate messages for cross-postings to mailing +# lists, or explicit replies to a post from you in addition to the +# mailing list message. +# +# The following recipe eliminates all duplicate messages. It tells +# formail(1) to keep a cache file of the Message-IDs of the most recent +# messages you've received. Since Message-IDs are guaranteed unique for +# distinct messages, they are ideally suited to weed out duplicates. + +# LOCKFILE=$MAILDIR/.msgid.lock +:0 Wh: $MAILDIR/.msgid.lock +| formail -D 655360 $MAILDIR/log/msgid -- 2.23.0

1 0

[PATCH v2 compass-ci 1/6] fetch-mail: config file for fetchmail
by Luan Shengde 15 Dec '20

15 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/fetchmailrc.template | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/fetch-mail/fetchmailrc.template diff --git a/container/fetch-mail/fetchmailrc.template b/container/fetch-mail/fetchmailrc.template new file mode 100644 index 0000000..8c7e82b --- /dev/null +++ b/container/fetch-mail/fetchmailrc.template @@ -0,0 +1,5 @@ +poll imap.qq.com with proto IMAP +username "FETCH_MAIL_ADDRESS" +password "FETCH_MAIL_AUTH_CODE" +mda "/usr/bin/procmail -f %F -d %T" +nokeep -- 2.23.0

1 0