- Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..5bd1a33 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,19 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:latest +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 2002 mailer +RUN yum update -y +RUN yum install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer +RUN chmod +x /home/mailer/run.sh -- 2.23.0

1 0

[PATCH v4 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 16 Dec '20

16 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100755 index 0000000..c79b054 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +mail_conf () +{ + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf +fetchmail -d 100 -- 2.23.0

1 0

[PATCH compass-ci] web_backend.rb: fix error message
by Li Yuanchao 16 Dec '20

16 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 94883a3..07e2e9e 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -457,7 +457,7 @@ def query_testboxes body = testboxes_body rescue StandardError => e warn e.message - return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get repos error'] + return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'get testboxes error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] end -- 2.23.0

1 0

[PATCH compass-ci] result-webdav/start: add REDIS_PORT and REDIS_HOST env
by Cui Yili 16 Dec '20

16 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/start | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/container/result-webdav/start b/container/result-webdav/start index d7b1097..bbbbb24 100755 --- a/container/result-webdav/start +++ b/container/result-webdav/start @@ -5,6 +5,9 @@ . $CCI_SRC/container/defconfig.sh +: ${REDIS_HOST:=172.17.0.1} +: ${REDIS_PORT:=6379} + docker_rm result-webdav cmd=( @@ -12,6 +15,8 @@ cmd=( --name result-webdav --restart=always -p 3080:3080 + -e REDIS_HOST=$REDIS_HOST + -e REDIS_PORT=$REDIS_PORT -v $PWD/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf -v /etc/localtime:/etc/localtime:ro -v /srv/result/:/srv/result/ -- 2.23.0

1 0

[PATCH compass-ci] assign-account: generate ssh keys for user by default
by Luan Shengde 16 Dec '20

16 Dec '20

[why] request is: when user submit jobs, the ssh keys are already there. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 10 ++-------- container/assign-account/get_account_info.rb | 6 ++---- 2 files changed, 4 insertions(+), 12 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 766e1b3..21285e7 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -61,7 +61,6 @@ stdin_info = {} email_info = {} my_info_es = {} conf_info = { - 'gen_sshkey' => false, 'enable_login' => true, 'is_update_account' => false } @@ -83,13 +82,12 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' - opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" opts.banner += " -n is required when assigning account with -e\n" opts.banner += " -s is optional when use -e\n" - opts.banner += " -g is optional, used to generate sshkey for user\n" opts.banner += " -u is required when updating an account\n" opts.banner += ' -l is optional, used to enable/disable login permission' @@ -141,10 +139,6 @@ options = OptionParser.new do |opts| init_info(mail_content, email_info, my_info, my_info_es) end - opts.on('-g', '--gen-sshkey', 'generate jumper rsa public/private key and return pubkey') do - conf_info['gen_sshkey'] = true - end - opts.on('-u', '--update', 'updata configurations') do conf_info['is_update_account'] = true end diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 9a7acda..5096734 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -64,7 +64,7 @@ class AccountStorage def read_account_info available_dir = File.join(@account_dir, 'available-users') - files = Dir.entries(available_dir) + files = Dir.open(available_dir).to_a files -= ['.', '..'] message = 'no more available users' @@ -112,9 +112,7 @@ class AccountStorage config_authorized_key(login_name) config_yaml(login_name) permit_login_config(login_name) - if @data.key?('gen_sshkey') && @data['gen_sshkey'] - my_jumper_pubkey = generate_ssh_key(login_name) - end + my_jumper_pubkey = generate_ssh_key(login_name) jumper_account_info = { 'my_login_name' => login_name, -- 2.23.0

1 0

[PATCH compass-ci] src/lib: add common deps packages
by Zhang Yu 16 Dec '20

16 Dec '20

[why] when run job with initramfs, some command like time and perf are not found [how] add depends package of initramfs-common to install the common package Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- src/lib/job.cr | 1 + 1 file changed, 1 insertion(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0083a7d..1c30f38 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -497,6 +497,7 @@ class Job private def get_depends_initrd(program_params, initrd_deps_arr, initrd_pkg_arr) initrd_http_prefix = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" mount_type = os_mount == "cifs" ? "nfs" : os_mount.dup + program_params["initramfs-common"] = JSON::Any.new("") if os_mount == "initramfs" program_params.keys.each do |program| if program =~ /^(.*)-\d+$/ -- 2.23.0

2 2

[PATCH v3 compass-ci 6/6] fetch-mail: start file
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/start | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100755 container/fetch-mail/start diff --git a/container/fetch-mail/start b/container/fetch-mail/start new file mode 100755 index 0000000..45f3910 --- /dev/null +++ b/container/fetch-mail/start @@ -0,0 +1,36 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'fetch-mail' + +names = Set.new %w[ + FETCH_MAIL_ADDRESS + FETCH_MAIL_AUTH_CODE + FETCH_MAIL_DIR + FETCH_MAIL_BOX +] + +defaults = relevant_defaults(names) + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + -u mailer + --name=fetch-mail + -d +] + env + %W[ + -v #{defaults['FETCH_MAIL_DIR']}:#{defaults['FETCH_MAIL_DIR']}:rw + -w /home/mailer/ + fetch-mail +] + +cmd += ['sh', '-c', 'sh run.sh'] + +system(*cmd) -- 2.23.0

2 2

[PATCH compass-ci] scheduler: move and refactor function 'auto_submit_idle_job'
by Ren Wen 16 Dec '20

16 Dec '20

Idle job will be submitted when there's no job to consume, so move this function to 'find_job_boot.cr'. Refactor this function for subqueue: for idle job, subqueue is 'idle'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 14 -------------- src/scheduler/find_job_boot.cr | 7 +++++++ 2 files changed, 7 insertions(+), 14 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 10a51ab..a1ef28e 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -67,20 +67,6 @@ class Sched @redis.hash_del("sched/host2queues", hostname) end - def auto_submit_idle_job(tbox_group) - full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" - extra_job_fields = [ - "idle_job=true", - "MASTER_FLUENTD_HOST=#{ENV["MASTER_FLUENTD_HOST"]}", - "MASTER_FLUENTD_PORT=#{ENV["MASTER_FLUENTD_PORT"]}", - ] - - Jobfile::Operate.auto_submit_job( - full_path_patterns, - "testbox: #{tbox_group}", - extra_job_fields) if Dir.glob(full_path_patterns).size > 0 - end - def update_tbox_wtmp testbox = "" hash = Hash(String, String).new diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index fcc7ea9..d5ef505 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -230,4 +230,11 @@ class Sched return job end + + private def auto_submit_idle_job(testbox) + full_path_patterns = "#{CCI_REPOS}/#{LAB_REPO}/allot/idle/#{testbox}/*.yaml" + fields = ["testbox=#{testbox}", "subqueue=idle"] + + Jobfile::Operate.auto_submit_job(full_path_patterns, fields) if Dir.glob(full_path_patterns).size > 0 + end end -- 2.23.0

1 0

[PATCH lkp-tests 3/3] tests/build-pkg: disable 2>&1 when run makepkg
by Lin Jiaxin 16 Dec '20

16 Dec '20

Make error will redirect itself to stdout now. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index df518727..e3bf31d9 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -126,7 +126,7 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH lkp-tests 2/3] revoke commit: 2dad3102dc5c11d02e66eab9840bcdffb11e93f0
by Lin Jiaxin 16 Dec '20

16 Dec '20

[Why] We modify the make source code let it's stderr to stdout when compile error. So no need for this commit. before: all stderr => stdout => file: build-pkg => stats/build-pkg -- when run makepkg But stats/build-pkg can only analyze make error. So it ignores some other errors. after: unknown error => stderr => file: stderr => stats/stderr make stderr => stdout => file: build-pkg => stats/build-pkg Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- sbin/makepkg | 18 +++--------------- tests/build-pkg | 4 ++-- 2 files changed, 5 insertions(+), 17 deletions(-) diff --git a/sbin/makepkg b/sbin/makepkg index 131ae690..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -1761,19 +1761,11 @@ run_build() { [[ -d /usr/lib/ccache/bin ]] && export PATH="/usr/lib/ccache/bin:$PATH" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "build" 2>&1 - else - run_function_safe "build" - fi + run_function_safe "build" } run_check() { - if [[ $STDERR_REDIR == y ]];then - run_function_safe "check" 2>&1 - else - run_function_safe "check" - fi + run_function_safe "check" } run_package() { @@ -1784,11 +1776,7 @@ run_package() { pkgfunc="package_$1" fi - if [[ $STDERR_REDIR == y ]];then - run_function_safe "$pkgfunc" 2>&1 - else - run_function_safe "$pkgfunc" - fi + run_function_safe "$pkgfunc" } build_id() { diff --git a/tests/build-pkg b/tests/build-pkg index dbde3e54..df518727 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -125,8 +125,8 @@ build_source_pkg() [ -n "$PKGBUILD_TAG" ] && replace_source "PKGBUILD-$PKGBUILD_TAG" cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

1 0

[PATCH compass-ci] git_mirror.rb: improve how to get filenames
by Ren Wen 16 Dec '20

16 Dec '20

Dir.children returns filenames except for “.” and “..”. Signed-off-by: Ren Wen <15991987063(a)163.com> --- lib/git_mirror.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9f254f5..dfcc016 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,7 +130,7 @@ class MirrorMain def traverse_repodir(repodir) if File.directory? repodir load_defaults(repodir) - entry_list = Dir.entries(repodir) - Array['.', '..', 'DEFAULTS', '.ignore', '.git'] + entry_list = Dir.children(repodir) - Array['DEFAULTS', '.ignore', '.git'] entry_list.each do |entry| traverse_repodir("#{repodir}/#{entry}") end -- 2.23.0

1 0

[PATCH lkp-tests] tests: fix error when running host-info in vm
by Zhang Yu 16 Dec '20

16 Dec '20

Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 0ab8e6c97..7513d6073 100755 --- a/tests/host-info +++ b/tests/host-info @@ -102,8 +102,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX - is_virt && return - is_docker && return + [[ -e /dev/ipmi0 ]] || return + [[ -e /dev/ipmi/0 ]] || return + [[ -e /dev/ipmidev/0 ]] || return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip: $ip" -- 2.23.0

3 2

[PATCH v2 compass-ci] doc/job: add SCHED_HOST, SCHED_PORT description
by Hu Xuejiao 16 Dec '20

16 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- doc/job/SCHED_HOST-SCHED_PORT.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/job/SCHED_HOST-SCHED_PORT.md diff --git a/doc/job/SCHED_HOST-SCHED_PORT.md b/doc/job/SCHED_HOST-SCHED_PORT.md new file mode 100644 index 0000000..74c628a --- /dev/null +++ b/doc/job/SCHED_HOST-SCHED_PORT.md @@ -0,0 +1,15 @@ +# SCHED_HOST, SCHED_PORT +Meaning: +- The job task depends on scheduler service, which will distribute tasks to test + machine. All tasks will run in test machine. +- `Scheduler` is one of our service, `SCHED` is the abbreviation of `Scheduler` + these variables mean: + SCHED_HOST: the scheduler service ip. + SCHED_PORT: the scheduler service port. + +Usage example: + +```$LKP_SRC/include/lab/z9 +SCHED_HOST: api.compass-ci.openeuler.org +SCHED_PORT: 11306 +``` -- 2.23.0

4 3

[PATCH v3 compass-ci 4/6] fetch-mail: Dockerfile
by Luan Shengde 16 Dec '20

16 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/fetch-mail/Dockerfile diff --git a/container/fetch-mail/Dockerfile b/container/fetch-mail/Dockerfile new file mode 100644 index 0000000..7be6b21 --- /dev/null +++ b/container/fetch-mail/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos +MAINTAINER Luan Shengde <shdluan(a)163.com> +ENV DEBIAN_FRONTEND noninteractive + +RUN useradd -u 2002 mailer +RUN yum update -y +RUN yum install -y procmail fetchmail + +COPY fetchmailrc.template /home/mailer/.fetchmailrc +COPY procmailrc.template /home/mailer/.procmailrc +COPY run.sh /home/mailer/ + +RUN chmod 600 /home/mailer/.fetchmailrc +RUN chmod 600 /home/mailer/.procmailrc +RUN chown -R mailer:mailer /home/mailer -- 2.23.0

2 2

[PATCH v3 compass-ci 3/6] fetch-mail: runner for fetch-mail service
by Luan Shengde 16 Dec '20

16 Dec '20

config fetchmail and procmail run fetchmail command to fetch emails Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/fetch-mail/run.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 container/fetch-mail/run.sh diff --git a/container/fetch-mail/run.sh b/container/fetch-mail/run.sh new file mode 100755 index 0000000..d7f9f38 --- /dev/null +++ b/container/fetch-mail/run.sh @@ -0,0 +1,14 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +function mail_conf { + sed -i "s|FETCH_MAIL_ADDRESS|${FETCH_MAIL_ADDRESS}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_AUTH_CODE|${FETCH_MAIL_AUTH_CODE}|g" '.fetchmailrc' + sed -i "s|FETCH_MAIL_DIR|${FETCH_MAIL_DIR}|g" '.procmailrc' + sed -i "s|FETCH_MAIL_BOX|${FETCH_MAIL_BOX}|g" '.procmailrc' +} + +mail_conf + +fetchmail -d 100 -- 2.23.0

2 2

[PATCH compass-ci] git_mirror.rb: ignore all hidden files when load upstream-repos
by Li Yuanchao 16 Dec '20

16 Dec '20

When vim file, there would be a hidden file, which is not yaml like. Such files should be ignored when load upstream-repos. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index dee0285..389b185 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,6 +130,8 @@ class MirrorMain load_defaults(repodir) entry_list = Dir.entries(repodir) - Array['.', '..', 'DEFAULTS', '.ignore', '.git'] entry_list.each do |entry| + next if entry.start_with? '.' + traverse_repodir("#{repodir}/#{entry}") end else -- 2.23.0

1 0

[PATCH compass-ci] scheduler: refactor function 'auto_submit_job'
by Ren Wen 16 Dec '20

16 Dec '20

Now can use an array contains all fields you want to add. Example: auto_submit_job("job.yaml", ["field01=value01", "field02=value02", ...]) Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 2 +- src/scheduler/jobfile_operate.cr | 7 ++++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 5a11523..fcc7ea9 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -33,7 +33,7 @@ class Sched Jobfile::Operate.auto_submit_job( "#{ENV["LKP_SRC"]}/jobs/host-info.yaml", - "testbox: #{host}") + ["testbox=#{host}"]) end def rand_queues(queues) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index df79d3b..65d9b17 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -61,10 +61,11 @@ module Jobfile::Operate return target_path end - def self.auto_submit_job(job_file, override_parameter, other_parameters = nil) + # *fields* should be: ["field01=value01", "field02=value02", ...] + def self.auto_submit_job(job_file, fields : Array(String) = ["testbox=vm-2p8g"]) cmd = "#{ENV["LKP_SRC"]}/sbin/submit " - cmd += other_parameters.join(" ") if other_parameters - cmd += " -s '#{override_parameter}' #{job_file}" + cmd += "#{job_file} " + cmd += fields.join(" ") puts `#{cmd}` end end -- 2.23.0

1 0

[PATCH lkp-tests] lib/upload: change the key of upload results
by Wei Jihui 16 Dec '20

16 Dec '20

the key of upload results is changed from access_key to job id Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job-init.sh | 2 +- lib/upload.sh | 18 +++++++++--------- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/lib/job-init.sh b/lib/job-init.sh index 40534af3..dc4a65d9 100755 --- a/lib/job-init.sh +++ b/lib/job-init.sh @@ -55,7 +55,7 @@ setup_result_service() supports_raw_upload && result_service=raw_upload && return - is_docker || [ -n "$access_key" ] || { + is_docker || [ -n "$id" ] || { if [ "$os_mount" = 'nfs' ] || [ -z "$os_mount" ]; then supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return elif [ "$os_mount" = 'cifs' ] || [ -z "$os_mount" ]; then diff --git a/lib/upload.sh b/lib/upload.sh index a5daceda..0115e350 100755 --- a/lib/upload.sh +++ b/lib/upload.sh @@ -78,9 +78,9 @@ upload_one_curl() ( cd $(dirname "$1") dir=$(basename "$1") - if [ -n "$access_key" ]; then - find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; - find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; + if [ -n "$id" ]; then + find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "JOBID=$id" \; + find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "JOBID=$id" \; else find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" \; @@ -88,8 +88,8 @@ upload_one_curl() ) else [ -s "$src" ] || return - if [ -n "$$access_key" ]; then - curl -sSf -T "$src" $http_url/ --cookie "ACCESSKEY=$access_key" + if [ -n "$id" ]; then + curl -sSf -T "$src" $http_url/ --cookie "JOBID=$id" else curl -sSf -T "$src" $http_url/ fi @@ -111,8 +111,8 @@ upload_files_curl() for dir in $(echo $target_directory | tr '/' ' ') do job_result_root=$job_result_root/$dir/ - if [ -n "$$access_key" ]; then - curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "ACCESSKEY=$access_key" >/dev/null + if [ -n "$id" ]; then + curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "JOBID=$id" >/dev/null else curl -sSf -X MKCOL "$http_host$job_result_root" >/dev/null fi @@ -178,12 +178,12 @@ upload_files() upload_files_copy "$@" return } - if has_cmd rsync && has_rsync_server && [ -z "$access_key" ]; then + if has_cmd rsync && has_rsync_server && [ -z "$id" ]; then upload_files_rsync "$@" return fi - if has_cmd lftp && [ -z "$access_key" ]; then + if has_cmd lftp && [ -z "$id" ]; then upload_files_lftp "$@" return fi -- 2.23.0

1 0

为什么你一定要参加 openEuler Summit 的 TC、Maintainer、SIG 组工作会议
by public openEuler 16 Dec '20

16 Dec '20

嗨，各位 openEuler 社区的开发者：很高兴能够向你们汇报 openEuler Summit 的会议进度。对于各位来说，本周最重要的事情是 TC、Maintainer和SIG 组开放工作会议的议程已经上线了。 *为什么你一定要参与 TC、Maintainer、SIG组开放工作会议？* 1. 你可以清楚的了解 openEuler 社区的运作模式 2. 你可以和 TC、Maintainer、SIG 的成员共同探讨openEuler 社区的下一个版本的计划和技术路线 3. 可以跟社区里的技术大佬进行面对面交流。 4. 更好的规划自己在社区中的成长路线 *会议时间和房间号：* TC 开放工作会议 Maintainer 开放工作会议 SIG 组开放工作会议时间 12/24 13:00-14:30 12/24 15:00-17:00 12/25 13:00-15:30 房间号悦府6和悦府7 悦府6和悦府7 大宴会厅B *TC **开放工作会议议程：* 1. 讨论 openEuler 支持 App Stream 机制 2. 申请成立智能运维 SIG 3. 讨论从 Gitee issue 整体切换到bugzilla 4. 讨论2021年 TC 合作的开展方式 5. …… *Maintainer **开放工作会议议程* 1. 开场 & 颁发 Maintainer 纪念徽章 2. openEuler 社区运作分析 3. 我在 openEuler 社区担任 Maintainer 的那些事儿 4. 自由问答 5. 总结 *参与本次 SIG 组开放工作会议的 SIG 组：* 1. sig-DDE 2. sig-UKUI 3. sig-Ha 4. sig-arrch32 5. A-Tune 6. iSulad & Container 7. sig-ai-bitdata 8. sig-security-facility 9. sig-confidential-computing 10. Compiler 11. Doc 12. Infrastructure 13. security-committee 14. sig-Compatibility-Infra 15. sig-QA 16. sig-release-management 17. Kernel 18. Virt *openEuler Summit **专属伴手礼：* [image: image.png] *openEuler **卫衣* [image: image.png] *openEuler Maintainer **纪念徽章* *欢迎通过openEuler 社区开发者专属的报名渠道报名：* https://etherpad.openeuler.org/p/openEuler-Summit-2020 openEuler Summit 官网： https://openeuler.org/zh/interaction/summit-list/

1 0

[PATCH v2 lkp-tests] tests: fix error when running host-info in vm or docker
by Zhang Yu 16 Dec '20

16 Dec '20

[why] Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory [how] Run ipmi command only on physical machines Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/host-info b/tests/host-info index 84f796dcb..6ac20c120 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,5 +1,7 @@ #!/bin/bash +. $LKP_SRC/lib/env.sh + get_memory() { memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') @@ -101,8 +103,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX + is_virt || is_docker && return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') - [ -n "$ip" ] && echo "ipmi_ip:" $ip + [ -n "$ip" ] && echo "ipmi_ip: $ip" } -- 2.23.0

3 5

[PATCH v2 lkp-tests] lib/env.sh: fix the check result of is_virt
by Zhang Yu 16 Dec '20

16 Dec '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/env.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/env.sh b/lib/env.sh index 78cfd6f57..fc34aaa8d 100755 --- a/lib/env.sh +++ b/lib/env.sh @@ -69,7 +69,8 @@ is_virt() # run as root [ -n "$(virt-what)" ] else - grep -q -w hypervisor /proc/cpuinfo + grep -q -w hypervisor /proc/cpuinfo && return 1 + return 0 fi } -- 2.23.0

2 3

[PATCH compass-ci] git_mirror.rb: gitclone.com does not works well
by Li Yuanchao 16 Dec '20

16 Dec '20

First, repos clone from gitclone.com don't have all history commits, which would make some pkgbuild fail. Second, gitclone.com seems will refuse too frequent download, when use it in git_mirror, often failed. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9f254f5..dee0285 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -34,8 +34,6 @@ class GitMirror def get_url(url) if url.include?('gitee.com/') && File.exist?("/srv/git/#{url.delete_prefix('https://')}") url = "/srv/git/#{url.delete_prefix('https://')}" - elsif url.include?('://github.com') - url = "https://gitclone.com/#{url.split('://')[1]}" end return url end -- 2.23.0

1 0

Re: [PATCH lkp-tests] pkg/: add PKGBUILD for AvxToNeon
by Zhang Yale 16 Dec '20

16 Dec '20

On Tue, Dec 15, 2020 at 05:17:21PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > pkg/api-avx2neon/PKGBUILD | 19 +++++++++++++++++++ > 1 file changed, 19 insertions(+) > create mode 100644 pkg/api-avx2neon/PKGBUILD > >diff --git a/pkg/api-avx2neon/PKGBUILD b/pkg/api-avx2neon/PKGBUILD >new file mode 100644 >index 00000000..1451c01b >--- /dev/null >+++ b/pkg/api-avx2neon/PKGBUILD >@@ -0,0 +1,19 @@ >+pkgname=api-avx2neon >+pkgver=1 ^ May be pkgver is "git". Thanks, Yale >+pkgrel=0 >+arch=('i386' 'x86_64') >+url="https://github.com/kunpengcompute/AvxToNeon.git" >+license=('GPL') >+source=("https://github.com/kunpengcompute/AvxToNeon.git") >+md5sums=('SKIP') >+ >+build() { >+ cd "$srcdir/AvxToNeon/tests" >+ make >+} >+ >+package() { >+ benchmark_path="${pkgdir}/lkp/benchmarks/${pkgname}" >+ mkdir -p "${benchmark_path}" >+ cp -af "${srcdir}/AvxToNeon/"* "${benchmark_path}" >+} >-- >2.23.0 >

1 0

[PATCH v2 compass-ci 1/2] sbin/build-my-info.rb: building my_info from configuration files
by Liu Yinsi 16 Dec '20

16 Dec '20

before: users build info by input my_name, my_email, then write into ES and yaml file. after: no need users to input, get defaults admin account info from /etc/compass-ci/defaults/admin-account.yaml, then write into ES. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/build-my-info.rb | 54 +++++++++++++++++++++---------------------- 1 file changed, 26 insertions(+), 28 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index 73f7390..d0cc7ec 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -3,48 +3,46 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'set' require 'json' require 'fileutils' require 'io/console' require_relative '../lib/es_client' - -print 'email: ' -my_email = $stdin.echo = gets.chomp -print 'name: ' -my_name = $stdin.echo = gets.chomp -my_uuid = %x(uuidgen).chomp - +require_relative '../container/defconfig' + +names = Set.new %w[ + my_email + my_name + my_token +] + +defaults = relevant_defaults(names) +my_email = defaults['my_email'] +my_name = defaults['my_name'] +my_token = defaults['my_token'] my_info = { 'my_email' => my_email, 'my_name' => my_name, - 'my_uuid' => my_uuid + 'my_token' => my_token, + 'my_login_name' => '', + 'my_commit_url' => '', + 'my_ssh_pubkey' => [] } -def store_account_info(my_info) - es = ESClient.new(index: 'accounts') - es.put_source_by_id(my_info['my_email'], my_info) +def es + ESClient.new(index: 'accounts') end -def config_default_yaml(my_info) - yaml_dir = "#{ENV['HOME']}/.config/compass-ci/defaults" - FileUtils.mkdir_p yaml_dir unless File.directory? yaml_dir - yaml_file = "#{yaml_dir}/account.yaml" - FileUtils.touch(yaml_file) unless File.exist? yaml_file - - yaml_info = YAML.load_file(yaml_file) || {} - yaml_info.update my_info +def store_account_info(my_info) + return if check_account_exist(my_info['my_email']) - File.open(yaml_file, 'w') do |f| - f.puts yaml_info.to_yaml - end + es.put_source_by_id(my_info['my_email'], my_info) end -def complete_my_info(my_info) - my_info['my_login_name'] = '' - my_info['my_commit_url'] = '' - my_info['my_ssh_pubkey'] = [] +def check_account_exist(email) + return true if es.query_by_id(email) + + return false end -config_default_yaml(my_info) -complete_my_info(my_info) store_account_info(my_info) -- 2.23.0

1 0

[PATCH v3 compass-ci] container/defconfig.rb: fix rubocop warning
by Liu Yinsi 16 Dec '20

16 Dec '20

W: Lint/UselessAssignment: Useless assignment to variable - kb. Use _ or _kb as a variable name to indicate that it won't be used. File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } ^^ C: Style/Semicolon: Do not use semicolons to terminate expressions. File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } ^ Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index 482dbc8..d1e2af0 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -35,11 +35,11 @@ def docker_rm(container) end def meminfo_hash - File.readlines('/proc/meminfo').map { |s| k, v, kb = s.split; { k.chomp(':') => v } } + YAML.load_file('/proc/meminfo') end def get_available_memory - memtotal = meminfo_hash[0]['MemTotal'].to_f / 1048576 + memtotal = meminfo_hash['MemTotal'].to_f / 1048576 # set container available memory size, minimum size is 1024m, maximum size is 30720m, # take the middle value according to the system memory size. -- 2.23.0

1 0