- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib/git_bisect.rb: delete --first-parent in git start bisect command
by Cao Xueliang 07 Feb '21

07 Feb '21

The option will follow only the first parent commit upon seeing a merge commit, but we need bisect the commits in the merge, so, delete the option. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 40f6b49..dd0b782 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -89,7 +89,7 @@ class GitBisect puts "bad_commit: #{@bad_commit}" puts "good_commit: #{@good_commit}" - result = `git -C #{@work_dir} bisect start --first-parent #{@bad_commit} #{@good_commit}` + result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` -- 2.23.0

1 0

[PATCH v4 compass-ci] sbin: add script for download os rootfs
by Liu Yinsi 07 Feb '21

07 Feb '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100755 index 0000000..208ac7d --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +os=${1:-openeuler} +os_arch=${2:-aarch64} +os_version=${3:-20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p /srv/os/${rootfs_dir} + cd /srv/os/$rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${rootfs_dir}.cgz" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

4 8

[PATCH compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty after finish build container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..e77c19e 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -5,3 +5,5 @@ git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile + +rm -rf openresty -- 2.23.0

3 3

[PATCH compass-ci 1/4] container/defconfig: add function docker_skip_rebuild
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute `sparrow/4-docker/buildall run-only`, if caller script export env DOCKER_SKIP_REBUILD, then image already is exist, just exit build script, not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 6 ++++++ container/defconfig.sh | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index f9b0824..1a655a3 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -47,3 +47,9 @@ def get_available_memory # take the middle value according to the system memory size. [1024, 30720, Math.sqrt(memtotal) * 1024].sort[1].to_i end + +def docker_skip_rebuild(tag) + if ENV.include? 'DOCKER_SKIP_REBUILD' + exit 1 if system "docker image inspect #{tag} > /dev/null 2>&1" + end +end diff --git a/container/defconfig.sh b/container/defconfig.sh index 54e65b5..7612e2d 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -41,3 +41,12 @@ push_image() docker tag "$src_tag" "$dst_tag" docker push "$dst_tag" } + +docker_skip_rebuild() +{ + tag=$1 + [ $DOCKER_SKIP_REBUILD ] && { + docker image inspect $tag > /dev/null 2>&1 + [ "$?" == "0" ] && exit 1 + } +} -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] sparrow/dev-env: add the local-testbox-env.yaml
by Cao Xueliang 07 Feb '21

07 Feb '21

The scheduler will read local-testbox-env.yaml, then add the content to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 8812839..31a12db 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -59,6 +59,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/local-testbox-env.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] scheduler: add testbox env in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the testbox env use the submit script in lkp-tests, but the submit don't need to do that, so, we add the testbox env in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/testbox_env.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/testbox_env.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..5f5f53b 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/testbox_env.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(testbox_env) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/testbox_env.cr b/src/scheduler/testbox_env.cr new file mode 100644 index 0000000..a378e2a --- /dev/null +++ b/src/scheduler/testbox_env.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def testbox_env(flag = "local") + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/#{flag}-testbox-env.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: set default testbox to dc-8g
by Wei Jihui 07 Feb '21

07 Feb '21

set testbox to dc-8g when no testbox input Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6d..eacb7ec9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -345,26 +345,24 @@ class Job def load_hosts_config return if @job.include?(:no_defaults) - return if @load_hosts_done - return unless check_set_tbox_group + check_set_tbox_group hosts_file = get_hosts_file if hosts_file hwconfig = load_yaml(hosts_file, {}) @job[source_file_symkey(hosts_file)] = nil @job.merge!(hwconfig) { |_k, a, _b| a } # job's key/value has priority over hwconfig - @load_hosts_done = true end @job['arch'] ||= 'aarch64' end def check_set_tbox_group - if @job.include?('testbox') - @job['tbox_group'] = tbox_group(@job['testbox']) - return true + unless @job.include?('testbox') + @job[comment_to_symbol('set default testbox')] = nil + @job['testbox'] = 'dc-8g' end - return @job.include?('tbox_group') + @job['tbox_group'] = tbox_group(@job['testbox']) end def get_hosts_file @@ -745,7 +743,6 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| - @load_hosts_done = false @job = deepcopy(job) @job.merge!(hash) load_defaults -- 2.23.0

1 0

[PATCH v2 compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..5d30486 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

1 0

[PATCH compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..a881d67 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

2 1

[PATCH lkp-tests] lib/bootstrap.sh: add step: setup dns
by Yu Chuan 07 Feb '21

07 Feb '21

[Why] Since we have added a dns server, so we need to add support for dns function in lkp-tests framework. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 1127de140a2b..06aa09bf65b2 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -283,6 +283,13 @@ setup_hosts() ln -fs $tmpfs_hosts /etc/hosts } +setup_dns() +{ + [ -n "$DNS_HOST" ] || return 0 + + echo "nameserver ${DNS_HOST}" > /etc/resolv.conf +} + export_ip_mac() { if has_cmd ip; then @@ -739,6 +746,7 @@ boot_init() setup_hostname setup_hosts + setup_dns add_lkp_user -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: delete blank line
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 464153d8f..b5b070f01 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -27,7 +27,6 @@ linux_dep() esac } - mac_dep() { brew install ruby -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: add submit depends
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 012c45499..464153d8f 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -12,13 +12,13 @@ linux_dep() case $ID in ubuntu|debian) export DEBIAN_FRONTEND=noninteractive - sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ + sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ uuid-runtime ;; openEuler|fedora|rhel|centos) if type dnf > /dev/null 2>&1; then - sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux else - sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux fi ;; *) -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result.rb: use upstream url and commit replace git url
by Cao Xueliang 07 Feb '21

07 Feb '21

We should show the upstream url in the bisect report, but the git url is our git server url, so, replace git url to the upstream url. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 8 ++++++++ lib/mail_bisect_result.rb | 3 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 5d13074..40f6b49 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -23,6 +23,7 @@ class GitBisect set_build_pkg_dir set_upstream set_pkgbuild_repo + set_upstream_url set_work_dir set_bad_commit set_good_commit @@ -56,6 +57,12 @@ class GitBisect @upstream_repo_git = "git://#{GIT_MIRROR_HOST}/#{@upstream_repo}" end + def set_upstream_url + @upstream_url = @bad_job['upstream_url'] + puts "upstream_url: #{@upstream_url}" + raise 'upstream url is null' unless @upstream_url + end + def set_work_dir @work_dir = Utils.clone_repo(@upstream_repo_git, @upstream_commit) puts "work_dir: #{@work_dir}" @@ -106,6 +113,7 @@ class GitBisect 'work_dir' => @work_dir, 'bisect_error' => bisect_error, 'all_errors' => all_errors, + 'upstream_url' => @upstream_url 'pkgbuild_repo' => @pkgbuild_repo, 'first_bad_commit_result_root' => first_bad_commit_result_root] end diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 637c008..5e1103c 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -16,6 +16,7 @@ class MailBisectResult @commit_id = bisect_info['commit'] @all_errors = bisect_info['all_errors'] @bisect_error = bisect_info['bisect_error'] + @upstream_url = bisect_info['upstream_url'] @pkgbuild_repo = bisect_info['pkgbuild_repo'] @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] @git_commit = GitCommit.new(@work_dir, @commit_id) @@ -46,7 +47,7 @@ class MailBisectResult Hi #{(a)git_commit.author_name}, We found some error/warning(s) and the first bad commit in the below project: - git url: #{(a)git_commit.url} + git url: #{@upstream_url/@commit_id} git commit: #{@commit_id} ("#{(a)git_commit.subject}") All error/warning(s) (new ones prefixed by >>): -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends/cci-makepkg: add git depend
by Wei Jihui 07 Feb '21

07 Feb '21

user maybe use git cmd when wirte PKGBUILD Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/cci-makepkg | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-makepkg b/distro/depends/cci-makepkg index 4ed4f19e..855ac8ce 100644 --- a/distro/depends/cci-makepkg +++ b/distro/depends/cci-makepkg @@ -1,3 +1,4 @@ +git curl gnupg gettext -- 2.23.0

2 1

[PATCH compass-ci 1/2] scheduler: add public service info in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the service info use the submit script in lkp-tests, but the lkp-tests don't know the service info for a remote user, so, we add the public service info in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/public_service.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/public_service.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..9829e81 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/public_service.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(public_service) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/public_service.cr b/src/scheduler/public_service.cr new file mode 100644 index 0000000..e3b888b --- /dev/null +++ b/src/scheduler/public_service.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def public_service + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/public_service.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

2 2

[PATCH lkp-tests] change the submit option from '-l' to '-r'
by Li Ping 05 Feb '21

05 Feb '21

[why] -l means lftp lftp is just one of the clients, "download results" should use more clear and descriptive name. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/monitor.rb | 6 +++--- sbin/submit | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 9091c92a9..45f720f20 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -96,7 +96,7 @@ class Monitor end def set_result_root(data) - return unless @action['lftp_result'] + return unless @action['mirror_result'] return unless data['log'] data = JSON.parse(data['log']) @@ -113,7 +113,7 @@ class Monitor system "lftp -c mirror #{url} >/dev/null 2>&1" end - def lftp(data) + def result(data) if @result_root lftp_mirror else @@ -163,7 +163,7 @@ class Monitor output(data) connect(data, ws) - lftp(data) + result(data) stop(data, ws) if @action['stop'] end diff --git a/sbin/submit b/sbin/submit index e8b176489..a6f0cd387 100755 --- a/sbin/submit +++ b/sbin/submit @@ -52,8 +52,8 @@ options = OptionParser.new do |opts| actions << 'connect' end - opts.on('-l', '--lftp', 'lftp mirror job result') do - actions << 'lftp_result' + opts.on('-r', '--result', 'mirror job result dir') do + actions << 'mirror_result' end opts.on('-m', '--monitor', "monitor job status: use -m 'KEY: VALUE' to add rule") do -- 2.23.0

1 0

[PATCH lab-z9] the file of testboxes allocation scheme
by Zhang Dewan 05 Feb '21

05 Feb '21

how about this ? it's not easy to show the true purpose of all the machines. sometimes machine borrow out, user maybe install OS on the disk, when it's back, we should set pxe boot, and reboot it. Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- allot/testboxes-allocation-scheme | 246 ++++++++++++++++++++++++++++++ 1 file changed, 246 insertions(+) create mode 100644 allot/testboxes-allocation-scheme diff --git a/allot/testboxes-allocation-scheme b/allot/testboxes-allocation-scheme new file mode 100644 index 0000000..db2940b --- /dev/null +++ b/allot/testboxes-allocation-scheme @@ -0,0 +1,246 @@ +# x86_64: 2288hv5-2s44p-384g +2288hv5-2s44p-384g--b1 + Board-test +2288hv5-2s44p-384g--b2 + pub-tbox + Board-test +2288hv5-2s44p-384g--b3 + VNC-server + Board-test +2288hv5-2s44p-384g--b4 + Openshift-analyse + Board-test +2288hv5-2s44p-384g--b5 + pub-tbox + Board-test +2288hv5-2s44p-384g--b6 + pub-tbox + Board-test +2288hv5-2s44p-384g--b7 + Board-test +# aarch64: taishan200-2280-2s48p-256g +taishan200-2280-2s48p-256g--a11 + multi-docker +taishan200-2280-2s48p-256g--a12 + multi-docker +taishan200-2280-2s48p-256g--a13 + multi-docker +taishan200-2280-2s48p-256g--a14 + multi-docker +taishan200-2280-2s48p-256g--a15 + multi-docker +taishan200-2280-2s48p-256g--a16 + multi-docker +taishan200-2280-2s48p-256g--a17 + multi-docker +taishan200-2280-2s48p-256g--a18 + multi-docker +taishan200-2280-2s48p-256g--a19 + multi-docker +taishan200-2280-2s48p-256g--a20 + multi-docker +taishan200-2280-2s48p-256g--a21 + pub-tbox +taishan200-2280-2s48p-256g--a22 + pub-tbox +taishan200-2280-2s48p-256g--a24 + long-runtime +taishan200-2280-2s48p-256g--a25 + long-runtime +taishan200-2280-2s48p-256g--a26 + long-runtime +taishan200-2280-2s48p-256g--a27 + long-runtime +taishan200-2280-2s48p-256g--a28 + long-runtime +taishan200-2280-2s48p-256g--a29 + long-runtime +taishan200-2280-2s48p-256g--a30 + long-runtime +taishan200-2280-2s48p-256g--a31 + long-runtime +taishan200-2280-2s48p-256g--a32 + long-runtime +taishan200-2280-2s48p-256g--a33 + long-runtime +taishan200-2280-2s48p-256g--a34 + long-runtime +taishan200-2280-2s48p-256g--a35 + long-runtime +taishan200-2280-2s48p-256g--a49 + pub-tbox +taishan200-2280-2s48p-256g--a53 + pub-tbox +taishan200-2280-2s48p-256g--a60 + multi-qemu +taishan200-2280-2s48p-256g--a61 + multi-qemu +taishan200-2280-2s48p-256g--a62 + multi-qemu +taishan200-2280-2s48p-256g--a63 + pub-tbox +taishan200-2280-2s48p-256g--a64 + pub-tbox +taishan200-2280-2s48p-256g--a65 + pub-tbox +taishan200-2280-2s48p-256g--a66 + pub-tbox +taishan200-2280-2s48p-256g--a67 + pub-tbox +taishan200-2280-2s48p-256g--a68 + pub-tbox +taishan200-2280-2s48p-256g--a69 + pub-tbox +taishan200-2280-2s48p-256g--a70 + pub-tbox +taishan200-2280-2s48p-256g--a71 + pub-tbox +taishan200-2280-2s48p-256g--a84 + long-runtime +taishan200-2280-2s48p-256g--a85 + long-runtime +taishan200-2280-2s48p-256g--a86 + long-runtime +taishan200-2280-2s48p-256g--a87 + long-runtime +taishan200-2280-2s48p-256g--a88 + long-runtime +taishan200-2280-2s48p-256g--a89 + long-runtime +taishan200-2280-2s48p-256g--a90 + long-runtime +taishan200-2280-2s48p-256g--a91 + long-runtime +taishan200-2280-2s48p-256g--a92 + long-runtime +taishan200-2280-2s48p-256g--a93 + long-runtime +taishan200-2280-2s48p-256g--a94 + long-runtime +taishan200-2280-2s48p-256g--a96 + long-runtime +taishan200-2280-2s48p-256g--a97 + long-runtime +taishan200-2280-2s48p-256g--a98 + long-runtime + Board-test +taishan200-2280-2s48p-256g--a99 + long-runtime +taishan200-2280-2s48p-256g--a100 + long-runtime +taishan200-2280-2s48p-256g--a101 + long-runtime +taishan200-2280-2s48p-256g--a103 + long-runtime +taishan200-2280-2s48p-256g--a104 + long-runtime +taishan200-2280-2s48p-256g--a106 + long-runtime +taishan200-2280-2s48p-256g--a107 + long-runtime +# aarch64: taishan200-2280-2s48p-512g +taishan200-2280-2s48p-512g--a102 + long-runtime +taishan200-2280-2s48p-512g--a105 + long-runtime +taishan200-2280-2s48p-512g--a95 + long-runtime +# aarch64: taishan200-2280-2s64p-256g +taishan200-2280-2s64p-256g--a3 + Board-test +taishan200-2280-2s64p-256g--a4 + pub-tbox +taishan200-2280-2s64p-256g--a5 + pub-tbox +taishan200-2280-2s64p-256g--a6 + pub-tbox +taishan200-2280-2s64p-256g--a7 + pub-tbox +taishan200-2280-2s64p-256g--a8 + pub-tbox +taishan200-2280-2s64p-256g--a9 + pub-tbox +taishan200-2280-2s64p-256g--a10 + pub-tbox +taishan200-2280-2s64p-256g--a36 + pub-tbox +taishan200-2280-2s64p-256g--a37 + pub-tbox +taishan200-2280-2s64p-256g--a38 + pub-tbox +taishan200-2280-2s64p-256g--a39 + pub-tbox +taishan200-2280-2s64p-256g--a40 + pub-tbox +taishan200-2280-2s64p-256g--a41 + pub-tbox +taishan200-2280-2s64p-256g--a42 + pub-tbox +taishan200-2280-2s64p-256g--a43 + pub-tbox +taishan200-2280-2s64p-256g--a44 + pub-tbox +taishan200-2280-2s64p-256g--a45 + pub-tbox +taishan200-2280-2s64p-256g--a46 + pub-tbox +taishan200-2280-2s64p-256g--a78 + pub-tbox +taishan200-2280-2s64p-256g--a79 + pub-tbox +taishan200-2280-2s64p-256g--a80 + pub-tbox +taishan200-2280-2s64p-256g--a81 + pub-tbox +taishan200-2280-2s64p-256g--a82 + pub-tbox +taishan200-2280-2s64p-256g--a83 + pub-tbox +taishan200-2280-2s64p-256g--a108 + long-runtime +taishan200-2280-2s64p-256g--a109 + long-runtime +taishan200-2280-2s64p-256g--a110 + long-runtime +taishan200-2280-2s64p-256g--a111 + long-runtime +taishan200-2280-2s64p-256g--a112 + long-runtime +taishan200-2280-2s64p-256g--a113 + long-runtime +taishan200-2280-2s64p-256g--a114 + long-runtime +taishan200-2280-2s64p-256g--a115 + long-runtime +taishan200-2280-2s64p-256g--a116 + long-runtime +taishan200-2280-2s64p-256g--a117 + long-runtime +taishan200-2280-2s64p-256g--a118 + long-runtime +taishan200-2280-2s64p-256g--a119 + long-runtime +taishan200-2280-2s64p-256g--a120 + long-runtime +taishan200-2280-2s64p-256g--a121 + long-runtime +taishan200-2280-2s64p-256g--a122 + long-runtime +taishan200-2280-2s64p-256g--a123 + long-runtime +taishan200-2280-2s64p-256g--a124 + long-runtime +taishan200-2280-2s64p-256g--a125 + long-runtime +taishan200-2280-2s64p-256g--a126 + long-runtime +taishan200-2280-2s64p-256g--a127 + long-runtime +taishan200-2280-2s64p-256g--a128 + long-runtime +taishan200-2280-2s64p-256g--a129 + long-runtime +taishan200-2280-2s64p-256g--a130 + long-runtime +taishan200-2280-2s64p-256g--a131 + long-runtime -- 2.23.0

2 2

[PATCH compass-ci 2/2] sparrow/dev-env: add the public_service.yaml
by Cao Xueliang 05 Feb '21

05 Feb '21

The scheduler will read public_service.yaml, then add the public service info to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..a7eae4d 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -54,6 +54,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/public_service.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: delete read "/etc/compass-ci/defaults/*.yaml"
by Cao Xueliang 05 Feb '21

05 Feb '21

The compass-ci/scheduler will read the yamls, so, delete it in lkp-tests Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/job.rb | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6dc..015fb840f 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -420,8 +420,7 @@ class Job def load_self_config self_config_path = "#{ENV['HOME']}/.config/compass-ci" - Dir.glob(['/etc/compass-ci/defaults/*.yaml', - "#{self_config_path}/defaults/*.yaml"]).each do |file| + Dir.glob(["#{self_config_path}/defaults/*.yaml"]).each do |file| load_one_defaults(file, @job) end -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: change the document name from Chinese to English
by Li Ping 05 Feb '21

05 Feb '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- .../manual/install-cci-client.md | 0 .../manual/log-in-machine-debug.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" => doc/manual/install-cci-client.md (100%) rename "doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" => doc/manual/log-in-machine-debug.md (100%) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" b/doc/manual/install-cci-client.md similarity index 100% rename from "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" rename to doc/manual/install-cci-client.md diff --git "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" b/doc/manual/log-in-machine-debug.md similarity index 100% rename from "doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" rename to doc/manual/log-in-machine-debug.md -- 2.23.0

1 0

[PATCH v2 compass-ci] container/srv-http: download h5ai repo
by Liu Yinsi 05 Feb '21

05 Feb '21

before: copy h5ai repo from our server /c/h5ai into image after: download h5ai repo when build container [why] download h5ai in docker image, make build script more clean. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/srv-http/Dockerfile | 8 ++++---- container/srv-http/build | 7 ------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/container/srv-http/Dockerfile b/container/srv-http/Dockerfile index 987e40e..6564730 100644 --- a/container/srv-http/Dockerfile +++ b/container/srv-http/Dockerfile @@ -5,16 +5,16 @@ FROM alpine:3.9 ADD root / -RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm \ - && mkdir /run/nginx/ \ - && mkdir /h5ai +RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm git \ + && mkdir /run/nginx/ RUN sed -i '/\[global\]/a daemonize = no' /etc/php7/php-fpm.conf RUN sed -i "s/user = nobody/user = nginx/g" /etc/php7/php-fpm.d/www.conf RUN sed -i "s/group = nobody/group = nginx/g" /etc/php7/php-fpm.d/www.conf +RUN git clone https://gitee.com/luweitao_y/h5ai.git /h5ai + WORKDIR /h5ai -COPY h5ai /h5ai # build h5ai package RUN npm install \ diff --git a/container/srv-http/build b/container/srv-http/build index 054e826..3a500ea 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,11 +2,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${H5AI_SRC:=/c/h5ai} - -cp -r $H5AI_SRC ./ - docker build -t srv-http . - -# clean h5ai folder -rm -rf ./h5ai -- 2.23.0

1 0

[PATCH compass-ci] container/srv-http: download h5ai repo
by Liu Yinsi 05 Feb '21

05 Feb '21

before: copy h5ai repo from our server /c/h5ai into image after: download h5ai repo when build container [why] download h5ai in docker image, make build script more clear. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/srv-http/Dockerfile | 8 ++++---- container/srv-http/build | 7 ------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/container/srv-http/Dockerfile b/container/srv-http/Dockerfile index 987e40e..1f9b5ca 100644 --- a/container/srv-http/Dockerfile +++ b/container/srv-http/Dockerfile @@ -5,16 +5,16 @@ FROM alpine:3.9 ADD root / -RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm \ - && mkdir /run/nginx/ \ - && mkdir /h5ai +RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm git\ + && mkdir /run/nginx/ RUN sed -i '/\[global\]/a daemonize = no' /etc/php7/php-fpm.conf RUN sed -i "s/user = nobody/user = nginx/g" /etc/php7/php-fpm.d/www.conf RUN sed -i "s/group = nobody/group = nginx/g" /etc/php7/php-fpm.d/www.conf +RUN git clone https://gitee.com/luweitao_y/h5ai.git /h5ai + WORKDIR /h5ai -COPY h5ai /h5ai # build h5ai package RUN npm install \ diff --git a/container/srv-http/build b/container/srv-http/build index 054e826..3a500ea 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,11 +2,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${H5AI_SRC:=/c/h5ai} - -cp -r $H5AI_SRC ./ - docker build -t srv-http . - -# clean h5ai folder -rm -rf ./h5ai -- 2.23.0

2 2

[PATCH v3 compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 05 Feb '21

05 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..8812839 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml + +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) +[ -z $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 05 Feb '21

05 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..4503080 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,11 @@ +# please fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix code error
by Liu Yinsi 05 Feb '21

05 Feb '21

[error] lys@crystal ~/compass-ci/container/register-accounts% ./start register-accounts register-accounts bash-5.1# ./register-accounts awk: /etc/compass-ci/defaults/*.yaml: No such file or directory Traceback (most recent call last): 10: from ./register-accounts:8:in `<main>' 9: from ./register-accounts:8:in `require_relative' 8: from /c/compass-ci/sbin/build-my-info.rb:55:in `<top (required)>' 7: from /c/compass-ci/sbin/build-my-info.rb:48:in `build_my_info' 6: from /c/compass-ci/lib/build_my_info_client.rb:30:in `search_my_info' 5: from /c/compass-ci/lib/es_query.rb:58:in `query_by_id' 4: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-api-7.10.1/lib/elasticsearch/api/actions/search.rb:103:in `search' 3: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/client.rb:182:in `perform_request' 2: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request' 1: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:347:in `perform_request' /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error': [400] {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"LOhdoqAnRYG09044aFiHUw","index":"accounts"},{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"d1F49OYCTxmCIDe3HZDKAw","index":"accounts1"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"accounts","node":"keFEKD-WTBe0tHF4fbS4MA","reason":{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"LOhdoqAnRYG09044aFiHUw","index":"accounts","caused_by":{"type":"illegal_argument_exception","reason":"Ids can't be empty"}}},{"shard":0,"index":"accounts1","node":"keFEKD-WTBe0tHF4fbS4MA","reason":{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"d1F49OYCTxmCIDe3HZDKAw","index":"accounts1","caused_by":{"type":"illegal_argument_exception","reason":"Ids can't be empty"}}}]},"status":400} (Elasticsearch::Transport::Transport::Errors::BadRequest) [why] sbin/build-my-info.rb: ... option = { 'my_email' => `git config --global user.email`.chomp, 'my_name' => `git config --global user.name`.chomp, 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } ... build_my_info(option) when call sbin/build-my-info.rb with params my_info the defaults option already exists in build-my-info.rb, then will cause conflict, so call lib/build_my_info_client.rb Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/register-accounts/register-accounts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index e4d39ff..b55e293 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,7 +5,7 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" +require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" content = YAML.load_file('/etc/compass-ci/register/register.yaml') content.each do |_key, value| @@ -13,7 +13,9 @@ content.each do |_key, value| my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] + my_info['my_token'] = %x(uuidgen).chomp - build_my_info(my_info) + build_my_info = BuildMyInfo.new(my_info['my_email']) + build_my_info.config_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH lkp-tests 2/2] installer/kali: add kali installer symlink
by Wang Yong 05 Feb '21

05 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/kali | 1 + 1 file changed, 1 insertion(+) create mode 120000 distro/installer/kali diff --git a/distro/installer/kali b/distro/installer/kali new file mode 120000 index 000000000..b2f7fd3e9 --- /dev/null +++ b/distro/installer/kali @@ -0,0 +1 @@ +debian \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests 1/2] distro/kali: add kali distro file
by Wang Yong 05 Feb '21

05 Feb '21

add kali pack method Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/kali | 1 + 1 file changed, 1 insertion(+) create mode 120000 distro/kali diff --git a/distro/kali b/distro/kali new file mode 120000 index 000000000..b2f7fd3e9 --- /dev/null +++ b/distro/kali @@ -0,0 +1 @@ +debian \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests] stats/ansible_stats_openeuler: stats result for ansible migrating
by Wang Chenglong 05 Feb '21

05 Feb '21

[How] ansible_stats_openeuler ${job_output} ./ansible_stats_openeuler /srv/result/ansible_test/2021-02-04/vm-2p16g--wcl/openeuler-20.03-aarch64/linux-system-roles-storage/crystal.974233/output Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_stats_openeuler | 48 +++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100755 stats/ansible_stats_openeuler diff --git a/stats/ansible_stats_openeuler b/stats/ansible_stats_openeuler new file mode 100755 index 000000000..5a6870eef --- /dev/null +++ b/stats/ansible_stats_openeuler @@ -0,0 +1,48 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' + +job_error_msg = `grep -E 'FAILED! | failed: .*}$' #{ARGV[0]} | awk -F'=>' '{$1="";print $0}'` +error_msg_json = JSON.parse(job_error_msg) + +unless error_msg_json['msg'].nil? + puts "config.error.no-file-find-by.#{$1}: 1" if error_msg_json['msg'].match(/No file was found when using (.+)\./) + + puts "yaml.error.syntax-problem.[#{$1}]: 1" if error_msg_json['msg'].match(/The error was: (.+)/) + + if error_msg_json['msg'].match(/No such file or directory/) + puts "command.error.no-such-command.#{error_msg_json['cmd']}: 1" + end + + if error_msg_json['msg'].match(/Failed to download metadata for repo '(.+)': /) + puts "download.error.failed-to-download.#{$1}: 1" + end + + puts "script.error [#{error_msg_json['cmd']}]: 1" if error_msg_json['msg'].match(/non-zero return code/) + + puts "download.error.http-error-404 [#{$1}]: 1" if error_msg_json['msg'].match(/Failure downloading (.+), /) + + if error_msg_json['msg'].match(/Could not find the requested service firewall: (.+)/) + puts "firewall.error.could-not-find-server.#{$1}: 1" + end + + puts "service.error.unable-to-star-service.#{$1}: 1" if error_msg_json['msg'].match(/Unable to start service (.+):/) + + puts 'system.error.not-supporte: 1' if error_msg_json['msg'].match(/is not supporte/) + + puts "depsolve.error.depend-package-error.#{$1}: 1" if error_msg_json['msg'].match(/needed by (.+)/) + + puts 'read.error.read-timed-out: 1' if error_msg_json['msg'].match(/failed to create temporary content file/) +end + +error_msg_json['failures']&.each do |items| + items.match(/No package (.+) available/) + puts "yum.error.no-package-match.#{$1}: 1" +end + +unless error_msg_json['message'].nil? + puts "yaml.error.not-find-yml.#{$1}: 1" if a['message'].to_s.match(/Could not find or access (.+)/) +end -- 2.23.0

1 0

[PATCH compass-ci] providers/lib: fix wget in multi-process mode
by Xiao Shenwei 05 Feb '21

05 Feb '21

[why] Redirecting output to ‘wget-log’. Redirecting output to ‘wget-log.1’. Redirecting output to ‘wget-log.2’. Redirecting output to ‘wget-log.3’. [how] append info to log file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/resource.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/lib/resource.rb b/providers/lib/resource.rb index bcce612..4279766 100644 --- a/providers/lib/resource.rb +++ b/providers/lib/resource.rb @@ -56,7 +56,7 @@ class Resource end def load_file(url) - system "wget --timestamping --progress=bar:force #{url}" + system "wget --timestamping -a #{(a)hostname}.log --progress=bar:force #{url}" basename = File.basename(url) file_size = %x(ls -s --block-size=M "#{basename}").chomp @logger.info("Load file size: #{file_size}") -- 2.23.0

1 0

[PATCH lab-z9] add physical machine information
by Bai Jing 05 Feb '21

05 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- hosts/taishan200-2280-2s64p-256g--a123 | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a123 diff --git a/hosts/taishan200-2280-2s64p-256g--a123 b/hosts/taishan200-2280-2s64p-256g--a123 new file mode 100644 index 0000000..f765092 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a123 @@ -0,0 +1,19 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-3500003998881eb19 + - /dev/disk/by-id/scsi-3500003998881e9a1 +mac_addr: + - 44:a1:91:9d:f5:4a + - 44:a1:91:9d:f5:4b + - 44:a1:91:9d:f5:4c + - 44:a1:91:9d:f5:4d + - 44:a1:91:e5:a8:e1 + - 44:a1:91:e5:a8:e2 + - 44:a1:91:e5:a8:e3 + - 44:a1:91:e5:a8:e4 +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.12.4 -- 2.23.0

1 0

[PATCH compass-ci 4/4] sparrow/4-docker: optimise function log_info() calling
by Liu Yinsi 05 Feb '21

05 Feb '21

before: just execute $container/build script, echo "build $container" after: only execute $container/build -> `docker build -t`, echo "build $container" [why] when execute `sparrow/4-docker/buildall run-only`, will execute $container/build script, but if image already exist, actually will not build container, it will exit code "1", so only exit code "0", call log_info "finish build $container" Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 83ecc9f..b24eca5 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -29,13 +29,9 @@ do_one_build() mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" - - log_info "start build $container." - [ -x build ] && ./build + [ "$?" == 0 ] && log_info "finish build $container" [ -x install ] && ./install - - log_info "finish build $container." ) lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

1 0

[PATCH compass-ci 3/4] sparrow/4-docker: export env DOCKER_SKIP_REBUILD
by Liu Yinsi 05 Feb '21

05 Feb '21

if env DOCKER_SKIP_REBUILD exists, then check image exists in $container/build script by call docker_skip_rebuild(), then not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index d4e9419..83ecc9f 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -70,11 +70,9 @@ do container_name=$(basename $dir) [ "$container_name" == 'ssh-r' ] && continue - docker images | grep -wq "$container_name" - if [ "$?" != 0 ] || [ "$action" != "run-only" ]; then - build_depends $dir & - wait - fi + [ "$action" == "run-only" ] && export DOCKER_SKIP_REBUILD=DOCKER_SKIP_REBUILD + build_depends $dir & + wait start_depends $dir done -- 2.23.0

1 0

[PATCH compass-ci 2/4] container: call docker_skip_rebuild()
by Liu Yinsi 05 Feb '21

05 Feb '21

prevent rebuild container when execute `sparrow/4-docker/buildall run-only` Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/add-software/build | 4 ++++ container/alpine/build | 4 ++++ container/archlinux/build | 4 ++++ container/assign-account/build | 4 ++++ container/assist-result/build | 4 ++++ container/assistant/build | 4 ++++ container/auto-submit/build | 4 ++++ container/conserver/build | 4 ++++ container/crystal-base/build | 4 ++++ container/crystal-compiler/build | 4 ++++ container/crystal-shards/build | 3 +++ container/debian/build | 4 ++++ container/delimiter/build | 2 ++ container/dnsmasq/build | 4 ++++ container/dracut-initrd/build | 4 ++++ container/es/build | 2 ++ container/extract-stats/build | 4 ++++ container/fetch-mail/build | 4 ++++ container/fluentd-base/build | 4 ++++ container/git-daemon/build | 4 ++++ container/git-mirror/build | 4 ++++ container/gitcache/build | 4 ++++ container/initrd-cifs/build | 4 ++++ container/initrd-http/build | 4 ++++ container/kibana/build | 4 ++++ container/lifecycle/build | 2 ++ container/lkp-initrd/build | 4 ++++ container/logging-es/build | 2 ++ container/mail-robot/build | 4 ++++ container/manjaro/build | 4 ++++ container/master-fluentd/build | 4 ++++ container/minio/build | 4 ++++ container/monitoring/build | 2 ++ container/netdata-slave/build | 4 ++++ container/netdata/build | 4 ++++ container/ntp-server/build | 4 ++++ container/openresty-proxy-cache/build | 4 ++++ container/os-cifs/build | 4 ++++ container/os-http/build | 4 ++++ container/os-nfs/build | 4 ++++ container/qcow2rootfs/build | 4 ++++ container/qemu-efi/build | 4 ++++ container/rabbitmq/build | 4 ++++ container/redis/build | 4 ++++ container/register-accounts/build | 4 ++++ container/registry/build | 4 ++++ container/remote-git/build | 4 ++++ container/result-cifs/build | 4 ++++ container/result-webdav/build | 4 ++++ container/rsync-server/build | 4 ++++ container/scheduler-dev/build | 4 ++++ container/scheduler/build | 2 ++ container/send-internet-mail/build | 4 ++++ container/send-mail/build | 4 ++++ container/shellcheck/build | 4 ++++ container/srv-http/build | 4 ++++ container/ssh-r/build | 4 ++++ container/sub-fluentd/build | 4 ++++ container/submit/build | 4 ++++ container/taskqueue/build | 2 ++ container/upload-libvirt-xml/build | 4 ++++ container/web-backend-nginx/build | 4 ++++ container/web-backend/build | 4 ++++ container/webhook/build | 4 ++++ 64 files changed, 241 insertions(+) diff --git a/container/add-software/build b/container/add-software/build index 234c8b9..bc3c95a 100755 --- a/container/add-software/build +++ b/container/add-software/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "addsoftware:auto" + docker build -t addsoftware:auto . diff --git a/container/alpine/build b/container/alpine/build index bd277c3..3168799 100755 --- a/container/alpine/build +++ b/container/alpine/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:testbed" + docker build -t alpine:testbed . diff --git a/container/archlinux/build b/container/archlinux/build index 9749489..6db5a68 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "archlinux:testbed" + docker build --build-arg ARCH=$(arch) -t archlinux:testbed . diff --git a/container/assign-account/build b/container/assign-account/build index 3984341..480c410 100755 --- a/container/assign-account/build +++ b/container/assign-account/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "assign-account:latest" + docker build -t assign-account:latest . diff --git a/container/assist-result/build b/container/assist-result/build index 130127b..600b80a 100755 --- a/container/assist-result/build +++ b/container/assist-result/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "assist-result" + docker build -t assist-result . diff --git a/container/assistant/build b/container/assistant/build index 7778c9f..89a8a97 100755 --- a/container/assistant/build +++ b/container/assistant/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:assistant" + docker build -t debian:assistant . diff --git a/container/auto-submit/build b/container/auto-submit/build index 96383e3..48ff111 100755 --- a/container/auto-submit/build +++ b/container/auto-submit/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:auto-submit" + docker build -t alpine:auto-submit . diff --git a/container/conserver/build b/container/conserver/build index 0700c5e..a7e3867 100755 --- a/container/conserver/build +++ b/container/conserver/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "conserver_server" + ./generate_conserver.rb || exit 1 docker build -t conserver_server . diff --git a/container/crystal-base/build b/container/crystal-base/build index 4df65a5..642e190 100755 --- a/container/crystal-base/build +++ b/container/crystal-base/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-base" + if [ -n "$SQUID_HOST" ] && [ -n "$SQUID_PORT" ]; then proxy="http://$SQUID_HOST:$SQUID_PORT/" else diff --git a/container/crystal-compiler/build b/container/crystal-compiler/build index cee43eb..570fe98 100755 --- a/container/crystal-compiler/build +++ b/container/crystal-compiler/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-compiler" + docker build -t alpine:crystal-compiler . diff --git a/container/crystal-shards/build b/container/crystal-shards/build index 784f571..7c2a712 100755 --- a/container/crystal-shards/build +++ b/container/crystal-shards/build @@ -6,6 +6,9 @@ # If job.yaml does not have these two parameters, git will use the local proxy cache, # GITCACHE_HOST defaults to the local ip, GITCACHE_PORT default port is 5000 . ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-shards" + load_cci_defaults docker build --build-arg GITCACHE_HOST=$GITCACHE_HOST --build-arg GITCACHE_PORT=$GITCACHE_PORT -t alpine:crystal-shards . diff --git a/container/debian/build b/container/debian/build index 6bff579..eac9af2 100755 --- a/container/debian/build +++ b/container/debian/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:testbed" + docker build -t debian:testbed . diff --git a/container/delimiter/build b/container/delimiter/build index f7f53e7..606149d 100755 --- a/container/delimiter/build +++ b/container/delimiter/build @@ -4,6 +4,8 @@ . ../defconfig.sh +docker_skip_rebuild "delimiter" + load_cci_defaults docker build -t delimiter . diff --git a/container/dnsmasq/build b/container/dnsmasq/build index 8a22575..b5738bc 100755 --- a/container/dnsmasq/build +++ b/container/dnsmasq/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "dnsmasq:alpine" + [[ -e /tftpboot/boot.ipxe ]] || { cp tftpboot/boot.ipxe /tftpboot/boot.ipxe } diff --git a/container/dracut-initrd/build b/container/dracut-initrd/build index 181b896..f02cea9 100755 --- a/container/dracut-initrd/build +++ b/container/dracut-initrd/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:dracut" + docker build -t debian:dracut . diff --git a/container/es/build b/container/es/build index 17a225c..c7d7115 100755 --- a/container/es/build +++ b/container/es/build @@ -5,6 +5,8 @@ require_relative '../defconfig.rb' +docker_skip_rebuild "es643b:alpine311" + available_memory = get_available_memory system "docker build -t es643b:alpine311 --build-arg MEMORY=#{available_memory} --network=host ." diff --git a/container/extract-stats/build b/container/extract-stats/build index 1aa9982..c315f0c 100755 --- a/container/extract-stats/build +++ b/container/extract-stats/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "extract-stats" + CCI_SRC=$CCI_SRC bash $CCI_SRC/container/compile . diff --git a/container/fetch-mail/build b/container/fetch-mail/build index e72bed8..9b1a501 100755 --- a/container/fetch-mail/build +++ b/container/fetch-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "fetch-mail:latest" + docker build -t fetch-mail:latest . diff --git a/container/fluentd-base/build b/container/fluentd-base/build index 32e0a38..72b85bf 100755 --- a/container/fluentd-base/build +++ b/container/fluentd-base/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "fluentd-base:alpine" + system 'docker build -t fluentd-base:alpine .' diff --git a/container/git-daemon/build b/container/git-daemon/build index d81e4b9..5ba9a52 100755 --- a/container/git-daemon/build +++ b/container/git-daemon/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "apline311:git-daemon" + docker build -t apline311:git-daemon . diff --git a/container/git-mirror/build b/container/git-mirror/build index 66967e4..c302ac7 100755 --- a/container/git-mirror/build +++ b/container/git-mirror/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine38:git-mirror" + docker build -t alpine38:git-mirror . diff --git a/container/gitcache/build b/container/gitcache/build index 5135f7c..d7f3523 100755 --- a/container/gitcache/build +++ b/container/gitcache/build @@ -3,4 +3,8 @@ # Copyright (C) 2019-2020 Eric D.little51 # SPDX-License-Identifier: GPL-3.0 +. ../defconfig.sh + +docker_skip_rebuild "gitcache" + docker build -t gitcache . diff --git a/container/initrd-cifs/build b/container/initrd-cifs/build index 4b93647..19d5125 100755 --- a/container/initrd-cifs/build +++ b/container/initrd-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "initrd-cifs" + docker build -t initrd-cifs . diff --git a/container/initrd-http/build b/container/initrd-http/build index 1b039dc..22babcd 100755 --- a/container/initrd-http/build +++ b/container/initrd-http/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "initrd-http" + docker build -t initrd-http . diff --git a/container/kibana/build b/container/kibana/build index 52d5a2a..9a1fb6d 100755 --- a/container/kibana/build +++ b/container/kibana/build @@ -3,6 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "kibana:7.6.2" + BASE_IMAGE_DICT = { 'aarch64' => 'gagara/kibana-oss-arm64:7.6.2', 'x86_64' => 'kibana:7.6.2' }.freeze diff --git a/container/lifecycle/build b/container/lifecycle/build index bfb7490..00a9c84 100755 --- a/container/lifecycle/build +++ b/container/lifecycle/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' +docker_skip_rebuild "lifecycle-#{LIFECYCLE_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "docker build -t lifecycle-#{LIFECYCLE_PORT} ." diff --git a/container/lkp-initrd/build b/container/lkp-initrd/build index 2383323..cfba617 100755 --- a/container/lkp-initrd/build +++ b/container/lkp-initrd/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:lkp" + docker build -t alpine:lkp . diff --git a/container/logging-es/build b/container/logging-es/build index 6465ace..3be841a 100755 --- a/container/logging-es/build +++ b/container/logging-es/build @@ -5,6 +5,8 @@ require_relative '../defconfig.rb' +docker_skip_rebuild "logging-es:7.6.2" + BASE_IMAGE_DICT = { 'aarch64' => 'gagara/elasticsearch-oss-arm64:7.6.2', 'x86_64' => 'elasticsearch:7.6.2' diff --git a/container/mail-robot/build b/container/mail-robot/build index c564add..4483e89 100755 --- a/container/mail-robot/build +++ b/container/mail-robot/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "mail-robot:latest" + docker build -t mail-robot:latest . diff --git a/container/manjaro/build b/container/manjaro/build index 114e550..6aeb3c6 100755 --- a/container/manjaro/build +++ b/container/manjaro/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "manjaro:testbed" + declare -A BASE_IMAGE_DICT BASE_IMAGE_DICT=( diff --git a/container/master-fluentd/build b/container/master-fluentd/build index 7d0291d..987cfcd 100755 --- a/container/master-fluentd/build +++ b/container/master-fluentd/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "master-fluentd:alpine" + system 'docker build -t master-fluentd:alpine .' diff --git a/container/minio/build b/container/minio/build index d1ccd06..b0909e9 100755 --- a/container/minio/build +++ b/container/minio/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "jessestuart/minio" + docker pull jessestuart/minio diff --git a/container/monitoring/build b/container/monitoring/build index bcb0c37..121ede0 100755 --- a/container/monitoring/build +++ b/container/monitoring/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) MONITOR_PORT = (defaults['MONITOR_PORT'] || '11310') +docker_skip_rebuild "monitoring-#{MONITOR_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "docker build -t monitoring-#{MONITOR_PORT} ." diff --git a/container/netdata-slave/build b/container/netdata-slave/build index 1a84550..cb53364 100755 --- a/container/netdata-slave/build +++ b/container/netdata-slave/build @@ -2,4 +2,8 @@ # Origin: https://hub.docker.com/r/netdata/netdata # SPDX-License-Identifier: GPL-3.0+ +. ../defconfig.sh + +docker_skip_rebuild "netdata/netdata-slave" + docker build -t netdata/netdata-slave . diff --git a/container/netdata/build b/container/netdata/build index 809b445..6aa9a17 100755 --- a/container/netdata/build +++ b/container/netdata/build @@ -2,4 +2,8 @@ # Origin: https://hub.docker.com/r/netdata/netdata # SPDX-License-Identifier: GPL-3.0+ +. ../defconfig.sh + +docker_skip_rebuild "netdata/netdatalkp" + docker build -t netdata/netdatalkp . diff --git a/container/ntp-server/build b/container/ntp-server/build index 7826fb2..d52f9dd 100755 --- a/container/ntp-server/build +++ b/container/ntp-server/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "ntp-server" + docker build -t ntp-server . diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 507c4fe..7665209 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "openresty:proxy_cache" + [ -d "openresty" ] && rm -rf openresty git clone https://gitee.com/cuiyili/openresty.git || exit 1 diff --git a/container/os-cifs/build b/container/os-cifs/build index 77be8e9..de44c44 100755 --- a/container/os-cifs/build +++ b/container/os-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-cifs" + docker build -t os-cifs . diff --git a/container/os-http/build b/container/os-http/build index c1cb008..367748e 100755 --- a/container/os-http/build +++ b/container/os-http/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-http" + docker build -t os-http . diff --git a/container/os-nfs/build b/container/os-nfs/build index eea5a7d..e0fd476 100755 --- a/container/os-nfs/build +++ b/container/os-nfs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-nfs" + docker build -t os-nfs . diff --git a/container/qcow2rootfs/build b/container/qcow2rootfs/build index 565de24..7f31bb4 100755 --- a/container/qcow2rootfs/build +++ b/container/qcow2rootfs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "qcow2rootfs:auto" + docker build -t qcow2rootfs:auto . diff --git a/container/qemu-efi/build b/container/qemu-efi/build index cac9426..4f1d488 100755 --- a/container/qemu-efi/build +++ b/container/qemu-efi/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:qemu-efi" + [ $(arch) == "aarch64" ] && uefi=qemu-efi-aarch64 [ $(arch) == "x86_64" ] && uefi=ovmf diff --git a/container/rabbitmq/build b/container/rabbitmq/build index c0bc179..930c112 100755 --- a/container/rabbitmq/build +++ b/container/rabbitmq/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "rabbitmq:3-management" + docker pull rabbitmq:3-management diff --git a/container/redis/build b/container/redis/build index e5230b0..490f99d 100755 --- a/container/redis/build +++ b/container/redis/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "redis507n:alpine311" + docker build -t redis507n:alpine311 . diff --git a/container/register-accounts/build b/container/register-accounts/build index e1dcfa4..2364b34 100755 --- a/container/register-accounts/build +++ b/container/register-accounts/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "register-accounts" + docker build -t register-accounts . diff --git a/container/registry/build b/container/registry/build index ca8680a..2edc2c1 100755 --- a/container/registry/build +++ b/container/registry/build @@ -2,4 +2,8 @@ # Origin: https://docs.docker.com/registry # SPDX-License-Identifier: Apache license +. ../defconfig.sh + +docker_skip_rebuild "registry" + docker pull registry diff --git a/container/remote-git/build b/container/remote-git/build index dad1a9b..727c7f9 100755 --- a/container/remote-git/build +++ b/container/remote-git/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:remote-git" + docker build -t debian:remote-git . diff --git a/container/result-cifs/build b/container/result-cifs/build index d97b6dd..fd9d2df 100755 --- a/container/result-cifs/build +++ b/container/result-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "result-cifs" + docker build -t result-cifs . diff --git a/container/result-webdav/build b/container/result-webdav/build index 7d70ae6..b5be9e9 100755 --- a/container/result-webdav/build +++ b/container/result-webdav/build @@ -4,4 +4,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "result-webdav" + docker build -t result-webdav . diff --git a/container/rsync-server/build b/container/rsync-server/build index 72bfc47..5d36265 100755 --- a/container/rsync-server/build +++ b/container/rsync-server/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "rsync_server" + docker build -t rsync_server . diff --git a/container/scheduler-dev/build b/container/scheduler-dev/build index 1e4c57e..56d27c0 100755 --- a/container/scheduler-dev/build +++ b/container/scheduler-dev/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:scheduler-dev" + docker build -t alpine:scheduler-dev . diff --git a/container/scheduler/build b/container/scheduler/build index 367aa4e..412988a 100755 --- a/container/scheduler/build +++ b/container/scheduler/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) SCHED_PORT = (defaults['SCHED_PORT'] || '3000') +docker_skip_rebuild "sch-ruby-a:v0.00d-#{SCHED_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "cp #{CCI_SRC}/sbin/create-job-cpio.sh ." diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build index 078adaf..07ca143 100755 --- a/container/send-internet-mail/build +++ b/container/send-internet-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "send-internet-mail:latest" + docker build -t send-internet-mail:latest . diff --git a/container/send-mail/build b/container/send-mail/build index 9159641..8ae9788 100755 --- a/container/send-mail/build +++ b/container/send-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "send-mail:latest" + docker build -t send-mail:latest . diff --git a/container/shellcheck/build b/container/shellcheck/build index 854bae6..f295e51 100755 --- a/container/shellcheck/build +++ b/container/shellcheck/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "shellcheck" + docker build -t shellcheck . diff --git a/container/srv-http/build b/container/srv-http/build index 054e826..c8c9aa4 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "srv-http" + : ${H5AI_SRC:=/c/h5ai} cp -r $H5AI_SRC ./ diff --git a/container/ssh-r/build b/container/ssh-r/build index 4bec543..2ad1bd3 100755 --- a/container/ssh-r/build +++ b/container/ssh-r/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "ssh-r:0.001" + if [ ! -f "known_auth_keys" ]; then echo "-----> ERROR: no testbox rsa public key file 'known_auth_keys' at currrent DIR" exit 1 diff --git a/container/sub-fluentd/build b/container/sub-fluentd/build index 797cd0d..1ec2246 100755 --- a/container/sub-fluentd/build +++ b/container/sub-fluentd/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "sub-fluentd:alpine" + system 'docker build -t sub-fluentd:alpine .' diff --git a/container/submit/build b/container/submit/build index fba4281..dff884a 100755 --- a/container/submit/build +++ b/container/submit/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "submit:latest" + docker build -t submit:latest . diff --git a/container/taskqueue/build b/container/taskqueue/build index 3a5d4c9..4f69f85 100755 --- a/container/taskqueue/build +++ b/container/taskqueue/build @@ -10,6 +10,8 @@ load_cci_defaults service_port=${TASKQUEUE_PORT:-3060} +docker_skip_rebuild "taskqueue-alpine:v0.001-${service_port}" + if [[ ! -f "taskqueue" ]]; then $CCI_SRC/container/compile . else diff --git a/container/upload-libvirt-xml/build b/container/upload-libvirt-xml/build index d39838b..0a22173 100755 --- a/container/upload-libvirt-xml/build +++ b/container/upload-libvirt-xml/build @@ -4,6 +4,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "upload-libvirt-xml" + # The libvirt xml upload has nothing to do with the job. It renames the uploaded file and does not currently have permission to upload. # it is a transitional service. Two different functions use the same port service, diff --git a/container/web-backend-nginx/build b/container/web-backend-nginx/build index 4c9d3fe..d282570 100755 --- a/container/web-backend-nginx/build +++ b/container/web-backend-nginx/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "web-backend-nginx" + docker build -t web-backend-nginx . diff --git a/container/web-backend/build b/container/web-backend/build index 671fc97..c7f7d8a 100755 --- a/container/web-backend/build +++ b/container/web-backend/build @@ -1,4 +1,8 @@ #!/bin/sh # SPDX-License-Identifier: MulanPSL-2.0+ +. ../defconfig.sh + +docker_skip_rebuild "debian:web-backend" + docker build -t debian:web-backend . diff --git a/container/webhook/build b/container/webhook/build index bb5c0a7..a3475c4 100755 --- a/container/webhook/build +++ b/container/webhook/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "node:webhook" + docker build -t node:webhook . -- 2.23.0

1 0

[PATCH compass-ci] providers: add my-libvirt script for user
by Xiao Shenwei 05 Feb '21

05 Feb '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/my-libvirt | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100755 providers/my-libvirt diff --git a/providers/my-libvirt b/providers/my-libvirt new file mode 100755 index 0000000..f1ba374 --- /dev/null +++ b/providers/my-libvirt @@ -0,0 +1,13 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative './libvirt/run' + +tbox_group = 'vt-2p16g' +hostname = "#{tbox_group}.#{ENV['USER']}-#{Process.pid}" +# specify which queues will be request, use "," to separate more than 2 values +queues = "#{tbox_group}~#{ENV['USER']}" + +main(hostname, queues) -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: replace gem source
by Liu Yinsi 05 Feb '21

05 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/common | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 2d7c9e3..fca591c 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -2,6 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ gem install --no-user-install git activesupport rest-client cucumber json faye-websocket elasticsearch grep -q '^lkp:' /etc/passwd || useradd -u 1090 lkp -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] doc: add job/rootfs_disk.md
by Yu Chuan 05 Feb '21

05 Feb '21

Add description of rootfs_disk field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/rootfs_disk.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 doc/job/rootfs_disk.md diff --git a/doc/job/rootfs_disk.md b/doc/job/rootfs_disk.md new file mode 100644 index 000000000000..72762cf72f2f --- /dev/null +++ b/doc/job/rootfs_disk.md @@ -0,0 +1,27 @@ +# Summary +========= + +`rootfs_disk` defines a disk which storing multiple rootfs, used for jobs which need boot from disk. + +Each of these rootfs corresponds to an logical volume, such as follow: + /dev/mapper/ + . + ├── os-openeuler_aarch64_20.03 + ├── os-openeuler_aarch64_20.03_{timestamp} + ├── os-openeuler_aarch64_20.03_sp1 + ├── os-openeuler_aarch64_20.03_sp1_{timestamp} + └── ... + +Addition: +1. `/dev/mapper/os-{os}_{os_arch}_{os_version}_{timestamp}` + - is a readonly logical volume + - used for backup and rollback + +2. `/dev/mapper/os-{os}_{os_arch}_{os_version}` + - is a read write logical volume + - use for boot and run job + - defaultly, this logical volume will be wiped before every job run + +3. every logical volume size: 10G + +4. more info can refer os_mount.md -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] doc: add job/os_mount.md
by Yu Chuan 05 Feb '21

05 Feb '21

Add description of os_mount field in job.yaml. This patch is the description when os_mount=local, and other values of os_mount will be added in the patch later. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 89 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 doc/job/os_mount.md diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md new file mode 100644 index 000000000000..257bc12619fe --- /dev/null +++ b/doc/job/os_mount.md @@ -0,0 +1,89 @@ +# Summary +========= + +`os_mount` defines the mount method of testbox's root partition. + +It has the following optional values: + - nfs + - cifs + - initramfs + - container + - local + +Usage example: + - submit iperf.yaml testbox=vm-2p8g os_mount=nfs + - submit iperf.yaml testbox=vm-2p8g os_mount=cifs + - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs + - submit iperf.yaml testbox=dc-8g os_mount=container + - submit iperf.yaml testbox=vm-2p8g os_mount=local + +# Optional Values +================= + +## Work flow when os_mount=local + +1. user submit job with os_mount: local + - optional kernel cmdline params: # can be added in {kernel_append_root} field of job.yaml + - save_root: give an iconic string, then the root partition data used by this job will be retained. + and you can use the root partition data of this job in the futher job by {use_root}. + - use_root : give an iconic string, then the root partition data generated by + the pervious job which has the {save_root} field will be used. + - tips: + - value of {save_root} and {use_root} need to be iconic enough, uuid is recommended. + - value of {save_root} and {use_root} will be the suffix of logical volume. + +2. scheduler return the ipxe_str to testbox + ``` + dhcp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz + imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + boot + ``` + +3. dracut step of boot + ``` + if have {use_root}; then + base_lv_name=openeuler_aarch64_20.03_{value_use_root} + lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 + + if have {save_root}; then + boot_lv_name=openeuler_aarch64_20.03_{value_save_root} + + if {save_root} == {use_root}; then + boot from /dev/mappaer/os-{boot_lv_name} + fi + + lvremove -f /dev/mappaer/os-{boot_lv_name} + create /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} + boot from /dev/mappaer/os-{boot_lv_name} + else + boot_lv_name=openeuler_aarch64_20.03_{value_use_root} + boot from /dev/mappaer/os-{boot_lv_name} + fi + + else: + base_lv_name=openeuler_aarch64_20.03_{timestamp} + lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { + # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} + mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt + mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly + lvchange -p r /dev/mapper/os-{base_lv_name} + } + + boot_lv_name=openeuler_aarch64_20.03 + if have {save_root}; then + boot_lv_name=openeuler_aarch64_20.03_{value_save_root} + fi + + lvremove -f /dev/mappaer/os-{boot_lv_name} + create /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} + boot from /dev/mappaer/os-{boot_lv_name} + fi + ``` + +4. execute the job -- 2.23.0

1 0

[PATCH v6 compass-ci] src/lib/job: add check runtime upper limit
by Luan Shengde 05 Feb '21

05 Feb '21

add limit for borrowing machine: runtime <= 10 days only jobs that runtime value is less than or equal to 10 days are allowed for borrowing machine. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- src/lib/job.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..61f18d6 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -143,6 +143,7 @@ class Job check_required_keys() check_account_info() + check_run_time() set_defaults() end @@ -401,6 +402,20 @@ class Job @hash.delete("my_token") end + private def check_run_time + # only job.yaml for borrowing machine has the key: ssh_pub_key + return unless @hash.has_key?("ssh_pub_key") + + # runtime value is converted to second. + max_run_time = 10 * 24 * 3600 + + error_msg = "The maximum runtime should no more than 10 days." + + # case runtime/sleep value count is more than 10 days, job will fail. + raise error_msg if @hash["runtime"].to_i > max_run_time + raise error_msg if @hash["sleep"].to_i > max_run_time + end + private def is_valid_account?(account_info) return false unless account_info.is_a?(JSON::Any) -- 2.23.0

2 1

[PATCH v2 compass-ci] sbin/compare remove change of failure metric
by Lu Weitao 05 Feb '21

05 Feb '21

[Why] We only display the failure metrics that belong the latest job, aimed to express the latest job exists error. so the change rate of failure metric is useless like: centos openeuler metric -------------------- ------------------------------ ------------------------------ fails:runs change fails:runs | | | 1:12 -8.3% 0:24 stderr./lkp/lkp/src/lib/env.sh:line#:/proc/sys/kernel/nmi_watchdog:Permission_denied [How] centos openeuler metric -------------------- ------------------------------ ------------------------------ fails:runs fails:runs | | 1:12 0:24 stderr./lkp/lkp/src/lib/env.sh:line#:/proc/sys/kernel/nmi_watchdog:Permission_denied centos openeuler metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 2.638681e+10 ± 23% -0.1% 2.636343e+10 ± 22% iperf.tcp.receiver.bps 2.638853e+10 ± 23% +0.1% 2.641691e+10 ± 22% iperf.tcp.sender.bps 6.80 ± 141% -50.4% 3.37 ± 223% iperf.time.elapsed_time Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 58 ++++++++--------------------------------- 1 file changed, 11 insertions(+), 47 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 6c88384..ffc6500 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -83,15 +83,12 @@ def get_values(value_list, success) end def get_compare_value(base_value_average, value_average, success) - # get compare value(change or reproduction) + # get compare value(change) # - if success - return if base_value_average.zero? + return unless success + return if base_value_average.zero? - (100 * value_average / base_value_average - 100).round(1) - else - (100 * (value_average - base_value_average)).round(1) - end + (100 * value_average / base_value_average - 100).round(1) end def field_changed?(base_values, values, success, field, options) @@ -265,7 +262,7 @@ def get_html_header(matrixes_titles, success) compare_title = CHANGE_STR else common_title = FAILS_RUNS_STR - compare_title = REPRODUCTION_STR + compare_title = '' end header = get_html_index(matrixes_titles) @@ -551,35 +548,6 @@ def format_fails_runs(fails, runs) fails_str + ':' + runs_str end -def get_reproduction_index_str(reproduction, compare_index) - reproduction_str = format('%+.1f%%', reproduction) - reproduction_index = compare_index - reproduction_str.index('.') - if reproduction_index.negative? - reproduction_str = format('%+.1e%%', reproduction).sub('e+0', 'e+').sub('e-0', 'e-') - reproduction_index = compare_index - reproduction_str.index('e') - 1 - end - return reproduction_index, reproduction_str -end - -def get_reprodcution(reproduction, compare_index) - if reproduction - reproduction_index, reproduction_str = get_reproduction_index_str(reproduction, compare_index) - space_str = ' ' * (SUB_SHORT_COLUMN_WIDTH - reproduction_str.length) - reproduction_str = space_str.insert(reproduction_index, reproduction_str) - else - reproduction_str = format("%-#{SUB_SHORT_COLUMN_WIDTH}s", '0') - end - reproduction_str -end - -def format_reproduction(reproduction, theme, compare_index) - color = get_compare_value_color(reproduction, theme) - colorize( - color, - get_reprodcution(reproduction, compare_index) - ) -end - def get_change_index_str(change, compare_index) change_str = format('%+.1f%%', change) change_index = compare_index - change_str.index('.') @@ -696,7 +664,7 @@ def get_title_name(success) compare_title = CHANGE_STR else common_title = FAILS_RUNS_STR - compare_title = REPRODUCTION_STR + compare_title = '' end return common_title, compare_title end @@ -721,14 +689,14 @@ def get_base_matrix_title_symbol(common_index, success) title_symbol end -def get_other_matrixes_title_symbol(_compare_title, matrixes_number, common_index, success) +def get_other_matrixes_title_symbol(compare_title, matrixes_number, common_index, success) title_symbol = ' ' * ( (INTERVAL_WIDTH + COLUMN_WIDTH) * matrixes_number ) start_point = 0 common_symbol = success ? '\\' : '|' - compare_symbol = '|' + compare_symbol = compare_title.empty? ? '' : '|' compare_index = SUB_SHORT_COLUMN_WIDTH / 2 (matrixes_number - 1).times do |_| @@ -789,12 +757,8 @@ def get_success_str(values, index, theme, compare_index) (change_str || '') + stddev_str end -def get_failure_str(values, index, theme, compare_index) - unless index.zero? - reproduction_str = format_reproduction( - values[:reproduction], theme, compare_index - ) - end +def get_failure_str(values, index) + reproduction_str = format("%-#{SUB_SHORT_COLUMN_WIDTH}s", '') unless index.zero? fails_runs_str = format_fails_runs( values[:fails], @@ -813,7 +777,7 @@ def get_values_str(matrixes, success, theme) ) + INTERVAL_BLANK else get_failure_str( - values, index, theme, compare_index + values, index ) + INTERVAL_BLANK end end -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/libvirt: consume job by libvirt
by Xiao Shenwei 05 Feb '21

05 Feb '21

function: 1. check host is the capability of executing tasks? - check libvirt - check qemu-kvm 2. request boot.libvirt and load files based on response 3. generate a domain.xml and start a vm used ruby-libvirt Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/run.rb | 68 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/providers/libvirt/run.rb b/providers/libvirt/run.rb index a500c31..2883197 100755 --- a/providers/libvirt/run.rb +++ b/providers/libvirt/run.rb @@ -9,6 +9,8 @@ require_relative "#{ENV['CCI_SRC']}/lib/log" require_relative "#{ENV['CCI_SRC']}/lib/sched_client" require_relative "#{ENV['CCI_SRC']}/providers/lib/context" require_relative "#{ENV['CCI_SRC']}/providers/lib/resource" +require_relative "#{ENV['CCI_SRC']}/providers/lib/domain" +require_relative "#{ENV['CCI_SRC']}/providers/lib/libvirt" def create_logger(hostname) filename = "#{hostname}.log" @@ -48,17 +50,75 @@ def request_job(context, sched_client, logger) return response end +def parse_response(response, context, resource) + context.merge!(response) + resource.parse_response(response) + context.merge!(resource.info) +end + +def generate_domain(context, logger) + domain = Domain.new(context, logger) + xml_path = '' + begin + domain.generate + xml_path = domain.save_to('domain.xml') + rescue StandardError => e + logger.error(e) + logger.error('please check your xml file format') + ensure + upload_log(context) + end + return xml_path +end + +def upload_log(context, upload_qemu_log: false) + id = context.info['job_id'] + host = context.info['LKP_SERVER'] + port = context.info['RESULT_WEBDAV_PORT'] + result_root = context.info['result_root'] + result_url = "http://#{host}:#{port}#{result_root}" + log_name = "#{context.info['hostname']}.log" + system "curl -sSf -T #{log_name} #{result_url}/ --cookie 'JOBID= #{id}'" + return unless upload_qemu_log + + qemu_log_name = "/var/log/libvirt/qemu/#{id}.log" + return unless FileTest.exist?(qemu_log_name) + + system "curl -sSf -T #{qemu_log_name} #{result_url}/ --cookie 'JOBID= #{id}'" +end + +def start(context, logger, libvirt) + xml_path = generate_domain(context, logger) + return if xml_path.empty? + + begin + libvirt.define(xml_path) + libvirt.create + libvirt.wait + rescue StandardError => e + logger.error(e) + ensure + upload_log(context, upload_qemu_log: true) + end +end + +def clean(context, sched_client, libvirt) + sched_client.delete_mac2host(context.info['mac']) + sched_client.delete_host2queues(context.info['hostname']) + libvirt.close +end + def main(hostname, queues) logger = create_logger(hostname) mac = compute_mac hostname context = Context.new(mac, hostname, queues) + libvirt = LibvirtConnect.new resource = Resource.new(hostname, logger) sched_client = SchedClient.new response = request_job(context, sched_client, logger) return if response.nil? - resource.parse_response(response) - context.merge!(resource.info) - sched_client.delete_mac2host(mac) - sched_client.delete_host2queues(hostname) + parse_response(response, context, resource) + start(context, logger, libvirt) + clean(context, sched_client, libvirt) end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 05 Feb '21

05 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..f1e946c 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) + +[ -z $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 05 Feb '21

05 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..e0197b9 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,12 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH v2 compass-ci] create-image: fix error path
by Wang Chenglong 05 Feb '21

05 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/osimage/openeuler/create-image | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/osimage/openeuler/create-image b/container/osimage/openeuler/create-image index 7f3d982..ed0bc14 100755 --- a/container/osimage/openeuler/create-image +++ b/container/osimage/openeuler/create-image @@ -37,7 +37,7 @@ setup_login() { pack_cgz() { echo "Packing package. Please wait." - find / ! -path "/tmp/${IMAGE_NAME}" | grep -vf /tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > /tmp/${IMAGE_NAME} + find ./ ! -path "./tmp/${IMAGE_NAME}" | grep -vf ./tmp/files-to-exclude | cpio -o -Hnewc | gzip -9 > ./tmp/${IMAGE_NAME} chmod 644 /tmp/${IMAGE_NAME} } -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: use -qq for apt-get install
by Xu Xijian 04 Feb '21

04 Feb '21

Just remove >/dev/null 2>&1, use -qq for apt-get install will not output except for errors. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 75148a1..7394575 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -5,9 +5,9 @@ export DEBIAN_FRONTEND=noninteractive apt-get update -apt-get install -y --fix-missing --no-install-recommends apt-utils >/dev/null 2>&1 -apt-get install -y --fix-missing nfs-common netbase cifs-utils kmod rsync -apt-get install -y --fix-missing dracut dracut-network dracut-config-generic +apt-get install -y --no-install-recommends --fix-missing -qq apt-utils +apt-get install -y nfs-common netbase cifs-utils kmod rsync util-linux --fix-missing +apt-get install -y dracut dracut-network dracut-config-generic --fix-missing apt-get clean rm -rf /var/lib/apt/lists/* -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] container/dracut-initrd: add --fix-missing when apt-get install
by Xu Xijian 04 Feb '21

04 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 7e2bf51..75148a1 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -5,9 +5,9 @@ export DEBIAN_FRONTEND=noninteractive apt-get update -apt-get install -y --no-install-recommends apt-utils >/dev/null 2>&1 -apt-get install -y nfs-common netbase cifs-utils kmod rsync -apt-get install -y dracut dracut-network dracut-config-generic +apt-get install -y --fix-missing --no-install-recommends apt-utils >/dev/null 2>&1 +apt-get install -y --fix-missing nfs-common netbase cifs-utils kmod rsync +apt-get install -y --fix-missing dracut dracut-network dracut-config-generic apt-get clean rm -rf /var/lib/apt/lists/* -- 2.23.0

2 2

[PATCH compass-ci] git_mirror.rb: fix that git clone failure isn't known
by Li Yuanchao 04 Feb '21

04 Feb '21

Some repo git clone failed just puts "Cloning into '/srv/git/t/twitch/twitch.git'..." and nothing more. So can not take "stderr.include?('fatal')" as the condition to judge fail or not. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 8898a44..940d5e2 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -39,14 +39,12 @@ class GitMirror end def git_clone(url, mirror_dir) - ret = false url = get_url(Array(url)[0]) 10.times do - stderr = %x(git clone --mirror --depth 1 #{url} #{mirror_dir} 2>&1) - ret = !stderr.include?('fatal') - break if ret + %x(git clone --mirror --depth 1 #{url} #{mirror_dir} 2>&1) + return true if File.directory?(mirror_dir) && File.exist?("#{mirror_dir}/config") end - return ret + return false end def git_fetch(mirror_dir) -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: puts the info about the define_file
by Li Ping 04 Feb '21

04 Feb '21

[why] test the function "submit -a" always need to check define_files in job.sh puts the info about the define_files when submit job.yaml is better. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index 9174cadc7..f19d1e6dc 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -680,6 +680,7 @@ class Job Dir.glob("#{LKP_SRC}/{*,*/*}/{#{program},#{program}-dev,#{program}.*}{,/*}").each do |file| file = File.realpath(file) if git_files_list.include?(file) + puts "define_file: #{file}" relative_path = file.sub("#{LKP_SRC}/", '') @job["define_files"][relative_path] = File.read(file) end -- 2.23.0

1 0

[PATCH v2 compass-ci] container/redis: delete rebundant code
by Liu Yinsi 04 Feb '21

04 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/redis/start | 5 ----- 1 file changed, 5 deletions(-) diff --git a/container/redis/start b/container/redis/start index 34266a0..5e2dc44 100755 --- a/container/redis/start +++ b/container/redis/start @@ -22,8 +22,3 @@ cmd=( ) "${cmd[@]}" - -# test -echo you can use cmd: redis-cli to start a redis client -echo and then use cmd: keys * - -- 2.23.0

1 0

[PATCH v2 compass-ci] git_mirror.rb: optimize the use of hash.merge
by Li Yuanchao 04 Feb '21

04 Feb '21

1. hash_c = hash_a.merge(hash_b) will return a new hash which contains key-values of both hash_a and hash_b, and if the two hashes have some same keys, the key-values of hash_b will be kept in hash_c. So that if a repo file has the same key with DEFAULTS file, its own will be kept. 2. fix that if there's no DEFAULTS file, the merge_defaults method will return nil. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index fa2e0cc..8898a44 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -453,10 +453,12 @@ class MirrorMain def merge_defaults(git_repo) defaults_key = File.dirname(git_repo) while defaults_key != '.' - return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + return @defaults[defaults_key].merge(@git_info[git_repo]) if @defaults[defaults_key] defaults_key = File.dirname(defaults_key) end - return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + return @defaults['default'].merge(@git_info[git_repo]) if @defaults['default'] + + return @git_info[git_repo] end end -- 2.23.0

2 1

[PATCH compass-ci] redis: fix PoolTimeoutError
by Wu Zhende 04 Feb '21

04 Feb '21

[Error] Exception: No free connection (used 64 of 64) after timeout of 0.01s (Redis::PoolTimeoutError) [How] I looked at the number of concurrent Redis visits on our cluster, and it peaked at over 900. The scheduler and TaskQueue are about 50/50. So increase the size of the Redis connection pool. Adapt to our concurrency Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 1 + src/scheduler/redis_client.cr | 9 +++++++-- src/taskqueue/constants.cr | 2 +- 3 files changed, 9 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index b90759f..dfec365 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -32,6 +32,7 @@ class Sched def initialize(env : HTTP::Server::Context) @es = Elasticsearch::Client.new + Redis::Client.set_pool_size(1000) @redis = Redis::Client.instance @task_queue = TaskQueueAPI.new @rgc = RemoteGitClient.new diff --git a/src/scheduler/redis_client.cr b/src/scheduler/redis_client.cr index a8d9dcb..030d1b7 100644 --- a/src/scheduler/redis_client.cr +++ b/src/scheduler/redis_client.cr @@ -12,13 +12,18 @@ class Redis::Client class_property :client HOST = (ENV.has_key?("REDIS_HOST") ? ENV["REDIS_HOST"] : JOB_REDIS_HOST) PORT = (ENV.has_key?("REDIS_PORT") ? ENV["REDIS_PORT"] : JOB_REDIS_PORT).to_i32 + @@size = 25 def self.instance Singleton::Of(self).instance end - def initialize(host = HOST, port = PORT) - @client = Redis::PooledClient.new(host: host, port: port, pool_size: 25, pool_timeout: 0.01) + def initialize(host = HOST, port = PORT, pool_size = @@size) + @client = Redis::PooledClient.new(host: host, port: port, pool_size: pool_size, pool_timeout: 0.01) + end + + def self.set_pool_size(pool_size) + @@size = pool_size end def keys(pattern) diff --git a/src/taskqueue/constants.cr b/src/taskqueue/constants.cr index c702382..6240c54 100644 --- a/src/taskqueue/constants.cr +++ b/src/taskqueue/constants.cr @@ -11,7 +11,7 @@ REDIS_PORT = 6379 # delimiter and exttract-ststs will loop consume job # when use 32 (scheduler use 25), meet Exception: # Exception: No free connection (used 32 of 32) -REDIS_POOL_NUM = 64 +REDIS_POOL_NUM = 1000 REDIS_POOL_TIMEOUT = 10 # ms -- 2.23.0

1 0

[PATCH lkp-tests] tests/monitor-job: fix lkp-tests gem depend
by Wei Jihui 04 Feb '21

04 Feb '21

faye-websocket is needed, but elasticserch is not needed. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/monitor-job | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/monitor-job b/tests/monitor-job index 0f495ae2..5da992fa 100755 --- a/tests/monitor-job +++ b/tests/monitor-job @@ -19,7 +19,7 @@ set_scheduler_config() install_gem_pkgs() { gem sources -r http://rubygems.org -a https://gems.ruby-china.com/ - gem install activesupport git rest-client elasticsearch + gem install git activesupport rest-client faye-websocket } build_my_info() -- 2.23.0

1 0

[PATCH v2 compass-ci] container/register-accounts: fix no my_token
by Luan Shengde 04 Feb '21

04 Feb '21

the lib/build_my_info_client will not generate my_token automatically. and lib/build_my_info_client do not support update if account exists in the ES. use lib/build-my-info, it will generate my_token if it's not in my_info. fix rubocop warning: register-accounts:11:18: W: Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used. content.each do |key, value| ^^^ Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index 519f289..e4d39ff 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,16 +5,15 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" +require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" content = YAML.load_file('/etc/compass-ci/register/register.yaml') -content.each do |key, value| +content.each do |_key, value| my_info = {} my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] - build_my_info = BuildMyInfo.new(my_info['my_email']) - build_my_info.store_account_info(my_info) + build_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH compass-ci] submit/Dockerfile: better way to replace gem source
by Cui Yili 04 Feb '21

04 Feb '21

Signed-off-by: Cui Yili <2268260388(a)qq.com> --- container/submit/Dockerfile | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile index 3ca13f4..6a021d4 100644 --- a/container/submit/Dockerfile +++ b/container/submit/Dockerfile @@ -16,8 +16,7 @@ RUN apt-get update && \ # it may take a long time, even break up if you use default source address for the gem. # delete the following step if you do not want to use the default one. -RUN gem sources --add https://gems.ruby-china.com/ && \ - gem sources --remove https://rubygems.org/ +RUN gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ RUN gem install git faye-websocket activesupport rest-client -- 2.23.0

1 0

[PATCH compass-ci] providers/libvirt: consume job by libvirt
by Xiao Shenwei 04 Feb '21

04 Feb '21

function: 1. check host is the capability of executing tasks? - check libvirt - check qemu-kvm 2. request boot.libvirt and load files based on response 3. generate a domain.xml and start a vm used ruby-libvirt Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/run.rb | 68 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/providers/libvirt/run.rb b/providers/libvirt/run.rb index a500c31..2883197 100755 --- a/providers/libvirt/run.rb +++ b/providers/libvirt/run.rb @@ -9,6 +9,8 @@ require_relative "#{ENV['CCI_SRC']}/lib/log" require_relative "#{ENV['CCI_SRC']}/lib/sched_client" require_relative "#{ENV['CCI_SRC']}/providers/lib/context" require_relative "#{ENV['CCI_SRC']}/providers/lib/resource" +require_relative "#{ENV['CCI_SRC']}/providers/lib/domain" +require_relative "#{ENV['CCI_SRC']}/providers/lib/libvirt" def create_logger(hostname) filename = "#{hostname}.log" @@ -48,17 +50,75 @@ def request_job(context, sched_client, logger) return response end +def parse_response(response, context, resource) + context.merge!(response) + resource.parse_response(response) + context.merge!(resource.info) +end + +def generate_domain(context, logger) + domain = Domain.new(context, logger) + xml_path = '' + begin + domain.generate + xml_path = domain.save_to('domain.xml') + rescue StandardError => e + logger.error(e) + logger.error('please check your xml file format') + ensure + upload_log(context) + end + return xml_path +end + +def upload_log(context, upload_qemu_log: false) + id = context.info['job_id'] + host = context.info['LKP_SERVER'] + port = context.info['RESULT_WEBDAV_PORT'] + result_root = context.info['result_root'] + result_url = "http://#{host}:#{port}#{result_root}" + log_name = "#{context.info['hostname']}.log" + system "curl -sSf -T #{log_name} #{result_url}/ --cookie 'JOBID= #{id}'" + return unless upload_qemu_log + + qemu_log_name = "/var/log/libvirt/qemu/#{id}.log" + return unless FileTest.exist?(qemu_log_name) + + system "curl -sSf -T #{qemu_log_name} #{result_url}/ --cookie 'JOBID= #{id}'" +end + +def start(context, logger, libvirt) + xml_path = generate_domain(context, logger) + return if xml_path.empty? + + begin + libvirt.define(xml_path) + libvirt.create + libvirt.wait + rescue StandardError => e + logger.error(e) + ensure + upload_log(context, upload_qemu_log: true) + end +end + +def clean(context, sched_client, libvirt) + sched_client.delete_mac2host(context.info['mac']) + sched_client.delete_host2queues(context.info['hostname']) + libvirt.close +end + def main(hostname, queues) logger = create_logger(hostname) mac = compute_mac hostname context = Context.new(mac, hostname, queues) + libvirt = LibvirtConnect.new resource = Resource.new(hostname, logger) sched_client = SchedClient.new response = request_job(context, sched_client, logger) return if response.nil? - resource.parse_response(response) - context.merge!(resource.info) - sched_client.delete_mac2host(mac) - sched_client.delete_host2queues(hostname) + parse_response(response, context, resource) + start(context, logger, libvirt) + clean(context, sched_client, libvirt) end -- 2.23.0

1 0

[PATCH lkp-tests] distro/archlinux: redirect pacman output
by Wang Yong 04 Feb '21

04 Feb '21

[before] $ pacman -Qq --noconfirm wget wget $ echo $? 0 [after] $ pacman -Qq --noconfirm wget > /dev/null $ echo $? 0 Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/archlinux b/distro/archlinux index 43c3aa13a..59c8da1d6 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -88,7 +88,7 @@ distro_install_depends() for pkg in $packages do - pacman -Qq --noconfirm $pkg || pacman -Sy --noconfirm $pkg + pacman -Qq --noconfirm $pkg > /dev/null || pacman -Sy --noconfirm $pkg done done } -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/lib: introduce libvirt.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

gem install ruby-libvirt ruby-libvirt providers Ruby binding for libvirt. you can called it based on a domain.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/libvirt.rb | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 providers/lib/libvirt.rb diff --git a/providers/lib/libvirt.rb b/providers/lib/libvirt.rb new file mode 100644 index 0000000..e4e1240 --- /dev/null +++ b/providers/lib/libvirt.rb @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'libvirt' + +# connect libvirt daemon +class LibvirtConnect + def initialize + @conn = Libvirt.open('qemu:///system') + end + + def define(xml) + @dom = @conn.define_domain_xml(File.read(xml)) + end + + def create + @dom.create + end + + def wait + loop do + sleep 10 + break unless @dom.active? + end + end + + def close + @conn.close + end +end -- 2.23.0

1 0

[PATCH compass-ci] providers/lib: introduce libvirt.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

gem install ruby-libvirt ruby-libvirt providers Ruby binding for libvirt. you can called it based on a domain.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/libvirt.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 providers/lib/libvirt.rb diff --git a/providers/lib/libvirt.rb b/providers/lib/libvirt.rb new file mode 100644 index 0000000..6f39651 --- /dev/null +++ b/providers/lib/libvirt.rb @@ -0,0 +1,33 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'libvirt' + +# connect libvirt daemon +class LibvirtConnect + def initialize + @conn = Libvirt.open('qemu:///system') + end + + def define(xml) + @dom = @conn.define_domain_xml(File.read(xml)) + end + + def create + @dom.create + end + + def wait + loop do + sleep 10 + unless @dom.active? + break + end + end + end + + def close + @conn.close + end +end -- 2.23.0

1 0

[PATCH lkp-tests] distro/*: fix output information
by Wang Yong 04 Feb '21

04 Feb '21

[before] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench.cgz [after] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench-cpu.cgz Thanks, Yong --- distro/aliyun | 2 +- distro/archlinux | 2 +- distro/clear | 2 +- distro/debian | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/distro/aliyun b/distro/aliyun index 598243d8e..83727ce32 100755 --- a/distro/aliyun +++ b/distro/aliyun @@ -94,7 +94,7 @@ yum_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.rpm > $pack_to/.${BM_NAME}.packages diff --git a/distro/archlinux b/distro/archlinux index d7b3d8d78..43c3aa13a 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -64,7 +64,7 @@ pacman_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.pkg.tar* > $pack_to/.${BM_NAME}.packages diff --git a/distro/clear b/distro/clear index f21de89c3..5d6fb0e46 100755 --- a/distro/clear +++ b/distro/clear @@ -135,7 +135,7 @@ pack_bundles_files() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } # 1. install all un-installed bundles and record it to uninstalled_bundles diff --git a/distro/debian b/distro/debian index 313f73e35..21dafe21b 100755 --- a/distro/debian +++ b/distro/debian @@ -120,7 +120,7 @@ pack() chown .lkp $pack_to/${BM_NAME}_$date.cgz \ $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } resolve_depends() -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 04 Feb '21

04 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..d4d70c3 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) + +[ ! -n $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 04 Feb '21

04 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..e0197b9 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,12 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin/auto_submit: don't use auto_submit.yaml any more
by Li Yuanchao 04 Feb '21

04 Feb '21

Now DEFAULTS files are added to upstream-repos to keep submit commands. And these info would be taken by messages which the container git-mirror send. So auto_submit parses the messages and get the submit commands. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 14 +++++--------- sbin/auto_submit.yaml | 25 ------------------------- 2 files changed, 5 insertions(+), 34 deletions(-) delete mode 100644 sbin/auto_submit.yaml diff --git a/sbin/auto_submit b/sbin/auto_submit index c59deb7..410445d 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -17,7 +17,6 @@ class AutoSubmit connection.start channel = connection.create_channel @queue = channel.queue('new_refs') - @repo2job = YAML.load_file("#{$PROGRAM_NAME}.yaml") end def get_pkgbuild_repo(repo_array) @@ -57,20 +56,17 @@ class AutoSubmit return unless pkgbuild_repo submit_argv.push("pkgbuild_repo=#{pkgbuild_repo}") - repo2job_key = @repo2job[git_repo] ? git_repo : 'archlinux' - else - return unless @repo2job[git_repo] - - repo2job_key = git_repo end - [submit_argv, repo2job_key] + submit_argv end def submit_job(newrefs_info) - submit_argv, repo2job_key = get_argvs(newrefs_info) + submit_argv = get_argvs(newrefs_info) return unless submit_argv - @repo2job[repo2job_key].each do |argv_config| + return unless newrefs_info['submit'] + + newrefs_info['submit'].each do |argv_config| argvs = Array.new(submit_argv) argvs.push(argv_config) submit(newrefs_info, argvs) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml deleted file mode 100644 index c368ad6..0000000 --- a/sbin/auto_submit.yaml +++ /dev/null @@ -1,25 +0,0 @@ -a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml -c/compass-ci/compass-ci: -- testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -l/linux/linux: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/linux-next: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/openeuler-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/debian-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/archlinux-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=dc-8g os=openeuler os_version=20.03-pre os_mount=container os_arch=aarch64 docker_image=openeuler:20.03-pre queue=dc-build-pkg build-pkg.yaml -- testbox=dc-8g os=archlinux os_version=2021-01-06-pre os_mount=container os_arch=aarch64 docker_image=archlinux:2021-01-06-pre queue=dc-build-pkg build-pkg.yaml -- 2.23.0

1 0

[PATCH compass-ci 1/2] git_mirror.rb: optimize the use of hash.merge
by Li Yuanchao 04 Feb '21

04 Feb '21

1. hash_a.merge(hash_b), this means if the two hashes have some same keys, the last one will cover the first one. I change it so that if a repo file has the same key with DEFAULTS file, its own will be kept. 2. fix that if there's no DEFAULTS file, the merge_defaults method will return nil. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index fa2e0cc..8898a44 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -453,10 +453,12 @@ class MirrorMain def merge_defaults(git_repo) defaults_key = File.dirname(git_repo) while defaults_key != '.' - return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + return @defaults[defaults_key].merge(@git_info[git_repo]) if @defaults[defaults_key] defaults_key = File.dirname(defaults_key) end - return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + return @defaults['default'].merge(@git_info[git_repo]) if @defaults['default'] + + return @git_info[git_repo] end end -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix no my_token
by Luan Shengde 04 Feb '21

04 Feb '21

the lib/build_my_info_client will not generate my_token automatically. and lib/build_my_info_client do not support update if account exists. in the ES. use lib/build-my-info, it will gererate my_token if it's not in my_info. fix rubocop warning: register-accounts:11:18: W: Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used. content.each do |key, value| ^^^ Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index 519f289..e4d39ff 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,16 +5,15 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" +require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" content = YAML.load_file('/etc/compass-ci/register/register.yaml') -content.each do |key, value| +content.each do |_key, value| my_info = {} my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] - build_my_info = BuildMyInfo.new(my_info['my_email']) - build_my_info.store_account_info(my_info) + build_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..749f3667b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,12 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + case line.chomp! + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0

[PATCH v6 compass-ci 3/3] container/submit: add source file for debian packages
by Luan Shengde 04 Feb '21

04 Feb '21

it may take a long time, even break up if use the default source address to install bebian backages. use the source address inland will resolv the problem. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/conf/sources.list | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/submit/conf/sources.list diff --git a/container/submit/conf/sources.list b/container/submit/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/submit/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH v6 compass-ci 2/3] container/submit: executable file submit
by Luan Shengde 04 Feb '21

04 Feb '21

executable file for container: submit file 'submit' is used as the command installed in the container. it can be run as the real command submit itself in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..6236892 --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M%S) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /srv/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0

[PATCH v6 compass-ci 1/3] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..3ca13f4 --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +COPY conf/sources.list /etc/apt/ + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /srv + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

1 0

[PATCH compass-ci] container/redis: use echo "xxx" for safety reason
by Liu Yinsi 04 Feb '21

04 Feb '21

before: echo you can use cmd: redis-cli to start a redis client echo and then use cmd: keys * after: echo "you can use cmd: redis-cli to start a redis client" echo "and then use cmd: keys *" [why] when use echo $string, if $string include '*', will be parsed, just like: root@test-lys /c/compass-ci/container/redis# ./start ... and then use cmd: keys build Dockerfile start Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/redis/start | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/redis/start b/container/redis/start index 34266a0..4611fe4 100755 --- a/container/redis/start +++ b/container/redis/start @@ -24,6 +24,6 @@ cmd=( "${cmd[@]}" # test -echo you can use cmd: redis-cli to start a redis client -echo and then use cmd: keys * +echo "you can use cmd: redis-cli to start a redis client" +echo "and then use cmd: keys *" -- 2.23.0

2 2

[PATCH compass-ci] sparrow/5-build: download os rootfs
by Liu Yinsi 04 Feb '21

04 Feb '21

default download os rootfs openeuler aarch64 20.03 for boot testbox. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/os-ready | 5 +++++ sparrow/install-tiny | 1 + 2 files changed, 6 insertions(+) create mode 100755 sparrow/5-build/os-ready diff --git a/sparrow/5-build/os-ready b/sparrow/5-build/os-ready new file mode 100755 index 0000000..48a740b --- /dev/null +++ b/sparrow/5-build/os-ready @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/download-rootfs diff --git a/sparrow/install-tiny b/sparrow/install-tiny index a291549..5f026e5 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -19,4 +19,5 @@ cd $CCI_SRC/sparrow || exit . /etc/profile.d/compass.sh 4-docker/buildall 5-build/ipxe +5-build/os-ready 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH compass-ci] sparrow/5-build: register my info
by Liu Yinsi 04 Feb '21

04 Feb '21

register user own account info to ES. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/register-account | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 sparrow/5-build/register-account diff --git a/sparrow/5-build/register-account b/sparrow/5-build/register-account new file mode 100755 index 0000000..1a23e07 --- /dev/null +++ b/sparrow/5-build/register-account @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/build-my-info.rb -- 2.23.0

1 0

[PATCH lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..95e2a272b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,13 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + line.chomp! + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0

[PATCH v3 lkp-tests] stats: add stats script for dockerfile os migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] ^[[91mError: Unable to find a match: supervisor ^[[91m/bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/stats/openeuler_docker b/stats/openeuler_docker index e31934573..95e2a272b 100755 --- a/stats/openeuler_docker +++ b/stats/openeuler_docker @@ -2,7 +2,12 @@ while (line = STDIN.gets) line.chomp! - next unless line =~ /build.fail/ - - puts line + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end end -- 2.23.0

1 0

[PATCH compass-ci] delimiter/utils.rb: delete the nil elements in array
by Cao Xueliang 04 Feb '21

04 Feb '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 3120ca6..0930dd7 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -29,7 +29,7 @@ module Utils commits << temp_commit if temp_commit end commits << get_last_commit(work_dir, commit) if commits.empty? - return commits.to_a + return commits.to_a.compact end def get_day_ago_commit(work_dir, commit, day_ago) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: add document
by Luan Shengde 04 Feb '21

04 Feb '21

add chinese document for apply account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.zh.md | 56 ++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 doc/manual/apply-account.zh.md diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md new file mode 100644 index 0000000..52d43ba --- /dev/null +++ b/doc/manual/apply-account.zh.md @@ -0,0 +1,56 @@ +# 账号申请 + +首先您需要一个账号来提交job，同时用于跟踪您的资源消费情况。 +因为目前我们的测试机资源池有限，因此需要做一些使用限制。 + +我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 +你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 + +我们同时也为我们的合作者提供了免费的账号。 +这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 + +## 1. 发送邮件进行账号申请 + +邮件模板： + +--- + From: {{ David Rientjes <rientjes(a)google.com> }} + To: compass-ci-robot(a)qq.com + Subject: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ attach your ssh pubkey file(s), e.g. ~/.ssh/id_rsa.pub }} + +--- + +- 使用您的信息替换{{ }}中的内容 + +- 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” + +- my oss commit: 使用您的邮件和名字提交的代码的地址。 + 您的仓库需要在平台测试覆盖的范围内，并且注册到： + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + 如果贡献的OSS工程仓库不在upstream-repos中，请参考： + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + 将仓库注册到compass-ci测试池中。 + +## 2. 接收邮件 + +申请邮件成功后，您将收到反馈邮件，包括： + + my_name + my_email + my_token + lab + +## 3. 一次性配置 + +**根据步骤2收到邮件的提示，配置您的本地环境： + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + setup ~/.config/compass-ci/defaults/account.yaml + +## 4. 现在开始 + +[submit a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

2 2

[PATCH compass-ci] doc/manual: add document apply-account.zh.md
by Luan Shengde 04 Feb '21

04 Feb '21

add chinese document Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.zh.md | 58 ++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/manual/apply-account.zh.md diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md new file mode 100644 index 0000000..5e6605d --- /dev/null +++ b/doc/manual/apply-account.zh.md @@ -0,0 +1,58 @@ +# 账号申请 + +首先您需要一个账号，可以用来提交job，同时跟踪您的资源消费情况。 +因为目前我们的测试机资源池有限，因此需要做一些使用限制。 + +我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 +你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 + +我们同时也为我们的合作者提供了免费的账号。 +这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 + +## 1. 发送邮件进行账号申请 + +邮件模板： + +--- + 发件人: {{ David Rientjes <rientjes(a)google.com> }} + 收件人: compass-ci-robot(a)qq.com + 邮件标题: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ 将您的ssh公钥添加为邮件附件，例如 ~/.ssh/id_rsa.pub }} + +--- + +- 使用您的信息替换{{ }}中的内容 +- 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” +- my oss commit: 使用您的邮件和名字提交的代码的地址。 + + + 您的仓库需要在平台测试覆盖的范围内，并且注册到： + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + 如果贡献的OSS工程仓库不在upstream-repos中，请参考： + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + 将仓库注册到compass-ci测试池中。 + +## 2. 接收邮件 + +申请邮件成功后，您将收到反馈邮件，邮件内容包括以下信息： + + my_name + my_email + my_token + lab + +## 3. 一次性配置 + +根据步骤2收到邮件的提示，配置您的本地环境： + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/{{ lab }}.yaml + +## 4. 现在开始 + +[提交job到compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit-job.en.md) + -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/manual/apply-account: optimize document
by Luan Shengde 04 Feb '21

04 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index fda2eba..974fe63 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -43,6 +43,7 @@ Which contains account information for you: my_name my_email my_token + lab ## step 3: one-time setup @@ -50,6 +51,7 @@ Which contains account information for you: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/{{ lab }}.yaml ## step 4: try it out -- 2.23.0

1 0

[PATCH lkp-tests] distro/debian: use function instead of command
by Wang Yong 04 Feb '21

04 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/debian | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/debian b/distro/debian index 9b6636f48..313f73e35 100755 --- a/distro/debian +++ b/distro/debian @@ -171,7 +171,7 @@ distro_install_depends() add_i386_package update - apt-get install -yf + fixup_preinstall for script in $scripts do script=$(basename $script) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: optimize document
by Luan Shengde 04 Feb '21

04 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index fda2eba..ac7ce5e 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -43,6 +43,7 @@ Which contains account information for you: my_name my_email my_token + lab ## step 3: one-time setup @@ -50,6 +51,7 @@ Which contains account information for you: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/${lab}.yaml ## step 4: try it out -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] container/dracut-initrd: add rsync module to initrd
by Xu Xijian 04 Feb '21

04 Feb '21

Change filename add-cifs.conf to add-modules.conf, then add rsync module to dracut by put "rsync" into add-modules.conf. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/conf/{add-cifs.conf => add-modules.conf} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename container/dracut-initrd/conf/{add-cifs.conf => add-modules.conf} (100%) diff --git a/container/dracut-initrd/conf/add-cifs.conf b/container/dracut-initrd/conf/add-modules.conf similarity index 100% rename from container/dracut-initrd/conf/add-cifs.conf rename to container/dracut-initrd/conf/add-modules.conf -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/write-PKGBUILD: fix documentation errors
by Lin Jiaxin 04 Feb '21

04 Feb '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- doc/manual/write-PKGBUILD.en.md | 5 +++-- doc/manual/write-PKGBUILD.zh.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/manual/write-PKGBUILD.en.md b/doc/manual/write-PKGBUILD.en.md index c0c25e9..c259399 100644 --- a/doc/manual/write-PKGBUILD.en.md +++ b/doc/manual/write-PKGBUILD.en.md @@ -13,8 +13,9 @@ PKGBUILD includes variables and functions. - pkgrel: Mandatory. It indicates the release number of a software package. - arch: Mandatory. It indicates the architecture sequence of a software package. - depends: Optional. It indicates the name of the dependency package required for running a software test. -- makedepends: Optional. It indicates the list of files required for building a software package. -- source: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. +- makedepends: Optional. It indicates the name of the dependency package required for building a software package. +- source: Optional. It indicates the list of files required for building a software package. +- md5sums: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. ## Defining a Function diff --git a/doc/manual/write-PKGBUILD.zh.md b/doc/manual/write-PKGBUILD.zh.md index 414bfdc..ff53f0f 100644 --- a/doc/manual/write-PKGBUILD.zh.md +++ b/doc/manual/write-PKGBUILD.zh.md @@ -13,8 +13,9 @@ PKGBUILD�� - pkgrel: �� - arch: �� - depends: �� -- makedepends: �� -- source: ��MD5�� +- makedepends: �� +- source: �� +- md5sums: ��MD5�� ## �� -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: add --fix-missing when apt-get install
by Xu Xijian 04 Feb '21

04 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 7e2bf51..75148a1 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -5,9 +5,9 @@ export DEBIAN_FRONTEND=noninteractive apt-get update -apt-get install -y --no-install-recommends apt-utils >/dev/null 2>&1 -apt-get install -y nfs-common netbase cifs-utils kmod rsync -apt-get install -y dracut dracut-network dracut-config-generic +apt-get install -y --fix-missing --no-install-recommends apt-utils >/dev/null 2>&1 +apt-get install -y --fix-missing nfs-common netbase cifs-utils kmod rsync +apt-get install -y --fix-missing dracut dracut-network dracut-config-generic apt-get clean rm -rf /var/lib/apt/lists/* -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/lib: introduce domain.rb action.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

function: based on response of boot.libvirt generate a domain.xml refer-to: commit 457644c0486e0786ed8b563c6e4928445eea0207 if you want to test with your domain.xml, define vt field in job.yaml vt: domain: xxx/xxx.xml if not will use default options to generate a domain.xml TODO: case 1: Impact of single factors on tasks: vt: cpu: xxx/cpu.xml case 2: Impact of multiple factors on tasks: vt: cpu: xxx/cpu.xml disk: xxx/disk.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/action.rb | 75 +++++++++++++++++++++++++++++++++++++++++ providers/lib/domain.rb | 75 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) create mode 100644 providers/lib/action.rb create mode 100644 providers/lib/domain.rb diff --git a/providers/lib/action.rb b/providers/lib/action.rb new file mode 100644 index 0000000..526d84d --- /dev/null +++ b/providers/lib/action.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# expand each element of the domain.xml +module Action + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + def expand_erb(file) + @context.expand_erb(File.read(file)) + end + + def domain + @doc = Nokogiri::XML(expand_erb(@domain_option)) + end + + def name + @doc.xpath('//domain/name').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/name.xml") + end + + def os + @doc.xpath('//domain/os').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/os.xml") + end + + def emulator + @doc.xpath('//domain/devices/emulator').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/emulator.xml") + end + + def cpu + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/cpu.xml") + end + + def memory + @doc.xpath('//domain/memory').remove + @doc.xpath('//domain/currentMemory').remove + @doc.xpath('//domain/maxMemory').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/memory.xml") + end + + def devices + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/devices.xml") + end + + def disk + @context.info['disk'] ||= nil + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/disk.xml") + end + + def serial + @doc.xpath('//domain/devices/serial').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/serial.xml") + end + + def interface + @doc.xpath('//domain/devices/interface').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/interface.xml") + end + + def active + @doc.xpath('//domain/on_poweroff').remove + @doc.xpath('//domain/on_reboot').remove + @doc.xpath('//domain/on_crash').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/active.xml") + end + + def clock + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/clock.xml") + end + + def seclabel + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/seclabel.xml") + end +end diff --git a/providers/lib/domain.rb b/providers/lib/domain.rb new file mode 100644 index 0000000..8df9c06 --- /dev/null +++ b/providers/lib/domain.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'set' +require 'nokogiri' +require 'yaml' +require_relative 'action' +require_relative "#{ENV['LKP_SRC']}/lib/hashugar" + +# generate a domain.xml +class Domain + include Action + + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + OPTION_FILE = "#{TEMPLATE_DIR}/options.yaml" + def initialize(context, logger) + @doc = nil + @context = context + @logger = logger + @options = Hashugar.new(YAML.safe_load(File.read(OPTION_FILE))) + end + + def generate + domain_option + if user_domain? + default_option + return + end + replaceable_option + default_option + end + + def save_to(filename) + File.open(filename, 'w') do |f| + f.puts @doc.to_xml + end + File.realpath(filename) + end + + private + + def user_domain? + @context.info['vt'].key?('domain') + end + + def load_xml_file(filepath) + host = @context.info['SRV_HTTP_HOST'] + port = @context.info['SRV_HTTP_PORT'] + system "wget --timestamping --progress=bar:force http://#{host}:#{port}/cci/libvirt-xml/#{filepath}" + %x(basename #{filepath}).chomp + end + + def domain_option + @domain_option = "#{TEMPLATE_DIR}/#{(a)options.domain}.xml" + if user_domain? + @domain_option = load_xml_file((a)context.info['vt']['domain']) + end + @logger.info("Domain option: #{@domain_option}") + domain + end + + def default_option + @options.default.each do |one| + instance_eval one + end + end + + def replaceable_option + @options.replaceable.each do |one| + instance_eval one + end + end +end -- 2.23.0

1 0

[PATCH v5 compass-ci 1/4] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..5ea0fcc --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +RUN echo 'deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n \ +\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n' \ +> /etc/apt/sources.list + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /root + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

2 2

[PATCH compass-ci] doc/write-PKGBUILD: fix documentation errors
by Lin Jiaxin 03 Feb '21

03 Feb '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- doc/manual/write-PKGBUILD.en.md | 5 +++-- doc/manual/write-PKGBUILD.zh.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/manual/write-PKGBUILD.en.md b/doc/manual/write-PKGBUILD.en.md index c0c25e9..c259399 100644 --- a/doc/manual/write-PKGBUILD.en.md +++ b/doc/manual/write-PKGBUILD.en.md @@ -13,8 +13,9 @@ PKGBUILD includes variables and functions. - pkgrel: Mandatory. It indicates the release number of a software package. - arch: Mandatory. It indicates the architecture sequence of a software package. - depends: Optional. It indicates the name of the dependency package required for running a software test. -- makedepends: Optional. It indicates the list of files required for building a software package. -- source: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. +- makedepends: Optional. It indicates the name of the dependency package required for building a software package. +- source: Optional. It indicates the list of files required for building a software package. +- md5sums: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. ## Defining a Function diff --git a/doc/manual/write-PKGBUILD.zh.md b/doc/manual/write-PKGBUILD.zh.md index 414bfdc..ff53f0f 100644 --- a/doc/manual/write-PKGBUILD.zh.md +++ b/doc/manual/write-PKGBUILD.zh.md @@ -13,8 +13,9 @@ PKGBUILD�� - pkgrel: �� - arch: �� - depends: �� -- makedepends: �� -- source: ��MD5�� +- makedepends: �� +- source: �� +- md5sums: ��MD5�� ## �� -- 2.23.0

1 0

[PATCH lkp-tests] distro/archlinux: fix pacman search range
by Wang Yong 03 Feb '21

03 Feb '21

[before] $ pacman -Qqs gzip gzip zlib [after] $ pacman -Qq gzip gzip we need list local package accurately -s, --search <regexp> Search each locally-installed package for names or descriptions that match regexp. When including multiple search terms, only packages with descriptions matching ALL of those terms are returned. ref: https://archlinux.org/pacman/pacman.8.html Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/archlinux b/distro/archlinux index 3942080a6..d7b3d8d78 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -88,7 +88,7 @@ distro_install_depends() for pkg in $packages do - pacman -Qqs --noconfirm $pkg || pacman -Sy --noconfirm $pkg + pacman -Qq --noconfirm $pkg || pacman -Sy --noconfirm $pkg done done } -- 2.23.0

1 0

[PATCH v5 compass-ci 3/3] container/defconfig: load config yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

Load "/etc/compass-ci/register/*.yaml", get register account info from cci defaults config. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index fd13312..f9b0824 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -9,6 +9,7 @@ def cci_defaults hash = {} Dir.glob(['/etc/compass-ci/defaults/*.yaml', '/etc/compass-ci/accounts/*.yaml', + '/etc/compass-ci/register/*.yaml', "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| hash.update YAML.load_file(file) || {} end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] container/auto-submit: bind mount volume
by Liu Yinsi 03 Feb '21

03 Feb '21

mount /etc/compass-ci/register, to get auto-submit account info(my_name, my_email) from config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/auto-submit/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/auto-submit/start b/container/auto-submit/start index 4632ab0..79463f6 100755 --- a/container/auto-submit/start +++ b/container/auto-submit/start @@ -21,6 +21,7 @@ cmd = %W[ -v #{ENV['LKP_SRC']}:#{DOCKER_LKP} -v /etc/localtime:/etc/localtime:ro -v /etc/compass-ci/defaults:/etc/compass-ci/defaults:ro + -v /etc/compass-ci/register:/etc/compass-ci/register:ro -v /srv/git:/srv/git -w /c/compass-ci/sbin alpine:auto-submit -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] sbin/auto-submit: config account yaml
by Liu Yinsi 03 Feb '21

03 Feb '21

[how] before: define my_name, my_email, lab, then store into ES and config yaml after: find my_name, my_email from /etc/compass-ci/register/*.yaml, then config yaml [why] store account info into ES, this part already done by container service register-accounts, so when start auto-submit service, no need to store data, only config yaml. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/auto_submit | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 5648d59..c59deb7 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -6,7 +6,9 @@ require 'bunny' require 'json' require 'yaml' +require 'set' require_relative '../lib/build_my_info_client' +require "#{ENV['CCI_SRC']}/container/defconfig" # receive message and auto submit job class AutoSubmit @@ -86,15 +88,23 @@ class AutoSubmit end end -option = { - my_name: 'Auto Submit Robot', - my_email: 'autosubmit@localhost', - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp -} +def my_email + names = Set.new %w[auto-submit] + + defaults = relevant_defaults(names) + defaults['auto-submit']['my_email'] +end + +def config_yaml + build_my_info = BuildMyInfo.new(my_email) + + my_info = build_my_info.search_my_info + build_my_info.config_default_yaml(my_info) + build_my_info.config_lab_yaml(my_info) +end # fix verification problems when submit jobs by auto-submit service -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +config_yaml auto_submitter = AutoSubmit.new auto_submitter.listen -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] sparrow/1-storage: optimise format
by Liu Yinsi 03 Feb '21

03 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 47 ++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 0e8d722..0713d7e 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -1,26 +1,37 @@ #! /bin/bash -chown lkp:lkp /srv/result -chown lkp:lkp /srv/initrd -chown -R lkp:lkp /srv/es -chown -R mailer:team /srv/cci/Maildir -chown lkp:lkp /srv/cache/netdata_cache -chown lkp:lkp /srv/cache/netdata_lib -chown lkp:lkp /srv/cci/serial/fluentd-pos -chown lkp:lkp /srv/cci/libvirt-xml +chmod_dir=( + /srv/result + /srv/es/logging-es + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/logs + /srv/cci/libvirt-xml + /srv/cci/serial/fluentd-pos + /srv/cci/Maildir +) + +chmod 775 "${chmod_dir[@]}" + +chown_dir=( + /srv/result + /srv/initrd + /srv/es + /srv/cache/netdata_cache + /srv/cache/netdata_lib + /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml +) + +chown lkp:lkp "${chown_dir[@]}" + chown lkp:committer /srv/git -chgrp team /srv/cci/serial/logs -chmod 775 /srv/result -chmod 775 /srv/es/logging-es -chmod 775 /srv/es -chmod 775 /srv/cache/netdata_cache -chmod 775 /srv/cache/netdata_lib -chmod 775 /srv/cci/serial/logs -chmod 775 /srv/cci/libvirt-xml -chmod 775 /srv/cci/serial/fluentd-pos -chmod -R 775 /srv/cci/Maildir +chown mailer:team /srv/cci/Maildir +chgrp team /srv/cci/serial/logs chgrp team /srv/dc + chmod g+ws /srv/dc find /tftpboot -type d | xargs chmod 775 -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: optimize load config
by Wei Jihui 03 Feb '21

03 Feb '21

[why] current load config order can not load physical machine [how] optimize load config for: 1. load /etc/compass-ci/defaults/*.yaml load ~/.config/compass-ci/defaults/*.yaml --> get lab 2. load ~/.config/compass-ci/include/lab/$lab.yaml -->get my_token 3. now user get testbox, load load hosts config 4. load other file: like lkp-tests/include/* Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 28 +++++++++------------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 477a8a12..9174cadc 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -367,26 +367,15 @@ class Job return @job.include?('tbox_group') end - def get_lab_hosts_file - lab_hosts_file_name = @job['testbox'] - lab_hosts_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{lab_hosts_file_name}" - - if File.file?(lab_hosts_file) - lab_hosts_file - else - nil - end - end - def get_hosts_file - lab_hosts_file = get_lab_hosts_file - return lab_hosts_file if lab_hosts_file + testbox_file = "#{ENV['CCI_REPOS']}/lab-#{@job['lab']}/hosts/#{@job['testbox']}" + return testbox_file if File.file?(testbox_file) - hosts_file_name = @job['tbox_group'].split(/\.|--/)[0] - hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" - return hosts_file if File.file?(hosts_file) + tbox_group = @job['tbox_group'].split(/\.|--/)[0] + tbox_group_file = "#{LKP_SRC}/hosts/#{tbox_group}" + return tbox_group_file if File.file?(tbox_group_file) - raise ArgumentError, "hosts file not exist: #{hosts_file_name}, maybe need check testbox field" + raise ArgumentError, "hosts file not exist: #{tbox_group}, maybe need check testbox field" end def include_files @@ -438,6 +427,9 @@ class Job lab_yaml = File.join(self_config_path, 'include/lab', "#{@defaults['lab']}.yaml") load_one_defaults(lab_yaml, @job) + + merge_defaults + load_hosts_config end def load_defaults(first_time = true) @@ -749,7 +741,6 @@ class Job each_job_init job = deepcopy(@job) load_defaults - load_hosts_config each_job_init each_job(&block) @jobs.each do |hash| @@ -757,7 +748,6 @@ class Job @job = deepcopy(job) @job.merge!(hash) load_defaults - load_hosts_config each_job_init each_job(&block) end -- 2.23.0

1 0

[PATCH v3 compass-ci] delimiter/utils.rb: fix get the wrong job id when submit job failed
by Cao Xueliang 03 Feb '21

03 Feb '21

If we submit job failed, we get the response "xxx job id=0, error: xxx". We should match the response at first and return nil, otherwise we will get the wrong job id. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41b65de..3120ca6 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -69,6 +69,7 @@ module Utils save_job_to_yaml(job, PROCESS_JOB_YAML) response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" + return nil if response =~ /job id=0/ return $1 if response =~ /job id=(.*)/ return nil -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

stores xml files required by libvirt Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

2 2

[PATCH v2 lkp-tests] installer/oracle: add shebang
by Wang Yong 03 Feb '21

03 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/oracle | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/installer/oracle b/distro/installer/oracle index b09abc945..11df716d6 100755 --- a/distro/installer/oracle +++ b/distro/installer/oracle @@ -1,3 +1,5 @@ +#!/bin/sh + if ! rpm -q epel-release >/dev/null; then yum install -y epel-release yum makecache fast -- 2.23.0

1 0

[PATCH lkp-tests] installer/oracle: add shebang
by Wang Yong 03 Feb '21

03 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/oracle | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/installer/oracle b/distro/installer/oracle index b09abc945..0831a5b6c 100755 --- a/distro/installer/oracle +++ b/distro/installer/oracle @@ -1,3 +1,5 @@ +#!/bin/bash + if ! rpm -q epel-release >/dev/null; then yum install -y epel-release yum makecache fast -- 2.23.0

2 2

[PATCH compass-ci] dnsmasq: open the dns function
by Yu Chuan 03 Feb '21

03 Feb '21

[Why] Generally, dnsmasq is used for a internal network lab, and as the number of machines in the lab increases, we need an internal dns server to perform dns relay and forward function. [How] 1. open dns function: - set 'port=53'; 2. set the upstream dns server of dnsmasq: - add param: 'resolv-conf'; - add resolv.conf and mount it to container. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/conf/etc/resolv.conf | 1 + container/dnsmasq/dnsmasq.d/dnsmasq.conf | 7 ++++++- container/dnsmasq/start | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 container/dnsmasq/conf/etc/resolv.conf diff --git a/container/dnsmasq/conf/etc/resolv.conf b/container/dnsmasq/conf/etc/resolv.conf new file mode 100644 index 000000000000..bad8aa25de25 --- /dev/null +++ b/container/dnsmasq/conf/etc/resolv.conf @@ -0,0 +1 @@ +nameserver 114.114.114.114 diff --git a/container/dnsmasq/dnsmasq.d/dnsmasq.conf b/container/dnsmasq/dnsmasq.d/dnsmasq.conf index 6141faf0436b..877216d8abdb 100644 --- a/container/dnsmasq/dnsmasq.d/dnsmasq.conf +++ b/container/dnsmasq/dnsmasq.d/dnsmasq.conf @@ -1,4 +1,9 @@ -port=0 +# open dns function +port=53 + +# setup the upstream dns server of dnsmasq +resolv-file=/etc/resolv.conf + interface=br0 enable-tftp diff --git a/container/dnsmasq/start b/container/dnsmasq/start index ff4247046eb1..a75ff209e64f 100755 --- a/container/dnsmasq/start +++ b/container/dnsmasq/start @@ -27,6 +27,7 @@ cmd=( --publish 67:67/udp --publish 69:69/udp -v ${WORKSPACE}/dnsmasq.d:/etc/dnsmasq.d + -v ${WORKSPACE}/conf/etc/resolv.conf:/etc/resolv.conf:ro # specify the upstream dns server of dnsmasq -v ${DNSMASQ_CONF_DIR}/lease_dir:/var/lib/misc # persist the dhcp lease file -v /etc/localtime:/etc/localtime:ro -v /tftpboot:/tftpboot:ro -- 2.23.0

1 0

[PATCH v2 compass-ci] delimiter/utils.rb: deal submit job failed
by Cao Xueliang 03 Feb '21

03 Feb '21

If we submit job failed, we get the response "xxx job id=0, error: xxx", we should match the response at first and return nil. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 41b65de..3120ca6 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -69,6 +69,7 @@ module Utils save_job_to_yaml(job, PROCESS_JOB_YAML) response = %x(#{LKP_SRC}/sbin/submit #{PROCESS_JOB_YAML}) puts "submit job response: #{response}" + return nil if response =~ /job id=0/ return $1 if response =~ /job id=(.*)/ return nil -- 2.23.0

1 0

[PATCH v3 compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty before git clone. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..507c4fe 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ -d "openresty" ] && rm -rf openresty + git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

1 0

[PATCH v5 compass-ci 4/4] container/submit: README file
by Luan Shengde 03 Feb '21

03 Feb '21

help file for the container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 container/submit/README.md diff --git a/container/submit/README.md b/container/submit/README.md new file mode 100644 index 0000000..6772961 --- /dev/null +++ b/container/submit/README.md @@ -0,0 +1,26 @@ +# Install lkp-tests and its depends in container. +# With docker container, will suit most of the linux OSes. +# Avoid much repetitive testing work to adapt various OSes. + +Prepare work: + apply account + config default yaml + generate ssh keys + +Installation: + git clone https://gitee.com/wu_fengguang/compass-ci.git + + cd compass-ci/container/submit + ./build + + Add soft link for executable file 'submit' to $PATH: + + ln -s $PWD/submit /usr/bin/submit + +Usage: + You can directly use the command 'submit' to submit jobs. + It's the same as you install the lkp-tests at your local host. + + Example: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v5 compass-ci 3/4] container/submit: executable file submit
by Luan Shengde 03 Feb '21

03 Feb '21

executable file for container: submit file 'submit' is used as the command installed in the container. it can be run as the real command submit itself in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..4e7e24f --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M%S) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /root/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0