- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..5d30486 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

1 0

[PATCH compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..a881d67 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

2 1

[PATCH lkp-tests] lib/bootstrap.sh: add step: setup dns
by Yu Chuan 07 Feb '21

07 Feb '21

[Why] Since we have added a dns server, so we need to add support for dns function in lkp-tests framework. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 1127de140a2b..06aa09bf65b2 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -283,6 +283,13 @@ setup_hosts() ln -fs $tmpfs_hosts /etc/hosts } +setup_dns() +{ + [ -n "$DNS_HOST" ] || return 0 + + echo "nameserver ${DNS_HOST}" > /etc/resolv.conf +} + export_ip_mac() { if has_cmd ip; then @@ -739,6 +746,7 @@ boot_init() setup_hostname setup_hosts + setup_dns add_lkp_user -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: delete blank line
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 464153d8f..b5b070f01 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -27,7 +27,6 @@ linux_dep() esac } - mac_dep() { brew install ruby -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: add submit depends
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 012c45499..464153d8f 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -12,13 +12,13 @@ linux_dep() case $ID in ubuntu|debian) export DEBIAN_FRONTEND=noninteractive - sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ + sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ uuid-runtime ;; openEuler|fedora|rhel|centos) if type dnf > /dev/null 2>&1; then - sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux else - sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux fi ;; *) -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result.rb: use upstream url and commit replace git url
by Cao Xueliang 07 Feb '21

07 Feb '21

We should show the upstream url in the bisect report, but the git url is our git server url, so, replace git url to the upstream url. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 8 ++++++++ lib/mail_bisect_result.rb | 3 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 5d13074..40f6b49 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -23,6 +23,7 @@ class GitBisect set_build_pkg_dir set_upstream set_pkgbuild_repo + set_upstream_url set_work_dir set_bad_commit set_good_commit @@ -56,6 +57,12 @@ class GitBisect @upstream_repo_git = "git://#{GIT_MIRROR_HOST}/#{@upstream_repo}" end + def set_upstream_url + @upstream_url = @bad_job['upstream_url'] + puts "upstream_url: #{@upstream_url}" + raise 'upstream url is null' unless @upstream_url + end + def set_work_dir @work_dir = Utils.clone_repo(@upstream_repo_git, @upstream_commit) puts "work_dir: #{@work_dir}" @@ -106,6 +113,7 @@ class GitBisect 'work_dir' => @work_dir, 'bisect_error' => bisect_error, 'all_errors' => all_errors, + 'upstream_url' => @upstream_url 'pkgbuild_repo' => @pkgbuild_repo, 'first_bad_commit_result_root' => first_bad_commit_result_root] end diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 637c008..5e1103c 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -16,6 +16,7 @@ class MailBisectResult @commit_id = bisect_info['commit'] @all_errors = bisect_info['all_errors'] @bisect_error = bisect_info['bisect_error'] + @upstream_url = bisect_info['upstream_url'] @pkgbuild_repo = bisect_info['pkgbuild_repo'] @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] @git_commit = GitCommit.new(@work_dir, @commit_id) @@ -46,7 +47,7 @@ class MailBisectResult Hi #{(a)git_commit.author_name}, We found some error/warning(s) and the first bad commit in the below project: - git url: #{(a)git_commit.url} + git url: #{@upstream_url/@commit_id} git commit: #{@commit_id} ("#{(a)git_commit.subject}") All error/warning(s) (new ones prefixed by >>): -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends/cci-makepkg: add git depend
by Wei Jihui 07 Feb '21

07 Feb '21

user maybe use git cmd when wirte PKGBUILD Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/cci-makepkg | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-makepkg b/distro/depends/cci-makepkg index 4ed4f19e..855ac8ce 100644 --- a/distro/depends/cci-makepkg +++ b/distro/depends/cci-makepkg @@ -1,3 +1,4 @@ +git curl gnupg gettext -- 2.23.0

2 1

[PATCH compass-ci 1/2] scheduler: add public service info in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the service info use the submit script in lkp-tests, but the lkp-tests don't know the service info for a remote user, so, we add the public service info in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/public_service.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/public_service.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..9829e81 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/public_service.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(public_service) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/public_service.cr b/src/scheduler/public_service.cr new file mode 100644 index 0000000..e3b888b --- /dev/null +++ b/src/scheduler/public_service.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def public_service + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/public_service.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

2 2

[PATCH lkp-tests] change the submit option from '-l' to '-r'
by Li Ping 05 Feb '21

05 Feb '21

[why] -l means lftp lftp is just one of the clients, "download results" should use more clear and descriptive name. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/monitor.rb | 6 +++--- sbin/submit | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 9091c92a9..45f720f20 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -96,7 +96,7 @@ class Monitor end def set_result_root(data) - return unless @action['lftp_result'] + return unless @action['mirror_result'] return unless data['log'] data = JSON.parse(data['log']) @@ -113,7 +113,7 @@ class Monitor system "lftp -c mirror #{url} >/dev/null 2>&1" end - def lftp(data) + def result(data) if @result_root lftp_mirror else @@ -163,7 +163,7 @@ class Monitor output(data) connect(data, ws) - lftp(data) + result(data) stop(data, ws) if @action['stop'] end diff --git a/sbin/submit b/sbin/submit index e8b176489..a6f0cd387 100755 --- a/sbin/submit +++ b/sbin/submit @@ -52,8 +52,8 @@ options = OptionParser.new do |opts| actions << 'connect' end - opts.on('-l', '--lftp', 'lftp mirror job result') do - actions << 'lftp_result' + opts.on('-r', '--result', 'mirror job result dir') do + actions << 'mirror_result' end opts.on('-m', '--monitor', "monitor job status: use -m 'KEY: VALUE' to add rule") do -- 2.23.0

1 0

[PATCH lab-z9] the file of testboxes allocation scheme
by Zhang Dewan 05 Feb '21

05 Feb '21

how about this ? it's not easy to show the true purpose of all the machines. sometimes machine borrow out, user maybe install OS on the disk, when it's back, we should set pxe boot, and reboot it. Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- allot/testboxes-allocation-scheme | 246 ++++++++++++++++++++++++++++++ 1 file changed, 246 insertions(+) create mode 100644 allot/testboxes-allocation-scheme diff --git a/allot/testboxes-allocation-scheme b/allot/testboxes-allocation-scheme new file mode 100644 index 0000000..db2940b --- /dev/null +++ b/allot/testboxes-allocation-scheme @@ -0,0 +1,246 @@ +# x86_64: 2288hv5-2s44p-384g +2288hv5-2s44p-384g--b1 + Board-test +2288hv5-2s44p-384g--b2 + pub-tbox + Board-test +2288hv5-2s44p-384g--b3 + VNC-server + Board-test +2288hv5-2s44p-384g--b4 + Openshift-analyse + Board-test +2288hv5-2s44p-384g--b5 + pub-tbox + Board-test +2288hv5-2s44p-384g--b6 + pub-tbox + Board-test +2288hv5-2s44p-384g--b7 + Board-test +# aarch64: taishan200-2280-2s48p-256g +taishan200-2280-2s48p-256g--a11 + multi-docker +taishan200-2280-2s48p-256g--a12 + multi-docker +taishan200-2280-2s48p-256g--a13 + multi-docker +taishan200-2280-2s48p-256g--a14 + multi-docker +taishan200-2280-2s48p-256g--a15 + multi-docker +taishan200-2280-2s48p-256g--a16 + multi-docker +taishan200-2280-2s48p-256g--a17 + multi-docker +taishan200-2280-2s48p-256g--a18 + multi-docker +taishan200-2280-2s48p-256g--a19 + multi-docker +taishan200-2280-2s48p-256g--a20 + multi-docker +taishan200-2280-2s48p-256g--a21 + pub-tbox +taishan200-2280-2s48p-256g--a22 + pub-tbox +taishan200-2280-2s48p-256g--a24 + long-runtime +taishan200-2280-2s48p-256g--a25 + long-runtime +taishan200-2280-2s48p-256g--a26 + long-runtime +taishan200-2280-2s48p-256g--a27 + long-runtime +taishan200-2280-2s48p-256g--a28 + long-runtime +taishan200-2280-2s48p-256g--a29 + long-runtime +taishan200-2280-2s48p-256g--a30 + long-runtime +taishan200-2280-2s48p-256g--a31 + long-runtime +taishan200-2280-2s48p-256g--a32 + long-runtime +taishan200-2280-2s48p-256g--a33 + long-runtime +taishan200-2280-2s48p-256g--a34 + long-runtime +taishan200-2280-2s48p-256g--a35 + long-runtime +taishan200-2280-2s48p-256g--a49 + pub-tbox +taishan200-2280-2s48p-256g--a53 + pub-tbox +taishan200-2280-2s48p-256g--a60 + multi-qemu +taishan200-2280-2s48p-256g--a61 + multi-qemu +taishan200-2280-2s48p-256g--a62 + multi-qemu +taishan200-2280-2s48p-256g--a63 + pub-tbox +taishan200-2280-2s48p-256g--a64 + pub-tbox +taishan200-2280-2s48p-256g--a65 + pub-tbox +taishan200-2280-2s48p-256g--a66 + pub-tbox +taishan200-2280-2s48p-256g--a67 + pub-tbox +taishan200-2280-2s48p-256g--a68 + pub-tbox +taishan200-2280-2s48p-256g--a69 + pub-tbox +taishan200-2280-2s48p-256g--a70 + pub-tbox +taishan200-2280-2s48p-256g--a71 + pub-tbox +taishan200-2280-2s48p-256g--a84 + long-runtime +taishan200-2280-2s48p-256g--a85 + long-runtime +taishan200-2280-2s48p-256g--a86 + long-runtime +taishan200-2280-2s48p-256g--a87 + long-runtime +taishan200-2280-2s48p-256g--a88 + long-runtime +taishan200-2280-2s48p-256g--a89 + long-runtime +taishan200-2280-2s48p-256g--a90 + long-runtime +taishan200-2280-2s48p-256g--a91 + long-runtime +taishan200-2280-2s48p-256g--a92 + long-runtime +taishan200-2280-2s48p-256g--a93 + long-runtime +taishan200-2280-2s48p-256g--a94 + long-runtime +taishan200-2280-2s48p-256g--a96 + long-runtime +taishan200-2280-2s48p-256g--a97 + long-runtime +taishan200-2280-2s48p-256g--a98 + long-runtime + Board-test +taishan200-2280-2s48p-256g--a99 + long-runtime +taishan200-2280-2s48p-256g--a100 + long-runtime +taishan200-2280-2s48p-256g--a101 + long-runtime +taishan200-2280-2s48p-256g--a103 + long-runtime +taishan200-2280-2s48p-256g--a104 + long-runtime +taishan200-2280-2s48p-256g--a106 + long-runtime +taishan200-2280-2s48p-256g--a107 + long-runtime +# aarch64: taishan200-2280-2s48p-512g +taishan200-2280-2s48p-512g--a102 + long-runtime +taishan200-2280-2s48p-512g--a105 + long-runtime +taishan200-2280-2s48p-512g--a95 + long-runtime +# aarch64: taishan200-2280-2s64p-256g +taishan200-2280-2s64p-256g--a3 + Board-test +taishan200-2280-2s64p-256g--a4 + pub-tbox +taishan200-2280-2s64p-256g--a5 + pub-tbox +taishan200-2280-2s64p-256g--a6 + pub-tbox +taishan200-2280-2s64p-256g--a7 + pub-tbox +taishan200-2280-2s64p-256g--a8 + pub-tbox +taishan200-2280-2s64p-256g--a9 + pub-tbox +taishan200-2280-2s64p-256g--a10 + pub-tbox +taishan200-2280-2s64p-256g--a36 + pub-tbox +taishan200-2280-2s64p-256g--a37 + pub-tbox +taishan200-2280-2s64p-256g--a38 + pub-tbox +taishan200-2280-2s64p-256g--a39 + pub-tbox +taishan200-2280-2s64p-256g--a40 + pub-tbox +taishan200-2280-2s64p-256g--a41 + pub-tbox +taishan200-2280-2s64p-256g--a42 + pub-tbox +taishan200-2280-2s64p-256g--a43 + pub-tbox +taishan200-2280-2s64p-256g--a44 + pub-tbox +taishan200-2280-2s64p-256g--a45 + pub-tbox +taishan200-2280-2s64p-256g--a46 + pub-tbox +taishan200-2280-2s64p-256g--a78 + pub-tbox +taishan200-2280-2s64p-256g--a79 + pub-tbox +taishan200-2280-2s64p-256g--a80 + pub-tbox +taishan200-2280-2s64p-256g--a81 + pub-tbox +taishan200-2280-2s64p-256g--a82 + pub-tbox +taishan200-2280-2s64p-256g--a83 + pub-tbox +taishan200-2280-2s64p-256g--a108 + long-runtime +taishan200-2280-2s64p-256g--a109 + long-runtime +taishan200-2280-2s64p-256g--a110 + long-runtime +taishan200-2280-2s64p-256g--a111 + long-runtime +taishan200-2280-2s64p-256g--a112 + long-runtime +taishan200-2280-2s64p-256g--a113 + long-runtime +taishan200-2280-2s64p-256g--a114 + long-runtime +taishan200-2280-2s64p-256g--a115 + long-runtime +taishan200-2280-2s64p-256g--a116 + long-runtime +taishan200-2280-2s64p-256g--a117 + long-runtime +taishan200-2280-2s64p-256g--a118 + long-runtime +taishan200-2280-2s64p-256g--a119 + long-runtime +taishan200-2280-2s64p-256g--a120 + long-runtime +taishan200-2280-2s64p-256g--a121 + long-runtime +taishan200-2280-2s64p-256g--a122 + long-runtime +taishan200-2280-2s64p-256g--a123 + long-runtime +taishan200-2280-2s64p-256g--a124 + long-runtime +taishan200-2280-2s64p-256g--a125 + long-runtime +taishan200-2280-2s64p-256g--a126 + long-runtime +taishan200-2280-2s64p-256g--a127 + long-runtime +taishan200-2280-2s64p-256g--a128 + long-runtime +taishan200-2280-2s64p-256g--a129 + long-runtime +taishan200-2280-2s64p-256g--a130 + long-runtime +taishan200-2280-2s64p-256g--a131 + long-runtime -- 2.23.0

2 2

[PATCH compass-ci 2/2] sparrow/dev-env: add the public_service.yaml
by Cao Xueliang 05 Feb '21

05 Feb '21

The scheduler will read public_service.yaml, then add the public service info to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..a7eae4d 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -54,6 +54,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/public_service.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: delete read "/etc/compass-ci/defaults/*.yaml"
by Cao Xueliang 05 Feb '21

05 Feb '21

The compass-ci/scheduler will read the yamls, so, delete it in lkp-tests Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/job.rb | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6dc..015fb840f 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -420,8 +420,7 @@ class Job def load_self_config self_config_path = "#{ENV['HOME']}/.config/compass-ci" - Dir.glob(['/etc/compass-ci/defaults/*.yaml', - "#{self_config_path}/defaults/*.yaml"]).each do |file| + Dir.glob(["#{self_config_path}/defaults/*.yaml"]).each do |file| load_one_defaults(file, @job) end -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: change the document name from Chinese to English
by Li Ping 05 Feb '21

05 Feb '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- .../manual/install-cci-client.md | 0 .../manual/log-in-machine-debug.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" => doc/manual/install-cci-client.md (100%) rename "doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" => doc/manual/log-in-machine-debug.md (100%) diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" b/doc/manual/install-cci-client.md similarity index 100% rename from "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" rename to doc/manual/install-cci-client.md diff --git "a/doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" b/doc/manual/log-in-machine-debug.md similarity index 100% rename from "doc/manual/\345\246\202\344\275\225\347\231\273\345\275\225\346\265\213\350\257\225\346\234\272\350\260\203\346\265\213\344\273\273\345\212\241.md" rename to doc/manual/log-in-machine-debug.md -- 2.23.0

1 0

[PATCH v2 compass-ci] container/srv-http: download h5ai repo
by Liu Yinsi 05 Feb '21

05 Feb '21

before: copy h5ai repo from our server /c/h5ai into image after: download h5ai repo when build container [why] download h5ai in docker image, make build script more clean. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/srv-http/Dockerfile | 8 ++++---- container/srv-http/build | 7 ------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/container/srv-http/Dockerfile b/container/srv-http/Dockerfile index 987e40e..6564730 100644 --- a/container/srv-http/Dockerfile +++ b/container/srv-http/Dockerfile @@ -5,16 +5,16 @@ FROM alpine:3.9 ADD root / -RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm \ - && mkdir /run/nginx/ \ - && mkdir /h5ai +RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm git \ + && mkdir /run/nginx/ RUN sed -i '/\[global\]/a daemonize = no' /etc/php7/php-fpm.conf RUN sed -i "s/user = nobody/user = nginx/g" /etc/php7/php-fpm.d/www.conf RUN sed -i "s/group = nobody/group = nginx/g" /etc/php7/php-fpm.d/www.conf +RUN git clone https://gitee.com/luweitao_y/h5ai.git /h5ai + WORKDIR /h5ai -COPY h5ai /h5ai # build h5ai package RUN npm install \ diff --git a/container/srv-http/build b/container/srv-http/build index 054e826..3a500ea 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,11 +2,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${H5AI_SRC:=/c/h5ai} - -cp -r $H5AI_SRC ./ - docker build -t srv-http . - -# clean h5ai folder -rm -rf ./h5ai -- 2.23.0

1 0

[PATCH compass-ci] container/srv-http: download h5ai repo
by Liu Yinsi 05 Feb '21

05 Feb '21

before: copy h5ai repo from our server /c/h5ai into image after: download h5ai repo when build container [why] download h5ai in docker image, make build script more clear. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/srv-http/Dockerfile | 8 ++++---- container/srv-http/build | 7 ------- 2 files changed, 4 insertions(+), 11 deletions(-) diff --git a/container/srv-http/Dockerfile b/container/srv-http/Dockerfile index 987e40e..1f9b5ca 100644 --- a/container/srv-http/Dockerfile +++ b/container/srv-http/Dockerfile @@ -5,16 +5,16 @@ FROM alpine:3.9 ADD root / -RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm \ - && mkdir /run/nginx/ \ - && mkdir /h5ai +RUN apk add --no-cache nginx php7 php7-fpm php7-session php7-json php7-exif php7-imagick php7-gd php7-fileinfo nodejs npm git\ + && mkdir /run/nginx/ RUN sed -i '/\[global\]/a daemonize = no' /etc/php7/php-fpm.conf RUN sed -i "s/user = nobody/user = nginx/g" /etc/php7/php-fpm.d/www.conf RUN sed -i "s/group = nobody/group = nginx/g" /etc/php7/php-fpm.d/www.conf +RUN git clone https://gitee.com/luweitao_y/h5ai.git /h5ai + WORKDIR /h5ai -COPY h5ai /h5ai # build h5ai package RUN npm install \ diff --git a/container/srv-http/build b/container/srv-http/build index 054e826..3a500ea 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,11 +2,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${H5AI_SRC:=/c/h5ai} - -cp -r $H5AI_SRC ./ - docker build -t srv-http . - -# clean h5ai folder -rm -rf ./h5ai -- 2.23.0

2 2

[PATCH v3 compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 05 Feb '21

05 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..8812839 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml + +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) +[ -z $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 05 Feb '21

05 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..4503080 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,11 @@ +# please fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix code error
by Liu Yinsi 05 Feb '21

05 Feb '21

[error] lys@crystal ~/compass-ci/container/register-accounts% ./start register-accounts register-accounts bash-5.1# ./register-accounts awk: /etc/compass-ci/defaults/*.yaml: No such file or directory Traceback (most recent call last): 10: from ./register-accounts:8:in `<main>' 9: from ./register-accounts:8:in `require_relative' 8: from /c/compass-ci/sbin/build-my-info.rb:55:in `<top (required)>' 7: from /c/compass-ci/sbin/build-my-info.rb:48:in `build_my_info' 6: from /c/compass-ci/lib/build_my_info_client.rb:30:in `search_my_info' 5: from /c/compass-ci/lib/es_query.rb:58:in `query_by_id' 4: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-api-7.10.1/lib/elasticsearch/api/actions/search.rb:103:in `search' 3: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/client.rb:182:in `perform_request' 2: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request' 1: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:347:in `perform_request' /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error': [400] {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"LOhdoqAnRYG09044aFiHUw","index":"accounts"},{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"d1F49OYCTxmCIDe3HZDKAw","index":"accounts1"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"accounts","node":"keFEKD-WTBe0tHF4fbS4MA","reason":{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"LOhdoqAnRYG09044aFiHUw","index":"accounts","caused_by":{"type":"illegal_argument_exception","reason":"Ids can't be empty"}}},{"shard":0,"index":"accounts1","node":"keFEKD-WTBe0tHF4fbS4MA","reason":{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"d1F49OYCTxmCIDe3HZDKAw","index":"accounts1","caused_by":{"type":"illegal_argument_exception","reason":"Ids can't be empty"}}}]},"status":400} (Elasticsearch::Transport::Transport::Errors::BadRequest) [why] sbin/build-my-info.rb: ... option = { 'my_email' => `git config --global user.email`.chomp, 'my_name' => `git config --global user.name`.chomp, 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } ... build_my_info(option) when call sbin/build-my-info.rb with params my_info the defaults option already exists in build-my-info.rb, then will cause conflict, so call lib/build_my_info_client.rb Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/register-accounts/register-accounts | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index e4d39ff..b55e293 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,7 +5,7 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" +require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" content = YAML.load_file('/etc/compass-ci/register/register.yaml') content.each do |_key, value| @@ -13,7 +13,9 @@ content.each do |_key, value| my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] + my_info['my_token'] = %x(uuidgen).chomp - build_my_info(my_info) + build_my_info = BuildMyInfo.new(my_info['my_email']) + build_my_info.config_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH lkp-tests 2/2] installer/kali: add kali installer symlink
by Wang Yong 05 Feb '21

05 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/installer/kali | 1 + 1 file changed, 1 insertion(+) create mode 120000 distro/installer/kali diff --git a/distro/installer/kali b/distro/installer/kali new file mode 120000 index 000000000..b2f7fd3e9 --- /dev/null +++ b/distro/installer/kali @@ -0,0 +1 @@ +debian \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests 1/2] distro/kali: add kali distro file
by Wang Yong 05 Feb '21

05 Feb '21

add kali pack method Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/kali | 1 + 1 file changed, 1 insertion(+) create mode 120000 distro/kali diff --git a/distro/kali b/distro/kali new file mode 120000 index 000000000..b2f7fd3e9 --- /dev/null +++ b/distro/kali @@ -0,0 +1 @@ +debian \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests] stats/ansible_stats_openeuler: stats result for ansible migrating
by Wang Chenglong 05 Feb '21

05 Feb '21

[How] ansible_stats_openeuler ${job_output} ./ansible_stats_openeuler /srv/result/ansible_test/2021-02-04/vm-2p16g--wcl/openeuler-20.03-aarch64/linux-system-roles-storage/crystal.974233/output Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_stats_openeuler | 48 +++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100755 stats/ansible_stats_openeuler diff --git a/stats/ansible_stats_openeuler b/stats/ansible_stats_openeuler new file mode 100755 index 000000000..5a6870eef --- /dev/null +++ b/stats/ansible_stats_openeuler @@ -0,0 +1,48 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' + +job_error_msg = `grep -E 'FAILED! | failed: .*}$' #{ARGV[0]} | awk -F'=>' '{$1="";print $0}'` +error_msg_json = JSON.parse(job_error_msg) + +unless error_msg_json['msg'].nil? + puts "config.error.no-file-find-by.#{$1}: 1" if error_msg_json['msg'].match(/No file was found when using (.+)\./) + + puts "yaml.error.syntax-problem.[#{$1}]: 1" if error_msg_json['msg'].match(/The error was: (.+)/) + + if error_msg_json['msg'].match(/No such file or directory/) + puts "command.error.no-such-command.#{error_msg_json['cmd']}: 1" + end + + if error_msg_json['msg'].match(/Failed to download metadata for repo '(.+)': /) + puts "download.error.failed-to-download.#{$1}: 1" + end + + puts "script.error [#{error_msg_json['cmd']}]: 1" if error_msg_json['msg'].match(/non-zero return code/) + + puts "download.error.http-error-404 [#{$1}]: 1" if error_msg_json['msg'].match(/Failure downloading (.+), /) + + if error_msg_json['msg'].match(/Could not find the requested service firewall: (.+)/) + puts "firewall.error.could-not-find-server.#{$1}: 1" + end + + puts "service.error.unable-to-star-service.#{$1}: 1" if error_msg_json['msg'].match(/Unable to start service (.+):/) + + puts 'system.error.not-supporte: 1' if error_msg_json['msg'].match(/is not supporte/) + + puts "depsolve.error.depend-package-error.#{$1}: 1" if error_msg_json['msg'].match(/needed by (.+)/) + + puts 'read.error.read-timed-out: 1' if error_msg_json['msg'].match(/failed to create temporary content file/) +end + +error_msg_json['failures']&.each do |items| + items.match(/No package (.+) available/) + puts "yum.error.no-package-match.#{$1}: 1" +end + +unless error_msg_json['message'].nil? + puts "yaml.error.not-find-yml.#{$1}: 1" if a['message'].to_s.match(/Could not find or access (.+)/) +end -- 2.23.0

1 0

[PATCH compass-ci] providers/lib: fix wget in multi-process mode
by Xiao Shenwei 05 Feb '21

05 Feb '21

[why] Redirecting output to ‘wget-log’. Redirecting output to ‘wget-log.1’. Redirecting output to ‘wget-log.2’. Redirecting output to ‘wget-log.3’. [how] append info to log file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/resource.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/lib/resource.rb b/providers/lib/resource.rb index bcce612..4279766 100644 --- a/providers/lib/resource.rb +++ b/providers/lib/resource.rb @@ -56,7 +56,7 @@ class Resource end def load_file(url) - system "wget --timestamping --progress=bar:force #{url}" + system "wget --timestamping -a #{(a)hostname}.log --progress=bar:force #{url}" basename = File.basename(url) file_size = %x(ls -s --block-size=M "#{basename}").chomp @logger.info("Load file size: #{file_size}") -- 2.23.0

1 0

[PATCH lab-z9] add physical machine information
by Bai Jing 05 Feb '21

05 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- hosts/taishan200-2280-2s64p-256g--a123 | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 hosts/taishan200-2280-2s64p-256g--a123 diff --git a/hosts/taishan200-2280-2s64p-256g--a123 b/hosts/taishan200-2280-2s64p-256g--a123 new file mode 100644 index 0000000..f765092 --- /dev/null +++ b/hosts/taishan200-2280-2s64p-256g--a123 @@ -0,0 +1,19 @@ +memory: 128G +nr_hdd_partitions: 2 +hdd_partitions: + - /dev/disk/by-id/scsi-3500003998881eb19 + - /dev/disk/by-id/scsi-3500003998881e9a1 +mac_addr: + - 44:a1:91:9d:f5:4a + - 44:a1:91:9d:f5:4b + - 44:a1:91:9d:f5:4c + - 44:a1:91:9d:f5:4d + - 44:a1:91:e5:a8:e1 + - 44:a1:91:e5:a8:e2 + - 44:a1:91:e5:a8:e3 + - 44:a1:91:e5:a8:e4 +arch: aarch64 +nr_node: 4 +nr_cpu: 128 +model_name: Kunpeng-920 +ipmi_ip: 9.3.12.4 -- 2.23.0

1 0

[PATCH compass-ci 4/4] sparrow/4-docker: optimise function log_info() calling
by Liu Yinsi 05 Feb '21

05 Feb '21

before: just execute $container/build script, echo "build $container" after: only execute $container/build -> `docker build -t`, echo "build $container" [why] when execute `sparrow/4-docker/buildall run-only`, will execute $container/build script, but if image already exist, actually will not build container, it will exit code "1", so only exit code "0", call log_info "finish build $container" Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 83ecc9f..b24eca5 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -29,13 +29,9 @@ do_one_build() mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" - - log_info "start build $container." - [ -x build ] && ./build + [ "$?" == 0 ] && log_info "finish build $container" [ -x install ] && ./install - - log_info "finish build $container." ) lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

1 0

[PATCH compass-ci 3/4] sparrow/4-docker: export env DOCKER_SKIP_REBUILD
by Liu Yinsi 05 Feb '21

05 Feb '21

if env DOCKER_SKIP_REBUILD exists, then check image exists in $container/build script by call docker_skip_rebuild(), then not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index d4e9419..83ecc9f 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -70,11 +70,9 @@ do container_name=$(basename $dir) [ "$container_name" == 'ssh-r' ] && continue - docker images | grep -wq "$container_name" - if [ "$?" != 0 ] || [ "$action" != "run-only" ]; then - build_depends $dir & - wait - fi + [ "$action" == "run-only" ] && export DOCKER_SKIP_REBUILD=DOCKER_SKIP_REBUILD + build_depends $dir & + wait start_depends $dir done -- 2.23.0

1 0