February 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 2/2] sparrow/dev-env: add the local-testbox-env.yaml
by Cao Xueliang 07 Feb '21

07 Feb '21

The scheduler will read local-testbox-env.yaml, then add the content to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 8812839..31a12db 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -59,6 +59,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/local-testbox-env.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] scheduler: add testbox env in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the testbox env use the submit script in lkp-tests, but the submit don't need to do that, so, we add the testbox env in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/testbox_env.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/testbox_env.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..5f5f53b 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/testbox_env.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(testbox_env) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/testbox_env.cr b/src/scheduler/testbox_env.cr new file mode 100644 index 0000000..a378e2a --- /dev/null +++ b/src/scheduler/testbox_env.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def testbox_env(flag = "local") + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/#{flag}-testbox-env.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: set default testbox to dc-8g
by Wei Jihui 07 Feb '21

07 Feb '21

set testbox to dc-8g when no testbox input Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6d..eacb7ec9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -345,26 +345,24 @@ class Job def load_hosts_config return if @job.include?(:no_defaults) - return if @load_hosts_done - return unless check_set_tbox_group + check_set_tbox_group hosts_file = get_hosts_file if hosts_file hwconfig = load_yaml(hosts_file, {}) @job[source_file_symkey(hosts_file)] = nil @job.merge!(hwconfig) { |_k, a, _b| a } # job's key/value has priority over hwconfig - @load_hosts_done = true end @job['arch'] ||= 'aarch64' end def check_set_tbox_group - if @job.include?('testbox') - @job['tbox_group'] = tbox_group(@job['testbox']) - return true + unless @job.include?('testbox') + @job[comment_to_symbol('set default testbox')] = nil + @job['testbox'] = 'dc-8g' end - return @job.include?('tbox_group') + @job['tbox_group'] = tbox_group(@job['testbox']) end def get_hosts_file @@ -745,7 +743,6 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| - @load_hosts_done = false @job = deepcopy(job) @job.merge!(hash) load_defaults -- 2.23.0

1 0

[PATCH v2 compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..5d30486 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

1 0

[PATCH compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..a881d67 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

2 1

[PATCH lkp-tests] lib/bootstrap.sh: add step: setup dns
by Yu Chuan 07 Feb '21

07 Feb '21

[Why] Since we have added a dns server, so we need to add support for dns function in lkp-tests framework. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 1127de140a2b..06aa09bf65b2 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -283,6 +283,13 @@ setup_hosts() ln -fs $tmpfs_hosts /etc/hosts } +setup_dns() +{ + [ -n "$DNS_HOST" ] || return 0 + + echo "nameserver ${DNS_HOST}" > /etc/resolv.conf +} + export_ip_mac() { if has_cmd ip; then @@ -739,6 +746,7 @@ boot_init() setup_hostname setup_hosts + setup_dns add_lkp_user -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: delete blank line
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 464153d8f..b5b070f01 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -27,7 +27,6 @@ linux_dep() esac } - mac_dep() { brew install ruby -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: add submit depends
by Wang Yong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index 012c45499..464153d8f 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -12,13 +12,13 @@ linux_dep() case $ID in ubuntu|debian) export DEBIAN_FRONTEND=noninteractive - sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ + sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ uuid-runtime ;; openEuler|fedora|rhel|centos) if type dnf > /dev/null 2>&1; then - sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux else - sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp + sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux fi ;; *) -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result.rb: use upstream url and commit replace git url
by Cao Xueliang 07 Feb '21

07 Feb '21

We should show the upstream url in the bisect report, but the git url is our git server url, so, replace git url to the upstream url. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 8 ++++++++ lib/mail_bisect_result.rb | 3 ++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 5d13074..40f6b49 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -23,6 +23,7 @@ class GitBisect set_build_pkg_dir set_upstream set_pkgbuild_repo + set_upstream_url set_work_dir set_bad_commit set_good_commit @@ -56,6 +57,12 @@ class GitBisect @upstream_repo_git = "git://#{GIT_MIRROR_HOST}/#{@upstream_repo}" end + def set_upstream_url + @upstream_url = @bad_job['upstream_url'] + puts "upstream_url: #{@upstream_url}" + raise 'upstream url is null' unless @upstream_url + end + def set_work_dir @work_dir = Utils.clone_repo(@upstream_repo_git, @upstream_commit) puts "work_dir: #{@work_dir}" @@ -106,6 +113,7 @@ class GitBisect 'work_dir' => @work_dir, 'bisect_error' => bisect_error, 'all_errors' => all_errors, + 'upstream_url' => @upstream_url 'pkgbuild_repo' => @pkgbuild_repo, 'first_bad_commit_result_root' => first_bad_commit_result_root] end diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 637c008..5e1103c 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -16,6 +16,7 @@ class MailBisectResult @commit_id = bisect_info['commit'] @all_errors = bisect_info['all_errors'] @bisect_error = bisect_info['bisect_error'] + @upstream_url = bisect_info['upstream_url'] @pkgbuild_repo = bisect_info['pkgbuild_repo'] @first_bad_commit_result_root = bisect_info['first_bad_commit_result_root'] @git_commit = GitCommit.new(@work_dir, @commit_id) @@ -46,7 +47,7 @@ class MailBisectResult Hi #{(a)git_commit.author_name}, We found some error/warning(s) and the first bad commit in the below project: - git url: #{(a)git_commit.url} + git url: #{@upstream_url/@commit_id} git commit: #{@commit_id} ("#{(a)git_commit.subject}") All error/warning(s) (new ones prefixed by >>): -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends/cci-makepkg: add git depend
by Wei Jihui 07 Feb '21

07 Feb '21

user maybe use git cmd when wirte PKGBUILD Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/cci-makepkg | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-makepkg b/distro/depends/cci-makepkg index 4ed4f19e..855ac8ce 100644 --- a/distro/depends/cci-makepkg +++ b/distro/depends/cci-makepkg @@ -1,3 +1,4 @@ +git curl gnupg gettext -- 2.23.0

2 1