- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib/git_bisect.rb: delete --first-parent in git start bisect command
by Cao Xueliang 07 Feb '21

07 Feb '21

The option will follow only the first parent commit upon seeing a merge commit, but we need bisect the commits in the merge, so, delete the option. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 40f6b49..dd0b782 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -89,7 +89,7 @@ class GitBisect puts "bad_commit: #{@bad_commit}" puts "good_commit: #{@good_commit}" - result = `git -C #{@work_dir} bisect start --first-parent #{@bad_commit} #{@good_commit}` + result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` -- 2.23.0

1 0

[PATCH v4 compass-ci] sbin: add script for download os rootfs
by Liu Yinsi 07 Feb '21

07 Feb '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100755 index 0000000..208ac7d --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +os=${1:-openeuler} +os_arch=${2:-aarch64} +os_version=${3:-20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p /srv/os/${rootfs_dir} + cd /srv/os/$rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${rootfs_dir}.cgz" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

4 8

[PATCH compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty after finish build container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..e77c19e 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -5,3 +5,5 @@ git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile + +rm -rf openresty -- 2.23.0

3 3

[PATCH compass-ci 1/4] container/defconfig: add function docker_skip_rebuild
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute `sparrow/4-docker/buildall run-only`, if caller script export env DOCKER_SKIP_REBUILD, then image already is exist, just exit build script, not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 6 ++++++ container/defconfig.sh | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index f9b0824..1a655a3 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -47,3 +47,9 @@ def get_available_memory # take the middle value according to the system memory size. [1024, 30720, Math.sqrt(memtotal) * 1024].sort[1].to_i end + +def docker_skip_rebuild(tag) + if ENV.include? 'DOCKER_SKIP_REBUILD' + exit 1 if system "docker image inspect #{tag} > /dev/null 2>&1" + end +end diff --git a/container/defconfig.sh b/container/defconfig.sh index 54e65b5..7612e2d 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -41,3 +41,12 @@ push_image() docker tag "$src_tag" "$dst_tag" docker push "$dst_tag" } + +docker_skip_rebuild() +{ + tag=$1 + [ $DOCKER_SKIP_REBUILD ] && { + docker image inspect $tag > /dev/null 2>&1 + [ "$?" == "0" ] && exit 1 + } +} -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] sparrow/dev-env: add the local-testbox-env.yaml
by Cao Xueliang 07 Feb '21

07 Feb '21

The scheduler will read local-testbox-env.yaml, then add the content to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 8812839..31a12db 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -59,6 +59,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/local-testbox-env.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] scheduler: add testbox env in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the testbox env use the submit script in lkp-tests, but the submit don't need to do that, so, we add the testbox env in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/testbox_env.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/testbox_env.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..5f5f53b 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/testbox_env.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(testbox_env) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/testbox_env.cr b/src/scheduler/testbox_env.cr new file mode 100644 index 0000000..a378e2a --- /dev/null +++ b/src/scheduler/testbox_env.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def testbox_env(flag = "local") + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/#{flag}-testbox-env.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: set default testbox to dc-8g
by Wei Jihui 07 Feb '21

07 Feb '21

set testbox to dc-8g when no testbox input Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6d..eacb7ec9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -345,26 +345,24 @@ class Job def load_hosts_config return if @job.include?(:no_defaults) - return if @load_hosts_done - return unless check_set_tbox_group + check_set_tbox_group hosts_file = get_hosts_file if hosts_file hwconfig = load_yaml(hosts_file, {}) @job[source_file_symkey(hosts_file)] = nil @job.merge!(hwconfig) { |_k, a, _b| a } # job's key/value has priority over hwconfig - @load_hosts_done = true end @job['arch'] ||= 'aarch64' end def check_set_tbox_group - if @job.include?('testbox') - @job['tbox_group'] = tbox_group(@job['testbox']) - return true + unless @job.include?('testbox') + @job[comment_to_symbol('set default testbox')] = nil + @job['testbox'] = 'dc-8g' end - return @job.include?('tbox_group') + @job['tbox_group'] = tbox_group(@job['testbox']) end def get_hosts_file @@ -745,7 +743,6 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| - @load_hosts_done = false @job = deepcopy(job) @job.merge!(hash) load_defaults -- 2.23.0

1 0

[PATCH v2 compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..5d30486 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

1 0

[PATCH compass-ci] update os list
by Wang Chenglong 07 Feb '21

07 Feb '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/support-os-os_arch-os_version.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/support-os-os_arch-os_version.md b/doc/support-os-os_arch-os_version.md index 6532f92..a881d67 100644 --- a/doc/support-os-os_arch-os_version.md +++ b/doc/support-os-os_arch-os_version.md @@ -1,8 +1,8 @@ ``` os os_arch os_version supported-os_mount openeuler aarch64 1.0 cifs/nfs, - 20.03 cifs/nfs, - 20.09 cifs/nfs, + 20.03 cifs/nfs, ,initramfs + 20.09 cifs/nfs, ,initramfs mainline-dailybuild cifs/nfs, 20.03-dailybuild cifs/nfs, 20.09-dailybuild cifs/nfs, @@ -10,13 +10,13 @@ openeuler aarch64 1.0 cifs/nfs, x86 20.03 cifs/nfs, 20.09 cifs/nfs, centos aarch64 7 container, - 7.6.1810 cifs/nfs, - 7.8.2003 cifs/nfs, + 7.6.1810 cifs/nfs, ,initramfs + 7.8.2003 cifs/nfs, ,initramfs 8.1.1911 cifs/nfs, 8 container, x86 7.6.1810 cifs/nfs, 7.8.2003 cifs/nfs, -debian aarch64 sid cifs/nfs, +debian aarch64 sid cifs/nfs, ,initramfs x86 sid cifs/nfs, archlinux aarch64 2020-11-12 cifs/nfs, ``` -- 2.23.0

2 1

[PATCH lkp-tests] lib/bootstrap.sh: add step: setup dns
by Yu Chuan 07 Feb '21

07 Feb '21

[Why] Since we have added a dns server, so we need to add support for dns function in lkp-tests framework. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- lib/bootstrap.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 1127de140a2b..06aa09bf65b2 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -283,6 +283,13 @@ setup_hosts() ln -fs $tmpfs_hosts /etc/hosts } +setup_dns() +{ + [ -n "$DNS_HOST" ] || return 0 + + echo "nameserver ${DNS_HOST}" > /etc/resolv.conf +} + export_ip_mac() { if has_cmd ip; then @@ -739,6 +746,7 @@ boot_init() setup_hostname setup_hosts + setup_dns add_lkp_user -- 2.23.0

1 0