- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 1/2] container/register-accounts: fix env lab not exist
by Liu Yinsi 07 Feb '21

07 Feb '21

container/register-accounts: ... my_info['lab'] = ENV['lab'] ... [why] when use env lab in container, must add into env in start script, or can't get env lab. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/register-accounts/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/register-accounts/start b/container/register-accounts/start index b1ebe98..c2acc75 100755 --- a/container/register-accounts/start +++ b/container/register-accounts/start @@ -9,6 +9,7 @@ require_relative '../defconfig.rb' names = Set.new %w[ ES_HOST ES_PORT + lab ] defaults = relevant_defaults(names) -- 2.23.0

1 0

[PATCH compass-ci] lib/job.cr: add to adjust local mount when get_initrds
by Xu Xijian 07 Feb '21

07 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index c013aa7..89c92c1 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -509,7 +509,7 @@ class Job if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) - elsif "#{os_mount}" == "nfs" || "#{os_mount}" == "cifs" + elsif "#{os_mount}" == "nfs" || "#{os_mount}" == "cifs" || "#{os_mount}" == "local" temp_initrds.concat(nfs_cifs_initrds()) end -- 2.23.0

2 2

[PATCH v2 compass-ci] doc/manual: add document apply-account.zh.md
by Luan Shengde 07 Feb '21

07 Feb '21

add Chinese document for applying account. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.zh.md | 58 ++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/manual/apply-account.zh.md diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md new file mode 100644 index 0000000..5e6605d --- /dev/null +++ b/doc/manual/apply-account.zh.md @@ -0,0 +1,58 @@ +# 账号申请 + +首先您需要一个账号，可以用来提交job，同时跟踪您的资源消费情况。 +因为目前我们的测试机资源池有限，因此需要做一些使用限制。 + +我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 +你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 + +我们同时也为我们的合作者提供了免费的账号。 +这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 + +## 1. 发送邮件进行账号申请 + +邮件模板： + +--- + 发件人: {{ David Rientjes <rientjes(a)google.com> }} + 收件人: compass-ci-robot(a)qq.com + 邮件标题: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ 将您的ssh公钥添加为邮件附件，例如 ~/.ssh/id_rsa.pub }} + +--- + +- 使用您的信息替换{{ }}中的内容 +- 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” +- my oss commit: 使用您的邮件和名字提交的代码的地址。 + + + 您的仓库需要在平台测试覆盖的范围内，并且注册到： + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + 如果贡献的OSS工程仓库不在upstream-repos中，请参考： + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + 将仓库注册到compass-ci测试池中。 + +## 2. 接收邮件 + +申请邮件成功后，您将收到反馈邮件，邮件内容包括以下信息： + + my_name + my_email + my_token + lab + +## 3. 一次性配置 + +根据步骤2收到邮件的提示，配置您的本地环境： + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/{{ lab }}.yaml + +## 4. 现在开始 + +[提交job到compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit-job.en.md) + -- 2.23.0

1 0

[PATCH compass-ci] lib/git_bisect.rb: delete --first-parent in git start bisect command
by Cao Xueliang 07 Feb '21

07 Feb '21

The option will follow only the first parent commit upon seeing a merge commit, but we need bisect the commits in the merge, so, delete the option. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 40f6b49..dd0b782 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -89,7 +89,7 @@ class GitBisect puts "bad_commit: #{@bad_commit}" puts "good_commit: #{@good_commit}" - result = `git -C #{@work_dir} bisect start --first-parent #{@bad_commit} #{@good_commit}` + result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` -- 2.23.0

1 0

[PATCH v4 compass-ci] sbin: add script for download os rootfs
by Liu Yinsi 07 Feb '21

07 Feb '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100755 index 0000000..208ac7d --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +os=${1:-openeuler} +os_arch=${2:-aarch64} +os_version=${3:-20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p /srv/os/${rootfs_dir} + cd /srv/os/$rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${rootfs_dir}.cgz" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

4 8

[PATCH compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty after finish build container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..e77c19e 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -5,3 +5,5 @@ git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile + +rm -rf openresty -- 2.23.0

3 3

[PATCH compass-ci 1/4] container/defconfig: add function docker_skip_rebuild
by Liu Yinsi 07 Feb '21

07 Feb '21

when execute `sparrow/4-docker/buildall run-only`, if caller script export env DOCKER_SKIP_REBUILD, then image already is exist, just exit build script, not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 6 ++++++ container/defconfig.sh | 9 +++++++++ 2 files changed, 15 insertions(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index f9b0824..1a655a3 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -47,3 +47,9 @@ def get_available_memory # take the middle value according to the system memory size. [1024, 30720, Math.sqrt(memtotal) * 1024].sort[1].to_i end + +def docker_skip_rebuild(tag) + if ENV.include? 'DOCKER_SKIP_REBUILD' + exit 1 if system "docker image inspect #{tag} > /dev/null 2>&1" + end +end diff --git a/container/defconfig.sh b/container/defconfig.sh index 54e65b5..7612e2d 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -41,3 +41,12 @@ push_image() docker tag "$src_tag" "$dst_tag" docker push "$dst_tag" } + +docker_skip_rebuild() +{ + tag=$1 + [ $DOCKER_SKIP_REBUILD ] && { + docker image inspect $tag > /dev/null 2>&1 + [ "$?" == "0" ] && exit 1 + } +} -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] sparrow/dev-env: add the local-testbox-env.yaml
by Cao Xueliang 07 Feb '21

07 Feb '21

The scheduler will read local-testbox-env.yaml, then add the content to the job yaml. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 8812839..31a12db 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -59,6 +59,23 @@ ES_PORT: $ES_PORT SRV_HTTP_PORT: $SRV_HTTP_PORT EOF +cat > /etc/compass-ci/defaults/local-testbox-env.yaml <<-EOF +LKP_SERVER: $server_ip +GIT_SERVER: $GIT_SERVER +INITRD_HTTP_HOST: $initrd_http_host +INITRD_HTTP_PORT: $initrd_http_port +GITCACHE_HOST: $gitcache_host +GITCACHE_PORT: $gitcache_port +SEND_MAIL_HOST: $server_ip +SEND_MAIL_PORT: $SEND_MAIL_PORT +ASSIST_RESULT_HOST: $server_ip +ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT +RESULT_WEBDAV_HOST: $server_ip +RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT +SRV_HTTP_HOST: $server_ip +SRV_HTTP_PORT: $SRV_HTTP_PORT +EOF + cat > /etc/profile.d/compass.sh <<'EOF' export LKP_SRC=/c/lkp-tests export CCI_SRC=/c/compass-ci -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] scheduler: add testbox env in scheduler
by Cao Xueliang 07 Feb '21

07 Feb '21

We add the testbox env use the submit script in lkp-tests, but the submit don't need to do that, so, we add the testbox env in scheduler. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 ++ src/scheduler/testbox_env.cr | 17 +++++++++++++++++ 2 files changed, 19 insertions(+) create mode 100644 src/scheduler/testbox_env.cr diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..5f5f53b 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -9,6 +9,7 @@ require "scheduler/constants.cr" require "scheduler/jobfile_operate.cr" require "scheduler/kernel_params.cr" require "scheduler/pp_params.cr" +require "scheduler/testbox_env.cr" require "../scheduler/elasticsearch_client" require "./json_logger" @@ -52,6 +53,7 @@ class Job def initialize(job_content : JSON::Any, id) @hash = job_content.as_h + @hash.merge!(testbox_env) @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new diff --git a/src/scheduler/testbox_env.cr b/src/scheduler/testbox_env.cr new file mode 100644 index 0000000..a378e2a --- /dev/null +++ b/src/scheduler/testbox_env.cr @@ -0,0 +1,17 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +class Job + def testbox_env(flag = "local") + hash = Hash(String, JSON::Any).new + + yaml_any = File.open("/etc/compass-ci/defaults/#{flag}-testbox-env.yaml") do |content| + YAML.parse(content).as_h? + end + return hash unless yaml_any + + hash.merge!(Hash(String, JSON::Any).from_json(yaml_any.to_json)) + + hash + end +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: set default testbox to dc-8g
by Wei Jihui 07 Feb '21

07 Feb '21

set testbox to dc-8g when no testbox input Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index f19d1e6d..eacb7ec9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -345,26 +345,24 @@ class Job def load_hosts_config return if @job.include?(:no_defaults) - return if @load_hosts_done - return unless check_set_tbox_group + check_set_tbox_group hosts_file = get_hosts_file if hosts_file hwconfig = load_yaml(hosts_file, {}) @job[source_file_symkey(hosts_file)] = nil @job.merge!(hwconfig) { |_k, a, _b| a } # job's key/value has priority over hwconfig - @load_hosts_done = true end @job['arch'] ||= 'aarch64' end def check_set_tbox_group - if @job.include?('testbox') - @job['tbox_group'] = tbox_group(@job['testbox']) - return true + unless @job.include?('testbox') + @job[comment_to_symbol('set default testbox')] = nil + @job['testbox'] = 'dc-8g' end - return @job.include?('tbox_group') + @job['tbox_group'] = tbox_group(@job['testbox']) end def get_hosts_file @@ -745,7 +743,6 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| - @load_hosts_done = false @job = deepcopy(job) @job.merge!(hash) load_defaults -- 2.23.0

1 0