- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 04 Feb '21

04 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..d4d70c3 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) + +[ ! -n $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 04 Feb '21

04 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..e0197b9 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,12 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin/auto_submit: don't use auto_submit.yaml any more
by Li Yuanchao 04 Feb '21

04 Feb '21

Now DEFAULTS files are added to upstream-repos to keep submit commands. And these info would be taken by messages which the container git-mirror send. So auto_submit parses the messages and get the submit commands. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 14 +++++--------- sbin/auto_submit.yaml | 25 ------------------------- 2 files changed, 5 insertions(+), 34 deletions(-) delete mode 100644 sbin/auto_submit.yaml diff --git a/sbin/auto_submit b/sbin/auto_submit index c59deb7..410445d 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -17,7 +17,6 @@ class AutoSubmit connection.start channel = connection.create_channel @queue = channel.queue('new_refs') - @repo2job = YAML.load_file("#{$PROGRAM_NAME}.yaml") end def get_pkgbuild_repo(repo_array) @@ -57,20 +56,17 @@ class AutoSubmit return unless pkgbuild_repo submit_argv.push("pkgbuild_repo=#{pkgbuild_repo}") - repo2job_key = @repo2job[git_repo] ? git_repo : 'archlinux' - else - return unless @repo2job[git_repo] - - repo2job_key = git_repo end - [submit_argv, repo2job_key] + submit_argv end def submit_job(newrefs_info) - submit_argv, repo2job_key = get_argvs(newrefs_info) + submit_argv = get_argvs(newrefs_info) return unless submit_argv - @repo2job[repo2job_key].each do |argv_config| + return unless newrefs_info['submit'] + + newrefs_info['submit'].each do |argv_config| argvs = Array.new(submit_argv) argvs.push(argv_config) submit(newrefs_info, argvs) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml deleted file mode 100644 index c368ad6..0000000 --- a/sbin/auto_submit.yaml +++ /dev/null @@ -1,25 +0,0 @@ -a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml -c/compass-ci/compass-ci: -- testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -l/linux/linux: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/linux-next: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/openeuler-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/debian-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/archlinux-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=dc-8g os=openeuler os_version=20.03-pre os_mount=container os_arch=aarch64 docker_image=openeuler:20.03-pre queue=dc-build-pkg build-pkg.yaml -- testbox=dc-8g os=archlinux os_version=2021-01-06-pre os_mount=container os_arch=aarch64 docker_image=archlinux:2021-01-06-pre queue=dc-build-pkg build-pkg.yaml -- 2.23.0

1 0

[PATCH compass-ci 1/2] git_mirror.rb: optimize the use of hash.merge
by Li Yuanchao 04 Feb '21

04 Feb '21

1. hash_a.merge(hash_b), this means if the two hashes have some same keys, the last one will cover the first one. I change it so that if a repo file has the same key with DEFAULTS file, its own will be kept. 2. fix that if there's no DEFAULTS file, the merge_defaults method will return nil. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index fa2e0cc..8898a44 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -453,10 +453,12 @@ class MirrorMain def merge_defaults(git_repo) defaults_key = File.dirname(git_repo) while defaults_key != '.' - return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + return @defaults[defaults_key].merge(@git_info[git_repo]) if @defaults[defaults_key] defaults_key = File.dirname(defaults_key) end - return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + return @defaults['default'].merge(@git_info[git_repo]) if @defaults['default'] + + return @git_info[git_repo] end end -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix no my_token
by Luan Shengde 04 Feb '21

04 Feb '21

the lib/build_my_info_client will not generate my_token automatically. and lib/build_my_info_client do not support update if account exists. in the ES. use lib/build-my-info, it will gererate my_token if it's not in my_info. fix rubocop warning: register-accounts:11:18: W: Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used. content.each do |key, value| ^^^ Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index 519f289..e4d39ff 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,16 +5,15 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" +require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" content = YAML.load_file('/etc/compass-ci/register/register.yaml') -content.each do |key, value| +content.each do |_key, value| my_info = {} my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] - build_my_info = BuildMyInfo.new(my_info['my_email']) - build_my_info.store_account_info(my_info) + build_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..749f3667b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,12 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + case line.chomp! + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0

[PATCH v6 compass-ci 3/3] container/submit: add source file for debian packages
by Luan Shengde 04 Feb '21

04 Feb '21

it may take a long time, even break up if use the default source address to install bebian backages. use the source address inland will resolv the problem. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/conf/sources.list | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/submit/conf/sources.list diff --git a/container/submit/conf/sources.list b/container/submit/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/submit/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH v6 compass-ci 2/3] container/submit: executable file submit
by Luan Shengde 04 Feb '21

04 Feb '21

executable file for container: submit file 'submit' is used as the command installed in the container. it can be run as the real command submit itself in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..6236892 --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M%S) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /srv/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0

[PATCH v6 compass-ci 1/3] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..3ca13f4 --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +COPY conf/sources.list /etc/apt/ + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /srv + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

1 0

[PATCH compass-ci] container/redis: use echo "xxx" for safety reason
by Liu Yinsi 04 Feb '21

04 Feb '21

before: echo you can use cmd: redis-cli to start a redis client echo and then use cmd: keys * after: echo "you can use cmd: redis-cli to start a redis client" echo "and then use cmd: keys *" [why] when use echo $string, if $string include '*', will be parsed, just like: root@test-lys /c/compass-ci/container/redis# ./start ... and then use cmd: keys build Dockerfile start Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/redis/start | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/redis/start b/container/redis/start index 34266a0..4611fe4 100755 --- a/container/redis/start +++ b/container/redis/start @@ -24,6 +24,6 @@ cmd=( "${cmd[@]}" # test -echo you can use cmd: redis-cli to start a redis client -echo and then use cmd: keys * +echo "you can use cmd: redis-cli to start a redis client" +echo "and then use cmd: keys *" -- 2.23.0

2 2

[PATCH compass-ci] sparrow/5-build: download os rootfs
by Liu Yinsi 04 Feb '21

04 Feb '21

default download os rootfs openeuler aarch64 20.03 for boot testbox. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/os-ready | 5 +++++ sparrow/install-tiny | 1 + 2 files changed, 6 insertions(+) create mode 100755 sparrow/5-build/os-ready diff --git a/sparrow/5-build/os-ready b/sparrow/5-build/os-ready new file mode 100755 index 0000000..48a740b --- /dev/null +++ b/sparrow/5-build/os-ready @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/download-rootfs diff --git a/sparrow/install-tiny b/sparrow/install-tiny index a291549..5f026e5 100755 --- a/sparrow/install-tiny +++ b/sparrow/install-tiny @@ -19,4 +19,5 @@ cd $CCI_SRC/sparrow || exit . /etc/profile.d/compass.sh 4-docker/buildall 5-build/ipxe +5-build/os-ready 7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH compass-ci] sparrow/5-build: register my info
by Liu Yinsi 04 Feb '21

04 Feb '21

register user own account info to ES. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/register-account | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 sparrow/5-build/register-account diff --git a/sparrow/5-build/register-account b/sparrow/5-build/register-account new file mode 100755 index 0000000..1a23e07 --- /dev/null +++ b/sparrow/5-build/register-account @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +$CCI_SRC/sbin/build-my-info.rb -- 2.23.0

1 0

[PATCH lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 13 +++++++++++++ 1 file changed, 13 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..95e2a272b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,13 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + line.chomp! + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0

[PATCH v3 lkp-tests] stats: add stats script for dockerfile os migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] ^[[91mError: Unable to find a match: supervisor ^[[91m/bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/stats/openeuler_docker b/stats/openeuler_docker index e31934573..95e2a272b 100755 --- a/stats/openeuler_docker +++ b/stats/openeuler_docker @@ -2,7 +2,12 @@ while (line = STDIN.gets) line.chomp! - next unless line =~ /build.fail/ - - puts line + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end end -- 2.23.0

1 0

[PATCH compass-ci] delimiter/utils.rb: delete the nil elements in array
by Cao Xueliang 04 Feb '21

04 Feb '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 3120ca6..0930dd7 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -29,7 +29,7 @@ module Utils commits << temp_commit if temp_commit end commits << get_last_commit(work_dir, commit) if commits.empty? - return commits.to_a + return commits.to_a.compact end def get_day_ago_commit(work_dir, commit, day_ago) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: add document
by Luan Shengde 04 Feb '21

04 Feb '21

add chinese document for apply account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.zh.md | 56 ++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 doc/manual/apply-account.zh.md diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md new file mode 100644 index 0000000..52d43ba --- /dev/null +++ b/doc/manual/apply-account.zh.md @@ -0,0 +1,56 @@ +# 账号申请 + +首先您需要一个账号来提交job，同时用于跟踪您的资源消费情况。 +因为目前我们的测试机资源池有限，因此需要做一些使用限制。 + +我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 +你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 + +我们同时也为我们的合作者提供了免费的账号。 +这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 + +## 1. 发送邮件进行账号申请 + +邮件模板： + +--- + From: {{ David Rientjes <rientjes(a)google.com> }} + To: compass-ci-robot(a)qq.com + Subject: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ attach your ssh pubkey file(s), e.g. ~/.ssh/id_rsa.pub }} + +--- + +- 使用您的信息替换{{ }}中的内容 + +- 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” + +- my oss commit: 使用您的邮件和名字提交的代码的地址。 + 您的仓库需要在平台测试覆盖的范围内，并且注册到： + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + 如果贡献的OSS工程仓库不在upstream-repos中，请参考： + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + 将仓库注册到compass-ci测试池中。 + +## 2. 接收邮件 + +申请邮件成功后，您将收到反馈邮件，包括： + + my_name + my_email + my_token + lab + +## 3. 一次性配置 + +**根据步骤2收到邮件的提示，配置您的本地环境： + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + setup ~/.config/compass-ci/defaults/account.yaml + +## 4. 现在开始 + +[submit a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

2 2

[PATCH compass-ci] doc/manual: add document apply-account.zh.md
by Luan Shengde 04 Feb '21

04 Feb '21

add chinese document Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.zh.md | 58 ++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/manual/apply-account.zh.md diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md new file mode 100644 index 0000000..5e6605d --- /dev/null +++ b/doc/manual/apply-account.zh.md @@ -0,0 +1,58 @@ +# 账号申请 + +首先您需要一个账号，可以用来提交job，同时跟踪您的资源消费情况。 +因为目前我们的测试机资源池有限，因此需要做一些使用限制。 + +我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 +你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 + +我们同时也为我们的合作者提供了免费的账号。 +这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 + +## 1. 发送邮件进行账号申请 + +邮件模板： + +--- + 发件人: {{ David Rientjes <rientjes(a)google.com> }} + 收件人: compass-ci-robot(a)qq.com + 邮件标题: apply account + + my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + + {{ 将您的ssh公钥添加为邮件附件，例如 ~/.ssh/id_rsa.pub }} + +--- + +- 使用您的信息替换{{ }}中的内容 +- 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” +- my oss commit: 使用您的邮件和名字提交的代码的地址。 + + + 您的仓库需要在平台测试覆盖的范围内，并且注册到： + [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). + 如果贡献的OSS工程仓库不在upstream-repos中，请参考： + [test-oss-project](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/… + 将仓库注册到compass-ci测试池中。 + +## 2. 接收邮件 + +申请邮件成功后，您将收到反馈邮件，邮件内容包括以下信息： + + my_name + my_email + my_token + lab + +## 3. 一次性配置 + +根据步骤2收到邮件的提示，配置您的本地环境： + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/{{ lab }}.yaml + +## 4. 现在开始 + +[提交job到compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/submit-job.en.md) + -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/manual/apply-account: optimize document
by Luan Shengde 04 Feb '21

04 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index fda2eba..974fe63 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -43,6 +43,7 @@ Which contains account information for you: my_name my_email my_token + lab ## step 3: one-time setup @@ -50,6 +51,7 @@ Which contains account information for you: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/{{ lab }}.yaml ## step 4: try it out -- 2.23.0

1 0

[PATCH lkp-tests] distro/debian: use function instead of command
by Wang Yong 04 Feb '21

04 Feb '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/debian | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/distro/debian b/distro/debian index 9b6636f48..313f73e35 100755 --- a/distro/debian +++ b/distro/debian @@ -171,7 +171,7 @@ distro_install_depends() add_i386_package update - apt-get install -yf + fixup_preinstall for script in $scripts do script=$(basename $script) -- 2.23.0

1 0

[PATCH compass-ci] doc/manual/apply-account: optimize document
by Luan Shengde 04 Feb '21

04 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/manual/apply-account.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index fda2eba..ac7ce5e 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -43,6 +43,7 @@ Which contains account information for you: my_name my_email my_token + lab ## step 3: one-time setup @@ -50,6 +51,7 @@ Which contains account information for you: git clone https://gitee.com/wu_fengguang/lkp-tests.git setup ~/.config/compass-ci/defaults/account.yaml + ~/.config/compass-ci/include/lab/${lab}.yaml ## step 4: try it out -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] container/dracut-initrd: add rsync module to initrd
by Xu Xijian 04 Feb '21

04 Feb '21

Change filename add-cifs.conf to add-modules.conf, then add rsync module to dracut by put "rsync" into add-modules.conf. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/conf/{add-cifs.conf => add-modules.conf} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename container/dracut-initrd/conf/{add-cifs.conf => add-modules.conf} (100%) diff --git a/container/dracut-initrd/conf/add-cifs.conf b/container/dracut-initrd/conf/add-modules.conf similarity index 100% rename from container/dracut-initrd/conf/add-cifs.conf rename to container/dracut-initrd/conf/add-modules.conf -- 2.23.0

1 0

[PATCH v2 compass-ci] doc/write-PKGBUILD: fix documentation errors
by Lin Jiaxin 04 Feb '21

04 Feb '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- doc/manual/write-PKGBUILD.en.md | 5 +++-- doc/manual/write-PKGBUILD.zh.md | 5 +++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/doc/manual/write-PKGBUILD.en.md b/doc/manual/write-PKGBUILD.en.md index c0c25e9..c259399 100644 --- a/doc/manual/write-PKGBUILD.en.md +++ b/doc/manual/write-PKGBUILD.en.md @@ -13,8 +13,9 @@ PKGBUILD includes variables and functions. - pkgrel: Mandatory. It indicates the release number of a software package. - arch: Mandatory. It indicates the architecture sequence of a software package. - depends: Optional. It indicates the name of the dependency package required for running a software test. -- makedepends: Optional. It indicates the list of files required for building a software package. -- source: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. +- makedepends: Optional. It indicates the name of the dependency package required for building a software package. +- source: Optional. It indicates the list of files required for building a software package. +- md5sums: Optional. It indicates the MD5 hash value of each source file to verify the integrity of the source file during the build process. ## Defining a Function diff --git a/doc/manual/write-PKGBUILD.zh.md b/doc/manual/write-PKGBUILD.zh.md index 414bfdc..ff53f0f 100644 --- a/doc/manual/write-PKGBUILD.zh.md +++ b/doc/manual/write-PKGBUILD.zh.md @@ -13,8 +13,9 @@ PKGBUILD�� - pkgrel: �� - arch: �� - depends: �� -- makedepends: �� -- source: ��MD5�� +- makedepends: �� +- source: �� +- md5sums: ��MD5�� ## �� -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: add --fix-missing when apt-get install
by Xu Xijian 04 Feb '21

04 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 7e2bf51..75148a1 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -5,9 +5,9 @@ export DEBIAN_FRONTEND=noninteractive apt-get update -apt-get install -y --no-install-recommends apt-utils >/dev/null 2>&1 -apt-get install -y nfs-common netbase cifs-utils kmod rsync -apt-get install -y dracut dracut-network dracut-config-generic +apt-get install -y --fix-missing --no-install-recommends apt-utils >/dev/null 2>&1 +apt-get install -y --fix-missing nfs-common netbase cifs-utils kmod rsync +apt-get install -y --fix-missing dracut dracut-network dracut-config-generic apt-get clean rm -rf /var/lib/apt/lists/* -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/lib: introduce domain.rb action.rb
by Xiao Shenwei 04 Feb '21

04 Feb '21

function: based on response of boot.libvirt generate a domain.xml refer-to: commit 457644c0486e0786ed8b563c6e4928445eea0207 if you want to test with your domain.xml, define vt field in job.yaml vt: domain: xxx/xxx.xml if not will use default options to generate a domain.xml TODO: case 1: Impact of single factors on tasks: vt: cpu: xxx/cpu.xml case 2: Impact of multiple factors on tasks: vt: cpu: xxx/cpu.xml disk: xxx/disk.xml Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/lib/action.rb | 75 +++++++++++++++++++++++++++++++++++++++++ providers/lib/domain.rb | 75 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 150 insertions(+) create mode 100644 providers/lib/action.rb create mode 100644 providers/lib/domain.rb diff --git a/providers/lib/action.rb b/providers/lib/action.rb new file mode 100644 index 0000000..526d84d --- /dev/null +++ b/providers/lib/action.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# expand each element of the domain.xml +module Action + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + def expand_erb(file) + @context.expand_erb(File.read(file)) + end + + def domain + @doc = Nokogiri::XML(expand_erb(@domain_option)) + end + + def name + @doc.xpath('//domain/name').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/name.xml") + end + + def os + @doc.xpath('//domain/os').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/os.xml") + end + + def emulator + @doc.xpath('//domain/devices/emulator').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/emulator.xml") + end + + def cpu + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/cpu.xml") + end + + def memory + @doc.xpath('//domain/memory').remove + @doc.xpath('//domain/currentMemory').remove + @doc.xpath('//domain/maxMemory').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/memory.xml") + end + + def devices + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/devices.xml") + end + + def disk + @context.info['disk'] ||= nil + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/disk.xml") + end + + def serial + @doc.xpath('//domain/devices/serial').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/serial.xml") + end + + def interface + @doc.xpath('//domain/devices/interface').remove + @doc.xpath('//domain/devices')[0].add_child expand_erb("#{TEMPLATE_DIR}/interface.xml") + end + + def active + @doc.xpath('//domain/on_poweroff').remove + @doc.xpath('//domain/on_reboot').remove + @doc.xpath('//domain/on_crash').remove + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/active.xml") + end + + def clock + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/clock.xml") + end + + def seclabel + @doc.root.add_child expand_erb("#{TEMPLATE_DIR}/seclabel.xml") + end +end diff --git a/providers/lib/domain.rb b/providers/lib/domain.rb new file mode 100644 index 0000000..8df9c06 --- /dev/null +++ b/providers/lib/domain.rb @@ -0,0 +1,75 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'set' +require 'nokogiri' +require 'yaml' +require_relative 'action' +require_relative "#{ENV['LKP_SRC']}/lib/hashugar" + +# generate a domain.xml +class Domain + include Action + + TEMPLATE_DIR = "#{ENV['CCI_SRC']}/providers/libvirt/templates" + OPTION_FILE = "#{TEMPLATE_DIR}/options.yaml" + def initialize(context, logger) + @doc = nil + @context = context + @logger = logger + @options = Hashugar.new(YAML.safe_load(File.read(OPTION_FILE))) + end + + def generate + domain_option + if user_domain? + default_option + return + end + replaceable_option + default_option + end + + def save_to(filename) + File.open(filename, 'w') do |f| + f.puts @doc.to_xml + end + File.realpath(filename) + end + + private + + def user_domain? + @context.info['vt'].key?('domain') + end + + def load_xml_file(filepath) + host = @context.info['SRV_HTTP_HOST'] + port = @context.info['SRV_HTTP_PORT'] + system "wget --timestamping --progress=bar:force http://#{host}:#{port}/cci/libvirt-xml/#{filepath}" + %x(basename #{filepath}).chomp + end + + def domain_option + @domain_option = "#{TEMPLATE_DIR}/#{(a)options.domain}.xml" + if user_domain? + @domain_option = load_xml_file((a)context.info['vt']['domain']) + end + @logger.info("Domain option: #{@domain_option}") + domain + end + + def default_option + @options.default.each do |one| + instance_eval one + end + end + + def replaceable_option + @options.replaceable.each do |one| + instance_eval one + end + end +end -- 2.23.0

1 0

[PATCH v5 compass-ci 1/4] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..5ea0fcc --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +RUN echo 'deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n \ +\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n' \ +> /etc/apt/sources.list + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /root + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

2 2

2024

2023

2022

2021

2020

Compass-ci