November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 1/2] srv-http/start: add os images data download directory
by Cui Yili 29 Jan '21

29 Jan '21

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/srv-http/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start b/container/srv-http/start index b56fe4d..8c8ca0b 100755 --- a/container/srv-http/start +++ b/container/srv-http/start @@ -14,6 +14,7 @@ cmd=( -v /etc/localtime:/etc/localtime:ro -v /srv/result:/srv/result:ro -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro + -v /srv/os:/srv/os:ro -d srv-http ) -- 2.23.0

3 4

[PATCH lkp-tests 1/2] tests: replace *.tar.* address with local address
by Liu Shaofei 01 Dec '20

01 Dec '20

When the address of source field is the format of "*.tar.*", we need to replace it with local address. The following example: source=("https://github.com/azukiapp/${pkgname}/archive/v${pkgver}.tar.gz") => source=("http://${LKP_SERVER}:8800/initrd/build-tar/${pkgname}/v${pkgver}.tar.gz") Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tests/build-pkg b/tests/build-pkg index bdaef035..731ac409 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -98,6 +98,20 @@ replace_source() sed -i "s|^source=[^)]*|$1|g" "$2" } +# replace *.tar.* address from source field in PKGBUILD +replace_tar_source() +{ + echo "source=(" > PKGBUILD.src + for url in ${source[@]} + do + echo "$url" | egrep '.+\.tar\..+|.+\.tar' && { + url="\"http://${LKP_SERVER}:8800/initrd/build-tar/${pkgname}/${url##*/}\"" + } + echo "$url" >> PKGBUILD.src + done + echo ")" >> PKGBUILD.src +} + build_source_pkg() { local repo_dir="" @@ -116,6 +130,8 @@ build_source_pkg() [ -n "$PKGBUILD_TAG" ] && replace_source "${upstream_source}" "PKGBUILD-$PKGBUILD_TAG" } + replace_tar_source + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 -- 2.23.0

2 2

[PATCH v2 compass-ci 1/2] compare_matrixes.rb: put compare matrix with more members first
by Li Yuanchao 01 Dec '20

01 Dec '20

As hash table is disordered, the output of compare matrixes sometimes would put one member matrix first, that is not a good comparison. We should put good comparison first. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/compare_matrixes.rb | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index d8112e9..8232513 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -383,24 +383,33 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group +# the result with more comparison objects first def compare_group_matrices(group_matrices, suites_list, options) result_str = '' - group_matrices.each do |k, v| - matrices_list = [] - matrices_titles = [] - v.each do |dim, matrix| - matrices_titles << dim - matrices_list << matrix - end - if options[:no_print] - result_str += compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) - else - print compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) - end + group_matrices_array = sort_by_matrice_size(group_matrices) + group_matrices_array.each do |matrice_kv| + result_str += get_matrix_str(matrice_kv[0], matrice_kv[1], suites_list, options) end result_str end +def get_matrix_str(matrice_key, matrice_value, suites_list, options) + m_list = [] + m_titles = [] + matrice_value.each do |dim, matrix| + m_titles << dim + m_list << matrix + end + return compare_matrixes(m_list, suites_list.shift, m_titles, matrice_key, options: options) if options[:no_print] + + print compare_matrixes(m_list, suites_list.shift, m_titles, matrice_key, options: options) +end + +# big size first +def sort_by_matrice_size(group_matrices) + group_matrices.sort { |a, b| b[1].size <=> a[1].size } +end + # input: groups_matrices # { # group_key_1 => { -- 2.23.0

2 2

[PATCH compass-ci] delete source and add getting started
by Li Ping 01 Dec '20

01 Dec '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- README.md | 37 +++++++++++++++---------------------- 1 file changed, 15 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 80e0766..128eba0 100644 --- a/README.md +++ b/README.md @@ -10,50 +10,43 @@ Compass-CI 是一个可持续集成的软件平台。为开发者提供针对上 - **测试服务** - 使用Compass-CI 基于开源软件 PR 触发[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… +使用Compass-CI 基于开源软件 PR 触发[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… - **调测环境登录** - 使用 SSH [登录测试环境进行调测](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +使用 SSH [登录测试环境进行调测](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… - **测试结果分析** - 通过 [web](https://compass-ci.openeuler.org) 接口，对历史测试结果进行分析和比较。 +通过 [Web](https://compass-ci.openeuler.org) 接口，对历史测试结果进行分析和比较。 - **测试结果复现** - Compass-CI 把测试过程中的各项环境参数保存在任务结果输出的job.yaml文件中，开发者可以通过重新提交job.yaml复现测试结果。 +Compass-CI 把测试过程中的各项环境参数保存在任务结果输出的job.yaml文件中，开发者可以通过重新提交job.yaml复现测试结果。 - **辅助定位** - Compass-CI 可以识别自动化构建测试过程中的错误，触发基于 git tree 的测试，找出引入问题的commit。 +Compass-CI 可以识别自动化构建测试过程中的错误，触发基于 git tree 的测试，找出引入问题的commit。 -## 使用 Compass-CI -> 您只需要先注册自己的仓库，当您的仓库有 commit 提交时，构建测试会自动执行，并且可在我们的网站中查看结果。 +## Getting started -- 注册自己的仓库 +- **自动化测试** - 如果您想在 `git push` 的时候, 自动触发测试, 那么需要把您的公开 git url 添加到如下仓库 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos)。 - ```bash - git clone https://gitee.com/wu_fengguang/upstream-repos.git - less upstream-repos/README.md - ``` +1. 添加待测试仓库 URL 到 [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos.git) 仓库，[编写测试用例并添加到仓库](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/…, 详细流程请查看[这篇文档](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/man… -- `git push` - - 更新仓库，自动触发测试。 +2. git push 更新仓库，自动触发测试。 -- 在网页中搜索并查看结果 - - web: https://compass-ci.openeuler.org/jobs +3. 在网页中搜索并查看结果 web: https://compass-ci.openeuler.org/jobs -## Source +- **手动提交测试任务** -我们最新的，最完整的源码都存放在 [Gitee](https://gitee.com/wu_fengguang/compass-ci.git) 上，Fork 我们吧！ +1. [安装Compass-CI客户端](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/… +2. [编写测试用例](https/blob/master/doc/add-testcase.md), [手动提交测试任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… +3. 在网页中搜索并查看结果 web: https://compass-ci.openeuler.org/jobs ## Contributing to Compass-CI -我们非常欢迎有新的贡献者，我们也很乐意为我们的贡献者提供一些指导 +我们非常欢迎有新的贡献者，我们也很乐意为我们的贡献者提供一些指导，Compass-CI 主要是使用 Ruby 和 Crystal 开发的一个项目，我们使用 rubocop和 ameba 来规范我d的代码风格，具体说明可以查看 [rococop源码仓](https://github.com/rubocop-hq/rubocop)和[ameba源码仓](https://githu… ## Website -- 2.23.0

1 1

[PATCH compass-ci] doc/manual: add 本地安装compass-ci客户端.md
by Li Ping 30 Nov '20

30 Nov '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- ...ci\345\256\242\346\210\267\347\253\257.md" | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" new file mode 100644 index 0000000..43a6bf5 --- /dev/null +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -0,0 +1,28 @@ +# 本地安装compass-ci客户端 + +Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客户端，通过本地安装 lkp-tests 可以手动提交测试任务 + +:exclamation: 前提条件：按照 [apply-ssh-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master… 完成如下动作： +- send apply account email +- receive email from compass-ci(a)qq.com + +## Getting started + +1. setup default config + run the following command to add the below setup to default config file + mkdir -p ~/.config/compass-ci/defaults/ + cat >> ~/.config/compass-ci/defaults/${USER}.yaml <<-EOF + my_uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx + my_email: xxx + my_name: xx + EOF + +2. download lkp-tests and dependencies + run the following command to install the lkp-test and effect the configuration + git clone https://gitee.com/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source ~/.bashrc && source ~/.bash_profile + +3. submit job + Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: add 本地暗转compass-ci客户端.md
by Li Ping 30 Nov '20

30 Nov '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- ...ci\345\256\242\346\210\267\347\253\257.md" | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 "doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" new file mode 100644 index 0000000..43a6bf5 --- /dev/null +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -0,0 +1,28 @@ +# 本地安装compass-ci客户端 + +Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客户端，通过本地安装 lkp-tests 可以手动提交测试任务 + +:exclamation: 前提条件：按照 [apply-ssh-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master… 完成如下动作： +- send apply account email +- receive email from compass-ci(a)qq.com + +## Getting started + +1. setup default config + run the following command to add the below setup to default config file + mkdir -p ~/.config/compass-ci/defaults/ + cat >> ~/.config/compass-ci/defaults/${USER}.yaml <<-EOF + my_uuid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx + my_email: xxx + my_name: xx + EOF + +2. download lkp-tests and dependencies + run the following command to install the lkp-test and effect the configuration + git clone https://gitee.com/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source ~/.bashrc && source ~/.bash_profile + +3. submit job + Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -- 2.23.0

1 1

[PATCH v2 lkp-tests] job.rb: disable submit warn
by Wei Jihui 30 Nov '20

30 Nov '20

[why] it has a warn when submit job: WARN -- skip non-executable */lkp-tests/monitors/plain/wait [how] lkp-tests/monitors/plain/wait is link to bin/event/wakeup but bin/event/wakeup does not exist. so we will do not check link file when it real file is not exist. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index 8d1f7469..f9e244a9 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -115,6 +115,11 @@ def __create_programs_hash(glob, lkp_src) next if path =~ /\.[0-9]+$/ unless File.executable?(path) + if File.symlink?(path) + real_path = File.readlink(path) + real_path = File.join(File.dirname(path), real_path) if real_path =~ /^\.\.|^\w/ + next unless File.exist?(real_path) + end log_warn "skip non-executable #{path}" unless path =~ /\.cr$/ next end -- 2.23.0

1 0

[PATCH lkp-tests 2/2] sbin: load PKGBUILD.src content
by Liu Shaofei 30 Nov '20

30 Nov '20

The contents of PKGBUILD.src is the following example: source=( "http://172.168.131.113:8800/initrd/build-tar/ansible/ansible-2.9.4.tar.gz" ) Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- sbin/makepkg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sbin/makepkg b/sbin/makepkg index 6f8b95af..498b4872 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -895,6 +895,7 @@ update_pkgver() { sed --follow-symlinks -i "s:^pkgrel=[^ ]*:pkgrel=1:" "$BUILDFILE" source_safe "$BUILDFILE" [[ -n $PKGBUILD_TAG ]] && source_safe "$BUILDFILE-$PKGBUILD_TAG" + [[ -f PKGBUILD.src ]] && source_safe "PKGBUILD.src" local fullver=$(get_full_version) msg "$(gettext "Updated version: %s")" "$pkgbase $fullver" else @@ -3699,6 +3700,7 @@ else fi source_buildfile "$BUILDFILE" [[ -n $PKGBUILD_TAG ]] && source_buildfile "$BUILDFILE-$PKGBUILD_TAG" + [[ -f PKGBUILD.src ]] && source_buildfile PKGBUILD.src fi # set defaults if they weren't specified in buildfile -- 2.23.0

1 0

[PATCH lab-z9] hosts/vm-2p16g: add vm-2p16g file
by Xiao Shenwei 30 Nov '20

30 Nov '20

some job need big memory like build-pkg Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- hosts/vm-2p16g | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 hosts/vm-2p16g diff --git a/hosts/vm-2p16g b/hosts/vm-2p16g new file mode 100644 index 0000000..10847a7 --- /dev/null +++ b/hosts/vm-2p16g @@ -0,0 +1,6 @@ +arch: aarch64 +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 16G -- 2.23.0

1 0

[PATCH lkp-tests] hosts/vm-2p16g: add vm-2p16g file
by Xiao Shenwei 30 Nov '20

30 Nov '20

some job need big memory like build-pkg Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- hosts/vm-2p16g | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 hosts/vm-2p16g diff --git a/hosts/vm-2p16g b/hosts/vm-2p16g new file mode 100644 index 00000000..42845072 --- /dev/null +++ b/hosts/vm-2p16g @@ -0,0 +1,5 @@ +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 16G -- 2.23.0

1 0