- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] distro/*: fix output information
by Wang Yong 04 Feb '21

04 Feb '21

[before] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench.cgz [after] package installed to /initrd/deps/nfs/$os/$os_arch/$os_version/sysbench-cpu/sysbench-cpu.cgz Thanks, Yong --- distro/aliyun | 2 +- distro/archlinux | 2 +- distro/clear | 2 +- distro/debian | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/distro/aliyun b/distro/aliyun index 598243d8e..83727ce32 100755 --- a/distro/aliyun +++ b/distro/aliyun @@ -94,7 +94,7 @@ yum_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.rpm > $pack_to/.${BM_NAME}.packages diff --git a/distro/archlinux b/distro/archlinux index d7b3d8d78..43c3aa13a 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -64,7 +64,7 @@ pacman_pack() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz ls $target_dir/*.pkg.tar* > $pack_to/.${BM_NAME}.packages diff --git a/distro/clear b/distro/clear index f21de89c3..5d6fb0e46 100755 --- a/distro/clear +++ b/distro/clear @@ -135,7 +135,7 @@ pack_bundles_files() ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } # 1. install all un-installed bundles and record it to uninstalled_bundles diff --git a/distro/debian b/distro/debian index 313f73e35..21dafe21b 100755 --- a/distro/debian +++ b/distro/debian @@ -120,7 +120,7 @@ pack() chown .lkp $pack_to/${BM_NAME}_$date.cgz \ $pack_to/${BM_NAME}.cgz || return - echo package installed to $pack_to/${BM_NAME}.cgz + echo package installed to $pack_to/${benchmark}.cgz } resolve_depends() -- 2.23.0

1 0

[PATCH compass-ci 2/2] sparrow/3-code: set server_ip
by Liu Yinsi 04 Feb '21

04 Feb '21

[how] before: server_ip is the local host ip after: server_ip from config yaml /etc/compass-ci/setup.yaml [why] if deploy compass-ci client and server not in a same machine, need to config server ip in config yaml, or client can't access server services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 46f450d..d4d70c3 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,8 +2,13 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +mkdir -p /etc/compass-ci +cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml +server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) + +[ ! -n $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') + server_name=$(hostname | cut -f1 -d.) -server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') : ${sched_host:=$server_ip} : ${sched_port:=3000} : ${os_http_host:=$server_ip} -- 2.23.0

1 0

[PATCH compass-ci 1/2] sparrow: add initial config setup.yaml
by Liu Yinsi 04 Feb '21

04 Feb '21

[why] before locally deploy compass-ci, fill config setup.yaml firstly, will make the whole process cleaner. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..e0197b9 --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,12 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. + +# account name +my_name: xxx + +# email address +my_email: xxx(a)xxx.com + +# when you deploy compass-ci client, fill the compass-ci server's ip, +# then client can access server service. +server_ip: -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin/auto_submit: don't use auto_submit.yaml any more
by Li Yuanchao 04 Feb '21

04 Feb '21

Now DEFAULTS files are added to upstream-repos to keep submit commands. And these info would be taken by messages which the container git-mirror send. So auto_submit parses the messages and get the submit commands. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 14 +++++--------- sbin/auto_submit.yaml | 25 ------------------------- 2 files changed, 5 insertions(+), 34 deletions(-) delete mode 100644 sbin/auto_submit.yaml diff --git a/sbin/auto_submit b/sbin/auto_submit index c59deb7..410445d 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -17,7 +17,6 @@ class AutoSubmit connection.start channel = connection.create_channel @queue = channel.queue('new_refs') - @repo2job = YAML.load_file("#{$PROGRAM_NAME}.yaml") end def get_pkgbuild_repo(repo_array) @@ -57,20 +56,17 @@ class AutoSubmit return unless pkgbuild_repo submit_argv.push("pkgbuild_repo=#{pkgbuild_repo}") - repo2job_key = @repo2job[git_repo] ? git_repo : 'archlinux' - else - return unless @repo2job[git_repo] - - repo2job_key = git_repo end - [submit_argv, repo2job_key] + submit_argv end def submit_job(newrefs_info) - submit_argv, repo2job_key = get_argvs(newrefs_info) + submit_argv = get_argvs(newrefs_info) return unless submit_argv - @repo2job[repo2job_key].each do |argv_config| + return unless newrefs_info['submit'] + + newrefs_info['submit'].each do |argv_config| argvs = Array.new(submit_argv) argvs.push(argv_config) submit(newrefs_info, argvs) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml deleted file mode 100644 index c368ad6..0000000 --- a/sbin/auto_submit.yaml +++ /dev/null @@ -1,25 +0,0 @@ -a/AvxToNeon/AvxToNeon: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 api-avx2neon.yaml -c/compass-ci/compass-ci: -- testbox=taishan200-2280-2s64p-256g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -- testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml -l/linux/linux: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/linux-next: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/openeuler-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/debian-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -l/linux/archlinux-kernel: -- testbox=vm-2p32g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 build-linux.yaml -- testbox=vm-2p32g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-linux.yaml -archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=cifs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 queue=vm-build-pkg build-pkg.yaml -- testbox=dc-8g os=openeuler os_version=20.03-pre os_mount=container os_arch=aarch64 docker_image=openeuler:20.03-pre queue=dc-build-pkg build-pkg.yaml -- testbox=dc-8g os=archlinux os_version=2021-01-06-pre os_mount=container os_arch=aarch64 docker_image=archlinux:2021-01-06-pre queue=dc-build-pkg build-pkg.yaml -- 2.23.0

1 0

[PATCH compass-ci 1/2] git_mirror.rb: optimize the use of hash.merge
by Li Yuanchao 04 Feb '21

04 Feb '21

1. hash_a.merge(hash_b), this means if the two hashes have some same keys, the last one will cover the first one. I change it so that if a repo file has the same key with DEFAULTS file, its own will be kept. 2. fix that if there's no DEFAULTS file, the merge_defaults method will return nil. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index fa2e0cc..8898a44 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -453,10 +453,12 @@ class MirrorMain def merge_defaults(git_repo) defaults_key = File.dirname(git_repo) while defaults_key != '.' - return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + return @defaults[defaults_key].merge(@git_info[git_repo]) if @defaults[defaults_key] defaults_key = File.dirname(defaults_key) end - return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + return @defaults['default'].merge(@git_info[git_repo]) if @defaults['default'] + + return @git_info[git_repo] end end -- 2.23.0

1 0

[PATCH compass-ci] container/register-accounts: fix no my_token
by Luan Shengde 04 Feb '21

04 Feb '21

the lib/build_my_info_client will not generate my_token automatically. and lib/build_my_info_client do not support update if account exists. in the ES. use lib/build-my-info, it will gererate my_token if it's not in my_info. fix rubocop warning: register-accounts:11:18: W: Lint/UnusedBlockArgument: Unused block argument - key. If it's necessary, use _ or _key as an argument name to indicate that it won't be used. content.each do |key, value| ^^^ Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index 519f289..e4d39ff 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -5,16 +5,15 @@ require 'yaml' require_relative "#{ENV['CCI_SRC']}/lib/es_client" -require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" +require_relative "#{ENV['CCI_SRC']}/sbin/build-my-info" content = YAML.load_file('/etc/compass-ci/register/register.yaml') -content.each do |key, value| +content.each do |_key, value| my_info = {} my_info['my_email'] = value['my_email'] my_info['my_name'] = value['my_name'] my_info['lab'] = ENV['lab'] - build_my_info = BuildMyInfo.new(my_info['my_email']) - build_my_info.store_account_info(my_info) + build_my_info(my_info) end sleep -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats: add stats script for dockerfile migrating
by Bai Jing 04 Feb '21

04 Feb '21

[input] Error: Unable to find a match: supervisor /bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..749f3667b --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,12 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + case line.chomp! + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end +end -- 2.23.0

1 0

[PATCH v6 compass-ci 3/3] container/submit: add source file for debian packages
by Luan Shengde 04 Feb '21

04 Feb '21

it may take a long time, even break up if use the default source address to install bebian backages. use the source address inland will resolv the problem. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/conf/sources.list | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/submit/conf/sources.list diff --git a/container/submit/conf/sources.list b/container/submit/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/submit/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH v6 compass-ci 2/3] container/submit: executable file submit
by Luan Shengde 04 Feb '21

04 Feb '21

executable file for container: submit file 'submit' is used as the command installed in the container. it can be run as the real command submit itself in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..6236892 --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M%S) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /srv/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0

[PATCH v6 compass-ci 1/3] container/submit: Dockerfile
by Luan Shengde 04 Feb '21

04 Feb '21

Instal lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..3ca13f4 --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time, even break up if you use default source address to install pkgs. +# you can delete the following step if you do not want to use it. +COPY conf/sources.list /etc/apt/ + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /srv + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

1 0