- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 3/4] container/submit: submit file
by Luan Shengde 01 Feb '21

01 Feb '21

executable file for container: submit it can be run as the real command submit in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..20fdf61 --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,19 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +suffix=$(date +%m%d-%H%M) +cmd=( + docker run + --rm + --name=submit-$USER-$suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /root/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

1 0

[PATCH v3 compass-ci 2/4] container/submit: build file
by Luan Shengde 01 Feb '21

01 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/submit/build diff --git a/container/submit/build b/container/submit/build new file mode 100755 index 0000000..fba4281 --- /dev/null +++ b/container/submit/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t submit:latest . -- 2.23.0

1 0

[PATCH v3 compass-ci 1/4] container/submit: Dockerfile
by Luan Shengde 01 Feb '21

01 Feb '21

Install lkp-tests and its deps in the container instead of install them at localhost. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..f957af8 --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,36 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +# it may take a long time if use default source address if in zh. +# you can delete the following step if you do not want to use it. +RUN echo 'deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n \ +\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n' \ +> /etc/apt/sources.list + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +# it may take a long time, even break up if you use default source address for the gem. +# delete the following step if you do not want to use the default one. +RUN gem sources --add https://gems.ruby-china.com/ && \ + gem sources --remove https://rubygems.org/ + +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /root + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

1 0

[PATCH compass-ci] git-mirror: fix build error
by Yu Chuan 01 Feb '21

01 Feb '21

[Why] After `apt add ruby-dev` is executed, the `gem` command can be available. [Error Msg] Step 6/7 : RUN apk update && gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && apk upgrade && apk add--no-cache git && apk add ruby-dev make gcc g++ openssh ---> Running in 4c1f8708c072 fetch http://mirrors.huaweicloud.com/alpine/v3.8/main/aarch64/APKINDEX.tar.gz fetch http://mirrors.huaweicloud.com/alpine/v3.8/community/aarch64/APKINDEX.tar.gz v3.8.5-65-g097f0358a9 [http://mirrors.huaweicloud.com/alpine/v3.8/main] v3.8.5-37-gf06ffe835a [http://mirrors.huaweicloud.com/alpine/v3.8/community] OK: 9385 distinct packages available /bin/sh: gem: not found The command '/bin/sh -c apk update && gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && apk upgrade && apk add --no-cache git && apk add ruby-dev make gcc g++ openssh' returned a non-zero code: 127 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/git-mirror/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/git-mirror/Dockerfile b/container/git-mirror/Dockerfile index 2eac95c54ca1..18613eddf30e 100644 --- a/container/git-mirror/Dockerfile +++ b/container/git-mirror/Dockerfile @@ -10,10 +10,10 @@ RUN adduser -u 1090 -D lkp RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk update && \ - gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ apk upgrade && \ apk add --no-cache git && \ - apk add ruby-dev make gcc g++ openssh + apk add ruby-dev make gcc g++ openssh && \ + gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ RUN umask 002 && \ gem install bunny json PriorityQueue elasticsearch --no-rdoc --no-ri -- 2.23.0

2 3

[PATCH v2 compass-ci] git-mirror: fix build error
by Yu Chuan 01 Feb '21

01 Feb '21

[Why] After `apt add ruby-dev` is executed, the `gem` command can be available. And 'change gem source' should also be before executing 'gem install xxx'. [Error Msg] Step 6/7 : RUN apk update && gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && apk upgrade && apk add--no-cache git && apk add ruby-dev make gcc g++ openssh ---> Running in 4c1f8708c072 fetch http://mirrors.huaweicloud.com/alpine/v3.8/main/aarch64/APKINDEX.tar.gz fetch http://mirrors.huaweicloud.com/alpine/v3.8/community/aarch64/APKINDEX.tar.gz v3.8.5-65-g097f0358a9 [http://mirrors.huaweicloud.com/alpine/v3.8/main] v3.8.5-37-gf06ffe835a [http://mirrors.huaweicloud.com/alpine/v3.8/community] OK: 9385 distinct packages available /bin/sh: gem: not found The command '/bin/sh -c apk update && gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && apk upgrade && apk add --no-cache git && apk add ruby-dev make gcc g++ openssh' returned a non-zero code: 127 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/git-mirror/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/git-mirror/Dockerfile b/container/git-mirror/Dockerfile index 2eac95c54ca1..127d0109d929 100644 --- a/container/git-mirror/Dockerfile +++ b/container/git-mirror/Dockerfile @@ -10,11 +10,11 @@ RUN adduser -u 1090 -D lkp RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk update && \ - gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ apk upgrade && \ apk add --no-cache git && \ apk add ruby-dev make gcc g++ openssh RUN umask 002 && \ + gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ gem install bunny json PriorityQueue elasticsearch --no-rdoc --no-ri -- 2.23.0

1 0

[PATCH compass-ci 1/4] container/submit: Dockerfile
by Luan Shengde 01 Feb '21

01 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/Dockerfile | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 container/submit/Dockerfile diff --git a/container/submit/Dockerfile b/container/submit/Dockerfile new file mode 100644 index 0000000..e4c2cbd --- /dev/null +++ b/container/submit/Dockerfile @@ -0,0 +1,31 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian + +MAINTAINER Luan Shengde <shdluan(a)163.com> + +ENV DEBIAN_FRONTEND noninteractive + +RUN echo 'deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n \ +\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib\n \ +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib\n' \ +> /etc/apt/sources.list + +RUN apt-get update && \ + apt-get install -y git ruby ruby-dev make libssl-dev gcc g++ sudo uuid-runtime + +RUN gem sources --add https://gems.ruby-china.com/ +RUN gem sources --remove https://rubygems.org/ +RUN gem install git faye-websocket activesupport rest-client + +WORKDIR /root + +RUN git clone https://gitee.com/wu_fengguang/lkp-tests.git +RUN cd lkp-tests && make install -- 2.23.0

2 4

[PATCH compass-ci 2/2] lib/build_my_info_client: fix always generate new my_token
by Luan Shengde 01 Feb '21

01 Feb '21

[why] everytime invoking the script, it will generate a new my_token for an existing user [how] for old user, only if it has added a new token for the class, the patch will update the my_token, else it will use the old one in the ES. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/build_my_info_client.rb | 41 ++++++++++++++++++++++++++----------- 1 file changed, 29 insertions(+), 12 deletions(-) mode change 100644 => 100755 lib/build_my_info_client.rb diff --git a/lib/build_my_info_client.rb b/lib/build_my_info_client.rb old mode 100644 new mode 100755 index c946bec..9d2aaea --- a/lib/build_my_info_client.rb +++ b/lib/build_my_info_client.rb @@ -21,17 +21,16 @@ require_relative '../lib/es_client' # build_my_info = BuildMyInfo.new(email, name, lab) # build_my_info.config_my_info class BuildMyInfo - def initialize(my_email, my_name, lab, my_token = nil) + def initialize(my_email, my_name, lab, my_token) @lab = lab || 'nolab' - @my_token = my_token || %x(uuidgen).chomp @my_info = { 'my_email' => my_email, - 'my_name' => my_name, - 'my_token' => @my_token, - 'my_login_name' => nil, - 'my_commit_url' => nil, - 'my_ssh_pubkey' => [] + 'my_name' => my_name } + unless my_token.nil? + @my_info['my_token'] = my_token + end + @es = ESClient.new(index: 'accounts') end def config_default_yaml @@ -50,6 +49,26 @@ class BuildMyInfo end end + def search_my_info + @es.query_by_id(@my_info['my_email']) || {} + end + + def update_from_es(my_info_es) + my_info_es.update @my_info + @my_info.update my_info_es + end + + def complete_my_info + my_info_es = search_my_info + update_from_es(my_info_es) unless my_info_es.empty? + + @my_info['my_token'] = %x(uuidgen).chomp if @my_info['my_token'].nil? + end + + def store_account_info + @es.put_source_by_id(@my_info['my_email'], @my_info) + end + def config_lab_yaml lab_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/include/lab" FileUtils.mkdir_p lab_yaml_dir unless File.directory? lab_yaml_dir @@ -64,14 +83,12 @@ class BuildMyInfo end end - def store_account_info - es = ESClient.new(index: 'accounts') - es.put_source_by_id(@my_info['my_email'], @my_info) - end - def config_my_info + complete_my_info + config_default_yaml config_lab_yaml + store_account_info end end -- 2.23.0

1 0

[PATCH compass-ci 1/2] sbin/build-my-info: optimize options
by Luan Shengde 01 Feb '21

01 Feb '21

optimize options: add option -t: generate new my_token add option -s: search my_info for the email add option -h: output help message enable user search my_info for the email enable reset my_token for my_info Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 56 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 48 insertions(+), 8 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index fd75f51..1b99408 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -8,26 +8,66 @@ require_relative '../lib/build_my_info_client' require 'optparse' option = { - my_name: `git config --global user.name`.chomp, - my_email: `git config --global user.email`.chomp, - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp + 'my_email' => `git config --global user.email`.chomp, + 'my_name' => `git config --global user.name`.chomp, + 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } +es_search = false + options = OptionParser.new do |opts| + opts.banner = "Usage: build-my-info [-e email] [-n name] [-l lab] [-t]\n" + opts.banner += " build-my-info [-e email] -s\n" + + opts.separator '' + opts.separator 'options:' + opts.on('-e email', 'my_email') do |email| - option[:my_email] = email + option['my_email'] = email end opts.on('-n name', 'my_name') do |name| - option[:my_name] = name + option['my_name'] = name end opts.on('-l lab', 'lab') do |lab| - option[:lab] = lab + option['lab'] = lab + end + + opts.on('-t', 'my_token') do + option['my_token'] = %x(uuidgen).chomp + end + + opts.on('-s', 'search info for the email address') do + es_search = true + end + + opts.on_tail('-h', 'show this message') do + puts opts + exit end end +ARGV << '-h' if ARGV.empty? + options.parse! -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +build_my_info = BuildMyInfo.new(option['my_email'], option['my_name'], option['lab'], option['my_token']) + +if es_search + my_info = build_my_info.search_my_info + if my_info.empty? + puts "No info for #{option['my_email']} yet.\n" + else + formatter = "%-15s %-15s\n" + my_info.each do |k, v| + printf formatter, "#{k}:", v + end + end +else + begin + build_my_info.config_my_info + rescue StandardError => e + puts e.backtrace + end +end -- 2.23.0

1 0

help star
by Wu Fengguang 01 Feb '21

01 Feb '21

Yu Chuan is our help star, thanks Yu Chuan! 21.1 Wu Fengguang 17.1 Yu Chuan 11.3 Wei Jihui 11.1 Cao Xueliang 9.4 Liu Shaofei 8.3 Zhang Dewan 7.5 Luan Shengde 7.3 Liu Yinsi 6.9 Xu Xijian 6.6 Xiao Shenwei 5.4 Wu Zhende 5.1 Li Yuanchao 4.6 Lu Weitao 4.5 Lin Jiaxin 4.2 Bai Jing 4.0 Wang Yong 3.6 Du Kaitian 3.4 Li Ping 2.7 Ren Wen 2.1 Wang Chenglong 1.6 Cui Yili 1.1 Lu Kaiyi 0.9 Zhang Yale 0.0 Tong Qunfeng 0.0 Hu Xuejiao 0.0 Wang Shuai 0.0 Cui Xucui

1 0

[PATCH compass-ci] sparrow/install-server: add script to deploy compass-ci container services
by Liu Yinsi 01 Feb '21

01 Feb '21

[why] For user locally deploy compass-ci, some users' hosts can not access internet. [how] deploy all container services in the host machine, other functions: submit job and run qemu test, will deploy in other machine which can access internet by $LKP_SRC/setup/simply-ci. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 sparrow/install-server diff --git a/sparrow/install-server b/sparrow/install-server new file mode 100755 index 0000000..c532956 --- /dev/null +++ b/sparrow/install-server @@ -0,0 +1,20 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# For your reference only. +# It's better to run the below scripts step by step. + +[[ $CCI_SRC ]] || export CCI_SRC=$(dirname $(dirname $(realpath $0))) +cd $CCI_SRC/sparrow || exit + +0-package/common +1-storage/tiny +1-storage/permission +2-network/br0 +2-network/iptables +2-network/nfs +2-network/cifs +3-code/dev-env +. /etc/profile.d/compass.sh +4-docker/buildall run-only +7-systemd/systemd-setup -- 2.23.0

1 0