December 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 1/3] doc/job: add SCHED_HOST, SCHED_PORT description
by Hu Xuejiao 24 Dec '20

24 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- doc/job/SCHED_HOST-SCHED_PORT.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/job/SCHED_HOST-SCHED_PORT.md diff --git a/doc/job/SCHED_HOST-SCHED_PORT.md b/doc/job/SCHED_HOST-SCHED_PORT.md new file mode 100644 index 0000000..74c628a --- /dev/null +++ b/doc/job/SCHED_HOST-SCHED_PORT.md @@ -0,0 +1,15 @@ +# SCHED_HOST, SCHED_PORT +Meaning: +- The job task depends on scheduler service, which will distribute tasks to test + machine. All tasks will run in test machine. +- `Scheduler` is one of our service, `SCHED` is the abbreviation of `Scheduler` + these variables mean: + SCHED_HOST: the scheduler service ip. + SCHED_PORT: the scheduler service port. + +Usage example: + +```$LKP_SRC/include/lab/z9 +SCHED_HOST: api.compass-ci.openeuler.org +SCHED_PORT: 11306 +``` -- 2.23.0

1 0

[PATCH lkp-tests] lib/bootstrap.sh: fix redirect_stdout_stderr for docker
by Xiao Shenwei 24 Dec '20

24 Dec '20

[why] exec command will redirect stdout stderr to /tmp/stdout, /tmp/stderr but docker log based on /dev/stdout and /dev/stderr, it cause the task status cannot be viewed from the console, which is difficult to debug. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/bootstrap.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index ad1c4a2f2..c9a9f89f9 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -315,6 +315,13 @@ announce_bootup() redirect_stdout_stderr() { if is_docker; then + [ -e /tmp/stdout ] || touch /tmp/stdout + [ -e /tmp/stderr ] || touch /tmp/stderr + tail -f /tmp/stdout > /dev/stdout & + echo $! >> /tmp/pid-tail-global + tail -f /tmp/stderr > /dev/stderr & + echo $! >> /tmp/pid-tail-global + exec > /tmp/stdout exec 2> /tmp/stderr return -- 2.23.0

1 0

[PATCH v2 lkp-tests] adaptation/archlinux: add archlinux mapping
by Wang Yong 24 Dec '20

24 Dec '20

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/archlinux | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/distro/adaptation/archlinux b/distro/adaptation/archlinux index 89163cf41..c52212e08 100644 --- a/distro/adaptation/archlinux +++ b/distro/adaptation/archlinux @@ -72,3 +72,11 @@ libtirpc: libtirpc1 initramfs-tools: openssh-server: openssh openssh-client: openssh +autopoint: gettext +g++: gcc +libarchive13: libarchive +libbz2-1.0: bzip2 +libssl: openssl +libtool-bin: libtool +linux-perf: perf +ruby-dev: ruby -- 2.23.0

1 0

[PATCH compass-ci] lib/env.cr: rename to web_env.cr
by Wu Zhende 24 Dec '20

24 Dec '20

[Why] env.cr is the same as the file name provided by the crystal. [Error] /c/cci/scheduler $ crystal a.cr Showing last frame. Use --error-trace for full trace. In /usr/lib/crystal/core/json/from_json.cr:140:5 140 | def Set.new(pull : JSON::PullParser) Error: undefined constant Set Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 2 +- src/lib/{env.cr => web_env.cr} | 0 src/scheduler/scheduler.cr | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename src/lib/{env.cr => web_env.cr} (100%) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6caa5b9..2d9dba6 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -5,7 +5,7 @@ require "kemal" require "yaml" require "./job" -require "./env" +require "./web_env" require "./block_helper" require "./taskqueue_api" require "./remote_git_client" diff --git a/src/lib/env.cr b/src/lib/web_env.cr similarity index 100% rename from src/lib/env.cr rename to src/lib/web_env.cr diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 2103f7c..6b032ee 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -3,7 +3,7 @@ require "kemal" -require "../lib/env" +require "../lib/web_env" require "../lib/sched" require "../lib/json_logger" -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] sparrow/4-docker/buildall: build dist image when it doesn't exist
by Xu Xijian 24 Dec '20

24 Dec '20

If the dist docker image doesn't exist for some reason, run build script to create it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 32801d1..9e66ac9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -32,7 +32,8 @@ do_one() ( cd $container [ "$container_name" == 'ssh-r' ] && exit - if [ "$action" != "reboot" ]; then + docker images | grep -w -q "$container_name" + if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] code-spec.md: adjust markdown document format
by Xu Xijian 24 Dec '20

24 Dec '20

In markdown documents, text for bash should be contained in block as belows: ```SHELL content SHELL Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/code-spec.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index d44c6fe..a717f64 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -21,10 +21,12 @@ code注意事项 - puts/printf字符串应当容易搜索(grep)。特别注意包含足够长的固定内容substring. 例如： +```SHELL die "test yaml$${test_yaml}$ not exist in ${LKP_SRC}/jobs !!!" ==> die "cannot find test yaml ($test_yaml) in $LKP_SRC/jobs" ^^^^^^^^^^^^^^^^^^^^^ grep-friendly fixed substring +``` 重构友好编码 ============ -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_client: add send_mail_text function
by Luan Shengde 24 Dec '20

24 Dec '20

for user use mail_client to send mail with 'mail content' data type Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail_client.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/mail_client.rb b/lib/mail_client.rb index b13492f..f63e65d 100644 --- a/lib/mail_client.rb +++ b/lib/mail_client.rb @@ -24,4 +24,9 @@ class MailClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_encode") resource.post(Base64.encode64(mail_data)) end + + def send_mail_text(mail_text) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_text") + resource.post(mail_text) + end end -- 2.23.0

1 0

[PATCH lkp-tests] tests/cci-depends: update depends for cci-depends
by Wang Yong 24 Dec '20

24 Dec '20

[Why] cci-depends need base packages to generate cgz file, and mount operation is deprecated Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/cci-depends | 3 ++- tests/cci-depends | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/distro/depends/cci-depends b/distro/depends/cci-depends index 1f7f73ac3..0dc9d1501 100644 --- a/distro/depends/cci-depends +++ b/distro/depends/cci-depends @@ -1,2 +1,3 @@ -cifs-utils cpio +curl +gzip diff --git a/tests/cci-depends b/tests/cci-depends index 42d5ef676..8baf023f3 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -44,6 +44,8 @@ check_shared_package "$benchmark" && { share_debian_package +distro_install_depends cci-depends + fixup_distro_mirror update -- 2.23.0

1 0

[PATCH v3 compass-ci] doc: add runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 24 Dec '20

24 Dec '20

User can specify the borrow time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 9 +++++---- doc/manual/borrow-machine.zh.md | 9 +++++---- 2 files changed, 10 insertions(+), 8 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index bed9fcc..b14ea0a 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -203,6 +203,7 @@ Ensure that you have performed the following operations according to the [apply- suite: borrow testcase: borrow + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 3f500e2..80c15d1 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -207,6 +207,7 @@ suite: borrow testcase: borrow + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH v2 compass-ci] lib: add build_my_info_client
by Luan Shengde 24 Dec '20

24 Dec '20

purpose: for user build their infos when installing compassci - store my_info to ES - write my_info to default/lab yaml file if not, job will fail for verification problems for user private environment Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/build_my_info_client.rb | 79 +++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100755 lib/build_my_info_client.rb diff --git a/lib/build_my_info_client.rb b/lib/build_my_info_client.rb new file mode 100755 index 0000000..e4a4335 --- /dev/null +++ b/lib/build_my_info_client.rb @@ -0,0 +1,79 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'fileutils' +require 'io/console' +require_relative '../lib/es_client' + +# purpose: +# build my_info +# store my_info to ES +# write my_info to default/lab yaml file +# for user install compass-ci at their local server +# user meet verification problems when submit jobs +# add calling in installing script to meet the verification function +# usage: +# require_relative 'build_my_info_client' +# +# build_my_info = BuildMyInfo.new(email, name, lab) +# build_my_info.config_my_info +class BuildMyInfo + def initialize(my_email, my_name, lab) + @lab = lab || 'z9' + @my_info = { + 'my_email' => my_email, + 'my_name' => my_name, + 'my_token' => %x(uuidgen).chomp + } + end + + def config_default_yaml + default_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/defaults" + FileUtils.mkdir_p default_yaml_dir unless File.directory? default_yaml_dir + default_yaml_file = "#{default_yaml_dir}/account.yaml" + FileUtils.touch(default_yaml_file) unless File.exist? default_yaml_file + + default_yaml_info = YAML.load_file(default_yaml_file) || {} + default_yaml_info['my_email'] = @my_info['my_email'] + default_yaml_info['my_name'] = @my_info['my_name'] + + File.open(default_yaml_file, 'w') do |f| + f.puts default_yaml_info.to_yaml + end + end + + def config_lab_yaml + lab_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/include/lab" + FileUtils.mkdir_p lab_yaml_dir unless File.directory? lab_yaml_dir + lab_yaml_file = "#{lab_yaml_dir}/#{(a)lab}.yaml" + FileUtils.touch(lab_yaml_file) unless File.exist? lab_yaml_file + + lab_yaml_info = YAML.load_file(lab_yaml_file) || {} + lab_yaml_info['my_token'] = @my_info['my_token'] + + File.open(lab_yaml_file, 'w') do |f| + f.puts lab_yaml_info.to_yaml + end + end + + def complete_my_info + @my_info['my_login_name'] = '' + @my_info['my_commit_url'] = '' + @my_info['my_ssh_pubkey'] = [] + end + + def store_account_info + es = ESClient.new(index: 'accounts') + es.put_source_by_id(@my_info['my_email'], @my_info) + end + + def config_my_info + config_default_yaml + config_lab_yaml + complete_my_info + store_account_info + end +end -- 2.23.0

1 0