- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 1/2] doc: add job/os_mount.md
by Yu Chuan 03 Feb '21

03 Feb '21

Add description of os_mount field in job.yaml. This patch is the description when os_mount=local, and other values of os_mount will be added in the patch later. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 62 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 doc/job/os_mount.md diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md new file mode 100644 index 000000000000..141335c3ee4b --- /dev/null +++ b/doc/job/os_mount.md @@ -0,0 +1,62 @@ +# Summary +========= + +`os_mount` defines the mount method of tbox's root partition. + +It has the following optional values: + - nfs + - cifs + - initramfs + - container + - local + +Usage example: + - submit iperf.yaml testbox=vm-2p8g os_mount=nfs + - submit iperf.yaml testbox=vm-2p8g os_mount=cifs + - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs + - submit iperf.yaml testbox=dc-8g os_mount=container + - submit iperf.yaml testbox=vm-2p8g os_mount=local + +# Optional Values +================= + +## Work flow when os_mount=local + +1. user submit job with os_mount: local + optional kernel cmdline params: # can be added in kernel_append_root field of job.yaml + - keep_last_root: keep the original data of disk what last job generated, don't wipe it + +2. scheduler return the ipxe_str to tbox + ``` + dhcp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/initrd.lkp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/boot/vmlinuz + imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + boot + ``` + +3. boot + - check if `/dev/mapper/os-openeuler_aarch64_20.03_{timestamp}` exist # according the kernel cmdline + - if exist: + - if `keep_last_root` not in kernel cmdline params: + - re-create(lvremove and lvcreate) rw logical volume /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if `keep_last_root` in kernel cmdline params: + - if /dev/mapper/os-openeuler_aarch64_20.03 exists: + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if /dev/mapper/os-openeuler_aarch64_20.03 not exists: + - generate /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03-{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if not exist: + ``` + mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt + create /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} according rootfs_disk. # logical volume size: 10G + mkdir /mnt1, mount /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} /mnt1 + rsync -az /mnt/. /mnt1/ + umount /mnt /mnt1 + lvchange -p r /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + creat rw logical volume /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + boot from /dev/mapper/os-openeuler_aarch64_20.03 + ``` + +4. execute the job -- 2.23.0

2 3

[PATCH compass-ci] register-accounts: sync register-accounts
by Luan Shengde 03 Feb '21

03 Feb '21

[why] the required file lib/build_my_info_client has been updated. and the invoking way has some change. just do update according to the change. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index ff69032..519f289 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -9,7 +9,12 @@ require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" content = YAML.load_file('/etc/compass-ci/register/register.yaml') content.each do |key, value| - build_my_info = BuildMyInfo.new(value['my_email'], value['my_name'], ENV['lab']) - build_my_info.store_account_info + my_info = {} + my_info['my_email'] = value['my_email'] + my_info['my_name'] = value['my_name'] + my_info['lab'] = ENV['lab'] + + build_my_info = BuildMyInfo.new(my_info['my_email']) + build_my_info.store_account_info(my_info) end sleep -- 2.23.0

2 1

[PATCH v2 compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty before git clone. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..2c982f7 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +rm -rf openresty + git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

1 0

[PATCH compass-ci] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

add work dir and permission to use libvirt to boot testbox. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

2 2

Re: [PATCH compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

>> git clone https://gitee.com/cuiyili/openresty.git || exit 1 >> docker build -t openresty:proxy_cache . -f openresty/Dockerfile >>+rm -rf openresty > >How about move the 'rm' step at the before of 'docker build'? docker build need openrestry/Dockerfile, this file come from git clone https://gitee.com/cuiyili/openresty.git, you mean move before of git clone? it looks not make any difference. Thanks, Yinsi > >-------- >Thanks >Yu Chuan > >>-- >>2.23.0 >>

2 2

[PATCH v5 compass-ci 3/3] mail-robot: delete useless file
by Luan Shengde 03 Feb '21

03 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/gitee-commit-url-check.rb | 45 ------------------- 1 file changed, 45 deletions(-) delete mode 100755 container/mail-robot/lib/gitee-commit-url-check.rb diff --git a/container/mail-robot/lib/gitee-commit-url-check.rb b/container/mail-robot/lib/gitee-commit-url-check.rb deleted file mode 100755 index af31e17..0000000 --- a/container/mail-robot/lib/gitee-commit-url-check.rb +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env ruby -# SPDX-License-Identifier: MulanPSL-2.0+ -# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -# frozen_string_literal: true - -require 'json' -require 'mail' - -# used to check commit available for hub gitee.com -# it is called when: -# - ParseApplyAccountEmail execute commit_url_availability -# - hub is gitee.com -# gitee_commit_check -# clone the repo -# check commit available -class GiteeCommitUrlCheck - def initialize(my_info, url, base_url) - @my_info = my_info - @url = url - @base_url = base_url - end - - def gitee_commit_check - repo_url = [@base_url, 'git'].join('.') - repo_dir = repo_url.split('/')[-1] - commit_id = @url.split('/')[-1] - - Dir.chdir '/tmp' - %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) - - email_index = %x(/usr/bin/git -C #{repo_dir} show #{commit_id}).index @my_info['my_email'] - - FileUtils.rm_rf repo_dir - - gitee_commit_exist(email_index) - end - - def gitee_commit_exist(email_index) - return if email_index - - error_message = 'COMMIT_URL_NOT_AVAILABLE' - - raise error_message - end -end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] mail-robot: optimize assign-account-fail-email
by Luan Shengde 03 Feb '21

03 Feb '21

add message for newly added error types. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 242dfe1..3597159 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -33,17 +33,20 @@ def email_err_message(message) https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… EMAIL_MESSAGE - when 'COMMIT_URL_NOT_AVAILABLE' + when 'COMMIT_AUTHOR_ERROR' err_message = <<~EMAIL_MESSAGE - We cannot confirm the commit url matches your email. + We cannot confirm the commit author matches your email. + Make sure it is truely submitted with your email. - Make sure the commit exists. - Make sure it is truely submitted with your email. - - Go to the commit history and find your commit. - Enter the commit and you will get your commit url. + Choose a commit that was submitted with your own email and try again. + EMAIL_MESSAGE + when 'NO_COMMIT_ID' + err_message = <<~EMAIL_MESSAGE + There is no such commit ID in the repo. - Try again after you have correctted the url + Please check the commit ID exists. + or + You have write the commit ID correctly. EMAIL_MESSAGE when 'NO_PUBKEY' err_message = <<~EMAIL_MESSAGE -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] mail-robot: fix bug for parsing email
by Luan Shengde 03 Feb '21

03 Feb '21

[why] for non-gitee repos: we may sometimes can not get the author email from the http request feedback when checking the commit. users may still receive a fail email even if they send 'apply account' email correctly. [how] both gitee and non-gitee will clone the repo tree and check the commit: - check commit id exists - check commit author matches the email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 41 ++++++++++--------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index a41a89a..c7fe9ea 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -6,7 +6,6 @@ require 'json' require 'mail' require 'rest-client' -require_relative 'gitee-commit-url-check' # used to parse mail_content for my_commit_url and my_ssh_pubkey # be called by AssignAccount when it needs to extract required data: @@ -91,29 +90,33 @@ class ParseApplyAccountEmail end def commit_url_availability(url, base_url) - hub_name = url.split('/')[2] - - # it requires authentication when sending request to get the commit information - # clone the repo and then validate the commit for the email address - if hub_name.eql? 'gitee.com' - gitee_commit(url, base_url) - else - non_gitee_commit(url) - end - end + repo_url = [base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] + commit_id = url.split('/')[-1] + + %x(/usr/bin/git -C /tmp clone --bare #{repo_url} #{repo_dir}) + + # get all commit IDs and check commit id exists + repo_commits = %x(/usr/bin/git -C /tmp/#{repo_dir} log --pretty=format:'%H').split(/\n/) + check_commit_exist(commit_id, repo_commits) - def gitee_commit(url, base_url) - my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) - my_gitee_commit.gitee_commit_check + # get the auther's email for the commit + author_email = %x(/usr/bin/git -C /tmp/#{repo_dir} show -s --format=%aE #{commit_id}).chomp + check_commit_email(author_email) + + FileUtils.rm_rf "/tmp/#{repo_dir}" end - def non_gitee_commit(url) - url_fdback = RestClient.get(url).body - email_index = url_fdback.index @my_info['my_email'] + def check_commit_exist(commit_id, repo_commits) + return if repo_commits.include? commit_id.chomp + + raise 'NO_COMMIT_ID' + end - return unless email_index.nil? + def check_commit_email(author_email) + return if author_email.eql? @my_info['my_email'].chomp - raise 'COMMIT_URL_NOT_AVAILABLE' + raise 'COMMIT_AUTHOR_ERROR' end def parse_pub_key -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker: delete duplicate functions code
by Liu Yinsi 03 Feb '21

03 Feb '21

git show d265997c: buildall: avoid repeated building containers in sparrow git show 67ad8b6e: sparrow/4-docker/buildall: prevent duplicate build [why] delete one of the preceding patches because the two patches have the same functions. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 -- 1 file changed, 2 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 727abb9..c838f86 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -25,7 +25,6 @@ do_one() local container=$1 local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock - echo "$started_containers" | grep -w -q "$container_name" && return 0 log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && @@ -42,7 +41,6 @@ do_one() [ "$container_name" == 'initrd-lkp' ] && ./run ) log_info "finish build $container." - started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH compass-ci] sparrow/4-docker: move confusing log_info()
by Liu Yinsi 03 Feb '21

03 Feb '21

[how] before: whether build container or not, all will call log_info(): "start/finish build $container" after: only when actually build container, will call log_info(): "start/finish build $container" [why] if not build container, but echo "start build $container", it's very confusing, mistakenly think that repeatedly build container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index c838f86..c676e6f 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -26,21 +26,24 @@ do_one() local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock - log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" [ "$container_name" == 'ssh-r' ] && exit docker images | grep -wq "$container_name" if [ "$?" != 0 ] || [ "$action" != "run-only" ]; then + + log_info "start build $container." + [ -x build ] && ./build [ -x install ] && ./install + + log_info "finish build $container." fi [ -x first-run ] && ./first-run [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run ) - log_info "finish build $container." lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH lkp-tests] iso2rootfs: add new config_rootfs field: enable_repos
by Yu Chuan 03 Feb '21

03 Feb '21

[Why] Not all sources of one repo file are enabled, sometimes we need to enable all sources to install the specified pkg. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 1 + tests/iso2rootfs | 12 ++++++++++++ 2 files changed, 13 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 2a6455245584..55c4345fad94 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -32,6 +32,7 @@ iso2rootfs: # - dns: will config /etc/resolv.conf. # - no_selinux: will disable selinux in /etc/selinux/config. # - no_fstab: will comment all line in /etc/fstab. + # - enable_repos: will enable all repo file of result rootfs. config_rootfs: ## install pkgs for result rootfs diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 65a1935b7f49..06c8565110ab 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -393,11 +393,23 @@ install_pkgs_for_rootfs() fi } +enable_repos() +{ + echo "${config_rootfs}" | grep -qw 'enable_repos' || return 0 + + ls ${ROOTFS_DES_DIR}/etc/yum.repos.d/*.repo > /dev/null 2>&1 && { + sed -i 's/^enabled=0$/enabled=1/g' ${ROOTFS_DES_DIR}/etc/yum.repos.d/*.repo + } + + return 0 +} + config_rootfs() { config_dns_resolver disable_selinux disable_fstab + enable_repos install_pkgs_for_rootfs } -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] sbin/build-my-info: optimize
by Luan Shengde 03 Feb '21

03 Feb '21

optimize options: add option -t: generate new my_token add option -h: output help message add search my_info from ES and update my_info before store it. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 38 ++++++++++++++++++++++++++++++-------- 1 file changed, 30 insertions(+), 8 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index fd75f51..dbe093d 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -8,26 +8,48 @@ require_relative '../lib/build_my_info_client' require 'optparse' option = { - my_name: `git config --global user.name`.chomp, - my_email: `git config --global user.email`.chomp, - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp + 'my_email' => `git config --global user.email`.chomp, + 'my_name' => `git config --global user.name`.chomp, + 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } options = OptionParser.new do |opts| + opts.banner = "Usage: build-my-info [-e email] [-n name] [-l lab] [-t]\n" + + opts.separator '' + opts.separator 'options:' + opts.on('-e email', 'my_email') do |email| - option[:my_email] = email + option['my_email'] = email end opts.on('-n name', 'my_name') do |name| - option[:my_name] = name + option['my_name'] = name end opts.on('-l lab', 'lab') do |lab| - option[:lab] = lab + option['lab'] = lab + end + + opts.on('-t', 'my_token') do + option['my_token'] = %x(uuidgen).chomp + end + + opts.on_tail('-h', 'show this message') do + puts opts + exit end end options.parse! -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +def build_my_info(option) + build_my_info = BuildMyInfo.new(option['my_email']) + info_es = build_my_info.search_my_info + info_es.update option + info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? + + build_my_info.config_my_info(info_es) +end + +build_my_info(option) -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/build_my_info_client: optimize
by Luan Shengde 03 Feb '21

03 Feb '21

cancel modify my_info enable user invoke each method alone. add search method: my_info Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/build_my_info_client.rb | 53 +++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 29 deletions(-) mode change 100644 => 100755 lib/build_my_info_client.rb diff --git a/lib/build_my_info_client.rb b/lib/build_my_info_client.rb old mode 100644 new mode 100755 index c946bec..6d35e97 --- a/lib/build_my_info_client.rb +++ b/lib/build_my_info_client.rb @@ -18,60 +18,55 @@ require_relative '../lib/es_client' # usage: # require_relative 'build_my_info_client' # -# build_my_info = BuildMyInfo.new(email, name, lab) -# build_my_info.config_my_info +# build_my_info = BuildMyInfo.new(my_info['email']) +# build_my_info.config_my_info(my_info) class BuildMyInfo - def initialize(my_email, my_name, lab, my_token = nil) - @lab = lab || 'nolab' - @my_token = my_token || %x(uuidgen).chomp - @my_info = { - 'my_email' => my_email, - 'my_name' => my_name, - 'my_token' => @my_token, - 'my_login_name' => nil, - 'my_commit_url' => nil, - 'my_ssh_pubkey' => [] - } + def initialize(my_email) + @my_email = my_email + @es = ESClient.new(index: 'accounts') end - def config_default_yaml + def search_my_info + @es.query_by_id(@my_email) || {} + end + + def store_account_info(my_info) + @es.put_source_by_id(@my_email, my_info) + end + + def config_default_yaml(my_info) default_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/defaults" FileUtils.mkdir_p default_yaml_dir unless File.directory? default_yaml_dir default_yaml_file = "#{default_yaml_dir}/account.yaml" FileUtils.touch(default_yaml_file) unless File.exist? default_yaml_file default_yaml_info = YAML.load_file(default_yaml_file) || {} - default_yaml_info['my_email'] = @my_info['my_email'] - default_yaml_info['my_name'] = @my_info['my_name'] - default_yaml_info['lab'] = @lab + default_yaml_info['my_email'] = my_info['my_email'] + default_yaml_info['my_name'] = my_info['my_name'] + default_yaml_info['lab'] = my_info['lab'] File.open(default_yaml_file, 'w') do |f| f.puts default_yaml_info.to_yaml end end - def config_lab_yaml + def config_lab_yaml(my_info) lab_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/include/lab" FileUtils.mkdir_p lab_yaml_dir unless File.directory? lab_yaml_dir - lab_yaml_file = "#{lab_yaml_dir}/#{(a)lab}.yaml" + lab_yaml_file = "#{lab_yaml_dir}/#{my_info['lab']}.yaml" FileUtils.touch(lab_yaml_file) unless File.exist? lab_yaml_file lab_yaml_info = YAML.load_file(lab_yaml_file) || {} - lab_yaml_info['my_token'] = @my_info['my_token'] + lab_yaml_info['my_token'] = my_info['my_token'] File.open(lab_yaml_file, 'w') do |f| f.puts lab_yaml_info.to_yaml end end - def store_account_info - es = ESClient.new(index: 'accounts') - es.put_source_by_id(@my_info['my_email'], @my_info) - end - - def config_my_info - config_default_yaml - config_lab_yaml - store_account_info + def config_my_info(my_info) + store_account_info(my_info) + config_default_yaml(my_info) + config_lab_yaml(my_info) end end -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/monitor-job: fix rm gem sources
by Wei Jihui 03 Feb '21

03 Feb '21

gem sources -c can not remove sources, use -r Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/monitor-job | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/monitor-job b/tests/monitor-job index 5fd2d73c..0f495ae2 100755 --- a/tests/monitor-job +++ b/tests/monitor-job @@ -18,7 +18,7 @@ set_scheduler_config() install_gem_pkgs() { - gem sources -c -a https://gems.ruby-china.com/ + gem sources -r http://rubygems.org -a https://gems.ruby-china.com/ gem install activesupport git rest-client elasticsearch } -- 2.23.0

1 0

[PATCH lkp-tests] tests/monitor-job: fix rm gem sources
by Wei Jihui 03 Feb '21

03 Feb '21

gem sources -c can not remove sources, use -r Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/monitor-job | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/monitor-job b/tests/monitor-job index 5fd2d73c..4a5d8562 100755 --- a/tests/monitor-job +++ b/tests/monitor-job @@ -18,7 +18,8 @@ set_scheduler_config() install_gem_pkgs() { - gem sources -c -a https://gems.ruby-china.com/ + gem sources -r http://rubygems.org + gem sources -a https://gems.ruby-china.com/ gem install activesupport git rest-client elasticsearch } -- 2.23.0

2 1

[PATCH compass-ci] delimiter: bisect run script catch the exceptions and exit -1
by Cao Xueliang 02 Feb '21

02 Feb '21

Git bisect tag the commit good/bad/skip according to the bisect run script exit code. Exit code 0 is good, 125 is skip, the others between 1 and 127 is bad. When the ruby script raise a exception exit code is 1, git bisect will tag the commit to bad, but we should abort the bisect process. So we catch the exceptions and exit -1 in bisect run script. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/find-commit/bisect_run_script.rb | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/src/delimiter/find-commit/bisect_run_script.rb b/src/delimiter/find-commit/bisect_run_script.rb index 4877d79..8cdf675 100755 --- a/src/delimiter/find-commit/bisect_run_script.rb +++ b/src/delimiter/find-commit/bisect_run_script.rb @@ -36,9 +36,13 @@ class GitBisectRun end end -job_id = ARGV[0] -error_id = ARGV[1] -work_dir = ARGV[2] - -run = GitBisectRun.new job_id, error_id, work_dir -run.git_bisect +begin + job_id = ARGV[0] + error_id = ARGV[1] + work_dir = ARGV[2] + + run = GitBisectRun.new job_id, error_id, work_dir + run.git_bisect +rescue + exit -1 +end -- 2.23.0

1 0

[PATCH v2 lkp-tests] stats: add stats script for dockerfile os migrating
by Bai Jing 02 Feb '21

02 Feb '21

[input] ^[[91mError: Unable to find a match: supervisor ^[[91m/bin/sh: tar: command not found [output] sh.command-not-found.tar: 1 yum.error.Unable-to-find-a-match.supervisor: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/stats/openeuler_docker b/stats/openeuler_docker index e31934573..95e2a272b 100755 --- a/stats/openeuler_docker +++ b/stats/openeuler_docker @@ -2,7 +2,12 @@ while (line = STDIN.gets) line.chomp! - next unless line =~ /build.fail/ - - puts line + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] doc: add job/rootfs_disk.md
by Yu Chuan 02 Feb '21

02 Feb '21

Add description of rootfs_disk field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/rootfs_disk.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 doc/job/rootfs_disk.md diff --git a/doc/job/rootfs_disk.md b/doc/job/rootfs_disk.md new file mode 100644 index 000000000000..72762cf72f2f --- /dev/null +++ b/doc/job/rootfs_disk.md @@ -0,0 +1,27 @@ +# Summary +========= + +`rootfs_disk` defines a disk which storing multiple rootfs, used for jobs which need boot from disk. + +Each of these rootfs corresponds to an logical volume, such as follow: + /dev/mapper/ + . + ├── os-openeuler_aarch64_20.03 + ├── os-openeuler_aarch64_20.03_{timestamp} + ├── os-openeuler_aarch64_20.03_sp1 + ├── os-openeuler_aarch64_20.03_sp1_{timestamp} + └── ... + +Addition: +1. `/dev/mapper/os-{os}_{os_arch}_{os_version}_{timestamp}` + - is a readonly logical volume + - used for backup and rollback + +2. `/dev/mapper/os-{os}_{os_arch}_{os_version}` + - is a read write logical volume + - use for boot and run job + - defaultly, this logical volume will be wiped before every job run + +3. every logical volume size: 10G + +4. more info can refer os_mount.md -- 2.23.0

1 0

[PATCH v4 compass-ci 1/3] mail-robot: fix bug for parsing email
by Luan Shengde 02 Feb '21

02 Feb '21

[why] for non-gitee repos: we may sometimes can not get the author email from the http request feedback when checking the commit. users may still receive a fail email even if they send 'apply account' email correctly. [how] both gitee and non-gitee will clone the repo tree and check the commit: - check commit id exists - check commit author matches the email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 41 ++++++++++--------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index a41a89a..c389a09 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -6,7 +6,6 @@ require 'json' require 'mail' require 'rest-client' -require_relative 'gitee-commit-url-check' # used to parse mail_content for my_commit_url and my_ssh_pubkey # be called by AssignAccount when it needs to extract required data: @@ -91,29 +90,33 @@ class ParseApplyAccountEmail end def commit_url_availability(url, base_url) - hub_name = url.split('/')[2] - - # it requires authentication when sending request to get the commit information - # clone the repo and then validate the commit for the email address - if hub_name.eql? 'gitee.com' - gitee_commit(url, base_url) - else - non_gitee_commit(url) - end - end + repo_url = [base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] + commit_id = url.split('/')[-1] + + %x(/usr/bin/git -C /tmp clone --bare #{repo_url} #{repo_dir}) + + # get all commit IDs and check commit id exists + repo_commits = %x(git -C /tmp/#{repo_dir} log --pretty=format:'%H').split(/\n/) + check_commit_exist(commit_id, repo_commits) - def gitee_commit(url, base_url) - my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) - my_gitee_commit.gitee_commit_check + # get the auther's email for the commit + author_email = %x(/usr/bin/git -C /tmp/#{repo_dir} show -s --format=%aE #{commit_id}).chomp + check_commit_email(author_email) + + FileUtils.rm_rf "/tmp/#{repo_dir}" end - def non_gitee_commit(url) - url_fdback = RestClient.get(url).body - email_index = url_fdback.index @my_info['my_email'] + def check_commit_exist(commit_id, repo_commits) + return if repo_commits.include? commit_id.chomp + + raise 'NO_COMMIT_ID' + end - return unless email_index.nil? + def check_commit_email(author_email) + return if author_email.eql? @my_info['my_email'].chomp - raise 'COMMIT_URL_NOT_AVAILABLE' + raise 'COMMIT_AUTHOR_ERROR' end def parse_pub_key -- 2.23.0

2 1

[PATCH v4 compass-ci 3/3] mail-robot: delete useless file
by Luan Shengde 02 Feb '21

02 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/gitee-commit-url-check.rb | 45 ------------------- 1 file changed, 45 deletions(-) delete mode 100755 container/mail-robot/lib/gitee-commit-url-check.rb diff --git a/container/mail-robot/lib/gitee-commit-url-check.rb b/container/mail-robot/lib/gitee-commit-url-check.rb deleted file mode 100755 index af31e17..0000000 --- a/container/mail-robot/lib/gitee-commit-url-check.rb +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env ruby -# SPDX-License-Identifier: MulanPSL-2.0+ -# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -# frozen_string_literal: true - -require 'json' -require 'mail' - -# used to check commit available for hub gitee.com -# it is called when: -# - ParseApplyAccountEmail execute commit_url_availability -# - hub is gitee.com -# gitee_commit_check -# clone the repo -# check commit available -class GiteeCommitUrlCheck - def initialize(my_info, url, base_url) - @my_info = my_info - @url = url - @base_url = base_url - end - - def gitee_commit_check - repo_url = [@base_url, 'git'].join('.') - repo_dir = repo_url.split('/')[-1] - commit_id = @url.split('/')[-1] - - Dir.chdir '/tmp' - %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) - - email_index = %x(/usr/bin/git -C #{repo_dir} show #{commit_id}).index @my_info['my_email'] - - FileUtils.rm_rf repo_dir - - gitee_commit_exist(email_index) - end - - def gitee_commit_exist(email_index) - return if email_index - - error_message = 'COMMIT_URL_NOT_AVAILABLE' - - raise error_message - end -end -- 2.23.0

1 0

[PATCH v4 compass-ci 2/3] mail-robot: optimize assign-account-fail-email
by Luan Shengde 02 Feb '21

02 Feb '21

add message for newly added error types. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 242dfe1..3597159 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -33,17 +33,20 @@ def email_err_message(message) https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… EMAIL_MESSAGE - when 'COMMIT_URL_NOT_AVAILABLE' + when 'COMMIT_AUTHOR_ERROR' err_message = <<~EMAIL_MESSAGE - We cannot confirm the commit url matches your email. + We cannot confirm the commit author matches your email. + Make sure it is truely submitted with your email. - Make sure the commit exists. - Make sure it is truely submitted with your email. - - Go to the commit history and find your commit. - Enter the commit and you will get your commit url. + Choose a commit that was submitted with your own email and try again. + EMAIL_MESSAGE + when 'NO_COMMIT_ID' + err_message = <<~EMAIL_MESSAGE + There is no such commit ID in the repo. - Try again after you have correctted the url + Please check the commit ID exists. + or + You have write the commit ID correctly. EMAIL_MESSAGE when 'NO_PUBKEY' err_message = <<~EMAIL_MESSAGE -- 2.23.0

1 0

[PATCH compass-ci 2/2] sbin/build-my-info: optimize
by Luan Shengde 02 Feb '21

02 Feb '21

optimize options: add option -t: generate new my_token add option -h: output help message add search my_info from ES update my_info before config my_info Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 39 +++++++++++++++++++++++++++++++-------- 1 file changed, 31 insertions(+), 8 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index fd75f51..078e390 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -8,26 +8,49 @@ require_relative '../lib/build_my_info_client' require 'optparse' option = { - my_name: `git config --global user.name`.chomp, - my_email: `git config --global user.email`.chomp, - lab: `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp + 'my_email' => `git config --global user.email`.chomp, + 'my_name' => `git config --global user.name`.chomp, + 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } options = OptionParser.new do |opts| + opts.banner = "Usage: build-my-info [-e email] [-n name] [-l lab] [-t]\n" + opts.banner += " build-my-info [-e email] -s\n" + + opts.separator '' + opts.separator 'options:' + opts.on('-e email', 'my_email') do |email| - option[:my_email] = email + option['my_email'] = email end opts.on('-n name', 'my_name') do |name| - option[:my_name] = name + option['my_name'] = name end opts.on('-l lab', 'lab') do |lab| - option[:lab] = lab + option['lab'] = lab + end + + opts.on('-t', 'my_token') do + option['my_token'] = %x(uuidgen).chomp + end + + opts.on_tail('-h', 'show this message') do + puts opts + exit end end options.parse! -build_my_info = BuildMyInfo.new(option[:my_email], option[:my_name], option[:lab]) -build_my_info.config_my_info +def build_my_info(option) + build_my_info = BuildMyInfo.new(option['my_email']) + info_es = build_my_info.search_my_info + info_es.update option + info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? + + build_my_info.config_my_info(info_es) +end + +build_my_info(option) -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/build_my_info_client: optimize
by Luan Shengde 02 Feb '21

02 Feb '21

cancel modify my_info enable user invoke each method alone. add search method: my_info Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/build_my_info_client.rb | 53 +++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 29 deletions(-) mode change 100644 => 100755 lib/build_my_info_client.rb diff --git a/lib/build_my_info_client.rb b/lib/build_my_info_client.rb old mode 100644 new mode 100755 index c946bec..6d35e97 --- a/lib/build_my_info_client.rb +++ b/lib/build_my_info_client.rb @@ -18,60 +18,55 @@ require_relative '../lib/es_client' # usage: # require_relative 'build_my_info_client' # -# build_my_info = BuildMyInfo.new(email, name, lab) -# build_my_info.config_my_info +# build_my_info = BuildMyInfo.new(my_info['email']) +# build_my_info.config_my_info(my_info) class BuildMyInfo - def initialize(my_email, my_name, lab, my_token = nil) - @lab = lab || 'nolab' - @my_token = my_token || %x(uuidgen).chomp - @my_info = { - 'my_email' => my_email, - 'my_name' => my_name, - 'my_token' => @my_token, - 'my_login_name' => nil, - 'my_commit_url' => nil, - 'my_ssh_pubkey' => [] - } + def initialize(my_email) + @my_email = my_email + @es = ESClient.new(index: 'accounts') end - def config_default_yaml + def search_my_info + @es.query_by_id(@my_email) || {} + end + + def store_account_info(my_info) + @es.put_source_by_id(@my_email, my_info) + end + + def config_default_yaml(my_info) default_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/defaults" FileUtils.mkdir_p default_yaml_dir unless File.directory? default_yaml_dir default_yaml_file = "#{default_yaml_dir}/account.yaml" FileUtils.touch(default_yaml_file) unless File.exist? default_yaml_file default_yaml_info = YAML.load_file(default_yaml_file) || {} - default_yaml_info['my_email'] = @my_info['my_email'] - default_yaml_info['my_name'] = @my_info['my_name'] - default_yaml_info['lab'] = @lab + default_yaml_info['my_email'] = my_info['my_email'] + default_yaml_info['my_name'] = my_info['my_name'] + default_yaml_info['lab'] = my_info['lab'] File.open(default_yaml_file, 'w') do |f| f.puts default_yaml_info.to_yaml end end - def config_lab_yaml + def config_lab_yaml(my_info) lab_yaml_dir = "#{ENV['HOME']}/.config/compass-ci/include/lab" FileUtils.mkdir_p lab_yaml_dir unless File.directory? lab_yaml_dir - lab_yaml_file = "#{lab_yaml_dir}/#{(a)lab}.yaml" + lab_yaml_file = "#{lab_yaml_dir}/#{my_info['lab']}.yaml" FileUtils.touch(lab_yaml_file) unless File.exist? lab_yaml_file lab_yaml_info = YAML.load_file(lab_yaml_file) || {} - lab_yaml_info['my_token'] = @my_info['my_token'] + lab_yaml_info['my_token'] = my_info['my_token'] File.open(lab_yaml_file, 'w') do |f| f.puts lab_yaml_info.to_yaml end end - def store_account_info - es = ESClient.new(index: 'accounts') - es.put_source_by_id(@my_info['my_email'], @my_info) - end - - def config_my_info - config_default_yaml - config_lab_yaml - store_account_info + def config_my_info(my_info) + store_account_info(my_info) + config_default_yaml(my_info) + config_lab_yaml(my_info) end end -- 2.23.0

1 0

[PATCH lkp-tests] stats: add stats script for dockerfile os migrating
by Bai Jing 02 Feb '21

02 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/stats/openeuler_docker b/stats/openeuler_docker index e31934573..95e2a272b 100755 --- a/stats/openeuler_docker +++ b/stats/openeuler_docker @@ -2,7 +2,12 @@ while (line = STDIN.gets) line.chomp! - next unless line =~ /build.fail/ - - puts line + case line + when /Error: Unable to find a match: (.+)/ + $1.split.each do |repo| + puts "yum.error.Unable-to-find-a-match.#{repo}: 1" + end + when /\/bin\/sh: (.+): command not found/ + puts "sh.command-not-found.#{$1}: 1" + end end -- 2.23.0

1 1

[PATCH compass-ci] doc/learning-resources.md: add some commonly used tmux commands
by Lin Jiaxin 02 Feb '21

02 Feb '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- doc/learning-resources.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/learning-resources.md b/doc/learning-resources.md index e631f4a..3a164b5 100644 --- a/doc/learning-resources.md +++ b/doc/learning-resources.md @@ -119,10 +119,14 @@ Tmux ## tmux keys (customized) + ctrl-s ? list-keys ctrl-s c create new window + ctrl-s x kill-pane alt-j/k switch to left/right window alt-1/2/3... switch to the 1st, 2nd, 3rd, ... window ctrl-s ctrl-u page up (press ctrl-u to continue paging up, <Enter> to exit) + ctrl-s [/] copy-mode/paste-buffer + shift-(left mouse button/right mouse button) copy/paste to/from system clipboard Mutt ==== -- 2.23.0

1 0

[PATCH lkp-tests] etc/makepkg.conf: disable strip symbols
by Wei Jihui 02 Feb '21

02 Feb '21

[why] Strip symbols missing when exec cci-makepkg, file size is changed. [How] according to https://man.archlinux.org/man/makepkg.conf.5.en: Strip symbols from binaries and libraries. If you frequently use a debugger on programs or libraries, it may be helpful to disable this option. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- etc/makepkg.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/etc/makepkg.conf b/etc/makepkg.conf index 5d73399d..1ff5d558 100644 --- a/etc/makepkg.conf +++ b/etc/makepkg.conf @@ -83,7 +83,7 @@ BUILDENV=(!distcc color !ccache check !sign) #-- purge: Remove files specified by PURGE_TARGETS #-- debug: Add debugging flags as specified in DEBUG_* variables # -OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug) +OPTIONS=(!strip docs !libtool !staticlibs emptydirs zipman purge !debug) #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512 INTEGRITY_CHECK=(md5) -- 2.23.0

2 1

[PATCH compass-ci] git_mirror.rb: merge default info when load upstream-repos
by Li Yuanchao 02 Feb '21

02 Feb '21

Load DEFAULTS files in upstream-repos. Merge the most close DEFAULTS file with git_info[git_repo]. The DEFAULTS files would keep common things like submit commands which used by many repos. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index d089b1c..fa2e0cc 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,8 +130,8 @@ class MirrorMain defaults_file = "#{repodir}/DEFAULTS" return unless File.exist?(defaults_file) - project = repodir.delete_prefix("#{REPO_DIR}/") - @defaults[project] = YAML.safe_load(File.open(defaults_file)) + defaults_key = repodir == REPO_DIR ? 'default' : repodir.delete_prefix("#{REPO_DIR}/") + @defaults[defaults_key] = YAML.safe_load(File.open(defaults_file)) end def traverse_repodir(repodir) @@ -225,7 +225,7 @@ class MirrorMain git_repo = "#{project}/#{fork_name}" @git_info[git_repo] = YAML.safe_load(File.open(repodir)) @git_info[git_repo]['git_repo'] = git_repo - @git_info[git_repo].merge!(@defaults[project]) if @defaults[project] + @git_info[git_repo] = merge_defaults(git_repo) fork_stat_init(git_repo) @priority_queue.push git_repo, @priority @priority += 1 @@ -447,3 +447,16 @@ class MirrorMain return update_year_hash(new_refs_count, year, month, day) end end + +# main thread +class MirrorMain + def merge_defaults(git_repo) + defaults_key = File.dirname(git_repo) + while defaults_key != '.' + return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + + defaults_key = File.dirname(defaults_key) + end + return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + end +end -- 2.23.0

1 0

[PATCH compass-ci 1/2] doc: add job/os_mount.md
by Yu Chuan 02 Feb '21

02 Feb '21

Add description of os_mount field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 62 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 doc/job/os_mount.md diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md new file mode 100644 index 000000000000..a4d832e07955 --- /dev/null +++ b/doc/job/os_mount.md @@ -0,0 +1,62 @@ +# Summary +========= + +`os_mount` defines the mount method of tbox's root partition. + +It has the following optional values: + - nfs + - cifs + - initramfs + - container + - local + +Usage example: + - submit iperf.yaml testbox=vm-2p8g os_mount=nfs + - submit iperf.yaml testbox=vm-2p8g os_mount=cifs + - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs + - submit iperf.yaml testbox=vm-2p8g os_mount=container + - submit iperf.yaml testbox=vm-2p8g os_mount=local + +# Optional Values +================= + +## Work flow when os_mount=local + +1. user submit job with os_mount: local + optional kernel cmdline params: # can be added in kernel_append_root field of job.yaml + - keep_last_root: keep the original data of disk what last job generated, don't wipe it + +2. scheduler return the ipxe_str to tbox + ``` + dhcp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/initrd.lkp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/boot/vmlinuz + imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + boot + ``` + +3. boot + - check if `/dev/mapper/os-openeuler_aarch64_20.03_{timestamp}` exist # according the kernel cmdline + - if exist: + - if `keep_last_root` not in kernel cmdline params: + - re-create(lvremove and lvcreate) lvm rw snapshot /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if `keep_last_root` in kernel cmdline params: + - if /dev/mapper/os-openeuler_aarch64_20.03 exists: + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if /dev/mapper/os-openeuler_aarch64_20.03 not exists: + - generate /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03-{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if not exist: + ``` + mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt + create /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} according rootfs_disk. # lvm size: 10G + mkdir /mnt1, mount /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} /mnt1 + rsync -az /mnt/. /mnt1/ + umount /mnt /mnt1 + lvchange -p r /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + create lvm rw snapshot /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + boot from /dev/mapper/os-openeuler_aarch64_20.03 + ``` + +4. execute the job -- 2.23.0

3 4

[PATCH compass-ci] container/defconfig: fix directory name spell error
by Liu Yinsi 02 Feb '21

02 Feb '21

container/defconfig.rb: Dir.glob(... '/etc/compass-ci/accounts/*.yaml', ...) [why] two file defconfig.sh and defconfig.rb, all load /etc/compass-ci/accounts/*.yaml, so the directory name should be same. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/defconfig.sh b/container/defconfig.sh index 4a5f5fa..54e65b5 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -10,7 +10,7 @@ load_cci_defaults() yaml_file=( /etc/compass-ci/defaults/*.yaml - /etc/compass-ci/account/*.yaml + /etc/compass-ci/accounts/*.yaml "$HOME"/.config/compass-ci/defaults/*.yaml ) for i in "${yaml_file[@]}" -- 2.23.0

1 0

[PATCH compass-ci 2/2] doc: add job/rootfs_disk.md
by Yu Chuan 02 Feb '21

02 Feb '21

Add description of rootfs_disk field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/rootfs_disk.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 doc/job/rootfs_disk.md diff --git a/doc/job/rootfs_disk.md b/doc/job/rootfs_disk.md new file mode 100644 index 000000000000..8236e24facb8 --- /dev/null +++ b/doc/job/rootfs_disk.md @@ -0,0 +1,25 @@ +# Summary +========= + +`rootfs_disk` defines a disk which storing multiple rootfs, used for jobs which need boot from disk. + +Each of these rootfs corresponds to an logical volume, such as follow: + /dev/mapper/ + . + ├── os-openeuler_aarch64_20.03 + ├── os-openeuler_aarch64_20.03_{timestamp} + ├── os-openeuler_aarch64_20.03_sp1 + ├── os-openeuler_aarch64_20.03_sp1_{timestamp} + └── ... + +Addition: +1. `/dev/mapper/os-{os}_{os_arch}_{os_version}_{timestamp}` + - is a readonly logical volume + - used for backup and rollback + +2. `/dev/mapper/os-{os}_{os_arch}_{os_version}` + - is a read write logical volume + - use for boot and run job + - defaultly, this logical volume will be wiped before every job run + +3. more info can refer os_mount.md -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] service/lifecycle: to process the machine that times out
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/lifecycle.cr | 51 ++++++++++++++++++++++++++++++++++++++ src/lifecycle/lifecycle.cr | 8 ++++++ 2 files changed, 59 insertions(+) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index af5cd07..75a18ec 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -4,6 +4,7 @@ require "kemal" require "yaml" +require "./scheduler_api" require "./web_env" require "../scheduler/elasticsearch_client" @@ -14,6 +15,7 @@ class Lifecycle @es = Elasticsearch::Client.new @env = env @log = env.log.as(JSONLogger) + @scheduler_api = SchedulerAPI.new end def alive(version) @@ -36,4 +38,53 @@ class Lifecycle } @es.search("testbox", query) end + + def get_timeout_testbox(size) + # why use "now+8h" + # The time zone in the ES database is 0 + # add 8 hours to the current time and convert it to time zone 0 + query = { + "size" => size, + "query" =>{ + "range" => { + "deadline" => { + "lt" => "now+8h" + } + } + } + } + + results = @es.search("testbox", query) + results["hits"]["hits"] + end + + def process_timeout_testbox + size = @env.params.query["size"]? || 20 + results = get_timeout_testbox(size) + return results unless results.is_a?(JSON::Any) + + results.as_a.each do |result| + hostname = result["_id"].to_s + hash = result["_source"] + close_lifecycle(hostname, hash, "timeout") + end + end + + # 1.reboot testbox, do in the fature + # 2.update testbox info in es + # 3.close job + def close_lifecycle(hostname, hash, close_reason) + state = hash["state"]? == "rebooting" ? hash["state"] : "forcibly_rebooting" + testbox_update_info = { + "deadline" => nil, + "state" => state + } + @es.update_tbox(hostname, testbox_update_info) + + # If the value of state is rebooting, + # the job has been closed and don't need to close it again + return if state == "rebooting" + + @scheduler_api.close_job(hash["job_id"], close_reason).status_message + end end diff --git a/src/lifecycle/lifecycle.cr b/src/lifecycle/lifecycle.cr index 4154a27..13d36b7 100644 --- a/src/lifecycle/lifecycle.cr +++ b/src/lifecycle/lifecycle.cr @@ -29,4 +29,12 @@ module Cycle get "/get_running_testbox" do |env| env.lifecycle.get_running_testbox.to_json end + + # curl http://localhost:11311/process_timeout_testbox?size=10 + # size indicates the number of records to be processed at a time + get "/process_timeout_testbox" do |env| + env.lifecycle.process_timeout_testbox + + "done" + end end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/scheduler_api.cr: added parameter "job_state"
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/scheduler_api.cr | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/lib/scheduler_api.cr b/src/lib/scheduler_api.cr index b8dd514..0fdcd12 100644 --- a/src/lib/scheduler_api.cr +++ b/src/lib/scheduler_api.cr @@ -10,10 +10,14 @@ class SchedulerAPI @host = ENV.has_key?("SCHED_HOST") ? ENV["SCHED_HOST"] : "172.17.0.1" end - def close_job(job_id) + def close_job(job_id, job_state = nil) + url = "/~lkp/cgi-bin/lkp-post-run?job_id=#{job_id}" + url += "&job_state=#{job_state}" if job_state + client = HTTP::Client.new(@host, port: @port) - response = client.get("/~lkp/cgi-bin/lkp-post-run?job_id=#{job_id}") + response = client.get(url) client.close() + return response end end -- 2.23.0

1 0

[PATCH v2 compass-ci] lib/job.cr: add a new os_mount named local
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 2 +- src/scheduler/kernel_params.cr | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 964c520..c013aa7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -344,7 +344,7 @@ class Job end # defaults to the 1st value - VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container"] + VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container", "local"] private def set_os_mount if is_docker_job? diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index dea9215..ed64bea 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -18,6 +18,7 @@ class Job "cifs" => "root=cifs://#{OS_HTTP_HOST}#{os_real_path}" + ",guest,ro,hard,vers=1.0,noacl,nouser_xattr", "initramfs" => "rdinit=/sbin/init prompt_ramdisk=0", + "local" => "root=/dev/mapper/os-#{self.os}_#{self.os_arch}_#{self.os_version}", "container" => "", } -- 2.23.0

1 0

[PATCH v4 compass-ci] providers/libvirt: split domain.xml to some common templates
by Xiao Shenwei 02 Feb '21

02 Feb '21

[why] libvirt can boot a vm based on a xml file named domain.xml. it contains many elements like os, device, cpu. to meet a requirements of a single module or a combination of multiple modules. split the domain.xml may be better. here are some scenarios: case 1: the user provides a complete domain.xml file case 2: the user only provides a cpu.xml template case 3: the user provides cpu.xml and disk.xml or other templates Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/templates/active.xml | 3 +++ providers/libvirt/templates/clock.xml | 5 +++++ providers/libvirt/templates/cpu.xml | 3 +++ providers/libvirt/templates/devices.xml | 3 +++ providers/libvirt/templates/disk.xml | 7 +++++++ providers/libvirt/templates/domain.xml | 10 ++++++++++ providers/libvirt/templates/emulator.xml | 1 + providers/libvirt/templates/interface.xml | 6 ++++++ providers/libvirt/templates/memory.xml | 1 + providers/libvirt/templates/name.xml | 1 + providers/libvirt/templates/options.yaml | 23 +++++++++++++++++++++++ providers/libvirt/templates/os.xml | 7 +++++++ providers/libvirt/templates/seclabel.xml | 4 ++++ providers/libvirt/templates/serial.xml | 4 ++++ 14 files changed, 78 insertions(+) create mode 100644 providers/libvirt/templates/active.xml create mode 100644 providers/libvirt/templates/clock.xml create mode 100644 providers/libvirt/templates/cpu.xml create mode 100644 providers/libvirt/templates/devices.xml create mode 100644 providers/libvirt/templates/disk.xml create mode 100644 providers/libvirt/templates/domain.xml create mode 100644 providers/libvirt/templates/emulator.xml create mode 100644 providers/libvirt/templates/interface.xml create mode 100644 providers/libvirt/templates/memory.xml create mode 100644 providers/libvirt/templates/name.xml create mode 100644 providers/libvirt/templates/options.yaml create mode 100644 providers/libvirt/templates/os.xml create mode 100644 providers/libvirt/templates/seclabel.xml create mode 100644 providers/libvirt/templates/serial.xml diff --git a/providers/libvirt/templates/active.xml b/providers/libvirt/templates/active.xml new file mode 100644 index 0000000..8ed7ee4 --- /dev/null +++ b/providers/libvirt/templates/active.xml @@ -0,0 +1,3 @@ +<on_poweroff>destroy</on_poweroff> +<on_reboot>destroy</on_reboot> +<on_crash>destroy</on_crash> diff --git a/providers/libvirt/templates/clock.xml b/providers/libvirt/templates/clock.xml new file mode 100644 index 0000000..5124f33 --- /dev/null +++ b/providers/libvirt/templates/clock.xml @@ -0,0 +1,5 @@ +<clock offset='localtime'> + <timer name='rtc' tickpolicy='catchup' track='guest'/> + <timer name='hpet' present='no'/> + <timer name='pit' tickpolicy='delay'/> +</clock> diff --git a/providers/libvirt/templates/cpu.xml b/providers/libvirt/templates/cpu.xml new file mode 100644 index 0000000..1f95dc6 --- /dev/null +++ b/providers/libvirt/templates/cpu.xml @@ -0,0 +1,3 @@ +<vcpu placement='static'><%= nr_cpu %></vcpu> +<cpu mode='host-passthrough' check='none'> +</cpu> diff --git a/providers/libvirt/templates/devices.xml b/providers/libvirt/templates/devices.xml new file mode 100644 index 0000000..83cd847 --- /dev/null +++ b/providers/libvirt/templates/devices.xml @@ -0,0 +1,3 @@ +<devices> + <graphics type='vnc' autoport='yes' listen='0.0.0.0'><graphics> +</devices> diff --git a/providers/libvirt/templates/disk.xml b/providers/libvirt/templates/disk.xml new file mode 100644 index 0000000..7aecef0 --- /dev/null +++ b/providers/libvirt/templates/disk.xml @@ -0,0 +1,7 @@ +<% if disk %> +<disk type='block' device='disk'> + <driver name='qemu' type='raw'/> + <source dev='<%= disk %>'/> + <target dev='vda' bus='virtio'/> +</disk> +<% end %> diff --git a/providers/libvirt/templates/domain.xml b/providers/libvirt/templates/domain.xml new file mode 100644 index 0000000..a3fdb63 --- /dev/null +++ b/providers/libvirt/templates/domain.xml @@ -0,0 +1,10 @@ +<?xml version="1.0" character="utf-8"?> +<domain type='kvm'> + <resource> + <partition>/machine</partition> + </resource> + <features> + <acpi/> + <gic version='3'/> + </features> +</domain> diff --git a/providers/libvirt/templates/emulator.xml b/providers/libvirt/templates/emulator.xml new file mode 100644 index 0000000..e915215 --- /dev/null +++ b/providers/libvirt/templates/emulator.xml @@ -0,0 +1 @@ +<emulator><%=qemu_path%></emulator> diff --git a/providers/libvirt/templates/interface.xml b/providers/libvirt/templates/interface.xml new file mode 100644 index 0000000..5d4c04f --- /dev/null +++ b/providers/libvirt/templates/interface.xml @@ -0,0 +1,6 @@ +<interface type='bridge'> + <source bridge='br0'/> + <mac address='<%= mac %>'/> + <target dev='vnet0'/> + <model type='virtio'/> +</interface> diff --git a/providers/libvirt/templates/memory.xml b/providers/libvirt/templates/memory.xml new file mode 100644 index 0000000..34aec49 --- /dev/null +++ b/providers/libvirt/templates/memory.xml @@ -0,0 +1 @@ +<memory unit='GiB'><%= memory.to_i %></memory> diff --git a/providers/libvirt/templates/name.xml b/providers/libvirt/templates/name.xml new file mode 100644 index 0000000..5dc98f4 --- /dev/null +++ b/providers/libvirt/templates/name.xml @@ -0,0 +1 @@ +<name><%= job_id %></name> diff --git a/providers/libvirt/templates/options.yaml b/providers/libvirt/templates/options.yaml new file mode 100644 index 0000000..c41ee29 --- /dev/null +++ b/providers/libvirt/templates/options.yaml @@ -0,0 +1,23 @@ +# Domain option: +# use templates/domain.xml if user not specified. +domain: domain + +# Default options: +# these elements must be replaced for each domain.xml. +default: + - name + - os + - memory + - serial + - interface + - active + - emulator + +# Replaceable options: +# these elements can be replaced by user configurations. +replaceable: + - devices + - cpu + - seclabel + - disk + - clock diff --git a/providers/libvirt/templates/os.xml b/providers/libvirt/templates/os.xml new file mode 100644 index 0000000..ee16126 --- /dev/null +++ b/providers/libvirt/templates/os.xml @@ -0,0 +1,7 @@ +<os> + <type arch='<%= arch %>' machine='virt'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader> + <kernel><%= kernel %></kernel> + <initrd><%= initrd %></initrd> + <cmdline><%= cmdline %></cmdline> +</os> diff --git a/providers/libvirt/templates/seclabel.xml b/providers/libvirt/templates/seclabel.xml new file mode 100644 index 0000000..5c10a77 --- /dev/null +++ b/providers/libvirt/templates/seclabel.xml @@ -0,0 +1,4 @@ +<seclabel type='dynamic' model='dac' relabel='yes'> + <label>+0:+0</label> + <imagelabel>+0:+0</imagelabel> +</seclabel> diff --git a/providers/libvirt/templates/serial.xml b/providers/libvirt/templates/serial.xml new file mode 100644 index 0000000..2edc882 --- /dev/null +++ b/providers/libvirt/templates/serial.xml @@ -0,0 +1,4 @@ +<serial type='file'> + <source path="<%= log_file %>"/> + <target port="0"/> +</serial> -- 2.23.0

1 0

[PATCH v4 compass-ci 3/4] container/submit: submit file
by Luan Shengde 02 Feb '21

02 Feb '21

executable file for container: submit it can be run as the real command submit in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..3f9539f --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /root/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

2 2

[PATCH v3 compass-ci 1/3] mail-robot: fix bug for parsing email
by Luan Shengde 02 Feb '21

02 Feb '21

[why] for non-gitee repos: we may sometimes can not get the author email from the http request feedback when checking the commit. users may still receive a fail email even if they send 'apply account' email correctly. [how] both gitee and non-gitee will clone the repo tree and check the commit: - check commit id exists - check commit author matches the email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 42 ++++++++++--------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index a41a89a..8baf7b7 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -6,7 +6,6 @@ require 'json' require 'mail' require 'rest-client' -require_relative 'gitee-commit-url-check' # used to parse mail_content for my_commit_url and my_ssh_pubkey # be called by AssignAccount when it needs to extract required data: @@ -91,29 +90,34 @@ class ParseApplyAccountEmail end def commit_url_availability(url, base_url) - hub_name = url.split('/')[2] - - # it requires authentication when sending request to get the commit information - # clone the repo and then validate the commit for the email address - if hub_name.eql? 'gitee.com' - gitee_commit(url, base_url) - else - non_gitee_commit(url) - end - end + repo_url = [base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] + commit_id = url.split('/')[-1] + + Dir.chdir '/tmp' + %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) + + # get all commit IDs and check commit id exists + repo_commits = %x(git -C #{repo_dir} log --pretty=format:'%H').split(/\n/) + check_commit_exist(commit_id, repo_commits) - def gitee_commit(url, base_url) - my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) - my_gitee_commit.gitee_commit_check + # get the auther's email for the commit + author_email = %x(/usr/bin/git -C #{repo_dir} show -s --format=%aE #{commit_id}).chomp + check_commit_email(author_email) + + FileUtils.rm_rf repo_dir end - def non_gitee_commit(url) - url_fdback = RestClient.get(url).body - email_index = url_fdback.index @my_info['my_email'] + def check_commit_exist(commit_id, repo_commits) + return if repo_commits.include? commit_id.chomp + + raise 'NO_COMMIT_ID' + end - return unless email_index.nil? + def check_commit_email(author_email) + return if author_email.eql? @my_info['my_email'].chomp - raise 'COMMIT_URL_NOT_AVAILABLE' + raise 'COMMIT_AUTHOR_ERROR' end def parse_pub_key -- 2.23.0

2 2

[PATCH lkp-tests] lib/job.rb: delete fields that need to be reloaded for reproduction task
by Li Ping 02 Feb '21

02 Feb '21

[why] load job from result's job yaml has key "my_email", like my_email: compass-ci-robot(a)qq.com my_name: compass-ci-robot lab: z9 reproduction task: my_email: xxxxx my_name: xxxx lab: xxx ==> check the field my_email fail each of these fields needs to be reset when submit job.yaml for reprodution task. however, these fields already exist in yaml file, can not to be reload, so delete these fields when submit reproduction job. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/job.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index a03261537..2b122c01d 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -259,6 +259,11 @@ class Job hash.delete('override') end + if hash.has_key?('id') + d_keys = ["my_name", "my_email", "lab"] + hash.delete_if { |k| d_keys.include? k } + end + revise_hash(hash, load_include_yamls(@default_yamls), false) unless @default_yamls.empty? revise_hash(hash, load_include_yamls(@override_yamls), true) unless @override_yamls.empty? revise_hash(hash, @overrides, true) unless @overrides.empty? -- 2.23.0

1 0

[PATCH v3 compass-ci 2/3] mail-robot: optimize assign-account-fail-email
by Luan Shengde 02 Feb '21

02 Feb '21

add message for newly added error types. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 242dfe1..3597159 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -33,17 +33,20 @@ def email_err_message(message) https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… EMAIL_MESSAGE - when 'COMMIT_URL_NOT_AVAILABLE' + when 'COMMIT_AUTHOR_ERROR' err_message = <<~EMAIL_MESSAGE - We cannot confirm the commit url matches your email. + We cannot confirm the commit author matches your email. + Make sure it is truely submitted with your email. - Make sure the commit exists. - Make sure it is truely submitted with your email. - - Go to the commit history and find your commit. - Enter the commit and you will get your commit url. + Choose a commit that was submitted with your own email and try again. + EMAIL_MESSAGE + when 'NO_COMMIT_ID' + err_message = <<~EMAIL_MESSAGE + There is no such commit ID in the repo. - Try again after you have correctted the url + Please check the commit ID exists. + or + You have write the commit ID correctly. EMAIL_MESSAGE when 'NO_PUBKEY' err_message = <<~EMAIL_MESSAGE -- 2.23.0

2 2

[PATCH v4 lkp-tests] tests/cci-depends: fix generate symlink cgz file
by Wang Yong 02 Feb '21

02 Feb '21

[before] $ ll $LKP_SRC/distro/depends/sysbench-cpu sysbench-cpu -> sysbench $ tree /srv/initrd/deps/$os/$os_arch/$os_version/ . ├── sysbench_20200903.cgz ├── sysbench.cgz -> sysbench_20200903.cgz └── sysbench-cpu.cgz -> sysbench.cgz all cgz file under one directory, so we can generate symlink when depends file is a symlink [after] $ tree /srv/initrd/deps/$os/$os_arch/$os_version/sysbench-cpu . ├── sysbench-cpu_20210128.cgz └── sysbench-cpu.cgz -> sysbench-cpu_20210128.cgz now we upload cgz file through curl, it's not appropriate to generate symlink directly, now generate original cgz file and rename it Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 12 ++++++++++++ tests/cci-depends | 9 ++------- 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/distro/common b/distro/common index 1fd7df4e8..94d18f461 100755 --- a/distro/common +++ b/distro/common @@ -94,3 +94,15 @@ upload_to_target_dir() upload_one_curl "${dir}/${file}" "${dir}" >/dev/null } + +rename_cgz_file() +{ + [ "${BM_NAME}" == "${benchmark}" ] && return + + cd "$pack_to" && { + [ -f "${BM_NAME}".cgz ] || return + rm "${BM_NAME}".cgz + date=$(ls -F | grep -v "@" | grep -Eo "[0-9]{8}") + mv "${BM_NAME}"_"$date".cgz "${benchmark}"_"$date".cgz + } +} diff --git a/tests/cci-depends b/tests/cci-depends index 8ac1d7e58..4a7b78a41 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -38,13 +38,6 @@ umask 002 chown .lkp "$pack_to" } -check_shared_package "$benchmark" && { - ln -sf $BM_NAME.cgz $pack_to/$benchmark.cgz - echo "create symlink for shared package: $benchmark -> $BM_NAME" - - exit 0 -} - share_debian_package distro_install_depends cci-depends @@ -59,4 +52,6 @@ fixup_arch_install $pack_arch pack_benchmark_deps +rename_cgz_file + upload_to_target_dir ${pack_to} -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/cci-depends: fix generate symlink cgz file
by Wang Yong 02 Feb '21

02 Feb '21

[before] $ ll $LKP_SRC/distro/depends/sysbench-cpu sysbench-cpu -> sysbench $ tree /srv/initrd/deps/$os/$os_arch/$os_version/ . ├── sysbench_20200903.cgz ├── sysbench.cgz -> sysbench_20200903.cgz └── sysbench-cpu.cgz -> sysbench.cgz all cgz file under one directory, so we can generate symlink when depends file is a symlink [after] $ tree /srv/initrd/deps/$os/$os_arch/$os_version/sysbench-cpu . ├── sysbench-cpu_20210128.cgz └── sysbench-cpu.cgz -> sysbench-cpu_20210128.cgz now we upload cgz file through curl, it's not appropriate to generate symlink directly, now generate original cgz file and rename it Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 11 +++++++++++ tests/cci-depends | 9 ++------- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/distro/common b/distro/common index 1fd7df4e8..f3a9c3071 100755 --- a/distro/common +++ b/distro/common @@ -94,3 +94,14 @@ upload_to_target_dir() upload_one_curl "${dir}/${file}" "${dir}" >/dev/null } + +rename_cgz_file() +{ + [ "${BM_NAME}" == "${benchmark}" ] && return + + cd "$pack_to" + [ -f "${BM_NAME}".cgz ] || return + rm "${BM_NAME}".cgz + date=$(ls -F | grep -v "@" | grep -Eo "[0-9]{8}") + mv "${BM_NAME}"_"$date".cgz "${benchmark}"_"$date".cgz +} diff --git a/tests/cci-depends b/tests/cci-depends index 8ac1d7e58..4a7b78a41 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -38,13 +38,6 @@ umask 002 chown .lkp "$pack_to" } -check_shared_package "$benchmark" && { - ln -sf $BM_NAME.cgz $pack_to/$benchmark.cgz - echo "create symlink for shared package: $benchmark -> $BM_NAME" - - exit 0 -} - share_debian_package distro_install_depends cci-depends @@ -59,4 +52,6 @@ fixup_arch_install $pack_arch pack_benchmark_deps +rename_cgz_file + upload_to_target_dir ${pack_to} -- 2.23.0

2 2

[PATCH compass-ci] fix spell error
by Bai Jing 02 Feb '21

02 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- doc/tutorial.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index bf72630..bb2951c 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -174,7 +174,7 @@ job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAM - 通过 qemu.img(qcow2格式)获取 rootfs (openEuler为例) - 1) 下载 openEule r官方网站提供的 qcow2 格式镜像 + 1) 下载 openEuler官方网站提供的 qcow2 格式镜像 ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… ``` -- 2.23.0

1 0

[PATCH compass-ci] sbin/es-find order by start_time as default
by Lu Weitao 02 Feb '21

02 Feb '21

[Why] Many time, user would see the latest jobs [How] Set 'start_time' as default keyword for order with 'desc' Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/es-find | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/es-find b/sbin/es-find index c57c02e..ed54955 100755 --- a/sbin/es-find +++ b/sbin/es-find @@ -72,7 +72,7 @@ def assign_range(value) range_value end -options = { nr_jobs: 10, desc_keyword: nil } +options = { nr_jobs: 10, desc_keyword: 'start_time' } range = nil opt_parser = OptionParser.new do |opts| -- 2.23.0

2 2

[PATCH v3 compass-ci] sparrow/install-server: add script to deploy compass-ci server
by Liu Yinsi 02 Feb '21

02 Feb '21

Classify compass-ci by functions: Client functions: submit job and boot testbox Server functions: run containers services Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 sparrow/install-server diff --git a/sparrow/install-server b/sparrow/install-server new file mode 100755 index 0000000..c532956 --- /dev/null +++ b/sparrow/install-server @@ -0,0 +1,20 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# For your reference only. +# It's better to run the below scripts step by step. + +[[ $CCI_SRC ]] || export CCI_SRC=$(dirname $(dirname $(realpath $0))) +cd $CCI_SRC/sparrow || exit + +0-package/common +1-storage/tiny +1-storage/permission +2-network/br0 +2-network/iptables +2-network/nfs +2-network/cifs +3-code/dev-env +. /etc/profile.d/compass.sh +4-docker/buildall run-only +7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/install-server: add script to deploy compass-ci container services
by Liu Yinsi 02 Feb '21

02 Feb '21

[why] For user locally deploy compass-ci, some users' hosts can not access internet. [how] deploy all container services in the host machine, other functions: submit job and run qemu test, will deploy in other machine which can access internet by $LKP_SRC/setup/simply-ci. sparrow/install-tiny: one-click locally deploy compass-ci, do job test in locally compass-ci platform. sparrow/install-server: one-click locally deploy compass-ci server, just running containers services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 sparrow/install-server diff --git a/sparrow/install-server b/sparrow/install-server new file mode 100755 index 0000000..c532956 --- /dev/null +++ b/sparrow/install-server @@ -0,0 +1,20 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# For your reference only. +# It's better to run the below scripts step by step. + +[[ $CCI_SRC ]] || export CCI_SRC=$(dirname $(dirname $(realpath $0))) +cd $CCI_SRC/sparrow || exit + +0-package/common +1-storage/tiny +1-storage/permission +2-network/br0 +2-network/iptables +2-network/nfs +2-network/cifs +3-code/dev-env +. /etc/profile.d/compass.sh +4-docker/buildall run-only +7-systemd/systemd-setup -- 2.23.0

2 2

[PATCH v5 compass-ci 3/3] replace log.sh
by Liu Yinsi 02 Feb '21

02 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 2 +- container/result-cifs/start | 2 +- providers/my-qemu.sh | 4 +--- providers/qemu/kvm.sh | 3 +-- providers/qemu/pxe.sh | 4 +--- sparrow/4-docker/buildall | 6 +++--- 8 files changed, 10 insertions(+), 15 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 0b9601a..6a91b37 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-cifs/start b/container/os-cifs/start index 09ae3d2..85017cc 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-nfs/start b/container/os-nfs/start index 9539561..1f4ed42 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { log_error "kernel module nfs is missing (try: sudo modprobe nfs)" diff --git a/container/result-cifs/start b/container/result-cifs/start index de38ca2..7ce066a 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 113b3ef..3e3b2f4 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -2,9 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh help() { diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index dd657cb..103e9e6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -6,9 +6,8 @@ : ${nr_cpu:=1} : ${memory:=1G} -: ${LKP_SRC:="/c/lkp-tests"} -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh check_logfile() { diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index 0ba052c..3ab3c2b 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -4,9 +4,7 @@ # - nr_cpu # - memory -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh : ${nr_cpu:=1} : ${memory:=1G} diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 3c0b7b0..727abb9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -5,7 +5,7 @@ CONTAINER_PATH="$CCI_SRC/container" export action=$1 -[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" +source ${CCI_SRC}/lib/log.sh build_depends() { @@ -27,7 +27,7 @@ do_one() lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock echo "$started_containers" | grep -w -q "$container_name" && return 0 - [ -n "$LKP_SRC" ] && log_info "start build $container." + log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" @@ -41,7 +41,7 @@ do_one() [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run ) - [ -n "$LKP_SRC" ] && log_info "finish build $container." + log_info "finish build $container." started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH v5 compass-ci 2/3] container: delete redundant log error info
by Liu Yinsi 02 Feb '21

02 Feb '21

[how] before: log_error "[ERROR] kernel module ..." after: log_error "kernel module ..." [why] log_error(): echo "$(date +'%F %T') [ERROR] -- $*" >&2 when call log_error(), already has complete log format, so delete redundant "[ERROR]" in parmas. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 4 ++-- container/result-cifs/start | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 119d1b1..0b9601a 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-cifs/start b/container/os-cifs/start index 47da260..09ae3d2 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-nfs/start b/container/os-nfs/start index 8c08db8..9539561 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -6,11 +6,11 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { - log_error "[ERROR] kernel module nfs is missing (try: sudo modprobe nfs)" + log_error "kernel module nfs is missing (try: sudo modprobe nfs)" exit 1 } lsmod | grep -q "^nfsd\s" || { - log_error "[ERROR] kernel module nfsd is missing (try: sudo modprobe nfsd)" + log_error "kernel module nfsd is missing (try: sudo modprobe nfsd)" exit 1 } diff --git a/container/result-cifs/start b/container/result-cifs/start index 4f8c2b9..de38ca2 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } -- 2.23.0

2 1

[PATCH v5 compass-ci 1/3] lib: add log.sh
by Liu Yinsi 02 Feb '21

02 Feb '21

Records the run logs of the shell program Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/log.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 lib/log.sh diff --git a/lib/log.sh b/lib/log.sh new file mode 100644 index 0000000..626a6e4 --- /dev/null +++ b/lib/log.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +log_info() +{ + date +"[INFO] %F %T ${0##*/}: $*" +} + +log_debug() +{ + date +"[DEBUG] %F %T ${0##*/}: $*" +} + +log_warn() +{ + date +"[WARN] %F %T ${0##*/}: $*" >&2 +} + +log_error() +{ + date +"[ERROR] %F %T ${0##*/}: $*" >&2 +} -- 2.23.0

2 1

[PATCH v5 compass-ci] sbin: add script for download os rootfs
by Liu Yinsi 02 Feb '21

02 Feb '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100755 index 0000000..e1d4091 --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +os=${1:-openeuler} +os_arch=${2:-aarch64} +os_version=${3:-20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=/srv/os/${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p $rootfs_dir + cd $rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${os}/${os_arch}/${…" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

1 0

[PATCH compass-ci] service/lifecycle: change port
by Wu Zhende 02 Feb '21

02 Feb '21

Port 11311 has been used by email service Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/lifecycle/build | 2 +- container/lifecycle/start | 2 +- src/lifecycle/constants.cr | 2 +- src/lifecycle/lifecycle.cr | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/container/lifecycle/build b/container/lifecycle/build index ba68a85..bfb7490 100755 --- a/container/lifecycle/build +++ b/container/lifecycle/build @@ -9,7 +9,7 @@ require_relative '../defconfig.rb' names = Set.new ['LIFECYCLE_PORT'] defaults = relevant_defaults(names) -LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11311' +LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." diff --git a/container/lifecycle/start b/container/lifecycle/start index 7e67b3c..6e9b4fb 100755 --- a/container/lifecycle/start +++ b/container/lifecycle/start @@ -13,7 +13,7 @@ names = Set.new %w[ defaults = relevant_defaults(names) env = docker_env(defaults) -LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11311' +LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' docker_rm "lifecycle-#{LIFECYCLE_PORT}" cmd = %W[ diff --git a/src/lifecycle/constants.cr b/src/lifecycle/constants.cr index a428251..137b7f9 100644 --- a/src/lifecycle/constants.cr +++ b/src/lifecycle/constants.cr @@ -1,4 +1,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -LIFECYCLE_PORT = (ENV.has_key?("LIFECYCLE_PORT") ? ENV["LIFECYCLE_PORT"] : 11311).to_i32 +LIFECYCLE_PORT = (ENV.has_key?("LIFECYCLE_PORT") ? ENV["LIFECYCLE_PORT"] : 11312).to_i32 diff --git a/src/lifecycle/lifecycle.cr b/src/lifecycle/lifecycle.cr index 4154a27..da42fda 100644 --- a/src/lifecycle/lifecycle.cr +++ b/src/lifecycle/lifecycle.cr @@ -25,7 +25,7 @@ module Cycle end # find the testbox that are performing jobs - # curl http://localhost:11311/get_running_testbox?size=10&from=0 + # curl http://localhost:11312/get_running_testbox?size=10&from=0 get "/get_running_testbox" do |env| env.lifecycle.get_running_testbox.to_json end -- 2.23.0

1 0

[PATCH v4 compass-ci 4/4] container/submit: README file
by Luan Shengde 02 Feb '21

02 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 container/submit/README.md diff --git a/container/submit/README.md b/container/submit/README.md new file mode 100644 index 0000000..6772961 --- /dev/null +++ b/container/submit/README.md @@ -0,0 +1,26 @@ +# Install lkp-tests and its depends in container. +# With docker container, will suit most of the linux OSes. +# Avoid much repetitive testing work to adapt various OSes. + +Prepare work: + apply account + config default yaml + generate ssh keys + +Installation: + git clone https://gitee.com/wu_fengguang/compass-ci.git + + cd compass-ci/container/submit + ./build + + Add soft link for executable file 'submit' to $PATH: + + ln -s $PWD/submit /usr/bin/submit + +Usage: + You can directly use the command 'submit' to submit jobs. + It's the same as you install the lkp-tests at your local host. + + Example: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0