- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] register-accounts: sync register-accounts
by Luan Shengde 03 Feb '21

03 Feb '21

[why] the required file lib/build_my_info_client has been updated. and the invoking way has some change. just do update according to the change. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/register-accounts/register-accounts | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/container/register-accounts/register-accounts b/container/register-accounts/register-accounts index ff69032..519f289 100755 --- a/container/register-accounts/register-accounts +++ b/container/register-accounts/register-accounts @@ -9,7 +9,12 @@ require_relative "#{ENV['CCI_SRC']}/lib/build_my_info_client" content = YAML.load_file('/etc/compass-ci/register/register.yaml') content.each do |key, value| - build_my_info = BuildMyInfo.new(value['my_email'], value['my_name'], ENV['lab']) - build_my_info.store_account_info + my_info = {} + my_info['my_email'] = value['my_email'] + my_info['my_name'] = value['my_name'] + my_info['lab'] = ENV['lab'] + + build_my_info = BuildMyInfo.new(my_info['my_email']) + build_my_info.store_account_info(my_info) end sleep -- 2.23.0

2 1

[PATCH v2 compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

when execute openresty-proxy-cache/build, if openresty already exists [error] fatal: destination path 'openresty' already exists and is not an empty directory. [how] delete openresty before git clone. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 0846197..2c982f7 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +rm -rf openresty + git clone https://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

1 0

[PATCH compass-ci] sparrow/1-storage: add work dir and set permission
by Liu Yinsi 03 Feb '21

03 Feb '21

add work dir and permission to use libvirt to boot testbox. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index d1367cf..0e8d722 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -7,6 +7,7 @@ chown -R mailer:team /srv/cci/Maildir chown lkp:lkp /srv/cache/netdata_cache chown lkp:lkp /srv/cache/netdata_lib chown lkp:lkp /srv/cci/serial/fluentd-pos +chown lkp:lkp /srv/cci/libvirt-xml chown lkp:committer /srv/git chgrp team /srv/cci/serial/logs chmod 775 /srv/result @@ -15,6 +16,7 @@ chmod 775 /srv/es chmod 775 /srv/cache/netdata_cache chmod 775 /srv/cache/netdata_lib chmod 775 /srv/cci/serial/logs +chmod 775 /srv/cci/libvirt-xml chmod 775 /srv/cci/serial/fluentd-pos chmod -R 775 /srv/cci/Maildir diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..8fd3652 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -17,6 +17,7 @@ dirs=( /srv/cci/scheduler /srv/cci/serial/logs /srv/cci/serial/fluentd-pos + /srv/cci/libvirt-xml /tftpboot /c /etc/qemu -- 2.23.0

2 2

Re: [PATCH compass-ci] container/openresty-proxy-cache: fix build error
by Liu Yinsi 03 Feb '21

03 Feb '21

>> git clone https://gitee.com/cuiyili/openresty.git || exit 1 >> docker build -t openresty:proxy_cache . -f openresty/Dockerfile >>+rm -rf openresty > >How about move the 'rm' step at the before of 'docker build'? docker build need openrestry/Dockerfile, this file come from git clone https://gitee.com/cuiyili/openresty.git, you mean move before of git clone? it looks not make any difference. Thanks, Yinsi > >-------- >Thanks >Yu Chuan > >>-- >>2.23.0 >>

2 2

[PATCH v5 compass-ci 3/3] mail-robot: delete useless file
by Luan Shengde 03 Feb '21

03 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/gitee-commit-url-check.rb | 45 ------------------- 1 file changed, 45 deletions(-) delete mode 100755 container/mail-robot/lib/gitee-commit-url-check.rb diff --git a/container/mail-robot/lib/gitee-commit-url-check.rb b/container/mail-robot/lib/gitee-commit-url-check.rb deleted file mode 100755 index af31e17..0000000 --- a/container/mail-robot/lib/gitee-commit-url-check.rb +++ /dev/null @@ -1,45 +0,0 @@ -#!/usr/bin/env ruby -# SPDX-License-Identifier: MulanPSL-2.0+ -# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -# frozen_string_literal: true - -require 'json' -require 'mail' - -# used to check commit available for hub gitee.com -# it is called when: -# - ParseApplyAccountEmail execute commit_url_availability -# - hub is gitee.com -# gitee_commit_check -# clone the repo -# check commit available -class GiteeCommitUrlCheck - def initialize(my_info, url, base_url) - @my_info = my_info - @url = url - @base_url = base_url - end - - def gitee_commit_check - repo_url = [@base_url, 'git'].join('.') - repo_dir = repo_url.split('/')[-1] - commit_id = @url.split('/')[-1] - - Dir.chdir '/tmp' - %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) - - email_index = %x(/usr/bin/git -C #{repo_dir} show #{commit_id}).index @my_info['my_email'] - - FileUtils.rm_rf repo_dir - - gitee_commit_exist(email_index) - end - - def gitee_commit_exist(email_index) - return if email_index - - error_message = 'COMMIT_URL_NOT_AVAILABLE' - - raise error_message - end -end -- 2.23.0

1 0

[PATCH v5 compass-ci 2/3] mail-robot: optimize assign-account-fail-email
by Luan Shengde 03 Feb '21

03 Feb '21

add message for newly added error types. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 242dfe1..3597159 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -33,17 +33,20 @@ def email_err_message(message) https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… EMAIL_MESSAGE - when 'COMMIT_URL_NOT_AVAILABLE' + when 'COMMIT_AUTHOR_ERROR' err_message = <<~EMAIL_MESSAGE - We cannot confirm the commit url matches your email. + We cannot confirm the commit author matches your email. + Make sure it is truely submitted with your email. - Make sure the commit exists. - Make sure it is truely submitted with your email. - - Go to the commit history and find your commit. - Enter the commit and you will get your commit url. + Choose a commit that was submitted with your own email and try again. + EMAIL_MESSAGE + when 'NO_COMMIT_ID' + err_message = <<~EMAIL_MESSAGE + There is no such commit ID in the repo. - Try again after you have correctted the url + Please check the commit ID exists. + or + You have write the commit ID correctly. EMAIL_MESSAGE when 'NO_PUBKEY' err_message = <<~EMAIL_MESSAGE -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] mail-robot: fix bug for parsing email
by Luan Shengde 03 Feb '21

03 Feb '21

[why] for non-gitee repos: we may sometimes can not get the author email from the http request feedback when checking the commit. users may still receive a fail email even if they send 'apply account' email correctly. [how] both gitee and non-gitee will clone the repo tree and check the commit: - check commit id exists - check commit author matches the email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 41 ++++++++++--------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index a41a89a..c7fe9ea 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -6,7 +6,6 @@ require 'json' require 'mail' require 'rest-client' -require_relative 'gitee-commit-url-check' # used to parse mail_content for my_commit_url and my_ssh_pubkey # be called by AssignAccount when it needs to extract required data: @@ -91,29 +90,33 @@ class ParseApplyAccountEmail end def commit_url_availability(url, base_url) - hub_name = url.split('/')[2] - - # it requires authentication when sending request to get the commit information - # clone the repo and then validate the commit for the email address - if hub_name.eql? 'gitee.com' - gitee_commit(url, base_url) - else - non_gitee_commit(url) - end - end + repo_url = [base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] + commit_id = url.split('/')[-1] + + %x(/usr/bin/git -C /tmp clone --bare #{repo_url} #{repo_dir}) + + # get all commit IDs and check commit id exists + repo_commits = %x(/usr/bin/git -C /tmp/#{repo_dir} log --pretty=format:'%H').split(/\n/) + check_commit_exist(commit_id, repo_commits) - def gitee_commit(url, base_url) - my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) - my_gitee_commit.gitee_commit_check + # get the auther's email for the commit + author_email = %x(/usr/bin/git -C /tmp/#{repo_dir} show -s --format=%aE #{commit_id}).chomp + check_commit_email(author_email) + + FileUtils.rm_rf "/tmp/#{repo_dir}" end - def non_gitee_commit(url) - url_fdback = RestClient.get(url).body - email_index = url_fdback.index @my_info['my_email'] + def check_commit_exist(commit_id, repo_commits) + return if repo_commits.include? commit_id.chomp + + raise 'NO_COMMIT_ID' + end - return unless email_index.nil? + def check_commit_email(author_email) + return if author_email.eql? @my_info['my_email'].chomp - raise 'COMMIT_URL_NOT_AVAILABLE' + raise 'COMMIT_AUTHOR_ERROR' end def parse_pub_key -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker: delete duplicate functions code
by Liu Yinsi 03 Feb '21

03 Feb '21

git show d265997c: buildall: avoid repeated building containers in sparrow git show 67ad8b6e: sparrow/4-docker/buildall: prevent duplicate build [why] delete one of the preceding patches because the two patches have the same functions. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 -- 1 file changed, 2 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 727abb9..c838f86 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -25,7 +25,6 @@ do_one() local container=$1 local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock - echo "$started_containers" | grep -w -q "$container_name" && return 0 log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && @@ -42,7 +41,6 @@ do_one() [ "$container_name" == 'initrd-lkp' ] && ./run ) log_info "finish build $container." - started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH compass-ci] sparrow/4-docker: move confusing log_info()
by Liu Yinsi 03 Feb '21

03 Feb '21

[how] before: whether build container or not, all will call log_info(): "start/finish build $container" after: only when actually build container, will call log_info(): "start/finish build $container" [why] if not build container, but echo "start build $container", it's very confusing, mistakenly think that repeatedly build container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index c838f86..c676e6f 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -26,21 +26,24 @@ do_one() local container_name=$(basename $container) lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock - log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" [ "$container_name" == 'ssh-r' ] && exit docker images | grep -wq "$container_name" if [ "$?" != 0 ] || [ "$action" != "run-only" ]; then + + log_info "start build $container." + [ -x build ] && ./build [ -x install ] && ./install + + log_info "finish build $container." fi [ -x first-run ] && ./first-run [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run ) - log_info "finish build $container." lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH lkp-tests] iso2rootfs: add new config_rootfs field: enable_repos
by Yu Chuan 03 Feb '21

03 Feb '21

[Why] Not all sources of one repo file are enabled, sometimes we need to enable all sources to install the specified pkg. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 1 + tests/iso2rootfs | 12 ++++++++++++ 2 files changed, 13 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 2a6455245584..55c4345fad94 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -32,6 +32,7 @@ iso2rootfs: # - dns: will config /etc/resolv.conf. # - no_selinux: will disable selinux in /etc/selinux/config. # - no_fstab: will comment all line in /etc/fstab. + # - enable_repos: will enable all repo file of result rootfs. config_rootfs: ## install pkgs for result rootfs diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 65a1935b7f49..06c8565110ab 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -393,11 +393,23 @@ install_pkgs_for_rootfs() fi } +enable_repos() +{ + echo "${config_rootfs}" | grep -qw 'enable_repos' || return 0 + + ls ${ROOTFS_DES_DIR}/etc/yum.repos.d/*.repo > /dev/null 2>&1 && { + sed -i 's/^enabled=0$/enabled=1/g' ${ROOTFS_DES_DIR}/etc/yum.repos.d/*.repo + } + + return 0 +} + config_rootfs() { config_dns_resolver disable_selinux disable_fstab + enable_repos install_pkgs_for_rootfs } -- 2.23.0

2 2

2025

2024

2023

2022

2021

2020

Compass-ci