- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/2] sparrow/3-code: delete special symbols in vars values
by Liu Yinsi 27 Apr '21

27 Apr '21

[why] special symbols will be parsed in shell command "docker build -t --build-arg $PASSWORD ...", caused failed to build images, so delete. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index b68d9e67..15d9bc45 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -190,7 +190,7 @@ auto-submit: EOF [[ -f /etc/compass-ci/passwd.yaml ]] || { -passwd=$(< /dev/urandom tr -dc "a-zA-Z0-9-=+@#_:/[](){}$%&.^*\!" | head -c 10;echo) +passwd=$(< /dev/urandom tr -dc "a-zA-Z0-9" | head -c 10; echo) cat > /etc/compass-ci/passwd.yaml <<EOF ES_USER: root -- 2.23.0

2 2

[PATCH compass-ci 1/2] sparrow/1-storage: set permission /etc/compass-ci/password.yaml 660
by Liu Yinsi 27 Apr '21

27 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 1 + sparrow/1-storage/tiny | 1 + 2 files changed, 2 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 3c229c5a..3a8bfdaa 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -44,5 +44,6 @@ chgrp team /srv/cci/serial/logs chgrp team /srv/dc chmod g+ws /srv/dc +chmod 660 /etc/compass-ci/password.yaml find /tftpboot -type d | xargs chmod 775 diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index c7636b02..1dbe8a32 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -22,6 +22,7 @@ dirs=( /c /c/git-repos /etc/qemu + /etc/compass-ci/password.yaml /srv/cci/Maildir/.compass-ci/new /srv/cci/Maildir/.compass-ci/cur /srv/cci/Maildir/.compass-ci/sent -- 2.23.0

2 2

[PATCH v2 compass-ci] src/lib/web_backend.rb: filter git_repos
by Li Yuanchao 27 Apr '21

27 Apr '21

To make sure git_repos displayed on the web are right. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 91287b4..066b781 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -403,6 +403,8 @@ def query_repos(must, from: 0, size: 1) repos = [] result['hits']['hits'].each do |r| r = r['_source'] + next unless r['git_repo'] =~ /^([a-z0-9])/ + repos << { git_url: get_repo_url(r['url']), git_repo: r['git_repo'] -- 2.23.0

1 0

[PATCH compass-ci] src/lib/web_backend.rb: filter git_repos
by Li Yuanchao 27 Apr '21

27 Apr '21

To make sure git_repos displayed on the web are right. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- src/lib/web_backend.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 91287b4..452e94a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -403,6 +403,8 @@ def query_repos(must, from: 0, size: 1) repos = [] result['hits']['hits'].each do |r| r = r['_source'] + next unless r['git_repo'] =~ /^([a-z]\/)/ + repos << { git_url: get_repo_url(r['url']), git_repo: r['git_repo'] -- 2.23.0

1 0

[PATCH compass-ci] sbin/es-*-mapping.sh: configuring ES authentication information
by Wu Zhende 27 Apr '21

27 Apr '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/es-regression-mapping.sh | 8 ++++++-- sbin/es-repo-mapping.sh | 8 ++++++-- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/sbin/es-regression-mapping.sh b/sbin/es-regression-mapping.sh index 5ae0f71..9d01bf9 100755 --- a/sbin/es-regression-mapping.sh +++ b/sbin/es-regression-mapping.sh @@ -1,14 +1,18 @@ #!/bin/sh +. $CCI_SRC/container/defconfig.sh + +load_service_authentication + # check whether regression index has created -status_code=$(curl -sSIL -w "%{http_code}\\n" -o /dev/null http://localhost:9200/regression) +status_code=$(curl -sSIL -u "${ES_USER}:${ES_PASSWORD}" -w "%{http_code}\\n" -o /dev/null http://localhost:9200/regression) if [ "$status_code" -eq 200 ] then echo "regression index has been created, exit." else echo "begin create index." - curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/regression' -d '{ + curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/regression' -u "${ES_USER}:${ES_PASSWORD}" -d '{ "mappings": { "dynamic": false, "properties": { diff --git a/sbin/es-repo-mapping.sh b/sbin/es-repo-mapping.sh index 290efb3..368dc94 100755 --- a/sbin/es-repo-mapping.sh +++ b/sbin/es-repo-mapping.sh @@ -2,13 +2,17 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. $CCI_SRC/container/defconfig.sh + +load_service_authentication + # Check if "repo" index already exists. -status_code=$(curl -sSIL -w "%{http_code}\n" -o /dev/null http://localhost:9200/repo) +status_code=$(curl -sSIL -u "${ES_USER}:${ES_PASSWORD}" -w "%{http_code}\n" -o /dev/null http://localhost:9200/repo) [ $status_code -eq 200 ] && echo '"repo" index already exists.' && exit # Create "repo" index. echo 'Start to create "repo" index.' -curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/repo' -d ' +curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/repo' -u "${ES_USER}:${ES_PASSWORD}" -d ' { "mappings": { "properties": { -- 2.23.0

1 0

[PATCH v5 lkp-tests 2/2] lib/rpm.sh: add add_repo function
by Wang Yong 27 Apr '21

27 Apr '21

add function for rpmbuild-pkg and install-rpm Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- lib/rpm.sh | 20 ++++++++++++++++++++ tests/install-rpm | 22 +--------------------- tests/rpmbuild-pkg | 6 ++++++ 3 files changed, 27 insertions(+), 21 deletions(-) create mode 100755 lib/rpm.sh diff --git a/lib/rpm.sh b/lib/rpm.sh new file mode 100755 index 000000000..42bc7f5e0 --- /dev/null +++ b/lib/rpm.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<-EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" + [${custom_repo_name[$i]}] + name=${custom_repo_name[$i]} + baseurl=${custom_repo_addr[$i]} + enabled=1 + gpgcheck=0 + priority=100 + + EOF + done +} diff --git a/tests/install-rpm b/tests/install-rpm index 13beecbee..7449b3bd1 100755 --- a/tests/install-rpm +++ b/tests/install-rpm @@ -9,6 +9,7 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh . $LKP_SRC/lib/log.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" : "${DISTRO:=openeuler}" @@ -19,29 +20,8 @@ [ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" [ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" -custom_repo_name=($custom_repo_name) -custom_repo_addr=($custom_repo_addr) rpm_name=($rpm_name) -add_repo() -{ - custom_repo_name=($custom_repo_name) - custom_repo_addr=($custom_repo_addr) - - for i in ${!custom_repo_name[@]} - do - cat <<-EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" - [${custom_repo_name[$i]}] - name=${custom_repo_name[$i]} - baseurl=${custom_repo_addr[$i]} - enabled=1 - gpgcheck=0 - priority=100 - - EOF - done -} - install_rpm() { log_info "Starting install test: yum install -y ${rpm_name[@]}" diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 312582d16..5c694b9d4 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -3,10 +3,15 @@ # - compat_os # - repo_name # - repo_addr +# - custom_repo_name +# - custom_repo_addr # Git repo jobs only use the first two parameters, SRPMs jobs use all +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" @@ -133,6 +138,7 @@ update_repo_mq() -d "{\"upload_rpms\": ["${full_list}"]}" } +add_repo [ -n "$repo_name" ] && from_srpm || from_git build_rpm upload_rpm_pkg -- 2.23.0

1 0

[PATCH v5 lkp-tests 1/2] tests/install-rpm: add suite install-rpm
by Wang Yong 27 Apr '21

27 Apr '21

add install-rpm to install/remove and ldd test for rpm packages Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/install-rpm | 1 + jobs/install-rpm.yaml | 4 ++ tests/install-rpm | 83 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+) create mode 100644 distro/depends/install-rpm create mode 100755 jobs/install-rpm.yaml create mode 100755 tests/install-rpm diff --git a/distro/depends/install-rpm b/distro/depends/install-rpm new file mode 100644 index 000000000..602479c9a --- /dev/null +++ b/distro/depends/install-rpm @@ -0,0 +1 @@ +dnf-plugins-core diff --git a/jobs/install-rpm.yaml b/jobs/install-rpm.yaml new file mode 100755 index 000000000..9afa9d992 --- /dev/null +++ b/jobs/install-rpm.yaml @@ -0,0 +1,4 @@ +suite: install-rpm +category: functional + +install-rpm: diff --git a/tests/install-rpm b/tests/install-rpm new file mode 100755 index 000000000..13beecbee --- /dev/null +++ b/tests/install-rpm @@ -0,0 +1,83 @@ +#!/bin/bash +# - compat_os +# - rpm_name +# - custom_repo_name +# - custom_repo_addr +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/log.sh + +: "${compat_os:=budding-openeuler}" +: "${DISTRO:=openeuler}" +: "${basearch:=$(arch)}" +. $LKP_SRC/distro/${DISTRO} + +[ -n "${rpm_name}" ] || die "rpm_name is empty" +[ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" +[ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" + +custom_repo_name=($custom_repo_name) +custom_repo_addr=($custom_repo_addr) +rpm_name=($rpm_name) + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<-EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" + [${custom_repo_name[$i]}] + name=${custom_repo_name[$i]} + baseurl=${custom_repo_addr[$i]} + enabled=1 + gpgcheck=0 + priority=100 + + EOF + done +} + +install_rpm() +{ + log_info "Starting install test: yum install -y ${rpm_name[@]}" + yum install -y ${rpm_name[@]} + + return 0 +} + +ldd_test() +{ + log_info "Starting ldd test" + for i in $(rpm -qa ${rpm_name[@]}) + do + ldd_list=$(rpm -ql $i | awk -F'/' '$3 ~ /bin|sbin|lib|lib64/') + [ -z "${ldd_list}" ] || { + for i in ${ldd_list[@]} + do + [[ -f "$i" && -x "$i" ]] && { + echo "ldd for $i" + ldd -d -r $i + } + done + } + done +} + +remove_rpm() +{ + log_info "Starting remove test: yum remove -y ${rpm_name[@]}" + yum remove -y ${rpm_name[@]} + + return 0 +} + +distro_install_depends install-rpm +add_repo +install_rpm +ldd_test +remove_rpm -- 2.23.0

1 0

[PATCH v5 lkp-tests] tests/qcow2rootfs-x86_64: fix failed to build container using absolute path
by Wang Chenglong 27 Apr '21

27 Apr '21

[Error] root@2288hv5-2s44p-384g--b6 /c/lkp-tests/tests# /c/compass-ci/container/qcow2rootfs/build /c/compass-ci/container/qcow2rootfs/build: line 5: ../defconfig.sh: No such file or directory /c/compass-ci/container/qcow2rootfs/build: line 7: docker_skip_rebuild: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/qcow2rootfs-x86_64 | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/tests/qcow2rootfs-x86_64 b/tests/qcow2rootfs-x86_64 index f698fb7ec..a783dc1a6 100755 --- a/tests/qcow2rootfs-x86_64 +++ b/tests/qcow2rootfs-x86_64 @@ -9,12 +9,17 @@ set_qcow2rootfs_tools() { export QCOW2_PATH="/c/compass-ci/container/qcow2rootfs" - git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 + export CCI_SRC="/c/compass-ci" + git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci rm ${QCOW2_PATH}/root/etc/yum.repos.d/* wget -O ${QCOW2_PATH}/root/etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo - ${QCOW2_PATH}/build + cd ${QCOW2_PATH} + ./build || { + echo "build container failed~" + exit 1 + } sed -i 's#$EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file#$EXTRACT_ROOT/bin/extract $EXTRACT_ROOT/qcow2-dir/$QCOW2_NAME $tar_out_file#' \ - ${QCOW2_PATH}/bin/mail + ${QCOW2_PATH}/bin/main } @@ -52,7 +57,13 @@ extract_rootfs() export ROOTFS_DIR="${qcow2_version}-$(date +%Y-%m-%d-%H-%M-%S)" export ROOTFS_PATH="${MOUNT_POINT}/openeuler/${qcow2_arch}/automatic-build/${ROOTFS_DIR}" mkdir -p $ROOTFS_PATH - ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} + cd $CCI_SRC/container/dracut-initrd + ./build || { + echo "build container failed~" + exit 1 + } + cd ${QCOW2_PATH} + ./run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} } set_resolv_config() -- 2.23.0

1 0

[PATCH v4 lkp-tests] lib/bootstrap.sh: fix the output error when ping not exist
by Wang Chenglong 27 Apr '21

27 Apr '21

[Why] If the ping command exists, then run test_ping, if not, then skip. [Error] [ 179.335889] Kernel tests: Boot OK! [ 180.180309] LKP: waiting for network... [ 182.991732] /lkp/lkp/src/lib/bootstrap.sh: line 90: ping: command not found [ 183.010815] /lkp/lkp/src/lib/bootstrap.sh: line 90: ping: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- lib/bootstrap.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 070dabf46..58f0531a7 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -88,6 +88,10 @@ net_devices_link() test_ping() { + has_cmd ping || { + echo "command not found: ping, skip ping test." + return 0 + } ping -c 1 -w 1 $LKP_SERVER > /dev/null } -- 2.23.0

1 0

[PATCH compass-ci] lib/compare_error_messages.rb: fix filename matching rule
by Lin Jiaxin 27 Apr '21

27 Apr '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 93b38ed..7c53580 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -37,7 +37,9 @@ module CEM error_lines = ErrorMessages.new(previous_result_file).obtain_error_messages_by_error_id(error_id, true) error_lines.each do |error_line| # "src/ssl_sock.c:1454:104: warning: unused parameter 'al' [-Wunused-parameter]" => "src/ssl_sock" - filenames << $1.chomp(File.extname($1)) if error_line =~ /(.*)(:\d+){2}: (error|warning):/ + # "/tmp/lkp/bart-git/src/bart/src/num/lapack.c:20:10: fatal error: lapacke.h: No such file or directory" + # => "/tmp/lkp/bart-git/src/bart/src/num/lapack" + filenames << $1.chomp(File.extname($1)) if error_line =~ /(.*)(:\d+){2}: ((fatal )?error|warning):/ end File.open(later_result_file).each_line do |line| -- 2.23.0

1 0

[PATCH compass-ci] assign-account/start: delete useless code
by Wu Zhende 27 Apr '21

27 Apr '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/assign-account/start | 2 -- 1 file changed, 2 deletions(-) diff --git a/container/assign-account/start b/container/assign-account/start index 42b34c2..b926853 100755 --- a/container/assign-account/start +++ b/container/assign-account/start @@ -4,8 +4,6 @@ . ../defconfig.sh -load_service_authentication - docker_rm assign-account cmd=( -- 2.23.0

1 0

[PATCH compass-ci] container/defconfig: fix file does not exist
by Wu Zhende 27 Apr '21

27 Apr '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/defconfig.rb | 5 ++++- container/defconfig.sh | 4 +++- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index 0a8e4d5..ba0e230 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -23,7 +23,10 @@ def relevant_defaults(names) end def relevant_service_authentication(names) - hash = YAML.load_file('/etc/compass-ci/passwd.yaml') || {} + file_name = '/etc/compass-ci/passwd.yaml' + return {} unless File.exist?(file_name) + + hash = YAML.load_file(file_name) || {} hash.select { |k, _| names.include? k } end diff --git a/container/defconfig.sh b/container/defconfig.sh index 64a434f..c217d5f 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -23,7 +23,9 @@ load_cci_defaults() load_service_authentication() { shopt -s nullglob - create_yaml_variables '/etc/compass-ci/passwd.yaml' + file_name='/etc/compass-ci/passwd.yaml' + [ -f $file_name ] || return + create_yaml_variables $file_name } docker_rm() -- 2.23.0

1 0

[PATCH v4 lkp-tests] lib/bootstrap.sh: fix the output error when ping not exist
by Wang Chenglong 27 Apr '21

27 Apr '21

[Why] If the ping command exists, then run test_ping, if not, then skip. [Error] [ 179.335889] Kernel tests: Boot OK! [ 180.180309] LKP: waiting for network... [ 182.991732] /lkp/lkp/src/lib/bootstrap.sh: line 90: ping: command not found [ 183.010815] /lkp/lkp/src/lib/bootstrap.sh: line 90: ping: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- lib/bootstrap.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 070dabf46..58f0531a7 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -88,6 +88,10 @@ net_devices_link() test_ping() { + has_cmd ping || { + echo "command not found: ping, skip ping test." + return 0 + } ping -c 1 -w 1 $LKP_SERVER > /dev/null } -- 2.23.0

2 2

[PATCH v4 lkp-tests 2/2] lib/rpm.sh: add add_repo function
by Wang Yong 27 Apr '21

27 Apr '21

add function for rpmbuild-pkg and install-rpm Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- lib/rpm.sh | 20 ++++++++++++++++++++ tests/install-rpm | 22 +--------------------- tests/rpmbuild-pkg | 6 ++++++ 3 files changed, 27 insertions(+), 21 deletions(-) create mode 100755 lib/rpm.sh diff --git a/lib/rpm.sh b/lib/rpm.sh new file mode 100755 index 000000000..4c4cbc078 --- /dev/null +++ b/lib/rpm.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<-EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" + [${custom_repo_name[$i]}] + name=${custom_repo_name[$i]} + baseurl=${custom_repo_addr[$i]} + enabled=1 + gpgcheck=0 + priority=100 + + EOF +done +} diff --git a/tests/install-rpm b/tests/install-rpm index d69d2d4c1..7449b3bd1 100755 --- a/tests/install-rpm +++ b/tests/install-rpm @@ -9,6 +9,7 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh . $LKP_SRC/lib/log.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" : "${DISTRO:=openeuler}" @@ -19,29 +20,8 @@ [ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" [ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" -custom_repo_name=($custom_repo_name) -custom_repo_addr=($custom_repo_addr) rpm_name=($rpm_name) -add_repo() -{ - custom_repo_name=($custom_repo_name) - custom_repo_addr=($custom_repo_addr) - - for i in ${!custom_repo_name[@]} - do - cat <<EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" -[${custom_repo_name[$i]}] -name=${custom_repo_name[$i]} -baseurl=${custom_repo_addr[$i]} -enabled=1 -gpgcheck=0 -priority=100 - -EOF - done -} - install_rpm() { log_info "Starting install test: yum install -y ${rpm_name[@]}" diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 312582d16..5c694b9d4 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -3,10 +3,15 @@ # - compat_os # - repo_name # - repo_addr +# - custom_repo_name +# - custom_repo_addr # Git repo jobs only use the first two parameters, SRPMs jobs use all +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" @@ -133,6 +138,7 @@ update_repo_mq() -d "{\"upload_rpms\": ["${full_list}"]}" } +add_repo [ -n "$repo_name" ] && from_srpm || from_git build_rpm upload_rpm_pkg -- 2.23.0

2 2

[PATCH v4 lkp-tests 1/2] tests/install-rpm: add suite install-rpm
by Wang Yong 27 Apr '21

27 Apr '21

add install-rpm to install/remove and ldd test for rpm packages Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/install-rpm | 1 + jobs/install-rpm.yaml | 4 ++ tests/install-rpm | 83 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+) create mode 100644 distro/depends/install-rpm create mode 100755 jobs/install-rpm.yaml create mode 100755 tests/install-rpm diff --git a/distro/depends/install-rpm b/distro/depends/install-rpm new file mode 100644 index 000000000..602479c9a --- /dev/null +++ b/distro/depends/install-rpm @@ -0,0 +1 @@ +dnf-plugins-core diff --git a/jobs/install-rpm.yaml b/jobs/install-rpm.yaml new file mode 100755 index 000000000..9afa9d992 --- /dev/null +++ b/jobs/install-rpm.yaml @@ -0,0 +1,4 @@ +suite: install-rpm +category: functional + +install-rpm: diff --git a/tests/install-rpm b/tests/install-rpm new file mode 100755 index 000000000..d69d2d4c1 --- /dev/null +++ b/tests/install-rpm @@ -0,0 +1,83 @@ +#!/bin/bash +# - compat_os +# - rpm_name +# - custom_repo_name +# - custom_repo_addr +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/log.sh + +: "${compat_os:=budding-openeuler}" +: "${DISTRO:=openeuler}" +: "${basearch:=$(arch)}" +. $LKP_SRC/distro/${DISTRO} + +[ -n "${rpm_name}" ] || die "rpm_name is empty" +[ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" +[ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" + +custom_repo_name=($custom_repo_name) +custom_repo_addr=($custom_repo_addr) +rpm_name=($rpm_name) + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" +[${custom_repo_name[$i]}] +name=${custom_repo_name[$i]} +baseurl=${custom_repo_addr[$i]} +enabled=1 +gpgcheck=0 +priority=100 + +EOF + done +} + +install_rpm() +{ + log_info "Starting install test: yum install -y ${rpm_name[@]}" + yum install -y ${rpm_name[@]} + + return 0 +} + +ldd_test() +{ + log_info "Starting ldd test" + for i in $(rpm -qa ${rpm_name[@]}) + do + ldd_list=$(rpm -ql $i | awk -F'/' '$3 ~ /bin|sbin|lib|lib64/') + [ -z "${ldd_list}" ] || { + for i in ${ldd_list[@]} + do + [[ -f "$i" && -x "$i" ]] && { + echo "ldd for $i" + ldd -d -r $i + } + done + } + done +} + +remove_rpm() +{ + log_info "Starting remove test: yum remove -y ${rpm_name[@]}" + yum remove -y ${rpm_name[@]} + + return 0 +} + +distro_install_depends install-rpm +add_repo +install_rpm +ldd_test +remove_rpm -- 2.23.0

2 2

[PATCH v2 compass-ci] sbin: configuring ES authentication information
by Wu Zhende 26 Apr '21

26 Apr '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/es-accounts-mapping.sh | 8 ++++++-- sbin/es-jobs-mapping.sh | 10 +++++++--- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/sbin/es-accounts-mapping.sh b/sbin/es-accounts-mapping.sh index c4ac2fc..580b459 100755 --- a/sbin/es-accounts-mapping.sh +++ b/sbin/es-accounts-mapping.sh @@ -2,15 +2,19 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. $CCI_SRC/container/defconfig.sh + +load_service_authentication + # check whether accounts index has created -status_code=$(curl -sSIL -w "%{http_code}\\n" -o /dev/null http://localhost:9200/accounts) +status_code=$(curl -sSIL -u "${ES_USER}:${ES_PASSWORD}" -w "%{http_code}\\n" -o /dev/null http://localhost:9200/accounts) if [ "$status_code" -eq 200 ] then echo "accounts index has been created, exit." else echo "begin create index." - curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/accounts' -d '{ + curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/accounts' -u "${ES_USER}:${ES_PASSWORD}" -d '{ "mappings": { "dynamic": false, "properties": { diff --git a/sbin/es-jobs-mapping.sh b/sbin/es-jobs-mapping.sh index 67b1ed1..15000d9 100755 --- a/sbin/es-jobs-mapping.sh +++ b/sbin/es-jobs-mapping.sh @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. $CCI_SRC/container/defconfig.sh + +load_service_authentication + # Determine whether curl is installed. If not, install curl. if ! [ -x "$(command -v curl)" ] then @@ -28,14 +32,14 @@ then fi # Determine whether jobs index has created -status_code=$(curl -sSIL -w "%{http_code}\n" -o /dev/null http://localhost:9200/jobs) +status_code=$(curl -sSIL -u "${ES_USER}:${ES_PASSWORD}" -w "%{http_code}\n" -o /dev/null http://localhost:9200/jobs) if [ $status_code -eq 200 ] then echo "jobs index has create, exit." else echo "jobs index not exists, begin create index." - curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/jobs' -d '{ + curl -sSH 'Content-Type: Application/json' -XPUT 'http://localhost:9200/jobs' -u "${ES_USER}:${ES_PASSWORD}" -d '{ "mappings": { "dynamic": false, "dynamic_templates": [ @@ -185,7 +189,7 @@ else echo "create jobs index failed." else echo "set index.mapping.total_fields.limit: 10000" - curl -sS -XPUT 127.0.0.1:9200/jobs/_settings -H 'Content-Type: application/json' \ + curl -sS -XPUT 127.0.0.1:9200/jobs/_settings -u "${ES_USER}:${ES_PASSWORD}" -H 'Content-Type: application/json' \ -d '{"index.mapping.total_fields.limit": 10000}' fi fi -- 2.23.0

2 1

[PATCH compass-ci] container/openresty-proxy-cache: fix failed to git clone
by Liu Yinsi 26 Apr '21

26 Apr '21

[error] /c/compass-ci/container/openresty-proxy-cache# ./build Cloning into 'openresty'... fatal: remote error: access denied or repository not exported: /gitee.com/cuiyili/openresty.git [why] lkp-tests/tests/deploy-ci: ... cat >> /etc/gitconfig <<-EOF [url "git://$GIT_SERVER/gitee.com"] insteadOf=https://gitee.com ... /etc/gitconfig cause failed to git clone https://gitee.com/cuiyili/openresty.git, https -> http to fix this problem. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/openresty-proxy-cache/build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 76652097..3635507f 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -8,6 +8,6 @@ docker_skip_rebuild "openresty:proxy_cache" [ -d "openresty" ] && rm -rf openresty -git clone https://gitee.com/cuiyili/openresty.git || exit 1 +git clone http://gitee.com/cuiyili/openresty.git || exit 1 docker build -t openresty:proxy_cache . -f openresty/Dockerfile -- 2.23.0

2 2

[PATCH compass-ci] container/srv-http: allow user download deps files
by Liu Yinsi 26 Apr '21

26 Apr '21

[why] run-ipconfig.cgz is necessary for user locally deploy compass-ci, when they want to run a qemu, so allow /srv/initrd/deps for download. See [PATCH v5 compass-ci] sbin/download-rootfs: download osimage { ... wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nfs…" ... } Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/srv-http/start-initrd | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start-initrd b/container/srv-http/start-initrd index 457465b..41958da 100755 --- a/container/srv-http/start-initrd +++ b/container/srv-http/start-initrd @@ -15,6 +15,7 @@ cmd=( -v /srv/initrd/dockerimage:/srv/initrd/dockerimage:ro -v /srv/initrd/rpmbuild-pkg:/srv/initrd/rpmbuild-pkg:ro -v /srv/initrd/osimage:/srv/initrd/osimage:ro + -v /srv/initrd/deps:/srv/initrd/deps:ro -v /etc/localtime:/etc/localtime:ro -d srv-http -- 2.23.0

1 0

[PATCH compass-ci] container/mail-robot: fix bug
by Luan Shengde 26 Apr '21

26 Apr '21

add init variable: err_message when execute line: err_message += <<~EMAIL_MESSAGE it may throw error: ERROR -- : exception while processing events: undefined method `+' for nil:NilClass Backtrace: Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-fail-eamil.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 31692358..a7f43f11 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -4,6 +4,7 @@ # frozen_string_literal: true def email_err_message(message) + err_message = '' case message when 'NO_COMMIT_URL' err_message = <<~EMAIL_MESSAGE -- 2.23.0

1 0

[PATCH compass-ci] sbin/download-rootfs: optimise error msg
by Liu Yinsi 26 Apr '21

26 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/download-rootfs b/sbin/download-rootfs index 4edd1952..0d0d1c7b 100755 --- a/sbin/download-rootfs +++ b/sbin/download-rootfs @@ -27,7 +27,7 @@ wget_download() { local download_url=$1 wget $download_url || { - log_error "failed to download os rootfs: $download_url" + log_error "failed to download file: $download_url" exit 1 } } -- 2.23.0

1 0

[PATCH v5 compass-ci] sbin/download-rootfs: download osimage
by Liu Yinsi 26 Apr '21

26 Apr '21

the default os_mount=initramfs, so download osimage to run qemu kvm. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 41 +++++++++++++++++++++++++++-------------- 1 file changed, 27 insertions(+), 14 deletions(-) diff --git a/sbin/download-rootfs b/sbin/download-rootfs index 68a68039..4edd1952 100755 --- a/sbin/download-rootfs +++ b/sbin/download-rootfs @@ -18,23 +18,36 @@ os_version=${3:-20.03} load_cci_defaults -rootfs_dir=/srv/os/${os}/${os_arch}/${os_version} +rootfs_dir="/srv/os/${os}/${os_arch}/${os_version}" +osimage_dir="/srv/initrd/osimage/${os}/${os_arch}/${os_version}" -download_os_rootfs() -{ - mkdir -p $rootfs_dir - cd $rootfs_dir +mkdir -p $rootfs_dir $osimage_dir - download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" - wget $download_url +wget_download() +{ + local download_url=$1 + wget $download_url || { + log_error "failed to download os rootfs: $download_url" + exit 1 + } +} - [ "$?" -ne 0 ] && { - log_error "failed to download os rootfs: $download_url" - exit 1 - } +download_os_rootfs() +{ + cd $rootfs_dir && { + wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz + } +} - gzip -dc ${os_version}.cgz | cpio -id - rm -rf ${os_version}.cgz +download_osimage() +{ + cd $osimage_dir && { + wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/osimage/…" + wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nfs…" + } } is_virtual() @@ -42,4 +55,4 @@ is_virtual() dmidecode -s system-product-name | grep -iq "virtual" } -is_virtual || download_os_rootfs +is_virtual || ( download_os_rootfs && download_osimage ) -- 2.23.0

2 2

[PATCH compass-ci] delimiter: delete unused require
by Cao Xueliang 26 Apr '21

26 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 1 - src/delimiter/utils.rb | 1 - 2 files changed, 2 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index c0c0a60..eaaee78 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -4,7 +4,6 @@ require 'json' require_relative 'error_messages' -require_relative 'sched_client' require_relative '../src/delimiter/utils' require_relative "#{ENV['LKP_SRC']}/lib/monitor" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 5ecbeb5..79bb658 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -8,7 +8,6 @@ require 'yaml' require 'fileutils' require_relative './constants' -require_relative '../../lib/sched_client' require_relative '../../lib/assist_result_client' require_relative '../../lib/compare_error_messages' require_relative "#{ENV['LKP_SRC']}/lib/monitor" -- 2.23.0

1 0

[PATCH v3 lkp-tests] lib/rpm.sh: add add_repo function
by Wang Yong 26 Apr '21

26 Apr '21

add function for rpmbuild-pkg and install-rpm Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- lib/rpm.sh | 20 ++++++++++++++++++++ tests/install-rpm | 22 +--------------------- tests/rpmbuild-pkg | 6 ++++++ 3 files changed, 27 insertions(+), 21 deletions(-) create mode 100755 lib/rpm.sh diff --git a/lib/rpm.sh b/lib/rpm.sh new file mode 100755 index 000000000..4c4cbc078 --- /dev/null +++ b/lib/rpm.sh @@ -0,0 +1,20 @@ +#!/bin/bash + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<-EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" + [${custom_repo_name[$i]}] + name=${custom_repo_name[$i]} + baseurl=${custom_repo_addr[$i]} + enabled=1 + gpgcheck=0 + priority=100 + + EOF +done +} diff --git a/tests/install-rpm b/tests/install-rpm index d69d2d4c1..7449b3bd1 100755 --- a/tests/install-rpm +++ b/tests/install-rpm @@ -9,6 +9,7 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh . $LKP_SRC/lib/log.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" : "${DISTRO:=openeuler}" @@ -19,29 +20,8 @@ [ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" [ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" -custom_repo_name=($custom_repo_name) -custom_repo_addr=($custom_repo_addr) rpm_name=($rpm_name) -add_repo() -{ - custom_repo_name=($custom_repo_name) - custom_repo_addr=($custom_repo_addr) - - for i in ${!custom_repo_name[@]} - do - cat <<EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" -[${custom_repo_name[$i]}] -name=${custom_repo_name[$i]} -baseurl=${custom_repo_addr[$i]} -enabled=1 -gpgcheck=0 -priority=100 - -EOF - done -} - install_rpm() { log_info "Starting install test: yum install -y ${rpm_name[@]}" diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 312582d16..5c694b9d4 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -3,10 +3,15 @@ # - compat_os # - repo_name # - repo_addr +# - custom_repo_name +# - custom_repo_addr # Git repo jobs only use the first two parameters, SRPMs jobs use all +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/rpm.sh : "${compat_os:=budding-openeuler}" @@ -133,6 +138,7 @@ update_repo_mq() -d "{\"upload_rpms\": ["${full_list}"]}" } +add_repo [ -n "$repo_name" ] && from_srpm || from_git build_rpm upload_rpm_pkg -- 2.23.0

1 1

[PATCH v2 lkp-tests 1/2] tests/install-rpm: add suite install-rpm
by Wang Yong 26 Apr '21

26 Apr '21

add install-rpm to install/remove and ldd test for rpm packages Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/install-rpm | 1 + jobs/install-rpm.yaml | 4 ++ tests/install-rpm | 83 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 88 insertions(+) create mode 100644 distro/depends/install-rpm create mode 100755 jobs/install-rpm.yaml create mode 100755 tests/install-rpm diff --git a/distro/depends/install-rpm b/distro/depends/install-rpm new file mode 100644 index 000000000..602479c9a --- /dev/null +++ b/distro/depends/install-rpm @@ -0,0 +1 @@ +dnf-plugins-core diff --git a/jobs/install-rpm.yaml b/jobs/install-rpm.yaml new file mode 100755 index 000000000..9afa9d992 --- /dev/null +++ b/jobs/install-rpm.yaml @@ -0,0 +1,4 @@ +suite: install-rpm +category: functional + +install-rpm: diff --git a/tests/install-rpm b/tests/install-rpm new file mode 100755 index 000000000..d69d2d4c1 --- /dev/null +++ b/tests/install-rpm @@ -0,0 +1,83 @@ +#!/bin/bash +# - compat_os +# - rpm_name +# - custom_repo_name +# - custom_repo_addr +# user can add custom_repo_name and custom_repo_addr to setup +# local repo, value can be an array + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/upload.sh +. $LKP_SRC/lib/log.sh + +: "${compat_os:=budding-openeuler}" +: "${DISTRO:=openeuler}" +: "${basearch:=$(arch)}" +. $LKP_SRC/distro/${DISTRO} + +[ -n "${rpm_name}" ] || die "rpm_name is empty" +[ -n "${custom_repo_name}" ] || die "custom_repo_name is empty" +[ -n "${custom_repo_addr}" ] || die "custom_repo_addr is empty" + +custom_repo_name=($custom_repo_name) +custom_repo_addr=($custom_repo_addr) +rpm_name=($rpm_name) + +add_repo() +{ + custom_repo_name=($custom_repo_name) + custom_repo_addr=($custom_repo_addr) + + for i in ${!custom_repo_name[@]} + do + cat <<EOF >> /etc/yum.repos.d/"${custom_repo_name[$i]}.repo" +[${custom_repo_name[$i]}] +name=${custom_repo_name[$i]} +baseurl=${custom_repo_addr[$i]} +enabled=1 +gpgcheck=0 +priority=100 + +EOF + done +} + +install_rpm() +{ + log_info "Starting install test: yum install -y ${rpm_name[@]}" + yum install -y ${rpm_name[@]} + + return 0 +} + +ldd_test() +{ + log_info "Starting ldd test" + for i in $(rpm -qa ${rpm_name[@]}) + do + ldd_list=$(rpm -ql $i | awk -F'/' '$3 ~ /bin|sbin|lib|lib64/') + [ -z "${ldd_list}" ] || { + for i in ${ldd_list[@]} + do + [[ -f "$i" && -x "$i" ]] && { + echo "ldd for $i" + ldd -d -r $i + } + done + } + done +} + +remove_rpm() +{ + log_info "Starting remove test: yum remove -y ${rpm_name[@]}" + yum remove -y ${rpm_name[@]} + + return 0 +} + +distro_install_depends install-rpm +add_repo +install_rpm +ldd_test +remove_rpm -- 2.23.0

2 2

[PATCH compass-ci] extract-stats: fix no new_error_id
by Cao Xueliang 26 Apr '21

26 Apr '21

regression_client need add auth info. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/extract-stats/regression_client.cr | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/src/extract-stats/regression_client.cr b/src/extract-stats/regression_client.cr index 586ea20..4ff8ab7 100644 --- a/src/extract-stats/regression_client.cr +++ b/src/extract-stats/regression_client.cr @@ -5,7 +5,13 @@ class RegressionClient HOST = (ENV.has_key?("ES_HOST") ? ENV["ES_HOST"] : JOB_ES_HOST) PORT = (ENV.has_key?("ES_PORT") ? ENV["ES_PORT"] : JOB_ES_PORT).to_i32 - def initialize(host = HOST, port = PORT) + def initialize(host = HOST, port = PORT, auth = true) + if auth + user = ENV["ES_USER"]? + password = ENV["ES_PASSWORD"]? + host = "#{user}:#{URI.encode_www_form(password)}@#{host}" if user && password + end + @client = Elasticsearch::API::Client.new({:host => host, :port => port}) end -- 2.23.0

1 0