March 2021 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] sparrow: add /srv/rabbitmq for rabbitmq container
by Cao Xueliang 18 Mar '21

18 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/1-storage/permission | 2 ++ sparrow/1-storage/tiny | 1 + 2 files changed, 3 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 0713d7e..26086d5 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -10,6 +10,7 @@ chmod_dir=( /srv/cci/libvirt-xml /srv/cci/serial/fluentd-pos /srv/cci/Maildir + /srv/rabbitmq ) chmod 775 "${chmod_dir[@]}" @@ -22,6 +23,7 @@ chown_dir=( /srv/cache/netdata_lib /srv/cci/serial/fluentd-pos /srv/cci/libvirt-xml + /srv/rabbitmq ) chown lkp:lkp "${chown_dir[@]}" diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index 8fd3652..9939d93 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -24,6 +24,7 @@ dirs=( /srv/cci/Maildir/.compass-ci/new /srv/cci/Maildir/.compass-ci/cur /srv/cci/Maildir/.compass-ci/sent + /srv/rabbitmq ) mkdir -p "${dirs[@]}" -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker/buildall: fix spelling error
by Liu Yinsi 18 Mar '21

18 Mar '21

before: initrd-lkp after: lkp-initrd Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index a7ace65..3cd5719 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -62,7 +62,7 @@ do_one_run() } [ -x first-run ] && ./first-run [ -x start ] && ./start - [ "$container_name" == 'initrd-lkp' ] && ./run + [ "$container_name" == 'lkp-initrd' ] && ./run ) } -- 2.23.0

1 0

[PATCH v5 lkp-tests] tests/ansible_test: ansible test case
by Wang Chenglong 18 Mar '21

18 Mar '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/ansible_test | 79 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100755 tests/ansible_test diff --git a/tests/ansible_test b/tests/ansible_test new file mode 100755 index 000000000..56926bbed --- /dev/null +++ b/tests/ansible_test @@ -0,0 +1,79 @@ +#!/bin/bash +# - ansible_repo + +. $LKP_SRC/lib/debug.sh + +check_vars() +{ + [ -n "$ansible_repo" ] || die "ansible_repo is empty" + benchmark=${ansible_repo##*/} + + ansible_path=/root/.ansible + [ -d "$ansible_path" ] || mkdir -p /root/.ansible +} + +install_ansible() +{ + if [ "$os" == "openeuler" ]; then + yum -y reinstall git + yum -y install python3 + pip3 install ansible + elif [ "$os" == "centos" ]; then + yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm + yum -y reinstall git + yum -y install ansible + elif [ "$os" == "debian" ]; then + apt-get install -y ansible + fi + + command -v ansible > /dev/null && { + ansible localhost -m ping + [ $? -eq 0 ] || { + echo "Ansible failed to install." + exit 1 + } + } +} + +download_ansible_role() +{ + ansible_roles_path=$ansible_path/roles + [ -d $ansible_roles_path ] || mkdir -p $ansible_roles_path + cd $ansible_roles_path + git clone git://github.com/$ansible_repo.git + if [ $? -eq 0 ]; then + echo "$benchmark is successfully cloned" + else + echo "Please check whether your ansible_repo is correct!" + exit 1 + fi +} + +ansible_playbook_run() +{ + cat <<-EOF > "$ansible_path"/site.yml + - hosts: localhost + roles: + - role: $benchmark + EOF + [ $? -eq 0 ] && { + echo "Playbook has been generated!" + } + ansible-playbook "$ansible_path"/site.yml + + if [ $? -eq 0 ]; then + echo "Your playbook is running successfully!" + else + echo "Your playbook is running failed!" && exit 1 + fi +} + +main() +{ + check_vars + install_ansible + download_ansible_role + ansible_playbook_run +} + +main -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/openeuler_docker: fix to build openeuler image error
by Liu Shaofei 18 Mar '21

18 Mar '21

^[[91m+ rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 ^[[0m^[[91merror: /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7: import read failed(2). ^[[0m Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/openeuler_docker | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/openeuler_docker b/tests/openeuler_docker index b0b5336a3..483f40135 100755 --- a/tests/openeuler_docker +++ b/tests/openeuler_docker @@ -78,6 +78,11 @@ handle_epelrpm() } } +handle_rpm_gpg() +{ + sed -i "s|RPM-GPG-KEY-CentOS-7|RPM-GPG-KEY-openEuler|g" "$dockerfile_name" +} + add_base_commands() { # grep -qw "groupadd*" "$file" && { @@ -95,6 +100,7 @@ handle_dockerfile() is_openeuler_build_fail=1 handle_FROM handle_epel + handle_rpm_gpg add_base_commands if docker_build "-openeuler"; then docker_run "-openeuler" -- 2.23.0

1 0

[PATCH lkp-tests] tests/openeuler_docker: fix to build openeuler image error
by Liu Shaofei 18 Mar '21

18 Mar '21

^[[91m+ rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 ^[[0m^[[91merror: /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7: import read failed(2). ^[[0m Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/openeuler_docker | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/openeuler_docker b/tests/openeuler_docker index b0b5336a3..139b5eb56 100755 --- a/tests/openeuler_docker +++ b/tests/openeuler_docker @@ -78,6 +78,13 @@ handle_epelrpm() } } +handle_rpm_gpg() +{ + grep -qw "RPM-GPG-KEY-CentOS-7" "$dockerfile_name" && { + sed -i "s|RPM-GPG-KEY-CentOS-7|PM-GPG-KEY-openEuler|g" "$dockerfile_name" + } +} + add_base_commands() { # grep -qw "groupadd*" "$file" && { @@ -95,6 +102,7 @@ handle_dockerfile() is_openeuler_build_fail=1 handle_FROM handle_epel + handle_rpm_gpg add_base_commands if docker_build "-openeuler"; then docker_run "-openeuler" -- 2.23.0

3 4

[PATCH compass-ci] fix network problem: add domestic resource url for Debian packages
by Luan Shengde 18 Mar '21

18 Mar '21

[why] the default url for Debian package resources may lead to download timeout problems: E: Release file for http://security.debian.org/debian-security/dists/buster/updates/InRelease is expired (invalid since 9h 8min 10s). Updates for this repository will not be applied. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/Dockerfile | 2 ++ container/mail-robot/conf/sources.list | 9 +++++++++ container/send-internet-mail/Dockerfile | 2 ++ container/send-internet-mail/conf/sources.list | 9 +++++++++ container/send-mail/Dockerfile | 2 ++ container/send-mail/conf/sources.list | 9 +++++++++ 6 files changed, 33 insertions(+) create mode 100644 container/mail-robot/conf/sources.list create mode 100644 container/send-internet-mail/conf/sources.list create mode 100644 container/send-mail/conf/sources.list diff --git a/container/mail-robot/Dockerfile b/container/mail-robot/Dockerfile index 4402543..e4b3e7c 100644 --- a/container/mail-robot/Dockerfile +++ b/container/mail-robot/Dockerfile @@ -7,6 +7,8 @@ MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive +COPY conf/sources.list* /etc/apt/ + RUN apt-get update && \ apt-get install -y git uuid-runtime ruby-listen ruby-json ruby-mail ruby-rest-client diff --git a/container/mail-robot/conf/sources.list b/container/mail-robot/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/mail-robot/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile index 29f1bc6..fa9ec2c 100644 --- a/container/send-internet-mail/Dockerfile +++ b/container/send-internet-mail/Dockerfile @@ -5,5 +5,7 @@ FROM debian MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive +COPY conf/sources.list* /etc/apt/ + RUN apt-get update && \ apt-get install -y ruby-mail ruby-json ruby-sinatra ruby-redis ruby-rufus-scheduler diff --git a/container/send-internet-mail/conf/sources.list b/container/send-internet-mail/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/send-internet-mail/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib diff --git a/container/send-mail/Dockerfile b/container/send-mail/Dockerfile index 29f1bc6..fa9ec2c 100644 --- a/container/send-mail/Dockerfile +++ b/container/send-mail/Dockerfile @@ -5,5 +5,7 @@ FROM debian MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive +COPY conf/sources.list* /etc/apt/ + RUN apt-get update && \ apt-get install -y ruby-mail ruby-json ruby-sinatra ruby-redis ruby-rufus-scheduler diff --git a/container/send-mail/conf/sources.list b/container/send-mail/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/send-mail/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH compass-ci] dracut-initrd: add xz-utils
by Wang Chenglong 18 Mar '21

18 Mar '21

[Why] wcl@crystal ~/compass-ci/container/dracut-initrd% ./run /srv/os/centos/x86_64/6.10/lib/modules/3.10.0-957.1.3.el7.x86_64 dracut: Executing: /usr/bin/dracut -o iscsi --omit-drivers iscsi --force --kver 3.10.0-957.1.3.el7.x86_64 -k /lib/modules/3.10.0-957.1.3.el7.x86_64 /boot/initramfs.lkp-3.10.0-957.1.3.el7.x86_64.img dracut: Kernel modules are compressed with xz, but xz is not available. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 616183f..b8eb6f5 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -6,7 +6,7 @@ export DEBIAN_FRONTEND=noninteractive apt-get update apt-get install -y --fix-missing --no-install-recommends -qq apt-utils \ -nfs-common netbase cifs-utils kmod rsync dracut dracut-network \ +nfs-common netbase cifs-utils kmod rsync dracut dracut-network xz-utils \ dracut-config-generic apt-get clean -- 2.23.0

2 1

[PATCH v3 lkp-tests] stat/ansible_test: stats result for ansible_test
by Wang Chenglong 18 Mar '21

18 Mar '21

The result of ansible_test are counted and recorded in the ES. query it with the 'es-jobs' command. sum.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng: 1 raw.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng.message: ["{\"changed\": false, \"msg\": \"Unsupported pkg mgr. This role is depends on apt, yum, or pkgng.\"}"] Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 47 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..d6acad1ab 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,40 @@ #!/usr/bin/env ruby -while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line +require 'json' + +def output_error(error_msg) + return if error_msg.nil? + + if error_msg.is_a? Array + error_msg.each do |i| + error_id = common_error_id i + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" + end + elsif error_msg.is_a? String + error_id = common_error_id error_msg + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" end end + +def common_error_id(line) + line.gsub!(/[^\w]/, '-') + line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one + line.gsub!(/^-|-$/, '') + line +end + +while (line = STDIN.gets) + next unless line =~ /(FAILED!|failed:).*=>(.*)/ + + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? + + ansible_failed_json = JSON.parse @ansible_failed_info + + output_error ansible_failed_json['msg'] + output_error ansible_failed_json['message'] + output_error ansible_failed_json['cmd'] + output_error ansible_failed_json['failures'] +end -- 2.23.0

1 0

[PATCH compass-ci] container/crystal-shards: replace https:// with git:// for building image
by Cao Xueliang 17 Mar '21

17 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/crystal-shards/Dockerfile | 7 +++++-- container/crystal-shards/git | 7 +++++++ 2 files changed, 12 insertions(+), 2 deletions(-) create mode 100755 container/crystal-shards/git diff --git a/container/crystal-shards/Dockerfile b/container/crystal-shards/Dockerfile index 2311554..15fe523 100644 --- a/container/crystal-shards/Dockerfile +++ b/container/crystal-shards/Dockerfile @@ -5,7 +5,10 @@ FROM alpine:crystal-base MAINTAINER chief <taxcom(a)tom.com> -copy shard.yml /usr/share/crystal/app/shard.yml +COPY git /usr/local/sbin/git +RUN echo -e "[url \"git://\"]\n\tinsteadOf = https://" >> /etc/gitconfig + +COPY shard.yml /usr/share/crystal/app/shard.yml WORKDIR /usr/share/crystal/app RUN shards @@ -14,7 +17,7 @@ sed -i '99s/arguments\[:id]/arguments\[:id]?/' /usr/share/crystal/app/lib/elast sed -i 's/, Utils.__listify(arguments\[:q].as(String))/ /' /usr/share/crystal/app/lib/elasticsearch-crystal/src/elasticsearch/api/actions/search.cr;\ sed -i '205a \ params.clear' /usr/share/crystal/app/lib/elasticsearch-crystal/src/elasticsearch/api/actions/search.cr; -copy shard-amqp.yml /usr/share/crystal/app/shard.yml +COPY shard-amqp.yml /usr/share/crystal/app/shard.yml RUN shards CMD ["bash"] diff --git a/container/crystal-shards/git b/container/crystal-shards/git new file mode 100755 index 0000000..7bb0b85 --- /dev/null +++ b/container/crystal-shards/git @@ -0,0 +1,7 @@ +#!/bin/bash + +if [[ "$*" = "ls-remote --get-url origin" ]]; then + /usr/bin/git "$@" | sed 's|git://|https://|g' +else + /usr/bin/git "$@" +fi -- 2.23.0

1 0

[PATCH compass-ci] container/crystal-shards: delete gitcache host and port
by Cao Xueliang 17 Mar '21

17 Mar '21

The gitcache service cannot be used to maintain branches when build crystal-shards images. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/crystal-shards/Dockerfile | 4 ---- container/crystal-shards/build | 2 +- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/container/crystal-shards/Dockerfile b/container/crystal-shards/Dockerfile index bdc5b66..2311554 100644 --- a/container/crystal-shards/Dockerfile +++ b/container/crystal-shards/Dockerfile @@ -5,10 +5,6 @@ FROM alpine:crystal-base MAINTAINER chief <taxcom(a)tom.com> -ARG GITCACHE_HOST -ARG GITCACHE_PORT -RUN [ -z "$GITCACHE_HOST" ] || echo -e "[url \"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig - copy shard.yml /usr/share/crystal/app/shard.yml WORKDIR /usr/share/crystal/app diff --git a/container/crystal-shards/build b/container/crystal-shards/build index 7c2a712..00153ab 100755 --- a/container/crystal-shards/build +++ b/container/crystal-shards/build @@ -11,4 +11,4 @@ docker_skip_rebuild "alpine:crystal-shards" load_cci_defaults -docker build --build-arg GITCACHE_HOST=$GITCACHE_HOST --build-arg GITCACHE_PORT=$GITCACHE_PORT -t alpine:crystal-shards . +docker build -t alpine:crystal-shards . -- 2.23.0

1 0