October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] sparrow: fix containers auto resume failed after reboot
by Liu Yinsi 26 Oct '20

26 Oct '20

call 4-docker/buildall to fix container auto resume depends issues. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 26 +++++++++++++++++--------- sparrow/7-systemd/cci-network | 9 +++++---- 2 files changed, 22 insertions(+), 13 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..26778a4 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -3,6 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. CONTAINER_PATH="$CCI_SRC/container" +export start_only=$1 build_depends() { @@ -30,20 +31,27 @@ do_one() ( cd $container [ "$container_name" == 'sub-fluentd' ] && exit - [ -x build ] && ./build - [ -x install ] && ./install + if [ -z "$start_only" ]; then + [ -x build ] && ./build + [ -x install ] && ./install + fi [ -x first-run ] && ./first-run [ -x start ] && ./start ) lockfile-remove --lock-name "$container_name".lock } -tmpdir=$(mktemp -d) +buildall() +{ + tmpdir=$(mktemp -d) -for dir in $CONTAINER_PATH/*/ -do - build_depends $dir -done + for dir in $CONTAINER_PATH/*/ + do + build_depends $dir + done + + wait + rm -fr $tmpdir +} -wait -rm -fr $tmpdir +buildall diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 4aadcdc..0d21a2d 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -4,13 +4,14 @@ #this file used in our own systemd unit file: cci-network.service. #contents as follows will be automatically executed after system reboot. -[[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci +. $CCI_SRC/container/defconfig.sh + +# export ENV +. /etc/profile.d/compass.sh $CCI_SRC/sparrow/2-network/br0 $CCI_SRC/sparrow/2-network/iptables $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs -# --restart=always option is not absolutely reliable -containers=($(docker ps -a |grep -v NAMES |awk '{print $NF}')) -[ -n "$containers" ] && docker start "${containers[@]}" >/dev/null 2>&1 +$CCI_SRC/sparrow/4-docker/buildall start_only -- 2.23.0

1 0

[PATCH lkp-tests] lib/monitor.rb: add stop reason
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] When the client disconnects from the ws, the reason is not printed. Like this: connect to ws://localhost:11310/filter ............. connection closed: Add stop reason, and it will be like: connect to ws://localhost:11310/filter .......... connection closed: timeout or connection closed: normal Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index 9ce1d475..c20794e8 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -27,6 +27,7 @@ class Monitor load_default @result = [] @stop_query = {} + @reason = nil end def load_default @@ -87,16 +88,12 @@ class Monitor ssh root@#{ssh_host} -p #{ssh_port} -o StrictHostKeyChecking=no" end - def stop_em(web_socket) - web_socket.close - EM.stop - end - - def stop(data, web_socket) + def stop(data, web_socket, code = 1000, reason = 'normal') @stop_query.each do |key, value| return false unless data[key] == value end - stop_em(web_socket) + @reason = reason + web_socket.close(code, reason) end def run(timeout = nil) @@ -111,7 +108,8 @@ class Monitor if timeout && timeout != 0 EM.add_timer(timeout) do @exit_status_code = 1 - stop_em(ws) + @reason = 'timeout' + ws.close(1000, @reason) end end @@ -131,7 +129,9 @@ class Monitor end ws.on :close do |event| - puts "connection closed: #{event.reason}" + reason = event.reason || @reason + puts "connection closed: #{reason}" + EM.stop end end return @exit_status_code -- 2.23.0

2 1

[PATCH lkp-tests 1/2] fix(distro/depends): delete unnecessary depends
by Wang Yong 26 Oct '20

26 Oct '20

[Why] ruby-full contains ruby-dev $ apt depends ruby-full ruby-full Depends: ruby Depends: ruby-dev Depends: ri [How] delete this package from depend file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/cci-deploy | 1 - 1 file changed, 1 deletion(-) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index dac682505..c797c1eea 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -4,7 +4,6 @@ g++ make docker.io ruby-full -ruby-dev bridge-utils qemu dmidecode -- 2.23.0

2 2

[PATCH compass-ci] 0-package/common: change the value of fluentd port
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] Prior to this, fluentd used port 24224 to collection docker logs. But now, fluentd change to sub-fluentd, and uses 24225. master-fluentd uses port 24224 to get docker logs from sub-fluentd. So docker should forward logs to sub-fluentd and uses 24225 port. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/0-package/common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 31588f4..523438c 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -31,7 +31,7 @@ cat > /etc/docker/daemon.json <<EOF { "log-driver": "fluentd", "log-opts":{ - "fluentd-address": "localhost:24224", + "fluentd-address": "localhost:24225", "fluentd-async-connect": "true", "tag": "{{.Name}}" }, -- 2.23.0

1 0

[PATCH lkp-tests] fix(adaptation/*): update adaptation for several OS
by Wang Yong 26 Oct '20

26 Oct '20

[Why] some packages in other OS have different name or some Debian packages no need add mapping in red-hat related OS. Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/centos | 10 ++++++++-- distro/adaptation/fedora | 6 +++++- distro/adaptation/kylin | 10 ++++++++++ distro/adaptation/openeuler | 9 +++++++++ distro/adaptation/oracle | 4 +++- 5 files changed, 35 insertions(+), 4 deletions(-) diff --git a/distro/adaptation/centos b/distro/adaptation/centos index f92cee2ff..e7fabf36d 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -8,7 +8,6 @@ build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup clang-6.0: clang cmake: cmake cmake3 -dbench=4.0-2: dbench dnsutils: bind-utils dpkg-dev: dpkg-devel e2fslibs-dev: e2fsprogs-devel @@ -297,7 +296,6 @@ libtasn1-6: libtasn1 libtimedate-perl: perl-TimeDate libtirpc-common: libtirpc libtirpc3: libtirpc -network-manager: network-manager-applet libv4l-dev: libv4l-devel libvirt-clients: libvirt-client libvirt-daemon-system: libvirt-daemon @@ -328,3 +326,11 @@ dh-python: dh-strip-nondeterminism: dh-systemd: guile-2.0: guile +libpython2.7: python-libs +network-manager: NetworkManager +init-system-helpers: +javahelper: +junit4: junit +liboath-dev: liboath-devel +lsb-release: redhat-lsb +linux-base: diff --git a/distro/adaptation/fedora b/distro/adaptation/fedora index dd274d0f1..38b7e9dc4 100644 --- a/distro/adaptation/fedora +++ b/distro/adaptation/fedora @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel freeglut3: freeglut fsmark: fs_mark @@ -158,3 +157,8 @@ libslang2-dev: slang-devel python-dev: python3-devel systemtap-sdt-dev: systemtap-sdt-devel initramfs-tools: +init-system-helpers: +network-manager: NetworkManager +javahelper: +junit4: junit +linux-base: diff --git a/distro/adaptation/kylin b/distro/adaptation/kylin index c60a7f3ee..f143fde96 100644 --- a/distro/adaptation/kylin +++ b/distro/adaptation/kylin @@ -137,3 +137,13 @@ uuid-runtime: uuid xfslibs-dev: xfsprogs-devel xutils-dev: imake xorg-x11-util-macros zlib1g-dev: zlib-devel +libpython2.7: +network-manager: NetworkManager +wpasupplicant: wpa_supplicant +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel +linux-base: diff --git a/distro/adaptation/openeuler b/distro/adaptation/openeuler index ca8688b5a..e72f34fd2 100644 --- a/distro/adaptation/openeuler +++ b/distro/adaptation/openeuler @@ -284,3 +284,12 @@ guile-2.0: guile libmysqlclient-dev: mysql-devel libipc-run-perl: libklibc-dev: +libpython2.7: +network-manager: NetworkManager +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel rubygem-rdoc +linux-base: diff --git a/distro/adaptation/oracle b/distro/adaptation/oracle index f7fcfaa84..af8415ec3 100644 --- a/distro/adaptation/oracle +++ b/distro/adaptation/oracle @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-uek-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel f2fs-tools: freeglut3: freeglut @@ -133,3 +132,6 @@ xutils-dev: imake xz-utils: xz zlib1g-dev: zlib-devel initramfs-tools: +init-system-helpers: +javahelper: +linux-base: -- 2.23.0

1 0

[PATCH lkp-tests 2/2] fix(distro/depends): move i386 depends to x86_64
by Wang Yong 26 Oct '20

26 Oct '20

[Why] i386 depends are not suitable for aarch64 [How] move these packages to x86_64 file for further support: commit 5b5f5d2b2e1948555dd01c12dc31e77442a542b8 depends: auto load $program.$(arch) Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/kernel-selftests | 22 ---------------------- distro/depends/kernel-selftests.x86_64 | 23 +++++++++++++++++++++++ 2 files changed, 23 insertions(+), 22 deletions(-) create mode 100644 distro/depends/kernel-selftests.x86_64 diff --git a/distro/depends/kernel-selftests b/distro/depends/kernel-selftests index 3d08c85da..b02e37e22 100644 --- a/distro/depends/kernel-selftests +++ b/distro/depends/kernel-selftests @@ -2,21 +2,8 @@ gcc-8 gcc-8-multilib libc6-dev libc-bin -libc6-dev-i386 (x86_64) -libc6-dev-x32 libc6/unstable -lib32gcc-8-dev -libx32gcc-8-dev -libc6-i386 libc6-dev -libc6-x32 -libx32gcc1 -libx32gomp1 -libx32itm1 -libx32atomic1 -libx32asan5 -libx32ubsan1 -libx32quadmath0 rsync make libcap-ng-dev @@ -55,12 +42,3 @@ sendip ipvsadm libmnl-dev libkeyutils-dev -lib64gcc1 (i386) -lib64gomp1 (i386) -lib64itm1 (i386) -lib64atomic1 (i386) -lib64asan3 (i386) -libc6-amd64/unstable (i386) -libc6-dev-amd64/unstable (i386) -lib64gcc-8-dev (i386) -libx32gcc-8-dev (i386) diff --git a/distro/depends/kernel-selftests.x86_64 b/distro/depends/kernel-selftests.x86_64 new file mode 100644 index 000000000..9491ac3b5 --- /dev/null +++ b/distro/depends/kernel-selftests.x86_64 @@ -0,0 +1,23 @@ +libc6-dev-i386 (x86_64) +libc6-dev-x32 +libc6-i386 +libc6-x32 +libx32gcc1 +libx32gomp1 +libx32itm1 +libx32atomic1 +libx32asan5 +libx32ubsan1 +libx32quadmath0 +lib64gcc1 (i386) +lib64gomp1 (i386) +lib64itm1 (i386) +lib64atomic1 (i386) +lib64asan3 (i386) +libc6-amd64/unstable (i386) +libc6-dev-amd64/unstable (i386) +lib64gcc-8-dev (i386) +libx32gcc-8-dev (i386) +lib32gcc-8-dev +libx32gcc-8-dev + -- 2.23.0

1 0

[PATCH v7 compass-ci 6/6] container/send-internet-mail: send-mail.md
by Luan Shengde 26 Oct '20

26 Oct '20

documentation for send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/send-mail.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/send-mail.md diff --git a/doc/send-mail.md b/doc/send-mail.md new file mode 100644 index 0000000..63716d7 --- /dev/null +++ b/doc/send-mail.md @@ -0,0 +1,58 @@ + +# send mail + +## purpose + +The service is used to send mails with formatted mail data +- send intranet mail +- send internet mail + +## send mail port +- send intranet mail: 11311 +- send internet mail: 49000 + +## send mail host +- host the service running on + +## data format +- send_mail_yaml + - data format + data='{ + "subject": "email subject", + "to": "email_to_addr", + "body": "email message" + }' + or + data=" + subject: email subject + to: email_to_addr + body: email message + " + +- send_mail_text + - data format + data=" + To: email_to_addr + Subject: email_subject + + mail_msg_line1 + mail_msg_line2 + ... + " + +## usage: +- send_mail_yaml + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_yaml' -d "$data" +- send_mail_text + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "$data" + +## example + + data = " + To: test_email(a)163.com + Subject: test mail 10 + + test msg 1010 + " + + %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") -- 2.23.0

1 0

[PATCH v7 compass-ci 4/6] container/send-internet-mail: build
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

1 0

[PATCH v7 compass-ci 3/6] container/send-internet-mail: Dockerfile
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..d151bda --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v7 compass-ci 2/6] container/send-internet-mail: send-internet-mail.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp config for internet email server invoke mail-post to hand the send mail request Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- .../send-internet-mail/send-internet-mail.rb | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 container/send-internet-mail/send-internet-mail.rb diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb new file mode 100644 index 0000000..3b1d159 --- /dev/null +++ b/container/send-internet-mail/send-internet-mail.rb @@ -0,0 +1,21 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

1 0