- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] 0-package/common: change the value of fluentd port
by Wu Zhende 26 Oct '20

26 Oct '20

[Why] Prior to this, fluentd used port 24224 to collection docker logs. But now, fluentd change to sub-fluentd, and uses 24225. master-fluentd uses port 24224 to get docker logs from sub-fluentd. So docker should forward logs to sub-fluentd and uses 24225 port. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/0-package/common | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/0-package/common b/sparrow/0-package/common index 31588f4..523438c 100755 --- a/sparrow/0-package/common +++ b/sparrow/0-package/common @@ -31,7 +31,7 @@ cat > /etc/docker/daemon.json <<EOF { "log-driver": "fluentd", "log-opts":{ - "fluentd-address": "localhost:24224", + "fluentd-address": "localhost:24225", "fluentd-async-connect": "true", "tag": "{{.Name}}" }, -- 2.23.0

1 0

[PATCH lkp-tests] fix(adaptation/*): update adaptation for several OS
by Wang Yong 26 Oct '20

26 Oct '20

[Why] some packages in other OS have different name or some Debian packages no need add mapping in red-hat related OS. Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/centos | 10 ++++++++-- distro/adaptation/fedora | 6 +++++- distro/adaptation/kylin | 10 ++++++++++ distro/adaptation/openeuler | 9 +++++++++ distro/adaptation/oracle | 4 +++- 5 files changed, 35 insertions(+), 4 deletions(-) diff --git a/distro/adaptation/centos b/distro/adaptation/centos index f92cee2ff..e7fabf36d 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -8,7 +8,6 @@ build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup clang-6.0: clang cmake: cmake cmake3 -dbench=4.0-2: dbench dnsutils: bind-utils dpkg-dev: dpkg-devel e2fslibs-dev: e2fsprogs-devel @@ -297,7 +296,6 @@ libtasn1-6: libtasn1 libtimedate-perl: perl-TimeDate libtirpc-common: libtirpc libtirpc3: libtirpc -network-manager: network-manager-applet libv4l-dev: libv4l-devel libvirt-clients: libvirt-client libvirt-daemon-system: libvirt-daemon @@ -328,3 +326,11 @@ dh-python: dh-strip-nondeterminism: dh-systemd: guile-2.0: guile +libpython2.7: python-libs +network-manager: NetworkManager +init-system-helpers: +javahelper: +junit4: junit +liboath-dev: liboath-devel +lsb-release: redhat-lsb +linux-base: diff --git a/distro/adaptation/fedora b/distro/adaptation/fedora index dd274d0f1..38b7e9dc4 100644 --- a/distro/adaptation/fedora +++ b/distro/adaptation/fedora @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel freeglut3: freeglut fsmark: fs_mark @@ -158,3 +157,8 @@ libslang2-dev: slang-devel python-dev: python3-devel systemtap-sdt-dev: systemtap-sdt-devel initramfs-tools: +init-system-helpers: +network-manager: NetworkManager +javahelper: +junit4: junit +linux-base: diff --git a/distro/adaptation/kylin b/distro/adaptation/kylin index c60a7f3ee..f143fde96 100644 --- a/distro/adaptation/kylin +++ b/distro/adaptation/kylin @@ -137,3 +137,13 @@ uuid-runtime: uuid xfslibs-dev: xfsprogs-devel xutils-dev: imake xorg-x11-util-macros zlib1g-dev: zlib-devel +libpython2.7: +network-manager: NetworkManager +wpasupplicant: wpa_supplicant +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel +linux-base: diff --git a/distro/adaptation/openeuler b/distro/adaptation/openeuler index ca8688b5a..e72f34fd2 100644 --- a/distro/adaptation/openeuler +++ b/distro/adaptation/openeuler @@ -284,3 +284,12 @@ guile-2.0: guile libmysqlclient-dev: mysql-devel libipc-run-perl: libklibc-dev: +libpython2.7: +network-manager: NetworkManager +gdbserver: gdb-gdbserver +init-system-helpers: +javahelper: +lsb-release: +openjdk: java-1.8.0-openjdk +ruby-full: ruby ruby-devel rubygem-rdoc +linux-base: diff --git a/distro/adaptation/oracle b/distro/adaptation/oracle index f7fcfaa84..af8415ec3 100644 --- a/distro/adaptation/oracle +++ b/distro/adaptation/oracle @@ -5,7 +5,6 @@ binutils-dev: binutils-devel btrfs-tools: btrfs-progs build-essential: make automake gcc gcc-c++ kernel-uek-devel rpm-build cgroup-bin: libcgroup -dbench=4.0-2: dbench dpkg-dev: dpkg-devel f2fs-tools: freeglut3: freeglut @@ -133,3 +132,6 @@ xutils-dev: imake xz-utils: xz zlib1g-dev: zlib-devel initramfs-tools: +init-system-helpers: +javahelper: +linux-base: -- 2.23.0

1 0

[PATCH lkp-tests 2/2] fix(distro/depends): move i386 depends to x86_64
by Wang Yong 26 Oct '20

26 Oct '20

[Why] i386 depends are not suitable for aarch64 [How] move these packages to x86_64 file for further support: commit 5b5f5d2b2e1948555dd01c12dc31e77442a542b8 depends: auto load $program.$(arch) Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/kernel-selftests | 22 ---------------------- distro/depends/kernel-selftests.x86_64 | 23 +++++++++++++++++++++++ 2 files changed, 23 insertions(+), 22 deletions(-) create mode 100644 distro/depends/kernel-selftests.x86_64 diff --git a/distro/depends/kernel-selftests b/distro/depends/kernel-selftests index 3d08c85da..b02e37e22 100644 --- a/distro/depends/kernel-selftests +++ b/distro/depends/kernel-selftests @@ -2,21 +2,8 @@ gcc-8 gcc-8-multilib libc6-dev libc-bin -libc6-dev-i386 (x86_64) -libc6-dev-x32 libc6/unstable -lib32gcc-8-dev -libx32gcc-8-dev -libc6-i386 libc6-dev -libc6-x32 -libx32gcc1 -libx32gomp1 -libx32itm1 -libx32atomic1 -libx32asan5 -libx32ubsan1 -libx32quadmath0 rsync make libcap-ng-dev @@ -55,12 +42,3 @@ sendip ipvsadm libmnl-dev libkeyutils-dev -lib64gcc1 (i386) -lib64gomp1 (i386) -lib64itm1 (i386) -lib64atomic1 (i386) -lib64asan3 (i386) -libc6-amd64/unstable (i386) -libc6-dev-amd64/unstable (i386) -lib64gcc-8-dev (i386) -libx32gcc-8-dev (i386) diff --git a/distro/depends/kernel-selftests.x86_64 b/distro/depends/kernel-selftests.x86_64 new file mode 100644 index 000000000..9491ac3b5 --- /dev/null +++ b/distro/depends/kernel-selftests.x86_64 @@ -0,0 +1,23 @@ +libc6-dev-i386 (x86_64) +libc6-dev-x32 +libc6-i386 +libc6-x32 +libx32gcc1 +libx32gomp1 +libx32itm1 +libx32atomic1 +libx32asan5 +libx32ubsan1 +libx32quadmath0 +lib64gcc1 (i386) +lib64gomp1 (i386) +lib64itm1 (i386) +lib64atomic1 (i386) +lib64asan3 (i386) +libc6-amd64/unstable (i386) +libc6-dev-amd64/unstable (i386) +lib64gcc-8-dev (i386) +libx32gcc-8-dev (i386) +lib32gcc-8-dev +libx32gcc-8-dev + -- 2.23.0

1 0

[PATCH v7 compass-ci 6/6] container/send-internet-mail: send-mail.md
by Luan Shengde 26 Oct '20

26 Oct '20

documentation for send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- doc/send-mail.md | 58 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 doc/send-mail.md diff --git a/doc/send-mail.md b/doc/send-mail.md new file mode 100644 index 0000000..63716d7 --- /dev/null +++ b/doc/send-mail.md @@ -0,0 +1,58 @@ + +# send mail + +## purpose + +The service is used to send mails with formatted mail data +- send intranet mail +- send internet mail + +## send mail port +- send intranet mail: 11311 +- send internet mail: 49000 + +## send mail host +- host the service running on + +## data format +- send_mail_yaml + - data format + data='{ + "subject": "email subject", + "to": "email_to_addr", + "body": "email message" + }' + or + data=" + subject: email subject + to: email_to_addr + body: email message + " + +- send_mail_text + - data format + data=" + To: email_to_addr + Subject: email_subject + + mail_msg_line1 + mail_msg_line2 + ... + " + +## usage: +- send_mail_yaml + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_yaml' -d "$data" +- send_mail_text + curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "$data" + +## example + + data = " + To: test_email(a)163.com + Subject: test mail 10 + + test msg 1010 + " + + %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") -- 2.23.0

1 0

[PATCH v7 compass-ci 4/6] container/send-internet-mail: build
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

1 0

[PATCH v7 compass-ci 3/6] container/send-internet-mail: Dockerfile
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..d151bda --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v7 compass-ci 2/6] container/send-internet-mail: send-internet-mail.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp config for internet email server invoke mail-post to hand the send mail request Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- .../send-internet-mail/send-internet-mail.rb | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 container/send-internet-mail/send-internet-mail.rb diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb new file mode 100644 index 0000000..3b1d159 --- /dev/null +++ b/container/send-internet-mail/send-internet-mail.rb @@ -0,0 +1,21 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

1 0

[PATCH v7 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..5832bfb --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,55 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

1 0

[PATCH v2 compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 26 Oct '20

26 Oct '20

[Why] Two things here i do: 1) alter 'node_macs' to 'direct_macs'; 2) add one field to jobs, 'direct_ips'. Because lkp use those 2 fields to bind 'direct_ips' to 'direct_macs', after this, nodes can communicate with each other when running cluster job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 692180f..c32091f 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,16 @@ class Sched # collect all job ids job_ids = [] of String + net_id = "192.168.222" + ip0 = cluster_config["ip0"]? + if ip0 + ip0 = ip0.as_i + else + ip0 = 1 + end + # steps for each host - cluster_config.each do |host, config| + cluster_config["nodes"].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +227,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 26 Oct '20

26 Oct '20

[WHY] the path of *.cgz file generated by build-pkg script cannot be obtained dring test. So the path of cgz need to be changed. [HOW] below is example for openeuler: /srv/initrd/build-pkg/ => /srv/initrd/build-pkg/initramfs/openeuler/aarch64/20.03/ Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..eca09e64 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,6 +2,7 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url @@ -21,10 +22,14 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" @@ -75,7 +80,7 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 } -- 2.23.0

3 5