- Compass-ci - mailweb.openeuler.org

[PATCH v7 compass-ci 4/6] container/send-internet-mail: build
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

1 0

[PATCH v7 compass-ci 3/6] container/send-internet-mail: Dockerfile
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..d151bda --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v7 compass-ci 2/6] container/send-internet-mail: send-internet-mail.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp config for internet email server invoke mail-post to hand the send mail request Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- .../send-internet-mail/send-internet-mail.rb | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) create mode 100644 container/send-internet-mail/send-internet-mail.rb diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb new file mode 100644 index 0000000..3b1d159 --- /dev/null +++ b/container/send-internet-mail/send-internet-mail.rb @@ -0,0 +1,21 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

1 0

[PATCH v7 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..5832bfb --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,55 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

1 0

[PATCH v2 compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 26 Oct '20

26 Oct '20

[Why] Two things here i do: 1) alter 'node_macs' to 'direct_macs'; 2) add one field to jobs, 'direct_ips'. Because lkp use those 2 fields to bind 'direct_ips' to 'direct_macs', after this, nodes can communicate with each other when running cluster job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 692180f..c32091f 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,16 @@ class Sched # collect all job ids job_ids = [] of String + net_id = "192.168.222" + ip0 = cluster_config["ip0"]? + if ip0 + ip0 = ip0.as_i + else + ip0 = 1 + end + # steps for each host - cluster_config.each do |host, config| + cluster_config["nodes"].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +227,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 26 Oct '20

26 Oct '20

[WHY] the path of *.cgz file generated by build-pkg script cannot be obtained dring test. So the path of cgz need to be changed. [HOW] below is example for openeuler: /srv/initrd/build-pkg/ => /srv/initrd/build-pkg/initramfs/openeuler/aarch64/20.03/ Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..eca09e64 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,6 +2,7 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url @@ -21,10 +22,14 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" @@ -75,7 +80,7 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 } -- 2.23.0

3 5

[PATCH v6 compass-ci 6/7] container/send-internet-mail: start
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/start | 42 ++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 container/send-internet-mail/start diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start new file mode 100755 index 0000000..6ea55d6 --- /dev/null +++ b/container/send-internet-mail/start @@ -0,0 +1,42 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'send-internet-mail' + +names = Set.new %w[ + ROBOT_EMAIL_ADDRESS_INTERNET + ROBOT_EMAIL_PASSWORD_INTERNET + SEND_MAIL_PORT_INTERNET +] + +defaults = relevant_defaults(names) + +ROBOT_EMAIL_ADDRESS = defaults['ROBOT_EMAIL_ADDRESS_INTERNET'] +ROBOT_EMAIL_PASSWORD = defaults['ROBOT_EMAIL_PASSWORD_INTERNET'] +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] + +cmd = %W[ + docker run + --restart=always + --name=send-internet-mail + -d + -e ROBOT_EMAIL_ADDRESS=#{ROBOT_EMAIL_ADDRESS} + -e ROBOT_EMAIL_PASSWORD=#{ROBOT_EMAIL_PASSWORD} + -e CCI_SRC=/c/compass-ci + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -p #{SEND_MAIL_PORT}:#{SEND_MAIL_PORT} + -v #{ENV['CCI_SRC']}:/c/compass-ci + -v /etc/localtime:/etc/localtime:ro + -w /c/compass-ci + --log-driver json-file + send-internet-mail +] + +cmd += ['ruby', "container/send-internet-mail/run.rb"] + +system(*cmd) -- 2.23.0

2 2

[PATCH v6 compass-ci 3/7] container/send-internet-mail: run.rb
by Luan Shengde 26 Oct '20

26 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 2

[PATCH v6 compass-ci 2/7] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

2 2

[PATCH v6 compass-ci 1/7] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..9fcf1fe --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,64 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = %x(/sbin/ip route | awk '/default/ {print $3}').chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

2 4