October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v7 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 55 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..5832bfb --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,55 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

1 0

[PATCH v2 compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 26 Oct '20

26 Oct '20

[Why] Two things here i do: 1) alter 'node_macs' to 'direct_macs'; 2) add one field to jobs, 'direct_ips'. Because lkp use those 2 fields to bind 'direct_ips' to 'direct_macs', after this, nodes can communicate with each other when running cluster job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 692180f..c32091f 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,16 @@ class Sched # collect all job ids job_ids = [] of String + net_id = "192.168.222" + ip0 = cluster_config["ip0"]? + if ip0 + ip0 = ip0.as_i + else + ip0 = 1 + end + # steps for each host - cluster_config.each do |host, config| + cluster_config["nodes"].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +227,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: modify save location for *.cgz file
by Liu Shaofei 26 Oct '20

26 Oct '20

[WHY] the path of *.cgz file generated by build-pkg script cannot be obtained dring test. So the path of cgz need to be changed. [HOW] below is example for openeuler: /srv/initrd/build-pkg/ => /srv/initrd/build-pkg/initramfs/openeuler/aarch64/20.03/ Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..eca09e64 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,6 +2,7 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url @@ -21,10 +22,14 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" @@ -75,7 +80,7 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 } -- 2.23.0

3 5

[PATCH v6 compass-ci 6/7] container/send-internet-mail: start
by Luan Shengde 26 Oct '20

26 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/start | 42 ++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 container/send-internet-mail/start diff --git a/container/send-internet-mail/start b/container/send-internet-mail/start new file mode 100755 index 0000000..6ea55d6 --- /dev/null +++ b/container/send-internet-mail/start @@ -0,0 +1,42 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig' + +docker_rm 'send-internet-mail' + +names = Set.new %w[ + ROBOT_EMAIL_ADDRESS_INTERNET + ROBOT_EMAIL_PASSWORD_INTERNET + SEND_MAIL_PORT_INTERNET +] + +defaults = relevant_defaults(names) + +ROBOT_EMAIL_ADDRESS = defaults['ROBOT_EMAIL_ADDRESS_INTERNET'] +ROBOT_EMAIL_PASSWORD = defaults['ROBOT_EMAIL_PASSWORD_INTERNET'] +SEND_MAIL_PORT = defaults['SEND_MAIL_PORT_INTERNET'] + +cmd = %W[ + docker run + --restart=always + --name=send-internet-mail + -d + -e ROBOT_EMAIL_ADDRESS=#{ROBOT_EMAIL_ADDRESS} + -e ROBOT_EMAIL_PASSWORD=#{ROBOT_EMAIL_PASSWORD} + -e CCI_SRC=/c/compass-ci + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -p #{SEND_MAIL_PORT}:#{SEND_MAIL_PORT} + -v #{ENV['CCI_SRC']}:/c/compass-ci + -v /etc/localtime:/etc/localtime:ro + -w /c/compass-ci + --log-driver json-file + send-internet-mail +] + +cmd += ['ruby', "container/send-internet-mail/run.rb"] + +system(*cmd) -- 2.23.0

2 2

[PATCH v6 compass-ci 3/7] container/send-internet-mail: run.rb
by Luan Shengde 26 Oct '20

26 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 2

[PATCH v6 compass-ci 2/7] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 26 Oct '20

26 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

2 2

[PATCH v6 compass-ci 1/7] container/send-internet-mail: mail-post.rb
by Luan Shengde 26 Oct '20

26 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 64 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..9fcf1fe --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,64 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = %x(/sbin/ip route | awk '/default/ {print $3}').chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.is_a? Hash + + mail_info = { + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + + send_mail(mail_info) +end + +def check_send_mail(mail_info) + raise 'no/empty subject.' if mail_info['subject'].nil? || mail_info['subject'].empty? + raise 'no/empty email_to address.' if mail_info['to'].nil? || mail_info['to'].empty? + raise 'no/empty email content.' if mail_info['body'].nil? || mail_info['body'].empty? +end + +def send_mail(mail_info) + check_send_mail(mail_info) + mail = Mail.new do + from ENV['ROBOT_EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

2 4

[PATCH lkp-tests] fio: fio test arg add iosched
by Wei Jihui 26 Oct '20

26 Oct '20

set iosched=my-deadline by test require Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- jobs/fio-basic-openeuler.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/jobs/fio-basic-openeuler.yaml b/jobs/fio-basic-openeuler.yaml index d2cc277b..55c98f97 100644 --- a/jobs/fio-basic-openeuler.yaml +++ b/jobs/fio-basic-openeuler.yaml @@ -7,6 +7,8 @@ disk: - 1HDD fs: - ext4 +iosched: +- mq-deadline nr_task: 24 fio-setup-basic: @@ -26,7 +28,6 @@ fio-setup-basic: bs: - 4k - 16k - - 32k - 64k - 128k - 256k -- 2.23.0

2 1

[PATCH v3 lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 26 Oct '20

26 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..dc4d157d 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -138,7 +138,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/"${os_arch}"-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +177,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/"${os_arch}"-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

1 0

git count 2020-10-23
by Wu Fengguang 26 Oct '20

26 Oct '20

Jihui is code star of the week, thanks Jihui! wfg@crystal ~/todo% bin/git-count 13 Wu Fengguang 12 Wei Jihui 10 Wu Zhende 6 Xiao Shenwei 6 Li Ping 5 Cao Xueliang 5 Li Yuanchao 4 Zhang Yu 4 Yu Chuan 4 Ren Wen 3 Bai Jing 3 Xu Xijian 3 Hu Xuejiao 3 Cui Yili 2 Zhang Yale 2 Wang Chenglong 1 Lu Weitao 1 Luan Shengde 1 Zhang Yuhang 1 Xie Zheng 0 chief 0 Du Kaitian 0 Liu Shaofei 0 Liu Yinsi 0 Ren Zhiwen 0 Sun Lijun 0 Sun Yukui 0 Shi Zhichao 0 Wang Yong 0 Lin Jiaxin 0 Liang Tao 0 Zhang Dewan 0 Lu Kaiyi 0 Zhou Shengrui

2 1