- Compass-ci - mailweb.openeuler.org

[PATCH v5 compass-ci 5/6] container/send-internet-mail: build
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/send-internet-mail/build diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build new file mode 100755 index 0000000..078adaf --- /dev/null +++ b/container/send-internet-mail/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t send-internet-mail:latest . -- 2.23.0

2 1

[PATCH v2 lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 23 Oct '20

23 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..424615be 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -3,6 +3,7 @@ # - test_memory_size # - mode # - test +# - os_arch ## lmbench is a suite of simple, portable, ANSI/C microbenchmarks for ## UNIX/POSIX. In general, it measures two key features: latency and @@ -138,7 +139,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/$os_arch-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +178,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/$os_arch-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

2 1

[PATCH compass-ci] container/debian: solve build error
by Cui Yili 23 Oct '20

23 Oct '20

Executing "container/debian/build", error when apt-get installs the package: E: Could not configure 'libc6:arm64'. E: Could not perform immediate configuration on 'libnss-nis:arm64'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2) Increase the parameter "-o APT::Immediate-Configure=false" when apt-get installs the package. reference: https://www.debian.org/releases/wheezy/armhf/release-notes/ch-upgrading.en.… Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/debian/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/debian/Dockerfile b/container/debian/Dockerfile index 1d310da..21b6a22 100644 --- a/container/debian/Dockerfile +++ b/container/debian/Dockerfile @@ -12,6 +12,6 @@ COPY root / RUN sed -i "s/^exit [0-9]*/exit 0/" /usr/sbin/policy-rc.d RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ + apt-get install -o APT::Immediate-Configure=false -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ mkdir -p /run/sshd -- 2.23.0

2 1

[PATCH v3 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexpected problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

2 1

[PATCH v3 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

2 1

[PATCH compass-ci] sparrow/0-package/os: skip when intall failed
by Xu Xijian 23 Oct '20

23 Oct '20

When install several packages, if one of them is installed failed, should go ahead to install next packages. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- sparrow/0-package/os/centos | 2 +- sparrow/0-package/os/debian | 2 +- sparrow/0-package/os/openEuler | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..4243f99 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -22,4 +22,4 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..16ef0a9 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -20,4 +20,4 @@ pkgs=( busybox ) -apt-get install -y "${pkgs[@]}" +apt-get install -y --fix-missing "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..ba298c8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -21,7 +21,7 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -- 2.23.0

2 1

[PATCH v3 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..992d4a14 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n $block_size ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

Re: [PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Zhang Yu 23 Oct '20

23 Oct '20

> log_echo "$cmd &" > $cmd > $TMP/iozone-$i & > i=$((i+1)) >+ iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest You should add a judge, if $block_size isn't in yaml file, this maybe error. Thanks, Zhangyu

1 0

[PATCH v5 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 23 Oct '20

23 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 68 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..e62640a --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,68 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.class.eql? Hash + + mail_info = { + 'references' => data['references'], + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'references' => data.references, + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + check_send_mail(mail_info) +end + +def check_send_mail(mail_info) + mail_info['from'] = ENV['ROBOT_EMAIL_ADDRESS'] + + raise 'no subject.' if mail_info['subject'].nil? + raise 'no email address.' if mail_info['to'].nil? + raise 'no email content.' if mail_info['body'].nil? + + send_mail(mail_info) +end + +def send_mail(mail_info) + mail = Mail.new do + references mail_info['references'] + from mail_info['from'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

3 6

[PATCH v2 lkp-tests] tests: add "arch" field with physical machine
by Zhang Yu 23 Oct '20

23 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..c0e17f26 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,18 +1,20 @@ #!/bin/bash -get_node() +get_mem() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + lscpu=$(lscpu) + echo "nr_node: $(echo "$lscpu" | grep "NUMA node(s)" | awk '{print $3}')" + echo "nr_cpu: $(echo "$lscpu" | grep "^CPU(s)" | awk '{print $2}')" + echo "arch: $(echo "$lscpu" | grep "Architecture" | awk '{print $2}')" + echo "model_name: $(echo "$lscpu" | grep "Model name" | awk '{print $3}')" } get_nr_disk() @@ -101,11 +103,11 @@ get_ipmi_ip() main() { - get_node + get_cpu + get_mem get_nr_disk get_partitions get_mac - get_model_name get_ipmi_ip } -- 2.23.0

5 4

[PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/iozone b/tests/iozone index 3095ce80..0a813621 100755 --- a/tests/iozone +++ b/tests/iozone @@ -11,6 +11,7 @@ do log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) + iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest done log_eval wait -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

1 0

[PATCH v5 compass-ci 2/6] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 23 Oct '20

23 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

3 6

[PATCH lkp-tests] chore: install iproute2
by Liu Yinsi 23 Oct '20

23 Oct '20

command ip used in sparrow/2-network/br0. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/cci-deploy | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index dac68250..01085f59 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -9,3 +9,4 @@ bridge-utils qemu dmidecode lockfile-progs +iproute2 -- 2.23.0

3 5

[PATCH compass-ci] sparrow/0-package: adapt to x86_64 architectures
by Liu Yinsi 23 Oct '20

23 Oct '20

lockfile-create download link compatible with x86_64 and aarch64 architectures. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index bd1f728..e408d77 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -24,7 +24,8 @@ pkgs=( yum install -y "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(uname -p) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

3 3

[PATCH v2 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexcepted problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 3/6] container/send-internet-mail: run.rb
by Luan Shengde 23 Oct '20

23 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 1

[PATCH lkp-tests] api-avx2neon: delete unnecessary fields
by Li Yuanchao 23 Oct '20

23 Oct '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 4/6] container/send-internet-mail: Dockerfile
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..598064b --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail && \ + apt-get install -y ruby-json && \ + apt-get install -y ruby-sinatra -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 23 Oct '20

23 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. Ips can bind to on macs after setting 'direct_mac' and 'direct_ips' for cluster jobsI [HowTo] 1) Need a cluster config file in $LKP_SRC/cluster, for example: ```yaml assumption: <assumption01> switch: switch01 ip0: 3 --- vm-2p8g--renwen: roles: [ server ] macs: [ "ec:f4:bb:cb:7b:92" ] vm-2p4g--renwen: roles: [ client ] macs: [ "ec:f4:bb:cb:54:90" ] ``` nr_node=2 cs1 switch=sw1 ip0=0 ($ip_prefix.0-.1) cs2 switch=sw1 ip0=2 ($ip_prefix.2-.3) "switch": only need if 2+ clusters share the same switch "ip0": only need if shared switch, and != 0 "assumption": the info about this cluster config 2) Scheduler get config from config file and set 'direct_macs' and 'direct_ips' for jobs. 3) When running a cluster job, bind direct_ips to direct_ macs in test environment. 4) Test client connect with daemon client to test job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 070a16b..8922874 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse_all(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,13 @@ class Sched # collect all job ids job_ids = [] of String + basic_config = cluster_config[0] + net_id = basic_config["net_id"]? || "192.168.222" + ip0 = basic_config["ip0"].as_i + # switch = basic_config["switch"].to_s # useless now + # steps for each host - cluster_config.each do |host, config| + cluster_config[-1].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +224,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

2 2

[PATCH v4 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 23 Oct '20

23 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 ++++ tests/iso2rootfs | 76 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 74 insertions(+), 8 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..e8a027b0dad2 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,12 +53,18 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t for yaml_t in "${yaml_vars[@]}" do - [ -n "$(eval echo "\$${yaml_t}")" ] || die "var$${yaml_t}$ is none" + [ -n "$(eval echo "\$${yaml_t}")" ] || die "cannot fine value of var: ${yaml_t}." done } @@ -133,7 +139,7 @@ mount_rootfs() install_pkgs "cifs-utils" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac @@ -164,7 +170,7 @@ check_sha256sum_update() if [ -n "$SHA256SUM_CACHE" ] then - [ -n "$SHA256SUM_NET" ] || die "get net sha256sum failed !!!" + [ -n "$SHA256SUM_NET" ] || die "cannot get sha256sum of dailybuild iso." if [ "$SHA256SUM_CACHE" == "$SHA256SUM_NET" ] then @@ -185,7 +191,9 @@ check_os() { log_info "starting check os ..." echo "${ALLOW_OS[@]}" | grep -wq "${os}" || - die "current host $${os}$ is not in $${ALLOW_OS[@]}$, exit !!!" + die "current os is not in allow os. + current os: ${os}. + allow os: $${ALLOW_OS[@]}$" } get_cgz() @@ -210,12 +218,12 @@ install_pkgs() local pkg_installer command -v yum && pkg_installer="yum" - [[ -n "$pkg_installer" ]] || die "can not find pkg installer !!!" + [[ -n "$pkg_installer" ]] || die "cannot find pkg installer." local pt for pt do - "$pkg_installer" install -y "$pt" || die "install pkg$$pt$ failed !!!" + "$pkg_installer" install -y "$pt" || die "cannot install pkg: $pt." done } @@ -364,7 +372,7 @@ test_rootfs() root_path="cifs:${ROOTFS_SERVER_PATH}/$(basename "${ROOTFS_DES_DIR}"),${ROOTFS_MOUNT_PARAM}" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac get_qemu_efi_fd @@ -404,6 +412,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "cannot find value of var: LKP_SRC." + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "cannot find test yaml in LKP_SRC/jobs. + test yaml: ${test_yaml}." + + generate_submit_append_str + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +473,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +492,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

2 1

[PATCH compass-ci] job.cr: initrd modules and headers with initramfs
by Xu Xijian 23 Oct '20

23 Oct '20

When run job with initramfs, we should initrd modules.cgz and headers.cgz too. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..75d75e5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -303,10 +303,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-default.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-default.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 1

[PATCH compass-ci] sparrow: install iproute
by Liu Yinsi 23 Oct '20

23 Oct '20

sparrow/2-network/br0 needs ip command. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..adfdfd5 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -20,6 +20,7 @@ pkgs=( liblockfile lockfile-progs busybox + iproute ) yum install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..48bb4c5 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -18,6 +18,7 @@ pkgs=( lockfile-progs bridge-utils busybox + iproute2 ) apt-get install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..bd1f728 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -19,6 +19,7 @@ pkgs=( redis wget busybox + iproute ) yum install -y "${pkgs[@]}" -- 2.23.0

1 0

[PATCH lkp-tests] lib/monitor.rb: only one way to ssh
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] Now, all the scenarios use "sshr" to login testbox. So delete the direct way to ssh. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index fcf7e220..9ce1d475 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -75,18 +75,9 @@ class Monitor return unless data['log'] data = JSON.parse(data['log']) - if job['uuid'] - # means Internet user - # using the redirection service to log in - return unless data['ssh_port'] + return unless data['ssh_port'] - ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) - else - # intranet user: direct login internal IP - return unless data['ip'] - - ssh_connect(data['ip'], 22, web_socket) - end + ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) end def ssh_connect(ssh_host, ssh_port, web_socket) -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: add 2 fields into job: direct_macs and direct_ips
by Ren Wen 22 Oct '20

22 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. After adding 2 fileds: 'direct_macs' and 'direct_ips', can bind ips to macs. [HowTo] When you test a cluster job, need to config cluster file correctly. Below is the template: ```yaml <hostname>: roles: [ <"server">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] <hostname>: roles: [ <"client">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] ``` For 'ips': 1) should be the intranet IP 2) all ips in cluster file should be in the same network segment 3) subnet mask is 24. [InFuture] Set default ips when there's no ips in cluster file or only one host has ips. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..a26091b 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -219,7 +219,8 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + job["direct_macs"] = config["macs"].as_a.join(" ") + job["direct_ips"] = config["ips"].as_a.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

4 4

[PATCH lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 22 Oct '20

22 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..f09c49e2 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -138,7 +138,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/aarch64-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +177,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/aarch64-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

3 3

[PATCH compass-ci] sbin: complier => compiler
by Cao Xueliang 22 Oct '20

22 Oct '20

[why] The container crystal-compiler build a image and named alpine:crystal-compiler, the sbin crystal-format use the image. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sbin/crystal-format | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/crystal-format b/sbin/crystal-format index 14f82b2..99f7a6a 100755 --- a/sbin/crystal-format +++ b/sbin/crystal-format @@ -15,7 +15,7 @@ cmd=( --rm -v $v_dir:$v_dir -w $PWD - alpine:crystal-complier + alpine:crystal-compiler crystal tool format "$@" ) -- 2.23.0

3 2

[PATCH compass-ci] auto_submit: testbox 2p8g => 2p16g for build-pkg
by Li Yuanchao 22 Oct '20

22 Oct '20

[why] 2p8g testbox only 1.3G left for build-pkg job, it's not enough. and there is error info write error: No space left on device Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index 1096014..edfbfb1 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -4,4 +4,4 @@ compass-ci/compass-ci: - testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=vm-2p8g os=centos os_version=7.6 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml archlinux: -- testbox=vm-2p8g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

1 0

[PATCH v2 lkp-tests] lib: fix btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 22 Oct '20

22 Oct '20

When job.yaml contains "fs: **", and /srv/initrd/deps/**/**/fs.cgz(btrfs-progs), it will execute mkfs.btrfs -f $rootfs_partition. but job.yaml don't container "fs: **" to appear warning: mkfs.btrfs command not found Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/bootstrap.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..16789c5c 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,6 +456,7 @@ mount_rootfs() ROOTFS_DIR=/opt/rootfs mkdir -p $ROOTFS_DIR mount $rootfs_partition $ROOTFS_DIR || { + command -v mkfs.btrfs >/dev/null || return mkfs.btrfs -f $rootfs_partition mount $rootfs_partition $ROOTFS_DIR } -- 2.23.0

3 2

[PATCH lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 22 Oct '20

22 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..f0c2878c 100755 --- a/tests/iozone +++ b/tests/iozone @@ -7,7 +7,7 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

2 1

[PATCH lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 22 Oct '20

22 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..d17137e8 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,13 +1,17 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m -disk: 2HDD +disk: 1HDD fs: -- xfs - ext4 -- btrfs iosched: - kyber -- 2.23.0

2 1

[PATCH lab-z9] hosts: add common configs
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-512g | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 hosts/taishan200-2280-2s48p-512g diff --git a/hosts/taishan200-2280-2s48p-512g b/hosts/taishan200-2280-2s48p-512g new file mode 100644 index 0000000..5a4126a --- /dev/null +++ b/hosts/taishan200-2280-2s48p-512g @@ -0,0 +1,4 @@ +nr_cpu: 96 +nr_node: 4 +memory: 512G +arch: aarch64 -- 2.23.0

1 0

[PATCH compass-ci] git_mirror: put fork_stat to es
by Li Yuanchao 22 Oct '20

22 Oct '20

[why] We want to know whether git-mirror works well, so save last 10 times fetch_time and new_refs_time, and put them to es Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 593d1eb..2b78523 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -98,8 +98,10 @@ class MirrorMain @fork_stat[stat_key] = { queued: false, priority: 0, - fetch_time: nil, - new_refs_time: nil + fetch_time: [], + offset_fetch: 0, + new_refs_time: [], + offset_new_refs: 0 } end @@ -152,10 +154,9 @@ class MirrorMain feedback_info = @feedback_queue.pop(true) git_repo = feedback_info[:git_repo] - @fork_stat[git_repo][:queued] = false + update_fork_stat(git_repo, feedback_info[:possible_new_refs]) return unless feedback_info[:possible_new_refs] - @fork_stat[git_repo][:priority] += 1 return reload_fork_info if git_repo == 'upstream-repos/upstream-repos' new_refs = check_new_refs(git_repo) @@ -196,8 +197,8 @@ class MirrorMain @git_info[git_repo] = YAML.safe_load(File.open(repodir)) @git_info[git_repo]['git_repo'] = git_repo @git_info[git_repo].merge!(@defaults[project]) if @defaults[project] - es_repo_update(git_repo, @git_info[git_repo]) fork_stat_init(git_repo) + es_repo_update(git_repo) @priority_queue.push git_repo, @priority @priority += 1 end @@ -255,11 +256,35 @@ class MirrorMain end end - def es_repo_update(git_repo, repo_info) + def es_repo_update(git_repo) + repo_info = @git_info[git_repo].merge(@fork_stat[git_repo]) + repo_info.delete(:cur_refs) if repo_info.key?(:cur_refs) body = { "doc": repo_info, "doc_as_upsert": true } @es_client.update(index: 'repo', type: '_doc', id: git_repo, body: body) end + + def update_stat_fetch(git_repo) + @fork_stat[git_repo][:queued] = false + offset_fetch = @fork_stat[git_repo][:offset_fetch] + offset_fetch = 0 if offset_fetch >= 10 + @fork_stat[git_repo][:fetch_time][offset_fetch] = Time.now.to_s + @fork_stat[git_repo][:offset_fetch] = offset_fetch + 1 + end + + def update_stat_new_refs(git_repo) + @fork_stat[git_repo][:priority] += 1 + offset_new_refs = @fork_stat[git_repo][:offset_new_refs] + offset_new_refs = 0 if offset_new_refs >= 10 + @fork_stat[git_repo][:new_refs_time][offset_new_refs] = Time.now.to_s + @fork_stat[git_repo][:offset_new_refs] = offset_new_refs + 1 + end + + def update_fork_stat(git_repo, possible_new_refs) + update_stat_fetch(git_repo) + update_stat_new_refs(git_repo) if possible_new_refs + es_repo_update(git_repo) + end end -- 2.23.0

1 0

[PATCH v2 compass-ci] /src/lib/web_backend.rb: support multi-fields search in "/get_jobs"
by Zhang Yuhang 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 46 ++++++++++++++++++++++++------------------ 1 file changed, 26 insertions(+), 20 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 48cfadb..c3f6bef 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -248,23 +248,32 @@ def es_search(must, size, from) return es_query(query)['hits']['hits'], total end -def get_job(result) - job = {} - ALL_FIELDS.each do |f| - job[f] = result[f] +def get_jobs_result(result) + jobs = [] + result.each do |r| + job = {} + ALL_FIELDS.each do |f| + job[f] = r['_source'][f] + end + jobs << job end - job + jobs end -def search_job(git_repo, page_size, page_num) +def search_job(condition_fields, page_size, page_num) must = [] - must << { regexp: { upstream_repo: ".*#{git_repo}.*" } } if git_repo - jobs = [] - result, total = es_search(must, page_size, page_num * page_size) - result.each do |r| - jobs << get_job(r['_source']) + FIELDS.each do |field| + value = space_to_nil(condition_fields[field]) + next unless value + + must << if field.to_s == 'upstream_repo' + { regexp: { field => ".*#{value}.*" } } + else + { term: { field => value } } + end end - return jobs, total + result, total = es_search(must, page_size, page_num * page_size) + return get_jobs_result(result), total end def get_banner(git_repo, branches) @@ -291,17 +300,14 @@ def get_optimize_jobs_braches(jobs) end def get_jobs_body(params) - git_repo = space_to_nil(params[:upstream_repo]) - page_size = get_positive_number(params[:page_size], 20) - page_num = get_positive_number(params[:page_num], 1) - 1 - jobs, total = search_job(git_repo, page_size, page_num) + page_size = get_positive_number(params.delete(:page_size), 20) + page_num = get_positive_number(params.delete(:page_num), 1) - 1 + jobs, total = search_job(params, page_size, page_num) jobs, branches = get_optimize_jobs_braches(jobs) { total: total, - filter: { - upstream_repo: git_repo - }, - banner: get_banner(git_repo, branches), + filter: params, + banner: get_banner(params[:upstream_repo], branches), jobs: jobs, fields: FIELDS }.to_json -- 2.23.0

1 0

[PATCH v4 lkp-tests] sshd: send email to users for login services
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 99 +++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 74 insertions(+), 25 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..82db61a6 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key -# email +# my_email +# sshr_port +# sshr_port_len +# sshr_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -11,40 +21,79 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { - deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") - data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh - -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: - - Login: - ssh root@$PUB_IP - Due time: - $deadline - - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch + data="To: $my_email +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_username: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: + + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch Regards -compass-ci +Compass-Ci " } +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Applying $testbox failed + +Dear $my_username: + Sorry to inform you that your application failed. + You may need to wait a moment, or check whether your pub_key exists. + +Regards +Compass-Ci +" +} + +compose_email() +{ + deadline=$(date -d "$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$pub_key" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

1 0

[PATCH v3 lkp-tests] sshd: send emails to login services for users
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 98 ++++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 74 insertions(+), 24 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index 98f31d77..1ebc9d00 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,6 +1,16 @@ #!/bin/sh # pub_key -# email +# my_email +# sshr_port +# sshr_port_len +# sshr_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -11,40 +21,80 @@ run_ssh() systemctl start sshd } -compose_email() +data_success() { deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") - data="To: $email -Subject: [NOTIFY compass-ci] $testbox ready to ssh - -Dear $email: - Thanks for your participation in Kunpeng and software ecosystem! - According to your application, $testbox has been provisioned. - The datails are as follows: - - Login: - ssh root@$PUB_IP - Due time: - $deadline - - HW: - nr_cpu: $nr_cpu - memory: $memory - testbox: $testbox - OS: - $os $os_version $os_arch + data="To: $my_email +Subject: [NOTIFY Compass-ci] $testbox ready to use + +Dear $my_username: + Thanks for your participation in software ecosystem! + According to your application, $testbox has been provisioned. + The datails are as follows: + + Login: + ssh root(a)api.compass-ci.openeuler.org -p $port + Due time: + $deadline + HW: + nr_cpu: $nr_cpu + memory: $memory + testbox: $testbox + OS: + $os $os_version $os_arch + +Regards +Compass-Ci +" +} + +data_failure() +{ + data="To: $my_email +Subject: [NOTIFY Compass-ci] Applying $testbox failed + +Dear $my_username: + Sorry to inform you that your application failed. + You may need to wait a moment, or check whether your pub_key exists. Regards -compass-ci +Compass-Ci " } +compose_email() +{ + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") + if [ -n "$port" ]; then + data_success + else + data_failure + fi +} + +set_port() +{ + [ -n "$pub_key" ] || return + + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] && return + done + port="" +} + run_email() { - [ -n "$email" ] || return 0 + [ -n "$my_email" ] || return 0 + my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email - curl -XPOST "$MAIL_HOST:$MAIL_PORT/send_mail_text" -d "$data" + curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" } run_ssh +set_port +report_ssh_port $port run_email -- 2.23.0

2 2

[PATCH compass-ci] lib/sched.cr: add result_root output logs
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] The result_root directory of the job stores the running results. It plays an important role in locating problems. So puts this info to docker logs, submit -m can show result_root. [Output Example] {"job_id"=>"z9.1", "message"=>"", "job_state"=>"submit", "result_root"=>"/srv/result/iperf/vm-2p8g/2020-10-22/z9.1"} {"job_id"=>"z9.1", "result_root"=>"/srv/result/iperf/vm-2p8g/2020-10-22/z9.1", "job_state"=>"set result root"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..070a16b 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -227,6 +227,7 @@ class Sched "job_id" => job_id, "message" => message.to_s, "job_state" => "submit", + "result_root" => "/srv#{job.result_root}" } return job_messages if response["error"]? end @@ -278,6 +279,7 @@ class Sched "job_id" => job_id, "message" => message.to_s, "job_state" => "submit", + "result_root" => "/srv#{job.result_root}" }] end @@ -441,6 +443,7 @@ class Sched if job job.update({"testbox" => testbox}) job.set_result_root + puts %({"job_id": "#{job_id}", "result_root": "/srv#{job.result_root}", "job_state": "set result root"}) @redis.set_job(job) end return job -- 2.23.0

1 0

[PATCH lkp-tests] fixed a bug that modified shellrc in root-path when make install
by Xie Zheng 22 Oct '20

22 Oct '20

--- Makefile | 4 ++-- sbin/set-env.sh | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Makefile b/Makefile index 37387809..907dba9e 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ ifeq ($(TARGET_DIR_BIN), ) TARGET_DIR_BIN := /usr/local/bin endif -ifneq ($(shell whoami), "root") +ifneq ($(shell whoami), root) TARGET_DIR_BIN := $${HOME}/bin endif @@ -14,8 +14,8 @@ subsystem: install: mkdir -p $(TARGET_DIR_BIN) ln -sf $(shell pwd)/bin/lkp $(TARGET_DIR_BIN)/lkp - bash sbin/install-dependencies.sh bash sbin/set-env.sh + sudo bash sbin/install-dependencies.sh .PHONY: doc doc: diff --git a/sbin/set-env.sh b/sbin/set-env.sh index b9046519..a8b161fa 100644 --- a/sbin/set-env.sh +++ b/sbin/set-env.sh @@ -5,7 +5,7 @@ write_shellrc() { echo "export LKP_SRC=$PWD" >> $(shell_profile) - echo "export PATH=\$PATH:$PWD/sbin:$PWD/bin" >> $(shell_profile) + echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $(shell_profile) } write_host() -- 2.23.0

1 0

[PATCH lab-z9] hosts: add "arch" field to host files
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/dc-1g | 1 + hosts/dc-2g | 1 + hosts/dc-4g | 1 + hosts/dc-8g | 1 + hosts/taishan200-2280-2s48p-256g | 1 + hosts/taishan200-2280-2s48p-256g--a101 | 1 + hosts/taishan200-2280-2s48p-256g--a102 | 1 + hosts/taishan200-2280-2s48p-256g--a11 | 1 + hosts/taishan200-2280-2s48p-256g--a13 | 1 + hosts/taishan200-2280-2s48p-256g--a14 | 1 + hosts/taishan200-2280-2s48p-256g--a15 | 1 + hosts/taishan200-2280-2s48p-256g--a17 | 1 + hosts/taishan200-2280-2s48p-256g--a18 | 1 + hosts/taishan200-2280-2s48p-256g--a19 | 1 + hosts/taishan200-2280-2s48p-256g--a21 | 1 + hosts/taishan200-2280-2s48p-256g--a22 | 1 + hosts/taishan200-2280-2s48p-256g--a24 | 1 + hosts/taishan200-2280-2s48p-256g--a25 | 1 + hosts/taishan200-2280-2s48p-256g--a26 | 1 + hosts/taishan200-2280-2s48p-256g--a27 | 1 + hosts/taishan200-2280-2s48p-256g--a28 | 1 + hosts/taishan200-2280-2s48p-256g--a29 | 1 + hosts/taishan200-2280-2s48p-256g--a30 | 1 + hosts/taishan200-2280-2s48p-256g--a31 | 1 + hosts/taishan200-2280-2s48p-256g--a32 | 1 + hosts/taishan200-2280-2s48p-256g--a33 | 1 + hosts/taishan200-2280-2s48p-256g--a34 | 1 + hosts/taishan200-2280-2s48p-256g--a52 | 1 + hosts/taishan200-2280-2s48p-256g--a53 | 1 + hosts/taishan200-2280-2s48p-256g--a60 | 1 + hosts/taishan200-2280-2s48p-256g--a61 | 1 + hosts/taishan200-2280-2s48p-256g--a62 | 1 + hosts/taishan200-2280-2s48p-256g--a66 | 1 + hosts/taishan200-2280-2s48p-256g--a67 | 1 + hosts/taishan200-2280-2s48p-256g--a68 | 1 + hosts/taishan200-2280-2s48p-256g--a69 | 1 + hosts/taishan200-2280-2s48p-256g--a70 | 1 + hosts/taishan200-2280-2s48p-256g--a71 | 1 + hosts/taishan200-2280-2s48p-256g--a95 | 1 + hosts/taishan200-2280-2s64p-256g | 1 + hosts/taishan200-2280-2s64p-256g--a10 | 1 + hosts/taishan200-2280-2s64p-256g--a3 | 1 + hosts/taishan200-2280-2s64p-256g--a36 | 1 + hosts/taishan200-2280-2s64p-256g--a4 | 1 + hosts/taishan200-2280-2s64p-256g--a40 | 1 + hosts/taishan200-2280-2s64p-256g--a41 | 1 + hosts/taishan200-2280-2s64p-256g--a42 | 1 + hosts/taishan200-2280-2s64p-256g--a8 | 1 + hosts/taishan200-2280-2s64p-256g--a9 | 1 + hosts/vm-1p1g | 1 + hosts/vm-1p1g-pxe | 1 + hosts/vm-2p1g | 1 + hosts/vm-2p1g-pxe | 1 + hosts/vm-2p4g | 1 + hosts/vm-2p4g-pxe | 1 + hosts/vm-2p8g | 1 + hosts/vm-2p8g-pxe | 1 + hosts/vm-hi1620-1p1g | 1 + hosts/vm-hi1620-2p1g | 1 + hosts/vm-hi1620-2p4g | 1 + hosts/vm-hi1620-2p8g | 1 + 61 files changed, 61 insertions(+) diff --git a/hosts/dc-1g b/hosts/dc-1g index ec1e59b..afeba49 100644 --- a/hosts/dc-1g +++ b/hosts/dc-1g @@ -1 +1,2 @@ +arch: aarch64 memory: 1g diff --git a/hosts/dc-2g b/hosts/dc-2g index b9509a2..6956411 100644 --- a/hosts/dc-2g +++ b/hosts/dc-2g @@ -1 +1,2 @@ +arch: aarch64 memory: 2g diff --git a/hosts/dc-4g b/hosts/dc-4g index 52e0636..1407332 100644 --- a/hosts/dc-4g +++ b/hosts/dc-4g @@ -1 +1,2 @@ +arch: aarch64 memory: 4g diff --git a/hosts/dc-8g b/hosts/dc-8g index 333d625..ec61c1d 100644 --- a/hosts/dc-8g +++ b/hosts/dc-8g @@ -1 +1,2 @@ +arch: aarch64 memory: 8g diff --git a/hosts/taishan200-2280-2s48p-256g b/hosts/taishan200-2280-2s48p-256g index abba938..642e8e1 100644 --- a/hosts/taishan200-2280-2s48p-256g +++ b/hosts/taishan200-2280-2s48p-256g @@ -1,3 +1,4 @@ +arch: aarch64 nr_cpu: 96 nr_node: 4 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a101 b/hosts/taishan200-2280-2s48p-256g--a101 index 1226913..f8a0429 100644 --- a/hosts/taishan200-2280-2s48p-256g--a101 +++ b/hosts/taishan200-2280-2s48p-256g--a101 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a102 b/hosts/taishan200-2280-2s48p-256g--a102 index 267a962..e1d3f10 100644 --- a/hosts/taishan200-2280-2s48p-256g--a102 +++ b/hosts/taishan200-2280-2s48p-256g--a102 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G diff --git a/hosts/taishan200-2280-2s48p-256g--a11 b/hosts/taishan200-2280-2s48p-256g--a11 index a281e24..619dc06 100644 --- a/hosts/taishan200-2280-2s48p-256g--a11 +++ b/hosts/taishan200-2280-2s48p-256g--a11 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 index 991694c..468c175 100644 --- a/hosts/taishan200-2280-2s48p-256g--a13 +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 index d6b2970..24ea9dc 100644 --- a/hosts/taishan200-2280-2s48p-256g--a14 +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 index d66b003..9c28258 100644 --- a/hosts/taishan200-2280-2s48p-256g--a15 +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 index 7485faa..16b065d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a17 +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 index 8673d64..9d1ec01 100644 --- a/hosts/taishan200-2280-2s48p-256g--a18 +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 index 8ffac96..217eaa9 100644 --- a/hosts/taishan200-2280-2s48p-256g--a19 +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 index b1fae8b..738738d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a21 +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 index 2d46733..bb29d94 100644 --- a/hosts/taishan200-2280-2s48p-256g--a22 +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index d666366..60ea1d2 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index 8e43697..916513e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 index 15cc4e0..4a5a0f3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a26 +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 index e6b3135..129589c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a27 +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 index f27d5e0..f8a77f8 100644 --- a/hosts/taishan200-2280-2s48p-256g--a28 +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 index 74c614f..37814cc 100644 --- a/hosts/taishan200-2280-2s48p-256g--a29 +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 index 568316f..f722edb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a30 +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 index 4f49685..7d6c67c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a31 +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 index ac0279e..fb17e38 100644 --- a/hosts/taishan200-2280-2s48p-256g--a32 +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 index 20b8323..a3a6b03 100644 --- a/hosts/taishan200-2280-2s48p-256g--a33 +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 index b657141..5a15949 100644 --- a/hosts/taishan200-2280-2s48p-256g--a34 +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 index 339b2a8..2bd7096 100644 --- a/hosts/taishan200-2280-2s48p-256g--a52 +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 index e1dca3a..bf0894b 100644 --- a/hosts/taishan200-2280-2s48p-256g--a53 +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 index 9a92a85..2255f2f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a60 +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 index bef5d8a..73079cb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a61 +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 index b41eaf7..abaac6e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a62 +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index c9fb714..fce93d1 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index a5ab220..6b32609 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index afd0101..4bdd8f4 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index 8ac90cc..3d3be80 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 463febf..89c3c77 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 index 501ea4b..3121212 100644 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ b/hosts/taishan200-2280-2s48p-256g--a71 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G diff --git a/hosts/taishan200-2280-2s48p-256g--a95 b/hosts/taishan200-2280-2s48p-256g--a95 index 3fd8124..213d10f 100644 --- a/hosts/taishan200-2280-2s48p-256g--a95 +++ b/hosts/taishan200-2280-2s48p-256g--a95 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 512G diff --git a/hosts/taishan200-2280-2s64p-256g b/hosts/taishan200-2280-2s64p-256g index ab798a3..16349d9 100644 --- a/hosts/taishan200-2280-2s64p-256g +++ b/hosts/taishan200-2280-2s64p-256g @@ -1,3 +1,4 @@ +arch: aarch64 nr_cpu: 128 nr_node: 4 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index f99a406..cdb32f5 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a3 b/hosts/taishan200-2280-2s64p-256g--a3 index 744978e..8f34f3f 100644 --- a/hosts/taishan200-2280-2s64p-256g--a3 +++ b/hosts/taishan200-2280-2s64p-256g--a3 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a36 b/hosts/taishan200-2280-2s64p-256g--a36 index 51ba3a4..9757144 100644 --- a/hosts/taishan200-2280-2s64p-256g--a36 +++ b/hosts/taishan200-2280-2s64p-256g--a36 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a4 b/hosts/taishan200-2280-2s64p-256g--a4 index 0f73c52..bcfaea7 100644 --- a/hosts/taishan200-2280-2s64p-256g--a4 +++ b/hosts/taishan200-2280-2s64p-256g--a4 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 index 6825c38..d9a9937 100644 --- a/hosts/taishan200-2280-2s64p-256g--a40 +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 index 5937093..91531fd 100644 --- a/hosts/taishan200-2280-2s64p-256g--a41 +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 index 1f5d3f0..ed1f188 100644 --- a/hosts/taishan200-2280-2s64p-256g--a42 +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a8 b/hosts/taishan200-2280-2s64p-256g--a8 index 64f2750..e515306 100644 --- a/hosts/taishan200-2280-2s64p-256g--a8 +++ b/hosts/taishan200-2280-2s64p-256g--a8 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 index b21e4d3..ce3aa0a 100644 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ b/hosts/taishan200-2280-2s64p-256g--a9 @@ -1,3 +1,4 @@ +arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G diff --git a/hosts/vm-1p1g b/hosts/vm-1p1g index c64cc36..3af3646 100644 --- a/hosts/vm-1p1g +++ b/hosts/vm-1p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-1p1g-pxe b/hosts/vm-1p1g-pxe index 233f639..4571c23 100644 --- a/hosts/vm-1p1g-pxe +++ b/hosts/vm-1p1g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p1g b/hosts/vm-2p1g index c781033..0293ce9 100644 --- a/hosts/vm-2p1g +++ b/hosts/vm-2p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p1g-pxe b/hosts/vm-2p1g-pxe index 6129d20..7f76598 100644 --- a/hosts/vm-2p1g-pxe +++ b/hosts/vm-2p1g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p4g b/hosts/vm-2p4g index 7e9570d..219c4c9 100644 --- a/hosts/vm-2p4g +++ b/hosts/vm-2p4g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p4g-pxe b/hosts/vm-2p4g-pxe index 8ba4e58..5f0de24 100644 --- a/hosts/vm-2p4g-pxe +++ b/hosts/vm-2p4g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-2p8g b/hosts/vm-2p8g index e5d5f84..6249322 100644 --- a/hosts/vm-2p8g +++ b/hosts/vm-2p8g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_node: 1 diff --git a/hosts/vm-2p8g-pxe b/hosts/vm-2p8g-pxe index ae83d6f..9b09402 100644 --- a/hosts/vm-2p8g-pxe +++ b/hosts/vm-2p8g-pxe @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: pxe nr_node: 1 diff --git a/hosts/vm-hi1620-1p1g b/hosts/vm-hi1620-1p1g index bf2a7b3..2226bdc 100644 --- a/hosts/vm-hi1620-1p1g +++ b/hosts/vm-hi1620-1p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 1 diff --git a/hosts/vm-hi1620-2p1g b/hosts/vm-hi1620-2p1g index 1c4786a..e2952eb 100644 --- a/hosts/vm-hi1620-2p1g +++ b/hosts/vm-hi1620-2p1g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p4g b/hosts/vm-hi1620-2p4g index f2a5b5e..83d8133 100644 --- a/hosts/vm-hi1620-2p4g +++ b/hosts/vm-hi1620-2p4g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p8g b/hosts/vm-hi1620-2p8g index 1780524..a4a0577 100644 --- a/hosts/vm-hi1620-2p8g +++ b/hosts/vm-hi1620-2p8g @@ -1,3 +1,4 @@ +arch: aarch64 provider: qemu template: kvm nr_cpu: 2 -- 2.23.0

1 0

[PATCH compass-ci] sched: no need limit access_key update
by Cao Xueliang 22 Oct '20

22 Oct '20

[why] When update result_root the access_key need update too. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index c6fb0fd..8f5dd36 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -191,7 +191,7 @@ class Job end private def set_access_key - self["access_key"] = "#{Random::Secure.hex(10)}-#{id}" unless @hash["access_key"]? + self["access_key"] = "#{Random::Secure.hex(10)}-#{id}" self["access_key_file"] = File.join("/srv/", "#{result_root}", ".#{access_key}") end -- 2.23.0

1 0

[PATCH v2 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 22 Oct '20

22 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 +++++ tests/iso2rootfs | 60 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..9a40f7b07892 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,6 +53,12 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t @@ -404,6 +410,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "no LKP_SRC !!!" + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "test yaml$${test_yaml}$ exist in {LKP_SRC}/jobs !!!" + + generate_submit_append_str + [ -z "${SUBMIT_APPEND_STR}" ] || SUBMIT_APPEND_STR="-a ${SUBMIT_APPEND_STR}" + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +471,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +490,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

3 5

[PATCH compass-ci] code-spec: grep-friendly fixed substring
by Wu Fengguang 22 Oct '20

22 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/code-spec.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index 37c86b6..6f2dddd 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -19,6 +19,13 @@ code注意事项 - 如果注释仅仅是代码的简单重复, 不提供额外信息量, 不要写. +- puts/printf字符串应当容易搜索(grep)。特别注意包含足够长的固定内容substring. +例如： + die "test yaml$${test_yaml}$ not exist in ${LKP_SRC}/jobs !!!" +==> + die "cannot find test yaml ($test_yaml) in $LKP_SRC/jobs" + ^^^^^^^^^^^^^^^^^^^^^ grep-friendly fixed substring + 重构友好编码 ============ -- 2.23.0

1 0

[PATCH v2 lkp-tests] sshd: send emails to login services for users
by Zhang Yale 22 Oct '20

22 Oct '20

[why] Provide a login port for users by email, determine whether to send the ssh_port message based on the "pub_key". Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- daemon/sshd | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/daemon/sshd b/daemon/sshd index cd57be4f..1ebc9d00 100755 --- a/daemon/sshd +++ b/daemon/sshd @@ -1,16 +1,16 @@ #!/bin/sh # pub_key # my_email -# my_uuid -# ssh_port_len -# ssh_port_base +# sshr_port +# sshr_port_len +# sshr_port_base . $LKP_SRC/lib/http.sh sshr_ip=$LKP_SERVER -sshr_port=5050 -[ -n "$ssh_port_len" ] || ssh_port_len=2000 -[ -n "$ssh_port_base" ] || ssh_port_base=50000 +[ -n "$sshr_port" ] || sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 run_ssh() { @@ -23,6 +23,7 @@ run_ssh() data_success() { + deadline=$(date -d "+$runtime seconds" +"%Y-%m-%d %H:%M:%S") data="To: $my_email Subject: [NOTIFY Compass-ci] $testbox ready to use @@ -43,7 +44,7 @@ Dear $my_username: $os $os_version $os_arch Regards -Compass-ci +Compass-Ci " } @@ -54,10 +55,10 @@ Subject: [NOTIFY Compass-ci] Applying $testbox failed Dear $my_username: Sorry to inform you that your application failed. - You may need to wait for a while and then try again. + You may need to wait a moment, or check whether your pub_key exists. Regards -Compass-ci +Compass-Ci " } @@ -73,11 +74,11 @@ compose_email() set_port() { - [ -n "$my_uuid" ] || return + [ -n "$pub_key" ] || return for i in $(seq 1 10) do - port=$(($(date +%s%N)%"$ssh_port_len"+"$ssh_port_base")) + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null [ $? -eq 0 ] && return @@ -87,7 +88,7 @@ set_port() run_email() { - [ -n "$my_email" ] && [ -n "$my_uuid" ] || return 0 + [ -n "$my_email" ] || return 0 my_username=$(echo "$my_email" | awk -F '@' '{print $1}') compose_email curl -XPOST "${MAIL_HOST:-$LKP_SERVER}:${MAIL_PORT:-11312}/send_mail_text" -d "$data" -- 2.23.0

2 2

[PATCH v2 lkp-tests] sbin/submit: -m automatically stop monitoring
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] When use "submit -m job.yaml" to monitor job state, it can't auto stop. Need to enter "Ctrl + c" manually. It's not convenient. So add the function of automatically stopping when "job_state=extract_finished" is received. [Output] connect to ws://localhost:11310/filter {"job_id"=>"z9.105511", "message"=>"", "job_state"=>"submit"} {"job_id"=>"z9.105511", "job_state"=>"boot"} {"job_id"=>"z9.105511", "job_state"=>"download"} {"time"=>"2020-10-22 11:14:55", "mac"=>"xx", "ip"=>"xx", "job_id"=>"z9.105511", "state"=>"running", "testbox"=>"vm-2p8g-3871595"} {"job_state"=>"running", "job_id"=>"z9.105511"} {"job_state"=>"post_run", "job_id"=>"z9.105511"} {"start_time"=>"2020-10-22 03:14:56", "end_time"=>"2020-10-22 03:15:26", "loadavg"=>"1.26 0.38 0.13 2/87 1268", "job_id"=>"z9.105511"} {"job_state"=>"finished", "job_id"=>"z9.105511"} {"job_id"=>"z9.105511", "job_state"=>"complete"} {"time"=>"2020-10-22 11:15:34", "mac"=>"xx", "ip"=>"xx", "job_id"=>"z9.105511", "state"=>"rebooting", "testbox"=>"vm-2p8g-3871595"} {"job_id"=>"z9.105511", "job_state"=>"extract_finished"} connection closed: Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/monitor | 6 ++++++ sbin/submit | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/sbin/monitor b/sbin/monitor index c19a6056..def47bf6 100755 --- a/sbin/monitor +++ b/sbin/monitor @@ -14,6 +14,7 @@ opt_action = {'output' => true} opt_job = {} opt_monitor_url = nil opt_timeout = 0 +opt_stop_query = {} options = OptionParser.new do |opts| opts.banner = 'Usage: monitor [options] <filter>' @@ -49,6 +50,10 @@ options = OptionParser.new do |opts| opts.on('-j', '--job json', 'set job') do |job| opt_job.merge! YAML.load job end + + opts.on('-s', '--stop-query json', 'add query to stop monitor') do |stop_query| + opt_stop_query.merge! YAML.load stop_query + end end options.parse!(ARGV) @@ -75,5 +80,6 @@ monitor.monitor_url = opt_monitor_url if opt_monitor_url monitor.overrides = opt_set_key_value monitor.action = opt_action monitor.job = opt_job +monitor.stop_query = opt_stop_query monitor.run(opt_timeout) diff --git a/sbin/submit b/sbin/submit index 8b46db32..c49e1e9a 100755 --- a/sbin/submit +++ b/sbin/submit @@ -16,7 +16,7 @@ opt_output_dir = nil opt_auto_define_files = false opt_monitor = false opt_monitor_query = {} -actions = ['output'] +actions = ['output', 'stop'] options = OptionParser.new do |opts| opts.banner = 'Usage: submit [options] jobs...' @@ -157,6 +157,6 @@ end if opt_monitor opt_monitor_query.merge!({'job_id' => job_ids[0]}) - cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_json_list[0]}' -a #{actions.join(',')}" + cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_json_list[0]}' -a #{actions.join(',')} -s 'job_state: extract_finished'" exec cmd end -- 2.23.0

1 0

[PATCH lkp-tests] tests: add "arch" field with physical machine
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..d12ebc7e 100755 --- a/tests/host-info +++ b/tests/host-info @@ -10,9 +10,10 @@ get_node() echo "memory: $memory" } -get_model_name() +get_model_and_arch() { echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + echo "arch: $(lscpu |grep "Architecture" | awk '{print $2}')" } get_nr_disk() @@ -105,7 +106,7 @@ main() get_nr_disk get_partitions get_mac - get_model_name + get_model_and_arch get_ipmi_ip } -- 2.23.0

2 2

[PATCH lkp-tests] sbin/submit: -m automatically stop monitoring
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] When use "submit -m job.yaml" to monitor job state, it can't auto stop. Need to enter "Ctrl + c" manually. It's not convenient. So add the function of automatically stopping when "job_state=extract_finished" is received. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/monitor | 6 ++++++ sbin/submit | 4 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/sbin/monitor b/sbin/monitor index c19a6056..def47bf6 100755 --- a/sbin/monitor +++ b/sbin/monitor @@ -14,6 +14,7 @@ opt_action = {'output' => true} opt_job = {} opt_monitor_url = nil opt_timeout = 0 +opt_stop_query = {} options = OptionParser.new do |opts| opts.banner = 'Usage: monitor [options] <filter>' @@ -49,6 +50,10 @@ options = OptionParser.new do |opts| opts.on('-j', '--job json', 'set job') do |job| opt_job.merge! YAML.load job end + + opts.on('-s', '--stop-query json', 'add query to stop monitor') do |stop_query| + opt_stop_query.merge! YAML.load stop_query + end end options.parse!(ARGV) @@ -75,5 +80,6 @@ monitor.monitor_url = opt_monitor_url if opt_monitor_url monitor.overrides = opt_set_key_value monitor.action = opt_action monitor.job = opt_job +monitor.stop_query = opt_stop_query monitor.run(opt_timeout) diff --git a/sbin/submit b/sbin/submit index 8b46db32..c49e1e9a 100755 --- a/sbin/submit +++ b/sbin/submit @@ -16,7 +16,7 @@ opt_output_dir = nil opt_auto_define_files = false opt_monitor = false opt_monitor_query = {} -actions = ['output'] +actions = ['output', 'stop'] options = OptionParser.new do |opts| opts.banner = 'Usage: submit [options] jobs...' @@ -157,6 +157,6 @@ end if opt_monitor opt_monitor_query.merge!({'job_id' => job_ids[0]}) - cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_json_list[0]}' -a #{actions.join(',')}" + cmd = "#{LKP_SRC}/sbin/monitor -f '#{opt_monitor_query.to_json}' -j '#{job_json_list[0]}' -a #{actions.join(',')} -s 'job_state: extract_finished'" exec cmd end -- 2.23.0

2 1

[PATCH lkp-tests] hackbench-20.yaml: fix "datesize" to "datasize"
by Zhang Yale 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- jobs/hackbench-20.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/hackbench-20.yaml b/jobs/hackbench-20.yaml index b69e8c77..12943c04 100644 --- a/jobs/hackbench-20.yaml +++ b/jobs/hackbench-20.yaml @@ -11,7 +11,7 @@ nr_threads: - 10 - 20 iterations: 10 -datesize: 1000 +datasize: 1000 runtime: 300 hackbench: -- 2.23.0

1 0

[PATCH lab-z9] hosts: add hostfile for a105-a107
by Zhang Yu 22 Oct '20

22 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-256g--a105 | 24 ++++++++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a106 | 24 ++++++++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a107 | 24 ++++++++++++++++++++++++ 3 files changed, 72 insertions(+) create mode 100644 hosts/taishan200-2280-2s48p-256g--a105 create mode 100644 hosts/taishan200-2280-2s48p-256g--a106 create mode 100644 hosts/taishan200-2280-2s48p-256g--a107 diff --git a/hosts/taishan200-2280-2s48p-256g--a105 b/hosts/taishan200-2280-2s48p-256g--a105 new file mode 100644 index 0000000..3c28d2d --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a105 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 512G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b891945d + - /dev/disk/by-id/scsi-350000399b8919135 + - /dev/disk/by-id/scsi-350000399b8919699 + - /dev/disk/by-id/scsi-350000399b89190d5 +mac_addr: + - 44:67:47:d7:76:c4 + - 44:67:47:d7:76:c5 + - 44:67:47:d7:76:c6 + - 44:67:47:d7:76:c7 + - 44:67:47:d7:76:c8 + - 44:67:47:d7:76:c9 + - 44:67:47:d7:76:ca + - 44:67:47:d7:76:cb + - cc:05:77:ee:b2:8f + - cc:05:77:ee:b2:90 + - cc:05:77:ee:b2:91 + - cc:05:77:ee:b2:92 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.10 diff --git a/hosts/taishan200-2280-2s48p-256g--a106 b/hosts/taishan200-2280-2s48p-256g--a106 new file mode 100644 index 0000000..a7b44a0 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a106 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b883bf99 + - /dev/disk/by-id/scsi-350000399b883ba89 + - /dev/disk/by-id/scsi-350000399b883b94d + - /dev/disk/by-id/scsi-350000399b883b981 +mac_addr: + - 44:67:47:86:1c:30 + - 44:67:47:86:1c:31 + - 44:67:47:86:1c:32 + - 44:67:47:86:1c:33 + - 44:67:47:86:1c:34 + - 44:67:47:86:1c:35 + - 44:67:47:86:1c:36 + - 44:67:47:86:1c:37 + - cc:05:77:ee:b7:df + - cc:05:77:ee:b7:e0 + - cc:05:77:ee:b7:e1 + - cc:05:77:ee:b7:e2 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.11 diff --git a/hosts/taishan200-2280-2s48p-256g--a107 b/hosts/taishan200-2280-2s48p-256g--a107 new file mode 100644 index 0000000..9d20e57 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a107 @@ -0,0 +1,24 @@ +nr_node: 4 +nr_cpu: 96 +memory: 256G +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8917c39 + - /dev/disk/by-id/scsi-350000399b8918005 + - /dev/disk/by-id/scsi-350000399b8900a4d + - /dev/disk/by-id/scsi-350000399b8900799 +mac_addr: + - 84:46:fe:73:a2:71 + - 84:46:fe:73:a2:72 + - 84:46:fe:73:a2:73 + - 84:46:fe:73:a2:74 + - 84:46:fe:73:a2:75 + - 84:46:fe:73:a2:76 + - 84:46:fe:73:a2:77 + - 84:46:fe:73:a2:78 + - 44:a1:91:35:1a:70 + - 44:a1:91:35:1a:71 + - 44:a1:91:35:1a:72 + - 44:a1:91:35:1a:73 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.12 -- 2.23.0

2 1

[PATCH v3 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 21 Oct '20

21 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 +++++ tests/iso2rootfs | 60 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 65 insertions(+), 1 deletion(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..85e3a73e8640 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,6 +53,12 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t @@ -404,6 +410,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "no LKP_SRC !!!" + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "test yaml$${test_yaml}$ not exist in {LKP_SRC}/jobs !!!" + + generate_submit_append_str + [ -z "${SUBMIT_APPEND_STR}" ] || SUBMIT_APPEND_STR="-a ${SUBMIT_APPEND_STR}" + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +471,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +490,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

1 0

[PATCH compass-ci] container/result-webdav: fix curl 403 forbidden error
by Cao Xueliang 21 Oct '20

21 Oct '20

[why] We cant find access_key_file in upload dest dir, because access_key_file was created in job result root dir, [how] 1. Add job id to the access_key. 2. Use job id to find the job result root dir in upload uri. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/result-webdav/nginx.conf | 8 +++++--- src/lib/job.cr | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/result-webdav/nginx.conf b/container/result-webdav/nginx.conf index 9297ae8..5ef3647 100644 --- a/container/result-webdav/nginx.conf +++ b/container/result-webdav/nginx.conf @@ -25,10 +25,12 @@ http { access_by_lua_block { accesskey = ngx.var.cookie_ACCESSKEY - uri=ngx.var.request_uri - path=string.match(uri,"%g*/") + job_id = string.sub(accesskey, string.find(accesskey,"-") - string.len(accesskey)) + uri = ngx.var.request_uri + path = string.match(uri,"%g*/") + root_path = string.sub(path, 1, string.find(path, job_id) - 1)..job_id.."/" - accesskey_file="/srv/"..path.."."..accesskey + accesskey_file = "/srv/"..root_path.."."..accesskey local f = io.open(accesskey_file) if not f then diff --git a/src/lib/job.cr b/src/lib/job.cr index 2cb8a2c..f6c1762 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -191,7 +191,7 @@ class Job end private def set_access_key - self["access_key"] = "#{Random::Secure.hex(10)}" unless @hash["access_key"]? + self["access_key"] = "#{Random::Secure.hex(10)}-#{id}" unless @hash["access_key"]? self["access_key_file"] = File.join("/srv/", "#{result_root}", ".#{access_key}") end -- 2.23.0

1 0