- Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 23 Oct '20

23 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..424615be 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -3,6 +3,7 @@ # - test_memory_size # - mode # - test +# - os_arch ## lmbench is a suite of simple, portable, ANSI/C microbenchmarks for ## UNIX/POSIX. In general, it measures two key features: latency and @@ -138,7 +139,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/$os_arch-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +178,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/$os_arch-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

2 1

[PATCH compass-ci] container/debian: solve build error
by Cui Yili 23 Oct '20

23 Oct '20

Executing "container/debian/build", error when apt-get installs the package: E: Could not configure 'libc6:arm64'. E: Could not perform immediate configuration on 'libnss-nis:arm64'. Please see man 5 apt.conf under APT::Immediate-Configure for details. (2) Increase the parameter "-o APT::Immediate-Configure=false" when apt-get installs the package. reference: https://www.debian.org/releases/wheezy/armhf/release-notes/ch-upgrading.en.… Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/debian/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/debian/Dockerfile b/container/debian/Dockerfile index 1d310da..21b6a22 100644 --- a/container/debian/Dockerfile +++ b/container/debian/Dockerfile @@ -12,6 +12,6 @@ COPY root / RUN sed -i "s/^exit [0-9]*/exit 0/" /usr/sbin/policy-rc.d RUN apt-get update && \ apt-get install -y --no-install-recommends apt-utils > /dev/null 2>&1 && \ - apt-get install -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ + apt-get install -o APT::Immediate-Configure=false -y runit openssh-server zsh vim rsync git make gcc g++ tzdata sudo && \ mkdir -p /run/sshd -- 2.23.0

2 1

[PATCH v3 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexpected problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

2 1

[PATCH v3 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

2 1

[PATCH compass-ci] sparrow/0-package/os: skip when intall failed
by Xu Xijian 23 Oct '20

23 Oct '20

When install several packages, if one of them is installed failed, should go ahead to install next packages. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- sparrow/0-package/os/centos | 2 +- sparrow/0-package/os/debian | 2 +- sparrow/0-package/os/openEuler | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..4243f99 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -22,4 +22,4 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..16ef0a9 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -20,4 +20,4 @@ pkgs=( busybox ) -apt-get install -y "${pkgs[@]}" +apt-get install -y --fix-missing "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..ba298c8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -21,7 +21,7 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -- 2.23.0

2 1

[PATCH v3 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..992d4a14 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,5 @@ #!/bin/sh +# - block_size ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +8,11 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + if [ -n $block_size ]; then + cmd="iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest" + else + cmd="iozone -a -f $mnt/iozonetest" + fi log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

Re: [PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Zhang Yu 23 Oct '20

23 Oct '20

> log_echo "$cmd &" > $cmd > $TMP/iozone-$i & > i=$((i+1)) >+ iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest You should add a judge, if $block_size isn't in yaml file, this maybe error. Thanks, Zhangyu

1 0

[PATCH v5 compass-ci 1/6] container/send-internet-mail: mail-post.rb
by Luan Shengde 23 Oct '20

23 Oct '20

get request for send mail analysis email data send mail default use local defined smtp setup when required internet-smtp, will override the local setup and use internet-smtp instead Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/mail-post.rb | 68 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100755 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb new file mode 100755 index 0000000..e62640a --- /dev/null +++ b/lib/mail-post.rb @@ -0,0 +1,68 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'sinatra' +require 'json' +require 'yaml' +require 'open3' +require 'mail' + +set :bind, '0.0.0.0' +set :port, ENV['SEND_MAIL_PORT'] + +mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp + +smtp = { + address: mail_server, + enable_starttls_auto: false +} + +Mail.defaults { delivery_method :smtp, smtp } + +post '/send_mail_yaml' do + data = YAML.safe_load request.body.read + raise TypeError, data, 'request data type error' unless data.class.eql? Hash + + mail_info = { + 'references' => data['references'], + 'subject' => data['subject'], + 'to' => data['to'], + 'body' => data['body'] + } + check_send_mail(mail_info) +end + +post '/send_mail_text' do + data = Mail.read_from_string(request.body.read) + + mail_info = { + 'references' => data.references, + 'subject' => data.subject, + 'to' => data.to, + 'body' => data.body.decoded + } + check_send_mail(mail_info) +end + +def check_send_mail(mail_info) + mail_info['from'] = ENV['ROBOT_EMAIL_ADDRESS'] + + raise 'no subject.' if mail_info['subject'].nil? + raise 'no email address.' if mail_info['to'].nil? + raise 'no email content.' if mail_info['body'].nil? + + send_mail(mail_info) +end + +def send_mail(mail_info) + mail = Mail.new do + references mail_info['references'] + from mail_info['from'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

3 6

[PATCH v2 lkp-tests] tests: add "arch" field with physical machine
by Zhang Yu 23 Oct '20

23 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..c0e17f26 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,18 +1,20 @@ #!/bin/bash -get_node() +get_mem() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + lscpu=$(lscpu) + echo "nr_node: $(echo "$lscpu" | grep "NUMA node(s)" | awk '{print $3}')" + echo "nr_cpu: $(echo "$lscpu" | grep "^CPU(s)" | awk '{print $2}')" + echo "arch: $(echo "$lscpu" | grep "Architecture" | awk '{print $2}')" + echo "model_name: $(echo "$lscpu" | grep "Model name" | awk '{print $3}')" } get_nr_disk() @@ -101,11 +103,11 @@ get_ipmi_ip() main() { - get_node + get_cpu + get_mem get_nr_disk get_partitions get_mac - get_model_name get_ipmi_ip } -- 2.23.0

5 4

[PATCH v2 lkp-tests 2/2] tests/iozone: add specified parameter for iozone tool
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for 4g size and different block_size. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/iozone b/tests/iozone index 3095ce80..0a813621 100755 --- a/tests/iozone +++ b/tests/iozone @@ -11,6 +11,7 @@ do log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) + iozone -r $block_size -s 4g -i 0 -i 1 -i 2 -f $mnt/iozonetest done log_eval wait -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/2] jobs/iozone.yaml: add specified block_size parameter
by Lu Kaiyi 23 Oct '20

23 Oct '20

[why] need to test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/jobs/iozone.yaml b/jobs/iozone.yaml index f281186b..f464280c 100644 --- a/jobs/iozone.yaml +++ b/jobs/iozone.yaml @@ -1,6 +1,12 @@ suite: iozone testcase: iozone category: benchmark +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m disk: 2HDD -- 2.23.0

1 0

[PATCH v5 compass-ci 2/6] container/send-internet-mail: internet-smtp.rb
by Luan Shengde 23 Oct '20

23 Oct '20

setup smtp for sending internet email: smtp setup for: smtp.qq.com Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- lib/internet-smtp.rb | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 lib/internet-smtp.rb diff --git a/lib/internet-smtp.rb b/lib/internet-smtp.rb new file mode 100644 index 0000000..2e51d8f --- /dev/null +++ b/lib/internet-smtp.rb @@ -0,0 +1,18 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['ROBOT_EMAIL_ADDRESS'], + password: ENV['ROBOT_EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } -- 2.23.0

3 6

[PATCH lkp-tests] chore: install iproute2
by Liu Yinsi 23 Oct '20

23 Oct '20

command ip used in sparrow/2-network/br0. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/cci-deploy | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-deploy b/distro/depends/cci-deploy index dac68250..01085f59 100644 --- a/distro/depends/cci-deploy +++ b/distro/depends/cci-deploy @@ -9,3 +9,4 @@ bridge-utils qemu dmidecode lockfile-progs +iproute2 -- 2.23.0

3 5

[PATCH compass-ci] sparrow/0-package: adapt to x86_64 architectures
by Liu Yinsi 23 Oct '20

23 Oct '20

lockfile-create download link compatible with x86_64 and aarch64 architectures. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index bd1f728..e408d77 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -24,7 +24,8 @@ pkgs=( yum install -y "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(uname -p) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

3 3

[PATCH v2 lkp-tests] api-avx2neon: delete unnecessary fields in api-avx2neon.yaml
by Li Yuanchao 23 Oct '20

23 Oct '20

These fields will be configured in compass-ci/sbin/auto_submit.yaml Both configured in two places may cause unexcepted problems Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 3/6] container/send-internet-mail: run.rb
by Luan Shengde 23 Oct '20

23 Oct '20

invoke mail-post/internet-smtp to run the send mail service Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/run.rb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 container/send-internet-mail/run.rb diff --git a/container/send-internet-mail/run.rb b/container/send-internet-mail/run.rb new file mode 100644 index 0000000..bd98da8 --- /dev/null +++ b/container/send-internet-mail/run.rb @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +# mail-post have default smtp setup for sending intranet email +# internet-smtp is smtp config file for sending internet email +# when sending internet email, require internet-smtp, smtp setup in +# internet-smtp will override the default smtp setup in mail-post + +require 'sinatra' + +require "#{ENV['CCI_SRC']}/lib/mail-post" +require "#{ENV['CCI_SRC']}/lib/internet-smtp" -- 2.23.0

2 1

[PATCH lkp-tests] api-avx2neon: delete unnecessary fields
by Li Yuanchao 23 Oct '20

23 Oct '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- jobs/api-avx2neon.yaml | 7 ------- 1 file changed, 7 deletions(-) diff --git a/jobs/api-avx2neon.yaml b/jobs/api-avx2neon.yaml index 24de6913..f3b46bf0 100644 --- a/jobs/api-avx2neon.yaml +++ b/jobs/api-avx2neon.yaml @@ -1,11 +1,4 @@ suite: api-avx2neon category: functional -testbox: vm-pxe-hi1620-1p1g -arch: aarch64 - -os: centos -os_arch: aarch64 -os_version: 7 - api-avx2neon: -- 2.23.0

1 0

[PATCH v5 compass-ci 4/6] container/send-internet-mail: Dockerfile
by Luan Shengde 23 Oct '20

23 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..598064b --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER shdluan(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail && \ + apt-get install -y ruby-json && \ + apt-get install -y ruby-sinatra -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: cluster job needs more fields
by Ren Wen 23 Oct '20

23 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. Ips can bind to on macs after setting 'direct_mac' and 'direct_ips' for cluster jobsI [HowTo] 1) Need a cluster config file in $LKP_SRC/cluster, for example: ```yaml assumption: <assumption01> switch: switch01 ip0: 3 --- vm-2p8g--renwen: roles: [ server ] macs: [ "ec:f4:bb:cb:7b:92" ] vm-2p4g--renwen: roles: [ client ] macs: [ "ec:f4:bb:cb:54:90" ] ``` nr_node=2 cs1 switch=sw1 ip0=0 ($ip_prefix.0-.1) cs2 switch=sw1 ip0=2 ($ip_prefix.2-.3) "switch": only need if 2+ clusters share the same switch "ip0": only need if shared switch, and != 0 "assumption": the info about this cluster config 2) Scheduler get config from config file and set 'direct_macs' and 'direct_ips' for jobs. 3) When running a cluster job, bind direct_ips to direct_ macs in test environment. 4) Test client connect with daemon client to test job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 070a16b..8922874 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -144,7 +144,7 @@ class Sched def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)).as_h + return YAML.parse_all(File.read(cluster_file_path)) end def get_commit_date(job) @@ -198,8 +198,13 @@ class Sched # collect all job ids job_ids = [] of String + basic_config = cluster_config[0] + net_id = basic_config["net_id"]? || "192.168.222" + ip0 = basic_config["ip0"].as_i + # switch = basic_config["switch"].to_s # useless now + # steps for each host - cluster_config.each do |host, config| + cluster_config[-1].as_h.each do |host, config| tbox_group = host.to_s job_id = add_task(tbox_group, lab) @@ -219,7 +224,15 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + direct_macs = config["macs"].as_a + direct_ips = [] of String + direct_macs.size.times do + raise "Host id is greater than 254, host_id: #{ip0}" if ip0 > 254 + direct_ips << "#{net_id}.#{ip0}" + ip0 += 1 + end + job["direct_macs"] = direct_macs.join(" ") + job["direct_ips"] = direct_ips.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

2 2

[PATCH v4 lkp-tests] feat(iso2rootfs): add step: submit rootfs test job after rootfs is made
by Yu Chuan 23 Oct '20

23 Oct '20

[Why] The produced dailybuild rootfs needs further testing by submit a job. [How] 1. Configure lkp env by {LKP_SRC}/sbin/install-dependencies.sh 2. Generate global compass-ci yaml. - The path of yaml: - /etc/compass-ci/defaults/sparrow.yaml - The yaml have two necessary fields: - SCHED_HOST - SCHED_PORT - The value of these two fields are consistent with the value in job.yaml of iso2rootfs. 3. Submit the job.yaml of rootfs test job with append follow params: - os - os_arch - os_version - os_mount - testbox Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 6 ++++ tests/iso2rootfs | 76 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 74 insertions(+), 8 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index f413c9cee1f0..644c64be06b4 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -11,3 +11,9 @@ iso2rootfs: rootfs_path: initrd_http_host: initrd_http_port: + test_yaml: + test_os: + test_os_arch: + test_os_version: + test_os_mount: + test_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index bf1f995e78df..e8a027b0dad2 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -53,12 +53,18 @@ check_yaml_vars() "rootfs_path" "initrd_http_host" "initrd_http_port" + "test_yaml" + "test_os" + "test_os_arch" + "test_os_version" + "test_os_mount" + "test_testbox" ) local yaml_t for yaml_t in "${yaml_vars[@]}" do - [ -n "$(eval echo "\$${yaml_t}")" ] || die "var$${yaml_t}$ is none" + [ -n "$(eval echo "\$${yaml_t}")" ] || die "cannot fine value of var: ${yaml_t}." done } @@ -133,7 +139,7 @@ mount_rootfs() install_pkgs "cifs-utils" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac @@ -164,7 +170,7 @@ check_sha256sum_update() if [ -n "$SHA256SUM_CACHE" ] then - [ -n "$SHA256SUM_NET" ] || die "get net sha256sum failed !!!" + [ -n "$SHA256SUM_NET" ] || die "cannot get sha256sum of dailybuild iso." if [ "$SHA256SUM_CACHE" == "$SHA256SUM_NET" ] then @@ -185,7 +191,9 @@ check_os() { log_info "starting check os ..." echo "${ALLOW_OS[@]}" | grep -wq "${os}" || - die "current host $${os}$ is not in $${ALLOW_OS[@]}$, exit !!!" + die "current os is not in allow os. + current os: ${os}. + allow os: $${ALLOW_OS[@]}$" } get_cgz() @@ -210,12 +218,12 @@ install_pkgs() local pkg_installer command -v yum && pkg_installer="yum" - [[ -n "$pkg_installer" ]] || die "can not find pkg installer !!!" + [[ -n "$pkg_installer" ]] || die "cannot find pkg installer." local pt for pt do - "$pkg_installer" install -y "$pt" || die "install pkg$$pt$ failed !!!" + "$pkg_installer" install -y "$pt" || die "cannot install pkg: $pt." done } @@ -364,7 +372,7 @@ test_rootfs() root_path="cifs:${ROOTFS_SERVER_PATH}/$(basename "${ROOTFS_DES_DIR}"),${ROOTFS_MOUNT_PARAM}" ;; *) - die "rootfs_protocol is none, exit !!!" + die "cannot support the rootfs_protocol: ${rootfs_protocol}." ;; esac get_qemu_efi_fd @@ -404,6 +412,57 @@ test_rootfs() EOF } +############ submit test job yaml ############ +generate_global_yaml() +{ + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" + + mkdir -p "$(dirname "${config_yaml}")" + echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" + echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" +} + +generate_submit_append_str() +{ + local wait_add_vars=( + "os" + "os_arch" + "os_version" + "os_mount" + "testbox" + ) + for vt in "${wait_add_vars[@]}" + do + SUBMIT_APPEND_STR="${SUBMIT_APPEND_STR} ${vt}=$(eval echo "\$test_${vt}")" + done +} + +submit_test_job_yaml() +{ + log_info "starting submit ${test_yaml} ..." + + [ -d "${LKP_SRC}" ] || die "cannot find value of var: LKP_SRC." + + cd "${LKP_SRC}" && { + chmod +x ./sbin/install-dependencies.sh + ./sbin/install-dependencies.sh + + generate_global_yaml + } + + cd "${LKP_SRC}/jobs" && { + [ -f "${test_yaml}" ] || + die "cannot find test yaml in LKP_SRC/jobs. + test yaml: ${test_yaml}." + + generate_submit_append_str + + "${LKP_SRC}/sbin/submit" ${SUBMIT_APPEND_STR} "${test_yaml}" + } + + log_info "submit ${test_yaml} finished" +} + ############ post works ############ post_works() { @@ -414,7 +473,7 @@ post_works() local soft_link="${iso_version}-dailybuild" cd "${ROOTFS_LOCAL_PATH}" && rm -f "${soft_link}" && - ln -s "$(basename ${ROOTFS_DES_DIR})" "${soft_link}" + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" cd / && umount "${ROOTFS_LOCAL_PATH}" @@ -433,6 +492,7 @@ main() run_qcow2rootfs test_rootfs + submit_test_job_yaml post_works } -- 2.23.0

2 1

[PATCH compass-ci] job.cr: initrd modules and headers with initramfs
by Xu Xijian 23 Oct '20

23 Oct '20

When run job with initramfs, we should initrd modules.cgz and headers.cgz too. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..75d75e5 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -303,10 +303,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-default.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-default.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 1

[PATCH compass-ci] sparrow: install iproute
by Liu Yinsi 23 Oct '20

23 Oct '20

sparrow/2-network/br0 needs ip command. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..adfdfd5 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -20,6 +20,7 @@ pkgs=( liblockfile lockfile-progs busybox + iproute ) yum install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..48bb4c5 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -18,6 +18,7 @@ pkgs=( lockfile-progs bridge-utils busybox + iproute2 ) apt-get install -y "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..bd1f728 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -19,6 +19,7 @@ pkgs=( redis wget busybox + iproute ) yum install -y "${pkgs[@]}" -- 2.23.0

1 0

[PATCH lkp-tests] lib/monitor.rb: only one way to ssh
by Wu Zhende 22 Oct '20

22 Oct '20

[Why] Now, all the scenarios use "sshr" to login testbox. So delete the direct way to ssh. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/monitor.rb | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/lib/monitor.rb b/lib/monitor.rb index fcf7e220..9ce1d475 100755 --- a/lib/monitor.rb +++ b/lib/monitor.rb @@ -75,18 +75,9 @@ class Monitor return unless data['log'] data = JSON.parse(data['log']) - if job['uuid'] - # means Internet user - # using the redirection service to log in - return unless data['ssh_port'] + return unless data['ssh_port'] - ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) - else - # intranet user: direct login internal IP - return unless data['ip'] - - ssh_connect(data['ip'], 22, web_socket) - end + ssh_connect(job['SCHED_HOST'], data['ssh_port'], web_socket) end def ssh_connect(ssh_host, ssh_port, web_socket) -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: add 2 fields into job: direct_macs and direct_ips
by Ren Wen 22 Oct '20

22 Oct '20

[Why] Nodes need to communicate with each other when testing cluster jobs. After adding 2 fileds: 'direct_macs' and 'direct_ips', can bind ips to macs. [HowTo] When you test a cluster job, need to config cluster file correctly. Below is the template: ```yaml <hostname>: roles: [ <"server">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] <hostname>: roles: [ <"client">, ... ] macs: [ <mac>, ... ] ips: [ <ip>, ... ] ``` For 'ips': 1) should be the intranet IP 2) all ips in cluster file should be in the same network segment 3) subnet mask is 24. [InFuture] Set default ips when there's no ips in cluster file or only one host has ips. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 7215df5..a26091b 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -219,7 +219,8 @@ class Sched job["testbox"] = tbox_group job.update_tbox_group(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") - job["node_macs"] = config["macs"].as_a.join(" ") + job["direct_macs"] = config["macs"].as_a.join(" ") + job["direct_ips"] = config["ips"].as_a.join(" ") response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") -- 2.23.0

4 4

[PATCH lkp-tests] tests: fix lmbench3 tool adapt to aarch64 framwork
by Bai Jing 22 Oct '20

22 Oct '20

error: cat: results/x84_64-linux-gnu/taishan200-2280-2s48p-256g--a28.0 No such file or directory Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/lmbench3 | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lmbench3 b/tests/lmbench3 index 8570a35a..f09c49e2 100755 --- a/tests/lmbench3 +++ b/tests/lmbench3 @@ -138,7 +138,7 @@ run_lmbench3_development() eval "$test=yes" - sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/x86_64-linux-gnu/lmbench + sed -i '/lat_pagefault -P $SYNC_MAX $FILE/i [ -f $FILE ] || dd if=/dev/zero of=$FILE count=1 bs=1G' bin/aarch64-linux-gnu/lmbench log_eval " ( echo $nr_threads @@ -177,7 +177,7 @@ run_lmbench3_development() show_result() { - cat results/x86_64-linux-gnu/$testbox.0 || { + cat results/aarch64-linux-gnu/$testbox.0 || { echo "no test result found" exit -1 } -- 2.23.0

3 3

2024

2023

2022

2021

2020

Compass-ci