November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v4 lkp-tests 2/2] tests/iozone: modify the way of parsing parameter
by Lu Kaiyi 05 Nov '20

05 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool also need change the way of parsing parameter. [how] modify the way of parsing parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 66 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..e126948d 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,52 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + # reset IFS to split $test by "," and then restore default + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + case $ele in + "write_rewrite") args+=" -i 0" + ;; + "read_reread") args+=" -i 1" + ;; + "random_read_write") args+=" -i 2" + ;; + "read_backwards") args+=" -i 3" + ;; + "rewrite_record") args+=" -i 4" + ;; + "stride_read") args+=" -i 5" + ;; + "fwrite_refwrite") args+=" -i 6" + ;; + "fread_refread") args+=" -i 7" + ;; + "random_mix") args+=" -i 8" + ;; + "pwrite_repwrite") args+=" -i 9" + ;; + "pread_repread") args+=" -i 10" + ;; + "pwritev_repwritev") args+=" -i 11" + ;; + "preadv_repreadv") args+=" -i 12" + ;; + esac + done else args+=" -a" fi -- 2.23.0

2 1

[PATCH v2 compass-ci 06/10] container/mail-robot: run.rb
by Luan Shengde 05 Nov '20

05 Nov '20

invoke: send-apply-account-email assign-jumper-account parse-apply-account-email apply-account mail-robot Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/run.rb | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100755 container/mail-robot/run.rb diff --git a/container/mail-robot/run.rb b/container/mail-robot/run.rb new file mode 100755 index 0000000..8ac1d6e --- /dev/null +++ b/container/mail-robot/run.rb @@ -0,0 +1,10 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative 'send-apply-account-email' +require_relative 'assign-jumper-account' +require_relative 'parse-apply-account-email' +require_relative 'apply-account' +require_relative '../../lib/mail-robot' -- 2.23.0

3 2

[PATCH lkp-tests 1/3] lib/job2sh.rb: fix bug: can not find the file:pkg/$program/PKGBUILD
by Li Ping 05 Nov '20

05 Nov '20

[why] when the $dir not exist, cat > $dir/$file << 'EOF' xxx EOF will echo error message: can not find the file $dir/$file Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job2sh.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job2sh.rb b/lib/job2sh.rb index 9c95ff4b..7e6a5d7c 100755 --- a/lib/job2sh.rb +++ b/lib/job2sh.rb @@ -350,6 +350,7 @@ class Job2sh < Job out_line out_line "\texport define_files='#{define_files.keys.join ' '}'" define_files.each do |file, val| + out_line "\tmkdir -p $LKP_SRC/$(dirname #{file})" out_line "\tcat > $LKP_SRC/#{file} <<'EOF'" out_line val out_line "EOF" -- 2.23.0

2 1

[PATCH v4 lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 05 Nov '20

05 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH v1 lkp-tests] tests: add hackbench-git test script
by Zhang Yale 05 Nov '20

05 Nov '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/hackbench-git | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100755 tests/hackbench-git diff --git a/tests/hackbench-git b/tests/hackbench-git new file mode 100755 index 00000000..bb66bd1d --- /dev/null +++ b/tests/hackbench-git @@ -0,0 +1,33 @@ +#!/bin/bash +# - runtime +# - nr_threads +# - mode +# - ipc +# - iterations +# - loops + +## Hackbench-git is an old version, It is both a benchmark and a stress test for the Linux kernel scheduler. +## It's main job is to create a specified number of pairs of schedulable entities +## (either threads or traditional processes) which communicate via either sockets or +## pipes and time how long it takes for each pair to send data back and forth. + +[ "$ipc" = 'pipe' ] && ipc_option='-pipe' + +[ -n "$runtime" ] || runtime=600 +[ -n "$iterations" ] || iterations=10 + + +run_hackbench() +{ + # args: groups number, mode [process/thread], times + local hackbench_path + [ -f /lkp/benchmarks/hackbench-git/hackbench/hackbench ] && hackbench_path=/lkp/benchmarks/hackbench-git/hackbench/hackbench + echo "--------$ipc $mode num=$nr_threads--------" + for i in $(seq 1 $iterations); do + log_cmd $hackbench_path $ipc_option $nr_threads $mode $loops + done +} + +run_hackbench + + -- 2.23.0

2 2

[PATCH lkp-tests 2/3] lib/job.rb: fix bug: add_define_files not work
by Li Ping 05 Nov '20

05 Nov '20

[why] multi-task job.yaml like this: program-0: xxx: program-1: xxx: the add_define_files will not work, because the following paths actually don't exist: $LKP_SRC/*/$program-$d $LKP_SRC/*/*/$program-$d $LKP_SRC/*/$program-$d/* $LKP_SRC/*/*/$program-$d/* Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index 5c2c9517..2c06a475 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -626,6 +626,7 @@ class Job programs_params_hash = @job["pp"] || {} program_name_list = programs_params_hash.keys || [] program_name_list.each do |program| + program = $1 if program =~ /^(.*)-\d+?$/ # if job is run as makepkg, makepkg-deps, pack-deps, cci-makepkg, cci-depends # then use makepkg's benchmark as program to get define_files -- 2.23.0

1 1

[PATCH lkp-tests] lib/job.rb: fix bug: add_define_files not work
by Li Ping 05 Nov '20

05 Nov '20

[why] multi-task job.yaml like this: program-0: xxx: program-1: xxx: the add_define_files will not work, because the following paths actually don't exist: $LKP_SRC/*/$program-$d $LKP_SRC/*/*/$program-$d $LKP_SRC/*/$program-$d/* $LKP_SRC/*/*/$program-$d/* Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index fb826b8c..254562f7 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -620,6 +620,7 @@ class Job programs_params_hash = @job["pp"] || {} program_name_list = programs_params_hash.keys || [] program_name_list.each do |program| + program = $1 if program =~ /^(.*)-\d+$/ # if job is run as makepkg, makepkg-deps, pack-deps, cci-makepkg, cci-depends # then use makepkg's benchmark as program to get define_files -- 2.23.0

1 0

[PATCH v2 compass-ci 01/10] lib/mail-robot.rb
by Luan Shengde 05 Nov '20

05 Nov '20

mail robot for listening the mail dir monitor mailbox check new added email file in new email directory handle new email file check to apply account only if email's subject match 'apply account' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail-robot.rb | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100755 lib/mail-robot.rb diff --git a/lib/mail-robot.rb b/lib/mail-robot.rb new file mode 100755 index 0000000..ff1dbd8 --- /dev/null +++ b/lib/mail-robot.rb @@ -0,0 +1,45 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' + +MAILDIR = '/srv/cci/Maildir/.compass-ci' + +def monitor_dir(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + next if added.empty? + + added.each do |mail_file| + begin + handle_new_email(mail_file, mail_drafts) + rescue StandardError => e + puts e.message + puts e.backtrace + end + end + end + listener.start + sleep +end + +def handle_new_email(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + check_to_apply_account(mail_content) + + FileUtils.mv(mail_file, mail_drafts) +end + +def check_to_apply_account(mail_content) + return unless mail_content.subject == 'apply account' + + assign_uuid = AssignAccount.new(mail_content) + assign_uuid.check_to_send_account +end + +monitor_dir("#{MAILDIR}/new", "#{MAILDIR}/cur") -- 2.23.0

2 1

[PATCH lkp-tests] lib/job.rb: fix bug: add_define_files not work
by Li Ping 05 Nov '20

05 Nov '20

[why] multi-task job.yaml like this: program-0: xxx: program-1: xxx: the add_define_files will not work, because the following paths actually don't exist: $LKP_SRC/*/$program-$d $LKP_SRC/*/*/$program-$d $LKP_SRC/*/$program-$d/* $LKP_SRC/*/*/$program-$d/* Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index fb826b8c..12391b39 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -620,6 +620,7 @@ class Job programs_params_hash = @job["pp"] || {} program_name_list = programs_params_hash.keys || [] program_name_list.each do |program| + program = $1 if program =~ /^(.*)-\d+?$/ # if job is run as makepkg, makepkg-deps, pack-deps, cci-makepkg, cci-depends # then use makepkg's benchmark as program to get define_files -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: change the way get hostname
by Li Ping 05 Nov '20

05 Nov '20

[why] we can get hostname from the repo lab-z9 or get hostname from lkp-tests/hosts --- lib/job.rb | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 3383897c..75882075 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -334,20 +334,14 @@ class Job end def get_hosts_file + lab_hosts_file = get_lab_hosts_file + return lab_hosts_file if lab_hosts_file + hosts_file_name = @job['tbox_group'].split('--')[0] hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" + return hosts_file if File.file?(hosts_file) - lab_hosts_file = get_lab_hosts_file - if lab_hosts_file - hosts_file = lab_hosts_file - end - - if File.file?(hosts_file) - hosts_file - else - puts("hosts_file not exist: #{hosts_file}, maybe need check testbox field") - nil - end + puts("hosts_file not exist: #{hosts_file_name}, maybe need check testbox field") end def include_files -- 2.23.0

2 1