- Compass-ci - mailweb.openeuler.org

[PATCH v6 lkp-tests] tests/build-pkg: change location for *.cgz file
by Liu Shaofei 05 Nov '20

05 Nov '20

[why] the path of *.cgz file generated by build-pkg script cannot be obtained during test. So the path of cgz file need to be changed. [how] below is example for openeuler: - path: /srv/initrd/build-pkg/initramfs/openeuler/aarch64/20.03/nasm-git - filename: latest.cgz -> f71820d6055373c9b9381615e28a28f307bb2df4.cgz f71820d6055373c9b9381615e28a28f307bb2df4.cgz v1.8.1.cgz -> 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz 92c89f904a14d4bfc25abcd2c3d5c2da35a589d6.cgz v1.5.1.cgz -> df0584e46db9e7b412626deab10b923504745b2c.cgz df0584e46db9e7b412626deab10b923504745b2c.cgz Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index b10b1497..7c494a15 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,10 +2,12 @@ # - os # - os_arch # - os_version +# - os_mount # - pkgbuild_repo # - upstream_repo # - upstream_url # - upstream_commit +# - upstream_tag ## makepkg is a script that automates the building of packages; it can download and validate source files, ## check dependencies, configure build-time settings, compile the sources, install into a temporary root, @@ -21,15 +23,19 @@ check_vars() [ -n "$os_version" ] || die "os_version is empty" [ -n "$pkgbuild_repo" ] || die "pkgbuild_repo is empty" [ -n "$upstream_commit" ] || die "upstream_commit is empty" + [ -n "$os_mount" ] || die "os_mount is empty" } mount_dest() { + [[ "$os_mount" = "cifs" ]] && os_mount="nfs" + pack_to=${os_mount}/${os}/${os_arch}/${os_version} + PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" [ -n "$LKP_SERVER" ] && { - mount $LKP_SERVER:$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } } @@ -75,12 +81,22 @@ build_source_pkg() sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD } - cgz_name="$PKG_MNT/${pkgname}/${upstream_commit}.cgz" + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --check --config $LKP_SRC/etc/makepkg.conf 2>&1 + $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 +} + +create_softlink() +{ + [ -n "$upstream_tag" ] || return 0 + [ -e "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" ] && return 0 + + ln -sf "$(basename $(realpath ${cgz_name}))" "$PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz" || return 0 + echo "create soft link: $PKG_MNT/${pack_to}/${pkgname}/${upstream_tag}.cgz -> $(basename $(realpath ${cgz_name}))" } check_vars mount_dest request_pkg build_source_pkg +create_softlink -- 2.23.0

3 3

[PATCH lkp-tests 3/3] lib/job.rb: change the way get hostname
by Li Ping 05 Nov '20

05 Nov '20

[why] we can get hostname from the repo lab-z9 or get hostname from lkp-tests/hosts before: get hostname from lab-z9, but the hostname not exist in lkp-tests/hosts the job will puts the error message: hosts_file not exist ... but there is no need to puts such error message, if we have got the hostname from lab-z9 --- lib/job.rb | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 2c06a475..12391b39 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -334,20 +334,14 @@ class Job end def get_hosts_file + lab_hosts_file = get_lab_hosts_file + return lab_hosts_file if lab_hosts_file + hosts_file_name = @job['tbox_group'].split('--')[0] hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" + return hosts_file if File.file?(hosts_file) - lab_hosts_file = get_lab_hosts_file - if lab_hosts_file - hosts_file = lab_hosts_file - end - - if File.file?(hosts_file) - hosts_file - else - puts("hosts_file not exist: #{hosts_file}, maybe need check testbox field") - nil - end + puts("hosts_file not exist: #{hosts_file_name}, maybe need check testbox field") end def include_files -- 2.23.0

4 5

[PATCH v3 lkp-tests 3/3] lib/job2sh.rb: fix bug: can not find the file:pkg/$program/PKGBUILD
by Li Ping 05 Nov '20

05 Nov '20

[why] when the $dir not exist, cat > $dir/$file << 'EOF' xxx EOF will echo error message: can not find the file $dir/$file Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job2sh.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job2sh.rb b/lib/job2sh.rb index 9c95ff4b..7e6a5d7c 100755 --- a/lib/job2sh.rb +++ b/lib/job2sh.rb @@ -350,6 +350,7 @@ class Job2sh < Job out_line out_line "\texport define_files='#{define_files.keys.join ' '}'" define_files.each do |file, val| + out_line "\tmkdir -p $LKP_SRC/$(dirname #{file})" out_line "\tcat > $LKP_SRC/#{file} <<'EOF'" out_line val out_line "EOF" -- 2.23.0

2 3

[PATCH v5 compass-ci 1/3] container/assign-account: assign-account.rb
by Luan Shengde 05 Nov '20

05 Nov '20

disable assigning account for user if there is no: - my_email - my_name - my_uuid [why]: my_email, my_name, my_uuid is required when initialize the default config file [how]: check if the parsed data has keys: - my_email - my_name - my_uuid Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/assign-account/assign-account.rb | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/container/assign-account/assign-account.rb b/container/assign-account/assign-account.rb index e356c18..834f666 100755 --- a/container/assign-account/assign-account.rb +++ b/container/assign-account/assign-account.rb @@ -7,7 +7,7 @@ require 'sinatra' require 'open3' require 'json' require 'yaml' -require_relative 'get_account_info.rb' +require_relative 'get_account_info' set :bind, '0.0.0.0' set :port, 29999 @@ -17,8 +17,17 @@ get '/assign_account' do data = YAML.safe_load request.body.read rescue StandardError => e puts e.message + puts e.backtrace end + check_to_assign_account(data) +end + +def check_to_assign_account(data) + lacked_info = %w[my_email my_name my_uuid] - data.keys + error_message = "lack of my infos: #{lacked_info.join(', ')}." + raise error_message unless lacked_info.empty? + ref_account_info = AccountStorage.new(data) account_info = ref_account_info.setup_jumper_account_info -- 2.23.0

2 2

[PATCH v4 lkp-tests 2/2] tests/iozone: modify the way of parsing parameter
by Lu Kaiyi 05 Nov '20

05 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool also need change the way of parsing parameter. [how] modify the way of parsing parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 66 ++++++++++++++++++++++++++++++---------------------- 1 file changed, 38 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..e126948d 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,52 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + # reset IFS to split $test by "," and then restore default + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + case $ele in + "write_rewrite") args+=" -i 0" + ;; + "read_reread") args+=" -i 1" + ;; + "random_read_write") args+=" -i 2" + ;; + "read_backwards") args+=" -i 3" + ;; + "rewrite_record") args+=" -i 4" + ;; + "stride_read") args+=" -i 5" + ;; + "fwrite_refwrite") args+=" -i 6" + ;; + "fread_refread") args+=" -i 7" + ;; + "random_mix") args+=" -i 8" + ;; + "pwrite_repwrite") args+=" -i 9" + ;; + "pread_repread") args+=" -i 10" + ;; + "pwritev_repwritev") args+=" -i 11" + ;; + "preadv_repreadv") args+=" -i 12" + ;; + esac + done else args+=" -a" fi -- 2.23.0

2 1

[PATCH v2 compass-ci 06/10] container/mail-robot: run.rb
by Luan Shengde 05 Nov '20

05 Nov '20

invoke: send-apply-account-email assign-jumper-account parse-apply-account-email apply-account mail-robot Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/run.rb | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100755 container/mail-robot/run.rb diff --git a/container/mail-robot/run.rb b/container/mail-robot/run.rb new file mode 100755 index 0000000..8ac1d6e --- /dev/null +++ b/container/mail-robot/run.rb @@ -0,0 +1,10 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative 'send-apply-account-email' +require_relative 'assign-jumper-account' +require_relative 'parse-apply-account-email' +require_relative 'apply-account' +require_relative '../../lib/mail-robot' -- 2.23.0

3 2

[PATCH lkp-tests 1/3] lib/job2sh.rb: fix bug: can not find the file:pkg/$program/PKGBUILD
by Li Ping 05 Nov '20

05 Nov '20

[why] when the $dir not exist, cat > $dir/$file << 'EOF' xxx EOF will echo error message: can not find the file $dir/$file Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job2sh.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job2sh.rb b/lib/job2sh.rb index 9c95ff4b..7e6a5d7c 100755 --- a/lib/job2sh.rb +++ b/lib/job2sh.rb @@ -350,6 +350,7 @@ class Job2sh < Job out_line out_line "\texport define_files='#{define_files.keys.join ' '}'" define_files.each do |file, val| + out_line "\tmkdir -p $LKP_SRC/$(dirname #{file})" out_line "\tcat > $LKP_SRC/#{file} <<'EOF'" out_line val out_line "EOF" -- 2.23.0

2 1

[PATCH v4 lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 05 Nov '20

05 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH v1 lkp-tests] tests: add hackbench-git test script
by Zhang Yale 05 Nov '20

05 Nov '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/hackbench-git | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100755 tests/hackbench-git diff --git a/tests/hackbench-git b/tests/hackbench-git new file mode 100755 index 00000000..bb66bd1d --- /dev/null +++ b/tests/hackbench-git @@ -0,0 +1,33 @@ +#!/bin/bash +# - runtime +# - nr_threads +# - mode +# - ipc +# - iterations +# - loops + +## Hackbench-git is an old version, It is both a benchmark and a stress test for the Linux kernel scheduler. +## It's main job is to create a specified number of pairs of schedulable entities +## (either threads or traditional processes) which communicate via either sockets or +## pipes and time how long it takes for each pair to send data back and forth. + +[ "$ipc" = 'pipe' ] && ipc_option='-pipe' + +[ -n "$runtime" ] || runtime=600 +[ -n "$iterations" ] || iterations=10 + + +run_hackbench() +{ + # args: groups number, mode [process/thread], times + local hackbench_path + [ -f /lkp/benchmarks/hackbench-git/hackbench/hackbench ] && hackbench_path=/lkp/benchmarks/hackbench-git/hackbench/hackbench + echo "--------$ipc $mode num=$nr_threads--------" + for i in $(seq 1 $iterations); do + log_cmd $hackbench_path $ipc_option $nr_threads $mode $loops + done +} + +run_hackbench + + -- 2.23.0

2 2

[PATCH lkp-tests 2/3] lib/job.rb: fix bug: add_define_files not work
by Li Ping 05 Nov '20

05 Nov '20

[why] multi-task job.yaml like this: program-0: xxx: program-1: xxx: the add_define_files will not work, because the following paths actually don't exist: $LKP_SRC/*/$program-$d $LKP_SRC/*/*/$program-$d $LKP_SRC/*/$program-$d/* $LKP_SRC/*/*/$program-$d/* Signed-off-by: Li Ping <15396232681(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index 5c2c9517..2c06a475 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -626,6 +626,7 @@ class Job programs_params_hash = @job["pp"] || {} program_name_list = programs_params_hash.keys || [] program_name_list.each do |program| + program = $1 if program =~ /^(.*)-\d+?$/ # if job is run as makepkg, makepkg-deps, pack-deps, cci-makepkg, cci-depends # then use makepkg's benchmark as program to get define_files -- 2.23.0

1 1