- Compass-ci - mailweb.openeuler.org

[PATCH v4 lkp-tests 1/2] tests/iozone: explicit support test=all
by Lu Kaiyi 25 Nov '20

25 Nov '20

[example] iozone-bs.yaml ... suite: iozone category: benchmark file_size: 4g test: all block_size: - 64k ... [output] ... Run began: Sat Nov 21 16:48:22 2020 Record Size 64 kB File size set to 4194304 kB Command line used: iozone -r 64k -s 4g -f /fs/sdb/iozonetest Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 4194304 64 13650392 11529057 7878373 7763936 7166639 10774159 7520609 16188351 7362482 9751567 9453804 7981606 7932187 --- tests/iozone | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/iozone b/tests/iozone index b6a1e206..bafc0087 100755 --- a/tests/iozone +++ b/tests/iozone @@ -19,6 +19,8 @@ if [ -n "$block_size" ]; then for ele in ${array[@]} do case $ele in + "all") args+=" -a" + ;; "write") args+=" -i 0" ;; "read") args+=" -i 1" -- 2.23.0

2 1

[PATCH v2 lkp-tests] lkp-tests/*: update mount option
by Wang Yong 24 Nov '20

24 Nov '20

[why] refer-to: compass-ci: commit 96584b45e528a4bad1f9 [how] add port=446 option to mount.cifs Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/cci-depends | 2 +- tests/cci-makepkg | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/cci-depends b/tests/cci-depends index 4b885dc0c..d5c21541d 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -30,7 +30,7 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} -mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" +mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" umask 002 diff --git a/tests/cci-makepkg b/tests/cci-makepkg index e49f66998..d08f23718 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -36,7 +36,7 @@ cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" [ -n "$LKP_SERVER" ] && { is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" + mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" } curl_pkgfile() -- 2.23.0

1 0

[PATCH lkp-tests] lkp-tests/*: update AUR Helper for archlinux
by Wang Yong 24 Nov '20

24 Nov '20

yaourt is unmaintained and yay also support native pacman operation, so only minor changes are needed Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 15 +++++++++++++++ distro/installer/archlinux | 15 +++++++++------ lib/install.sh | 2 +- 3 files changed, 25 insertions(+), 7 deletions(-) diff --git a/distro/archlinux b/distro/archlinux index 408c5fa23..8e3fbbce7 100755 --- a/distro/archlinux +++ b/distro/archlinux @@ -109,3 +109,18 @@ pack_benchmark() ) done } + +add_aur_helper() +{ + pacman -Q yay 2>/dev/null && return + + local user=lkp + local dir=/home/${user} + + mkdir -p ${dir} && chown ${user}:${user} ${dir} + echo "lkp ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers + + pacman -S --needed --noconfirm git base-devel 2>/dev/null + su - ${user} -c "git clone https://aur.archlinux.org/yay.git" + su - ${user} -c "cd yay && makepkg -si --needed --noconfirm" +} diff --git a/distro/installer/archlinux b/distro/installer/archlinux index 27cdc63cf..af98e466f 100755 --- a/distro/installer/archlinux +++ b/distro/installer/archlinux @@ -2,12 +2,15 @@ # Archlinux package installation -# yaourt is a wrapper for pacman that adds automated access -# to the AUR. -# See: https://wiki.archlinux.org/index.php/Yaourt +# Yet Another Yogurt - An AUR Helper Written in Go +# Yay is based on the design of yaourt, apacman and pacaur +# See: https://github.com/Jguer/yay -# yaourt not support root to build package +# Yay not support root to build package # Use regular user to do this -regular_user=`grep 1000 /etc/passwd | awk -F':' '{ print $1 }'` +. $LKP_SRC/distro/${DISTRO} +add_aur_helper -sudo -u $regular_user yaourt -S --noconfirm $* +regular_user=`grep 1090 /etc/passwd | awk -F':' '{ print $1 }'` + +sudo -u $regular_user yay -S --needed --noconfirm $* diff --git a/lib/install.sh b/lib/install.sh index 881ade9f4..bf7bf21d7 100755 --- a/lib/install.sh +++ b/lib/install.sh @@ -17,7 +17,7 @@ sync_distro_sources() yum update fi ;; redhat) yum update ;; - archlinux) yaourt -Sy ;; + archlinux) yay -Sy ;; opensuse) zypper update ;; oracle) yum update ;; -- 2.23.0

1 1

[PATCH v3 lkp-tests] tests/iozone: explicit support test=all and add invalid test option identification
by Lu Kaiyi 24 Nov '20

24 Nov '20

[example] iozone-bs.yaml ... suite: iozone category: benchmark file_size: 4g test: all block_size: - 64k ... [output] ... Run began: Sat Nov 21 16:48:22 2020 Record Size 64 kB File size set to 4194304 kB Command line used: iozone -r 64k -s 4g -f /fs/sdb/iozonetest Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 4194304 64 13650392 11529057 7878373 7763936 7166639 10774159 7520609 16188351 7362482 9751567 9453804 7981606 7932187 --- tests/iozone | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/iozone b/tests/iozone index b6a1e206..5186766f 100755 --- a/tests/iozone +++ b/tests/iozone @@ -19,6 +19,8 @@ if [ -n "$block_size" ]; then for ele in ${array[@]} do case $ele in + "all") args+=" -a" + ;; "write") args+=" -i 0" ;; "read") args+=" -i 1" @@ -45,6 +47,9 @@ if [ -n "$block_size" ]; then ;; "preadv") args+=" -i 12" ;; + *) echo "invalid test option, please check again!" + exit 1 + ;; esac done else -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: avoid saving empty dictionary
by Wei Jihui 24 Nov '20

24 Nov '20

[why] when add a program to pp, it will declare { pp: program: {} } if the program does not has args, it will keep {} to pp. [how] give a nil to program, then add program args when the program adds args. it will: { pp: program: } Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 2e94316c..84ab5056 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -580,11 +580,14 @@ class Job program_name = hash_key_re_string(h, p_n, @job['pp'].keys) next if program_name.nil? - @job["pp"][program_name] = Hash.new() + @job['pp'][program_name] = nil options_array = options_hash.keys options_array.each do |option| option_value = get_program_option_value(@job, program_name, option) - @job["pp"][program_name][option] = option_value unless option_value.nil? + next if option_value.nil? + + @job['pp'][program_name] ||= {} + @job['pp'][program_name][option] = option_value end end end -- 2.23.0

1 0

[PATCH v3 compass-ci] optimize document for apply account
by Luan Shengde 24 Nov '20

24 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index d718d0d..a4902f5 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -36,20 +36,12 @@ for external user email subject: apply account email body: - my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + example: + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… attachment: ssh pubkey file - it will return email with my_email, my_name, my_uuid if successfully applied account -2. prepare to submit job - execute on user's local host - - config default yaml file: - ~/.config/compass-ci/default/account.yaml - - prepare job.yaml - - submit job - -3. apply account email format - example: + - email example: To: compass-ci(a)qq.com Subject: apply account @@ -58,5 +50,19 @@ for external user # We'll validate whether the URL contains your email. # for example, my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - # attach your ssh pub key to the email as attachment + +2. successfully applied account email + it will return an email with following information: + - my_email + - my_name + - my_uuid + - SCHED_HOST + - SCHED_PORT + +3. environment configuration + follow steps in the email to finish the following configuration + - setup default yaml + ~/.config/compass-ci/default/account.yaml + - download and install lkp-tests + - prepare to submit jobs -- 2.23.0

1 0

[PATCH v10 compass-ci 2/2] lib: fix group compare no output when dimension number less than 2
by Lu Kaiyi 24 Nov '20

24 Nov '20

[why] When do group compare, if returned groups of auto_group like below (the dimension number less than 2 for group_key): groups = { group_key_1 => { dimension_1 => matrix_1 }, group_key_2 => {} } the group_key will be deleted, then, group compare will have no output. [how] Reduce dimension number limit from 2 to 1. [example] lukaiyi@z9 ~% compare "suite=iperf" -d "os" in z9 [before] <null> [after] os_arch=aarch64/os_version=20/pp.iperf.protocol=tcp/pp.iperf.runtime=300/tbox_group=vm-hi1620-2p8g--liangt openeuler metric -------------------- ------------------------------ fails:runs | 18:18 stderr.EOF' 18:18 stderr.has_stderr openeuler metric -------------------- ------------------------------ %stddev \ 3.496630e+10 ± 19% iperf.tcp.receiver.bps 3.497117e+10 ± 19% iperf.tcp.sender.bps 300.23 iperf.time.elapsed_time 300.23 iperf.time.elapsed_time.max ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 2 +- lib/params_group.rb | 3 +-- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 2065039..82cf033 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,7 +116,7 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| - if value.size < 2 + if value.empty? groups.delete(group_key) next end diff --git a/lib/params_group.rb b/lib/params_group.rb index 40191a9..48d651b 100644 --- a/lib/params_group.rb +++ b/lib/params_group.rb @@ -34,8 +34,7 @@ def auto_group(jobs_list, dimensions) return [] if dimensions.empty? jobs_list = extract_jobs_list(jobs_list) - groups = group(jobs_list, dimensions) - return remove_singleton(groups) + group(jobs_list, dimensions) end def extract_jobs_list(jobs_list) -- 2.23.0

1 0

[PATCH v10 compass-ci 1/2] lib/matrix2.rb: refactor performance optimization
by Lu Kaiyi 24 Nov '20

24 Nov '20

[why] Avoid to execute some unnecessary operations and improve performance. [how] Move the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index ff4ea31..2065039 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

1 0

[PATCH v2 lkp-tests] spec/submit: add job yaml for multi-args test
by Bai Jing 24 Nov '20

24 Nov '20

[test cmd] respec $LKP_SRC/spec/submit_spec.rb [output] lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml lkp-tests/spec/submit/matrices-args/unixbench-spec.yaml => /lkp-tests/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml Signed-off-by: Bai Jing <799286817(a)qq.com> --- spec/submit/matrices-args/unixbench-spec.yaml | 10 +++ ...unixbench-spec-1-300-whetstone-double.yaml | 78 +++++++++++++++++++ .../unixbench-spec-3-300-shell1.yaml | 78 +++++++++++++++++++ spec/submit_spec.rb | 4 + 4 files changed, 170 insertions(+) create mode 100644 spec/submit/matrices-args/unixbench-spec.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml create mode 100644 spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml diff --git a/spec/submit/matrices-args/unixbench-spec.yaml b/spec/submit/matrices-args/unixbench-spec.yaml new file mode 100644 index 00000000..39447528 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec.yaml @@ -0,0 +1,10 @@ +suite: unixbench +category: benchmark + +runtime: 300s + +nr_task test: + - 1 whetstone-double + - 3 shell1 + +unixbench: diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml new file mode 100644 index 00000000..abc7fd53 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-1-300-whetstone-double.yaml @@ -0,0 +1,78 @@ +--- +suite: unixbench +category: benchmark +runtime: 300 +unixbenc: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '1' +test: whetstone-double +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kms: +boot-tim: +uptim: +iosta: +heartbea: +vmsta: +numa-numasta: +numa-vmsta: +numa-meminf: +proc-vmsta: +proc-sta: +meminf: +slabinf: +interrupt: +lock_sta: +latency_stat: +softirq: +bdi_dev_mappin: +diskstat: +nfssta: +cpuidl: +cpufreq-stat: +sched_debu: +perf-sta: +mpsta: +perf-profil: +cpufreq_governo: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: whetstone-double + nr_task: '1' + cpufreq_governor: {} +monitors: + kms: + boot-tim: + uptim: + iosta: + heartbea: + vmsta: + numa-numasta: + numa-vmsta: + numa-meminf: + proc-vmsta: + proc-sta: + meminf: + slabinf: + interrupt: + lock_sta: + latency_stat: + softirq: + bdi_dev_mappin: + diskstat: + nfssta: + cpuidl: + cpufreq-stat: + sched_debu: + perf-sta: + mpsta: + perf-profil: diff --git a/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml new file mode 100644 index 00000000..559a04d2 --- /dev/null +++ b/spec/submit/matrices-args/unixbench-spec/unixbench-spec-3-300-shell1.yaml @@ -0,0 +1,78 @@ +--- +suite: unixbench +category: benchmark +runtime: 300 +unixbenc: +lab: spec_lab +testbox: vm-2p8g--spec_submit +nr_task: '3' +test: shell1 +tbox_group: vm-2p8g--spec_submit +provider: qemu +template: kvm +nr_node: 1 +nr_cpu: 2 +memory: 8G +arch: aarch64 +kms: +boot-tim: +uptim: +iosta: +heartbea: +vmsta: +numa-numasta: +numa-vmsta: +numa-meminf: +proc-vmsta: +proc-sta: +meminf: +slabinf: +interrupt: +lock_sta: +latency_stat: +softirq: +bdi_dev_mappin: +diskstat: +nfssta: +cpuidl: +cpufreq-stat: +sched_debu: +perf-sta: +mpsta: +perf-profil: +cpufreq_governo: +need_kconfig: +- CONFIG_KVM_GUEST=y +pp: + unixbench: + runtime: 300 + test: shell1 + nr_task: '3' + cpufreq_governor: {} +monitors: + kms: + boot-tim: + uptim: + iosta: + heartbea: + vmsta: + numa-numasta: + numa-vmsta: + numa-meminf: + proc-vmsta: + proc-sta: + meminf: + slabinf: + interrupt: + lock_sta: + latency_stat: + softirq: + bdi_dev_mappin: + diskstat: + nfssta: + cpuidl: + cpufreq-stat: + sched_debu: + perf-sta: + mpsta: + perf-profil: diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index f365f35b..ebebad5d 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -40,4 +40,8 @@ describe 'submit job spec' do it 'link jobs spec' do submit_job('link_jobs') end + + it 'matrices-args spec' do + submit_job('matrices-args') + end end -- 2.23.0

1 0

[PATCH v2 compass-ci] optimize document for apply account
by Luan Shengde 24 Nov '20

24 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index d718d0d..1844c86 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -36,20 +36,12 @@ for external user email subject: apply account email body: - my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + example: + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… attachment: ssh pubkey file - it will return email with my_email, my_name, my_uuid if successfully applied account -2. prepare to submit job - execute on user's local host - - config default yaml file: - ~/.config/compass-ci/default/account.yaml - - prepare job.yaml - - submit job - -3. apply account email format - example: + - email example: To: compass-ci(a)qq.com Subject: apply account @@ -58,5 +50,19 @@ for external user # We'll validate whether the URL contains your email. # for example, my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - # attach your ssh pub key to the email as attachment + +2. successfully applied account email + it will return an email with following infomation: + - my_email + - my_name + - my_uuid + - SCHED_HOST + - SCHED_PORT + +3. environment configuration + follow steps in the email to finish the following configuration + - setup default yaml + ~/.config/compass-ci/default/account.yaml + - download lkp-tests and dependencies + - prepare to submit jobs -- 2.23.0

2 2