- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 2/2] tests/iozone: modify parameter parse process
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] iozone-bs.yaml has combined multiple parameter to single, so, iozone tool need change parse process. [how] modify parse process Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 50 ++++++++++++++++++++++---------------------------- 1 file changed, 22 insertions(+), 28 deletions(-) diff --git a/tests/iozone b/tests/iozone index 88a92a18..19c246f1 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,42 +1,36 @@ #!/bin/sh # - block_size # - file_size -# - write_rewrite -# - read_reread -# - random_read_write -# - read_backwards -# - rewrite_record -# - stride_read -# - fwrite_refwrite -# - fread_refread -# - random_mix -# - pwrite_repwrite -# - pread_repread -# - pwritev_repwritev -# - preadv_repreadv +# - test ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh - args="iozone" if [ -n "$block_size" ]; then args+=" -r $block_size" - [ -n "$file_size" ] && args+=" -s $file_size" - [ -n "$write_rewrite" ] && args+=" -i 0" - [ -n "$read_reread" ] && args+=" -i 1" - [ -n "$random_read_write" ] && args+=" -i 2" - [ -n "$read_backwards" ] && args+=" -i 3" - [ -n "$rewrite_record" ] && args+=" -i 4" - [ -n "$stride_read" ] && args+=" -i 5" - [ -n "$fwrite_refwrite" ] && args+=" -i 6" - [ -n "$fread_refread" ] && args+=" -i 7" - [ -n "$random_mix" ] && args+=" -i 8" - [ -n "$pwrite_repwrite" ] && args+=" -i 9" - [ -n "$pread_repread" ] && args+=" -i 10" - [ -n "$pwritev_repwritev" ] && args+=" -i 11" - [ -n "$preadv_repreadv" ] && args+=" -i 12" + [ -n "$file_size" ] && args+=" -s $file_size" + OLD_IFS="$IFS" + IFS="," + array=($test) + IFS="$OLD_IFS" + for ele in ${array[@]} + do + [ "$ele" == "write_rewrite" ] && args+=" -i 0" + [ "$ele" == "read_reread" ] && args+=" -i 1" + [ "$ele" == "random_read_write" ] && args+=" -i 2" + [ "$ele" == "read_backwards" ] && args+=" -i 3" + [ "$ele" == "rewrite_record" ] && args+=" -i 4" + [ "$ele" == "stride_read" ] && args+=" -i 5" + [ "$ele" == "fwrite_refwrite" ] && args+=" -i 6" + [ "$ele" == "fread_refread" ] && args+=" -i 7" + [ "$ele" == "random_mix" ] && args+=" -i 8" + [ "$ele" == "pwrite_repwrite" ] && args+=" -i 9" + [ "$ele" == "pread_repread" ] && args+=" -i 10" + [ "$ele" == "pwritev_repwritev" ] && args+=" -i 11" + [ "$ele" == "preadv_repreadv" ] && args+=" -i 12" + done else args+=" -a" fi -- 2.23.0

1 0

[PATCH lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH v3 lkp-tests] jobs/hackbench-git.yaml: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 lkp-tests] jobs: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..6494bc8 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file=/var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

3 2

[PATCH v3 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..cdba89c 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file = /var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

1 0

[PATCH v2 compass-ci] container: fix failed to build archlinux images
by Liu Yinsi 02 Nov '20

02 Nov '20

[why] when build archlinux images in x86_64 machine error: error: failed retrieving file 'core.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update core (failed to retrieve some files) error: failed retrieving file 'extra.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirror.archlinuxarm.org : Resolving timed out after 10000 milliseconds error: failed to update extra (download library error) error: failed retrieving file 'community.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update community (failed to retrieve some files) error: failed to synchronize all databases The command '/bin/sh -c pacman --needed --noprogressbar --noconfirm -Syu && pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk' returned a non-zero code: 1 becaue archlinux mirror not support x86_64 machine. [how] when deploy in x86_64 machine, change mirror http://mirrors.tuna.tsinghua.edu.cn/archlinuxarm/$arch/$repo => http://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/archlinux/build | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/archlinux/build b/container/archlinux/build index 81feda2..f96ad40 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ "$(arch)" == "x86_64" ] && + sed -i 's#archlinuxarm/$arch/$repo#archlinux/$repo/os/$arch#g' root/etc/pacman.d/mirrorlist + docker build -t archlinux:testbed . -- 2.23.0

3 4

[PATCH v2 compass-ci] src/lib/web_backend.rb: "/compare_candidates" tbox_group regex error
by Zhang Yuhang 02 Nov '20

02 Nov '20

[error info] 1. /\d+$/ will be matched, such as "xxx123". 2. `index = "xxx123".index('--') || "xxx123".rindex('-')` - index equal nil. 3. `r = r[0, nil]` - raise a TypeError. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index c3f6bef..55c5e4a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -47,7 +47,7 @@ end def filter_tbox_group(es_result) result = Set.new es_result.each do |r| - if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$)/ + if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$)/ index = r.index('--') || r.rindex('-') r = r[0, index] end -- 2.23.0

3 4

[PATCH v6 compass-ci 2/3] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 02 Nov '20

02 Nov '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..8bf7a65 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -87,6 +87,10 @@ class Job kernel_append_root kernel_params docker_image + kernel_version + vmlinuz_path + modules_path + headers_path ) macro method_missing(call) @@ -135,6 +139,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,9 +298,22 @@ class Job return true end + private def set_kernel_version + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + if self["kernel_version"]? + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz-#{kernel_version}" + self["modules_path"] = "#{boot_dir}/modules-#{kernel_version}.cgz" + self["headers_path"] = "#{boot_dir}/headers-#{kernel_version}.cgz" + else + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz" + self["modules_path"] = "#{boot_dir}/modules.cgz" + self["headers_path"] = "#{boot_dir}/headers.cgz" + end + end + private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{vmlinuz_path}")}" end private def common_initrds @@ -316,6 +334,10 @@ class Job "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{modules_path}")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{headers_path}")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 2

[PATCH lkp-tests] jobs/iozone-bs.yaml: fix mapper_parsing_exception
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] when key:value of fs: xfs in same line for .yaml file, submit job will cause some error occurred like below: submit /home/lukaiyi/lkp-tests/jobs/iozone-bs.yaml failed, got job_id=z9.144590, error: [{"type" => "mapper_parsing_exception", "reason" => "failed to parse [pp.fs.fs]"}] [how] change key and value of fs: xfs to different line, same to iosched: kyber Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index 868979fc..e2cd9f48 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -14,6 +14,11 @@ block_size: - 16m disk: 1HDD -fs: xfs -iosched: kyber + +fs: +- xfs + +iosched: +- kyber + iozone: -- 2.23.0

1 0