November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 1/2] jobs/iozone-bs.yaml: combine iozone's multiple -i parameter to single
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] avoid explosion of parameter for iozone-bs.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index e2cd9f48..f9ab305f 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -2,9 +2,7 @@ suite: iozone category: benchmark file_size: 4g -write_rewrite: true -read_reread: true -random_read_write: true +test: write_rewrite,read_reread,random_read_write block_size: - 64k -- 2.23.0

1 0

[PATCH v3 lkp-tests] jobs/hackbench-git.yaml: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 lkp-tests] jobs: fix testsuite and datasize filed
by Zhang Yale 02 Nov '20

02 Nov '20

[Why] hackbench-git is an old version required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/hackbench-git.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/hackbench-git.yaml b/jobs/hackbench-git.yaml index 12943c04..4a4b6a06 100644 --- a/jobs/hackbench-git.yaml +++ b/jobs/hackbench-git.yaml @@ -1,4 +1,4 @@ -suite: hackbench +suite: hackbench-git category: benchmark disable_latency_stats: 1 @@ -11,10 +11,10 @@ nr_threads: - 10 - 20 iterations: 10 -datasize: 1000 +loops: 1000 runtime: 300 -hackbench: +hackbench-git: mode: - process - threads -- 2.23.0

1 1

[PATCH v2 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..6494bc8 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file=/var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

3 2

[PATCH v3 compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 02 Nov '20

02 Nov '20

[why] Modify data upload directory to upload qemu image. qemu image uploaded by the user, used for testing [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..cdba89c 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file = /var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/result/upload/ read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..b24605b 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/result/upload:/srv/result/upload -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

1 0

[PATCH v2 compass-ci] container: fix failed to build archlinux images
by Liu Yinsi 02 Nov '20

02 Nov '20

[why] when build archlinux images in x86_64 machine error: error: failed retrieving file 'core.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'core.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update core (failed to retrieve some files) error: failed retrieving file 'extra.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'extra.db' from mirror.archlinuxarm.org : Resolving timed out after 10000 milliseconds error: failed to update extra (download library error) error: failed retrieving file 'community.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : The requested URL returned error: 404 error: failed to update community (failed to retrieve some files) error: failed to synchronize all databases The command '/bin/sh -c pacman --needed --noprogressbar --noconfirm -Syu && pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk' returned a non-zero code: 1 becaue archlinux mirror not support x86_64 machine. [how] when deploy in x86_64 machine, change mirror http://mirrors.tuna.tsinghua.edu.cn/archlinuxarm/$arch/$repo => http://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/archlinux/build | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/archlinux/build b/container/archlinux/build index 81feda2..f96ad40 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +[ "$(arch)" == "x86_64" ] && + sed -i 's#archlinuxarm/$arch/$repo#archlinux/$repo/os/$arch#g' root/etc/pacman.d/mirrorlist + docker build -t archlinux:testbed . -- 2.23.0

3 4

[PATCH v2 compass-ci] src/lib/web_backend.rb: "/compare_candidates" tbox_group regex error
by Zhang Yuhang 02 Nov '20

02 Nov '20

[error info] 1. /\d+$/ will be matched, such as "xxx123". 2. `index = "xxx123".index('--') || "xxx123".rindex('-')` - index equal nil. 3. `r = r[0, nil]` - raise a TypeError. Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index c3f6bef..55c5e4a 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -47,7 +47,7 @@ end def filter_tbox_group(es_result) result = Set.new es_result.each do |r| - if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$)/ + if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$)/ index = r.index('--') || r.rindex('-') r = r[0, index] end -- 2.23.0

3 4

[PATCH v6 compass-ci 2/3] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 02 Nov '20

02 Nov '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 0fd57e0..8bf7a65 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -87,6 +87,10 @@ class Job kernel_append_root kernel_params docker_image + kernel_version + vmlinuz_path + modules_path + headers_path ) macro method_missing(call) @@ -135,6 +139,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,9 +298,22 @@ class Job return true end + private def set_kernel_version + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + if self["kernel_version"]? + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz-#{kernel_version}" + self["modules_path"] = "#{boot_dir}/modules-#{kernel_version}.cgz" + self["headers_path"] = "#{boot_dir}/headers-#{kernel_version}.cgz" + else + self["vmlinuz_path"] = "#{boot_dir}/vmlinuz" + self["modules_path"] = "#{boot_dir}/modules.cgz" + self["headers_path"] = "#{boot_dir}/headers.cgz" + end + end + private def set_kernel_uri self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{vmlinuz_path}")}" end private def common_initrds @@ -316,6 +334,10 @@ class Job "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{modules_path}")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{headers_path}")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 2

[PATCH lkp-tests] jobs/iozone-bs.yaml: fix mapper_parsing_exception
by Lu Kaiyi 02 Nov '20

02 Nov '20

[why] when key:value of fs: xfs in same line for .yaml file, submit job will cause some error occurred like below: submit /home/lukaiyi/lkp-tests/jobs/iozone-bs.yaml failed, got job_id=z9.144590, error: [{"type" => "mapper_parsing_exception", "reason" => "failed to parse [pp.fs.fs]"}] [how] change key and value of fs: xfs to different line, same to iosched: kyber Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml index 868979fc..e2cd9f48 100644 --- a/jobs/iozone-bs.yaml +++ b/jobs/iozone-bs.yaml @@ -14,6 +14,11 @@ block_size: - 16m disk: 1HDD -fs: xfs -iosched: kyber + +fs: +- xfs + +iosched: +- kyber + iozone: -- 2.23.0

1 0

[PATCH compass-ci] based on queue field add the job to specified queue
by Xiao Shenwei 02 Nov '20

02 Nov '20

add queue field into job.yaml, refer to commit-id 912214cf97c125ed5387dc20ce394ee6db2b11b6 then use queue field replace tbox_group to add the job Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 80cf0b0..b4745aa 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -278,24 +278,24 @@ class Sched # success: [{"job_id" => job_id, "message" => "", job_state => "submit"}] # failure: [{"job_id" => "0", "message" => err_msg, job_state => "submit"}] def submit_single_job(job) - tbox_group = job.tbox_group + queue = job.queue return [{ "job_id" => "0", - "message" => "get tbox group failed", + "message" => "get queue failed", "job_state" => "submit", - }] unless tbox_group + }] unless queue # only single job will has "idle job" and "execute rate limiter" if job["idle_job"].empty? - tbox_group += "#{job.get_uuid_tag}" + queue += "#{job.get_uuid_tag}" else - tbox_group = "#{tbox_group}/idle" + queue = "#{queue}/idle" end - job_id = add_task(tbox_group, job.lab) + job_id = add_task(queue, job.lab) return [{ "job_id" => "0", - "message" => "add task queue sched/#{tbox_group} failed", + "message" => "add task queue sched/#{queue} failed", "job_state" => "submit", }] unless job_id @@ -311,9 +311,9 @@ class Sched end # return job_id - def add_task(tbox_group, lab) + def add_task(queue, lab) task_desc = JSON.parse(%({"domain": "compass-ci", "lab": "#{lab}"})) - response = @task_queue.add_task("sched/#{tbox_group}", task_desc) + response = @task_queue.add_task("sched/#{queue}", task_desc) JSON.parse(response[1].to_json)["id"].to_s if response[0] == 200 end -- 2.23.0

1 0