November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] dracut-initrd: delete unused kernel module iscsi from dracut
by Xu Xijian 18 Nov '20

18 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/dracut-initrd/run b/container/dracut-initrd/run index b0c9b30..6301ee2 100755 --- a/container/dracut-initrd/run +++ b/container/dracut-initrd/run @@ -43,7 +43,7 @@ cmd=( -v $root/lib/modules:/lib/modules debian:dracut bash -c - "dracut --force --kver $kver -k $kernel_modules $initrd_output && + "dracut -o iscsi --omit-drivers iscsi --force --kver $kver -k $kernel_modules $initrd_output && chmod 644 $initrd_output" # example: -- 2.23.0

1 0

[PATCH v6 lkp-tests] tests/cci-makepkg: update gcc for centos7 when use initramfs os
by Lin Jiaxin 18 Nov '20

18 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 530c9fef..905ad689 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -88,13 +88,22 @@ update_shared_pkg() echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } +centos_update_gcc() +{ + if grep -sqF "CentOS Linux release 7" /etc/centos-release; then + [ "$os_mount" = "initramfs" ] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable + fi +} + [ -n "$PKGBUILD_TAG" ] && download_pkgfile distro_install_depends lkp-dev -[ $os = "centos" ] && -grep -sqF "CentOS Linux release 7" /etc/centos-release && -source /opt/rh/devtoolset-8/enable +[ "$os" = "centos" ] && centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

2 1

[PATCH compass-ci] updata document for apply account
by Luan Shengde 18 Nov '20

18 Nov '20

internal user: CLI tool: answerback-email external user: send 'apply account' email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/apply-ssh-account.md | 49 ++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/doc/apply-ssh-account.md b/doc/apply-ssh-account.md index 69d8df6..d718d0d 100644 --- a/doc/apply-ssh-account.md +++ b/doc/apply-ssh-account.md @@ -1,18 +1,22 @@ -# apply for ssh account +# apply for jumper account ## purpose -The ssh account is used to login our jumper VM, where you can +The jumper account is used to login the jumper VM, where you can - submit a job - ssh into a testbox +- read output of the job ## steps overview -1. send email to apply for ssh account - If approved, you'll get a response email with: +### manually assign account +for internal user +1. run assign account tool + usage: + answerback-mail.rb [-e|--email email] [-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey] - login command: - ssh -p port account@server_ip + login command: + ssh -p port account@server_ip 2. login authentication - use pub key @@ -20,18 +24,39 @@ The ssh account is used to login our jumper VM, where you can 3. submit jobs - prepare job.yaml - - run job + - submit job - command: submit job.yaml -## example apply email +### apply account with email +for external user +1. send apply account email + - send apply account email to: compass-ci(a)qq.com + attention: + email subject: + apply account + email body: + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + attachment: + ssh pubkey file + it will return email with my_email, my_name, my_uuid if successfully applied account + +2. prepare to submit job + execute on user's local host + - config default yaml file: + ~/.config/compass-ci/default/account.yaml + - prepare job.yaml + - submit job + +3. apply account email format + example: - To: team(a)crystal.ci - Subject: apply ssh account + To: compass-ci(a)qq.com + Subject: apply account # Show a commit URL that you contributed to an OSS project # We'll validate whether the URL contains your email. # for example, - commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… - # attach your ssh pub key as file (optionally but highly recommended) + # attach your ssh pub key to the email as attachment -- 2.23.0

1 0

[PATCH v5 lkp-tests] tests/cci-makepkg: update gcc for centos7 when use initramfs os
by Lin Jiaxin 18 Nov '20

18 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 530c9fef..865b2a9d 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -88,13 +88,22 @@ update_shared_pkg() echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } +centos_update_gcc() +{ + if $(grep -sqF "CentOS Linux release 7" /etc/centos-release ); then + [ "$os_mount" = "initramfs" ] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable + fi +} + [ -n "$PKGBUILD_TAG" ] && download_pkgfile distro_install_depends lkp-dev -[ $os = "centos" ] && -grep -sqF "CentOS Linux release 7" /etc/centos-release && -source /opt/rh/devtoolset-8/enable +[ "$os" = "centos" ] && centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

2 1

[PATCH compass-ci] container: add other group read permission
by Bai Jing 18 Nov '20

18 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- container/conserver/conserver | 1 + 1 file changed, 1 insertion(+) diff --git a/container/conserver/conserver b/container/conserver/conserver index 94b1909..6886127 100755 --- a/container/conserver/conserver +++ b/container/conserver/conserver @@ -2,4 +2,5 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +umask 0002 conserver -o -O1 -C /etc/conserver.cf -- 2.23.0

1 0

[PATCH v4 lkp-tests 2/2] tests/cci-makepkg: update gcc for centos7 when use initramfs os
by Lin Jiaxin 18 Nov '20

18 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a62b5be0..b090559f 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -88,13 +88,22 @@ update_shared_pkg() echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } +centos_update_gcc() +{ + if $(grep -sqF "CentOS Linux release 7" /etc/centos-release ); then + [[ "$os_mount" = "initramfs" ]] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable + fi +} + [ -n "$PKGBUILD_TAG" ] && get_pkgfile distro_install_depends lkp-dev -[ $os = "centos" ] && -grep -sqF "CentOS Linux release 7" /etc/centos-release && -source /opt/rh/devtoolset-8/enable +[ $os = "centos" ] && centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

2 1

[PATCH compass-ci] remove actual ip in compass-ci code
by Xiao Shenwei 18 Nov '20

18 Nov '20

obtain the actual ip address from configuration file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- src/features/jobs/right_iperf.yaml | 2 +- src/scheduler/README.md | 2 +- user-client/helper/iperf.yaml | 2 +- user-client/jobs/iperf-pxe.yaml | 2 +- user-client/jobs/iperf-vm.yaml | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..ca86e88 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -39,7 +39,7 @@ the returned data for setup_jumper_account_info like: { "account" => "guest", "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "jumper_ip" => "0.0.0.0", "jumper_port" => "10000" } diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index f81af38..e239ccd 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/src/scheduler/README.md b/src/scheduler/README.md index fdc7027..8e3a197 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -6,7 +6,7 @@ - And set the environment variables needed for running (modify "/etc/compass-ci/defaults/crystal.yaml", and run $CCI_SRC/container/defconfig.rb | defconfig.sh) LKP_SRC: ~/code/lkp-tests # path to your lkp-tests repo CCI_SRC: ~/code/compass-ci # path to your compass-ci repo - SCHED_HOST: 172.168.131.131 # hostname or ip of the scheduler + SCHED_HOST: 172.17.0.1 # hostname or ip of the scheduler SCHED_PORT: 3000 # port of the scheduler JOB_INDEX_TYPE: "jobs/_doc" # es job document index and type - '$'{variable} is used at shell command, means the value of variable diff --git a/user-client/helper/iperf.yaml b/user-client/helper/iperf.yaml index 1509470..30e929f 100644 --- a/user-client/helper/iperf.yaml +++ b/user-client/helper/iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-pxe.yaml b/user-client/jobs/iperf-pxe.yaml index 209c5d6..cb5a513 100644 --- a/user-client/jobs/iperf-pxe.yaml +++ b/user-client/jobs/iperf-pxe.yaml @@ -53,7 +53,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-vm.yaml b/user-client/jobs/iperf-vm.yaml index 434b964..c976bed 100644 --- a/user-client/jobs/iperf-vm.yaml +++ b/user-client/jobs/iperf-vm.yaml @@ -34,7 +34,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x -- 2.23.0

2 1

[PATCH v5 compass-ci] container: fix failed to build archlinux images in x86_64 machine
by Liu Yinsi 18 Nov '20

18 Nov '20

[why] when build archlinux images in x86_64 machine error: failed retrieving file 'community.db' from mirrors.tuna.tsinghua.edu.cn : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirrors.163.com : The requested URL returned error: 404 error: failed retrieving file 'community.db' from mirror.archlinuxarm.org : The requested URL returned error:404 error: failed to update community (failed to retrieve some files) error: failed to synchronize all databases The command '/bin/sh -c pacman --needed --noprogressbar --noconfirm -Syu && pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk' returned a non-zero code: 1 because archlinux mirrorlist not support x86_64 machine. [how] 1. add 2 folder root/aarch64/etc/pacman.d/mirrorlist and root/x86_64/etc/pacman.d/mirrorlist; In addition, 'COPY' in Dockerfile, the directory itself is not copied, only its contents. Please see: https://docs.docker.com/engine/reference/builder/#copy 2. use $(arch) to choose mirrorlist according to different system architecture. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/archlinux/Dockerfile | 6 +++++- container/archlinux/build | 2 +- .../archlinux/root/{ => aarch64}/etc/pacman.d/mirrorlist | 0 container/archlinux/root/x86_64/etc/pacman.d/mirrorlist | 1 + 4 files changed, 7 insertions(+), 2 deletions(-) rename container/archlinux/root/{ => aarch64}/etc/pacman.d/mirrorlist (100%) create mode 100644 container/archlinux/root/x86_64/etc/pacman.d/mirrorlist diff --git a/container/archlinux/Dockerfile b/container/archlinux/Dockerfile index c0f05d3..1f80ae0 100644 --- a/container/archlinux/Dockerfile +++ b/container/archlinux/Dockerfile @@ -5,7 +5,11 @@ FROM lopsided/archlinux MAINTAINER Wu Fenguang <wfg(a)mail.ustc.edu.cn> -COPY root / +ARG ARCH + +COPY root/$ARCH / + RUN chmod 755 /etc + RUN pacman --needed --noprogressbar --noconfirm -Syu && \ pacman --needed --noprogressbar --noconfirm -S bash zsh git openssh rsync make gcc tzdata sudo coreutils util-linux vim gawk diff --git a/container/archlinux/build b/container/archlinux/build index 81feda2..9749489 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -docker build -t archlinux:testbed . +docker build --build-arg ARCH=$(arch) -t archlinux:testbed . diff --git a/container/archlinux/root/etc/pacman.d/mirrorlist b/container/archlinux/root/aarch64/etc/pacman.d/mirrorlist similarity index 100% rename from container/archlinux/root/etc/pacman.d/mirrorlist rename to container/archlinux/root/aarch64/etc/pacman.d/mirrorlist diff --git a/container/archlinux/root/x86_64/etc/pacman.d/mirrorlist b/container/archlinux/root/x86_64/etc/pacman.d/mirrorlist new file mode 100644 index 0000000..556fac8 --- /dev/null +++ b/container/archlinux/root/x86_64/etc/pacman.d/mirrorlist @@ -0,0 +1 @@ +Server = http://mirrors.tuna.tsinghua.edu.cn/archlinux/$repo/os/$arch -- 2.23.0

2 1

[PATCH v2 lkp-tests] lib/job.rb: change the way to analyze the testbox field
by Li Ping 18 Nov '20

18 Nov '20

If user wants to reproduce the result directly in the result directory. [problem] /srv/result/iperf/vm-2p8g/2020-11-09/z9.147520% submit job.yaml [Error] Traceback (most recent call last): 9: from /home/liping/lkp-tests/sbin/submit:96:in `<main>' 8: from /home/liping/lkp-tests/sbin/submit:96:in `each' 7: from /home/liping/lkp-tests/sbin/submit:106:in `block in <main>' 6: from /home/liping/lkp-tests/lib/job.rb:699:in `each_jobs' 5: from /home/liping/lkp-tests/lib/job.rb:400:in `load_defaults' 4: from /home/liping/lkp-tests/lib/job.rb:400:in `each' 3: from /home/liping/lkp-tests/lib/job.rb:401:in `block in load_defaults' 2: from /home/liping/lkp-tests/lib/job.rb:381:in `load_one_defaults' 1: from /home/liping/lkp-tests/lib/job.rb:306:in `load_hosts_config' /home/liping/lkp-tests/lib/job.rb:344:in `get_hosts_file': hosts file not exist: vm-2p8g.taishan200-2280-2s64p-256g, maybe need check testbox field (ArgumentError) --- lib/job.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 01719c2e..2e94316c 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -337,7 +337,7 @@ class Job lab_hosts_file = get_lab_hosts_file return lab_hosts_file if lab_hosts_file - hosts_file_name = @job['tbox_group'].split('--')[0] + hosts_file_name = @job['tbox_group'].split(/\.|--/)[0] hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" return hosts_file if File.file?(hosts_file) -- 2.23.0

2 2

Re: [PATCH v6 lkp-tests] add: sysbench-threads-git result parsing file the colon is not required. add sysbench-threads-git result parse file
by Li Ping 18 Nov '20

18 Nov '20

Reply-To: In-Reply-To: <20201118065209.1071046-1-1034502035(a)qq.com> On Wed, Nov 18, 2020 at 02:52:09PM +0800, Hu Xuejiao wrote: >[input] >General statistics: >total time: 0.3987s >total number of events: 10000 >total time taken by event execution: 0.3971 >response time: >min: 0.04ms > It has written in your patch. >[output] >total time: 0.3987s > >Signed-off-by: Hu XueJiao <1034502035(a)qq.com> >--- > stats/sysbench-threads-git | 14 ++++++++++++++ > 1 file changed, 14 insertions(+) > create mode 100755 stats/sysbench-threads-git > >diff --git a/stats/sysbench-threads-git b/stats/sysbench-threads-git >new file mode 100755 >index 00000000..afe5d664 >--- /dev/null >+++ b/stats/sysbench-threads-git >@@ -0,0 +1,14 @@ >+#!/usr/bin/env ruby >+ >+# General statistics: >+# total time: 0.2671s >+# total number of events: 10000 >+# total time taken by event execution: 0.3971s >+# response time: >+# min: 0.04ms >+ >+$stdin.each_line do |line| >+ next unless line =~ /^\s+total time:\s+([0-9.]+)/ >+ >+ puts "total time: #{$1.to_f}s" >+end >-- >2.23.0 >

2 2