- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci] sparrow/0-package: fix: failed to download lockfile-create in x86_64
by Liu Yinsi 30 Oct '20

30 Oct '20

[what] when execute sparrow/4-docker/buildall in x86_64 machine [error] lockfile-create: command not found [how] use arch=$(arch), support download lockfile-create command in aarch64 and x86_64. [why] weget link and rpm not support ${arch}, must use arch=$(arch) Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index ba298c8..31d795d 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -23,7 +23,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(arch) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

5 7

Re: [PATCH v2 compass-ci] parrow/0-package: fix lockfile-create download failed in x86_64
by Liu Yinsi 30 Oct '20

30 Oct '20

>>+wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… > >Why not use $(arch) directly? good Thanks, Yinsi > >>-rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm >>+rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm > >Thanks, >Zhangyu

1 0

[PATCH v2 compass-ci] parrow/0-package: fix lockfile-create download failed in x86_64
by Liu Yinsi 30 Oct '20

30 Oct '20

[how] use arch=$(arch), support download lockfile-create command in aarch64 and x86_64. [why] weget link and rpm not support ${arch}, must use arch=$(arch) Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index ba298c8..31d795d 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -23,7 +23,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(arch) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm -- 2.23.0

3 3

[PATCH v4 compass-ci] sparrow/0-package: fix: failed to download lockfile-create in x86_64
by Liu Yinsi 30 Oct '20

30 Oct '20

[what] when execute sparrow/4-docker/buildall in x86_64 machine [error] lockfile-create: command not found [how] 1. wget link and rpm not support ${arch}, must use arch=$(arch) to support download lockfile-create command in aarch64 and x86_64. 2. delete rpm file after download. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index ba298c8..ea4b29a 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -23,7 +23,9 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… +arch=$(arch) +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/${arch}/main/releas… -rpm -i lib64lockfile1-1.16-1-omv4002.aarch64.rpm lockfile-progs-0.1.18-5-omv4002.aarch64.rpm +rpm -i lib64lockfile1-1.16-1-omv4002.${arch}.rpm lockfile-progs-0.1.18-5-omv4002.${arch}.rpm +rm -rf *.rpm -- 2.23.0

2 1

[PATCH lab-z9] hosts: add host file a103-a107
by Zhang Yu 30 Oct '20

30 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-256g--a103 | 22 ++++++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a104 | 22 ++++++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a106 | 22 ++++++++++++++++++++++ hosts/taishan200-2280-2s48p-256g--a107 | 22 ++++++++++++++++++++++ hosts/taishan200-2280-2s48p-512g--a105 | 22 ++++++++++++++++++++++ 5 files changed, 110 insertions(+) create mode 100644 hosts/taishan200-2280-2s48p-256g--a103 create mode 100644 hosts/taishan200-2280-2s48p-256g--a104 create mode 100644 hosts/taishan200-2280-2s48p-256g--a106 create mode 100644 hosts/taishan200-2280-2s48p-256g--a107 create mode 100644 hosts/taishan200-2280-2s48p-512g--a105 diff --git a/hosts/taishan200-2280-2s48p-256g--a103 b/hosts/taishan200-2280-2s48p-256g--a103 new file mode 100644 index 0000000..304c6fc --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a103 @@ -0,0 +1,22 @@ +<<: hosts/taishan200-2280-2s48p-256g +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b88bdc69 + - /dev/disk/by-id/scsi-350000399b88bdd85 + - /dev/disk/by-id/scsi-350000399b88bdc4d + - /dev/disk/by-id/scsi-350000399b88bdc7d +mac_addr: + - 44:67:47:c9:d8:b8 + - 44:67:47:c9:d8:b9 + - 44:67:47:c9:d8:ba + - 44:67:47:c9:d8:bb + - 44:67:47:c9:d8:bc + - 44:67:47:c9:d8:bd + - 44:67:47:c9:d8:be + - 44:67:47:c9:d8:bf + - cc:05:77:ee:b5:4f + - cc:05:77:ee:b5:50 + - cc:05:77:ee:b5:51 + - cc:05:77:ee:b5:52 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.8 diff --git a/hosts/taishan200-2280-2s48p-256g--a104 b/hosts/taishan200-2280-2s48p-256g--a104 new file mode 100644 index 0000000..a8a2789 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a104 @@ -0,0 +1,22 @@ +<<: hosts/taishan200-2280-2s48p-256g +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8892431 + - /dev/disk/by-id/scsi-350000399b8892f81 + - /dev/disk/by-id/scsi-350000399b8891c49 + - /dev/disk/by-id/scsi-350000399b88930d1 +mac_addr: + - 44:67:47:d7:6d:f0 + - 44:67:47:d7:6d:f1 + - 44:67:47:d7:6d:f2 + - 44:67:47:d7:6d:f3 + - 44:67:47:d7:6d:f4 + - 44:67:47:d7:6d:f5 + - 44:67:47:d7:6d:f6 + - 44:67:47:d7:6d:f7 + - cc:05:77:ee:b9:6f + - cc:05:77:ee:b9:70 + - cc:05:77:ee:b9:71 + - cc:05:77:ee:b9:72 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.9 diff --git a/hosts/taishan200-2280-2s48p-256g--a106 b/hosts/taishan200-2280-2s48p-256g--a106 new file mode 100644 index 0000000..ad587b3 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a106 @@ -0,0 +1,22 @@ +<<: hosts/taishan200-2280-2s48p-256g +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b883bf99 + - /dev/disk/by-id/scsi-350000399b883ba89 + - /dev/disk/by-id/scsi-350000399b883b94d + - /dev/disk/by-id/scsi-350000399b883b981 +mac_addr: + - 44:67:47:86:1c:30 + - 44:67:47:86:1c:31 + - 44:67:47:86:1c:32 + - 44:67:47:86:1c:33 + - 44:67:47:86:1c:34 + - 44:67:47:86:1c:35 + - 44:67:47:86:1c:36 + - 44:67:47:86:1c:37 + - cc:05:77:ee:b7:df + - cc:05:77:ee:b7:e0 + - cc:05:77:ee:b7:e1 + - cc:05:77:ee:b7:e2 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.11 diff --git a/hosts/taishan200-2280-2s48p-256g--a107 b/hosts/taishan200-2280-2s48p-256g--a107 new file mode 100644 index 0000000..fba1937 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-256g--a107 @@ -0,0 +1,22 @@ +<<: hosts/taishan200-2280-2s48p-256g +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b8917c39 + - /dev/disk/by-id/scsi-350000399b8918005 + - /dev/disk/by-id/scsi-350000399b8900a4d + - /dev/disk/by-id/scsi-350000399b8900799 +mac_addr: + - 84:46:fe:73:a2:71 + - 84:46:fe:73:a2:72 + - 84:46:fe:73:a2:73 + - 84:46:fe:73:a2:74 + - 84:46:fe:73:a2:75 + - 84:46:fe:73:a2:76 + - 84:46:fe:73:a2:77 + - 84:46:fe:73:a2:78 + - 44:a1:91:35:1a:70 + - 44:a1:91:35:1a:71 + - 44:a1:91:35:1a:72 + - 44:a1:91:35:1a:73 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.12 diff --git a/hosts/taishan200-2280-2s48p-512g--a105 b/hosts/taishan200-2280-2s48p-512g--a105 new file mode 100644 index 0000000..096b5b0 --- /dev/null +++ b/hosts/taishan200-2280-2s48p-512g--a105 @@ -0,0 +1,22 @@ +<<: hosts/taishan200-2280-2s48p-512g +nr_hdd_partitions: 4 +hdd_partitions: + - /dev/disk/by-id/scsi-350000399b891945d + - /dev/disk/by-id/scsi-350000399b8919135 + - /dev/disk/by-id/scsi-350000399b8919699 + - /dev/disk/by-id/scsi-350000399b89190d5 +mac_addr: + - 44:67:47:d7:76:c4 + - 44:67:47:d7:76:c5 + - 44:67:47:d7:76:c6 + - 44:67:47:d7:76:c7 + - 44:67:47:d7:76:c8 + - 44:67:47:d7:76:c9 + - 44:67:47:d7:76:ca + - 44:67:47:d7:76:cb + - cc:05:77:ee:b2:8f + - cc:05:77:ee:b2:90 + - cc:05:77:ee:b2:91 + - cc:05:77:ee:b2:92 +model_name: Kunpeng-920 +ipmi_ip: 9.3.10.10 -- 2.23.0

1 0

[PATCH v8 compass-ci] sparrow: fix containers auto resume failed after reboot
by Liu Yinsi 30 Oct '20

30 Oct '20

[why] some containers have start-depends, docker start "${containers[@]}" also cause container service resume failed, need start its dependence firstly. [how] 1. add param(action) to buildall script. 2. if first deploy cci, $action is none, buildall script will execute: - $CONTAINER/build; - $CONTAINER/install; - $CONTAINER/first-run; - $CONTAINER/start script. 3. if reboot, $action will be 'reboot', buildall script will execute: - $CONTAINER/first-run; - $CONTAINER/start script. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 7 +++++-- sparrow/7-systemd/cci-network | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..6e0a2f8 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -3,6 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. CONTAINER_PATH="$CCI_SRC/container" +export action=$1 build_depends() { @@ -30,8 +31,10 @@ do_one() ( cd $container [ "$container_name" == 'sub-fluentd' ] && exit - [ -x build ] && ./build - [ -x install ] && ./install + if [ "$action" != "reboot" ]; then + [ -x build ] && ./build + [ -x install ] && ./install + fi [ -x first-run ] && ./first-run [ -x start ] && ./start ) diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 4aadcdc..6e59dbb 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -4,7 +4,8 @@ #this file used in our own systemd unit file: cci-network.service. #contents as follows will be automatically executed after system reboot. -[[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci +# export ENV +. /etc/profile.d/compass.sh $CCI_SRC/sparrow/2-network/br0 $CCI_SRC/sparrow/2-network/iptables @@ -12,5 +13,5 @@ $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs # --restart=always option is not absolutely reliable -containers=($(docker ps -a |grep -v NAMES |awk '{print $NF}')) -[ -n "$containers" ] && docker start "${containers[@]}" >/dev/null 2>&1 +# container need to start its dependences firstly. +$CCI_SRC/sparrow/4-docker/buildall reboot -- 2.23.0

3 2

[PATCH compass-ci] delimiter: refactor init job content
by Cao Xueliang 30 Oct '20

30 Oct '20

[why] There are tow file need init job content, then use a function to wrap the code. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 8 +------- src/delimiter/find-commit/bisect_run_script.rb | 3 +-- src/delimiter/utils.rb | 18 ++++++++++++++++++ 3 files changed, 20 insertions(+), 9 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index c548861..9033db2 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -35,13 +35,7 @@ class GitBisect end def set_bad_job - @bad_job = @es.query_by_id @bad_job_id - raise "es query job id: #{@bad_job_id} failed!" unless @bad_job - - @bad_job.delete('uuid') - @bad_job.delete('stats') - @bad_job.delete('id') - @bad_job.delete('error_ids') + @bad_job = Utils.init_job_content(@bad_job_id) end def set_upstream diff --git a/src/delimiter/find-commit/bisect_run_script.rb b/src/delimiter/find-commit/bisect_run_script.rb index d5e4226..4877d79 100755 --- a/src/delimiter/find-commit/bisect_run_script.rb +++ b/src/delimiter/find-commit/bisect_run_script.rb @@ -18,8 +18,7 @@ class GitBisectRun end def git_bisect - job = @es.query_by_id @job_id - job.delete('stats') if job.key?('stats') + job = Utils.init_job_content(@job_id) commit = `git -C #{@work_dir} log --pretty=format:"%H" -1` job['upstream_commit'] = commit get_bisect_status job diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 82cb918..679c8d9 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -87,5 +87,23 @@ module Utils return 'good' end + + def init_job_content(job_id) + es = ESQuery.new + job = es.query_by_id(job_id) + raise "es query job id: #{job_id} failed!" unless job + + job['bad_job_id'] = job_id + job['suite'] = 'bisect' + + job.delete('id') + job.delete('uuid') + job.delete('error_ids') + job.delete('start_time') + job.delete('end_time') + job.delete('loadavg') + + return job + end end end -- 2.23.0

3 4

[PATCH lkp-tests] jobs: add spawn test item
by Bai Jing 30 Oct '20

30 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/unixbench.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/unixbench.yaml b/jobs/unixbench.yaml index 2b833fcb..947ac7a1 100644 --- a/jobs/unixbench.yaml +++ b/jobs/unixbench.yaml @@ -15,7 +15,7 @@ unixbench: - shell8 - pipe - context1 - # - spawn + - spawn - syscall - dhry2reg - fstime -- 2.23.0

1 0

[PATCH compass-ci] sparrow/4-docker/buildall: container sub-fluentd don't exit
by Wu Zhende 30 Oct '20

30 Oct '20

[Why] Now, primary server also need to startup sub-fluentd. It collects Docker logs and VM serial port logs on the server. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sparrow/4-docker/buildall | 1 - 1 file changed, 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..2d39b1a 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -29,7 +29,6 @@ do_one() mkdir $tmpdir/$container_name 2>/dev/null && ( cd $container - [ "$container_name" == 'sub-fluentd' ] && exit [ -x build ] && ./build [ -x install ] && ./install [ -x first-run ] && ./first-run -- 2.23.0

2 1

[PATCH compass-ci] addsoftware/run: change the example os version
by Xu Xijian 30 Oct '20

30 Oct '20

[why] As the old os version centos/aarch64/76 is not exist, change the version number 76 in given example. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- container/add-software/run | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/add-software/run b/container/add-software/run index 99490ee..f4c6833 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -5,8 +5,8 @@ [ -L "$1/vmlinuz" -a -n "$2" ] || { cat << 'EOF' Example usage: - 1> ./run /os/centos/aarch64/76 pack1 pack2 ... - 2> ./run /os/centos/aarch64/76 $(show-depends-packages centos) + 1> ./run /os/centos/aarch64/7.6.1810 pack1 pack2 ... + 2> ./run /os/centos/aarch64/7.6.1810 $(show-depends-packages centos) centos is an adaption file contain packages mapping from debian to centos, the whole path is '$LKP_SRC/distro/adaptation/centos'. EOF -- 2.23.0

2 1

[PATCH v5 lkp-tests 2/2] tests/iozone: add specified parameter for iozone test
by Lu Kaiyi 30 Oct '20

30 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..175b206a 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,13 +1,46 @@ #!/bin/sh - +# - block_size +# - file_size +# - write_rewrite +# - read_reread +# - random_read_write +# - read_backwards +# - rewrite_record +# - stride_read +# - fwrite_refwrite +# - fread_refread +# - random_mix +# - pwrite_repwrite +# - pread_repread +# - pwritev_repwritev +# - preadv_repreadv ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. . $LKP_SRC/lib/reproduce-log.sh +cmd="iozone" +[ -n "$block_size" ] && cmd+=" -r $block_size" +[ -n "$file_size" ] && cmd+=" -s $file_size" +[ -n "$write_rewrite" ] && cmd+=" -i 0" +[ -n "$read_reread" ] && cmd+=" -i 1" +[ -n "$random_read_write" ] && cmd+=" -i 2" +[ -n "$read_backwards" ] && cmd+=" -i 3" +[ -n "$rewrite_record" ] && cmd+=" -i 4" +[ -n "$stride_read" ] && cmd+=" -i 5" +[ -n "$fwrite_refwrite" ] && cmd+=" -i 6" +[ -n "$fread_refread" ] && cmd+=" -i 7" +[ -n "$random_mix" ] && cmd+=" -i 8" +[ -n "$pwrite_repwrite" ] && cmd+=" -i 9" +[ -n "$pread_repread" ] && cmd+=" -i 10" +[ -n "$pwritev_repwritev" ] && cmd+=" -i 11" +[ -n "$preadv_repreadv" ] && cmd+=" -i 12" + +[ "${#cmd}" == 6 ] && cmd+=" -a" + for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + cmd+=" -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

1 0

[PATCH v5 lkp-tests 1/2] jobs/iozone_with_block_size.yaml: add iozone_with_block_size.yaml
by Lu Kaiyi 30 Oct '20

30 Oct '20

[why] need test different block_size value Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- jobs/iozone-bs.yaml | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 jobs/iozone-bs.yaml diff --git a/jobs/iozone-bs.yaml b/jobs/iozone-bs.yaml new file mode 100644 index 00000000..e2cd9f48 --- /dev/null +++ b/jobs/iozone-bs.yaml @@ -0,0 +1,24 @@ +suite: iozone +category: benchmark + +file_size: 4g +write_rewrite: true +read_reread: true +random_read_write: true + +block_size: +- 64k +- 256k +- 1m +- 4m +- 16m + +disk: 1HDD + +fs: +- xfs + +iosched: +- kyber + +iozone: -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/3-code: fix no such file or directory
by Zhang Yale 30 Oct '20

30 Oct '20

[Why] when run dev-env in centos system error like this: ./dev-env:line 41: ./os/centos: No such file or directory. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- sparrow/3-code/dev-env | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 11c7fb0..62de23d 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -38,4 +38,7 @@ EOF source /etc/os-release -. $(dirname ${BASH_SOURCE[0]})/os/${ID} +os_id=$(dirname ${BASH_SOURCE[0]})/os/${ID} +[ -f "${os_id}" ] && { + . os_id +} -- 2.23.0

3 3

[PATCH lkp-tests] hackbench/PKGBUILD: fix source and build function
by Zhang Yale 30 Oct '20

30 Oct '20

[Why] Old source "https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-1.8.tar.gz" is not available, and need build before package. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- pkg/hackbench/PKGBUILD | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/pkg/hackbench/PKGBUILD b/pkg/hackbench/PKGBUILD index 797a00e6..871a26db 100644 --- a/pkg/hackbench/PKGBUILD +++ b/pkg/hackbench/PKGBUILD @@ -4,9 +4,14 @@ pkgrel=1 arch=('i386' 'x86_64') url="https://www.kernel.org/pub/linux/utils/rt-tests" license=('GPL') -source=("https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-${pkgver}.tar.gz") +source=("https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/snapshot/rt-test…") sha256sums=('SKIP') +build(){ + cd "$srcdir/rt-tests-${pkgver}" + gcc -o hackbench hackbench.c -lpthread +} + package() { mkdir -p "${pkgdir}/lkp/benchmarks/${pkgname}" cd "$srcdir/rt-tests-${pkgver}" -- 2.23.0

5 7

[PATCH v7 compass-ci] sparrow: fix containers auto resume failed after reboot
by Liu Yinsi 30 Oct '20

30 Oct '20

[why] some containers have start-depends, docker start "${containers[@]}" also cause container service resume failed, need start its dependence firstly. [how] before: 4-docker/buildall => execute $CONTAINER/build, install, first-run, start script after: 4-docker/buildall reboot => only execute $CONTAINER/first-run, start script Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 7 +++++-- sparrow/7-systemd/cci-network | 7 ++++--- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 230c6ac..6e0a2f8 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -3,6 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. CONTAINER_PATH="$CCI_SRC/container" +export action=$1 build_depends() { @@ -30,8 +31,10 @@ do_one() ( cd $container [ "$container_name" == 'sub-fluentd' ] && exit - [ -x build ] && ./build - [ -x install ] && ./install + if [ "$action" != "reboot" ]; then + [ -x build ] && ./build + [ -x install ] && ./install + fi [ -x first-run ] && ./first-run [ -x start ] && ./start ) diff --git a/sparrow/7-systemd/cci-network b/sparrow/7-systemd/cci-network index 4aadcdc..6e59dbb 100755 --- a/sparrow/7-systemd/cci-network +++ b/sparrow/7-systemd/cci-network @@ -4,7 +4,8 @@ #this file used in our own systemd unit file: cci-network.service. #contents as follows will be automatically executed after system reboot. -[[ $CCI_SRC ]] || CCI_SRC=/c/compass-ci +# export ENV +. /etc/profile.d/compass.sh $CCI_SRC/sparrow/2-network/br0 $CCI_SRC/sparrow/2-network/iptables @@ -12,5 +13,5 @@ $CCI_SRC/sparrow/2-network/nfs $CCI_SRC/sparrow/2-network/cifs # --restart=always option is not absolutely reliable -containers=($(docker ps -a |grep -v NAMES |awk '{print $NF}')) -[ -n "$containers" ] && docker start "${containers[@]}" >/dev/null 2>&1 +# container need to start its dependences firstly. +$CCI_SRC/sparrow/4-docker/buildall reboot -- 2.23.0

2 2

[PATCH compass-ci] sparrow/3-code/debian: fix format problem.
by Zhang Yale 30 Oct '20

30 Oct '20

Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- sparrow/3-code/os/debian | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/3-code/os/debian b/sparrow/3-code/os/debian index efb4fcb..80dca70 100644 --- a/sparrow/3-code/os/debian +++ b/sparrow/3-code/os/debian @@ -1,5 +1,5 @@ #!/bin/bash -cat >> /etc/default/docker <<EOF +cat > /etc/default/docker << EOF DOCKER_OPTS="--dns 114.114.114.114" EOF -- 2.23.0

5 5

[PATCH v6 compass-ci 1/5] container/ntp-server: Dockerfile
by Wang Chenglong 30 Oct '20

30 Oct '20

[What] The docker file for build a container to provide network time synchronization service for virtual and physical machines. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/Dockerfile | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100644 container/ntp-server/Dockerfile diff --git a/container/ntp-server/Dockerfile b/container/ntp-server/Dockerfile new file mode 100644 index 0000000..76f2a0e --- /dev/null +++ b/container/ntp-server/Dockerfile @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:7 +MAINTAINER Wang Chenglong <18509160991(a)163.com> + +RUN yum repolist +RUN yum -y install ntp && \ + yum clean all +RUN ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime + +COPY ntp.conf /etc/ntp.conf +COPY start-server.sh / + +EXPOSE 123 + +ENTRYPOINT ["/start-server.sh"] +CMD ["sh"] -- 2.23.0

2 2

[PATCH lkp-tests] bin/lkp-setup-rootfs: fix auto submit makepkg task
by Li Ping 30 Oct '20

30 Oct '20

[why] the define_files will generate the PKGBUILD and when run define_files_auto_pack will automatic run makepkg even though the job is not for makepkg --- bin/lkp-setup-rootfs | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 2890601e..a3663b42 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -187,14 +187,14 @@ define_files_pkg_pack() ( local benchmark=${1#pkg/} export benchmark=${benchmark%/PKGBUILD} - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) } # There may be $program-dev or $program-dev-dev in the # "$LKP_SRC/distro/depends" directory, # when the value passed by $define_files is "distro/depends/$program-dev", -# you don't know which script in "makepkg" or "pack-deps" to execute, +# you don't know which script in "cci-makepkg" or "pack-deps" to execute, # so it needs to be handled sparately. # Save the result to # "/srv/initrd/deps/$os/$os_arch/$os_version/$deps_benchmark.cgz", @@ -208,7 +208,7 @@ define_files_deps_pack() [ $suffix == "dev" ] && [ -f "$LKP_SRC/pkg/$pkg_benchmark/PKGBUILD" ] && { ( export benchmark=$pkg_benchmark - $LKP_SRC/tests/makepkg + $LKP_SRC/tests/cci-makepkg ) return 0 } @@ -228,10 +228,10 @@ define_files_auto_pack() do [ -f "$LKP_SRC/$path" ] || continue - [ ${path##*/} = PKGBUILD ] && { - define_files_pkg_pack $path + [ ${path##*/} = PKGBUILD ] && [ $suite = 'cci-makepkg' ] && { + define_files_pkg_pack $path } - [ ${path#distro/depends/} != $path ] && { + [ ${path#distro/depends/} != $path ] && [ $suite = 'cci-depends' ] && { define_files_deps_pack $path } done -- 2.23.0

2 1

[PATCH v6 compass-ci 5/5] container/ntp-server: start
by Wang Chenglong 30 Oct '20

30 Oct '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/start | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100755 container/ntp-server/start diff --git a/container/ntp-server/start b/container/ntp-server/start new file mode 100755 index 0000000..c6f04a6 --- /dev/null +++ b/container/ntp-server/start @@ -0,0 +1,17 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +SCRIPT_DIR=$(dirname $(realpath $0)) +. $(dirname $SCRIPT_DIR)/defconfig.sh + +docker_rm ntp-server +cmd=( + docker run + -id + --name=ntp-server + -p 123:123/udp + ntp-server +) + +"${cmd[@]}" -- 2.23.0

3 4

[PATCH v3 lkp-tests] fix: btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 29 Oct '20

29 Oct '20

[why] mkfs.btrfs "command not found" when os_mount=initramfs [how] it will user mkfs.ext4 because mkfs.ext4 is common, other mkfs.*** is not considered [how] mkfs.btrfs and mkfs.ext4 is no exist, it will return Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/bootstrap.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..9744cb84 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,7 +456,12 @@ mount_rootfs() ROOTFS_DIR=/opt/rootfs mkdir -p $ROOTFS_DIR mount $rootfs_partition $ROOTFS_DIR || { - mkfs.btrfs -f $rootfs_partition + if has_cmd mkfs.btrfs; then + mkfs.btrfs -f $rootfs_partition + else + mkfs.ext4 -F $rootfs_partition || return + fi + mount $rootfs_partition $ROOTFS_DIR } mkdir -p $ROOTFS_DIR/tmp -- 2.23.0

3 2

[PATCH v2 lkp-tests] hackbench/PKGBUILD: fix source and build function
by Zhang Yale 29 Oct '20

29 Oct '20

[Why] Old source "https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-1.8.tar.gz" is not available, and need build before package. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- pkg/hackbench/PKGBUILD | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/pkg/hackbench/PKGBUILD b/pkg/hackbench/PKGBUILD index 797a00e6..7e3b52e9 100644 --- a/pkg/hackbench/PKGBUILD +++ b/pkg/hackbench/PKGBUILD @@ -4,10 +4,17 @@ pkgrel=1 arch=('i386' 'x86_64') url="https://www.kernel.org/pub/linux/utils/rt-tests" license=('GPL') -source=("https://www.kernel.org/pub/linux/utils/rt-tests/rt-tests-${pkgver}.tar.gz") +source=("https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git/snapshot/rt-test…") sha256sums=('SKIP') -package() { +build() +{ + cd "$srcdir/rt-tests-${pkgver}" + gcc -o hackbench hackbench.c -lpthread +} + +package() +{ mkdir -p "${pkgdir}/lkp/benchmarks/${pkgname}" cd "$srcdir/rt-tests-${pkgver}" make install DESTDIR="${pkgdir}/lkp/benchmarks/${pkgname}" -- 2.23.0

3 4

[PATCH v2 compass-ci] save hostname and queues relation in redis
by Xiao Shenwei 29 Oct '20

29 Oct '20

[why] the test machine has the ability to know which queues should be requested [how] for VM HW, we will register mac2hostname and hostname2queues. then they can find queues based on mac. API: boot.ipxe/mac/${mac} for docker, we will register hostname2queues, then container can find queues based on hostname. API: boot.container/hostname/${hostname} Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/sched.cr | 8 ++++++++ src/scheduler/scheduler.cr | 24 ++++++++++++++++++++++++ 2 files changed, 32 insertions(+) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index b2ef6ce..9407ddc 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -34,10 +34,18 @@ class Sched @redis.hash_set("sched/mac2host", normalize_mac(mac), hostname) end + def set_host2queues(hostname : String, queues : String) + @redis.hash_set("sched/host2queues", hostname, queues) + end + def del_host_mac(mac : String) @redis.hash_del("sched/mac2host", normalize_mac(mac)) end + def del_host2queues(hostname : String) + @redis.hash_del("sched/host2queues", hostname) + end + # return: # Hash(String, Hash(String, String)) def get_cluster_state(cluster_id) diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 371e4ed..a946ea0 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -113,6 +113,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/set_host2queues?queues=vm-2p8g.aarch64&host=vm-2p8g.a…" + put "/set_host2queues" do |env| + env.response.headers["Connection"] = "close" + if (client_queues = env.params.query["queues"]?) && (client_host = env.params.query["host"]?) + sched.set_host2queues(client_host, client_queues) + + "Done" + else + "No yet" + end + end + # curl -X PUT "http://localhost:3000/del_host_mac?mac=00-01-02-03-04-05" put "/del_host_mac" do |env| env.response.headers["Connection"] = "close" @@ -125,6 +137,18 @@ module Scheduler end end + # curl -X PUT "http://localhost:3000/del_host2queues?host=vm-2p8g.aarch64" + put "/del_host2queues" do |env| + env.response.headers["Connection"] = "close" + if client_host = env.params.query["host"]? + sched.del_host2queues(client_host) + + "Done" + else + "No yet!" + end + end + # client(runner) report job's status # /~lkp/cgi-bin/lkp-jobfile-append-var # ?job_file=/lkp/scheduled/job.yaml&job_state=running&job_id=10 -- 2.23.0

3 4

[PATCH v2 lkp-tests] fix: btrfs-progs is not installed when os_mount=initramfs
by Hu Xuejiao 29 Oct '20

29 Oct '20

[why] mkfs.btrfs "command not found" when os_mount=initramfs [how] it will user mkfs.ext4. because mkfs.ext4 is common, other mkfs.*** is not considered [how] mkfs.btrf and mkfs.ext4 is no exist, it will return Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/bootstrap.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d68..9744cb84 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -456,7 +456,12 @@ mount_rootfs() ROOTFS_DIR=/opt/rootfs mkdir -p $ROOTFS_DIR mount $rootfs_partition $ROOTFS_DIR || { - mkfs.btrfs -f $rootfs_partition + if has_cmd mkfs.btrfs; then + mkfs.btrfs -f $rootfs_partition + else + mkfs.ext4 -F $rootfs_partition || return + fi + mount $rootfs_partition $ROOTFS_DIR } mkdir -p $ROOTFS_DIR/tmp -- 2.23.0

3 2

[PATCH compass-ci] container/scheduler/start: fix duplicate -e
by Wu Zhende 29 Oct '20

29 Oct '20

[Why] "env = docker_env(defaults)" already add "-e xxx". Don't need to add in cmd again. SCHED_HOST/SCHED_PORT/GIT_SERVER/MASTER_FLUENTD_HOST/MASTER_FLUENTD_HOST may add duplicate -e. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/scheduler/start | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/container/scheduler/start b/container/scheduler/start index 6898d4b..70a3496 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -32,15 +32,17 @@ defaults = relevant_defaults(names) defaults['SSHR_PORT'] ||= 5050 defaults['SSHR_PORT_BASE'] ||= 50000 defaults['SSHR_PORT_LEN'] ||= 2000 +defaults['SCHED_PORT'] ||= '3000' +defaults['SCHED_HOST'] ||= '172.17.0.1' +defaults['MASTER_FLUENTD_PORT'] ||= '24224' env = docker_env(defaults) CCI_REPOS = ENV['CCI_REPOS'] || '/c' DEFAULT_LKP = "#{CCI_REPOS}/lkp-tests" DEFAULT_CCI = "#{CCI_REPOS}/compass-ci" LAB = defaults['lab'] || 'z9' -SCHED_PORT = defaults['SCHED_PORT'] || '3000' -SCHED_HOST = defaults['SCHED_HOST'] || '172.17.0.1' -MASTER_FLUENTD_PORT = defaults['MASTER_FLUENTD_PORT'] || '24224' +SCHED_PORT = defaults['SCHED_PORT'] +SCHED_HOST = defaults['SCHED_HOST'] DEFAULT_CONFIG_DIR = '/etc/compass-ci/defaults' docker_rm "s001-alpine-#{SCHED_PORT}" @@ -61,11 +63,6 @@ cmd = %W[ -e CCI_REPOS=#{CCI_REPOS} -e LKP_SRC=#{DEFAULT_LKP} -e CCI_SRC=#{DEFAULT_CCI} - -e SCHED_HOST=#{SCHED_HOST} - -e SCHED_PORT=#{SCHED_PORT} - -e GIT_SERVER=#{defaults["GIT_SERVER"]} - -e MASTER_FLUENTD_HOST=#{defaults["MASTER_FLUENTD_HOST"]} - -e MASTER_FLUENTD_PORT=#{MASTER_FLUENTD_PORT} -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro -- 2.23.0

1 0

[PATCH lkp-tests] tests: add get_cpu function
by Zhang Yu 29 Oct '20

29 Oct '20

[why] Obtain more host information from 'lscpu' Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..84f796dc 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,18 +1,25 @@ #!/bin/bash -get_node() +get_memory() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + echo $1: $(lscpu | awk -F': *' '/'$2':/{print $2}') +} + +get_cpu_info() +{ + cpu_info=("arch Architecture" "nr_node NUMA\snode$s$" "nr_cpu ^CPU$s$" "model_name Model\sname") + for info in "${cpu_info[@]}" + do + get_cpu $info + done } get_nr_disk() @@ -101,11 +108,11 @@ get_ipmi_ip() main() { - get_node + get_memory get_nr_disk get_partitions get_mac - get_model_name + get_cpu_info get_ipmi_ip } -- 2.23.0

3 4