November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 3/3] kernel_version.md: delete unused modules.cgz and headers.cgz
by Xu Xijian 19 Nov '20

19 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index 3cf5e26..3e94ca8 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -30,11 +30,9 @@ Related files: - In initramfs boot process, every kernel version is related with a vmlinuz, module and headers. - Files like below under $boot_dir, an example $boot_dir can be "/srv/os/openeuler/aarch64/20.03/boot". ├── headers-4.19.90-2003.cgz -├── headers.cgz -> headers-4.19.90-2003.cgz ├── modules-4.19.90-2003.cgz -├── modules.cgz -> modules-4.19.90-2003.cgz ├── vmlinuz-4.19.90-2003 -└── vmlinuz -> vmlinuz-4.19.90-2003 +└── vmlinuz -> vmlinuz-4.19.90-2003 Usage example: - submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 -- 2.23.0

1 0

[PATCH v3 compass-ci 1/3] dracut-initrd: delete unused kernel module iscsi from dracut
by Xu Xijian 19 Nov '20

19 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/dracut-initrd/run b/container/dracut-initrd/run index b0c9b30..6301ee2 100755 --- a/container/dracut-initrd/run +++ b/container/dracut-initrd/run @@ -43,7 +43,7 @@ cmd=( -v $root/lib/modules:/lib/modules debian:dracut bash -c - "dracut --force --kver $kver -k $kernel_modules $initrd_output && + "dracut -o iscsi --omit-drivers iscsi --force --kver $kver -k $kernel_modules $initrd_output && chmod 644 $initrd_output" # example: -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] dracut-initrd: delete unused kernel module iscsi from dracut
by Xu Xijian 19 Nov '20

19 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/dracut-initrd/run b/container/dracut-initrd/run index b0c9b30..6301ee2 100755 --- a/container/dracut-initrd/run +++ b/container/dracut-initrd/run @@ -43,7 +43,7 @@ cmd=( -v $root/lib/modules:/lib/modules debian:dracut bash -c - "dracut --force --kver $kver -k $kernel_modules $initrd_output && + "dracut -o iscsi --omit-drivers iscsi --force --kver $kver -k $kernel_modules $initrd_output && chmod 644 $initrd_output" # example: -- 2.23.0

1 0

[PATCH lkp-tests 2/6] iso2rootfs: allow non-dailybuild jobs to be submitted
by Yu Chuan 19 Nov '20

19 Nov '20

[Why] We can only generate the rootfs of dailybuild, so decouple the code related to dailybuild. [How] Use 'iso_url_file' as a switch. - If this field is specified, so generate the rootfs of dailybuild; - If not, generate the rootfs of the official iso of the target version. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 4 +++- tests/iso2rootfs | 24 +++++++++++++----------- 2 files changed, 16 insertions(+), 12 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index d4f0ffc0e80c..4e8729997a00 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -2,7 +2,6 @@ suite: iso2rootfs category: benchmark iso2rootfs: iso2rootfs_cgz: - iso_url_file: iso_os: iso_arch: iso_version: @@ -12,6 +11,9 @@ iso2rootfs: initrd_http_host: initrd_http_port: + # dailybuild related fields + iso_url_file: + # submit test yaml related fields test_yaml: test_os: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 9c71b8427d90..d42c38909f68 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -44,7 +44,6 @@ check_yaml_vars() "os_arch" "os_version" "iso2rootfs_cgz" - "iso_url_file" "iso_os" "iso_arch" "iso_version" @@ -149,8 +148,6 @@ mount_rootfs() get_cache_iso_checksum() { - mount_rootfs - CHECKSUM_FILE_CACHE="${ROOTFS_LOCAL_PATH}/${iso_version}-latest.sha256sum" [ ! -f "${CHECKSUM_FILE_CACHE}" ] || SHA256SUM_CACHE=$(awk '{print $1}' "$CHECKSUM_FILE_CACHE") @@ -243,7 +240,10 @@ config_iso2rootfs() pre_works() { check_yaml_vars - check_sha256sum_update + + mount_rootfs + [ -n "${iso_url_file}" ] && check_sha256sum_update + check_os config_iso2rootfs @@ -286,7 +286,7 @@ config_iso2qcow2() config_pip_proxy config_iso_conf - config_iso_url + [ -n "${iso_url_file}" ] && config_iso_url } run_iso2qcow2() @@ -332,7 +332,7 @@ config_qcow2rootfs() run_qcow2rootfs() { log_info "starting run qcow2rootfs ..." - local qcow2_path="${I2Q_SRC}/output/${ISO_NAME%.iso}.qcow2" + local qcow2_path="$(ls ${I2Q_SRC}/output/*.qcow2)" cd "${CCI_SRC}/container/qcow2rootfs" ./run "${qcow2_path}" "${ROOTFS_DES_DIR}" @@ -501,12 +501,14 @@ post_works() { log_info "starting post works ..." - mv "${CHECKSUM_FILE_CACHE}.tmp" "${CHECKSUM_FILE_CACHE}" + [ -n "${iso_url_file}" ] && { + mv "${CHECKSUM_FILE_CACHE}.tmp" "${CHECKSUM_FILE_CACHE}" - local soft_link="${iso_version}-dailybuild" - cd "${ROOTFS_LOCAL_PATH}" && - rm -f "${soft_link}" && - ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" + local soft_link="${iso_version}-dailybuild" + cd "${ROOTFS_LOCAL_PATH}" && + rm -f "${soft_link}" && + ln -s "$(basename "${ROOTFS_DES_DIR}")" "${soft_link}" + } cd / && umount "${ROOTFS_LOCAL_PATH}" -- 2.23.0

4 8

[PATCH v4 lkp-tests] jobs: add new job yaml for lmbench3 performance test
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3-performance.yaml | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 jobs/lmbench3-performance.yaml diff --git a/jobs/lmbench3-performance.yaml b/jobs/lmbench3-performance.yaml new file mode 100644 index 00000000..60ef80fe --- /dev/null +++ b/jobs/lmbench3-performance.yaml @@ -0,0 +1,16 @@ +suite: lmbench3 +category: benchmark + +test_memory_size: 4096 + +# Test mode all, hardware, os were commented temporally, since it will take very long time +# lmbench3: +# - mode: all +# - mode: hardware +# - mode: os + +nr_threads: + - 1 + +lmbench3: + mode: all -- 2.23.0

1 0

[PATCH v2 lkp-tests] add on_fail to job2sh when sh_on_fail is not nil
by Hu Xuejiao 19 Nov '20

19 Nov '20

[description] When sh_on_fail is nil, on_fail does not need to add to the job.yaml Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/submit | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/sbin/submit b/sbin/submit index a325e912..9bba1018 100755 --- a/sbin/submit +++ b/sbin/submit @@ -121,11 +121,14 @@ ARGV.each do |jobfile| 'job2sh' => { 'run_job' => sh_run_job, 'extract_stats' => sh_extract_stats, - 'define_files' => sh_define_files, - 'on_fail' => sh_on_fail + 'define_files' => sh_define_files } } + if sh_on_fail + sh_hash['job2sh']['on_fail'] = sh_on_fail + end + # merge job info job_hash = job.to_hash job_hash_list << job_hash -- 2.23.0

1 0

[PATCH v7 compass-ci 2/2] lib/matrix2.rb: refactor performance optimization
by Lu Kaiyi 19 Nov '20

19 Nov '20

[why] Avoid to execute some unnecessary operations and improve performance. [how] Keep the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 62d174c..86cfc93 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

2 2

[PATCH lkp-tests] borrow-1h.yaml: fix unable to receive email
by Zhang Yale 19 Nov '20

19 Nov '20

[why] $my_email is used in sshd, so borrow_1h.yaml should use my_email field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/borrow-1h.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml index c9d6ff8e..6c2284f3 100644 --- a/jobs/borrow-1h.yaml +++ b/jobs/borrow-1h.yaml @@ -9,6 +9,6 @@ sshd: nil end %> - email: + my_email: # sleep at the bottom sleep: 1h -- 2.23.0

2 2

[PATCH v5 compass-ci] sparrow: fix 3-code/dev-env no such file error
by Liu Yinsi 19 Nov '20

19 Nov '20

[why] when execute 3-code/dev-env in openeuler and centos machine, error message as follows: ./dev-env: line 45: ./os/openEuler: No such file or directory the purpose of ". $(dirname ${BASH_SOURCE[0]})/os/${ID}" is to fix crystal-shards build error when adapt in debian. but it's unnecessary for other linux os. Please see: [PATCH compass-ci 3/4] sparrow: add domain name resolution for Docker images [how] hide this redundant and confusion error message. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 8928816..08084b0 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -44,4 +44,4 @@ EOF source /etc/os-release -. $(dirname ${BASH_SOURCE[0]})/os/${ID} +. $(dirname ${BASH_SOURCE[0]})/os/${ID} 2> /dev/null -- 2.23.0

3 2

[PATCH v3 lkp-tests] jobs: add new job yaml for lmbench3 performance test
by Bai Jing 19 Nov '20

19 Nov '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- jobs/lmbench3-performance.yaml | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100644 jobs/lmbench3-performance.yaml diff --git a/jobs/lmbench3-performance.yaml b/jobs/lmbench3-performance.yaml new file mode 100644 index 00000000..76e5348b --- /dev/null +++ b/jobs/lmbench3-performance.yaml @@ -0,0 +1,17 @@ +suite: lmbench3 +category: benchmark + +test_memory_size: 4096 + +# Test mdoe all, hardware, os were commented temporally, since it will take very long time +# more than 2 hours on test machine lkp-bdw-ep4. +# lmbench3: +# - mode: all +# - mode: hardware +# - mode: os + +nr_threads: +- 1 + +lmbench3: + mode: all -- 2.23.0

1 0