October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] lib/job.rb: check pp key
by Wei Jihui 13 Oct '20

13 Oct '20

next when check key is nil, donot add it to pp Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 3c555e13..c3966546 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -568,7 +568,9 @@ class Job init_program_options() for_each_in(@job, @referenced_programs.keys) do |_pk, h, p_n, p_args| k = hash_key_re_string(@job, p_n, @job['pp'].keys) - if k && !(a)job[k].is_a?(Hash) + next if k.nil? + + unless @job[k].is_a?(Hash) @job['pp'][k] = @job[k] next end -- 2.23.0

1 0

[PATCH lkp-tests] /lkp-tests/test: add test-tools
by Li Ping 13 Oct '20

13 Oct '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- tests/test-tools | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 tests/test-tools diff --git a/tests/test-tools b/tests/test-tools new file mode 100755 index 00000000..9ca5d228 --- /dev/null +++ b/tests/test-tools @@ -0,0 +1,37 @@ +#!/bin/bash +# - testsuite +# - testcase +# - all + +cd $BENCHMARK_ROOT/test-tools/mugen || exit + +set_for_env() +{ + bash runoet.sh -c + source ~/.bash_profile +} + +test_one_case() +{ + + bash runoet.sh -f $testsuite -r $testcase -x +} + +test_all_cases() +{ + bash runoet.sh -a +} + +run_test() +{ + set_for_env + [ -n "$all" ] && { + test_all_cases + return + } + [ -n "$testsuite" ] || testsuite=smoke-testing + [ -n "$testcase" ] || testcase=oe_test_dd_001 + test_one_case +} + +run_test -- 2.23.0

4 4

[PATCH lkp-tests] add PKGBUILD for test-tools
by Li Ping 13 Oct '20

13 Oct '20

Signed-off-by: Li Ping <15396232681(a)163.com> --- pkg/test-tools/PKGBUILD | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 pkg/test-tools/PKGBUILD diff --git a/pkg/test-tools/PKGBUILD b/pkg/test-tools/PKGBUILD new file mode 100644 index 00000000..6908c193 --- /dev/null +++ b/pkg/test-tools/PKGBUILD @@ -0,0 +1,19 @@ +pkgname=test-tools +pkgver=git +pkgrel=1 +pkgdesc="smoke test" +url="https://gitee.com/openeuler/test-tools.git" +arch=aarch64 +license=('GPL') +source=("https://gitee.com/openeuler/test-tools.git" + "https://gitee.com/openeuler/integration-test.git" +) +md5sums=('SKIP' 'SKIP') + +package() { + test_tools_path="${pkgdir}/lkp/benchmarks/${pkgname}" + mkdir -p "${test_tools_path}" + cp -af "${srcdir}/test-tools/"* "${test_tools_path}" + cp -af "${srcdir}/integration-test/testcases"* "${test_tools_path}/mugen" + cp -af "${srcdir}/integration-test/suite2cases"* "${test_tools_path}/mugen" +} -- 2.23.0

2 1

[PATCH compass-ci] user-client/maintain: use tbox to test os rootfs
by Liu Yinsi 13 Oct '20

13 Oct '20

1. vm-hi1620-2p8g--USER -> vm-2p8g--taishan200-2280-2s48p-256g--a71 2. test in tbox, so remove my-qemu.sh 3. get job run time by es-find Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- .../maintain/walk-os-test/iperf-walk-os.yaml | 2 +- .../maintain/walk-os-test/walk-os-iperf-test | 19 +++++++++++++++---- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/user-client/maintain/walk-os-test/iperf-walk-os.yaml b/user-client/maintain/walk-os-test/iperf-walk-os.yaml index 2991743..b215730 100644 --- a/user-client/maintain/walk-os-test/iperf-walk-os.yaml +++ b/user-client/maintain/walk-os-test/iperf-walk-os.yaml @@ -7,7 +7,7 @@ if role server: if role client: iperf: protocol: tcp -testbox: vm-hi1620-2p8g--USER +testbox: vm-2p8g--taishan200-2280-2s48p-256g--a71 os: OS os_arch: OS_ARCH os_version: OS_VER diff --git a/user-client/maintain/walk-os-test/walk-os-iperf-test b/user-client/maintain/walk-os-test/walk-os-iperf-test index 21e1428..9c2b612 100755 --- a/user-client/maintain/walk-os-test/walk-os-iperf-test +++ b/user-client/maintain/walk-os-test/walk-os-iperf-test @@ -43,12 +43,23 @@ create_test_yaml() { echo "$hwarch $os/$os_arch/$os_ver" } +get_job_state() { + job_state='' + until [ -n "$job_state" ] + do + job_state=$(es-find id=$job_id| grep job_state) + sleep 5 + done +} + job_test() { local job_id bef aft - job_id=$(submit test.yaml |awk '{print $NF}') - bef=$(date +%s) - ./my-qemu.sh - aft=$(date +%s) + job_id=$(submit test.yaml |awk -F '=' '{print $NF}') + get_job_state + start_time=$(es-find id=$job_id| grep start_time| awk -F '"' '{print $4}') + end_time=$(es-find id=$job_id| grep end_time| awk -F '"' '{print $4}') + bef=$(date -d "$start_time" +%s) + aft=$(date -d "$end_time" +%s) logging "$1" "$2" "$3" "$(sec2date $bef)" "$(sec2date $aft)" "$((aft - bef))" "$job_id" } -- 2.23.0

2 1

[PATCH compass-ci] sched: when os_mount is container no need initrd.lkp
by Cao Xueliang 13 Oct '20

13 Oct '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index be3446e..70f2505 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -324,7 +324,7 @@ class Job if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) - else + elsif "#{os_mount}" == "nfs" or "#{os_mount}" == "cifs" temp_initrds.concat(nfs_cifs_initrds()) end -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/extract-stats add 'SRV_HTTP_HOST/PORT' in environment variables
by Lu Weitao 13 Oct '20

13 Oct '20

background: extract-stats service use mail-job tool that need get SRV_HTTP_HOST/PORT from ENV for display job result url in email Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/extract-stats/start | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/container/extract-stats/start b/container/extract-stats/start index 6bb9f10..e5ee777 100755 --- a/container/extract-stats/start +++ b/container/extract-stats/start @@ -4,6 +4,7 @@ # frozen_string_literal: true require 'set' +require 'open3' require_relative '../defconfig.rb' names = Set.new %w[ @@ -25,6 +26,8 @@ DEFAULT_LKP = '/c/lkp-tests' DEFAULT_CCI = '/c/compass-ci' FLUENTD_HOST = defaults['FLUENTD_HOST'] || '172.17.0.1' FLUENTD_PORT = defaults['FLUENTD_PORT'] || '24224' +SRV_HTTP_HOST = defaults['SRV_HTTP_HOST'] || Open3.capture3('curl ip.sb')[0].chomp +SRV_HTTP_PORT = defaults['SRV_HTTP_PORT'] || '11300' docker_rm "extract-stats" cmd = %w[ @@ -35,6 +38,8 @@ cmd = %w[ ] + env + %W[ -e LKP_SRC=#{DEFAULT_LKP} -e CCI_SRC=#{DEFAULT_CCI} + -e SRV_HTTP_HOST=#{SRV_HTTP_HOST} + -e SRV_HTTP_PORT=#{SRV_HTTP_PORT} -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} -v /etc/localtime:/etc/localtime:ro -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} -- 2.23.0

1 0

[PATCH compass-ci] job.cr: optimize set_pp_initrd() for good reading
by Xu Xijian 13 Oct '20

13 Oct '20

Define vars for twice used string and adjust code for good reading. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index be3446e..79778e4 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -387,16 +387,13 @@ class Job if @hash["pp"]? program_params = @hash["pp"].as_h program_params.keys.each do |program| - dest_file = "#{mount_type}/#{os_dir}/#{program}" - if File.exists?("#{ENV["LKP_SRC"]}/distro/depends/#{program}") && - File.exists?("#{SRV_INITRD}/deps/#{dest_file}.cgz") - initrd_deps_arr << "#{initrd_http_prefix}" + - JobHelper.service_path("#{SRV_INITRD}/deps/#{dest_file}.cgz") + deps_dest_file = "#{SRV_INITRD}/deps/#{mount_type}/#{os_dir}/#{program}.cgz" + pkg_dest_file = "#{SRV_INITRD}/pkg/#{mount_type}/#{os_dir}/#{program}.cgz" + if File.exists?("#{ENV["LKP_SRC"]}/distro/depends/#{program}") && File.exists?(deps_dest_file) + initrd_deps_arr << "#{initrd_http_prefix}" + JobHelper.service_path(deps_dest_file) end - if File.exists?("#{ENV["LKP_SRC"]}/pkg/#{program}") && - File.exists?("#{SRV_INITRD}/pkg/#{dest_file}.cgz") - initrd_pkg_arr << "#{initrd_http_prefix}" + - JobHelper.service_path("#{SRV_INITRD}/pkg/#{dest_file}.cgz") + if File.exists?("#{ENV["LKP_SRC"]}/pkg/#{program}") && File.exists?(pkg_dest_file) + initrd_pkg_arr << "#{initrd_http_prefix}" + JobHelper.service_path(pkg_dest_file) end end end -- 2.23.0

1 0

[PATCH compass-ci] fix(dnsmasq): dnsmasq does not assign boot.ipxe to the physical machine
by Yu Chuan 13 Oct '20

13 Oct '20

When there are multiple dhcp-boot in the configuration file, the dnsmasq will send the boot file corresponding to the last matched tag. Reference: - https://wiki.fogproject.org/wiki/index.php?title=ProxyDHCP_with_dnsmasq Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/dnsmasq.d/dnsmasq.conf | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/container/dnsmasq/dnsmasq.d/dnsmasq.conf b/container/dnsmasq/dnsmasq.d/dnsmasq.conf index df5bc6814e22..6141faf0436b 100644 --- a/container/dnsmasq/dnsmasq.d/dnsmasq.conf +++ b/container/dnsmasq/dnsmasq.d/dnsmasq.conf @@ -22,10 +22,12 @@ dhcp-match=set:pxeclient,60,PXEClient* dhcp-match=set:pxeclient-arm64,93,11 dhcp-match=set:pxeclient-x64,93,7 -dhcp-boot=tag:ipxe,boot.ipxe dhcp-boot=tag:pxeclient-arm64,/tftpboot/ipxe/bin-arm64-efi/snp.efi dhcp-boot=tag:pxeclient-x64,/tftpboot/ipxe/bin-x86_64-efi/snp.efi +# dhcp-boot of boot.ipxe must be on the last line. +dhcp-boot=tag:ipxe,boot.ipxe + log-queries log-facility=/var/log/dnsmasq/dnsmasq.log # more info: -- 2.23.0

1 0

[PATCH compass-ci] use local docker repository
by Xiao Shenwei 13 Oct '20

13 Oct '20

pull image from local repository, if failed will pull from docker hub usage: docker-pull NAME[:TAG] eg: docker-pull centos:7 docker-pull dc-centos:7 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- sbin/docker-pull | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100755 sbin/docker-pull diff --git a/sbin/docker-pull b/sbin/docker-pull new file mode 100755 index 0000000..61ead7e --- /dev/null +++ b/sbin/docker-pull @@ -0,0 +1,31 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +load_cci_defaults + +: ${DOCKER_REGISTRY_HOST:="127.0.0.1"} +: ${DOCKER_REGISTRY_PORT:=5001} + +image_name=$1 + +local_repository() +{ + echo "pull image from local repository" + docker pull $DOCKER_REGISTRY_HOST:$DOCKER_REGISTRY_PORT/$image_name +} + +docker_hub() +{ + echo "pull image from docker hub" + docker pull $image_name +} + +main() +{ + local_repository || docker_hub +} + +main -- 2.23.0

1 0

[PATCH lkp-tests 3/3] fix(iso2rootfs): solve possible problems with the checksum file
by Yu Chuan 13 Oct '20

13 Oct '20

If the remote server happens to release a new iso while the program is running. The checksum file will be the same as the newly released checksum file, not the same as the checksum file when the program was first executed. It's clearly not normal. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- tests/iso2rootfs | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 735c2b9aabcd..5184041abf2b 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -66,7 +66,8 @@ get_daily_iso_checksum() ISO_NAME=$(basename "$ISO_URL") ISO_CHECKSUM_URL="${ISO_URL}.sha256sum" - SHA256SUM_NET=$(curl -s "$ISO_CHECKSUM_URL" | awk '{print $1}') + curl -o "${CHECKSUM_FILE_CACHE}.tmp" "${ISO_CHECKSUM_URL}" + SHA256SUM_NET=$(awk '{print $1}' "${CHECKSUM_FILE_CACHE}.tmp") } mount_rootfs() @@ -118,6 +119,7 @@ check_sha256sum_update() if [ "$SHA256SUM_CACHE" == "$SHA256SUM_NET" ] then log_info "${iso_os} haven't release new iso, no need to generate rootfs" + rm -f "${CHECKSUM_FILE_CACHE}.tmp" exit 0 else log_info "${iso_os} release a new iso, start to generate rootfs ..." @@ -354,17 +356,11 @@ EOF } ############ post works ############ -update_sha256sum_file() -{ - [ -f "${CHECKSUM_FILE_CACHE}" ] && rm -f "${CHECKSUM_FILE_CACHE}" - wget -c -O "${CHECKSUM_FILE_CACHE}" "${ISO_CHECKSUM_URL}" -} - post_works() { log_info "starting post works ..." - update_sha256sum_file + mv "${CHECKSUM_FILE_CACHE}.tmp" "${CHECKSUM_FILE_CACHE}" umount "${ROOTFS_LOCAL_PATH}" -- 2.23.0

1 0