- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/2] lib/sched_client.rb: obtain host port from the configuration file
by Xiao Shenwei 25 Jan '21

25 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/sched_client.rb | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/sched_client.rb b/lib/sched_client.rb index f1c095d..fec4b14 100644 --- a/lib/sched_client.rb +++ b/lib/sched_client.rb @@ -3,12 +3,11 @@ # frozen_string_literal: true require 'rest-client' +require_relative 'constants' # sched client class class SchedClient - HOST = (ENV.key?('SCHED_HOST') ? ENV['SCHED_HOST'] : '172.17.0.1') - PORT = (ENV.key?('SCHED_PORT') ? ENV['SCHED_PORT'] : 3000).to_i - def initialize(host = HOST, port = PORT) + def initialize(host = SCHED_HOST, port = SCHED_PORT) @host = host @port = port end -- 2.23.0

1 0

[PATCH v3 compass-ci] lib/compare_error_messages.rb: return true if job's field job_state is finished
by Lin Jiaxin 25 Jan '21

25 Jan '21

The job is considered to be credible when its' field job_state is finished. Return true early and no need to search for error filename. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 5f74be7..9c864a3 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -13,9 +13,12 @@ require_relative "#{ENV['LKP_SRC']}/lib/common" # the later_job_id is credible. def credible?(previous_job_id, later_job_id, error_id) es = ESQuery.new + later_es_result = es.query_by_id(later_job_id) + + return true if later_es_result['job_state'] == 'finished' previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') - later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') + later_result_file = File.join('/srv', later_es_result['result_root'], 'build-pkg') filenames_check = filenames_check(previous_result_file, later_result_file, error_id) -- 2.23.0

1 0

[PATCH v6 compass-ci] providers/qemu: fix script hangs when no job now
by Liu Yinsi 25 Jan '21

25 Jan '21

[error] when no job now, $initrds is null, execute my-qemu.sh will hangs. lys@crystal ~/compass-ci/providers% ./my-qemu.sh hostname: vm-2p8g.lys-3917841 mac: 0a-4b-33-a0-96-69 DoneDone % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 79 100 79 0 0 19750 0 --:--:-- --:--:-- --:--:-- 19750 lys@crystal ~/compass-ci/providers/qemu% cat --help cat Copy standard input to standard output. when input is null, script will hang to waiting for input. [how] function set_options(): cat $initrds > $initrd before: execut cat command, then check after: check kernel, check initrds, if not null, execute cat command. in addition, check_option_value -> check_kernel, check_initrds, check_qemu convenient to call as needed. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 4d05964..b4e87fc 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -53,7 +53,7 @@ parse_ipxe_script() done < $ipxe_script } -check_option_value() +check_kernel() { [ -n "$kernel" ] || { log_info "Can not find job for current hostname: $hostname." @@ -64,10 +64,16 @@ check_option_value() log_error "Can not find kernel file or kernel file is empty: $kernel." exit 1 } - +} + +check_qemu() +{ # debian has both qemu-system-x86_64 and qemu-system-riscv64 command [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 +} +check_initrds() +{ [ -n "$initrds" ] || { log_error "The current initrds is null." exit 1 @@ -118,6 +124,8 @@ set_qemu() do command -v "$qemu" > /dev/null && break done + + check_qemu } print_message() @@ -209,8 +217,10 @@ write_logfile parse_ipxe_script +check_kernel +check_initrds + set_options -check_option_value print_message -- 2.23.0

2 2

[PATCH v2 compass-ci] lib/compare_error_messages.rb: return true if job state = finished
by Lin Jiaxin 25 Jan '21

25 Jan '21

The job is considered to be credible when its' state = finished. Return true early and no need to search for error filename. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 5f74be7..9c864a3 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -13,9 +13,12 @@ require_relative "#{ENV['LKP_SRC']}/lib/common" # the later_job_id is credible. def credible?(previous_job_id, later_job_id, error_id) es = ESQuery.new + later_es_result = es.query_by_id(later_job_id) + + return true if later_es_result['job_state'] == 'finished' previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') - later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') + later_result_file = File.join('/srv', later_es_result['result_root'], 'build-pkg') filenames_check = filenames_check(previous_result_file, later_result_file, error_id) -- 2.23.0

2 2

[PATCH v1 compass-ci] lib: introduce log.rb
by Xiao Shenwei 25 Jan '21

25 Jan '21

Records the run logs of the ruby program message format: 2021-01-25 11:00:09 +0800 INFO -- SCHED_PORT: 3000 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/log.rb | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 lib/log.rb diff --git a/lib/log.rb b/lib/log.rb new file mode 100644 index 0000000..762c0b2 --- /dev/null +++ b/lib/log.rb @@ -0,0 +1,33 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' + +# records the run logs of the ruby program +class Log < Logger + def initialize(filename) + File.delete(filename) if FileTest.exist?(filename) + super(filename) + format + end + + private + + def format + self.datetime_format = '%Y-%m-%d %H:%M:%s' + self.formatter = proc do |severity, datetime, _progname, msg| + transform_msg(msg).map { |m| "#{datetime} #{severity} -- #{m}\n" }.join + end + end + + def transform_msg(msg) + msg = if msg.is_a? Exception + ["#{msg.backtrace.first}: #{msg.message.split("\n").first} (#{msg.class.name})", + msg.backtrace[1..-1].map { |m| "\tfrom #{m}" }].flatten + else + msg.to_s.split("\n") + end + return msg + end +end -- 2.23.0

2 3

[PATCH compass-ci] lib/compare_error_messages.rb: return true if job state = finished
by Lin Jiaxin 25 Jan '21

25 Jan '21

The job is considered to be credible when its' state = finished. Return true early and no need to search for error filename. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 5f74be7..978d6ba 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -14,6 +14,8 @@ require_relative "#{ENV['LKP_SRC']}/lib/common" def credible?(previous_job_id, later_job_id, error_id) es = ESQuery.new + return true if es.query_by_id(later_job_id)['job_state'] == 'finished' + previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') -- 2.23.0

2 1

[PATCH lab-z9] hosts: replace field rootfs_partition to rootfs_disk of all hosts file
by Yu Chuan 25 Jan '21

25 Jan '21

[Why] 1. The current value of rootfs_partition field is actually disk, not the partition. 2. We now need auto install iso to the specific disk of testbox, so in order to avoid conflict with rootfs_partition in lkp-tests framework, we use rootfs_disk instead of rootfs_partition. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- hosts/taishan200-2280-2s48p-256g--a11 | 2 +- hosts/taishan200-2280-2s48p-256g--a12 | 2 +- hosts/taishan200-2280-2s48p-256g--a13 | 2 +- hosts/taishan200-2280-2s48p-256g--a14 | 2 +- hosts/taishan200-2280-2s48p-256g--a15 | 2 +- hosts/taishan200-2280-2s48p-256g--a17 | 2 +- hosts/taishan200-2280-2s48p-256g--a18 | 2 +- hosts/taishan200-2280-2s48p-256g--a19 | 2 +- hosts/taishan200-2280-2s48p-256g--a21 | 2 +- hosts/taishan200-2280-2s48p-256g--a22 | 2 +- hosts/taishan200-2280-2s48p-256g--a24 | 2 +- hosts/taishan200-2280-2s48p-256g--a25 | 2 +- hosts/taishan200-2280-2s48p-256g--a26 | 2 +- hosts/taishan200-2280-2s48p-256g--a27 | 2 +- hosts/taishan200-2280-2s48p-256g--a28 | 2 +- hosts/taishan200-2280-2s48p-256g--a29 | 2 +- hosts/taishan200-2280-2s48p-256g--a30 | 2 +- hosts/taishan200-2280-2s48p-256g--a31 | 2 +- hosts/taishan200-2280-2s48p-256g--a32 | 2 +- hosts/taishan200-2280-2s48p-256g--a33 | 2 +- hosts/taishan200-2280-2s48p-256g--a34 | 2 +- hosts/taishan200-2280-2s48p-256g--a52 | 2 +- hosts/taishan200-2280-2s48p-256g--a53 | 2 +- hosts/taishan200-2280-2s48p-256g--a60 | 2 +- hosts/taishan200-2280-2s48p-256g--a61 | 2 +- hosts/taishan200-2280-2s48p-256g--a62 | 2 +- hosts/taishan200-2280-2s48p-256g--a66 | 2 +- hosts/taishan200-2280-2s48p-256g--a67 | 2 +- hosts/taishan200-2280-2s48p-256g--a68 | 2 +- hosts/taishan200-2280-2s48p-256g--a69 | 2 +- hosts/taishan200-2280-2s48p-256g--a70 | 2 +- hosts/taishan200-2280-2s64p-256g--a10 | 2 +- hosts/taishan200-2280-2s64p-256g--a3 | 2 +- hosts/taishan200-2280-2s64p-256g--a36 | 2 +- hosts/taishan200-2280-2s64p-256g--a4 | 2 +- hosts/taishan200-2280-2s64p-256g--a40 | 2 +- hosts/taishan200-2280-2s64p-256g--a41 | 2 +- hosts/taishan200-2280-2s64p-256g--a42 | 2 +- hosts/taishan200-2280-2s64p-256g--a8 | 2 +- hosts/taishan200-2280-2s64p-256g--a9 | 2 +- 40 files changed, 40 insertions(+), 40 deletions(-) diff --git a/hosts/taishan200-2280-2s48p-256g--a11 b/hosts/taishan200-2280-2s48p-256g--a11 index 619dc063cf0f..412154f8c8e8 100644 --- a/hosts/taishan200-2280-2s48p-256g--a11 +++ b/hosts/taishan200-2280-2s48p-256g--a11 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8891431-part1 - /dev/disk/by-id/scsi-350000399b8890e09 - /dev/disk/by-id/scsi-350000399b8890cd1 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8727929 mac_addr: - cc:05:77:ab:97:62 diff --git a/hosts/taishan200-2280-2s48p-256g--a12 b/hosts/taishan200-2280-2s48p-256g--a12 index ce014c133bb6..3baa8f187ba3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a12 +++ b/hosts/taishan200-2280-2s48p-256g--a12 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c500c276c347 - /dev/disk/by-id/scsi-35000c500c276eb87 - /dev/disk/by-id/scsi-35000c500c276b6ff -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500c276f47b mac_addr: - 44:67:47:97:22:37 diff --git a/hosts/taishan200-2280-2s48p-256g--a13 b/hosts/taishan200-2280-2s48p-256g--a13 index 468c175a3111..4b9548fa032e 100644 --- a/hosts/taishan200-2280-2s48p-256g--a13 +++ b/hosts/taishan200-2280-2s48p-256g--a13 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c500c2799bb3 - /dev/disk/by-id/scsi-35000c500c278b18b - /dev/disk/by-id/scsi-35000c500c279f96b -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500c2789b4b mac_addr: - 44:67:47:97:2b:bf diff --git a/hosts/taishan200-2280-2s48p-256g--a14 b/hosts/taishan200-2280-2s48p-256g--a14 index 24ea9dc8daf5..b96e79c9029a 100644 --- a/hosts/taishan200-2280-2s48p-256g--a14 +++ b/hosts/taishan200-2280-2s48p-256g--a14 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4475 - /dev/disk/by-id/scsi-350000399b88b4371 - /dev/disk/by-id/scsi-350000399b88b432d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88b4531 mac_addr: - 44:67:47:c9:e9:c2 diff --git a/hosts/taishan200-2280-2s48p-256g--a15 b/hosts/taishan200-2280-2s48p-256g--a15 index 9c28258def58..a1d63ed4fcc5 100644 --- a/hosts/taishan200-2280-2s48p-256g--a15 +++ b/hosts/taishan200-2280-2s48p-256g--a15 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b841fff5 - /dev/disk/by-id/scsi-350000399b841fcb9 - /dev/disk/by-id/scsi-350000399b8420a0d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b841ff55 mac_addr: - 44:67:47:85:d4:9e diff --git a/hosts/taishan200-2280-2s48p-256g--a17 b/hosts/taishan200-2280-2s48p-256g--a17 index 16b065db0f66..9437ac9ea397 100644 --- a/hosts/taishan200-2280-2s48p-256g--a17 +++ b/hosts/taishan200-2280-2s48p-256g--a17 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b44cd - /dev/disk/by-id/scsi-350000399b88b4541 - /dev/disk/by-id/scsi-350000399b88b4495 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88b48d1 mac_addr: - 44:67:47:c9:ea:1c diff --git a/hosts/taishan200-2280-2s48p-256g--a18 b/hosts/taishan200-2280-2s48p-256g--a18 index 9d1ec017438a..0567352ac36b 100644 --- a/hosts/taishan200-2280-2s48p-256g--a18 +++ b/hosts/taishan200-2280-2s48p-256g--a18 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b891b4d1 - /dev/disk/by-id/scsi-350000399b8890d59 - /dev/disk/by-id/scsi-350000399b8890f65 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b891b3c5 mac_addr: - 84:46:fe:3d:a7:e9 diff --git a/hosts/taishan200-2280-2s48p-256g--a19 b/hosts/taishan200-2280-2s48p-256g--a19 index 217eaa9ec067..7e86603af883 100644 --- a/hosts/taishan200-2280-2s48p-256g--a19 +++ b/hosts/taishan200-2280-2s48p-256g--a19 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8923939 - /dev/disk/by-id/scsi-350000399b8420b2d - /dev/disk/by-id/scsi-350000399b891b5c5 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b841f769 mac_addr: - 44:67:47:85:d5:48 diff --git a/hosts/taishan200-2280-2s48p-256g--a21 b/hosts/taishan200-2280-2s48p-256g--a21 index 738738d134cb..72fb511ae1bb 100644 --- a/hosts/taishan200-2280-2s48p-256g--a21 +++ b/hosts/taishan200-2280-2s48p-256g--a21 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf2d5 - /dev/disk/by-id/scsi-350000399b88bf5a5 - /dev/disk/by-id/scsi-350000399b88bf279 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88bf24d mac_addr: - 44:67:47:86:1b:e0 diff --git a/hosts/taishan200-2280-2s48p-256g--a22 b/hosts/taishan200-2280-2s48p-256g--a22 index bb29d9432b52..73a2799006a3 100644 --- a/hosts/taishan200-2280-2s48p-256g--a22 +++ b/hosts/taishan200-2280-2s48p-256g--a22 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919069 - /dev/disk/by-id/scsi-350000399b8918c15 - /dev/disk/by-id/scsi-350000399b8918c79 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8918cb9 mac_addr: - 44:67:47:d7:6d:14 diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 index 60ea1d2a1588..8f78184d4da0 100644 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ b/hosts/taishan200-2280-2s48p-256g--a24 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8917d59 - /dev/disk/by-id/scsi-350000399b89185d9 - /dev/disk/by-id/scsi-350000399b891790d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8917ef1 mac_addr: - 44:67:47:c9:d8:f4 diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 index 916513e68f0e..d8f721c58888 100644 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ b/hosts/taishan200-2280-2s48p-256g--a25 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88b4499 - /dev/disk/by-id/scsi-350000399b88b4455 - /dev/disk/by-id/scsi-350000399b88b442d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88b44f9 mac_addr: - 44:67:47:c9:ea:08 diff --git a/hosts/taishan200-2280-2s48p-256g--a26 b/hosts/taishan200-2280-2s48p-256g--a26 index 4a5a0f350bec..210e15d2c724 100644 --- a/hosts/taishan200-2280-2s48p-256g--a26 +++ b/hosts/taishan200-2280-2s48p-256g--a26 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf131 - /dev/disk/by-id/scsi-350000399b88bfd8d - /dev/disk/by-id/scsi-350000399b88bfe55 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88bef31 mac_addr: - 44:67:47:c9:e9:40 diff --git a/hosts/taishan200-2280-2s48p-256g--a27 b/hosts/taishan200-2280-2s48p-256g--a27 index 129589cde687..5d5559e35597 100644 --- a/hosts/taishan200-2280-2s48p-256g--a27 +++ b/hosts/taishan200-2280-2s48p-256g--a27 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918ea9 - /dev/disk/by-id/scsi-350000399b8918bad - /dev/disk/by-id/scsi-350000399b8918d05 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8919415 mac_addr: - 44:67:47:d7:6d:3c diff --git a/hosts/taishan200-2280-2s48p-256g--a28 b/hosts/taishan200-2280-2s48p-256g--a28 index f8a77f8408b9..2e32147db8ff 100644 --- a/hosts/taishan200-2280-2s48p-256g--a28 +++ b/hosts/taishan200-2280-2s48p-256g--a28 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdf1eb07 - /dev/disk/by-id/scsi-35000c500cdef4c6f - /dev/disk/by-id/scsi-35000c500cdefac2b -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500cdefa75f mac_addr: - 5c:e8:83:31:c8:20 diff --git a/hosts/taishan200-2280-2s48p-256g--a29 b/hosts/taishan200-2280-2s48p-256g--a29 index 37814cc0e043..fff6a831f2aa 100644 --- a/hosts/taishan200-2280-2s48p-256g--a29 +++ b/hosts/taishan200-2280-2s48p-256g--a29 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918425 - /dev/disk/by-id/scsi-350000399b891841d - /dev/disk/by-id/scsi-350000399b8917859 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8917a51 mac_addr: - 44:67:47:c9:d8:ea diff --git a/hosts/taishan200-2280-2s48p-256g--a30 b/hosts/taishan200-2280-2s48p-256g--a30 index f722edbd7795..713d7af7017d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a30 +++ b/hosts/taishan200-2280-2s48p-256g--a30 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b881 - /dev/disk/by-id/scsi-350000399b88bf83d - /dev/disk/by-id/scsi-350000399b883bbbd -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88bf48d mac_addr: - 44:67:47:86:1b:cc diff --git a/hosts/taishan200-2280-2s48p-256g--a31 b/hosts/taishan200-2280-2s48p-256g--a31 index 7d6c67cc8400..c112fe2c6235 100644 --- a/hosts/taishan200-2280-2s48p-256g--a31 +++ b/hosts/taishan200-2280-2s48p-256g--a31 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b883b929 - /dev/disk/by-id/scsi-350000399b883bbe1 - /dev/disk/by-id/scsi-350000399b883c059 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b883bc9d mac_addr: - 44:67:47:86:1b:ae diff --git a/hosts/taishan200-2280-2s48p-256g--a32 b/hosts/taishan200-2280-2s48p-256g--a32 index fb17e38aebb1..212db948e368 100644 --- a/hosts/taishan200-2280-2s48p-256g--a32 +++ b/hosts/taishan200-2280-2s48p-256g--a32 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bf141 - /dev/disk/by-id/scsi-350000399b88bee25 - /dev/disk/by-id/scsi-350000399b88bea59 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88be1d9 mac_addr: - 44:67:47:c9:db:38 diff --git a/hosts/taishan200-2280-2s48p-256g--a33 b/hosts/taishan200-2280-2s48p-256g--a33 index a3a6b0306ad4..96bb5a4b5d1d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a33 +++ b/hosts/taishan200-2280-2s48p-256g--a33 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919555 - /dev/disk/by-id/scsi-350000399b8918c4d - /dev/disk/by-id/scsi-350000399b8919c3d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8919389 mac_addr: - 44:67:47:d7:6c:c4 diff --git a/hosts/taishan200-2280-2s48p-256g--a34 b/hosts/taishan200-2280-2s48p-256g--a34 index 5a15949fe123..727f870fb7e6 100644 --- a/hosts/taishan200-2280-2s48p-256g--a34 +++ b/hosts/taishan200-2280-2s48p-256g--a34 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be219 - /dev/disk/by-id/scsi-350000399b88bdf71 - /dev/disk/by-id/scsi-350000399b88bdb49 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b891784d mac_addr: - 44:67:47:c9:d9:26 diff --git a/hosts/taishan200-2280-2s48p-256g--a52 b/hosts/taishan200-2280-2s48p-256g--a52 index 2bd70962a573..0deb149ef6fa 100644 --- a/hosts/taishan200-2280-2s48p-256g--a52 +++ b/hosts/taishan200-2280-2s48p-256g--a52 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c500cdefd89b - /dev/disk/by-id/scsi-35000c500cdefac67 - /dev/disk/by-id/scsi-35000c500cdedee6f -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500cdedfdaf mac_addr: - 5c:e8:83:31:c8:d4 diff --git a/hosts/taishan200-2280-2s48p-256g--a53 b/hosts/taishan200-2280-2s48p-256g--a53 index bf0894b46f99..ca075737a4e6 100644 --- a/hosts/taishan200-2280-2s48p-256g--a53 +++ b/hosts/taishan200-2280-2s48p-256g--a53 @@ -5,7 +5,7 @@ memory: 256G nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6aa4e3 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500bd681acf mac_addr: - 5c:e8:83:73:25:c9 diff --git a/hosts/taishan200-2280-2s48p-256g--a60 b/hosts/taishan200-2280-2s48p-256g--a60 index 2255f2f52478..13612be3785d 100644 --- a/hosts/taishan200-2280-2s48p-256g--a60 +++ b/hosts/taishan200-2280-2s48p-256g--a60 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd67640b - /dev/disk/by-id/scsi-35000c500bd6707a7 - /dev/disk/by-id/scsi-35000c500bd680dc3 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500bd683a73 mac_addr: - e0:00:84:2b:51:7d diff --git a/hosts/taishan200-2280-2s48p-256g--a61 b/hosts/taishan200-2280-2s48p-256g--a61 index 73079cbebb0a..1619d17f99e9 100644 --- a/hosts/taishan200-2280-2s48p-256g--a61 +++ b/hosts/taishan200-2280-2s48p-256g--a61 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399886a7869 - /dev/disk/by-id/scsi-350000399886acc5d - /dev/disk/by-id/scsi-350000399886b6861 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399886ac70d mac_addr: - 44:67:47:c9:e4:b8 diff --git a/hosts/taishan200-2280-2s48p-256g--a62 b/hosts/taishan200-2280-2s48p-256g--a62 index abaac6e80c02..f1d72af845d0 100644 --- a/hosts/taishan200-2280-2s48p-256g--a62 +++ b/hosts/taishan200-2280-2s48p-256g--a62 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b892d131 - /dev/disk/by-id/scsi-350000399b892d619 - /dev/disk/by-id/scsi-350000399b892dc1d -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b892dbb5 mac_addr: - 44:67:47:c9:e3:3a diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 index fce93d166425..24d9f04e52ec 100644 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ b/hosts/taishan200-2280-2s48p-256g--a66 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-35000c5009a671773 - /dev/disk/by-id/scsi-35000c5009a690c53 - /dev/disk/by-id/scsi-35000c5009a6713fb -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c5009a65963f mac_addr: - 5c:e8:83:31:c8:8e diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 index 6b326099609c..35bf63983e71 100644 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ b/hosts/taishan200-2280-2s48p-256g--a67 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88aed21 - /dev/disk/by-id/scsi-350000399b88af325 - /dev/disk/by-id/scsi-350000399b88af699 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88af421 mac_addr: - 44:67:47:c9:e2:2c diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 index 4bdd8f4fa83b..7c5091311f76 100644 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ b/hosts/taishan200-2280-2s48p-256g--a68 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b8909f81 - /dev/disk/by-id/scsi-350000399b8909f79 - /dev/disk/by-id/scsi-350000399b8909605 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8909b85 mac_addr: - 84:46:fe:89:b6:e6 diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 index 3d3be8042fb2..c796b53bb529 100644 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ b/hosts/taishan200-2280-2s48p-256g--a69 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b88be92d - /dev/disk/by-id/scsi-350000399b88befc5 - /dev/disk/by-id/scsi-350000399b88be701 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b88bea5d mac_addr: - 44:67:47:c9:d9:4e diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 index 89c3c772fa7b..6064b87f718c 100644 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ b/hosts/taishan200-2280-2s48p-256g--a70 @@ -7,7 +7,7 @@ hdd_partitions: - /dev/disk/by-id/scsi-350000399b890931d - /dev/disk/by-id/scsi-350000399b8909d79 - /dev/disk/by-id/scsi-350000399b8909ac5 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399b8909c99 mac_addr: - 84:46:fe:89:b7:22 diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 index cdb32f54343f..c6e6b56dc6f1 100644 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ b/hosts/taishan200-2280-2s64p-256g--a10 @@ -3,7 +3,7 @@ nr_node: 4 nr_cpu: 128 memory: 256G nr_ssd_partitions: 1 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399c8030fd5 ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB44333 diff --git a/hosts/taishan200-2280-2s64p-256g--a3 b/hosts/taishan200-2280-2s64p-256g--a3 index 8f34f3f8bdec..1c7a2e9fc4f2 100644 --- a/hosts/taishan200-2280-2s64p-256g--a3 +++ b/hosts/taishan200-2280-2s64p-256g--a3 @@ -7,7 +7,7 @@ ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part1 - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part2 - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part3 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399981347d5 mac_addr: - 68:4a:ae:ca:e8:ca diff --git a/hosts/taishan200-2280-2s64p-256g--a36 b/hosts/taishan200-2280-2s64p-256g--a36 index 97571449df9c..1b0bf294956d 100644 --- a/hosts/taishan200-2280-2s64p-256g--a36 +++ b/hosts/taishan200-2280-2s64p-256g--a36 @@ -3,7 +3,7 @@ nr_node: 4 nr_cpu: 128 memory: 256G nr_ssd_partitions: 1 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-3500003999813487d ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB42949-part1 diff --git a/hosts/taishan200-2280-2s64p-256g--a4 b/hosts/taishan200-2280-2s64p-256g--a4 index bcfaea74799d..b3a4ad7b2da0 100644 --- a/hosts/taishan200-2280-2s64p-256g--a4 +++ b/hosts/taishan200-2280-2s64p-256g--a4 @@ -7,7 +7,7 @@ ssd_partitions: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part1 - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part2 - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part3 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399c802cfb9 mac_addr: - 68:4a:ae:ca:e8:a2 diff --git a/hosts/taishan200-2280-2s64p-256g--a40 b/hosts/taishan200-2280-2s64p-256g--a40 index d9a993725dbc..f30f7fb4f8ea 100644 --- a/hosts/taishan200-2280-2s64p-256g--a40 +++ b/hosts/taishan200-2280-2s64p-256g--a40 @@ -5,7 +5,7 @@ memory: 256G nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-3500003999812ed81 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399c8020115 mac_addr: - 84:46:fe:89:be:8e diff --git a/hosts/taishan200-2280-2s64p-256g--a41 b/hosts/taishan200-2280-2s64p-256g--a41 index 91531fded637..ab2073edd46c 100644 --- a/hosts/taishan200-2280-2s64p-256g--a41 +++ b/hosts/taishan200-2280-2s64p-256g--a41 @@ -5,7 +5,7 @@ memory: 256G nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8919495 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000039998130a85 mac_addr: - 84:46:fe:89:bc:4a diff --git a/hosts/taishan200-2280-2s64p-256g--a42 b/hosts/taishan200-2280-2s64p-256g--a42 index ed1f188b00f3..ecaccae5bda0 100644 --- a/hosts/taishan200-2280-2s64p-256g--a42 +++ b/hosts/taishan200-2280-2s64p-256g--a42 @@ -5,7 +5,7 @@ memory: 256G nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8918a11 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-350000399c802ee75 mac_addr: - cc:05:77:fe:d8:64 diff --git a/hosts/taishan200-2280-2s64p-256g--a8 b/hosts/taishan200-2280-2s64p-256g--a8 index e515306aba96..8a02f712cd31 100644 --- a/hosts/taishan200-2280-2s64p-256g--a8 +++ b/hosts/taishan200-2280-2s64p-256g--a8 @@ -2,7 +2,7 @@ arch: aarch64 nr_node: 4 nr_cpu: 128 memory: 256G -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43325 mac_addr: - 84:46:fe:26:12:f5 diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 index ce3aa0a99073..fd8473fb4076 100644 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ b/hosts/taishan200-2280-2s64p-256g--a9 @@ -5,7 +5,7 @@ memory: 256G nr_hdd_partitions: 1 hdd_partitions: - /dev/disk/by-id/scsi-35000c500bd6a1347 -rootfs_partition: +rootfs_disk: - /dev/disk/by-id/scsi-35000c500bd62acb3 mac_addr: - cc:05:77:fe:d4:0e -- 2.23.0

1 0

[PATCH v2 lkp-tests] iso2rootfs: fix domain name cannot be resolved problem
by Yu Chuan 25 Jan '21

25 Jan '21

[Why] We have added some nameservers to /etc/resolv.conf of rootfs, but this file will be overwritten when NetworkManager service starts. [How] Add config option `dns=none` to /etc/NetworkManager/NetworkManager.conf. [Reference] dns in NetworkManager.conf Set the DNS (resolv.conf) processing mode. default: The default if the key is not specified. NetworkManager will update resolv.conf to reflect the nameservers provided by currently active connections. dnsmasq: NetworkManager will run dnsmasq as a local caching nameserver, using a "split DNS" configuration if you are connected to a VPN, and then update resolv.conf to point to the local nameserver. none: NetworkManager will not modify resolv.conf. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- tests/iso2rootfs | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/iso2rootfs b/tests/iso2rootfs index cb077fa28b0f..65a1935b7f49 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -358,6 +358,10 @@ run_qcow2rootfs() config_dns_resolver() { echo "${config_rootfs}" | grep -qw 'dns' || return 0 + + grep -qs "^dns=none" $config_file || + sed -i '/^\[main\]/adns=none' "${ROOTFS_DES_DIR}/etc/NetworkManager/NetworkManager.conf" + cat <<-EOF > "${ROOTFS_DES_DIR}/etc/resolv.conf" nameserver 114.114.114.114 nameserver 8.8.8.8 -- 2.23.0

1 0

[PATCH lkp-tests] iso2rootfs: fix domain name cannot be resolved problem
by Yu Chuan 25 Jan '21

25 Jan '21

[Why] We have added some nameservers to /etc/resolv.conf of rootfs, but this file will be overwritten when NetworkManager service starts. [How] Add config option `dns=none` to /etc/NetworkManager/NetworkManager.conf. [Reference] dns in NetworkManager.conf Set the DNS (resolv.conf) processing mode. default: The default if the key is not specified. NetworkManager will update resolv.conf to reflect the nameservers provided by currently active connections. dnsmasq: NetworkManager will run dnsmasq as a local caching nameserver, using a "split DNS" configuration if you are connected to a VPN, and then update resolv.conf to point to the local nameserver. none: NetworkManager will not modify resolv.conf. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- tests/iso2rootfs | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/iso2rootfs b/tests/iso2rootfs index cb077fa28b0f..cde1d12e4cd3 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -358,6 +358,9 @@ run_qcow2rootfs() config_dns_resolver() { echo "${config_rootfs}" | grep -qw 'dns' || return 0 + + grep "^dns=none" $config_file || sed -i '/^\[main\]/adns=none' "${ROOTFS_DES_DIR}/etc/NetworkManager/NetworkManager.conf" + cat <<-EOF > "${ROOTFS_DES_DIR}/etc/resolv.conf" nameserver 114.114.114.114 nameserver 8.8.8.8 -- 2.23.0

2 2

[PATCH v2 compass-ci] lib/matrix2.rb support merge message|log line to stats
by Lu Weitao 25 Jan '21

25 Jan '21

[Example] { "iperf.tcp.sender.bps": 34804801216.197174, "iperf.tcp.receiver.bps": 34804762215.18231, "iperf.run.message": "error message xxx", "iperf.run.log": "error log xxx" } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/matrix2.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 8444334..5920d62 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -10,6 +10,8 @@ require "#{LKP_SRC}/lib/matrix" require_relative './params_group.rb' def set_pre_value(item, value, sample_size) + return value if value.is_a?(String) + if value.size == 1 value[0] elsif independent_counter? item -- 2.23.0

1 0

[PATCH v3 lkp-tests] sbin/dump-stats: keep message|log line
by Lu Weitao 25 Jan '21

25 Jan '21

[Example] { "iperf.tcp.sender.bps": [ 34804801216.197174 ], "iperf.tcp.receiver.bps": [ 34804762215.18231 ], "iperf.run.message": "error message xxx", "iperf.run.log": "error log xxx" } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/dump-stat | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/sbin/dump-stat b/sbin/dump-stat index b5abf5a9b..6e10e432d 100755 --- a/sbin/dump-stat +++ b/sbin/dump-stat @@ -59,6 +59,13 @@ while (line = STDIN.gets) exit 1 end + + # keep message | log line which key end with .message|.log + if k.end_with?('.message', '.log') + result[k] = v + next + end + # only number is valid unless v.numeric? invalid_records.push record_index @@ -84,6 +91,8 @@ min_cols_stat = '' max_cols_stat = '' zero_stats = [] result.each do |key, val| + next if val.is_a?(String) + if max_cols < val.size max_cols = val.size max_cols_stat = key -- 2.23.0

1 0

[PATCH v2 lkp-tests] params/timeout: convert timeout into seconds
by Wu Zhende 25 Jan '21

25 Jan '21

It's hard to process the "timeout" in different units in the code, and convert them into integers in seconds. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- params/timeout | 1 + 1 file changed, 1 insertion(+) create mode 120000 params/timeout diff --git a/params/timeout b/params/timeout new file mode 120000 index 000000000..9e1976b33 --- /dev/null +++ b/params/timeout @@ -0,0 +1 @@ +runtime \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests 2/2] params/timeout: convert timeout into seconds
by Wu Zhende 25 Jan '21

25 Jan '21

It's hard to process the "timeout" in different units in the code, and convert them into integers in seconds. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- params/timeout | 1 + 1 file changed, 1 insertion(+) create mode 120000 params/timeout diff --git a/params/timeout b/params/timeout new file mode 120000 index 000000000..9e1976b33 --- /dev/null +++ b/params/timeout @@ -0,0 +1 @@ +runtime \ No newline at end of file -- 2.23.0

1 0

[PATCH lkp-tests 1/2] sbin/submit: add submit_id
by Wu Zhende 25 Jan '21

25 Jan '21

A globally "submit_id" is automatically generated for each submit command and automatically added to jobs. Indicates the jobs of the same batch submitted by the command. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/submit | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sbin/submit b/sbin/submit index a92556fff..5273896b6 100755 --- a/sbin/submit +++ b/sbin/submit @@ -103,6 +103,9 @@ def find_jobfile(jobfile) exit 1 end +submit_id = %x(uuidgen).chomp +puts "submit_id=#{submit_id}" + ARGV.each do |jobfile| jobfile = find_jobfile(jobfile) jobs = Job2sh.new @@ -113,6 +116,7 @@ ARGV.each do |jobfile| jobs['testbox'] = opt_set_key_value['testbox'] if opt_set_key_value['testbox'] jobs['tbox_group'] = tbox_group(jobs['testbox']) if jobs.include?('testbox') jobs['node_roles'] ||= 'server client' if jobs['cluster'] + jobs['submit_id'] = submit_id jobs.each_jobs do |job| raise 'Please configure SCHED_HOST' unless job['SCHED_HOST'] raise 'Please configure SCHED_PORT' unless job['SCHED_PORT'] -- 2.23.0

1 0

[PATCH compass-ci] taskqueue: add start dependent container: redis
by Yu Chuan 25 Jan '21

25 Jan '21

[Why] The container taskqueue start depends on redis, so add redis to taskqueue/start-depends. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/taskqueue/start-depends | 1 + 1 file changed, 1 insertion(+) create mode 100755 container/taskqueue/start-depends diff --git a/container/taskqueue/start-depends b/container/taskqueue/start-depends new file mode 100755 index 000000000000..7800f0fad3ff --- /dev/null +++ b/container/taskqueue/start-depends @@ -0,0 +1 @@ +redis -- 2.23.0

1 0

[PATCH v2 lkp-tests] sbin/dump-stats: keep message line
by Lu Weitao 25 Jan '21

25 Jan '21

[Example] { "iperf.tcp.sender.bps": [ 34804801216.197174 ], "iperf.tcp.receiver.bps": [ 34804762215.18231 ], "iperf.run.message": "error xxx" } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/dump-stat | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/sbin/dump-stat b/sbin/dump-stat index b5abf5a9b..5d565e36a 100755 --- a/sbin/dump-stat +++ b/sbin/dump-stat @@ -59,6 +59,12 @@ while (line = STDIN.gets) exit 1 end + + if k.end_with?('message') + result[k] = v + next + end + # only number is valid unless v.numeric? invalid_records.push record_index @@ -84,6 +90,8 @@ min_cols_stat = '' max_cols_stat = '' zero_stats = [] result.each do |key, val| + next if val.is_a?(String) + if max_cols < val.size max_cols = val.size max_cols_stat = key -- 2.23.0

2 2

[PATCH v4 compass-ci 1/3] providers/libvirt: add a libvirt provider
by Xiao Shenwei 25 Jan '21

25 Jan '21

libvirt is a vm management tool, it based on a domain.xml to run vm each element of xml can be parsed qemu-kvm parameter. so we can configure xml file to run different type of vm Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/client.rb | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 providers/libvirt/client.rb diff --git a/providers/libvirt/client.rb b/providers/libvirt/client.rb new file mode 100755 index 0000000..29cdaf3 --- /dev/null +++ b/providers/libvirt/client.rb @@ -0,0 +1,23 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative 'lib/mylog' +require_relative 'lib/consumer' + +def main(hostname, queues) + puts "cat #{hostname}.log" + logger = Mylog.new("#{hostname}.log") + consumer = Consumer.new(hostname, queues, logger) + response = consumer.connect + if response.nil? || response['job_id'].empty? + logger.warn('no job now') + puts '----------' + puts 'no job now' + puts '----------' + return + end + ## clean env + consumer.clean_info +end -- 2.23.0

2 2

[PATCH compass-ci] taskqueue: add build dependent container: redis
by Yu Chuan 25 Jan '21

25 Jan '21

[Why] The container taskqueue depends on redis, so add redis to taskqueue/build-depends. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/taskqueue/build-depends | 1 + 1 file changed, 1 insertion(+) diff --git a/container/taskqueue/build-depends b/container/taskqueue/build-depends index 5dd403e166af..a961b878fb44 100755 --- a/container/taskqueue/build-depends +++ b/container/taskqueue/build-depends @@ -1 +1,2 @@ scheduler-dev +redis -- 2.23.0

2 4

[PATCH v4 compass-ci 3/3] libvirt/lib: request scheduler to consume job
by Xiao Shenwei 23 Jan '21

23 Jan '21

request url: http://$SCHED_HOST:$SCHED_PORT/boot.libvirt/mac/$mac Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/consumer.rb | 85 +++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 providers/libvirt/lib/consumer.rb diff --git a/providers/libvirt/lib/consumer.rb b/providers/libvirt/lib/consumer.rb new file mode 100644 index 0000000..f1477fe --- /dev/null +++ b/providers/libvirt/lib/consumer.rb @@ -0,0 +1,85 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'yaml' +require 'json' +require 'rest-client' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +# register hostname mac queues +# request scheduler +class Consumer + attr_reader :info + + def initialize(hostname, queues, logger) + @logger = logger + @info = { 'hostname' => hostname, 'queues' => queues } + host_exists + parse_mac_from_hostname + defconfig + register_info + end + + def connect + res = RestClient.get("http://#{@sched_host}:#{@sched_port}/boot.libvirt/mac/#{@info['mac']}") + if res.empty? + @logger.error('Can not connect scheduler') + raise 'Can not connect scheduler' + end + JSON.parse(res) + end + + def clean_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host_mac?mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host2queues?host=#{@info['hostname']}", {} + ) + end + + private + + def defconfig + names = Set.new %w[ + SCHED_HOST + SCHED_PORT + ] + defaults = relevant_defaults(names) + @sched_host = defaults['SCHED_HOST'] || '172.17.0.1' + @sched_port = defaults['SCHED_PORT'] || 3000 + @logger.info("SCHED_HOST: #{@sched_host}") + @logger.info("SCHED_PORT: #{@sched_port}") + end + + def parse_mac_from_hostname + # TODO + # use digest/md5 replace + cmd = %(echo #{@info['hostname']} | + md5sum | + sed "s/^\$..\$\$..\$\$..\$\$..\$\$..\$.*$/0a-\\1-\\2-\\3-\\4-\\5/" + ) + @info['mac'] = %x(#{cmd}).chomp + @logger.info("Mac address: #{@info['mac']}") + end + + def register_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host_mac?hostname=#{@info['hostname']}&mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host2queues?host=#{@info['hostname']}&queues=#{@info['queues']}", {} + ) + end + + def host_exists + @info['host'] = @info['hostname'].split('.')[0] + host_file = "#{ENV['LKP_SRC']}/hosts/#{@info['host']}" + return if FileTest.exists?(host_file) + + @logger.error("The file does't exist: #{host_file}") + raise "The file does't exist: #{host_file}" + end +end -- 2.23.0

2 1

[PATCH v4 compass-ci 2/3] libvirt/lib: introduce mylog.rb
by Xiao Shenwei 23 Jan '21

23 Jan '21

[why] records run logs 1. easy debugging 2. the log will be upload to result_root in this way, user can get message when error occurred [how] custom format of log like this: 2021-01-22 10:11:15 +0800 INFO -- SCHED_HOST: 3000 Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/mylog.rb | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 providers/libvirt/lib/mylog.rb diff --git a/providers/libvirt/lib/mylog.rb b/providers/libvirt/lib/mylog.rb new file mode 100644 index 0000000..38f8c98 --- /dev/null +++ b/providers/libvirt/lib/mylog.rb @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' + +# define a logger format +class Mylog < Logger + def initialize(filename) + File.delete(filename) if File.exist?(filename) + super(filename) + format + end + + private + + def format + self.datetime_format = '%Y-%m-%d %H:%M:%s' + self.formatter = proc do |severity, datetime, _progname, msg| + msg = if msg.is_a? Exception + ["#{msg.backtrace.first}: #{msg.message.split("\n").first} (#{msg.class.name})", + msg.backtrace[1..-1].map { |m| "\tfrom #{m}" }].flatten + else + msg.to_s.split("\n") + end + msg.map { |m| "#{datetime} #{severity} -- #{m}\n" }.join + end + end +end -- 2.23.0

2 1

[PATCH compass-ci 2/2] sbin/quick_fetch: add an interface script for git_mirror speed up
by Li Yuanchao 23 Jan '21

23 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/quick_fetch | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 sbin/quick_fetch diff --git a/sbin/quick_fetch b/sbin/quick_fetch new file mode 100755 index 0000000..e8378ec --- /dev/null +++ b/sbin/quick_fetch @@ -0,0 +1,23 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'bunny' + +# usage: quick_fetch git_repo:$git_repo +# or: quick_fetch $url +# quick_fetch https://github.com/Siguyi/AvxToNeon +# quick_fetch git_repo:u/upstream-repos/upstream-repos +def send_url_of_repo + connection = Bunny.new('amqp://172.17.0.1:5672') + connection.start + channel = connection.create_channel + + queue = channel.queue('web_hook') + message = ARGV[0] + queue.publish(message) + connection.close +end + +send_url_of_repo -- 2.23.0

1 0

[PATCH compass-ci 1/2] git_mirror.rb: add one more parameter choice when use webhook
by Li Yuanchao 23 Jan '21

23 Jan '21

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 6d1624f..a111e51 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -318,6 +318,8 @@ class MirrorMain # url: https://github.com/berkeley-abc/abc git_repo: a/abc/abc # url: https://github.com/Siguyi/AvxToNeon git_repo: a/AvxToNeon/Siguyi def get_git_repo(webhook_url) + return webhook_url.split(':')[1].gsub(' ', '') if webhook_url =~ /^(git_repo:)/ + fork_name, project = webhook_url.split('/')[-2, 2] git_repo = "#{project[0].downcase}/#{project}/#{fork_name}" -- 2.23.0

1 0

[PATCH v3 compass-ci 2/3] bvirt/lib: introduce mylog.rb
by Xiao Shenwei 23 Jan '21

23 Jan '21

[why] records run logs 1. easy debugging 2. the log will be upload to result_root in the fulture [how] custom format of log like this: 2021-01-22 10:11:15 +0800 INFO -- SCHED_HOST: 3000 in this way, user can perceived message when error occurred Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/mylog.rb | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 providers/libvirt/lib/mylog.rb diff --git a/providers/libvirt/lib/mylog.rb b/providers/libvirt/lib/mylog.rb new file mode 100644 index 0000000..38f8c98 --- /dev/null +++ b/providers/libvirt/lib/mylog.rb @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' + +# define a logger format +class Mylog < Logger + def initialize(filename) + File.delete(filename) if File.exist?(filename) + super(filename) + format + end + + private + + def format + self.datetime_format = '%Y-%m-%d %H:%M:%s' + self.formatter = proc do |severity, datetime, _progname, msg| + msg = if msg.is_a? Exception + ["#{msg.backtrace.first}: #{msg.message.split("\n").first} (#{msg.class.name})", + msg.backtrace[1..-1].map { |m| "\tfrom #{m}" }].flatten + else + msg.to_s.split("\n") + end + msg.map { |m| "#{datetime} #{severity} -- #{m}\n" }.join + end + end +end -- 2.23.0

2 2

[PATCH compass-ci] sparrow/README.md: improve readability.
by Li Ping 23 Jan '21

23 Jan '21

[why] Markdown does not translate every line break character in a paragraph into a <br/> tag. ref: http://markdown.cn/ obviously, in some paragraphs of this markdown document, the newline effect is not shown, content would be displayed as a single line, readability would be reduced. --- sparrow/README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sparrow/README.md b/sparrow/README.md index c7450fd..1906f10 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -72,8 +72,10 @@ ``` 执行上述命令正常情况下会提示信息如下: + ``` submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Error resolving real path of '/srv/os/openeuler/aarch64/20.03/boot/vmlinuz': No such file or directory + ``` compass-ci搭建完毕，执行步骤4下载所需要的rootfs文件就可以开始进行测试了。 4. 下载rootfs文件（根据所需要的rootfs在[该目录](http://124.90.34.227:11300/os/)下获取对应版本的cgz文件） -- 2.23.0

1 0

[PATCH v3 compass-ci 3/3] libvirt/lib: request scheduler to consume job
by Xiao Shenwei 23 Jan '21

23 Jan '21

request url: http://#{@sched_host}:#{@sched_port}/boot.libvirt/mac/#{@info['mac']} Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/consumer.rb | 85 +++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) create mode 100644 providers/libvirt/lib/consumer.rb diff --git a/providers/libvirt/lib/consumer.rb b/providers/libvirt/lib/consumer.rb new file mode 100644 index 0000000..f1477fe --- /dev/null +++ b/providers/libvirt/lib/consumer.rb @@ -0,0 +1,85 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'yaml' +require 'json' +require 'rest-client' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +# register hostname mac queues +# request scheduler +class Consumer + attr_reader :info + + def initialize(hostname, queues, logger) + @logger = logger + @info = { 'hostname' => hostname, 'queues' => queues } + host_exists + parse_mac_from_hostname + defconfig + register_info + end + + def connect + res = RestClient.get("http://#{@sched_host}:#{@sched_port}/boot.libvirt/mac/#{@info['mac']}") + if res.empty? + @logger.error('Can not connect scheduler') + raise 'Can not connect scheduler' + end + JSON.parse(res) + end + + def clean_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host_mac?mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/del_host2queues?host=#{@info['hostname']}", {} + ) + end + + private + + def defconfig + names = Set.new %w[ + SCHED_HOST + SCHED_PORT + ] + defaults = relevant_defaults(names) + @sched_host = defaults['SCHED_HOST'] || '172.17.0.1' + @sched_port = defaults['SCHED_PORT'] || 3000 + @logger.info("SCHED_HOST: #{@sched_host}") + @logger.info("SCHED_PORT: #{@sched_port}") + end + + def parse_mac_from_hostname + # TODO + # use digest/md5 replace + cmd = %(echo #{@info['hostname']} | + md5sum | + sed "s/^\$..\$\$..\$\$..\$\$..\$\$..\$.*$/0a-\\1-\\2-\\3-\\4-\\5/" + ) + @info['mac'] = %x(#{cmd}).chomp + @logger.info("Mac address: #{@info['mac']}") + end + + def register_info + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host_mac?hostname=#{@info['hostname']}&mac=#{@info['mac']}", {} + ) + RestClient.put( + "http://#{@sched_host}:#{@sched_port}/set_host2queues?host=#{@info['hostname']}&queues=#{@info['queues']}", {} + ) + end + + def host_exists + @info['host'] = @info['hostname'].split('.')[0] + host_file = "#{ENV['LKP_SRC']}/hosts/#{@info['host']}" + return if FileTest.exists?(host_file) + + @logger.error("The file does't exist: #{host_file}") + raise "The file does't exist: #{host_file}" + end +end -- 2.23.0

1 0

2025

2024

2023

2022

2021

2020

Compass-ci