- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 2/2] tests/deploy-cci: mount os rootfs
by Liu Yinsi 22 Jan '21

22 Jan '21

[why] mount our server's os rootfs directory is a dedicated way for $LKP_SRC/test/deploy-cci, it is not a general way for deploy Compass CI with one click, put it in tests/deploy-cci more reasonable, not in $CCI_SRC/sparrow script. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- tests/deploy-cci | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/deploy-cci b/tests/deploy-cci index 8e6acbac4..e2a7bd320 100755 --- a/tests/deploy-cci +++ b/tests/deploy-cci @@ -35,5 +35,13 @@ deploy() ./install-tiny } +mount_rootfs() +{ + mkdir -p /srv/{os,initrd} + mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr + mount -t cifs //$LKP_SERVER/initrd /srv/initrd -o port=446,guest,ro,hard,vers=1.0,noacl,nouser_xattr +} + export_env +mount_rootfs deploy -- 2.23.0

1 0

[PATCH lkp-tests 1/2] distro/depends: fix failed to mount
by Liu Yinsi 22 Jan '21

22 Jan '21

[error] root@test-lys /c/compass-ci/container/os-cifs# mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr mount: /srv: bad option; for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program. [why] missing depends cifs-utils causes mount error Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- distro/depends/deploy-cci | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/deploy-cci b/distro/depends/deploy-cci index 5664e303b..c0d6a5310 100644 --- a/distro/depends/deploy-cci +++ b/distro/depends/deploy-cci @@ -1 +1,2 @@ git +cifs-utils -- 2.23.0

1 0

[PATCH compass-ci 2/2] lib/compare_error_messages.rb: return false when matching error filename failed
by Lin Jiaxin 22 Jan '21

22 Jan '21

If error_id = 'build-pkg.c++:fatal-error:switch'~x16~xe6~xab~xaa'does-not-start-with'-'', cannot get the error filename and return an empty hash. The empty hash is not credible too. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 2e35019..5f74be7 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -17,7 +17,9 @@ def credible?(previous_job_id, later_job_id, error_id) previous_result_file = File.join('/srv', es.query_by_id(previous_job_id)['result_root'], 'build-pkg') later_result_file = File.join('/srv', es.query_by_id(later_job_id)['result_root'], 'build-pkg') - return false if filenames_check(previous_result_file, later_result_file, error_id).value?(false) + filenames_check = filenames_check(previous_result_file, later_result_file, error_id) + + return false if filenames_check.empty? || filenames_check.value?(false) return true end -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/compare_error_messages.rb: remove filename extension when matching error filename
by Lin Jiaxin 22 Jan '21

22 Jan '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/compare_error_messages.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/compare_error_messages.rb b/lib/compare_error_messages.rb index 92533b8..2e35019 100644 --- a/lib/compare_error_messages.rb +++ b/lib/compare_error_messages.rb @@ -28,8 +28,8 @@ def filenames_check(previous_result_file, later_result_file, error_id) error_lines = ErrorMessages.new(previous_result_file).obtain_error_messages_by_error_id(error_id, true) error_lines.each do |error_line| - # "src/ssl_sock.c:1454:104: warning: unused parameter 'al' [-Wunused-parameter]" => "src/ssl_sock.c" - filenames << $1 if error_line =~ /(.*)(:\d+){2}: (error|warning):/ + # "src/ssl_sock.c:1454:104: warning: unused parameter 'al' [-Wunused-parameter]" => "src/ssl_sock" + filenames << $1.chomp(File.extname($1)) if error_line =~ /(.*)(:\d+){2}: (error|warning):/ end File.open(later_result_file).each_line do |line| -- 2.23.0

1 0

[PATCH v2 compass-ci] src/lib/job: add check runtime upper limit
by Luan Shengde 22 Jan '21

22 Jan '21

add limit for borrowing machine: runtime <= 10 days the job will break up if the value of runtime in the job yaml is greater than 10 days Signed-off-by: Luan Shengde <shdluan(a)163.com> --- src/lib/job.cr | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 9b52d88..bb9b792 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -64,6 +64,7 @@ class Job return unless @hash.has_key?("my_uuid") || @hash.has_key?("my_token") check_account_info() + check_run_time() set_sshr_info() return end @@ -410,6 +411,15 @@ class Job @hash.delete("my_token") end + private def check_run_time + return unless @hash["testcase"] == "borrow" + + max_run_time = 10 * 24 * 3600 + + error_msg = "The maximum runtime should no more than 10 days." + raise error_msg if hash["runtime"].to_i > max_run_time + end + private def is_valid_account?(account_info) return false unless account_info.is_a?(JSON::Any) -- 2.23.0

1 0

[PATCH lkp-tests] sbin/dump-stats: support dump 'message'
by Lu Weitao 22 Jan '21

22 Jan '21

[Example] { "iperf.tcp.sender.bps": [ 34804801216.197174 ], "iperf.tcp.receiver.bps": [ 34804762215.18231 ], "iperf.run.message": "error xxx" } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/dump-stat | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/sbin/dump-stat b/sbin/dump-stat index b5abf5a9b..6ebda1f29 100755 --- a/sbin/dump-stat +++ b/sbin/dump-stat @@ -61,10 +61,8 @@ while (line = STDIN.gets) end # only number is valid unless v.numeric? - invalid_records.push record_index - warn_stat "invalid stats value: #{v}", monitor - - exit 1 + result[k] = v + next end v = v.index('.') ? v.to_f : v.to_i @@ -84,6 +82,8 @@ min_cols_stat = '' max_cols_stat = '' zero_stats = [] result.each do |key, val| + next if val.is_a?(String) + if max_cols < val.size max_cols = val.size max_cols_stat = key -- 2.23.0

1 1

[PATCH v2 compass-ci 2/3] libvirt/lib: introduce mylog.rb
by Xiao Shenwei 22 Jan '21

22 Jan '21

[why] records run logs 1. easy debugging 2. the log will be upload to result_root in the fulture [how] custom format of log like this: 2021-01-22 10:11:15 +0800 INFO -- SCHED_HOST: 3000 in this way, user can perceived message when error occurred Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/lib/mylog.rb | 35 ++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 providers/libvirt/lib/mylog.rb diff --git a/providers/libvirt/lib/mylog.rb b/providers/libvirt/lib/mylog.rb new file mode 100644 index 0000000..141dd80 --- /dev/null +++ b/providers/libvirt/lib/mylog.rb @@ -0,0 +1,35 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'logger' + +# define a logger format +class Mylog < Logger + def initialize(filename) + @name = filename + rm_exist? + super(filename) + format + end + + private + + def rm_exist? + if FileTest.exist?(@name) + File.delete(@name) + end + end + + def format + self.datetime_format = '%Y-%m-%d %H:%M:%s' + self.formatter = proc do |severity, datetime, _progname, msg| + msg = if msg.is_a? Exception + ["#{msg.backtrace.first}: #{msg.message.split("\n").first} (#{msg.class.name})", msg.backtrace[1..-1].map { |m| "\tfrom #{m}" }].flatten + else + msg.to_s.split("\n") + end + msg.map { |m| "#{datetime} #{severity} -- #{m}\n" }.join + end + end +end -- 2.23.0

2 2

[PATCH lkp-tests] setup/simplify-ci: set default values
by Liu Yinsi 22 Jan '21

22 Jan '21

[why] when call simplify-ci individually to deploy environment, SCHED_HOST, SCHED_PORT, GIT_SERVER, these variables are undefined in a very clean machine, execute the script will failed, so set default values is necessary. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- setup/simplify-ci | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/setup/simplify-ci b/setup/simplify-ci index e3ee3f772..df18272f3 100755 --- a/setup/simplify-ci +++ b/setup/simplify-ci @@ -2,8 +2,18 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +: ${SCHED_HOST:=172.17.0.1} +: ${SCHED_PORT:=3000} + git_ci() { + if [ -n "$GIT_SERVER" ]; then + cat >> /etc/gitconfig <<-EOF + [url "git://$GIT_SERVER/gitee.com"] + insteadOf=https://gitee.com + EOF + fi + git clone git://$GIT_SERVER/gitee.com/wu_fengguang/compass-ci.git /c/compass-ci } -- 2.23.0

1 0

[PATCH compass-ci] src/lib/job: add check runtime upper limit
by Luan Shengde 22 Jan '21

22 Jan '21

the maximum time for borrowing machine should no more than 10 days. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- src/lib/job.cr | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 9b52d88..e9e693c 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -64,6 +64,7 @@ class Job return unless @hash.has_key?("my_uuid") || @hash.has_key?("my_token") check_account_info() + check_run_time() set_sshr_info() return end @@ -410,6 +411,19 @@ class Job @hash.delete("my_token") end + private def check_run_time + return unless @hash["testcase"] == "borrow" + + if @hash["runtime"].ends_with? "d" + hour = hash["runtime"][0..-2].to_i * 24 + elsif @hash["runtime"].ends_with? "h" + hour = hash["runtime"][0..-2].to_i + end + + error_msg = "The maximum runtime should no more than 10 days." + raise error_msg if hour.is_a?(Int32) && hour >= 240 + end + private def is_valid_account?(account_info) return false unless account_info.is_a?(JSON::Any) -- 2.23.0

1 0

[PATCH compass-ci] lib/matrix2.rb support merge 'message' to stats
by Lu Weitao 22 Jan '21

22 Jan '21

[Example] { "iperf.tcp.sender.bps": 34804801216.197174, "iperf.tcp.receiver.bps": 34804762215.18231, "iperf.iperf.run.message": "error xxx" } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/matrix2.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 8444334..5920d62 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -10,6 +10,8 @@ require "#{LKP_SRC}/lib/matrix" require_relative './params_group.rb' def set_pre_value(item, value, sample_size) + return value if value.is_a?(String) + if value.size == 1 value[0] elsif independent_counter? item -- 2.23.0

1 0