August 2020 - Crystal-ci - mailweb.openeuler.org

[PATCH v4 lkp-tests] lib/job-init.sh: choose result service by $os_mount
by Hu Xuejiao 21 Aug '20

21 Aug '20

1. if no $RESULT_SERVER, default to $LKP_SERVER. 2. result service according to $os_mount, if no $os_mount, default original judge. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/job-init.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/job-init.sh b/lib/job-init.sh index 0732d4b9..42cfc2bd 100755 --- a/lib/job-init.sh +++ b/lib/job-init.sh @@ -51,9 +51,14 @@ setup_result_service() { [ -n "$result_service" ] && return [ -n "$NO_NETWORK" ] && return 1 + [ -n "$RESULT_SERVER" ] || $RESULT_SERVER=$LKP_SERVER + + if [ "$os_mount" == 'nfs' ] || [ -z "$os_mount" ]; then + supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return + elif [ "$os_mount" == 'cifs' ] || [ -z "$os_mount" ]; then + supports_netfs 'cifs' && result_service=//$RESULT_SERVER/result && return + fi - supports_netfs 'nfs' && result_service=$LKP_SERVER:/result && return - supports_netfs 'cifs' && result_service=//$LKP_SERVER/result && return supports_raw_upload && result_service=raw_upload && return return 1 -- 2.23.0

2 1

Re: [PATCH v4 lkp-tests] lib/job-init.sh: choose result service by $os_mount
by Liu Shaofei 21 Aug '20

21 Aug '20

the title look not good, perhaps you should use the words "choose different service by os_mount". Thanks, Liushaofei

2 1

[PATCH v4 todo] todo/bin: fix git-count error
by Bai Jing 21 Aug '20

21 Aug '20

problem: sh: line 0: cd: /home/baijing/.am/todo: No such file or directory Signed-off-by: Bai Jing <baijing14(a)huawei.com> --- bin/git-count | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/bin/git-count b/bin/git-count index 2b1359b..21fd944 100755 --- a/bin/git-count +++ b/bin/git-count @@ -56,9 +56,12 @@ end count_by_name.sort_by { |k, _v| k } +%x(mkdir -p #{ENV['HOME']}/.am) unless File.exist?("#{ENV['HOME']}/.am") repos.each do |repo, files| - %x(git -C ~/.am/#{repo} pull --rebase) - %x(cd ~/.am/#{repo} && git shortlog -ens --no-merges --since "#{days} days ago" -- #{files}).each_line do |line| + %x(cd #{ENV['HOME']}/.am/ && git clone file:///c/#{repo}.git) unless File.exist?("#{ENV['HOME']}/.am/#{repo}") + %x(git -C ~/.am/#{repo} pull --rebase 2>&1) + file = %x(cd ~/.am/#{repo} && git shortlog -ens --no-merges --since "#{days} days ago" -- #{files}) + file.each_line do |line| if line =~ /(\d+)\t(.*) <(.*)>/ count = $1.to_i name = $2 -- 2.23.0

2 2

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Wu Fengguang 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 10:16:52PM +0800, Lu Kaiyi wrote: >On Thu, Aug 20, 2020 at 07:45:33PM +0800, Wu Fengguang wrote: >>Kaiyi, 你做这么多转换, 用了之前我提到的两个工具了吧? >> > >To tell the truth, still not. but there's no much syntax difference between >ruby and crystal. 手工活尽量避免~ 这是一种态度. Thanks, Fengguang

1 0

Re: [PATCH lkp-tests 2/3] delete the redundant /
by Liu Shaofei 21 Aug '20

21 Aug '20

the title look not good, maybe can change "delete extra character". Thanks, Liushaofei

1 0

Re: [PATCH distro-info] get tag_prefix in a more reasonable way
by Du Kaitian 21 Aug '20

21 Aug '20

On Tue, Aug 18, 2020 at 12:20:47PM +0800, Li Ping wrote: >On Tue, Aug 18, 2020 at 10:02:45AM +0800, Wu Fengguang wrote: >>CC Shaofei. > >I have already CC All. > >> >>On Mon, Aug 17, 2020 at 07:32:45PM +0800, Li Ping wrote: >>>Previously, the tag_prefix was the first matched tag_prefix in git_tag. >>>It is easy to make errors >>> >>>So, I get ten matched tag_prefix in git_tag and choose the one that >>>appears the most >>>--- >>> archlinux/parse-git_tag.rb | 21 +++++++++++++++------ >>> 1 file changed, 15 insertions(+), 6 deletions(-) >>> >>>diff --git a/archlinux/parse-git_tag.rb b/archlinux/parse-git_tag.rb >>>index 85ab41e..22054ea 100755 >>>--- a/archlinux/parse-git_tag.rb >>>+++ b/archlinux/parse-git_tag.rb >>>@@ -35,15 +35,24 @@ rescue StandardError >>> end >>> >>> def write_config(config) >>>+ git_tags = {} >>>+ git_tags.default = ["", 0] >>>+ i = 0 >>> config['git_tag'].each do |tag| >>> next unless tag.to_s =~ /^([-_.a-zA-Z]*)\d+([._]?)/ >>>- >>>- config['tag_prefix'] = ("^#{Regexp.last_match(1)}" if Regexp.last_match(1)) >>>- config['separator'] = Regexp.last_match(2) >>>- File.open(file, 'w') do |f| >>>- f.write(config.to_yaml) >> >>swap the below 2 lines, to >>>+ git_tags["^#{Regexp.last_match(1)}"][0] = Regexp.last_match(2) >>>+ git_tags["^#{Regexp.last_match(1)}"][1] += 1 >> >> Move i += 1 here, like this > >Thank you for your reminder > >>>+ i += 1 >>>+ break if i > 9 >> >>>+ end >>>+ git_tags.each do |t, n| >>>+ if n[1] > git_tags[config['tag_prefix']][1] >> >>config['tag_prefix'] may be nil > >OK, I will add a code: >config['tag_prefix'] = "^#{Regexp.last_match(1)}" > >Thanks, >Ping 这个语句能够确保 config['tag_prefix'] 不为空么 >> >>>+ config['tag_prefix'] = t >>>+ config['separator'] = git_tags[t][0] >>> end >>>- break >>>+ end >>>+ File.open(file, 'w') do |f| >>>+ f.write(config.to_yaml) >>> end >>> end 这个想法很好，将 File.open 放到循环外面. 谢谢 jimmy >>> >>>-- >>>2.23.0 >>>

1 0

[PATCH crystal-ci] display colorful comparing results
by Lu Weitao 21 Aug '20

21 Aug '20

sample: sbin/compare "id=15104" "id=15409" --color 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 35110937.00 +2.0% 35854560.00 atomic.score 1.00 [93m+200.0% [0m 3.00 atomic.threads 11.14 +3.3% 11.51 atomic.time.elapsed_time 11.14 +3.3% 11.51 atomic.time.elapsed_time.max 22.00 [93m+27.0% [0m 28.00 atomic.time.involuntary_context_switches 4288.00 +1.0% 4352.00 atomic.time.maximum_resident_set_size 668.00 +0.0% 672.00 atomic.time.minor_page_faults 0.00 0 0.03 atomic.time.system_time 94.00 +10.0% 104.00 atomic.time.voluntary_context_switches 11.35 [93m+19.1% [0m 13.52 boot-time.boot 14.90 +14.8% 17.10 boot-time.idle 6990.00 [93m+56.0% [0m 10971.00 interrupts.2:GICv3.27.Level.arch_timer 463.00 [93m+313.0% [0m 1913.00 interrupts.38:GICv3.36.Level.virtio0 138.00 -3.0% 135.00 interrupts.45:ITS-MSI.32769.Edge.virtio1-control 3613.00 [93m+51.0% [0m 5473.00 interrupts.CPU0.2:GICv3.27.Level.arch_timer 138.00 -3.0% 135.00 interrupts.CPU0.45:ITS-MSI.32769.Edge.virtio1-control 1834.00 [93m+61.0% [0m 2968.00 interrupts.CPU0.IPI0:Rescheduling_interrupts 24.00 [91m-34.0% [0m 16.00 interrupts.CPU0.IPI1:Function_call_interrupts 3377.00 [93m+62.0% [0m 5498.00 interrupts.CPU1.2:GICv3.27.Level.arch_timer 463.00 [93m+313.0% [0m 1913.00 interrupts.CPU1.38:GICv3.36.Level.virtio0 1858.00 -8.0% 1723.00 interrupts.CPU1.IPI0:Rescheduling_interrupts 28.00 +7.0% 30.00 interrupts.CPU1.IPI1:Function_call_interrupts 3692.00 [93m+27.0% [0m 4691.00 interrupts.IPI0:Rescheduling_interrupts 52.00 -12.0% 46.00 interrupts.IPI1:Function_call_interrupts 375264.00 +8.0% 408058.00 meminfo.Active ... Signed-off-by: Lu Weitao <luweitao2(a)huawei.com> --- lib/compare.rb | 8 ++++---- lib/compare_matrixes.rb | 7 +++---- sbin/compare | 18 ++++++++++++++++-- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 4ef21b8..48367d9 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -16,10 +16,10 @@ require_relative './constants.rb' # - multiple conditions: "os=centos,debian suite=iperf,atomic" # -def compare_matrices_list(argv, common_conditions) +def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list) + compare_matrixes(matrices_list, options: options) end def parse_argv(argv, common_conditions) @@ -53,11 +53,11 @@ end # - multiple dimensions: "os os_version ..." # -def compare_group(argv, dimensions) +def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices) + compare_group_matrices(groups_matrices, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 9068073..b892826 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -187,7 +187,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_key = nil, options = {}) +def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # # @matrixes_list: list consisting of matrix @@ -199,7 +199,6 @@ def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_ warn 'Matrix cannot be empty!' return end - options = { 'perf-profile': 5, theme: :none }.merge(options) matrixes_values = get_matrixes_values(matrixes_list, options) remove_unchanged_field(matrixes_values) @@ -375,7 +374,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices) +def compare_group_matrices(group_matrices, options) group_matrices.each do |k, v| matrices_list = [] matrices_titles = [] @@ -384,7 +383,7 @@ def compare_group_matrices(group_matrices) matrices_list << matrix end - compare_matrixes(matrices_list, matrices_titles, k) + compare_matrixes(matrices_list, matrices_titles, k, options: options) end end diff --git a/sbin/compare b/sbin/compare index bc63ace..c171894 100755 --- a/sbin/compare +++ b/sbin/compare @@ -10,6 +10,7 @@ # compare "commit=a12d232e" "commit=b3bacc31" # compare "os=debian" "os=centos" -c "suite=iperf" # compare "os=centos" -d "os_version os_arch" +# compare "os=centos" -d "os_version os_arch" --color -t light require 'optparse' require_relative '../lib/compare.rb' @@ -17,6 +18,9 @@ require_relative '../lib/compare.rb' common_conditions = '' is_group = false dimensions = nil +theme = :classic +color = false +options = {} opt_parser = OptionParser.new do |opts| opts.banner = 'Usage: compare "conditions" ... [option]' @@ -36,6 +40,14 @@ opt_parser = OptionParser.new do |opts| is_group = true end + opts.on('-t', '--theme theme', 'theme to compare: classic/focus_good/focus_bad/striking/light') do |t| + theme = t + end + + opts.on('--color', 'turn on colorful display with current theme') do + color = true + end + opts.on_tail('-h', '--help', 'show this message') do puts opts exit @@ -49,8 +61,10 @@ argv = if ARGV == [] end opt_parser.parse!(argv) +options = { theme: theme } if color + if is_group - compare_group(argv, dimensions) + compare_group(argv, dimensions, options) else - compare_matrices_list(argv, common_conditions) + compare_matrices_list(argv, common_conditions, options) end -- 2.23.0

2 2

[PATCH lkp-tests 2/3] delete the redundant /
by Zhou Shengrui 21 Aug '20

21 Aug '20

--- stats/crystal/mysql-server.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr index dae06a35..82956c3d 100755 --- a/stats/crystal/mysql-server.cr +++ b/stats/crystal/mysql-server.cr @@ -1,7 +1,7 @@ #!/usr/bin/env crystal # frozen_string_literal: true -require "../..//lib/string_ext" +require "../../lib/string_ext" results = [] of String -- 2.23.0

1 0

[PATCH lkp-tests 1/3] mysql-server with crystal
by Zhou Shengrui 21 Aug '20

21 Aug '20

--- stats/crystal/mysql-server.cr | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100755 stats/crystal/mysql-server.cr diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr new file mode 100755 index 00000000..dae06a35 --- /dev/null +++ b/stats/crystal/mysql-server.cr @@ -0,0 +1,26 @@ +#!/usr/bin/env crystal +# frozen_string_literal: true + +require "../..//lib/string_ext" + +results = [] of String + +while (line = STDIN.gets) + line = line.remediate_invalid_byte_sequence(replace: "_") unless line.valid_encoding? + line = line.to_s + + case line + # The output is as below: + # 200706 14:35:48 [ 99%] main.subquery_sj_innodb_all w8 [ pass ] 140 + # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 + # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 + when /\[\s*\d+\%\]/ + results << line.split("]")[1].split[0] + ".pass: 1" if line.includes?("[ pass ]") + results << line.split("]")[1].split[0] + ".fail: 1" if line.includes?("[ fail ]") + results << line.split("]")[1].split[0] + ".skip: 1" if line.includes?("[ skipped ]") + end +end + +results.each do |item| + puts item +end -- 2.23.0

1 0

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Lu Kaiyi 21 Aug '20

21 Aug '20

On Fri, Aug 21, 2020 at 02:26:38PM +0800, Zhou Shengrui wrote: >On Thu, Aug 20, 2020 at 07:04:40PM +0800, Lu Kaiyi wrote: >>fix syntax problem in line with crystal >> >>1. single-quotes to double-quotes >>2. $PROGRAM_NAME to PROGRAM_NAME >>3. stats = [] to stats = [] of String >>4. add type = "" before using >> >>Signed-off-by: Lu Kaiyi <lukaiyi(a)huawei.com> >>--- >> stats/crystal/test-bpf.cr | 39 +++++++++++++++++++-------------------- >> 1 file changed, 19 insertions(+), 20 deletions(-) >> >>diff --git a/stats/crystal/test-bpf.cr b/stats/crystal/test-bpf.cr >>index f5b13a09..3941a110 100644 >>--- a/stats/crystal/test-bpf.cr >>+++ b/stats/crystal/test-bpf.cr >>@@ -1,46 +1,45 @@ >>-#!/usr/bin/env ruby >>+#!/usr/bin/env crystal >> >>-LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) >>+LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.real_path(PROGRAM_NAME))) >> >>-require "#{LKP_SRC}/lib/string_ext" >>+require "../../lib/string_ext" >> >>-stats = [] >>+stats = [] of String >> success_test = 0 >> fail_test = 0 >> is_divided = false >> >> while (line = STDIN.gets) >>- line = line.remediate_invalid_byte_sequence(replace: '_') unless line.valid_encoding? >>- >> case line >> when /test_bpf: #[0-9]+ (.+) jited:.+ PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) jited:.+ FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".pass: 1" >> when /test_bpf: #[0-9]+ check: (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".fail: 1" >> when /test_bpf: #[0-9]+ (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+)/ >>- type = 'check:_' + $1.strip.tr(' ', '_') >>+ type = "check:_" + $1.strip.tr(" ", "_") >> is_divided = true >> when /test_bpf: #[0-9]+ (.+)/ >>- type = $1.strip.tr(' ', '_') >>+ type = $1.strip.tr(" ", "_") >> is_divided = true > >"type" should not be null here,the content of "type" will be lost. >I think you can put it at the beginning of the program. > >Thanks >Shengrui > >> when /test_bpf: Summary: ([0-9]+) PASSED, ([0-9]+) FAILED, \[.+ JIT'ed\]$/ >> success_test = $1 >> fail_test = $2 >> when /(PASS|FAIL)/ >>+ type = "" >> if is_divided >> stats << type + ".#{$1.downcase}: 1" >> is_divided = false >>-- >>2.23.0 >>

1 0