- Crystal-ci - mailweb.openeuler.org

Re: command
by Wu Fengguang 21 Aug '20

21 Aug '20

On Fri, Aug 21, 2020 at 10:17:00AM +0800, Luan Shengde wrote: >On Fri, Aug 21, 2020 at 09:46:17AM +0800, Wu Fengguang wrote: >>As of version 18.03, you can use host.docker.internal as the host's IP. >> >>https://stackoverflow.com/questions/22944631/how-to-get-the-ip-address-of-t… > >excelent! >I will try it. >It seems that we need to add line in /etc/hosts: >host_id host.docker.internal which /etc/hosts? I just tried debian docker, the DNS is not available: wfg@debian ~% ping host.docker.internal ping: host.docker.internal: Name or service not known Thanks, Fengguang

2 2

[PATCH todo] todo: add 20200826-3D:load-default-config-when-submit-job.md
by Wei Jihui 21 Aug '20

21 Aug '20

Signed-off-by: Wei Jihui <weijihui2(a)huawei.com> --- ...-3D:load-default-config-when-submit-job.md | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 people/WeiJihui/todo/20200826-3D:load-default-config-when-submit-job.md diff --git a/people/WeiJihui/todo/20200826-3D:load-default-config-when-submit-job.md b/people/WeiJihui/todo/20200826-3D:load-default-config-when-submit-job.md new file mode 100644 index 0000000..71712b7 --- /dev/null +++ b/people/WeiJihui/todo/20200826-3D:load-default-config-when-submit-job.md @@ -0,0 +1,20 @@ +######## +# 1. Background +We has three places to place ours jobs's params when submit job. +- job.yaml +- /etc/crystal-ci/defaults/*.yaml +- #{ENV['HOME']}/.config/crystal-ci/defaults/*.yaml + +But same params must be place in job.yaml, or it will be raise error, +such as "testbox", it is because programs does not load last two yaml files +at the first. + +# 2. Solution +It should be change the load order, load all yaml files first. +order: +- load_default: + - load job yaml + - load /etc/crystal-ci/defaults/*.yaml + - load #{ENV['HOME']}/.config/crystal-ci/defaults/*.yaml +-load_hosts_config + -- 2.23.0

1 0

[PATCH v4 lkp-tests] lib/job-init.sh: choose result service by $os_mount
by Hu Xuejiao 21 Aug '20

21 Aug '20

1. if no $RESULT_SERVER, default to $LKP_SERVER. 2. result service according to $os_mount, if no $os_mount, default original judge. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/job-init.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/job-init.sh b/lib/job-init.sh index 0732d4b9..42cfc2bd 100755 --- a/lib/job-init.sh +++ b/lib/job-init.sh @@ -51,9 +51,14 @@ setup_result_service() { [ -n "$result_service" ] && return [ -n "$NO_NETWORK" ] && return 1 + [ -n "$RESULT_SERVER" ] || $RESULT_SERVER=$LKP_SERVER + + if [ "$os_mount" == 'nfs' ] || [ -z "$os_mount" ]; then + supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return + elif [ "$os_mount" == 'cifs' ] || [ -z "$os_mount" ]; then + supports_netfs 'cifs' && result_service=//$RESULT_SERVER/result && return + fi - supports_netfs 'nfs' && result_service=$LKP_SERVER:/result && return - supports_netfs 'cifs' && result_service=//$LKP_SERVER/result && return supports_raw_upload && result_service=raw_upload && return return 1 -- 2.23.0

2 1

Re: [PATCH v4 lkp-tests] lib/job-init.sh: choose result service by $os_mount
by Liu Shaofei 21 Aug '20

21 Aug '20

the title look not good, perhaps you should use the words "choose different service by os_mount". Thanks, Liushaofei

2 1

[PATCH v4 todo] todo/bin: fix git-count error
by Bai Jing 21 Aug '20

21 Aug '20

problem: sh: line 0: cd: /home/baijing/.am/todo: No such file or directory Signed-off-by: Bai Jing <baijing14(a)huawei.com> --- bin/git-count | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/bin/git-count b/bin/git-count index 2b1359b..21fd944 100755 --- a/bin/git-count +++ b/bin/git-count @@ -56,9 +56,12 @@ end count_by_name.sort_by { |k, _v| k } +%x(mkdir -p #{ENV['HOME']}/.am) unless File.exist?("#{ENV['HOME']}/.am") repos.each do |repo, files| - %x(git -C ~/.am/#{repo} pull --rebase) - %x(cd ~/.am/#{repo} && git shortlog -ens --no-merges --since "#{days} days ago" -- #{files}).each_line do |line| + %x(cd #{ENV['HOME']}/.am/ && git clone file:///c/#{repo}.git) unless File.exist?("#{ENV['HOME']}/.am/#{repo}") + %x(git -C ~/.am/#{repo} pull --rebase 2>&1) + file = %x(cd ~/.am/#{repo} && git shortlog -ens --no-merges --since "#{days} days ago" -- #{files}) + file.each_line do |line| if line =~ /(\d+)\t(.*) <(.*)>/ count = $1.to_i name = $2 -- 2.23.0

2 2

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Wu Fengguang 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 10:16:52PM +0800, Lu Kaiyi wrote: >On Thu, Aug 20, 2020 at 07:45:33PM +0800, Wu Fengguang wrote: >>Kaiyi, 你做这么多转换, 用了之前我提到的两个工具了吧? >> > >To tell the truth, still not. but there's no much syntax difference between >ruby and crystal. 手工活尽量避免~ 这是一种态度. Thanks, Fengguang

1 0

Re: [PATCH lkp-tests 2/3] delete the redundant /
by Liu Shaofei 21 Aug '20

21 Aug '20

the title look not good, maybe can change "delete extra character". Thanks, Liushaofei

1 0

Re: [PATCH distro-info] get tag_prefix in a more reasonable way
by Du Kaitian 21 Aug '20

21 Aug '20

On Tue, Aug 18, 2020 at 12:20:47PM +0800, Li Ping wrote: >On Tue, Aug 18, 2020 at 10:02:45AM +0800, Wu Fengguang wrote: >>CC Shaofei. > >I have already CC All. > >> >>On Mon, Aug 17, 2020 at 07:32:45PM +0800, Li Ping wrote: >>>Previously, the tag_prefix was the first matched tag_prefix in git_tag. >>>It is easy to make errors >>> >>>So, I get ten matched tag_prefix in git_tag and choose the one that >>>appears the most >>>--- >>> archlinux/parse-git_tag.rb | 21 +++++++++++++++------ >>> 1 file changed, 15 insertions(+), 6 deletions(-) >>> >>>diff --git a/archlinux/parse-git_tag.rb b/archlinux/parse-git_tag.rb >>>index 85ab41e..22054ea 100755 >>>--- a/archlinux/parse-git_tag.rb >>>+++ b/archlinux/parse-git_tag.rb >>>@@ -35,15 +35,24 @@ rescue StandardError >>> end >>> >>> def write_config(config) >>>+ git_tags = {} >>>+ git_tags.default = ["", 0] >>>+ i = 0 >>> config['git_tag'].each do |tag| >>> next unless tag.to_s =~ /^([-_.a-zA-Z]*)\d+([._]?)/ >>>- >>>- config['tag_prefix'] = ("^#{Regexp.last_match(1)}" if Regexp.last_match(1)) >>>- config['separator'] = Regexp.last_match(2) >>>- File.open(file, 'w') do |f| >>>- f.write(config.to_yaml) >> >>swap the below 2 lines, to >>>+ git_tags["^#{Regexp.last_match(1)}"][0] = Regexp.last_match(2) >>>+ git_tags["^#{Regexp.last_match(1)}"][1] += 1 >> >> Move i += 1 here, like this > >Thank you for your reminder > >>>+ i += 1 >>>+ break if i > 9 >> >>>+ end >>>+ git_tags.each do |t, n| >>>+ if n[1] > git_tags[config['tag_prefix']][1] >> >>config['tag_prefix'] may be nil > >OK, I will add a code: >config['tag_prefix'] = "^#{Regexp.last_match(1)}" > >Thanks, >Ping 这个语句能够确保 config['tag_prefix'] 不为空么 >> >>>+ config['tag_prefix'] = t >>>+ config['separator'] = git_tags[t][0] >>> end >>>- break >>>+ end >>>+ File.open(file, 'w') do |f| >>>+ f.write(config.to_yaml) >>> end >>> end 这个想法很好，将 File.open 放到循环外面. 谢谢 jimmy >>> >>>-- >>>2.23.0 >>>

1 0

[PATCH crystal-ci] display colorful comparing results
by Lu Weitao 21 Aug '20

21 Aug '20

sample: sbin/compare "id=15104" "id=15409" --color 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 35110937.00 +2.0% 35854560.00 atomic.score 1.00 [93m+200.0% [0m 3.00 atomic.threads 11.14 +3.3% 11.51 atomic.time.elapsed_time 11.14 +3.3% 11.51 atomic.time.elapsed_time.max 22.00 [93m+27.0% [0m 28.00 atomic.time.involuntary_context_switches 4288.00 +1.0% 4352.00 atomic.time.maximum_resident_set_size 668.00 +0.0% 672.00 atomic.time.minor_page_faults 0.00 0 0.03 atomic.time.system_time 94.00 +10.0% 104.00 atomic.time.voluntary_context_switches 11.35 [93m+19.1% [0m 13.52 boot-time.boot 14.90 +14.8% 17.10 boot-time.idle 6990.00 [93m+56.0% [0m 10971.00 interrupts.2:GICv3.27.Level.arch_timer 463.00 [93m+313.0% [0m 1913.00 interrupts.38:GICv3.36.Level.virtio0 138.00 -3.0% 135.00 interrupts.45:ITS-MSI.32769.Edge.virtio1-control 3613.00 [93m+51.0% [0m 5473.00 interrupts.CPU0.2:GICv3.27.Level.arch_timer 138.00 -3.0% 135.00 interrupts.CPU0.45:ITS-MSI.32769.Edge.virtio1-control 1834.00 [93m+61.0% [0m 2968.00 interrupts.CPU0.IPI0:Rescheduling_interrupts 24.00 [91m-34.0% [0m 16.00 interrupts.CPU0.IPI1:Function_call_interrupts 3377.00 [93m+62.0% [0m 5498.00 interrupts.CPU1.2:GICv3.27.Level.arch_timer 463.00 [93m+313.0% [0m 1913.00 interrupts.CPU1.38:GICv3.36.Level.virtio0 1858.00 -8.0% 1723.00 interrupts.CPU1.IPI0:Rescheduling_interrupts 28.00 +7.0% 30.00 interrupts.CPU1.IPI1:Function_call_interrupts 3692.00 [93m+27.0% [0m 4691.00 interrupts.IPI0:Rescheduling_interrupts 52.00 -12.0% 46.00 interrupts.IPI1:Function_call_interrupts 375264.00 +8.0% 408058.00 meminfo.Active ... Signed-off-by: Lu Weitao <luweitao2(a)huawei.com> --- lib/compare.rb | 8 ++++---- lib/compare_matrixes.rb | 7 +++---- sbin/compare | 18 ++++++++++++++++-- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 4ef21b8..48367d9 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -16,10 +16,10 @@ require_relative './constants.rb' # - multiple conditions: "os=centos,debian suite=iperf,atomic" # -def compare_matrices_list(argv, common_conditions) +def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list) + compare_matrixes(matrices_list, options: options) end def parse_argv(argv, common_conditions) @@ -53,11 +53,11 @@ end # - multiple dimensions: "os os_version ..." # -def compare_group(argv, dimensions) +def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices) + compare_group_matrices(groups_matrices, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 9068073..b892826 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -187,7 +187,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_key = nil, options = {}) +def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # # @matrixes_list: list consisting of matrix @@ -199,7 +199,6 @@ def compare_matrixes(matrixes_list, matrixes_titles = matrixes_list.size, group_ warn 'Matrix cannot be empty!' return end - options = { 'perf-profile': 5, theme: :none }.merge(options) matrixes_values = get_matrixes_values(matrixes_list, options) remove_unchanged_field(matrixes_values) @@ -375,7 +374,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices) +def compare_group_matrices(group_matrices, options) group_matrices.each do |k, v| matrices_list = [] matrices_titles = [] @@ -384,7 +383,7 @@ def compare_group_matrices(group_matrices) matrices_list << matrix end - compare_matrixes(matrices_list, matrices_titles, k) + compare_matrixes(matrices_list, matrices_titles, k, options: options) end end diff --git a/sbin/compare b/sbin/compare index bc63ace..c171894 100755 --- a/sbin/compare +++ b/sbin/compare @@ -10,6 +10,7 @@ # compare "commit=a12d232e" "commit=b3bacc31" # compare "os=debian" "os=centos" -c "suite=iperf" # compare "os=centos" -d "os_version os_arch" +# compare "os=centos" -d "os_version os_arch" --color -t light require 'optparse' require_relative '../lib/compare.rb' @@ -17,6 +18,9 @@ require_relative '../lib/compare.rb' common_conditions = '' is_group = false dimensions = nil +theme = :classic +color = false +options = {} opt_parser = OptionParser.new do |opts| opts.banner = 'Usage: compare "conditions" ... [option]' @@ -36,6 +40,14 @@ opt_parser = OptionParser.new do |opts| is_group = true end + opts.on('-t', '--theme theme', 'theme to compare: classic/focus_good/focus_bad/striking/light') do |t| + theme = t + end + + opts.on('--color', 'turn on colorful display with current theme') do + color = true + end + opts.on_tail('-h', '--help', 'show this message') do puts opts exit @@ -49,8 +61,10 @@ argv = if ARGV == [] end opt_parser.parse!(argv) +options = { theme: theme } if color + if is_group - compare_group(argv, dimensions) + compare_group(argv, dimensions, options) else - compare_matrices_list(argv, common_conditions) + compare_matrices_list(argv, common_conditions, options) end -- 2.23.0

2 2

[PATCH lkp-tests 2/3] delete the redundant /
by Zhou Shengrui 21 Aug '20

21 Aug '20

--- stats/crystal/mysql-server.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr index dae06a35..82956c3d 100755 --- a/stats/crystal/mysql-server.cr +++ b/stats/crystal/mysql-server.cr @@ -1,7 +1,7 @@ #!/usr/bin/env crystal # frozen_string_literal: true -require "../..//lib/string_ext" +require "../../lib/string_ext" results = [] of String -- 2.23.0

1 0

[PATCH lkp-tests 1/3] mysql-server with crystal
by Zhou Shengrui 21 Aug '20

21 Aug '20

--- stats/crystal/mysql-server.cr | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100755 stats/crystal/mysql-server.cr diff --git a/stats/crystal/mysql-server.cr b/stats/crystal/mysql-server.cr new file mode 100755 index 00000000..dae06a35 --- /dev/null +++ b/stats/crystal/mysql-server.cr @@ -0,0 +1,26 @@ +#!/usr/bin/env crystal +# frozen_string_literal: true + +require "../..//lib/string_ext" + +results = [] of String + +while (line = STDIN.gets) + line = line.remediate_invalid_byte_sequence(replace: "_") unless line.valid_encoding? + line = line.to_s + + case line + # The output is as below: + # 200706 14:35:48 [ 99%] main.subquery_sj_innodb_all w8 [ pass ] 140 + # 200706 14:35:52 [ 99%] main.ssl_dynamic_persisted w1 [ fail ] 5953 + # 200706 14:35:59 [ 99%] main.mysql_upgrade_grant w7 [ skipped ] 80805 + when /\[\s*\d+\%\]/ + results << line.split("]")[1].split[0] + ".pass: 1" if line.includes?("[ pass ]") + results << line.split("]")[1].split[0] + ".fail: 1" if line.includes?("[ fail ]") + results << line.split("]")[1].split[0] + ".skip: 1" if line.includes?("[ skipped ]") + end +end + +results.each do |item| + puts item +end -- 2.23.0

1 0

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Lu Kaiyi 21 Aug '20

21 Aug '20

On Fri, Aug 21, 2020 at 02:26:38PM +0800, Zhou Shengrui wrote: >On Thu, Aug 20, 2020 at 07:04:40PM +0800, Lu Kaiyi wrote: >>fix syntax problem in line with crystal >> >>1. single-quotes to double-quotes >>2. $PROGRAM_NAME to PROGRAM_NAME >>3. stats = [] to stats = [] of String >>4. add type = "" before using >> >>Signed-off-by: Lu Kaiyi <lukaiyi(a)huawei.com> >>--- >> stats/crystal/test-bpf.cr | 39 +++++++++++++++++++-------------------- >> 1 file changed, 19 insertions(+), 20 deletions(-) >> >>diff --git a/stats/crystal/test-bpf.cr b/stats/crystal/test-bpf.cr >>index f5b13a09..3941a110 100644 >>--- a/stats/crystal/test-bpf.cr >>+++ b/stats/crystal/test-bpf.cr >>@@ -1,46 +1,45 @@ >>-#!/usr/bin/env ruby >>+#!/usr/bin/env crystal >> >>-LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) >>+LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.real_path(PROGRAM_NAME))) >> >>-require "#{LKP_SRC}/lib/string_ext" >>+require "../../lib/string_ext" >> >>-stats = [] >>+stats = [] of String >> success_test = 0 >> fail_test = 0 >> is_divided = false >> >> while (line = STDIN.gets) >>- line = line.remediate_invalid_byte_sequence(replace: '_') unless line.valid_encoding? >>- >> case line >> when /test_bpf: #[0-9]+ (.+) jited:.+ PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) jited:.+ FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".pass: 1" >> when /test_bpf: #[0-9]+ check: (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".fail: 1" >> when /test_bpf: #[0-9]+ (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+)/ >>- type = 'check:_' + $1.strip.tr(' ', '_') >>+ type = "check:_" + $1.strip.tr(" ", "_") >> is_divided = true >> when /test_bpf: #[0-9]+ (.+)/ >>- type = $1.strip.tr(' ', '_') >>+ type = $1.strip.tr(" ", "_") >> is_divided = true > >"type" should not be null here,the content of "type" will be lost. >I think you can put it at the beginning of the program. > >Thanks >Shengrui > >> when /test_bpf: Summary: ([0-9]+) PASSED, ([0-9]+) FAILED, \[.+ JIT'ed\]$/ >> success_test = $1 >> fail_test = $2 >> when /(PASS|FAIL)/ >>+ type = "" >> if is_divided >> stats << type + ".#{$1.downcase}: 1" >> is_divided = false >>-- >>2.23.0 >>

1 0

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Lu Kaiyi 21 Aug '20

21 Aug '20

On Fri, Aug 21, 2020 at 02:26:38PM +0800, Zhou Shengrui wrote: >On Thu, Aug 20, 2020 at 07:04:40PM +0800, Lu Kaiyi wrote: >>fix syntax problem in line with crystal >> >>1. single-quotes to double-quotes >>2. $PROGRAM_NAME to PROGRAM_NAME >>3. stats = [] to stats = [] of String >>4. add type = "" before using >> >>Signed-off-by: Lu Kaiyi <lukaiyi(a)huawei.com> >>--- >> stats/crystal/test-bpf.cr | 39 +++++++++++++++++++-------------------- >> 1 file changed, 19 insertions(+), 20 deletions(-) >> >>diff --git a/stats/crystal/test-bpf.cr b/stats/crystal/test-bpf.cr >>index f5b13a09..3941a110 100644 >>--- a/stats/crystal/test-bpf.cr >>+++ b/stats/crystal/test-bpf.cr >>@@ -1,46 +1,45 @@ >>-#!/usr/bin/env ruby >>+#!/usr/bin/env crystal >> >>-LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) >>+LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.real_path(PROGRAM_NAME))) >> >>-require "#{LKP_SRC}/lib/string_ext" >>+require "../../lib/string_ext" >> >>-stats = [] >>+stats = [] of String >> success_test = 0 >> fail_test = 0 >> is_divided = false >> >> while (line = STDIN.gets) >>- line = line.remediate_invalid_byte_sequence(replace: '_') unless line.valid_encoding? >>- >> case line >> when /test_bpf: #[0-9]+ (.+) jited:.+ PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) jited:.+ FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".pass: 1" >> when /test_bpf: #[0-9]+ check: (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << 'check:_' + type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << "check:_" + type + ".fail: 1" >> when /test_bpf: #[0-9]+ (.+) PASS$/ >>- type = $1.tr(' ', '_') >>- stats << type + '.pass: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".pass: 1" >> when /test_bpf: #[0-9]+ (.+) FAIL/ >>- type = $1.tr(' ', '_') >>- stats << type + '.fail: 1' >>+ type = $1.tr(" ", "_") >>+ stats << type + ".fail: 1" >> when /test_bpf: #[0-9]+ check: (.+)/ >>- type = 'check:_' + $1.strip.tr(' ', '_') >>+ type = "check:_" + $1.strip.tr(" ", "_") >> is_divided = true >> when /test_bpf: #[0-9]+ (.+)/ >>- type = $1.strip.tr(' ', '_') >>+ type = $1.strip.tr(" ", "_") >> is_divided = true > >"type" should not be null here,the content of "type" will be lost. >I think you can put it at the beginning of the program. > >Thanks >Shengrui > that's right. thank you for pointing out. Thanks, Kaiyi > >> when /test_bpf: Summary: ([0-9]+) PASSED, ([0-9]+) FAILED, \[.+ JIT'ed\]$/ >> success_test = $1 >> fail_test = $2 >> when /(PASS|FAIL)/ >>+ type = "" >> if is_divided >> stats << type + ".#{$1.downcase}: 1" >> is_divided = false >>-- >>2.23.0 >>

1 0

Re: [PATCH crystal-ci] fix: some error in run job
by Wu Fengguang 21 Aug '20

21 Aug '20

>[ 8.061040] dracut-pre-pivot[653]: mkdir: cannot create directory >'/sysroot/tmp': File exists >[ 8.061818] dracut-pre-pivot[653]: mkdir: cannot create directory >'/sysroot/usr': File exists /usr 分明应该是目录. >[ 8.062597] dracut-pre-pivot[653]: mkdir: cannot create directory >'/sysroot/var': File exists >[ 8.063374] dracut-pre-pivot[653]: mkdir: cannot create directory >'/sysroot/vmlinuz': File exists > >可是这些目录应该有吧，也会报错。这是啥环境? 怎么复现? ls -l /sysroot 是啥调试输出? Thanks, Fengguang

2 1

[PATCH todo] mutt-git-am : fix shellcheck warning
by Bai Jing 21 Aug '20

21 Aug '20

problem: repos=$(cd $HOME/.am && echo *) ^-- SC2086: Double quote to prevent globbing and word splitting. ^-- SC2035: Use ./*glob* or --*glob* so names with dashes won't become options. Signed-off-by: Bai Jing <baijing14(a)huawei.com> --- bin/mutt-git-am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/mutt-git-am b/bin/mutt-git-am index c2e8ebf..7a16105 100755 --- a/bin/mutt-git-am +++ b/bin/mutt-git-am @@ -8,7 +8,7 @@ patch=/tmp/$$.patch cat > $patch -repos=$(cd $HOME/.am && echo *) +repos=$(cd "$HOME"/.am && ls) repo=$(grep -m1 -o "Subject: .*] " $patch | grep -Eo "(${repos// /|})") cd "$HOME/.am/$repo" || exit -- 2.23.0

1 0

Re: [PATCH lkp-tests 2/2] stats/crystal: test-bpf to test-bpf.cr
by Zhou Shengrui 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 07:04:40PM +0800, Lu Kaiyi wrote: >fix syntax problem in line with crystal > >1. single-quotes to double-quotes >2. $PROGRAM_NAME to PROGRAM_NAME >3. stats = [] to stats = [] of String >4. add type = "" before using > >Signed-off-by: Lu Kaiyi <lukaiyi(a)huawei.com> >--- > stats/crystal/test-bpf.cr | 39 +++++++++++++++++++-------------------- > 1 file changed, 19 insertions(+), 20 deletions(-) > >diff --git a/stats/crystal/test-bpf.cr b/stats/crystal/test-bpf.cr >index f5b13a09..3941a110 100644 >--- a/stats/crystal/test-bpf.cr >+++ b/stats/crystal/test-bpf.cr >@@ -1,46 +1,45 @@ >-#!/usr/bin/env ruby >+#!/usr/bin/env crystal > >-LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.reapath($PROGRAM_NAME))) >+LKP_SRC = ENV["LKP_SRC"] || File.dirname(File.dirname(File.real_path(PROGRAM_NAME))) > >-require "#{LKP_SRC}/lib/string_ext" >+require "../../lib/string_ext" > >-stats = [] >+stats = [] of String > success_test = 0 > fail_test = 0 > is_divided = false > > while (line = STDIN.gets) >- line = line.remediate_invalid_byte_sequence(replace: '_') unless line.valid_encoding? >- > case line > when /test_bpf: #[0-9]+ (.+) jited:.+ PASS$/ >- type = $1.tr(' ', '_') >- stats << type + '.pass: 1' >+ type = $1.tr(" ", "_") >+ stats << type + ".pass: 1" > when /test_bpf: #[0-9]+ (.+) jited:.+ FAIL/ >- type = $1.tr(' ', '_') >- stats << type + '.fail: 1' >+ type = $1.tr(" ", "_") >+ stats << type + ".fail: 1" > when /test_bpf: #[0-9]+ check: (.+) PASS$/ >- type = $1.tr(' ', '_') >- stats << 'check:_' + type + '.pass: 1' >+ type = $1.tr(" ", "_") >+ stats << "check:_" + type + ".pass: 1" > when /test_bpf: #[0-9]+ check: (.+) FAIL/ >- type = $1.tr(' ', '_') >- stats << 'check:_' + type + '.fail: 1' >+ type = $1.tr(" ", "_") >+ stats << "check:_" + type + ".fail: 1" > when /test_bpf: #[0-9]+ (.+) PASS$/ >- type = $1.tr(' ', '_') >- stats << type + '.pass: 1' >+ type = $1.tr(" ", "_") >+ stats << type + ".pass: 1" > when /test_bpf: #[0-9]+ (.+) FAIL/ >- type = $1.tr(' ', '_') >- stats << type + '.fail: 1' >+ type = $1.tr(" ", "_") >+ stats << type + ".fail: 1" > when /test_bpf: #[0-9]+ check: (.+)/ >- type = 'check:_' + $1.strip.tr(' ', '_') >+ type = "check:_" + $1.strip.tr(" ", "_") > is_divided = true > when /test_bpf: #[0-9]+ (.+)/ >- type = $1.strip.tr(' ', '_') >+ type = $1.strip.tr(" ", "_") > is_divided = true > when /test_bpf: Summary: ([0-9]+) PASSED, ([0-9]+) FAILED, \[.+ JIT'ed\]$/ > success_test = $1 > fail_test = $2 > when /(PASS|FAIL)/ >+ type = "" "type" should not be null here,the content of "type" will be lost. I think you can put it at the beginning of the program. Thanks Shengrui > if is_divided > stats << type + ".#{$1.downcase}: 1" > is_divided = false >-- >2.23.0 >

1 0

[PATCH todo] todo/bin: fix shellcheck warning
by Zhang Yu 21 Aug '20

21 Aug '20

Signed-off-by: Zhang Yu <zhangyu424(a)huawei.com> --- bin/remind-credit | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/bin/remind-credit b/bin/remind-credit index 887bd42..88a629c 100755 --- a/bin/remind-credit +++ b/bin/remind-credit @@ -3,14 +3,14 @@ credit_file=$(date -d Friday +%Y-%m-%d.md) -cd /c/todo.git +cd /c/todo.git || exit for p in $(git ls-tree --name-only HEAD people/) do [ "${p#people/.}" != "$p" ] && continue - [ -n "$(git ls-tree --name-only HEAD $p/credit/$credit_file)" ] && continue + [[ "$(git ls-tree --name-only HEAD "$p"/credit/"$credit_file")" ]] && continue - name=`basename $p` + name=$(basename "$p") # get mail name from /etc/people2mail.yaml mail_to=$(awk "/^$name: /{print \$2}" '/etc/people2mail.yaml') @@ -24,7 +24,7 @@ do # mail to $mail_to mail_subject="[CREDIT] vim people/$name/credit/$credit_file" - mail -s "$mail_subject" $mail_to <<-EOF + mail -s "$mail_subject" "$mail_to" <<-EOF Please create/push ASAP(ref: /etc/user-fullnames): cd ~/todo -- 2.23.0

1 0

[PATCH crystal-ci] sparrow/0-package: add /srv/osimage in tiny
by Hu Xuejiao 21 Aug '20

21 Aug '20

Because /srv/osimage is required to mapped in os-cifs-smb.conf/start. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/1-storage/tiny | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index 175787c..eb6f8b1 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -10,6 +10,7 @@ dirs=( /srv/initrd/pkg /srv/initrd/deps /srv/os + /srv/osimage /srv/redis /srv/result /srv/cci/scheduler -- 2.23.0

3 3

Re: [CREDIT] vim people/ZhangYu/credit/2020-08-21.md
by Wu Fengguang 21 Aug '20

21 Aug '20

On Fri, Aug 21, 2020 at 09:05:25AM +0800, Zhang Yu wrote: >On Fri, Aug 21, 2020 at 09:00:02AM +0800, Crystal TEAM wrote: >>Please create/push ASAP(ref: /etc/user-fullnames): >> >>cd ~/todo >>vim -O people/ZhangYu/credit/2020-08-21.md /etc/user-fullnames >>git add people/ZhangYu/credit/2020-08-21.md >>git commit -m "people who help me this week" >>git push > >This remind email maybe have some problem, I've created and pushed >credit yesterday. Thanks for the report. Fixed. Thanks, Fengguang

1 0

Re: [PATCH v2 crystal-ci] container/os-cifs: add cifs-/initrd/osimage/result in start
by Wu Fengguang 21 Aug '20

21 Aug '20

title is a bit mess. On Thu, Aug 20, 2020 at 09:02:23PM +0800, Hu Xuejiao wrote: >Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> >--- > container/os-cifs/start | 3 +++ > 1 file changed, 3 insertions(+) > >diff --git a/container/os-cifs/start b/container/os-cifs/start >index a26a58f..c1b3a28 100755 >--- a/container/os-cifs/start >+++ b/container/os-cifs/start >@@ -6,6 +6,9 @@ cmd=( > -p 445:445 > -v $PWD/smb.conf:/etc/samba/smb.conf > -v /srv/os:/srv/os:ro >+ -v /srv/osimage:/srv/osimage That can be removed. >+ -v /srv/initrd:/srv/initrd >+ -v /srv/result:/srv/result > --name samba > --restart=always > alpine/samba >-- >2.23.0 >

1 0

Re: [PATCH v2 crystal-ci] container/os-cifs: add cifs-initrd/osimage/result in smb.conf
by Wu Fengguang 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 08:46:25PM +0800, Hu Xuejiao wrote: >Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> >--- > container/os-cifs/smb.conf | 26 +++++++++++++++++++++++++- > 1 file changed, 25 insertions(+), 1 deletion(-) > >diff --git a/container/os-cifs/smb.conf b/container/os-cifs/smb.conf >index e026cdb..819b702 100644 >--- a/container/os-cifs/smb.conf >+++ b/container/os-cifs/smb.conf >@@ -19,7 +19,7 @@ > unix extensions = yes > [os] > path = /srv/os/ >- comment = initrd >+ comment = os > browseable = yes > read only = yes > public = yes >@@ -27,3 +27,27 @@ > # that can only be accessed by root. > force user = root > force group = root >+[initrd] >+ path = /srv/initrd/ >+ comment = initrd >+ browseable = yes >+ read only = yes >+ public = yes >+ force user = root >+ force group = root root 不合适. >+[osimage] >+ path = /srv/osimage/ /srv/osimage => /srv/initrd >+ comment = osimage osimage => initrd osimage是为了兼容性创造的initrd的一个别名. >+ browseable = yes >+ read only = yes >+ public = yes >+ force user = root >+ force group = root >+[result] >+ path = /srv/result/ >+ comment = result >+ browseable = yes >+ read only = yes >+ public = yes >+ force user = root >+ force group = root >-- >2.23.0 >

1 0

Re: [PATCH todo] todo: fix git-count error
by Wu Fengguang 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 08:21:18PM +0800, Bai Jing wrote: >On Thu, Aug 20, 2020 at 04:58:11PM +0800, Wu Fengguang wrote: >>两段unless File.exist?代码有重复, 一段是不是就够了? >> >>%x => system 更好些 >> >>>+unless File.exist?("#{ENV['HOME']}/.am") >>>+ repos.each_key do |repo| >>>+ %x(mkdir -p ~/.am && cd ~/.am && git clone file:///c/#{repo}.git) >>>+ end >>>+end >>>+ >>> count_by_name.sort_by { |k, _v| k } >>> >>> repos.each do |repo, files| >>>+ unless File.exist?("#{ENV['HOME']}/.am/#{repo}") >>>+ %x(cd ~/.am/ && git clone file:///c/#{repo}.git) >>>+ end >>>+ >>> %x(git -C ~/.am/#{repo} pull --rebase) >>> %x(cd ~/.am/#{repo} && git shortlog -ens --no-merges --since "#{days} days ago" -- #{files}).each_line do |line| > >command : system("\"cd ~/.am/#{repo} && git shortlog -ens --no-merges --since >'#{days} days ago' -- #{files}\"") 是不是 \" 多余了 >I tried use system to replace "%x", but this command show below error. > >error : sh: cd ~/.am/ameba && git shortlog -ens --no-merges --since '1 days >ago' -- : No such file or directory

1 0

Re: [PATCH v5 iso2qcow2 4/6] feat: add os related conf and lib
by Wu Fengguang 21 Aug '20

21 Aug '20

On Thu, Aug 20, 2020 at 09:05:58PM +0800, Yu Chuan wrote: >>>>>这种有一般性, 网上搜一搜, 不需要自己写. >>>>> >>>>>比如 >>>>>~/lkp-tests/lib/detect-system.sh >>>>> >>>>>>+function get_host_info() { >>>>>>+ HOST_OS=$(cat /etc/os-release |grep ^NAME|awk -F '"' '{print $2}'| tr '[:upper:]' '[:lower:]') >>>>>>+ HOST_ARCH=$(arch) >>>>>>+ HOST_VERSION=$(cat /etc/os-release |grep ^VERSION_ID|awk -F '"' '{print $2}') >>>>>>+ check_host >>>>>>+} >>>> >>>>ok, i'll googled about this. >>> >>>Hi Fengguang, >>> >>>我搜了下网上相关功能的实现，以及 ~/lkp-tests/lib/detect-system.sh. 发 >>>现比较健壮可靠的方案是逐个系统进行判断。 >>> >>>所以我准备按如下方案进行实现，你看下是否可行？ >> >>detect-system.sh 还需要修改吗? > >yes, i tested this, and here is the output: > > [root@test ~]# cat ./get-info.sh > #!/bin/bash > > export LKP_SRC="/root/lkp-tests" > > source ${LKP_SRC}/lib/detect-system.sh > > detect_system > detect_system_arch > > echo _system_arch: $_system_arch > echo _system_name: $_system_name > echo _system_name_lowercase: $_system_name_lowercase > echo _system_version: $_system_version > [root@test ~]# sh ./get-info.sh > _system_arch: aarch64 > _system_name: openEuler > _system_name_lowercase: openeuler > _system_version: unknown > >主要是有两个问题： >1. system version 无法获取到。如20.03 这个需要直接 fix lkp-tests >2. detect-system.sh需要用到lkp-tests库中的其他代码，如果要使用这种方式， > 那么就需要分析其依赖，并将用到的所有代码都拷贝一份到iso2qcow2库中？可以啊. 依赖就是 . $LKP_SRC/lib/env.sh 注意license兼容性即可. Thanks, Fengguang >> >>>由于iso2qcow2对环境运行系统做了限制（conf/os/support-host.conf），所以 >>>关于操作系统信息的获得，准备参考~/lkp-tests/lib/detect-system.sh，使用 >>>“if-elif-fi”方式进行收集。 >>> >>>目前支持的系统仅为"openeuler-aarch64-20.03"，所以"if-elif-fi"只需要支持 >>>openeuler，随着以后支持系统的增多，再对此方法进行修正，增加支持新系统。 >>

1 0

[PATCH v3 lkp-tests] lib: add $RESULT_SERVER and update mount.cifs-comment job-init.sh
by Hu Xuejiao 21 Aug '20

21 Aug '20

For result mount in test-machine to add $RESULT_SERVER, $RESULT_SERVER is written in deploy-cci.yaml, and run clif for update mount.cifs-comment. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- lib/job-init.sh | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/lib/job-init.sh b/lib/job-init.sh index 0732d4b9..eeac693e 100755 --- a/lib/job-init.sh +++ b/lib/job-init.sh @@ -51,11 +51,15 @@ setup_result_service() { [ -n "$result_service" ] && return [ -n "$NO_NETWORK" ] && return 1 + [ -n "$RESULT_SERVER" ] || $RESULT_SERVER=$LKP_SERVER - supports_netfs 'nfs' && result_service=$LKP_SERVER:/result && return - supports_netfs 'cifs' && result_service=//$LKP_SERVER/result && return - supports_raw_upload && result_service=raw_upload && return + if [ "$os_mount" == 'nfs' ] || [ -z "$os_mount" ]; then + supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return + elif [ "$os_mount" == 'cifs' ] || [ -z "$os_mount" ]; then + supports_netfs 'cifs' && result_service=//$RESULT_SERVER/result && return + fi + supports_raw_upload && result_service=raw_upload && return return 1 } @@ -94,7 +98,7 @@ mount_result_root() //*/*) result_fs=cifs modprobe cifs 2>/dev/null - local cifs_mount_option='-o guest' + local cifs_mount_option='-o guest,ro,hard,vers=1.0,noacl,nouser_xattr' [ -n "$LKP_CIFS_PORT" ] && cifs_mount_option="$cifs_mount_option,port=$LKP_CIFS_PORT" mount.cifs $cifs_mount_option $result_service $RESULT_MNT || return ;; -- 2.23.0

4 3

[PATCH todo] help me
by Hu Xuejiao 21 Aug '20

21 Aug '20

Signed-off-by: Hu Xuejiao <huxuejiao1(a)huawei.com> --- people/HuXuejiao/credit/2020-08-21.md | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 people/HuXuejiao/credit/2020-08-21.md diff --git a/people/HuXuejiao/credit/2020-08-21.md b/people/HuXuejiao/credit/2020-08-21.md new file mode 100644 index 0000000..c312427 --- /dev/null +++ b/people/HuXuejiao/credit/2020-08-21.md @@ -0,0 +1,4 @@ +Wei Jihui +Yu Chuan +Cao Xueliang +Liu Yinsi -- 2.23.0

1 0