- Compass-ci - mailweb.openeuler.org

[PATCH v6 compass-ci] sparrow/4-docker: fix logical error
by Liu Yinsi 14 Jan '21

14 Jan '21

[error] root@taishan200-2280-2s64p-256g--a9 /c/compass-ci/sparrow/4-docker# ./buildall 2021-01-14 03:32:28 INFO -- start build /c/compass-ci/container/archlinux/. Unable to find image 'archlinux:testbed' locally docker: Error response from daemon: manifest for archlinux:testbed not found. See 'docker run --help'. [why] before: if docker images not exist, not build images after: if docker images not exist, build images Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index f24226c..a60067f 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -33,7 +33,7 @@ do_one() cd "$container" [ "$container_name" == 'ssh-r' ] && exit docker images | grep -wq "$container_name" - if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then + if [ "$?" != 0 ] || [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

1 0

Re: [PATCH v2 lkp-tests 1/2] tests: modify api-avx2neon
by Wu Fengguang 14 Jan '21

14 Jan '21

>>>+ echo "api-avx2neon running" >>>+ ./test >> >>What if ./test exit 1? > > this script captures and counts its own error output What if ./test exit for other (unexpected) reason? Then we miss a failure case. >>>+ echo "api-avx2neon finished"

1 0

[PATCH v3 lkp-tests] stats: add stats script for openeuler_docker test
by Bai Jing 14 Jan '21

14 Jan '21

[input] ^[[0mwordpress.build.fail: 1 [output] wordpress.build.fail: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..e31934573 --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + line.chomp! + next unless line =~ /build.fail/ + + puts line +end -- 2.23.0

2 1

[PATCH compass-ci] qeumu/kvm.sh: modify the comment
by Hu Xuejiao 14 Jan '21

14 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- providers/qemu/kvm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index ce3b75f..8b05f83 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -53,7 +53,7 @@ parse_ipxe_script() check_option_value() { [ -s "$kernel" ] || { - echo "The kernel does not exist" + echo "The kernel does not exist: $kernel" exit } -- 2.23.0

2 1

[PATCH v2 lkp-tests] stats: add stats script for openeuler_docker test
by Bai Jing 14 Jan '21

14 Jan '21

[input] ^[[0mwordpress_build_on_fail: 1 [output] wordpress_build_on_fail: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..3f8012a8c --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + line.chomp! + next unless line =~ /build_on_fail/ + + puts line +end -- 2.23.0

2 2

Re: [PATCH v2 lkp-tests 2/2] lib: fix 409 conflict
by Wu Fengguang 14 Jan '21

14 Jan '21

TO ALL. On Thu, Jan 14, 2021 at 10:55:46AM +0800, Li Ping wrote: >On Thu, Jan 14, 2021 at 09:08:50AM +0800, Wu Fengguang wrote: >>On Wed, Jan 13, 2021 at 09:19:47PM +0800, Li Ping wrote: >>>[why] >>>find a -type d >>>a >>>a/b >>>a/c >>>find a -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; >>>==> curl: (22) The requested URL returned error: 409 Conflict >>>reason: upload an empty folder causes this error. >> >>I suspect 409 happens because server side already have that dir. >>It's like you run > > That's what I thought at first, then I make new dir, > upload the new dir also causes this error. It does not happen if you didn't say it. That's the golden rule of mailing list. Please document such important points in the changelog in future. Thanks, Fengguang

1 0

Re: [PATCH v2 lkp-tests 2/2] lib: fix 409 conflict
by Wu Fengguang 14 Jan '21

14 Jan '21

On Thu, Jan 14, 2021 at 10:55:46AM +0800, Li Ping wrote: >On Thu, Jan 14, 2021 at 09:08:50AM +0800, Wu Fengguang wrote: >>On Wed, Jan 13, 2021 at 09:19:47PM +0800, Li Ping wrote: >>>[why] >>>find a -type d >>>a >>>a/b >>>a/c >>>find a -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; >>>==> curl: (22) The requested URL returned error: 409 Conflict >>>reason: upload an empty folder causes this error. >> >>I suspect 409 happens because server side already have that dir. >>It's like you run > > That's what I thought at first, then I make new dir, > upload the new dir also causes this error. > > if I create new file under the dir would not cause this error. > so i think there is no need to upload empty folder at all. No "think" please. You should give clear statement, like "webdav supports auto-and-recursively create dirs when uploading some dir_level1/dir_level2/file". And prove it with commandline examples or webdav API definitions. Whether or not your patch is right, the changelog is not clear enough to prove whether it's "definitely/always right" or "right by chance". >> >> mkdir a => success >> mkdir a => 409 >> >>But will the server side auto create new subdirs if they do not >>already exist? Probably the below find will fail. > >> >>Please experiment with deeper dirs, like a/b/c/file > > Actually, I have experimented with deeper dirs Then please showcase it in changelog. BTW, please try this example https://stackoverflow.com/questions/42941176/how-to-create-directories-recu… If our webdav server can auto create subdirs, then what's the difference in our server comparing to the standard/old one? Thanks, Fengguang

1 0

[PATCH compass-ci] lib/error_messages.rb: uniform naming style
by Lin Jiaxin 14 Jan '21

14 Jan '21

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/error_messages.rb | 14 +++++++------- lib/git_bisect.rb | 2 +- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/lib/error_messages.rb b/lib/error_messages.rb index 5bbc329..3e1564b 100644 --- a/lib/error_messages.rb +++ b/lib/error_messages.rb @@ -8,8 +8,8 @@ require_relative "#{ENV['LKP_SRC']}/lib/common" # usage: # foo = ErrorMessages.new(build-pkg) # build-pkg is a file that includes errors generated by make -# errorid_array = foo.obtain_error_messages_by_errorid(errorid) -# errorid_array includes all error_messages that error same as errorid +# error_id_array = foo.obtain_error_messages_by_error_id(error_id) +# error_id_array includes all error_messages that error same as error_id # error_messages_hash = foo.obtain_error_messages # error_messages_hash.each { |k,v| puts k; puts; v.each {|m| puts m } } # output: @@ -40,15 +40,15 @@ class ErrorMessages return @error_messages end - def obtain_error_messages_by_errorid(errorid) - error_messages_by_errorid = [] + def obtain_error_messages_by_error_id(error_id) + error_messages_by_error_id = [] error_messages = obtain_error_messages error_messages.each do |k, v| - if ('build-pkg.' + build_pkg_error_id(k)) == errorid - error_messages_by_errorid += v.to_a + if ('build-pkg.' + build_pkg_error_id(k)) == error_id + error_messages_by_error_id += v.to_a end end - error_messages_by_errorid + error_messages_by_error_id end private diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 2043730..2fee60c 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -97,7 +97,7 @@ class GitBisect return nil unless temp[0].include?('is the first bad commit') || temp[-1].include?('bisect run success') first_bad_commit = Utils.parse_first_bad_commit(result) - bisect_error = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_errorid(@error_id) + bisect_error = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_error_id(@error_id) all_errors = Utils.obt_errors(@work_dir, first_bad_commit) first_bad_commit_result_root = Utils.obt_result_root_by_commit(first_bad_commit) FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) -- 2.23.0

1 0

Re: [PATCH lkp-tests] lib/job.sh: modify the generated file suffix
by Cao Xueliang 14 Jan '21

14 Jan '21

On Thu, Jan 14, 2021 at 03:44:10PM +0800, Li Ping wrote: >[why] >user extract $program result actually generating a YAML file. >and compass-ci/sbin/result2stats have already support merge yaml file > >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > lib/job.sh | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > >diff --git a/lib/job.sh b/lib/job.sh >index 03c49e0ad..dc65f68d8 100755 >--- a/lib/job.sh >+++ b/lib/job.sh >@@ -318,5 +318,5 @@ run_test() > run_target_stats() > { > local script_name=$1 >- $LKP_SRC/stats/$script_name < $TMP_RESULT_ROOT/$script_name > $TMP_RESULT_ROOT/$script_name.json >+ $LKP_SRC/stats/$script_name < $TMP_RESULT_ROOT/$script_name > $TMP_RESULT_ROOT/$script_name.yaml I don't think this can change json to yaml format. Thanks, Xueliang > } >-- >2.23.0 >

1 0

[PATCH v9 compass-ci 2/2] lib/es_query.rb: parse es_result to a Hash result
by Lu Kaiyi 14 Jan '21

14 Jan '21

[why] make es result more easy access. [example] input: es_result = [{"key"=>"build-pkg", "doc_count"=>354526, "all_job_state"=> {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=> [{"key"=>"failed", "doc_count"=>5708}, {"key"=>"finished", "doc_count"=>1033}, {"key"=>"incomplete", "doc_count"=>204}, {"key"=>"submit", "doc_count"=>136}, ... output: result_hash = {"build-pkg"=> {"failed"=>5708, "finished"=>1033, "incomplete"=>204, "submit"=>136, "OOM"=>11, "post_run"=>2}, "cci-depends"=> {"finished"=>1785, "failed"=>675, ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/es_query.rb | 46 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 91aa6ab..928bd68 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -128,7 +128,7 @@ class ESQuery result = @client.search(index: @index + '*', body: query)['aggregations']["all_#{field1}"]['buckets'] return nil if result.empty? - result + parse_fields(result) end end @@ -216,3 +216,47 @@ def build_aggs_from_fields(fields) aggs_hash['aggs']["all_#{field}"].merge!(sub_aggs) if sub_aggs aggs_hash end + +# input: +# es_result = +# [{"key"=>"build-pkg", +# "doc_count"=>354526, +# "all_job_state"=> +# {"doc_count_error_upper_bound"=>0, +# "sum_other_doc_count"=>0, +# "buckets"=> +# [{"key"=>"failed", "doc_count"=>5708}, +# {"key"=>"finished", "doc_count"=>1033}, +# {"key"=>"incomplete", "doc_count"=>204}, +# {"key"=>"submit", "doc_count"=>136}, +# ... +# +# output: +# result_hash = +# {"build-pkg"=> +# {"failed"=>5708, +# "finished"=>1033, +# "incomplete"=>204, +# "submit"=>136, +# "OOM"=>11, +# "post_run"=>2}, +# "cci-depends"=> +# {"finished"=>1785, +# "failed"=>675, +# ... +def parse_fields(es_result) + result_hash = {} + es_result.each do |result| + key = result['key'] + sub_field = result.keys.detect { |field| field.start_with?('all_') } + + if sub_field + all_field = result[sub_field]['buckets'] + result_hash[key] = parse_fields(all_field) + else + result_hash[key] = result['doc_count'] + end + end + + result_hash +end -- 2.23.0

1 0