- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 1/2] scheduler/elasticsearch_client.cr: add function get_account
by Wu Zhende 21 Nov '20

21 Nov '20

Add a new function "get_account" to get account info by "my_email". Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/scheduler/elasticsearch_client.cr | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/scheduler/elasticsearch_client.cr b/src/scheduler/elasticsearch_client.cr index ec6de0d..6ea8e2d 100644 --- a/src/scheduler/elasticsearch_client.cr +++ b/src/scheduler/elasticsearch_client.cr @@ -69,6 +69,16 @@ class Elasticsearch::Client return job end + def get_account(my_email : String) + if @client.exists({:index => "accounts", :type => "_doc", :id => my_email}) + response = @client.get_source({:index => "accounts", :type => "_doc", :id => my_email}) + else + response = {"_id" => my_email, "found" => false} + end + + return response + end + private def create(job_content : JSON::Any, job_id : String) return @client.create( { -- 2.23.0

1 0

[PATCH lkp-tests] lkp-tests/*: add archlinux install pp support
by Wang Yong 21 Nov '20

21 Nov '20

archlinux use pacman to install pp Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- bin/lkp-setup-rootfs | 1 + lib/bootstrap.sh | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 98c63bae9..0eabea755 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -261,6 +261,7 @@ while true; do fi install_rpms install_deb + install_pkgs echo $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-lkp $LKP_DEBUG_PREFIX $LKP_SRC/bin/run-lkp $job diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index a7776d687..ce34a6329 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -705,3 +705,10 @@ boot_init() mount_rootfs } + +install_pkgs() +{ + [ -d /opt/pkgs ] || return + echo "install $(ls /opt/pkgs/*.pkg.tar*)" + pacman -U --needed --noconfirm /opt/pkgs/*.pkg.tar* +} -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/iozone: explicit support test=all for iozone
by Lu Kaiyi 21 Nov '20

21 Nov '20

[example] iozone-bs.yaml ... suite: iozone category: benchmark file_size: 4g test: all block_size: - 64k ... [output] ... Run began: Sat Nov 21 16:48:22 2020 Record Size 64 kB File size set to 4194304 kB Command line used: iozone -r 64k -s 4g -f /fs/sdb/iozonetest Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 4194304 64 13650392 11529057 7878373 7763936 7166639 10774159 7520609 16188351 7362482 9751567 9453804 7981606 7932187 iozone test complete. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/iozone b/tests/iozone index b6a1e206..bafc0087 100755 --- a/tests/iozone +++ b/tests/iozone @@ -19,6 +19,8 @@ if [ -n "$block_size" ]; then for ele in ${array[@]} do case $ele in + "all") args+=" -a" + ;; "write") args+=" -i 0" ;; "read") args+=" -i 1" -- 2.23.0

1 0

[PATCH lkp-tests] tests/iozone: explicit support test=all for iozone
by Lu Kaiyi 21 Nov '20

21 Nov '20

[example] iozone-bs.yaml ... suite: iozone category: benchmark file_size: 4g test: all block_size: - 64k ... [output] ... Run began: Sat Nov 21 16:10:59 2020 Record Size 64 kB File size set to 4194304 kB Command line used: iozone -r 64k -s 4g -f /fs/sda/iozonetest Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 4194304 64 13566379 14323483 8305849 8185793 7770078 13206535 4562592 15656107 4448821 4868781 4908219 4736366 4720413 iozone test complete. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/iozone b/tests/iozone index b6a1e206..48688ebb 100755 --- a/tests/iozone +++ b/tests/iozone @@ -19,6 +19,8 @@ if [ -n "$block_size" ]; then for ele in ${array[@]} do case $ele in + "all") args+= " -a" + ;; "write") args+=" -i 0" ;; "read") args+=" -i 1" -- 2.23.0

1 0

[PATCH v2 compass-ci 3/3] container/git-mirror: fix that can't get group id of committer in z9
by Li Yuanchao 21 Nov '20

21 Nov '20

[why] In z9, 'committer' is only a group, not a user, so command 'id -g committer' can't get id of committer group Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/start | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/git-mirror/start b/container/git-mirror/start index e1d7d8f..f5321a1 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -10,7 +10,7 @@ DOCKER_REPO = '/c/upstream-repos' docker_rm "git-mirror" lkp = %x(id -u lkp) -committer = %x(id -g committer) +committer = %x(cat /etc/group | grep '^committer' | awk -F ':' '{print $3}') cmd = %W[ docker run -- 2.23.0

3 4

[PATCH v9 compass-ci 2/3] lib/matrix2.rb: refactor performance optimization
by Lu Kaiyi 21 Nov '20

21 Nov '20

[why] Avoid to execute some unnecessary operations and improve performance. [how] Move the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index ff4ea31..2065039 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

1 0

[PATCH v6 lkp-tests] add: sysbench-threads-git result parsing file
by Hu Xuejiao 21 Nov '20

21 Nov '20

[input] General statistics: total time: 0.3987s total number of events: 10000 total time taken by event execution: 0.3971 response time: min: 0.04ms [output] total time: 0.3987s Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- stats/sysbench-threads-git | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100755 stats/sysbench-threads-git diff --git a/stats/sysbench-threads-git b/stats/sysbench-threads-git new file mode 100755 index 00000000..afe5d664 --- /dev/null +++ b/stats/sysbench-threads-git @@ -0,0 +1,14 @@ +#!/usr/bin/env ruby + +# General statistics: +# total time: 0.2671s +# total number of events: 10000 +# total time taken by event execution: 0.3971s +# response time: +# min: 0.04ms + +$stdin.each_line do |line| + next unless line =~ /^\s+total time:\s+([0-9.]+)/ + + puts "total time: #{$1.to_f}s" +end -- 2.23.0

2 1

[PATCH v2 lkp-tests] stats/lmbench3: add new parsing parameters for performance test
by Bai Jing 21 Nov '20

21 Nov '20

[input] Memory read bandwidth 0.000512 16353.73 0.001024 16577.89 0.002048 16752.28 [output] L1_$: 1.539 L2_$: 0.539 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/lmbench3 | 62 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 60 insertions(+), 2 deletions(-) diff --git a/stats/lmbench3 b/stats/lmbench3 index 211c3abc..750e8bfc 100755 --- a/stats/lmbench3 +++ b/stats/lmbench3 @@ -21,6 +21,27 @@ def largest_bandwidth bandwidth end +def designated_file_size + designated_hash = {} + result_str = '' + + $stdin.each_line do |line| + result_str += line + break if line =~ /(^\s|^\t)/ + end + + result_str.split("\n").each do |e| + designated_hash['L1_$'] = e.split[1] if e =~ /0.01758/ + designated_hash['Rand_mem'] = e.split[1] if e =~ /512.0000/ + designated_hash['Mmap_Latency'] = e.split[1] if e =~ /536.870912/ + designated_hash['L2_$'] = e.split[1] if e =~ /0.05078/ + end + + designated_hash +end + +null_io_array = [] + while (line = STDIN.gets) line = line.resolve_invalid_bytes case line @@ -32,14 +53,15 @@ while (line = STDIN.gets) # Simple fstat: 0.3517 microseconds # Simple open/close: 1.3696 microseconds when /^Simple (\S+): (\d+.\d+) microseconds$/ + null_io_array << $2.to_f if $1.to_s == 'read' || $1.to_s == 'write' puts "syscall.#{$1}.latency.us: #{$2}" # Extract select test result. # Select on 100 fd's: 1.2293 microseconds # Select on 100 tcp fd's: 5.0377 microseconds - when /^Select on 100 fd\'s: (\d+.\d+) microseconds$/ + when /^Select on 100 fd's: (\d+.\d+) microseconds$/ puts "Select.100fd.latency.us: #{$1}" - when /^Select on 100 tcp fd\'s: (\d+.\d+) microseconds$/ + when /^Select on 100 tcp fd's: (\d+.\d+) microseconds$/ puts "Select.100tcp.latency.us: #{$1}" # Extract proc test result. @@ -198,5 +220,41 @@ while (line = STDIN.gets) # ... when /^"Mmap read open2close bandwidth$/ puts "MMAP.read_open2close.bandwidth.MB/sec: #{largest_bandwidth}" + + # "Random Memory load latency" + # 416.00000 92.824 + # 448.00000 92.772 + # 512.00000 92.640 + # ... + when /^Random load latency$/ + result_rand_mem = designated_file_size + puts "Rand_mem: #{result_rand_mem['Rand_mem']}" + + # "Memory load latency" + # 0.00049 1.539 + # 0.00098 1.539 + # 512.00000 74.421 + # ... + when /^Memory load latency$/ + result_rand_mem = designated_file_size + puts "L1_$: #{result_rand_mem['L1_$']}" + puts "L2_$: #{result_rand_mem['L2_$']}" + + # "mappings" + # 67.108864 82 + # 134.217728 151 + # 268.435456 309 + # 536.870912 615 + # ... + when /^"mappings$/ + result_map = designated_file_size + puts "Mmap_Latency: #{result_map['Mmap_Latency']}" end end + +num = 0 +null_io_array.each do |v| + num += v +end +null_num = format('%.4f', num / 2).to_f +puts "null_io: #{null_num}" -- 2.23.0

3 6

git count
by Wu Fengguang 20 Nov '20

20 Nov '20

Shengde is our code star, thanks Shengde! wfg@crystal ~/todo% bin/git-count 21 Luan Shengde 13 Sun Yukui 10 Li Yuanchao 8 Hu Xuejiao 8 Xiao Shenwei 8 Zhang Yale 8 Yu Chuan 8 Lin Jiaxin 7 Bai Jing 7 Liu Yinsi 6 Liu Shaofei 6 Zhang Yu 5 Cao Xueliang 5 Cui Yili 5 Wang Yong 4 Xu Xijian 4 Wei Jihui 4 Lu Kaiyi 3 Wu Zhende 2 Lu Weitao 2 Wang Chenglong 2 Zhang Yuhang 1 Wu Fengguang 1 Ren Wen 1 Li Ping

1 0

review star
by Wu Fengguang 20 Nov '20

20 Nov '20

Xueliang is our review star, thanks Xueliang! wfg@crystal /c/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 51 44 107 34 Wu Fengguang 118 37 48 24 Cao Xueliang 99 35 50 20 Wei Jihui 29 18 22 14 Xu Xijian 33 6 18 9 Xiao Shenwei 75 23 54 9 Luan Shengde 47 19 21 7 Liu Shaofei 61 14 13 7 Lu Kaiyi 31 13 17 6 Zhang Yu 16 6 11 6 Zhang Yuhang 52 9 11 5 Bai Jing 18 5 10 4 Wang Yong 34 6 11 4 Wang Chenglong 22 14 17 4 Cui Yili 56 7 10 4 Lin Jiaxin 13 3 5 4 Lu Weitao 19 6 8 3 Wu Zhende 61 7 9 3 Hu Xuejiao 47 3 8 2 Yu Chuan 31 7 9 1 Liu Yinsi 8 0 0 0 Li Ping 1 1 1 0 Zhang Dewan 12 2 3 0 Ren Wen 28 9 9 -2 Zhang Yale 23 5 9 -3 Li Yuanchao 41 15 26 -3 Sun Yukui

1 0