- Compass-ci - mailweb.openeuler.org

review star 2020-09-29
by Wu Fengguang 29 Sep '20

29 Sep '20

Yuchuan is our review star, thanks Yuchuan! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 401 212 425 75 Wu Fengguang 129 23 56 27 Tong Qunfeng 28 17 52 12 Shi Zhichao 49 22 43 24 Du Kaitian 67 14 41 10 Yu Chuan 28 18 27 7 Xu Xijian 41 12 21 7 Xiao Shenwei 24 14 20 5 Wei Jihui 21 11 20 6 Li Yuanchao 20 10 15 0 Zhang Yu 8 8 14 2 Sun Yukui 32 10 14 1 Liu Yinsi 54 5 11 3 Hu Xuejiao 33 4 10 1 Cao Xueliang 90 5 9 1 Luan Shengde 44 7 8 0 Wu Zhende 13 5 7 0 Xie Zheng 116 5 7 0 Ren Wen 13 7 7 0 Wang Yong 9 3 6 3 Zhou Shengrui 17 3 6 0 Liu Shaofei 69 1 5 1 Lu Weitao 67 1 5 3 Zhang Yuhang 20 3 3 1 Zhang Yale 29 1 3 1 Li Ping 30 1 2 6 Cui Yili 15 6 2 2 Lu Kaiyi 13 1 1 0 Lin Jiaxin 29 1 1 0 Wang Chenglong 1 0 0 0 Liang Tao 8 0 0 0 Zhang Dewan 22 3 0 1 Bai Jing

1 0

help star 2020-09-18
by Wu Fengguang 29 Sep '20

29 Sep '20

Jihui is our help star, thanks to kind and helpful Jihui! wfg@crystal ~/todo% bin/help-star people/*/credit/2020-09-18.md 32.6 Wu Fengguang 20.1 Shi Zhichao 18.1 Wei Jihui 17.6 Xu Xijian 15.1 Bai Jing 13.4 Yu Chuan 12.2 Zhang Dewan 11.8 Lu Weitao 10.4 Tong Qunfeng 9.8 Xiao Shenwei 9.6 Cui Yili 9.0 Liu Shaofei 8.5 Du Kaitian 8.4 Liu Yinsi 7.7 Zhang Yu 7.5 Wu Zhende 7.3 Wang Yong 7.1 Cao Xueliang 6.9 Sun Yukui 6.6 Hu Xuejiao 6.5 Wang Chenglong 6.1 Li Yuanchao 5.8 Zhou Shengrui 4.7 Lin Jiaxin 3.6 Liang Tao 3.5 Luan Shengde 3.1 Lu Kaiyi 2.5 Zhang Yale 1.4 Lu Shengde 1.2 Ren Wen 1.1 Zhang Yuhang 0.8 Li Ping 0.0 Ren Zhiwen 0.0 Sun Lijun 0.0 Wang Shuai 0.0 Cui Xucui 0.0 Xie Zheng

1 0

git count 2020-09-29
by Wu Fengguang 29 Sep '20

29 Sep '20

Renwen is code star of the week, thanks Renwen! wfg@crystal ~/todo% bin/git-count 11 40 Ren Wen 18 Wu Fengguang 12 Lu Weitao 11 Wu Zhende 10 Hu Xuejiao 8 Zhang Yuhang 7 Cao Xueliang 7 Liu Yinsi 7 Xiao Shenwei 6 Yu Chuan 5 Bai Jing 5 Zhang Yu 4 Xu Xijian 4 Wei Jihui 4 Cui Yili 4 Wang Yong 4 Li Yuanchao 3 Wang Chenglong 3 Luan Shengde 3 Lin Jiaxin 3 Xie Zheng 2 Zhang Yale 2 Li Ping 2 Lu Kaiyi 1 Du Kaitian 1 Liu Shaofei 1 Liang Tao 1 Zhang Dewan 0 chief 0 Ren Zhiwen 0 Sun Lijun 0 Sun Yukui 0 Shi Zhichao 0 Zhou Shengrui

1 0

[PATCH v2 compass-ci] sparrow/0-package: delete the repeate install package
by Zhang Yu 29 Sep '20

29 Sep '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- sparrow/0-package/os/debian | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 8301500..ef90e50 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -13,11 +13,10 @@ pkgs=( docker.io ruby-full ruby-dev - bridge-utils qemu + bridge-utils dmidecode lockfile-progs - bridge-utils ) apt-get install -y "${pkgs[@]}" -- 2.23.0

3 2

[PATCH compass-ci] git_mirror: feat: reload fork_info when /upstream-repos update
by Li Yuanchao 29 Sep '20

29 Sep '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index ad27880..88a94ac 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -147,6 +147,8 @@ class MirrorMain end def send_message(feedback_info) + feedback_info.merge!(@git_info[feedback_info[:git_repo]]) + feedback_info.delete(:cur_refs) message = feedback_info.to_json @message_queue.publish(message) end @@ -158,12 +160,12 @@ class MirrorMain @fork_stat[feedback_info[:git_repo]][:queued] = false return unless feedback_info[:possible_new_refs] + return reload_fork_info if feedback_info[:git_repo] == 'upstream-repos/upstream-repos' + new_refs = check_new_refs(feedback_info[:git_repo]) return if new_refs[:heads].empty? feedback_info[:new_refs] = new_refs - feedback_info.merge!(@git_info[feedback_info[:git_repo]]) - feedback_info.delete(:cur_refs) send_message(feedback_info) end @@ -219,4 +221,27 @@ class MirrorMain @git_info[git_repo][:cur_refs] = cur_refs return new_refs end + + def reload_fork_info + upstream_repos = 'upstream-repos/upstream-repos' + if @git_info[upstream_repos][:cur_refs].empty? + @git_info[upstream_repos][:cur_refs] = get_cur_refs(upstream_repos) + else + old_commit = @git_info[upstream_repos][:cur_refs][:heads]['refs/heads/master'] + new_refs = check_new_refs(upstream_repos) + new_commit = new_refs[:heads]['refs/heads/master'] + changed_files = %x(git -C /srv/git/#{upstream_repos}.git diff --name-only #{old_commit}...#{new_commit}) + reload(changed_files) + end + end + + def reload(file_list) + system("git -C #{REPO_DIR} pull") + file_list.each_line do |file| + next if File.basename(file) == '.ignore' + + repo_dir = "#{REPO_DIR}/#{file}" + load_repo_file(repo_dir, File.dirname(file), File.basename(file)) if File.file?(repo_dir) + end + end end -- 2.23.0

2 1

[PATCH v4 compass-ci] fix: sched: submit idle jobs
by Ren Wen 29 Sep '20

29 Sep '20

1) add field 'idle_job' for idle jobs, value is always 'true'. 2) add two fields 'FLUENTD_SERVER_*' for monitoring logs of virtual machines started by physical machine. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 15f5c9e..309df04 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -468,9 +468,16 @@ class Sched def auto_submit_idle_job(tbox_group) full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" + extra_job_fields = [ + "idle_job=true", + "FLUENTD_SERVER_HOST=#{ENV["FLUENTD_SERVER_HOST"]}", + "FLUENTD_SERVER_PORT=#{ENV["FLUENTD_SERVER_PORT"]}", + ] + Jobfile::Operate.auto_submit_job( full_path_patterns, - "testbox: #{tbox_group}") if Dir.glob(full_path_patterns).size > 0 + "testbox: #{tbox_group}", + extra_job_fields) if Dir.glob(full_path_patterns).size > 0 end private def add_kernel_console_param(arch_tmp) @@ -520,7 +527,7 @@ class Sched "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/vmlinuz")}" else response += "kernel #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" end response += " user=lkp" response += " job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" -- 2.23.0

1 0

[PATCH compass-ci] check path of the upstream_repo under /srv/git/
by Li Ping 29 Sep '20

29 Sep '20

--- lib/git.rb | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/lib/git.rb b/lib/git.rb index db97c47..a181ec9 100644 --- a/lib/git.rb +++ b/lib/git.rb @@ -3,8 +3,34 @@ # wrap common git commands class GitCommit def initialize(repo, commit) - @git_prefix = "git -C /srv/git/#{repo}.git" + @repo = repo @commit = commit + @git_prefix = cd_repo_path + end + + def check_repo + fork_name = @repo.split('/')[0] + project = @repo.split('/')[1] + first_letter = project[0] + fork_name = 'linus' if project == 'linux' + path_a = find_path("/srv/git/#{first_letter}/#{project}/#{project}.git") + path_b = find_path("/srv/git/#{first_letter}/#{project}/#{fork_name}.git") + path_c = find_path("/srv/git/#{project}/#{fork_name}.git") + path_d = find_path("/srv/git/#{project}/#{project}.git") + path_a || path_b || path_c || path_d + end + + def find_path(path) + return false if Dir.glob(path).empty? + + Dir.glob(path).first + end + + def cd_repo_path + find_repo = check_repo + return unless find_repo + + "git -C #{find_repo}" end def author_name -- 2.23.0

5 13

[PATCH v3 compass-ci] fix: sched: submit idle jobs
by Ren Wen 29 Sep '20

29 Sep '20

1) submit all idles jobs once use '*' -> submit each idle job once until all done. 2) add two fields into job if job's filename is 'multi-qemu.yaml'. 3) add field 'idle_job' for idle jobs, value is always 'true'. Two extra fields are used for: monitoring logs of virtual machines started by physical machine. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 15f5c9e..6ca20d7 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -467,10 +467,18 @@ class Sched end def auto_submit_idle_job(tbox_group) - full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" - Jobfile::Operate.auto_submit_job( - full_path_patterns, - "testbox: #{tbox_group}") if Dir.glob(full_path_patterns).size > 0 + idle_job_path = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/" + + return nil unless Dir.exists?(idle_job_path) + jobs = Dir[idle_job_path + "*.yaml"] + jobs.each do |job| + extra_job_fields = ["idle_job=true"] + if Path[job].basename == "multi-qemu.yaml" + extra_job_fields += ["FLUENTD_SERVER_HOST=#{ENV["FLUENTD_SERVER_HOST"]}", + "FLUENTD_SERVER_PORT=#{ENV["FLUENTD_SERVER_PORT"]}"] + end + Jobfile::Operate.auto_submit_job(job, "testbox: #{tbox_group}", extra_job_fields) + end end private def add_kernel_console_param(arch_tmp) @@ -520,7 +528,7 @@ class Sched "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/vmlinuz")}" else response += "kernel #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" end response += " user=lkp" response += " job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" -- 2.23.0

2 3

[PATCH compass-ci] fix: sched: submit idle jobs
by Ren Wen 29 Sep '20

29 Sep '20

1) submit all idles jobs once use '*' -> submit each idle job once until all done. 2) add two fields into job if job's name is multi-qemu. job's name 'multi-qemu' is case-insensitive matching now. 3) `.yaml` and `.yml` can both match. Two extra fields are used for: monitoring logs of virtual machines started by physical machine. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 15f5c9e..c27dd99 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -467,10 +467,20 @@ class Sched end def auto_submit_idle_job(tbox_group) - full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" - Jobfile::Operate.auto_submit_job( - full_path_patterns, - "testbox: #{tbox_group}") if Dir.glob(full_path_patterns).size > 0 + idle_job_path = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/" + + if Dir.exists?(idle_job_path) + jobs = Dir[idle_job_path + "*.y{a,}ml"] + extra_job_fields = ["FLUENTD_SERVER_HOST=#{ENV["FLUENTD_SERVER_HOST"]}", + "FLUENTD_SERVER_PORT=#{ENV["FLUENTD_SERVER_PORT"]}"] + + jobs.each do |job| + if Path[job].basename =~ /^multi-qemu\.ya?ml/i + Jobfile::Operate.auto_submit_job(job, "testbox: #{tbox_group}", extra_job_fields) + end + Jobfile::Operate.auto_submit_job(job, "testbox: #{tbox_group}") + end + end end private def add_kernel_console_param(arch_tmp) @@ -520,7 +530,7 @@ class Sched "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/vmlinuz")}" else response += "kernel #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" end response += " user=lkp" response += " job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" -- 2.23.0

3 10

[PATCH compass-ci] check if author's email address exist in upstream repos
by Li Ping 29 Sep '20

29 Sep '20

--- lib/git.rb | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/lib/git.rb b/lib/git.rb index a181ec9..4939c94 100644 --- a/lib/git.rb +++ b/lib/git.rb @@ -2,7 +2,7 @@ # wrap common git commands class GitCommit - def initialize(repo, commit) + def initialize(repo, commit = nil) @repo = repo @commit = commit @git_prefix = cd_repo_path @@ -28,32 +28,42 @@ class GitCommit def cd_repo_path find_repo = check_repo - return unless find_repo + message = "The repo is not found in the upstream-repo" + + raise StandardError, message unless find_repo "git -C #{find_repo}" end + def check_author_in_upstream_repos(email) + author_email = %x(#{@git_prefix} log | grep '^Author'| grep #{email}).chomp + + message = "The author's email address is not found in the upstream-repo" + + raise StandardError, message if author_email.empty? + end + def author_name - `#{@git_prefix} log -n1 --pretty=format:'%an' #{(a)commit}`.chomp + %x(#{git_prefix} log -n1 --pretty=format:'%an' #{(a)commit}).chomp end def author_email - `#{@git_prefix} log -n1 --pretty=format:'%ae' #{(a)commit}`.chomp + %x(#{@git_prefix} log -n1 --pretty=format:'%ae' #{(a)commit}).chomp end def subject - `#{@git_prefix} log -n1 --pretty=format:'%s' #{(a)commit}`.chomp + %x(#{@git_prefix} log -n1 --pretty=format:'%s' #{(a)commit}).chomp end def commit_time - `#{@git_prefix} log -n1 --pretty=format:'%ci' #{(a)commit}`.chomp + %x(#{@git_prefix} log -n1 --pretty=format:'%ci' #{(a)commit}).chomp end def url - `#{@git_prefix} remote -v`.split[1] + %x(#{@git_prefix} remote -v).split[1] end def diff - `#{@git_prefix} diff -1 #{@commit}~..#{@commit}`.chomp + %x(#{@git_prefix} diff -1 #{@commit}~..#{@commit}).chomp end end -- 2.23.0

1 0