November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] spec/submit_spec.rb: update submit_job
by Hu Xuejiao 28 Nov '20

28 Nov '20

[why] Originally submit_job is too complicated, it need to be simplified. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- spec/submit_spec.rb | 55 +++++++++++++++++---------------------------- 1 file changed, 21 insertions(+), 34 deletions(-) diff --git a/spec/submit_spec.rb b/spec/submit_spec.rb index 4096cb73..28aca220 100644 --- a/spec/submit_spec.rb +++ b/spec/submit_spec.rb @@ -20,40 +20,27 @@ def traverse_file(output_dir) end end -def submit_job(submit_item) - submit_yaml_path = File.join("#{LKP_SRC}/spec/submit", submit_item) - Dir.glob("#{submit_yaml_path}/*.yaml").each do |yaml_file| - output_dir = File.join(submit_yaml_path, File.basename(yaml_file, '.yaml')) - submit_cmd = [ - "#{LKP_SRC}/sbin/submit", - '-o', output_dir, - '-s', 'lab: spec_lab', - '-s', 'testbox: vm-2p8g--spec_submit', - yaml_file - ] - system(*submit_cmd) - traverse_file(output_dir) +def submit_job() + submit_path = "#{LKP_SRC}/spec/submit" + Dir.foreach(submit_path) do | sub | + next if sub == '.' || sub == '..' + + path = "#{submit_path}/#{sub}" + if File.directory?(path) + Dir.glob("#{path}/*.yaml").each do |yaml_file| + output_dir = File.join(path, File.basename(yaml_file, '.yaml')) + submit_cmd = [ + "#{LKP_SRC}/sbin/submit", + '-o', output_dir, + '-s', 'lab: spec_lab', + '-s', 'testbox: vm-2p8g--spec_submit', + yaml_file + ] + system(*submit_cmd) + traverse_file(output_dir) + end + end end end -describe 'submit job spec' do - it 'link jobs spec' do - submit_job('link_jobs') - end - - it 'link matrix' do - submit_job('matrix') - end - - it 'separate yaml spec' do - submit_job('separate_yaml') - end - - it 'job on fail' do - submit_job('job_on_fail') - end - - it 'merge yaml' do - submit_job('merge_yaml') - end -end +submit_job -- 2.23.0

1 0

[PATCH v4 compass-ci] answerback-email: fix bugs
by Luan Shengde 28 Nov '20

28 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original value is defined nil for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..badbdea 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -121,12 +121,15 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + puts "-l: bad value #{value}, please use y|n\n\n" + puts opts.banner + return 0 end end @@ -136,6 +139,12 @@ options = OptionParser.new do |opts| end end +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +161,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] answerback-email: fix bugs
by Luan Shengde 28 Nov '20

28 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original data is defined nil, but not empty for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index badbdea..874003e 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -219,4 +219,4 @@ def store_account_info(my_info) es.put_source_by_id(my_info['my_email'], my_info) end -send_account(my_info, conf_info, email_info, my_info_es, stdin_info) +# send_account(my_info, conf_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 1

[PATCH v3 compass-ci 1/2] answerback-email: fix bugs
by Luan Shengde 28 Nov '20

28 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original data is defined nil, but not empty for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..badbdea 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -121,12 +121,15 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + puts "-l: bad value #{value}, please use y|n\n\n" + puts opts.banner + return 0 end end @@ -136,6 +139,12 @@ options = OptionParser.new do |opts| end end +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +161,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 1

[PATCH v2 compass-ci] answerback-email: fix bugs
by Luan Shengde 28 Nov '20

28 Nov '20

1. set default option -h for tool [why] the tool will show help message if no option specified the tool will get no email address without both '-e' and '-f' [how] add '-h' as default option if no option specified options will be reset to '-h' if both '-e' and '-f' are not specified 2. fix undefined method 'empty' [why] the original data is defined nil, but not empty for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..badbdea 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -80,7 +80,7 @@ end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] [-n|--name name] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [--login y|n] [--update]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-g|--gen-sshkey] [-l|--login y|n] [-u|--update]\n" opts.banner += ' answerback-mail.rb [-f|--raw-email email_file] ' opts.banner += "[-g|--gen-sshkey] [--login y|n] [--update]\n" opts.banner += " -e|-f is required when applying account or updating account\n" @@ -121,12 +121,15 @@ options = OptionParser.new do |opts| end opts.on('-l value', '--login value', 'enable/disable login, value: y|n') do |value| - if value.downcase == 'y' + case value + when 'y', 'Y' conf_info['enable_login'] = true - elsif value.downcase == 'n' + when 'n', 'N' conf_info['enable_login'] = false else - raise 'invalid parameter, please use y|n' + puts "-l: bad value #{value}, please use y|n\n\n" + puts opts.banner + return 0 end end @@ -136,6 +139,12 @@ options = OptionParser.new do |opts| end end +if ARGV.empty? + ARGV << '-h' +elsif (['-e', '-f'] - ARGV).eql? ['-e', '-f'] + ARGV.clear + ARGV << '-h' +end options.parse!(ARGV) def apply_account(my_info, conf_info) @@ -152,7 +161,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 1

[PATCH compass-ci] params_group.rb: don't use os_version as filter when dimension is os
by Li Yuanchao 28 Nov '20

28 Nov '20

When compare with dimension 'os', different operating systems can not have the same os_version, so there is no data for compare. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/params_group.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/params_group.rb b/lib/params_group.rb index 48d651b..cde40af 100644 --- a/lib/params_group.rb +++ b/lib/params_group.rb @@ -98,6 +98,7 @@ def get_group_dimension_params(job, dimensions) dimension_list = [] dimensions.each do |dimension| dimension_list << all_group_params.delete(dimension) if all_group_params.key?(dimension) + all_group_params.delete('os_version') if dimension == 'os' end [all_group_params, dimension_list.join('|')] end -- 2.23.0

1 0

[PATCH compass-ci] lib/matrix2.rb: refactor performance optimization
by Lu Kaiyi 28 Nov '20

28 Nov '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 82cf033..2507b77 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -61,9 +61,7 @@ def create_stats(result_root) end def samples_fill_missing_zeros(value, size) - samples = value || [0] * size - samples << 0 while samples.size < size - samples + value.concat([0] * (size - value.size)) end # input: job_list -- 2.23.0

1 0

[PATCH compass-ci] scheduler: fix condition when setting default queue
by Ren Wen 28 Nov '20

28 Nov '20

Hash.*has_key?* returns true although there is no value of the key. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index e836d1f..55bac7f 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -247,7 +247,7 @@ class Job end private def set_queue - return if @hash.has_key?("queue") + return unless self["queue"].empty? # set default value self["queue"] = tbox_group -- 2.23.0

3 3

[PATCH compass-ci] answerback-email: fix bugs
by Luan Shengde 28 Nov '20

28 Nov '20

1. add default option for tool [why] the tool will show help message if no option specified [how] if no option specified, it will add '-h' as default option 2. fix undefined method 'empty' [why] the original data is defined nil, but not empty for my_email when execute answerback-email with no -e or -f, it will throw error as: raceback (most recent call last): 2: from ./answerback-email.rb:213:in `<main>' 1: from ./answerback-email.rb:184:in `send_account' ./answerback-email.rb:155:in `check_my_email': undefined method `empty?' for nil:NilClass (NoMethodError) [how] use '.nil?' instead of '.empty?' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c6bfdd3..1aaad75 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -136,7 +136,10 @@ options = OptionParser.new do |opts| end end -options.parse!(ARGV) +if ARGV.empty? + ARGV << '-h' + options.parse!(ARGV) +end def apply_account(my_info, conf_info) apply_info = {} @@ -152,7 +155,7 @@ def check_my_email(my_info) message += "use -e to add an email address for applying account\n" message += 'or use -f to add an email file' - raise message if my_info['my_email'].empty? + raise message if my_info['my_email'].nil? end def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) -- 2.23.0

1 0

[PATCH v2 compass-ci] scheduler: fix the paths to find cluster config file
by Ren Wen 27 Nov '20

27 Nov '20

To find cluster config file at *lab-z9/cluster/* first, *lkp-tests/cluster/* otherwise. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/constants.cr | 3 +++ src/scheduler/submit_job.cr | 14 +++++++++----- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index 1b37333..1afdda0 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -24,6 +24,9 @@ OS_HTTP_PORT = (ENV.has_key?("OS_HTTP_PORT") ? ENV["OS_HTTP_PORT"] : 8000).to_i3 SRV_OS = "/srv/os" SRV_INITRD = "/srv/initrd" +CCI_REPOS = "/c" +LAB_REPO = "lab-z9" + INITRD_HTTP_PREFIX = "http://#{INITRD_HTTP_HOST}:#{INITRD_HTTP_PORT}" OS_HTTP_PREFIX = "http://#{OS_HTTP_HOST}:#{OS_HTTP_PORT}" SCHED_HTTP_PREFIX = "http://#{SCHED_HOST}:#{SCHED_PORT}" diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 2836275..c60b99d 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -15,7 +15,7 @@ class Sched else cluster_config = get_cluster_config(cluster_file, job.lkp_initrd_user, - job.os_arch) + job.os_arch).not_nil! return submit_cluster_job(job, cluster_config) end rescue ex @@ -152,12 +152,16 @@ class Sched @es.set_job_content(job) end - # get cluster config using own lkp_src cluster file, - # a hash type will be returned def get_cluster_config(cluster_file, lkp_initrd_user, os_arch) lkp_src = Jobfile::Operate.prepare_lkp_tests(lkp_initrd_user, os_arch) - cluster_file_path = Path.new(lkp_src, "cluster", cluster_file) - return YAML.parse(File.read(cluster_file_path)) + + cluster_file_paths = [ + Path.new(CCI_REPOS, LAB_REPO, "cluster", cluster_file), + Path.new(lkp_src, "cluster", cluster_file) + ] + cluster_file_paths.each do |f| + return YAML.parse(File.read(f)) if File.file?(f) + end end def get_commit_date(job) -- 2.23.0

1 0