- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] bin/lkp-setup-rootfs: modify test-machine is not restarted
by Hu Xuejiao 11 Dec '20

11 Dec '20

[why] When force_reboot is not added to the job.yaml, the test-machine is "no KERNEL found". Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- bin/lkp-setup-rootfs | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index 0eabea75..cca1b53b 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -61,7 +61,10 @@ job_does_not_need_reboot() job_force_reboot() { - grep -q "^force_reboot: \+1$" $job + grep -q "^force_reboot: 1$" $job && return 0 + grep -q "^do_not_reboot: 1$" $job || return 0 + + return 1 } reboot_for_next_job() -- 2.23.0

1 0

[PATCH lkp-tests] tests/build-pkg: fix syntax errors caused by missing double quotes
by Lin Jiaxin 11 Dec '20

11 Dec '20

error: PKGBUILD.src: line 2: syntax error near unexpected token `&' PKGBUILD.src: line 2: `glassfish-5.1.0.zip::http://www.eclipse.org/downloads/download.php?file=/glassfish/glassfish-5.1.0.zip&r=1' PKGBUILD.src: line 3: glassfish5.service: command not found PKGBUILD.src: line 4: syntax error near unexpected token `)' PKGBUILD.src: line 4: `)' Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 8a138055..0b2794b1 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -112,7 +112,7 @@ replace_source() } url=$(echo "$url" | sed 's|https://|http://|g') - echo "$url" >> $1 + echo "\"$url\"" >> $1 done echo ")" >> $1 } -- 2.23.0

3 3

[PATCH v2 lkp-tests] tests/build-pkg: fix syntax errors caused by missing quotes
by Lin Jiaxin 11 Dec '20

11 Dec '20

error: PKGBUILD.src: line 2: syntax error near unexpected token `&' PKGBUILD.src: line 2: `glassfish-5.1.0.zip::http://www.eclipse.org/downloads/download.php?file=/glassfish/glassfish-5.1.0.zip&r=1' PKGBUILD.src: line 3: glassfish5.service: command not found PKGBUILD.src: line 4: syntax error near unexpected token `)' PKGBUILD.src: line 4: `)' Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 8a138055..aeaf1778 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -112,7 +112,7 @@ replace_source() } url=$(echo "$url" | sed 's|https://|http://|g') - echo "$url" >> $1 + echo \'$url\' >> $1 done echo ")" >> $1 } -- 2.23.0

1 0

[PATCH compass-ci 2/2] add options for compare_by_template
by Lu Kaiyi 11 Dec '20

11 Dec '20

if theme is json like below: compare -t compare_template.yaml --theme json display json format output result, otherwise, display table format output result by default. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare.rb | 4 ++-- lib/compare_matrixes.rb | 11 ++++++++--- sbin/compare | 2 +- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index f34e239..db264f8 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -96,11 +96,11 @@ end # unit: KB/s # -def compare_by_template(template) +def compare_by_template(template, options) template_params = load_template(template) groups_matrices = create_groups_matrices(template_params) compare_results = compare_metrics_values(groups_matrices) - show_compare_result(compare_results, template_params) + show_compare_result(compare_results, template_params, options) end def load_template(template) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index f7bb56a..db102f1 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -507,11 +507,16 @@ def get_dimensions_combination(dimension_list) dims end -def show_compare_result(metrics_compare_results, template_params) +def show_compare_result(metrics_compare_results, template_params, options) + theme = options[:theme] formatter = FormatEchartData.new(metrics_compare_results, template_params) echart_results = formatter.format_for_echart - table_data = FormatTableData.new(echart_results) - table_data.show_table + if theme == 'json' + print JSON.pretty_generate(echart_results) + else + table_data = FormatTableData.new(echart_results) + table_data.show_table + end end # Format Fields diff --git a/sbin/compare b/sbin/compare index c13ec1a..e8d1ac4 100755 --- a/sbin/compare +++ b/sbin/compare @@ -67,7 +67,7 @@ opt_parser.parse!(argv) options = { theme: colorful } if colorful if template - compare_by_template(template) + compare_by_template(template, options) elsif is_group compare_group(argv, dimensions, options) else -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib: friendly show compare template result in command line
by Lu Kaiyi 11 Dec '20

11 Dec '20

[why] friendly show compare template result in command line. [how] rehandle the json format result of "compare -t <compare_template.yaml>" with terminal-table in Ruby. [usage] compare -t <compare_template.yaml> [example] compare -t compare_template.yaml in z9 [input] a compare_template.yaml like below: compare_metrics: - fio.write_iops - fio.read_iops filter: suite: - fio-basic os_arch: - aarch64 - x86 compare_dimensions: - os: debian os_version: sid - os: openeuler os_version: 20.03 x_params: - bs - test_size title: Hackbench Performance Testing unit: KB/s [output] +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | Hackbench Performance Testing | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | fio.read_iops | 4k|1G | 4k|80G | 16k|1G | 32k|1G | 64k|1G | 128k|1G | 256k|1G | 512k|1G | 1024k|1G | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ | openeuler 20.03 average | 144076.29 | 11601.10 | 37865.30 | 21145.10 | 14010.34 | 6701.24 | 3205.08 | 1367.48 | 673.33 | | openeuler 20.03 standard_deviation | 195.00 | 0.00 | 214.00 | 205.00 | 188.00 | 183.00 | 180.00 | 191.00 | 191.00 | +------------------------------------+-----------+----------+----------+----------+----------+---------+---------+---------+----------+ +-------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | Hackbench Performance Testing | +-------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | fio.write_iops | 4k|1G | 16k|1G | 32k|1G | 64k|1G | 128k|1G | 256k|1G | 512k|1G | 1024k|1G | +-------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ | centos 7.6 average | 345243.03 | 142698.79 | 62108.35 | 34747.73 | 26330.19 | 10317.85 | 7471.71 | 3558.30 | | centos 7.6 standard_deviation | 97.00 | 95.00 | 122.00 | 125.00 | 100.00 | 130.00 | 101.00 | 103.00 | +-------------------------------+-----------+-----------+----------+----------+----------+----------+---------+----------+ Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare_data_format.rb | 53 ++++++++++++++++++++++++++++++++++++++ lib/compare_matrixes.rb | 3 ++- 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/lib/compare_data_format.rb b/lib/compare_data_format.rb index bf4013b..aa5179f 100644 --- a/lib/compare_data_format.rb +++ b/lib/compare_data_format.rb @@ -2,6 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'terminal-table' # ---------------------------------------------------------------------------------------------------- # format compare results for a specific format # @@ -66,6 +67,58 @@ class FormatEchartData end end +class FormatTableData + def initialize(result_hash) + @title = result_hash['title'] + @tables = result_hash['tables'] + end + + def show_table + @tables.each do |table_title, table| + @tb = Terminal::Table.new + set_field_names(table_title, table) + set_table_title + add_row(table) + set_align_column + print_table + end + end + + def set_table_title + @tb.title = @title + end + + def set_field_names(table_title, table) + field_names = [table_title] + field_names.concat(table['average']['source'][0]) + @tb.add_row(field_names) + @tb.add_separator + end + + def add_row(table) + row_names = %w[average standard_deviation] + row_names.each do |row_name| + row = table[row_name]['source'][1] + row_title = [row[0], row_name].join(' ') + format_data_row = row[1..-1] + format_data_row.map! { |data| format('%.2f', data) } + @tb.add_row([row_title, *format_data_row]) + end + end + + def set_align_column + @tb.number_of_columns.times do |index| + @tb.align_column(index + 1, :right) + end + @tb.align_column(0, :left) + end + + def print_table + puts @tb + puts + end +end + # input: x_params_list # eg: ["1G|4K", "1G|1024k", "1G|128k", 2G|4k] # output: diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 5293989..f7bb56a 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -510,7 +510,8 @@ end def show_compare_result(metrics_compare_results, template_params) formatter = FormatEchartData.new(metrics_compare_results, template_params) echart_results = formatter.format_for_echart - print JSON.pretty_generate(echart_results) + table_data = FormatTableData.new(echart_results) + table_data.show_table end # Format Fields -- 2.23.0

1 0

help star
by Wu Fengguang 11 Dec '20

11 Dec '20

Xueliang is our help star, thanks Xueliang! wfg@crystal ~/todo% bin/help-star people/*/credit/2020-12-0[34].md 22.7 Cao Xueliang 20.0 Wei Jihui 19.8 Wu Zhende 16.0 Luan Shengde 13.6 Wu Fengguang 13.3 Lu Weitao 11.9 Xiao Shenwei 11.0 Bai Jing 9.3 Yu Chuan 7.7 Zhang Yu 7.4 Ren Wen 6.7 Zhang Yale 6.7 Cui Yili 6.2 Xu Xijian 6.2 Lin Jiaxin 5.7 Liu Yinsi 5.6 Zhang Dewan 5.6 Wang Yong 5.1 Lu Kaiyi 4.4 Li Ping 4.2 Hu Xuejiao 3.8 Liu Shaofei 3.5 Du Kaitian 2.6 Sun Yukui 0.9 Li Yuanchao 0.0 Tong Qunfeng 0.0 Wang Chenglong 0.0 Wang Shuai 0.0 Cui Xucui

1 0

review star
by Wu Fengguang 11 Dec '20

11 Dec '20

Shenwei is our review star, thanks Shenwei! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 19 12 34 21 Xiao Shenwei 95 30 44 19 Wei Jihui 28 16 25 17 Liu Shaofei 79 13 16 16 Cao Xueliang 20 4 18 13 Wang Yong 41 16 36 10 Wu Fengguang 34 9 10 9 Zhang Yu 22 7 18 9 Luan Shengde 30 4 17 8 Yu Chuan 26 3 8 7 Sun Yukui 19 5 7 6 Wu Zhende 18 4 4 5 Cui Yili 8 8 8 4 Li Ping 33 6 5 4 Hu Xuejiao 63 7 11 4 Wang Chenglong 19 7 15 4 Lu Kaiyi 6 2 4 3 Xu Xijian 13 1 2 2 Lu Weitao 14 4 5 2 Lin Jiaxin 17 5 5 2 Liu Yinsi 15 1 1 1 Bai Jing 4 0 0 0 Li Yuanchao 33 2 0 0 Zhang Yale 2 0 0 0 Zhang Dewan 20 0 0 0 Ren Wen

1 0

git count
by Wu Fengguang 11 Dec '20

11 Dec '20

Zhende is our code star again, thanks Zhende! wfg@crystal ~/todo% bin/git-count 13 Wu Fengguang 12 Wu Zhende 9 Cao Xueliang 9 Zhang Yale 7 Hu Xuejiao 6 Bai Jing 6 Liu Shaofei 6 Lu Weitao 6 Liu Yinsi 6 Wei Jihui 6 Zhang Yu 6 Luan Shengde 5 Yu Chuan 5 Cui Yili 5 Wang Chenglong 5 Li Yuanchao 5 Lu Kaiyi 4 Wang Yong 4 Lin Jiaxin 3 Sun Yukui 3 Ren Wen 2 Xu Xijian 2 Xiao Shenwei 1 Li Ping 0 Tong Qunfeng 0 Du Kaitian 0 Zhang Dewan

1 0

[PATCH compass-ci] logging: fix time difference of eight hours
by Wu Zhende 11 Dec '20

11 Dec '20

If the time zone information is not contained, the ES saves time as the UTC time by default. As a result, the kibana display time is eight hours later. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/master-fluentd/docker-fluentd.conf | 2 +- src/lib/json_logger.cr | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/container/master-fluentd/docker-fluentd.conf b/container/master-fluentd/docker-fluentd.conf index b3daeb4..7bf2229 100644 --- a/container/master-fluentd/docker-fluentd.conf +++ b/container/master-fluentd/docker-fluentd.conf @@ -11,7 +11,7 @@ @type record_transformer enable_ruby <record> - time ${time.strftime('%Y-%m-%dT%H:%M:%S')} + time ${time.strftime('%Y-%m-%dT%H:%M:%S.%3N+0800')} </record> </filter> diff --git a/src/lib/json_logger.cr b/src/lib/json_logger.cr index dd87020..6d61197 100644 --- a/src/lib/json_logger.cr +++ b/src/lib/json_logger.cr @@ -18,7 +18,7 @@ class JSONLogger < Logger Logger::Formatter.new do | severity, datetime, progname, msg, io| get_env_info(@env.as(HTTP::Server::Context)) if @env level_num = severity.to_i32 - datetime = datetime.to_s("%Y-%m-%dT%H:%M:%S") + datetime = datetime.to_s("%Y-%m-%dT%H:%M:%S.%3N+0800") logger_hash = JSON.parse(%({"level_num": #{level_num}, "level": "#{severity}", "time": "#{datetime}" -- 2.23.0

2 1

[PATCH lkp-tests] fix: depends/lkp: add ntpdate server
by Wang Chenglong 11 Dec '20

11 Dec '20

[Why] In $LKP_SRC/lib/bootstrap.s use ntpdate command to synchronize time. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- distro/depends/lkp | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/lkp b/distro/depends/lkp index 6c8b6bf2..750b5c59 100644 --- a/distro/depends/lkp +++ b/distro/depends/lkp @@ -4,3 +4,4 @@ time kmod gzip which +ntpdate -- 2.23.0

1 0

[PATCH compass-ci 1/2] scheduler: subqueue: consume subqueue
by Ren Wen 11 Dec '20

11 Dec '20

Find subqueues related to the host default queues of all users, and consume the job from random one. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/find_job_boot.cr | 20 +++++++++++++++++++- src/scheduler/redis_client.cr | 4 ++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 6f51640..77071db 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -56,7 +56,25 @@ class Sched queues << item.strip end - return rand_queues(queues) + user_queues = [] of String + queues.each do |queue| + matched_queues = @redis.keys("*/sched/#{queue}*") + matched_queues.each do |mq| + user_queues << "#{mq}" + end + end + + user_queues = Set.new(user_queues) + user_queues = user_queues.map do |queue| + if queue =~ /sched\/(.+)\/ready$/ + $1 + else + "" + end + end + user_queues.reject!(&.empty?) + + return rand_queues(user_queues) end def get_job_from_queues(queues, testbox) diff --git a/src/scheduler/redis_client.cr b/src/scheduler/redis_client.cr index b816d67..1253108 100644 --- a/src/scheduler/redis_client.cr +++ b/src/scheduler/redis_client.cr @@ -16,6 +16,10 @@ class Redis::Client @client = Redis::PooledClient.new(host: host, port: port, pool_size: 25, pool_timeout: 0.01) end + def keys(pattern) + @client.keys(pattern) + end + def hash_set(key : String, field, value) @client.hset(key, field.to_s, value.to_s) end -- 2.23.0

2 2

[PATCH v3 compass-ci 2/2] scheduler: subqueue: change queue to queue/subqueue
by Ren Wen 11 Dec '20

11 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/close_job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index d071d69..2c33ffd 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -14,7 +14,7 @@ class Sched end response = @task_queue.hand_over_task( - "sched/#{job.queue}", "extract_stats", job_id + "sched/#{job.queue}/#{job.subqueue}", "extract_stats", job_id ) if response[0] != 201 raise "#{response}" -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] scheduler: subqueue: submit user's job to subqueue
by Ren Wen 11 Dec '20

11 Dec '20

Every user has a subqueue, user's email by default. Queue will be "queue/subqueue" when submitting job. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 6 ++++++ src/scheduler/submit_job.cr | 2 +- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 7ded6d8..0083a7d 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -80,6 +80,7 @@ class Job testbox lab queue + subqueue initrd_pkg initrd_deps initrds_uri @@ -161,6 +162,7 @@ class Job set_lkp_server() set_sshr_info() set_queue() + set_subqueue() end private def set_kernel @@ -266,6 +268,10 @@ class Job end end + private def set_subqueue + self["subqueue"] = self["my_email"] + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index f93ad2b..3909cb7 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -107,7 +107,7 @@ class Sched # success: [{"job_id" => job_id, "message" => "", job_state => "submit"}] # failure: [{"job_id" => "0", "message" => err_msg, job_state => "submit"}] def submit_single_job(job) - queue = job.queue + queue = "#{job.queue}/#{job.subqueue}" # only single job will has "idle job" and "execute rate limiter" if job["idle_job"].empty? -- 2.23.0

1 0

Re: [PATCH v4 compass-ci] doc/job: add result_root.md
by Zhang Yu 11 Dec '20

11 Dec '20

>>+ result_root: /result/host-info/2020-12-09/vm-2p8g/openeuler-20.03-aarch64/crystal.166995 > > the example missing the field $pp_params. good Thanks, Zhangyu > >>-- >>2.23.0 >>

1 0

[PATCH compass-ci] src/scheduler: fix ameba warning
by Zhang Yu 11 Dec '20

11 Dec '20

Inspecting 1 file. create_job_cpio.cr:83:35 [C] Layout/TrailingWhitespace: Trailing whitespace detected > job_content.delete("job2sh") ^ Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- src/scheduler/create_job_cpio.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/scheduler/create_job_cpio.cr b/src/scheduler/create_job_cpio.cr index cfa40ab..1101a0f 100644 --- a/src/scheduler/create_job_cpio.cr +++ b/src/scheduler/create_job_cpio.cr @@ -80,7 +80,7 @@ class Sched # put job2sh in an array if job_content.has_key?("job2sh") tmp_job_sh_content = job_content["job2sh"] - job_content.delete("job2sh") + job_content.delete("job2sh") job_sh_array = [] of JSON::Any tmp_job_sh_content.as_h.each do |_key, val| -- 2.23.0

1 0

[PATCH v4 compass-ci] doc/job: add result_root.md
by Zhang Yu 11 Dec '20

11 Dec '20

Add a document for introducing the result_root Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- doc/job/result_root.md | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 doc/job/result_root.md diff --git a/doc/job/result_root.md b/doc/job/result_root.md new file mode 100644 index 0000000..c2b82c5 --- /dev/null +++ b/doc/job/result_root.md @@ -0,0 +1,10 @@ +# result_root + +Meaning: +- When a test job is executed, some result files will be generated and stored in the "result_root" + directory, you can check the result file in the web page or manually go to the "result_root" + directory for a check +- The default value of this field is: + result_root: /result/$suite/$submit_date/$tbox_group/$rootfs/$pp_params/$id + Here is an example of this field: + result_root: /result/host-info/2020-12-09/vm-2p8g/openeuler-20.03-aarch64/crystal.166995 -- 2.23.0

1 0

[PATCH v3 compass-ci] doc/job: add result_root.md
by Zhang Yu 11 Dec '20

11 Dec '20

Add a document for introducing the result_root Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- doc/job/result_root.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/job/result_root.md diff --git a/doc/job/result_root.md b/doc/job/result_root.md new file mode 100644 index 0000000..c3e0a01 --- /dev/null +++ b/doc/job/result_root.md @@ -0,0 +1,8 @@ +# result_root + +Meaning: +- When a test job is executed, some result file will generated and stored in the "result_root" + directory, you can check the result file in the web page or manually go to the "result_root" + directory for a check +- The default value of this field is: + result_root: "/result/#{suite}/#{submit_date}/#{tbox_group}/#{rootfs}"/#{pp_params}/#{id}" -- 2.23.0

2 2

[PATCH v2 compass-ci] container: add prompt message
by Bai Jing 11 Dec '20

11 Dec '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- container/conserver/generate_conserver.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/container/conserver/generate_conserver.rb b/container/conserver/generate_conserver.rb index 4e923c2..a0ee23b 100755 --- a/container/conserver/generate_conserver.rb +++ b/container/conserver/generate_conserver.rb @@ -12,9 +12,12 @@ def generate_conserver(lab) return unless File.exist?('conserver-head.cf') FileUtils.cp('conserver-head.cf', 'conserver.cf') - + host_dir = "#{CCI_REPOS}/lab-#{lab}/hosts/" - return unless Dir.exist?(host_dir) + unless Dir.exist?(host_dir) + puts "please check whether the #{host_dir} directory exists " + return + end Dir.each_child(host_dir) do |host| ipmi_ip = YAML.load_file("#{host_dir}#{host}")['ipmi_ip'] -- 2.23.0

1 0

[PATCH compass-ci] How to add a machine to compass-ci
by Zhang Dewan 10 Dec '20

10 Dec '20

Signed-off-by: Zhang Dewan <513619464(a)qq.com> How a new machine add to compass-ci Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- ...00\350\210\254\350\277\207\347\250\213.md" | 54 +++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 "doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" diff --git "a/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" "b/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" new file mode 100644 index 0000000..340ffea --- /dev/null +++ "b/doc/manual/\346\267\273\345\212\240\346\234\272\345\231\250\345\210\260compass-ci\347\232\204\344\270\200\350\210\254\350\277\207\347\250\213.md" @@ -0,0 +1,54 @@ + +��--�� + +## ��: +- ��compass-ci +- ��scheduler�� + +## �� +### 1. �� +��z9��redis-db. + +x.x.x.x:xxxx> `hgetall "sched/mac2host"` # ��redis, �� + +### 2. �� +��scheduler�� +`curl -X PUT "http://${SCHED_HOST:-172.17.0.1}:${SCHED_PORT:-3000}/set_host_mac?hostname=${hostname}&mac=${mac}"` +`curl -X PUT "http://${SCHED_HOST:-172.17.0.1}:${SCHED_PORT:-3000}/set_host2queues?host=${hostname}&queues=${api_queues}"` + +��mac��hostname��api_queues + +#### 2.1 �� +##### 2.1.1 mac +��mac�� + +###### 2.1.1.1 ��mac��TaiShan�� +- ��bmc��--��--��--��--��--port1�� +- ��bios��Advanced--Network Device List�� +- ��bmc�� +`ipmitool -I lanplus -H $cur_ip -U $iBMC_user -P $iBMC_passwd raw 0x30 0x90 0x01 0x03 0x0${nic_idx} 0x00|awk '{print $5,$6,$7,$8,$9,$10}' |sed 's/\s/\-/g'` +- ��os�� +`ip link | grep "^2:" -A 1| grep link |awk '{print $2}` + +##### 2.1.2 hostname +��mac��hostname�� + +###### 2.1.2.1 ��hostname��Taishan�� +` taishan200-2280-${cpuNum}s${coreNum}p-${memSize:-256}g--${aN}` +> taishan200-2280 �� +> cpuNum cpu�� +> coreNum cpu�� +> memSize �� +> aN �� + +#### 2.1.3 api_queues +��hostname�� +��taishan200-2280-2s64p-256g--a100, ��api_queues��taishan200-2280-2s64p-256g + +## �� +### 1. �� +��pxe��scheduler�� + +### 2. ��PXE�� +- ��pxe�� +- �� -- 2.23.0

2 1

[PATCH lkp-tests] bisect: bisect test use top level key
by Cao Xueliang 10 Dec '20

10 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 2 -- tests/bisect | 6 +++--- 2 files changed, 3 insertions(+), 5 deletions(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 8b0001df..5fab30bf 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -2,5 +2,3 @@ suite: bisect docker_image: delimiter:latest testbox: dc-1g--delimiter bisect: - job_id: - error_id: diff --git a/tests/bisect b/tests/bisect index ab280b95..33303296 100755 --- a/tests/bisect +++ b/tests/bisect @@ -1,13 +1,13 @@ #!/usr/bin/env ruby -# - job_id -# - error_id +# bad_job_id +# error_id # SPDX-License-Identifier: MulanPSL-2.0+ # frozen_string_literal: true require_relative "#{ENV['CCI_SRC']}/lib/git_bisect" require_relative "#{ENV['CCI_SRC']}/lib/mail_bisect_result" -task = {'job_id' => ENV['job_id'], 'error_id' => ENV['error_id']} +task = {'bad_job_id' => ENV['bad_job_id'], 'error_id' => ENV['error_id']} gb = GitBisect.new(task) result = gb.find_first_bad_commit -- 2.23.0

1 0

[PATCH compass-ci] container: add prompt message
by Bai Jing 10 Dec '20

10 Dec '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- container/conserver/generate_conserver.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/container/conserver/generate_conserver.rb b/container/conserver/generate_conserver.rb index 4e923c2..8914247 100755 --- a/container/conserver/generate_conserver.rb +++ b/container/conserver/generate_conserver.rb @@ -12,9 +12,12 @@ def generate_conserver(lab) return unless File.exist?('conserver-head.cf') FileUtils.cp('conserver-head.cf', 'conserver.cf') - + host_dir = "#{CCI_REPOS}/lab-#{lab}/hosts/" - return unless Dir.exist?(host_dir) + unless Dir.exist?(host_dir) do + puts "please check whether the #{host_dir} directory exists " + return + end Dir.each_child(host_dir) do |host| ipmi_ip = YAML.load_file("#{host_dir}#{host}")['ipmi_ip'] -- 2.23.0

1 0

[PATCH v5 lkp-tests] lkp-tests/tests: add rpmbuild-pkg to build RPM
by Zhang Yale 10 Dec '20

10 Dec '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/rpmbuild-pkg | 59 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) create mode 100755 tests/rpmbuild-pkg diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg new file mode 100755 index 00000000..156ca0f6 --- /dev/null +++ b/tests/rpmbuild-pkg @@ -0,0 +1,59 @@ +#!/bin/bash +# - rpm_repo + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/upload.sh + +[ -n "$rpm_repo" ] || die "rpm_repo is empty" +benchmark=${rpm_repo##*/} +sync_dest="/initrd/rpmbuild-pkg/${os_mount}/${os}/${os_arch}/${os_version}/${benchmark}" + +init_workspace() +{ + # generate workspace in ${HOME} + rpmdev-setuptree +} + +get_pkgfile() +{ + curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ + -d '{"git_repo": "'${rpm_repo}'", "git_command": ["git-show", "HEAD:'$1'"]}' -o "${2}" +} + +download_rpm_repo() +{ + filelist=$(curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ + -d '{"git_repo": "'${rpm_repo}'", "git_command": ["git-ls-files", "."]}') + for pkgfile in ${filelist[*]} + do + local dir="SOURCES" + echo $pkgfile | egrep "\.spec$" && dir="SPECS" + get_pkgfile "$pkgfile" "${HOME}/rpmbuild/${dir}/$pkgfile" + done +} + +build_rpm() +{ + spec_dir=${HOME}/rpmbuild/SPECS/$benchmark.spec + sed -i 's/^$Source[^ ]*:[ \t]*$https/\1http/g' `grep http -rl $benchmark.spec` + # Install build depends + yum-builddep -y $spec_dir + # Download tar.gz to default path ${HOME}/rpmbuild/SOURCE + spectool -g -R $spec_dir + # Building rpm or srpm packages + rpmbuild -ba $spec_dir +} + +upload_rpm_pkg() +{ + # rpm package will be generated in "${HOME}/rpmbuild/SRPMS" and "${HOME}/rpmbuild/RPMS" + for file in $(find ${HOME}/rpmbuild/ -type f -name "*.rpm") + do + upload_one_curl ${file} ${sync_dest} + done +} + +init_workspace +download_rpm_repo +build_rpm +upload_rpm_pkg -- 2.23.0

1 0

[PATCH v2 lkp-tests 3/3] iso2rootfs: allow install some packages to result rootfs
by Yu Chuan 10 Dec '20

10 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 4 ++++ tests/iso2rootfs | 34 ++++++++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 6 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index c67faf44147d..a84f7583de94 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -34,6 +34,10 @@ iso2rootfs: # - no_fstab: will comment all line in /etc/fstab. config_rootfs: + ## install pkgs for result rootfs + ## - example: vim, git, xxx + rootfs_install_pkgs: + ################# # dailybuild related fields ################# diff --git a/tests/iso2rootfs b/tests/iso2rootfs index cd64044f0833..8625b6484d89 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -203,17 +203,24 @@ get_cgz() tar -xf "${cgz_path_t}" -C "${CGZ_PATH}" } -install_pkgs() +get_pkg_installer() { - local pkg_installer - command -v yum && pkg_installer="yum" + local installer + + has_cmd yum && installer=yum + has_cmd apt && installer=apt - [[ -n "$pkg_installer" ]] || die "cannot find pkg installer." + [ -n "$installer" ] || die "cannot find pkg installer." + echo $installer +} + +install_pkgs() +{ + local installer=$(get_pkg_installer) - local pt for pt do - "$pkg_installer" install -y "$pt" || die "cannot install pkg: $pt." + "$installer" install -y "$pt" || die "cannot install pkg: $pt." done } @@ -367,11 +374,26 @@ disable_fstab() sed -i 's/^$[^#]$/# \1/g' "${ROOTFS_DES_DIR}/etc/fstab" } +install_pkgs_for_rootfs() +{ + [ -n "$rootfs_install_pkgs" ] || return 0 + + local installer=$(get_pkg_installer) + + if [ "$installer" == "yum" ]; then + "$installer" install -y --skip-broken --installroot=${ROOTFS_DES_DIR} ${rootfs_install_pkgs//,/ } + elif [ "$installer" == "apt" ]; then + "$installer" install -y --ignore-missing --fix-missing --fix-broken -o RootDir=${ROOTFS_DES_DIR} ${rootfs_install_pkgs//,/ } + fi +} + config_rootfs() { config_dns_resolver disable_selinux disable_fstab + + install_pkgs_for_rootfs } ############ test rootfs ############ -- 2.23.0

1 0

[PATCH v2 lkp-tests 2/3] iso2rootfs: add three required fields when submit test job.yaml
by Yu Chuan 10 Dec '20

10 Dec '20

[Why] Since the ability `verify account information` was added on submit job.yaml, so add three required fields when submit test job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 52 ++++++++++++++++++++++++++++++++++---------- tests/iso2rootfs | 14 ++++++++++-- 2 files changed, 52 insertions(+), 14 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 9123bef24ae5..c67faf44147d 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,35 +1,63 @@ suite: iso2rootfs category: benchmark iso2rootfs: + ################# # iso related fields to be used to generate rootfs + ################# + iso_os: iso_arch: iso_version: + ################# # place the result rootfs related fields - ## 1. result rootfs will be placed in the following localtion on the - ## remote file server: - ## - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ - ## 2. remote file server protocols current supported: - ## - nfs - ## - cifs + ################# + + # 1. Result rootfs will be placed in the following location on the + # remote file server: + # - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ + # 2. Remote file server protocols current supported: + # - nfs + # - cifs rootfs_protocol: rootfs_server: rootfs_path: + ################# # config rootfs related fields - ## supported fields: - ## - dns: will config /etc/resolv.conf. - ## - no_selinux: will disable selinux in /etc/selinux/config. - ## - no_fstab: will comment all line in /etc/fstab. + ################# + + # you can config add some configurations of the result rootfs. + # supported fields: + # - dns: will config /etc/resolv.conf. + # - no_selinux: will disable selinux in /etc/selinux/config. + # - no_fstab: will comment all line in /etc/fstab. config_rootfs: + ################# # dailybuild related fields + ################# + + # dailybuild_iso_url_file: + # - The `dailybuild iso url file` content is the url of a iso. + # - The iso checksum file also exists on the network, and the checksum file path is "{dailybuild_iso_url}.check256sum" + # - demo: + # root@localhost ~% curl http://1.1.1.1/dailybuilds/openEuler-20.03-LTS-aarch64-LTS/release_iso + # http://1.1.1.1//dailybuilds/1970-01-01-00-00-00/openEuler-20.03-LTS-aarch64… dailybuild_iso_url_file: + ################# # submit test yaml related fields - ## 1. you can add as many jobs as you like. - ## 2. the following three fields is requied for every test job. + ################# + + # three required fields when submit test yaml + submit_email: + submit_name: + submit_uuid: + + # submit target tests yaml + ## 1. You can add as many jobs as you like. + ## 2. The following three fields is required for every test job. test1_yaml: test1_os_mount: test1_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index ce02080875ce..cd64044f0833 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -448,11 +448,21 @@ EOF ############ submit test job yaml ############ generate_global_yaml() { + [ -n "${submit_email}" ] || die "cannot find value of var: submit_email" + [ -n "${submit_name}" ] || die "cannot find value of var: submit_name" + [ -n "${submit_uuid}" ] || die "cannot find value of var: submit_uuid" + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" mkdir -p "$(dirname "${config_yaml}")" - echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" - echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" + + cat <<-EOF > "${config_yaml}" + SCHED_HOST: ${SCHED_HOST} + SCHED_PORT: ${SCHED_PORT} + my_email: ${submit_email} + my_name: ${submit_name} + my_uuid: ${submit_uuid} + EOF } generate_submit_append_str() -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/3] iso2rootfs: result rootfs support disable fstab
by Yu Chuan 10 Dec '20

10 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 1 + tests/iso2rootfs | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 2c4d3a93ff62..9123bef24ae5 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -21,6 +21,7 @@ iso2rootfs: ## supported fields: ## - dns: will config /etc/resolv.conf. ## - no_selinux: will disable selinux in /etc/selinux/config. + ## - no_fstab: will comment all line in /etc/fstab. config_rootfs: # dailybuild related fields diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 9adbc930b48f..ce02080875ce 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -361,10 +361,17 @@ disable_selinux() sed -i 's/^SELINUX=.*/SELINUX=disabled/g' "${ROOTFS_DES_DIR}/etc/selinux/config" } +disable_fstab() +{ + echo "${config_rootfs}" | grep -qw 'no_fstab' || return 0 + sed -i 's/^$[^#]$/# \1/g' "${ROOTFS_DES_DIR}/etc/fstab" +} + config_rootfs() { config_dns_resolver disable_selinux + disable_fstab } ############ test rootfs ############ -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] scheduler: subqueue: consume jobs from subqueue
by Ren Wen 10 Dec '20

10 Dec '20

Find subqueues related to the host default queues of all users, and consume according to random sequence. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/constants.cr | 2 ++ src/scheduler/find_job_boot.cr | 11 ++++++++++- src/scheduler/redis_client.cr | 4 ++++ 3 files changed, 16 insertions(+), 1 deletion(-) diff --git a/src/scheduler/constants.cr b/src/scheduler/constants.cr index b2ea831..1146b21 100644 --- a/src/scheduler/constants.cr +++ b/src/scheduler/constants.cr @@ -1,6 +1,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +require "../taskqueue/constants" + JOB_REDIS_HOST = "172.17.0.1" JOB_REDIS_PORT = 6379 JOB_REDIS_PORT_DEBUG = 6380 diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 3e33604..fb94e91 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -56,7 +56,16 @@ class Sched queues << item.strip end - return rand_queues(queues) + sub_queues = [] of String + queues.each do |queue| + matched_queues = @redis.keys("#{QUEUE_NAME_BASE}/sched/#{queue}/*/ready") + matched_queues.each do |mq| + match_data = "#{mq}".match(%r(^#{QUEUE_NAME_BASE}/sched/(#{queue}/.+)/ready$)) + sub_queues << match_data[1] if match_data + end + end + + return rand_queues(sub_queues) end def get_job_from_queues(queues, testbox) diff --git a/src/scheduler/redis_client.cr b/src/scheduler/redis_client.cr index b816d67..1253108 100644 --- a/src/scheduler/redis_client.cr +++ b/src/scheduler/redis_client.cr @@ -16,6 +16,10 @@ class Redis::Client @client = Redis::PooledClient.new(host: host, port: port, pool_size: 25, pool_timeout: 0.01) end + def keys(pattern) + @client.keys(pattern) + end + def hash_set(key : String, field, value) @client.hset(key, field.to_s, value.to_s) end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] scheduler: subqueue: submit user's job to subqueue
by Ren Wen 10 Dec '20

10 Dec '20

Every user has a subqueue, user's email by default. Queue will be "queue/subqueue" when submitting job. Example: queue : vm-2p8g.aarch64 subqueue: zs(a)xxx.com will submit to: vm-2p8g.aarch64/zs(a)xxx.com Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/job.cr | 9 +++++++++ src/scheduler/submit_job.cr | 2 ++ 2 files changed, 11 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 7ded6d8..5e56fff 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -80,6 +80,7 @@ class Job testbox lab queue + subqueue initrd_pkg initrd_deps initrds_uri @@ -161,6 +162,7 @@ class Job set_lkp_server() set_sshr_info() set_queue() + set_subqueue() end private def set_kernel @@ -266,6 +268,13 @@ class Job end end + private def set_subqueue + return unless self["subqueue"].empty? + + # set default subqueue + self["subqueue"] = self["my_email"] + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index f93ad2b..7841736 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -108,6 +108,8 @@ class Sched # failure: [{"job_id" => "0", "message" => err_msg, job_state => "submit"}] def submit_single_job(job) queue = job.queue + subqueue = job.subqueue + queue += "/#{subqueue}" # only single job will has "idle job" and "execute rate limiter" if job["idle_job"].empty? -- 2.23.0

1 0

[PATCH compass-ci] minio/start: add localtime
by Cui Yili 10 Dec '20

10 Dec '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/minio/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/minio/start b/container/minio/start index 348bff7..1798cfa 100755 --- a/container/minio/start +++ b/container/minio/start @@ -9,6 +9,7 @@ cmd=( docker run -d -p 11321:9000 --name minio + -v /etc/localtime:/etc/localtime:ro -v /srv/minio/:/srv/minio/ -e "MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY}" -e "MINIO_SECRET_KEY=${MINIO_SECRET_KEY}" -- 2.23.0

1 0

Re: [PATCH lkp-tests] jobs: rename pub_key to ssh_pub_key
by Zhang Yale 10 Dec '20

10 Dec '20

On Thu, Dec 10, 2020 at 05:10:47PM +0800, Li Ping wrote: >On Thu, Dec 10, 2020 at 03:31:54PM +0800, Zhang Yale wrote: >>As top level job key, the name should be more exact. > > if you make changes in these files, > you should also synchronize the changes in those > documents. > and remind everyone. > > Thanks, > Ping The relevant document has been modified by someone else. Thanks, Yale > >> >>Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> >>--- >> jobs/borrow-10d.yaml | 2 +- >> jobs/borrow-1d.yaml | 2 +- >> jobs/borrow-1h.yaml | 2 +- >> 3 files changed, 3 insertions(+), 3 deletions(-) >> >>diff --git a/jobs/borrow-10d.yaml b/jobs/borrow-10d.yaml >>index a2b50025..e0e35e17 100644 >>--- a/jobs/borrow-10d.yaml >>+++ b/jobs/borrow-10d.yaml >>@@ -1,7 +1,7 @@ >> suite: borrow >> testcase: borrow >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>diff --git a/jobs/borrow-1d.yaml b/jobs/borrow-1d.yaml >>index 7767f2d7..c31d7ebd 100644 >>--- a/jobs/borrow-1d.yaml >>+++ b/jobs/borrow-1d.yaml >>@@ -3,7 +3,7 @@ testcase: borrow >> >> timeout: 1d >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml >>index f2fedd20..d6b72c43 100644 >>--- a/jobs/borrow-1h.yaml >>+++ b/jobs/borrow-1h.yaml >>@@ -1,7 +1,7 @@ >> suite: borrow >> testcase: borrow >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>-- >>2.23.0 >>

1 0

Re: [PATCH lkp-tests] jobs: rename pub_key to ssh_pub_key
by Zhang Yale 10 Dec '20

10 Dec '20

On Thu, Dec 10, 2020 at 05:10:47PM +0800, Li Ping wrote: >On Thu, Dec 10, 2020 at 03:31:54PM +0800, Zhang Yale wrote: >>As top level job key, the name should be more exact. > > if you make changes in these files, > you should also synchronize the changes in those > documents. > and remind everyone. > > Thanks, > Ping Ok, I got it. Thanks, Yale > >> >>Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> >>--- >> jobs/borrow-10d.yaml | 2 +- >> jobs/borrow-1d.yaml | 2 +- >> jobs/borrow-1h.yaml | 2 +- >> 3 files changed, 3 insertions(+), 3 deletions(-) >> >>diff --git a/jobs/borrow-10d.yaml b/jobs/borrow-10d.yaml >>index a2b50025..e0e35e17 100644 >>--- a/jobs/borrow-10d.yaml >>+++ b/jobs/borrow-10d.yaml >>@@ -1,7 +1,7 @@ >> suite: borrow >> testcase: borrow >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>diff --git a/jobs/borrow-1d.yaml b/jobs/borrow-1d.yaml >>index 7767f2d7..c31d7ebd 100644 >>--- a/jobs/borrow-1d.yaml >>+++ b/jobs/borrow-1d.yaml >>@@ -3,7 +3,7 @@ testcase: borrow >> >> timeout: 1d >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml >>index f2fedd20..d6b72c43 100644 >>--- a/jobs/borrow-1h.yaml >>+++ b/jobs/borrow-1h.yaml >>@@ -1,7 +1,7 @@ >> suite: borrow >> testcase: borrow >> >>-pub_key: <%= >>+ssh_pub_key: <%= >> begin >> File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp >> rescue >>-- >>2.23.0 >>

1 0

[PATCH lkp-tests] lib/job.rb: delete useless var
by Wei Jihui 10 Dec '20

10 Dec '20

@job2 is {} all the time, so it is useless. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 5 ----- 1 file changed, 5 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index e94b6c85..b41d55a7 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -433,7 +433,6 @@ class Job i = include_files job = deepcopy(@job) - revise_hash(job, deepcopy(@job2), true) job['___'] = nil @jobx = job if @job.include?(:expand_params) @@ -498,8 +497,6 @@ class Job return unless first_time - revise_hash(@job, @job2, true) - return if @overrides.empty? key = comment_to_symbol('user overrides') @@ -729,7 +726,6 @@ class Job def each_jobs(&block) each_job_init job = deepcopy(@job) - @job2 = {} load_defaults load_hosts_config each_job_init @@ -738,7 +734,6 @@ class Job @load_hosts_done = false @job = deepcopy(job) @job.merge!(hash) - @job2 = {} load_defaults load_hosts_config each_job_init -- 2.23.0

1 0

[PATCH lkp-tests] jobs: rename pub_key to ssh_pub_key
by Zhang Yale 10 Dec '20

10 Dec '20

As top level job key, the name should be more exact. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/borrow-10d.yaml | 2 +- jobs/borrow-1d.yaml | 2 +- jobs/borrow-1h.yaml | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/jobs/borrow-10d.yaml b/jobs/borrow-10d.yaml index a2b50025..e0e35e17 100644 --- a/jobs/borrow-10d.yaml +++ b/jobs/borrow-10d.yaml @@ -1,7 +1,7 @@ suite: borrow testcase: borrow -pub_key: <%= +ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/jobs/borrow-1d.yaml b/jobs/borrow-1d.yaml index 7767f2d7..c31d7ebd 100644 --- a/jobs/borrow-1d.yaml +++ b/jobs/borrow-1d.yaml @@ -3,7 +3,7 @@ testcase: borrow timeout: 1d -pub_key: <%= +ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml index f2fedd20..d6b72c43 100644 --- a/jobs/borrow-1h.yaml +++ b/jobs/borrow-1h.yaml @@ -1,7 +1,7 @@ suite: borrow testcase: borrow -pub_key: <%= +ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue -- 2.23.0

1 0

[PATCH compass-ci] container/defconfig: fix bug
by Luan Shengde 10 Dec '20

10 Dec '20

[why] sometimes we may add new yaml file at $HOME/.config/compass-ci/defaults/ if we reference the defconfig.rb before we put anything to the file, error may occur: TypeError: no implicit conversion of false into Hash if the file is not in a standard format of yaml file, error may occur: TypeError: no implicit conversion of String into Hash [how] add update {} for the hash if YAML.load_file() end with errors Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/defconfig.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index c0768ca..d353bb3 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -9,7 +9,7 @@ def cci_defaults hash = {} Dir.glob(['/etc/compass-ci/defaults/*.yaml', "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| - hash.update YAML.load_file(file) + hash.update YAML.load_file(file) || {} end hash end -- 2.23.0

1 0

[PATCH lkp-tests 3/3] iso2rootfs: allow install some packages to result rootfs
by Yu Chuan 10 Dec '20

10 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 4 ++++ tests/iso2rootfs | 34 ++++++++++++++++++++++++++++------ 2 files changed, 32 insertions(+), 6 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index c67faf44147d..a84f7583de94 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -34,6 +34,10 @@ iso2rootfs: # - no_fstab: will comment all line in /etc/fstab. config_rootfs: + ## install pkgs for result rootfs + ## - example: vim, git, xxx + rootfs_install_pkgs: + ################# # dailybuild related fields ################# diff --git a/tests/iso2rootfs b/tests/iso2rootfs index dbb409d16923..c8c9318436c9 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -203,17 +203,24 @@ get_cgz() tar -xf "${cgz_path_t}" -C "${CGZ_PATH}" } -install_pkgs() +get_pkg_installer() { - local pkg_installer - command -v yum && pkg_installer="yum" + local installer + + [ -d "/etc/yum/" ] && installer=yum + [ -d "/etc/apt/" ] && installer=apt - [[ -n "$pkg_installer" ]] || die "cannot find pkg installer." + [ -n "$installer" ] || die "cannot find pkg installer." + echo $installer +} + +install_pkgs() +{ + local installer=$(get_pkg_installer) - local pt for pt do - "$pkg_installer" install -y "$pt" || die "cannot install pkg: $pt." + "$installer" install -y "$pt" || die "cannot install pkg: $pt." done } @@ -367,11 +374,26 @@ disable_fstab() sed -i 's/^$[^#]$/#\1/g' "${ROOTFS_DES_DIR}/etc/fstab" } +rootfs_install_pkgs() +{ + [ -n "$rootfs_install_pkgs" ] || return 0 + + local installer=$(get_pkg_installer) + + if [ "$installer" == "yum" ]; then + "$installer" install -y --installroot=${ROOTFS_DES_DIR} ${rootfs_install_pkgs//,/ } + elif [ "$installer" == "apt" ]; then + "$installer" install -y -o RootDir=${ROOTFS_DES_DIR} ${rootfs_install_pkgs//,/ } + fi +} + config_rootfs() { config_dns_resolver disable_selinux disable_fstab + + rootfs_install_pkgs } ############ test rootfs ############ -- 2.23.0

2 2

[PATCH v2 lkp-tests 1/3] sbin/set-env.sh: modify the method of adding env-variables
by Hu Xuejiao 10 Dec '20

10 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/set-env.sh | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/sbin/set-env.sh b/sbin/set-env.sh index a8b161fa..18024e1a 100644 --- a/sbin/set-env.sh +++ b/sbin/set-env.sh @@ -2,10 +2,16 @@ . lib/env.sh -write_shellrc() +write_shell_profile() { - echo "export LKP_SRC=$PWD" >> $(shell_profile) - echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $(shell_profile) + if [ $# -gt 0 ]; then + echo $@ >> $(shell_profile) + else + echo "export LKP_SRC=$PWD" >> $(shell_profile) + echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $(shell_profile) + fi + + source $(shell_profile) } write_host() @@ -28,5 +34,5 @@ write_host() EOF } -write_shellrc +write_shell_profile write_host -- 2.23.0

2 1

[PATCH lkp-tests 1/3] iso2rootfs: result rootfs support disable fstab
by Yu Chuan 10 Dec '20

10 Dec '20

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 1 + tests/iso2rootfs | 7 +++++++ 2 files changed, 8 insertions(+) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 2c4d3a93ff62..9123bef24ae5 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -21,6 +21,7 @@ iso2rootfs: ## supported fields: ## - dns: will config /etc/resolv.conf. ## - no_selinux: will disable selinux in /etc/selinux/config. + ## - no_fstab: will comment all line in /etc/fstab. config_rootfs: # dailybuild related fields diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 9adbc930b48f..c03235eb2d2b 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -361,10 +361,17 @@ disable_selinux() sed -i 's/^SELINUX=.*/SELINUX=disabled/g' "${ROOTFS_DES_DIR}/etc/selinux/config" } +disable_fstab() +{ + echo "${config_rootfs}" | grep -qw 'no_fstab' || return 0 + sed -i 's/^$[^#]$/#\1/g' "${ROOTFS_DES_DIR}/etc/fstab" +} + config_rootfs() { config_dns_resolver disable_selinux + disable_fstab } ############ test rootfs ############ -- 2.23.0

2 2

[PATCH v2 compass-ci] manual/如何申请测试机.md: change the borrow-1h.yaml to latest
by Zhang Yale 10 Dec '20

10 Dec '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.zh.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 04be9cf..578bfe3 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -207,14 +207,14 @@ suite: borrow testcase: borrow + pub_key: <%= + begin + File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp + rescue + nil + end + %> sshd: - pub_key: <%= - begin - File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp - rescue - nil - end - %> # sleep at the bottom sleep: 1h hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml -- 2.23.0

2 2

[PATCH compass-ci] doc/tutorial.md: update cci-depends tutorial
by Wang Yong 10 Dec '20

10 Dec '20

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- doc/tutorial.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index e6f8491..f89f070 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -271,11 +271,9 @@ job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAM `vim lkp-tests/distro/depends/test` 2) 将测试用例所需debian下的包名填入创建的文件中 `echo "apache2" >> lkp-tests/distro/depends/test` - 3) 生成lkp-aarch64.cgz - `./crystal-ci/container/lkp-initrd/run` - 4) 提交测试用例 - `submit test.yaml` - 5) 查看结果 + 3) 提交测试用例 + `submit -a test.yaml` + 4) 查看结果 https://compass-ci.openeuler.org/jobs -- 2.23.0

2 4

[PATCH compass-ci] arch-nr_node-nr_cpu-memory.md: change 'SHELL' to 'YAML'
by Xu Xijian 10 Dec '20

10 Dec '20

The content format between ```SHELL and ``` is YAML, so modify the keyword. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/arch-nr_node-nr_cpu-memory.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/job/arch-nr_node-nr_cpu-memory.md b/doc/job/arch-nr_node-nr_cpu-memory.md index 72cf497..7a1d396 100644 --- a/doc/job/arch-nr_node-nr_cpu-memory.md +++ b/doc/job/arch-nr_node-nr_cpu-memory.md @@ -3,7 +3,7 @@ Meaning: - Each test machine has some basic configurations, such as arch, nr_node, nr_cpu, memory these fields mean: -```SHELL +```YAML arch: the architecture of the machine. nr_node: number of NUMA nodes. nr_cpu: number of logical CPUs. @@ -12,7 +12,7 @@ Meaning: - These fields are from the file which in $LAB_SRC/hosts. - Here is an example of these fields in the host file: -```SHELL +```YAML arch: aarch64 nr_node: 4 nr_cpu: 96 -- 2.23.0

1 0

[PATCH v2 compass-ci] auto_submit.yaml: add os archlinux for buildpkg
by Li Yuanchao 10 Dec '20

10 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index b53b647..d0f4fbe 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -5,4 +5,5 @@ c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=nfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=nfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

1 0

[PATCH compass-ci] arch-nr_node-nr_cpu-memory.md: adjust markdown document format
by Xu Xijian 10 Dec '20

10 Dec '20

In markdown documents, text for bash should be contained in block as belows: ```SHELL content ``` Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/arch-nr_node-nr_cpu-memory.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/doc/job/arch-nr_node-nr_cpu-memory.md b/doc/job/arch-nr_node-nr_cpu-memory.md index c788607..72cf497 100644 --- a/doc/job/arch-nr_node-nr_cpu-memory.md +++ b/doc/job/arch-nr_node-nr_cpu-memory.md @@ -3,14 +3,18 @@ Meaning: - Each test machine has some basic configurations, such as arch, nr_node, nr_cpu, memory these fields mean: +```SHELL arch: the architecture of the machine. nr_node: number of NUMA nodes. nr_cpu: number of logical CPUs. memory: memory size of the machine. +``` - These fields are from the file which in $LAB_SRC/hosts. - Here is an example of these fields in the host file: +```SHELL arch: aarch64 nr_node: 4 nr_cpu: 96 memory: 256G +``` -- 2.23.0

2 2

[PATCH compass-ci] auto_submit.yaml: add os archlinux for buildpkg
by Li Yuanchao 10 Dec '20

10 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sbin/auto_submit.yaml b/sbin/auto_submit.yaml index b53b647..24e524b 100644 --- a/sbin/auto_submit.yaml +++ b/sbin/auto_submit.yaml @@ -5,4 +5,5 @@ c/compass-ci/compass-ci: - testbox=taishan200-2280-2s64p-256g os=centos os_version=7.6.1810 os_mount=initramfs os_arch=aarch64 deploy-cci.yaml - testbox=taishan200-2280-2s64p-256g os=debian os_version=sid os_mount=initramfs os_arch=aarch64 deploy-cci.yaml archlinux: -- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=openeuler os_version=20.03 os_mount=initramfs os_arch=aarch64 build-pkg.yaml +- testbox=vm-2p16g os=archlinux os_version=2020-11-12 os_mount=initramfs os_arch=aarch64 build-pkg.yaml -- 2.23.0

3 2

[PATCH compass-ci] container/git-mirror: add openssh
by Li Yuanchao 10 Dec '20

10 Dec '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/git-mirror/Dockerfile b/container/git-mirror/Dockerfile index 0828be5..8b2a50a 100644 --- a/container/git-mirror/Dockerfile +++ b/container/git-mirror/Dockerfile @@ -13,7 +13,7 @@ RUN apk update && \ echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ apk upgrade && \ apk add --no-cache git && \ - apk add ruby-dev make gcc g++ + apk add ruby-dev make gcc g++ openssh RUN umask 002 && \ gem install bunny json PriorityQueue elasticsearch --no-rdoc --no-ri -- 2.23.0

1 0

[PATCH lkp-tests] tests/*: use curl to upload cgz file
by Wang Yong 10 Dec '20

10 Dec '20

mount directory need OS install nfs-utils/cifs-utils firstly, now only use curl to upload cgz file Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/build-pkg | 12 ------------ tests/cci-depends | 7 ------- tests/cci-makepkg | 14 -------------- 3 files changed, 33 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 8a138055f..fcb10f11f 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -18,7 +18,6 @@ ## See: https://wiki.archlinux.org/index.php/Makepkg . $LKP_SRC/lib/debug.sh -. $LKP_SRC/lib/env.sh . $LKP_SRC/distro/common check_vars() @@ -31,13 +30,6 @@ check_vars() [ -n "$os_mount" ] || die "os_mount is empty" } -mount_pkg_mnt() -{ - [ -n "$LKP_SERVER" ] && { - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" - } -} - mount_dest() { # the same image is mounted to cifs and nfs, the generated cgz files @@ -47,8 +39,6 @@ mount_dest() PKG_MNT=/initrd/build-pkg mkdir -p "$PKG_MNT" - - is_docker || mount_pkg_mnt } get_pkgfile() @@ -140,8 +130,6 @@ build_source_pkg() [ "$?" == 0 ] || exit 1 create_softlink - - is_docker || exit 0 upload_to_target_dir ${PKG_MNT}/${pack_to}/${pkgname} } diff --git a/tests/cci-depends b/tests/cci-depends index ffa585b8c..42d5ef676 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -7,7 +7,6 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/misc-base.sh -. $LKP_SRC/lib/mount.sh [ -n "$benchmark" ] || die "benchmark is empty" [ -n "$os_mount" ] || die "os_mount is empty" @@ -29,11 +28,6 @@ pack_arch=$os_arch . $LKP_SRC/distro/$DISTRO -is_docker || { - is_mount_point ${DEPS_MNT} && umount ${DEPS_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$DEPS_MNT $DEPS_MNT || die "Failed to run mount" -} - umask 002 [[ -d "$pack_to" ]] || { @@ -62,5 +56,4 @@ fixup_arch_install $pack_arch pack_benchmark_deps -is_docker || exit 0 upload_to_target_dir ${pack_to} diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 0354af89d..a50a40c20 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -13,7 +13,6 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/misc-base.sh . $LKP_SRC/lib/env.sh -. $LKP_SRC/lib/mount.sh [ -n "$benchmark" ] || die "benchmark is empty" [ -n "$os_mount" ] || die "os_mount is empty" @@ -34,17 +33,6 @@ pack_to=${os_mount}/${os}/${os_arch}/${os_version}/${benchmark} cd $LKP_SRC/pkg/$benchmark || die "pkg is empty" -mount_pkg_mnt() -{ - is_mount_point ${PKG_MNT} && umount ${PKG_MNT} - mount -t cifs -o guest,vers=1.0,noacl,nouser_xattr,port=446 //$LKP_SERVER$PKG_MNT $PKG_MNT || die "Failed to run mount" -} - -is_docker || { - [ -n "$LKP_SERVER" ] && mount_pkg_mnt -} - - curl_pkgfile() { curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' \ @@ -148,8 +136,6 @@ build_source_pkg() [ "$?" == 0 ] || exit 1 update_softlink - - is_docker || exit 0 upload_to_target_dir ${sync_dest} } -- 2.23.0

1 0

[PATCH compass-ci] job: rename pub_key to ssh_pub_key
by Wu Fengguang 10 Dec '20

10 Dec '20

As top level job key, the name should be more exact. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/borrow-machine.en.md | 2 +- doc/manual/borrow-machine.zh.md | 2 +- src/lib/job.cr | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index 4c3b348..b9c2bc4 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -203,7 +203,7 @@ Ensure that you have performed the following operations according to the [apply- suite: borrow testcase: borrow - pub_key: <%= + ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index d5f9c50..06d4db6 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -207,7 +207,7 @@ suite: borrow testcase: borrow - pub_key: <%= + ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp rescue diff --git a/src/lib/job.cr b/src/lib/job.cr index 7746889..7ded6d8 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -199,7 +199,7 @@ class Job end private def set_sshr_info - return unless @hash["pub_key"]? + return unless @hash["ssh_pub_key"]? self["sshr_port"] = ENV["SSHR_PORT"] self["sshr_port_base"] = ENV["SSHR_PORT_BASE"] @@ -211,7 +211,7 @@ class Job end private def set_my_ssh_pubkey - pub_key = @hash["pub_key"]?.to_s + pub_key = @hash["ssh_pub_key"]?.to_s update_account_my_pub_key(pub_key) @hash["my_ssh_pubkey"] = @account_info["my_ssh_pubkey"] -- 2.23.0

1 0

[PATCH compass-ci] delimiter: submit a job to handle the delimiter task
by Cao Xueliang 10 Dec '20

10 Dec '20

Before: we call the function to handle the delimiter task. After: we submit a job to handle the delimiter task. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter.rb | 17 +++++------------ src/delimiter/constants.rb | 2 +- src/delimiter/delimiter.rb | 12 +++--------- 3 files changed, 9 insertions(+), 22 deletions(-) diff --git a/src/delimiter.rb b/src/delimiter.rb index bb73c5f..bd40775 100644 --- a/src/delimiter.rb +++ b/src/delimiter.rb @@ -2,18 +2,11 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -require_relative './delimiter/constants' require_relative './delimiter/delimiter' -START_PROCESS_COUNT.times do - begin - Process.fork do - delimiter = Delimiter.new - delimiter.start_delimit - end - rescue StandardError => e - puts e - end +begin + delimiter = Delimiter.new + delimiter.start_delimit +rescue StandardError => e + puts e end - -sleep() diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index c243df5..ebf90cc 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -2,7 +2,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -START_PROCESS_COUNT = 10 +LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' diff --git a/src/delimiter/delimiter.rb b/src/delimiter/delimiter.rb index 67a621d..3dcb3de 100644 --- a/src/delimiter/delimiter.rb +++ b/src/delimiter/delimiter.rb @@ -21,20 +21,14 @@ class Delimiter # consume delimiter task queue task = consume_delimiter_queue unless task - sleep(2) + sleep(60) next end - # find first bad commit based on the task - git_bisect = GitBisect.new task - result = git_bisect.find_first_bad_commit - - # send mail - mbr = MailBisectResult.new result - mbr.create_send_email + %x(#{LKP_SRC}/sbin/submit job_id=#{task['job_id']} error_id=#{task['error_id']}) rescue StandardError => e puts e - sleep(30) + sleep(60) end end end -- 2.23.0

1 0

[PATCH compass-ci] doc/manual: fix URLs in files
by Wu Fengguang 10 Dec '20

10 Dec '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/manual/apply-account.md | 4 ++-- doc/manual/borrow-machine.en.md | 4 ++-- doc/manual/borrow-machine.zh.md | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index c1b5206..d9a03c8 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -36,7 +36,7 @@ Which contains account information for you: ## step 4: try it out -Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… +Now try [submitting a job to compass-ci](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… ## notice: @@ -44,4 +44,4 @@ In case that you received a failed email with message as below: The repo url for your commit is not in our upstream-repo list. -Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… +Try to [register repo to upstream-repos](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index e177269..4c3b348 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -1,6 +1,6 @@ # Prerequisites -Ensure that you have performed the following operations according to the [apply-for-an-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/mas…: +Ensure that you have performed the following operations according to the [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc…: - Send an email to apply for an account. - Receive an email from compass-ci(a)qq.com. @@ -223,7 +223,7 @@ Ensure that you have performed the following operations according to the [apply- * Guide to the **submit** Command - Reference: [submit Command Description.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/m… + Reference: [submit-job.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/ma… * What Are the testbox Options? diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 578bfe3..d5f9c50 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -227,7 +227,7 @@ * Submit命令指导 - 参考文档：[submit命令详解.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/… + 参考文档：[submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… * testbox有什么可选项 -- 2.23.0

1 0

[PATCH lkp-tests 2/3] iso2rootfs: add three required fields when submit test job.yaml
by Yu Chuan 10 Dec '20

10 Dec '20

[Why] Since the ability `verify account information` was added on submit job.yaml, so add three required fields when submit test job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- jobs/iso2rootfs.yaml | 52 ++++++++++++++++++++++++++++++++++---------- tests/iso2rootfs | 14 ++++++++++-- 2 files changed, 52 insertions(+), 14 deletions(-) diff --git a/jobs/iso2rootfs.yaml b/jobs/iso2rootfs.yaml index 9123bef24ae5..c67faf44147d 100644 --- a/jobs/iso2rootfs.yaml +++ b/jobs/iso2rootfs.yaml @@ -1,35 +1,63 @@ suite: iso2rootfs category: benchmark iso2rootfs: + ################# # iso related fields to be used to generate rootfs + ################# + iso_os: iso_arch: iso_version: + ################# # place the result rootfs related fields - ## 1. result rootfs will be placed in the following localtion on the - ## remote file server: - ## - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ - ## 2. remote file server protocols current supported: - ## - nfs - ## - cifs + ################# + + # 1. Result rootfs will be placed in the following location on the + # remote file server: + # - {remote_file_server}/{rootfs_path}/{iso_os}/{iso_arch}/ + # 2. Remote file server protocols current supported: + # - nfs + # - cifs rootfs_protocol: rootfs_server: rootfs_path: + ################# # config rootfs related fields - ## supported fields: - ## - dns: will config /etc/resolv.conf. - ## - no_selinux: will disable selinux in /etc/selinux/config. - ## - no_fstab: will comment all line in /etc/fstab. + ################# + + # you can config add some configurations of the result rootfs. + # supported fields: + # - dns: will config /etc/resolv.conf. + # - no_selinux: will disable selinux in /etc/selinux/config. + # - no_fstab: will comment all line in /etc/fstab. config_rootfs: + ################# # dailybuild related fields + ################# + + # dailybuild_iso_url_file: + # - The `dailybuild iso url file` content is the url of a iso. + # - The iso checksum file also exists on the network, and the checksum file path is "{dailybuild_iso_url}.check256sum" + # - demo: + # root@localhost ~% curl http://1.1.1.1/dailybuilds/openEuler-20.03-LTS-aarch64-LTS/release_iso + # http://1.1.1.1//dailybuilds/1970-01-01-00-00-00/openEuler-20.03-LTS-aarch64… dailybuild_iso_url_file: + ################# # submit test yaml related fields - ## 1. you can add as many jobs as you like. - ## 2. the following three fields is requied for every test job. + ################# + + # three required fields when submit test yaml + submit_email: + submit_name: + submit_uuid: + + # submit target tests yaml + ## 1. You can add as many jobs as you like. + ## 2. The following three fields is required for every test job. test1_yaml: test1_os_mount: test1_testbox: diff --git a/tests/iso2rootfs b/tests/iso2rootfs index c03235eb2d2b..dbb409d16923 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -448,11 +448,21 @@ EOF ############ submit test job yaml ############ generate_global_yaml() { + [ -n "${submit_email}" ] || die "cannot find value of var: submit_email" + [ -n "${submit_name}" ] || die "cannot find value of var: submit_name" + [ -n "${submit_uuid}" ] || die "cannot find value of var: submit_uuid" + local config_yaml="/etc/compass-ci/defaults/sparrow.yaml" mkdir -p "$(dirname "${config_yaml}")" - echo "SCHED_HOST: ${SCHED_HOST}" >> "${config_yaml}" - echo "SCHED_PORT: ${SCHED_PORT}" >> "${config_yaml}" + + cat <<-EOF > "${config_yaml}" + SCHED_HOST: ${SCHED_HOST} + SCHED_PORT: ${SCHED_PORT} + my_email: ${submit_email} + my_name: ${submit_name} + my_uuid: ${submit_uuid} + EOF } generate_submit_append_str() -- 2.23.0

1 0

[PATCH lkp-tests] jobs/rpmbuild-pkg.yaml: change the rpm_repo field to level-2 parameter
by Zhang Yale 10 Dec '20

10 Dec '20

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/rpmbuild-pkg.yaml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/jobs/rpmbuild-pkg.yaml b/jobs/rpmbuild-pkg.yaml index 8b5fc39b..19fd5ec2 100644 --- a/jobs/rpmbuild-pkg.yaml +++ b/jobs/rpmbuild-pkg.yaml @@ -1,6 +1,5 @@ suite: rpmbuild-pkg category: functional -rpm_repo: - rpmbuild-pkg: + rpm_repo: -- 2.23.0

1 0

[PATCH compass-ci 4/4] docker-rootfs: list of packages that need to install
by Wang Chenglong 10 Dec '20

10 Dec '20

write the software in this file, it will install into rootfs. the kernel and kernel-devel is necessary. They provide the default kernel as well as kernel source code. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/packages-to-install | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 container/docker-rootfs/packages-to-install diff --git a/container/docker-rootfs/packages-to-install b/container/docker-rootfs/packages-to-install new file mode 100644 index 0000000..d7e890f --- /dev/null +++ b/container/docker-rootfs/packages-to-install @@ -0,0 +1,3 @@ +kernel +kernel-devel +openssh-server -- 2.23.0

2 1