October 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 3/5] container/send-internet-mail: Dockerfile
by Luan Shengde 20 Oct '20

20 Oct '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/send-internet-mail/Dockerfile | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/send-internet-mail/Dockerfile diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile new file mode 100644 index 0000000..dfa0d46 --- /dev/null +++ b/container/send-internet-mail/Dockerfile @@ -0,0 +1,9 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM debian +MAINTAINER luanshd0525(a)163.com +ENV DEBIAN_FRONTEND noninteractive + +RUN apt-get update && \ + apt-get install -y ruby-mail ruby-json ruby-sinatra -- 2.23.0

1 0

[PATCH v3 compass-ci 2/5] container/send-internet-mail: send-internet-mail.rb
by Luan Shengde 20 Oct '20

20 Oct '20

set up smtp config for smtp.qq.com send mail with recieved email data Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- .../send-internet-mail/send-internet-mail.rb | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100755 container/send-internet-mail/send-internet-mail.rb diff --git a/container/send-internet-mail/send-internet-mail.rb b/container/send-internet-mail/send-internet-mail.rb new file mode 100755 index 0000000..d8c8d64 --- /dev/null +++ b/container/send-internet-mail/send-internet-mail.rb @@ -0,0 +1,31 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'mail' + +# setup smtp config +smtp = { + address: 'smtp.qq.com', + port: 25, + domain: 'qq.com', + user_name: ENV['EMAIL_ADDRESS'], + password: ENV['EMAIL_PASSWORD'], + openssl_verify_mode: 'none', + enable_starttls_auto: true +} + +Mail.defaults { delivery_method :smtp, smtp } + +# send mail +def send_mail(mail_info) + mail = Mail.new do + references mail_info['references'] + from ENV['EMAIL_ADDRESS'] + subject mail_info['subject'] + to mail_info['to'] + body mail_info['body'] + end + mail.deliver! +end -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: merge overrides in advance
by Wei Jihui 20 Oct '20

20 Oct '20

[why] we have a new job yaml like: override: nr_task: - 100% - 50% << : jobs/unixbench.yaml overrides args:[100%, 50%] shoule be add job at the first time, so it will split to atomic job, nr_task:[nr_cpu*100%, nr_cpu*50%] job has overrides, so del revise_hash(job, deepcopy(@overrides), true), otherwise it will override job with nr_task:[100%, 50%], expand_params can not deal this list directly. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 2fd4567e..bca0a100 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -249,6 +249,7 @@ class Job @overrides.merge!(hash['override']){ |_key, a, _b| a} hash.delete('override') end + hash.merge!(@overrides) @jobs << hash end rescue StandardError => e @@ -404,7 +405,6 @@ class Job i = include_files job = deepcopy(@job) revise_hash(job, deepcopy(@job2), true) - revise_hash(job, deepcopy(@overrides), true) job['___'] = nil @jobx = job if @job.include?(:expand_params) -- 2.23.0

2 1

[PATCH lkp-tests] submit: mark .yaml when there is no =
by Wu Fengguang 20 Oct '20

20 Oct '20

This feels more symmetric logic. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index 2475466f..a749ea20 100755 --- a/sbin/submit +++ b/sbin/submit @@ -60,7 +60,6 @@ options.parse!(ARGV) seen_job_yaml = false ARGV.delete_if do |arg| - seen_job_yaml = true if arg =~ /\.yaml$/ if arg.index '=' if seen_job_yaml opt_set_key_value.merge! YAML.load arg.sub(/=/, ': ') @@ -69,6 +68,7 @@ ARGV.delete_if do |arg| end true else + seen_job_yaml = true if arg =~ /\.yaml$/ false end end -- 2.23.0

1 0

[PATCH lkp-tests] submit: rename find_job_yaml to seen_job_yaml
by Wu Fengguang 20 Oct '20

20 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- sbin/submit | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/sbin/submit b/sbin/submit index 979e3dca..2475466f 100755 --- a/sbin/submit +++ b/sbin/submit @@ -58,11 +58,11 @@ end options.parse!(ARGV) -find_job_yaml = false +seen_job_yaml = false ARGV.delete_if do |arg| - find_job_yaml = true if arg =~ /\.yaml$/ + seen_job_yaml = true if arg =~ /\.yaml$/ if arg.index '=' - if find_job_yaml + if seen_job_yaml opt_set_key_value.merge! YAML.load arg.sub(/=/, ': ') else opt_cmdline_defaults.merge! YAML.load arg.sub(/=/, ': ') -- 2.23.0

1 0

[PATCH v4 lkp-tests] tests: add get_ipmi_ip function to host-info
by Zhang Yu 20 Oct '20

20 Oct '20

output: ipmi_ip: 9.3.4.1 Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tests/host-info b/tests/host-info index a57855dc..5a4f590c 100755 --- a/tests/host-info +++ b/tests/host-info @@ -80,6 +80,24 @@ get_partitions() fi } +get_ipmi_ip() +{ + # ipmitool lan print + # Set in Progress : Set Complete + # IP Address Source : Static Address + # IP Address : 9.3.4.1 + # Subnet Mask : 255.255.0.0 + # MAC Address : e0:00:84:2b:51:7b + # SNMP Community String : TrapAdmin12#$ + # IP Header : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10 + # Default Gateway IP : 9.3.0.1 + # 802.1q VLAN ID : Disabled + # RMCP+ Cipher Suites : 0,1,2,3,17 + # Cipher Suite Priv Max : XuuaXXXXXXXXXXX + imp_ip=$(ipmitool lan print |awk 'NR==3{print $4}') + [ $? -eq 0 ] && echo "ipmi_ip:" $imp_ip +} + main() { get_node @@ -87,6 +105,7 @@ main() get_partitions get_mac get_model_name + get_ipmi_ip } main -- 2.23.0

4 5

[PATCH compass-ci] code-spec: add reuse rule
by Wu Fengguang 20 Oct '20

20 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/code-spec.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index 52af5e6..37c86b6 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -1,6 +1,9 @@ code注意事项 ============ +- 在开始设计、编码之前，请善用搜索，善于复用现成code。 + 大部分基本问题在网上有现成的答案，往往还是最优解。 + - 代码应当放进函数里不要在脚本顶层直接写逻辑 - 代码应当放进lib/里脚本本身最小化, 仅解析命令行参数, 调用lib/入口函数 -- 2.23.0

1 0

[PATCH compass-ci] /src/lib/web_backend.rb: support multi-fields search in "/get_jobs"
by Zhang Yuhang 20 Oct '20

20 Oct '20

Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/web_backend.rb | 49 ++++++++++++++++++++++++------------------ 1 file changed, 28 insertions(+), 21 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 48cfadb..c8b6b66 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -47,7 +47,7 @@ end def filter_tbox_group(es_result) result = Set.new es_result.each do |r| - if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-([a-zA-Z]+)|(\d+)$)/ + if r =~ /(^.+--.+$)|(^vm-.*-\d\w*-(([a-zA-Z]+)|(\d+))$)/ index = r.index('--') || r.rindex('-') r = r[0, index] end @@ -245,26 +245,36 @@ def es_search(must, size, from) } return {}, total if wrong_size?(size, from) + p query return es_query(query)['hits']['hits'], total end -def get_job(result) - job = {} - ALL_FIELDS.each do |f| - job[f] = result[f] +def get_jobs_result(result) + jobs = [] + result.each do |r| + job = {} + ALL_FIELDS.each do |f| + job[f] = r['_source'][f] + end + jobs << job end - job + jobs end -def search_job(git_repo, page_size, page_num) +def search_job(fields, page_size, page_num) must = [] - must << { regexp: { upstream_repo: ".*#{git_repo}.*" } } if git_repo - jobs = [] - result, total = es_search(must, page_size, page_num * page_size) - result.each do |r| - jobs << get_job(r['_source']) + FIELDS.each do |field| + value = space_to_nil(fields[field]) + next unless value + + must << if field == 'upstream_repo' + { regexp: { field => ".*#{value}.*" } } + else + { term: { field => value } } + end end - return jobs, total + result, total = es_search(must, page_size, page_num * page_size) + return get_jobs_result(result), total end def get_banner(git_repo, branches) @@ -291,17 +301,14 @@ def get_optimize_jobs_braches(jobs) end def get_jobs_body(params) - git_repo = space_to_nil(params[:upstream_repo]) - page_size = get_positive_number(params[:page_size], 20) - page_num = get_positive_number(params[:page_num], 1) - 1 - jobs, total = search_job(git_repo, page_size, page_num) + page_size = get_positive_number(params.delete(:page_size), 20) + page_num = get_positive_number(params.delete(:page_num), 1) - 1 + jobs, total = search_job(params, page_size, page_num) jobs, branches = get_optimize_jobs_braches(jobs) { total: total, - filter: { - upstream_repo: git_repo - }, - banner: get_banner(git_repo, branches), + filter: params, + banner: get_banner(params[:upstream_repo], branches), jobs: jobs, fields: FIELDS }.to_json -- 2.23.0

2 6

[PATCH v2 compass-ci] service/scheduler: add ssh port to job content
by Wu Zhende 20 Oct '20

20 Oct '20

[Why] Network users need to use sshr to login testbox. So testbox should set sshr. And testbox get sshr info from job content. Scheduler needs to add ssh info to job content Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/scheduler/start | 4 ++++ src/lib/job.cr | 6 ++++++ 2 files changed, 10 insertions(+) diff --git a/container/scheduler/start b/container/scheduler/start index 81e7139..1183cde 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -24,10 +24,14 @@ names = Set.new %w[ GIT_SERVER FLUENTD_SERVER_HOST FLUENTD_SERVER_PORT + SSH_PORT_BASE + SSH_PORT_LEN lab ] defaults = relevant_defaults(names) +defaults["SSH_PORT_BASE"] = 50000 unless defaults["SSH_PORT_BASE"] +defaults["SSH_PORT_LEN"]= 2000 unless defaults["SSH_PORT_LEN"] env = docker_env(defaults) CCI_REPOS = ENV['CCI_REPOS'] || '/c' diff --git a/src/lib/job.cr b/src/lib/job.cr index 1e44d15..bd20828 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_append_root() set_kernel_params() set_lkp_server() + set_ssh_port() if self["uuid"]? end private def append_init_field @@ -168,6 +169,11 @@ class Job end end + private def set_ssh_port + self["ssh_port_base"] = ENV["SSH_PORT_BASE"] + self["ssh_port_len"] = ENV["SSH_PORT_LEN"] + end + private def set_os_dir self["os_dir"] = "#{os}/#{os_arch}/#{os_version}" end -- 2.23.0

2 1

[PATCH compass-ci] code-spec: add rules
by Wu Fengguang 20 Oct '20

20 Oct '20

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- doc/code-spec.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index d405cdd..52af5e6 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -2,12 +2,18 @@ code注意事项 ============ - 代码应当放进函数里不要在脚本顶层直接写逻辑 + - 代码应当放进lib/里脚本本身最小化, 仅解析命令行参数, 调用lib/入口函数 + - 如果出现了全局变量一般意味着需要写一个 class/module 来放这个状态量了 + - 一个函数, 不要超过10行代码, 该原则适用于所有ruby/crystal/shell各类程序. + - 起准确和有意义的变量/函数名字, 名字要让人看出赋予他们的确切意义 + - 函数要有分层结构. 每个函数须有明确的定位: 业务层逻辑, 还是底层支持性routine. 一个函数不能同时做两个层次的事. 同一函数内避免混杂不同类型的事务. + - 如果注释仅仅是代码的简单重复, 不提供额外信息量, 不要写. 重构友好编码 @@ -31,6 +37,7 @@ PATCH注意事项 ============= - PATCH title 应当简要说明做了什么 (what) + - PATCH changelog / code comment 应当解释读者无法从代码简单推断的 - 背景 - 目的 @@ -39,9 +46,18 @@ PATCH注意事项 - 实际例子 - 改进数字 - bug场景, 出错消息原文 + - PATCH 应当加上作者签名 (Signed-off-by: Name <email>) + - PATCH 最好不超过100行, 一个PATCH只做一件事, 避免混杂多个逻辑变更 +- changelog/comment里的话，需理顺逻辑关系，让人一看就知道在讲什么， + 比如问题、原因、条件、结果、预期、假设、场景、目的、副作用、权衡， + 哪些是因果关系，哪些是可能，哪些是肯定。 + +- 论述问题的时候，引用必要的代码或output，帮助明确逻辑流/数据流。 + 引文加TAB缩进显示。 + PATCH参考材料 ============= -- 2.23.0

1 0