- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 5/6] mail-robot: add error code/message
by Luan Shengde 28 Dec '20

28 Dec '20

add error message for each error code before building the email_message. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 87 +++++++++++++++---- 1 file changed, 69 insertions(+), 18 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 593f345..5871a6a 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -3,32 +3,83 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true -def build_apply_account_fail_email(my_info, message) - email_msg = <<~EMAIL_MESSAGE - To: #{my_info['my_email']} - Subject: [compass-ci] apply account failed +def email_err_message(message) + case message + when 'NO_COMMIT_URL' + err_message = <<~EMAIL_MESSAGE + No commit url found. + You should add a commit url for the 'apply account' email. - Dear user: + For example: + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + EMAIL_MESSAGE + when 'URL_PREFIX_ERR' + err_message = <<~EMAIL_MESSAGE + Please add a correct prefix for the commit url. + + my oss commit + + For example: + + my oss commit: https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… + EMAIL_MESSAGE + when 'NOT_REGISTERED' + err_message = <<~EMAIL_MESSAGE + Your repo has not been registered to our upstream-repos yet. + You should register your repo to the upstream first. + Try again after you have done it. + + Reference the following url to learn how to register the repo: + + https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… + EMAIL_MESSAGE + when 'COMMIT_URL_NOT_AVAILABLE' + err_message = <<~EMAIL_MESSAGE + We cannot confirm the commit url matches your email. + + Make sure the commit exists. + Make sure it is truely submitted with you email. - Your application for account failed with following error: + Go to the commit history and find your commit. + Enter the commit and you will get your commit url. - #{message} + Try again after you have correctted the url + EMAIL_MESSAGE + when 'NO_PUBKEY' + err_message = <<~EMAIL_MESSAGE + No pub_key found. + Please add a pubkey to the email and then try again. + EMAIL_MESSAGE + when 'PUBKEY_NAME_ERR' + err_message = <<~EMAIL_MESSAGE + Pubkey file name error, keep it's name as when it was generated. + The pubkey file name should like: - In order to successfully apply an account, please pay attention to the following points: + id_{{ xxx }}.pub + EMAIL_MESSAGE + end - 1. mail subject - The subject should exactly: apply account + err_message += <<~EMAIL_MESSAGE - 2. commit url - When you writing the url, add prefix: my oss commit - example: - my oss commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… + Manual for how to apply account: + + https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-acco… + EMAIL_MESSAGE + + err_message +end + +def build_apply_account_fail_email(my_info, message) + err_message = email_err_message(message) + email_msg = <<~EMAIL_MESSAGE + To: #{my_info['my_email']} + Subject: [compass-ci] apply account failed + + Dear user: - attention: - Ensure you commit url exist and available to access. + Your application for account failed. - 3. ssh pubkey - You need to add a pubkey as an attachment to the email. + #{err_message} regards compass-ci -- 2.23.0

1 0

[PATCH compass-ci 4/6] mail-robot: optimize message message
by Luan Shengde 28 Dec '20

28 Dec '20

optimize mail message for successfully applied account [why] its used linux command 'curl' to send request data when sending email. so the ${HOME} and ${SHELL##*/} is treated as a system variable and replaced with the variable value. as a result, we will send a unexpected email. to avoid this problem, need add '\\' before '$' in the old. [how] use rest-client instead of command 'curl' to send the request. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index 466586d..ee3b574 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -37,7 +37,7 @@ def build_apply_account_email(my_info) git clone https://gitee.com/wu_fengguang/lkp-tests.git cd lkp-tests make install - source \\${HOME}/.\\${SHELL##*/}rc + source ${HOME}/.${SHELL##*/}rc 3) submit job reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md -- 2.23.0

1 0

[PATCH compass-ci 3/6] mail-robot: optimize apply-jumper-account
by Luan Shengde 28 Dec '20

28 Dec '20

use rest-client instead of linux command 'curl' to send http request Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-jumper-account.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/mail-robot/lib/apply-jumper-account.rb b/container/mail-robot/lib/apply-jumper-account.rb index 44f412b..73d2946 100755 --- a/container/mail-robot/lib/apply-jumper-account.rb +++ b/container/mail-robot/lib/apply-jumper-account.rb @@ -22,8 +22,8 @@ class ApplyJumperAccount end def apply_jumper_account - account_info_str = %x(curl -XGET "#{@jumper_host}:#{@jumper_port}/assign_account" \ - -d '#{(a)my_info.to_json}') + assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" + account_info_str = RestClient.post assign_account_url, @my_info.to_json account_info = JSON.parse account_info_str account_info_exist(account_info) -- 2.23.0

1 0

[PATCH compass-ci 2/6] mail-robot: optimize apply-account
by Luan Shengde 28 Dec '20

28 Dec '20

use rest-client instead of linux command 'curl' to send request Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb index aea1aa5..e8288d7 100755 --- a/container/mail-robot/lib/apply-account.rb +++ b/container/mail-robot/lib/apply-account.rb @@ -5,6 +5,7 @@ require 'json' require 'mail' +require 'rest-client' require_relative '../../../lib/es_client' require_relative 'assign-account-email' require_relative 'assign-account-fail-eamil' @@ -131,6 +132,7 @@ class ApplyAccount build_apply_account_fail_email(@my_info, error_message) end - %x(curl -XPOST "#{@send_mail_host}:#{@send_mail_port}/send_mail_text" -d "#{email_message}") + send_mail_url = "#{@send_mail_host}:#{@send_mail_port}/send_mail_text" + RestClient.post send_mail_url, email_message end end -- 2.23.0

1 0

[PATCH compass-ci 1/6] mail-robot: update Dockerfile
by Luan Shengde 28 Dec '20

28 Dec '20

[why] the old way is use linux command 'curl' to send http request. it's not human friendly to read for user [how] install rest-client instead of linux command 'curl' Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/Dockerfile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/container/mail-robot/Dockerfile b/container/mail-robot/Dockerfile index 76dc94d..f0aebbd 100644 --- a/container/mail-robot/Dockerfile +++ b/container/mail-robot/Dockerfile @@ -8,7 +8,8 @@ MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \ - apt-get install -y git uuid-runtime curl ruby-listen ruby-json ruby-mail + apt-get install -y git uuid-runtime ruby-listen ruby-json ruby-mail ruby-rest-client + RUN echo ':sources: ["http://rubygems.org"]' >> ~/.gemrc && \ gem install fileutils elasticsearch activesupport -- 2.23.0

1 0

[PATCH compass-ci] src/lib/web_backend.rb: fix spell error
by Lu Kaiyi 28 Dec '20

28 Dec '20

Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- src/lib/web_backend.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 0ca71e1..ea67064 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -328,7 +328,7 @@ def get_banner(git_repo, branches) } end -def get_optimize_jobs_braches(jobs) +def get_optimize_jobs_branches(jobs) branch_set = Set.new jobs.size.times do |i| branch = jobs[i]['upstream_branch'].to_s @@ -347,7 +347,7 @@ def get_jobs_body(params) page_size = get_positive_number(params.delete(:page_size), 20) page_num = get_positive_number(params.delete(:page_num), 1) - 1 jobs, total = search_job(params, page_size, page_num) - jobs, branches = get_optimize_jobs_braches(jobs) + jobs, branches = get_optimize_jobs_branches(jobs) { total: total, filter: params, -- 2.23.0

1 0

[PATCH compass-ci] 1-storage/tiny: add mount path
by Hu Xuejiao 26 Dec '20

26 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/1-storage/tiny | 3 +++ 1 file changed, 3 insertions(+) diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index e589937..eb7270d 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -26,3 +26,6 @@ dirs=( ) mkdir -p "${dirs[@]}" + +mount -t cifs //$LKP_SERVER/os /srv/os -o guest,ro,hard,vers=1.0,noacl,nouser_xattr +mount -t cifs //$LKP_SERVER/initrd /srv/initrd -o port=446,guest,ro,hard,vers=1.0,noacl,nouser_xattr -- 2.23.0

1 0

[PATCH compass-ci 3/4] assign-account: optimize answerback-email
by Luan Shengde 26 Dec '20

26 Dec '20

[why] the old way to send request is use linux command 'curl', it's not good for reading and has low efficiency use ruby gem resource rest-client instead of linux command Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 21285e7..a45fef2 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -11,6 +11,7 @@ require 'json' require 'mail' require 'set' require 'optparse' +require 'rest-client' require_relative '../defconfig' require_relative '../../lib/es_client' require_relative 'build-send-account-email' @@ -180,7 +181,8 @@ def apply_account(my_info, conf_info) apply_info.update conf_info apply_info['lab'] = LAB - account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') + assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" + account_info_str = RestClient.post assign_account_url, apply_info.to_json JSON.parse account_info_str end @@ -285,7 +287,8 @@ def send_mail(my_info, account_info, conf_info) build_message(my_info['my_email'], account_info) end - %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") + send_mail_url = "#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text" + RestClient.post send_mail_url, message end def store_account_info(my_info) -- 2.23.0

1 0

[PATCH compass-ci 1/4] assign-account: update Dockerfile
by Luan Shengde 26 Dec '20

26 Dec '20

[why] no need to install command 'curl' in the container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/Dockerfile b/container/assign-account/Dockerfile index 12faafa..1d9c36c 100644 --- a/container/assign-account/Dockerfile +++ b/container/assign-account/Dockerfile @@ -8,7 +8,7 @@ ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ - apt-get install -y ssh ruby-sinatra ruby-json curl && \ + apt-get install -y ssh ruby-sinatra ruby-json && \ ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ echo 'Asia/Shanghai' > /etc/timezone COPY assign-account.rb /usr/local/bin/ -- 2.23.0

1 0

[PATCH v3 lkp-tests] sbin/run_job: Automatic task execution
by Hu Xuejiao 26 Dec '20

26 Dec '20

[purpose] Verify that a local job can be consumed properly Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/run_job | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100755 sbin/run_job diff --git a/sbin/run_job b/sbin/run_job new file mode 100755 index 00000000..c6cf9e85 --- /dev/null +++ b/sbin/run_job @@ -0,0 +1,73 @@ +#!/bin/bash + +cci_makepkg_file="$LKP_SRC/jobs/cci-makepkg.yaml" +grep "atomic" $cci_makepkg_file || sed -i 's|benchmark:|& atomic|g' $cci_makepkg_file + +submit_one_job() +{ + cmd=( + submit -m $yaml + os=${single_arg[0]} + os_arch=${single_arg[1]} + os_version=${single_arg[2]} + os_mount=${single_arg[3]} + testbox=$testbox + queue="${tbox_group}.${HOSTNAME}" + ) + output=$("${cmd[@]}") + echo "$output" | grep -o -E 'job_state=finished[^\"]*' +} + +submit_job() +{ + yaml_args=("$cci_makepkg_file" "host-info.yaml") + for yaml in "${yaml_args[@]}" + do + submit_one_job & + done +} + +run_qemu() +{ + local nr_vm=4 + local testbox="vm-2p8g" + local tbox_group="vm-2p8g" + local hostname="${tbox_group}.${HOSTNAME}" + local queues="${tbox_group}.${HOSTNAME}" + + $CCI_SRC/providers/multi-qemu -n "$hostname" -c "$nr_vm" -q "$queues" + + qemu_args=( + "openeuler aarch64 20.03 cifs" + "centos aarch64 7.6.1810 initramfs" + ) + for qemu_arg in "${qemu_args[@]}" + do + local single_arg=($(echo $qemu_arg)) + submit_job + done +} + +run_docker() +{ + local nr_container=2 + local tbox_group="dc-8g" + local testbox="dc-8g" + local hostname="${tbox_group}.${HOSTNAME}" + local queues="${tbox_group}.${HOSTNAME}" + + $CCI_SRC/providers/multi-docker -n $hostname -c $nr_container -q $queues + + docker_args=( + "centos aarch64 7.6.1810 container" + ) + for docker_arg in "${docker_args[@]}" + do + local single_arg=($(echo $docker_arg)) + submit_job + done +} + +run_qemu +run_docker +wait -- 2.23.0

1 0