AssignUuid: read email content check email available check if base url in upstream-repos check if commit url valid check pub_key assign account generate uuid write account/uuid to es send uuid to user
Signed-off-by: Luan Shengde luanshengde2@huawei.com --- container/mail-robot/answerback-email.rb | 181 +++++++++++++++++++++++ 1 file changed, 181 insertions(+) create mode 100755 container/mail-robot/answerback-email.rb
diff --git a/container/mail-robot/answerback-email.rb b/container/mail-robot/answerback-email.rb new file mode 100755 index 0000000..2cd15ef --- /dev/null +++ b/container/mail-robot/answerback-email.rb @@ -0,0 +1,181 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require_relative '/compass-ci/lib/es_client.rb' + +# Assign uuid/account +class AssignUuid + def initialize(mail_content) + @mail_content = mail_content + end + + def build_message(sum_infos) + message = <<~EMAIL_MESSAGE + To: #{sum_infos['email']} + Message-ID: #{sum_infos['message_id']} + Subject: Account Ready + + Dear #{sum_infos['email'].split('@')[0]} + + Thank you for joining us. + You can use the following info to submit jobs: + + First: add the info to ~/.config/compass-ci/defaults/{USER}.yaml + + uuid: #{sum_infos['account_uuid']} + lab: z9 + + Second: download lkp-tests and dependencies + git clone https://gitee.com_/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source {HOME}/.bashrc && source {HOME}/.bash_profile + + Third: submit job + reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md + how to write job yaml + submit jobs/netperf.yaml + + Repalce the fields {...} with your local environment variable. + + regards + compass-ci + EMAIL_MESSAGE + + return message + end + + def url_addr + url_line = @mail_content.part[0].body.decoded.split(/\n/).find { |line| line =~ %r{https?://} } + message = 'No commit url found' + + raise message unless url_line + + url = url_line.split[-1] + + return url + end + + def check_email_available(email) + url = url_addr + + base_url = url.gsub(%r{/commit/[^/].*}, '') + + upstream_dir = '/upstream-repos' + check_url_in_upstream_repos(upstream_dir, base_url) + check_url_commit(url, base_url, email) + + return url + end + + def check_url_in_upstream_repos(upstream_dir, base_url) + Dir.chdir(upstream_dir) + match_out = %x(grep -rn #{base_url}) + + message = 'The url is not in upstream-repo list' + + raise StandardError, message, base_url if match_out.empty? + end + + def check_url_commit(url, base_url, email) + hub_name = url.split('/')[2] + check_gitee_commit(url, base_url, email) if hub_name.eql? 'gitee.com' + check_non_gitee_commit(url, email) unless hub_name.eql? 'gitee.com' + end + + def check_non_gitee_commit(url, email) + url_fdback = %x(curl #{url}) + email_index = url_fdback.index email + + message = 'No email matched from the commit_id feedback.' + raise StandardError, message, url if email_index.nil? + end + + def check_gitee_commit(url, base_url, email) + repo_info = { + 'repos_dir' => '/repos_dir', + 'repo_dir' => url.split('/')[-3], + 'repo_url' => [base_url, 'git'].join('.'), + 'repo_path' => File.join('/repos_dir', url.split('/')[-3]), + 'commit_id' => url.split('/')[-1] + } + check_gitee_repo_commit(url, email, repo_info) + end + + def check_gitee_repo_commit(url, email, repo_info) + Dir.chdir repo_info['repos_dir'] + %x(/usr/bin/git clone #{repo_info['repo_url']}) + email_index = %x(/usr/bin/git -C #{repo_info['repo_dir']} show #{repo_info['commit_id']}).index email + + FileUtils.rm_rf repo_info['repo_dir'] + + message = 'No email matched from commit_id feedback.' if email_index.nil? + raise StandardError, message, url if email_index.nil? + end + + def pub_key + pub_key = @mail_content.part[1].body.decoded.split(/\r|\n/).join if @mail_content.part[1].filename == 'id_rsa.pub' + + message = 'No pub_key found' + + raise message unless pub_key + + return pub_key + end + + def account_info(pub_key) + account_info_str = %x(curl -XGET "#{ENV['J_IP']}:#{ENV['J_PORT']}/assign_account" -d "pub_key: #{pub_key}") + account_info = JSON.parse account_info_str + + message = 'No available jumper account returned.' + raise StandardError, message, account_info if account_info['account'].empty? + + return account_info + end + + def build_sum_infos + email = @mail_content.from[0] + account_uuid = %x(uuidgen).chomp + message_id = @mail_content.message_id + sum_infos = { + 'email' => email, + 'message_id' => message_id, + 'url' => check_email_available(email), + 'acct_infos' => account_info(pub_key), + 'account_uuid' => account_uuid, + 'pub_key' => pub_key + } + store_account_info(sum_infos) + sum_infos + end + + def send_uuid(sum_infos) + message = build_message(sum_infos) + + send_mail_server = `/sbin/ip route |awk '/default/ {print $3}'`.chomp + %x(curl -XPOST "#{send_mail_server}:#{ENV['S_M_PORT']}/send_mail_text" -d "#{message}") + end + + def send_account + sum_infos = build_sum_infos + + send_uuid(sum_infos) + end + + def store_account_info(sum_infos) + account_uuid_info = { + 'email' => sum_infos['email'], + 'name' => sum_infos['email'].split('@')[0], + 'uuid' => sum_infos['account_uuid'], + 'user' => sum_infos['acct_infos']['account'], + 'my_commit_url' => sum_infos['url'] + } + + es = ESClient.new(index: 'accounts') + es.put_source_by_id(sum_infos['email'], account_uuid_info) + end +end
- def build_sum_infos
- email = @mail_content.from[0]
- account_uuid = %x(uuidgen).chomp
- message_id = @mail_content.message_id
- sum_infos = {
'email' => email,
'message_id' => message_id,
'url' => check_email_available(email),
'acct_infos' => account_info(pub_key),
'account_uuid' => account_uuid,
'pub_key' => pub_key
This creates sum_infos in one Hash format and convert it to another below. Why not create it in final form all at once?
- }
- store_account_info(sum_infos)
- sum_infos
- end
- def store_account_info(sum_infos)
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
- end
+end
2.23.0
On Wed, Oct 14, 2020 at 10:32:59AM +0800, Wu Fengguang wrote:
- def build_sum_infos
- email = @mail_content.from[0]
- account_uuid = %x(uuidgen).chomp
- message_id = @mail_content.message_id
- sum_infos = {
'email' => email,
'message_id' => message_id,
'url' => check_email_available(email),
'acct_infos' => account_info(pub_key),
'account_uuid' => account_uuid,
'pub_key' => pub_key
This creates sum_infos in one Hash format and convert it to another below. Why not create it in final form all at once?
OK, I will try to optimize it.
Thanks Luan Shengde
- }
- store_account_info(sum_infos)
- sum_infos
- end
- def store_account_info(sum_infos)
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
- end
+end
2.23.0
TO ALL.
踩过的坑, 影响决策的关键点, 请全部记录下来. 踩过的坑, 影响决策的关键点, 请全部记录下来. 踩过的坑, 影响决策的关键点, 请全部记录下来.
- def check_url_commit(url, base_url, email)
- hub_name = url.split('/')[2]
- check_gitee_commit(url, base_url, email) if hub_name.eql? 'gitee.com'
Should comment why gitee needs special handling. What's the tradeoff behind the code.
踩过的坑, 讨论的影响决策的关键点, 请全部记录下来.
否则的话, 别人光看代码会一头雾水, 难以理解和维护.
Thanks, Fengguang
While these JSON field names look logical in DB context, they could be confusing when used in other context.
- def store_account_info(sum_infos)
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value ============================================== my_email wfg@kernel.org my_name Wu Fengguang my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
where the below vars should be set in the user's defaults config (.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
Thanks, Fengguang
On Thu, Oct 15, 2020 at 10:31:19AM +0800, Wu Fengguang wrote:
While these JSON field names look logical in DB context, they could be confusing when used in other context.
- def store_account_info(sum_infos)
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang
will need user to add his/her name in the apply account email? if not, where can I get the "my_name"
my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
where the below vars should be set in the user's defaults config (.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
Thanks, Fengguang
On Thu, Oct 15, 2020 at 05:27:53PM +0800, Luan Shengde wrote:
On Thu, Oct 15, 2020 at 10:31:19AM +0800, Wu Fengguang wrote:
While these JSON field names look logical in DB context, they could be confusing when used in other context.
- def store_account_info(sum_infos)
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang
will need user to add his/her name in the apply account email?
Yes.
if not, where can I get the "my_name"
Named email address. Like "Wu Fengguang wfg@crystal.ci".
Thanks, Fengguang
my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
where the below vars should be set in the user's defaults config (.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
Thanks, Fengguang
On Thu, Oct 15, 2020 at 05:29:52PM +0800, Wu Fengguang wrote:
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang
will need user to add his/her name in the apply account email?
Yes.
if not, where can I get the "my_name"
Named email address. Like "Wu Fengguang wfg@crystal.ci".
but sometimes the named email address may in undesired mode in the accepted email file, like below:
/srv/cci/Maildir/.compass-ci/new% grep -rn "From: " 1600931612.3884974_1.z9:39:From: =?UTF-8?B?a2lkZHk=?= taxcom@tom.com 1601016368.1077922_1.z9:2,S:20:From: "LiPing" 15396232681@163.com 1601005606.322199_1.z9:2,S:22:From: =?UTF-8?B?5p2O6JCN?= 15396232681@163.com 1602749599.465580_1.z9:39:From: =?gb2312?B?0Lsg1f4=?= XieZheng@live.com 1602659539.1186432_1.z9:2,S:39:From: =?gb2312?B?0Lsg1f4=?= xiezheng@live.com
how about write add the user name as mail content part
Thanks Luan Shengde
Thanks, Fengguang
my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
where the below vars should be set in the user's defaults config (.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
Thanks, Fengguang
On Thu, Oct 15, 2020 at 06:56:42PM +0800, Luan Shengde wrote:
On Thu, Oct 15, 2020 at 05:29:52PM +0800, Wu Fengguang wrote:
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang
will need user to add his/her name in the apply account email?
Yes.
if not, where can I get the "my_name"
Named email address. Like "Wu Fengguang wfg@crystal.ci".
but sometimes the named email address may in undesired mode in the accepted email file, like below:
/srv/cci/Maildir/.compass-ci/new% grep -rn "From: " 1600931612.3884974_1.z9:39:From: =?UTF-8?B?a2lkZHk=?= taxcom@tom.com 1601016368.1077922_1.z9:2,S:20:From: "LiPing" 15396232681@163.com 1601005606.322199_1.z9:2,S:22:From: =?UTF-8?B?5p2O6JCN?= 15396232681@163.com 1602749599.465580_1.z9:39:From: =?gb2312?B?0Lsg1f4=?= XieZheng@live.com 1602659539.1186432_1.z9:2,S:39:From: =?gb2312?B?0Lsg1f4=?= xiezheng@live.com
Yeah real world can be complex.
how about write add the user name as mail content part
OK.
Thanks, Fengguang
my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
where the below vars should be set in the user's defaults config (.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
Thanks, Fengguang
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
so, we should also update es-mapping of accounts index in ES DB
where the below vars should be set in the user's defaults config
we could remind user set them in user's defaults config with email
(.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
above 3 field will keep consistent names throughout the project, include jobs index for job result mail, bisect result email, ... we can use the $my_email, right?
Thanks, Weitao
Thanks, Fengguang
On Thu, Oct 15, 2020 at 05:46:32PM +0800, Lu Weitao wrote:
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
so, we should also update es-mapping of accounts index in ES DB
Right.
where the below vars should be set in the user's defaults config
we could remind user set them in user's defaults config with email
Yeah. Shengde: should do that in robot reply email.
(.config/compass-ci/defaults/xxx.yaml), to be auto included in every job he/she submits.
my_name my_email my_uuid
above 3 field will keep consistent names throughout the project, include jobs index for job result mail, bisect result email, ... we can use the $my_email, right?
Right. It's the best way to 减轻认知负担.
Thanks, Fengguang
On Thu, Oct 15, 2020 at 07:24:19PM +0800, Wu Fengguang wrote:
On Thu, Oct 15, 2020 at 05:46:32PM +0800, Lu Weitao wrote:
- account_uuid_info = {
'email' => sum_infos['email'],
'name' => sum_infos['email'].split('@')[0],
'uuid' => sum_infos['account_uuid'],
'user' => sum_infos['acct_infos']['account'],
'my_commit_url' => sum_infos['url']
- }
- es = ESClient.new(index: 'accounts')
- es.put_source_by_id(sum_infos['email'], account_uuid_info)
Let's make the names more specific and use the specific names throughout the project consistently.
variable (or hash key) example value
my_email wfg@kernel.org my_name Wu Fengguang my_commit_url https://gitee.com/wu_fengguang/compass-ci/commit/b14b8b5b2ba3c47d1bd13eaf734... my_uuid 88c756ad-9707-491b-b406-13515ffcb887 my_login_name hi1000
so, we should also update es-mapping of accounts index in ES DB
Right.
ok, I will update them.
job he/she submits.
my_name my_email my_uuid
above 3 field will keep consistent names throughout the project, include jobs index for job result mail, bisect result email, ... we can use the $my_email, right?
Right. It's the best way to 减轻认知负担.
ok, got it
Thanks, Weitao
Thanks, Fengguang