- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 3/3] ssign-account: add config lab yaml file
by Luan Shengde 14 Dec '20

14 Dec '20

[why] key my_uuid is renamed to my_token item key my_token will be stored in file: ~/.config/compass-ci/include/lab/${lab}.yaml to do this change with no sense: for old user(update account): keep my_uuid in the default yaml add my_token to ~/.config/compass-ci/include/lab/${lab}.yaml my_token has the same value to my_uuid for new user: add my_email, my_name to the default yaml add my_token to ~/.config/compass-ci/include/lab/${lab}.yaml Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 47 ++++++++++++++++---- 1 file changed, 38 insertions(+), 9 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 95bec14..8015a5f 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -110,7 +110,7 @@ class AccountStorage jumper_host, jumper_port = read_jumper_info config_authorized_key(login_name) - config_default_yaml(login_name) + config_yaml(login_name) permit_login_config(login_name) if @data.key?('gen_sshkey') && @data['gen_sshkey'] my_jumper_pubkey = generate_ssh_key(login_name) @@ -127,6 +127,14 @@ class AccountStorage return jumper_account_info end + def config_yaml(login_name) + default_yaml = touch_default_yaml(login_name) + config_default_yaml(login_name, default_yaml) + + lab_yaml = touch_lab_yaml(login_name) + config_lab_yaml(login_name, lab_yaml) + end + def permit_login_config(login_name) if @data.key?('enable_login') && @data['enable_login'] %x(usermod -s /usr/bin/zsh #{login_name}) @@ -151,26 +159,47 @@ class AccountStorage def touch_default_yaml(login_name) default_yaml_dir = File.join('/home', login_name, '.config/compass-ci/defaults') + # before create the file, the directory need to be exists FileUtils.mkdir_p default_yaml_dir unless File.exist? default_yaml_dir default_yaml = File.join(default_yaml_dir, 'account.yaml') + # for we need to read the file before we open it + # need to firstly create the file FileUtils.touch default_yaml unless File.exist? default_yaml default_yaml end - def config_default_yaml(login_name) - default_yaml = touch_default_yaml(login_name) - + def config_default_yaml(login_name, default_yaml) account_yaml = YAML.load_file(default_yaml) || {} - # my_email, my_name, my_uuid is required to config default yaml file + # my_email, my_name is required to config default yaml file # they are added along with 'my_ssh_pubkey' when sending assign account request account_yaml['my_email'] = @data['my_email'] account_yaml['my_name'] = @data['my_name'] - account_yaml['my_uuid'] = @data['my_uuid'] - f = File.new(default_yaml, 'w') - f.puts account_yaml.to_yaml - f.close + File.open(default_yaml, 'w') do |f| + f.puts account_yaml.to_yaml + end + + FileUtils.chown_R(login_name, login_name, "/home/#{login_name}/.config") + end + + def touch_lab_yaml(login_name) + lab_yaml_dir = File.join('/home', login_name, '.config/compass-ci/include/lab') + FileUtils.mkdir_p lab_yaml_dir unless File.exist? lab_yaml_dir + + lab_yaml = File.join(lab_yaml_dir, "#{@data['lab']}.yaml") + FileUtils.touch lab_yaml unless File.exist? lab_yaml + lab_yaml + end + + def config_lab_yaml(login_name, lab_yaml) + lab_yaml_info = YAML.load_file(lab_yaml) || {} + lab_yaml_info['my_token'] = @data['my_token'] + + File.open(lab_yaml, 'w') do |f| + f.puts lab_yaml_info.to_yaml + end + FileUtils.chown_R(login_name, login_name, "/home/#{login_name}/.config") end -- 2.23.0

1 0

[PATCH v3 compass-ci 2/3] assign-account: rename my_uuid to my_token
by Luan Shengde 14 Dec '20

14 Dec '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/assign-account.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/assign-account.rb b/container/assign-account/assign-account.rb index c0753b0..4ed95c0 100755 --- a/container/assign-account/assign-account.rb +++ b/container/assign-account/assign-account.rb @@ -24,7 +24,7 @@ get '/assign_account' do end def assign_jumper_account(data) - lacked_info = %w[my_email my_name my_uuid] - data.keys + lacked_info = %w[my_email my_name my_token] - data.keys error_message = "lack of my infos: #{lacked_info.join(', ')}." raise error_message unless lacked_info.empty? -- 2.23.0

1 0

[PATCH v3 compass-ci 1/3] answerback-email: rename key my_uuid with my_token
by Luan Shengde 14 Dec '20

14 Dec '20

key my_uuid is renamed to my_token when applying/updating account, will use my_token instead to do this change with no sense for old user(update account) keep key my_uuid and add key my_token for new user just add key my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 25 ++++++++++++++------ 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index c08f10e..766e1b3 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -21,6 +21,7 @@ names = Set.new %w[ JUMPER_PORT SEND_MAIL_HOST SEND_MAIL_PORT + lab ] defaults = relevant_defaults(names) @@ -29,12 +30,14 @@ JUMPER_HOST = defaults['JUMPER_HOST'] JUMPER_PORT = defaults['JUMPER_PORT'] || 29999 SEND_MAIL_HOST = defaults['SEND_MAIL_HOST'] || 'localhost' SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 +LAB = defaults['lab'] +TOOL_SERVER = 'z9' my_info = { 'my_email' => nil, 'my_name' => nil, 'my_commit_url' => nil, - 'my_uuid' => nil, + 'my_token' => nil, 'my_login_name' => nil, 'my_ssh_pubkey' => [] } @@ -51,6 +54,8 @@ my_info = { # stdin_info > email_info > my_info_es # when assigning account or update conf for account # if they have the same key, my_info will use the value with higher priority. +# if execute update operations, if there are new key in stdin_info and email_info +# it will use the one with higher priority, and merge it with the the keys in my_info_es # conf_info is used to store keys used to config the account stdin_info = {} email_info = {} @@ -179,6 +184,7 @@ def apply_account(my_info, conf_info) apply_info = {} apply_info.update my_info apply_info.update conf_info + apply_info['lab'] = LAB account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{apply_info.to_json}') JSON.parse account_info_str @@ -200,9 +206,7 @@ def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) new_stdin_pubkey = stdin_info.delete 'new_ssh_pubkey' new_pubkey = new_stdin_pubkey || new_email_pubkey - my_info.update my_info_es unless my_info_es.empty? - my_info.update email_info unless email_info.empty? - my_info.update stdin_info unless stdin_info.empty? + update_my_info(my_info, my_info_es, email_info, stdin_info) return if new_pubkey.nil? return if my_info['my_ssh_pubkey'].include? new_pubkey @@ -210,6 +214,13 @@ def build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) my_info['my_ssh_pubkey'].insert(0, new_pubkey) end +def update_my_info(my_info, my_info_es, email_info, stdin_info) + my_info['my_token'] = my_info_es['my_uuid'] if my_info['my_token'].nil? + my_info.update my_info_es + my_info.update email_info + my_info.update stdin_info +end + def build_my_info_from_account_info(my_info, account_info, conf_info) unless account_info['my_jumper_pubkey'].nil? return if my_info['my_ssh_pubkey'][-1] == account_info['my_jumper_pubkey'] @@ -221,9 +232,9 @@ def build_my_info_from_account_info(my_info, account_info, conf_info) end def check_server - return true if ENV['HOSTNAME'] == 'z9' + return true if ENV['HOSTNAME'] == TOOL_SERVER - message = 'please run the tool on z9 server' + message = "please run the tool on #{TOOL_SERVER} server" puts message return false @@ -260,7 +271,7 @@ def send_account(my_info, conf_info, email_info, my_info_es, stdin_info) return unless check_my_email(my_info) return unless check_email_assigned_account(conf_info, my_info_es) - my_info['my_uuid'] = %x(uuidgen).chomp unless conf_info['is_update_account'] + my_info['my_token'] = %x(uuidgen).chomp unless conf_info['is_update_account'] build_my_info_from_input(my_info, email_info, my_info_es, stdin_info) return unless check_my_name_exist(my_info) -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] mail-robot: update email message
by Luan Shengde 14 Dec '20

14 Dec '20

update email message with new key: my_uuid => my_token Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/assign-account-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb index a88626c..834e96f 100755 --- a/container/mail-robot/lib/assign-account-email.rb +++ b/container/mail-robot/lib/assign-account-email.rb @@ -27,7 +27,7 @@ def build_apply_account_email(my_info) EOF mkdir -p ~/.config/compass-ci/include/lab cat >> ~/.config/compass-ci/include/lab/#{ENV['lab']}.yaml <<-EOF - my_uuid: #{my_info['my_uuid']} + my_token: #{my_info['my_token']} EOF 2) download lkp-tests and dependencies -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/defconfig.rb: add administrator account configuration files
by Liu Yinsi 14 Dec '20

14 Dec '20

sbin/build-my-info.rb need to use configuration files /etc/compass-ci/account/*.yaml, so add it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index 482dbc8..af7e248 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -8,6 +8,7 @@ require 'yaml' def cci_defaults hash = {} Dir.glob(['/etc/compass-ci/defaults/*.yaml', + '/etc/compass-ci/account/*.yaml', "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| hash.update YAML.load_file(file) || {} end -- 2.23.0

1 0

[PATCH compass-ci] docker-rootfs/setup-image: set the system time zone
by Wang Chenglong 14 Dec '20

14 Dec '20

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker-rootfs/setup-image | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/container/docker-rootfs/setup-image b/container/docker-rootfs/setup-image index 8e2e88a..787a42c 100755 --- a/container/docker-rootfs/setup-image +++ b/container/docker-rootfs/setup-image @@ -13,6 +13,11 @@ remove_file() rm -rf /.dockerenv } +set_localtime() +{ + ln -fs /usr/share/zoneinfo/Asia/Shanghai /etc/localtime +} + setup_login() { [ -n "$ROOT_PASSWD" ] || return @@ -32,5 +37,6 @@ pack_cgz() install_packages remove_file +set_localtime setup_login pack_cgz -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result: bisect email use new content
by Cao Xueliang 14 Dec '20

14 Dec '20

1. Add software name and first error message in subject 2. Add gcc version, error messages and job url in mail body Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 2 +- lib/mail_bisect_result.rb | 20 +++++++++----------- 2 files changed, 10 insertions(+), 12 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 189edb8..97ad467 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -95,7 +95,7 @@ class GitBisect error_messages = ErrorMessages.new(@build_pkg_dir).obtain_error_messages_by_errorid(@error_id) return Hash['repo' => @upstream_repo, 'commit' => first_bad_commit, - 'error_messages' => error_messages.join("\n")] + 'error_messages' => error_messages] end # first search the good commit in db diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 5a208c4..44a5645 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -19,23 +19,21 @@ class MailBisectResult end def compose_mail - subject = "[Compass-CI] #{(a)repo}.git: bisect result" + subject = "[Compass-CI][#{(a)repo.split('/')[1]}]: #{@error_messages[0]}" + job_url = "job url: http://183.134.196.212:11300/#{ENV['result_root']}\n" ? ENV['result_root'] : '' body = <<~BODY Hi #{(a)git_commit.author_name}, - Bisect completed for + git url: #{(a)git_commit.url} + git commit: #{@commit_id[0..11]} ("#{(a)git_commit.subject}") - url: #{(a)git_commit.url} - - This is a bisect email from compass-ci. We met some problems when test with new commits. - Would you help to check what happend? - After submitting a job we noticed an error response due to the commit: - - commit: #{@commit_id[0..11]} ("#{(a)git_commit.subject}") + gcc version: 7.3.0 error_messages: - #{@error_messages} + #{(a)error_messages.join("\n")} - https://gitee.com/openeuler/compass-ci + #{job_url} + Regards, + Compass CI team BODY to = 'caoxl(a)crystal.ci' @hash = { 'to' => to, 'body' => body, 'subject' => subject } -- 2.23.0

2 2

[PATCH lkp-tests] sbin/set-env.sh: modify script tightening specification
by Hu Xuejiao 14 Dec '20

14 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/set-env.sh | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/sbin/set-env.sh b/sbin/set-env.sh index 0b197afc..27471bb5 100644 --- a/sbin/set-env.sh +++ b/sbin/set-env.sh @@ -6,10 +6,10 @@ write_shell_profile() { shell_profile_file=$(shell_profile) if [ $# -gt 0 ]; then - echo $@ >> $shell_profile_file + echo $@ >> $shell_profile_file else - echo "export LKP_SRC=$PWD" >> $shell_profile_file - echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $shell_profile_file + echo "export LKP_SRC=$PWD" >> $shell_profile_file + echo "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" >> $shell_profile_file fi source $shell_profile_file @@ -18,11 +18,11 @@ write_shell_profile() write_host() { if is_system "Linux"; then - nr_cpu=$(nproc) - memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') + nr_cpu=$(nproc) + memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') else - nr_cpu=$(sysctl -n hw.logicalcpu) - memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') + nr_cpu=$(sysctl -n hw.logicalcpu) + memory_total=$(top -l 1 | grep MemRegions | awk '{print $2}') fi memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024}') memory=$(echo $memory_new | awk '{print int($0)+1}')G -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: add configuration files
by Liu Yinsi 14 Dec '20

14 Dec '20

scheduler need to read administrator account configuration info, such as account autosubmit. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/job.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job.rb b/lib/job.rb index b41d55a7c..e6a96fd1f 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -427,6 +427,7 @@ class Job end Dir.glob(["/etc/compass-ci/defaults/*.yaml", + "/etc/compass-ci/account/*.yaml", "#{ENV['HOME']}/.config/compass-ci/defaults/*.yaml"]).each do |file| load_one_defaults(file, @job) end -- 2.23.0

1 0

[PATCH compass-ci 2/2] container/auto-submit: fix failed to submit job in auto-submit container
by Liu Yinsi 14 Dec '20

14 Dec '20

[why] when submit job in container auto-submit, error message: /c/lkp-tests/sbin $ ./submit testbox=vm-2p8g iperf.yaml submit /c/lkp-tests/jobs/iperf.yaml failed, got job_id=0, error: Missing required job key: 'my_email' because auto-submit not mount directory /etc/compass-ci/account, it can't find autosubmit.yaml [how] mount /etc/compass-ci/account in auto-submit/start Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/auto-submit/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/auto-submit/start b/container/auto-submit/start index 4632ab0..c25fbdd 100755 --- a/container/auto-submit/start +++ b/container/auto-submit/start @@ -21,6 +21,7 @@ cmd = %W[ -v #{ENV['LKP_SRC']}:#{DOCKER_LKP} -v /etc/localtime:/etc/localtime:ro -v /etc/compass-ci/defaults:/etc/compass-ci/defaults:ro + -v /etc/compass-ci/account:/etc/compass-ci/account:ro -v /srv/git:/srv/git -w /c/compass-ci/sbin alpine:auto-submit -- 2.23.0

1 0