- Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 1/4] assign-account: Dockerfile add install ssh
by Luan Shengde 13 Nov '20

13 Nov '20

add install ssh to enable execute ssh-keygen in container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/assign-account/Dockerfile b/container/assign-account/Dockerfile index f84b5e6..12faafa 100644 --- a/container/assign-account/Dockerfile +++ b/container/assign-account/Dockerfile @@ -2,13 +2,13 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM debian -MAINTAINER luanshd0525(a)163.com +MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ - apt-get install -y ruby-sinatra ruby-json curl && \ + apt-get install -y ssh ruby-sinatra ruby-json curl && \ ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ echo 'Asia/Shanghai' > /etc/timezone COPY assign-account.rb /usr/local/bin/ -- 2.23.0

1 0

[PATCH v2 compass-ci] container: fix crystal-compiler/install error
by Liu Yinsi 13 Nov '20

13 Nov '20

when execute crystal-compiler/install, error message: ln: failed to create symbolic link '/usr/local/bin/crystal': File exists Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/crystal-compiler/install | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/crystal-compiler/install b/container/crystal-compiler/install index 0fa1943..4ec5c47 100755 --- a/container/crystal-compiler/install +++ b/container/crystal-compiler/install @@ -3,4 +3,4 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. DIR=$(dirname $(realpath $0)) -sudo ln -s $DIR/run /usr/local/bin/crystal +sudo ln -sf $DIR/run /usr/local/bin/crystal -- 2.23.0

1 0

[PATCH compass-ci 1/2] delimiter: set a timeout for monitor
by Cao Xueliang 13 Nov '20

13 Nov '20

[why] If the job is suspended for some reason, the delimiter thread will monitor it all the time, so, it should set a timeout for monitor. Now set the timeout to 3600s. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/constants.rb | 1 + src/delimiter/utils.rb | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index 93b633d..5257427 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -6,6 +6,7 @@ START_PROCESS_COUNT = 10 GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' +MONITOR_TIMEOUT = 3600 TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index 679c8d9..36954a1 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -55,7 +55,7 @@ module Utils monitor = Monitor.new("ws://#{MONITOR_HOST}:#{MONITOR_PORT}/filter") monitor.query = query monitor.action = { 'stop' => true } - return monitor.run + return monitor.run(MONITOR_TIMEOUT) end def submit_job(job) -- 2.23.0

2 3

[PATCH compass-ci] container: fix crystal-compiler/install error
by Liu Yinsi 13 Nov '20

13 Nov '20

[why] when execute crystal-compiler/install, error: ln: failed to create symbolic link '/usr/local/bin/crystal': File exists Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/crystal-compiler/install | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/crystal-compiler/install b/container/crystal-compiler/install index 0fa1943..4ec5c47 100755 --- a/container/crystal-compiler/install +++ b/container/crystal-compiler/install @@ -3,4 +3,4 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. DIR=$(dirname $(realpath $0)) -sudo ln -s $DIR/run /usr/local/bin/crystal +sudo ln -sf $DIR/run /usr/local/bin/crystal -- 2.23.0

2 2

[PATCH lab-z9] hosts: fix testbox included yaml error
by Wei Jihui 13 Nov '20

13 Nov '20

[Why] submit job with some physical machine's testbox will cause error: Included yaml file not found: 'hosts/taishan200-2280-2s48p-256g' (RuntimeError) [How] lkp-tests just include files that find $LKP_SRC and testbox's directory, so delete hosts is ok. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- hosts/taishan200-2280-2s48p-256g--a103 | 2 +- hosts/taishan200-2280-2s48p-256g--a104 | 2 +- hosts/taishan200-2280-2s48p-256g--a106 | 2 +- hosts/taishan200-2280-2s48p-256g--a107 | 2 +- hosts/taishan200-2280-2s48p-512g--a105 | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/hosts/taishan200-2280-2s48p-256g--a103 b/hosts/taishan200-2280-2s48p-256g--a103 index 304c6fc..9fcc718 100644 --- a/hosts/taishan200-2280-2s48p-256g--a103 +++ b/hosts/taishan200-2280-2s48p-256g--a103 @@ -1,4 +1,4 @@ -<<: hosts/taishan200-2280-2s48p-256g +<<: taishan200-2280-2s48p-256g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b88bdc69 diff --git a/hosts/taishan200-2280-2s48p-256g--a104 b/hosts/taishan200-2280-2s48p-256g--a104 index a8a2789..41cdbb0 100644 --- a/hosts/taishan200-2280-2s48p-256g--a104 +++ b/hosts/taishan200-2280-2s48p-256g--a104 @@ -1,4 +1,4 @@ -<<: hosts/taishan200-2280-2s48p-256g +<<: taishan200-2280-2s48p-256g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8892431 diff --git a/hosts/taishan200-2280-2s48p-256g--a106 b/hosts/taishan200-2280-2s48p-256g--a106 index ad587b3..95b2f05 100644 --- a/hosts/taishan200-2280-2s48p-256g--a106 +++ b/hosts/taishan200-2280-2s48p-256g--a106 @@ -1,4 +1,4 @@ -<<: hosts/taishan200-2280-2s48p-256g +<<: taishan200-2280-2s48p-256g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b883bf99 diff --git a/hosts/taishan200-2280-2s48p-256g--a107 b/hosts/taishan200-2280-2s48p-256g--a107 index fba1937..31b8dc6 100644 --- a/hosts/taishan200-2280-2s48p-256g--a107 +++ b/hosts/taishan200-2280-2s48p-256g--a107 @@ -1,4 +1,4 @@ -<<: hosts/taishan200-2280-2s48p-256g +<<: taishan200-2280-2s48p-256g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b8917c39 diff --git a/hosts/taishan200-2280-2s48p-512g--a105 b/hosts/taishan200-2280-2s48p-512g--a105 index 096b5b0..66a0aca 100644 --- a/hosts/taishan200-2280-2s48p-512g--a105 +++ b/hosts/taishan200-2280-2s48p-512g--a105 @@ -1,4 +1,4 @@ -<<: hosts/taishan200-2280-2s48p-512g +<<: taishan200-2280-2s48p-512g nr_hdd_partitions: 4 hdd_partitions: - /dev/disk/by-id/scsi-350000399b891945d -- 2.23.0

2 1

[PATCH lkp-tests] stats: add sysbench-mysql result parsing file
by Zhang Yu 13 Nov '20

13 Nov '20

[input] SQL statistics: queries performed: read: 208714422 write: 59632689 other: 29816345 total: 298163456 transactions: 14908172 (25024.26 per sec.) queries: 298163456 (496918.82 per sec.) ... [output] transactions: 25024.26 Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- stats/sysbench-mysql | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100755 stats/sysbench-mysql diff --git a/stats/sysbench-mysql b/stats/sysbench-mysql new file mode 100755 index 00000000..d58d089a --- /dev/null +++ b/stats/sysbench-mysql @@ -0,0 +1,34 @@ +#!/usr/bin/env ruby + +#SQL statistics: +# queries performed: +# read: 210212086 +# write: 60060591 +# other: 30030296 +# total: 300302973 +# transactions: 15015147 (25024.26 per sec.) +# queries: 300302973 (500485.16 per sec.) +# ignored errors: 2 (0.00 per sec.) +# reconnects:0 (0.00 per sec.) +# +#General statistics: +# total time: 600.0222s +# total number of events: 15015147 +# +#Latency (ms): +# min: 2.74 +# avg: 10.23 +# max: 2238.49 +# 95th percentile: 15.55 +# sum: 153573603.78 +# +#Threads fairness: +# events (avg/stddev): 58652.9180/4292.14 +# execution time (avg/stddev): 599.8969/0.01 + +while (line = STDIN.gets) + case line.chomp! + when /transactions:\s+\d+\s+\((\d+.?\d+)/ + puts "transactions: #{$1}" + end +end -- 2.23.0

3 5

[PATCH v3 compass-ci 4/4] ssign-account: build send account email
by Luan Shengde 13 Nov '20

13 Nov '20

build email message for successfuly assign account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../assign-account/send-account-email.rb | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100755 container/assign-account/send-account-email.rb diff --git a/container/assign-account/send-account-email.rb b/container/assign-account/send-account-email.rb new file mode 100755 index 0000000..7fe1447 --- /dev/null +++ b/container/assign-account/send-account-email.rb @@ -0,0 +1,30 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_message(email, account_info) + message = <<~EMAIL_MESSAGE + To: #{email} + Subject: [compass-ci] jumper account is ready + + Dear user: + + Thank you for joining us. + You can use the following command to login the jumper server: + + Login command: + ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} + + Account password: + #{account_info['my_password']} + + Suggest: + If you use the password to login, change it in time. + + regards + compass-ci + EMAIL_MESSAGE + + return message +end -- 2.23.0

2 2

[PATCH v3 compass-ci 3/4] assign-account: generate rsa keys
by Luan Shengde 13 Nov '20

13 Nov '20

generate rsa private/public key and return public key on jumper server [why] register the public key on testbox enable user one click to login in the testbox [how] generate rsa private/public key for user on jumper server return and register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 43 +++++++++++--------- 1 file changed, 24 insertions(+), 19 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..9cf6044 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -34,12 +34,13 @@ setup_jumper_account_info read_jumper_info config_default_yaml config_authorized_key + generate_ssh_key the returned data for setup_jumper_account_info like: { - "account" => "guest", - "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "my_login_name" => "test_user", + "password" => "Use pub_key to login", + "jumper_host" => "0.0.0.0", "jumper_port" => "10000" } @@ -94,30 +95,35 @@ class AccountStorage end def setup_jumper_account_info - account_info = read_account_info - jumper_info = read_jumper_info - pub_key = @data['my_ssh_pubkey'] unless @data['my_ssh_pubkey'].nil? + login_name, password = read_account_info + jumper_host, jumper_port = read_jumper_info + pub_key = @data['my_ssh_pubkey'][0] - login_name = account_info[0] - password = if pub_key.nil? - account_info[1] - else - 'Use pub_key to login' - end + password = 'Use pub_key to login' unless pub_key.nil? + + ssh_dir = File.join('/home/', login_name, '.ssh') + config_authorized_key(login_name, pub_key, ssh_dir) unless pub_key.nil? + config_default_yaml(login_name) + my_jumper_pubkey = generate_ssh_key(login_name, ssh_dir) if @data['gen_sshkey'].eql? true jumper_account_info = { 'my_login_name' => login_name, 'my_password' => password, - 'jumper_host' => jumper_info[0].chomp, - 'jumper_port' => jumper_info[1].chomp + 'jumper_host' => jumper_host, + 'jumper_port' => jumper_port, + 'my_jumper_pubkey' => my_jumper_pubkey } - config_authorized_key(login_name, pub_key) unless pub_key.nil? - config_default_yaml(login_name) - return jumper_account_info end + def generate_ssh_key(login_name, ssh_dir) + Dir.mkdir ssh_dir, 0o700 unless File.exist? ssh_dir + %x(ssh-keygen -f "#{ssh_dir}/id_rsa" -N '' -C "#{login_name}@account-vm") + %x(chown -R #{login_name}:#{login_name} #{ssh_dir}) + File.read("/home/#{login_name}/.ssh/id_rsa.pub") + end + def config_default_yaml(login_name) default_yaml_dir = File.join('/home', login_name, '.config/compass-ci/defaults') FileUtils.mkdir_p default_yaml_dir @@ -132,8 +138,7 @@ class AccountStorage %x(chown -R #{login_name}:#{login_name} "/home/#{login_name}/.config") end - def config_authorized_key(login_name, pub_key) - ssh_dir = File.join('/home/', login_name, '.ssh') + def config_authorized_key(login_name, pub_key, ssh_dir) Dir.mkdir ssh_dir, 0o700 Dir.chdir ssh_dir f = File.new('authorized_keys', 'w') -- 2.23.0

2 1

[PATCH v2 lkp-tests] tats: add sysbench-mysql result parsing file
by Zhang Yu 13 Nov '20

13 Nov '20

[input] SQL statistics: queries performed: read: 208714422 write: 59632689 other: 29816345 total: 298163456 transactions: 14908172 (25024.26 per sec.) queries: 298163456 (496918.82 per sec.) ... [output] transactions: 25024.26 Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- stats/sysbench-mysql | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100755 stats/sysbench-mysql diff --git a/stats/sysbench-mysql b/stats/sysbench-mysql new file mode 100755 index 00000000..c7283126 --- /dev/null +++ b/stats/sysbench-mysql @@ -0,0 +1,34 @@ +#!/usr/bin/env ruby + +# SQL statistics: +# queries performed: +# read: 210212086 +# write: 60060591 +# other: 30030296 +# total: 300302973 +# transactions: 15015147 (25024.26 per sec.) +# queries: 300302973 (500485.16 per sec.) +# ignored errors: 2 (0.00 per sec.) +# reconnects:0 (0.00 per sec.) +# +# General statistics: +# total time: 600.0222s +# total number of events: 15015147 +# +# Latency (ms): +# min: 2.74 +# avg: 10.23 +# max: 2238.49 +# 95th percentile: 15.55 +# sum: 153573603.78 +# +# Threads fairness: +# events (avg/stddev): 58652.9180/4292.14 +# execution time (avg/stddev): 599.8969/0.01 + +while (line = STDIN.gets) + case line.chomp! + when /transactions:\s+\d+\s+\((\d+.?\d+)/ + puts "transactions: #{$1}" + end +end -- 2.23.0

3 5

Re: [PATCH v2 lkp-tests] tats: add sysbench-mysql result parsing file ^ stats
by Zhang Yale 13 Nov '20

13 Nov '20

Reply-To: In-Reply-To: <20201112101355.3076124-1-2134782174(a)qq.com> On Thu, Nov 12, 2020 at 06:13:55PM +0800, Zhang Yu wrote: >[input] > SQL statistics: > queries performed: > read: 208714422 > write: 59632689 > other: 29816345 > total: 298163456 > transactions: 14908172 (25024.26 per sec.) > queries: 298163456 (496918.82 per sec.) > ... > >[output] > transactions: 25024.26 > >Signed-off-by: Zhang Yu <2134782174(a)qq.com> >--- > stats/sysbench-mysql | 34 ++++++++++++++++++++++++++++++++++ > 1 file changed, 34 insertions(+) > create mode 100755 stats/sysbench-mysql > >diff --git a/stats/sysbench-mysql b/stats/sysbench-mysql >new file mode 100755 >index 00000000..c7283126 >--- /dev/null >+++ b/stats/sysbench-mysql >@@ -0,0 +1,34 @@ >+#!/usr/bin/env ruby >+ >+# SQL statistics: >+# queries performed: >+# read: 210212086 >+# write: 60060591 >+# other: 30030296 >+# total: 300302973 >+# transactions: 15015147 (25024.26 per sec.) >+# queries: 300302973 (500485.16 per sec.) >+# ignored errors: 2 (0.00 per sec.) >+# reconnects:0 (0.00 per sec.) >+# >+# General statistics: >+# total time: 600.0222s >+# total number of events: 15015147 >+# >+# Latency (ms): >+# min: 2.74 >+# avg: 10.23 >+# max: 2238.49 >+# 95th percentile: 15.55 >+# sum: 153573603.78 >+# >+# Threads fairness: >+# events (avg/stddev): 58652.9180/4292.14 >+# execution time (avg/stddev): 599.8969/0.01 >+ >+while (line = STDIN.gets) >+ case line.chomp! >+ when /transactions:\s+\d+\s+\((\d+.?\d+)/ >+ puts "transactions: #{$1}" >+ end >+end >-- >2.23.0 >

1 1