- Compass-ci - mailweb.openeuler.org

[PATCH v6 compass-ci 3/3] kernel_version.md: change some detail to make this doc better
by Xu Xijian 13 Nov '20

13 Nov '20

1. add default vmlinuz in Related files example. 2. change the testbox's value and add queue key-value in submit cmd. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index f6a5f4d..3cf5e26 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -34,6 +34,7 @@ Related files: ├── modules-4.19.90-2003.cgz ├── modules.cgz -> modules-4.19.90-2003.cgz ├── vmlinuz-4.19.90-2003 +└── vmlinuz -> vmlinuz-4.19.90-2003 Usage example: -- submit iperf.yaml testbox=vm-2p8g--$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 +- submit iperf.yaml testbox=vm-2p8g queue=vm-2p8g~$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 -- 2.23.0

2 2

[PATCH v6 compass-ci 1/3] fix: avoid to parse iscsiroot in setup-dracut.sh
by Xu Xijian 13 Nov '20

13 Nov '20

[why] When start with a new kernel, the dracut will parse iscsiroot and halted, which is unnecessary for us. [errmsg] [ 0.976307] dracut: FATAL: iscsiroot requested but kernel/initrd does not support iscsi [ 0.977011] dracut: Refusing to continue ... [ 1.063137] reboot: System halted Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 2e676de..b9a7800 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -14,5 +14,6 @@ rm -rf /var/lib/apt/lists/* # Replace the runtime shell script with a custom shell script cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ +sed -i '/inst_hook cmdline 90 "$moddir\/parse-iscsiroot.sh"/d' /usr/lib/dracut/modules.d/95iscsi/module-setup.sh cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh -- 2.23.0

4 5

[PATCH v3 lkp-tests] test/cci-makepkg: fix a softlink will be established if the package fails
by Sun Yukui 13 Nov '20

13 Nov '20

[why] After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the softlink should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a105f189..c6fce675 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -92,10 +92,6 @@ cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy -echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf -PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf - update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return @@ -109,4 +105,13 @@ update_softlink() rm $sync_dest/$cgz_name } -update_softlink && echo "Makepkg finished successfully" +build_source_pkg() +{ + echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + + [ "$?" == 0 ] || update_softlink +} + +build_source_pkg -- 2.23.0

1 0

[PATCH lkp-tests] test/build-pkg: fix a softlink will be established if the package fails
by Sun Yukui 13 Nov '20

13 Nov '20

[why] After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the soft connection should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index ad8d4843..093bbe4d 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,19 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -86,24 +99,11 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 -} -create_softlink() -{ - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" + [ "$?" == 0 ] || create_softlink } check_vars mount_dest request_pkg build_source_pkg -create_softlink -- 2.23.0

2 1

[PATCH v4 lkp-tests] tats: add hackbench-git result parsing script
by Zhang Yale 13 Nov '20

13 Nov '20

[input] --------socket thread num=20-------- 2020-11-10 19:51:31 /lkp/benchmarks/hackbench-git/hackbench/hackbench 20 thread 1000 Running with 20*40 (== 800) tasks. Time: 0.495 ... 2020-11-10 19:51:38 /lkp/benchmarks/hackbench-git/hackbench/hackbench 20 thread 1000 Running with 20*40 (== 800) tasks. Time: 0.512 [output] socket_thread_num_800: 0.5016 Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/hackbench-git | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 stats/hackbench-git diff --git a/stats/hackbench-git b/stats/hackbench-git new file mode 100755 index 00000000..187f1c19 --- /dev/null +++ b/stats/hackbench-git @@ -0,0 +1,20 @@ +#!/usr/bin/env ruby + +LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) + +require "#{LKP_SRC}/lib/statistics" + +time = [] +while (line = gets) + case line + when /^-+(\w+ \w+ \w+)/ + args = $1.tr(' ', '_') + when /^Running with .* $== (\d+)$ tasks/ + tasks = $1.to_i + when /^Time:/ + seconds = line.split[1] + time << seconds.to_f + end +end + +puts "#{args}_#{tasks}: #{time.sum / time.size}" -- 2.23.0

1 1

[PATCH v4 compass-ci 4/4] assign-account: build send account email
by Luan Shengde 13 Nov '20

13 Nov '20

build email message for successfuly assign account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../build-send-account-email.rb | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100755 container/assign-account/build-send-account-email.rb diff --git a/container/assign-account/build-send-account-email.rb b/container/assign-account/build-send-account-email.rb new file mode 100755 index 0000000..3c41404 --- /dev/null +++ b/container/assign-account/build-send-account-email.rb @@ -0,0 +1,32 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_message(email, account_info) + message = <<~EMAIL_MESSAGE + To: #{email} + Subject: [compass-ci] jumper account is ready + + Dear user: + + Thank you for joining us. + You can use the following command to login the jumper server: + + Login command: + ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} + + Account password: + #{account_info['my_password']} + + You can use your pub_key to login if you have offered one. + + Suggest: + If you use the password to login, change it in time. + + regards + compass-ci + EMAIL_MESSAGE + + return message +end -- 2.23.0

1 0

[PATCH v4 compass-ci 3/4] assign-account: generate ssh keys
by Luan Shengde 13 Nov '20

13 Nov '20

generate private/public key on jumper server and return public key [why] register the public key to es, final register it to testbox enable user one click to login in the testbox [how] generate rsa private/public key for user on jumper server return and register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 41 +++++++++++--------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..3cb583b 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -34,12 +34,13 @@ setup_jumper_account_info read_jumper_info config_default_yaml config_authorized_key + generate_ssh_key the returned data for setup_jumper_account_info like: { - "account" => "guest", - "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "my_login_name" => "test_user", + "password" => "Use pub_key to login", + "jumper_host" => "0.0.0.0", "jumper_port" => "10000" } @@ -94,30 +95,33 @@ class AccountStorage end def setup_jumper_account_info - account_info = read_account_info - jumper_info = read_jumper_info - pub_key = @data['my_ssh_pubkey'] unless @data['my_ssh_pubkey'].nil? + login_name, password = read_account_info + jumper_host, jumper_port = read_jumper_info + pub_key = @data['my_ssh_pubkey'][0] - login_name = account_info[0] - password = if pub_key.nil? - account_info[1] - else - 'Use pub_key to login' - end + ssh_dir = File.join('/home/', login_name, '.ssh') + config_authorized_key(login_name, pub_key, ssh_dir) unless pub_key.nil? + config_default_yaml(login_name) + my_jumper_pubkey = generate_ssh_key(login_name, ssh_dir) if @data['gen_sshkey'].eql? true jumper_account_info = { 'my_login_name' => login_name, 'my_password' => password, - 'jumper_host' => jumper_info[0].chomp, - 'jumper_port' => jumper_info[1].chomp + 'jumper_host' => jumper_host, + 'jumper_port' => jumper_port, + 'my_jumper_pubkey' => my_jumper_pubkey } - config_authorized_key(login_name, pub_key) unless pub_key.nil? - config_default_yaml(login_name) - return jumper_account_info end + def generate_ssh_key(login_name, ssh_dir) + Dir.mkdir ssh_dir, 0o700 unless File.exist? ssh_dir + %x(ssh-keygen -f "#{ssh_dir}/id_rsa" -N '' -C "#{login_name}@account-vm") + %x(chown -R #{login_name}:#{login_name} #{ssh_dir}) + File.read("/home/#{login_name}/.ssh/id_rsa.pub") + end + def config_default_yaml(login_name) default_yaml_dir = File.join('/home', login_name, '.config/compass-ci/defaults') FileUtils.mkdir_p default_yaml_dir @@ -132,8 +136,7 @@ class AccountStorage %x(chown -R #{login_name}:#{login_name} "/home/#{login_name}/.config") end - def config_authorized_key(login_name, pub_key) - ssh_dir = File.join('/home/', login_name, '.ssh') + def config_authorized_key(login_name, pub_key, ssh_dir) Dir.mkdir ssh_dir, 0o700 Dir.chdir ssh_dir f = File.new('authorized_keys', 'w') -- 2.23.0

1 0

[PATCH v4 compass-ci 2/4] assign-account: add new option to generate ssh key
by Luan Shengde 13 Nov '20

13 Nov '20

add -g|--gen-sshkey option for answerback-email [why] enable one click login testbox from jumper server [how] add add -g|--gen-sshkey option to enable generate private/public key on jumper server register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 81 ++++++++++---------- 1 file changed, 40 insertions(+), 41 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 686a327..cf1b852 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -7,12 +7,22 @@ # samba mount # ssh logshn (huawei, ) (install pubkey / send password) +# entry point1: external user: email apply => mail robot +# entry point2: ./answerback-email: cli apply +# => params +# => REST api (params different) +# => robot email notify +# user_type: +# apply_from: mail/cli +# allow_login: + require 'json' require 'mail' require 'set' require 'optparse' require_relative '../defconfig' require_relative '../../lib/es_client' +require_relative 'build-send-account-email' names = Set.new %w[ JUMPER_HOST @@ -31,41 +41,49 @@ SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 my_info = { 'my_email' => nil, 'my_name' => nil, + 'my_commit_url' => nil, 'my_uuid' => %x(uuidgen).chomp, - 'my_ssh_pubkey' => nil + 'my_ssh_pubkey' => [], + 'gen_sshkey' => false } -def init_info(email_file, my_info) - mail_content = Mail.read(email_file) +def init_info(mail_content, my_info) my_info['my_email'] = mail_content.from[0] my_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') - my_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded - end + return if mail_content.attachments.empty? + + my_info['my_ssh_pubkey'] << mail_content.attachments[0].body.decoded end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey]\n" opts.banner += " -e or -f is required\n" - opts.banner += ' -s is optional when use -e' + opts.banner += " -s is optional when use -e\n" + opts.banner += ' -g is optional, used to generate sshkey for user' opts.separator '' opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| my_info['my_email'] = email_address - # when apply account with email address, will get no user name + # when apply account with email address, will get no user name and commit url my_info['my_name'] = '' + my_info['my_commit_url'] = '' end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| - my_info['my_ssh_pubkey'] = File.read(pub_key_file) + my_info['my_ssh_pubkey'] << File.read(pub_key_file).chomp end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| - init_info(email_file, my_info) + mail_content = Mail.read(email_file) + init_info(mail_content, my_info) + end + + opts.on('-g', '--gen-sshkey', 'generate jumper ras public/private key and return pubkey') do + my_info['gen_sshkey'] = true end opts.on_tail('-h', '--help', 'show this message') do @@ -76,32 +94,6 @@ end options.parse!(ARGV) -def build_message(email, account_info) - message = <<~EMAIL_MESSAGE - To: #{email} - Subject: [compass-ci] jumper account is ready - - Dear user: - - Thank you for joining us. - You can use the following command to login the jumper server: - - Login command: - ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} - - Account password: - #{account_info['my_password']} - - Suggest: - If you use the password to login, change it in time. - - regards - compass-ci - EMAIL_MESSAGE - - return message -end - def apply_account(my_info) account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{my_info.to_json}') JSON.parse account_info_str @@ -114,12 +106,19 @@ def send_account(my_info) raise message if my_info['my_email'].nil? account_info = apply_account(my_info) - # for manually assign account, there will be no my_commit_url - # but the key my_commit_url is required for es - my_info['my_commit_url'] = '' my_info['my_login_name'] = account_info['my_login_name'] - my_info.delete 'my_ssh_pubkey' + + unless account_info['my_jumper_pubkey'].nil? + my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'].chomp + end + + my_info.delete 'gen_sshkey' store_account_info(my_info) + + send_mail(my_info, account_info) +end + +def send_mail(my_info, account_info) message = build_message(my_info['my_email'], account_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") -- 2.23.0

1 0

[PATCH v4 compass-ci 1/4] assign-account: Dockerfile add install ssh
by Luan Shengde 13 Nov '20

13 Nov '20

add install ssh to enable execute ssh-keygen in container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/assign-account/Dockerfile b/container/assign-account/Dockerfile index f84b5e6..12faafa 100644 --- a/container/assign-account/Dockerfile +++ b/container/assign-account/Dockerfile @@ -2,13 +2,13 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM debian -MAINTAINER luanshd0525(a)163.com +MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ - apt-get install -y ruby-sinatra ruby-json curl && \ + apt-get install -y ssh ruby-sinatra ruby-json curl && \ ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ echo 'Asia/Shanghai' > /etc/timezone COPY assign-account.rb /usr/local/bin/ -- 2.23.0

1 0

[PATCH v2 compass-ci] container: fix crystal-compiler/install error
by Liu Yinsi 13 Nov '20

13 Nov '20

when execute crystal-compiler/install, error message: ln: failed to create symbolic link '/usr/local/bin/crystal': File exists Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/crystal-compiler/install | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/crystal-compiler/install b/container/crystal-compiler/install index 0fa1943..4ec5c47 100755 --- a/container/crystal-compiler/install +++ b/container/crystal-compiler/install @@ -3,4 +3,4 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. DIR=$(dirname $(realpath $0)) -sudo ln -s $DIR/run /usr/local/bin/crystal +sudo ln -sf $DIR/run /usr/local/bin/crystal -- 2.23.0

1 0