- Compass-ci - mailweb.openeuler.org

[PATCH v3 lkp-tests] distro/archlinux: add archlinux distro file
by Wang Yong 17 Nov '20

17 Nov '20

add archlinux pack method Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/archlinux | 103 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100755 distro/archlinux diff --git a/distro/archlinux b/distro/archlinux new file mode 100755 index 000000000..1e6d0f8a4 --- /dev/null +++ b/distro/archlinux @@ -0,0 +1,103 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +shopt -s nullglob + +. $LKP_SRC/lib/install.sh +. $LKP_SRC/distro/common +. $LKP_SRC/lib/debug.sh + +distro_install_depends() +{ + local script + local bm="$1" + local scripts=$(find $LKP_SRC/distro/depends/ -name "$bm" -o -name "${bm}.[0-9]") + + for script in $scripts + do + script=$(basename $script) + packages=$(get_dependency_packages $DISTRO $script) + + [ -z "$packages" ] && continue + + echo install packages for $script: $packages + + for pkg in $packages + do + pacman -Qqs ----noconfirm $pkg || pacman -Sy --noconfirm $pkg + done + done +} + +pack_benchmark() +{ + setup_proxy + + distro_install_depends lkp-dev + + # Process each benchmark + for BM_NAME in $benchmark + do + distro_install_depends $BM_NAME-dev || continue + echo $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ( + $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ) + done +} + +pacman_download() +{ + echo "pacman -Sw --noconfirm $*" + pacman -Sw --noconfirm "$@" +} + +pacman_pack() +{ + local target_dir=/opt/pkgs + local date=$(date +"%Y%m%d") + + local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz") + + [ "$downloaded_pkgs" ] || return + + mkdir -p $target_dir + + mv $downloaded_pkgs $target_dir + + find $target_dir | cpio --quiet -o -H newc --owner=root.root | gzip -n -9 >$pack_to/${BM_NAME}_$date.cgz + + ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + echo package installed to $pack_to/${BM_NAME}.cgz + + ls $target_dir/*.pkg.tar.gz > $pack_to/.${BM_NAME}.packages + + rm -rf $target_dir +} + +cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +} + +pack_benchmark_deps() +{ + setup_proxy + + for BM_NAME in $benchmark + do + + cleanup_downloaded_pkgs + + check_shared_package $BM_NAME + + packages=$(echo $(get_dependency_packages $DISTRO $BM_NAME)) + + if pacman_download $packages; then + pacman_pack + else + echo "failed to pack-deps $BM_NAME" >&2 + fi + done +} -- 2.23.0

1 1

[PATCH v4 compass-ci 1/4] git_mirror: pull submodule if repositories have
by Li Yuanchao 17 Nov '20

17 Nov '20

[why] Some repositories have submodules. Without these submodules, they can not be built successfully. Submodules of repositories are repositories too. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9d6a45d..55ec67d 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -56,7 +56,8 @@ class GitMirror def mirror_sync fork_info = @queue.pop - mirror_dir = "/srv/git/#{fork_info['git_repo']}.git" + mirror_dir = "/srv/git/#{fork_info['git_repo']}" + mirror_dir = "#{mirror_dir}.git" unless fork_info['is_submodule'] possible_new_refs = false if File.directory?(mirror_dir) possible_new_refs = git_fetch(mirror_dir) @@ -159,6 +160,8 @@ class MirrorMain feedback_info = @feedback_queue.pop(true) git_repo = feedback_info[:git_repo] + return if check_submodule(git_repo) + update_fork_stat(git_repo, feedback_info[:possible_new_refs]) return unless feedback_info[:possible_new_refs] @@ -219,6 +222,8 @@ class MirrorMain end def get_cur_refs(git_repo) + return if @git_info[git_repo]['is_submodule'] + mirror_dir = "/srv/git/#{git_repo}.git" show_ref_out = %x(git -C #{mirror_dir} show-ref --heads) cur_refs = { heads: {} } @@ -328,4 +333,32 @@ class MirrorMain end end end + + def handle_submodule(submodule, parent_project) + submodule.each_line do |line| + next unless line.include?('url = ') + + url = line.split(' = ')[1].chomp + git_repo = url.split('://')[1] if url.include?('://') + return unless git_repo + + @git_info[git_repo] = { 'url' => url, 'git_repo' => git_repo, 'is_submodule' => true } + fork_stat_init(git_repo) + @priority_queue.push git_repo, @priority + @priority += 1 + end + end + + def check_submodule(git_repo) + if @git_info[git_repo]['is_submodule'] + @fork_stat[git_repo][:queued] = false + return true + end + + mirror_dir = "/srv/git/#{git_repo}.git" + submodule = %x(git -C #{mirror_dir} show HEAD:.gitmodules 2>/dev/null) + return if submodule.empty? + + handle_submodule(submodule, File.basename(git_repo)) + end end -- 2.23.0

2 1

[PATCH v5 compass-ci 2/4] assign-account: add option to generate ssh key
by Luan Shengde 17 Nov '20

17 Nov '20

[why] enable one click login testbox from jumper server [how] add add -g|--gen-sshkey option to enable generate private/public key on jumper server register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 80 ++++++++++---------- 1 file changed, 38 insertions(+), 42 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 686a327..1eb2a10 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -7,12 +7,22 @@ # samba mount # ssh logshn (huawei, ) (install pubkey / send password) +# entry point1: external user: email apply => mail robot +# entry point2: ./answerback-email: cli apply +# => params +# => REST api (params different) +# => robot email notify +# user_type: +# apply_from: mail/cli +# allow_login: + require 'json' require 'mail' require 'set' require 'optparse' require_relative '../defconfig' require_relative '../../lib/es_client' +require_relative 'build-send-account-email' names = Set.new %w[ JUMPER_HOST @@ -31,41 +41,46 @@ SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 my_info = { 'my_email' => nil, 'my_name' => nil, + 'my_commit_url' => nil, 'my_uuid' => %x(uuidgen).chomp, - 'my_ssh_pubkey' => nil + 'my_ssh_pubkey' => [], + 'gen_sshkey' => false } -def init_info(email_file, my_info) - mail_content = Mail.read(email_file) +def init_info(mail_content, my_info) my_info['my_email'] = mail_content.from[0] my_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') - my_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded - end + return if mail_content.attachments.empty? + + my_info['my_ssh_pubkey'] << mail_content.attachments[0].body.decoded end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey]\n" opts.banner += " -e or -f is required\n" - opts.banner += ' -s is optional when use -e' + opts.banner += " -s is optional when use -e\n" + opts.banner += ' -g is optional, used to generate sshkey for user' opts.separator '' opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| my_info['my_email'] = email_address - # when apply account with email address, will get no user name - my_info['my_name'] = '' end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| - my_info['my_ssh_pubkey'] = File.read(pub_key_file) + my_info['my_ssh_pubkey'] << File.read(pub_key_file).chomp end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| - init_info(email_file, my_info) + mail_content = Mail.read(email_file) + init_info(mail_content, my_info) + end + + opts.on('-g', '--gen-sshkey', 'generate jumper ras public/private key and return pubkey') do + my_info['gen_sshkey'] = true end opts.on_tail('-h', '--help', 'show this message') do @@ -76,32 +91,6 @@ end options.parse!(ARGV) -def build_message(email, account_info) - message = <<~EMAIL_MESSAGE - To: #{email} - Subject: [compass-ci] jumper account is ready - - Dear user: - - Thank you for joining us. - You can use the following command to login the jumper server: - - Login command: - ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} - - Account password: - #{account_info['my_password']} - - Suggest: - If you use the password to login, change it in time. - - regards - compass-ci - EMAIL_MESSAGE - - return message -end - def apply_account(my_info) account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{my_info.to_json}') JSON.parse account_info_str @@ -114,12 +103,19 @@ def send_account(my_info) raise message if my_info['my_email'].nil? account_info = apply_account(my_info) - # for manually assign account, there will be no my_commit_url - # but the key my_commit_url is required for es - my_info['my_commit_url'] = '' my_info['my_login_name'] = account_info['my_login_name'] - my_info.delete 'my_ssh_pubkey' + + unless account_info['my_jumper_pubkey'].nil? + my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'].chomp + end + + my_info.delete 'gen_sshkey' store_account_info(my_info) + + send_mail(my_info, account_info) +end + +def send_mail(my_info, account_info) message = build_message(my_info['my_email'], account_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") -- 2.23.0

3 3

[PATCH v7 compass-ci 06/11] mail-robot: build success email
by Luan Shengde 17 Nov '20

17 Nov '20

build email message for successfully assigning account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/assign-account-email.rb | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100755 container/mail-robot/lib/assign-account-email.rb diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb new file mode 100755 index 0000000..1251d1b --- /dev/null +++ b/container/mail-robot/lib/assign-account-email.rb @@ -0,0 +1,53 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_apply_account_email(my_info) + email_msg = <<~EMAIL_MESSAGE + To: #{my_info['my_email']} + Subject: [compass-ci] Account Ready + + Dear #{my_info['my_name']}, + + Thank you for joining us. + + You can use the following info to submit jobs: + + notice: + the followings are ONE-TIME setup + + 1) setup default config + run the following command to add the below setup to default config file + + mkdir -p ~/.config/compass-ci/defaults/ + cat >> ~/.config/compass-ci/defaults/account.yaml <<-EOF + my_email: #{my_info['my_email']} + my_name: #{my_info['my_name']} + my_uuid: #{my_info['my_uuid']} + EOF + + 2) download lkp-tests and dependencies + run the following command to install the lkp-test and effect the configuration + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source ${HOME}/.bashrc && source ${HOME}/.bash_profile + + 3) submit job + reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md + + reference to 'how to write job yaml' section to write the job yaml + you can also reference to files in compass-ci/jobs as example. + + submit jobs, for example: + + submit jobs/iperf.yaml testbox=vm-2p8g~\\$USER + + regards + compass-ci + EMAIL_MESSAGE + + return email_msg +end -- 2.23.0

2 3

[PATCH v5 compass-ci 1/2] /lib: keep suite result in top for compare output
by Lu Kaiyi 17 Nov '20

17 Nov '20

[why] Currently, the compare output result default order by alphabet. If there are too many lines in result, it will be difficult for user to find the lines that starts with $suite in metric column. So, need reorder compare output result with keeping $suite item on the top of the table. [how] Divide the lines into two parts before print according to whether the line start with $suite or not, then combine output result into a whole to highlight the important info. [example] command: compare "id=z9.148096" "id=z9.148143" in z9 suites_list:["sysbench-mysql"] [before] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 ... 1266042.00 -18.3% 1034206.00 softirqs.RCU 547849.00 -27.4% 397593.00 softirqs.SCHED 159990.00 -80.5% 31181.00 softirqs.TASKLET 1617967.00 -23.0% 1245331.00 softirqs.TIMER 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 2149.23 -100.0% 0.00 time.elapsed_time 2149.23 -100.0% 0.00 time.elapsed_time.max 214458.00 -100.0% 0.00 time.involuntary_context_switches 2.00 -100.0% 0.00 time.major_page_faults 192832.00 -100.0% 0.00 time.maximum_resident_set_size ... [after] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 2894304.00 +12.8% 3265730.00 interrupts.4:GICv3.26.Level.arch_timer 11546998.00 -100.0% 0.00 interrupts.746:ITS-MSI.99092481.Edge.hns3-0000:bd:00.1-TxRx-0 7476064.00 -100.0% 0.00 interrupts.747:ITS-MSI.99092482.Edge.hns3-0000:bd:00.1-TxRx-2 ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare.rb | 16 ++++++++++------ lib/compare_matrixes.rb | 41 ++++++++++++++++++++++++++++------------- lib/matrix2.rb | 20 +++++++++++++++----- 3 files changed, 53 insertions(+), 24 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 645f682..f34e239 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -20,8 +20,8 @@ require 'yaml' def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) - matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list, options: options) + matrices_list, suite_list = create_matrices_list(condition_list) + compare_matrixes(matrices_list, suite_list, options: options) end def parse_argv(argv, common_conditions) @@ -37,12 +37,16 @@ end def create_matrices_list(conditions) matrices_list = [] + suite_list = [] es = ESQuery.new(ES_HOST, ES_PORT) conditions.each do |condition| query_results = es.multi_field_query(condition) - matrices_list << combine_query_data(query_results) + matrix, suites = combine_query_data(query_results) + matrices_list << matrix + suite_list.concat(suites) end - matrices_list + + return matrices_list, suite_list end # ------------------------------------------------------------------------------------------- @@ -58,8 +62,8 @@ end def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') - groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices, options) + groups_matrices, suites_list = create_groups_matrices_list(conditions, dims) + compare_group_matrices(groups_matrices, suites_list, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 119d42d..5039b01 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -128,7 +128,7 @@ def set_compare_values(index, values, field, success, options) end def get_values_by_field(matrixes_list, field, matrixes_size, success, options) - # get values by field, values struce example: values[0][:average] + # get values by field, values struct example: values[0][:average] # values = {} matrixes_list.length.times do |index| @@ -193,7 +193,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suite_list, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -208,6 +208,7 @@ def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, opti result_str = group_key ? "\n\n\n\n\n" + group_key : '' result_str += get_all_result_str( matrixes_values, + suite_list, matrixes_titles, matrixes_list.size, options[:theme] @@ -382,7 +383,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices, options) +def compare_group_matrices(group_matrices, suites_list, options) result_str = '' group_matrices.each do |k, v| matrices_list = [] @@ -392,9 +393,9 @@ def compare_group_matrices(group_matrices, options) matrices_list << matrix end if options[:no_print] - result_str += compare_matrixes(matrices_list, matrices_titles, k, options: options) + result_str += compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) else - print compare_matrixes(matrices_list, matrices_titles, k, options: options) + print compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) end end result_str @@ -807,26 +808,40 @@ def get_theme(matrixes_values, matrixes_titles, theme) return THEMES[:none] end -def get_all_result_str(matrixes_values, matrixes_titles, matrixes_number, theme) +def get_all_result_str(matrixes_values, suite_list, matrixes_titles, matrixes_number, theme) matrixes_titles ||= matrixes_number.times.to_a.map(&:to_s) theme = get_theme(matrixes_values, matrixes_titles, theme) return '' unless theme - failure_str = get_result_str(matrixes_values[false].sort, matrixes_titles, false, theme) - success_str = get_result_str(matrixes_values[true].sort, matrixes_titles, true, theme) + failure_str = get_result_str(matrixes_values[false].sort, suite_list, matrixes_titles, false, theme) + success_str = get_result_str(matrixes_values[true].sort, suite_list, matrixes_titles, true, theme) failure_str + success_str end -def get_result_str(values, matrixes_titles, success, theme) +def get_result_str(values, suite_list, matrixes_titles, success, theme) return '' if values.empty? + suite_set = Set.new(suite_list) result_str = "\n\n\n" common_title, compare_title = get_title_name(success) result_str += get_header(matrixes_titles, success, common_title, compare_title) + ranked_str = get_ranked_str(values, suite_set, success, theme) + result_str += ranked_str + result_str +end + +def get_ranked_str(values, suite_set, success, theme) + suite_str = '' + common_str = '' values.each do |field, matrixes| - result_str += get_values_str(matrixes, success, theme) - result_str += get_field_str(field) - result_str += "\n" + row = get_values_str(matrixes, success, theme) + row += get_field_str(field) + "\n" + field_start_with_suite = suite_set.any? { |suite| field.start_with?(suite) } + if field_start_with_suite + suite_str += row + else + common_str += row + end end - result_str + suite_str + common_str end diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 26f07fa..62d174c 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -76,7 +76,9 @@ end # } def create_matrix(job_list) matrix = {} + suites = [] job_list.each do |job| + suites << job['suite'] if job['suite'] stats = job['stats'] next unless stats @@ -89,7 +91,7 @@ def create_matrix(job_list) matrix.each_value do |value| samples_fill_missing_zeros(value, col_size) end - matrix + return matrix, suites end # input: query results from es_query @@ -110,15 +112,23 @@ end # ... # } def combine_group_query_data(query_data, dims) + suites_list = [] job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + suite_list = [] value.each do |dimension_key, jobs| - groups[group_key][dimension_key] = create_matrix(jobs) + groups[group_key][dimension_key], suites = create_matrix(jobs) + suite_list.concat(suites) end - groups.delete(group_key) if value.size < 2 + if value.size < 2 + groups.delete(group_key) + next + end + suites_list << suite_list end - groups + + return groups, suites_list end # input: @@ -139,7 +149,7 @@ def combine_group_jobs_list(query_data, groups_params, dimensions, metrics) groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) groups.each do |group_key, dims| dims.each do |dim_key, jobs| - groups[group_key][dim_key] = create_matrix(jobs) + groups[group_key][dim_key], = create_matrix(jobs) end end -- 2.23.0

2 2

[PATCH v6 compass-ci] src/lib/web_backend.rb: add date range limit feature
by Zhang Yuhang 17 Nov '20

17 Nov '20

[why] The "start_time" field need limit by date. example: GET /get_jobs?upstream_repo=a&page_size=12&start_date=2020-11-03&end_date=2020-11-03 Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- container/web-backend/web-backend | 57 +++++++++++++++++-------------- src/lib/web_backend.rb | 15 ++++++++ 2 files changed, 46 insertions(+), 26 deletions(-) diff --git a/container/web-backend/web-backend b/container/web-backend/web-backend index fcc765f..7db82ac 100755 --- a/container/web-backend/web-backend +++ b/container/web-backend/web-backend @@ -1,4 +1,6 @@ #!/usr/bin/ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true require 'sinatra' @@ -12,56 +14,59 @@ set :show_exceptions, false set :environment, :production # GET /compare_candidates -# return: {"query_conditions": {...}, "dimension": [...]} +# Response: +# - {"query_conditions": {...}, "dimension": [...]} get '/compare_candidates' do compare_candidates end # GET /compare?suite=borrow&dimension=tbox_group -# query_conditions: -# > must +# must: +# - query_conditions # - os / os_version / suite / os_arch / tbox_group -# -# dimension: -# > must +# - dimension # - os / os_version / os_arch / suite / tbox_group # -# return: compare result string / "No Data." / "No Difference." +# Response: +# - compare result string / "No Data." / "No Difference." get '/compare' do compare(params) end -# GET /get_jobs?upstream_repo=AvxToNeon/AvxToNeon&page_size=12&page_num=3 -# upstream_repo: -# > optional -# -# page_size: -# > optional +# GET /get_jobs?upstream_repo=a&page_size=12&start_date=2020-11-03&end_date=2020-11-03 +# optional: +# - upstream_repo +# - regex search +# - os +# - os_version +# - os_arch +# - suite +# - category +# - testbox +# - job_state +# - id +# - start_date +# - end_date +# - page_size # - default: 20 -# -# page_num: -# > optional +# - page_num # - default: 1 # -# return: +# Response: # - default: 20 jobs get '/get_jobs' do get_jobs(params) end # GET /get_repos?git_repo=a&page_size=12&page_num=3 -# git_repo: -# > optional -# -# page_size: -# > optional +# optional: +# - git_repo +# - page_size # - default: 20 -# -# page_num: -# > optional +# - page_num # - default: 1 # -# return: +# Response: # - default: 20 repos get '/get_repos' do get_repos(params) diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 55c5e4a..b8df6f4 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -1,3 +1,5 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true require 'json' @@ -260,6 +262,17 @@ def get_jobs_result(result) jobs end +def get_job_query_range(condition_fields) + range = { start_time: {} } + start_date = condition_fields[:start_date] + end_date = condition_fields[:end_date] + + range[:start_time][:gte] = "#{start_date} 00:00:00" if start_date + range[:start_time][:lte] = "#{end_date} 23:59:59" if end_date + + { range: range } +end + def search_job(condition_fields, page_size, page_num) must = [] FIELDS.each do |field| @@ -272,6 +285,8 @@ def search_job(condition_fields, page_size, page_num) { term: { field => value } } end end + range = get_job_query_range(condition_fields) + must << range if range[:range][:start_time] result, total = es_search(must, page_size, page_num * page_size) return get_jobs_result(result), total end -- 2.23.0

1 0

[PATCH lkp-tests] update compiling commands
by Hu Xuejiao 17 Nov '20

17 Nov '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- pkg/sysbench-threads-git/PKGBUILD | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkg/sysbench-threads-git/PKGBUILD b/pkg/sysbench-threads-git/PKGBUILD index 41ed9f46..c79d97b3 100644 --- a/pkg/sysbench-threads-git/PKGBUILD +++ b/pkg/sysbench-threads-git/PKGBUILD @@ -12,7 +12,7 @@ build() { cd "$srcdir/sysbench" ./autogen.sh - ./configure --prefix=/usr --without-gcc-arch --without-mysql + ./configure --build=arm --host=arm --without-mysql make } -- 2.23.0

2 1

[PATCH v7 compass-ci 02/11] mail-robot: check to apply account
by Luan Shengde 17 Nov '20

17 Nov '20

ApplyAccount check to apply account for user when listened new email with subject 'apply account' input: mail_content send_account parse email_content parse my_commit_url parse my_ssh_pubkey apply_my_account generate uuid apply account with my_info and my_ssh_pubkey my_info: - my_email - my_name - my_uuid store account info to es send email send success email send fail email usage: reference mail-robot.rb Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 113 ++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100755 container/mail-robot/lib/apply-account.rb diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb new file mode 100755 index 0000000..1b0f2ad --- /dev/null +++ b/container/mail-robot/lib/apply-account.rb @@ -0,0 +1,113 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require_relative '../../../lib/es_client' +require_relative 'assign-account-email' +require_relative 'assign-account-fail-eamil' +require_relative 'apply-jumper-account' +require_relative 'parse-apply-account-email' + +SEND_MAIL_PORT = ENV['SEND_MAIL_PORT'] || 49000 + +# assign uuid/account for user +# when mail-robot listened new email, and the email's subject +# exactly equal 'apply account', mail-robot will call this class +# entry point: send_account +# input: email's content +# +# send_account +# parse_commit_url_pub_key +# call ParseApplyAccountEmail to parse: +# - my_commit_url +# - my_ssh_pubkey +# apply_my_account +# call ApplyJumperAccount to apply new account +# required data: +# my_info: +# - my_email +# - my_name +# - my_uuid +# my_ssh_pubkey +# store_account_info +# call ESClient to store my_info +# my_info: +# - my_email +# - my_name +# - my_uuid +# - my_commit_url +# - my_login_name +# - my_ssh_pubkey +# send_mail +# when successfully applied an account +# call build_apply_account_email to send a successful email +# when rescued error message +# call build_apply_account_fail_email to send fail email +class ApplyAccount + def initialize(mail_content) + @send_mail_host = %x(/sbin/ip route | awk '/default/ {print $3}').chomp + @send_mail_port = SEND_MAIL_PORT + @mail_content = mail_content + + @my_info = { + 'my_email' => mail_content.from[0], + 'my_name' => mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, ''), + 'my_ssh_pubkey' => [] + } + end + + def check_to_send_account + my_commit_url, my_ssh_pubkey = parse_commit_url_pub_key + acct_info = apply_my_account(my_ssh_pubkey) + + @my_info['my_commit_url'] = my_commit_url + @my_info['my_login_name'] = acct_info['my_login_name'] + @my_info['my_ssh_pubkey'] << my_ssh_pubkey + + store_account_info + send_mail('') + rescue StandardError => e + puts e.message + puts e.backtrace + + send_mail(e.message) + end + + def parse_commit_url_pub_key + parse_apply_account_email = ParseApplyAccountEmail.new(@mail_content) + + my_commit_url = parse_apply_account_email.parse_commit_url + my_ssh_pubkey = parse_apply_account_email.parse_pub_key + + return my_commit_url, my_ssh_pubkey + end + + def apply_my_account(my_ssh_pubkey) + my_uuid = %x(uuidgen).chomp + + @my_info['my_uuid'] = my_uuid + + apply_account = ApplyJumperAccount.new(@my_info, my_ssh_pubkey) + acct_info = apply_account.apply_jumper_account + + return acct_info + end + + def store_account_info + es = ESClient.new(index: 'accounts') + es.put_source_by_id(@my_info['my_email'], @my_info) + end + + def send_mail(error_message) + email_message = if error_message.empty? + build_apply_account_email(@my_info) + else + build_apply_account_fail_email(@my_info, error_message) + end + + %x(curl -XPOST "#{@send_mail_host}:#{@send_mail_port}/send_mail_text" -d "#{email_message}") + end +end -- 2.23.0

2 2

[PATCH v5 compass-ci 2/2] lib/matrix2.rb: refactor performance optimization
by Lu Kaiyi 17 Nov '20

17 Nov '20

[why] Avoid to execute some unnecessary operations and improve performance. [how] Keep the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 62d174c..86cfc93 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

1 0

[PATCH compass-ci] git_mirror: fix that README.md in upstream-repos cause a load error
by Li Yuanchao 16 Nov '20

16 Nov '20

[why] As README.md or other docs in upstream-repos, and they are not yaml file. But git-mirror load all files in upstream-repos as yaml file, so there comes out a load error. Now files whose dirname is 'xxx/upstream-repos' will not be loaded. [errmsg] in `load_repo_file': undefined method `[]=' for x:Integer (NoMethodError) Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 9648063..72ca328 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -135,6 +135,8 @@ class MirrorMain traverse_repodir("#{repodir}/#{entry}") end else + return if File.dirname(repodir) == REPO_DIR + project = File.dirname(repodir).delete_prefix("#{REPO_DIR}/") fork_name = File.basename(repodir) load_repo_file(repodir, project, fork_name) -- 2.23.0

1 0

[PATCH compass-ci] container/minio: run minio container
by Cui Yili 16 Nov '20

16 Nov '20

Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/minio/start | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100755 container/minio/start diff --git a/container/minio/start b/container/minio/start new file mode 100755 index 0000000..348bff7 --- /dev/null +++ b/container/minio/start @@ -0,0 +1,18 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ../defconfig.sh +docker_rm minio + +cmd=( + docker run -d + -p 11321:9000 + --name minio + -v /srv/minio/:/srv/minio/ + -e "MINIO_ACCESS_KEY=${MINIO_ACCESS_KEY}" + -e "MINIO_SECRET_KEY=${MINIO_SECRET_KEY}" + jessestuart/minio server /srv/minio/ +) + +"${cmd[@]}" -- 2.23.0

2 1

[PATCH lkp-tests] tests/build-pkg: add download PKGBUILD-$tag function
by Liu Shaofei 16 Nov '20

16 Nov '20

[why] Support more version's PKGBUILD to build different version software. [how] Download the PKGBUILD-$tag file through remote-git server from "/srv/git/pkg/" usage: 1) append upstream_tag field from build-pkg.yaml 2) create git repo that named by software name of $upstream_repo in "/srv/git/pkg/", and add PKGBUILD-$tag file to the directory of "/srv/git/pkg/$upstream_repo". if not PKGBUILD-$tag, the software is built with PKGBUILD. Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- tests/build-pkg | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 1eed784f..5d2fb792 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -61,6 +61,15 @@ request_pkg() get_pkgfile "${pkgrepo}" "$pkgfile" done + # download PKGBUILD-$tag + benchmark="${upstream_repo##*/}" + benchmark="${benchmark%.git}" + [ -n "$upstream_tag" ] && { + get_pkgfile "pkg/$benchmark" "PKGBUILD-$upstream_tag" + [ -f "PKGBUILD-$upstream_tag" ] || die "PKGBUILD-$upstream_tag not exist" + [ -s "PKGBUILD-$upstream_tag" ] || die "PKGBUILD-$upstream_tag is empty" + } + [ -f PKGBUILD ] || die "PKGBUILD not exist" [ -s PKGBUILD ] || die "PKGBUILD is empty" } @@ -80,7 +89,10 @@ build_source_pkg() fi upstream_source="source=(\"${repo_dir}git://${LKP_SERVER}/${upstream_repo}#commit=${upstream_commit}\")" - sed -i "s|^source=.*|${upstream_source}|g" PKGBUILD + sed -i "s|^source=.*|${upstream_source}|g" "PKGBUILD" + [ -n "$upstream_tag" ] && { + sed -i "s|^source=.*|${upstream_source}|g" "PKGBUILD-$upstream_tag" + } } cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" -- 2.23.0

1 0

[PATCH lkp-tests 1/4] test/build-pkg: Refactor move create_softlink function position
by Sun Yukui 16 Nov '20

16 Nov '20

Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 29 +++++++++++++---------------- 1 file changed, 13 insertions(+), 16 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 1eed784f..2ec45b9c 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,19 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -88,22 +101,6 @@ build_source_pkg() $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 } -create_softlink() -{ - [ -e "$cgz_name" ] || return 0 - - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" -} - check_vars mount_dest request_pkg -- 2.23.0

2 2

[PATCH v7 compass-ci] assign-account: add option to generate ssh key
by Luan Shengde 16 Nov '20

16 Nov '20

[why] enable one click login testbox from jumper server [how] add -g|--gen-sshkey option to enable generate private/public key on jumper server register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 71 ++++++++------------ 1 file changed, 29 insertions(+), 42 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 686a327..5ddc451 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -13,6 +13,7 @@ require 'set' require 'optparse' require_relative '../defconfig' require_relative '../../lib/es_client' +require_relative 'build-send-account-email' names = Set.new %w[ JUMPER_HOST @@ -31,41 +32,46 @@ SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 my_info = { 'my_email' => nil, 'my_name' => nil, + 'my_commit_url' => nil, 'my_uuid' => %x(uuidgen).chomp, - 'my_ssh_pubkey' => nil + 'my_ssh_pubkey' => [], + 'gen_sshkey' => false } -def init_info(email_file, my_info) - mail_content = Mail.read(email_file) +def init_info(mail_content, my_info) my_info['my_email'] = mail_content.from[0] my_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') - my_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded - end + return if mail_content.attachments.empty? + + my_info['my_ssh_pubkey'] << mail_content.attachments[0].body.decoded end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey]\n" opts.banner += " -e or -f is required\n" - opts.banner += ' -s is optional when use -e' + opts.banner += " -s is optional when use -e\n" + opts.banner += ' -g is optional, used to generate sshkey for user' opts.separator '' opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| my_info['my_email'] = email_address - # when apply account with email address, will get no user name - my_info['my_name'] = '' end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| - my_info['my_ssh_pubkey'] = File.read(pub_key_file) + my_info['my_ssh_pubkey'] << File.read(pub_key_file).chomp end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| - init_info(email_file, my_info) + mail_content = Mail.read(email_file) + init_info(mail_content, my_info) + end + + opts.on('-g', '--gen-sshkey', 'generate jumper ras public/private key and return pubkey') do + my_info['gen_sshkey'] = true end opts.on_tail('-h', '--help', 'show this message') do @@ -76,32 +82,6 @@ end options.parse!(ARGV) -def build_message(email, account_info) - message = <<~EMAIL_MESSAGE - To: #{email} - Subject: [compass-ci] jumper account is ready - - Dear user: - - Thank you for joining us. - You can use the following command to login the jumper server: - - Login command: - ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} - - Account password: - #{account_info['my_password']} - - Suggest: - If you use the password to login, change it in time. - - regards - compass-ci - EMAIL_MESSAGE - - return message -end - def apply_account(my_info) account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{my_info.to_json}') JSON.parse account_info_str @@ -114,12 +94,19 @@ def send_account(my_info) raise message if my_info['my_email'].nil? account_info = apply_account(my_info) - # for manually assign account, there will be no my_commit_url - # but the key my_commit_url is required for es - my_info['my_commit_url'] = '' my_info['my_login_name'] = account_info['my_login_name'] - my_info.delete 'my_ssh_pubkey' + + unless account_info['my_jumper_pubkey'].nil? + my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'].chomp + end + + my_info.delete 'gen_sshkey' store_account_info(my_info) + + send_mail(my_info, account_info) +end + +def send_mail(my_info, account_info) message = build_message(my_info['my_email'], account_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") -- 2.23.0

1 0

[PATCH v4 lkp-tests] sbin/makepkg: download submodules code from LKP server
by Liu Shaofei 16 Nov '20

16 Nov '20

Some repositories have submodules. So when build packages, we need download submodules from LKP server, avoid the waste of time in downloading files from the remote end. Signed-off-by: Liu Shaofei <liushaofei5(a)huawei.com> --- sbin/makepkg | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sbin/makepkg b/sbin/makepkg index fde82212..6f8b95af 100755 --- a/sbin/makepkg +++ b/sbin/makepkg @@ -623,6 +623,11 @@ extract_git() { cd_safe "${dir##*/}" + # modify submodules address to local server address. + if [[ -f ".gitmodules" ]] && [[ -n ${LKP_SERVER} ]]; then + echo -e "[url \"git://${LKP_SERVER}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig + fi + local ref=origin/HEAD if [[ -n $fragment ]]; then case ${fragment%%=*} in -- 2.23.0

2 2

[PATCH v6 compass-ci 1/2] assign-account: add option to generate ssh key
by Luan Shengde 16 Nov '20

16 Nov '20

[why] enable one click login testbox from jumper server [how] add add -g|--gen-sshkey option to enable generate private/public key on jumper server register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 71 ++++++++------------ 1 file changed, 29 insertions(+), 42 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 686a327..5ddc451 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -13,6 +13,7 @@ require 'set' require 'optparse' require_relative '../defconfig' require_relative '../../lib/es_client' +require_relative 'build-send-account-email' names = Set.new %w[ JUMPER_HOST @@ -31,41 +32,46 @@ SEND_MAIL_PORT = defaults['SEND_MAIL_PORT'] || 49000 my_info = { 'my_email' => nil, 'my_name' => nil, + 'my_commit_url' => nil, 'my_uuid' => %x(uuidgen).chomp, - 'my_ssh_pubkey' => nil + 'my_ssh_pubkey' => [], + 'gen_sshkey' => false } -def init_info(email_file, my_info) - mail_content = Mail.read(email_file) +def init_info(mail_content, my_info) my_info['my_email'] = mail_content.from[0] my_info['my_name'] = mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, '') - my_info['my_ssh_pubkey'] = if mail_content.part[1].filename == 'id_rsa.pub' - mail_content.part[1].body.decoded - end + return if mail_content.attachments.empty? + + my_info['my_ssh_pubkey'] << mail_content.attachments[0].body.decoded end options = OptionParser.new do |opts| opts.banner = 'Usage: answerback-mail.rb [-e|--email email] ' - opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file]\n" + opts.banner += "[-s|--ssh-pubkey pub_key_file] [-f|--raw-email email_file] [-g|--gen-sshkey]\n" opts.banner += " -e or -f is required\n" - opts.banner += ' -s is optional when use -e' + opts.banner += " -s is optional when use -e\n" + opts.banner += ' -g is optional, used to generate sshkey for user' opts.separator '' opts.separator 'options:' opts.on('-e email_address', '--email email_address', 'appoint email address') do |email_address| my_info['my_email'] = email_address - # when apply account with email address, will get no user name - my_info['my_name'] = '' end opts.on('-s pub_key_file', '--ssh-pubkey pub_key_file', \ 'ssh pub_key file, enable password-less login') do |pub_key_file| - my_info['my_ssh_pubkey'] = File.read(pub_key_file) + my_info['my_ssh_pubkey'] << File.read(pub_key_file).chomp end opts.on('-f email_file', '--raw-email email_file', 'email file') do |email_file| - init_info(email_file, my_info) + mail_content = Mail.read(email_file) + init_info(mail_content, my_info) + end + + opts.on('-g', '--gen-sshkey', 'generate jumper ras public/private key and return pubkey') do + my_info['gen_sshkey'] = true end opts.on_tail('-h', '--help', 'show this message') do @@ -76,32 +82,6 @@ end options.parse!(ARGV) -def build_message(email, account_info) - message = <<~EMAIL_MESSAGE - To: #{email} - Subject: [compass-ci] jumper account is ready - - Dear user: - - Thank you for joining us. - You can use the following command to login the jumper server: - - Login command: - ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} - - Account password: - #{account_info['my_password']} - - Suggest: - If you use the password to login, change it in time. - - regards - compass-ci - EMAIL_MESSAGE - - return message -end - def apply_account(my_info) account_info_str = %x(curl -XGET '#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account' -d '#{my_info.to_json}') JSON.parse account_info_str @@ -114,12 +94,19 @@ def send_account(my_info) raise message if my_info['my_email'].nil? account_info = apply_account(my_info) - # for manually assign account, there will be no my_commit_url - # but the key my_commit_url is required for es - my_info['my_commit_url'] = '' my_info['my_login_name'] = account_info['my_login_name'] - my_info.delete 'my_ssh_pubkey' + + unless account_info['my_jumper_pubkey'].nil? + my_info['my_ssh_pubkey'] << account_info['my_jumper_pubkey'].chomp + end + + my_info.delete 'gen_sshkey' store_account_info(my_info) + + send_mail(my_info, account_info) +end + +def send_mail(my_info, account_info) message = build_message(my_info['my_email'], account_info) %x(curl -XPOST '#{SEND_MAIL_HOST}:#{SEND_MAIL_PORT}/send_mail_text' -d "#{message}") -- 2.23.0

2 2

[PATCH v6 compass-ci 2/2] assign-account: build email message for successfuly assign account
by Luan Shengde 16 Nov '20

16 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../build-send-account-email.rb | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100755 container/assign-account/build-send-account-email.rb diff --git a/container/assign-account/build-send-account-email.rb b/container/assign-account/build-send-account-email.rb new file mode 100755 index 0000000..3c41404 --- /dev/null +++ b/container/assign-account/build-send-account-email.rb @@ -0,0 +1,32 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_message(email, account_info) + message = <<~EMAIL_MESSAGE + To: #{email} + Subject: [compass-ci] jumper account is ready + + Dear user: + + Thank you for joining us. + You can use the following command to login the jumper server: + + Login command: + ssh -p #{account_info['jumper_port']} #{account_info['my_login_name']}@#{account_info['jumper_host']} + + Account password: + #{account_info['my_password']} + + You can use your pub_key to login if you have offered one. + + Suggest: + If you use the password to login, change it in time. + + regards + compass-ci + EMAIL_MESSAGE + + return message +end -- 2.23.0

1 0

[PATCH lkp-tests 3/4] test/cci-makepkg: Refactor
by Sun Yukui 16 Nov '20

16 Nov '20

Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a105f189..da7050f3 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -92,10 +92,6 @@ cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy -echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf -PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf - update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return @@ -109,4 +105,11 @@ update_softlink() rm $sync_dest/$cgz_name } -update_softlink && echo "Makepkg finished successfully" +build_source_pkg() +{ + echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ + $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf +} + +build_source_pkg -- 2.23.0

2 1

[PATCH v2 compass-ci] remove actual ip in compass-ci code
by Xiao Shenwei 16 Nov '20

16 Nov '20

obtain the actual ip address from configuration file, Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- doc/develop-environment.md | 2 +- src/features/jobs/right_iperf.yaml | 2 +- src/scheduler/README.md | 2 +- user-client/helper/iperf.yaml | 2 +- user-client/jobs/iperf-pxe.yaml | 2 +- user-client/jobs/iperf-vm.yaml | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..ca86e88 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -39,7 +39,7 @@ the returned data for setup_jumper_account_info like: { "account" => "guest", "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "jumper_ip" => "0.0.0.0", "jumper_port" => "10000" } diff --git a/doc/develop-environment.md b/doc/develop-environment.md index 8760878..ac408a2 100644 --- a/doc/develop-environment.md +++ b/doc/develop-environment.md @@ -4,7 +4,7 @@ cat >> ~/.ssh/config <<-EOF Host crystal - Hostname 124.90.34.227 + Hostname xxx.xxx.xxx.xxx Port 22113 EOF diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index f81af38..e239ccd 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/src/scheduler/README.md b/src/scheduler/README.md index fdc7027..8e3a197 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -6,7 +6,7 @@ - And set the environment variables needed for running (modify "/etc/compass-ci/defaults/crystal.yaml", and run $CCI_SRC/container/defconfig.rb | defconfig.sh) LKP_SRC: ~/code/lkp-tests # path to your lkp-tests repo CCI_SRC: ~/code/compass-ci # path to your compass-ci repo - SCHED_HOST: 172.168.131.131 # hostname or ip of the scheduler + SCHED_HOST: 172.17.0.1 # hostname or ip of the scheduler SCHED_PORT: 3000 # port of the scheduler JOB_INDEX_TYPE: "jobs/_doc" # es job document index and type - '$'{variable} is used at shell command, means the value of variable diff --git a/user-client/helper/iperf.yaml b/user-client/helper/iperf.yaml index 1509470..30e929f 100644 --- a/user-client/helper/iperf.yaml +++ b/user-client/helper/iperf.yaml @@ -30,7 +30,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-pxe.yaml b/user-client/jobs/iperf-pxe.yaml index 209c5d6..cb5a513 100644 --- a/user-client/jobs/iperf-pxe.yaml +++ b/user-client/jobs/iperf-pxe.yaml @@ -53,7 +53,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-vm.yaml b/user-client/jobs/iperf-vm.yaml index 434b964..c976bed 100644 --- a/user-client/jobs/iperf-vm.yaml +++ b/user-client/jobs/iperf-vm.yaml @@ -34,7 +34,7 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 +LKP_SERVER: LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x -- 2.23.0

2 2

[PATCH compass-ci 2/2] remove actual ip in compass-ci code
by Xiao Shenwei 16 Nov '20

16 Nov '20

if need ip: obtain the actual ip address from configuration file Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/assign-account/get_account_info.rb | 2 +- doc/develop-environment.md | 2 +- src/features/jobs/right_iperf.yaml | 2 -- src/scheduler/README.md | 2 +- user-client/helper/iperf.yaml | 1 - 5 files changed, 3 insertions(+), 6 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..ca86e88 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -39,7 +39,7 @@ the returned data for setup_jumper_account_info like: { "account" => "guest", "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "jumper_ip" => "0.0.0.0", "jumper_port" => "10000" } diff --git a/doc/develop-environment.md b/doc/develop-environment.md index 8760878..ac408a2 100644 --- a/doc/develop-environment.md +++ b/doc/develop-environment.md @@ -4,7 +4,7 @@ cat >> ~/.ssh/config <<-EOF Host crystal - Hostname 124.90.34.227 + Hostname xxx.xxx.xxx.xxx Port 22113 EOF diff --git a/src/features/jobs/right_iperf.yaml b/src/features/jobs/right_iperf.yaml index f81af38..db83832 100644 --- a/src/features/jobs/right_iperf.yaml +++ b/src/features/jobs/right_iperf.yaml @@ -30,8 +30,6 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x - diff --git a/src/scheduler/README.md b/src/scheduler/README.md index fdc7027..c92c1e8 100644 --- a/src/scheduler/README.md +++ b/src/scheduler/README.md @@ -6,7 +6,7 @@ - And set the environment variables needed for running (modify "/etc/compass-ci/defaults/crystal.yaml", and run $CCI_SRC/container/defconfig.rb | defconfig.sh) LKP_SRC: ~/code/lkp-tests # path to your lkp-tests repo CCI_SRC: ~/code/compass-ci # path to your compass-ci repo - SCHED_HOST: 172.168.131.131 # hostname or ip of the scheduler + SCHED_HOST: 172.17.0.1 # hostname or ip of the scheduler SCHED_PORT: 3000 # port of the scheduler JOB_INDEX_TYPE: "jobs/_doc" # es job document index and type - '$'{variable} is used at shell command, means the value of variable diff --git a/user-client/helper/iperf.yaml b/user-client/helper/iperf.yaml index 1509470..50c9ac6 100644 --- a/user-client/helper/iperf.yaml +++ b/user-client/helper/iperf.yaml @@ -30,7 +30,6 @@ proc-vmstat: proc-stat: meminfo: -LKP_SERVER: 172.168.131.113 LKP_CGI_PORT: 3000 result_root: /result/iperf LKP_DEBUG_PREFIX: bash -x -- 2.23.0

1 0

[PATCH compass-ci 1/2] delete unnecessary yaml files
by Xiao Shenwei 16 Nov '20

16 Nov '20

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- user-client/jobs/iperf-pxe.yaml | 59 --------------------------------- user-client/jobs/iperf-vm.yaml | 40 ---------------------- user-client/jobs/myjobs.yaml | 2 -- 3 files changed, 101 deletions(-) delete mode 100644 user-client/jobs/iperf-pxe.yaml delete mode 100644 user-client/jobs/iperf-vm.yaml delete mode 100644 user-client/jobs/myjobs.yaml diff --git a/user-client/jobs/iperf-pxe.yaml b/user-client/jobs/iperf-pxe.yaml deleted file mode 100644 index 209c5d6..0000000 --- a/user-client/jobs/iperf-pxe.yaml +++ /dev/null @@ -1,59 +0,0 @@ ---- - -#! jobs/iperf.yaml - -#this job yaml file is used to run qemu test by using pxe, in compass-ci/sparrow/6-test/qemu, -#usage like this: 0_addjob iperf-pxe.yaml - -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-pxe-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-pxe-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -iostat: -heartbeat: -vmstat: -numa-numastat: -numa-vmstat: -numa-meminfo: -proc-vmstat: -proc-stat: -meminfo: - -LKP_SERVER: 172.168.131.113 -LKP_CGI_PORT: 3000 -result_root: /result/iperf -LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/iperf-vm.yaml b/user-client/jobs/iperf-vm.yaml deleted file mode 100644 index 434b964..0000000 --- a/user-client/jobs/iperf-vm.yaml +++ /dev/null @@ -1,40 +0,0 @@ ---- - -#! jobs/iperf.yaml - -#this job yaml file is used to run qemu test by using kvm, in compass-ci/sparrow/6-test/qemu, -#usage like this: 0_addjob iperf-vm.yaml - -suite: iperf -testcase: iperf -category: benchmark -runtime: 300 -cluster: cs-localhost -if role server: - iperf-server: -if role client: - iperf: - protocol: tcp -job_origin: jobs/iperf.yaml -testbox: vm-2p8g-1 -arch: x86_64 -node_roles: server client - -#! include/category/benchmark -kmsg: -boot-time: -uptime: -iostat: -heartbeat: -vmstat: -numa-numastat: -numa-vmstat: -numa-meminfo: -proc-vmstat: -proc-stat: -meminfo: - -LKP_SERVER: 172.168.131.113 -LKP_CGI_PORT: 3000 -result_root: /result/iperf -LKP_DEBUG_PREFIX: bash -x diff --git a/user-client/jobs/myjobs.yaml b/user-client/jobs/myjobs.yaml deleted file mode 100644 index 73bbb63..0000000 --- a/user-client/jobs/myjobs.yaml +++ /dev/null @@ -1,2 +0,0 @@ -#! jobs/myjobs.yaml -test: -- 2.23.0

1 0

[PATCH compass-ci] container/git-mirror: get the lkp and comitter ID from host
by Cao Xueliang 16 Nov '20

16 Nov '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/git-mirror/start | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/container/git-mirror/start b/container/git-mirror/start index ee79578..6ec0001 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -9,11 +9,14 @@ DOCKER_CCI = '/c/compass-ci' DOCKER_REPO = '/c/upstream-repos' docker_rm "git-mirror" +lkp = %x(id -u lkp) +committer = %x(id -u committer) + cmd = %W[ docker run --restart=always --name git-mirror - -u 1090:1999 + -u #{lkp}:#{committer} -d -e CCI_SRC=#{DOCKER_CCI} -e REPO_SRC=#{DOCKER_REPO} -- 2.23.0

1 0

[PATCH lkp-tests] tests: add hackbench-git test script
by Zhang Yale 16 Nov '20

16 Nov '20

[Why] hackbench-git is an old version tool required by the customer. The hackbench-git usage: hackbench [-pipe] <num groups> [process|thread] [loops] So need to add new test script for hackbench-git. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- tests/hackbench-git | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) create mode 100755 tests/hackbench-git diff --git a/tests/hackbench-git b/tests/hackbench-git new file mode 100755 index 00000000..f2be0c9d --- /dev/null +++ b/tests/hackbench-git @@ -0,0 +1,30 @@ +#!/bin/sh +# - nr_threads +# - mode +# - ipc +# - iterations +# - loops + +## Hackbench-git is both a benchmark and a stress test for the Linux kernel scheduler. +## Its main job is to create a specified number of schedulable entities pairs. +## (either threads or traditional processes)which communicate via either sockets +## or pipes and time how long it takes for each pair to send data back and forth. + +[ "$ipc" = 'pipe' ] && ipc_option='-pipe' +[ -n "$iterations" ] || iterations=10 +[ -n "$loops" ] || loops=100 +[ -n "$nr_threads" ] || nr_threads=1 + +run_hackbench() +{ + # args: groups number, mode [process/thread], loops, ipc [pipe/socket] + hackbench_path="$BENCHMARK_ROOT"/hackbench-git/hackbench/hackbench + [ ! -x "${hackbench_path}" ] && exit 1 + + echo "--------$ipc $mode num=$nr_threads--------" + for i in $(seq 1 $iterations); do + log_cmd $hackbench_path $ipc_option $nr_threads $mode $loops + done +} + +run_hackbench -- 2.23.0

2 2

[PATCH compass-ci 2/2] lib/matrix2.rb: judge the value.size condition first
by Lu Kaiyi 16 Nov '20

16 Nov '20

[why] avoid to execute some unnecessary operations and improve performance. [how] keep the value.size condition judgement to the front of block. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/matrix2.rb | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 62d174c..86cfc93 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -116,15 +116,15 @@ def combine_group_query_data(query_data, dims) job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + if value.size < 2 + groups.delete(group_key) + next + end suite_list = [] value.each do |dimension_key, jobs| groups[group_key][dimension_key], suites = create_matrix(jobs) suite_list.concat(suites) end - if value.size < 2 - groups.delete(group_key) - next - end suites_list << suite_list end -- 2.23.0

2 1

[PATCH v3 lkp-tests] sysbench-mysql/PKGBUILD: fix "No such file or directory"
by Zhang Yu 16 Nov '20

16 Nov '20

[why] only openeuler have dir "/usr/local/mysql/lib/" Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- pkg/sysbench-mysql/PKGBUILD | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index d9f342a5..c214d8a2 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -18,6 +18,8 @@ build() { package() { cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + [[ $os == openeuler ]] && { + mkdir -p $pkgdir/usr/lib64 + cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + } } -- 2.23.0

1 0

[PATCH lkp-tests v3] depends/sysbench-mysql: add depends
by Zhang Yu 16 Nov '20

16 Nov '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- distro/depends/sysbench-mysql | 2 ++ 1 file changed, 2 insertions(+) diff --git a/distro/depends/sysbench-mysql b/distro/depends/sysbench-mysql index cfbc4518..b0d33ffe 100644 --- a/distro/depends/sysbench-mysql +++ b/distro/depends/sysbench-mysql @@ -1,2 +1,4 @@ libnuma1 xfsprogs +ethtool +libmariadb3 -- 2.23.0

1 0

[PATCH v2 lkp-tests] sysbench-mysql/PKGBUILD: fix "No such file or directory"
by Zhang Yu 16 Nov '20

16 Nov '20

[why] when run this PKGBUILD in centos error like: /lkp/lkp/src/pkg/sysbench/PKGBUILD: line 23: usr/local/mysql/lib/: No No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- pkg/sysbench-mysql/PKGBUILD | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index d9f342a5..119aea46 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -18,6 +18,8 @@ build() { package() { cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + [[ $os == openeuler ]] && { + mkdir -p $pkgdir/usr/lib64 + cp -r /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + } } -- 2.23.0

2 2

[PATCH v2 lkp-tests] add: sysbench-threads-git result parsing file
by Hu Xuejiao 16 Nov '20

16 Nov '20

[input] General statistics: total time: 0.3987s total number of events: 10000 total time taken by event execution: 0.3971 response time: min: 0.04ms [output] total time: 0.3987s Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- stats/sysbench-threads-git | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100755 stats/sysbench-threads-git diff --git a/stats/sysbench-threads-git b/stats/sysbench-threads-git new file mode 100755 index 00000000..4bf2ded9 --- /dev/null +++ b/stats/sysbench-threads-git @@ -0,0 +1,26 @@ +#!/usr/bin/env ruby + +# General statistics: +# total time: 0.3987s +# total number of events: 10000 +# total time taken by event execution: 0.3971s +# response time: +# min: 0.04ms + +results_total = {} + +def add_result(results, key, val) + results[key] ||= [] + results[key] << val +end + +$stdin.each_line do |line| + case line + when /total time:\s+([0-9.]+)/ + add_result(results_total, 'total time', $1.to_f) + end +end + +results_total.each do |key, vals| + puts "#{key}: #{vals.inject(0.0, :+)s}" +end -- 2.23.0

2 1

[PATCH lkp-tests 4/4] test/cci-makepkg: fix a softlink will be established if the package fails
by Sun Yukui 16 Nov '20

16 Nov '20

After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the softlink should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index da7050f3..c6fce675 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -110,6 +110,8 @@ build_source_pkg() echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + + [ "$?" == 0 ] || update_softlink } build_source_pkg -- 2.23.0

1 0

[PATCH lkp-tests 2/4] test/build-pkg: fix a softlink will be established if the package fails
by Sun Yukui 16 Nov '20

16 Nov '20

After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the softlink should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 2ec45b9c..093bbe4d 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -99,10 +99,11 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 + + [ "$?" == 0 ] || create_softlink } check_vars mount_dest request_pkg build_source_pkg -create_softlink -- 2.23.0

1 0

[PATCH v5 compass-ci 3/4] assign-account: generate ssh keys
by Luan Shengde 16 Nov '20

16 Nov '20

generate private/public key on jumper server and return public key [why] register the public key to es, final register it to testbox enable user one click to login in the testbox [how] generate private/public key for user on jumper server return and register the public key to es Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/get_account_info.rb | 43 +++++++++++--------- 1 file changed, 23 insertions(+), 20 deletions(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 137f5c6..0e91057 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -30,16 +30,17 @@ API: call graph: setup_jumper_account_info read_account_info - build_account_info + build_account_name read_jumper_info config_default_yaml config_authorized_key + generate_ssh_key the returned data for setup_jumper_account_info like: { - "account" => "guest", - "passwd" => "Use pub_key to login", - "jumper_ip" => "10.10.10.10", + "my_login_name" => "login_name", + "password" => "password", + "jumper_host" => "0.0.0.0", "jumper_port" => "10000" } @@ -94,30 +95,33 @@ class AccountStorage end def setup_jumper_account_info - account_info = read_account_info - jumper_info = read_jumper_info - pub_key = @data['my_ssh_pubkey'] unless @data['my_ssh_pubkey'].nil? + login_name, password = read_account_info + jumper_host, jumper_port = read_jumper_info + pub_key = @data['my_ssh_pubkey'][0] - login_name = account_info[0] - password = if pub_key.nil? - account_info[1] - else - 'Use pub_key to login' - end + ssh_dir = File.join('/home/', login_name, '.ssh') + config_authorized_key(login_name, pub_key, ssh_dir) unless pub_key.nil? + config_default_yaml(login_name) + my_jumper_pubkey = generate_ssh_key(login_name, ssh_dir) if @data['gen_sshkey'].eql? true jumper_account_info = { 'my_login_name' => login_name, 'my_password' => password, - 'jumper_host' => jumper_info[0].chomp, - 'jumper_port' => jumper_info[1].chomp + 'jumper_host' => jumper_host, + 'jumper_port' => jumper_port, + 'my_jumper_pubkey' => my_jumper_pubkey } - config_authorized_key(login_name, pub_key) unless pub_key.nil? - config_default_yaml(login_name) - return jumper_account_info end + def generate_ssh_key(login_name, ssh_dir) + Dir.mkdir ssh_dir, 0o700 unless File.exist? ssh_dir + %x(ssh-keygen -f "#{ssh_dir}/id_rsa" -N '' -C "#{login_name}@account-vm") + %x(chown -R #{login_name}:#{login_name} #{ssh_dir}) + File.read("/home/#{login_name}/.ssh/id_rsa.pub") + end + def config_default_yaml(login_name) default_yaml_dir = File.join('/home', login_name, '.config/compass-ci/defaults') FileUtils.mkdir_p default_yaml_dir @@ -132,8 +136,7 @@ class AccountStorage %x(chown -R #{login_name}:#{login_name} "/home/#{login_name}/.config") end - def config_authorized_key(login_name, pub_key) - ssh_dir = File.join('/home/', login_name, '.ssh') + def config_authorized_key(login_name, pub_key, ssh_dir) Dir.mkdir ssh_dir, 0o700 Dir.chdir ssh_dir f = File.new('authorized_keys', 'w') -- 2.23.0

1 0

[PATCH v5 compass-ci 1/4] assign-account: Dockerfile add install ssh
by Luan Shengde 16 Nov '20

16 Nov '20

add install ssh to enable execute ssh-keygen to generate ssh key in container Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/Dockerfile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/assign-account/Dockerfile b/container/assign-account/Dockerfile index f84b5e6..12faafa 100644 --- a/container/assign-account/Dockerfile +++ b/container/assign-account/Dockerfile @@ -2,13 +2,13 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM debian -MAINTAINER luanshd0525(a)163.com +MAINTAINER Luan Shengde <shdluan(a)163.com> ENV DEBIAN_FRONTEND noninteractive COPY conf/sources.list* /etc/apt/ RUN apt-get update && \ - apt-get install -y ruby-sinatra ruby-json curl && \ + apt-get install -y ssh ruby-sinatra ruby-json curl && \ ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && \ echo 'Asia/Shanghai' > /etc/timezone COPY assign-account.rb /usr/local/bin/ -- 2.23.0

1 0

[PATCH compass-ci 1/2] /lib: keep suite result in top for compare output
by Lu Kaiyi 16 Nov '20

16 Nov '20

[why] Currently, the compare output result default order by alphabet. If there are too many lines in result, it will be difficult for user to find the lines that starts with $suite in metric column. So, need reorder compare output result with keeping $suite item on the top of the table. [how] Divide the lines into two parts before print according to whether the line start with $suite or not, then combine output result into a whole to highlight the important info. [example] command: compare "id=z9.148096" "id=z9.148143" in z9 suites_list:["sysbench-mysql"] [before] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 ... 1266042.00 -18.3% 1034206.00 softirqs.RCU 547849.00 -27.4% 397593.00 softirqs.SCHED 159990.00 -80.5% 31181.00 softirqs.TASKLET 1617967.00 -23.0% 1245331.00 softirqs.TIMER 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 2149.23 -100.0% 0.00 time.elapsed_time 2149.23 -100.0% 0.00 time.elapsed_time.max 214458.00 -100.0% 0.00 time.involuntary_context_switches 2.00 -100.0% 0.00 time.major_page_faults 192832.00 -100.0% 0.00 time.maximum_resident_set_size ... [after] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 2894304.00 +12.8% 3265730.00 interrupts.4:GICv3.26.Level.arch_timer 11546998.00 -100.0% 0.00 interrupts.746:ITS-MSI.99092481.Edge.hns3-0000:bd:00.1-TxRx-0 7476064.00 -100.0% 0.00 interrupts.747:ITS-MSI.99092482.Edge.hns3-0000:bd:00.1-TxRx-2 ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare.rb | 16 ++++++++++------ lib/compare_matrixes.rb | 41 ++++++++++++++++++++++++++++------------- lib/matrix2.rb | 20 +++++++++++++++----- 3 files changed, 53 insertions(+), 24 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 645f682..f34e239 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -20,8 +20,8 @@ require 'yaml' def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) - matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list, options: options) + matrices_list, suite_list = create_matrices_list(condition_list) + compare_matrixes(matrices_list, suite_list, options: options) end def parse_argv(argv, common_conditions) @@ -37,12 +37,16 @@ end def create_matrices_list(conditions) matrices_list = [] + suite_list = [] es = ESQuery.new(ES_HOST, ES_PORT) conditions.each do |condition| query_results = es.multi_field_query(condition) - matrices_list << combine_query_data(query_results) + matrix, suites = combine_query_data(query_results) + matrices_list << matrix + suite_list.concat(suites) end - matrices_list + + return matrices_list, suite_list end # ------------------------------------------------------------------------------------------- @@ -58,8 +62,8 @@ end def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') - groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices, options) + groups_matrices, suites_list = create_groups_matrices_list(conditions, dims) + compare_group_matrices(groups_matrices, suites_list, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 119d42d..5039b01 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -128,7 +128,7 @@ def set_compare_values(index, values, field, success, options) end def get_values_by_field(matrixes_list, field, matrixes_size, success, options) - # get values by field, values struce example: values[0][:average] + # get values by field, values struct example: values[0][:average] # values = {} matrixes_list.length.times do |index| @@ -193,7 +193,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suite_list, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -208,6 +208,7 @@ def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, opti result_str = group_key ? "\n\n\n\n\n" + group_key : '' result_str += get_all_result_str( matrixes_values, + suite_list, matrixes_titles, matrixes_list.size, options[:theme] @@ -382,7 +383,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices, options) +def compare_group_matrices(group_matrices, suites_list, options) result_str = '' group_matrices.each do |k, v| matrices_list = [] @@ -392,9 +393,9 @@ def compare_group_matrices(group_matrices, options) matrices_list << matrix end if options[:no_print] - result_str += compare_matrixes(matrices_list, matrices_titles, k, options: options) + result_str += compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) else - print compare_matrixes(matrices_list, matrices_titles, k, options: options) + print compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) end end result_str @@ -807,26 +808,40 @@ def get_theme(matrixes_values, matrixes_titles, theme) return THEMES[:none] end -def get_all_result_str(matrixes_values, matrixes_titles, matrixes_number, theme) +def get_all_result_str(matrixes_values, suite_list, matrixes_titles, matrixes_number, theme) matrixes_titles ||= matrixes_number.times.to_a.map(&:to_s) theme = get_theme(matrixes_values, matrixes_titles, theme) return '' unless theme - failure_str = get_result_str(matrixes_values[false].sort, matrixes_titles, false, theme) - success_str = get_result_str(matrixes_values[true].sort, matrixes_titles, true, theme) + failure_str = get_result_str(matrixes_values[false].sort, suite_list, matrixes_titles, false, theme) + success_str = get_result_str(matrixes_values[true].sort, suite_list, matrixes_titles, true, theme) failure_str + success_str end -def get_result_str(values, matrixes_titles, success, theme) +def get_result_str(values, suite_list, matrixes_titles, success, theme) return '' if values.empty? + suite_set = Set.new(suite_list) result_str = "\n\n\n" common_title, compare_title = get_title_name(success) result_str += get_header(matrixes_titles, success, common_title, compare_title) + ranked_str = get_ranked_str(values, suite_set, success, theme) + result_str += ranked_str + result_str +end + +def get_ranked_str(values, suite_set, success, theme) + suite_str = '' + common_str = '' values.each do |field, matrixes| - result_str += get_values_str(matrixes, success, theme) - result_str += get_field_str(field) - result_str += "\n" + row = get_values_str(matrixes, success, theme) + row += get_field_str(field) + "\n" + field_start_with_suite = suite_set.any? { |suite| field.start_with?(suite) } + if field_start_with_suite + suite_str += row + else + common_str += row + end end - result_str + suite_str + common_str end diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 26f07fa..62d174c 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -76,7 +76,9 @@ end # } def create_matrix(job_list) matrix = {} + suites = [] job_list.each do |job| + suites << job['suite'] if job['suite'] stats = job['stats'] next unless stats @@ -89,7 +91,7 @@ def create_matrix(job_list) matrix.each_value do |value| samples_fill_missing_zeros(value, col_size) end - matrix + return matrix, suites end # input: query results from es_query @@ -110,15 +112,23 @@ end # ... # } def combine_group_query_data(query_data, dims) + suites_list = [] job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + suite_list = [] value.each do |dimension_key, jobs| - groups[group_key][dimension_key] = create_matrix(jobs) + groups[group_key][dimension_key], suites = create_matrix(jobs) + suite_list.concat(suites) end - groups.delete(group_key) if value.size < 2 + if value.size < 2 + groups.delete(group_key) + next + end + suites_list << suite_list end - groups + + return groups, suites_list end # input: @@ -139,7 +149,7 @@ def combine_group_jobs_list(query_data, groups_params, dimensions, metrics) groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) groups.each do |group_key, dims| dims.each do |dim_key, jobs| - groups[group_key][dim_key] = create_matrix(jobs) + groups[group_key][dim_key], = create_matrix(jobs) end end -- 2.23.0

1 0

[PATCH v2 compass-ci] container/minio: add minio container image
by Cui Yili 16 Nov '20

16 Nov '20

Add minio container image to store files. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/minio/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/minio/build diff --git a/container/minio/build b/container/minio/build new file mode 100644 index 0000000..d1ccd06 --- /dev/null +++ b/container/minio/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker pull jessestuart/minio -- 2.23.0

1 0

[PATCH v7 lkp-tests 3/3] execute on_fail when exit-code isn't 0
by Hu Xuejiao 16 Nov '20

16 Nov '20

[why] When a job-task fails(exit_code != 0), the on_fail function needs to be called to keep the program in the waiting state to user adjustment Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- bin/run-lkp | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/run-lkp b/bin/run-lkp index 26669331..c4d43db4 100755 --- a/bin/run-lkp +++ b/bin/run-lkp @@ -97,6 +97,7 @@ elif [ "$run_job_exit_code" != 0 ]; then else set_job_state 'failed' fi + $LKP_DEBUG_PREFIX $job_script on_fail elif [ -f '/var/log/wtmp' ] && has_cmd last && last -n1 | grep -q -E '(pts/|tty)[0-9].*(still logged in)'; then set_job_state 'disturbed' disturbed=1 -- 2.23.0

1 0

[PATCH v7 lkp-tests 2/3] submit on_fail to the scheduler
by Hu Xuejiao 16 Nov '20

16 Nov '20

[why] When the task is failed(state), it need in the waiting state [how] When job2sh.rb defined sh_on_state, it will be invoked by the scheduler. Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/submit | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index bc90e2e0..3052cf39 100755 --- a/sbin/submit +++ b/sbin/submit @@ -115,12 +115,14 @@ ARGV.each do |jobfile| sh_run_job = job.sh_run_job sh_extract_stats = job.sh_extract_stats sh_define_files = job.sh_define_files + sh_on_state = job.sh_on_state sh_hash = { 'job2sh' => { 'run_job' => sh_run_job, 'extract_stats' => sh_extract_stats, - 'define_files' => sh_define_files + 'define_files' => sh_define_files, + 'on_fail' => sh_on_state } } -- 2.23.0

1 0

[PATCH v7 lkp-tests 1/3] job2sh.rb: define on_state
by Hu Xuejiao 16 Nov '20

16 Nov '20

[why] If job.yaml is defined: on_fail: sleep(tests): 300(runtime) [how] By on_fail(sh_on_state), it will appear in job.sh: on_fail() { . $LKP_SRC/lib/job.sh . $LKP_SRC/lib/env.sh run_test $LKP_SRC/tests/wrapper sleep 300 } Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- lib/job2sh.rb | 30 +++++++++++++++++++++++++++--- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/lib/job2sh.rb b/lib/job2sh.rb index 7e6a5d7c..9c8b578d 100755 --- a/lib/job2sh.rb +++ b/lib/job2sh.rb @@ -142,7 +142,7 @@ class Job2sh < Job end def parse_one(ancestors, key, val, pass) - if ['pp', 'define_files', 'monitors'].include?(key) + if ['pp', 'define_files', 'monitors', 'on_fail'].include?(key) return false end tabs = indent(ancestors) @@ -266,11 +266,13 @@ class Job2sh < Job sh_run_job(job) local_script_lines += @script_lines - sh_extract_stats(job) local_script_lines += @script_lines sh_define_files() + local_script_lines += @script_lines + + sh_on_state(job) out_line '"$@"' @@ -358,6 +360,28 @@ class Job2sh < Job out_line "\tchmod +x $LKP_SRC/#{file}" end end - out_line '}' + out_line "}\n\n" + end + + def sh_on_state(job = nil) + @script_lines = [] + + @programs = available_programs(:workload_elements) + + job ||= (@jobx || @job).clone + + if job.include?('on_fail') + + @cur_func = :run_job + + out_line 'on_fail()' + out_line '{' + out_line + out_line "\t. $LKP_SRC/lib/job.sh" + out_line "\t. $LKP_SRC/lib/env.sh" + out_line + parse_hash [], job['on_fail'] + out_line "}\n\n" + end end end -- 2.23.0

1 0

[PATCH v2 lkp-tests] depends/sysbench-mysql: add depends
by Zhang Yu 16 Nov '20

16 Nov '20

[why] when run with initramfs, there occurred an error: /lkp/lkp/src/daemon/mysql: line 57: ethtool: command not found Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- distro/depends/sysbench-mysql | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/sysbench-mysql b/distro/depends/sysbench-mysql index cfbc4518..a248fd07 100644 --- a/distro/depends/sysbench-mysql +++ b/distro/depends/sysbench-mysql @@ -1,2 +1,3 @@ libnuma1 xfsprogs +ethtool -- 2.23.0

1 0

[PATCH lkp-tests] depends/sysbenhc-mysql: add depends
by Zhang Yu 16 Nov '20

16 Nov '20

[why] when run with initramfs, there occurred an error: /lkp/lkp/src/daemon/mysql: line 57: ethtool: command not found Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- distro/depends/sysbench-mysql | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/sysbench-mysql b/distro/depends/sysbench-mysql index cfbc4518..a248fd07 100644 --- a/distro/depends/sysbench-mysql +++ b/distro/depends/sysbench-mysql @@ -1,2 +1,3 @@ libnuma1 xfsprogs +ethtool -- 2.23.0

2 2

[PATCH v2 lkp-tests] tests/cci-makepkg: update gcc for centos7 when use initramfs os
by Lin Jiaxin 16 Nov '20

16 Nov '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/cci-makepkg | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 67d055a4..bdb2128d 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -88,13 +88,22 @@ update_shared_pkg() echo "update shared pkg link ${benchmark}.cgz -> ${bm_name}/${cgz_name}" } +centos_update_gcc() +{ + if $(grep -sqF "CentOS Linux release 7" /etc/centos-release ); then + [[ "$os_mount" = "initramfs" ]] && { + yum install -y centos-release-scl-rh + yum install -y "devtoolset-8-gcc*" + } + source /opt/rh/devtoolset-8/enable + fi +} + [ -n "$PKGBUILD_TAG" ] && get_pkgfile distro_install_depends lkp-dev -[ $os = "centos" ] && -grep -sqF "CentOS Linux release 7" /etc/centos-release && -source /opt/rh/devtoolset-8/enable +[ $os = "centos" ] && centos_update_gcc distro_install_depends ${benchmark}-dev -- 2.23.0

1 0

[PATCH lkp-tests] add: sysbench-threads-git result parsing file
by Hu Xuejiao 16 Nov '20

16 Nov '20

[input] General statistics: total time: 0.3987s total number of events: 10000 total time taken by event execution: 0.3971s response time: min: 0.04ms [output] total time: 0.3987s Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- stats/sysbench-threads-git | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 stats/sysbench-threads-git diff --git a/stats/sysbench-threads-git b/stats/sysbench-threads-git new file mode 100755 index 00000000..a8cf0b53 --- /dev/null +++ b/stats/sysbench-threads-git @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby + +LKP_SRC ||= ENV['LKP_SRC'] + +# General statistics: +# total time: 0.3987s +# total number of events: 10000 +# total time taken by event execution: 0.3971s +# response time: +# min: 0.04ms + +$results_total = {} + +def add_result(results, key, val) + results[key] ||= [] + results[key] << val +end + +$stdin.each_line do |line| + case line + when /total time:\s+([0-9.]+)/ + add_result($results_total, 'total time', $1.to_f) + end +end + +$results_total.each do |key, vals| + puts "#{key}: #{vals.inject(0.0, :+)s}" +end -- 2.23.0

4 4

[PATCH lkp-tests] pkg: fix "No such file or directory"
by Zhang Yu 16 Nov '20

16 Nov '20

[why] when run this PKGBUILD in centos error like: /lkp/lkp/src/pkg/sysbench/PKGBUILD: line 23: usr/local/mysql/lib/: No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- pkg/sysbench-mysql/PKGBUILD | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index d9f342a5..c214d8a2 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -18,6 +18,8 @@ build() { package() { cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + [[ $os == openeuler ]] && { + mkdir -p $pkgdir/usr/lib64 + cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ + } } -- 2.23.0

2 2

[PATCH v2 lkp-tests] stats: add libmicro result parsing file
by Bai Jing 16 Nov '20

16 Nov '20

[input] prc thr usecs/call samples errors cnt/samp getpid 1 1 0.15629 193 0 100000 [output] getpid: 0.15629 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/libmicro | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100755 stats/libmicro diff --git a/stats/libmicro b/stats/libmicro new file mode 100755 index 00000000..bbfb6db5 --- /dev/null +++ b/stats/libmicro @@ -0,0 +1,11 @@ +#!/usr/bin/env ruby + +# prc thr usecs/call samples errors cnt/samp +# close_tmp 1 1 0.61880 201 0 640 + +$stdin.each_line do |line| + case line + when /^(\w+)\s+\S+\s+\S+\s+(\S+)/ + puts "#{$1}: #{$2}" + end +end -- 2.23.0

3 2

[PATCH compass-ci] container/minio: add minio container
by Cui Yili 16 Nov '20

16 Nov '20

Add minio container to store files. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/minio/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/minio/build diff --git a/container/minio/build b/container/minio/build new file mode 100755 index 0000000..d1ccd06 --- /dev/null +++ b/container/minio/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker pull jessestuart/minio -- 2.23.0

2 1

[PATCH v4 compass-ci] /lib: keep suite result in top for compare output
by Lu Kaiyi 16 Nov '20

16 Nov '20

[why] Currently, the compare output result default order by alphabet. If there are too many lines in result, it will be difficult for user to find the lines that starts with $suite in metric column. So, need reorder compare output result with keeping $suite item on the top of the table. [how] Divide the lines into two parts before print according to whether the line start with $suite or not, then combine output result into a whole to highlight the important info. [example] command: compare "id=z9.148096" "id=z9.148143" in z9 suites_list:["sysbench-mysql"] [before] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 ... 1266042.00 -18.3% 1034206.00 softirqs.RCU 547849.00 -27.4% 397593.00 softirqs.SCHED 159990.00 -80.5% 31181.00 softirqs.TASKLET 1617967.00 -23.0% 1245331.00 softirqs.TIMER 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 2149.23 -100.0% 0.00 time.elapsed_time 2149.23 -100.0% 0.00 time.elapsed_time.max 214458.00 -100.0% 0.00 time.involuntary_context_switches 2.00 -100.0% 0.00 time.major_page_faults 192832.00 -100.0% 0.00 time.maximum_resident_set_size ... [after] 0 1 metric -------------------- ------------------------------ ------------------------------ %stddev change %stddev \ | \ 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time 2149.23 -100.0% 0.00 sysbench-mysql.time.elapsed_time.max 214458.00 -100.0% 0.00 sysbench-mysql.time.involuntary_context_switches 2.00 -100.0% 0.00 sysbench-mysql.time.major_page_faults 192832.00 -100.0% 0.00 sysbench-mysql.time.maximum_resident_set_size 90746.00 -100.0% 0.00 sysbench-mysql.time.minor_page_faults 65536.00 -100.0% 0.00 sysbench-mysql.time.page_size 201.00 -100.0% 0.00 sysbench-mysql.time.percent_of_cpu_this_job_got 2796.60 -100.0% 0.00 sysbench-mysql.time.system_time 1535.98 -100.0% 0.00 sysbench-mysql.time.user_time 282616989.00 -100.0% 0.00 sysbench-mysql.time.voluntary_context_switches 224.42 -74.1% 58.14 boot-time.boot 20400.95 -73.5% 5397.13 boot-time.idle 11.56 +83.3% 21.19 boot-time.kernel_boot 29424.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 4100080.00 -99.8% 10196.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 2894304.00 +12.8% 3265730.00 interrupts.4:GICv3.26.Level.arch_timer 11546998.00 -100.0% 0.00 interrupts.746:ITS-MSI.99092481.Edge.hns3-0000:bd:00.1-TxRx-0 7476064.00 -100.0% 0.00 interrupts.747:ITS-MSI.99092482.Edge.hns3-0000:bd:00.1-TxRx-2 ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare.rb | 16 ++++++++++------ lib/compare_matrixes.rb | 41 ++++++++++++++++++++++++++++------------- lib/matrix2.rb | 15 +++++++++++---- 3 files changed, 49 insertions(+), 23 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 645f682..813e08c 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -20,8 +20,8 @@ require 'yaml' def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) - matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list, options: options) + matrices_list, suites_list = create_matrices_list(condition_list) + compare_matrixes(matrices_list, suites_list, options: options) end def parse_argv(argv, common_conditions) @@ -37,12 +37,16 @@ end def create_matrices_list(conditions) matrices_list = [] + suites_list = [] es = ESQuery.new(ES_HOST, ES_PORT) conditions.each do |condition| query_results = es.multi_field_query(condition) - matrices_list << combine_query_data(query_results) + matrix, suites = combine_query_data(query_results) + matrices_list << matrix + suites_list.concat(suites) end - matrices_list + + return matrices_list, suites_list end # ------------------------------------------------------------------------------------------- @@ -58,8 +62,8 @@ end def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') - groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices, options) + groups_matrices, suites_list = create_groups_matrices_list(conditions, dims) + compare_group_matrices(groups_matrices, suites_list, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 119d42d..bd1e646 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -128,7 +128,7 @@ def set_compare_values(index, values, field, success, options) end def get_values_by_field(matrixes_list, field, matrixes_size, success, options) - # get values by field, values struce example: values[0][:average] + # get values by field, values struct example: values[0][:average] # values = {} matrixes_list.length.times do |index| @@ -193,7 +193,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suites_list, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -208,6 +208,7 @@ def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, opti result_str = group_key ? "\n\n\n\n\n" + group_key : '' result_str += get_all_result_str( matrixes_values, + suites_list, matrixes_titles, matrixes_list.size, options[:theme] @@ -382,7 +383,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices, options) +def compare_group_matrices(group_matrices, suites_list, options) result_str = '' group_matrices.each do |k, v| matrices_list = [] @@ -392,9 +393,9 @@ def compare_group_matrices(group_matrices, options) matrices_list << matrix end if options[:no_print] - result_str += compare_matrixes(matrices_list, matrices_titles, k, options: options) + result_str += compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) else - print compare_matrixes(matrices_list, matrices_titles, k, options: options) + print compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) end end result_str @@ -807,26 +808,40 @@ def get_theme(matrixes_values, matrixes_titles, theme) return THEMES[:none] end -def get_all_result_str(matrixes_values, matrixes_titles, matrixes_number, theme) +def get_all_result_str(matrixes_values, suites_list, matrixes_titles, matrixes_number, theme) matrixes_titles ||= matrixes_number.times.to_a.map(&:to_s) theme = get_theme(matrixes_values, matrixes_titles, theme) return '' unless theme - failure_str = get_result_str(matrixes_values[false].sort, matrixes_titles, false, theme) - success_str = get_result_str(matrixes_values[true].sort, matrixes_titles, true, theme) + failure_str = get_result_str(matrixes_values[false].sort, suites_list, matrixes_titles, false, theme) + success_str = get_result_str(matrixes_values[true].sort, suites_list, matrixes_titles, true, theme) failure_str + success_str end -def get_result_str(values, matrixes_titles, success, theme) +def get_result_str(values, suites_list, matrixes_titles, success, theme) return '' if values.empty? + suites_list.uniq! result_str = "\n\n\n" common_title, compare_title = get_title_name(success) result_str += get_header(matrixes_titles, success, common_title, compare_title) + ranked_str = get_ranked_str(values, suites_list, success, theme) + result_str += ranked_str + result_str +end + +def get_ranked_str(values, suites_list, success, theme) + suite_str = '' + common_str = '' values.each do |field, matrixes| - result_str += get_values_str(matrixes, success, theme) - result_str += get_field_str(field) - result_str += "\n" + row = get_values_str(matrixes, success, theme) + row += get_field_str(field) + "\n" + field_start_with_suite = suites_list.any? { |suite| field.start_with?(suite) } + if field_start_with_suite + suite_str += row + else + common_str += row + end end - result_str + suite_str + common_str end diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 26f07fa..479d280 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -76,7 +76,9 @@ end # } def create_matrix(job_list) matrix = {} + suites = [] job_list.each do |job| + suites << job['suite'] if job['suite'] stats = job['stats'] next unless stats @@ -89,7 +91,7 @@ def create_matrix(job_list) matrix.each_value do |value| samples_fill_missing_zeros(value, col_size) end - matrix + return matrix, suites end # input: query results from es_query @@ -110,15 +112,20 @@ end # ... # } def combine_group_query_data(query_data, dims) + suites_list = [] job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + suite_list = [] value.each do |dimension_key, jobs| - groups[group_key][dimension_key] = create_matrix(jobs) + groups[group_key][dimension_key], suites = create_matrix(jobs) + suite_list.concat(suites) end groups.delete(group_key) if value.size < 2 + suites_list << suite_list unless value.size < 2 end - groups + + return groups, suites_list end # input: @@ -139,7 +146,7 @@ def combine_group_jobs_list(query_data, groups_params, dimensions, metrics) groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) groups.each do |group_key, dims| dims.each do |dim_key, jobs| - groups[group_key][dim_key] = create_matrix(jobs) + groups[group_key][dim_key], = create_matrix(jobs) end end -- 2.23.0

3 4

[PATCH v4 compass-ci 4/4] container/git-mirrir: add Docker env GIT_ASKPASS
by Li Yuanchao 16 Nov '20

16 Nov '20

[why] some repositories need username and password, but we don't know. And the problem is, it will block there as we have no username and password. Add GIT_ASKPASS=/kill-parent.sh, will kill the block one, and go to next repository. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/Dockerfile | 3 +++ container/git-mirror/kill-parent.sh | 3 +++ container/git-mirror/start | 1 + 3 files changed, 7 insertions(+) create mode 100755 container/git-mirror/kill-parent.sh diff --git a/container/git-mirror/Dockerfile b/container/git-mirror/Dockerfile index 390bbcc..1180a7b 100644 --- a/container/git-mirror/Dockerfile +++ b/container/git-mirror/Dockerfile @@ -2,6 +2,9 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. FROM alpine:3.8 +MAINTAINER Li Yuanchao <lyc163mail(a)163.com> + +ADD kill-parent.sh / RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories diff --git a/container/git-mirror/kill-parent.sh b/container/git-mirror/kill-parent.sh new file mode 100755 index 0000000..6418592 --- /dev/null +++ b/container/git-mirror/kill-parent.sh @@ -0,0 +1,3 @@ +#!/bin/sh + +kill $PPID diff --git a/container/git-mirror/start b/container/git-mirror/start index dd4fd63..ee79578 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -17,6 +17,7 @@ cmd = %W[ -d -e CCI_SRC=#{DOCKER_CCI} -e REPO_SRC=#{DOCKER_REPO} + -e GIT_ASKPASS=/kill-parent.sh -v #{ENV['CCI_SRC']}:#{DOCKER_CCI} -v /etc/localtime:/etc/localtime:ro -v #{ENV['REPO_SRC']}:#{DOCKER_REPO} -- 2.23.0

1 0

[PATCH v4 compass-ci 3/4] git_mirror: use "gitclone.com" to make "git clone" faster
by Li Yuanchao 16 Nov '20

16 Nov '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 8f0497e..9648063 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -31,12 +31,18 @@ class GitMirror @feedback_queue.push(@feedback_info) end - def git_clone(url, mirror_dir) - ret = false - url = Array(url)[0] + def get_url(url) if url.include?('gitee.com/') && File.exist?("/srv/git/#{url.delete_prefix('https://')}") url = "/srv/git/#{url.delete_prefix('https://')}" + elsif url.include?('://github.com') + url = "https://gitclone.com/#{url.split('://')[1]}" end + return url + end + + def git_clone(url, mirror_dir) + ret = false + url = get_url(Array(url)[0]) 10.times do stderr = %x(git clone --mirror #{url} #{mirror_dir} 2>&1) ret = !stderr.include?('fatal') -- 2.23.0

1 0

[PATCH v4 compass-ci 2/4] git_mirror: fix that worker thread die after system(git clone)
by Li Yuanchao 16 Nov '20

16 Nov '20

[why] As Signal.trap(:SIGCHLD, 'SIG_IGN') has been set, the init process will handle the return status of child process, so ret = system(git clone xxx) can't get return value, which cause worker thread die. Use %x() instead of system(), and use stderr to check if 'git clone' succeed. So return value is not needed. [errmsg] in `system': No child process - Another thread waited the process started by system(). (Errno::ECHILD) Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 55ec67d..8f0497e 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -38,7 +38,8 @@ class GitMirror url = "/srv/git/#{url.delete_prefix('https://')}" end 10.times do - ret = system("git clone --mirror #{url} #{mirror_dir}") + stderr = %x(git clone --mirror #{url} #{mirror_dir} 2>&1) + ret = !stderr.include?('fatal') break if ret end FileUtils.rm_r(mirror_dir) unless ret -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add optimization parameters to job.yaml
by Zhang Yu 16 Nov '20

16 Nov '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- jobs/sysbench-mysql.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/jobs/sysbench-mysql.yaml b/jobs/sysbench-mysql.yaml index 83795a18..12c83087 100644 --- a/jobs/sysbench-mysql.yaml +++ b/jobs/sysbench-mysql.yaml @@ -2,6 +2,8 @@ suite: sysbench-mysql category: benchmark cluster: cs-lkp-hsw-ep5 +kernel_version: 4.19.90-mysql +kernel_custom_params: "sched_steal_node_limit=8" if role server: mysql: -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: support more testbox formats
by Li Ping 16 Nov '20

16 Nov '20

If user wants to reproduce the result directly in the result directory. [problem] /srv/result/iperf/vm-2p8g/2020-11-09/z9.147520% submit job.yaml output: /home/liping/lkp-tests/lib/job.rb:344:in `get_hosts_file': hosts file not exist: vm-2p8g.taishan200-2280-2s64p-256g, maybe need check testbox field (ArgumentError) --- lib/job.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index 01719c2e..6dd93f01 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -337,7 +337,7 @@ class Job lab_hosts_file = get_lab_hosts_file return lab_hosts_file if lab_hosts_file - hosts_file_name = @job['tbox_group'].split('--')[0] + hosts_file_name = @job['tbox_group'].split(/(\.|--)/)[0] hosts_file = "#{LKP_SRC}/hosts/#{hosts_file_name}" return hosts_file if File.file?(hosts_file) -- 2.23.0

2 2