November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v9 compass-ci 10/11] mail-robot: start file
by Luan Shengde 18 Nov '20

18 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..5827e18 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,37 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +docker_rm 'mail-robot' + +names = Set.new %w[ + JUMPER_HOST + JUMPER_PORT + SEND_MAIL_PORT +] + +defaults = relevant_defaults(names) +defaults['MAILDIR'] = '/srv/cci/Maildir/.compass-ci' + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + --name=mail-robot + -d +] + env + %W[ + -v /c/upstream-repos:/c/upstream-repos:ro + -v #{defaults['MAILDIR']}:#{defaults['MAILDIR']}:rw + -v #{ENV['CCI_SRC']}:/c/compass-ci:rw + -w /c/compass-ci/container/mail-robot + mail-robot +] + +cmd += ['ruby', 'mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH v9 compass-ci 09/11] mail-robot: build file
by Luan Shengde 18 Nov '20

18 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/build | 5 +++++ 1 file changed, 5 insertions(+) create mode 100755 container/mail-robot/build diff --git a/container/mail-robot/build b/container/mail-robot/build new file mode 100755 index 0000000..c564add --- /dev/null +++ b/container/mail-robot/build @@ -0,0 +1,5 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t mail-robot:latest . -- 2.23.0

1 0

[PATCH v9 compass-ci 07/11] mail-robot: build fail message
by Luan Shengde 18 Nov '20

18 Nov '20

build fail email message with raised error message - lack commit url - repo url for the commit url not in upstream - commit not exist or not available - lack pubkey - no more available account on jumper server Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 38 +++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100755 container/mail-robot/lib/assign-account-fail-eamil.rb diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb new file mode 100755 index 0000000..593f345 --- /dev/null +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -0,0 +1,38 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_apply_account_fail_email(my_info, message) + email_msg = <<~EMAIL_MESSAGE + To: #{my_info['my_email']} + Subject: [compass-ci] apply account failed + + Dear user: + + Your application for account failed with following error: + + #{message} + + In order to successfully apply an account, please pay attention to the following points: + + 1. mail subject + The subject should exactly: apply account + + 2. commit url + When you writing the url, add prefix: my oss commit + example: + my oss commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… + + attention: + Ensure you commit url exist and available to access. + + 3. ssh pubkey + You need to add a pubkey as an attachment to the email. + + regards + compass-ci + EMAIL_MESSAGE + + return email_msg +end -- 2.23.0

1 0

[PATCH v9 compass-ci 06/11] mail-robot: build success email
by Luan Shengde 18 Nov '20

18 Nov '20

build email message for successfully assigning account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/assign-account-email.rb | 53 +++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100755 container/mail-robot/lib/assign-account-email.rb diff --git a/container/mail-robot/lib/assign-account-email.rb b/container/mail-robot/lib/assign-account-email.rb new file mode 100755 index 0000000..b793232 --- /dev/null +++ b/container/mail-robot/lib/assign-account-email.rb @@ -0,0 +1,53 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +def build_apply_account_email(my_info) + email_msg = <<~EMAIL_MESSAGE + To: #{my_info['my_email']} + Subject: [compass-ci] Account Ready + + Dear #{my_info['my_name']}, + + Thank you for joining us. + + You can use the following info to submit jobs: + + notice: + the followings are ONE-TIME setup + + 1) setup default config + run the following command to add the below setup to default config file + + mkdir -p ~/.config/compass-ci/defaults/ + cat >> ~/.config/compass-ci/defaults/account.yaml <<-EOF + my_email: #{my_info['my_email']} + my_name: #{my_info['my_name']} + my_uuid: #{my_info['my_uuid']} + EOF + + 2) download lkp-tests and dependencies + run the following command to install the lkp-test and effect the configuration + + git clone https://gitee.com/wu_fengguang/lkp-tests.git + cd lkp-tests + make install + source ${HOME}/.bashrc && source ${HOME}/.bash_profile + + 3) submit job + reference: https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/tutorial.md + + reference to 'how to write job yaml' section to write the job yaml + you can also reference to files in lkp-tests/jobs as example. + + submit jobs, for example: + + submit -m iperf.yaml testbox=vm-2p8g queue=vm-2p8g~\\$USER + + regards + compass-ci + EMAIL_MESSAGE + + return email_msg +end -- 2.23.0

1 0

[PATCH v9 compass-ci 05/11] mail-robot: apply new account for user
by Luan Shengde 18 Nov '20

18 Nov '20

ApplyJumperAccount used to apply account for user input: my_info output: account_info apply_jumper_account apply account with my_info my_info: - my_email - my_name - my_uuid - my_ssh_pubkey account_info: - my_login_name - my_password - jumper_host - jumper_port - my_jumper_pubkey Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/apply-jumper-account.rb | 42 +++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 container/mail-robot/lib/apply-jumper-account.rb diff --git a/container/mail-robot/lib/apply-jumper-account.rb b/container/mail-robot/lib/apply-jumper-account.rb new file mode 100755 index 0000000..44f412b --- /dev/null +++ b/container/mail-robot/lib/apply-jumper-account.rb @@ -0,0 +1,42 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' + +JUMPER_HOST = ENV['JUMPER_HOST'] || 'api.compass-ci.openeuler.org' +JUMPER_PORT = ENV['JUMPER_PORT'] || 29999 + +# used to apply account +# be called after AssignAccount successfully parsed my_commit_url and my_ssh_pubkey +# apply_jumper_account +# apply jumper account with my_info and my_ssh_pubkey +# account_info_exist +# check account exists +class ApplyJumperAccount + def initialize(my_info) + @jumper_host = JUMPER_HOST + @jumper_port = JUMPER_PORT + @my_info = my_info.clone + end + + def apply_jumper_account + account_info_str = %x(curl -XGET "#{@jumper_host}:#{@jumper_port}/assign_account" \ + -d '#{(a)my_info.to_json}') + account_info = JSON.parse account_info_str + + account_info_exist(account_info) + + return account_info + end + + def account_info_exist(account_info) + return unless account_info['my_login_name'].nil? + + error_message = 'No more available jumper account.' + error_message += 'You may try again later or consulting the manager for a solution.' + + raise error_message + end +end -- 2.23.0

1 0

[PATCH v9 compass-ci 04/11] mail-robot: check commit url for gitee.com
by Luan Shengde 18 Nov '20

18 Nov '20

GiteeCommitUrlCheck used to check commit for gitee.com input: my_commit_url gitee_commit_check clone the repo and check commit exists and availalbe Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../mail-robot/lib/gitee-commit-url-check.rb | 47 +++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100755 container/mail-robot/lib/gitee-commit-url-check.rb diff --git a/container/mail-robot/lib/gitee-commit-url-check.rb b/container/mail-robot/lib/gitee-commit-url-check.rb new file mode 100755 index 0000000..72c948f --- /dev/null +++ b/container/mail-robot/lib/gitee-commit-url-check.rb @@ -0,0 +1,47 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' + +# used to check commit available for hub gitee.com +# it is called when: +# - ParseApplyAccountEmail execute commit_url_availability +# - hub is gitee.com +# gitee_commit_check +# clone the repo +# check commit available +class GiteeCommitUrlCheck + def initialize(my_info, url, base_url) + @my_info = my_info + @url = url + @base_url = base_url + end + + def gitee_commit_check + repo_dir = @url.split('/')[-3] + repo_url = [@base_url, 'git'].join('.') + commit_id = @url.split('/')[-1] + + Dir.chdir '/tmp' + %x(/usr/bin/git clone #{repo_url} #{repo_dir}) + + email_index = %x(/usr/bin/git -C #{repo_dir} show #{commit_id}).index @my_info['my_email'] + + FileUtils.rm_rf repo_dir + + gitee_commit_exist(email_index) + end + + def gitee_commit_exist(email_index) + return if email_index + + error_message = "We can not confirm whether the commit url matches your email.\n" + error_message += 'Make sure that the commit url is right,' + error_message += ' or it is truely submitted with your email.' + + raise error_message + end +end -- 2.23.0

1 0

[PATCH v9 compass-ci 03/11] mail-robot: parse apply account email
by Luan Shengde 18 Nov '20

18 Nov '20

ParseApplyAccountEmail used to parse my_email, my_name, my_commit_url and my_ssh_pubkey input: mail_content output: my_info - my_email - my_name - my_commit_url - my_ssh_pubkey init my_info parse_commit_url extract_commit_url extract my_commit_url from mail_content base_url_in_upstream_repos check whether the repo's url in upstream-repos commit_url_availability check the commit available - gitee.com call GiteeCommitUrlCheck to check commit available - non-gitee.com execute curl and check feedback for the commit parse_pub_key check attachment for my_ssh_pubkey Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 138 ++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100755 container/mail-robot/lib/parse-apply-account-email.rb diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb new file mode 100755 index 0000000..8c55d8f --- /dev/null +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -0,0 +1,138 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require_relative 'gitee-commit-url-check' + +# used to parse mail_content for my_commit_url and my_ssh_pubkey +# be called by AssignAccount when it needs to extract required data: +# - my_commit_url +# - my_ssh_pubkey +# +# input: mail_content +# output: my_commit_url, my_ssh_pubkey +# +# parse_commit_url +# parse my_commit_url from the mail_content and return it +# extract my_commit_url +# extract my_coomit_url from the mail_content +# check commit url exists +# base_url_in_upstream_repos +# check whether the base url in upstream-repos +# commit_url_availability +# check commit available +# gitee.com: +# call GiteeCommitUrlCheck to check the commit +# non-gitee.com: +# execute curl to check the commit +# parse_pub_key +# check my_ssh_pubkey exists and return it +class ParseApplyAccountEmail + def initialize(mail_content) + @mail_content = mail_content + + @my_info = { + 'my_email' => mail_content.from[0], + 'my_name' => mail_content.From.unparsed_value.gsub(/ <[^<>]*>/, ''), + 'my_ssh_pubkey' => [] + } + end + + def build_my_info + @my_info['my_commit_url'] = parse_commit_url + @my_info['my_ssh_pubkey'] << parse_pub_key + + return @my_info + end + + def extract_mail_content_line + mail_content_body = @mail_content.part[0].part[0].body.decoded || \ + @mail_content.part[0].body.decoded || \ + @mail_content.body.decoded + mail_content_line = mail_content_body.gsub(/\n/, '') + + return mail_content_line + end + + def extract_commit_url + mail_content_line = extract_mail_content_line + # the commit url should be headed with a prefix: my oss commit + # the commit url should be in a standart format, example: + # my oss commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… + unless mail_content_line.match?(%r{my oss commit:\s*https?://[^/]*/[^/]*/[^/]*/commit/[\w\d]{40}}) + error_message = "No matched commit url found.\n" + error_message += 'Ensure that you have add a right commit url, ' + error_message += "and with prefix 'my oss commit:'." + raise error_message + end + + mail_content_line.match(%r{https?://[^/]*/[^/]*/[^/]*/commit/[\w\d]{40}})[0] + end + + def parse_commit_url + url = extract_commit_url + base_url = url.gsub(%r{/commit/[\w\d]{40}$}, '') + + base_url_in_upstream_repos('/c/upstream-repos', base_url) + commit_url_availability(url, base_url) + + return url + end + + def base_url_in_upstream_repos(upstream_dir, base_url) + Dir.chdir(upstream_dir) + match_out = %x(grep -rn #{base_url}) + + return unless match_out.empty? + + error_message = "The repo url for your commit is not in our upstream-repo list.\n" + error_message += 'Use a new one, or consulting the manager for available repo list.' + + raise error_message + end + + def commit_url_availability(url, base_url) + hub_name = url.split('/')[2] + + # it requires authentication when execute curl to get the commit information + # clone the repo and then validate the commit for the email address + if hub_name.eql? 'gitee.com' + gitee_commit(url, base_url) + else + non_gitee_commit(url) + end + end + + def gitee_commit(url, base_url) + my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) + my_gitee_commit.gitee_commit_check + end + + def non_gitee_commit(url) + url_fdback = %x(curl #{url}) + email_index = url_fdback.index @my_info['my_email'] + + return unless email_index.nil? + + error_message = "We can not confirm the commit url matches your email.\n" + error_message += 'Make sure that the commit url is right,' + error_message += ' or it is truely submitted with you email.' + + raise error_message + end + + def parse_pub_key + error_message = "No pub_key found.\n" + error_message += 'Please add a pub_key as an attachment to your email.' + + raise error_message if @mail_content.attachments.empty? + raise error_message unless @mail_content.attachments[0].filename =~ /^id_.*\.pub$/ + + pub_key = @mail_content.attachments[0].body.decoded + + return pub_key + end +end -- 2.23.0

1 0

[PATCH v9 compass-ci 02/11] mail-robot: check to apply account
by Luan Shengde 18 Nov '20

18 Nov '20

ApplyAccount check to apply account for user when listened new email with subject 'apply account' input: mail_content output: send mail to user check_to_send_account parse_mail_content update my_info apply_my_account generate uuid apply account with my_info my_info: - my_email - my_name - my_uuid - my_ssh_pubkey store account info to es send email - send success email - send fail email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 110 ++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100755 container/mail-robot/lib/apply-account.rb diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb new file mode 100755 index 0000000..bf40429 --- /dev/null +++ b/container/mail-robot/lib/apply-account.rb @@ -0,0 +1,110 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'mail' +require_relative '../../../lib/es_client' +require_relative 'assign-account-email' +require_relative 'assign-account-fail-eamil' +require_relative 'apply-jumper-account' +require_relative 'parse-apply-account-email' + +SEND_MAIL_PORT = ENV['SEND_MAIL_PORT'] || 49000 + +# assign uuid/account for user +# when mail-robot listened new email, and the email's subject +# exactly equal 'apply account', mail-robot will call this class +# entry point: send_account +# input: email's content +# +# send_account +# parse_mail_content +# call ParseApplyAccountEmail to parse: +# - my_commit_url +# - my_ssh_pubkey +# apply_my_account +# call ApplyJumperAccount to apply new account +# required data: +# my_info: +# - my_email +# - my_name +# - my_uuid +# my_ssh_pubkey +# store_account_info +# call ESClient to store my_info +# my_info: +# - my_email +# - my_name +# - my_uuid +# - my_commit_url +# - my_login_name +# - my_ssh_pubkey +# send_mail +# when successfully applied an account +# call build_apply_account_email to send a successful email +# when rescued error message +# call build_apply_account_fail_email to send fail email +class ApplyAccount + def initialize(mail_content) + @send_mail_host = %x(/sbin/ip route | awk '/default/ {print $3}').chomp + @send_mail_port = SEND_MAIL_PORT + @mail_content = mail_content + + @my_info = {} + end + + def check_to_send_account + # in case of failed parsing, parse_mail_content will return none info + # in order to successfully send email for failed parsing + # firstly get my_email before execute parse_mail_content is needed + @my_info['my_email'] = @mail_content.from[0] + parse_mail_content + acct_info = apply_my_account + + @my_info['my_login_name'] = acct_info['my_login_name'] + + store_account_info + send_mail('') + rescue StandardError => e + puts e.message + puts e.backtrace + + send_mail(e.message) + end + + def parse_mail_content + parse_apply_account_email = ParseApplyAccountEmail.new(@mail_content) + + parsed_email_info = parse_apply_account_email.build_my_info + + @my_info.update parsed_email_info + end + + def apply_my_account + my_uuid = %x(uuidgen).chomp + + @my_info['my_uuid'] = my_uuid + + apply_account = ApplyJumperAccount.new(@my_info) + acct_info = apply_account.apply_jumper_account + + return acct_info + end + + def store_account_info + es = ESClient.new(index: 'accounts') + es.put_source_by_id(@my_info['my_email'], @my_info) + end + + def send_mail(error_message) + email_message = if error_message.empty? + build_apply_account_email(@my_info) + else + build_apply_account_fail_email(@my_info, error_message) + end + + %x(curl -XPOST "#{@send_mail_host}:#{@send_mail_port}/send_mail_text" -d "#{email_message}") + end +end -- 2.23.0

1 0

[PATCH v9 compass-ci 01/11] mail-robot: listen and handle new email
by Luan Shengde 18 Nov '20

18 Nov '20

listen the mailbox for new email handle_new_email handle the email with defined opetions - apply_account email subject: apply account Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/mail-robot.rb | 41 ++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100755 container/mail-robot/mail-robot.rb diff --git a/container/mail-robot/mail-robot.rb b/container/mail-robot/mail-robot.rb new file mode 100755 index 0000000..a809eae --- /dev/null +++ b/container/mail-robot/mail-robot.rb @@ -0,0 +1,41 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require 'yaml' +require 'listen' +require 'mail' +require 'fileutils' +require_relative 'lib/apply-account' + +MAILDIR = ENV['MAILDIR'] + +def monitor_new_email(mail_inbox, mail_drafts) + listener = Listen.to(mail_inbox) do |_modified, added, _removed| + next if added.empty? + + added.each do |mail_file| + handle_new_email(mail_file, mail_drafts) + end + end + listener.start + sleep +end + +def handle_new_email(mail_file, mail_drafts) + mail_content = Mail.read(mail_file) + apply_account(mail_content) + + FileUtils.mv(mail_file, mail_drafts) +end + +def apply_account(mail_content) + return unless mail_content.subject == 'apply account' + + assign_uuid = ApplyAccount.new(mail_content) + assign_uuid.check_to_send_account +end + +monitor_new_email("#{MAILDIR}/new", "#{MAILDIR}/cur") -- 2.23.0

1 0

[PATCH compass-ci] lib/compare_data_format: optimize the format for echart
by Lu Weitao 18 Nov '20

18 Nov '20

[Why] With template compare mode, the result(data_set) isn't friendly for echart [How] Make the result more friendly for echart like: { "title": "Hackbench Performance Testing", "unit": "KB/s", "x_name": "test_size|bs", "tables": { "fio.read_iops": { "average": { "dimensions": [ "compare_dimension", "openeuler 20.03" ], "source": [ ["1G|4k", "1G|16k", "1G|32k", "1G|64k", "1G|128k", "1G|256k", "1G|512k", "1G|1024k", "80G|4k"], ["openeuler 20.03", 144076.2903315, 37865.30472368628, 21145.10375497826, 14010.34254665909, 6701.240849466667, 3205.077255, 1367.476930860465, 673.3270888666667, 11601.099817 ] ] }, "standard_deviation": {...}, "change": {...} }, "fio.write_iops": {...} ... } } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_data_format.rb | 77 ++++++++++++++++++++++++++++++++++---- lib/compare_matrixes.rb | 3 +- 2 files changed, 71 insertions(+), 9 deletions(-) diff --git a/lib/compare_data_format.rb b/lib/compare_data_format.rb index 3d82550..b4f26ee 100644 --- a/lib/compare_data_format.rb +++ b/lib/compare_data_format.rb @@ -5,14 +5,75 @@ # ---------------------------------------------------------------------------------------------------- # format compare results for a specific format # +class FormatEchartData + def initialize(metrics_compare_results, template_params) + @metrics_compare_results = metrics_compare_results + @template_params = template_params + @data_set = {} + end -def format_for_echart(metrics_compare_results, template_params) - echart_result = {} - echart_result['title'] = template_params['title'] - echart_result['unit'] = template_params['unit'] - x_params = template_params['x_params'] - echart_result['x_name'] = x_params.join('|') if x_params - echart_result['tables'] = metrics_compare_results + def format_for_echart + echart_result = {} + echart_result['title'] = @template_params['title'] + echart_result['unit'] = @template_params['unit'] + @x_name = @template_params['x_params'] + echart_result['x_name'] = @x_name.join('|') if @x_name + echart_result['tables'] = convert_to_echart_dataset - echart_result + echart_result + end + + def convert_to_echart_dataset + @x_params = sort_x_params((a)metrics_compare_results.keys) + @metrics_compare_results.each_value do |metrics_values| + metrics_values.each do |metric, metric_value| + assign_echart_data_set(metric, metric_value) + end + end + + @data_set + end + + def assign_echart_data_set(metric, metric_value) + @data_set[metric] = {} + metric_value.each do |value_type, values| # value_type can be: average, standard_deviation, change + @data_set[metric][value_type] = { + 'dimensions' => ['compare_dimension'] + } + + dimension_list = values.keys.sort + @data_set[metric][value_type]['dimensions'] += dimension_list + @data_set[metric][value_type]['source'] = assign_echart_source(metric, value_type, dimension_list) + end + end + + def assign_echart_source(metric, value_type, dimensions) + source = [] + source << @x_params + dimensions.each do |dimension| + dimension_values = [dimension] + @x_params.each do |x_param| + if @metrics_compare_results[x_param][metric] + dimension_values << @metrics_compare_results[x_param][metric][value_type][dimension] + end + end + source << dimension_values + end + + source + end +end + +# input: x_params_list +# eg: ["1G|4K", "1G|1024k", "1G|128k", 2G|4k] +# output: +# ["1G|4K", "1G|128k", "1G|1024k", "2G|4k"] +def sort_x_params(x_params_list) + x_params_hash = {} + x_params_list.each do |param| + params = param.gsub(/[a-zA-Z]+$/, '0').split('|').map(&:to_i) + x_params_hash[params] = param + end + + x_params_hash.sort.map { |h| h[1] } end diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 119d42d..762c087 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -498,7 +498,8 @@ def get_dimentions_combination(dimension_list) end def show_compare_result(metrics_compare_results, template_params) - echart_results = format_for_echart(metrics_compare_results, template_params) + formatter = FormatEchartData.new(metrics_compare_results, template_params) + echart_results = formatter.format_for_echart print JSON.pretty_generate(echart_results) end -- 2.23.0

2 2

2024

2023

2022

2021

2020

Compass-ci November 2020