[Compass-ci] Re: [PATCH compass-ci 2/2] add template compare mode

On Tue, Nov 03, 2020 at 02:49:50PM +0800, Lu Weitao wrote:

add template compare mode in sbin/compare(command-line tool) by:

• load and parse compare.yaml
• calls lib/es_query.rb, query jobs data set from ES
• calls lib/matrix.rb, convert jobs data set to group matrices
• calls lib/compare_matrixes.rb, use groups matrices to compare

background: For support compare with user-defined template feature, the work-flow of user-defined template feature: load_compare_template.yaml --> query_results(ES) ---> auto group jobs_list ---> create groups_matrices ---> compare_values by each metrics ---> format/show results

example: compare -t/--template compare_template.yaml, the yaml like: compare_metrics: - fio.write_iops - fio.read_iops filter: suite: - fio-basic os_arch: - aarch64 - x86 compare_dimensions: - os: debian os_version: sid - os: openeuler os_version: 20.03 x_params: - bs - test_size

title: Hackbench Performance Testing unit: KB/s

output:{ "title": "Hackbench Performance Testing", "unit": "KB/s", "x_name": "bs|test_size", "tables": { "4k|1G": { "fio.write_iops": { "average": { "openeuler 20.03": 308298.5524761111, "debian sid": 220289.07563 }, "standard_deviation": { "openeuler 20.03": null, "debian sid": null }, "change": { "debian sid vs openeuler 20.03": -28.5 } } }, "16k|1G": {...}, ... } }

Signed-off-by: Lu Weitao luweitaobe@163.com

lib/compare.rb | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++ sbin/compare | 16 +++++++++++++--- 2 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/lib/compare.rb b/lib/compare.rb index 2837edc..26e6b49 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -6,6 +6,7 @@ require_relative './es_query.rb' require_relative './matrix2.rb' require_relative './compare_matrixes.rb' require_relative './constants.rb' +require 'yaml'

# ------------------------------------------------------------------------------------------- # compare_matrices_list @@ -66,3 +67,54 @@ def create_groups_matrices_list(conditions, dims) query_results = es.multi_field_query(conditions) combine_group_query_data(query_results, dims) end

+# ------------------------------------------------------------------------------------------- +# compare with user defined compare_template.yaml +# the data change flow: +# load_compare_template.yaml --> query_results(ES) ---> auto_group_jobs_list +# ---> groups_matrices ---> compare_values ---> format/show_results +#

+def compare_by_template(template)

• template_params = load_template(template)
• groups_matrices = create_groups_matrices(template_params)
• compare_values = compare_metrics_values(groups_matrices)
• chart_results = format_for_chart(compare_values, template_params)
• show_chart_compare_result(chart_results)

+end

+def load_template(template)

• unless File.file?(template)
• warn 'template does not exist'
• exit
• end
• YAML.safe_load(File.open(template))

you can use YAML.load_file here directly

Thanks Luan Shengde

+end

+# according to user's template to create groups_matrices +# input: template_params: Hash +# eg: +# { +# "compare_metrics"=>["fio.write_iops", "fio.read_iops"], +# "filter"=>[ +# {"suite"=>["fio-bisic"]}, +# {"os_arch"=>["aarch_64"]} +# ], +# "compare_dimensions"=>[ +# {"os"=>"openeuler", "os_version"=>20.03}, +# {"os"=>"centos", "os_version"=>7.6} +# ], +# "x_params"=>["block_size", "package_size"], +# "title"=>"Hackbench Performance Testing", +# "unit"=>"KB/s" +# } +def create_groups_matrices(template_params)

• es = ESQuery.new
• query_results = es.multi_field_query(template_params['filter'])
• combine_group_query_data(
• query_results,
• template_params['x_params'],
• template_params['compare_dimensions'],
• template_params['compare_metrics']
• )

+end diff --git a/sbin/compare b/sbin/compare index cf3980c..b1bc49a 100755 --- a/sbin/compare +++ b/sbin/compare @@ -6,6 +6,7 @@ # Usage: # compare "conditions_1" "conditions_2" ... -c "common_conditions" # compare "conditions" -d "dimensions" +# compare -t template.yaml # Eg: # compare "id=6000,6001" "id=7000,7001" # compare "commit=a12d232e" "commit=b3bacc31" @@ -21,6 +22,7 @@ is_group = false dimensions = nil colorful = nil options = {} +template = nil

opt_parser = OptionParser.new do |opts| opts.banner = 'Usage: compare "conditions" ... [option]' @@ -45,6 +47,10 @@ opt_parser = OptionParser.new do |opts| colorful = color end

• opts.on('-t', '--template template', 'compare with user-defined template') do |t|
• template = t
• end
• opts.on_tail('-h', '--help', 'show this message') do puts opts exit

@@ -60,8 +66,12 @@ opt_parser.parse!(argv)

options = { theme: colorful } if colorful

-if is_group

• compare_group(argv, dimensions, options)

+if template

• compare_by_template(template)

else

• compare_matrices_list(argv, common_conditions, options)

• if is_group
• compare_group(argv, dimensions, options)
• else
• compare_matrices_list(argv, common_conditions, options)
• end

end

2.23.0