[why] Currently, the output result of compare default order by alphabet. If there have too many lines in result, it will be difficult for user to find the lines that starts with $suite in metric column. [how] Divide the lines into two parts before print according to whether the line start with $suite or not, then combine output result into a whole to highlight the important info. [example] command: compare "os=openeuler" -d "tbox_group" in z9 one of group compare output result. suites_list:["iso2rootfs", "build-pkg", "deploy-cci"] Before: taishan200-2280-2s64p-256g--a101 vm-hi1620-2p8g--roota15 vm-hi1620-2p8g--roota14 taishan200-2280-2s64p-256g--a6 metric -------------------- ------------------------------ ------------------------------ ------------------------------ ------------------------------ %stddev change %stddev change %stddev change %stddev \ | \ | \ | \ 493.41 ± 108% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.boot 47114.17 ± 109% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.idle 11.57 -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.kernel_boot 0.00 0 401.60 ± 319% 0 553.75 ± 358% 0 0.00 build-pkg.time.elapsed_time 0.00 0 401.60 ± 319% 0 553.75 ± 358% 0 0.00 build-pkg.time.elapsed_time.max 0.00 0 54815.06 ± 124% 0 52113.21 ± 173% 0 0.00 build-pkg.time.file_system_inputs 0.00 0 3730.76 ± 362% 0 9382.32 ± 552% 0 0.00 build-pkg.time.involuntary_context_switches 0.00 0 7.12 ± 365% 0 6.78 ± 379% 0 0.00 build-pkg.time.major_page_faults 0.00 0 35358.13 ± 194% 0 42519.61 ± 208% 0 0.00 build-pkg.time.maximum_resident_set_size 0.00 0 56001.11 ± 360% 0 160421.51 ± 553% 0 0.00 build-pkg.time.minor_page_faults 0.00 0 65536.00 0 65536.00 0 0.00 build-pkg.time.page_size 0.00 0 7.58 ± 755% 0 40.12 ± 903% 0 0.00 build-pkg.time.user_time 0.00 0 16269.71 ± 318% 0 29779.86 ± 394% 0 0.00 build-pkg.time.voluntary_context_switches 0.00 0 0.00 0 0.00 0 5762.00 deploy-cci.time.elapsed_time 0.00 0 0.00 0 0.00 0 5762.00 deploy-cci.time.elapsed_time.max ... 5579.67 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.CPU95.IPI1:Function_call_interrupts 3152478.83 ± 215% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.IPI0:Rescheduling_interrupts 570272.33 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.IPI1:Function_call_interrupts 551.49 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.elapsed_time 551.49 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.elapsed_time.max 73066.67 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.file_system_inputs 124.50 ± 217% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.involuntary_context_switches 4.33 ± 172% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.major_page_faults 34816.00 ± 140% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.maximum_resident_set_size 23354.67 ± 202% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.minor_page_faults 65536.00 -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.page_size 9.50 ± 68% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.percent_of_cpu_this_job_got 4.27 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.system_time 5.20 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.user_time 172904.17 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.voluntary_context_switches 2651512.00 ± 157% -61.9% 1011304.90 ± 25% -60.9% 1037595.96 ± 25% +582.1% 18086008.00 meminfo.Active 2253381.67 ± 176% -69.2% 694564.75 ± 35% -68.1% 718592.59 ± 35% +668.1% 17308404.00 meminfo.Active(anon) 466201.00 ± 223% -100.0% 0.00 -100.0% 0.00 +2509.3% 12164514.00 meminfo.AnonHugePages 1328043.33 ± 148% -50.7% 655286.32 ± 36% -49.3% 673334.38 ± 34% +966.0% 14157259.00 meminfo.AnonPages 0.00 0 524032.00 0 524032.00 0 520576.00 meminfo.CmaFree ... After: taishan200-2280-2s64p-256g--a101 vm-hi1620-2p8g--roota15 vm-hi1620-2p8g--roota14 taishan200-2280-2s64p-256g--a6 metric -------------------- ------------------------------ ------------------------------ ------------------------------ ------------------------------ %stddev change %stddev change %stddev change %stddev \ | \ | \ | \ 0.00 0 401.60 ± 319% 0 553.75 ± 358% 0 0.00 build-pkg.time.elapsed_time 0.00 0 401.60 ± 319% 0 553.75 ± 358% 0 0.00 build-pkg.time.elapsed_time.max 0.00 0 54815.06 ± 124% 0 52113.21 ± 173% 0 0.00 build-pkg.time.file_system_inputs 0.00 0 3730.76 ± 362% 0 9382.32 ± 552% 0 0.00 build-pkg.time.involuntary_context_switches 0.00 0 7.12 ± 365% 0 6.78 ± 379% 0 0.00 build-pkg.time.major_page_faults 0.00 0 35358.13 ± 194% 0 42519.61 ± 208% 0 0.00 build-pkg.time.maximum_resident_set_size 0.00 0 56001.11 ± 360% 0 160421.51 ± 553% 0 0.00 build-pkg.time.minor_page_faults 0.00 0 65536.00 0 65536.00 0 0.00 build-pkg.time.page_size 0.00 0 7.58 ± 755% 0 40.12 ± 903% 0 0.00 build-pkg.time.user_time 0.00 0 16269.71 ± 318% 0 29779.86 ± 394% 0 0.00 build-pkg.time.voluntary_context_switches 0.00 0 0.00 0 0.00 0 5762.00 deploy-cci.time.elapsed_time 0.00 0 0.00 0 0.00 0 5762.00 deploy-cci.time.elapsed_time.max 0.00 0 0.00 0 0.00 0 1.00 deploy-cci.time.exit_status 0.00 0 0.00 0 0.00 0 503552.00 deploy-cci.time.file_system_inputs 0.00 0 0.00 0 0.00 0 3925.00 deploy-cci.time.involuntary_context_switches 0.00 0 0.00 0 0.00 0 28.00 deploy-cci.time.major_page_faults 0.00 0 0.00 0 0.00 0 271168.00 deploy-cci.time.maximum_resident_set_size 0.00 0 0.00 0 0.00 0 2280195.00 deploy-cci.time.minor_page_faults 0.00 0 0.00 0 0.00 0 65536.00 deploy-cci.time.page_size 0.00 0 0.00 0 0.00 0 3.00 deploy-cci.time.percent_of_cpu_this_job_got 0.00 0 0.00 0 0.00 0 17.61 deploy-cci.time.system_time 0.00 0 0.00 0 0.00 0 176.09 deploy-cci.time.user_time 0.00 0 0.00 0 0.00 0 384342.00 deploy-cci.time.voluntary_context_switches 551.49 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.elapsed_time 551.49 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.elapsed_time.max 73066.67 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.file_system_inputs 124.50 ± 217% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.involuntary_context_switches 4.33 ± 172% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.major_page_faults 34816.00 ± 140% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.maximum_resident_set_size 23354.67 ± 202% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.minor_page_faults 65536.00 -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.page_size 9.50 ± 68% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.percent_of_cpu_this_job_got 4.27 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.system_time 5.20 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.user_time 172904.17 ± 223% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 iso2rootfs.time.voluntary_context_switches 493.41 ± 108% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.boot 47114.17 ± 109% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.idle 11.57 -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 boot-time.kernel_boot 1073.50 ± 217% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.17:GICv3.23.Level.arm-pmu 1003392.50 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.226:ITS-MSI.65536001.Edge.hns3-0000:7d:00.0-TxRx-0 612284.17 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.4:GICv3.26.Level.arch_timer 8151.83 ± 213% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.CPU0.4:GICv3.26.Level.arch_timer 165405.33 ± 222% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.CPU0.IPI0:Rescheduling_interrupts 5886.83 ± 221% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.CPU0.IPI1:Function_call_interrupts 12477.50 ± 214% -100.0% 0.00 -100.0% 0.00 -100.0% 0.00 interrupts.CPU1.4:GICv3.26.Level.arch_timer ... Signed-off-by: Lu Kaiyi <2392863668@qq.com> --- lib/compare.rb | 15 +++++++++------ lib/compare_matrixes.rb | 33 ++++++++++++++++++++++----------- lib/matrix2.rb | 15 +++++++++++---- 3 files changed, 42 insertions(+), 21 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index 645f682..3db5169 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -20,8 +20,8 @@ require 'yaml' def compare_matrices_list(argv, common_conditions, options) condition_list = parse_argv(argv, common_conditions) - matrices_list = create_matrices_list(condition_list) - compare_matrixes(matrices_list, options: options) + matrices_list, suites_list = create_matrices_list(condition_list) + compare_matrixes(matrices_list, suites_list, options: options) end def parse_argv(argv, common_conditions) @@ -37,12 +37,15 @@ end def create_matrices_list(conditions) matrices_list = [] + suites_list = [] es = ESQuery.new(ES_HOST, ES_PORT) conditions.each do |condition| query_results = es.multi_field_query(condition) - matrices_list << combine_query_data(query_results) + matrix, suites = combine_query_data(query_results) + matrices_list << matrix + suites_list.concat(suites) end - matrices_list + return matrices_list, suites_list end # ------------------------------------------------------------------------------------------- @@ -58,8 +61,8 @@ end def compare_group(argv, dimensions, options) conditions = parse_conditions(argv) dims = dimensions.split(' ') - groups_matrices = create_groups_matrices_list(conditions, dims) - compare_group_matrices(groups_matrices, options) + groups_matrices, suites_list = create_groups_matrices_list(conditions, dims) + compare_group_matrices(groups_matrices, suites_list, options) end def create_groups_matrices_list(conditions, dims) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 119d42d..c1cee10 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -193,7 +193,7 @@ def matrixes_empty?(matrixes_list) return matrixes_list.any?(&:empty?) end -def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, options: {}) +def compare_matrixes(matrixes_list, suites_list, matrixes_titles = nil, group_key = nil, options: {}) # compare matrix in matrixes_list and print info # @matrixes_list: list consisting of matrix # @matrixes_titles: number or dimension of matrix @@ -208,6 +208,7 @@ def compare_matrixes(matrixes_list, matrixes_titles = nil, group_key = nil, opti result_str = group_key ? "\n\n\n\n\n" + group_key : '' result_str += get_all_result_str( matrixes_values, + suites_list, matrixes_titles, matrixes_list.size, options[:theme] @@ -382,7 +383,7 @@ end # compare each matrices_list within pre dimension of group matrices # input: group matrices # output: pre compare result of each group -def compare_group_matrices(group_matrices, options) +def compare_group_matrices(group_matrices, suites_list, options) result_str = '' group_matrices.each do |k, v| matrices_list = [] @@ -392,9 +393,9 @@ def compare_group_matrices(group_matrices, options) matrices_list << matrix end if options[:no_print] - result_str += compare_matrixes(matrices_list, matrices_titles, k, options: options) + result_str += compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) else - print compare_matrixes(matrices_list, matrices_titles, k, options: options) + print compare_matrixes(matrices_list, suites_list.shift, matrices_titles, k, options: options) end end result_str @@ -807,26 +808,36 @@ def get_theme(matrixes_values, matrixes_titles, theme) return THEMES[:none] end -def get_all_result_str(matrixes_values, matrixes_titles, matrixes_number, theme) +def get_all_result_str(matrixes_values, suites_list, matrixes_titles, matrixes_number, theme) matrixes_titles ||= matrixes_number.times.to_a.map(&:to_s) theme = get_theme(matrixes_values, matrixes_titles, theme) return '' unless theme - failure_str = get_result_str(matrixes_values[false].sort, matrixes_titles, false, theme) - success_str = get_result_str(matrixes_values[true].sort, matrixes_titles, true, theme) + failure_str = get_result_str(matrixes_values[false].sort, suites_list, matrixes_titles, false, theme) + success_str = get_result_str(matrixes_values[true].sort, suites_list, matrixes_titles, true, theme) failure_str + success_str end -def get_result_str(values, matrixes_titles, success, theme) +def get_result_str(values, suites_list, matrixes_titles, success, theme) return '' if values.empty? + suites_list.uniq! result_str = "\n\n\n" common_title, compare_title = get_title_name(success) result_str += get_header(matrixes_titles, success, common_title, compare_title) + suite_str = '' + common_str = '' values.each do |field, matrixes| - result_str += get_values_str(matrixes, success, theme) - result_str += get_field_str(field) - result_str += "\n" + row = get_values_str(matrixes, success, theme) + row += get_field_str(field) + "\n" + field_start_with_suite = suites_list.any? { |suite| field.start_with?(suite) } + if field_start_with_suite + suite_str += row + else + common_str += row + end end + + result_str += suite_str + common_str result_str end diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 26f07fa..479d280 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -76,7 +76,9 @@ end # } def create_matrix(job_list) matrix = {} + suites = [] job_list.each do |job| + suites << job['suite'] if job['suite'] stats = job['stats'] next unless stats @@ -89,7 +91,7 @@ def create_matrix(job_list) matrix.each_value do |value| samples_fill_missing_zeros(value, col_size) end - matrix + return matrix, suites end # input: query results from es_query @@ -110,15 +112,20 @@ end # ... # } def combine_group_query_data(query_data, dims) + suites_list = [] job_list = query_data['hits']['hits'] groups = auto_group(job_list, dims) groups.each do |group_key, value| + suite_list = [] value.each do |dimension_key, jobs| - groups[group_key][dimension_key] = create_matrix(jobs) + groups[group_key][dimension_key], suites = create_matrix(jobs) + suite_list.concat(suites) end groups.delete(group_key) if value.size < 2 + suites_list << suite_list unless value.size < 2 end - groups + + return groups, suites_list end # input: @@ -139,7 +146,7 @@ def combine_group_jobs_list(query_data, groups_params, dimensions, metrics) groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) groups.each do |group_key, dims| dims.each do |dim_key, jobs| - groups[group_key][dim_key] = create_matrix(jobs) + groups[group_key][dim_key], = create_matrix(jobs) end end -- 2.23.0