[PATCH v3 compass-ci 2/2] create matrices-groups according to group jobs
create matrices-groups according to auto group job_list background: For support compare with user-defined template feature, the work-flow are: load_compare_template.yaml --> query_results(ES) ---> auto group jobs_list ---> create groups_matrices ---> compare_values by each metrics ---> format/show results current patch is the step: create groups_matrices Signed-off-by: Lu Weitao <luweitaobe@163.com> --- lib/matrix2.rb | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 05f96dc..a284c98 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -120,3 +120,28 @@ def combine_group_query_data(query_data, dims) end groups end + +# combine_group_query_data: combine query data from ES to group matrix +# input: +# 1. query results(job_list) from es_query that will be auto group by auto_group_by_template() +# 2. params from user's template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# return: group_matrix of Hash(String, Hash(String, matrix)) +def combine_group_query_data(query_data, groups_params, dimensions, metrics) + job_list = query_data['hits']['hits'] + groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) + groups.each do |group_key, dimensions| + dimensions.each do |dimension_key, jobs| + groups[group_key][dimension_key] = create_matrix(jobs) + end + end + groups +end -- 2.23.0
On Tue, Nov 03, 2020 at 02:49:42PM +0800, Lu Weitao wrote:
create matrices-groups according to auto group job_list
background: For support compare with user-defined template feature, the work-flow are: load_compare_template.yaml --> query_results(ES) ---> auto group jobs_list ---> create groups_matrices ---> compare_values by each metrics ---> format/show results
current patch is the step: create groups_matrices
Signed-off-by: Lu Weitao <luweitaobe@163.com> --- lib/matrix2.rb | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 05f96dc..a284c98 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -120,3 +120,28 @@ def combine_group_query_data(query_data, dims) end groups end + +# combine_group_query_data: combine query data from ES to group matrix
combine_group_query_data: can be removed. This comment block is close enough to the function body.
+# input: +# 1. query results(job_list) from es_query that will be auto group by auto_group_by_template() +# 2. params from user's template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# return: group_matrix of Hash(String, Hash(String, matrix)) +def combine_group_query_data(query_data, groups_params, dimensions, metrics) + job_list = query_data['hits']['hits'] + groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) + groups.each do |group_key, dimensions| + dimensions.each do |dimension_key, jobs| + groups[group_key][dimension_key] = create_matrix(jobs) + end + end + groups +end -- 2.23.0
On Tue, Nov 03, 2020 at 02:58:09PM +0800, Wu Fengguang wrote:
On Tue, Nov 03, 2020 at 02:49:42PM +0800, Lu Weitao wrote:
create matrices-groups according to auto group job_list
background: For support compare with user-defined template feature, the work-flow are: load_compare_template.yaml --> query_results(ES) ---> auto group jobs_list ---> create groups_matrices ---> compare_values by each metrics ---> format/show results
current patch is the step: create groups_matrices
Signed-off-by: Lu Weitao <luweitaobe@163.com> --- lib/matrix2.rb | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 05f96dc..a284c98 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -120,3 +120,28 @@ def combine_group_query_data(query_data, dims) end groups end + +# combine_group_query_data: combine query data from ES to group matrix
combine_group_query_data: can be removed. This comment block is close enough to the function body.
ok Thanks, Weitao
On Tue, Nov 03, 2020 at 02:49:42PM +0800, Lu Weitao wrote:
create matrices-groups according to auto group job_list
background: For support compare with user-defined template feature, the work-flow are: load_compare_template.yaml --> query_results(ES) ---> auto group jobs_list ---> create groups_matrices ---> compare_values by each metrics ---> format/show results
current patch is the step: create groups_matrices
Signed-off-by: Lu Weitao <luweitaobe@163.com> --- lib/matrix2.rb | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 05f96dc..a284c98 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -120,3 +120,28 @@ def combine_group_query_data(query_data, dims) end groups end + +# combine_group_query_data: combine query data from ES to group matrix +# input: +# 1. query results(job_list) from es_query that will be auto group by auto_group_by_template() +# 2. params from user's template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# return: group_matrix of Hash(String, Hash(String, matrix)) +def combine_group_query_data(query_data, groups_params, dimensions, metrics) + job_list = query_data['hits']['hits'] + groups = auto_group_by_template(job_list, groups_params, dimensions, metrics)
The function name, "auto_group_by_template", I feel so wired, which sound like will do something, then return true/false. Thanks, Zhang Yuhang
+ groups.each do |group_key, dimensions| + dimensions.each do |dimension_key, jobs| + groups[group_key][dimension_key] = create_matrix(jobs) + end + end + groups +end -- 2.23.0
On Tue, Nov 03, 2020 at 03:07:30PM +0800, Zhang Yuhang wrote:
On Tue, Nov 03, 2020 at 02:49:42PM +0800, Lu Weitao wrote:
+# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# return: group_matrix of Hash(String, Hash(String, matrix)) +def combine_group_query_data(query_data, groups_params, dimensions, metrics) + job_list = query_data['hits']['hits'] + groups = auto_group_by_template(job_list, groups_params, dimensions, metrics)
The function name, "auto_group_by_template", I feel so wired, which sound like will do something, then return true/false.
actually, we can do like this, reference: todo/doc/the-art-of-readable-code.md ## 用具体的名字代替抽象的名字 在给变量、函数或者其他元素命名时，要把它描述的更具体而不是更抽象。 ### 例1 ServerCanStart() => 直接描述这个方法要做什么事情： CanListenOnPort() Thanks, Weitao
Thanks, Zhang Yuhang
participants (3)
-
Lu Weitao -
Wu Fengguang -
Zhang Yuhang