[PATCH compass-ci] create define_files in result_root
[Why] User need debug their result-parse-script which haven't commit in lkp-tests/stats. [How] We already support user write script as "define_file" => {"stats/$script_name" => "$script_content"} in job.yaml, the job.yaml will be save as job to ES DB, write the job['define_file']['stats/$script_name'](value) at $result_root/stats/$script, extract-service will parse result by $result_root/stats/$script(user's result-parse-script) firstly Signed-off-by: Lu Weitao <luweitaobe@163.com> --- src/extract-stats/stats_worker.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..ae5d703 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]? + define_files = job["define_files"]? + write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any) result_post_processing(job_id, result_root.to_s, queue_path) @tq.delete_task(queue_path + "/in_process", "#{job_id}") end @@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root) + + define_files.each do |k, v| + if k.match (/^stats/) + Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end + end + end end -- 2.23.0
On Mon, Nov 23, 2020 at 11:45:13AM +0800, Lu Weitao wrote:
[Why] User need debug their result-parse-script which haven't commit in lkp-tests/stats.
[How] We already support user write script as "define_file" => {"stats/$script_name" => "$script_content"} in job.yaml, the job.yaml will be save as job to ES DB, write the job['define_file']['stats/$script_name'](value) at $result_root/stats/$script, extract-service will parse result by $result_root/stats/$script(user's result-parse-script) firstly
Signed-off-by: Lu Weitao <luweitaobe@163.com> --- src/extract-stats/stats_worker.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..ae5d703 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]? + define_files = job["define_files"]? + write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any) result_post_processing(job_id, result_root.to_s, queue_path) @tq.delete_task(queue_path + "/in_process", "#{job_id}") end @@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root) + + define_files.each do |k, v| + if k.match (/^stats/) ^^^^^^^^^^^^^^^^^^^^^ one less indent if use next unless
next unless k.match (/^stats/)
+ Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end
Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| file.puts v end Thanks, Kaiyi
+ end + end end -- 2.23.0
@@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root) + + define_files.each do |k, v| + if k.match (/^stats/) ^^^^^^^^^^^^^^^^^^^^^ one less indent if use next unless
不需要next，k没有匹配上自动进入下次循环了
next unless k.match (/^stats/)
+ Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end
Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| file.puts v end
这里似乎没有变化？ Thanks, Weitao
Thanks, Kaiyi
+ end + end end -- 2.23.0
On Mon, Nov 23, 2020 at 11:45:13AM +0800, Lu Weitao wrote:
[Why] User need debug their result-parse-script which haven't commit in lkp-tests/stats.
[How] We already support user write script as "define_file" => {"stats/$script_name" => "$script_content"} in job.yaml, the job.yaml will be save as job to ES DB, write the job['define_file']['stats/$script_name'](value) at $result_root/stats/$script, extract-service will parse result by $result_root/stats/$script(user's result-parse-script) firstly
Signed-off-by: Lu Weitao <luweitaobe@163.com> --- src/extract-stats/stats_worker.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+)
diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..ae5d703 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]? + define_files = job["define_files"]? + write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any) result_post_processing(job_id, result_root.to_s, queue_path) @tq.delete_task(queue_path + "/in_process", "#{job_id}") end @@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root)
^------ delete "nil" ? function caller doesn't need to receive a return value.
+ define_files.each do |k, v| + if k.match (/^stats/)
^----- extra space. Thanks, Liushaofei
+ Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end + end + end end -- 2.23.0
+++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]? + define_files = job["define_files"]? + write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any) result_post_processing(job_id, result_root.to_s, queue_path) @tq.delete_task(queue_path + "/in_process", "#{job_id}") end @@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end + + def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any)) + return nil unless result_root && File.exists?(result_root)
^------ delete "nil" ? function caller doesn't need to receive a return value.
we can delete "nil" but write "nil" looks more clearly
+ define_files.each do |k, v| + if k.match (/^stats/)
^----- extra space.
OK Thanks, Weitao
Thanks, Liushaofei
+ Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats") + File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file| + file.puts v + end + end + end + end end -- 2.23.0
participants (3)
-
Liu Shaofei -
Lu Kaiyi -
Lu Weitao