[Compass-ci] Re: [PATCH v2 compass-ci] create define_files in result_root

On Mon, Nov 23, 2020 at 08:26:58PM +0800, Li Ping wrote:

On Mon, Nov 23, 2020 at 04:08:53PM +0800, Lu Weitao wrote:

...

[Why] User need debug their result-parse-script which haven't commit in lkp-tests/stats.

[How] We already support user write script as "define_file" => {"stats/$script_name" => "$script_content"} in job.yaml, the job.yaml will be save as job to ES DB, write the job['define_file']['stats/$script_name'](value) at $result_root/stats/$script, extract-stats service will parse result by $result_root/stats/$script(user's result-parse-script) firstly

Signed-off-by: Lu Weitao luweitaobe@163.com

src/extract-stats/stats_worker.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+)

diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index bcf6e52..74afee1 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -22,6 +22,8 @@ class StatsWorker if job_id # will consume the job by post-processing job = @es.get_job_content(job_id) result_root = job["result_root"]?

•      define_files = job["define_files"]?
  

•      write_user_stats(result_root.to_s, define_files.as_h) if define_files.is_a?(JSON::Any)
  

   Submit with the option '-a' is not recommended.
   User can bring their all scripts to the directory in lkp-tests
   when building package like:
   cp $testcase ==> /lkp/lkp/src/tests/$testcase
   Then unpacking, the stats/$testcase file would be found.
   so user would submit with the option '-a' in the first time,
   the next time they don't have to if the initrd_pkg has been
   generated.

   Thanks,
   Ping

got it

Thanks, Weitao

...

      result_post_processing(job_id, result_root.to_s, queue_path)
      @tq.delete_task(queue_path + "/in_process", "#{job_id}")
    end

@@ -139,4 +141,17 @@ class StatsWorker STDERR.puts e.message end end

• def write_user_stats(result_root : String, define_files : Hash(String, JSON::Any))
• return nil unless result_root && File.exists?(result_root)
• define_files.each do |k, v|

•  if k.match(/^stats/)
  

•    Dir.mkdir("#{result_root}/stats") unless File.exists?("#{result_root}/stats")
  

•    File.open("#{result_root}/#{k}", "w+", File::Permissions.new(0o775)) do |file|
  

•      file.puts v
  

•    end
  

•  end
  

• end
• end

end

2.23.0