[Compass-ci] Re: [PATCH v2 compass-ci] fix: sched: submit idle jobs

On Wed, Sep 30, 2020 at 07:30:48AM +0800, Wu Fengguang wrote:

On Mon, Sep 28, 2020 at 10:41:22PM +0800, Ren Wen wrote:

...

1. submit all idles jobs once use '*'

-> submit each idle job once until all done. 2) add two fields into job if job's filename is 'multi-qemu.yaml'. 3) add field 'idle_job' for idle jobs, value is always 'true'.

...

def auto_submit_idle_job(tbox_group)

• full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml"
• Jobfile::Operate.auto_submit_job(

•  full_path_patterns,
  

•  "testbox: #{tbox_group}") if Dir.glob(full_path_patterns).size > 0
  

• idle_job_path = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/"
• if Dir.exists?(idle_job_path)

•  jobs = Dir[idle_job_path + "*.yaml"]
  

•  jobs.each do |job|
  

•    extra_job_fields = ["idle_job=true"]
  

•    if Path[job].basename == "multi-qemu.yaml"
  

•      extra_job_fields += ["FLUENTD_SERVER_HOST=#{ENV["FLUENTD_SERVER_HOST"]}",
  

•                           "FLUENTD_SERVER_PORT=#{ENV["FLUENTD_SERVER_PORT"]}"]
  

•    end
  

•    Jobfile::Operate.auto_submit_job(job, "testbox: #{tbox_group}", extra_job_fields)
  

•  end
  

• end end

The above loop may take long time. Will it cause HTTP timeout?

not sure. we will patch this (submit at backgroud). - normal http timeout is about 1 minutes, but it can set to 1 hours. - it maybe need a "long" time to submit all idle jobs.

Thanks, Fengguang