[PATCH v2 compass-ci] add queue field into job.yaml
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end + private def set_queue + if @hash.has_key?("queue") + if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port()
+ set_queue()
the @hash is come from submit_job's job_content, that include ["queue"] - as default, no need to set_queue - here is default enry
end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue")
+ if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end
if has ["queue"], let it be, no need to recheck.
+ return + end + + # set default value
+ if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else
tbox_group? this is about how to get/set tbox_group. not set_queue.
+ self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
On Mon, Oct 26, 2020 at 12:28:47PM +0800, Tong Qunfeng wrote:
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port()
+ set_queue()
the @hash is come from submit_job's job_content, that include ["queue"] - as default, no need to set_queue - here is default enry
if user not write queue field on job.yaml we need set a default value so need a set_dafault_queue func
end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue")
+ if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end
if has ["queue"], let it be, no need to recheck.
got it,
+ return + end + + # set default value
+ if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else
tbox_group? this is about how to get/set tbox_group. not set_queue.
for HW: testbox: taishan200-2280-2p48g--a11 so it's queue default value is queue: taishan200-2280-2p48g--a11 the same as tbox_group Thanks, Shenwei
+ self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue")
return unless @hash.has_key?("queue") ?
+ if !@hash["queue"].to_s.starts_with?(tbox_group)
use unless instead of if !. Thanks Luan Shengde
+ raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
return unless @hash.has_key?("queue") ?
+ if !@hash["queue"].to_s.starts_with?(tbox_group)
use unless instead of if !.
good ! Thanks, Shenwei
Thanks Luan Shengde
+ raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
now what field to discribe in queue?
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue") + if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
On Mon, Oct 26, 2020 at 04:27:33PM +0800, Liu Shaofei wrote:
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
now what field to discribe in queue?
if a job add the queue, the queue name is "queues/sched/vm-2p8g~xsw1/ready" only use queue to replace tbox_group Thanks, Shenwei
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue") + if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value + if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
On Mon, Oct 26, 2020 at 11:44:25AM +0800, Xiao Shenwei wrote:
[why] after one job submit, it will be add specified queue before: use tbox_group to discriminate now : use queue field to discriminate
so, we should add this field into job.yaml it's come from user's specify(job.yaml) or dafault then put the task into the queue
Signed-off-by: Xiao Shenwei <xiaoshenwei96@163.com> --- src/lib/job.cr | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..14e6ec7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -139,6 +139,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end
private def append_init_field @@ -199,6 +200,22 @@ class Job self["result_service"] = "raw_upload" end
+ private def set_queue + if @hash.has_key?("queue") + if !@hash["queue"].to_s.starts_with?(tbox_group) + raise "an incorrect queue-prefix is specified: #{queue}" + end + return + end + + # set default value
+ if tbox_group.to_s.starts_with?("vm") || tbox_group.to_s.starts_with?("dc") + self["queue"] = "#{tbox_group}.#{arch}" + else + self["queue"] = tbox_group + end
this seems to be an isolate requirement: "append arch" to virtual "testbox"?
+ end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
+ end
this seems to be an isolate requirement: "append arch" to virtual "testbox"?
if a job need running on x86_64, then it's queue name should use vm-2p8g.x86_64 Thanks, Shenwei
+ end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
On Tue, Oct 27, 2020 at 02:46:24PM +0800, Xiao Shenwei wrote:
+ end
this seems to be an isolate requirement: "append arch" to virtual "testbox"?
if a job need running on x86_64, then it's queue name should use vm-2p8g.x86_64
yes. this is also "isolate requirments": 1. add to vm queue append with arch. 2. consume vm job with host's arch.
Thanks, Shenwei
+ end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
-- Thanks. chief <tongqunfeng@huawei.com>
On Wed, Oct 28, 2020 at 10:34:02AM +0800, Tong Qunfeng wrote:
On Tue, Oct 27, 2020 at 02:46:24PM +0800, Xiao Shenwei wrote:
+ end
this seems to be an isolate requirement: "append arch" to virtual "testbox"?
if a job need running on x86_64, then it's queue name should use vm-2p8g.x86_64
yes.
this is also "isolate requirments": 1. add to vm queue append with arch. 2. consume vm job with host's arch.
mac2queues xx:xx:xx:xx(aarch64 machine) => vm-2p8g.aarch64 xx:xx:xx:xx(x86_64 machine) => vm-2p8g.x86_64 this step will append on qemu.sh for HW, need register by myself Thanks, Shenwei
Thanks, Shenwei
+ end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0
-- Thanks. chief <tongqunfeng@huawei.com>
-- Thanks. chief <tongqunfeng@huawei.com>
participants (4)
-
Liu Shaofei -
Luan Shengde -
Tong Qunfeng -
Xiao Shenwei