- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib: optimize es query_by_id to adapt to change the es mapping
by Cao Xueliang 17 Dec '20

17 Dec '20

When we change es mapping, we can't use es.get_source, because the jobs is a alias for new jobs index, the old data at the old index. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/es_query.rb | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/lib/es_query.rb b/lib/es_query.rb index 68c70b5..3bea624 100644 --- a/lib/es_query.rb +++ b/lib/es_query.rb @@ -54,9 +54,12 @@ class ESQuery end def query_by_id(id) - @client.get_source({ index: @index, type: '_doc', id: id }) - rescue Elasticsearch::Transport::Transport::Errors::NotFound - nil + result = @client.search(index: @index + '*', + body: { query: { bool: { must: { term: { '_id' => id } } } }, + size: 1 })['hits']['hits'] + return nil unless result.size == 1 + + return result[0]['_source'] end end -- 2.23.0

1 0

[PATCH compass-ci] doc: and runtime field in borrow-machine.en.md and borrow-machine.zh.md
by Zhang Yale 17 Dec '20

17 Dec '20

User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- doc/manual/borrow-machine.en.md | 11 ++++++----- doc/manual/borrow-machine.zh.md | 11 ++++++----- 2 files changed, 12 insertions(+), 10 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index b9c2bc4..0277984 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -202,7 +202,8 @@ Ensure that you have performed the following operations according to the [apply- hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -212,10 +213,10 @@ Ensure that you have performed the following operations according to the [apply- %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # Use the VIM editor to change the value of the sleep field. + sleep: + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # Use the VIM editor to change the value of the runtime field. hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # After changing the value, submit the request again. hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 06d4db6..5bc951d 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -206,7 +206,8 @@ hi684@account-vm ~/lkp-tests/jobs% cat borrow-1h.yaml suite: borrow testcase: borrow - + + runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -216,10 +217,10 @@ %> sshd: # sleep at the bottom - sleep: 1h - hi684@account-vm ~/lkp-tests/jobs% grep sleep: borrow-1h.yaml - sleep: 1h - # ��vim��sleep�� + runtime: 1h + hi684@account-vm ~/lkp-tests/jobs% grep runtime: borrow-1h.yaml + runtime: 1h + # ��vim��runtime�� hi684@account-vm ~/lkp-tests/jobs% vim borrow-1h.yaml # ��submit�� hi684@account-vm ~/lkp-tests/jobs% submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends: add sshd depends file
by Wang Yong 17 Dec '20

17 Dec '20

add depends file for sshd Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/sshd | 1 + 1 file changed, 1 insertion(+) create mode 100644 distro/depends/sshd diff --git a/distro/depends/sshd b/distro/depends/sshd new file mode 100644 index 000000000..dd21b5aa9 --- /dev/null +++ b/distro/depends/sshd @@ -0,0 +1 @@ +openssh-server -- 2.23.0

2 2

[PATCH v2 compass-ci] scheduler: submit cluster job to "queue/subqueue"
by Ren Wen 17 Dec '20

17 Dec '20

Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/scheduler/submit_job.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 4303b1d..c322d0f 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -37,6 +37,7 @@ class Sched def submit_cluster_job(job, cluster_config) job_messages = Array(Hash(String, String)).new lab = job.lab + subqueue = job.subqueue # collect all job ids job_ids = [] of String @@ -52,7 +53,7 @@ class Sched # steps for each host cluster_config["nodes"].as_h.each do |host, config| queue = host.to_s - job_id = add_task(queue, lab) + job_id = add_task("#{queue}/#{subqueue}", lab) # return when job_id is '0' # 2 Questions: -- 2.23.0

1 0

[PATCH lkp-tests] jobs: add runtime field in borrow-*.yaml
by Zhang Yale 17 Dec '20

17 Dec '20

[why] 1. We need "$runtime" field to determine "Due time" value in email template received by users. 2. And the "$runtime" field is used in sleep script too. Therefore, sleep can be left blank, and pp.sleep can be parse successfully now. 3. User can specify the sleep time in "runtime" field. Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/borrow-10d.yaml | 3 ++- jobs/borrow-1d.yaml | 3 ++- jobs/borrow-1h.yaml | 3 ++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/jobs/borrow-10d.yaml b/jobs/borrow-10d.yaml index e0e35e17..88a137e0 100644 --- a/jobs/borrow-10d.yaml +++ b/jobs/borrow-10d.yaml @@ -1,6 +1,7 @@ suite: borrow testcase: borrow +runtime: 10d ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -10,4 +11,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 10d +sleep: diff --git a/jobs/borrow-1d.yaml b/jobs/borrow-1d.yaml index c31d7ebd..f79b440b 100644 --- a/jobs/borrow-1d.yaml +++ b/jobs/borrow-1d.yaml @@ -3,6 +3,7 @@ testcase: borrow timeout: 1d +runtime: 1d ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -12,4 +13,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 1d +sleep: diff --git a/jobs/borrow-1h.yaml b/jobs/borrow-1h.yaml index d6b72c43..d13af910 100644 --- a/jobs/borrow-1h.yaml +++ b/jobs/borrow-1h.yaml @@ -1,6 +1,7 @@ suite: borrow testcase: borrow +runtime: 1h ssh_pub_key: <%= begin File.read("#{ENV['HOME']}/.ssh/id_rsa.pub").chomp @@ -10,4 +11,4 @@ ssh_pub_key: <%= %> sshd: # sleep at the bottom -sleep: 1h +sleep: -- 2.23.0

2 2

[PATCH compass-ci] result-webdav/nginx.conf: error 500 at the second request
by Cui Yili 17 Dec '20

17 Dec '20

[why] the table in the function split is used as a variable elsewhere. error: Attempt to call field 'insert' (a nil value) [how] table => dirs Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/result-webdav/nginx.conf | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/container/result-webdav/nginx.conf b/container/result-webdav/nginx.conf index af27653..8b54314 100644 --- a/container/result-webdav/nginx.conf +++ b/container/result-webdav/nginx.conf @@ -61,13 +61,13 @@ http { ngx.exit(ngx.HTTP_FORBIDDEN) end - if string.find(res, ",") then - table = split(res,",") - for i = 1, #table do - if string.find(path, table[i]) then + if string.find(res, ",", 1, true) then + dirs = split(res, ",") + for i = 1, #dirs do + if string.find(path, dirs[i], 1, true) then return end - end + end else if string.find(path, res, 1, true) then return -- 2.23.0

2 1

[PATCH v1 compass-ci] delimiter: optimize delimiter utils function
by Cao Xueliang 17 Dec '20

17 Dec '20

1. Delete redundant bad job fields. 2. Add to make /tmp/lkp/result dir. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index b60ada0..e87bbbe 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -108,8 +108,11 @@ module Utils job.delete('error_ids') job.delete('start_time') + job.delete('job_state') job.delete('end_time') job.delete('loadavg') + job.delete('stats') + job.delete('id') return job end @@ -123,6 +126,7 @@ module Utils end def create_bisect_log(git_dir) + FileUtils.mkdir_p TMP_RESULT_ROOT unless File.exist? TMP_RESULT_ROOT log_file = File.join(TMP_RESULT_ROOT, "bisect.log") log_content = parse_bisect_log(git_dir) File.open(log_file, 'w') do |f| -- 2.23.0

1 0

[PATCH compass-ci] delimiter: optimize delimiter utils function
by Cao Xueliang 17 Dec '20

17 Dec '20

1. Delete redundant bad job fields. 2. Add to make /tmp/lkp/result dir. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/utils.rb | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index b60ada0..e87bbbe 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -108,8 +108,11 @@ module Utils job.delete('error_ids') job.delete('start_time') + job.delete('job_state') job.delete('end_time') job.delete('loadavg') + job.delete('stats') + job.delete('id') return job end @@ -123,6 +126,7 @@ module Utils end def create_bisect_log(git_dir) + FileUtils.mkdir_p TMP_RESULT_ROOT unless File.exist? TMP_RESULT_ROOT log_file = File.join(TMP_RESULT_ROOT, "bisect.log") log_content = parse_bisect_log(git_dir) File.open(log_file, 'w') do |f| -- 2.23.0

1 1

[PATCH v2 lkp-tests] tests: fix error when running host-info in vm
by Zhang Yu 17 Dec '20

17 Dec '20

Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tests/host-info b/tests/host-info index 0ab8e6c97..d1d0df449 100755 --- a/tests/host-info +++ b/tests/host-info @@ -102,11 +102,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX - is_virt && return - is_docker && return + [ -e /dev/ipmi0 ] || [ -e /dev/ipmi/0 ] || [ -e /dev/ipmidev/0 ] && return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip: $ip" - } main() -- 2.23.0

2 2

[PATCH v1 compass-ci] delimiter: error id use inspect to avoid special characters
by Cao Xueliang 17 Dec '20

17 Dec '20

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 2 +- src/delimiter/delimiter.rb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index 97ad467..bc1e035 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -79,7 +79,7 @@ class GitBisect result = `git -C #{@work_dir} bisect start #{@bad_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' - result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} "#{@error_id}" #{@work_dir}` + result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} #{(a)error_id.inspect} #{@work_dir}` end Utils.create_bisect_log(@work_dir) FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) diff --git a/src/delimiter/delimiter.rb b/src/delimiter/delimiter.rb index c1922e2..bf36db3 100644 --- a/src/delimiter/delimiter.rb +++ b/src/delimiter/delimiter.rb @@ -25,7 +25,7 @@ class Delimiter next end - %x(#{LKP_SRC}/sbin/submit bad_job_id=#{task['job_id']} error_id=#{task['error_id']} bisect.yaml) + %x(#{LKP_SRC}/sbin/submit bad_job_id=#{task['job_id']} error_id=#{task['error_id'].inspect} bisect.yaml) rescue StandardError => e puts e sleep(60) -- 2.23.0

1 0