- Compass-ci - mailweb.openeuler.org

[PATCH v7 compass-ci 01/12] send-internet-mail/Dockerfile: add install gem resources
by Luan Shengde 14 Jan '21

14 Jan '21

[why] redis: used to read/store email counts from/to redis queue rufus-scheduler: used to reset the queue as a timing task Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/send-internet-mail/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/send-internet-mail/Dockerfile b/container/send-internet-mail/Dockerfile index d151bda..03e3327 100644 --- a/container/send-internet-mail/Dockerfile +++ b/container/send-internet-mail/Dockerfile @@ -6,4 +6,4 @@ MAINTAINER shdluan(a)163.com ENV DEBIAN_FRONTEND noninteractive RUN apt-get update && \ - apt-get install -y ruby-mail ruby-json ruby-sinatra + apt-get install -y ruby-mail ruby-json ruby-sinatra ruby-redis ruby-rufus-scheduler -- 2.23.0

1 1

Re: [PATCH v2 lkp-tests] jobs: and ansible_test.yaml
by Liu Shaofei 14 Jan '21

14 Jan '21

^---- and => add Reply-To: In-Reply-To: <20210114021509.109179-1-ylzhangah(a)qq.com> On Thu, Jan 14, 2021 at 10:15:09AM +0800, Zhang Yale wrote: >Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> >--- > jobs/ansible_test.yaml | 6 ++++++ > 1 file changed, 6 insertions(+) > create mode 100644 jobs/ansible_test.yaml > >diff --git a/jobs/ansible_test.yaml b/jobs/ansible_test.yaml >new file mode 100644 >index 00000000..49f31562 >--- /dev/null >+++ b/jobs/ansible_test.yaml >@@ -0,0 +1,6 @@ >+suite: ansible_test >+category: functional >+ >+ansible_test: >+ ansible_repo: >+# - geerlingguy/ansible-role-jenkins ^---- whether to remove '#'? Thanks, Liushaofei >-- >2.23.0 >

2 1

[PATCH lkp-tests] stats: add ansible_test stats script
by Zhang Yale 14 Jan '21

14 Jan '21

output: ansible_test.total.nr_ok: 2 ansible_test.total.nr_changed: 0 ansible_test.total.nr_failed: 1 ansible_test.total.nr_skipped: 0 ansible_test.total.nr_rescued: 0 ansible_test.total.nr_ignored: 0 playbook_run_on_fail: 0 Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- stats/ansible_test | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100755 stats/ansible_test diff --git a/stats/ansible_test b/stats/ansible_test new file mode 100755 index 00000000..74ac32f1 --- /dev/null +++ b/stats/ansible_test @@ -0,0 +1,16 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + case line.chomp! + when /\: ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ + puts "ansible_test.total.nr_ok: #{$1}" + puts "ansible_test.total.nr_changed: #{$2}" + puts "ansible_test.total.nr_failed: #{$4}" + puts "ansible_test.total.nr_skipped: #{$5}" + puts "ansible_test.total.nr_rescued: #{$6}" + puts "ansible_test.total.nr_ignored: #{$7}" + when /.*playbook_run_on_fail/ + puts line + end +end + -- 2.23.0

1 1

[PATCH compass-ci] docker2rootfs: add unzip vmlinux function
by Wang Chenglong 14 Jan '21

14 Jan '21

[Why] Sometimes, the kernel is in vmlinux format and needs to be unpacked into vmlinuz format. Signeo-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker2rootfs/common | 57 ++++++++++++++++++++++++++++- container/docker2rootfs/run | 1 + container/docker2rootfs/setup-image | 2 +- 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/container/docker2rootfs/common b/container/docker2rootfs/common index e908079..66aba62 100755 --- a/container/docker2rootfs/common +++ b/container/docker2rootfs/common @@ -133,7 +133,7 @@ get_rootfs_kernel() local vmlinuz_file local vmlinuz kernel - vmlinuz_file=$(find ./boot -name "vmlinuz-*" | grep -v rescue) && export ROOTFS_VMLINUZ_FILE=$vmlinuz_file + vmlinuz_file=$(find ./boot -name "vmlinu[z|x]-*" | grep -v rescue) && export ROOTFS_VMLINUZ_FILE=$vmlinuz_file vmlinuz=$(basename "$vmlinuz_file") && export ROOTFS_VMLINUZ=$vmlinuz kernel=${vmlinuz:8} && export ROOTFS_KERNEL=$kernel @@ -155,3 +155,58 @@ create_links_vmlinuz_initrd() echo "[INFO] Create links to initrd.lkp and vmlinuz success!" } + +get_vmlinuz_start_postion() { + local vmlinuz=$1 + + local vmlinuz_start_line + local vmlinuz_start_postion + # '1f 8b 08' - https://www.filesignatures.net/index.php?page=search&search=1F8B08&mode=SIG + vmlinuz_start_line=$(od -A d -t x1 ${vmlinuz} | grep "1f 8b 08") + vmlinuz_start_postion=${vmlinuz_start_line%% *} + + [ -z ${vmlinuz_start_postion} ] && { + echo "[WARNING] identify vmlinuz failed." + export VMLINUZ_START_POSTION=-1 + return + } + + [ X${vmlinuz_start_postion} == X0000000 ] || { + local vmlinuz_start_postion_tmp + vmlinuz_start_postion_tmp=$(echo ${vmlinuz_start_line} | awk '{for(i=1;i<=NF;i++)if($i=="1f")print i-1}') + vmlinuz_start_postion=$((${vmlinuz_start_postion} + ${vmlinuz_start_postion_tmp} -1)) + } + + export VMLINUZ_START_POSTION=${vmlinuz_start_postion} +} + +unzip_vmlinuz() { + get_rootfs_kernel + + file ${ROOTFS_VMLINUZ_FILE} | grep -q gzip || return + + get_vmlinuz_start_postion ${ROOTFS_VMLINUZ_FILE} + [ $VMLINUZ_START_POSTION -eq -1 ] && return + + local unzip_str + + if [ $VMLINUZ_START_POSTION -eq 0 ]; then + unzip_str="dd if=$ROOTFS_VMLINUZ bs=1 | zcat > $ROOTFS_VMLINUZ.tmp" + else + unzip_str="dd if=$ROOTFS_VMLINUZ bs=1 skip=$VMLINUZ_START_POSTION | zcat > $ROOTFS_VMLINUZ.tmp" + fi + + echo "Unzipping vmlinuz..." + local cmds + cmds=( + cd $ROOTFS_DIR/boot + $unzip_str + mv $ROOTFS_VMLINUZ.tmp $ROOTFS_VMLINUZ + chmod o+r $ROOTFS_VMLINUZ + ) + if "${cmds[@]}" ; then + echo "[INFO] Unzipping vmlinuz success!" + else + echo "[WARNING] Unzipping vmlinuz failed!" + fi +} diff --git a/container/docker2rootfs/run b/container/docker2rootfs/run index 8a7b315..cdb19bf 100755 --- a/container/docker2rootfs/run +++ b/container/docker2rootfs/run @@ -31,3 +31,4 @@ start=( cp_rootfs $IMAGE_PACK $2 docker rm -f rootfs-docker create_links_vmlinuz_initrd $2 +unzip_vmlinuz diff --git a/container/docker2rootfs/setup-image b/container/docker2rootfs/setup-image index ffe4937..47bfb7e 100755 --- a/container/docker2rootfs/setup-image +++ b/container/docker2rootfs/setup-image @@ -38,7 +38,7 @@ setup_login() pack_cgz() { echo "Packing package. Please wait." - find ./ ! -path "./tmp/${IMAGE_PACK}" | cpio -o -Hnewc | gzip -9 >./tmp/$IMAGE_PACK + find ./ ! -path "./tmp/$IMAGE_PACK" | cpio -o -Hnewc | gzip -9 > "./tmp/$IMAGE_PACK" chmod 644 /tmp/${IMAGE_PACK} } -- 2.23.0

3 4

[PATCH v2 compass-ci] docker2rootfs: add unzip vmlinux function
by Wang Chenglong 14 Jan '21

14 Jan '21

[Why] Sometimes, the kernel is in vmlinux format and needs to be unpacked into vmlinuz format. Signeo-off-by: Wang Chenglong <18509160991(a)163.com> --- container/docker2rootfs/common | 57 ++++++++++++++++++++++++++++- container/docker2rootfs/run | 1 + container/docker2rootfs/setup-image | 2 +- 3 files changed, 58 insertions(+), 2 deletions(-) diff --git a/container/docker2rootfs/common b/container/docker2rootfs/common index c26bbc8..91d53e7 100755 --- a/container/docker2rootfs/common +++ b/container/docker2rootfs/common @@ -133,7 +133,7 @@ get_rootfs_kernel() local vmlinuz_file local vmlinuz kernel - vmlinuz_file=$(find ./boot -name "vmlinuz-*" | grep -v rescue) && export ROOTFS_VMLINUZ_FILE=$vmlinuz_file + vmlinuz_file=$(find ./boot -name "vmlinu[z|x]-*" | grep -v rescue) && export ROOTFS_VMLINUZ_FILE=$vmlinuz_file vmlinuz=$(basename "$vmlinuz_file") && export ROOTFS_VMLINUZ=$vmlinuz kernel=${vmlinuz:8} && export ROOTFS_KERNEL=$kernel @@ -155,3 +155,58 @@ create_links_vmlinuz_initrd() echo "[INFO] Create links to initrd.lkp and vmlinuz success!" } + +get_vmlinuz_start_postion() { + local vmlinuz=$1 + + local vmlinuz_start_line + local vmlinuz_start_postion + # '1f 8b 08' - https://www.filesignatures.net/index.php?page=search&search=1F8B08&mode=SIG + vmlinuz_start_line=$(od -A d -t x1 ${vmlinuz} | grep "1f 8b 08") + vmlinuz_start_postion=${vmlinuz_start_line%% *} + + [ -z ${vmlinuz_start_postion} ] && { + echo "[WARNING] identify vmlinuz failed." + export VMLINUZ_START_POSTION=-1 + return + } + + [ X${vmlinuz_start_postion} == X0000000 ] || { + local vmlinuz_start_postion_tmp + vmlinuz_start_postion_tmp=$(echo ${vmlinuz_start_line} | awk '{for(i=1;i<=NF;i++)if($i=="1f")print i-1}') + vmlinuz_start_postion=$((${vmlinuz_start_postion} + ${vmlinuz_start_postion_tmp} -1)) + } + + export VMLINUZ_START_POSTION=${vmlinuz_start_postion} +} + +unzip_vmlinuz() { + get_rootfs_kernel + + file ${ROOTFS_VMLINUZ_FILE} | grep -q gzip || return + + get_vmlinuz_start_postion ${ROOTFS_VMLINUZ_FILE} + [ $VMLINUZ_START_POSTION -eq -1 ] && return + + local unzip_str + + if [ $VMLINUZ_START_POSTION -eq 0 ]; then + unzip_str="dd if=$ROOTFS_VMLINUZ bs=1 | zcat > $ROOTFS_VMLINUZ.tmp" + else + unzip_str="dd if=$ROOTFS_VMLINUZ bs=1 skip=$VMLINUZ_START_POSTION | zcat > $ROOTFS_VMLINUZ.tmp" + fi + + echo "Unzipping vmlinuz..." + local cmds + cmds=( + cd $ROOTFS_DIR/boot + $unzip_str + mv $ROOTFS_VMLINUZ.tmp $ROOTFS_VMLINUZ + chmod o+r $ROOTFS_VMLINUZ + ) + if "${cmds[@]}" ; then + echo "[INFO] Unzipping vmlinuz success!" + else + echo "[WARNING] Unzipping vmlinuz failed!" + fi +} diff --git a/container/docker2rootfs/run b/container/docker2rootfs/run index 8a7b315..cdb19bf 100755 --- a/container/docker2rootfs/run +++ b/container/docker2rootfs/run @@ -31,3 +31,4 @@ start=( cp_rootfs $IMAGE_PACK $2 docker rm -f rootfs-docker create_links_vmlinuz_initrd $2 +unzip_vmlinuz diff --git a/container/docker2rootfs/setup-image b/container/docker2rootfs/setup-image index ffe4937..47bfb7e 100755 --- a/container/docker2rootfs/setup-image +++ b/container/docker2rootfs/setup-image @@ -38,7 +38,7 @@ setup_login() pack_cgz() { echo "Packing package. Please wait." - find ./ ! -path "./tmp/${IMAGE_PACK}" | cpio -o -Hnewc | gzip -9 >./tmp/$IMAGE_PACK + find ./ ! -path "./tmp/$IMAGE_PACK" | cpio -o -Hnewc | gzip -9 > "./tmp/$IMAGE_PACK" chmod 644 /tmp/${IMAGE_PACK} } -- 2.23.0

2 1

[PATCH lkp-tests] stats: add stats script for openeuler_docker test
by Bai Jing 14 Jan '21

14 Jan '21

[input] ^[[0mwordpress_build_on_fail: 1 [output] wordpress_build_on_fail: 1 Signed-off-by: Bai Jing <799286817(a)qq.com> --- stats/openeuler_docker | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100755 stats/openeuler_docker diff --git a/stats/openeuler_docker b/stats/openeuler_docker new file mode 100755 index 000000000..5ede8a240 --- /dev/null +++ b/stats/openeuler_docker @@ -0,0 +1,8 @@ +#!/usr/bin/env ruby + +while (line = STDIN.gets) + line.chomp! + next unless line =~ /.*_build_on_fail/ + + puts line +end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] delimiter: use assist-result container client to get job yaml
by Cao Xueliang 14 Jan '21

14 Jan '21

Before: Get the job yaml from es, we need deal some process products of job. After: Get the job yaml from assist-result container without the process products. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/constants.rb | 2 +- src/delimiter/utils.rb | 24 +++++++----------------- 2 files changed, 8 insertions(+), 18 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index ebf90cc..7b14558 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -6,7 +6,7 @@ LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' -DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'compass-ci(a)qq.com' +DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'caoxl78320(a)163.com' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index de5a302..61fe6ab 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -9,6 +9,7 @@ require 'fileutils' require_relative './constants' require_relative '../../lib/sched_client' +require_relative '../../lib/assist_result_client' require_relative '../../lib/compare_error_messages' require_relative "#{ENV['LKP_SRC']}/lib/monitor" @@ -121,9 +122,10 @@ module Utils end def init_job_content(job_id) - es = ESQuery.new - job = es.query_by_id(job_id) - raise "es query job id: #{job_id} failed!" unless job + job_yaml = AssistResult.new.get_job_yaml(job_id) + raise "get job yaml failed, job id: #{job_id} !" unless job_yaml + + job = JSON.parse job_yaml account_info = get_account_info raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info @@ -135,21 +137,9 @@ module Utils job['my_email'] = account_info['my_email'] job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id - job['testbox'] = job['config'] ? 'dc-16g' : 'dc-8g' - job['os'] = 'openeuler' - job['os_arch'] = 'aarch64' - job['os_mount'] = 'container' - job['os_version'] = '20.03-pre' - job['docker_image'] = 'openeuler:20.03-pre' - - job.delete('start_time') - job.delete('tbox_group') - job.delete('error_ids') - job.delete('job_state') - job.delete('end_time') - job.delete('loadavg') + + job.delete('subqueue') job.delete('queue') - job.delete('stats') job.delete('id') return job -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib: add assist_result_client for assist-result container
by Cao Xueliang 14 Jan '21

14 Jan '21

[why] Add client for assist-result container, others can call the API by the client. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/assist_result_client.rb | 22 ++++++++++++++++++++++ lib/constants.rb | 3 +++ 2 files changed, 25 insertions(+) create mode 100644 lib/assist_result_client.rb diff --git a/lib/assist_result_client.rb b/lib/assist_result_client.rb new file mode 100644 index 0000000..042b189 --- /dev/null +++ b/lib/assist_result_client.rb @@ -0,0 +1,22 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'rest-client' +require_relative 'constants' + +class AssistResult + def initialize(host = ASSIST_RESULT_HOST, port = ASSIST_RESULT_PORT) + @host = host + @port = port + end + + def get_job_yaml(job_id) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/get_job_yaml/#{job_id}") + response = resource.get() + return nil unless response.code == 200 + + return response.body + end +end diff --git a/lib/constants.rb b/lib/constants.rb index a274adc..ffc4f03 100644 --- a/lib/constants.rb +++ b/lib/constants.rb @@ -15,3 +15,6 @@ SRV_HTTP_HOST = config['SRV_HTTP_HOST'] || ENV['SRV_HTTP_HOST'] || '172.17.0.1' SRV_HTTP_PORT = config['SRV_HTTP_PORT'] || ENV['SRV_HTTP_PORT'] || 11300 SRV_HTTP_DOMAIN = config['SRV_HTTP_DOMAIN'] || ENV['SRV_HTTP_DOMAIN'] || 'api.compass-ci.openeuler.org' + +ASSIST_RESULT_HOST = config['ASSIST_RESULT_HOST'] || ENV['ASSIST_RESULT_HOST'] || '172.17.0.1' +ASSIST_RESULT_PORT = config['ASSIST_RESULT_PORT'] || ENV['ASSIST_RESULT_PORT'] || 8102 -- 2.23.0

1 0

[PATCH v2 lkp-tests] jobs: and ansible_test.yaml
by Zhang Yale 14 Jan '21

14 Jan '21

Signed-off-by: Zhang Yale <ylzhangah(a)qq.com> --- jobs/ansible_test.yaml | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 jobs/ansible_test.yaml diff --git a/jobs/ansible_test.yaml b/jobs/ansible_test.yaml new file mode 100644 index 00000000..49f31562 --- /dev/null +++ b/jobs/ansible_test.yaml @@ -0,0 +1,6 @@ +suite: ansible_test +category: functional + +ansible_test: + ansible_repo: +# - geerlingguy/ansible-role-jenkins -- 2.23.0

1 0

[PATCH v1 compass-ci 2/2] delimiter: use assist-result container client to get job yaml
by Cao Xueliang 14 Jan '21

14 Jan '21

Before: Get the job yaml from es, we need deal some process products of job. After: Get the job yaml from assist-result container without the process products. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/constants.rb | 2 +- src/delimiter/utils.rb | 24 +++++++----------------- 2 files changed, 8 insertions(+), 18 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index ebf90cc..7b14558 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -6,7 +6,7 @@ LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' -DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'compass-ci(a)qq.com' +DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'caoxl78320(a)163.com' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index de5a302..61fe6ab 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -9,6 +9,7 @@ require 'fileutils' require_relative './constants' require_relative '../../lib/sched_client' +require_relative '../../lib/assist_result_client' require_relative '../../lib/compare_error_messages' require_relative "#{ENV['LKP_SRC']}/lib/monitor" @@ -121,9 +122,10 @@ module Utils end def init_job_content(job_id) - es = ESQuery.new - job = es.query_by_id(job_id) - raise "es query job id: #{job_id} failed!" unless job + job_yaml = AssistResult.new.get_job_yaml(job_id) + raise "get job yaml failed, job id: #{job_id} !" unless job_yaml + + job = JSON.parse job_yaml account_info = get_account_info raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info @@ -135,21 +137,9 @@ module Utils job['my_email'] = account_info['my_email'] job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id - job['testbox'] = job['config'] ? 'dc-16g' : 'dc-8g' - job['os'] = 'openeuler' - job['os_arch'] = 'aarch64' - job['os_mount'] = 'container' - job['os_version'] = '20.03-pre' - job['docker_image'] = 'openeuler:20.03-pre' - - job.delete('start_time') - job.delete('tbox_group') - job.delete('error_ids') - job.delete('job_state') - job.delete('end_time') - job.delete('loadavg') + + job.delete('subqueue') job.delete('queue') - job.delete('stats') job.delete('id') return job -- 2.23.0

1 0

[PATCH v1 compass-ci 1/2] lib: add assist_result_client for assist_result container
by Cao Xueliang 14 Jan '21

14 Jan '21

[why] Add client for assist_result container, others can call the assist_result container API by the client. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/assist_result_client.rb | 22 ++++++++++++++++++++++ lib/constants.rb | 3 +++ 2 files changed, 25 insertions(+) create mode 100644 lib/assist_result_client.rb diff --git a/lib/assist_result_client.rb b/lib/assist_result_client.rb new file mode 100644 index 0000000..042b189 --- /dev/null +++ b/lib/assist_result_client.rb @@ -0,0 +1,22 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'rest-client' +require_relative 'constants' + +class AssistResult + def initialize(host = ASSIST_RESULT_HOST, port = ASSIST_RESULT_PORT) + @host = host + @port = port + end + + def get_job_yaml(job_id) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/get_job_yaml/#{job_id}") + response = resource.get() + return nil unless response.code == 200 + + return response.body + end +end diff --git a/lib/constants.rb b/lib/constants.rb index a274adc..ffc4f03 100644 --- a/lib/constants.rb +++ b/lib/constants.rb @@ -15,3 +15,6 @@ SRV_HTTP_HOST = config['SRV_HTTP_HOST'] || ENV['SRV_HTTP_HOST'] || '172.17.0.1' SRV_HTTP_PORT = config['SRV_HTTP_PORT'] || ENV['SRV_HTTP_PORT'] || 11300 SRV_HTTP_DOMAIN = config['SRV_HTTP_DOMAIN'] || ENV['SRV_HTTP_DOMAIN'] || 'api.compass-ci.openeuler.org' + +ASSIST_RESULT_HOST = config['ASSIST_RESULT_HOST'] || ENV['ASSIST_RESULT_HOST'] || '172.17.0.1' +ASSIST_RESULT_PORT = config['ASSIST_RESULT_PORT'] || ENV['ASSIST_RESULT_PORT'] || 8102 -- 2.23.0

1 0

[PATCH v3 compass-ci] tool/es-jobs: a new tool for summarizing jobs information
by Wu Zhende 14 Jan '21

14 Jan '21

Use es-jobs to summarizes information about a group of related jobs. [Example] es-jobs submit_id=xxx -f 'job_state,dmesg.timestamp:last' -r '-1' -s 'on_fail' means: 1. query the job whose submit_id is xxx 2. I want to get job_state,dmesg.timestamp:last 3. don't refine jobs 4. -s 'on_fail' to get data in stats which contain 'on_fail' [Output] { "job_state": [ "crystal.605716.failed" ], "dmesg.timestamp:last": [ 657.181408 ], "stats_filter_result": { "crystal.605716.openeuler_docker.\u001b[0mwordpress_build_on_fail": 1 }, "stats.count": { "stats.has_error": 1, "stats.has_error_jobs": [ "crystal.605716" ] } } Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- lib/es_jobs.rb | 139 +++++++++++++++++++++++++++++++++++++++++++++++++ sbin/es-jobs | 58 +++++++++++++++++++++ 2 files changed, 197 insertions(+) create mode 100644 lib/es_jobs.rb create mode 100755 sbin/es-jobs diff --git a/lib/es_jobs.rb b/lib/es_jobs.rb new file mode 100644 index 0000000..a90b1ba --- /dev/null +++ b/lib/es_jobs.rb @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: GPL-2.0-only +# frozen_string_literal: true + +LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' + +require "#{LKP_SRC}/lib/stats" +require_relative './es_query' + +# deal jobs search from es +class ESJobs + def initialize(es_query, my_refine = [], fields = [], stats_filter = []) + @es_query = es_query + @es = ESQuery.new(ES_HOST, ES_PORT) + @refine = my_refine + @fields = fields + @stats_filter = stats_filter + @stats_filter_result = {} + @refine_jobs = [] + @jobs = {} + @stats_level = { + 0 => 'stats.success', + 1 => 'stats.unknown', + 2 => 'stats.warning', + 3 => 'stats.has_error' + } + set_defaults + deal_jobs + end + + def set_defaults + query_result = @es.multi_field_query(@es_query) + query_result['hits']['hits'].each do |job| + @jobs[job['_id']] = job['_source'] + end + + @stats = { + 'stats.count' => Hash.new(0), + 'stats.sum' => Hash.new(0), + 'stats.avg' => Hash.new(0) + } + @result = {} + @fields.each do |field| + @result[field] = [] + end + end + + def add_result_fields(job, level) + return unless @refine.include?(level) || @refine.include?(-1) + + @refine_jobs << job['id'] + @fields.each do |field| + value = job[field] + if value + value = job['id'] + '.' + value if field == 'job_state' + @result[field] << value + end + + next unless job['stats'] + + @result[field] << job['stats'][field] if job['stats'][field] + end + end + + def deal_jobs + stats_count = Hash.new(0) + stats_jobs = {} + + @jobs.each do |job_id, job| + level = deal_stats(job) + add_result_fields(job, level) + + stat_key = @stats_level[level] + stat_jobs_key = stat_key + '_jobs' + + stats_count[stat_key] += 1 + stats_jobs[stat_jobs_key] ||= [] + stats_jobs[stat_jobs_key] << job_id + end + + @stats['stats.count'].merge!(stats_count) + @stats['stats.count'].merge!(stats_jobs) + end + + def deal_stats(job, level = 0) + return 1 unless job['stats'] + + job['stats'].each do |key, value| + match_stats_filter(key, value, job['id']) + calculate_stat(key, value) + level = get_stat_level(key, level) + end + return level + end + + def match_stats_filter(key, value, job_id) + @stats_filter.each do |filter| + next unless key.include?(filter) + + key = job_id + '.' + key + @stats_filter_result[key] = value + + break + end + end + + def calculate_stat(key, value) + if function_stat?(key) + return unless @fields.include?('stats.sum') + + @stats['stats.sum'][key] += value + else + return unless @fields.include?('stats.avg') + + @stats['stats.avg'][key] = (@stats['stats.avg'][key] + value) / 2 + end + end + + def get_stat_level(stat, level) + return level if level >= 3 + return 3 if stat.match(/error|fail/i) + return 2 if stat.match(/warn/i) + + return 0 + end + + def output + result = { + 'stats.count' => @stats['stats.count'] + } + + @stats.each do |key, value| + result[key] = value if @fields.include?(key) + end + + @result['stats_filter_result'] = @stats_filter_result unless @stats_filter.empty? + @result.merge!(result) + puts JSON.pretty_generate(@result) + end +end diff --git a/sbin/es-jobs b/sbin/es-jobs new file mode 100755 index 0000000..25cbe9c --- /dev/null +++ b/sbin/es-jobs @@ -0,0 +1,58 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'optparse' +require 'json' + +require_relative '../lib/es_jobs' + +def parse_argv + items = {} + ARGV.each do |item| + key, value = item.split('=') + if key && value + value_list = value.split(',') + items[key] = value_list.length > 1 ? value_list : value + end + end + items +end +opt_refine = [-1] +opt_fields = [] +opt_stats_filter = [] +opt_parser = OptionParser.new do |opts| + opts.banner = 'Usage: es-jobs [options] search_key1=val1[,val2..] ..' + + opts.separator 'search_key can be id, suite, os, etc.' + opts.separator 'how to use -r' + opts.separator 'like es-jobs submit_id=xxx -r "-1,0,1,2,3"' + opts.separator '-1 means not refine' + opts.separator '{ + -1 => null, + 0 => stats.succes, + 1 => stats.unknow, + 2 => stats.warning, + 3 => stats.has_error +}' + + opts.on('-r fields', '--refine fields', 'refine jobs') do |fields| + opt_refine = fields.split(',').map(&:to_i) + end + + opts.on('-f fields', '--fields fields', 'fields you want to see') do |fields| + opt_fields = fields.split(',') + end + + opts.on('-s fields', '--stats-filter fields', 'return data contains fields') do |fields| + opt_stats_filter = fields.split(',') + end +end + +opt_parser.parse!(ARGV) +items = parse_argv +raise 'Please enter a query' if items.empty? + +es_jobs = ESJobs.new(items, opt_refine, opt_fields, opt_stats_filter) +es_jobs.output -- 2.23.0

2 1

[PATCH v2 lkp-tests 2/2] modify reset-tbox's yaml and script
by Zhang Dewan 14 Jan '21

14 Jan '21

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- jobs/reset-tbox.yaml | 3 +-- tests/reset-tbox | 7 ++++--- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/jobs/reset-tbox.yaml b/jobs/reset-tbox.yaml index 9c3f4f01..2ceccdb9 100644 --- a/jobs/reset-tbox.yaml +++ b/jobs/reset-tbox.yaml @@ -1,10 +1,9 @@ suite: reset-tbox -testcase: reset-tbox testbox: dc-8g os_mount: container docker_image: 172.168.131.2:5001/reset-tbox -queue: reset +queue: reset-tbox host_name: reset-tbox: diff --git a/tests/reset-tbox b/tests/reset-tbox index b0f645d7..5b1279b6 100755 --- a/tests/reset-tbox +++ b/tests/reset-tbox @@ -7,11 +7,12 @@ hosts_file=/etc/lab/hosts_file # require $host_name host_ibmc_ip=$(grep -w $host_name $hosts_file | awk '{print $1}') -if [[ "$host_ibmc_ip" =~ ^[0-9]+.* ]]; then +if [[ "$host_ibmc_ip" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then echo ">>> exec ipmitool commamd!" ipmitool -I lanplus -H $host_ibmc_ip -U $iBMC_user -P $iBMC_passwd power reset - [ $? -eq 0 ] && echo "succeed!" + [ $? -eq 0 ] && echo "reboot $host_name succeed!" || \ + "echo reboot $host_name failed!" else - echo "specify host cannot found0!" + echo "specify host cannot found!" exit fi -- 2.23.0

2 2

[PATCH compass-ci 2/2] delimiter: use assist-result container client to get job yaml
by Cao Xueliang 14 Jan '21

14 Jan '21

Before: Get the job yaml from es, we need deal some process products of job. After: Get the job yaml from assist-result container without the process products. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/delimiter/constants.rb | 2 +- src/delimiter/utils.rb | 24 +++++++----------------- 2 files changed, 8 insertions(+), 18 deletions(-) diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index ebf90cc..7b14558 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -6,7 +6,7 @@ LKP_SRC = ENV['LKP_SRC'] || '/c/lkp-tests' GIT_MIRROR_HOST = ENV['GIT_MIRROR_HOST'] || '172.17.0.1' MONITOR_HOST = ENV['MONITOR_HOST'] || '172.17.0.1' MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' -DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'compass-ci(a)qq.com' +DELIMITER_ACCONUT = ENV['DELIMITER_ACCONUT'] || 'caoxl78320(a)163.com' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" diff --git a/src/delimiter/utils.rb b/src/delimiter/utils.rb index de5a302..61fe6ab 100644 --- a/src/delimiter/utils.rb +++ b/src/delimiter/utils.rb @@ -9,6 +9,7 @@ require 'fileutils' require_relative './constants' require_relative '../../lib/sched_client' +require_relative '../../lib/assist_result_client' require_relative '../../lib/compare_error_messages' require_relative "#{ENV['LKP_SRC']}/lib/monitor" @@ -121,9 +122,10 @@ module Utils end def init_job_content(job_id) - es = ESQuery.new - job = es.query_by_id(job_id) - raise "es query job id: #{job_id} failed!" unless job + job_yaml = AssistResult.new.get_job_yaml(job_id) + raise "get job yaml failed, job id: #{job_id} !" unless job_yaml + + job = JSON.parse job_yaml account_info = get_account_info raise "query #{DELIMITER_ACCONUT} account info failed!" unless account_info @@ -135,21 +137,9 @@ module Utils job['my_email'] = account_info['my_email'] job['my_token'] = account_info['my_token'] job['bad_job_id'] = job_id - job['testbox'] = job['config'] ? 'dc-16g' : 'dc-8g' - job['os'] = 'openeuler' - job['os_arch'] = 'aarch64' - job['os_mount'] = 'container' - job['os_version'] = '20.03-pre' - job['docker_image'] = 'openeuler:20.03-pre' - - job.delete('start_time') - job.delete('tbox_group') - job.delete('error_ids') - job.delete('job_state') - job.delete('end_time') - job.delete('loadavg') + + job.delete('subqueue') job.delete('queue') - job.delete('stats') job.delete('id') return job -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib: add assist_result_client for assist_result container
by Cao Xueliang 14 Jan '21

14 Jan '21

[why] Add client for assist_result container, others can call the assist_result container API by the client. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/assist_result_client.rb | 21 +++++++++++++++++++++ lib/constants.rb | 3 +++ 2 files changed, 24 insertions(+) create mode 100644 lib/assist_result_client.rb diff --git a/lib/assist_result_client.rb b/lib/assist_result_client.rb new file mode 100644 index 0000000..980486d --- /dev/null +++ b/lib/assist_result_client.rb @@ -0,0 +1,21 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2021 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true +require 'rest-client' +require_relative 'constants' + +class AssistResult + def initialize(host = ASSIST_RESULT_HOST, port = ASSIST_RESULT_PORT) + @host = host + @port = port + end + + def get_job_yaml(job_id) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/get_job_yaml/#{job_id}") + response = resource.get() + return nil unless response.code == 200 + + return response.body + end +end diff --git a/lib/constants.rb b/lib/constants.rb index a274adc..ffc4f03 100644 --- a/lib/constants.rb +++ b/lib/constants.rb @@ -15,3 +15,6 @@ SRV_HTTP_HOST = config['SRV_HTTP_HOST'] || ENV['SRV_HTTP_HOST'] || '172.17.0.1' SRV_HTTP_PORT = config['SRV_HTTP_PORT'] || ENV['SRV_HTTP_PORT'] || 11300 SRV_HTTP_DOMAIN = config['SRV_HTTP_DOMAIN'] || ENV['SRV_HTTP_DOMAIN'] || 'api.compass-ci.openeuler.org' + +ASSIST_RESULT_HOST = config['ASSIST_RESULT_HOST'] || ENV['ASSIST_RESULT_HOST'] || '172.17.0.1' +ASSIST_RESULT_PORT = config['ASSIST_RESULT_PORT'] || ENV['ASSIST_RESULT_PORT'] || 8102 -- 2.23.0

1 0

Re: [PATCH v2 lkp-tests 2/2] lib: fix 409 conflict
by Wu Fengguang 14 Jan '21

14 Jan '21

On Wed, Jan 13, 2021 at 09:19:47PM +0800, Li Ping wrote: >[why] >find a -type d >a >a/b >a/c >find a -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; >==> curl: (22) The requested URL returned error: 409 Conflict >reason: upload an empty folder causes this error. I suspect 409 happens because server side already have that dir. It's like you run mkdir a => success mkdir a => 409 But will the server side auto create new subdirs if they do not already exist? Probably the below find will fail. Please experiment with deeper dirs, like a/b/c/file Please be very careful and make sure you understand all the error messages, all situations and the root cause before going ahead changing something. >[how] >find a -type f -exec curl -sSf -X MKCOL "$http_url/{}" \; >a/b >a/c >All the files would be found, there is no need to upload empty folder. Thanks, Fengguang

1 0

Re: [PATCH v2 lkp-tests 1/2] tests: modify api-avx2neon
by Bai Jing 14 Jan '21

14 Jan '21

On Wed, Jan 13, 2021 at 09:19:46PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > distro/depends/api-avx2neon | 4 ++++ > tests/api-avx2neon | 33 ++++++++++++++++++++++----------- > 2 files changed, 26 insertions(+), 11 deletions(-) > create mode 100644 distro/depends/api-avx2neon > >diff --git a/distro/depends/api-avx2neon b/distro/depends/api-avx2neon >new file mode 100644 >index 000000000..d82d2f65e >--- /dev/null >+++ b/distro/depends/api-avx2neon >@@ -0,0 +1,4 @@ >+git >+make >+gcc >+g++ >diff --git a/tests/api-avx2neon b/tests/api-avx2neon >index 9a64f89bf..47421021c 100755 >--- a/tests/api-avx2neon >+++ b/tests/api-avx2neon >@@ -1,18 +1,29 @@ > #!/bin/bash > >+SCRIPT_DIR=$(dirname $(realpath $0)) >+cd "$SCRIPT_DIR" >+: "${git_url:=https://github.com/Siguyi/AvxToNeon}" > >-script=$(dirname $(realpath $0)) >-cd $script >+build() { >+ git clone "$git_url" >+ cd AvxToNeon/tests || exit > >-yum install -y git >-yum install -y make >+ if [ "${arch}" == "aarch64" ]; then >+ make >/dev/null 2>&1 >+ elif [ "${arch}" == "x86_64" ]; then >+ make ARCH=x86 >/dev/null 2>&1 >+ fi >+} > >-git clone https://github.com/Siguyi/AvxToNeon >+run() { >+ build >+ echo "api-avx2neon running" >+ ./test >+ echo "api-avx2neon finished" > >-cd AvxToNeon >-cd tests >-make >+ output=$(find / -name "output") >+ fail=$(grep 'AVX2NEONTest Complete' $output |awk '{print $8}') ^^ need space Thanks, Baijing >+ [ 0 -eq "$fail" ] || exit 1 >+} > >-echo "api-avx2neon running" >-./test >-echo "api-avx2neon finished" >+run >-- >2.23.0 >

1 0

Re: [PATCH v2 lkp-tests 1/2] tests: modify api-avx2neon
by Wang Yong 14 Jan '21

14 Jan '21

On Wed, Jan 13, 2021 at 09:19:46PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >+ if [ "${arch}" == "aarch64" ]; then >+ make >/dev/null 2>&1 >+ elif [ "${arch}" == "x86_64" ]; then >+ make ARCH=x86 >/dev/null 2>&1 >+ fi avx2neon use cross compile to build x86 version, not only require arch=x86_64 ref: Makefile Thanks, Yong

1 0

Re: [PATCH v2 lkp-tests 1/2] tests: modify api-avx2neon
by Wu Fengguang 14 Jan '21

14 Jan '21

On Wed, Jan 13, 2021 at 09:19:46PM +0800, Li Ping wrote: >Signed-off-by: Li Ping <1477412247(a)qq.com> >--- > distro/depends/api-avx2neon | 4 ++++ > tests/api-avx2neon | 33 ++++++++++++++++++++++----------- > 2 files changed, 26 insertions(+), 11 deletions(-) > create mode 100644 distro/depends/api-avx2neon > >diff --git a/distro/depends/api-avx2neon b/distro/depends/api-avx2neon >new file mode 100644 >index 000000000..d82d2f65e >--- /dev/null >+++ b/distro/depends/api-avx2neon >@@ -0,0 +1,4 @@ >+git >+make >+gcc >+g++ >diff --git a/tests/api-avx2neon b/tests/api-avx2neon >index 9a64f89bf..47421021c 100755 >--- a/tests/api-avx2neon >+++ b/tests/api-avx2neon >@@ -1,18 +1,29 @@ > #!/bin/bash > >+SCRIPT_DIR=$(dirname $(realpath $0)) >+cd "$SCRIPT_DIR" >+: "${git_url:=https://github.com/Siguyi/AvxToNeon}" > >-script=$(dirname $(realpath $0)) >-cd $script >+build() { >+ git clone "$git_url" Better to add "|| exit" here too. >+ cd AvxToNeon/tests || exit > >-yum install -y git >-yum install -y make >+ if [ "${arch}" == "aarch64" ]; then >+ make >/dev/null 2>&1 Why quiet all output and stderr? >+ elif [ "${arch}" == "x86_64" ]; then >+ make ARCH=x86 >/dev/null 2>&1 >+ fi >+} Thanks, Fengguang >-git clone https://github.com/Siguyi/AvxToNeon >+run() { >+ build >+ echo "api-avx2neon running" >+ ./test What if ./test exit 1? >+ echo "api-avx2neon finished" > >-cd AvxToNeon >-cd tests >-make >+ output=$(find / -name "output") output= => output_file= What if find returns nothing? >+ fail=$(grep 'AVX2NEONTest Complete' $output |awk '{print $8}') Please give example line that you want to grep/awk. >+ [ 0 -eq "$fail" ] || exit 1 That logic reads awkward. fail => complete? >+} > >-echo "api-avx2neon running" >-./test >-echo "api-avx2neon finished" >+run >-- >2.23.0 >

1 0

[PATCH v3 lkp-tests] tests/*: add makepkg -s operation in archlinux
by Wang Yong 13 Jan '21

13 Jan '21

use makepkg -s to auto install depends in archlinux ref: https://git.archlinux.org/pacman.git/plain/scripts/makepkg.sh.in Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- tests/build-pkg | 7 ++++++- tests/cci-makepkg | 12 +++++++++--- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 31762384d..554cd56e5 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -92,7 +92,9 @@ create_cgz_name() cgz_path="$PKG_MNT/${pack_to}/${pkgname}" cgz_name="${cgz_path}/${upstream_commit}.cgz" + pkg_args="-A --check --skippgpcheck" + [ "${os}" == "archlinux" ] && pkg_args="${pkg_args} -s --needed --noconfirm" [ -n "$git_tag" ] && { cgz_name="${cgz_path}/${git_tag}-${pkgrel}.cgz" @@ -153,7 +155,10 @@ build_source_pkg() create_cgz_name get_config - PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + local mark="true" + [ "${os}" == "archlinux" ] && local mark="pacman" + + PACMAN="${mark}" BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf local build_state="$?" diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 7d1e5cb15..ecbf72134 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -133,10 +133,16 @@ update_softlink() build_source_pkg() { - local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" - echo "$makepkg" + local mark="true" + local pkg_args="-A --check --skippgpcheck" + [ "${DISTRO}" == "archlinux" ] && { + local mark="pacman" + local pkg_args="${pkg_args} -s --needed --noconfirm" + } + local makepkg="$LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf" + echo "${makepkg}" - PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + PACMAN="${mark}" BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg [ "$?" == 0 ] || exit 1 update_softlink -- 2.23.0

1 0

[PATCH compass-ci 3/3] providers/qemu: check kernel
by Liu Yinsi 13 Jan '21

13 Jan '21

test: /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-x86_64/z9.351229 /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-riscv64/z9.351202 /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-aarch64/z9.351187 Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 8979da6..51357a6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -57,6 +57,9 @@ check_option_value() exit } + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 + [ -n "$initrds" ] || exit } @@ -104,9 +107,6 @@ set_qemu() do command -v "$qemu" > /dev/null && break done - - # debian has both qemu-system-x86_64 and qemu-system-riscv64 command - [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 } print_message() @@ -195,9 +195,9 @@ check_logfile write_logfile parse_ipxe_script -check_option_value set_options +check_option_value print_message -- 2.23.0

2 2

[PATCH v2 compass-ci 3/3] providers/qemu: check kernel
by Liu Yinsi 13 Jan '21

13 Jan '21

[why] check $kernel includes "riscv64" to set qemu command, this logic is suitable in check_option_value(), not in set_qemu, so move order. test: /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-x86_64/z9.351229 /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-riscv64/z9.351202 /srv/result/host-info/2021-01-13/vm-2p8g/debian-sid-aarch64/z9.351187 Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 8979da6..51357a6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -57,6 +57,9 @@ check_option_value() exit } + # debian has both qemu-system-x86_64 and qemu-system-riscv64 command + [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 + [ -n "$initrds" ] || exit } @@ -104,9 +107,6 @@ set_qemu() do command -v "$qemu" > /dev/null && break done - - # debian has both qemu-system-x86_64 and qemu-system-riscv64 command - [[ $kernel =~ 'riscv64' ]] && qemu=qemu-system-riscv64 } print_message() @@ -195,9 +195,9 @@ check_logfile write_logfile parse_ipxe_script -check_option_value set_options +check_option_value print_message -- 2.23.0

1 0

[PATCH v2 compass-ci 2/3] providers/qemu: check qemu command exist
by Liu Yinsi 13 Jan '21

13 Jan '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 9d15f7d..8979da6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -102,7 +102,7 @@ set_qemu() for qemu in "${qemus[@]}" do - [ -n "$(command -v ${qemu})" ] && break + command -v "$qemu" > /dev/null && break done # debian has both qemu-system-x86_64 and qemu-system-riscv64 command -- 2.23.0

1 0

[PATCH v2 compass-ci 1/3] providers/qemu: set qemu command
by Liu Yinsi 13 Jan '21

13 Jan '21

before: set qemu command by iterate qemus after: set qemu command by arch Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- providers/qemu/kvm.sh | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index 39a1dc1..9d15f7d 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -96,10 +96,8 @@ set_netdev() set_qemu() { qemus=( - qemu-system-aarch64 + qemu-system-$(arch) qemu-kvm - qemu-system-x86_64 - qemu-system-riscv64 ) for qemu in "${qemus[@]}" -- 2.23.0

1 0

[PATCH compass-ci] container/add-software: redirect redundant logs when clean dirs
by Xu Xijian 13 Jan '21

13 Jan '21

redundant logs as follows: rm: refusing to remove '.' or '..' directory: skipping 'srv/.' rm: refusing to remove '.' or '..' directory: skipping 'srv/..' rm: refusing to remove '.' or '..' directory: skipping 'opt/.' rm: refusing to remove '.' or '..' directory: skipping 'opt/..' rm: refusing to remove '.' or '..' directory: skipping 'mnt/.' rm: refusing to remove '.' or '..' directory: skipping 'mnt/..' rm: refusing to remove '.' or '..' directory: skipping 'media/.' rm: refusing to remove '.' or '..' directory: skipping 'media/..' rm: refusing to remove '.' or '..' directory: skipping 'home/.' rm: refusing to remove '.' or '..' directory: skipping 'home/..' rm: refusing to remove '.' or '..' directory: skipping 'dev/.' rm: refusing to remove '.' or '..' directory: skipping 'dev/..' rm: refusing to remove '.' or '..' directory: skipping 'proc/.' rm: refusing to remove '.' or '..' directory: skipping 'proc/..' rm: refusing to remove '.' or '..' directory: skipping 'run/.' rm: refusing to remove '.' or '..' directory: skipping 'run/..' rm: refusing to remove '.' or '..' directory: skipping 'sys/.' rm: refusing to remove '.' or '..' directory: skipping 'sys/..' Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/add-software/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/add-software/run b/container/add-software/run index 85f038b..840fb2c 100755 --- a/container/add-software/run +++ b/container/add-software/run @@ -43,7 +43,7 @@ install_cmds=( bash -c " yum install -y --skip-broken --installroot=/mnt/root $packages cd /mnt/root - rm -rf srv/{*,.*} opt/{*,.*} mnt/{*,.*} media/{*,.*} home/{*,.*} lost+found/{*,.*} dev/{*,.*} proc/{*,.*} run/{*,.*} sys/{*,.*} + rm -rf srv/{*,.*} opt/{*,.*} mnt/{*,.*} media/{*,.*} home/{*,.*} lost+found/{*,.*} dev/{*,.*} proc/{*,.*} run/{*,.*} sys/{*,.*} > /dev/null 2>&1 " ) -- 2.23.0

1 0