- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 1/2] src/lib(scheduer): change function name "*pushlish*" to "*publish*" for mq
by Xu Xijian 16 Mar '21

16 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/lifecycle.cr | 4 ++-- src/lib/mq.cr | 4 ++-- src/lib/sched.cr | 4 ++-- src/scheduler/close_job.cr | 2 +- src/scheduler/find_job_boot.cr | 2 +- src/scheduler/update_job_parameter.cr | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index 1eb6264..4bd2ac0 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -79,7 +79,7 @@ class Lifecycle "job_state" => "abnormal", "testbox" => testbox } - @mq.pushlish_confirm("job_mq", msg.to_json) + @mq.publish_confirm("job_mq", msg.to_json) @match[testbox].delete(id) end end @@ -313,7 +313,7 @@ class Lifecycle mq_queue = get_machine_reboot_queue(testbox) machine.as_h.delete("history") machine.as_h["testbox"] = JSON::Any.new(testbox) - @mq.pushlish_confirm(mq_queue, machine.to_json, durable: true) + @mq.publish_confirm(mq_queue, machine.to_json, durable: true) machine["state"] = "rebooting_queue" machine["time"] = Time.local.to_s("%Y-%m-%dT%H:%M:%S+0800") diff --git a/src/lib/mq.cr b/src/lib/mq.cr index 980585d..1c7f4b7 100644 --- a/src/lib/mq.cr +++ b/src/lib/mq.cr @@ -37,7 +37,7 @@ class MQClient Singleton::Of(self).instance end - def pushlish_confirm(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + def publish_confirm(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) q = @ch.queue(queue, passive, durable, exclusive, auto_delete) if durable q.publish_confirm(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) @@ -46,7 +46,7 @@ class MQClient end end - def pushlish(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + def publish(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) q = @ch.queue(queue, passive, durable, exclusive, auto_delete) if durable q.publish(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 9528f54..ac677de 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -219,9 +219,9 @@ class Sched return nil end - def mq_pushlish_confirm(queue, msg) + def mq_publish_confirm(queue, msg) 3.times do - @mq.pushlish_confirm(queue, msg) + @mq.publish_confirm(queue, msg) break rescue e res = @mq.reconnect diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index e014361..a57b233 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -45,7 +45,7 @@ class Sched "job_state" => "close", "time" => get_time } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end end end diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 4a8ea8a..416f8f4 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -32,7 +32,7 @@ class Sched "time" => get_time, "job_state" => "boot" } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end # auto submit a job to collect the host information. diff --git a/src/scheduler/update_job_parameter.cr b/src/scheduler/update_job_parameter.cr index d81f28b..92b905e 100644 --- a/src/scheduler/update_job_parameter.cr +++ b/src/scheduler/update_job_parameter.cr @@ -44,6 +44,6 @@ class Sched "job_state" => (@env.get?("job_state") || "update").to_s, "time" => @env.get?("time").to_s } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end end -- 2.23.0

1 0

[PATCH compass-ci] container/dracut-initrd: add e2fsck and basename into dracut module 00bash
by Xu Xijian 16 Mar '21

16 Mar '21

1. mke2fs is depended by e2fsck. 2. We've add basename into 98dracut module, but it's too late, so add it into an earlier module 00bash. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 494588a..48e8bcc 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -18,6 +18,7 @@ cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh sed -i "/install() {/a\ inst /usr/bin/awk" /usr/lib/dracut/modules.d/40network/module-setup.sh sed -i "/install() {/a\ inst /sbin/mke2fs /usr/bin/basename" /usr/lib/dracut/modules.d/98dracut-systemd/module-setup.sh +sed -i "/install() {/a\ inst /sbin/e2fsck /sbin/mke2fs /usr/bin/basename" /usr/lib/dracut/modules.d/00bash/module-setup.sh pre_mount_file="/usr/lib/dracut/modules.d/98dracut-systemd/dracut-pre-mount.sh" [ "$(sed -n '$p' $pre_mount_file)" = "exit 0" ] && sed -i '$d' "$pre_mount_file" cat set-local-sysroot.sh >> "$pre_mount_file" -- 2.23.0

2 2

[PATCH v3 lkp-tests] add test script for dockerfile os migrating
by Bai Jing 16 Mar '21

16 Mar '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/openeuler_docker | 151 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100755 tests/openeuler_docker diff --git a/tests/openeuler_docker b/tests/openeuler_docker new file mode 100755 index 000000000..b0b5336a3 --- /dev/null +++ b/tests/openeuler_docker @@ -0,0 +1,151 @@ +#!/bin/sh +# - dockerfile_url + +# exit 1 if openEuler failed +# exit 0 if +# - no centos support +# - centos failed + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/log.sh + +[ -n "$dockerfile_url" ] || die "dockerfile_url is empty" + +declare -i nr_openeuler_fail=0 +declare -i nr_openeuler_success=0 +declare -i is_openeuler_build_fail=0 +declare -i dockerfile_count=0 + +fit_repo() +{ + yum reinstall -y git + git clone "$dockerfile_url" "$BENCHMARK_ROOT/src/" + dockerfile_path="$(find "$BENCHMARK_ROOT/src/" -type f -name "*Dockerfile*" | xargs grep -HE '^FROM centos:(centos|)[78]')" + dockerfile_count=$(( $(echo $dockerfile_path | awk '{print NF}') / 2 )) + [ -n "$dockerfile_path" ] || { + echo "current-repo-not-support-centos: 1" + exit 0 + } +} + +openeuler_ready() +{ + docker load < $BENCHMARK_ROOT/openeuler_docker/openeuler-20.03-lts-v1.tar +} + +centos_docker() +{ + is_openeuler_build_fail=0 + index=1 + for file in $dockerfile_path + do + [[ $file =~ ^centos ]] && continue + file=$(echo "$file" | awk -F: '{print $1}') + dockerfile_name="${file##*/}" + cd "${file%/*}" || continue + image=$(echo "$file" | awk -F '/' '{print $5}' | tr 'A-Z' 'a-z') + [ -n "$image" ] && image=${image}_$index + if [ -f "build.sh" ] && [ -x "build.sh" ]; then + ./build.sh > /dev/null 2> /dev/null + else + docker_build > /dev/null 2> /dev/null + fi + + [ $? == 0 ] && { + handle_dockerfile + } + + let index++ + done +} + +handle_FROM() +{ + sed -i "s/^FROM .*/FROM openeuler-20.03-lts:v1/g" "$dockerfile_name" +} + +handle_epel() +{ + grep -qw "epel-release" "$dockerfile_name" && { + sed -i "s/epel-release/bash/g" "$dockerfile_name" + } +} + +handle_epelrpm() +{ + grep -qw "https://.*epel-release.*rpm" "$dockerfile_name" && { + sed -i "s|https://.*epel-release.*rpm|bash|g" "$dockerfile_name" + } +} + +add_base_commands() +{ + # grep -qw "groupadd*" "$file" && { + # fix missing useradd, groupadd, chpasswd, etc. commands + sed -i '/FROM /a\RUN yum -y install shadow tar' "$dockerfile_name" +} + +handle_epel_repo() +{ + sed -i '/.* wget .*(epel|CentOS-Base)\.repo/ s|^|#|g' "$dockerfile_name" +} + +handle_dockerfile() +{ + is_openeuler_build_fail=1 + handle_FROM + handle_epel + add_base_commands + if docker_build "-openeuler"; then + docker_run "-openeuler" + let nr_openeuler_success++ + else + let nr_openeuler_fail++ + fi +} + +docker_build() +{ + docker build -f "$dockerfile_name" -t "${image}$1" . + [ $? != 0 ] && { + [ "$is_openeuler_build_fail" == "1" ] && echo "${image}$1.build.fail: 1" + return 1 + } + [ "$is_openeuler_build_fail" == "1" ] && echo "${image}$1.build.success: 1" + is_openeuler_build_fail=0 +} + +docker_run() +{ + docker run -d --name "${image}" "${image}$1" + docker_stats=$(docker inspect -f '{{.State.Running}}' ${image}) + [ $docker_stats == false ] && { + echo "${image}.run.fail: 1" + return 1 + } + + echo "${image}.run.success: 1" +} + +handle_build_result() +{ + if [ "$nr_openeuler_fail" != 0 ]; then + echo "openeuler build failed times: $nr_openeuler_fail" + echo "openeuler_valid_url: $dockerfile_url" + exit 1 + # centos all success, openeuler all success + elif [ "$nr_openeuler_success" == "$dockerfile_count" ]; then + echo "openeuler_valid_url: $dockerfile_url" + exit 0 + fi +} + +docker_test() +{ + fit_repo + openeuler_ready + centos_docker + handle_build_result +} + +docker_test -- 2.23.0

1 0

[PATCH lkp-tests] add test script for dockerfile os migrating
by Bai Jing 16 Mar '21

16 Mar '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/openeuler_docker | 152 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 152 insertions(+) create mode 100755 tests/openeuler_docker diff --git a/tests/openeuler_docker b/tests/openeuler_docker new file mode 100755 index 000000000..74087ab62 --- /dev/null +++ b/tests/openeuler_docker @@ -0,0 +1,152 @@ +#!/bin/sh +# - dockerfile_url + +# exit 1 if openEuler failed +# exit 0 if +# - no centos support +# - centos failed + +. $LKP_SRC/lib/debug.sh +. $LKP_SRC/lib/log.sh + +[ -n "$dockerfile_url" ] || die "dockerfile_url is empty" + +declare -i nr_openeuler_fail=0 +declare -i nr_openeuler_success=0 +declare -i is_openeuler_build_fail=0 +declare -i dockerfile_count=0 + +fit_repo() +{ + yum reinstall -y git + git clone "$dockerfile_url" "$BENCHMARK_ROOT/src/" + dockerfile_path="$(find "$BENCHMARK_ROOT/src/" -type f -name "*Dockerfile*" | xargs grep -HE '^FROM centos:(centos|)[78]')" + dockerfile_count=$(( $(echo $dockerfile_path | awk '{print NF}') / 2 )) + [ -n "$dockerfile_path" ] || { + echo "current-repo-not-support-centos: 1" + exit 0 + } +} + +openeuler_ready() +{ + docker load < $BENCHMARK_ROOT/openeuler_docker/openeuler-20.03-lts-v1.tar +} + +centos_docker() +{ + is_openeuler_build_fail=0 + index=1 + for file in $dockerfile_path + do + [[ $file =~ ^centos ]] && continue + file=$(echo "$file" | awk -F: '{print $1}') + dockerfile_name="${file##*/}" + cd "${file%/*}" || continue + image=$(echo "$file" | awk -F '/' '{print $5}' | tr 'A-Z' 'a-z') + [ -n "$image" ] && image=${image}_$index + if [ -f "build.sh" ] && [ -x "build.sh" ]; then + ./build.sh > /dev/null 2> /dev/null + else + docker_build > /dev/null 2> /dev/null + fi + + [ $? == 0 ] && { + handle_dockerfile + } + + let index++ + done +} + + +handle_FROM() +{ + sed -i "s/^FROM .*/FROM openeuler-20.03-lts:v1/g" "$dockerfile_name" +} + +handle_epel() +{ + grep -qw "epel-release" "$dockerfile_name" && { + sed -i "s/epel-release/bash/g" "$dockerfile_name" + } +} + +handle_epelrpm() +{ + grep -qw "https://.*epel-release.*rpm" "$dockerfile_name" && { + sed -i "s|https://.*epel-release.*rpm|bash|g" "$dockerfile_name" + } +} + +add_base_commands() +{ + # grep -qw "groupadd*" "$file" && { + # fix missing useradd, groupadd, chpasswd, etc. commands + sed -i '/FROM /a\RUN yum -y install shadow tar' "$dockerfile_name" +} + +handle_epel_repo() +{ + sed -i '/.* wget .*(epel|CentOS-Base)\.repo/ s|^|#|g' "$dockerfile_name" +} + +handle_dockerfile() +{ + is_openeuler_build_fail=1 + handle_FROM + handle_epel + add_base_commands + if docker_build "-openeuler"; then + docker_run "-openeuler" + let nr_openeuler_success++ + else + let nr_openeuler_fail++ + fi +} + +docker_build() +{ + docker build -f "$dockerfile_name" -t "${image}$1" . + [ $? != 0 ] && { + [ "$is_openeuler_build_fail" == "1" ] && echo "${image}$1.build.fail: 1" + return 1 + } + [ "$is_openeuler_build_fail" == "1" ] && echo "${image}$1.build.success: 1" + is_openeuler_build_fail=0 +} + +docker_run() +{ + docker run -d --name "${image}" "${image}$1" + docker_stats=$(docker inspect -f '{{.State.Running}}' ${image}) + [ $docker_stats == false ] && { + echo "${image}.run.fail: 1" + return 1 + } + + echo "${image}.run.success: 1" +} + +handle_build_result() +{ + if [ "$nr_openeuler_fail" != 0 ]; then + echo "openeuler build failed times: $nr_openeuler_fail" + echo "openeuler_valid_url: $dockerfile_url" + exit 1 + # centos all success, openeuler all success + elif [ "$nr_openeuler_success" == "$dockerfile_count" ]; then + echo "openeuler_valid_url: $dockerfile_url" + exit 0 + fi +} + +docker_test() +{ + fit_repo + openeuler_ready + centos_docker + handle_build_result +} + +docker_test -- 2.23.0

2 2

[PATCH compass-ci] sparrow: change service default ports
by Cao Xueliang 16 Mar '21

16 Mar '21

Reference: 6354ba631e0fd5f795a1dc28b65e2002bd3de668 Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/extract-stats/start | 2 +- sparrow/3-code/dev-env | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/container/extract-stats/start b/container/extract-stats/start index 5033754..621099a 100755 --- a/container/extract-stats/start +++ b/container/extract-stats/start @@ -25,7 +25,7 @@ env = docker_env(defaults) DEFAULT_LKP = '/c/lkp-tests' DEFAULT_CCI = '/c/compass-ci' SRV_HTTP_RESULT_HOST = defaults['SRV_HTTP_RESULT_HOST'] || Open3.capture3('curl ip.sb')[0].chomp -SRV_HTTP_RESULT_PORT = defaults['SRV_HTTP_RESULT_PORT'] || '11300' +SRV_HTTP_RESULT_PORT = defaults['SRV_HTTP_RESULT_PORT'] || '20007' docker_rm "extract-stats" cmd = %w[ diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index b740a15..65061d8 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -15,10 +15,10 @@ server_name=$(hostname | cut -f1 -d.) : ${SRV_HTTP_OS_HOST:=$server_ip} : ${SRV_HTTP_GIT_HOST:=$server_ip} : ${SRV_HTTP_CCI_HOST:=$server_ip} -: ${SRV_HTTP_RESULT_PORT:=11300} -: ${SRV_HTTP_OS_PORT:=11304} -: ${SRV_HTTP_GIT_PORT:=11305} -: ${SRV_HTTP_CCI_PORT:=11307} +: ${SRV_HTTP_RESULT_PORT:=20007} +: ${SRV_HTTP_OS_PORT:=20009} +: ${SRV_HTTP_GIT_PORT:=20010} +: ${SRV_HTTP_CCI_PORT:=20011} : ${LOGGING_ES_HOST:=$server_ip} : ${LOGGING_ES_PORT:=9202} : ${INITRD_HTTP_HOST:=$server_ip} -- 2.23.0

1 0

[PATCH compass-ci] src/lib(scheduer): change function name "*pushlish*" to "*publish*" for mq
by Xu Xijian 16 Mar '21

16 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/lifecycle.cr | 4 ++-- src/lib/mq.cr | 4 ++-- src/lib/sched.cr | 4 ++-- src/scheduler/close_job.cr | 2 +- src/scheduler/find_job_boot.cr | 2 +- src/scheduler/update_job_parameter.cr | 2 +- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index 1eb6264..4bd2ac0 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -79,7 +79,7 @@ class Lifecycle "job_state" => "abnormal", "testbox" => testbox } - @mq.pushlish_confirm("job_mq", msg.to_json) + @mq.publish_confirm("job_mq", msg.to_json) @match[testbox].delete(id) end end @@ -313,7 +313,7 @@ class Lifecycle mq_queue = get_machine_reboot_queue(testbox) machine.as_h.delete("history") machine.as_h["testbox"] = JSON::Any.new(testbox) - @mq.pushlish_confirm(mq_queue, machine.to_json, durable: true) + @mq.publish_confirm(mq_queue, machine.to_json, durable: true) machine["state"] = "rebooting_queue" machine["time"] = Time.local.to_s("%Y-%m-%dT%H:%M:%S+0800") diff --git a/src/lib/mq.cr b/src/lib/mq.cr index 980585d..1c7f4b7 100644 --- a/src/lib/mq.cr +++ b/src/lib/mq.cr @@ -37,7 +37,7 @@ class MQClient Singleton::Of(self).instance end - def pushlish_confirm(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + def publish_confirm(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) q = @ch.queue(queue, passive, durable, exclusive, auto_delete) if durable q.publish_confirm(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) @@ -46,7 +46,7 @@ class MQClient end end - def pushlish(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) + def publish(queue, msg, passive = false, durable = false, exclusive = false, auto_delete = false) q = @ch.queue(queue, passive, durable, exclusive, auto_delete) if durable q.publish(msg, props: AMQ::Protocol::Properties.new(delivery_mode: 2)) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 9528f54..ac677de 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -219,9 +219,9 @@ class Sched return nil end - def mq_pushlish_confirm(queue, msg) + def mq_publish_confirm(queue, msg) 3.times do - @mq.pushlish_confirm(queue, msg) + @mq.publish_confirm(queue, msg) break rescue e res = @mq.reconnect diff --git a/src/scheduler/close_job.cr b/src/scheduler/close_job.cr index e014361..a57b233 100644 --- a/src/scheduler/close_job.cr +++ b/src/scheduler/close_job.cr @@ -45,7 +45,7 @@ class Sched "job_state" => "close", "time" => get_time } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end end end diff --git a/src/scheduler/find_job_boot.cr b/src/scheduler/find_job_boot.cr index 4a8ea8a..416f8f4 100644 --- a/src/scheduler/find_job_boot.cr +++ b/src/scheduler/find_job_boot.cr @@ -32,7 +32,7 @@ class Sched "time" => get_time, "job_state" => "boot" } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end # auto submit a job to collect the host information. diff --git a/src/scheduler/update_job_parameter.cr b/src/scheduler/update_job_parameter.cr index d81f28b..92b905e 100644 --- a/src/scheduler/update_job_parameter.cr +++ b/src/scheduler/update_job_parameter.cr @@ -44,6 +44,6 @@ class Sched "job_state" => (@env.get?("job_state") || "update").to_s, "time" => @env.get?("time").to_s } - spawn mq_pushlish_confirm(JOB_MQ, mq_msg.to_json) + spawn mq_publish_confirm(JOB_MQ, mq_msg.to_json) end end -- 2.23.0

1 0

[PATCH v3 lkp-tests] stat/ansible_test: stats result for ansible_test
by Wang Chenglong 16 Mar '21

16 Mar '21

The result of ansible_test are counted and recorded in the ES. query it with the 'es-jobs' command. sum.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng: 1 raw.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng.message: ["{\"changed\": false, \"msg\": \"Unsupported pkg mgr. This role is depends on apt, yum, or pkgng.\"}"] Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 47 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..d6acad1ab 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,40 @@ #!/usr/bin/env ruby -while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line +require 'json' + +def output_error(error_msg) + return if error_msg.nil? + + if error_msg.is_a? Array + error_msg.each do |i| + error_id = common_error_id i + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" + end + elsif error_msg.is_a? String + error_id = common_error_id error_msg + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" end end + +def common_error_id(line) + line.gsub!(/[^\w]/, '-') + line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one + line.gsub!(/^-|-$/, '') + line +end + +while (line = STDIN.gets) + next unless line =~ /(FAILED!|failed:).*=>(.*)/ + + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? + + ansible_failed_json = JSON.parse @ansible_failed_info + + output_error ansible_failed_json['msg'] + output_error ansible_failed_json['message'] + output_error ansible_failed_json['cmd'] + output_error ansible_failed_json['failures'] +end -- 2.23.0

1 0

[PATCH compass-ci] doc: update document
by Luan Shengde 16 Mar '21

16 Mar '21

update document for service new port: send-internet-mail: 49000 => 10001 Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/send-mail.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/send-mail.md b/doc/send-mail.md index 5af8d17..ffa5651 100644 --- a/doc/send-mail.md +++ b/doc/send-mail.md @@ -8,7 +8,7 @@ The service is used to send mails with formatted mail data ## send mail port - send intranet mail: 11311 -- send internet mail: 49000 +- send internet mail: 10001 ## send mail host - host the service running on @@ -58,5 +58,5 @@ The service is used to send mails with formatted mail data test msg 1010 " - %x(curl -XPOST 'localhost:49000/send_mail_text' -d "#{data}") + %x(curl -XPOST 'localhost:10001/send_mail_text' -d "#{data}") ``` -- 2.23.0

1 0

[PATCH v3 lkp-tests] stat/ansible_test: stats result for ansible_test
by Wang Chenglong 16 Mar '21

16 Mar '21

The result of ansible_test are counted and recorded in the ES. query it with the 'es-jobs' command. sum.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng: 1 raw.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng.message: ["{\"changed\": false, \"msg\": \"Unsupported pkg mgr. This role is depends on apt, yum, or pkgng.\"}"] Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 47 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..9e9e15fab 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,40 @@ #!/usr/bin/env ruby -while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line +require 'json' + +def output_error(error_msg) + return if error_msg.nil? + + if error_msg.is_a? Array + error_msg.each do |i| + error_id = common_error_id i + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" + end + elsif error_msg.is_a? String + error_id = common_error_id error_msg + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" end end + +def common_error_id(line) + line.gsub!(/[^\w]/, '-') + line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one + line.gsub!(/-$/, '') + line +end + +while (line = STDIN.gets) + next unless line =~ /(FAILED!|failed:).*=>(.*)/ + + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? + + ansible_failed_json = JSON.parse @ansible_failed_info + + output_error ansible_failed_json['msg'] + output_error ansible_failed_json['message'] + output_error ansible_failed_json['cmd'] + output_error ansible_failed_json['failures'] +end -- 2.23.0

1 1

[PATCH v2 lkp-tests] stat/ansible_test: stats result for ansible_test
by Wang Chenglong 16 Mar '21

16 Mar '21

The result of ansible_test are counted and recorded in the ES. query it with the 'es-jobs' command. sum.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng: 1 raw.stats.ansible_test.error.Unsupported-pkg-mgr-This-role-is-depends-on-apt-yum-or-pkgng.message: ["{\"changed\": false, \"msg\": \"Unsupported pkg mgr. This role is depends on apt, yum, or pkgng.\"}"] Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 47 +++++++++++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 11 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..119dc1d2a 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,40 @@ #!/usr/bin/env ruby -while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line +require 'json' + +def output_error(error_msg) + return if error_msg.nil? + + if error_msg.is_a? Array + error_msg.each do |i| + error_id = common_error_id i + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" + end + else + error_id = common_error_id error_msg + puts "error.#{error_id}: 1" + puts "error.#{error_id}.message: #{@ansible_failed_info}" end end + +def common_error_id(line) + line.gsub!(/[^\w]/, '-') + line.gsub!(/-+/, '-') # Replace multiple consecutive '-' with a single one + line.gsub!(/-$/, '') + line +end + +while (line = STDIN.gets) + next unless line =~ /(FAILED!|failed:).*=>(.*)/ + + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? + + ansible_failed_json = JSON.parse @ansible_failed_info + + output_error ansible_failed_json['msg'] + output_error ansible_failed_json['message'] + output_error ansible_failed_json['cmd'] + output_error ansible_failed_json['failures'] +end -- 2.23.0

2 2