- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] etcd/Dockerfile: change source address
by Wang Yong 30 Mar '21

30 Mar '21

accelerate download Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- container/etcd/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/etcd/Dockerfile b/container/etcd/Dockerfile index 3654fc4..b3957e8 100644 --- a/container/etcd/Dockerfile +++ b/container/etcd/Dockerfile @@ -8,7 +8,7 @@ MAINTAINER Cao Xueliang <caoxl78320(a)163.com> ARG VERSION RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ apk update && apk add ca-certificates && \ - wget https://github.com/etcd-io/etcd/releases/download/${VERSION}/etcd-${VERSION… && \ + wget https://repo.huaweicloud.com/etcd/${VERSION}/etcd-${VERSION}-linux-arm64.ta… && \ tar zxvf etcd-${VERSION}-linux-arm64.tar.gz && \ mv etcd-${VERSION}-linux-arm64/etcd* /bin/ && \ rm -Rf etcd-${VERSION}-linux-arm64* /var/cache/apk/* -- 2.23.0

1 0

[PATCH v2 compass-ci] container/ssh-r/start: rename var for vm_name to docker_name
by Wei Jihui 30 Mar '21

30 Mar '21

vm_name --> docker_name Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- container/ssh-r/start | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/container/ssh-r/start b/container/ssh-r/start index 68a63f2..47a29eb 100755 --- a/container/ssh-r/start +++ b/container/ssh-r/start @@ -4,14 +4,14 @@ . $CCI_SRC/container/defconfig.sh -vm_name=ssh_r +docker_name=ssh_r -docker_rm ${vm_name} +docker_rm ${docker_name} cmd=( docker run --restart=always - --name ${vm_name} + --name ${docker_name} -e GATEWAY_PORTS=true -e TCP_FORWARDING=true -d -- 2.23.0

2 1

[PATCH compass-ci] container/taskqueue: compiling code every time
by Wu Zhende 30 Mar '21

30 Mar '21

Delete the complied taskqueue file. Otherwise, the code will be updated and the taskqueue file will not be recomplied. Uses old code all the time. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/taskqueue/build | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/container/taskqueue/build b/container/taskqueue/build index 4f69f85..518a27e 100755 --- a/container/taskqueue/build +++ b/container/taskqueue/build @@ -12,10 +12,8 @@ service_port=${TASKQUEUE_PORT:-3060} docker_skip_rebuild "taskqueue-alpine:v0.001-${service_port}" -if [[ ! -f "taskqueue" ]]; then - $CCI_SRC/container/compile . -else - echo -e "\n\033[34m- Info: using \033[31mexists\033[34m taskqueue -\033[0m\n" -fi +$CCI_SRC/container/compile . docker build -t taskqueue-alpine:v0.001-${service_port} . + +rm taskqueue -- 2.23.0

1 0

[PATCH compass-ci] monitoring/parse_serial_logs.cr: fix useless arg
by Wei Jihui 30 Mar '21

30 Mar '21

host is not used at detect_start_or_end Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- src/monitoring/parse_serial_logs.cr | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/monitoring/parse_serial_logs.cr b/src/monitoring/parse_serial_logs.cr index 0589523..451312c 100644 --- a/src/monitoring/parse_serial_logs.cr +++ b/src/monitoring/parse_serial_logs.cr @@ -51,7 +51,7 @@ class SerialParser File.basename(msg["serial_path"].to_s) end - def detect_start_or_end(msg, host, pattern_list) + def detect_start_or_end(msg, pattern_list) message = msg["message"].to_s pattern_list.each do |pattern| matched = message.match(/.*(?<signal>#{pattern})/) @@ -60,7 +60,7 @@ class SerialParser end def delete_host(msg, host, signal) - boundary_signal = detect_start_or_end(msg, host, signal) + boundary_signal = detect_start_or_end(msg, signal) return unless boundary_signal @host2head.delete(host) -- 2.23.0

1 0

[PATCH lkp-tests] monitors/kmsg: not execute dmesg when in docker
by Lin Jiaxin 30 Mar '21

30 Mar '21

In order to avoid the following stderr: ==> /tmp/stderr <== dmesg: read kernel buffer failed: Operation not permitted Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- monitors/kmsg | 2 ++ 1 file changed, 2 insertions(+) diff --git a/monitors/kmsg b/monitors/kmsg index 4edd90a9..a1cf7121 100755 --- a/monitors/kmsg +++ b/monitors/kmsg @@ -6,6 +6,8 @@ # serial console has better availability. is_virt && [ -z "$LKP_LOCAL_RUN" ] && exec > /dev/ttyS1 +[ -f /.dockerenv ] && exit 0 + if dmesg --help 2>&1 | grep -q -- --follow; then exec dmesg --follow --decode else -- 2.23.0

2 1

[PATCH compass-ci] src/lib/lifecycle.cr: use close_job close job
by Wei Jihui 30 Mar '21

30 Mar '21

unified use close_job to close job, reduce code Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- src/lib/lifecycle.cr | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index 4bd2ac0..adcb965 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -177,8 +177,7 @@ class Lifecycle event_job_id = event["job_id"].to_s return unless @jobs.has_key?(event_job_id) - @jobs.delete(event_job_id) - spawn @scheduler_api.close_job(event_job_id, "abnormal", "lifecycle") + close_job(event_job_id, "abnormal") end def on_job_close(event) @@ -222,8 +221,7 @@ class Lifecycle return if machine_job_id.empty? return unless @jobs.has_key?(machine_job_id) - @jobs.delete(machine_job_id) - spawn @scheduler_api.close_job(machine_job_id, "abnormal", "lifecycle") + close_job(machine_job_id, "abnormal") end def max_time(times) -- 2.23.0

1 0

[PATCH compass-ci 3/4] container/rpm-repo: add rpm-repo.rb for update repodata
by Li Ping 30 Mar '21

30 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/rpm-repo/rpm-repo.rb | 73 ++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100755 container/rpm-repo/rpm-repo.rb diff --git a/container/rpm-repo/rpm-repo.rb b/container/rpm-repo/rpm-repo.rb new file mode 100755 index 0000000..18f0507 --- /dev/null +++ b/container/rpm-repo/rpm-repo.rb @@ -0,0 +1,73 @@ +#!/usr/bin/ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require "bunny" +require "fileutils" +require "json" + + +MQ_HOST = ENV['MQ_HOST'] || ENV['LKP_SERVER'] || '172.17.0.1' +MQ_PORT = ENV['MQ_PORT'] || 5672 + +class MQClient + def initialize(hostname = "172.17.0.1", port = "5672") + @conn = Bunny.new(hostname: hostname, port: port) + @conn.start + @channel = @conn.create_channel + end + + def queue(queue_name, opts = {}) + @channel.queue(queue_name, opts) + end + + def ack(delivery_info) + @channel.ack(delivery_info.delivery_tag) + end +end + +class HandleRepo + def initialize + @mq = MQClient.new(MQ_HOST, MQ_PORT) + @update = [] + end + + def handle_new_rpm + queue = @mq.queue("update_repo") + queue.subscribe({:block => true, :manual_ack => true}) do |info, _pro, msg| + rpm_info = JSON.parse(msg) + puts rpm_info + rpm_info["upload_rpms"].each do |rpm| + rpm_path = File.dirname(rpm).sub("upload", "testing") + FileUtils.mkdir_p(rpm_path) unless File.directory?(rpm_path) + + dest = File.join(rpm_path.to_s, File.basename(rpm)) + @update << dest + FileUtils.mv(rpm, dest) + system("createrepo --update $(dirname #{rpm_path})") + end + update_pub_dir + @mq.ack(info) + end + end + + def update_pub_dir + @update.each do |rpm| + pub_path = File.dirname(rpm).sub("testing", "pub") + FileUtils.mkdir_p(pub_path) unless File.directory?(pub_path) + + dest = File.join(pub_path, File.basename(rpm)) + FileUtils.cp(rpm, dest) + + repodata_dest = File.join(File.dirname(pub_path), "repodata") + repodata_src = File.dirname(rpm).sub("Packages", "repodata") + + FileUtils.rm_r(repodata_dest) if Dir.exist?(repodata_dest) + FileUtils.cp_r(repodata_src, File.dirname(repodata_dest)) + end + end +end + +hr = HandleRepo.new +hr.handle_new_rpm -- 2.23.0

4 5

[PATCH compass-ci 5/5] container/updaterepo: add update-repo start script
by Li Ping 30 Mar '21

30 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/updaterepo/start | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100755 container/updaterepo/start diff --git a/container/updaterepo/start b/container/updaterepo/start new file mode 100755 index 0000000..46e3ad6 --- /dev/null +++ b/container/updaterepo/start @@ -0,0 +1,24 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative '../defconfig.rb' + +docker_rm "update-repo" + +cmd = %W[ + docker run + --restart=always + --name update-repo + -u 1090:1090 + -p 8132:8132 + -d + -v /srv/rpm/:/srv/rpm/ + -v /etc/localtime:/etc/localtime:ro + update-repo +] + +cmd += ['sh', '-c', 'umask 002 && ./updaterepo'] + +system(*cmd) -- 2.23.0

2 2

[PATCH compass-ci 2/5] src/lib/updaterepo.cr: send messages to update-repo queue
by Li Ping 30 Mar '21

30 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- src/lib/updaterepo.cr | 60 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 src/lib/updaterepo.cr diff --git a/src/lib/updaterepo.cr b/src/lib/updaterepo.cr new file mode 100644 index 0000000..4826fac --- /dev/null +++ b/src/lib/updaterepo.cr @@ -0,0 +1,60 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +require "json" +require "kemal" +require "./json_logger" +require "./mq" + +class Repo + def initialize(env : HTTP::Server::Context) + @mq = MQClient.instance + @env = env + @log = JSONLogger.new + end + + def upload_repo + begin + body = @env.request.body.not_nil!.gets_to_end + data = JSON.parse(body.to_s).as_h? + rescue e + @log.error(e) + end + + puts "-" * 50 + puts "post body:", data + + begin + # check if the parameters are complete + check_params_complete(data) + mq_publish(data) + rescue e + response = { "errcode" => "101", "errmsg" => "upload rpm failed" } + @log.info(response.to_json) + puts "error message: #{e.message}" + end + end + + def mq_publish(data) + mq_msg = data + puts mq_msg.to_json + + spawn mq_publish_check("update_repo", mq_msg.to_json) + end + + def mq_publish_check(queue, msg) + 3.times do + @mq.publish_confirm(queue, msg) + break + rescue e + res = @mq.reconnect + sleep 5 + end + end + + def check_params_complete(params) + params = params.not_nil! + tmp_hash = {"errcode" => "101", "errmsg" => "no upload_rpms params"}.to_json + raise tmp_hash unless params["upload_rpms"]? + end +end -- 2.23.0

2 2

[PATCH v3 compass-ci 3/3] sparrow/3-code/dev-env: add key EMAIL_LIMIT_COUNT
by Luan Shengde 30 Mar '21

30 Mar '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sparrow/3-code/dev-env | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 27258fc..7327083 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -12,6 +12,7 @@ server_name=$(hostname | cut -f1 -d.) : ${OS_HTTP_HOST:=$server_ip} : ${send_mail_host:=$server_ip} : ${SEND_MAIL_PORT:=10001} +: ${EMAIL_LIMIT_COUNT:=10} : ${srv_http_result_host:=$server_ip} : ${srv_http_os_host:=$server_ip} : ${srv_http_git_host:=$server_ip} @@ -90,6 +91,7 @@ SRV_HTTP_CCI_PORT: $SRV_HTTP_CCI_PORT TASKQUEUE_HOST: $sched_host SEND_MAIL_HOST: $send_mail_host SEND_MAIL_PORT: $SEND_MAIL_PORT +EMAIL_LIMIT_COUNT: $EMAIL_LIMIT_COUNT LOGGING_ES_HOST: $LOGGING_ES_HOST LOGGING_ES_PORT: $LOGGING_ES_PORT INITRD_HTTP_HOST: $initrd_http_host -- 2.23.0

1 0