- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] 0-package/os/openEuler: update lockfile rpm download URL
by Liu Yinsi 27 Mar '21

27 Mar '21

The old URL is invalid. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 61a37c4..9cb7b04 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -28,8 +28,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… +wget http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

1 0

[PATCH compass-ci] container/web-backend: fix SyntaxError)
by Lu Weitao 27 Mar '21

27 Mar '21

[Why] Traceback (most recent call last): 2: from ./web-backend:9:in `<main>' 1: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require' /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require': /home/lwt/compass-ci/src/lib/web_backend.rb:681: syntax error, unexpected end-of-input, expecting keyword_end (SyntaxError) Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/web-backend/web-backend | 1 + src/lib/web_backend.rb | 1 + 2 files changed, 2 insertions(+) diff --git a/container/web-backend/web-backend b/container/web-backend/web-backend index c7d1ec6..1172160 100755 --- a/container/web-backend/web-backend +++ b/container/web-backend/web-backend @@ -170,6 +170,7 @@ end # } get '/get_jobs_summary' do group_jobs_stats(params) +end # GET /get_job_error?suite=virttest&tbox_group=vm-2p8g # Response diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 7f26b9c..932d362 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -614,6 +614,7 @@ def group_jobs_stats(params) return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'group jobs table error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] +end # ------------------------------------------------------------------------------------------- # job error table like: -- 2.23.0

1 0

[PATCH v2 compass-ci] netdata: modify the service port
by Yu Chuan 27 Mar '21

27 Mar '21

[Why] According our latest port policy, modify the service port: - netdata: 20012 => 20013 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/netdata/start | 4 +++- sparrow/3-code/dev-env | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/container/netdata/start b/container/netdata/start index 18d225b8fcae..93eabb45dba7 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -4,13 +4,15 @@ . $CCI_SRC/container/defconfig.sh +load_cci_defaults + docker_rm netdata cmd=( docker run -d --restart=always --name netdata - -p 20012:19999 + -p ${NETDATA_PORT:=20013}:19999 -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /etc/localtime:/etc/localtime:ro diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f96972eb102d..209c6663238f 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -50,6 +50,7 @@ server_name=$(hostname | cut -f1 -d.) : ${REDIS_PORT:=6379} : ${MAILDIR:=/srv/cci/Maildir/.compass-ci} : ${SENT_MAILDIR:=/srv/cci/Maildir/.sent} +: ${NETDATA_PORT:=20013} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF -- 2.23.0

1 0

[PATCH compass-ci] netdata: modify the service port
by Yu Chuan 27 Mar '21

27 Mar '21

[Why] According our latest port policy, modify the service port: - netdata: 20012 => 20013 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/netdata/start | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/netdata/start b/container/netdata/start index 18d225b8fcae..4b2d8962204e 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -10,7 +10,7 @@ cmd=( docker run -d --restart=always --name netdata - -p 20012:19999 + -p 20013:19999 -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /etc/localtime:/etc/localtime:ro -- 2.23.0

2 2

[PATCH lkp-tests] sbin/submit: fix error parsing parameter
by Wu Zhende 27 Mar '21

27 Mar '21

[Input] submit a="11 22 33" job.yaml [Current key-value] {"a" => "1122 33"} [Correct key-value] {"a" => "11 22 33"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index 5a87d781a..d2d9bec76 100755 --- a/sbin/submit +++ b/sbin/submit @@ -79,7 +79,7 @@ options.parse!(ARGV) seen_job_yaml = false ARGV.delete_if do |arg| if arg.index '=' - k, v = arg.sub(' ', '').split('=', 2) + k, v = arg.split('=', 2) if seen_job_yaml opt_set_key_value[k] = v else -- 2.23.0

2 1

[PATCH compass-ci] container: fix library libgc.so.1: No such file or directory
by Cao Xueliang 27 Mar '21

27 Mar '21

use the compile env as the runtime env in monitoring and lifecycle container. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/lifecycle/Dockerfile | 4 ++-- container/monitoring/Dockerfile | 6 +----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/container/lifecycle/Dockerfile b/container/lifecycle/Dockerfile index aa3e737..d8f8234 100644 --- a/container/lifecycle/Dockerfile +++ b/container/lifecycle/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:scheduler-dev MAINTAINER Wu Zhende <wuzhende666(a)163.com> @@ -13,7 +13,7 @@ RUN apk update && \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client activesupport git json yaml + gem install yaml ENV LIFECYCLE_RUNTIME_DIR /c/cci/lifecycle diff --git a/container/monitoring/Dockerfile b/container/monitoring/Dockerfile index 0a5dc22..80d2ec9 100644 --- a/container/monitoring/Dockerfile +++ b/container/monitoring/Dockerfile @@ -1,4 +1,4 @@ -FROM alpine:3.11 +FROM alpine:scheduler-dev RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories @@ -9,10 +9,6 @@ RUN apk add --no-cache 'ruby-dev' \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client && \ - gem install activesupport && \ - gem install git && \ - gem install json && \ gem install yaml -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code/dev-env: rename defaults vars
by Liu Yinsi 27 Mar '21

27 Mar '21

before: : {$A:=$server_ip}; $A=$A after: : {$a:=$server_ip}; $A=$a [why] when submit a job to run a testbox, sched will get variables(from /etc/compass-ci/scheduler), and export them in a running testbox env, but for selftest, it will deploy a new Compass CI, no need to use variables from the host(crystal or z9), it should be the testbox own server ip, so change defaults variables name. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f96972e..f03fcb8 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -10,23 +10,23 @@ server_name=$(hostname | cut -f1 -d.) : ${ES_HOST:=$server_ip} : ${ES_PORT:=9200} : ${OS_HTTP_HOST:=$server_ip} -: ${SEND_MAIL_HOST:=$server_ip} +: ${send_mail_host:=$server_ip} : ${SEND_MAIL_PORT:=10001} -: ${SRV_HTTP_RESULT_HOST:=$server_ip} -: ${SRV_HTTP_OS_HOST:=$server_ip} -: ${SRV_HTTP_GIT_HOST:=$server_ip} -: ${SRV_HTTP_CCI_HOST:=$server_ip} +: ${srv_http_result_host:=$server_ip} +: ${srv_http_os_host:=$server_ip} +: ${srv_http_git_host:=$server_ip} +: ${srv_http_cci_host:=$server_ip} : ${SRV_HTTP_RESULT_PORT:=20007} : ${SRV_HTTP_OS_PORT:=20009} : ${SRV_HTTP_GIT_PORT:=20010} : ${SRV_HTTP_CCI_PORT:=20011} : ${LOGGING_ES_HOST:=$server_ip} : ${LOGGING_ES_PORT:=9202} -: ${INITRD_HTTP_HOST:=$server_ip} +: ${initrd_http_host:=$server_ip} : ${INITRD_HTTP_PORT:=8800} -: ${ASSIST_RESULT_HOST:=$server_ip} +: ${assist_result_host:=$server_ip} : ${ASSIST_RESULT_PORT:=8102} -: ${RESULT_WEBDAV_HOST:=$server_ip} +: ${result_webdav_host:=$server_ip} : ${RESULT_WEBDAV_PORT:=3080} : ${MASTER_FLUENTD_HOST:=$server_ip} : ${MASTER_FLUENTD_PORT:=24224} @@ -34,7 +34,7 @@ server_name=$(hostname | cut -f1 -d.) : ${DOCKER_REGISTRY_PORT:=5001} : ${LOCAL_SEND_MAIL_PORT:=11311} : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} -: ${GIT_SERVER:=$server_ip} +: ${git_server:=$server_ip} : ${SCHED_HTTPS_HOST:=$server_ip} : ${SCHED_HTTPS_PORT:=20002} : ${DOCKER_PROXY_HOST:=$server_ip} @@ -76,28 +76,28 @@ ES_PORT: $ES_PORT SCHED_HOST: $sched_host SCHED_PORT: $sched_port LKP_SERVER: $server_ip -GIT_SERVER: $GIT_SERVER +GIT_SERVER: $git_server OS_HTTP_HOST: $OS_HTTP_HOST GITCACHE_HOST: $gitcache_host GITCACHE_PORT: $gitcache_port -SRV_HTTP_RESULT_HOST: $SRV_HTTP_RESULT_HOST -SRV_HTTP_OS_HOST: $SRV_HTTP_OS_HOST -SRV_HTTP_GIT_HOST: $SRV_HTTP_GIT_HOST -SRV_HTTP_CCI_HOST: $SRV_HTTP_CCI_HOST +SRV_HTTP_RESULT_HOST: $srv_http_result_host +SRV_HTTP_OS_HOST: $srv_http_os_host +SRV_HTTP_GIT_HOST: $srv_http_git_host +SRV_HTTP_CCI_HOST: $srv_http_cci_host SRV_HTTP_RESULT_PORT: $SRV_HTTP_RESULT_PORT SRV_HTTP_OS_PORT: $SRV_HTTP_OS_PORT SRV_HTTP_GIT_PORT: $SRV_HTTP_GIT_PORT SRV_HTTP_CCI_PORT: $SRV_HTTP_CCI_PORT TASKQUEUE_HOST: $sched_host -SEND_MAIL_HOST: $SEND_MAIL_HOST +SEND_MAIL_HOST: $send_mail_host SEND_MAIL_PORT: $SEND_MAIL_PORT LOGGING_ES_HOST: $LOGGING_ES_HOST LOGGING_ES_PORT: $LOGGING_ES_PORT -INITRD_HTTP_HOST: $INITRD_HTTP_HOST +INITRD_HTTP_HOST: $initrd_http_host INITRD_HTTP_PORT: $INITRD_HTTP_PORT -ASSIST_RESULT_HOST: $ASSIST_RESULT_HOST +ASSIST_RESULT_HOST: $assist_result_host ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT -RESULT_WEBDAV_HOST: $RESULT_WEBDAV_HOST +RESULT_WEBDAV_HOST: $result_webdav_host RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT MASTER_FLUENTD_HOST: $MASTER_FLUENTD_HOST MASTER_FLUENTD_PORT: $MASTER_FLUENTD_PORT -- 2.23.0

3 3

[PATCH compass-ci] src/lib: add default prefix for move queue
by Cao Xueliang 26 Mar '21

26 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/etcd_client.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr index 637b738..7891eb3 100644 --- a/src/lib/etcd_client.cr +++ b/src/lib/etcd_client.cr @@ -39,6 +39,8 @@ class EtcdClient end def move(f_queue, t_queue, value) + f_queue = "#{BASE}/#{f_queue}" unless f_queue.starts_with?(BASE) + t_queue = "#{BASE}/#{t_queue}" unless t_queue.starts_with?(BASE) @etcd.kv.move(f_queue, t_queue, value) end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package/openEuler: update wget URL
by Liu Yinsi 26 Mar '21

26 Mar '21

The ole URL doesn't work, so update valid RUL Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 61a37c4..0b0348d 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -28,8 +28,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

1 1

[PATCH compass-ci] src/lib: add etcd client
by Cao Xueliang 26 Mar '21

26 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/constants.cr | 8 +++++++ src/lib/etcd_client.cr | 51 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100644 src/lib/constants.cr create mode 100644 src/lib/etcd_client.cr diff --git a/src/lib/constants.cr b/src/lib/constants.cr new file mode 100644 index 0000000..d6a96d7 --- /dev/null +++ b/src/lib/constants.cr @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ or GPL-2.0 +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +BASE = "/queues" + +ETCD_HOST = "172.17.0.1" +ETCD_PORT = 2379 +ETCD_VERSION = "v3beta" diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr new file mode 100644 index 0000000..637b738 --- /dev/null +++ b/src/lib/etcd_client.cr @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ or GPL-2.0 +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +require "etcd" + +require "./constants" + +class EtcdClient + def initialize + host = (ENV.has_key?("ETCD_HOST") ? ENV["ETCD_HOST"] : ETCD_HOST) + port = (ENV.has_key?("ETCD_PORT") ? ENV["ETCD_PORT"].to_i32 : ETCD_PORT) + version = (ENV.has_key?("ETCD_VERSION") ? ENV["ETCD_VERSION"] : ETCD_VERSION) + @etcd = Etcd.client(host, port, version) + end + + def put(queue, content) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.put_not_exists(queue, content) + end + + def delete(queue) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.delete(queue) + end + + def range(queue) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.range(queue) + end + + def range_prefix(prefix) + prefix = "#{BASE}/#{prefix}" unless prefix.starts_with?(BASE) + @etcd.kv.range_prefix(prefix) + end + + def update(queue, value) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.put(queue, value) + end + + def move(f_queue, t_queue, value) + @etcd.kv.move(f_queue, t_queue, value) + end + + def watch_prefix(prefix, **opts, &block : Array(Etcd::Model::WatchEvent) -> Void) + prefix = "#{BASE}/#{prefix}" unless prefix.starts_with?(BASE) + @etcd.watch.watch_prefix(prefix, **opts, &block) + end + +end + -- 2.23.0

1 0