- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/2] lib/mail-post/email_limit_queue: use ENV instead of static value
by Luan Shengde 29 Mar '21

29 Mar '21

use ENV variable instead of static value. enable user-defined new value without edit the code. just restart the service. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail-post/email_limit_queue.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/mail-post/email_limit_queue.rb b/lib/mail-post/email_limit_queue.rb index c7e3954..62e9634 100755 --- a/lib/mail-post/email_limit_queue.rb +++ b/lib/mail-post/email_limit_queue.rb @@ -54,9 +54,9 @@ class EmailRateLimit end def change_queue(email) - return unless (@redis.hget 'email_in_limit', email).to_i >= 10 + return unless (@redis.hget 'email_in_limit', email).to_i >= ENV['EMAIL_LIMIT_COUNT'].to_i @redis.hdel 'email_in_limit', email - @redis.hset 'email_out_limit', email, 10 + @redis.hset 'email_out_limit', email, ENV['EMAIL_LIMIT_COUNT'] end end -- 2.23.0

1 0

[PATCH compass-ci] container/gitcache: delete gitcache docker that is no longer in use
by Cui Yili 28 Mar '21

28 Mar '21

Signed-off-by: Cui Yili <2268260388(a)qq.com> --- container/gitcache/Dockerfile | 15 --------------- container/gitcache/build | 10 ---------- container/gitcache/start | 20 -------------------- 3 files changed, 45 deletions(-) delete mode 100644 container/gitcache/Dockerfile delete mode 100755 container/gitcache/build delete mode 100755 container/gitcache/start diff --git a/container/gitcache/Dockerfile b/container/gitcache/Dockerfile deleted file mode 100644 index d92a141..0000000 --- a/container/gitcache/Dockerfile +++ /dev/null @@ -1,15 +0,0 @@ -# Origin: https://github.com/git-cloner/gitcache -# Copyright (C) 2019-2020 Eric D.little51 -# SPDX-License-Identifier: GPL-3.0 - -FROM golang:alpine - -ENV GO111MODULE on -ENV GOPROXY https://goproxy.cn - - -RUN apk add --no-cache git && \ - git clone https://github.com/git-cloner/gitcache /gitcache && \ - cd /gitcache && go build - -ENTRYPOINT ["/gitcache/gitcache"] diff --git a/container/gitcache/build b/container/gitcache/build deleted file mode 100755 index d7f3523..0000000 --- a/container/gitcache/build +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/bash -# Origin: https://github.com/git-cloner/gitcache -# Copyright (C) 2019-2020 Eric D.little51 -# SPDX-License-Identifier: GPL-3.0 - -. ../defconfig.sh - -docker_skip_rebuild "gitcache" - -docker build -t gitcache . diff --git a/container/gitcache/start b/container/gitcache/start deleted file mode 100755 index 0e1de1b..0000000 --- a/container/gitcache/start +++ /dev/null @@ -1,20 +0,0 @@ -#!/bin/bash -# Origin: https://github.com/git-cloner/gitcache -# Copyright (C) 2019-2020 Eric D.little51 -# SPDX-License-Identifier: GPL-3.0 - -. $CCI_SRC/container/defconfig.sh - -docker_rm gitcache - -cmd=( - docker run -dt - --name gitcache - --restart=always - -p 5000:5000 - -v /srv/cache/gitcache/:/var/gitcache/ - -v /etc/localtime:/etc/localtime:ro - gitcache -) - -"${cmd[@]}" -- 2.23.0

2 1

[PATCH compass-ci] container: add etcd container
by Cao Xueliang 28 Mar '21

28 Mar '21

add etcd container Dockerfile, build and start script. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/etcd/Dockerfile | 16 ++++++++++++++++ container/etcd/build | 12 ++++++++++++ container/etcd/start | 28 ++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+) create mode 100644 container/etcd/Dockerfile create mode 100755 container/etcd/build create mode 100755 container/etcd/start diff --git a/container/etcd/Dockerfile b/container/etcd/Dockerfile new file mode 100644 index 0000000..3654fc4 --- /dev/null +++ b/container/etcd/Dockerfile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM alpine:3.13 + +MAINTAINER Cao Xueliang <caoxl78320(a)163.com> + +ARG VERSION +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories && \ + apk update && apk add ca-certificates && \ + wget https://github.com/etcd-io/etcd/releases/download/${VERSION}/etcd-${VERSION… && \ + tar zxvf etcd-${VERSION}-linux-arm64.tar.gz && \ + mv etcd-${VERSION}-linux-arm64/etcd* /bin/ && \ + rm -Rf etcd-${VERSION}-linux-arm64* /var/cache/apk/* + +VOLUME /data diff --git a/container/etcd/build b/container/etcd/build new file mode 100755 index 0000000..6ae25dd --- /dev/null +++ b/container/etcd/build @@ -0,0 +1,12 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ../defconfig.sh + +version=v3.4.15 +docker_skip_rebuild "etcd-${version}:alpine" + +docker build -t etcd-${version}:alpine --build-arg VERSION=${version} . + + diff --git a/container/etcd/start b/container/etcd/start new file mode 100755 index 0000000..7657b86 --- /dev/null +++ b/container/etcd/start @@ -0,0 +1,28 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. $CCI_SRC/container/defconfig.sh + +docker_rm etcd-server + +cmd=( + docker run + -it + -p 2379:2379 + -p 2380:2380 + -p 4001:4001 + -p 7001:7001 + -v /srv/etcd/dir:/data + -e ETCD_UNSUPPORTED_ARCH=arm64 + --name etcd-server + etcd-v3.4.15:alpine + /bin/etcd + --name node-1 + --data-dir /data + --initial-cluster-state new + --listen-client-urls http://0.0.0.0:2379 + --advertise-client-urls http://0.0.0.0:2379 +) + +"${cmd[@]}" -- 2.23.0

1 0

[PATCH compass-ci] 0-package/os/openEuler: update lockfile rpm download URL
by Liu Yinsi 27 Mar '21

27 Mar '21

The old URL is invalid. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 61a37c4..9cb7b04 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -28,8 +28,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… +wget http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

1 0

[PATCH compass-ci] container/web-backend: fix SyntaxError)
by Lu Weitao 27 Mar '21

27 Mar '21

[Why] Traceback (most recent call last): 2: from ./web-backend:9:in `<main>' 1: from /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require' /usr/share/rubygems/rubygems/core_ext/kernel_require.rb:59:in `require': /home/lwt/compass-ci/src/lib/web_backend.rb:681: syntax error, unexpected end-of-input, expecting keyword_end (SyntaxError) Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- container/web-backend/web-backend | 1 + src/lib/web_backend.rb | 1 + 2 files changed, 2 insertions(+) diff --git a/container/web-backend/web-backend b/container/web-backend/web-backend index c7d1ec6..1172160 100755 --- a/container/web-backend/web-backend +++ b/container/web-backend/web-backend @@ -170,6 +170,7 @@ end # } get '/get_jobs_summary' do group_jobs_stats(params) +end # GET /get_job_error?suite=virttest&tbox_group=vm-2p8g # Response diff --git a/src/lib/web_backend.rb b/src/lib/web_backend.rb index 7f26b9c..932d362 100644 --- a/src/lib/web_backend.rb +++ b/src/lib/web_backend.rb @@ -614,6 +614,7 @@ def group_jobs_stats(params) return [500, headers.merge('Access-Control-Allow-Origin' => '*'), 'group jobs table error'] end [200, headers.merge('Access-Control-Allow-Origin' => '*'), body] +end # ------------------------------------------------------------------------------------------- # job error table like: -- 2.23.0

1 0

[PATCH v2 compass-ci] netdata: modify the service port
by Yu Chuan 27 Mar '21

27 Mar '21

[Why] According our latest port policy, modify the service port: - netdata: 20012 => 20013 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/netdata/start | 4 +++- sparrow/3-code/dev-env | 1 + 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/container/netdata/start b/container/netdata/start index 18d225b8fcae..93eabb45dba7 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -4,13 +4,15 @@ . $CCI_SRC/container/defconfig.sh +load_cci_defaults + docker_rm netdata cmd=( docker run -d --restart=always --name netdata - -p 20012:19999 + -p ${NETDATA_PORT:=20013}:19999 -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /etc/localtime:/etc/localtime:ro diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f96972eb102d..209c6663238f 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -50,6 +50,7 @@ server_name=$(hostname | cut -f1 -d.) : ${REDIS_PORT:=6379} : ${MAILDIR:=/srv/cci/Maildir/.compass-ci} : ${SENT_MAILDIR:=/srv/cci/Maildir/.sent} +: ${NETDATA_PORT:=20013} mkdir -p /etc/compass-ci/defaults cat > /etc/compass-ci/defaults/$server_name.yaml <<EOF -- 2.23.0

1 0

[PATCH compass-ci] netdata: modify the service port
by Yu Chuan 27 Mar '21

27 Mar '21

[Why] According our latest port policy, modify the service port: - netdata: 20012 => 20013 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/netdata/start | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/netdata/start b/container/netdata/start index 18d225b8fcae..4b2d8962204e 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -10,7 +10,7 @@ cmd=( docker run -d --restart=always --name netdata - -p 20012:19999 + -p 20013:19999 -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /etc/localtime:/etc/localtime:ro -- 2.23.0

2 2

[PATCH lkp-tests] sbin/submit: fix error parsing parameter
by Wu Zhende 27 Mar '21

27 Mar '21

[Input] submit a="11 22 33" job.yaml [Current key-value] {"a" => "1122 33"} [Correct key-value] {"a" => "11 22 33"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index 5a87d781a..d2d9bec76 100755 --- a/sbin/submit +++ b/sbin/submit @@ -79,7 +79,7 @@ options.parse!(ARGV) seen_job_yaml = false ARGV.delete_if do |arg| if arg.index '=' - k, v = arg.sub(' ', '').split('=', 2) + k, v = arg.split('=', 2) if seen_job_yaml opt_set_key_value[k] = v else -- 2.23.0

2 1

[PATCH compass-ci] container: fix library libgc.so.1: No such file or directory
by Cao Xueliang 27 Mar '21

27 Mar '21

use the compile env as the runtime env in monitoring and lifecycle container. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/lifecycle/Dockerfile | 4 ++-- container/monitoring/Dockerfile | 6 +----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/container/lifecycle/Dockerfile b/container/lifecycle/Dockerfile index aa3e737..d8f8234 100644 --- a/container/lifecycle/Dockerfile +++ b/container/lifecycle/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:scheduler-dev MAINTAINER Wu Zhende <wuzhende666(a)163.com> @@ -13,7 +13,7 @@ RUN apk update && \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client activesupport git json yaml + gem install yaml ENV LIFECYCLE_RUNTIME_DIR /c/cci/lifecycle diff --git a/container/monitoring/Dockerfile b/container/monitoring/Dockerfile index 0a5dc22..80d2ec9 100644 --- a/container/monitoring/Dockerfile +++ b/container/monitoring/Dockerfile @@ -1,4 +1,4 @@ -FROM alpine:3.11 +FROM alpine:scheduler-dev RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories @@ -9,10 +9,6 @@ RUN apk add --no-cache 'ruby-dev' \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client && \ - gem install activesupport && \ - gem install git && \ - gem install json && \ gem install yaml -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code/dev-env: rename defaults vars
by Liu Yinsi 27 Mar '21

27 Mar '21

before: : {$A:=$server_ip}; $A=$A after: : {$a:=$server_ip}; $A=$a [why] when submit a job to run a testbox, sched will get variables(from /etc/compass-ci/scheduler), and export them in a running testbox env, but for selftest, it will deploy a new Compass CI, no need to use variables from the host(crystal or z9), it should be the testbox own server ip, so change defaults variables name. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f96972e..f03fcb8 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -10,23 +10,23 @@ server_name=$(hostname | cut -f1 -d.) : ${ES_HOST:=$server_ip} : ${ES_PORT:=9200} : ${OS_HTTP_HOST:=$server_ip} -: ${SEND_MAIL_HOST:=$server_ip} +: ${send_mail_host:=$server_ip} : ${SEND_MAIL_PORT:=10001} -: ${SRV_HTTP_RESULT_HOST:=$server_ip} -: ${SRV_HTTP_OS_HOST:=$server_ip} -: ${SRV_HTTP_GIT_HOST:=$server_ip} -: ${SRV_HTTP_CCI_HOST:=$server_ip} +: ${srv_http_result_host:=$server_ip} +: ${srv_http_os_host:=$server_ip} +: ${srv_http_git_host:=$server_ip} +: ${srv_http_cci_host:=$server_ip} : ${SRV_HTTP_RESULT_PORT:=20007} : ${SRV_HTTP_OS_PORT:=20009} : ${SRV_HTTP_GIT_PORT:=20010} : ${SRV_HTTP_CCI_PORT:=20011} : ${LOGGING_ES_HOST:=$server_ip} : ${LOGGING_ES_PORT:=9202} -: ${INITRD_HTTP_HOST:=$server_ip} +: ${initrd_http_host:=$server_ip} : ${INITRD_HTTP_PORT:=8800} -: ${ASSIST_RESULT_HOST:=$server_ip} +: ${assist_result_host:=$server_ip} : ${ASSIST_RESULT_PORT:=8102} -: ${RESULT_WEBDAV_HOST:=$server_ip} +: ${result_webdav_host:=$server_ip} : ${RESULT_WEBDAV_PORT:=3080} : ${MASTER_FLUENTD_HOST:=$server_ip} : ${MASTER_FLUENTD_PORT:=24224} @@ -34,7 +34,7 @@ server_name=$(hostname | cut -f1 -d.) : ${DOCKER_REGISTRY_PORT:=5001} : ${LOCAL_SEND_MAIL_PORT:=11311} : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} -: ${GIT_SERVER:=$server_ip} +: ${git_server:=$server_ip} : ${SCHED_HTTPS_HOST:=$server_ip} : ${SCHED_HTTPS_PORT:=20002} : ${DOCKER_PROXY_HOST:=$server_ip} @@ -76,28 +76,28 @@ ES_PORT: $ES_PORT SCHED_HOST: $sched_host SCHED_PORT: $sched_port LKP_SERVER: $server_ip -GIT_SERVER: $GIT_SERVER +GIT_SERVER: $git_server OS_HTTP_HOST: $OS_HTTP_HOST GITCACHE_HOST: $gitcache_host GITCACHE_PORT: $gitcache_port -SRV_HTTP_RESULT_HOST: $SRV_HTTP_RESULT_HOST -SRV_HTTP_OS_HOST: $SRV_HTTP_OS_HOST -SRV_HTTP_GIT_HOST: $SRV_HTTP_GIT_HOST -SRV_HTTP_CCI_HOST: $SRV_HTTP_CCI_HOST +SRV_HTTP_RESULT_HOST: $srv_http_result_host +SRV_HTTP_OS_HOST: $srv_http_os_host +SRV_HTTP_GIT_HOST: $srv_http_git_host +SRV_HTTP_CCI_HOST: $srv_http_cci_host SRV_HTTP_RESULT_PORT: $SRV_HTTP_RESULT_PORT SRV_HTTP_OS_PORT: $SRV_HTTP_OS_PORT SRV_HTTP_GIT_PORT: $SRV_HTTP_GIT_PORT SRV_HTTP_CCI_PORT: $SRV_HTTP_CCI_PORT TASKQUEUE_HOST: $sched_host -SEND_MAIL_HOST: $SEND_MAIL_HOST +SEND_MAIL_HOST: $send_mail_host SEND_MAIL_PORT: $SEND_MAIL_PORT LOGGING_ES_HOST: $LOGGING_ES_HOST LOGGING_ES_PORT: $LOGGING_ES_PORT -INITRD_HTTP_HOST: $INITRD_HTTP_HOST +INITRD_HTTP_HOST: $initrd_http_host INITRD_HTTP_PORT: $INITRD_HTTP_PORT -ASSIST_RESULT_HOST: $ASSIST_RESULT_HOST +ASSIST_RESULT_HOST: $assist_result_host ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT -RESULT_WEBDAV_HOST: $RESULT_WEBDAV_HOST +RESULT_WEBDAV_HOST: $result_webdav_host RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT MASTER_FLUENTD_HOST: $MASTER_FLUENTD_HOST MASTER_FLUENTD_PORT: $MASTER_FLUENTD_PORT -- 2.23.0

3 3