- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] netdata: modify the service port
by Yu Chuan 27 Mar '21

27 Mar '21

[Why] According our latest port policy, modify the service port: - netdata: 20012 => 20013 Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/netdata/start | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/netdata/start b/container/netdata/start index 18d225b8fcae..4b2d8962204e 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -10,7 +10,7 @@ cmd=( docker run -d --restart=always --name netdata - -p 20012:19999 + -p 20013:19999 -v /proc:/host/proc:ro -v /sys:/host/sys:ro -v /etc/localtime:/etc/localtime:ro -- 2.23.0

2 2

[PATCH lkp-tests] sbin/submit: fix error parsing parameter
by Wu Zhende 27 Mar '21

27 Mar '21

[Input] submit a="11 22 33" job.yaml [Current key-value] {"a" => "1122 33"} [Correct key-value] {"a" => "11 22 33"} Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- sbin/submit | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/submit b/sbin/submit index 5a87d781a..d2d9bec76 100755 --- a/sbin/submit +++ b/sbin/submit @@ -79,7 +79,7 @@ options.parse!(ARGV) seen_job_yaml = false ARGV.delete_if do |arg| if arg.index '=' - k, v = arg.sub(' ', '').split('=', 2) + k, v = arg.split('=', 2) if seen_job_yaml opt_set_key_value[k] = v else -- 2.23.0

2 1

[PATCH compass-ci] container: fix library libgc.so.1: No such file or directory
by Cao Xueliang 27 Mar '21

27 Mar '21

use the compile env as the runtime env in monitoring and lifecycle container. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/lifecycle/Dockerfile | 4 ++-- container/monitoring/Dockerfile | 6 +----- 2 files changed, 3 insertions(+), 7 deletions(-) diff --git a/container/lifecycle/Dockerfile b/container/lifecycle/Dockerfile index aa3e737..d8f8234 100644 --- a/container/lifecycle/Dockerfile +++ b/container/lifecycle/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:scheduler-dev MAINTAINER Wu Zhende <wuzhende666(a)163.com> @@ -13,7 +13,7 @@ RUN apk update && \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client activesupport git json yaml + gem install yaml ENV LIFECYCLE_RUNTIME_DIR /c/cci/lifecycle diff --git a/container/monitoring/Dockerfile b/container/monitoring/Dockerfile index 0a5dc22..80d2ec9 100644 --- a/container/monitoring/Dockerfile +++ b/container/monitoring/Dockerfile @@ -1,4 +1,4 @@ -FROM alpine:3.11 +FROM alpine:scheduler-dev RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories @@ -9,10 +9,6 @@ RUN apk add --no-cache 'ruby-dev' \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client && \ - gem install activesupport && \ - gem install git && \ - gem install json && \ gem install yaml -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code/dev-env: rename defaults vars
by Liu Yinsi 27 Mar '21

27 Mar '21

before: : {$A:=$server_ip}; $A=$A after: : {$a:=$server_ip}; $A=$a [why] when submit a job to run a testbox, sched will get variables(from /etc/compass-ci/scheduler), and export them in a running testbox env, but for selftest, it will deploy a new Compass CI, no need to use variables from the host(crystal or z9), it should be the testbox own server ip, so change defaults variables name. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index f96972e..f03fcb8 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -10,23 +10,23 @@ server_name=$(hostname | cut -f1 -d.) : ${ES_HOST:=$server_ip} : ${ES_PORT:=9200} : ${OS_HTTP_HOST:=$server_ip} -: ${SEND_MAIL_HOST:=$server_ip} +: ${send_mail_host:=$server_ip} : ${SEND_MAIL_PORT:=10001} -: ${SRV_HTTP_RESULT_HOST:=$server_ip} -: ${SRV_HTTP_OS_HOST:=$server_ip} -: ${SRV_HTTP_GIT_HOST:=$server_ip} -: ${SRV_HTTP_CCI_HOST:=$server_ip} +: ${srv_http_result_host:=$server_ip} +: ${srv_http_os_host:=$server_ip} +: ${srv_http_git_host:=$server_ip} +: ${srv_http_cci_host:=$server_ip} : ${SRV_HTTP_RESULT_PORT:=20007} : ${SRV_HTTP_OS_PORT:=20009} : ${SRV_HTTP_GIT_PORT:=20010} : ${SRV_HTTP_CCI_PORT:=20011} : ${LOGGING_ES_HOST:=$server_ip} : ${LOGGING_ES_PORT:=9202} -: ${INITRD_HTTP_HOST:=$server_ip} +: ${initrd_http_host:=$server_ip} : ${INITRD_HTTP_PORT:=8800} -: ${ASSIST_RESULT_HOST:=$server_ip} +: ${assist_result_host:=$server_ip} : ${ASSIST_RESULT_PORT:=8102} -: ${RESULT_WEBDAV_HOST:=$server_ip} +: ${result_webdav_host:=$server_ip} : ${RESULT_WEBDAV_PORT:=3080} : ${MASTER_FLUENTD_HOST:=$server_ip} : ${MASTER_FLUENTD_PORT:=24224} @@ -34,7 +34,7 @@ server_name=$(hostname | cut -f1 -d.) : ${DOCKER_REGISTRY_PORT:=5001} : ${LOCAL_SEND_MAIL_PORT:=11311} : ${LOCAL_ROBOT_EMAIL_ADDRESS:=Crystal TEAM} -: ${GIT_SERVER:=$server_ip} +: ${git_server:=$server_ip} : ${SCHED_HTTPS_HOST:=$server_ip} : ${SCHED_HTTPS_PORT:=20002} : ${DOCKER_PROXY_HOST:=$server_ip} @@ -76,28 +76,28 @@ ES_PORT: $ES_PORT SCHED_HOST: $sched_host SCHED_PORT: $sched_port LKP_SERVER: $server_ip -GIT_SERVER: $GIT_SERVER +GIT_SERVER: $git_server OS_HTTP_HOST: $OS_HTTP_HOST GITCACHE_HOST: $gitcache_host GITCACHE_PORT: $gitcache_port -SRV_HTTP_RESULT_HOST: $SRV_HTTP_RESULT_HOST -SRV_HTTP_OS_HOST: $SRV_HTTP_OS_HOST -SRV_HTTP_GIT_HOST: $SRV_HTTP_GIT_HOST -SRV_HTTP_CCI_HOST: $SRV_HTTP_CCI_HOST +SRV_HTTP_RESULT_HOST: $srv_http_result_host +SRV_HTTP_OS_HOST: $srv_http_os_host +SRV_HTTP_GIT_HOST: $srv_http_git_host +SRV_HTTP_CCI_HOST: $srv_http_cci_host SRV_HTTP_RESULT_PORT: $SRV_HTTP_RESULT_PORT SRV_HTTP_OS_PORT: $SRV_HTTP_OS_PORT SRV_HTTP_GIT_PORT: $SRV_HTTP_GIT_PORT SRV_HTTP_CCI_PORT: $SRV_HTTP_CCI_PORT TASKQUEUE_HOST: $sched_host -SEND_MAIL_HOST: $SEND_MAIL_HOST +SEND_MAIL_HOST: $send_mail_host SEND_MAIL_PORT: $SEND_MAIL_PORT LOGGING_ES_HOST: $LOGGING_ES_HOST LOGGING_ES_PORT: $LOGGING_ES_PORT -INITRD_HTTP_HOST: $INITRD_HTTP_HOST +INITRD_HTTP_HOST: $initrd_http_host INITRD_HTTP_PORT: $INITRD_HTTP_PORT -ASSIST_RESULT_HOST: $ASSIST_RESULT_HOST +ASSIST_RESULT_HOST: $assist_result_host ASSIST_RESULT_PORT: $ASSIST_RESULT_PORT -RESULT_WEBDAV_HOST: $RESULT_WEBDAV_HOST +RESULT_WEBDAV_HOST: $result_webdav_host RESULT_WEBDAV_PORT: $RESULT_WEBDAV_PORT MASTER_FLUENTD_HOST: $MASTER_FLUENTD_HOST MASTER_FLUENTD_PORT: $MASTER_FLUENTD_PORT -- 2.23.0

3 3

[PATCH compass-ci] src/lib: add default prefix for move queue
by Cao Xueliang 26 Mar '21

26 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/etcd_client.cr | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr index 637b738..7891eb3 100644 --- a/src/lib/etcd_client.cr +++ b/src/lib/etcd_client.cr @@ -39,6 +39,8 @@ class EtcdClient end def move(f_queue, t_queue, value) + f_queue = "#{BASE}/#{f_queue}" unless f_queue.starts_with?(BASE) + t_queue = "#{BASE}/#{t_queue}" unless t_queue.starts_with?(BASE) @etcd.kv.move(f_queue, t_queue, value) end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package/openEuler: update wget URL
by Liu Yinsi 26 Mar '21

26 Mar '21

The ole URL doesn't work, so update valid RUL Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/openEuler | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 61a37c4..0b0348d 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -28,8 +28,8 @@ pkgs=( yum install -y --skip-broken "${pkgs[@]}" -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… -wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… +wget http://rpmfind.net/linux/openmandriva/cooker/repository/$(arch)/main/releas… rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm rm -rf *.rpm -- 2.23.0

1 1

[PATCH compass-ci] src/lib: add etcd client
by Cao Xueliang 26 Mar '21

26 Mar '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/constants.cr | 8 +++++++ src/lib/etcd_client.cr | 51 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100644 src/lib/constants.cr create mode 100644 src/lib/etcd_client.cr diff --git a/src/lib/constants.cr b/src/lib/constants.cr new file mode 100644 index 0000000..d6a96d7 --- /dev/null +++ b/src/lib/constants.cr @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ or GPL-2.0 +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +BASE = "/queues" + +ETCD_HOST = "172.17.0.1" +ETCD_PORT = 2379 +ETCD_VERSION = "v3beta" diff --git a/src/lib/etcd_client.cr b/src/lib/etcd_client.cr new file mode 100644 index 0000000..637b738 --- /dev/null +++ b/src/lib/etcd_client.cr @@ -0,0 +1,51 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ or GPL-2.0 +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +require "etcd" + +require "./constants" + +class EtcdClient + def initialize + host = (ENV.has_key?("ETCD_HOST") ? ENV["ETCD_HOST"] : ETCD_HOST) + port = (ENV.has_key?("ETCD_PORT") ? ENV["ETCD_PORT"].to_i32 : ETCD_PORT) + version = (ENV.has_key?("ETCD_VERSION") ? ENV["ETCD_VERSION"] : ETCD_VERSION) + @etcd = Etcd.client(host, port, version) + end + + def put(queue, content) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.put_not_exists(queue, content) + end + + def delete(queue) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.delete(queue) + end + + def range(queue) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.range(queue) + end + + def range_prefix(prefix) + prefix = "#{BASE}/#{prefix}" unless prefix.starts_with?(BASE) + @etcd.kv.range_prefix(prefix) + end + + def update(queue, value) + queue = "#{BASE}/#{queue}" unless queue.starts_with?(BASE) + @etcd.kv.put(queue, value) + end + + def move(f_queue, t_queue, value) + @etcd.kv.move(f_queue, t_queue, value) + end + + def watch_prefix(prefix, **opts, &block : Array(Etcd::Model::WatchEvent) -> Void) + prefix = "#{BASE}/#{prefix}" unless prefix.starts_with?(BASE) + @etcd.watch.watch_prefix(prefix, **opts, &block) + end + +end + -- 2.23.0

1 0

[PATCH compass-ci] lib/dump_stat.rb: fix name error
by Lu Weitao 26 Mar '21

26 Mar '21

[Why] /home/baijing/compass-ci/lib/dump_stat.rb:171:in `number?': undefined local variable or method `monitor' for DumpStat:Module (NameError) Did you mean? @monitor Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/dump_stat.rb | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/dump_stat.rb b/lib/dump_stat.rb index 1c0695e..de0168e 100644 --- a/lib/dump_stat.rb +++ b/lib/dump_stat.rb @@ -55,7 +55,7 @@ module DumpStat if value.is_a?(String) value = check_string_value(k, value, @monitor) next unless value - return nil unless number?(value, @invalid_records) + return nil unless number?(value, @invalid_records, @record_index, @monitor) value = value.index('.') ? value.to_f : value.to_i elsif value.is_a?(Array) @@ -64,7 +64,7 @@ module DumpStat value[i] = check_string_value(k, value[i], @monitor) next unless value[i] - return nil unless number?(value[i], @invalid_records) + return nil unless number?(value[i], @invalid_records, @record_index, @monitor) value[i] = value[i].index('.') ? value[i].to_f : value[i].to_i valid_stats_verification(k, value[i]) @@ -165,7 +165,7 @@ def check_string_value(key, value, monitor) end # only number is valid -def number?(value, invalid_records) +def number?(value, invalid_records, record_index, monitor) unless value.numeric? invalid_records.push record_index warn_stat "invalid stats value: #{value}", monitor -- 2.23.0

1 0

[PATCH compass-ci 3/3] container/scheduler: fix library libgc.so.1: No such file or directory
by Cao Xueliang 26 Mar '21

26 Mar '21

use the compile env as the runtime env Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/scheduler/Dockerfile | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/container/scheduler/Dockerfile b/container/scheduler/Dockerfile index 7fb8e35..229a9c7 100644 --- a/container/scheduler/Dockerfile +++ b/container/scheduler/Dockerfile @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM alpine:scheduler-dev RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories @@ -12,10 +12,6 @@ RUN apk add --no-cache 'ruby-dev' \ RUN umask 002 && \ gem sources -r https://rubygems.org/ -a https://gems.ruby-china.com/ && \ - gem install rest-client && \ - gem install activesupport && \ - gem install git && \ - gem install json && \ gem install yaml -- 2.23.0

1 1

[PATCH compass-ci 2/3] container/crystal-shards: update crystal version from 0.33 to 0.35
by Cao Xueliang 26 Mar '21

26 Mar '21

add shard-amqp.yml to shard.yml, delete shard-amqp.yml add crystal-etcd to shard.yml Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- container/crystal-shards/Dockerfile | 3 --- container/crystal-shards/shard-amqp.yml | 7 ------- container/crystal-shards/shard.yml | 9 ++++++--- 3 files changed, 6 insertions(+), 13 deletions(-) delete mode 100644 container/crystal-shards/shard-amqp.yml diff --git a/container/crystal-shards/Dockerfile b/container/crystal-shards/Dockerfile index 15fe523..d3e321f 100644 --- a/container/crystal-shards/Dockerfile +++ b/container/crystal-shards/Dockerfile @@ -17,8 +17,5 @@ sed -i '99s/arguments\[:id]/arguments\[:id]?/' /usr/share/crystal/app/lib/elast sed -i 's/, Utils.__listify(arguments\[:q].as(String))/ /' /usr/share/crystal/app/lib/elasticsearch-crystal/src/elasticsearch/api/actions/search.cr;\ sed -i '205a \ params.clear' /usr/share/crystal/app/lib/elasticsearch-crystal/src/elasticsearch/api/actions/search.cr; -COPY shard-amqp.yml /usr/share/crystal/app/shard.yml -RUN shards - CMD ["bash"] diff --git a/container/crystal-shards/shard-amqp.yml b/container/crystal-shards/shard-amqp.yml deleted file mode 100644 index aea87cc..0000000 --- a/container/crystal-shards/shard-amqp.yml +++ /dev/null @@ -1,7 +0,0 @@ -name: monitor -version: 0.1.0 - -dependencies: - amqp-client: - github: cloudamqp/amqp-client.cr - version: 0.5.14 diff --git a/container/crystal-shards/shard.yml b/container/crystal-shards/shard.yml index ca8f72e..3466bc5 100644 --- a/container/crystal-shards/shard.yml +++ b/container/crystal-shards/shard.yml @@ -8,7 +8,7 @@ targets: scheduler: main: src/scheduler.cr -crystal: 0.33.0 +crystal: 0.35.0 license: Mulan PSL v2 @@ -17,10 +17,8 @@ dependencies: github: kemalcr/kemal redis: github: stefanwille/crystal-redis - version: ~> 2.5.3 elasticsearch-crystal: github: paktek123/elasticsearch-crystal - version: ~> 0.14 any_merge: github: icyleaf/any_merge branch: master @@ -32,3 +30,8 @@ dependencies: github: z64/rate_limiter singleton: github: waterlink/singleton.cr + etcd: + github: PythonEngineer007/crystal-etcd + branch: master + amqp-client: + github: cloudamqp/amqp-client.cr -- 2.23.0

1 0