February 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v4 compass-ci 2/3] container: call docker_skip_rebuild()
by Liu Yinsi 09 Feb '21

09 Feb '21

prevent rebuild container when execute `sparrow/4-docker/buildall run-only` Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/add-software/build | 4 ++++ container/alpine/build | 4 ++++ container/archlinux/build | 4 ++++ container/assign-account/build | 4 ++++ container/assist-result/build | 4 ++++ container/assistant/build | 4 ++++ container/auto-submit/build | 4 ++++ container/conserver/build | 4 ++++ container/crystal-base/build | 4 ++++ container/crystal-compiler/build | 4 ++++ container/crystal-shards/build | 3 +++ container/debian/build | 4 ++++ container/delimiter/build | 2 ++ container/dnsmasq/build | 4 ++++ container/dracut-initrd/build | 4 ++++ container/es/build | 2 ++ container/extract-stats/build | 4 ++++ container/fetch-mail/build | 4 ++++ container/fluentd-base/build | 4 ++++ container/git-daemon/build | 4 ++++ container/git-mirror/build | 4 ++++ container/gitcache/build | 4 ++++ container/initrd-cifs/build | 4 ++++ container/initrd-http/build | 4 ++++ container/kibana/build | 4 ++++ container/lifecycle/build | 2 ++ container/lkp-initrd/build | 4 ++++ container/logging-es/build | 2 ++ container/mail-robot/build | 4 ++++ container/manjaro/build | 4 ++++ container/master-fluentd/build | 4 ++++ container/minio/build | 4 ++++ container/monitoring/build | 2 ++ container/netdata-slave/build | 4 ++++ container/netdata/build | 4 ++++ container/ntp-server/build | 4 ++++ container/openresty-proxy-cache/build | 4 ++++ container/os-cifs/build | 4 ++++ container/os-http/build | 4 ++++ container/os-nfs/build | 4 ++++ container/qcow2rootfs/build | 4 ++++ container/qemu-efi/build | 4 ++++ container/rabbitmq/build | 4 ++++ container/redis/build | 4 ++++ container/register-accounts/build | 4 ++++ container/registry/build | 4 ++++ container/remote-git/build | 4 ++++ container/result-cifs/build | 4 ++++ container/result-webdav/build | 4 ++++ container/rsync-server/build | 4 ++++ container/scheduler-dev/build | 4 ++++ container/scheduler/build | 2 ++ container/send-internet-mail/build | 4 ++++ container/send-mail/build | 4 ++++ container/shellcheck/build | 4 ++++ container/srv-http/build | 4 ++++ container/ssh-r/build | 4 ++++ container/sub-fluentd/build | 4 ++++ container/submit/build | 4 ++++ container/taskqueue/build | 2 ++ container/upload-libvirt-xml/build | 4 ++++ container/web-backend-nginx/build | 4 ++++ container/web-backend/build | 4 ++++ container/webhook/build | 4 ++++ 64 files changed, 241 insertions(+) diff --git a/container/add-software/build b/container/add-software/build index 234c8b9..bc3c95a 100755 --- a/container/add-software/build +++ b/container/add-software/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "addsoftware:auto" + docker build -t addsoftware:auto . diff --git a/container/alpine/build b/container/alpine/build index bd277c3..3168799 100755 --- a/container/alpine/build +++ b/container/alpine/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:testbed" + docker build -t alpine:testbed . diff --git a/container/archlinux/build b/container/archlinux/build index 9749489..6db5a68 100755 --- a/container/archlinux/build +++ b/container/archlinux/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "archlinux:testbed" + docker build --build-arg ARCH=$(arch) -t archlinux:testbed . diff --git a/container/assign-account/build b/container/assign-account/build index 3984341..480c410 100755 --- a/container/assign-account/build +++ b/container/assign-account/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "assign-account:latest" + docker build -t assign-account:latest . diff --git a/container/assist-result/build b/container/assist-result/build index 130127b..600b80a 100755 --- a/container/assist-result/build +++ b/container/assist-result/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "assist-result" + docker build -t assist-result . diff --git a/container/assistant/build b/container/assistant/build index 7778c9f..89a8a97 100755 --- a/container/assistant/build +++ b/container/assistant/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:assistant" + docker build -t debian:assistant . diff --git a/container/auto-submit/build b/container/auto-submit/build index 96383e3..48ff111 100755 --- a/container/auto-submit/build +++ b/container/auto-submit/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:auto-submit" + docker build -t alpine:auto-submit . diff --git a/container/conserver/build b/container/conserver/build index 0700c5e..a7e3867 100755 --- a/container/conserver/build +++ b/container/conserver/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "conserver_server" + ./generate_conserver.rb || exit 1 docker build -t conserver_server . diff --git a/container/crystal-base/build b/container/crystal-base/build index 4df65a5..642e190 100755 --- a/container/crystal-base/build +++ b/container/crystal-base/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-base" + if [ -n "$SQUID_HOST" ] && [ -n "$SQUID_PORT" ]; then proxy="http://$SQUID_HOST:$SQUID_PORT/" else diff --git a/container/crystal-compiler/build b/container/crystal-compiler/build index cee43eb..570fe98 100755 --- a/container/crystal-compiler/build +++ b/container/crystal-compiler/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-compiler" + docker build -t alpine:crystal-compiler . diff --git a/container/crystal-shards/build b/container/crystal-shards/build index 784f571..7c2a712 100755 --- a/container/crystal-shards/build +++ b/container/crystal-shards/build @@ -6,6 +6,9 @@ # If job.yaml does not have these two parameters, git will use the local proxy cache, # GITCACHE_HOST defaults to the local ip, GITCACHE_PORT default port is 5000 . ../defconfig.sh + +docker_skip_rebuild "alpine:crystal-shards" + load_cci_defaults docker build --build-arg GITCACHE_HOST=$GITCACHE_HOST --build-arg GITCACHE_PORT=$GITCACHE_PORT -t alpine:crystal-shards . diff --git a/container/debian/build b/container/debian/build index 6bff579..eac9af2 100755 --- a/container/debian/build +++ b/container/debian/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:testbed" + docker build -t debian:testbed . diff --git a/container/delimiter/build b/container/delimiter/build index f7f53e7..606149d 100755 --- a/container/delimiter/build +++ b/container/delimiter/build @@ -4,6 +4,8 @@ . ../defconfig.sh +docker_skip_rebuild "delimiter" + load_cci_defaults docker build -t delimiter . diff --git a/container/dnsmasq/build b/container/dnsmasq/build index 8a22575..b5738bc 100755 --- a/container/dnsmasq/build +++ b/container/dnsmasq/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "dnsmasq:alpine" + [[ -e /tftpboot/boot.ipxe ]] || { cp tftpboot/boot.ipxe /tftpboot/boot.ipxe } diff --git a/container/dracut-initrd/build b/container/dracut-initrd/build index 181b896..f02cea9 100755 --- a/container/dracut-initrd/build +++ b/container/dracut-initrd/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:dracut" + docker build -t debian:dracut . diff --git a/container/es/build b/container/es/build index 17a225c..c7d7115 100755 --- a/container/es/build +++ b/container/es/build @@ -5,6 +5,8 @@ require_relative '../defconfig.rb' +docker_skip_rebuild "es643b:alpine311" + available_memory = get_available_memory system "docker build -t es643b:alpine311 --build-arg MEMORY=#{available_memory} --network=host ." diff --git a/container/extract-stats/build b/container/extract-stats/build index 1aa9982..c315f0c 100755 --- a/container/extract-stats/build +++ b/container/extract-stats/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "extract-stats" + CCI_SRC=$CCI_SRC bash $CCI_SRC/container/compile . diff --git a/container/fetch-mail/build b/container/fetch-mail/build index e72bed8..9b1a501 100755 --- a/container/fetch-mail/build +++ b/container/fetch-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "fetch-mail:latest" + docker build -t fetch-mail:latest . diff --git a/container/fluentd-base/build b/container/fluentd-base/build index 32e0a38..72b85bf 100755 --- a/container/fluentd-base/build +++ b/container/fluentd-base/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "fluentd-base:alpine" + system 'docker build -t fluentd-base:alpine .' diff --git a/container/git-daemon/build b/container/git-daemon/build index d81e4b9..5ba9a52 100755 --- a/container/git-daemon/build +++ b/container/git-daemon/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "apline311:git-daemon" + docker build -t apline311:git-daemon . diff --git a/container/git-mirror/build b/container/git-mirror/build index 66967e4..c302ac7 100755 --- a/container/git-mirror/build +++ b/container/git-mirror/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine38:git-mirror" + docker build -t alpine38:git-mirror . diff --git a/container/gitcache/build b/container/gitcache/build index 5135f7c..d7f3523 100755 --- a/container/gitcache/build +++ b/container/gitcache/build @@ -3,4 +3,8 @@ # Copyright (C) 2019-2020 Eric D.little51 # SPDX-License-Identifier: GPL-3.0 +. ../defconfig.sh + +docker_skip_rebuild "gitcache" + docker build -t gitcache . diff --git a/container/initrd-cifs/build b/container/initrd-cifs/build index 4b93647..19d5125 100755 --- a/container/initrd-cifs/build +++ b/container/initrd-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "initrd-cifs" + docker build -t initrd-cifs . diff --git a/container/initrd-http/build b/container/initrd-http/build index 1b039dc..22babcd 100755 --- a/container/initrd-http/build +++ b/container/initrd-http/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "initrd-http" + docker build -t initrd-http . diff --git a/container/kibana/build b/container/kibana/build index 52d5a2a..9a1fb6d 100755 --- a/container/kibana/build +++ b/container/kibana/build @@ -3,6 +3,10 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "kibana:7.6.2" + BASE_IMAGE_DICT = { 'aarch64' => 'gagara/kibana-oss-arm64:7.6.2', 'x86_64' => 'kibana:7.6.2' }.freeze diff --git a/container/lifecycle/build b/container/lifecycle/build index bfb7490..00a9c84 100755 --- a/container/lifecycle/build +++ b/container/lifecycle/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' +docker_skip_rebuild "lifecycle-#{LIFECYCLE_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "docker build -t lifecycle-#{LIFECYCLE_PORT} ." diff --git a/container/lkp-initrd/build b/container/lkp-initrd/build index 2383323..cfba617 100755 --- a/container/lkp-initrd/build +++ b/container/lkp-initrd/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:lkp" + docker build -t alpine:lkp . diff --git a/container/logging-es/build b/container/logging-es/build index 6465ace..3be841a 100755 --- a/container/logging-es/build +++ b/container/logging-es/build @@ -5,6 +5,8 @@ require_relative '../defconfig.rb' +docker_skip_rebuild "logging-es:7.6.2" + BASE_IMAGE_DICT = { 'aarch64' => 'gagara/elasticsearch-oss-arm64:7.6.2', 'x86_64' => 'elasticsearch:7.6.2' diff --git a/container/mail-robot/build b/container/mail-robot/build index c564add..4483e89 100755 --- a/container/mail-robot/build +++ b/container/mail-robot/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "mail-robot:latest" + docker build -t mail-robot:latest . diff --git a/container/manjaro/build b/container/manjaro/build index 114e550..6aeb3c6 100755 --- a/container/manjaro/build +++ b/container/manjaro/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "manjaro:testbed" + declare -A BASE_IMAGE_DICT BASE_IMAGE_DICT=( diff --git a/container/master-fluentd/build b/container/master-fluentd/build index 7d0291d..987cfcd 100755 --- a/container/master-fluentd/build +++ b/container/master-fluentd/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "master-fluentd:alpine" + system 'docker build -t master-fluentd:alpine .' diff --git a/container/minio/build b/container/minio/build index d1ccd06..b0909e9 100755 --- a/container/minio/build +++ b/container/minio/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "jessestuart/minio" + docker pull jessestuart/minio diff --git a/container/monitoring/build b/container/monitoring/build index bcb0c37..121ede0 100755 --- a/container/monitoring/build +++ b/container/monitoring/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) MONITOR_PORT = (defaults['MONITOR_PORT'] || '11310') +docker_skip_rebuild "monitoring-#{MONITOR_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "docker build -t monitoring-#{MONITOR_PORT} ." diff --git a/container/netdata-slave/build b/container/netdata-slave/build index 1a84550..cb53364 100755 --- a/container/netdata-slave/build +++ b/container/netdata-slave/build @@ -2,4 +2,8 @@ # Origin: https://hub.docker.com/r/netdata/netdata # SPDX-License-Identifier: GPL-3.0+ +. ../defconfig.sh + +docker_skip_rebuild "netdata/netdata-slave" + docker build -t netdata/netdata-slave . diff --git a/container/netdata/build b/container/netdata/build index 809b445..6aa9a17 100755 --- a/container/netdata/build +++ b/container/netdata/build @@ -2,4 +2,8 @@ # Origin: https://hub.docker.com/r/netdata/netdata # SPDX-License-Identifier: GPL-3.0+ +. ../defconfig.sh + +docker_skip_rebuild "netdata/netdatalkp" + docker build -t netdata/netdatalkp . diff --git a/container/ntp-server/build b/container/ntp-server/build index 7826fb2..d52f9dd 100755 --- a/container/ntp-server/build +++ b/container/ntp-server/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "ntp-server" + docker build -t ntp-server . diff --git a/container/openresty-proxy-cache/build b/container/openresty-proxy-cache/build index 507c4fe..7665209 100755 --- a/container/openresty-proxy-cache/build +++ b/container/openresty-proxy-cache/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "openresty:proxy_cache" + [ -d "openresty" ] && rm -rf openresty git clone https://gitee.com/cuiyili/openresty.git || exit 1 diff --git a/container/os-cifs/build b/container/os-cifs/build index 77be8e9..de44c44 100755 --- a/container/os-cifs/build +++ b/container/os-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-cifs" + docker build -t os-cifs . diff --git a/container/os-http/build b/container/os-http/build index c1cb008..367748e 100755 --- a/container/os-http/build +++ b/container/os-http/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-http" + docker build -t os-http . diff --git a/container/os-nfs/build b/container/os-nfs/build index eea5a7d..e0fd476 100755 --- a/container/os-nfs/build +++ b/container/os-nfs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "os-nfs" + docker build -t os-nfs . diff --git a/container/qcow2rootfs/build b/container/qcow2rootfs/build index 565de24..7f31bb4 100755 --- a/container/qcow2rootfs/build +++ b/container/qcow2rootfs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "qcow2rootfs:auto" + docker build -t qcow2rootfs:auto . diff --git a/container/qemu-efi/build b/container/qemu-efi/build index cac9426..4f1d488 100755 --- a/container/qemu-efi/build +++ b/container/qemu-efi/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:qemu-efi" + [ $(arch) == "aarch64" ] && uefi=qemu-efi-aarch64 [ $(arch) == "x86_64" ] && uefi=ovmf diff --git a/container/rabbitmq/build b/container/rabbitmq/build index c0bc179..930c112 100755 --- a/container/rabbitmq/build +++ b/container/rabbitmq/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "rabbitmq:3-management" + docker pull rabbitmq:3-management diff --git a/container/redis/build b/container/redis/build index e5230b0..490f99d 100755 --- a/container/redis/build +++ b/container/redis/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "redis507n:alpine311" + docker build -t redis507n:alpine311 . diff --git a/container/register-accounts/build b/container/register-accounts/build index e1dcfa4..2364b34 100755 --- a/container/register-accounts/build +++ b/container/register-accounts/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "register-accounts" + docker build -t register-accounts . diff --git a/container/registry/build b/container/registry/build index ca8680a..2edc2c1 100755 --- a/container/registry/build +++ b/container/registry/build @@ -2,4 +2,8 @@ # Origin: https://docs.docker.com/registry # SPDX-License-Identifier: Apache license +. ../defconfig.sh + +docker_skip_rebuild "registry" + docker pull registry diff --git a/container/remote-git/build b/container/remote-git/build index dad1a9b..727c7f9 100755 --- a/container/remote-git/build +++ b/container/remote-git/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "debian:remote-git" + docker build -t debian:remote-git . diff --git a/container/result-cifs/build b/container/result-cifs/build index d97b6dd..fd9d2df 100755 --- a/container/result-cifs/build +++ b/container/result-cifs/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "result-cifs" + docker build -t result-cifs . diff --git a/container/result-webdav/build b/container/result-webdav/build index 7d70ae6..b5be9e9 100755 --- a/container/result-webdav/build +++ b/container/result-webdav/build @@ -4,4 +4,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "result-webdav" + docker build -t result-webdav . diff --git a/container/rsync-server/build b/container/rsync-server/build index 72bfc47..5d36265 100755 --- a/container/rsync-server/build +++ b/container/rsync-server/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "rsync_server" + docker build -t rsync_server . diff --git a/container/scheduler-dev/build b/container/scheduler-dev/build index 1e4c57e..56d27c0 100755 --- a/container/scheduler-dev/build +++ b/container/scheduler-dev/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "alpine:scheduler-dev" + docker build -t alpine:scheduler-dev . diff --git a/container/scheduler/build b/container/scheduler/build index 367aa4e..412988a 100755 --- a/container/scheduler/build +++ b/container/scheduler/build @@ -11,6 +11,8 @@ defaults = relevant_defaults(names) SCHED_PORT = (defaults['SCHED_PORT'] || '3000') +docker_skip_rebuild "sch-ruby-a:v0.00d-#{SCHED_PORT}" + CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." system "cp #{CCI_SRC}/sbin/create-job-cpio.sh ." diff --git a/container/send-internet-mail/build b/container/send-internet-mail/build index 078adaf..07ca143 100755 --- a/container/send-internet-mail/build +++ b/container/send-internet-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "send-internet-mail:latest" + docker build -t send-internet-mail:latest . diff --git a/container/send-mail/build b/container/send-mail/build index 9159641..8ae9788 100755 --- a/container/send-mail/build +++ b/container/send-mail/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "send-mail:latest" + docker build -t send-mail:latest . diff --git a/container/shellcheck/build b/container/shellcheck/build index 854bae6..f295e51 100755 --- a/container/shellcheck/build +++ b/container/shellcheck/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "shellcheck" + docker build -t shellcheck . diff --git a/container/srv-http/build b/container/srv-http/build index 3a500ea..eb0dcbc 100755 --- a/container/srv-http/build +++ b/container/srv-http/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "srv-http" + docker build -t srv-http . diff --git a/container/ssh-r/build b/container/ssh-r/build index 4bec543..2ad1bd3 100755 --- a/container/ssh-r/build +++ b/container/ssh-r/build @@ -2,6 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "ssh-r:0.001" + if [ ! -f "known_auth_keys" ]; then echo "-----> ERROR: no testbox rsa public key file 'known_auth_keys' at currrent DIR" exit 1 diff --git a/container/sub-fluentd/build b/container/sub-fluentd/build index 797cd0d..1ec2246 100755 --- a/container/sub-fluentd/build +++ b/container/sub-fluentd/build @@ -3,4 +3,8 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require_relative '../defconfig' + +docker_skip_rebuild "sub-fluentd:alpine" + system 'docker build -t sub-fluentd:alpine .' diff --git a/container/submit/build b/container/submit/build index fba4281..dff884a 100755 --- a/container/submit/build +++ b/container/submit/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "submit:latest" + docker build -t submit:latest . diff --git a/container/taskqueue/build b/container/taskqueue/build index 3a5d4c9..4f69f85 100755 --- a/container/taskqueue/build +++ b/container/taskqueue/build @@ -10,6 +10,8 @@ load_cci_defaults service_port=${TASKQUEUE_PORT:-3060} +docker_skip_rebuild "taskqueue-alpine:v0.001-${service_port}" + if [[ ! -f "taskqueue" ]]; then $CCI_SRC/container/compile . else diff --git a/container/upload-libvirt-xml/build b/container/upload-libvirt-xml/build index d39838b..0a22173 100755 --- a/container/upload-libvirt-xml/build +++ b/container/upload-libvirt-xml/build @@ -4,6 +4,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "upload-libvirt-xml" + # The libvirt xml upload has nothing to do with the job. It renames the uploaded file and does not currently have permission to upload. # it is a transitional service. Two different functions use the same port service, diff --git a/container/web-backend-nginx/build b/container/web-backend-nginx/build index 4c9d3fe..d282570 100755 --- a/container/web-backend-nginx/build +++ b/container/web-backend-nginx/build @@ -2,4 +2,8 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "web-backend-nginx" + docker build -t web-backend-nginx . diff --git a/container/web-backend/build b/container/web-backend/build index 671fc97..c7f7d8a 100755 --- a/container/web-backend/build +++ b/container/web-backend/build @@ -1,4 +1,8 @@ #!/bin/sh # SPDX-License-Identifier: MulanPSL-2.0+ +. ../defconfig.sh + +docker_skip_rebuild "debian:web-backend" + docker build -t debian:web-backend . diff --git a/container/webhook/build b/container/webhook/build index bb5c0a7..a3475c4 100755 --- a/container/webhook/build +++ b/container/webhook/build @@ -2,5 +2,9 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +. ../defconfig.sh + +docker_skip_rebuild "node:webhook" + docker build -t node:webhook . -- 2.23.0

2 1

[PATCH lkp-tests] stats/ansible_test: stats result for ansible migrating
by Wang Chenglong 09 Feb '21

09 Feb '21

[How] ansible_stats_openeuler < ${job_output} Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 102 ++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 92 insertions(+), 10 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..556d4bcdd 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,97 @@ #!/usr/bin/env ruby +require 'json' + while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line + line = line.gsub(/'/, ' ') + ansible_stats_error = `echo '#{line}' |grep -E 'FAILED! |failed: .*}$' | awk -F'=>' '{$1="";print $0}'` + next if ansible_stats_error.empty? + + a = JSON.load(ansible_stats_error) + +# a['msg'].each_line do + unless a['msg'].nil? + + # When running some plug-in, the file was not found + if a['msg'].match(/No file was found when using (.+)\./) + puts "files.error.no-file-was-found-when-using.#{$1}: 1" + puts "files.error.no-file-was-found-when-using.#{$1}.message: #{ansible_stats_error}" + end + + # No such command + if a['msg'].match(/No such file or directory: b (.+) :/) + puts "command.error.no-such-file-or-directory.#{$1}: 1" + puts "command.error.no-such-file-or-directory.#{a['cmd']}.message: #{ansible_stats_error}" + end + + # The URL of the RPM is invalid + if a['msg'].match(/Failure downloading (.+), /) + puts "download.error.http-error-404 [#{$1}]: 1" + puts "download.error.http-error-404 [#{$1}]: #{ansible_stats_error}" + end + + # Failed to download the yum repodata + if a['msg'].match(/Failed to download metadata for repo (.+) :/) + puts "download.error.failed-to-download-metadata.#{$1}: 1" + puts "download.error.failed-to-download-metadata.#{$1}.message: #{ansible_stats_error}" + end + + if a['msg'].match(/Failed to download packages: Cannot download (.+):/) + puts "download.error.failed-to-download-package.#{$1}: 1" + puts "download.error.failed-to-download-package.#{$1}.message: #{ansible_stats_error}" + end + + # The task includes an option with an undefined variable. + if a['msg'].match(/: (.+) is undefined/) + puts "syntax.error.an-undefined-variable.#{$1}: 1" + puts "syntax.error.an-undefined-variable.#{$1}.message: #{ansible_stats_error}" + end + + # The command runing error. + if a['msg'].match(/non-zero return code/) + puts "command.error[#{a['cmd']}]: 1" + puts "command.error [#{a['cmd']}].message: #{ansible_stats_error}" + end + + # Some service could not fand the requested host. + if a['msg'].match(/Could not find the requested service (.+): host/) + puts "request.error.could-not-find-service-host.#{$1}: 1" + puts "request.error.could-not-find-service-host.#{$1}.messgae: #{ansible_stats_error}" + end + + # Unable to start some services + if a['msg'].match(/Unable to start service (.+):/) + puts "service.error.unable-to-star-service.#{$1}: 1" + puts "service.error.unable-to-star-service.#{$1}.message: #{ansible_stats_error}" + end + + # Some role in not sipported this system + if a['msg'].match(/(.+) is not supported/) + puts "system.error.not-supported.#{$1.gsub(/ /, '-')}: 1" + puts "system.error.not-supported.message: #{ansible_stats_error}" + end + + # failed to create temporary content file: The read operation timed out. + if a['msg'].match(/The read operation timed out/) + puts 'connection.error.read-timed-out: 1' + puts "read.error.read-timed-out.message: #{ansible_stats_error}" + end + end + + unless a['failures'].nil? + a['failures'].each do |items| + # Failed to install some of the specified packages. + items.match(/No package (.+) available/) + puts "yum.error.no-package-match.#{$1}: 1" + puts "yum.error.no-package-match.#{$1}.message: #{line}" + end + end + + next if a['message'].nil? + + # Could not find or access ansible yaml. + if a['message'].to_s.match(/Could not find or access (.+) /) + puts "yaml.error.not-find-yml.#{$1}: 1" + puts "yaml.error.not-find-yml.#{$1}.message: #{ansible_stats_error}" end end -- 2.23.0

1 1

[PATCH lab-z9] idle/*/multi-qemu.yaml: add disk
by Cui Yili 09 Feb '21

09 Feb '21

Signed-off-by: Cui Yili <2268260388(a)qq.com> --- .../multi-qemu.yaml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml b/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml index 513b631..8182a97 100644 --- a/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml +++ b/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml @@ -1,10 +1,9 @@ suite: multi-qemu category: functional -os: openeuler -os_version: 20.03 -os_arch: aarch64 -os_mount: initramfs +disk: 1HDD +fs: ext4 + simplify-ci: multi-qemu-0: nr_vm: 20 @@ -16,3 +15,8 @@ multi-qemu-2: nr_vm: 2 tbox_group: vm-1p1g sleep: 100d + +os: openeuler +os_version: 20.03 +os_arch: aarch64 +os_mount: initramfs -- 2.23.0

1 0

[PATCH lkp-tests] hosts: add dc-16g
by Cao Xueliang 09 Feb '21

09 Feb '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- hosts/dc-16g | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 hosts/dc-16g diff --git a/hosts/dc-16g b/hosts/dc-16g new file mode 100644 index 000000000..1f16373e6 --- /dev/null +++ b/hosts/dc-16g @@ -0,0 +1,3 @@ +nr_cpu: 16 +memory: 16g + -- 2.23.0

1 0

[PATCH v2 compass-ci] sbin/es-find: support count keyword
by Lu Weitao 09 Feb '21

09 Feb '21

[Example] es-find tbox_group=vm-2p8g -c/--count suite,job_state { "iperf": { "finished": 50, "failed": 40, "incomplete": 4, "boot": 3 }, "iperf-walk-os-test": { "finished": 167, "failed": 22 }, "borrow": { "finished": 47, "incomplete": 17, "failed": 2 }, ... } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/es-find | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/sbin/es-find b/sbin/es-find index ed54955..2c3b3dc 100755 --- a/sbin/es-find +++ b/sbin/es-find @@ -6,6 +6,7 @@ # Usage: # es-find key=val ... # es-find -r "start_time=2021-1-1 00:00:00,2021-1-1 23:59:59;..." +# es-find tbox_group=vm-2p8g -c suite,job_state # if given multi keys, they create AND conditions # if given val1,val2,... for a key, they create OR conditions # Eg: query (testbox is xxx or yyy) and (suite is aaa) and (user is bbb). @@ -74,6 +75,7 @@ end options = { nr_jobs: 10, desc_keyword: 'start_time' } range = nil +count = nil opt_parser = OptionParser.new do |opts| opts.banner = 'Usage: es-find [options] search_key1=val1[,val2..] ..' @@ -97,6 +99,10 @@ opt_parser = OptionParser.new do |opts| range = r end + opts.on('-c', '--count KEYWORD', 'count keyword: suite|job_state|testbox|id|...') do |c| + count = c + end + opts.on_tail('-h', '--help', 'show this message') do puts opts exit @@ -108,6 +114,11 @@ items = parse_argv items[:range] = prase_range(range) if range es = ESQuery.new(ES_HOST, ES_PORT) -query_result = es.multi_field_query(items, size: options[:nr_jobs], desc_keyword: options[:desc_keyword]) +if count + count_keywords = count.split(',') + query_result = es.query_fields(count_keywords, items) +else + query_result = es.multi_field_query(items, size: options[:nr_jobs], desc_keyword: options[:desc_keyword]) +end json_string = JSON.pretty_generate(query_result) puts json_string -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_bisect_result: fix PKGBUILD url 404
by Cao Xueliang 09 Feb '21

09 Feb '21

[error] http://api.compass-ci.openeuler.org:11300/archlinux/aur-v/vifm-git 404 [how] Need the add git to the url. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 5e1103c..d43785b 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -36,7 +36,7 @@ class MailBisectResult prefix_srv = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_PORT}" bisect_job_url = ENV['result_root'] ? "bisect job result directory:\n#{prefix_srv}#{ENV['result_root']}\n" : '' bisect_report_doc = "bisect email doc:\nhttps://gitee.com/wu_fengguang/compass-ci/blob/master/doc/bisect_emai…" - pkgbuild_repo_url = "PKGBUILD:\n#{prefix_srv}/#{@pkgbuild_repo}\n" + pkgbuild_repo_url = "PKGBUILD:\n#{prefix_srv}/git/#{@pkgbuild_repo}\n" first_bad_commit_job_url = "first bad commit job result directory:\n#{prefix_srv}#{@first_bad_commit_result_root}\n" data = <<~BODY -- 2.23.0

1 0

[PATCH lkp-tests] setup/simply-ci: fix failed to git clone
by Liu Yinsi 09 Feb '21

09 Feb '21

[error] Cloning into '/c/compass-ci'... fatal: unable to look up (port 9418) (Name or service not known) [how] before: git clone git://$GIT_SERVER/... after: git clone https://$GIT_SERVER/... Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- setup/simplify-ci | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup/simplify-ci b/setup/simplify-ci index df18272f3..076c357d9 100755 --- a/setup/simplify-ci +++ b/setup/simplify-ci @@ -14,7 +14,7 @@ git_ci() EOF fi - git clone git://$GIT_SERVER/gitee.com/wu_fengguang/compass-ci.git /c/compass-ci + git clone https://$GIT_SERVER/gitee.com/wu_fengguang/compass-ci.git /c/compass-ci } dev_env() -- 2.23.0

1 0

[PATCH compass-ci] docker/docker.rb: use curl instead of wget
by Lin Jiaxin 09 Feb '21

09 Feb '21

wget -q will turn off wget's output, even error. curl -sS can disable progress meter but still show error messages. So use curl -sS instead of wget -q. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- providers/docker/docker.rb | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/providers/docker/docker.rb b/providers/docker/docker.rb index 56c9754..f4e954e 100755 --- a/providers/docker/docker.rb +++ b/providers/docker/docker.rb @@ -49,8 +49,9 @@ def parse_response(url) return hash end -def wget_cmd(path, url, name) - system "wget -q -P #{path} #{url} && gzip -dc #{path}/#{name} | cpio -id -D #{path}" + +def curl_cmd(path, url, name) + system "curl -sS --create-dirs -o #{path}/#{name} #{url} && gzip -dc #{path}/#{name} | cpio -id -D #{path}" end def build_load_path(hostname) @@ -75,8 +76,8 @@ def load_initrds(load_path, hash) arch = RUBY_PLATFORM.split('-')[0] job_url = hash['job'] lkp_url = hash['lkp'] - wget_cmd(load_path, job_url, 'job.cgz') - wget_cmd(load_path, lkp_url, "lkp-#{arch}.cgz") + curl_cmd(load_path, job_url, 'job.cgz') + curl_cmd(load_path, lkp_url, "lkp-#{arch}.cgz") end def start_container(hostname, load_path, hash) -- 2.23.0

2 1

[PATCH compass-ci] sbin/auto_submit: add support for different branch different job
by Li Yuanchao 08 Feb '21

08 Feb '21

Now can support some branch use specific submit commands and others use common submit commands for the same repo. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 78 ++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 62 insertions(+), 16 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 410445d..6d29543 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -30,21 +30,6 @@ class AutoSubmit return pkgbuild_repo end - def submit(newrefs_info, submit_argv) - newrefs_info['new_refs']['heads'].each do |branch, commit_id| - inactive_time = %x(git -C /srv/git/#{newrefs_info['git_repo']}.git log --pretty=format:"%cr" -1 #{commit_id}) - next if inactive_time =~ /(month|year)/ - - real_argvs = Array.new(submit_argv) - real_argvs.push("upstream_branch=#{branch.delete_prefix('refs/heads/')}") - real_argvs.push("upstream_commit=#{commit_id}") - tag = %x(git -C /srv/git/#{newrefs_info['git_repo']}.git tag --points-at #{commit_id}) - real_argvs.push("upstream_tag=#{tag}") unless tag.empty? - - system(real_argvs.join(' ')) - end - end - def get_argvs(newrefs_info) git_repo = newrefs_info['git_repo'] puts "git_repo : #{git_repo}" @@ -60,16 +45,77 @@ class AutoSubmit submit_argv end + def submit_one_job(submit_argv, git_repo, branch, commit_id) + inactive_time = %x(git -C /srv/git/#{git_repo}.git log --pretty=format:"%cr" -1 #{commit_id}) + return if inactive_time =~ /(month|year)/ + + real_argvs = Array.new(submit_argv) + real_argvs.push("upstream_branch=#{branch}") + real_argvs.push("upstream_commit=#{commit_id}") + tag = %x(git -C /srv/git/#{git_repo}.git tag --points-at #{commit_id}) + real_argvs.push("upstream_tag=#{tag}") unless tag.empty? + + system(real_argvs.join(' ')) + end + + def get_branch_commands(submit_commands) + submit_commands.each do |element| + return element['branch_commands'] if element['branch_commands'] + end + return nil + end + + def submit_specific_branch(submit_argv, newrefs_info, branch_commands) + return newrefs_info['new_refs']['heads'] if branch_commands.nil? + + non_specific_commits = {} + newrefs_info['new_refs']['heads'].each do |branch, commit_id| + branch_name = branch.delete_prefix('refs/heads/') + if branch_commands[branch_name].nil? + non_specific_commits[branch] = commit_id + next + end + argvs = Array.new(submit_argv) + argvs.push(branch_commands[branch_name]) + submit_one_job(argvs, newrefs_info['git_repo'], branch_name, commit_id) + end + return non_specific_commits + end + + def submit_non_specific(non_specific_commits, git_repo, submit_argv) + non_specific_commits.each do |branch, commit_id| + branch = branch.delete_prefix('refs/heads/') + submit_one_job(submit_argv, git_repo, branch, commit_id) + end + end + + # Add support of different branch different job + # newrefs_info['submit'] is like: + # ["submit command 1", + # "submit command 2", + # ..., + # { "branch_commands" => { "branch_1" => "submit command 1", + # ..., + # "branch_n" => "submit command n" + # } + # } + # ] def submit_job(newrefs_info) submit_argv = get_argvs(newrefs_info) return unless submit_argv return unless newrefs_info['submit'] + branch_commands = get_branch_commands(newrefs_info['submit']) + non_specific_commits = submit_specific_branch(submit_argv, newrefs_info, branch_commands) + return if non_specific_commits.empty? + newrefs_info['submit'].each do |argv_config| + next if argv_config.is_a? Hash + argvs = Array.new(submit_argv) argvs.push(argv_config) - submit(newrefs_info, argvs) + submit_non_specific(non_specific_commits, newrefs_info['git_repo'], argvs) end end -- 2.23.0

1 0

[PATCH v4 compass-ci 1/3] container/defconfig: add function docker_skip_rebuild
by Liu Yinsi 08 Feb '21

08 Feb '21

When execute `sparrow/4-docker/buildall run-only`, if docker image already exists, just exit build script, not rebuild container. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.rb | 6 ++++++ container/defconfig.sh | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/container/defconfig.rb b/container/defconfig.rb index f9b0824..7dc9f33 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -47,3 +47,9 @@ def get_available_memory # take the middle value according to the system memory size. [1024, 30720, Math.sqrt(memtotal) * 1024].sort[1].to_i end + +def docker_skip_rebuild(tag) + return unless ENV['action'].include? 'run-only' + + exit 1 if system "docker image inspect #{tag} > /dev/null 2>&1" +end diff --git a/container/defconfig.sh b/container/defconfig.sh index 54e65b5..7c575ef 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -41,3 +41,11 @@ push_image() docker tag "$src_tag" "$dst_tag" docker push "$dst_tag" } + +docker_skip_rebuild() +{ + tag=$1 + [ "$action" != "run-only" ] && return + docker image inspect $tag > /dev/null 2>&1 + [ "$?" == "0" ] && exit 1 +} -- 2.23.0

1 0

[PATCH v4 compass-ci] container/register-accounts: fix failed to register account
by Liu Yinsi 08 Feb '21

08 Feb '21

[error] root@taishan200-2280-2s64p-256g--a9 /c/compass-ci/container/register-accounts# ./start register-accounts register-accounts /c/compass-ci/container/register-accounts # ./register-accounts awk: /etc/compass-ci/defaults/*.yaml: No such file or directory "option: {\"my_email\"=>\"\", \"my_name\"=>\"\", \"lab\"=>\"\"}" Traceback (most recent call last): 10: from ./register-accounts:8:in `<main>' 9: from ./register-accounts:8:in `require_relative' 8: from /c/compass-ci/sbin/build-my-info.rb:57:in `<top (required)>' 7: from /c/compass-ci/sbin/build-my-info.rb:48:in `build_my_info' 6: from /c/compass-ci/lib/build_my_info_client.rb:30:in `search_my_info' 5: from /c/compass-ci/lib/es_query.rb:58:in `query_by_id' 4: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-api-7.10.1/lib/elasticsearch/api/actions/search.rb:103:in `search' 3: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/client.rb:182:in `perform_request' 2: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/http/faraday.rb:37:in `perform_request' 1: from /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:347:in `perform_request' /usr/lib/ruby/gems/2.7.0/gems/elasticsearch-transport-7.10.1/lib/elasticsearch/transport/transport/base.rb:218:in `__raise_transport_error': [400] {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"MH_AlDBvTYa9sQhwsESXgQ","index":"accounts"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"accounts","node":"I1usO2axQ9-XZhzcX3oHxw","reason":{"type":"query_shard_exception","reason":"failed to create query: {\\n \\"bool\\" : {\\n \\"must\\" : [\\n {\\n \\"term\\" : {\\n \\"_id\\" : {\\n \\"value\\" : \\"\\",\\n \\"boost\\" : 1.0\\n }\\n }\\n }\\n ],\\n \\"adjust_pure_negative\\" : true,\\n \\"boost\\" : 1.0\\n }\\n}","index_uuid":"MH_AlDBvTYa9sQhwsESXgQ","index":"accounts","caused_by":{"type":"illegal_argument_exception","reason":"Ids can't be empty"}}}]},"status":400} (Elasticsearch::Transport::Transport::Errors::BadRequest) refer code: /c/compass-ci/sbin/build-my-info.rb: ... option = { 'my_email' => `git config --global user.email`.chomp, 'my_name' => `git config --global user.name`.chomp, 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp } ... build_my_info(option) ... [why] when call build-my-info.rb, we just want call function build_my_info(), no need to execute option, in caller script already define option values [how] use `if $PROGRAM_NAME == __FILE__`, when sbin/build-my-info.rb is not executed independently but invoked by other files, avoid to execute some part code. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/build-my-info.rb | 72 ++++++++++++++++++++++--------------------- 1 file changed, 37 insertions(+), 35 deletions(-) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index dbe093d..c7a7798 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -7,49 +7,51 @@ require_relative '../lib/es_client' require_relative '../lib/build_my_info_client' require 'optparse' -option = { - 'my_email' => `git config --global user.email`.chomp, - 'my_name' => `git config --global user.name`.chomp, - 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp -} +def build_my_info(option) + build_my_info = BuildMyInfo.new(option['my_email']) + info_es = build_my_info.search_my_info + info_es.update option + info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? -options = OptionParser.new do |opts| - opts.banner = "Usage: build-my-info [-e email] [-n name] [-l lab] [-t]\n" + build_my_info.config_my_info(info_es) +end - opts.separator '' - opts.separator 'options:' +if $PROGRAM_NAME == __FILE__ + option = { + 'my_email' => `git config --global user.email`.chomp, + 'my_name' => `git config --global user.name`.chomp, + 'lab' => `awk '/^lab:\s/ {print $2; exit}' /etc/compass-ci/defaults/*.yaml`.chomp + } - opts.on('-e email', 'my_email') do |email| - option['my_email'] = email - end + options = OptionParser.new do |opts| + opts.banner = "Usage: build-my-info [-e email] [-n name] [-l lab] [-t]\n" - opts.on('-n name', 'my_name') do |name| - option['my_name'] = name - end + opts.separator '' + opts.separator 'options:' - opts.on('-l lab', 'lab') do |lab| - option['lab'] = lab - end + opts.on('-e email', 'my_email') do |email| + option['my_email'] = email + end - opts.on('-t', 'my_token') do - option['my_token'] = %x(uuidgen).chomp - end + opts.on('-n name', 'my_name') do |name| + option['my_name'] = name + end - opts.on_tail('-h', 'show this message') do - puts opts - exit - end -end + opts.on('-l lab', 'lab') do |lab| + option['lab'] = lab + end -options.parse! + opts.on('-t', 'my_token') do + option['my_token'] = %x(uuidgen).chomp + end -def build_my_info(option) - build_my_info = BuildMyInfo.new(option['my_email']) - info_es = build_my_info.search_my_info - info_es.update option - info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? + opts.on_tail('-h', 'show this message') do + puts opts + exit + end + end - build_my_info.config_my_info(info_es) -end + options.parse! -build_my_info(option) + build_my_info(option) +end -- 2.23.0

1 0

[PATCH lkp-tests] sbin/install-dependencies.sh: delete redundant code
by Wang Yong 08 Feb '21

08 Feb '21

write_shell_profile function will source profile, no need source twice Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index ed8aeb927..b687dc0d6 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -65,7 +65,6 @@ set_env() write_host write_shell_profile "export LKP_SRC=$PWD" write_shell_profile "export PATH=\$PATH:\$LKP_SRC/sbin:\$LKP_SRC/bin" - source $(shell_profile) } set_env -- 2.23.0

1 0

[PATCH v3 lkp-tests] sbin/install-dependencies.sh: change install way
by Wang Yong 08 Feb '21

08 Feb '21

[before] install packages from $DISTRO detect [after] install packages from Package Manager it's not convenient when install packages in a new $DISTRO Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- sbin/install-dependencies.sh | 32 ++++++++++++++++++++------------ 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/sbin/install-dependencies.sh b/sbin/install-dependencies.sh index b5b070f01..ed8aeb927 100755 --- a/sbin/install-dependencies.sh +++ b/sbin/install-dependencies.sh @@ -5,28 +5,36 @@ PROJECT_DIR=$(dirname $SCRIPT_DIR) . $PROJECT_DIR/lib/env.sh -# choose install function base on DISTRIBUTION +# choose install function base on common Package Manager linux_dep() { - source /etc/os-release - case $ID in - ubuntu|debian) + get_package_manager + + case "$installer" in + apt-get) export DEBIAN_FRONTEND=noninteractive - sudo apt-get install -yqm ruby-dev libssl-dev gcc g++ uuid-runtime + sudo "$installer" install -yqm ruby-dev libssl-dev gcc g++ uuid-runtime + ;; + dnf|yum) + sudo "$installer" install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux ;; - openEuler|fedora|rhel|centos) - if type dnf > /dev/null 2>&1; then - sudo dnf install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux - else - sudo yum install -y --skip-broken ruby rubygems gcc gcc-c++ make ruby-devel git lftp util-linux - fi + pacman) + sudo "$installer" -Sy --noconfirm --needed ruby rubygems gcc make git lftp util-linux ;; *) - echo "$ID not support! please install dependencies manually." && exit 1 + echo "Unknown Package Manager! please install dependencies manually." && exit 1 ;; esac } +get_package_manager() +{ + has_cmd "yum" && installer="yum" + has_cmd "dnf" && installer="dnf" && return + has_cmd "apt-get" && installer="apt-get" && return + has_cmd "pacman" && installer="pacman" && return +} + mac_dep() { brew install ruby -- 2.23.0

2 2

[PATCH compass-ci] sbin/build-my-info: fix no key: my_ssh_pubkey
by Luan Shengde 08 Feb '21

08 Feb '21

[why] my_ssh_pubkey is needed sometimes when submit jobs: $ submit -m -c job.yaml -i ~/lkp-tests/jobs/ssh-on-fail.yaml submit_id=f87d2951-c226-4e51-a3e4-c71818a4c974 submit job.yaml failed, got job id=0, error: Missing hash key: "my_ssh_pubkey" [how] add initialize my_ssh_pubkey when building my_info. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index dbe093d..6be08a4 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -48,6 +48,7 @@ def build_my_info(option) info_es = build_my_info.search_my_info info_es.update option info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? + info_es['my_ssh_pubkey'] = [] build_my_info.config_my_info(info_es) end -- 2.23.0

1 0

[PATCH v2 lkp-tests] lib/common.rb: build_pkg_error_id function add filtering rule
by Lin Jiaxin 08 Feb '21

08 Feb '21

Some software use tempfile when build. The directory is random, thus every error will be a new error_id. In fact, they are same. Now remove the random part except /tmp/lkp (makepkg argument: BUILDDIR). /tmp/gitstatus-build.BOpUcJeETC/libgit2/src/futils.c:1143:16: warning: comparison of integer expressions of different signedness: 'uint64_t' {aka 'const long unsigned int'} and 'long int' [-Wsign-compare] => libgit2/src/futils.c:warning:comparison-of-integer-expressions-of-different-signedness:'uint64_t'{aka'const-long-unsigned-int'}and'long-int'[-Wsign-compare] Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- lib/common.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/common.rb b/lib/common.rb index cc64eb17..7deee62e 100755 --- a/lib/common.rb +++ b/lib/common.rb @@ -339,5 +339,6 @@ def build_pkg_error_id(line) line = line.tr(" ", "-") line = line.gsub(/[-_.,;:#!\[(]+$/, "") line = line.gsub(/([-_.,;:#!]){3,}/, ":") + line = line.gsub(/^\/tmp\/(?!lkp).*?\//, "") line end -- 2.23.0

1 0

[PATCH lab-z9] add host-info of b5
by Zhang Dewan 08 Feb '21

08 Feb '21

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- hosts/2288hv5-2s44p-384g--b5 | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 hosts/2288hv5-2s44p-384g--b5 diff --git a/hosts/2288hv5-2s44p-384g--b5 b/hosts/2288hv5-2s44p-384g--b5 new file mode 100644 index 0000000..5dbd623 --- /dev/null +++ b/hosts/2288hv5-2s44p-384g--b5 @@ -0,0 +1,14 @@ +memory: 384G +nr_ssd_partitions: 1 +ssd_partitions: + - /dev/disk/by-id/ata-MTFDDAK480TDC-1AT1ZABYY_20032653CBD6 +mac_addr: + - 6c:eb:b6:50:c0:a9 + - 6c:eb:b6:50:c0:aa + - 64:2c:ac:de:ce:51 + - 64:2c:ac:de:ce:d3 +arch: x86_64 +nr_node: 2 +nr_cpu: 88 +model_name: Intel(R) Xeon(R) Gold 6266C CPU @ 3.00GHz +ipmi_ip: 9.3.3.10 -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends/cci-makepkg: add git depend
by Wei Jihui 08 Feb '21

08 Feb '21

user maybe use git cmd when write PKGBUILD Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/cci-makepkg | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/depends/cci-makepkg b/distro/depends/cci-makepkg index 4ed4f19e..855ac8ce 100644 --- a/distro/depends/cci-makepkg +++ b/distro/depends/cci-makepkg @@ -1,3 +1,4 @@ +git curl gnupg gettext -- 2.23.0

1 0

[PATCH lkp-tests] add tests script for openeuler os migration
by Bai Jing 08 Feb '21

08 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- tests/openeuler_docker | 113 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100755 tests/openeuler_docker diff --git a/tests/openeuler_docker b/tests/openeuler_docker new file mode 100755 index 000000000..69d8a58a6 --- /dev/null +++ b/tests/openeuler_docker @@ -0,0 +1,113 @@ +#!/bin/sh +# - dockerfile_url + +. $LKP_SRC/lib/debug.sh + +[ -n "$dockerfile_url" ] || die "dockerfile_url is empty " + +cd $BENCHMARK_ROOT + +fit_repo() +{ + yum reinstall -y git + git clone "$dockerfile_url" || exit 0 + file_dir=$(find ./ -type f -name Dockerfile | xargs grep -E '^FROM centos:(centos|)[78]') + [ -n "$file_dir" ] || exit 0 +} + +openeuler_ready() +{ + docker load < $BENCHMARK_ROOT/openeuler_docker/openeuler-20.03-lts-v1.tar +} + +centos_docker() +{ + flag=0 + for file in $(find ./ -name Dockerfile | xargs grep -E '^FROM centos:(centos|)[78]'| awk -F: '{print $1}') + do + file=${file#*/} + image=$(echo "$file" | awk -F'/' '{print $2}' | tr 'A-Z' 'a-z') + + ( + cd $BENCHMARK_ROOT/${file%/*} + if [ -f "build.sh" ] && [ -x "build.sh" ] + then + ./build.sh > /dev/null 2> /dev/null && handle_dockerfile + else + docker_build > /dev/null 2> /dev/null && handle_dockerfile + fi + ) + done +} + +handle_FROM() +{ + sed -i "s/^FROM .*/FROM openeuler-20.03-lts:v1/g" Dockerfile + +} + +handle_epel() +{ + grep -qw "epel-release" Dockerfile && { + sed -i "s/epel-release/bash/g" Dockerfile + } +} + +handle_epelrpm() +{ + + grep -qw "https://.*epel-release.*rpm" Dockerfile && { + sed -i "s|https://.*epel-release.*rpm|bash|g" Dockerfile + + } +} + +handle_groupadd() +{ + grep -qw "groupadd*" Dockerfile && { + sed -i '/groupadd*/i\ yum -y install shadow && \\' Dockerfile + } + +} + +handle_dockerfile() +{ + flag=1 + handle_FROM + handle_epel + handle_groupadd + docker_build "-openeuler" + [ $? -ne 0 ] || docker_run "-openeuler" +} + +docker_build() +{ + docker build -t "${image}$1" . + [ $? -ne 0 ] && { + [ "$flag" == "1" ] && echo "${image}$1.build.fail: 1" + return 1 + } + [ "$flag" == "1" ] && echo "${image}$1.build.success: 1" + flag=0 +} + +docker_run() +{ + docker run -d --name "${image}" "${image}$1" + docker_stats=$(docker inspect ${image} | grep Running | awk -F ' ' '{print $2}' | tr -d ',') + [ $docker_stats == false ] && { + echo "${image}.run.fail: 1" + return 1 + } + + echo "${image}.run.success: 1" +} + +docker_test() +{ + fit_repo + openeuler_ready + centos_docker +} + +docker_test -- 2.23.0

2 1

[PATCH compass-ci] answerback-email: add store lab to ES
by Luan Shengde 08 Feb '21

08 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 8dceee1..e2672aa 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -177,9 +177,9 @@ options.parse!(ARGV) def apply_account(my_info, conf_info) apply_info = {} + my_info['lab'] = LAB apply_info.update my_info apply_info.update conf_info - apply_info['lab'] = LAB assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" account_info_str = RestClient.post assign_account_url, apply_info.to_json -- 2.23.0

1 0

[PATCH v3 compass-ci 3/3] lib/job.cr: refactor the matching string action of os_mount
by Xu Xijian 08 Feb '21

08 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..b724b07 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -509,7 +509,7 @@ class Job if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) - elsif "#{os_mount}" == "nfs" || "#{os_mount}" == "cifs" || "#{os_mount}" == "local" + elsif ["nfs", "cifs", "local"].includes? "#{os_mount}" temp_initrds.concat(nfs_cifs_initrds()) end -- 2.23.0

1 0

[PATCH compass-ci] sbin/es-find: support count keyword
by Lu Weitao 08 Feb '21

08 Feb '21

[Example] es-find tbox_group=vm2p8g -c/--count suite,job_state { "iperf": { "finished": 50, "failed": 40, "incomplete": 4, "boot": 3 }, "iperf-walk-os-test": { "finished": 167, "failed": 22 }, "borrow": { "finished": 47, "incomplete": 17, "failed": 2 }, ... } Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/es-find | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/sbin/es-find b/sbin/es-find index ed54955..2c3b3dc 100755 --- a/sbin/es-find +++ b/sbin/es-find @@ -6,6 +6,7 @@ # Usage: # es-find key=val ... # es-find -r "start_time=2021-1-1 00:00:00,2021-1-1 23:59:59;..." +# es-find tbox_group=vm-2p8g -c suite,job_state # if given multi keys, they create AND conditions # if given val1,val2,... for a key, they create OR conditions # Eg: query (testbox is xxx or yyy) and (suite is aaa) and (user is bbb). @@ -74,6 +75,7 @@ end options = { nr_jobs: 10, desc_keyword: 'start_time' } range = nil +count = nil opt_parser = OptionParser.new do |opts| opts.banner = 'Usage: es-find [options] search_key1=val1[,val2..] ..' @@ -97,6 +99,10 @@ opt_parser = OptionParser.new do |opts| range = r end + opts.on('-c', '--count KEYWORD', 'count keyword: suite|job_state|testbox|id|...') do |c| + count = c + end + opts.on_tail('-h', '--help', 'show this message') do puts opts exit @@ -108,6 +114,11 @@ items = parse_argv items[:range] = prase_range(range) if range es = ESQuery.new(ES_HOST, ES_PORT) -query_result = es.multi_field_query(items, size: options[:nr_jobs], desc_keyword: options[:desc_keyword]) +if count + count_keywords = count.split(',') + query_result = es.query_fields(count_keywords, items) +else + query_result = es.multi_field_query(items, size: options[:nr_jobs], desc_keyword: options[:desc_keyword]) +end json_string = JSON.pretty_generate(query_result) puts json_string -- 2.23.0

1 0

[PATCH compass-ci] lib/stats.rb: use Set save stat program
by Lu Weitao 08 Feb '21

08 Feb '21

[Why] we use some key of job.yaml as stat program, its may duplicate with default stat program for example: job.yaml include suite: openeuler_docker openeuler_docker: xxx we always insert $suite to stats_list so the 'openeuler_docker' will be extracted twice [How] use Set replace to Array Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/stats.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/stats.rb b/lib/stats.rb index acd807a..c01d38d 100644 --- a/lib/stats.rb +++ b/lib/stats.rb @@ -43,7 +43,7 @@ class Stats end def assign_stats_list - stats_list = [] + stats_list = Set.new stats_list << @job['suite'] stats_list << ['time', @job['suite'] + '.time'] stats_list << 'stderr' -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] lib/job.cr: refactor the matching string action of os_mount
by Xu Xijian 08 Feb '21

08 Feb '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/lib/job.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 89c92c1..597c88c 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -509,7 +509,7 @@ class Job if "#{os_mount}" == "initramfs" temp_initrds.concat(initramfs_initrds()) - elsif "#{os_mount}" == "nfs" || "#{os_mount}" == "cifs" || "#{os_mount}" == "local" + elsif ["nfs", "cifs", "local"].index "#{os_mount}" temp_initrds.concat(nfs_cifs_initrds()) end -- 2.23.0

2 2

[PATCH compass-ci] container/dracut-initrd: fix: awk: command not found
by Xu Xijian 08 Feb '21

08 Feb '21

[Why] When start an os, it will output awk: command not found in dracut stage. See errlog, this happens in 40network module, so let's install awk. [errlog] [ 7.417877] dracut-initqueue[368]: /lib/net-lib.sh: line 125: awk: command not found Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index f7119ec..8f03fa8 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -16,3 +16,4 @@ rm -rf /var/lib/apt/lists/* cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh +sed -i "/install() {/ainst /usr/bin/awk" /usr/lib/dracut/modules.d/40network/module-setup.sh -- 2.23.0

2 2

[PATCH compass-ci] doc/manual/borrow-machine: improve readability
by Li Ping 08 Feb '21

08 Feb '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- doc/manual/borrow-machine.en.md | 18 +++++++++--------- doc/manual/borrow-machine.zh.md | 18 +++++++++--------- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index 7e66f40..72bcdcb 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -64,24 +64,24 @@ If you have not completed above works, reference to [apply-account.md](https://g # FAQ -## Customize the borrowing time +* Customize the borrowing time Find key in the yaml file and edit its value according to your requirement. - - The borrowing period can be calculated in days and hours. - - The maximum period is no more than 10 day. + The borrowing period can be calculated in days and hours. + The maximum period is no more than 10 day. -## Guidance for command submit +* Guidance for command submit See the usage and options for command 'submit' the command below: - submit -h + submit -h Reference the following line to learn the advanced usage for command 'submit': + + [submit detailed usage](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/sub… - [submit detailed usage](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/sub… - -## Available testbox +* Available testbox For a full list of testbox, reference to https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts @@ -96,6 +96,6 @@ If you have not completed above works, reference to [apply-account.md](https://g > - You will need to wait if there are already tasks in the task queue for the machine. > - Your job will be randomly assigned to a machine that meets the requirements if the testbox name is not end with '-axx'. -## Specify the OS +* Specify the OS About supportted `os`, `os_arch`, `os_version`, reference to [os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/ma… diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 73aea6b..c19d8a5 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -3,7 +3,7 @@ - 申请account - 配置默认yaml文件 -如果未完成以上步骤，请参考[apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.md)完成操作。 +如果未完成以上步骤，请参考 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… 完成操作。 # 申请测试机 @@ -62,14 +62,14 @@ # FAQ -## 自定义借用时长 +* 自定义借用时长 - 在借用机器的yaml实例文件中，找到’runtime‘字段，根据需求修改借用时长。 + 在借用机器的yaml实例文件中，找到 **runtime** 字段，根据需求修改借用时长。 - - 借用机器可以按照小时（h）、天（d）来计算。 - - 借用时长最多不超过10天。 + 借用机器可以按照小时（h）、天（d）来计算。 + 借用时长最多不超过10天。 -## submit命令指导 +* submit命令指导学习submit命令，您可以使用以下命令查看submit命令的各项参数及使用方法： @@ -77,9 +77,9 @@ 也可以参考submit命令手册学习submit命令高级用法： - [submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… + [submit命令详解](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… -## 可选的testbox +* 可选的testbox 查看可选的testbox列表，请参考：https://gitee.com/wu_fengguang/lab-z9/tree/master/hosts @@ -95,6 +95,6 @@ > - 物理机的testbox若选择以`--axx`结尾的，则表示指定到了具体的某一个物理机。若此物理机任务队列中已经有任务在排队，则需要等待队列中前面的任务执行完毕后，才会轮到你提交的borrow任务。 > - 物理机的testbox若不选择以`-axx`结尾的，表示不指定具体的某一个物理机。则此时集群中的空闲物理机会即时被分配执行你的borrow任务。 -## 如何 borrow 指定的操作系统 +* 如何 borrow 指定的操作系统关于支持的`os`, `os_arch`, `os_version`，参见：[os-os_verison-os_arch.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/job/os-os_verison-os_arch.md) -- 2.23.0

1 0