- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] job.cr: add key "kernel_version" for initramfs
by Xu Xijian 29 Oct '20

29 Oct '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..c8b9dc6 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -85,6 +85,7 @@ class Job kernel_append_root kernel_params docker_image + kernel_version ) macro method_missing(call) @@ -281,8 +282,9 @@ class Job end private def set_kernel_uri + vmlinuz_sub_path = "#{kernel_version}" == "default" ? "vmlinuz" : "boot/vmlinuz-#{kernel_version}" self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/#{vmlinuz_sub_path}")}" end private def kernel_common_params @@ -303,10 +305,16 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" + self["kernel_version"] ||= "default" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-#{kernel_version}.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-#{kernel_version}.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? -- 2.23.0

2 1

[PATCH v2 compass-ci] container/logging-es: optimize bulk write performance
by Wu Zhende 29 Oct '20

29 Oct '20

[Why] fluentd uses "bulk" to continuously write data to es. The es write capability needs to be optimized to adapt to high-concurrency scenarios. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/logging-es/Dockerfile | 12 ++++++++---- container/logging-es/start | 12 ++++++++++++ 2 files changed, 20 insertions(+), 4 deletions(-) diff --git a/container/logging-es/Dockerfile b/container/logging-es/Dockerfile index 7f91f65..7199cba 100644 --- a/container/logging-es/Dockerfile +++ b/container/logging-es/Dockerfile @@ -7,10 +7,14 @@ FROM gagara/elasticsearch-oss-arm64:7.6.2 MAINTAINER Wu Zhende <wuzhende666(a)163.com> -RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a path.data: /srv/es/logging-es' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a node.name: node-1' /usr/share/elasticsearch/config/elasticsearch.yml; -RUN sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml; +RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a path.data: /srv/es/logging-es' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a node.name: node-1' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a indices.memory.index_buffer_size: 30%' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a thread_pool.write.queue_size: 2000' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i "s/-Xms1g/-Xms32g/g" /usr/share/elasticsearch/config/jvm.options && \ + sed -i "s/-Xmx1g/-Xmx32g/g" /usr/share/elasticsearch/config/jvm.options RUN mkdir /usr/share/elasticsearch/tmp && \ chown -R 1090:1090 /usr/share/elasticsearch diff --git a/container/logging-es/start b/container/logging-es/start index 9be7a5b..25ed36b 100755 --- a/container/logging-es/start +++ b/container/logging-es/start @@ -19,3 +19,15 @@ cmd=( ) "${cmd[@]}" + +echo "Please wait 60s for setting index" +sleep 60 + +curl -H "Content-Type: application/json" -XPUT "localhost:9202/_settings" -d ' +{ + "index": { + "refresh_interval": "120s", + "translog.durability": "async", + "translog.sync_interval": "120s", + "translog.flush_threshold_size": "1024mb"} +}' -- 2.23.0

2 1

[PATCH v4 lkp-tests 2/2] tests/iozone: add specified parameter for iozone test
by Lu Kaiyi 29 Oct '20

29 Oct '20

[why] need to test write, rewrite, read, reread, random write, random read for different block_size. [How] add judement for variable $block_size to execute iozone command with specificed parameter. Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- tests/iozone | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/tests/iozone b/tests/iozone index 3095ce80..c3b758e9 100755 --- a/tests/iozone +++ b/tests/iozone @@ -1,4 +1,9 @@ #!/bin/sh +# - block_size +# - file_size +# - write_rewrite +# - read_reread +# - random_read_write ## IOzone is a filesystem benchmark tool. The benchmark generates ## and measures a variety of file operations. @@ -7,7 +12,17 @@ for mnt in $mount_points do - cmd="iozone -a -f $mnt/iozonetest" + cmd="iozone" + if [ -n "$block_size" ]; then + cmd+=" -r $block_size" + [ -n "$file_size" ] && cmd+=" -s $file_size" + [ -n "$write_rewrite" ] && cmd+=" -i $write_rewrite" + [ -n "$read_reread" ] && cmd+=" -i $read_reread" + [ -n "$random_read_write" ] && cmd+=" -i $random_read_write" + else + cmd+=" -a" + fi + cmd+=" -f $mnt/iozonetest" log_echo "$cmd &" $cmd > $TMP/iozone-$i & i=$((i+1)) -- 2.23.0

2 1

[PATCH lab-z9] hosts: delete duplicate arches
by Hu Xuejiao 29 Oct '20

29 Oct '20

Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- hosts/vm-hi1620-1p1g | 1 - hosts/vm-hi1620-2p1g | 1 - hosts/vm-hi1620-2p4g | 1 - hosts/vm-hi1620-2p8g | 1 - 4 files changed, 4 deletions(-) diff --git a/hosts/vm-hi1620-1p1g b/hosts/vm-hi1620-1p1g index 2226bdc..bf2a7b3 100644 --- a/hosts/vm-hi1620-1p1g +++ b/hosts/vm-hi1620-1p1g @@ -1,4 +1,3 @@ -arch: aarch64 provider: qemu template: kvm nr_cpu: 1 diff --git a/hosts/vm-hi1620-2p1g b/hosts/vm-hi1620-2p1g index e2952eb..1c4786a 100644 --- a/hosts/vm-hi1620-2p1g +++ b/hosts/vm-hi1620-2p1g @@ -1,4 +1,3 @@ -arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p4g b/hosts/vm-hi1620-2p4g index 83d8133..f2a5b5e 100644 --- a/hosts/vm-hi1620-2p4g +++ b/hosts/vm-hi1620-2p4g @@ -1,4 +1,3 @@ -arch: aarch64 provider: qemu template: kvm nr_cpu: 2 diff --git a/hosts/vm-hi1620-2p8g b/hosts/vm-hi1620-2p8g index a4a0577..1780524 100644 --- a/hosts/vm-hi1620-2p8g +++ b/hosts/vm-hi1620-2p8g @@ -1,4 +1,3 @@ -arch: aarch64 provider: qemu template: kvm nr_cpu: 2 -- 2.23.0

2 1

[PATCH v2 compass-ci 2/2] job.cr: add a key "kernel_version" for initramfs
by Xu Xijian 29 Oct '20

29 Oct '20

When run job with initramfs, there is a default combination of kernel, modules and headers, and I add a key "kernel_version" to make it optional. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- src/lib/job.cr | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 7fbc414..88f64ab 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -86,6 +86,7 @@ class Job kernel_append_root kernel_params docker_image + kernel_version ) macro method_missing(call) @@ -134,6 +135,7 @@ class Job set_result_service() set_os_mount() set_depends_initrd() + set_kernel_version() set_initrds_uri() set_kernel_uri() set_kernel_append_root() @@ -293,8 +295,9 @@ class Job end private def set_kernel_uri + vmlinuz_sub_path = (("#{kernel_version}" == "default") ? "vmlinuz" : "boot/vmlinuz-#{kernel_version}") self["kernel_uri"] = "kernel #{OS_HTTP_PREFIX}" + - "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{os_dir}/#{vmlinuz_sub_path}")}" end private def kernel_common_params @@ -315,10 +318,15 @@ class Job temp_initrds = [] of String osimage_dir = "#{SRV_INITRD}/osimage/#{os_dir}" + boot_dir = "#{SRV_OS}/#{os_dir}/boot" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/current")}" temp_initrds << "#{INITRD_HTTP_PREFIX}" + "#{JobHelper.service_path("#{osimage_dir}/run-ipconfig.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/modules-#{kernel_version}.cgz")}" + temp_initrds << "#{OS_HTTP_PREFIX}" + + "#{JobHelper.service_path("#{boot_dir}/headers-#{kernel_version}.cgz")}" temp_initrds.concat(initrd_deps.split(/ /)) unless initrd_deps.empty? temp_initrds.concat(initrd_pkg.split(/ /)) unless initrd_pkg.empty? @@ -359,6 +367,10 @@ class Job return basenames end + private def set_kernel_version + self["kernel_version"] ||= "default" + end + private def set_initrds_uri uris = "" -- 2.23.0

2 2

[PATCH v2 compass-ci 1/2] sparrow/0-package/os: skip when intsall failed
by Xu Xijian 29 Oct '20

29 Oct '20

[why] When install several packages, if one of them is installed failed, should go ahead to install next packages. [how] Add "--skip-broken" for "yum install" cmd. Add "--fix-missing" for "apt-get install" cmd. Signed-off-by: Xu Xijian <xuxijian(a)huawei.com> --- sparrow/0-package/os/centos | 2 +- sparrow/0-package/os/debian | 2 +- sparrow/0-package/os/openEuler | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 261592c..4243f99 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -22,4 +22,4 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 020c34a..16ef0a9 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -20,4 +20,4 @@ pkgs=( busybox ) -apt-get install -y "${pkgs[@]}" +apt-get install -y --fix-missing "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 5c64fe8..ba298c8 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -21,7 +21,7 @@ pkgs=( busybox ) -yum install -y "${pkgs[@]}" +yum install -y --skip-broken "${pkgs[@]}" wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… wget http://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/releas… -- 2.23.0

1 0

[PATCH v2 compass-ci] feat: container/ntp-server: network time synchronization
by Wang Chenglong 29 Oct '20

29 Oct '20

[Why] By container to provide network time synchronization service for virtual and physical machines. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/Dockerfile | 16 ++++++++ container/ntp-server/build | 5 +++ container/ntp-server/ntp.conf | 57 ++++++++++++++++++++++++++++ container/ntp-server/start | 16 ++++++++ container/ntp-server/start-server.sh | 7 ++++ 5 files changed, 101 insertions(+) create mode 100644 container/ntp-server/Dockerfile create mode 100755 container/ntp-server/build create mode 100644 container/ntp-server/ntp.conf create mode 100755 container/ntp-server/start create mode 100755 container/ntp-server/start-server.sh diff --git a/container/ntp-server/Dockerfile b/container/ntp-server/Dockerfile new file mode 100644 index 0000000..6d95bf2 --- /dev/null +++ b/container/ntp-server/Dockerfile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:7 + +RUN yum repolist +RUN yum -y install ntp \ + && yum clean all + +COPY ntp.conf /etc/ntp.conf +COPY start-server.sh / + +EXPOSE 123 + +ENTRYPOINT ["/start-server.sh"] +CMD ["sh"] diff --git a/container/ntp-server/build b/container/ntp-server/build new file mode 100755 index 0000000..c02152a --- /dev/null +++ b/container/ntp-server/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t ntp_server . diff --git a/container/ntp-server/ntp.conf b/container/ntp-server/ntp.conf new file mode 100644 index 0000000..3152fd3 --- /dev/null +++ b/container/ntp-server/ntp.conf @@ -0,0 +1,57 @@ +For more information about this file, see the ntp.conf(5) man page. + +# Record the frequency of the system clock. +driftfile /var/lib/ntp/drift + +# Permit time synchronization with our time source, but do not +# permit the source to query or modify the service on this system. +restrict default nomodify notrap nopeer noepeer noquery + +# Permit association with pool servers. +restrict source nomodify notrap noepeer noquery + +# Permit all access over the loopback interface. This could +# be tightened as well, but to do so would effect some of +# the administrative functions. +restrict 127.0.0.1 +restrict ::1 + +# Hosts on local network are less restricted. +restrict 172.0.0.0 mask 255.0.0.0 nomodify notrap + +# Use public servers from the pool.ntp.org project. +# Please consider joining the pool (http://www.pool.ntp.org/join.html) +# pool 2.openEuler.pool.ntp.org iburst +server 0.centos.pool.ntp.org +server 1.centos.pool.ntp.org +server 2.centos.pool.ntp.org +server 3.centos.pool.ntp.org + +server 127.127.1.0 +fudge 127.127.1.0 stratum 8 + + +# Reduce the maximum number of servers used from the pool. +tos maxclock 5 + +# Enable public key cryptography. +# crypto + +includefile /etc/ntp/crypto/pw + +# Key file containing the keys and key identifiers used when operating +# with symmetric key cryptography. +keys /etc/ntp/keys + +# Specify the key identifiers which are trusted. +# trustedkey 4 8 42 + +# Specify the key identifier to use with the ntpdc utility. +# requestkey 8 + +# Specify the key identifier to use with the ntpq utility. +# controlkey 8 + +# Enable writing of statistics records. +# statistics clockstats cryptostats loopstats peerstats + diff --git a/container/ntp-server/start b/container/ntp-server/start new file mode 100755 index 0000000..6e7479e --- /dev/null +++ b/container/ntp-server/start @@ -0,0 +1,16 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ../defconfig.sh + +docker_rm ntp_server +cmd=( + docker run + -id + --name=ntp_server + -p 123:123/udp + ntp_server:latest +) + +"${cmd[@]}" diff --git a/container/ntp-server/start-server.sh b/container/ntp-server/start-server.sh new file mode 100755 index 0000000..8f35f22 --- /dev/null +++ b/container/ntp-server/start-server.sh @@ -0,0 +1,7 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +/usr/sbin/ntpd -g -l /var/log/ntpstats/ntpd.log + +exec "$@" -- 2.23.0

2 2

[PATCH compass-ci] feat: container/ntp-server: network time synchronization
by Wang Chenglong 29 Oct '20

29 Oct '20

By container to provides network time synchronization service for virtual and physical machines. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/ntp-server/Dockerfile | 16 ++++++++ container/ntp-server/build | 5 +++ container/ntp-server/ntp.conf | 57 ++++++++++++++++++++++++++++ container/ntp-server/start | 16 ++++++++ container/ntp-server/start-server.sh | 7 ++++ 5 files changed, 101 insertions(+) create mode 100644 container/ntp-server/Dockerfile create mode 100755 container/ntp-server/build create mode 100644 container/ntp-server/ntp.conf create mode 100755 container/ntp-server/start create mode 100755 container/ntp-server/start-server.sh diff --git a/container/ntp-server/Dockerfile b/container/ntp-server/Dockerfile new file mode 100644 index 0000000..6d95bf2 --- /dev/null +++ b/container/ntp-server/Dockerfile @@ -0,0 +1,16 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +FROM centos:7 + +RUN yum repolist +RUN yum -y install ntp \ + && yum clean all + +COPY ntp.conf /etc/ntp.conf +COPY start-server.sh / + +EXPOSE 123 + +ENTRYPOINT ["/start-server.sh"] +CMD ["sh"] diff --git a/container/ntp-server/build b/container/ntp-server/build new file mode 100755 index 0000000..c02152a --- /dev/null +++ b/container/ntp-server/build @@ -0,0 +1,5 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +docker build -t ntp_server . diff --git a/container/ntp-server/ntp.conf b/container/ntp-server/ntp.conf new file mode 100644 index 0000000..3152fd3 --- /dev/null +++ b/container/ntp-server/ntp.conf @@ -0,0 +1,57 @@ +For more information about this file, see the ntp.conf(5) man page. + +# Record the frequency of the system clock. +driftfile /var/lib/ntp/drift + +# Permit time synchronization with our time source, but do not +# permit the source to query or modify the service on this system. +restrict default nomodify notrap nopeer noepeer noquery + +# Permit association with pool servers. +restrict source nomodify notrap noepeer noquery + +# Permit all access over the loopback interface. This could +# be tightened as well, but to do so would effect some of +# the administrative functions. +restrict 127.0.0.1 +restrict ::1 + +# Hosts on local network are less restricted. +restrict 172.0.0.0 mask 255.0.0.0 nomodify notrap + +# Use public servers from the pool.ntp.org project. +# Please consider joining the pool (http://www.pool.ntp.org/join.html) +# pool 2.openEuler.pool.ntp.org iburst +server 0.centos.pool.ntp.org +server 1.centos.pool.ntp.org +server 2.centos.pool.ntp.org +server 3.centos.pool.ntp.org + +server 127.127.1.0 +fudge 127.127.1.0 stratum 8 + + +# Reduce the maximum number of servers used from the pool. +tos maxclock 5 + +# Enable public key cryptography. +# crypto + +includefile /etc/ntp/crypto/pw + +# Key file containing the keys and key identifiers used when operating +# with symmetric key cryptography. +keys /etc/ntp/keys + +# Specify the key identifiers which are trusted. +# trustedkey 4 8 42 + +# Specify the key identifier to use with the ntpdc utility. +# requestkey 8 + +# Specify the key identifier to use with the ntpq utility. +# controlkey 8 + +# Enable writing of statistics records. +# statistics clockstats cryptostats loopstats peerstats + diff --git a/container/ntp-server/start b/container/ntp-server/start new file mode 100755 index 0000000..6e7479e --- /dev/null +++ b/container/ntp-server/start @@ -0,0 +1,16 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +. ../defconfig.sh + +docker_rm ntp_server +cmd=( + docker run + -id + --name=ntp_server + -p 123:123/udp + ntp_server:latest +) + +"${cmd[@]}" diff --git a/container/ntp-server/start-server.sh b/container/ntp-server/start-server.sh new file mode 100755 index 0000000..8f35f22 --- /dev/null +++ b/container/ntp-server/start-server.sh @@ -0,0 +1,7 @@ +#!/bin/sh +# SPDX-License-Identifier: MulanPSL-2.0+ +# # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +/usr/sbin/ntpd -g -l /var/log/ntpstats/ntpd.log + +exec "$@" -- 2.23.0

2 2

[PATCH compass-ci] sparrow: open the masquerade at public zone
by Hu Xuejiao 29 Oct '20

29 Oct '20

Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/2-network/iptables | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/2-network/iptables b/sparrow/2-network/iptables index e5955d0..48794f5 100755 --- a/sparrow/2-network/iptables +++ b/sparrow/2-network/iptables @@ -25,3 +25,4 @@ DOCKER0_SUBNET=172.17.0.0/16 firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" +firewall-cmd --zone=public --add-masquerade -- 2.23.0

1 0

[PATCH lkp-tests] jobs/build-pkg.yaml: add pkgbuild_repo examples
by Lin Jiaxin 29 Oct '20

29 Oct '20

[why] pkgbuild_repo field's parameter format is not uniform for different git repository. Novices are easy to make mistakes. [how] Add examples of three types of git repository used by build-pkg. Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- jobs/build-pkg.yaml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/jobs/build-pkg.yaml b/jobs/build-pkg.yaml index 5888838a..b17fd2ad 100644 --- a/jobs/build-pkg.yaml +++ b/jobs/build-pkg.yaml @@ -1,4 +1,9 @@ suite: build-pkg category: functional +# pkgbuild_repo examples +# pkgbuild_repo: archlinux/aur-j/java8-openjdk +# pkgbuild_repo: archlinux/community//keepalived/trunk +# pkgbuild_repo: archlinux/packages//prometheus-node-exporter/trunk + build-pkg: -- 2.23.0

3 4

[PATCH compass-ci] sched: consume job from queues
by Cao Xueliang 29 Oct '20

29 Oct '20

[why] Before, we consume job from the queue which has relation with tbox_group. Now, we consume job from the queue which is setted by mac2queues. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/lib/sched.cr | 69 +++++++++++++++++++++++++++++++++----- src/scheduler/scheduler.cr | 2 +- 2 files changed, 62 insertions(+), 9 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index c32091f..d8d0338 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -240,10 +240,10 @@ class Sched response = add_job(job, job_id) message = (response["error"]? ? response["error"]["root_cause"] : "") job_messages << { - "job_id" => job_id, - "message" => message.to_s, - "job_state" => "submit", - "result_root" => "/srv#{job.result_root}" + "job_id" => job_id, + "message" => message.to_s, + "job_state" => "submit", + "result_root" => "/srv#{job.result_root}", } return job_messages if response["error"]? end @@ -292,10 +292,10 @@ class Sched message = (response["error"]? ? response["error"]["root_cause"] : "") return [{ - "job_id" => job_id, - "message" => message.to_s, - "job_state" => "submit", - "result_root" => "/srv#{job.result_root}" + "job_id" => job_id, + "message" => message.to_s, + "job_state" => "submit", + "result_root" => "/srv#{job.result_root}", }] end @@ -378,6 +378,59 @@ class Sched end end + def rand_queues(queues) + return queues if queues.empty? + + queues_size = queues.size + base = Random.rand(queues_size) + temp_queues = [] of String + + (0..queues_size - 1).each do |index| + temp_queues << queues[(index + base) % queues_size] + end + + return temp_queues + end + + def get_queues(mac) + queues = [] of String + + queues_str = @redis.hash_get("sched/mac2queues", normalize_mac(mac)) + return queues unless queues_str + + queues_str.split(',', remove_empty: true) do |item| + queues << item.strip + end + + return rand_queues(queues) + end + + def get_job_from_queues(queues, testbox) + job = nil + + queues.each do |queue| + job = prepare_job("sched/#{queue}", testbox) + return job if job + end + + return job + end + + def get_job_boot(env : HTTP::Server::Context) + value = env.params.url["value"] + boot_type = env.params.url["boot_type"] + testbox = @redis.hash_get("sched/mac2host", normalize_mac(value)) + + queues = get_queues(value) + job = get_job_from_queues(queues, testbox) + + if job + Jobfile::Operate.create_job_cpio(job.dump_to_json_any, Kemal.config.public_folder) + end + + return boot_content(job, boot_type) + end + def find_job_boot(env : HTTP::Server::Context) api_param = env.params.url["value"] diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 371e4ed..1bfea02 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -51,7 +51,7 @@ module Scheduler # /boot.yyy/mac/${mac} get "/boot.:boot_type/:parameter/:value" do |env| env.response.headers["Connection"] = "close" - response = sched.find_job_boot(env) + response = sched.get_job_boot(env) job_id = response[/tmpfs\/(.*)\/job\.cgz/, 1]? puts %({"job_id": "#{job_id}", "job_state": "boot"}) if job_id -- 2.23.0

3 2

[PATCH compass-ci] delimiter: delete tbox_group settings
by Cao Xueliang 29 Oct '20

29 Oct '20

[why] Use the original tbox_group of the job. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/git_bisect.rb | 4 +--- src/delimiter/constants.rb | 1 - src/delimiter/find-commit/bisect_run_script.rb | 9 +++------ 3 files changed, 4 insertions(+), 10 deletions(-) diff --git a/lib/git_bisect.rb b/lib/git_bisect.rb index d9e5cf4..c548861 100644 --- a/lib/git_bisect.rb +++ b/lib/git_bisect.rb @@ -42,7 +42,6 @@ class GitBisect @bad_job.delete('stats') @bad_job.delete('id') @bad_job.delete('error_ids') - @bad_job['tbox_group'] = DELIMITER_TBOX_GROUP end def set_upstream @@ -75,8 +74,7 @@ class GitBisect result = `git -C #{@work_dir} bisect start #{@upstream_commit} #{@good_commit}` temp = result.split(/\n/) if temp[0].include? 'Bisecting' - result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} "#{@error_id}" \ - "#{DELIMITER_TBOX_GROUP}" #{@work_dir}` + result = `git -C #{@work_dir} bisect run #{BISECT_RUN_SCRIPT} #{@bad_job_id} "#{@error_id}" #{@work_dir}` end FileUtils.rm_r(@work_dir) if Dir.exist?(@work_dir) puts "\nbisect result: #{result}" diff --git a/src/delimiter/constants.rb b/src/delimiter/constants.rb index dafc2c8..93b633d 100644 --- a/src/delimiter/constants.rb +++ b/src/delimiter/constants.rb @@ -9,4 +9,3 @@ MONITOR_PORT = ENV['MONITOR_PORT'] || '11310' TMEP_GIT_BASE = '/c/public_git' DELIMITER_TASK_QUEUE = 'delimiter' BISECT_RUN_SCRIPT = "#{ENV['CCI_SRC']}/src/delimiter/find-commit/bisect_run_script.rb" -DELIMITER_TBOX_GROUP = 'vm-2p8g--delimiter' diff --git a/src/delimiter/find-commit/bisect_run_script.rb b/src/delimiter/find-commit/bisect_run_script.rb index 539599f..d5e4226 100755 --- a/src/delimiter/find-commit/bisect_run_script.rb +++ b/src/delimiter/find-commit/bisect_run_script.rb @@ -10,18 +10,16 @@ require_relative "#{ENV['CCI_SRC']}/src/delimiter/utils" # git bisect run class GitBisectRun - def initialize(job_id, error_id, tbox_group, work_dir) + def initialize(job_id, error_id, work_dir) @es = ESQuery.new @job_id = job_id @error_id = error_id - @tbox_group = tbox_group @work_dir = work_dir end def git_bisect job = @es.query_by_id @job_id job.delete('stats') if job.key?('stats') - job['tbox_group'] = @tbox_group commit = `git -C #{@work_dir} log --pretty=format:"%H" -1` job['upstream_commit'] = commit get_bisect_status job @@ -41,8 +39,7 @@ end job_id = ARGV[0] error_id = ARGV[1] -tbox_group = ARGV[2] -work_dir = ARGV[3] +work_dir = ARGV[2] -run = GitBisectRun.new job_id, error_id, tbox_group, work_dir +run = GitBisectRun.new job_id, error_id, work_dir run.git_bisect -- 2.23.0

1 0

[PATCH compass-ci] container/rsync-server: modify data upload directory
by Cui Yili 29 Oct '20

29 Oct '20

[why] Modify data upload directory to upload qemu image. [how] Change the rsync.conf configuration and docker operating parameters. Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/rsync-server/rsyncd.conf | 6 ++++-- container/rsync-server/start | 4 ++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/container/rsync-server/rsyncd.conf b/container/rsync-server/rsyncd.conf index 528e8bd..dd8f128 100644 --- a/container/rsync-server/rsyncd.conf +++ b/container/rsync-server/rsyncd.conf @@ -1,10 +1,12 @@ uid = lkp gid = lkp +port = 11387 use chroot = no +log file=/var/log/rsync.log -[result] -path = /result +[upload] +path = /srv/initrd/qemu-image read only = no max connections = 200 refuse options = delete diff --git a/container/rsync-server/start b/container/rsync-server/start index 4124192..6fa3de2 100755 --- a/container/rsync-server/start +++ b/container/rsync-server/start @@ -9,9 +9,9 @@ cmd=( docker run -id --name=rsync_server - -v /srv/result:/result + -v /srv/initrd/qemu-image:/srv/initrd/qemu-image -v /etc/localtime:/etc/localtime:ro - -p 873:873 + -p 11387:11387 rsync_server:latest ) -- 2.23.0

2 1

[PATCH compass-ci 1/2] auto group jobs with user-defined template
by Lu Weitao 29 Oct '20

29 Oct '20

auto group jobs with user-defined template to support template compare mode background: before compare, create job-groups for all jobs that query from ES with user's filter then, create matrices-groups according job-groups finally, compare each group matrices Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/params_group.rb | 120 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 120 insertions(+) diff --git a/lib/params_group.rb b/lib/params_group.rb index 0f07d26..4d10dbe 100644 --- a/lib/params_group.rb +++ b/lib/params_group.rb @@ -113,3 +113,123 @@ end def remove_singleton(groups) groups.delete_if { |_k, v| v.length < 2 } end + +# -------------------------------------------------------------------------------------------------- +# auto group job_list by user's template +# Exammple: +# Input: +# 1. jobs_list. +# 2. params from template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# Output: +# eg: +# { +# '4k|1G' => { +# 'openeuler 20.03' => [ +# {'stats' => {'fio.write_iops' => 312821.002387, 'fio.read_iops' => 212821.2387}}, +# {'stats' => {'fio.write_iops' => 289661.878453}}, +# ... +# ], +# 'contos 7.6' => [...] +# }, +# '16K|1G' => {...}, +# ... +# } +# + +def auto_group_by_template(jobs_list, group_params, dimensions, metrics) + job_list = extract_jobs_list(jobs_list) + get_group_by_template(job_list, group_params, dimensions, metrics) +end + +def get_group_by_template(job_list, group_params, dimensions, metrics) + groups = {} + job_list.each do |job| + next unless job['stats'] + + group_key = get_user_group_key(job, group_params) + dimension = get_user_dimension(job, dimensions) + next unless group_key && dimension + + new_job = get_new_job_by_metrics(job, metrics) + next if new_job.empty? + + groups[group_key] ||= {} + groups[group_key][dimension] ||= [] + groups[group_key][dimension] << new_job + end + groups +end + +# @group_params Array(String) +# eg: +# ['block_size', 'package_size'] +# return eg: +# '4K|1G' +def get_user_group_key(job, group_params) + group_key_list = [] + group_params.each do |param| + if job.key?(param) + group_key_list << job[param] + next + end + job.each_value do |v| + if v.is_a?(Hash) && v.key?(param) + group_key_list << v[param] + break + end + end + end + return nil if group_key_list.size < group_params.size + + group_key_list.join('|') +end + +# @dimension Array(Hash) +# eg: +# [ +# {os => openeuler, os_version => 20.03}, +# {os => centos, os_version => 7.6} +# ] +# return eg: +# 'openeuler 20.03' +def get_user_dimension(job, dimensions) + dimension_list = [] + + dimensions.each do |dim| + dim.each_key do |key| + if job[key] == dim[key] + dimension_list << job[key] + end + end + break unless dimension_list.empty? + end + return nil if dimension_list.size < dimensions.size + + dimension_list.join(' ') +end + +# @metrics Array(String) +# eg: +# ["fio.read_iops", "fio.write_iops"] +# return new_job +# eg: +# {'stats' => {'fio.write_iops' => 312821.002387, 'fio.read_iops' => 212821.2387}}, +def get_new_job_by_metrics(job, metrics) + new_job = {} + metrics.each do |metric| + if job['stats'].key?(metric) + new_job['stats'] ||= {} + new_job['stats'][metric] = job['stats'][metric] + end + end + new_job +end -- 2.23.0

4 6

[PATCH lab-z9] hosts: delete repet arch field
by Zhang Yu 29 Oct '20

29 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-256g | 1 - hosts/taishan200-2280-2s64p-256g | 1 - 2 files changed, 2 deletions(-) diff --git a/hosts/taishan200-2280-2s48p-256g b/hosts/taishan200-2280-2s48p-256g index 642e8e1..b326025 100644 --- a/hosts/taishan200-2280-2s48p-256g +++ b/hosts/taishan200-2280-2s48p-256g @@ -2,4 +2,3 @@ arch: aarch64 nr_cpu: 96 nr_node: 4 memory: 256G -arch: aarch64 diff --git a/hosts/taishan200-2280-2s64p-256g b/hosts/taishan200-2280-2s64p-256g index 16349d9..7c7eb5c 100644 --- a/hosts/taishan200-2280-2s64p-256g +++ b/hosts/taishan200-2280-2s64p-256g @@ -2,4 +2,3 @@ arch: aarch64 nr_cpu: 128 nr_node: 4 memory: 256G -arch: aarch64 -- 2.23.0

2 3

[PATCH compass-ci 2/2] create matrices-groups according to group jobs
by Lu Weitao 29 Oct '20

29 Oct '20

create matrices-groups according to group jobs for support user-defined template compare mode background: before compare, create job-groups for all jobs that query from ES with user's filter then, create matrices-groups according job-groups finally, compare each group matrices Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/matrix2.rb | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 05f96dc..61517d9 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -120,3 +120,27 @@ def combine_group_query_data(query_data, dims) end groups end + +# input: +# 1. query results from es_query +# 2. params from template include: +# groups_params(x_params): +# eg: ['block_size', 'package_size'] +# dimensions: +# eg: [ +# {'os' => 'openeuler', 'os_version' => '20.03'}, +# {'os' => 'centos', 'os_version' => '7.6'} +# ] +# metrics: +# eg: ['fio.read_iops', 'fio_write_iops'] +# return: group_matrix of Hash(String, Hash(String, matrix)) +def combine_user_group_query_data(query_data, groups_params, dimensions, metrics) + job_list = query_data['hits']['hits'] + groups = auto_group_by_template(job_list, groups_params, dimensions, metrics) + groups.each do |group_key, value| + value.each do |dimension_key, jobs| + groups[group_key][dimension_key] = create_matrix(jobs) + end + end + groups +end -- 2.23.0

3 4

[PATCH compass-ci] container/netdata: use json-file save logs
by Wu Zhende 29 Oct '20

29 Oct '20

[Why] netdata is aperformance check tool and does not need to send logs to fluentd for analysis. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/netdata/start | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/netdata/start b/container/netdata/start index a3009a9..58fb675 100755 --- a/container/netdata/start +++ b/container/netdata/start @@ -17,6 +17,8 @@ cmd=( -v /var/run/docker.sock:/var/run/docker.sock:ro -v /srv/cache/netdata_cache:/var/cache/netdata -v /srv/cache/netdata_lib:/var/lib/netdata + --log-driver json-file + --log-opt max-size=1g --cap-add SYS_PTRACE --security-opt apparmor=unconfined netdata/netdatalkp -- 2.23.0

1 0

[PATCH lab-z9] hosts: Delete duplicate field
by Zhang Yu 29 Oct '20

29 Oct '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- hosts/taishan200-2280-2s48p-256g | 1 - hosts/taishan200-2280-2s64p-256g | 1 - 2 files changed, 2 deletions(-) diff --git a/hosts/taishan200-2280-2s48p-256g b/hosts/taishan200-2280-2s48p-256g index 642e8e1..b326025 100644 --- a/hosts/taishan200-2280-2s48p-256g +++ b/hosts/taishan200-2280-2s48p-256g @@ -2,4 +2,3 @@ arch: aarch64 nr_cpu: 96 nr_node: 4 memory: 256G -arch: aarch64 diff --git a/hosts/taishan200-2280-2s64p-256g b/hosts/taishan200-2280-2s64p-256g index 16349d9..7c7eb5c 100644 --- a/hosts/taishan200-2280-2s64p-256g +++ b/hosts/taishan200-2280-2s64p-256g @@ -2,4 +2,3 @@ arch: aarch64 nr_cpu: 128 nr_node: 4 memory: 256G -arch: aarch64 -- 2.23.0

1 0

[PATCH lkp-tests] tests: add get_cpu function
by Zhang Yu 29 Oct '20

29 Oct '20

[why] Obtain more host information from 'lscpu' Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/tests/host-info b/tests/host-info index 46f43f93..84f796dc 100755 --- a/tests/host-info +++ b/tests/host-info @@ -1,18 +1,25 @@ #!/bin/bash -get_node() +get_memory() { - echo "nr_node: $(grep -Eo 'Node [0-9]+,' /proc/zoneinfo | uniq | wc -l)" - echo "nr_cpu: $(nproc)" memory_total=$(cat /proc/meminfo |grep MemTotal | awk '{print $2}') memory_new=$(awk 'BEGIN{printf "%0.2f", '$memory_total'/1024/1024/16}') memory=$(echo `expr $(echo $memory_new | awk '{print int($0)+1}') \* 16`)G echo "memory: $memory" } -get_model_name() +get_cpu() { - echo "model_name: $(lscpu |grep "Model name" | awk '{print $3}')" + echo $1: $(lscpu | awk -F': *' '/'$2':/{print $2}') +} + +get_cpu_info() +{ + cpu_info=("arch Architecture" "nr_node NUMA\snode$s$" "nr_cpu ^CPU$s$" "model_name Model\sname") + for info in "${cpu_info[@]}" + do + get_cpu $info + done } get_nr_disk() @@ -101,11 +108,11 @@ get_ipmi_ip() main() { - get_node + get_memory get_nr_disk get_partitions get_mac - get_model_name + get_cpu_info get_ipmi_ip } -- 2.23.0

1 0

[PATCH v6 compass-ci] add queue field into job.yaml
by Xiao Shenwei 29 Oct '20

29 Oct '20

use queue field decide which queue to enter queue field come from user's specify(job.yaml) or dafault Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- src/lib/job.cr | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 8f5dd36..7fbc414 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -73,6 +73,7 @@ class Job tbox_group testbox lab + queue initrd_pkg initrd_deps initrds_uri @@ -139,6 +140,7 @@ class Job set_kernel_params() set_lkp_server() set_sshr_port() + set_queue() end private def append_init_field @@ -199,6 +201,16 @@ class Job self["result_service"] = "raw_upload" end + private def set_queue + return if @hash.has_key?("queue") + + # set default value + self["queue"] = tbox_group + if tbox_group.to_s.starts_with?(/(vm|dc)-/) + self["queue"] = "#{tbox_group}.#{arch}" + end + end + # if not assign tbox_group, set it to a match result from testbox # ?if job special testbox, should we just set tbox_group=textbox private def update_tbox_group_from_testbox -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code: add centos in os file
by Zhang Yale 29 Oct '20

29 Oct '20

[Why] when run dev-env in centos system, error like this: ./dev-env:line 41: ./os/centos: No such file or directory. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- sparrow/3-code/os/centos | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 sparrow/3-code/os/centos diff --git a/sparrow/3-code/os/centos b/sparrow/3-code/os/centos new file mode 100644 index 0000000..efb4fcb --- /dev/null +++ b/sparrow/3-code/os/centos @@ -0,0 +1,6 @@ +#!/bin/bash +cat >> /etc/default/docker <<EOF +DOCKER_OPTS="--dns 114.114.114.114" +EOF + +systemctl restart docker -- 2.23.0

4 6

[PATCH v2 compass-ci] LICENSE: add document license
by Bai Jing 29 Oct '20

29 Oct '20

Signed-off-by: Bai Jing <799286817(a)qq.com> --- LICENSES/CCBY-4.0 | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 LICENSES/CCBY-4.0 diff --git a/LICENSES/CCBY-4.0 b/LICENSES/CCBY-4.0 new file mode 100644 index 0000000..abaf8d0 --- /dev/null +++ b/LICENSES/CCBY-4.0 @@ -0,0 +1,4 @@ +Copyright (c) 2020 Huawei Technologies Co., Ltd. +Your copying, use, modification, and distribution of this document is subject to the Creative Commons Attribution-ShareAlike 4.0 International Public License ("CC BY-SA 4.0"). +For ease of understanding, you can visit https://creativecommons.org/licenses/by-sa/4.0/to learn about the outline of CC BY-SA 4.0(but not a substitute). +The complete agreement of CC BY-SA 4.0 can be obtained from the following website: https://creativecommons.org/licenses/by-sa/4.0/legalcode. -- 2.23.0

2 2

[PATCH lkp-tests] lib/job.rb: expand load hosts file
by Wei Jihui 28 Oct '20

28 Oct '20

load_yaml(yaml_file, template_context) if template_context is not nil, it will expand read the yaml. this will read "<< : hosts/$tbox_group" in hosts file. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index f24435a0..5c2c9517 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -305,7 +305,7 @@ class Job hosts_file = get_hosts_file if hosts_file - hwconfig = load_yaml(hosts_file, nil) + hwconfig = load_yaml(hosts_file, {}) @job[source_file_symkey(hosts_file)] = nil @job.merge!(hwconfig) { |_k, a, _b| a } # job's key/value has priority over hwconfig @load_hosts_done = true -- 2.23.0

1 0

[PATCH v4 compass-ci] fix: sched: add 2 fields into cluster state
by Ren Wen 28 Oct '20

28 Oct '20

[Why] 1) Each node will report it's infos (roles, ip, direct_ips, direct_macs) when running cluster job, so also add these 2 parameters into node state: "direct_ips", "direct_macs". Cluster test scripts needs "direct_server_ips" as server client's ip to connect to server to do a test. When request state is "roles_ip", return clients two variables: server, direct_server_ips. Clients will export them. 2) Alter funtion "get_ip"'s name and it's return value to fit for changes. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 21 +++++++++++++-------- 1 file changed, 13 insertions(+), 8 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index c32091f..4221b73 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -99,25 +99,30 @@ class Sched when "write_state" node_roles = env.params.query["node_roles"] node_ip = env.params.query["ip"] + direct_ips = env.params.query["direct_ips"] + direct_macs = env.params.query["direct_macs"] + update_cluster_state(cluster_id, job_id, "roles", node_roles) update_cluster_state(cluster_id, job_id, "ip", node_ip) + update_cluster_state(cluster_id, job_id, "direct_ips", direct_ips) + update_cluster_state(cluster_id, job_id, "direct_macs", direct_macs) when "roles_ip" role = "server" - server_ip = get_ip(cluster_id, role) - return "server=#{server_ip}" + role_state = get_role_state(cluster_id, role) + raise "Missing #{role} state in cluster state" unless role_state + return "server=#{role_state["ip"]}\n" \ + "direct_server_ips=#{role_state["direct_ips"]}" end # show cluster state return @redis.hash_get("sched/cluster_state", cluster_id) end - # get the ip of role from cluster_state - def get_ip(cluster_id, role) + # get the node state of role from cluster_state + private def get_role_state(cluster_id, role) cluster_state = get_cluster_state(cluster_id) - cluster_state.each_value do |config| - if %(#{config["roles"]}) == role - return config["ip"] - end + cluster_state.each_value do |role_state| + return role_state if role_state["roles"] == role end end -- 2.23.0

1 0

[PATCH compass-ci] container/srv-http: add new web directory
by Cui Yili 28 Oct '20

28 Oct '20

[why] The "/srv/initrd/qemu-image/" directory needs to be accessible through the web to obtain qemu image. [how] docker start add "-v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro" Signed-off-by: cuiyili <2268260388(a)qq.com> --- container/srv-http/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start b/container/srv-http/start index 2b5bd1d..b56fe4d 100755 --- a/container/srv-http/start +++ b/container/srv-http/start @@ -13,6 +13,7 @@ cmd=( -v /srv/pub:/srv/pub:ro -v /etc/localtime:/etc/localtime:ro -v /srv/result:/srv/result:ro + -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro -d srv-http ) -- 2.23.0

1 0