- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] doc: update sparrow README.md
by Liu Yinsi 31 Mar '21

31 Mar '21

Locally deploy compass-ci just for sparrow, if users want to use other services, prompt users to apply for account and borrow machine. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/README.md | 50 ++++++++++++----------------------------------- 1 file changed, 13 insertions(+), 37 deletions(-) diff --git a/sparrow/README.md b/sparrow/README.md index d7b8e81..b161877 100644 --- a/sparrow/README.md +++ b/sparrow/README.md @@ -41,7 +41,7 @@ 3. 创建工作目录并克隆 compass-ci 项目代码 ```bash mkdir /c/ && ln -s /c/compass-ci /c/cci - git clone https://gitee.com/wu_fengguang/compass-ci.git + git clone https://gitee.com/wu_fengguang/compass-ci.git ``` 4. 编辑setup.yaml配置用户名和邮箱 @@ -57,9 +57,10 @@ cd compass-ci/sparrow && ./install-tiny ``` -#### 提交测试任务前的准备 +#### 提交测试任务 +本文以/c/lkp-tests/jobs/目录下已有的通用测试用例iperf.yaml为例 -1. 测试环境是否可以提交job测试 +1. 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… ```bash submit iperf.yaml ``` @@ -68,42 +69,17 @@ ``` submit_id=bf5e7ad7-839d-48ec-a033-23281323c750 submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.1 - ``` - compass-ci搭建完毕，下面就可以开始进行测试了。 - -#### 提交测试任务到本地compass-ci -本文以测试用例iperf.yaml为例 - -1. [使用 compass-ci 平台测试开源项目](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/… - -2. 根据测试需要[编写测试用例](https://gitee.com/wu_fengguang/lkp-tests/blob/master/doc/add… - -3. 使用[submit命令](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manu… - ```bash - submit iperf.yaml + submit /c/lkp-tests/jobs/iperf.yaml, got job id=nolab.2 ``` -#### 运行测试任务并查看任务结果 - -1. 运行测试任务 - ```bash - cd /c/compass-ci/providers/ && ./my-qemu.sh - ``` - -2. 在本地/srv/result/目录下根据测试用例名称/日期/[testbox](https://gitee.com/wu_fengguang/lab-… [查看任务结果](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/br… +2. 查看任务结果 +任务结果文件会生成在本地/srv/result目录下，根据上一步骤显示出的job id可以查看对应任务的结果文件output(可使用tab键自动补全多级目录) ```bash - cd /srv/result/iperf/2020-12-29/vm-2p8g/openeuler-20.03-aarch64/nolab.1 - cat output + cd /srv/result/iperf/2021-03-18/dc-8g/centos-7-aarch64/tcp-300/nolab.1 && cat output + cd /srv/result/iperf/2021-03-18/dc-8g/centos-7-aarch64/udp-300/nolab.2 && cat output ``` + + >**说明：** + >output 文件记录了用例的执行过程，文件最后部分一般会有 check_exit_code 这个状态码，非 0 代表测试用例错误。 - >**说明：** - >[登陆测试环境调测任务](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manua… - -# FAQ - -* 选择下载os rootfs - - 启动qemu测试机需要[os rootfs文件](http://api.compass-ci.openeuler.org:11304/os/)， - 一键部署默认下载os rootfs为openeuler aarch64 20.03，当部署完毕后，可在该目录下查看: /srv/os/openeuler/aarch64/20.03 - 如需要其他os rootfs，可使用该脚本下载: /c/compass-ci/sbin/download-rootfs +体验更多服务例如[自动化测试](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-project.zh.md)、[调测环境登录](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/log-in-machine-debug.md)、[测试结果分析](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/compare-results.zh.md)等，请[申请账号](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/apply-account.zh.md)、[申请测试机](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/borrow-machine.zh.md)。 -- 2.23.0

1 1

[PATCH compass-ci 1/2] lib/stats_wrapper.rb: fix TypeError
by Lu Weitao 31 Mar '21

31 Mar '21

[Why] linjiaxin@crystal ~/crystal.116494% ~/compass-ci/sbin/result2stats ./ Traceback (most recent call last): 7: from /home/linjiaxin/compass-ci/sbin/result2stats:37:in `<main>' 6: from /home/linjiaxin/compass-ci/lib/stats.rb:47:in `extract_stats' 5: from /usr/share/ruby/set.rb:338:in `each' 4: from /usr/share/ruby/set.rb:338:in `each_key' 3: from /home/linjiaxin/compass-ci/lib/stats.rb:49:in `block in extract_stats' 2: from /home/linjiaxin/compass-ci/lib/stats_wrapper.rb:61:in `wrapper_func' 1: from /home/linjiaxin/compass-ci/lib/stats_wrapper.rb:287:in `delete_log_package' /home/linjiaxin/compass-ci/lib/stats_wrapper.rb:287:in `exist?': no implicit conversion of nil into String (TypeError) Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/stats_wrapper.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/stats_wrapper.rb b/lib/stats_wrapper.rb index faa81df..6a08797 100644 --- a/lib/stats_wrapper.rb +++ b/lib/stats_wrapper.rb @@ -284,7 +284,7 @@ module StatsWrapper # delete temporarily extracted dmesg above File.delete(@dmesg_log) if File.exist?("#{(a)dmesg_log}.xz") - File.delete(@tmpfile) if File.exist?(@tmpfile) + File.delete(@tmpfile) if @tmpfile && File.exist?(@tmpfile) end end -- 2.23.0

2 1

[PATCH compass-ci 2/2] lib/matrix2.rb: fix NoMethodError
by Lu Weitao 31 Mar '21

31 Mar '21

[Why] linjiaxin@crystal ~/crystal.116494% ~/compass-ci/sbin/result2stats ./ Traceback (most recent call last): 7: from /home/linjiaxin/compass-ci/sbin/result2stats:38:in `<main>' 6: from /home/linjiaxin/compass-ci/lib/matrix2.rb:52:in `create_stats' 5: from /home/linjiaxin/compass-ci/lib/matrix2.rb:52:in `each' 4: from /home/linjiaxin/compass-ci/lib/matrix2.rb:58:in `block in create_stats' 3: from /home/linjiaxin/compass-ci/lib/matrix2.rb:30:in `extract_pre_result' 2: from /home/linjiaxin/compass-ci/lib/matrix2.rb:30:in `each' 1: from /home/linjiaxin/compass-ci/lib/matrix2.rb:33:in `block in extract_pre_result' /home/linjiaxin/compass-ci/lib/matrix2.rb:22:in `set_pre_value': undefined method `/' for #<String:0x0000aaabcdd1a140> (NoMethodError) Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/matrix2.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/matrix2.rb b/lib/matrix2.rb index 5e0ad21..12245a8 100644 --- a/lib/matrix2.rb +++ b/lib/matrix2.rb @@ -10,7 +10,7 @@ require "#{LKP_SRC}/lib/matrix" require_relative './params_group.rb' def set_pre_value(item, value, sample_size) - return value if value.is_a?(String) + return value if item.end_with?('.message', '.log') if value.size == 1 value[0] -- 2.23.0

2 1

[PATCH lkp-tests 2/2] stats: add build-pkg.rb to generate error_id with message
by Lin Jiaxin 31 Mar '21

31 Mar '21

output like this: "build-pkg.include/common/ist.h:warning:unused-parameter‘count’[-Wunused-parameter].message": [ "include/common/ist.h:488:74: warning: unused parameter ‘count’ [-Wunused-parameter]\n static inline struct ist ist2str(char *dst, const struct ist src, size_t count)\n ^~~~~\n", "include/common/ist.h:533:77: warning: unused parameter ‘count’ [-Wunused-parameter]\n static inline struct ist ist2str_lc(char *dst, const struct ist src, size_t count )\n ^~~~~\n", "include/common/ist.h:578:77: warning: unused parameter ‘count’ [-Wunused-parameter]\n static inline struct ist ist2str_uc(char *dst, const struct ist src, size_t count )\n ^~~~~\n" ] Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- stats/build-pkg.rb | 73 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100755 stats/build-pkg.rb diff --git a/stats/build-pkg.rb b/stats/build-pkg.rb new file mode 100755 index 00000000..a987e318 --- /dev/null +++ b/stats/build-pkg.rb @@ -0,0 +1,73 @@ +#!/usr/bin/env ruby + +LKP_SRC ||= ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) + +require 'set' +require "#{LKP_SRC}/lib/common" + +class ErrorMessages + def initialize + @seqno = '' + @in_stderr = false + @error_line = '' + @error_details = '' + @error_message = {} + @error_messages = Hash.new { |h, k| h[k] = Set.new } + end + + def obtain_error_messages(log_lines) + log_lines.each do |line| + next if extract_error_message(line) + next unless @in_stderr + next unless @error_message['error_line'] =~ /(error|warning):[^:]/i + + error_id = build_pkg_error_id(@error_message['error_line']) + @error_messages[error_id] << @error_message['error_line'] + @error_message['error_details'] + end + return @error_messages + end + + private + + def extract_error_message(line) + if line =~ /^ / + @error_details += line + else + update_error_message(line) + case line + when /^## ______________([0-9.]+):stderr$/ + @in_stderr = true + @seqno = $1 + return true + + when /^## ______________#{@seqno}:enderr$/ + @in_stderr = false + @seqno = '' + add_error_message + end + end + end + + def update_error_message(line) + @error_message['error_line'] = @error_line + @error_message['error_details'] = @error_details + @error_line = line + @error_details = '' + end + + def add_error_message + return unless @error_message['error_line'] =~ /(error|warning):[^:]/i + + error_id = build_pkg_error_id(@error_message['error_line']) + @error_messages[error_id] << @error_message['error_line'] + @error_message['error_details'] + end +end + +def build_pkg(log_lines) + message = {} + ErrorMessages.new.obtain_error_messages(log_lines).each do |k, v| + message["#{k}.message"] = v.to_a + end + + return message +end -- 2.23.0

1 0

[PATCH lkp-tests 1/2] stats/build-pkg: don't initialize LKP_SRC repeatedly
by Lin Jiaxin 31 Mar '21

31 Mar '21

error message: /home/linjiaxin/lkp-tests/stats/build-pkg.rb:3: warning: already initialized constant LKP_SRC Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- stats/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/stats/build-pkg b/stats/build-pkg index 764594b7..0caf1a73 100755 --- a/stats/build-pkg +++ b/stats/build-pkg @@ -1,6 +1,6 @@ #!/usr/bin/env ruby -LKP_SRC = ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) +LKP_SRC ||= ENV['LKP_SRC'] || File.dirname(File.dirname(File.realpath($PROGRAM_NAME))) require 'set' require "#{LKP_SRC}/lib/common" -- 2.23.0

1 0

[PATCH compass-ci 2/4] src/scheduler/kernel_params.cr: add param pv_device into dracut initrd for local mount
by Xu Xijian 31 Mar '21

31 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/kernel_params.cr | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index 8ad5029..9e2f58b 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -9,7 +9,8 @@ class Job os_info = "#{os}_#{os_arch}_#{os_version}" use_root_partition = "/dev/mapper/os-#{os_info}_#{src_lv_suffix}" if @hash["src_lv_suffix"]? != nil save_root_partition = "/dev/mapper/os-#{os_info}_#{boot_lv_suffix}" if @hash["boot_lv_suffix"]? != nil - return "#{common_params} local use_root_partition=#{use_root_partition} save_root_partition=#{save_root_partition} rw" + pv_device = "#{self.pv_device}" if @hash["pv_device"]? != nil + return "#{common_params} local use_root_partition=#{use_root_partition} save_root_partition=#{save_root_partition} pv_device=#{pv_device} rw" end private def kernel_custom_params -- 2.23.0

2 2

[PATCH compass-ci 4/4] container/dracut-initrd: enable creating pv and vg for lv to support local mount in HW machine
by Xu Xijian 31 Mar '21

31 Mar '21

[why] If a HW machine has no physical volume or volume group for logical volume, we should support to create them by job param, this param named as pv_device(default empty and do nothing). If a usable $pv_device is given, dracut will check the device and ensure pv and vg. Usage: submit job.yaml ... pv_device=/dev/sda Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- .../dracut-initrd/bin/set-local-sysroot.sh | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/container/dracut-initrd/bin/set-local-sysroot.sh b/container/dracut-initrd/bin/set-local-sysroot.sh index 835bdd2..2a07270 100644 --- a/container/dracut-initrd/bin/set-local-sysroot.sh +++ b/container/dracut-initrd/bin/set-local-sysroot.sh @@ -14,9 +14,36 @@ analyse_kernel_cmdline_params() { sync_src_lv() { local src_lv="$1" + local vg_name="os" [ -e "$src_lv" ] && return + # need create volume group, usually in first use of this machine. $pv_device e.g. /dev/sda + pv_device="$(getarg pv_device=)" + [ -n "$pv_device" ] && { + [ -b "$pv_device" ] || { + echo "warn dracut: FATAL: device not found: $pv_device, reboot" + reboot + } + + # ensure the physical disk has been initialized as physical volume + real_pv_device="$(lvm pvs | grep -w $pv_device | awk '{print $1}')" + [ "$real_pv_device" = "$pv_device" ] || { + lvm pvcreate "$pv_device" || reboot + } + + # ensure the volume group $vg_name exists + real_vg_name="$(lvm pvs | grep -w $vg_name | awk '{print $2}')" + [ "$real_vg_name" = "$vg_name" ] || { + lvm vgcreate "$vg_name" "$pv_device" || reboot + } + } + + lvm vgs "$vg_name" || { + echo "warn dracut: FATAL: vg os not found, reboot" + reboot + } + # create logical volume src_lv_devname="$(basename $src_lv)" lvm lvcreate -y -L 10G --name "${src_lv_devname#os-}" os -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] container/es: upgrade version
by Wu Zhende 31 Mar '21

31 Mar '21

Upgrade to 7.11.1. Matches the version of kibana so that data can be displayed on kibana. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/es/Dockerfile | 48 ++++++++++++-------------------------- container/es/build | 4 ++-- container/es/start | 2 +- container/logging-es/build | 8 +++---- container/logging-es/start | 2 +- 5 files changed, 23 insertions(+), 41 deletions(-) diff --git a/container/es/Dockerfile b/container/es/Dockerfile index ed02490..44d87ee 100644 --- a/container/es/Dockerfile +++ b/container/es/Dockerfile @@ -1,41 +1,24 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -FROM alpine:3.11 +FROM elasticsearch:7.11.1@sha256:d52cda1e73d1b1915ba2d76ca1e426620c7b5d6942d9d2f432259503974ba786 ARG MEMORY -RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories - -RUN apk add --no-cache elasticsearch curl - -RUN rm -rf /etc/init.d/elasticsearch \ - && rm -rf /usr/share/java/elasticsearch/plugins \ - && mv /usr/share/java/elasticsearch /usr/share/es \ - && echo "===> Creating Elasticsearch Paths..." \ - && for path in \ - /srv/es \ - /usr/share/es/logs \ - /usr/share/es/config \ - /usr/share/es/config/scripts \ - /usr/share/es/tmp \ - /usr/share/es/plugins \ - ; do \ - mkdir -p "$path"; \ - done \ - && cp /etc/elasticsearch/*.* /usr/share/es/config \ - && chown -R 1090:1090 /usr/share/es \ - && chown -R 1090:1090 /srv/es; - -RUN sed -i 's:#path.data\: /path/to/data:path.data\: /srv/es:' /usr/share/es/config/elasticsearch.yml; -RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/es/config/elasticsearch.yml; -RUN sed -i "s/-Xms256m/-Xms${MEMORY}m/g" /usr/share/es/config/jvm.options -RUN sed -i "s/-Xmx256m/-Xmx${MEMORY}m/g" /usr/share/es/config/jvm.options - -WORKDIR /usr/share/es - -ENV PATH /usr/share/es/bin:$PATH -ENV ES_TMPDIR /usr/share/es/tmp +RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a path.data: /srv/es' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a node.name: node-1' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i "s/-Xms256m/-Xms${MEMORY}m/g" /usr/share/elasticsearch/config/jvm.options && \ + sed -i "s/-Xmx256m/-Xmx${MEMORY}m/g" /usr/share/elasticsearch/config/jvm.options + +RUN mkdir /usr/share/elasticsearch/tmp && \ + chown -R 1090:1090 /usr/share/elasticsearch + +WORKDIR /usr/share/elasticsearch + +ENV PATH /usr/share/elasticsearch/bin:$PATH +ENV ES_TMPDIR /usr/share/elasticsearch/tmp VOLUME ["/srv/es"] @@ -43,4 +26,3 @@ EXPOSE 9200 9300 USER 1090 CMD ["elasticsearch"] - diff --git a/container/es/build b/container/es/build index c7d7115..db5145f 100755 --- a/container/es/build +++ b/container/es/build @@ -5,8 +5,8 @@ require_relative '../defconfig.rb' -docker_skip_rebuild "es643b:alpine311" +docker_skip_rebuild "es:7.11.1" available_memory = get_available_memory -system "docker build -t es643b:alpine311 --build-arg MEMORY=#{available_memory} --network=host ." +system "docker build -t es:7.11.1 --build-arg MEMORY=#{available_memory} --network=host ." diff --git a/container/es/start b/container/es/start index 67d6531..3aa9525 100755 --- a/container/es/start +++ b/container/es/start @@ -15,7 +15,7 @@ cmd=( -v /srv/es:/srv/es -v /etc/localtime:/etc/localtime:ro --name es-server01 - es643b:alpine311 + es:7.11.1 ) "${cmd[@]}" diff --git a/container/logging-es/build b/container/logging-es/build index 3be841a..b50830e 100755 --- a/container/logging-es/build +++ b/container/logging-es/build @@ -5,15 +5,15 @@ require_relative '../defconfig.rb' -docker_skip_rebuild "logging-es:7.6.2" +docker_skip_rebuild "logging-es:7.11.1" BASE_IMAGE_DICT = { - 'aarch64' => 'gagara/elasticsearch-oss-arm64:7.6.2', - 'x86_64' => 'elasticsearch:7.6.2' + 'aarch64' => 'elasticsearch:7.11.1@sha256:d52cda1e73d1b1915ba2d76ca1e426620c7b5d6942d9d2f432259503974ba786', + 'x86_64' => 'elasticsearch:7.11.1' }.freeze BASE_IMAGE = BASE_IMAGE_DICT[%x(arch).chomp] available_memory = get_available_memory -system "docker build -t logging-es:7.6.2 --build-arg BASE_IMAGE=#{BASE_IMAGE} --build-arg MEMORY=#{available_memory} ." +system "docker build -t logging-es:7.11.1 --build-arg BASE_IMAGE=#{BASE_IMAGE} --build-arg MEMORY=#{available_memory} ." diff --git a/container/logging-es/start b/container/logging-es/start index 9be7a5b..05ac46f 100755 --- a/container/logging-es/start +++ b/container/logging-es/start @@ -15,7 +15,7 @@ cmd=( -p 9302:9300 -v /srv/es/logging-es:/srv/es/logging-es --name logging-es - logging-es:7.6.2 + logging-es:7.11.1 ) "${cmd[@]}" -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] container/kibana: upgrade and addition
by Wu Zhende 31 Mar '21

31 Mar '21

1. upgrade from 7.6.2 to 7.11.1 2. add a new kibana for data. one display log and one display data 3. delete logtrail plugin because it don't support in new kibana version Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/kibana-logging/Dockerfile | 12 ++++++++ container/kibana-logging/build | 16 ++++++++++ container/kibana-logging/start | 31 +++++++++++++++++++ container/kibana-logging/start-depends | 1 + container/kibana/Dockerfile | 10 ------- container/kibana/build | 8 ++--- container/kibana/logtrail.json | 41 -------------------------- container/kibana/start | 14 ++++----- container/kibana/start-depends | 2 +- 9 files changed, 72 insertions(+), 63 deletions(-) create mode 100644 container/kibana-logging/Dockerfile create mode 100755 container/kibana-logging/build create mode 100755 container/kibana-logging/start create mode 100755 container/kibana-logging/start-depends delete mode 100644 container/kibana/logtrail.json diff --git a/container/kibana-logging/Dockerfile b/container/kibana-logging/Dockerfile new file mode 100644 index 0000000..970eb5a --- /dev/null +++ b/container/kibana-logging/Dockerfile @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +ARG BASE_IMAGE + +FROM ${BASE_IMAGE} + +# docker image borrowed from hub.docker.com/r/gagara/kibana-oss-arm64 + +MAINTAINER Wu Zhende <wuzhende666(a)163.com> + +RUN sed -i 's/server.host: "0"/server.host: "0.0.0.0"/' config/kibana.yml diff --git a/container/kibana-logging/build b/container/kibana-logging/build new file mode 100755 index 0000000..9c75aeb --- /dev/null +++ b/container/kibana-logging/build @@ -0,0 +1,16 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require_relative '../defconfig' + +docker_skip_rebuild "kibana:7.11.1" + +BASE_IMAGE_DICT = { + 'aarch64' => 'jamesgarside/kibana:7.11.1', + 'x86_64' => 'kibana:7.11.1' }.freeze + +BASE_IMAGE = BASE_IMAGE_DICT[%x(arch).chomp] + +system "docker build -t kibana:7.11.1 --build-arg BASE_IMAGE=#{BASE_IMAGE} ." diff --git a/container/kibana-logging/start b/container/kibana-logging/start new file mode 100755 index 0000000..f2169a6 --- /dev/null +++ b/container/kibana-logging/start @@ -0,0 +1,31 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +names = Set.new %w[ + LOGGING_ES_HOST + LOGGING_ES_PORT +] + +defaults = relevant_defaults(names) +LOGGING_ES_HOST = defaults['LOGGING_ES_HOST'] || '172.17.0.1' +LOGGING_ES_PORT = defaults['LOGGING_ES_PORT'] || '9202' + +docker_rm 'kibana-logging' + +cmd = %W[ + docker run + --restart=always + --name kibana-logging + -v /etc/localtime:/etc/localtime:ro + -d + -e ELASTICSEARCH_HOSTS=http://#{LOGGING_ES_HOST}:#{LOGGING_ES_PORT} + -p 20000:5601 + kibana:7.11.1 +] + +system(*cmd) diff --git a/container/kibana-logging/start-depends b/container/kibana-logging/start-depends new file mode 100755 index 0000000..66c1996 --- /dev/null +++ b/container/kibana-logging/start-depends @@ -0,0 +1 @@ +logging-es diff --git a/container/kibana/Dockerfile b/container/kibana/Dockerfile index 60b889d..970eb5a 100644 --- a/container/kibana/Dockerfile +++ b/container/kibana/Dockerfile @@ -10,13 +10,3 @@ FROM ${BASE_IMAGE} MAINTAINER Wu Zhende <wuzhende666(a)163.com> RUN sed -i 's/server.host: "0"/server.host: "0.0.0.0"/' config/kibana.yml - -USER root -RUN yum -y install wget \ - && wget https://github.com/sivasamyk/logtrail/releases/download/v0.1.31/logtrail-7.… -O /logtrail-7.6.2-0.1.31.zip - -USER 1090 -RUN ./bin/kibana-plugin install file:///logtrail-7.6.2-0.1.31.zip \ - && rm -rf /tmp/logtrail-7.6.2-0.1.31.zip - -COPY --chown=1090:1090 logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json diff --git a/container/kibana/build b/container/kibana/build index 9a1fb6d..9c75aeb 100755 --- a/container/kibana/build +++ b/container/kibana/build @@ -5,12 +5,12 @@ require_relative '../defconfig' -docker_skip_rebuild "kibana:7.6.2" +docker_skip_rebuild "kibana:7.11.1" BASE_IMAGE_DICT = { - 'aarch64' => 'gagara/kibana-oss-arm64:7.6.2', - 'x86_64' => 'kibana:7.6.2' }.freeze + 'aarch64' => 'jamesgarside/kibana:7.11.1', + 'x86_64' => 'kibana:7.11.1' }.freeze BASE_IMAGE = BASE_IMAGE_DICT[%x(arch).chomp] -system "docker build -t kibana:7.6.2 --build-arg BASE_IMAGE=#{BASE_IMAGE} ." +system "docker build -t kibana:7.11.1 --build-arg BASE_IMAGE=#{BASE_IMAGE} ." diff --git a/container/kibana/logtrail.json b/container/kibana/logtrail.json deleted file mode 100644 index c63f2ac..0000000 --- a/container/kibana/logtrail.json +++ /dev/null @@ -1,41 +0,0 @@ -{ - "version": 2, - "index_patterns": [{ - "es": { - "default_index": "*", - "allow_url_parameter": false, - "timezone": "CST" - }, - "tail_interval_in_seconds": 10, - "es_index_time_offset_in_seconds": 0, - "display_timezone": "CST", - "display_timestamp_format": "YYYY MMM DD HH:mm:ss", - "max_buckets": 500, - "nested_objects": false, - "default_time_range_in_days": 30, - "max_hosts": 100, - "max_events_to_keep_in_viewer": 10000, - "default_search": "", - "fields": { - "mapping": { - "timestamp": "time", - "display_timestamp" : "time", - "hostname": "container_name", - "program": "tags", - "message": "log" - }, - "message_format": "{{{log}}}", - "keyword_suffix": "keyword" - }, - "color_mapping": { - "field": "level", - "mapping": { - "ERROR": "#FF0000", - "WARN": "#FFEF96", - "DEBUG": "#B5E7A0", - "TRACE": "#CFE0E8", - "INFO": "#339999" - } - } - }] -} diff --git a/container/kibana/start b/container/kibana/start index 26a4944..63cb8f8 100755 --- a/container/kibana/start +++ b/container/kibana/start @@ -7,13 +7,13 @@ require 'set' require_relative '../defconfig.rb' names = Set.new %w[ - LOGGING_ES_HOST - LOGGING_ES_PORT + ES_HOST + ES_PORT ] defaults = relevant_defaults(names) -LOGGING_ES_HOST = defaults['LOGGING_ES_HOST'] || '172.17.0.1' -LOGGING_ES_PORT = defaults['LOGGING_ES_PORT'] || '9202' +ES_HOST = defaults['ES_HOST'] || '172.17.0.1' +ES_PORT = defaults['ES_PORT'] || '9200' docker_rm 'kibana' @@ -23,9 +23,9 @@ cmd = %W[ --name kibana -v /etc/localtime:/etc/localtime:ro -d - -e ELASTICSEARCH_HOSTS=http://#{LOGGING_ES_HOST}:#{LOGGING_ES_PORT} - -p 20000:5601 - kibana:7.6.2 + -e ELASTICSEARCH_HOSTS=http://#{ES_HOST}:#{ES_PORT} + -p 20017:5601 + kibana:7.11.1 ] system(*cmd) diff --git a/container/kibana/start-depends b/container/kibana/start-depends index 66c1996..8357fca 100755 --- a/container/kibana/start-depends +++ b/container/kibana/start-depends @@ -1 +1 @@ -logging-es +es -- 2.23.0

1 0

[PATCH compass-ci] lib/git_mirror.rb: fix no positive? method error
by Li Yuanchao 31 Mar '21

31 Mar '21

The result of query es changed, the result['hits']['total'] changes from a number to a hash, so the positive? method can not be used. Now use the count method which contained by es_client. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index c967c8b..7f36be3 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -414,9 +414,9 @@ class MirrorMain new_refs_count: {} } query = { query: { match: { _id: git_repo } } } - result = @es_client.search(index: 'repo', body: query)['hits'] - return fork_stat unless result['total'].positive? + return fork_stat unless @es_client.count(index: 'repo', body: query)['count'].positive? + result = @es_client.search(index: 'repo', body: query)['hits'] fork_stat.each_key do |key| fork_stat[key] = result['hits'][0]['_source'][key.to_s] || fork_stat[key] end -- 2.23.0

1 0