November 2020 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci] sbin/compare optimize help message
by Lu Weitao 13 Nov '20

13 Nov '20

[why] compare tool support output result with json | html format, but have no corresponding help message. [how] add the corresponding help message in --theme option [example] compare id=crystal.109482 id=crystal.109483 --theme json Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/compare | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/sbin/compare b/sbin/compare index b13682e..baa90d4 100755 --- a/sbin/compare +++ b/sbin/compare @@ -12,7 +12,7 @@ # compare "commit=a12d232e" "commit=b3bacc31" # compare "os=debian" "os=centos" -c "suite=iperf" # compare "os=centos" -d "os_version os_arch" -# compare "os=centos" -d "os_version os_arch" --color "classic" +# compare "os=centos" -d "os_version os_arch" --theme "classic" require 'optparse' require_relative '../lib/compare.rb' @@ -42,9 +42,9 @@ opt_parser = OptionParser.new do |opts| is_group = true end - opts.on('--color color', 'turn on colorful display with theme: classic|focus_good|focus_bad', - '|striking|light|none') do |color| - colorful = color + opts.on('--theme theme', 'turn on colorful display with theme: classic|focus_good|focus_bad', + '|striking|light|json|html') do |theme| + colorful = theme end opts.on('-t', '--template template', 'compare with user-defined template') do |t| -- 2.23.0

2 2

[PATCH v2 compass-ci] container/master-fluentd: fix stop sending logs to elasticsearch
by Wu Zhende 13 Nov '20

13 Nov '20

[Why] fluentd stop sending logs to elasticsearch after a few hours [Error] fluent.warn: [es] failed to flush the buffer. retry_time=19 error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch, Connection refused - connect(2) for 172.17.0.2:9200 (Errno::ECONNREFUSED)" [Reference] https://github.com/fluent/fluentd/issues/2334 https://github.com/uken/fluent-plugin-elasticsearch Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/master-fluentd/docker-fluentd.conf | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/master-fluentd/docker-fluentd.conf b/container/master-fluentd/docker-fluentd.conf index 62c0319..5de7901 100644 --- a/container/master-fluentd/docker-fluentd.conf +++ b/container/master-fluentd/docker-fluentd.conf @@ -49,6 +49,9 @@ ssl_verify false log_es_400_reason true with_transporter_log true + reconnect_on_error true + reload_on_failure true + reload_connections false </store> <store> -- 2.23.0

1 0

[PATCH v6 compass-ci 2/3] jobfile_operate.cr: optimize the function self.parse_one
by Xu Xijian 12 Nov '20

12 Nov '20

Optimize the structure of this function: 1. reduce if nesting 2. simplify the code for good reading Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/jobfile_operate.cr | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 4ec9f96..16a4d06 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -56,16 +56,10 @@ module Jobfile::Operate end def self.parse_one(script_lines, key, val) - return false if val.as_h? - - if valid_shell_variable?(key) - value = if val.as_a? - shell_escape(val.as_a) - else - shell_escape(val.to_s) - end - script_lines << "\texport #{key}=" + value if value - end + return false if val.as_h? || !valid_shell_variable?(key) + + value = shell_escape(val.as_a? || val.to_s) + script_lines << "\texport #{key}=" + value if value end def self.sh_export_top_env(job_content : Hash) -- 2.23.0

1 0

[PATCH v5 compass-ci 3/3] kernel_version.md: add default vmlinuz in Related files example
by Xu Xijian 12 Nov '20

12 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index f6a5f4d..f9c1a8b 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -34,6 +34,7 @@ Related files: ├── modules-4.19.90-2003.cgz ├── modules.cgz -> modules-4.19.90-2003.cgz ├── vmlinuz-4.19.90-2003 +└── vmlinuz -> vmlinuz-4.19.90-2003 Usage example: - submit iperf.yaml testbox=vm-2p8g--$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 -- 2.23.0

3 4

[PATCH compass-ci] container/master-fluentd: fix stop sending logs to elasticsearch
by Wu Zhende 12 Nov '20

12 Nov '20

[Why] fluentd stop sending logs to elasticsearch after a few hours [Error] fluent.warn: [es] failed to flush the buffer. retry_time=19 error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure error="could not push logs to Elasticsearch, Connection refused - connect(2) for 172.17.0.2:9200 (Errno::ECONNREFUSED)" [Reference] https://github.com/fluent/fluentd/issues/2334 https://github.com/uken/fluent-plugin-elasticsearch Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/master-fluentd/docker-fluentd.conf | 3 +++ 1 file changed, 3 insertions(+) diff --git a/container/master-fluentd/docker-fluentd.conf b/container/master-fluentd/docker-fluentd.conf index 62c0319..5de7901 100644 --- a/container/master-fluentd/docker-fluentd.conf +++ b/container/master-fluentd/docker-fluentd.conf @@ -49,6 +49,9 @@ ssl_verify false log_es_400_reason true with_transporter_log true + reconnect_on_error true + reload_on_failure true + reload_connections false </store> <store> -- 2.23.0

2 1

[PATCH v5 compass-ci 2/3] jobfile_operate.cr: optimize the function self.parse_one
by Xu Xijian 12 Nov '20

12 Nov '20

Optimize the structure of this function: 1. reduce if nesting 2. simplify the code for good reading Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/jobfile_operate.cr | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 4ec9f96..16a4d06 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -56,16 +56,10 @@ module Jobfile::Operate end def self.parse_one(script_lines, key, val) - return false if val.as_h? - - if valid_shell_variable?(key) - value = if val.as_a? - shell_escape(val.as_a) - else - shell_escape(val.to_s) - end - script_lines << "\texport #{key}=" + value if value - end + return false if val.as_h? || !valid_shell_variable?(key) + + value = shell_escape(val.as_a? || val.to_s) + script_lines << "\texport #{key}=" + value if value end def self.sh_export_top_env(job_content : Hash) -- 2.23.0

1 0

[PATCH v5 compass-ci 1/3] fix: avoid to parse iscsiroot in setup-dracut.sh
by Xu Xijian 12 Nov '20

12 Nov '20

[why] When start with a new kernel, the dracut will parse iscsiroot and halted, which is unnecessary for us. [errmsg] [ 0.976307] dracut: FATAL: iscsiroot requested but kernel/initrd does not support iscsi [ 0.977011] dracut: Refusing to continue ... [ 1.063137] reboot: System halted Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 2e676de..b9a7800 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -14,5 +14,6 @@ rm -rf /var/lib/apt/lists/* # Replace the runtime shell script with a custom shell script cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ +sed -i '/inst_hook cmdline 90 "$moddir\/parse-iscsiroot.sh"/d' /usr/lib/dracut/modules.d/95iscsi/module-setup.sh cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh -- 2.23.0

1 0

[PATCH v2 lkp-tests] test/build-pkg: fix a softlink will be established if the package fails
by Sun Yukui 12 Nov '20

12 Nov '20

[why] After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the soft connection should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index ad8d4843..093bbe4d 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,19 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -86,24 +99,11 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 -} -create_softlink() -{ - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" + [ "$?" == 0 ] || create_softlink } check_vars mount_dest request_pkg build_source_pkg -create_softlink -- 2.23.0

4 4

[PATCH v4 compass-ci 1/3] kernel_version.md: add default vmlinuz in Related files example
by Xu Xijian 12 Nov '20

12 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/job/kernel_version.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/job/kernel_version.md b/doc/job/kernel_version.md index f6a5f4d..23c132a 100644 --- a/doc/job/kernel_version.md +++ b/doc/job/kernel_version.md @@ -34,6 +34,7 @@ Related files: ├── modules-4.19.90-2003.cgz ├── modules.cgz -> modules-4.19.90-2003.cgz ├── vmlinuz-4.19.90-2003 +├── vmlinuz -> vmlinuz-4.19.90-2003 Usage example: - submit iperf.yaml testbox=vm-2p8g--$USER os=openeuler os_arch=aarch64 os_version=20.03 runtime=20 kernel_version=4.19.90-2003 -- 2.23.0

3 4

[PATCH v4 compass-ci 2/3] jobfile_operate.cr: reduce if nesting levels
by Xu Xijian 12 Nov '20

12 Nov '20

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- src/scheduler/jobfile_operate.cr | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/src/scheduler/jobfile_operate.cr b/src/scheduler/jobfile_operate.cr index 4ec9f96..16a4d06 100644 --- a/src/scheduler/jobfile_operate.cr +++ b/src/scheduler/jobfile_operate.cr @@ -56,16 +56,10 @@ module Jobfile::Operate end def self.parse_one(script_lines, key, val) - return false if val.as_h? - - if valid_shell_variable?(key) - value = if val.as_a? - shell_escape(val.as_a) - else - shell_escape(val.to_s) - end - script_lines << "\texport #{key}=" + value if value - end + return false if val.as_h? || !valid_shell_variable?(key) + + value = shell_escape(val.as_a? || val.to_s) + script_lines << "\texport #{key}=" + value if value end def self.sh_export_top_env(job_content : Hash) -- 2.23.0

2 2