- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci 2/2] sparrow/3-code: delete special symbols in vars values
by Liu Yinsi 28 Apr '21

28 Apr '21

[why] special symbols will be parsed in shell command "docker build -t --build-arg $PASSWORD ...", caused failed to build images, so delete. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index fe86b6cf..014221ee 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -190,7 +190,7 @@ auto-submit: EOF [[ -f /etc/compass-ci/passwd.yaml ]] || { -passwd=$(< /dev/urandom tr -dc "a-zA-Z0-9-=+@#_:/[](){}$%&.^*\!" | head -c 10;echo) +passwd=$(< /dev/urandom tr -dc "a-zA-Z0-9" | head -c 10; echo) cat > /etc/compass-ci/passwd.yaml <<EOF ES_USER: root -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] sparrow/1-storage: set permission /etc/compass-ci/password.yaml 660
by Liu Yinsi 28 Apr '21

28 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/1-storage/permission | 1 + sparrow/1-storage/tiny | 1 + 2 files changed, 2 insertions(+) diff --git a/sparrow/1-storage/permission b/sparrow/1-storage/permission index 3c229c5a..3a8bfdaa 100755 --- a/sparrow/1-storage/permission +++ b/sparrow/1-storage/permission @@ -44,5 +44,6 @@ chgrp team /srv/cci/serial/logs chgrp team /srv/dc chmod g+ws /srv/dc +chmod 660 /etc/compass-ci/password.yaml find /tftpboot -type d | xargs chmod 775 diff --git a/sparrow/1-storage/tiny b/sparrow/1-storage/tiny index c7636b02..1dbe8a32 100755 --- a/sparrow/1-storage/tiny +++ b/sparrow/1-storage/tiny @@ -22,6 +22,7 @@ dirs=( /c /c/git-repos /etc/qemu + /etc/compass-ci/password.yaml /srv/cci/Maildir/.compass-ci/new /srv/cci/Maildir/.compass-ci/cur /srv/cci/Maildir/.compass-ci/sent -- 2.23.0

1 0

[PATCH compass-ci] container: add build depends
by Liu Yinsi 28 Apr '21

28 Apr '21

delimiter's and updaterepo's base image is scheduler-dev. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/delimiter/build-depends | 1 + container/updaterepo/build-depends | 1 + 2 files changed, 2 insertions(+) create mode 100755 container/updaterepo/build-depends diff --git a/container/delimiter/build-depends b/container/delimiter/build-depends index 14bc599b..2ec2b37a 100755 --- a/container/delimiter/build-depends +++ b/container/delimiter/build-depends @@ -1 +1,2 @@ registry +scheduler-dev diff --git a/container/updaterepo/build-depends b/container/updaterepo/build-depends new file mode 100755 index 00000000..5dd403e1 --- /dev/null +++ b/container/updaterepo/build-depends @@ -0,0 +1 @@ +scheduler-dev -- 2.23.0

1 0

[PATCH lkp-tests] lib/bootstrap.sh: remove packages after installation
by Wei Jihui 28 Apr '21

28 Apr '21

[why] /opt/rpms is for save rpm package which the task depends. but if task is cci-depends, it mv $downloaded_rpms to /opt/rpms, then pack /opt/rpms/*.rpm, deps cgz will obtain useless rpm from the task depends. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/bootstrap.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 58f0531a7..417c8662a 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -411,6 +411,7 @@ install_rpms() [ -d /opt/rpms ] || return echo "install $(ls /opt/rpms/*.rpm)" rpm -ivh --force --ignoresize /opt/rpms/*.rpm + rm -f /opt/rpms/*.rpm } try_get_and_set_distro() -- 2.23.0

1 0

[PATCH lkp-tests] add email feedback for rpmbuild-pkg
by Li Ping 28 Apr '21

28 Apr '21

tests/rpmbuild-pkg: add a new input parameter email [usage]: rpmbuild-pkg: email: 1 when job failed, send an rpmbuild failed email. when job run success, send an rpmbuild success email. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/email.sh | 41 +++++++++++++++++++++++++++++++++++++++++ tests/rpmbuild-pkg | 10 ++++++++++ 2 files changed, 51 insertions(+) diff --git a/lib/email.sh b/lib/email.sh index b5e56df79..613d9fd70 100755 --- a/lib/email.sh +++ b/lib/email.sh @@ -96,6 +96,47 @@ Compass-CI " } +git_pretty_format() +{ + curl -sS -H 'Content-Type: Application/json' -XPOST "$LKP_SERVER"':8100/git_command' -d '{"git_repo": "'${upstream_dir}/${upstream_repo}.git'", "git_command": ["git-log","-n1","--pretty=format:'$1'", "'${upstream_commit}'"]}' +} + +rpmbuild_success_content() +{ + local author_email=$(git_pretty_format "%ae") + local author_name=$(git_pretty_format "%an") + email_content="To: $author_email +Subject: [NOTIFY Compass-CI] rpmbuild report + +Dear $author_name: + + Your RPM Package is successfully built. + + You can click the following link to view details: + http://api.compass-ci.openeuler.org:20012/rpm/pub/${os}-${os_version}/${com… + +Regards +Compass-CI +" +} + +rpmbuild_failed_content() +{ + local author_email=$(git_pretty_format "%ae") + local author_name=$(git_pretty_format "%an") + email_content="To: $author_email +Subject: [NOTIFY Compass-CI] rpmbuild report + +Dear $author_name: + + Sorry to inform you that RPM Package built failed, you can click the following link to view details. + http://api.compass-ci.openeuler.org:${SRV_HTTP_RESULT_PORT:-20007}$result_r… + +Regards +Compass-CI +" +} + send_email() { local subject=$1 diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 3dd75b896..8edcb9d1a 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -5,6 +5,7 @@ # - repo_addr # - custom_repo_name # - custom_repo_addr +# - email # Git repo jobs only use the first two parameters, SRPMs jobs use all # user can add custom_repo_name and custom_repo_addr to setup # local repo, value can be an array @@ -12,6 +13,7 @@ . $LKP_SRC/lib/debug.sh . $LKP_SRC/lib/upload.sh . $LKP_SRC/lib/rpm.sh +. $LKP_SRC/lib/email.sh : "${compat_os:=budding-openeuler}" @@ -19,6 +21,13 @@ dest_dir="/rpm/upload/${os}-${os_version}/${compat_os}" +die() +{ + [ -n "$email" ] && send_email "rpmbuild_failed" + echo "$@" 1>&2 + exit 99 +} + from_git() { package_name=${upstream_repo##*/} @@ -143,3 +152,4 @@ add_repo build_rpm upload_rpm_pkg update_repo_mq +[ -n "$email" ] && send_email "rpmbuild_success" -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] kibana/dashboard_import: import kibana dashboards
by Wu Zhende 28 Apr '21

28 Apr '21

Use this script can import kibana dashboards by json files. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/kibana/dashboard_import | 40 +++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 container/kibana/dashboard_import diff --git a/container/kibana/dashboard_import b/container/kibana/dashboard_import new file mode 100755 index 0000000..e20e8ff --- /dev/null +++ b/container/kibana/dashboard_import @@ -0,0 +1,40 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'pathname' +require_relative '../defconfig.rb' + +names = Set.new %w[ + KIBANA_HOST + KIBANA_PORT + ES_USER + ES_PASSWORD +] + +defaults = relevant_defaults(names) +KIBANA_HOST = defaults['KIBANA_HOST'] || '172.17.0.1' +KIBANA_PORT = defaults['KIBANA_PORT'] || '20017' + +service_authentication = relevant_service_authentication(names) + +files = ARGV +if files.empty? + profile = Pathname.new(File.dirname(__FILE__)).realpath + files << profile + 'data.json' + files << profile + 'resource.json' + files << profile + 'all.json' +end + +cmd = "curl -XPOST -u #{service_authentication['ES_USER']}:#{service_authentication['ES_PASSWORD']}" +cmd += " http://#{KIBANA_HOST}:#{KIBANA_PORT}/api/kibana/dashboards/import" +cmd += " -H 'Content-Type: application/json' -H 'kbn-xsrf: reporting'" + +files.each do |f| + new_cmd = cmd + new_cmd += " -d @#{f}" + puts new_cmd + system new_cmd +end -- 2.23.0

2 1

[PATCH compass-ci] scheduler-https/start: fix start error
by Wu Zhende 28 Apr '21

28 Apr '21

Missing ENV key: "NODE_PORT" Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/scheduler-https/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/scheduler-https/start b/container/scheduler-https/start index 5e79d2b..59e7278 100755 --- a/container/scheduler-https/start +++ b/container/scheduler-https/start @@ -69,6 +69,7 @@ cmd = %W[ ] + env + %W[ -e LKP_SRC=#{DEFAULT_LKP} -e CCI_SRC=#{DEFAULT_CCI} + -e NODE_PORT=#{SCHED_HTTPS_PORT} -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} -v #{CCI_REPOS}/lab-#{LAB}:#{CCI_REPOS}/lab-#{LAB} -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] kibana/dashboard_export: exporting kibana dashboards
by Wu Zhende 28 Apr '21

28 Apr '21

Use this script can export kibana dashboards by ids. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/kibana/dashboard_export | 33 +++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100755 container/kibana/dashboard_export diff --git a/container/kibana/dashboard_export b/container/kibana/dashboard_export new file mode 100755 index 0000000..1532aae --- /dev/null +++ b/container/kibana/dashboard_export @@ -0,0 +1,33 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +names = Set.new %w[ + KIBANA_HOST + KIBANA_PORT + ES_USER + ES_PASSWORD +] + +defaults = relevant_defaults(names) +KIBANA_HOST = defaults['KIBANA_HOST'] || '172.17.0.1' +KIBANA_PORT = defaults['KIBANA_PORT'] || '20017' + +service_authentication = relevant_service_authentication(names) + +dashboard_ids = ARGV +dashboard_ids = %w[data resource all] if dashboard_ids.empty? + +cmd = "curl -u #{service_authentication['ES_USER']}:#{service_authentication['ES_PASSWORD']}" +cmd += " http://#{KIBANA_HOST}:#{KIBANA_PORT}/api/kibana/dashboards/export?dashboard=" + +dashboard_ids.each do |id| + new_cmd = cmd + new_cmd += "#{id} >> #{id}.json" + puts new_cmd + system new_cmd +end -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] kibana/dashboard_import: import kibana dashboards
by Wu Zhende 28 Apr '21

28 Apr '21

Use this script can import kibana dashboards by json files. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/kibana/dashboard_import | 44 +++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100755 container/kibana/dashboard_import diff --git a/container/kibana/dashboard_import b/container/kibana/dashboard_import new file mode 100755 index 0000000..a6129a5 --- /dev/null +++ b/container/kibana/dashboard_import @@ -0,0 +1,44 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require 'pathname' +require_relative '../defconfig.rb' + +names = Set.new %w[ + KIBANA_HOST + KIBANA_PORT + ES_USER + ES_PASSWORD +] + +defaults = relevant_defaults(names) +KIBANA_HOST = defaults['KIBANA_HOST'] || '172.17.0.1' +KIBANA_PORT = defaults['KIBANA_PORT'] || '20017' + +service_authentication = relevant_service_authentication(names) + +files = [] +ARGV.each do |f| + files >> f +end + +if files.empty? + profile = Pathname.new(File.dirname(__FILE__)).realpath + files << profile + 'data.json' + files << profile + 'resource.json' + files << profile + 'all.json' +end + +cmd = "curl -XPOST -u #{service_authentication['ES_USER']}:#{service_authentication['ES_PASSWORD']}" +cmd += " http://#{KIBANA_HOST}:#{KIBANA_PORT}/api/kibana/dashboards/import" +cmd += " -H 'Content-Type: application/json' -H 'kbn-xsrf: reporting'" + +files.each do |f| + new_cmd = cmd + new_cmd += " -d @#{f}" + puts new_cmd + system new_cmd +end -- 2.23.0

2 2

[PATCH v2 compass-ci 1/2] kibana/dashboard_export: exporting kibana dashboards
by Wu Zhende 28 Apr '21

28 Apr '21

Use this script can export kibana dashboards by ids. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/kibana/dashboard_export | 37 +++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/kibana/dashboard_export diff --git a/container/kibana/dashboard_export b/container/kibana/dashboard_export new file mode 100755 index 0000000..d6e0054 --- /dev/null +++ b/container/kibana/dashboard_export @@ -0,0 +1,37 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative '../defconfig.rb' + +names = Set.new %w[ + KIBANA_HOST + KIBANA_PORT + ES_USER + ES_PASSWORD +] + +defaults = relevant_defaults(names) +KIBANA_HOST = defaults['KIBANA_HOST'] || '172.17.0.1' +KIBANA_PORT = defaults['KIBANA_PORT'] || '20017' + +service_authentication = relevant_service_authentication(names) + +dashboard_ids = [] +ARGV.each do |i| + dashboard_ids << i +end + +dashboard_ids = %w[data resource all] if dashboard_ids.empty? + +cmd = "curl -u #{service_authentication['ES_USER']}:#{service_authentication['ES_PASSWORD']}" +cmd += " http://#{KIBANA_HOST}:#{KIBANA_PORT}/api/kibana/dashboards/export?dashboard=" + +dashboard_ids.each do |id| + new_cmd = cmd + new_cmd += "#{id} >> #{id}.json" + puts new_cmd + system new_cmd +end -- 2.23.0

2 1