- Compass-ci - mailweb.openeuler.org

[PATCH v4 lkp-tests 1/4] test/build-pkg: Refactor move create_softlink function position
by Sun Yukui 17 Nov '20

17 Nov '20

[why] The create_softlink function needs to be called in build_source_pkg function. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 1eed784f..6426f2da 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,21 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + [ -e "$cgz_name" ] || return 0 + + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -88,22 +103,6 @@ build_source_pkg() $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 } -create_softlink() -{ - [ -e "$cgz_name" ] || return 0 - - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" -} - check_vars mount_dest request_pkg -- 2.23.0

1 0

[PATCH v3 lkp-tests 4/4] test/cci-makepkg: fix a softlink will be established if the package fails
by Sun Yukui 17 Nov '20

17 Nov '20

After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the softlink should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index 43ba04ad..a3b5e960 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -110,7 +110,8 @@ build_source_pkg(){ echo "$makepkg" PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + + [ "$?" == 0 ] && update_softlink } build_source_pkg -update_softlink && echo "Makepkg finished successfully" -- 2.23.0

1 0

[PATCH v3 lkp-tests 3/4] test/cci-makepkg: Refactor encapsulate the statement as a function
by Sun Yukui 17 Nov '20

17 Nov '20

[why] Encapsulating statements as functions is important for a better syntax. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/cci-makepkg | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/tests/cci-makepkg b/tests/cci-makepkg index a105f189..43ba04ad 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -92,10 +92,6 @@ cgz_name="${pkgver:-0}-${pkgrel:-0}.cgz" setup_proxy -echo $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf -PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" \ - $LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf - update_softlink() { [ -e "$sync_dest/$cgz_name" ] || return @@ -109,4 +105,12 @@ update_softlink() rm $sync_dest/$cgz_name } +build_source_pkg(){ + local makepkg="$LKP_SRC/sbin/makepkg -A --skippgpcheck --config $LKP_SRC/etc/makepkg.conf" + echo "$makepkg" + + PACMAN=true BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg +} + +build_source_pkg update_softlink && echo "Makepkg finished successfully" -- 2.23.0

1 0

[PATCH v3 lkp-tests 2/4] test/build-pkg: fix a softlink will be established if the package fails
by Sun Yukui 17 Nov '20

17 Nov '20

After packaging fails, a softlink will be established, but this is wrong. If the packaging fails, the softlink should not be established. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index 6426f2da..4aaa4fef 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -101,10 +101,11 @@ build_source_pkg() cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 + + [ "$?" == 0 ] && create_softlink } check_vars mount_dest request_pkg build_source_pkg -create_softlink -- 2.23.0

1 0

[PATCH v3 lkp-tests 1/4] test/build-pkg: Refactor move create_softlink function position
by Sun Yukui 17 Nov '20

17 Nov '20

[why] The create_softlink function needs to be called in build_source_pkg function. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 1eed784f..6426f2da 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,21 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + [ -e "$cgz_name" ] || return 0 + + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -88,22 +103,6 @@ build_source_pkg() $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 } -create_softlink() -{ - [ -e "$cgz_name" ] || return 0 - - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" -} - check_vars mount_dest request_pkg -- 2.23.0

1 0

[PATCH v2 lkp-tests 1/4] test/build-pkg: Refactor move create_softlink function position
by Sun Yukui 17 Nov '20

17 Nov '20

[why] The create_softlink function needs to be called in build_source_pkg function. Signed-off-by: Sun Yukui <sun.yukui(a)foxmail.com> --- tests/build-pkg | 31 +++++++++++++++---------------- 1 file changed, 15 insertions(+), 16 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index 1eed784f..6426f2da 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -65,6 +65,21 @@ request_pkg() [ -s PKGBUILD ] || die "PKGBUILD is empty" } +create_softlink() +{ + [ -e "$cgz_name" ] || return 0 + + local soft_path="$PKG_MNT/${pack_to}/${pkgname}" + local bm_name=$(basename $(realpath ${cgz_name})) + + if [ -n "$upstream_tag" ]; then + echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" + fi + + echo "create soft link: latest.cgz -> ${bm_name}" + ln -sf "${bm_name}" "${soft_path}/latest.cgz" +} build_source_pkg() { @@ -88,22 +103,6 @@ build_source_pkg() $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf 2>&1 } -create_softlink() -{ - [ -e "$cgz_name" ] || return 0 - - local soft_path="$PKG_MNT/${pack_to}/${pkgname}" - local bm_name=$(basename $(realpath ${cgz_name})) - - if [ -n "$upstream_tag" ]; then - echo "create soft link: ${upstream_tag}.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/${upstream_tag}.cgz" - fi - - echo "create soft link: latest.cgz -> ${bm_name}" - ln -sf "${bm_name}" "${soft_path}/latest.cgz" -} - check_vars mount_dest request_pkg -- 2.23.0

2 1

[PATCH v8 lkp-tests 3/3] execute on_fail when exit-code isn't 0
by Hu Xuejiao 17 Nov '20

17 Nov '20

[why] When a job-task fails(exit_code != 0), the on_fail function needs to be called to keep the program in the waiting state to user adjustment Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- bin/run-lkp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/bin/run-lkp b/bin/run-lkp index 26669331..b422a7eb 100755 --- a/bin/run-lkp +++ b/bin/run-lkp @@ -97,6 +97,10 @@ elif [ "$run_job_exit_code" != 0 ]; then else set_job_state 'failed' fi + + source $job_script + [ "$(type -t on_fail)" = "function" ] && $LKP_DEBUG_PREFIX $job_script on_fail + elif [ -f '/var/log/wtmp' ] && has_cmd last && last -n1 | grep -q -E '(pts/|tty)[0-9].*(still logged in)'; then set_job_state 'disturbed' disturbed=1 -- 2.23.0

2 1

[PATCH v4 lkp-tests] add: sysbench-threads-git result parsing file
by Hu Xuejiao 17 Nov '20

17 Nov '20

[input] General statistics: total time: 0.3987s total number of events: 10000 total time taken by event execution: 0.3971 response time: min: 0.04ms [output] total time: 0.3987s Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- stats/sysbench-threads-git | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 stats/sysbench-threads-git diff --git a/stats/sysbench-threads-git b/stats/sysbench-threads-git new file mode 100755 index 00000000..08faf4e0 --- /dev/null +++ b/stats/sysbench-threads-git @@ -0,0 +1,20 @@ +#!/usr/bin/env ruby + +# General statistics: +# total time: 0.2671s +# total number of events: 10000 +# total time taken by event execution: 0.3971s +# response time: +# min: 0.04ms + +results_total = {} + +$stdin.each_line do |line| + next unless line =~ /total time:\s+([0-9.]+)/ + + results_total['total_time'] = $1.to_f +end + +results_total.each do |key, vals| + puts "#{key}: #{vals}s" +end -- 2.23.0

2 1

[PATCH v8 compass-ci 11/11] mail-robot: documentation for service
by Luan Shengde 17 Nov '20

17 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/mail-robot.md | 98 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 doc/mail-robot.md diff --git a/doc/mail-robot.md b/doc/mail-robot.md new file mode 100644 index 0000000..ec1a866 --- /dev/null +++ b/doc/mail-robot.md @@ -0,0 +1,98 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ + +# email monitor robot + +## purpose + +apply account/uuid for user + uuid: used for user to submit jobs + account: used for user to check the test data + +## usage + +build email as the following section: allowed email format +send mail to: compass-ci(a)qq.com + +## allowed email format: + +subject: apply account +mail content + my oss commit: commit_url + example: + my oss commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… +attahcment + ssh pub_key + +## steps overview + +1. mail-robot + monitor_new_email + - the monitor will listen to the mailbox for new email files + handle_new_email: + read email file content + apply_account + invoke AssignAccount for new account + +2. apply-account + init my_info + - my_email + - my_name + check_to_send_account + - parse_mail_content + invoke ParseApplyAccountEmail + parse_commit_url + extract_commit_url + check whether there is a standard commit url + base_url_in_upstream_repos + check whether the repo's url in upstream-repos + commit_url_availability + check whether the commit url valid + - gitee_commit + clone the repo and check the commit + - non_gitee_commit(url) + check the commit with curl + + parse_pub_key + check whether there has an attachment file to the email file + attachment: + first attachment + + - apply_my_account + my_uuid + generate uuid + apply_account + invoke ApplyJumperAccount to apply new account with my_info and pub_key + my_info: + - my_email + - my_name + - my_uuid + apply_jumper_account + required data: pub_key + complete my_info + my_info add: + - my_login_name + - my_commit_url + store my_info + invoke es to store the apply infos + + - rescue error + error type: + - no commit url + - commit url not in upstream-repos + - commit url not available + - no pub_key + - no more available account + build_error_email with raised error message + + - send_mail + - build success email + - build failed email + send mail + +3. continue the monitor + continue to monitor the mailbox for new email file + cycle run step 1 and 2 if matched email files + +4. service log + use the following command to get the logs + docker logs -f --tail=100 fluentd | grep mail-robot -- 2.23.0

1 0

[PATCH v8 compass-ci 10/11] mail-robot: start file
by Luan Shengde 17 Nov '20

17 Nov '20

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..5827e18 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,37 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'set' +require_relative "#{ENV['CCI_SRC']}/container/defconfig" + +docker_rm 'mail-robot' + +names = Set.new %w[ + JUMPER_HOST + JUMPER_PORT + SEND_MAIL_PORT +] + +defaults = relevant_defaults(names) +defaults['MAILDIR'] = '/srv/cci/Maildir/.compass-ci' + +env = docker_env(defaults) + +cmd = %w[ + docker run + --restart=always + --name=mail-robot + -d +] + env + %W[ + -v /c/upstream-repos:/c/upstream-repos:ro + -v #{defaults['MAILDIR']}:#{defaults['MAILDIR']}:rw + -v #{ENV['CCI_SRC']}:/c/compass-ci:rw + -w /c/compass-ci/container/mail-robot + mail-robot +] + +cmd += ['ruby', 'mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0