- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] PACKAGER use default value "Compass-CI" when use makepkg
by Lin Jiaxin 02 Mar '21

02 Mar '21

If the value of PACKAGER is not specified in job.yaml, set it to the default value of "Compass-CI". [Now] % cat .PKGINFO | grep packager packager = Unknown Packager [After] % cat .PKGINFO | grep packager packager = Compass-CI Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- tests/cci-makepkg | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index f3545a19..6a083a9b 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -159,7 +159,7 @@ build_source_pkg() export V=1 - PACMAN="${mark}" BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" \ + PACMAN="${mark}" BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" PACKAGER="${PACKAGER:-Compass-CI}" \ $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf local build_state="$?" diff --git a/tests/cci-makepkg b/tests/cci-makepkg index ecbf7213..556de6b0 100755 --- a/tests/cci-makepkg +++ b/tests/cci-makepkg @@ -142,7 +142,7 @@ build_source_pkg() local makepkg="$LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf" echo "${makepkg}" - PACMAN="${mark}" BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" $makepkg + PACMAN="${mark}" BUILDDIR=$TMP CARCH=$arch PKGEXT=.cgz CGZDEST="$sync_dest/$cgz_name" PACKAGER="${PACKAGER:-Compass-CI}" $makepkg [ "$?" == 0 ] || exit 1 update_softlink -- 2.23.0

1 0

[PATCH lkp-tests] pkg/sysbench-mysql: fix sysbench-mysql
by Wei Jihui 02 Mar '21

02 Mar '21

install mysql by distro/depends/sysbench-mysql-dev:libmariadb-dev so no need specify mysql path when install sysbench Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/sysbench-mysql/PKGBUILD | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/pkg/sysbench-mysql/PKGBUILD b/pkg/sysbench-mysql/PKGBUILD index c214d8a2..c240330f 100644 --- a/pkg/sysbench-mysql/PKGBUILD +++ b/pkg/sysbench-mysql/PKGBUILD @@ -8,18 +8,16 @@ license=('GPL') source=("https://github.com/akopytov/sysbench/archive/$pkgver.$pkgrel.tar.gz") md5sums=('2912bfe7238cac7351459019a84e2557') -build() { +build() +{ cd "$pkgname-$pkgver.$pkgrel" ./autogen.sh - ./configure --with-mysql-includes=/usr/local/mysql/include/ --with-mysql-libs=/usr/local/mysql/lib/ --disable-dependency-tracking + ./configure --disable-dependency-tracking make -j4 } -package() { +package() +{ cd "$pkgname-$pkgver.$pkgrel" make DESTDIR=$pkgdir install - [[ $os == openeuler ]] && { - mkdir -p $pkgdir/usr/lib64 - cp -rpf /usr/local/mysql/lib/* $pkgdir/usr/lib64/ - } } -- 2.23.0

1 0

[PATCH compass-ci] container/git-mirror: mount /etc/compass-ci/defaults
by Li Yuanchao 02 Mar '21

02 Mar '21

As git-mirror need to read upstream-config there. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/git-mirror/start | 1 + 1 file changed, 1 insertion(+) diff --git a/container/git-mirror/start b/container/git-mirror/start index a618ec3..ec42595 100755 --- a/container/git-mirror/start +++ b/container/git-mirror/start @@ -23,6 +23,7 @@ cmd = %W[ -e GIT_ASKPASS=/kill-parent.sh -v #{ENV['CCI_SRC']}:#{DOCKER_CCI} -v /etc/localtime:/etc/localtime:ro + -v /etc/compass-ci/defaults:/etc/compass-ci/defaults:ro -v #{ENV['REPO_SRC']}:#{DOCKER_REPO} -v /srv/git:/srv/git -w /c/compass-ci/sbin -- 2.23.0

1 0

[PATCH compass-ci] lib/es_jobs.rb: don't sum '.message'
by Wang Chenglong 02 Mar '21

02 Mar '21

[Why] The 'xxx.message' in es-jobs just show the error message, it doesn't has to be summed. before: sum.stats.ansible_test.error.No-package-yum-utils-available.message: 1 sum.stats.ansible_test.error.No-package-yum-utils-available: 1 raw.stats.ansible_test.error.No-package-yum-utils-available.message: ["{\"ansible_loop_var\": \"item\", \"changed\": false, \"failures\": [\"No package yum-utils available.\"], \"item\": [\"yum-utils\", \"device-mapper-persistent-data\", \"lvm2\"], \"msg\": \"Failed to install some of the specified packages\", \"rc\": 1, \"results\": []}"] raw.stats.ansible_test.error.No-package-yum-utils-available: [1] after: sum.stats.ansible_test.error.No-package-yum-utils-available: 1 raw.stats.ansible_test.error.No-package-yum-utils-available.message: ["{\"ansible_loop_var\": \"item\", \"changed\": false, \"failures\": [\"No package yum-utils available.\"], \"item\": [\"yum-utils\", \"device-mapper-persistent-data\", \"lvm2\"], \"msg\": \"Failed to install some of the specified packages\", \"rc\": 1, \"results\": []}"] raw.stats.ansible_test.error.No-package-yum-utils-available: [1] Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- lib/es_jobs.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/es_jobs.rb b/lib/es_jobs.rb index c5c0f0d..b0d0fb2 100644 --- a/lib/es_jobs.rb +++ b/lib/es_jobs.rb @@ -124,6 +124,8 @@ class ESJobs def stats_count(result) result['raw.stats'].each do |key, value| + next if key.end_with?('.message') + if function_stat?(key) result['sum.stats'][key] = value.compact.size else -- 2.23.0

2 2

[PATCH lkp-tests] lib/bootstrap.sh: change $HOSTNAME suffix
by Wang Yong 02 Mar '21

02 Mar '21

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- lib/bootstrap.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index 76f07e6a7..b2ded1893 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -283,8 +283,8 @@ setup_hosts() else cp /etc/hosts $tmpfs_hosts fi - echo "127.0.0.1 $HOSTNAME.sh.intel.com $HOSTNAME" >> $tmpfs_hosts - echo "::1 $HOSTNAME.sh.intel.com $HOSTNAME" >> $tmpfs_hosts + echo "127.0.0.1 $HOSTNAME.compass-ci.net $HOSTNAME" >> $tmpfs_hosts + echo "::1 $HOSTNAME.compass-ci.net $HOSTNAME" >> $tmpfs_hosts ln -fs $tmpfs_hosts /etc/hosts } -- 2.23.0

2 2

[PATCH compass-ci] container/srv-http: add /srv/initrd/rpmbuild-pkg
by Li Ping 02 Mar '21

02 Mar '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- container/srv-http/start-initrd | 1 + 1 file changed, 1 insertion(+) diff --git a/container/srv-http/start-initrd b/container/srv-http/start-initrd index 7c73ea1..495cc25 100755 --- a/container/srv-http/start-initrd +++ b/container/srv-http/start-initrd @@ -12,6 +12,7 @@ cmd=( -p 11303:11300 -v /srv/initrd/qemu-image:/srv/initrd/qemu-image:ro -v /srv/initrd/dockerimage:/srv/initrd/dockerimage:ro + -v /srv/initrd/rpmbuild-pkg:/srv/initrd/rpmbuild-pkg:ro -v /etc/localtime:/etc/localtime:ro -d srv-http -- 2.23.0

2 1

[PATCH v2 compass-ci] lib: parse mail list from yaml by type as to and bcc mail list
by Cao Xueliang 02 Mar '21

02 Mar '21

Yaml file define: delimiter: to: mai1, mail3 bcc: mail2, mail3 The mail_bisect_result calls parse_mail_list('delimiter') to get the delimiter mail list. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 14 +++++++++++--- lib/mail_list.yaml | 3 +++ lib/parse_mail_list.rb | 12 ++++++++++++ 3 files changed, 26 insertions(+), 3 deletions(-) create mode 100644 lib/mail_list.yaml create mode 100644 lib/parse_mail_list.rb diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 104f098..23a8ade 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -6,6 +6,7 @@ require_relative 'git' require_relative 'es_query' require_relative 'constants' require_relative 'mail_client' +require_relative 'parse_mail_list' require_relative 'assign_account_client' # compose and send email for bisect result @@ -22,10 +23,17 @@ class MailBisectResult @git_commit = GitCommit.new(@work_dir, @commit_id) @to = @git_commit.author_email # now send mail to review - @bcc = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' + @bcc = 'caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' + end + + def parse_mail_info + mail_hash = parse_mail_list('delimiter') + @to = mail_hash['to'] if mail_hash.key?('to') + @bcc = mail_hash['bcc'] if mail_hash.key?('bcc') end def create_send_email + parse_mail_info send_report_mail(compose_mail) send_account_mail rm_work_dir @@ -35,7 +43,7 @@ class MailBisectResult subject = "[Compass-CI][#{(a)repo.split('/')[1]}] #{@commit_id[0..9]} #{(a)bisect_error[0].split("\n")[0]}" prefix_srv = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_PORT}" bisect_job_url = ENV['result_root'] ? "bisect job result directory:\n#{prefix_srv}#{ENV['result_root']}\n" : '' - bisect_report_doc = "bisect email doc:\nhttps://gitee.com/wu_fengguang/compass-ci/blob/master/doc/bisect_emai…" + report_doc = "bisect email doc:\nhttps://gitee.com/wu_fengguang/compass-ci/blob/master/doc/bisect_emai…" pkgbuild_repo_url = "PKGBUILD:\n#{prefix_srv}/git/#{@pkgbuild_repo}\n" first_bad_commit_job_url = "first bad commit job result directory:\n#{prefix_srv}#{@first_bad_commit_result_root}\n" @@ -58,7 +66,7 @@ class MailBisectResult #{pkgbuild_repo_url} #{first_bad_commit_job_url} #{bisect_job_url} - #{bisect_report_doc} + #{report_doc} Regards, Compass CI team BODY diff --git a/lib/mail_list.yaml b/lib/mail_list.yaml new file mode 100644 index 0000000..d982c93 --- /dev/null +++ b/lib/mail_list.yaml @@ -0,0 +1,3 @@ +delimiter: + to: caoxl78320(a)163.com + bcc: caoxl78320(a)163.com, wfg(a)mail.ustc.edu.cn, ljx.joe(a)qq.com, huming15(a)163.com diff --git a/lib/parse_mail_list.rb b/lib/parse_mail_list.rb new file mode 100644 index 0000000..1a76a1e --- /dev/null +++ b/lib/parse_mail_list.rb @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# frozen_string_literal: true + +require 'yaml' + +def parse_mail_list(type) + content = {} + mail_list_yaml = './mail_list.yaml' + content = YAML.safe_load(File.open(mail_list_yaml)) if FileTest.exists?(mail_list_yaml) + + return content[type] || {} +end -- 2.23.0

1 1

[PATCH v3 compass-ci] lib: parse mail list from yaml by type as to and bcc mail list
by Cao Xueliang 02 Mar '21

02 Mar '21

Yaml file define: delimiter: to: mail1, mail3 bcc: mail2, mail3 The mail_bisect_result calls parse_mail_list('delimiter') to get the delimiter mail list. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/mail_bisect_result.rb | 14 +++++++++++--- lib/mail_list.yaml | 3 +++ lib/parse_mail_list.rb | 12 ++++++++++++ 3 files changed, 26 insertions(+), 3 deletions(-) create mode 100644 lib/mail_list.yaml create mode 100644 lib/parse_mail_list.rb diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb index 104f098..23a8ade 100644 --- a/lib/mail_bisect_result.rb +++ b/lib/mail_bisect_result.rb @@ -6,6 +6,7 @@ require_relative 'git' require_relative 'es_query' require_relative 'constants' require_relative 'mail_client' +require_relative 'parse_mail_list' require_relative 'assign_account_client' # compose and send email for bisect result @@ -22,10 +23,17 @@ class MailBisectResult @git_commit = GitCommit.new(@work_dir, @commit_id) @to = @git_commit.author_email # now send mail to review - @bcc = 'caoxl(a)crystal.ci, caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' + @bcc = 'caoxl78320(a)163.com, huming15(a)163.com, wfg(a)mail.ustc.edu.cn' + end + + def parse_mail_info + mail_hash = parse_mail_list('delimiter') + @to = mail_hash['to'] if mail_hash.key?('to') + @bcc = mail_hash['bcc'] if mail_hash.key?('bcc') end def create_send_email + parse_mail_info send_report_mail(compose_mail) send_account_mail rm_work_dir @@ -35,7 +43,7 @@ class MailBisectResult subject = "[Compass-CI][#{(a)repo.split('/')[1]}] #{@commit_id[0..9]} #{(a)bisect_error[0].split("\n")[0]}" prefix_srv = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_PORT}" bisect_job_url = ENV['result_root'] ? "bisect job result directory:\n#{prefix_srv}#{ENV['result_root']}\n" : '' - bisect_report_doc = "bisect email doc:\nhttps://gitee.com/wu_fengguang/compass-ci/blob/master/doc/bisect_emai…" + report_doc = "bisect email doc:\nhttps://gitee.com/wu_fengguang/compass-ci/blob/master/doc/bisect_emai…" pkgbuild_repo_url = "PKGBUILD:\n#{prefix_srv}/git/#{@pkgbuild_repo}\n" first_bad_commit_job_url = "first bad commit job result directory:\n#{prefix_srv}#{@first_bad_commit_result_root}\n" @@ -58,7 +66,7 @@ class MailBisectResult #{pkgbuild_repo_url} #{first_bad_commit_job_url} #{bisect_job_url} - #{bisect_report_doc} + #{report_doc} Regards, Compass CI team BODY diff --git a/lib/mail_list.yaml b/lib/mail_list.yaml new file mode 100644 index 0000000..d982c93 --- /dev/null +++ b/lib/mail_list.yaml @@ -0,0 +1,3 @@ +delimiter: + to: caoxl78320(a)163.com + bcc: caoxl78320(a)163.com, wfg(a)mail.ustc.edu.cn, ljx.joe(a)qq.com, huming15(a)163.com diff --git a/lib/parse_mail_list.rb b/lib/parse_mail_list.rb new file mode 100644 index 0000000..1a76a1e --- /dev/null +++ b/lib/parse_mail_list.rb @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: MulanPSL-2.0+ +# frozen_string_literal: true + +require 'yaml' + +def parse_mail_list(type) + content = {} + mail_list_yaml = './mail_list.yaml' + content = YAML.safe_load(File.open(mail_list_yaml)) if FileTest.exists?(mail_list_yaml) + + return content[type] || {} +end -- 2.23.0

1 0

[PATCH v2 lkp-tests] daemon: add mysql
by Wei Jihui 02 Mar '21

02 Mar '21

add mysql for sysbench-mysql test Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- daemon/mysql | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100755 daemon/mysql diff --git a/daemon/mysql b/daemon/mysql new file mode 100755 index 00000000..89d2318c --- /dev/null +++ b/daemon/mysql @@ -0,0 +1,104 @@ +#!/bin/sh +# - nic +# - device +# mysql_user +# mysql_host +# mysql_port +# mysql_db +# mysql_password +. $LKP_SRC/lib/common.sh + +: "${mysql_user:=root}" +: "${mysql_host:=$server}" +: "${mysql_port:=3306}" +: "${mysql_db:=sysbench_1}" +: "${mysql_password:=mysql_password}" + +create_basedir() +{ + mkdir -p /data + mkfs.xfs -f /dev/"$device" + mount /dev/"$device" /data + + mkdir -p /data/mysql/data /data/mysql/share /data/mysql/tmp /data/mysql/run /data/mysql/log + + useradd mysql + touch /data/mysql/log/mysql.log + chown -R mysql:mysql /data + chown -R mysql:mysql /data/mysql/log/mysql.log +} + +start_mysql_server() +{ + if [ ! -f $mysqld ] || [ ! -f $mysql ] || [ ! -f $mysql_server ]; then + echo "mysql is not installed!" && exit 1 + fi + + "$mysqld" --user=mysql --basedir=/data/mysql --datadir=/data/mysql/data/ --initialize --port="$mysql_port" + sleep 10 + + "$mysql_server" start + sleep 10 + + "$mysql" -u "$mysql_user" -e "flush privileges; use mysql; alter user 'root'@'localhost' identified by '$mysql_password'; flush privileges;" + sed -i '/^skip-grant-tables/d' /etc/my.cnf + "$mysql_server" restart + sleep 10 +} + +create_database() +{ + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "use mysql; update user set host ='%' where user='root'; flush privileges;" + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "create database $mysql_db;" + "$mysql_server" stop + sleep 10 +} + +disable_irqbalance() +{ + systemctl stop firewalld >/dev/null 2>&1 + iptables -F >/dev/null 2>&1 + pkill -9 irqbalance >/dev/null 2>&1 + systemctl start tuned >/dev/null 2>&1 +} + +intrrupt_nic_binding() +{ + local core_ranges='55-79 93-95' + ethtool -L $nic combined 28 + for core_range in $core_ranges + do + $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + done + ethtool -l $nic +} + +net_io_cache_setting() +{ + echo deadline > /sys/block/"$device"/queue/scheduler + echo 2048 > /sys/block/"$device"/queue/nr_requests +} + +getcpu_process_bind_core() +{ + cd /getcpu || exit + insmod getcpu-hi.ko + numactl -C 0-92 -i 0-3 "$mysqld" --defaults-file=/etc/my.cnf --port="$mysql_port" & + sleep 20 +} + +main() +{ + local mysql="/usr/local/mysql/bin/mysql" + local mysqld="/usr/local/mysql/bin/mysqld" + local mysql_server="/usr/local/mysql/support-files/mysql.server" + + create_basedir + start_mysql_server + create_database + disable_irqbalance + intrrupt_nic_binding + net_io_cache_setting + getcpu_process_bind_core +} +main -- 2.23.0

2 1

[PATCH compass-ci 1/2] sbin/upstream-config: add upstream-config file
by Li Yuanchao 02 Mar '21

02 Mar '21

To support more than one upstream repos warehouse Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/upstream-config | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 sbin/upstream-config diff --git a/sbin/upstream-config b/sbin/upstream-config new file mode 100644 index 0000000..e3b167a --- /dev/null +++ b/sbin/upstream-config @@ -0,0 +1,10 @@ +--- +upstreams: +- url: https://gitee.com/wu_fengguang/upstream-repos.git + location: upstream +- url: https://gitee.com/wu_fengguang/ansible-repos.git + location: ansible +- url: https://gitee.com/wu_fengguang/dockerfile-repos.git + location: dockerfile +- url: https://gitee.com/wu_fengguang/pkgbuild-repos.git + location: pkgbuild -- 2.23.0

2 3