- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci 2/2] lib/compare_matrixes.rb: fix rubocop warning
by Lu Weitao 25 Feb '21

25 Feb '21

[Why] /home/lwt/compass-ci/lib/compare_matrixes.rb:586:14: C: Style/StringLiterals: Prefer single-quoted strings when you don't need string interpolation or special symbols. return " < 1% " if stddev_percent < 1 Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index d004099..6700126 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -584,7 +584,7 @@ def format_stddev_percent(stddev_percent, average_width) percent_width = SUB_LONG_COLUMN_WIDTH - average_width if stddev_percent if stddev_percent != 0 - return " < 1% " if stddev_percent < 1 + return ' < 1% ' if stddev_percent < 1 percent_str = get_suitable_number_str( stddev_percent.abs, -- 2.23.0

1 0

[PATCH compass-ci 1/2] lib/compare_matrixes.rb skip .message|.log metric
by Lu Weitao 25 Feb '21

25 Feb '21

[Why] the metric end with .message|.log, no need to compare Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- lib/compare_matrixes.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index ffc6500..d004099 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -163,6 +163,7 @@ def get_matrixes_values(matrixes_list, latest_jobs, options) matrixes_size = get_matrixes_size(matrixes_list) get_matrixes_fields(matrixes_list).each do |field| next if field == 'stats_source' + next if field.end_with?('.message', '.log') success = successful?(field) next unless success || latest_failure?(field, latest_jobs) -- 2.23.0

1 0

[PATCH v5 lkp-tests] jobs: add os-matrix-*.yaml
by Li Ping 25 Feb '21

25 Feb '21

1 0

[PATCH lkp-tests] pkg/mysql: change mysql default port
by Wei Jihui 25 Feb '21

25 Feb '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- pkg/mysql/PKGBUILD | 2 +- pkg/mysql/my.cnf | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/pkg/mysql/PKGBUILD b/pkg/mysql/PKGBUILD index 0f7d8bc8..977ad47e 100644 --- a/pkg/mysql/PKGBUILD +++ b/pkg/mysql/PKGBUILD @@ -9,7 +9,7 @@ source=("https://cdn.mysql.com/archives/mysql-8.0/mysql-boost-$pkgver.tar.gz" "https://mirrors.huaweicloud.com/kunpeng/archive/openEuler/patch/database/my…" "https://github.com/thkukuk/rpcsvc-proto/releases/download/v1.4/rpcsvc-proto…" "my.cnf") -md5sums=('7472a25d91973cbba5d8a8f176a3080b' '6e853850efbd6f887ac3e13b1bf99cc4' 'f13eb11de53616017f7428cc06d4e242' 'df1022ccc5d0aff30b8e505735b4a98b' '7bb0372ec4e9281813a9afc67c6cfc20') +md5sums=('7472a25d91973cbba5d8a8f176a3080b' '6e853850efbd6f887ac3e13b1bf99cc4' 'f13eb11de53616017f7428cc06d4e242' 'df1022ccc5d0aff30b8e505735b4a98b' '6d545f2b9a5ef6989c675063a328e407') prepare() { diff --git a/pkg/mysql/my.cnf b/pkg/mysql/my.cnf index 9732af40..acca75ae 100644 --- a/pkg/mysql/my.cnf +++ b/pkg/mysql/my.cnf @@ -13,7 +13,7 @@ socket=/data/mysql/run/mysql.sock tmpdir=/data/mysql/tmp datadir=/data/mysql/data default_authentication_plugin=mysql_native_password -port=3309 +port=3306 user=mysql memlock=on bind-address=0.0.0.0 -- 2.23.0

3 2

[PATCH lkp-tests] daemon: add mysql
by Wei Jihui 25 Feb '21

25 Feb '21

daemon/mysql is for start mysql server Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- daemon/mysql | 104 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100755 daemon/mysql diff --git a/daemon/mysql b/daemon/mysql new file mode 100755 index 00000000..0291d3e9 --- /dev/null +++ b/daemon/mysql @@ -0,0 +1,104 @@ +#!/bin/sh +# - nic +# - device +# mysql_user +# mysql_host +# mysql_port +# mysql_db +# mysql_password +. $LKP_SRC/lib/common.sh + +: "${mysql_user:=root}" +: "${mysql_host:=$server}" +: "${mysql_port:=3306}" +: "${mysql_db:=sysbench_1}" +: "${mysql_password:=mysql_password}" + +create_basedir() +{ + mkdir -p /data + mkfs.xfs -f /dev/"$device" + mount /dev/"$device" /data + + mkdir -p /data/mysql/data /data/mysql/share /data/mysql/tmp /data/mysql/run /data/mysql/log + + useradd mysql + touch /data/mysql/log/mysql.log + chown -R mysql:mysql /data + chown -R mysql:mysql /data/mysql/log/mysql.log +} + +start_mysql_server() +{ + if [ ! -f $mysqld ] || [ ! -f $mysql ] || [ ! -f $mysql_server ]; then + echo "$mysql is not installed!" >&2 && exit 1 + fi + + "$mysqld" --user=mysql --basedir=/data/mysql --datadir=/data/mysql/data/ --initialize + sleep 10 + + "$mysql_server" start + sleep 10 + + "$mysql" -u "$mysql_user" -e "flush privileges; use mysql; alter user 'root'@'localhost' identified by '$mysql_password'; flush privileges;" + sed -i '/^skip-grant-tables/d' /etc/my.cnf + "$mysql_server" restart + sleep 10 +} + +create_database() +{ + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "use mysql; update user set host ='%' where user='root'; flush privileges;" + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "create database $mysql_db;" + "$mysql_server" stop + sleep 10 +} + +disable_irqbalance() +{ + systemctl stop firewalld + iptables -F + pkill -9 irqbalance + systemctl start tuned +} + +intrrupt_nic_binding() +{ + local core_ranges='55-79 93-95' + ethtool -L $nic combined 28 + for core_range in $core_ranges + do + $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + done + ethtool -l $nic +} + +net_io_cache_setting() +{ + echo deadline > /sys/block/"$device"/queue/scheduler + echo 2048 >/sys/block/"$device"/queue/nr_requests +} + +getcpu_process_bind_core() +{ + cd /getcpu || exit + insmod getcpu-hi.ko + numactl -C 0-92 -i 0-3 "mysqld" --defaults-file=/etc/my.cnf & + sleep 20 +} + +main() +{ + local mysql="/usr/local/mysql/bin/mysql" + local mysqld="/usr/local/mysql/bin/mysqld" + local mysql_server="/usr/local/mysql/support-files/mysql.server" + + create_basedir + start_mysql_server + create_database + disable_irqbalance + intrrupt_nic_binding + net_io_cache_setting + getcpu_process_bind_core +} +main -- 2.23.0

2 1

[PATCH lkp-tests 2/3] tests/sysbench-mysql: skip clean up data
by Wei Jihui 25 Feb '21

25 Feb '21

comment out unnecessary opration: cleanup data Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/sysbench-mysql | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tests/sysbench-mysql b/tests/sysbench-mysql index 1d424aaa..5dd8686f 100755 --- a/tests/sysbench-mysql +++ b/tests/sysbench-mysql @@ -85,9 +85,13 @@ run_sysbench_step() run_sysbench_mysql() { - run_sysbench_step parallel_prepare.lua ${args1[@]} prepare + echo "data is being prepared, this will take some time..." + run_sysbench_step parallel_prepare.lua ${args1[@]} prepare > /dev/null + run_sysbench_step oltp.lua ${args2[@]} run - run_sysbench_step oltp.lua ${args2[@]} cleanup + + # clean up test data will take some time, this step can be skipped + # run_sysbench_step oltp.lua ${args2[@]} cleanup > /dev/null } stop_firewalld -- 2.23.0

2 1

[PATCH lkp-tests 1/3] jobs/sysbench-mysql.yaml: udpate test args
by Wei Jihui 25 Feb '21

25 Feb '21

1. change cluster to cs-s1-a102-c1 2. add kernel args 3. add mysql args 4. add sysctl for performance optimized 5. add server nic and device, this can be sured by cluster Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- jobs/sysbench-mysql.yaml | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/jobs/sysbench-mysql.yaml b/jobs/sysbench-mysql.yaml index 83795a18..607acf12 100644 --- a/jobs/sysbench-mysql.yaml +++ b/jobs/sysbench-mysql.yaml @@ -1,10 +1,35 @@ suite: sysbench-mysql category: benchmark -cluster: cs-lkp-hsw-ep5 +cluster: cs-s1-a102-c1 +kernel_version: 4.19.90-mysql +kernel_custom_params: sched_steal_node_limit=8 + +mysql_user: +mysql_host: +mysql_port: +mysql_db: +mysql_password: if role server: + sysctl: + sc_numa_balancing: 0 + sc_sched_autogroup_enabled: 0 + sc_sched_min_granularity_ns: 10000000 + sc_sched_wakeup_granularity_ns: 15000000 + sc_threads-max: 52130 + sc_tcp_max_syn_backlog: 360000 + sc_dirty_ratio: 5 + sc_somaxconn: 1024 + sc_rmem_max: 16777216 + sc_wmem_max: 16777216 + sc_swappiness: 1 + sc_ipv4_tcp_rmem: 4096 87380 16777216 + sc_ipv4_tcp_wmem: 4096 87380 16777216 + sc_sched_features: "NO_WAKEUP_PREEMPTION STEAL" mysql: + nic: enp189s0f1 + device: nvme0n1 if role client: sysbench-mysql: @@ -17,6 +42,6 @@ if role client: runtime: 600 rand_type: special rand_spec_pct: 100 - oltp_read_only: off + oltp_read_only: false report_interval: 1 events: 2000000000 -- 2.23.0

2 1

[PATCH v3 compass-ci] doc/stats-rule.md
by Wang Chenglong 25 Feb '21

25 Feb '21

[Why] The rules to follow when writing the stats script. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/stats-rule.md | 47 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 doc/stats-rule.md diff --git a/doc/stats-rule.md b/doc/stats-rule.md new file mode 100644 index 0000000..eaa2586 --- /dev/null +++ b/doc/stats-rule.md @@ -0,0 +1,47 @@ +${LKP_SERVER}/stats/${script} stat�� +========================================== + +## ��es�� + +1.��error_id�� + - original: {"changed": false, "failures": ["No package yum-plugin-priorities available."], "msg": "Failed to + install some of the specified packages", "rc": 1, "results": []} + + - error_id: error.No-package-yum-plugin-priorities-available: 1 + +2.�� + - ��error_id�� + # �� + # {"changed": false, "cmd": "csf -v", "msg": "[Errno 2] No such file or directory: b'csf': b'csf'", "rc": 2} + when /(No such file or directory: b'.+')/ + output_error $1 + end + + - ��jobs��'��'��es��error_id.��stats/${script}�� + +3.��error message�� + # original error message + # {"changed": false, "msg": "Failed to download packages: Cannot download 7.10.2/filebeat-7.10.2-aarch64.rpm: + # All mirrors were tried", "results": []} + when /(Failed to download packages: Cannot download .+):/ + output_error $1 + end + +4.��error_id�� + - ��'-'�� + - ��'#'. + - ��'-',��. + + def common_error_id(line) + line = line.chomp + line.gsub!(/ /, '-') + line.gsub!(/-+/, '-') # ��'-', �� + line.gsub!(/\b-[a-z][0-9]\.[0-9]\.[0-9]+/, "#") # dgraph-linux-amd64-v0.8.2.tar.gz + line.gsub!(/:/, '') # error.:-'__mysql_packages'-is-undefined: 1 + line.gsub!(/,/, '') + line.gsub!(/'/, '') # Failed-to-download-metadata-for-repo-'dockerrepo: 1 + line.gsub!(/!/, '') # Distribution-openEuler-is-not-supported-by-this-role!: 1 + line.gsub!(/{%/, ' ') + line.gsub!(/%}/, ' ') + line + end -- 2.23.0

2 1

[PATCH compass-ci] container/dracut-initrd: add mke2fs to module 98dracut-systemd
by Xu Xijian 25 Feb '21

25 Feb '21

1. align cmd "inst /usr/bin/awk" when insert it into module 40network/module-setup.sh. 2. add mke2fs to module 98dracut-systemd, it's used to make filesystem type. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/bin/setup-dracut.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/container/dracut-initrd/bin/setup-dracut.sh b/container/dracut-initrd/bin/setup-dracut.sh index 8f03fa8..849480d 100755 --- a/container/dracut-initrd/bin/setup-dracut.sh +++ b/container/dracut-initrd/bin/setup-dracut.sh @@ -16,4 +16,5 @@ rm -rf /var/lib/apt/lists/* cp -a /usr/local/bin/cifs-lib.sh /usr/lib/dracut/modules.d/95cifs/ cat overlay-lkp.sh >> /usr/lib/dracut/modules.d/90overlay-root/overlay-mount.sh -sed -i "/install() {/ainst /usr/bin/awk" /usr/lib/dracut/modules.d/40network/module-setup.sh +sed -i "/install() {/a\ inst /usr/bin/awk" /usr/lib/dracut/modules.d/40network/module-setup.sh +sed -i "/install() {/a\ inst /sbin/mke2fs" /usr/lib/dracut/modules.d/98dracut-systemd/module-setup.sh -- 2.23.0

1 0

[PATCH v3 lkp-tests] jobs: add os-matrix-arch.yaml
by Li Ping 25 Feb '21

25 Feb '21

1 0