- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] src/stats_worker:delete filter error_id and add log
by Cao Xueliang 02 Mar '21

02 Mar '21

We add the filter error_id in commit: 1c154d888fab1fce4623663b4a88b9a1630171c2, we don't need it now, so delete the filter error_id, add the new_error_id log for statistical. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- src/extract-stats/stats_worker.cr | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/src/extract-stats/stats_worker.cr b/src/extract-stats/stats_worker.cr index 01bae21..836d1f8 100644 --- a/src/extract-stats/stats_worker.cr +++ b/src/extract-stats/stats_worker.cr @@ -82,25 +82,20 @@ class StatsWorker } ) - error_id = select_error_id(check_new_error_ids(error_ids, job_id)) - if error_id + new_error_ids = check_new_error_ids(error_ids, job_id) + unless new_error_ids.empty? + sample_error_id = new_error_ids.sample STDOUT.puts "send a delimiter task: job_id is #{job_id}" - @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => error_id, + @tq.add_task(DELIMITER_TASK_QUEUE, JSON.parse({"error_id" => sample_error_id, "job_id" => job_id, "lab" => LAB}.to_json)) + msg = %({"job_id": "#{job_id}", "new_error_id": "#{sample_error_id}"}) + system "echo '#{msg}'" end msg = %({"job_id": "#{job_id}", "job_state": "extract_finished"}) system "echo '#{msg}'" end - def select_error_id(new_error_ids : Array) - new_error_ids.each do |error_id| - return error_id if /(cpp|c|h):(warning|error)/i =~ error_id - end - - return nil - end - def check_new_error_ids(error_ids : Array, job_id : String) new_error_ids = [] of String error_ids.each do |error_id| -- 2.23.0

1 0

[PATCH v3 compass-ci 4/4] container/dracut-initrd: add 90lkp module into Dockerfile
by Xu Xijian 02 Mar '21

02 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/Dockerfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/container/dracut-initrd/Dockerfile b/container/dracut-initrd/Dockerfile index 7f8448f..b8c551f 100644 --- a/container/dracut-initrd/Dockerfile +++ b/container/dracut-initrd/Dockerfile @@ -6,6 +6,8 @@ FROM debian MAINTAINER Wu Fenguang <wfg(a)mail.ustc.edu.cn> ADD bin /usr/local/bin +ADD modules.d /usr/lib/modules.d +COPY modules.d/90lkp /usr/lib/dracut/modules.d/90lkp COPY conf/sources.list* /etc/apt/ COPY conf/add-modules.conf /etc/dracut.conf.d/ WORKDIR /usr/local/bin -- 2.23.0

1 0

[PATCH v3 compass-ci 2/4] container/dracut-initrd: README of module 90lkp
by Xu Xijian 02 Mar '21

02 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/modules.d/90lkp/README | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 container/dracut-initrd/modules.d/90lkp/README diff --git a/container/dracut-initrd/modules.d/90lkp/README b/container/dracut-initrd/modules.d/90lkp/README new file mode 100644 index 0000000..7d7a205 --- /dev/null +++ b/container/dracut-initrd/modules.d/90lkp/README @@ -0,0 +1,5 @@ +dracut lkp module + +This module is to deploy lkp module into rootfs. +Module lkp(Linux Kernel Performance) is a test framework, which is used +to make self-defined test for os kernel. -- 2.23.0

1 0

[PATCH v3 compass-ci 1/4] container/dracut-initrd: add module 90lkp/module-setup.sh
by Xu Xijian 02 Mar '21

02 Mar '21

90lkp module a module to deploy lkp module into rootfs. module-setup.sh is a base setup script for every kernel module. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- .../modules.d/90lkp/module-setup.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100755 container/dracut-initrd/modules.d/90lkp/module-setup.sh diff --git a/container/dracut-initrd/modules.d/90lkp/module-setup.sh b/container/dracut-initrd/modules.d/90lkp/module-setup.sh new file mode 100755 index 0000000..c188600 --- /dev/null +++ b/container/dracut-initrd/modules.d/90lkp/module-setup.sh @@ -0,0 +1,19 @@ +#!/bin/bash + +check() { + return 0 +} + +depends() { + # We do not depend on any modules - just some root + return 0 +} + +# called by dracut +installkernel() { + return 0 +} + +install() { + inst_hook pre-pivot 10 "$moddir/lkp-deploy.sh" +} -- 2.23.0

1 0

[PATCH compass-ci 2/2] test2
by Xu Xijian 02 Mar '21

02 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/test2 | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 container/dracut-initrd/test2 diff --git a/container/dracut-initrd/test2 b/container/dracut-initrd/test2 new file mode 100644 index 0000000..e69de29 -- 2.23.0

1 1

[PATCH compass-ci 1/2] test1
by Xu Xijian 02 Mar '21

02 Mar '21

Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- container/dracut-initrd/test1 | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 container/dracut-initrd/test1 diff --git a/container/dracut-initrd/test1 b/container/dracut-initrd/test1 new file mode 100644 index 0000000..e69de29 -- 2.23.0

1 1

[PATCH v2 compass-ci 2/2] container/dracut-initrd: add module 90lkp/lkp-deploy.sh
by Xu Xijian 02 Mar '21

02 Mar '21

lkp-deploy.sh does the concrete work of 90lkp, it adds the lkp scripts into testbox's rootfs to run test job. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- .../modules.d/90lkp/lkp-deploy.sh | 48 +++++++++++++++++++ 1 file changed, 48 insertions(+) create mode 100755 container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh diff --git a/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh b/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh new file mode 100755 index 0000000..e88959b --- /dev/null +++ b/container/dracut-initrd/modules.d/90lkp/lkp-deploy.sh @@ -0,0 +1,48 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +. /lib/dracut-lib.sh + +# transfer LKP dirs +[ -d /lkp ] || return 0 + +cp -a /lkp "$NEWROOT"/ + +for i in /lkp/lkp/src/rootfs/addon/* /usr/local/* /opt/* +do + dir=$(basename "$i") + + [ "$i" != "${i%/\*}" ] && continue # skip: i='dir/*' + + [ -d "$NEWROOT/$dir" ] || mkdir -p "$NEWROOT/$dir" + + for j in "$i"/* "$i"/.??* + do + [ "$j" != "${j%/\*}" ] && continue # skip: j='dir/*' + + [ -f "$j" ] && { + cp -a "$j" "$NEWROOT/$dir"/ + continue + } + + subdir=$(basename "$j") + + [ -d "$NEWROOT/$dir/$subdir" ] || mkdir -p "$NEWROOT/$dir/$subdir" + + for k in "$j"/* + do + [ "$k" != "${k%/\*}" ] && continue # skip: k='dir/*' + + cp -a "$j"/* "$NEWROOT/$dir/$subdir"/ + done + done +done + +[ -d /usr/src ] && + cp -a /usr/src "$NEWROOT"/usr/ + +kmdir=/lib/modules/$(uname -r) +if test -d "$kmdir" && ! test -d "$NEWROOT/$kmdir"; then + cp -an "$kmdir" "$NEWROOT"/lib/modules/ + cp -an /lib/firmware "$NEWROOT"/lib/ +fi -- 2.23.0

2 2

[PATCH v3 lkp-tests] daemon: add mysql
by Wei Jihui 02 Mar '21

02 Mar '21

add mysql for sysbench-mysql test Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- daemon/mysql | 92 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100755 daemon/mysql diff --git a/daemon/mysql b/daemon/mysql new file mode 100755 index 00000000..a275a812 --- /dev/null +++ b/daemon/mysql @@ -0,0 +1,92 @@ +#!/bin/sh +# - nic +# - device +# mysql_user +# mysql_host +# mysql_port +# mysql_db +# mysql_password +. $LKP_SRC/lib/common.sh + +: "${mysql_user:=root}" +: "${mysql_host:=$server}" +: "${mysql_port:=3306}" +: "${mysql_db:=sysbench_test}" +: "${mysql_password:=mysql_password}" + +prepare() +{ + systemctl stop firewalld >/dev/null 2>&1 + iptables -F >/dev/null 2>&1 + systemctl start tuned >/dev/null 2>&1 +} + +insmod_getcpu() +{ + if [ -d "/getcpu" ]; then + cd /getcpu && insmod getcpu-hi.ko + else + echo "this test will not use getcpu optimize to bind core." + fi +} + +create_basedir() +{ + mkdir -p /data + mkfs.xfs -f /dev/"$device" + mount /dev/"$device" /data + + mkdir -p /data/mysql/data /data/mysql/share /data/mysql/tmp /data/mysql/run /data/mysql/log + + useradd mysql + touch /data/mysql/log/mysql.log + chown -R mysql:mysql /data + chown -R mysql:mysql /data/mysql/log/mysql.log +} + +configure_mysql() +{ + if [ ! -f $mysqld ] || [ ! -f $mysql ] || [ ! -f $mysql_server ]; then + echo "mysql is not installed!" && exit 1 + fi + + "$mysqld" --user=mysql --basedir=/data/mysql --datadir=/data/mysql/data/ --initialize --port="$mysql_port" + sleep 10 + + "$mysql_server" start + sleep 10 + + "$mysql" -u "$mysql_user" -e "flush privileges; use mysql; alter user 'root'@'localhost' identified by '$mysql_password'; flush privileges;" + sed -i '/^skip-grant-tables/d' /etc/my.cnf + "$mysql_server" restart + sleep 10 +} + +create_database() +{ + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "use mysql; update user set host ='%' where user='root'; flush privileges;" + "$mysql" -u "$mysql_user" -p"$mysql_password" -e "create database $mysql_db;" + "$mysql_server" stop + sleep 10 +} + +start_mysql() +{ + numactl -C 0-92 -i 0-3 "$mysqld" --defaults-file=/etc/my.cnf --port="$mysql_port" & + sleep 20 +} + +main() +{ + local mysql="/usr/local/mysql/bin/mysql" + local mysqld="/usr/local/mysql/bin/mysqld" + local mysql_server="/usr/local/mysql/support-files/mysql.server" + + prepare + insmod_getcpu + create_basedir + configure_mysql + create_database + start_mysql +} +main -- 2.23.0

1 0

[PATCH compass-ci 4/4] sparrow/5-build/register-account: check account info before register
by Liu Yinsi 02 Mar '21

02 Mar '21

for user locally deploy compass-ci client, if not fill config yaml $CCI_SRC/sparrow/setup.yaml, failed to register account, so check my_email, my_name. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/5-build/register-account | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/sparrow/5-build/register-account b/sparrow/5-build/register-account index 1a23e07..94b7407 100755 --- a/sparrow/5-build/register-account +++ b/sparrow/5-build/register-account @@ -2,4 +2,10 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -$CCI_SRC/sbin/build-my-info.rb +source ${CCI_SRC}/lib/log.sh + +if [ -n "$my_email" ] && [ -n "$my_name" ]; then + $CCI_SRC/sbin/build-my-info.rb -e $my_email -n $my_name +else + log_info "if you need to register account, please try: $CCI_SRC/sbin/build-my-info.rb -e \$my_email -n \$my_name" +fi -- 2.23.0

1 0

[PATCH compass-ci 3/4] sparrow/3-code/dev-env: delete redunbant code
by Liu Yinsi 02 Mar '21

02 Mar '21

0-package/read-config already copy $CCI_SRC/sparrow/setup.yaml to /etc/compass-ci/setup.yaml, so delete redunbant code. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 5 ----- 1 file changed, 5 deletions(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 698a4a2..df7b1ce 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -2,12 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -mkdir -p /etc/compass-ci -cp -a $CCI_SRC/sparrow/setup.yaml /etc/compass-ci/setup.yaml - -server_ip=$(awk '/^server_ip:\s/ {print $2; exit}' /etc/compass-ci/setup.yaml) [ -z $server_ip ] && server_ip=$(ip route get 1.2.3.4 | awk '{print $7; exit}') - server_name=$(hostname | cut -f1 -d.) : ${sched_host:=$server_ip} : ${sched_port:=3000} -- 2.23.0

1 0

2025

2024

2023

2022

2021

2020

Compass-ci