April 2021 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests 1/6] jobs: add job file for installing/testing mysql
by Luan Shengde 23 Apr '21

23 Apr '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- jobs/mysql-8-0-20.yaml | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 jobs/mysql-8-0-20.yaml diff --git a/jobs/mysql-8-0-20.yaml b/jobs/mysql-8-0-20.yaml new file mode 100644 index 000000000..1926c5547 --- /dev/null +++ b/jobs/mysql-8-0-20.yaml @@ -0,0 +1,34 @@ +suite: cci-makepkg +category: functional + +testbox: taishan200-2280-2s48p-256g--a18 + +os: openeuler +os_version: 20.03 +os_mount: initramfs + +mysql_user: +mysql_host: +mysql_port: +mysql_db: +mysql_password: +data_dev: sdb + +cci-makepkg: + benchmark: mysql-8-0-20 + +mysql-8-0-20: + +sysbench-mysql: + oltp_test_mode: complex + max_requests: 0 + mysql_table_engine: innodb + oltp_tables_count: 1000 + oltp_table_size: 100000 + nr_threads: 256 + runtime: 60 + rand_type: special + rand_spec_pct: 100 + oltp_read_only: false + report_interval: 1 + events: 20000 -- 2.23.0

1 1

[PATCH lkp-tests 6/6] tests: add specify mysql socket
by Luan Shengde 23 Apr '21

23 Apr '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- tests/sysbench-mysql | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/sysbench-mysql b/tests/sysbench-mysql index 0ade2dab9..d7b7fc148 100755 --- a/tests/sysbench-mysql +++ b/tests/sysbench-mysql @@ -14,6 +14,7 @@ # mysql_db # mysql_password # db_driver +# mysql_socket : "${mysql_user:=root}" : "${mysql_host:=$server}" @@ -30,6 +31,7 @@ : "${report_interval:=1}" : "${oltp_read_only:=off}" : "${events:=2000000000}" +: "${mysql_socket:=/data/mysql/run/mysql.sock}" args1=( --mysql-user=$mysql_user @@ -44,10 +46,12 @@ args1=( --rand-spec-pct=$rand_spec_pct --threads=$nr_threads --time=$runtime + --mysql-socket=$mysql_socket ) args2=( --mysql-user=$mysql_user + --mysql-socket=$mysql_socket --mysql-password=$mysql_password --mysql-host=$mysql_host --mysql-port=$mysql_port -- 2.23.0

1 0

[PATCH v3 lkp-tests] jobs/rpmbuild-pkg.yaml: remove upstream_repo
by Lin Jiaxin 23 Apr '21

23 Apr '21

Assigning a null value to upstream_repo is meaningless. And when use "submit upstream_repo=xxx rpmbuild-pkg.yaml" will make upstream_repo be null. Unless we put "upstream_repo=xxx" behind. Or declare it in yaml. [example] linjiaxin@crystal ~%/home/linjiaxin/lkp-tests/sbin/submit upstream_repo="a/azk/azk" upstream_url=https://github.com/azukiapp/azk pkgbuild_repo=archlinux/aur-a/azk-git testbox=dc-16g os=openeuler os_version=20.03-LTS os_mount=container os_arch=aarch64 docker_image=openeuler:20.03-LTS repo_name=aur rpmbuild-pkg.yaml upstream_branch=master upstream_commit=e72adb039e4dd64fd6ed81d82fd597c1d4efdf14 upstream_dir=upstream submit_id=d18fabcd-6302-464c-aa99-3ed9e610704b submit /home/linjiaxin/lkp-tests/jobs/rpmbuild-pkg.yaml, got job id=crystal.1594407 linjiaxin@crystal ~% es-find id=crystal.1594407 | grep upstream_repo "upstream_repo": null, Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- jobs/rpmbuild-pkg.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/jobs/rpmbuild-pkg.yaml b/jobs/rpmbuild-pkg.yaml index a70590afe..5860ba029 100644 --- a/jobs/rpmbuild-pkg.yaml +++ b/jobs/rpmbuild-pkg.yaml @@ -1,5 +1,4 @@ suite: rpmbuild-pkg category: functional -upstream_repo: rpmbuild-pkg: -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/qcow2rootfs-x86_64: fix failed to build container using absolute path
by Wang Chenglong 23 Apr '21

23 Apr '21

[Error] root@2288hv5-2s44p-384g--b6 /c/lkp-tests/tests# /c/compass-ci/container/qcow2rootfs/build /c/compass-ci/container/qcow2rootfs/build: line 5: ../defconfig.sh: No such file or directory /c/compass-ci/container/qcow2rootfs/build: line 7: docker_skip_rebuild: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/qcow2rootfs-x86_64 | 23 +++++++++++++++++------ 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/tests/qcow2rootfs-x86_64 b/tests/qcow2rootfs-x86_64 index f698fb7ec..6826e39ef 100755 --- a/tests/qcow2rootfs-x86_64 +++ b/tests/qcow2rootfs-x86_64 @@ -9,12 +9,17 @@ set_qcow2rootfs_tools() { export QCOW2_PATH="/c/compass-ci/container/qcow2rootfs" + export CCI_SRC="/c/compass-ci" git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 rm ${QCOW2_PATH}/root/etc/yum.repos.d/* wget -O ${QCOW2_PATH}/root/etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo - ${QCOW2_PATH}/build + cd ${QCOW2_PATH} + ./build || { + echo "build container failed~" + exit 1 + } sed -i 's#$EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file#$EXTRACT_ROOT/bin/extract $EXTRACT_ROOT/qcow2-dir/$QCOW2_NAME $tar_out_file#' \ - ${QCOW2_PATH}/bin/mail + ${QCOW2_PATH}/bin/main } @@ -51,15 +56,20 @@ extract_rootfs() { export ROOTFS_DIR="${qcow2_version}-$(date +%Y-%m-%d-%H-%M-%S)" export ROOTFS_PATH="${MOUNT_POINT}/openeuler/${qcow2_arch}/automatic-build/${ROOTFS_DIR}" + cd $CCI_SRC/container/dracut-initrd + ./build || { + echo "build container failed~" + exit 1 + } mkdir -p $ROOTFS_PATH - ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} + ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} || exit 1 } set_resolv_config() { - cat >${ROOTFS_PATH}/etc/resolv.conf<<-EOF - nameserver 114.114.114.114 - EOF + cat >${ROOTFS_PATH}/etc/resolv.conf<<-EOF + nameserver 114.114.114.114 + EOF } set_link() @@ -71,6 +81,7 @@ set_link() echo $ROOTFS_PATH } + set_qcow2rootfs_tools check_url download_qcow2_img -- 2.23.0

1 1

[PATCH lkp-tests 1/3] status 04-8
by Wang Chenglong 22 Apr '21

22 Apr '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 664fd53e0..0cbad6519 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -52,7 +52,7 @@ def parse_msg(ansible_failed_json) output_error $1 when /(Could not find .*):/ output_error $1 - when /(Unsupported parameters) for/ + when /(Unsupported parameters for.*module):/ output_error $1 when /aajdhch/ output_error $1 @@ -66,6 +66,14 @@ def parse_msg(ansible_failed_json) output_error $1 when /(Unable to stop.*):/ output_error $1 + when /AnsibleUndefinedVariable: (.*)/ + output_error $1 + when /(Failed to find handler .*)"\./ + output_error $1 + when /(Connection failure: .*)/ + output_error $1 + when /(Docker .*)\. / + output_error $1 when /non-zero return code|Failed to install some of the specified packages/ return else @@ -82,7 +90,8 @@ def parse_stderr(ansible_failed_json) output_error "gcc: Make error" else ansible_failed_json['stderr_lines'].each do |i| - next if i =~ /^ .*\~.*|^ .*\^$/ + next if i =~ /^ .*\~.*|^ .*\^$|\*.*/ + next if i =~ /Problems in request/ output_error i end @@ -109,21 +118,23 @@ def parse_failures(ansible_failed_json) end def parse_stdout(ansible_failed_json) - case ansible_failed_json['stdout'] - when ansible_failed_json.nil? - return - when ansible_failed_json['cmd'] =~ /make install/ - return - else - ansible_failed_json['stdout_lines'].each do |i| - next if i =~ /^$/ - next if ansible_failed_json['cmd'] =~ /make install/ - next if i =~ /Last metadata expiration/ - next if i =~ /^##.*/ - next if i =~ /^\+.*/ + if ansible_failed_json['stderr_lines'].nil? + case ansible_failed_json['stdout'] + when ansible_failed_json.nil? + return + when ansible_failed_json['cmd'] =~ /make install/ + return + else + ansible_failed_json['stdout_lines'].each do |i| + next if i =~ /^$/ + next if ansible_failed_json['cmd'] =~ /make install/ + next if i =~ /Last metadata expiration/ + next if i =~ /^##.*|^#.*/ + next if i =~ /^\+.*/ - output_error i - end + output_error i + end + end end end -- 2.23.0

1 1

[PATCH lkp-tests 2/3] handle_ansible 04-8
by Wang Chenglong 22 Apr '21

22 Apr '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- lib/handle_ansible.sh | 89 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 lib/handle_ansible.sh diff --git a/lib/handle_ansible.sh b/lib/handle_ansible.sh new file mode 100644 index 000000000..415abfba0 --- /dev/null +++ b/lib/handle_ansible.sh @@ -0,0 +1,89 @@ +#!/bin/bash + +epelrpm() +{ + sed -i "s/epel-release/bash/g" /root/.ansible/roles/*/tasks/*.yml +} + +include_vars() +{ + sed -i 's/{{ ansible_distribution_major_version }}/7/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ansible_distribution_major_version}}/7/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ ansible_distribution }}/CentOS/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ os_family | lower }}.yml/rhel.yml/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ ansible_distribution_file_variety }}/RedHat/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ os_name }}/redhat/g' /root/.ansible/roles/*/tasks/*.yml + sed -i 's/{{ os_release }}/rhel7/g' /root/.ansible/roles/*/tasks/*.yml + + grep with_first_found /root/.ansible/roles/*/tasks/*.yml + [ "$?" == "0" ] && { + sed -i '/with_first_found:/a\ - redhat-7.yml' /root/.ansible/roles/*/tasks/*.yml + } + + [ -d /root/.ansible/roles/*/vars/ansible_distribution ] && { + cp /root/.ansible/roles/*/vars/ansible_distribution/redhat.yml /root/.ansible/roles/*/vars/ansible_distribution/openeuler.yml + sed -i 's/centos/rhel' /root/.ansible/roles/*/vars/ansible_distribution/openeuler.yml + } + + cd /root/.ansible/roles/*/ + cp ./vars/RedHat-7.yml ./vars/RedHat-20.yml + cp ./vars/CentOS-7.yml ./vars/CentOS-20.yml + + cp ./vars/CentOS.yml ./vars/openEuler.yml + sed -i 's/\"7\"/\"20\"/g' ./vars/openEuler.yml + + cp vars/ansible_distribution/redhat.yml vars/ansible_distribution/openeuler.yml + cp vars/ansible_os_family/redhat.yml vars/ansible_os_family/openeuler.yml + + # GPG signature + grep 'state: present' ./tasks/install.yml + [ "$?" == "0" ] && { + sed -i '/yum:/a\ disable_gpg_check: yes' ./tasks/install.yml + } +} + +service_firewall() +{ + systemctl enable firewall +} + +service_redis() +{ + useradd redis -s /usr/sbin/nologin + mkdir -p /var/log/redis + chown -R redis:root /var/log/redis + mkdir -p /var/lib/redis + chown -R redis:root /var/lib/redis +} + +service_postgresql() +{ + sed -i 's/^replication_timeout/#replication_timeout/g' templates/postgresql.conf.j2 + sed -i 's/^checkpoint_segments/#checkpoint_segments/g' templates/postgresql.conf.j2 + sed -i 's/^sql_inheritance/#sql_inheritance/g' templates/postgresql.conf.j2 +} + +service_nfs() +{ + systemctl unmask network.target +} + +command_python() +{ + sed -i 's/3.6.5/3.7.5/g' /root/.ansible/roles/*/defaults/* + +} + + + + +handle_ansible() +{ + epelrpm + include_vars + service_firewall + service_redis + service_postgresql + service_nfs + command_python +} -- 2.23.0

1 0

[PATCH compass-ci] container/etcd: delete extra blank lines
by Liu Yinsi 22 Apr '21

22 Apr '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/etcd/entry.sh | 1 - container/etcd/etcd_compact.sh | 1 - 2 files changed, 2 deletions(-) diff --git a/container/etcd/entry.sh b/container/etcd/entry.sh index 16c8d00..d1c906d 100755 --- a/container/etcd/entry.sh +++ b/container/etcd/entry.sh @@ -5,4 +5,3 @@ crond /bin/etcd "$@" exec - diff --git a/container/etcd/etcd_compact.sh b/container/etcd/etcd_compact.sh index acc20d0..e414371 100755 --- a/container/etcd/etcd_compact.sh +++ b/container/etcd/etcd_compact.sh @@ -4,4 +4,3 @@ rev=$(etcdctl endpoint status --write-out="json"|egrep -o '"revision":[0-9]*'|egrep -o '[0-9].*') etcdctl compact $rev && etcdctl defrag - -- 2.23.0

1 0

[PATCH compass-ci] sparrow/3-code/dev-env: fix error no such file /etc/compass-ci/passwd.yaml
by Liu Yinsi 22 Apr '21

22 Apr '21

[error] Traceback (most recent call last): 4: from ./start:50:in `<main>' 3: from /c/compass-ci/container/defconfig.rb:25:in `relevant_service_authentication' 2: from /usr/share/ruby/psych.rb:497:in `load_file' 1: from /usr/share/ruby/psych.rb:497:in `open' /usr/share/ruby/psych.rb:497:in `initialize': No such file or directory @ rb_sysopen - /etc/compass-ci/passwd.yaml (Errno::ENOENT) Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index a478fffe..b68d9e67 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -189,6 +189,17 @@ auto-submit: my_name: auto-submit EOF +[[ -f /etc/compass-ci/passwd.yaml ]] || { +passwd=$(< /dev/urandom tr -dc "a-zA-Z0-9-=+@#_:/[](){}$%&.^*\!" | head -c 10;echo) + +cat > /etc/compass-ci/passwd.yaml <<EOF +ES_USER: root +ES_PASSWORD: $passwd +LOGGING_ES_USER: root +LOGGING_ES_PASSWORD: $passwd +EOF +} + source /etc/os-release path=$(dirname ${BASH_SOURCE[0]})/os/${ID} -- 2.23.0

1 0

[PATCH v2 lkp-tests] tests/qcow2rootfs-x86_64: fix failed to build container using absolute path
by Wang Chenglong 22 Apr '21

22 Apr '21

[Error] root@2288hv5-2s44p-384g--b6 /c/lkp-tests/tests# /c/compass-ci/container/qcow2rootfs/build /c/compass-ci/container/qcow2rootfs/build: line 5: ../defconfig.sh: No such file or directory /c/compass-ci/container/qcow2rootfs/build: line 7: docker_skip_rebuild: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/qcow2rootfs-x86_64 | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qcow2rootfs-x86_64 b/tests/qcow2rootfs-x86_64 index f698fb7ec..5895c6304 100755 --- a/tests/qcow2rootfs-x86_64 +++ b/tests/qcow2rootfs-x86_64 @@ -12,10 +12,10 @@ set_qcow2rootfs_tools() git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 rm ${QCOW2_PATH}/root/etc/yum.repos.d/* wget -O ${QCOW2_PATH}/root/etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo - ${QCOW2_PATH}/build + cd ${QCOW2_PATH} + ./build || exit 1 sed -i 's#$EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file#$EXTRACT_ROOT/bin/extract $EXTRACT_ROOT/qcow2-dir/$QCOW2_NAME $tar_out_file#' \ - ${QCOW2_PATH}/bin/mail - + ${QCOW2_PATH}/bin/main } check_url() @@ -52,7 +52,7 @@ extract_rootfs() export ROOTFS_DIR="${qcow2_version}-$(date +%Y-%m-%d-%H-%M-%S)" export ROOTFS_PATH="${MOUNT_POINT}/openeuler/${qcow2_arch}/automatic-build/${ROOTFS_DIR}" mkdir -p $ROOTFS_PATH - ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} + ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} || exit 1 } set_resolv_config() -- 2.23.0

1 1

[PATCH lkp-tests] tests/qcow2rootfs-x86_64: fix failed to build container using absolute path
by Wang Chenglong 22 Apr '21

22 Apr '21

[Error] root@2288hv5-2s44p-384g--b6 /c/lkp-tests/tests# /c/compass-ci/container/qcow2rootfs/build /c/compass-ci/container/qcow2rootfs/build: line 5: ../defconfig.sh: No such file or directory /c/compass-ci/container/qcow2rootfs/build: line 7: docker_skip_rebuild: command not found Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- tests/qcow2rootfs-x86_64 | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/qcow2rootfs-x86_64 b/tests/qcow2rootfs-x86_64 index f698fb7ec..2aecc3d4d 100755 --- a/tests/qcow2rootfs-x86_64 +++ b/tests/qcow2rootfs-x86_64 @@ -12,10 +12,10 @@ set_qcow2rootfs_tools() git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 rm ${QCOW2_PATH}/root/etc/yum.repos.d/* wget -O ${QCOW2_PATH}/root/etc/yum.repos.d/CentOS-Base.repo https://repo.huaweicloud.com/repository/conf/CentOS-7-reg.repo - ${QCOW2_PATH}/build + cd ${QCOW2_PATH} + ./build || exit 1 sed -i 's#$EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file#$EXTRACT_ROOT/bin/extract $EXTRACT_ROOT/qcow2-dir/$QCOW2_NAME $tar_out_file#' \ ${QCOW2_PATH}/bin/mail - } check_url() @@ -52,7 +52,7 @@ extract_rootfs() export ROOTFS_DIR="${qcow2_version}-$(date +%Y-%m-%d-%H-%M-%S)" export ROOTFS_PATH="${MOUNT_POINT}/openeuler/${qcow2_arch}/automatic-build/${ROOTFS_DIR}" mkdir -p $ROOTFS_PATH - ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} + ${QCOW2_PATH}/run ${IMG_PATH}/${QCOW2_NAME} ${ROOTFS_PATH} || exit 1 } set_resolv_config() -- 2.23.0

1 1