- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] jobs: dc-1g => dc-1g-bisect in bisect.yaml
by Cao Xueliang 06 Jan '21

06 Jan '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- jobs/bisect.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/jobs/bisect.yaml b/jobs/bisect.yaml index 11ffc7c05..8c2ee8f62 100644 --- a/jobs/bisect.yaml +++ b/jobs/bisect.yaml @@ -1,4 +1,4 @@ suite: bisect docker_image: delimiter:latest -testbox: dc-1g +testbox: dc-1g-bisect bisect: -- 2.23.0

2 3

[PATCH compass-ci] lib/job.cr: set docker job os
by Wu Zhende 06 Jan '21

06 Jan '21

docker job's os/os_version/os_mount don't need to define separately. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 2554db1..6f344fe 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -153,6 +153,7 @@ class Job private def set_defaults append_init_field() + set_docker_os() set_user_lkp_src() set_os_dir() set_submit_date() @@ -170,6 +171,14 @@ class Job set_subqueue() end + private def set_docker_os + return unless is_docker_job? + + os_info = docker_image.split(":") + self["os"] = os_info[0] + self["os_version"] = os_info[1] + end + private def set_kernel return if os_mount == "container" @@ -309,10 +318,23 @@ class Job @hash[key] = JSON::Any.new(value) if value end + private def is_docker_job? + if testbox =~ /^dc/ + return true + else + return false + end + end + # defaults to the 1st value VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container"] private def set_os_mount + if is_docker_job? + self["os_mount"] = "container" + return + end + if @hash["os_mount"]? if !VALID_OS_MOUNTS.includes?(@hash["os_mount"].to_s) raise "Invalid os_mount: #{@hash["os_mount"]}, should be in #{VALID_OS_MOUNTS}" -- 2.23.0

1 0

[PATCH v4 lkp-tests] tests: add httpress test script
by Wei Jihui 06 Jan '21

06 Jan '21

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- tests/httpress | 136 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 136 insertions(+) create mode 100755 tests/httpress diff --git a/tests/httpress b/tests/httpress new file mode 100755 index 00000000..462a2705 --- /dev/null +++ b/tests/httpress @@ -0,0 +1,136 @@ +#!/bin/sh +# - nic +# - nic_ip +# - request_ip +# - request_port +# - request_index +# - core_nums +# - protocol +# - alive_type + +. $LKP_SRC/lib/env.sh + +: "${request_index:=index.html}" +: "${core_nums:=4}" +: "${protocol:=http}" +: "${alive_type:=short}" + +core_range=0-15 + +http_long_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=50000000 + threads_num=1200 + concurrent_num=1200 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=50000000 + threads_num=400 + concurrent_num=400 + ethtool -L $nic combined 16 && /lkp/lkp/src/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + +http_short_requests() +{ + if [ "$core_nums" == 4 ]; then + requests_num=3000000 + threads_num=300 + concurrent_num=300 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=400 + concurrent_num=400 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=10000000 + threads_num=300 + concurrent_num=300 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + ethtool -L $nic combined 16 && $LKP_SRC/bin/set_nic_irq_affinity $core_range $nic + numactl -C 16-95 httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +https_long_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=5000000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=5000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=10000000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" -k "$request_url" + fi +} + +https_short_requests() +{ + if [ "$core_nums" == 4 ] || [ "$core_nums" == 8 ]; then + requests_num=500000 + threads_num=100 + concurrent_num=100 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 48 ]; then + requests_num=500000 + threads_num=600 + concurrent_num=600 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + elif [ "$core_nums" == 96 ]; then + requests_num=1000000 + threads_num=1200 + concurrent_num=1200 + httpress -n "$requests_num" -c "$concurrent_num" -t "$threads_num" "$request_url" + fi +} + +main() +{ + local request_url="$protocol://$request_ip:$request_port/$request_index" + + ip addr add $nic_ip/24 dev "$nic" + ip link set "$nic" up + + if [ "$protocol" == 'http' ]; then + if [ "$alive_type" == 'long' ]; then + http_long_requests + else + http_short_requests + fi + elif [ "$protocol" == 'https' ]; then + if [ "$alive_type" == 'long' ]; then + https_long_requests + else + https_short_requests + fi + fi +} + +main -- 2.23.0

2 1

[PATCH v5 compass-ci] tutorial.md: update initramfs tutorial.md
by Wang Chenglong 06 Jan '21

06 Jan '21

Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- doc/tutorial.md | 71 ++++++++++++------------------------------------- 1 file changed, 17 insertions(+), 54 deletions(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index bf72630..f7ca26e 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -150,61 +150,24 @@ job yaml ��[YAML](http://yaml.org/YAM #### �� - 1. �� os �� rootfs�� openEuler �� - - �� docker �� rootfs - - 1) �� openEuler �� docker �� - - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/docker_img/aarch64/openEuler… - ``` - 2) �� docker �� - ```bash - docker load -i openEuler-docker.aarch64 - ``` - - 3) �� openEuler �� - ```bash - docker run -id openeuler-20.03-lts - ``` - 4) �� docker �� rootfs - ```bash - docker cp -a docker run -d openeuler-20.03-lts:/ openEuler-rootfs - ``` - - �� qemu.img(qcow2��)�� rootfs (openEuler��) - - - 1) �� openEule r�� qcow2 �� - ```bash - wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… - ``` - 2) ��{compass-ci}/container/qcow2rootfs ��rootfs - ```bash - cd {compass-ci}/container/qcow2rootfs - ./run openEuler-20.03-LTS.aarch64.qcow2.xz /tmp/openEuler-rootfs - ``` - 2. ��rootfs - 1. ��chroot�� rootfs (�� root ��) - ```bash - chroot openEuler-rootfs - ``` - 2. �� - - a. �� root �� - b. �� ssh �� - c. �� - d. �� docker �� osimage �� - > 1. �� - > 2. �� centos ��rpm�� - > 3. �� yum �� - > 4. �� docker �� - - 3. �� rootfs�� - ```bash - cd $rootfs - find . | coip -o -Hnewc |gzip -9 > $os_name.cgz - ``` + 1. �� openEuler �� + ```bash + cd ${CCI_SRC}/rootfs/initramfs/openeuler/aarch64/20.03-LTS/ + ./run + ``` + + 2. ��/�� + �� ${CCI_SRC} ��rootfs�� rootfs ��/�� + ```bash + mkdir rootfs + cd rootfs + zcat ../$os_name.cgz | cpio -idmv + find ./lib/modules/* | cpio -o -Hnewc | gzip -9 > $HOME/modules-$os_name.cgz + find ./usr/src/kernels/* | cpio -o -Hnewc | gzip -9 > $HOME/headers-$os_name.cgz + ``` + #### FAQ + 1. �� Unable to mount root fs on unknown-block�� - �� ```bash -- 2.23.0

1 0

[PATCH v2 compass-ci] providers/docker: copy rootfs/addon/root/. to container
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 4388e19..b28b24c 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -3,6 +3,6 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. mkdir -p /root/.ssh -cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ +cp -r /lkp/lkp/src/rootfs/addon/root/. /root cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap -- 2.23.0

1 0

[PATCH compass-ci] providers/docker: copy rootfs/addon/root/. to container
by Xiao Shenwei 06 Jan '21

06 Jan '21

Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/docker/bin/entrypoint.sh | 2 +- providers/docker/run.sh | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/providers/docker/bin/entrypoint.sh b/providers/docker/bin/entrypoint.sh index 4388e19..b28b24c 100755 --- a/providers/docker/bin/entrypoint.sh +++ b/providers/docker/bin/entrypoint.sh @@ -3,6 +3,6 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. mkdir -p /root/.ssh -cp /lkp/lkp/src/rootfs/addon/root/.ssh/id_rsa /root/.ssh/ +cp -r /lkp/lkp/src/rootfs/addon/root/. /root cp -r /lkp/lkp/src/rootfs/addon/usr/* /usr sh /lkp/lkp/src/rootfs/addon/etc/init.d/lkp-bootstrap diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 62a1f9f..57a01c0 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -22,18 +22,19 @@ busybox_path=$(command -v busybox) cmd=( docker run --rm + --hostname $host -m $memory --tmpfs /tmp:rw,exec,nosuid,nodev -e CCI_SRC=/c/compass-ci -v ${load_path}/lkp:/lkp - -v ${DIR}/bin:/root/bin:ro + -v ${DIR}/bin:/root/sbin:ro -v $CCI_SRC:/c/compass-ci:ro -v /srv/git:/srv/git:ro -v /srv/result:/srv/result:ro -v ${busybox_path}:/usr/local/bin/busybox --oom-score-adj="-1000" ${docker_image} - /root/bin/entrypoint.sh + /root/sbin/entrypoint.sh ) "${cmd[@]}" -- 2.23.0

1 1

[PATCH compass-ci] mail: rename Compass-ci to Compass-CI
by Wu Fengguang 06 Jan '21

06 Jan '21

Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- lib/mail_job_result.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/mail_job_result.rb b/lib/mail_job_result.rb index 50e2f53..796a4f3 100644 --- a/lib/mail_job_result.rb +++ b/lib/mail_job_result.rb @@ -22,8 +22,8 @@ class MailJobResult def compose_mail set_submitter_info - subject = "[Compass-ci] job: #{@job_id} result" - signature = "Regards\nCompass-ci\nhttps://gitee.com/openeuler/compass-ci" + subject = "[Compass-CI] job: #{@job_id} result" + signature = "Regards\nCompass-CI\nhttps://gitee.com/openeuler/compass-ci" body = "Hi, Thanks for your participation in Kunpeng and software ecosystem! Your Job: #{@job_id} had finished. -- 2.23.0

1 0

[PATCH compass-ci] sparrow: add setup.yaml
by Liu Yinsi 06 Jan '21

06 Jan '21

[why] before deploy compass-ci, fill configuration setup.yaml firstly, will make the whole process cleaner. fill my_name and my_email, it's necessary to verify account when submit job. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/setup.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 sparrow/setup.yaml diff --git a/sparrow/setup.yaml b/sparrow/setup.yaml new file mode 100644 index 0000000..b1cac8c --- /dev/null +++ b/sparrow/setup.yaml @@ -0,0 +1,11 @@ +# please copy this template with the below commands, +# then fill the configuration according to comments. +# mkdir -p /etc/compass-ci/ +# cp setup.yaml /etc/compass-ci/setup.yaml +# vim /etc/compass-ci/setup.yaml + +# account name +my_name: xxxxxx + +# email address +my_email: xxx@xxx -- 2.23.0

3 4

[PATCH compass-ci] doc: update doc
by Luan Shengde 06 Jan '21

06 Jan '21

rename email address compass-ci-reply(a)qq.com => compass-ci-robot(a)qq.com Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/mail-robot.md | 2 +- doc/manual/apply-account.md | 2 +- doc/manual/borrow-machine.en.md | 2 +- doc/manual/borrow-machine.zh.md | 2 +- ...50\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index e97912c..6158b65 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -11,7 +11,7 @@ apply account/uuid for user ## usage build email as the following section: allowed email format -send mail to: compass-ci-reply(a)qq.com +send mail to: compass-ci-robot(a)qq.com ## allowed email format: diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 08e7618..a0b2c82 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -6,7 +6,7 @@ Email template: --- From: {{ David Rientjes <rientjes(a)google.com> }} - To: compass-ci-reply(a)qq.com + To: compass-ci-robot(a)qq.com Subject: apply account my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} diff --git a/doc/manual/borrow-machine.en.md b/doc/manual/borrow-machine.en.md index bed9fcc..1b4a220 100644 --- a/doc/manual/borrow-machine.en.md +++ b/doc/manual/borrow-machine.en.md @@ -3,7 +3,7 @@ Ensure that you have performed the following operations according to the [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc…: - Send an email to apply for an account. -- Receive an email from compass-ci-reply(a)qq.com. +- Receive an email from compass-ci-robot(a)qq.com. - Configure the local environment. # Applying for a Test Machine (VM) diff --git a/doc/manual/borrow-machine.zh.md b/doc/manual/borrow-machine.zh.md index 3f500e2..bb1c08f 100644 --- a/doc/manual/borrow-machine.zh.md +++ b/doc/manual/borrow-machine.zh.md @@ -2,7 +2,7 @@ 请确认您已按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… - send apply account email. -- receive email from compass-ci-reply(a)qq.com. +- receive email from compass-ci-robot(a)qq.com. - local environment configuration. diff --git "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" index 6f0736b..ad933b1 100644 --- "a/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" +++ "b/doc/manual/\346\234\254\345\234\260\345\256\211\350\243\205compass-ci\345\256\242\346\210\267\347\253\257.md" @@ -6,7 +6,7 @@ Compass-CI 将 [lkp-tests](https://gitee.com/wu_fengguang/lkp-tests) 作为客 :exclamation: 前提条件：按照 [apply-account.md](https://gitee.com/wu_fengguang/compass-ci/blob/master/doc… 完成如下动作： - send apply account email -- receive email from compass-ci-reply(a)qq.com +- receive email from compass-ci-robot(a)qq.com - get reply email and follow its instructions to - setup default config -- 2.23.0

1 0

[PATCH compass-ci] providers/docker: change the mount path
by Xiao Shenwei 06 Jan '21

06 Jan '21

[why] /root/bin the dir may be used by user to stores own file --- providers/docker/run.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/providers/docker/run.sh b/providers/docker/run.sh index 62a1f9f..57a01c0 100755 --- a/providers/docker/run.sh +++ b/providers/docker/run.sh @@ -22,18 +22,19 @@ busybox_path=$(command -v busybox) cmd=( docker run --rm + --hostname $host -m $memory --tmpfs /tmp:rw,exec,nosuid,nodev -e CCI_SRC=/c/compass-ci -v ${load_path}/lkp:/lkp - -v ${DIR}/bin:/root/bin:ro + -v ${DIR}/bin:/root/sbin:ro -v $CCI_SRC:/c/compass-ci:ro -v /srv/git:/srv/git:ro -v /srv/result:/srv/result:ro -v ${busybox_path}:/usr/local/bin/busybox --oom-score-adj="-1000" ${docker_image} - /root/bin/entrypoint.sh + /root/sbin/entrypoint.sh ) "${cmd[@]}" -- 2.23.0

1 0