- Compass-ci - mailweb.openeuler.org

[PATCH v2 lkp-tests] bin/run_job: Automatic task execution
by Hu Xuejiao 24 Dec '20

24 Dec '20

[purpose] Verify that a local job can be consumed properly Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sbin/run_job | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100755 sbin/run_job diff --git a/sbin/run_job b/sbin/run_job new file mode 100755 index 00000000..d45571b3 --- /dev/null +++ b/sbin/run_job @@ -0,0 +1,74 @@ +#!/bin/bash + +cci_makepkg_file="$LKP_SRC/jobs/cci-makepkg.yaml" +grep "atomic" $cci_makepkg_file || sed -i 's|benchmark:|& atomic|g' $cci_makepkg_file + +submit_one_job() +{ + cmd=( + submit -m $yaml + os=${single_arg[0]} + os_arch=${single_arg[1]} + os_version=${single_arg[2]} + os_mount=${single_arg[3]} + testbox=$testbox + queue="$tbox_group.$HOSTNAME" + ) + echo ${cmd[@]} + output=$("${cmd[@]}") + echo "$output" | grep -o -E 'job_state=finished[^\"]*' +} + +submit_job() +{ + yaml_args=("$cci_makepkg_file" "host-info.yaml") + for yaml in "${yaml_args[@]}" + do + submit_one_job & + done +} + +run_qemu() +{ + local nr_vm=4 + local testbox="vm-2p8g" + local tbox_group="vm-2p8g" + local hostname=$tbox_group.$HOSTNAME + local queues="$tbox_group.$HOSTNAME" + + $CCI_SRC/providers/multi-qemu -n "$hostname" -c "$nr_vm" -q "$queues" + + qemu_args=( + "openeuler aarch64 20.03 cifs" + "centos aarch64 7.6.1810 initramfs" + ) + for qemu_arg in "${qemu_args[@]}" + do + local single_arg=($(echo $qemu_arg)) + submit_job + done +} + +run_docker() +{ + local nr_container=2 + local tbox_group="dc-8g" + local testbox="dc-8g" + local hostname=$tbox_group.$HOSTNAME + local queues="$tbox_group.$HOSTNAME" + + $CCI_SRC/providers/multi-docker -n $hostname -c $nr_container -q $queues + + docker_args=( + "centos aarch64 7.6.1810 container" + ) + for docker_arg in "${docker_args[@]}" + do + local single_arg=($(echo $docker_arg)) + submit_job + done +} + +run_qemu +run_docker +wait -- 2.23.0

2 1

[PATCH compass-ci 2/3] 3-code/git: add repository version pointing
by Hu Xuejiao 24 Dec '20

24 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- sparrow/3-code/git | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/sparrow/3-code/git b/sparrow/3-code/git index 494107e..2159ddc 100755 --- a/sparrow/3-code/git +++ b/sparrow/3-code/git @@ -13,11 +13,18 @@ cd /c || exit # modify and manual run for now: [ -d "compass-ci/.git" ] || { git clone https://gitee.com/wu_fengguang/compass-ci.git - ln -s compass-ci cci + + cd /c/compass-ci || exit + [ -n "$compass_commitid" ] && git reset --hard "$compass_commitid" + + ln -s compass-ci /c/cci } [ -d "lkp-tests/.git" ] || { git clone https://gitee.com/wu_fengguang/lkp-tests.git + + cd /c/lkp-tests || exit + [ -n "$lkp_commitid" ] && git reset --hard "$lkp_commitid" } [ -d "upstream-repos/.git" ] || { -- 2.23.0

1 0

[PATCH compass-ci 1/3] doc/job: add SCHED_HOST, SCHED_PORT description
by Hu Xuejiao 24 Dec '20

24 Dec '20

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- doc/job/SCHED_HOST-SCHED_PORT.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 doc/job/SCHED_HOST-SCHED_PORT.md diff --git a/doc/job/SCHED_HOST-SCHED_PORT.md b/doc/job/SCHED_HOST-SCHED_PORT.md new file mode 100644 index 0000000..74c628a --- /dev/null +++ b/doc/job/SCHED_HOST-SCHED_PORT.md @@ -0,0 +1,15 @@ +# SCHED_HOST, SCHED_PORT +Meaning: +- The job task depends on scheduler service, which will distribute tasks to test + machine. All tasks will run in test machine. +- `Scheduler` is one of our service, `SCHED` is the abbreviation of `Scheduler` + these variables mean: + SCHED_HOST: the scheduler service ip. + SCHED_PORT: the scheduler service port. + +Usage example: + +```$LKP_SRC/include/lab/z9 +SCHED_HOST: api.compass-ci.openeuler.org +SCHED_PORT: 11306 +``` -- 2.23.0

1 0

[PATCH lkp-tests] lib/bootstrap.sh: fix redirect_stdout_stderr for docker
by Xiao Shenwei 24 Dec '20

24 Dec '20

[why] exec command will redirect stdout stderr to /tmp/stdout, /tmp/stderr but docker log based on /dev/stdout and /dev/stderr, it cause the task status cannot be viewed from the console, which is difficult to debug. Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- lib/bootstrap.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/bootstrap.sh b/lib/bootstrap.sh index ad1c4a2f2..c9a9f89f9 100755 --- a/lib/bootstrap.sh +++ b/lib/bootstrap.sh @@ -315,6 +315,13 @@ announce_bootup() redirect_stdout_stderr() { if is_docker; then + [ -e /tmp/stdout ] || touch /tmp/stdout + [ -e /tmp/stderr ] || touch /tmp/stderr + tail -f /tmp/stdout > /dev/stdout & + echo $! >> /tmp/pid-tail-global + tail -f /tmp/stderr > /dev/stderr & + echo $! >> /tmp/pid-tail-global + exec > /tmp/stdout exec 2> /tmp/stderr return -- 2.23.0

1 0

[PATCH v2 lkp-tests] adaptation/archlinux: add archlinux mapping
by Wang Yong 24 Dec '20

24 Dec '20

Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/adaptation/archlinux | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/distro/adaptation/archlinux b/distro/adaptation/archlinux index 89163cf41..c52212e08 100644 --- a/distro/adaptation/archlinux +++ b/distro/adaptation/archlinux @@ -72,3 +72,11 @@ libtirpc: libtirpc1 initramfs-tools: openssh-server: openssh openssh-client: openssh +autopoint: gettext +g++: gcc +libarchive13: libarchive +libbz2-1.0: bzip2 +libssl: openssl +libtool-bin: libtool +linux-perf: perf +ruby-dev: ruby -- 2.23.0

1 0

[PATCH compass-ci] lib/env.cr: rename to web_env.cr
by Wu Zhende 24 Dec '20

24 Dec '20

[Why] env.cr is the same as the file name provided by the crystal. [Error] /c/cci/scheduler $ crystal a.cr Showing last frame. Use --error-trace for full trace. In /usr/lib/crystal/core/json/from_json.cr:140:5 140 | def Set.new(pull : JSON::PullParser) Error: undefined constant Set Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/sched.cr | 2 +- src/lib/{env.cr => web_env.cr} | 0 src/scheduler/scheduler.cr | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename src/lib/{env.cr => web_env.cr} (100%) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 6caa5b9..2d9dba6 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -5,7 +5,7 @@ require "kemal" require "yaml" require "./job" -require "./env" +require "./web_env" require "./block_helper" require "./taskqueue_api" require "./remote_git_client" diff --git a/src/lib/env.cr b/src/lib/web_env.cr similarity index 100% rename from src/lib/env.cr rename to src/lib/web_env.cr diff --git a/src/scheduler/scheduler.cr b/src/scheduler/scheduler.cr index 2103f7c..6b032ee 100644 --- a/src/scheduler/scheduler.cr +++ b/src/scheduler/scheduler.cr @@ -3,7 +3,7 @@ require "kemal" -require "../lib/env" +require "../lib/web_env" require "../lib/sched" require "../lib/json_logger" -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] sparrow/4-docker/buildall: build dist image when it doesn't exist
by Xu Xijian 24 Dec '20

24 Dec '20

If the dist docker image doesn't exist for some reason, run build script to create it. Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- sparrow/4-docker/buildall | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 32801d1..9e66ac9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -32,7 +32,8 @@ do_one() ( cd $container [ "$container_name" == 'ssh-r' ] && exit - if [ "$action" != "reboot" ]; then + docker images | grep -w -q "$container_name" + if [ "$?" == 0 ] && [ "$action" != "reboot" ]; then [ -x build ] && ./build [ -x install ] && ./install fi -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] code-spec.md: adjust markdown document format
by Xu Xijian 24 Dec '20

24 Dec '20

In markdown documents, text for bash should be contained in block as belows: ```SHELL content SHELL Signed-off-by: Xu Xijian <hdxuxijian(a)163.com> --- doc/code-spec.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index d44c6fe..a717f64 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -21,10 +21,12 @@ code注意事项 - puts/printf字符串应当容易搜索(grep)。特别注意包含足够长的固定内容substring. 例如： +```SHELL die "test yaml$${test_yaml}$ not exist in ${LKP_SRC}/jobs !!!" ==> die "cannot find test yaml ($test_yaml) in $LKP_SRC/jobs" ^^^^^^^^^^^^^^^^^^^^^ grep-friendly fixed substring +``` 重构友好编码 ============ -- 2.23.0

1 0

[PATCH compass-ci] lib/mail_client: add send_mail_text function
by Luan Shengde 24 Dec '20

24 Dec '20

for user use mail_client to send mail with 'mail content' data type Signed-off-by: Luan Shengde <shdluan(a)163.com> --- lib/mail_client.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/mail_client.rb b/lib/mail_client.rb index b13492f..f63e65d 100644 --- a/lib/mail_client.rb +++ b/lib/mail_client.rb @@ -24,4 +24,9 @@ class MailClient resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_encode") resource.post(Base64.encode64(mail_data)) end + + def send_mail_text(mail_text) + resource = RestClient::Resource.new("http://#{@host}:#{@port}/send_mail_text") + resource.post(mail_text) + end end -- 2.23.0

1 0

[PATCH lkp-tests] tests/cci-depends: update depends for cci-depends
by Wang Yong 24 Dec '20

24 Dec '20

[Why] cci-depends need base packages to generate cgz file, and mount operation is deprecated Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/depends/cci-depends | 3 ++- tests/cci-depends | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/distro/depends/cci-depends b/distro/depends/cci-depends index 1f7f73ac3..0dc9d1501 100644 --- a/distro/depends/cci-depends +++ b/distro/depends/cci-depends @@ -1,2 +1,3 @@ -cifs-utils cpio +curl +gzip diff --git a/tests/cci-depends b/tests/cci-depends index 42d5ef676..8baf023f3 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -44,6 +44,8 @@ check_shared_package "$benchmark" && { share_debian_package +distro_install_depends cci-depends + fixup_distro_mirror update -- 2.23.0

1 0