- Compass-ci - mailweb.openeuler.org

[PATCH v6 lkp-tests] lib: add forward_port.sh
by Zhang Yale 15 Oct '20

15 Oct '20

Provide a login port for user, determine whether to send the ssh_port message based on the uuid. Signed-off-by: Zhang Yale <zhangyale3(a)huawei.com> --- lib/forward_port.sh | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100755 lib/forward_port.sh diff --git a/lib/forward_port.sh b/lib/forward_port.sh new file mode 100755 index 00000000..fa45953b --- /dev/null +++ b/lib/forward_port.sh @@ -0,0 +1,25 @@ +#!/bin/bash +# uuid +# sshr_port_len +# sshr_port_base + +. $LKP_SRC/lib/http.sh + +sshr_ip=$LKP_SERVER +sshr_port=5050 +[ -n "$sshr_port_len" ] || sshr_port_len=2000 +[ -n "$sshr_port_base" ] || sshr_port_base=50000 + +get_port() +{ + [ -n "$uuid" ] || return + for i in $(seq 1 10) + do + port=$(($(date +%s%N)%"$sshr_port_len"+"$sshr_port_base")) + ssh -o StrictHostKeyChecking=no -o ExitOnForwardFailure=yes -o TCPKeepAlive=yes \ + -Nf -R $port:localhost:22 sshr@"$sshr_ip" -p "$sshr_port" &>/dev/null + [ $? -eq 0 ] || continue + echo "$port" + break + done +} -- 2.23.0

2 4

[PATCH lkp-tests] fix(lkp-bootstrap.servce):$GOCACHE is not defined with jobs that use go-lang
by Zhou Shengrui 15 Oct '20

15 Oct '20

The lkp-bootstrap.service runs before login, and it's environment variables is not completely full.The jobs run with go-lang can't get $GOCACHE and $HOME, will prompt errors like below: build cache is required, but could not be located: GOCACHE is not defined and neither $XDG_CACHE_HOME nor $HOME are defined make: *** [Makefile:34: runc] Error 1 ==> ERROR: A failure occurred in build(). Aborting... Signed-off-by: Zhou Shengrui <zhoushengrui(a)qq.com> --- rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service index 50320140..92e7e5ee 100644 --- a/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service +++ b/rootfs/addon/usr/lib/systemd/system/lkp-bootstrap.service @@ -5,7 +5,7 @@ After=network.target [Service] Type=forking -ExecStart=/etc/init.d/lkp-bootstrap +ExecStart=su -c /etc/init.d/lkp-bootstrap IgnoreSIGPIPE=no KillMode=none TasksMax=infinity -- 2.23.0

3 2

[PATCH lkp-tests] lib/job.rb: revoke change for a9c727816f01a99
by Wei Jihui 15 Oct '20

15 Oct '20

commit id for a9c727816f01a99 is not good, so revoke it Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/lib/job.rb b/lib/job.rb index 6497f0f1..f91686b5 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -568,9 +568,7 @@ class Job init_program_options() for_each_in(@job, @referenced_programs.keys) do |_pk, h, p_n, p_args| k = hash_key_re_string(@job, p_n, @job['pp'].keys) - next if k.nil? - - unless @job[k].is_a?(Hash) + if k && !(a)job[k].is_a?(Hash) @job['pp'][k] = @job[k] next end -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: tbox -> tbox_group
by Ren Wen 15 Oct '20

15 Oct '20

fix an error. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 223a30c..a4ddcec 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -418,7 +418,7 @@ class Sched private def find_job(testbox : String, count = 1) tbox_group = JobHelper.match_tbox_group(testbox) - tbox = tbox.partition("--")[0] + tbox = tbox_group.partition("--")[0] boxes = [testbox, tbox_group, tbox] boxes.each do |box| -- 2.23.0

1 1

[PATCH compass-ci] fix: ssh-r: use reserved port range
by Tong Qunfeng 15 Oct '20

15 Oct '20

host reserved an port range (in sysctl.conf) <net.ipv4.ip_local_port_range = 1024 49999> ssh-r mapping the start 2000 port (50000 - 51999) Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- container/ssh-r/Dockerfile | 2 +- container/ssh-r/start | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/container/ssh-r/Dockerfile b/container/ssh-r/Dockerfile index 1c10e03..65a13aa 100644 --- a/container/ssh-r/Dockerfile +++ b/container/ssh-r/Dockerfile @@ -21,7 +21,7 @@ RUN chown -R sshr:sshr /home/sshr/.ssh/authorized_keys \ && chmod 600 /home/sshr/.ssh/authorized_keys \ && passwd -u sshr -EXPOSE 20000-22000 +EXPOSE 50000-51999 EXPOSE 22 COPY entry.sh /entry.sh diff --git a/container/ssh-r/start b/container/ssh-r/start index e962656..5c800f3 100755 --- a/container/ssh-r/start +++ b/container/ssh-r/start @@ -15,7 +15,7 @@ cmd=( -e GATEWAY_PORTS=true -e TCP_FORWARDING=true -d - -p 20000-22000:20000-22000 + -p 50000-51999:50000-51999 -p 5050:22 -v /etc/localtime:/etc/localtime:ro -v /srv/pub/sshr/keys/:/etc/ssh/keys -- 2.23.0

1 0

[PATCH lkp-tests 2/3] lib/job.rb: load host config with atomic job
by Wei Jihui 15 Oct '20

15 Oct '20

Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index 8a6d68a5..38df0368 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -696,9 +696,11 @@ class Job each_job_init each_job(&block) @jobs.each do |hash| + @load_hosts_done = false @job = deepcopy(job) @job2 = hash load_defaults + load_hosts_config each_job_init each_job(&block) end -- 2.23.0

2 1

[PATCH lkp-tests] feat(iso2rootfs): add a soft link to the newest version of rootfs dir
by Yu Chuan 15 Oct '20

15 Oct '20

Allow job can use the debug rootfs dir by specifying 'os_version' as '{os_version}-debug'. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- tests/iso2rootfs | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/tests/iso2rootfs b/tests/iso2rootfs index 5184041abf2b..7ab4a08b0d60 100755 --- a/tests/iso2rootfs +++ b/tests/iso2rootfs @@ -72,13 +72,14 @@ get_daily_iso_checksum() mount_rootfs() { + DEBUG_ROOTFS_DIR="debug-versions/dailybuild" case ${rootfs_protocol} in "nfs") - ROOTFS_SERVER_PATH="${rootfs_server}:${rootfs_path}/${iso_os}/${iso_arch}/debug-versions/dailybuild" + ROOTFS_SERVER_PATH="${rootfs_server}:${rootfs_path}/${iso_os}/${iso_arch}" install_pkgs "nfs-utils" ;; "cifs") - ROOTFS_SERVER_PATH="//${rootfs_server}/${rootfs_path}/${iso_os}/${iso_arch}/debug-versions/dailybuild" + ROOTFS_SERVER_PATH="//${rootfs_server}/${rootfs_path}/${iso_os}/${iso_arch}" ROOTFS_MOUNT_PARAM="guest,vers=1.0,noacl,nouser_xattr" install_pkgs "cifs-utils" ;; @@ -102,7 +103,7 @@ get_cache_iso_checksum() { mount_rootfs - CHECKSUM_FILE_CACHE="${ROOTFS_LOCAL_PATH}/${iso_version}-newest.sha256sum" + CHECKSUM_FILE_CACHE="${ROOTFS_LOCAL_PATH}/${DEBUG_ROOTFS_DIR}/${iso_version}-newest.sha256sum" [ ! -f "${CHECKSUM_FILE_CACHE}" ] || SHA256SUM_CACHE=$(awk '{print $1}' "$CHECKSUM_FILE_CACHE") } @@ -254,7 +255,7 @@ download_compass_ci() config_rootfs_dir() { - ROOTFS_DES_DIR=${ROOTFS_LOCAL_PATH}/${iso_version}-$(date "+%Y%m%d%H%M%S") + ROOTFS_DES_DIR=${ROOTFS_LOCAL_PATH}/${DEBUG_ROOTFS_DIR}/${iso_version}-$(date "+%Y%m%d%H%M%S") [ -d "${ROOTFS_DES_DIR}" ] && ROOTFS_DES_DIR="${ROOTFS_DES_DIR}-${HOSTNAME##*--}" mkdir -p "$ROOTFS_DES_DIR" @@ -309,10 +310,10 @@ test_rootfs() local root_path case ${rootfs_protocol} in "nfs") - root_path="${ROOTFS_SERVER_PATH}/$(basename "${ROOTFS_DES_DIR}")" + root_path="${ROOTFS_SERVER_PATH}/${DEBUG_ROOTFS_DIR}/$(basename "${ROOTFS_DES_DIR}")" ;; "cifs") - root_path="cifs:${ROOTFS_SERVER_PATH}/$(basename "${ROOTFS_DES_DIR}"),${ROOTFS_MOUNT_PARAM}" + root_path="cifs:${ROOTFS_SERVER_PATH}/${DEBUG_ROOTFS_DIR}/$(basename "${ROOTFS_DES_DIR}"),${ROOTFS_MOUNT_PARAM}" ;; *) die "rootfs_protocol is none, exit !!!" @@ -362,7 +363,12 @@ post_works() mv "${CHECKSUM_FILE_CACHE}.tmp" "${CHECKSUM_FILE_CACHE}" - umount "${ROOTFS_LOCAL_PATH}" + cd "${ROOTFS_LOCAL_PATH}" && + rm -f "${iso_version}-debug" && + ln -s "${DEBUG_ROOTFS_DIR}/$(basename ${ROOTFS_DES_DIR})" \ + "${iso_version}-debug" + + cd / && umount "${ROOTFS_LOCAL_PATH}" log_info "iso2rootfs finished" } -- 2.23.0

1 0

[PATCH compass-ci] fix: sched: find job from 2 types of tbox_group queues
by Ren Wen 15 Oct '20

15 Oct '20

Job is saved into tbox_group queues when submitting, so find job from tbox_group queues. When hostname is 'taishan200-2280-2s48p-256g--a11-12', first find job from 'taishan200-2280-2s48p-256g--a11', then find from 'taishan200-2280-2s48p-256g--a11-12'. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2fbf05d..2fab207 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -417,10 +417,15 @@ class Sched end private def find_job(testbox : String, count = 1) - tbox = JobHelper.match_tbox_group(testbox) - tbox_group = tbox.partition("--")[0] - - boxes = [testbox, tbox, tbox_group] + # For example: + # testbox : taishan200-2280-2s48p-256g--a11-12 + # tbox : taishan200-2280-2s48p-256g--a11 + # tbox_group : taishan200-2280-2s48p-256g + sep = "--" + tbox = testbox.sub(/(-\d+)+$/, "") + tbox_group = tbox.rchop("#{sep}#{tbox.split(sep)[-1]}") + + boxes = [tbox, tbox_group] boxes.each do |box| count.times do job = prepare_job("sched/#{box}", testbox) -- 2.23.0

2 2

[PATCH v2 compass-ci] fix: sched: fix variable name not find
by Ren Wen 15 Oct '20

15 Oct '20

Forget to modify variable name when exchanging 2 variable names. So fix it. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 223a30c..a4ddcec 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -418,7 +418,7 @@ class Sched private def find_job(testbox : String, count = 1) tbox_group = JobHelper.match_tbox_group(testbox) - tbox = tbox.partition("--")[0] + tbox = tbox_group.partition("--")[0] boxes = [testbox, tbox_group, tbox] boxes.each do |box| -- 2.23.0

1 0

[PATCH v3 compass-ci] fix: sched: find and submit idle job using tbox_group
by Ren Wen 15 Oct '20

15 Oct '20

Using tbox_group to find and submit idle jobs. Example: Here is a hostname: taishan200-2280-2s48p-256g--a11-<sn> Before: using taishan200-2280-2s48p-256g Now: using taishan200-2280-2s48p-256g--a11 Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 2fbf05d..223a30c 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -417,10 +417,10 @@ class Sched end private def find_job(testbox : String, count = 1) - tbox = JobHelper.match_tbox_group(testbox) - tbox_group = tbox.partition("--")[0] + tbox_group = JobHelper.match_tbox_group(testbox) + tbox = tbox.partition("--")[0] - boxes = [testbox, tbox, tbox_group] + boxes = [testbox, tbox_group, tbox] boxes.each do |box| count.times do job = prepare_job("sched/#{box}", testbox) -- 2.23.0

1 1