- Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci 2/2] container/es/start: call check_service_ready()
by Liu Yinsi 01 Mar '21

01 Mar '21

call container/defconfig.sh: check_service_ready() to accelerate es/logging-es startup time. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/es/start | 7 +++---- container/logging-es/start | 5 ++--- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/container/es/start b/container/es/start index d84ad33..67d6531 100755 --- a/container/es/start +++ b/container/es/start @@ -20,7 +20,6 @@ cmd=( "${cmd[@]}" -echo "Please wait 60s for setting index" -sleep 60 - -set_es_indices +# set index +check_service_ready 9200 +find $CCI_SRC/sbin/ -name "es-*-mapping.sh" -exec sh {} \; diff --git a/container/logging-es/start b/container/logging-es/start index 66464e6..efd89de 100755 --- a/container/logging-es/start +++ b/container/logging-es/start @@ -20,9 +20,8 @@ cmd=( "${cmd[@]}" -echo "Please wait 60s for setting index" -sleep 60 - +# set index +check_service_ready 9202 curl -sSH "Content-Type: application/json" -XPUT "localhost:9202/_settings" -d ' { "index": { -- 2.23.0

1 0

[PATCH v3 compass-ci 1/2] container/defconfig.sh: check service ready
by Liu Yinsi 01 Mar '21

01 Mar '21

[why] check service ready or not every 2s, if ok, continue next step, rather than sleep 60s, it can save time. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.sh | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/container/defconfig.sh b/container/defconfig.sh index c33a790..50446d8 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -28,9 +28,15 @@ docker_rm() docker rm -f $container } -set_es_indices() +check_service_ready() { - find $CCI_SRC/sbin/ -name "es-*-mapping.sh" -exec sh {} \; + local port=$1 + local i + for i in {1..30} + do + curl -s localhost:$port > /dev/null && return + sleep 2 + done } push_image() -- 2.23.0

1 0

[PATCH v3 compass-ci 2/2] sparrow/4-docker/buildall: add $action value "start-only"
by Liu Yinsi 01 Mar '21

01 Mar '21

[usage] ./buildall start-only [how] before: ./buildall run-only: call build_depends(), check images exist or not, then call start_depends() after: ./buildall start-only: only call start_depends() [why] for some users locally deploy compass-ci, they will not build images, and some users can't install lockfile-create command, only run containers, no need to check images exist or not, so only call start_depends(). Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index aa48bce..74016d7 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -69,7 +69,7 @@ tmpdir=$(mktemp -d) for dir in $CONTAINER_PATH/*/ do - build_depends $dir + [ "$action" != "start-only" ] && build_depends $dir start_depends $dir done -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] container/es/start: call check_es_ready()
by Liu Yinsi 01 Mar '21

01 Mar '21

call container/defconfig.sh:check_service_ready() to accelerate es/logging-es startup time. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/es/start | 7 +++---- container/logging-es/start | 5 ++--- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/container/es/start b/container/es/start index d84ad33..67d6531 100755 --- a/container/es/start +++ b/container/es/start @@ -20,7 +20,6 @@ cmd=( "${cmd[@]}" -echo "Please wait 60s for setting index" -sleep 60 - -set_es_indices +# set index +check_service_ready 9200 +find $CCI_SRC/sbin/ -name "es-*-mapping.sh" -exec sh {} \; diff --git a/container/logging-es/start b/container/logging-es/start index 66464e6..efd89de 100755 --- a/container/logging-es/start +++ b/container/logging-es/start @@ -20,9 +20,8 @@ cmd=( "${cmd[@]}" -echo "Please wait 60s for setting index" -sleep 60 - +# set index +check_service_ready 9202 curl -sSH "Content-Type: application/json" -XPUT "localhost:9202/_settings" -d ' { "index": { -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] container/defconfig.sh: check service ready
by Liu Yinsi 01 Mar '21

01 Mar '21

[why] check service ready or not every 2s, if ok, continue next step, rather than sleep 60s, it can save time. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.sh | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/container/defconfig.sh b/container/defconfig.sh index c33a790..50446d8 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -28,9 +28,15 @@ docker_rm() docker rm -f $container } -set_es_indices() +check_service_ready() { - find $CCI_SRC/sbin/ -name "es-*-mapping.sh" -exec sh {} \; + local port=$1 + local i + for i in {1..30} + do + curl -s localhost:$port > /dev/null && return + sleep 2 + done } push_image() -- 2.23.0

1 0

[PATCH compass-ci] lib/git_mirror.rb: fix that unexpected file would cause a load error
by Li Yuanchao 01 Mar '21

01 Mar '21

Sometimes there would be some not yaml like file in upstream-repos, such as README.md or user's temporary file. Now we'll do a judge after load and before any other actions, if it's not repo file, just return. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index 76cdd1b..22c4e07 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -223,7 +223,10 @@ end class MirrorMain def load_repo_file(repodir, project, fork_name) git_repo = "#{project}/#{fork_name}" - @git_info[git_repo] = YAML.safe_load(File.open(repodir)) + git_info = YAML.safe_load(File.open(repodir)) + return if git_info.nil? || git_info['url'].nil? + + @git_info[git_repo] = git_info @git_info[git_repo]['git_repo'] = git_repo @git_info[git_repo] = merge_defaults(git_repo, @git_info[git_repo]) fork_stat_init(git_repo) -- 2.23.0

2 2

[PATCH v2 compass-ci 1/2] sparrow/4-docker/buildall: check whether skip ssh-r
by Liu Yinsi 01 Mar '21

01 Mar '21

[why] for user locally deploy compass-ci, ssh-r is not exists, so check when ssh-r already exists, skip ssh-r. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index bb77a1f..f53bf4c 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -56,7 +56,7 @@ do_one_run() mkdir $tmpdir/start_$container_name 2>/dev/null && ( cd "$container" - [ "$container_name" == 'ssh-r' ] && exit + [ -n "$(docker ps -aqf name='ssh_r')" ] && exit [ -x first-run ] && ./first-run [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run -- 2.23.0

2 2

[PATCH v2 compass-ci 2/2] sparrow/4-docker/buildall: add $action value "start-only"
by Liu Yinsi 01 Mar '21

01 Mar '21

[usage] ./buildall start-only [how] before: ./buildall run-only: call build_depends(), check images exist or not, then call start_depends() after: ./buildall start-only: only call start_depends() [why] for some users locally deploy compass-ci, they will not build images, and some users can't install lockfile-create command, only run containers, no need to check images exist or not, so only call start_depends(). Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index f53bf4c..a933feb 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -67,7 +67,7 @@ tmpdir=$(mktemp -d) for dir in $CONTAINER_PATH/*/ do - build_depends $dir + [ "$action" != "start-only" ] && build_depends $dir start_depends $dir done -- 2.23.0

1 0

[PATCH compass-ci 1/2] container/defconfig.sh: accelerate es/logging-es startup time
by Liu Yinsi 01 Mar '21

01 Mar '21

before: es/logging-es startup time is 60s after: es/logging-es startup time is 10s [how] check es ready or not every 2s, if ok, just set index, rather than sleep 60s. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.sh | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/container/defconfig.sh b/container/defconfig.sh index c33a790..ee66ea5 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -28,9 +28,18 @@ docker_rm() docker rm -f $container } -set_es_indices() +check_es_ready() { - find $CCI_SRC/sbin/ -name "es-*-mapping.sh" -exec sh {} \; + port=$1 + for num in {1..30} + do + curl -s localhost:$port > /dev/null + if [ $? == "0" ]; then + break + else + sleep 2 + fi + done } push_image() -- 2.23.0

4 7

[PATCH compass-ci 2/2] sparrow/4-docker/buildall: add $action value "start-only"
by Liu Yinsi 01 Mar '21

01 Mar '21

[usage] ./buildall start-only [how] before: ./buildall run-only: call build_depends(), check images exist or not, then call start_depends() after: ./buildall start-only: only call start_depends() [why] for some users locally deploy compass-ci, they will not to build images, and some users can't install lockfile-create command, only run containers, no need to check images exist or not, so only call start_depends(). Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/4-docker/buildall | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index c9edbe9..6619600 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -68,7 +68,7 @@ tmpdir=$(mktemp -d) for dir in $CONTAINER_PATH/*/ do - build_depends $dir + [ "$action" != "start-only" ] && build_depends $dir start_depends $dir done -- 2.23.0

2 2