Compass-ci August 2021

compass-ci@openeuler.org

1 participants
2 discussions

Re: [PATCH compass-ci 2/2] modify url about srv-http
by Liu Shaofei 25 Aug '21

25 Aug '21

On Wed, Aug 25, 2021 at 04:55:29PM +0800, Lu Weitao wrote: >[Why] >compass-ci/container/srv-http may change the communication protocol: > http => https >it's depends on whether there are ssl certificates on the server >we need convert all 'http://compass-ci.api.openeuler.org.*' to >'https://compass-ci.api.openeuler.org.*' > >[How] >scope: gg http.*SRV_HTTP > >lib/mail_bisect_result.rb:44: prefix_srv_result = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_RESULT_PORT}" >lib/mail_bisect_result.rb:45: prefix_srv_git = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_GIT_PORT}" >sbin/download-rootfs:38: wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" >sbin/download-rootfs:48: wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/osimage/…" >sbin/download-rootfs:49: wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nfs…" >sparrow/0-package/os/openEuler:53: wget_rpm http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/d… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… >sparrow/0-package/os/openEuler:54: wget_rpm http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/d… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… > >Signed-off-by: Lu Weitao <luweitaobe(a)163.com> >--- > lib/constants.rb | 3 +++ > lib/mail_bisect_result.rb | 4 ++-- > sbin/download-rootfs | 6 +++--- > sparrow/0-package/os/openEuler | 4 ++-- > sparrow/local/compare-results.md | 2 +- > 5 files changed, 11 insertions(+), 8 deletions(-) > >diff --git a/lib/constants.rb b/lib/constants.rb >index d85ca5c3..38e1ec92 100644 >--- a/lib/constants.rb >+++ b/lib/constants.rb >@@ -70,6 +70,9 @@ SRV_HTTP_GIT_PORT ||= config['SRV_HTTP_GIT_PORT'] || ENV['SRV_HTTP_GIT_PORT'] || > > SRV_HTTP_DOMAIN ||= config['SRV_HTTP_DOMAIN'] || ENV['SRV_HTTP_DOMAIN'] || 'api.compass-ci.openeuler.org' > >+SRV_HTTP_PROTOCOL ||= >+ File.exist?('/etc/ssl/certs/web-backend.key') && File.exist?('/etc/ssl/certs/web-backend.crt') ? 'https' : 'http' >+ > ASSISTANT_HOST ||= config['ASSISTANT_HOST'] || ENV['ASSISTANT_HOST'] || '172.17.0.1' > ASSISTANT_PORT ||= config['ASSISTANT_PORT'] || ENV['ASSISTANT_PORT'] || 8101 > >diff --git a/lib/mail_bisect_result.rb b/lib/mail_bisect_result.rb >index 855fcdc9..cff3137b 100644 >--- a/lib/mail_bisect_result.rb >+++ b/lib/mail_bisect_result.rb >@@ -41,8 +41,8 @@ class MailBisectResult > > def compose_mail > subject = "[Compass-CI][#{(a)repo.split('/')[1]}] #{@commit_id[0..9]} #{(a)bisect_error[0].split("\n")[0]}" >- prefix_srv_result = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_RESULT_PORT}" >- prefix_srv_git = "http://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_GIT_PORT}" >+ prefix_srv_result = "#{SRV_HTTP_PROTOCOL}://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_RESULT_PORT}" >+ prefix_srv_git = "#{SRV_HTTP_PROTOCOL}://#{SRV_HTTP_DOMAIN}:#{SRV_HTTP_GIT_PORT}" > pkgbuild_repo_url = "we build project with this script:\n#{prefix_srv_git}/git/#{@pkgbuild_repo}/PKGBUILD\n" > first_bad_commit_job_url = "first bad commit result :\n#{prefix_srv_result}#{@first_bad_commit_result_root}/build-pkg\n" > >diff --git a/sbin/download-rootfs b/sbin/download-rootfs >index 0d0d1c7b..9f1d0070 100755 >--- a/sbin/download-rootfs >+++ b/sbin/download-rootfs >@@ -35,7 +35,7 @@ wget_download() > download_os_rootfs() > { > cd $rootfs_dir && { >- wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" >+ wget_download "https://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch…" > > gzip -dc ${os_version}.cgz | cpio -id > rm -rf ${os_version}.cgz >@@ -45,8 +45,8 @@ download_os_rootfs() > download_osimage() > { > cd $osimage_dir && { >- wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/osimage/…" >- wget_download "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nfs…" >+ wget_download "https://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/osimage…" >+ wget_download "https://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nf…" ^-----^-----^-------^--- need to change ${os_mount}/${os}/${os_version} ? Thanks, Liushaofei > } > } > >diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler >index 1c61e2c0..399e5e47 100755 >--- a/sparrow/0-package/os/openEuler >+++ b/sparrow/0-package/os/openEuler >@@ -50,8 +50,8 @@ public_network_ok && { > > # lib64lockfile1 from http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… > # lockfile-progs from http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… >- wget_rpm http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/d… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… >- wget_rpm http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/d… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… >+ wget_rpm https://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… >+ wget_rpm https://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT:-20008}/initrd/… || wget_rpm http://abf-downloads.openmandriva.org/rock/repository/$(arch)/main/release/… > > rpm -i --force lib64lockfile1-1.16-1-omv4002.$(arch).rpm lockfile-progs-0.1.18-5-omv4002.$(arch).rpm > rm -rf *.rpm >diff --git a/sparrow/local/compare-results.md b/sparrow/local/compare-results.md >index e555f32e..d82b0165 100644 >--- a/sparrow/local/compare-results.md >+++ b/sparrow/local/compare-results.md >@@ -6,7 +6,7 @@ > submit -m cci-depends.yaml cci-depends.benchmark=iperf > ``` > >-2. 编辑os-matrix-dc.yaml，指定使用centos：7和 [openEuler：20.03-pre](http://api.compass-ci.openeuler.org:20008/initrd/dockerimage/openeuler-pre.tar)（由于dockerhub没有openeuler20.03的镜像，可以下载该链接并使用docker load -i openeuler-pre.tar加载到本地），如需要使用其他os，参考如下文件格式修改即可。 >+2. 编辑os-matrix-dc.yaml，指定使用centos：7和 [openEuler：20.03-pre](https://api.compass-ci.openeuler.org:20008/initrd/dockerimage/openeuler-pre.tar)（由于dockerhub没有openeuler20.03的镜像，可以下载该链接并使用docker load -i openeuler-pre.tar加载到本地），如需要使用其他os，参考如下文件格式修改即可。 > ``` > cat > /c/lkp-tests/jobs/os-matrix-dc.yaml << EOF > os_arch: aarch64 >-- >2.23.0 >

1 0

Re: [PATCH compass-ci 1/2] container/srv-http: support https protocol
by Liu Shaofei 25 Aug '21

25 Aug '21

On Wed, Aug 25, 2021 at 04:55:28PM +0800, Lu Weitao wrote: >[Why] >Some clients(chrome|firefox bowser) can't access srv-http like: > http://api.compass-ci.openeuler.org/pub >because the url will be change as:(it's default behavior of bowser) > https://api.compass-ci.openeuler.org/pub > >so srv-http need support https access > >[How] >with start container/srv-http: > > check current server exists?: > /etc/ssl/certs/web-backend.crt > /etc/ssl/certs/web-backend.key > | | > v v > yes no > config nginx.conf like: config /nginx.conf like: > ... ... > listen $port ssl: listen $port; > ... > | > v > start nginx in container > >Signed-off-by: Lu Weitao <luweitaobe(a)163.com> >--- > container/srv-http/docker_run.sh | 9 +++++++++ > container/srv-http/root/sbin/entrypoint.sh | 13 ++++++++++++- > container/srv-http/start-cci | 2 ++ > container/srv-http/start-git | 2 ++ > container/srv-http/start-initrd | 2 ++ > container/srv-http/start-os | 2 ++ > container/srv-http/start-pub | 2 ++ > container/srv-http/start-result | 2 ++ > container/srv-http/start-rpm | 2 ++ > 9 files changed, 35 insertions(+), 1 deletion(-) > create mode 100644 container/srv-http/docker_run.sh > >diff --git a/container/srv-http/docker_run.sh b/container/srv-http/docker_run.sh >new file mode 100644 >index 00000000..fcd9083d >--- /dev/null >+++ b/container/srv-http/docker_run.sh >@@ -0,0 +1,9 @@ >+#!/bin/bash >+# SPDX-License-Identifier: MulanPSL-2.0+ >+# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. >+ >+mount_ssl(){ >+ if [ -f "/etc/ssl/certs/web-backend.key" ] && [ -f "/etc/ssl/certs/web-backend.crt" ]; then >+ echo "-v /etc/ssl/certs:/opt/cert" >+ fi defined var: ssl_path="/etc/ssl/certs/" , avoid long contents; >+} >diff --git a/container/srv-http/root/sbin/entrypoint.sh b/container/srv-http/root/sbin/entrypoint.sh >index ced6eda5..d0d354c2 100755 >--- a/container/srv-http/root/sbin/entrypoint.sh >+++ b/container/srv-http/root/sbin/entrypoint.sh >@@ -1,6 +1,17 @@ > #!/bin/sh > >-sed -i "s/listen 11300;/listen $LISTEN_PORT;/g" /etc/nginx/conf.d/default.conf >+ssl_conf="\ ssl_certificate /opt/cert/web-backend.crt;\n\ >+ ssl_certificate_key /opt/cert/web-backend.key;\n\ >+ ssl_session_timeout 5m;\n\ >+ ssl_ciphers BCDHE-RSA-AES128-GCM-SHA256:ECDHE:ECDH:AES:HIGH:!NULL:!aNULL:!MD5:!ADH:!RC4;\n\ >+ ssl_prefer_server_ciphers on;\n" >+ >+if [ -f "/opt/cert/web-backend.key" ] && [ -f "/opt/cert/web-backend.crt" ]; then >+ sed -i "s/listen 11300;/listen $LISTEN_PORT ssl;/g" /etc/nginx/conf.d/default.conf >+ sed -i "/server_name/a $ssl_conf" /etc/nginx/conf.d/default.conf >+else >+ sed -i "s/listen 11300;/listen $LISTEN_PORT;/g" /etc/nginx/conf.d/default.conf >+fi perhaps modify like this: listen="$LISTEN_PORT" if [ -f "/opt/cert/web-backend.key" ] && [ -f "/opt/cert/web-backend.crt" ]; then listen="$listen ssl" sed -i "/server_name/a $ssl_conf" /etc/nginx/conf.d/default.conf fi sed -i "s/listen 11300;/listen $listen;/g" /etc/nginx/conf.d/default.conf Thanks, Liushaofei > nginx > /usr/sbin/php-fpm7 >diff --git a/container/srv-http/start-cci b/container/srv-http/start-cci >index aa539d20..1f160f44 100755 >--- a/container/srv-http/start-cci >+++ b/container/srv-http/start-cci >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-cci > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20011:20011 > -v /srv/cci/libvirt-xml:/srv/cci/libvirt-xml:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-git b/container/srv-http/start-git >index 0b0ff477..5851af6f 100755 >--- a/container/srv-http/start-git >+++ b/container/srv-http/start-git >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-git > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20010:20010 > -v /srv/git/archlinux:/srv/git/archlinux:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-initrd b/container/srv-http/start-initrd >index 41958da2..c8e7eded 100755 >--- a/container/srv-http/start-initrd >+++ b/container/srv-http/start-initrd >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-initrd > > cmd=( >@@ -17,6 +18,7 @@ cmd=( > -v /srv/initrd/osimage:/srv/initrd/osimage:ro > -v /srv/initrd/deps:/srv/initrd/deps:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-os b/container/srv-http/start-os >index 5b63902e..c2012397 100755 >--- a/container/srv-http/start-os >+++ b/container/srv-http/start-os >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-os > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20009:20009 > -v /srv/os:/srv/os:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-pub b/container/srv-http/start-pub >index 6dfd5a44..2224737e 100755 >--- a/container/srv-http/start-pub >+++ b/container/srv-http/start-pub >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-pub > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20006:20006 > -v /srv/pub:/srv/pub:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-result b/container/srv-http/start-result >index 32bdf7df..c9e0eac2 100755 >--- a/container/srv-http/start-result >+++ b/container/srv-http/start-result >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-result > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20007:20007 > -v /srv/result:/srv/result:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >diff --git a/container/srv-http/start-rpm b/container/srv-http/start-rpm >index 2d38298c..62fb1385 100755 >--- a/container/srv-http/start-rpm >+++ b/container/srv-http/start-rpm >@@ -3,6 +3,7 @@ > # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. > > . $CCI_SRC/container/defconfig.sh >+. ./docker_run.sh > docker_rm srv-http-rpm > > cmd=( >@@ -13,6 +14,7 @@ cmd=( > -p 20012:20012 > -v /srv/rpm/pub:/srv/rpm/pub:ro > -v /etc/localtime:/etc/localtime:ro >+ $(mount_ssl) > -d > srv-http > ) >-- >2.23.0 >

1 0

2024

2023

2022

2021

2020

Compass-ci August 2021