January 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] lib/job.cr: use an independent interface for submit
by Wu Zhende 22 Jan '21

22 Jan '21

When we use Job.new, we need to do some extra processing only when we submit the job. Use interface "initialized?" to determine whether to submit a job. However, there is a vulnerability. If a job with all fields initialized is submitted, the job can be successfully submitted without checking the account. Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 29 ++++++++++------------------- src/scheduler/submit_job.cr | 1 + 2 files changed, 11 insertions(+), 19 deletions(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 9b52d88..964c520 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -55,25 +55,6 @@ class Job @es = Elasticsearch::Client.new @account_info = Hash(String, JSON::Any).new @log = JSONLogger.new - - # init job with "-1", or use the original job_content["id"] - id = "-1" if "#{id}" == "" - - if initialized? - if @hash["id"] == "#{id}" - return unless @hash.has_key?("my_uuid") || @hash.has_key?("my_token") - - check_account_info() - set_sshr_info() - return - end - end - - @hash["id"] = JSON::Any.new("#{id}") - - check_required_keys() - check_account_info() - set_defaults() end METHOD_KEYS = %w( @@ -155,6 +136,16 @@ class Job end end + def submit(id = nil) + # init job with "-1", or use the original job_content["id"] + id = "-1" if "#{id}" == "" + @hash["id"] = JSON::Any.new("#{id}") + + check_required_keys() + check_account_info() + set_defaults() + end + private def set_defaults append_init_field() set_docker_os() diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index bcb265f..1837cb1 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -7,6 +7,7 @@ class Sched job_content = JSON.parse(body) job = Job.new(job_content, job_content["id"]?) + job.submit(job_content["id"]?) job["commit_date"] = get_commit_date(job) cluster_file = job["cluster"] -- 2.23.0

1 0

[PATCH v2 compass-ci] add a comment, make it easier to understand
by Zhang Dewan 21 Jan '21

21 Jan '21

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- providers/qemu/kvm.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index b4e87fc..d445248 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -89,6 +89,7 @@ set_initrd() set_bios() { bios=/usr/share/qemu-efi-aarch64/QEMU_EFI.fd + # when arch='x86_64', use the following file: [ -f "$bios" ] || bios=/usr/share/ovmf/OVMF.fd } -- 2.23.0

1 0

[PATCH lkp-tests] expose/report error on install_proxy()
by Wu Fengguang 21 Jan '21

21 Jan '21

Carefully preserve error code from external commands, and return error code to function callers. Check/report error, and fail it early. Signed-off-by: Wu Fengguang <wfg(a)mail.ustc.edu.cn> --- bin/lkp-setup-rootfs | 28 +++++++++++++--------------- lib/http.sh | 8 ++++++++ 2 files changed, 21 insertions(+), 15 deletions(-) diff --git a/bin/lkp-setup-rootfs b/bin/lkp-setup-rootfs index cca1b53be..cf48c8f6b 100755 --- a/bin/lkp-setup-rootfs +++ b/bin/lkp-setup-rootfs @@ -103,8 +103,6 @@ wget_install_cgz() install_proxy_yum() { - [ -d "/etc/yum/" ] || return - echo "proxy=http://$SQUID_HOST:$SQUID_PORT" >> /etc/yum.conf if ls /etc/yum.repos.d/*.repo >/dev/null 2>&1; then sed -i 's/https:/http:/g' /etc/yum.repos.d/*.repo @@ -115,8 +113,6 @@ install_proxy_yum() install_proxy_apt() { - [ -d "/etc/apt/" ] || return - cat << EOF > /etc/apt/apt.conf.d/proxy Acquire::http::proxy "http://$SQUID_HOST:$SQUID_PORT/"; Acquire::ftp::proxy "ftp://$SQUID_HOST:$SQUID_PORT/"; @@ -133,8 +129,8 @@ EOF install_docker_proxy() { - [ -n "$DOCKER_PROXY_HOST" ] || return - [ -n "$DOCKER_PROXY_PORT" ] || return + [ -n "$DOCKER_PROXY_HOST" ] || return 0 + [ -n "$DOCKER_PROXY_PORT" ] || return 0 mkdir -p /etc/systemd/system/docker.service.d cat << EOF > /etc/systemd/system/docker.service.d/http-proxy.conf @@ -156,30 +152,32 @@ EOF modify_gem_source() { echo ':sources: ["http://rubygems.org"]' >> /root/.gemrc - - return 0 } install_goproxy() { - mkdir -p /root/.config/go/ + mkdir -p /root/.config/go/ || return echo "GOPROXY=https://goproxy.io,direct" >> /root/.config/go/env - - return 0 } install_proxy() { - modify_gem_source + modify_gem_source || return install_docker_proxy - install_goproxy + install_goproxy || return [ -z "$SQUID_HOST" ] && return [ -z "$SQUID_PORT" ] && return - install_proxy_apt || install_proxy_yum + if [ -d "/etc/yum/" ]; then + install_proxy_yum + elif [ -d "/etc/apt/" ]; then + install_proxy_apt + else + : + fi } # Save the result to @@ -251,7 +249,7 @@ chmod_root_ssh() chmod 600 /root/.ssh/id_rsa } -install_proxy +install_proxy || exit_with_job_state 'error:proxy' chmod_root_ssh while true; do diff --git a/lib/http.sh b/lib/http.sh index a0cb33fc2..523c86333 100755 --- a/lib/http.sh +++ b/lib/http.sh @@ -60,8 +60,16 @@ jobfile_append_var() http_get_cgi "cgi-bin/lkp-jobfile-append-var?$query_str" } +exit_with_job_state() +{ + local exit_code=$? + set_job_state "$1" + exit $exit_code +} + set_job_state() { + [ -n "$1" ] || return jobfile_append_var "job_state=$1" } -- 2.23.0

1 0

help star
by Wu Fengguang 21 Jan '21

21 Jan '21

Xueliang is our help star again, thanks Xueliang! 23.2 Wu Fengguang 15.9 Cao Xueliang 13.7 Yu Chuan 13.4 Wei Jihui 12.3 Xiao Shenwei 10.9 Luan Shengde 8.4 Liu Yinsi 8.4 Liu Shaofei 7.9 Lin Jiaxin 7.6 Cui Yili 7.5 Wu Zhende 6.9 Zhang Dewan 6.8 Lu Weitao 4.7 Zhang Yale 4.4 Ren Wen 4.2 Bai Jing 3.9 Du Kaitian 3.9 Xu Xijian 3.8 Li Yuanchao 3.4 Wang Yong 3.2 Hu Xuejiao 2.2 Lu Kaiyi 2.1 Li Ping 1.3 Wang Chenglong 0.0 Tong Qunfeng 0.0 Wang Shuai 0.0 Cui Xucui

1 0

review star
by Wu Fengguang 21 Jan '21

21 Jan '21

Jihui is our review star again, thanks Jihui! Review email records in the last 7 days total review paragraph point people 61 48 122 39 Wu Fengguang 96 31 48 36 Wei Jihui 93 19 24 33 Cao Xueliang 35 18 33 17 Xiao Shenwei 38 19 52 15 Luan Shengde 58 13 16 6 Ren Wen 29 5 9 5 Lin Jiaxin 41 8 14 5 Liu Yinsi 23 2 4 4 Yu Chuan 12 5 6 4 Xu Xijian 13 7 12 4 Liu Shaofei 3 3 4 4 Wang Yong 33 3 9 3 Zhang Yale 45 5 10 3 Wu Zhende 35 2 7 2 Wang Chenglong 39 2 4 2 Lu Kaiyi 34 4 3 2 Zhang Dewan 15 2 2 1 Hu Xuejiao 10 2 2 1 Bai Jing 9 1 1 1 Li Yuanchao 28 5 4 1 Cui Yili 11 0 0 0 Li Ping 8 4 9 0 Lu Weitao

1 0

git count
by Wu Fengguang 21 Jan '21

21 Jan '21

Xueliang is our code star, thanks Xueliang! 9 Cao Xueliang 8 Wu Zhende 7 Liu Yinsi 6 Wei Jihui 6 Ren Wen 4 Xu Xijian 4 Liu Shaofei 4 Zhang Yale 3 Yu Chuan 3 Lin Jiaxin 2 Bai Jing 2 Hu Xuejiao 2 Lu Weitao 2 Xiao Shenwei 2 Wang Chenglong 2 Li Yuanchao 1 Wu Fengguang 1 Wang Yong 1 Luan Shengde 1 Li Ping 1 Lu Kaiyi 0 Tong Qunfeng 0 Du Kaitian 0 Sun Yukui 0 Zhang Yu 0 Cui Yili 0 Zhang Dewan 0 Shi Zhichao

1 0

Re: [PATCH v4 lkp-tests] add option '-l' for submit
by Wu Fengguang 21 Jan '21

21 Jan '21

On Tue, Jan 05, 2021 at 10:03:22AM +0800, Li Ping wrote: >[Usage]: >submit -m -l job.yaml > ==> lftp -c mirror http://SRV_HTTP_HOST:SRV_HTTP_PORT/RESULT_ROOT -l = lftp? "download results" can be more meaningful feature name. lftp is just one of the clients. Thanks, Fengguang

2 1

[PATCH lkp-tests] daptation/debian: add pgbench dependency
by Hu Xuejiao 21 Jan '21

21 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- distro/adaptation/debian | 1 + 1 file changed, 1 insertion(+) diff --git a/distro/adaptation/debian b/distro/adaptation/debian index 90260f1d..15706cf6 100644 --- a/distro/adaptation/debian +++ b/distro/adaptation/debian @@ -21,3 +21,4 @@ libperl5: libperl5.24 perl-modules-5: perl-modules-5.24 libgcc-dev: libgcc-6-dev openjdk: openjdk-8-jdk +postgresql-server-dev-all: postgresql -- 2.23.0

1 0

Re: [PATCH v4 lkp-tests] add option '-l' for submit
by Wu Fengguang 21 Jan '21

21 Jan '21

TO ALL >+ srv_http_host = job['SRV_HTTP_HOST'] || '124.160.11.58' >+ srv_http_port = job['SRV_HTTP_PORT'] || '11300' DON't use IP address. Use domain name instead. Thanks, Fengguang

2 1

[PATCH lkp-tests] adaptation/debian: add pgbench dependency
by Hu Xuejiao 21 Jan '21

21 Jan '21

Signed-off-by: Hu XueJiao <1034502035(a)qq.com> --- distro/adaptation/centos | 1 + distro/adaptation/debian | 1 + 2 files changed, 2 insertions(+) diff --git a/distro/adaptation/centos b/distro/adaptation/centos index 2cfe3877..8fbae666 100644 --- a/distro/adaptation/centos +++ b/distro/adaptation/centos @@ -335,3 +335,4 @@ liboath-dev: liboath-devel lsb-release: redhat-lsb linux-base: openssh-client: openssh-clients +postgresql-server: postgresql diff --git a/distro/adaptation/debian b/distro/adaptation/debian index 90260f1d..15706cf6 100644 --- a/distro/adaptation/debian +++ b/distro/adaptation/debian @@ -21,3 +21,4 @@ libperl5: libperl5.24 perl-modules-5: perl-modules-5.24 libgcc-dev: libgcc-6-dev openjdk: openjdk-8-jdk +postgresql-server-dev-all: postgresql -- 2.23.0

1 0