- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] fix: helper: delete_task "-h" option and bugs
by Tong Qunfeng 30 Sep '20

30 Sep '20

when input single task_id, should use "<< task_id" not "<< [task_id]" Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- user-client/helper/redis_op_gc/delete_task.rb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/user-client/helper/redis_op_gc/delete_task.rb b/user-client/helper/redis_op_gc/delete_task.rb index 643d2b3..6534f95 100755 --- a/user-client/helper/redis_op_gc/delete_task.rb +++ b/user-client/helper/redis_op_gc/delete_task.rb @@ -6,17 +6,21 @@ require_relative './force_delete' -if ARGV.empty? +if ARGV.empty? || ARGV.include?("-h") || ARGV.include?("--help") puts "Usage: #{__FILE__} task_id[|task_ids]" puts ' delete special task with [task_id]' - puts ' or delete special task form a [task_ids] file' + puts ' or delete special task from [task_ids] file' + puts "\n--- example ---" + puts " input: #{__FILE__} z9.11734 nolab.11342" + puts 'output: <none>' + puts ' # delete (z9.11734 nolab.11342) at taskqueue and scheduler' exit end task_ids = [] ARGV.each do |id| task_ids += IO.readlines(id) if File.exist?(id) - task_ids << [id] unless File.exist?(id) + task_ids << id unless File.exist?(id) end i = 0 -- 2.23.0

1 0

[PATCH v3 lkp-tests] fix: run-ipconfig: add ntpdat
by Wang Chenglong 30 Sep '20

30 Sep '20

initramfs start, need to synchronize time after get the ip address Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- bin/run-ipconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/run-ipconfig b/bin/run-ipconfig index 0f80feaf..ad2a9808 100755 --- a/bin/run-ipconfig +++ b/bin/run-ipconfig @@ -20,3 +20,4 @@ read_ip() [ -n "$IP" ] || read_ip configure_networking +ntpdate -u ntp.api.bz -- 2.23.0

2 2

[PATCH v4 compass-ci 6/7] container/mail-robot: conf/sources.list
by Luan Shengde 30 Sep '20

30 Sep '20

sources list file for the image install packages Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/conf/sources.list | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/mail-robot/conf/sources.list diff --git a/container/mail-robot/conf/sources.list b/container/mail-robot/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/mail-robot/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH v4 compass-ci 5/7] container/mail-robot: start file
by Luan Shengde 30 Sep '20

30 Sep '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/start | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..56c4259 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# frozen_string_literal: true + +require_relative '../defconfig.rb' +docker_rm 'mail-robot' + +JUMPER_IP = '192.168.132.2' +JUMPER_PORT = '29999' +SEND_MAIL_PORT = '11311' + +cmd = %W[ + docker run + --restart=always + --name=mail-robot + -d + -e JUMPER_IP=#{JUMPER_IP} + -e JUMPER_PORT=#{JUMPER_PORT} + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -v /c/upstream-repos:/upstream-repos:ro + -v /srv/cci/Maildir:/Maildir:rw + -v /c/compass-ci:/compass-ci:ro + mail-robot +] + +cmd += ['bash', '-c', 'umask 002 && ruby ./mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci] doc/code-spec: add 文档与易用性
by Wu Fengguang 30 Sep '20

30 Sep '20

From: Wu Fengguang <wufengguang(a)huawei.com> Signed-off-by: Wu Fengguang <wufengguang(a)huawei.com> --- doc/code-spec.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index e783a0c..9805182 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -40,3 +40,16 @@ here: 2) 描述你的改动 3) 拆分你的改动 8）回复评审意见 + +文档与易用性 +============ + +- 每个feature都应当有 + - 用户文档 + - 设计文档 + - 指示对应的代码在哪里(class/func); 或者如果有多处代码互相配合, 指示用什么关键字grep能把它们串起来看 + +- 每个有命令行参数的脚本, 都应当支持help output (-h|--help|参数不够时显示帮助信息) + - dependencies/assumptions + - input parameters + - input/output example -- 2.23.0

1 0

[PATCH compass-ci] feat: helper: snapshot of taskqueue
by Tong Qunfeng 30 Sep '20

30 Sep '20

Give a snapshot information about taskqueue (data at redis sorted set queue "queues/id2content"). It will generate a file named taskqueue_snap_yyyymmdd.md. snapshot content examples: |id---|add_time----------|current queue name| 14703, 2020-07-22 14:33, sched/vm-hi1620-2p8g-xxx/in_process 14831, 2020-07-22 17:51, sched/vm-hi1620-2p8g-xxx/in_process crystal.89630, 2020-09-23 11:19, sched/vm-hi1620-2p8g/in_process crystal.89667, 2020-09-23 11:20, sched/vm-hi1620-2p8g/in_process Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- .../helper/redis_op_gc/taskqueue_snap.rb | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100755 user-client/helper/redis_op_gc/taskqueue_snap.rb diff --git a/user-client/helper/redis_op_gc/taskqueue_snap.rb b/user-client/helper/redis_op_gc/taskqueue_snap.rb new file mode 100755 index 0000000..ebf32b5 --- /dev/null +++ b/user-client/helper/redis_op_gc/taskqueue_snap.rb @@ -0,0 +1,44 @@ +#!/usr/bin/env ruby + +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require_relative './basic_env' + +# redis hash key "queues/id2content" is used by taskqueue +cmd = "#{CMD_BASE} queues/id2content , hgetall" +result = `#{cmd}`.chomp +results = result.split("\n") + +task_info = [] + +# format time to "2020-09-29 14:23" +i = 0 +task_num = results.size / 2 +while i < task_num + task_id = results[i] + task_content = JSON.parse(results[i + 1]) + + time = task_content['add_time'] + task_content['add_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + time = task_content['move_time'] + task_content['move_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + task_info << { 'id' => task_id, 'value' => task_content } + + i += 2 + set_progress(i, task_num) +end + +# sort and put short information to file. +task_info.sort! { |a, b| a['value']['add_time'] <=> b['value']['add_time'] } +today = Time.now.strftime('%Y%m%d') +File.open("taskqueue_snap_#{today}.md", 'w') do |f| + task_info.each do |task| + f.write(task['id'] + ', ' + task['value']['add_time'] \ + + ', ' + task['value']['queue'] + "\n") + end +end -- 2.23.0

2 1

[PATCH compass-ci] modify the origin address for fluentd-base image
by Xiao Shenwei 30 Sep '20

30 Sep '20

error message: ERROR: libcurl-7.67.0-r0: package mentioned in index not found refer to: https://mirrors.huaweicloud.com/alpine Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/fluentd-base/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/fluentd-base/Dockerfile b/container/fluentd-base/Dockerfile index ef671c2..62f55fd 100644 --- a/container/fluentd-base/Dockerfile +++ b/container/fluentd-base/Dockerfile @@ -5,7 +5,7 @@ FROM alpine:3.11 MAINTAINER Wu Zhende <wuzhende666(a)163.com> -RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|https://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk update -- 2.23.0

3 5

[PATCH v2 compass-ci] sparrow: start the firewall in adapting openeuler
by Hu Xuejiao 30 Sep '20

30 Sep '20

When firewalld is inactive, return code of 'systemctl status firewalld' will be 3, and install-tiny has set -e, so this will cause this process exit. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/2-network/iptables | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/sparrow/2-network/iptables b/sparrow/2-network/iptables index 092c4ca..b4dc2d2 100755 --- a/sparrow/2-network/iptables +++ b/sparrow/2-network/iptables @@ -12,15 +12,18 @@ iptables -I FORWARD 1 -j ACCEPT iptables -t nat -A POSTROUTING -o "$PUB_IFACE" -s $BR0_SUBNET -j MASQUERADE iptables -t nat -A POSTROUTING -o $BR0_IFACE -d $BR0_SUBNET -j MASQUERADE -systemctl status firewalld | grep -q "running" && { +if command -v "firewall-cmd" > /dev/null; then + systemctl start firewalld && { + DOCKER0_IFACE=docker + DOCKER0_SUBNET=172.17.0.0/16 - DOCKER0_IFACE=docker0 - DOCKER0_SUBNET=172.17.0.0/16 + iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE + iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE - iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE - iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE - - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" -} + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" + } +else + echo "firewall.service could not be found" +fi -- 2.23.0

3 3

[PATCH compass-ci 2/4] sparrow: add domain name resolution for Docker images
by Hu Xuejiao 30 Sep '20

30 Sep '20

When executing ./crystal-shards/build appear error: http://mirrors.huaweicloud.com/alpine/v3.11/main: temporary error(try again later) Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/3-code/dev-env | 3 +++ sparrow/3-code/os/debian | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 sparrow/3-code/os/debian diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index ae267bd..80ee2cf 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -32,3 +32,6 @@ export REPO_SRC=/c/upstream-repos export PATH="$PATH:$CCI_SRC/sbin:$LKP_SRC/sbin:$LKP_SRC/bin" EOF +source /etc/os-release + +. $(dirname ${BASH_SOURCE[0]})/os/${ID} diff --git a/sparrow/3-code/os/debian b/sparrow/3-code/os/debian new file mode 100644 index 0000000..171b979 --- /dev/null +++ b/sparrow/3-code/os/debian @@ -0,0 +1,4 @@ +#!/bin/bash +cat >> /etc/default/docker <<EOF +DOCKER_OPTS="--dns 114.114.114.114" +EOF -- 2.23.0

2 1

[PATCH compass-ci 1/4] container: consider the situation that GITCACHE_HOST is undefined
by Hu Xuejiao 30 Sep '20

30 Sep '20

Error Msg: Step 5/12 : RUN [ -n "$GITCACCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig ---> Running in 52ffe9f8ecd1 The command '/bin/sh -c [ -n "$GITCACHE_HOST" ] && echo -e "[url\"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig' reton-zero code: 1 Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- container/crystal-shards/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/crystal-shards/Dockerfile b/container/crystal-shards/Dockerfile index 0c5713c..bdc5b66 100644 --- a/container/crystal-shards/Dockerfile +++ b/container/crystal-shards/Dockerfile @@ -7,7 +7,7 @@ MAINTAINER chief <taxcom(a)tom.com> ARG GITCACHE_HOST ARG GITCACHE_PORT -RUN [ -n "$GITCACHE_HOST" ] && echo -e "[url \"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig +RUN [ -z "$GITCACHE_HOST" ] || echo -e "[url \"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig copy shard.yml /usr/share/crystal/app/shard.yml -- 2.23.0

2 1