- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] feat: helper: snapshot of taskqueue
by Tong Qunfeng 30 Sep '20

30 Sep '20

Give a snapshot information about taskqueue (data at redis sorted set queue "queues/id2content"). It will generate a file named taskqueue_snap_yyyymmdd.md. snapshot content examples: |id---|add_time----------|current queue name| 14703, 2020-07-22 14:33, sched/vm-hi1620-2p8g-xxx/in_process 14831, 2020-07-22 17:51, sched/vm-hi1620-2p8g-xxx/in_process crystal.89630, 2020-09-23 11:19, sched/vm-hi1620-2p8g/in_process crystal.89667, 2020-09-23 11:20, sched/vm-hi1620-2p8g/in_process Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- .../helper/redis_op_gc/taskqueue_snap.rb | 56 +++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100755 user-client/helper/redis_op_gc/taskqueue_snap.rb diff --git a/user-client/helper/redis_op_gc/taskqueue_snap.rb b/user-client/helper/redis_op_gc/taskqueue_snap.rb new file mode 100755 index 0000000..f0a7034 --- /dev/null +++ b/user-client/helper/redis_op_gc/taskqueue_snap.rb @@ -0,0 +1,56 @@ +#!/usr/bin/env ruby + +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require_relative './basic_env' + +if !ARGV.empty? + puts 'Usage:' + puts "input: #{__FILE__}" + puts 'output: taskqueue_snap_yyyymmdd.md' + puts ' # id, first add time, current queue' + puts ' # 14703, 2020-07-22 14:33, sched/vm-hi1620-2p8g-xxx/in_process' + puts ' # 14831, 2020-07-22 17:51, sched/vm-hi1620-2p8g-xxx/in_process' + puts ' # crystal.89630, 2020-09-23 11:19, sched/vm-hi1620-2p8g/in_process' + puts ' # crystal.89667, 2020-09-23 11:20, sched/vm-hi1620-2p8g/in_process' + exit +end + +# redis hash key "queues/id2content" is used by taskqueue +cmd = "#{CMD_BASE} queues/id2content , hgetall" +result = `#{cmd}`.chomp +results = result.split("\n") + +task_info = [] + +# format time to "2020-09-29 14:23" +i = 0 +task_num = results.size / 2 +while i < task_num + task_id = results[i] + task_content = JSON.parse(results[i + 1]) + + time = task_content['add_time'] + task_content['add_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + time = task_content['move_time'] + task_content['move_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + task_info << { 'id' => task_id, 'value' => task_content } + + i += 2 + set_progress(i, task_num) +end + +# sort and put short information to file. +task_info.sort! { |a, b| a['value']['add_time'] <=> b['value']['add_time'] } +today = Time.now.strftime('%Y%m%d') +File.open("taskqueue_snap_#{today}.md", 'w') do |f| + task_info.each do |task| + f.write(task['id'] + ', ' + task['value']['add_time'] \ + + ', ' + task['value']['queue'] + "\n") + end +end -- 2.23.0

2 2

[PATCH compass-ci 3/4] sparrow: start the firewall in adapting openeuler
by Hu Xuejiao 30 Sep '20

30 Sep '20

When firewalld is inactive, return code of 'systemctl status firewalld' will be 3, and install-tiny has set -e, so this will cause this process exit. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/2-network/iptables | 26 +++++++++++++++++--------- 1 file changed, 17 insertions(+), 9 deletions(-) diff --git a/sparrow/2-network/iptables b/sparrow/2-network/iptables index 092c4ca..831c1a3 100755 --- a/sparrow/2-network/iptables +++ b/sparrow/2-network/iptables @@ -12,15 +12,23 @@ iptables -I FORWARD 1 -j ACCEPT iptables -t nat -A POSTROUTING -o "$PUB_IFACE" -s $BR0_SUBNET -j MASQUERADE iptables -t nat -A POSTROUTING -o $BR0_IFACE -d $BR0_SUBNET -j MASQUERADE -systemctl status firewalld | grep -q "running" && { +command -v firewall-cmd > /dev/null || { + echo "firewalld.service could not be found" + exit 0 +} + +systemctl start firewalld +[ "$(systemctl is-active firewalld)" == "active" ] || { + echo "start firewalld.service failed" + exit 0 +} - DOCKER0_IFACE=docker0 - DOCKER0_SUBNET=172.17.0.0/16 +DOCKER0_IFACE=docker0 +DOCKER0_SUBNET=172.17.0.0/16 - iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE - iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE +iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE +iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" -} +firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" +firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" +firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" -- 2.23.0

3 2

[PATCH lab-z9] servers add disks, hosts file should update
by Zhang Dewan 30 Sep '20

30 Sep '20

Signed-off-by: Zhang Dewan <513619464(a)qq.com> --- hosts/taishan200-2280-2s48p-256g--a11 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a24 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a25 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a66 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a67 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a68 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a69 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a70 | 19 ------------------- hosts/taishan200-2280-2s48p-256g--a71 | 13 ------------- hosts/taishan200-2280-2s64p-256g--a10 | 20 -------------------- hosts/taishan200-2280-2s64p-256g--a3 | 25 ------------------------- hosts/taishan200-2280-2s64p-256g--a4 | 25 ------------------------- hosts/taishan200-2280-2s64p-256g--a8 | 16 ---------------- hosts/taishan200-2280-2s64p-256g--a9 | 17 ----------------- 14 files changed, 268 deletions(-) delete mode 100644 hosts/taishan200-2280-2s48p-256g--a11 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a24 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a25 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a66 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a67 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a68 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a69 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a70 delete mode 100644 hosts/taishan200-2280-2s48p-256g--a71 delete mode 100644 hosts/taishan200-2280-2s64p-256g--a10 delete mode 100644 hosts/taishan200-2280-2s64p-256g--a3 delete mode 100644 hosts/taishan200-2280-2s64p-256g--a4 delete mode 100644 hosts/taishan200-2280-2s64p-256g--a8 delete mode 100644 hosts/taishan200-2280-2s64p-256g--a9 diff --git a/hosts/taishan200-2280-2s48p-256g--a11 b/hosts/taishan200-2280-2s48p-256g--a11 deleted file mode 100644 index 9c0224e..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a11 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b8891431-part1 - - /dev/disk/by-id/scsi-350000399b8890e09 - - /dev/disk/by-id/scsi-350000399b8890cd1 - - /dev/disk/by-id/scsi-350000399b8727929 -mac_addr: - - cc:05:77:ab:97:62 - - cc:05:77:ab:97:63 - - cc:05:77:ab:97:64 - - cc:05:77:ab:97:65 - - cc:05:77:ab:97:66 - - cc:05:77:ab:97:67 - - cc:05:77:ab:97:68 - - cc:05:77:ab:97:69 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a24 b/hosts/taishan200-2280-2s48p-256g--a24 deleted file mode 100644 index 7dda921..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a24 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b8917d59 - - /dev/disk/by-id/scsi-350000399b891790d - - /dev/disk/by-id/scsi-350000399b89185d9 - - /dev/disk/by-id/scsi-350000399b8917ef1 -mac_addr: - - 44:67:47:c9:d8:f4 - - 44:67:47:c9:d8:f5 - - 44:67:47:c9:d8:f6 - - 44:67:47:c9:d8:f7 - - 44:67:47:c9:d8:f8 - - 44:67:47:c9:d8:f9 - - 44:67:47:c9:d8:fa - - 44:67:47:c9:d8:fb -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a25 b/hosts/taishan200-2280-2s48p-256g--a25 deleted file mode 100644 index 5d075b9..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a25 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b88b4499 - - /dev/disk/by-id/scsi-350000399b88b4455 - - /dev/disk/by-id/scsi-350000399b88b442d - - /dev/disk/by-id/scsi-350000399b88b44f9 -mac_addr: - - 44:67:47:c9:ea:08 - - 44:67:47:c9:ea:09 - - 44:67:47:c9:ea:0a - - 44:67:47:c9:ea:0b - - 44:67:47:c9:ea:0c - - 44:67:47:c9:ea:0d - - 44:67:47:c9:ea:0e - - 44:67:47:c9:ea:0f -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a66 b/hosts/taishan200-2280-2s48p-256g--a66 deleted file mode 100644 index 88ef791..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a66 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-35000c5009a671773 - - /dev/disk/by-id/scsi-35000c5009a690c53 - - /dev/disk/by-id/scsi-35000c5009a6713fb - - /dev/disk/by-id/scsi-35000c5009a65963f -mac_addr: - - 5c:e8:83:31:c8:8e - - 5c:e8:83:31:c8:8f - - 5c:e8:83:31:c8:90 - - 5c:e8:83:31:c8:91 - - 5c:e8:83:31:c8:92 - - 5c:e8:83:31:c8:93 - - 5c:e8:83:31:c8:94 - - 5c:e8:83:31:c8:95 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a67 b/hosts/taishan200-2280-2s48p-256g--a67 deleted file mode 100644 index 97ea498..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a67 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b88aed21 - - /dev/disk/by-id/scsi-350000399b88af325 - - /dev/disk/by-id/scsi-350000399b88af699 - - /dev/disk/by-id/scsi-350000399b88af421 -mac_addr: - - 44:67:47:c9:e2:2c - - 44:67:47:c9:e2:2d - - 44:67:47:c9:e2:2e - - 44:67:47:c9:e2:2f - - 44:67:47:c9:e2:30 - - 44:67:47:c9:e2:31 - - 44:67:47:c9:e2:32 - - 44:67:47:c9:e2:33 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a68 b/hosts/taishan200-2280-2s48p-256g--a68 deleted file mode 100644 index f3d50f4..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a68 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b8909f81 - - /dev/disk/by-id/scsi-350000399b8909f79 - - /dev/disk/by-id/scsi-350000399b8909605 - - /dev/disk/by-id/scsi-350000399b8909b85 -mac_addr: - - 84:46:fe:89:b6:e6 - - 84:46:fe:89:b6:e7 - - 84:46:fe:89:b6:e8 - - 84:46:fe:89:b6:e9 - - 84:46:fe:89:b6:ea - - 84:46:fe:89:b6:eb - - 84:46:fe:89:b6:ec - - 84:46:fe:89:b6:ed -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a69 b/hosts/taishan200-2280-2s48p-256g--a69 deleted file mode 100644 index 9e5734e..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a69 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b88be92d - - /dev/disk/by-id/scsi-350000399b88befc5 - - /dev/disk/by-id/scsi-350000399b88be701 - - /dev/disk/by-id/scsi-350000399b88bea5d -mac_addr: - - 44:67:47:c9:d9:4e - - 44:67:47:c9:d9:4f - - 44:67:47:c9:d9:50 - - 44:67:47:c9:d9:51 - - 44:67:47:c9:d9:52 - - 44:67:47:c9:d9:53 - - 44:67:47:c9:d9:54 - - 44:67:47:c9:d9:55 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a70 b/hosts/taishan200-2280-2s48p-256g--a70 deleted file mode 100644 index 0dd8208..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a70 +++ /dev/null @@ -1,19 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -nr_hdd_partitions: 4 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399b890931d - - /dev/disk/by-id/scsi-350000399b8909d79 - - /dev/disk/by-id/scsi-350000399b8909ac5 - - /dev/disk/by-id/scsi-350000399b8909c99 -mac_addr: - - 84:46:fe:89:b7:22 - - 84:46:fe:89:b7:23 - - 84:46:fe:89:b7:24 - - 84:46:fe:89:b7:25 - - 84:46:fe:89:b7:26 - - 84:46:fe:89:b7:27 - - 84:46:fe:89:b7:28 - - 84:46:fe:89:b7:29 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s48p-256g--a71 b/hosts/taishan200-2280-2s48p-256g--a71 deleted file mode 100644 index dde6205..0000000 --- a/hosts/taishan200-2280-2s48p-256g--a71 +++ /dev/null @@ -1,13 +0,0 @@ -nr_node: 4 -nr_cpu: 96 -memory: 256G -mac_addr: - - 44:67:47:d7:6d:e6 - - 44:67:47:d7:6d:e7 - - 44:67:47:d7:6d:e8 - - 44:67:47:d7:6d:e9 - - 44:67:47:d7:6d:ea - - 44:67:47:d7:6d:eb - - 44:67:47:d7:6d:ec - - 44:67:47:d7:6d:ed -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a10 b/hosts/taishan200-2280-2s64p-256g--a10 deleted file mode 100644 index 2784a67..0000000 --- a/hosts/taishan200-2280-2s64p-256g--a10 +++ /dev/null @@ -1,20 +0,0 @@ -nr_node: 4 -nr_cpu: 128 -memory: 256G -nr_ssd_partitions: 1 -ssd_partitions: - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB44333 -mac_addr: - - 44:67:47:86:2c:95 - - 44:67:47:86:2c:96 - - 44:67:47:86:2c:97 - - 44:67:47:86:2c:98 - - ac:b3:b5:1e:d6:8b - - ac:b3:b5:1e:d6:8c - - ac:b3:b5:1e:d6:8d - - ac:b3:b5:1e:d6:8e - - ac:b3:b5:1e:d5:4b - - ac:b3:b5:1e:d5:4c - - ac:b3:b5:1e:d5:4d - - ac:b3:b5:1e:d5:4e -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a3 b/hosts/taishan200-2280-2s64p-256g--a3 deleted file mode 100644 index 4bfa8ed..0000000 --- a/hosts/taishan200-2280-2s64p-256g--a3 +++ /dev/null @@ -1,25 +0,0 @@ -nr_node: 4 -nr_cpu: 128 -memory: 256G -nr_hdd_partitions: 1 -nr_ssd_partitions: 3 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399981347d5 -ssd_partitions: - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part1 - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part2 - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43895-part3 -mac_addr: - - 68:4a:ae:ca:e8:ca - - 68:4a:ae:ca:e8:cb - - 68:4a:ae:ca:e8:cc - - 68:4a:ae:ca:e8:cd - - ac:b3:b5:1f:74:de - - ac:b3:b5:1f:74:df - - ac:b3:b5:1f:74:e0 - - ac:b3:b5:1f:74:e1 - - ac:b3:b5:1f:74:ee - - ac:b3:b5:1f:74:ef - - ac:b3:b5:1f:74:f0 - - ac:b3:b5:1f:74:f1 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a4 b/hosts/taishan200-2280-2s64p-256g--a4 deleted file mode 100644 index e0062d9..0000000 --- a/hosts/taishan200-2280-2s64p-256g--a4 +++ /dev/null @@ -1,25 +0,0 @@ -nr_node: 4 -nr_cpu: 128 -memory: 256G -nr_hdd_partitions: 1 -nr_ssd_partitions: 3 -hdd_partitions: - - /dev/disk/by-id/scsi-350000399c802cfb9 -ssd_partitions: - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part1 - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part2 - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43892-part3 -mac_addr: - - 68:4a:ae:ca:e8:a2 - - 68:4a:ae:ca:e8:a3 - - 68:4a:ae:ca:e8:a4 - - 68:4a:ae:ca:e8:a5 - - ac:b3:b5:1f:74:6e - - ac:b3:b5:1f:74:6f - - ac:b3:b5:1f:74:70 - - ac:b3:b5:1f:74:71 - - ac:b3:b5:1f:7d:4e - - ac:b3:b5:1f:7d:4f - - ac:b3:b5:1f:7d:50 - - ac:b3:b5:1f:7d:51 -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a8 b/hosts/taishan200-2280-2s64p-256g--a8 deleted file mode 100644 index 9386034..0000000 --- a/hosts/taishan200-2280-2s64p-256g--a8 +++ /dev/null @@ -1,16 +0,0 @@ -nr_node: 4 -nr_cpu: 128 -memory: 256G -nr_ssd_partitions: 1 -ssd_partitions: - - /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA2MB43325 -mac_addr: - - 84:46:fe:26:12:f5 - - 84:46:fe:26:12:f6 - - 84:46:fe:26:12:f7 - - 84:46:fe:26:12:f8 - - ac:b3:b5:1e:b6:7b - - ac:b3:b5:1e:b6:7c - - ac:b3:b5:1e:b6:7d - - ac:b3:b5:1e:b6:7e -model_name: Kunpeng-920 diff --git a/hosts/taishan200-2280-2s64p-256g--a9 b/hosts/taishan200-2280-2s64p-256g--a9 deleted file mode 100644 index 19f4d75..0000000 --- a/hosts/taishan200-2280-2s64p-256g--a9 +++ /dev/null @@ -1,17 +0,0 @@ -nr_node: 4 -nr_cpu: 128 -memory: 128G -nr_hdd_partitions: 2 -hdd_partitions: - - /dev/disk/by-id/scsi-35000c500bd6a1347 - - /dev/disk/by-id/scsi-35000c500bd62acb3 -mac_addr: - - cc:05:77:fe:d4:0e - - cc:05:77:fe:d4:0f - - cc:05:77:fe:d4:10 - - cc:05:77:fe:d4:11 - - 44:a1:91:35:1a:70 - - 44:a1:91:35:1a:71 - - 44:a1:91:35:1a:72 - - 44:a1:91:35:1a:73 -model_name: Kunpeng-920 -- 2.23.0

1 0

[PATCH compass-ci] fix: helper: delete_task "-h" option and bugs
by Tong Qunfeng 30 Sep '20

30 Sep '20

when input single task_id, should use "<< task_id" not "<< [task_id]" Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- user-client/helper/redis_op_gc/delete_task.rb | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/user-client/helper/redis_op_gc/delete_task.rb b/user-client/helper/redis_op_gc/delete_task.rb index 643d2b3..6534f95 100755 --- a/user-client/helper/redis_op_gc/delete_task.rb +++ b/user-client/helper/redis_op_gc/delete_task.rb @@ -6,17 +6,21 @@ require_relative './force_delete' -if ARGV.empty? +if ARGV.empty? || ARGV.include?("-h") || ARGV.include?("--help") puts "Usage: #{__FILE__} task_id[|task_ids]" puts ' delete special task with [task_id]' - puts ' or delete special task form a [task_ids] file' + puts ' or delete special task from [task_ids] file' + puts "\n--- example ---" + puts " input: #{__FILE__} z9.11734 nolab.11342" + puts 'output: <none>' + puts ' # delete (z9.11734 nolab.11342) at taskqueue and scheduler' exit end task_ids = [] ARGV.each do |id| task_ids += IO.readlines(id) if File.exist?(id) - task_ids << [id] unless File.exist?(id) + task_ids << id unless File.exist?(id) end i = 0 -- 2.23.0

1 0

[PATCH v3 lkp-tests] fix: run-ipconfig: add ntpdat
by Wang Chenglong 30 Sep '20

30 Sep '20

initramfs start, need to synchronize time after get the ip address Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- bin/run-ipconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/run-ipconfig b/bin/run-ipconfig index 0f80feaf..ad2a9808 100755 --- a/bin/run-ipconfig +++ b/bin/run-ipconfig @@ -20,3 +20,4 @@ read_ip() [ -n "$IP" ] || read_ip configure_networking +ntpdate -u ntp.api.bz -- 2.23.0

2 2

[PATCH v4 compass-ci 6/7] container/mail-robot: conf/sources.list
by Luan Shengde 30 Sep '20

30 Sep '20

sources list file for the image install packages Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/conf/sources.list | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 container/mail-robot/conf/sources.list diff --git a/container/mail-robot/conf/sources.list b/container/mail-robot/conf/sources.list new file mode 100644 index 0000000..8179e01 --- /dev/null +++ b/container/mail-robot/conf/sources.list @@ -0,0 +1,9 @@ +deb [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib + +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-updates main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian/ buster-backports main non-free contrib +deb-src [trusted=yes] http://mirrors.163.com/debian-security/ buster/updates main non-free contrib -- 2.23.0

1 0

[PATCH v4 compass-ci 5/7] container/mail-robot: start file
by Luan Shengde 30 Sep '20

30 Sep '20

Signed-off-by: Luan Shengde <luanshengde2(a)huawei.com> --- container/mail-robot/start | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100755 container/mail-robot/start diff --git a/container/mail-robot/start b/container/mail-robot/start new file mode 100755 index 0000000..56c4259 --- /dev/null +++ b/container/mail-robot/start @@ -0,0 +1,28 @@ +#!/usr/bin/env ruby +# SPDX-License-Identifier: MulanPSL-2.0+ +# frozen_string_literal: true + +require_relative '../defconfig.rb' +docker_rm 'mail-robot' + +JUMPER_IP = '192.168.132.2' +JUMPER_PORT = '29999' +SEND_MAIL_PORT = '11311' + +cmd = %W[ + docker run + --restart=always + --name=mail-robot + -d + -e JUMPER_IP=#{JUMPER_IP} + -e JUMPER_PORT=#{JUMPER_PORT} + -e SEND_MAIL_PORT=#{SEND_MAIL_PORT} + -v /c/upstream-repos:/upstream-repos:ro + -v /srv/cci/Maildir:/Maildir:rw + -v /c/compass-ci:/compass-ci:ro + mail-robot +] + +cmd += ['bash', '-c', 'umask 002 && ruby ./mail-robot.rb'] + +system(*cmd) -- 2.23.0

1 0

[PATCH compass-ci] doc/code-spec: add 文档与易用性
by Wu Fengguang 30 Sep '20

30 Sep '20

From: Wu Fengguang <wufengguang(a)huawei.com> Signed-off-by: Wu Fengguang <wufengguang(a)huawei.com> --- doc/code-spec.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/doc/code-spec.md b/doc/code-spec.md index e783a0c..9805182 100644 --- a/doc/code-spec.md +++ b/doc/code-spec.md @@ -40,3 +40,16 @@ here: 2) 描述你的改动 3) 拆分你的改动 8）回复评审意见 + +文档与易用性 +============ + +- 每个feature都应当有 + - 用户文档 + - 设计文档 + - 指示对应的代码在哪里(class/func); 或者如果有多处代码互相配合, 指示用什么关键字grep能把它们串起来看 + +- 每个有命令行参数的脚本, 都应当支持help output (-h|--help|参数不够时显示帮助信息) + - dependencies/assumptions + - input parameters + - input/output example -- 2.23.0

1 0

[PATCH compass-ci] feat: helper: snapshot of taskqueue
by Tong Qunfeng 30 Sep '20

30 Sep '20

Give a snapshot information about taskqueue (data at redis sorted set queue "queues/id2content"). It will generate a file named taskqueue_snap_yyyymmdd.md. snapshot content examples: |id---|add_time----------|current queue name| 14703, 2020-07-22 14:33, sched/vm-hi1620-2p8g-xxx/in_process 14831, 2020-07-22 17:51, sched/vm-hi1620-2p8g-xxx/in_process crystal.89630, 2020-09-23 11:19, sched/vm-hi1620-2p8g/in_process crystal.89667, 2020-09-23 11:20, sched/vm-hi1620-2p8g/in_process Signed-off-by: Tong Qunfeng <taxcom(a)tom.com> --- .../helper/redis_op_gc/taskqueue_snap.rb | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100755 user-client/helper/redis_op_gc/taskqueue_snap.rb diff --git a/user-client/helper/redis_op_gc/taskqueue_snap.rb b/user-client/helper/redis_op_gc/taskqueue_snap.rb new file mode 100755 index 0000000..ebf32b5 --- /dev/null +++ b/user-client/helper/redis_op_gc/taskqueue_snap.rb @@ -0,0 +1,44 @@ +#!/usr/bin/env ruby + +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# frozen_string_literal: true + +require 'json' +require_relative './basic_env' + +# redis hash key "queues/id2content" is used by taskqueue +cmd = "#{CMD_BASE} queues/id2content , hgetall" +result = `#{cmd}`.chomp +results = result.split("\n") + +task_info = [] + +# format time to "2020-09-29 14:23" +i = 0 +task_num = results.size / 2 +while i < task_num + task_id = results[i] + task_content = JSON.parse(results[i + 1]) + + time = task_content['add_time'] + task_content['add_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + time = task_content['move_time'] + task_content['move_time'] = Time.at(time.to_f).strftime('%Y-%m-%d %H:%M') unless time.nil? + + task_info << { 'id' => task_id, 'value' => task_content } + + i += 2 + set_progress(i, task_num) +end + +# sort and put short information to file. +task_info.sort! { |a, b| a['value']['add_time'] <=> b['value']['add_time'] } +today = Time.now.strftime('%Y%m%d') +File.open("taskqueue_snap_#{today}.md", 'w') do |f| + task_info.each do |task| + f.write(task['id'] + ', ' + task['value']['add_time'] \ + + ', ' + task['value']['queue'] + "\n") + end +end -- 2.23.0

2 1

[PATCH compass-ci] modify the origin address for fluentd-base image
by Xiao Shenwei 30 Sep '20

30 Sep '20

error message: ERROR: libcurl-7.67.0-r0: package mentioned in index not found refer to: https://mirrors.huaweicloud.com/alpine Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- container/fluentd-base/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/fluentd-base/Dockerfile b/container/fluentd-base/Dockerfile index ef671c2..62f55fd 100644 --- a/container/fluentd-base/Dockerfile +++ b/container/fluentd-base/Dockerfile @@ -5,7 +5,7 @@ FROM alpine:3.11 MAINTAINER Wu Zhende <wuzhende666(a)163.com> -RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|http://mirrors.huaweicloud.com|g' /etc/apk/repositories +RUN sed -ri.origin 's|^https?://dl-cdn.alpinelinux.org|https://mirrors.huaweicloud.com|g' /etc/apk/repositories RUN apk update -- 2.23.0

3 5

[PATCH v2 compass-ci] sparrow: start the firewall in adapting openeuler
by Hu Xuejiao 30 Sep '20

30 Sep '20

When firewalld is inactive, return code of 'systemctl status firewalld' will be 3, and install-tiny has set -e, so this will cause this process exit. Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/2-network/iptables | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/sparrow/2-network/iptables b/sparrow/2-network/iptables index 092c4ca..b4dc2d2 100755 --- a/sparrow/2-network/iptables +++ b/sparrow/2-network/iptables @@ -12,15 +12,18 @@ iptables -I FORWARD 1 -j ACCEPT iptables -t nat -A POSTROUTING -o "$PUB_IFACE" -s $BR0_SUBNET -j MASQUERADE iptables -t nat -A POSTROUTING -o $BR0_IFACE -d $BR0_SUBNET -j MASQUERADE -systemctl status firewalld | grep -q "running" && { +if command -v "firewall-cmd" > /dev/null; then + systemctl start firewalld && { + DOCKER0_IFACE=docker + DOCKER0_SUBNET=172.17.0.0/16 - DOCKER0_IFACE=docker0 - DOCKER0_SUBNET=172.17.0.0/16 + iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE + iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE - iptables -t nat -A POSTROUTING -o $PUB_IFACE -s $DOCKER0_SUBNET -j MASQUERADE - iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERADE - - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" - firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" -} + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" + firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" + } +else + echo "firewall.service could not be found" +fi -- 2.23.0

3 3

[PATCH compass-ci 2/4] sparrow: add domain name resolution for Docker images
by Hu Xuejiao 30 Sep '20

30 Sep '20

When executing ./crystal-shards/build appear error: http://mirrors.huaweicloud.com/alpine/v3.11/main: temporary error(try again later) Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/3-code/dev-env | 3 +++ sparrow/3-code/os/debian | 4 ++++ 2 files changed, 7 insertions(+) create mode 100644 sparrow/3-code/os/debian diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index ae267bd..80ee2cf 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -32,3 +32,6 @@ export REPO_SRC=/c/upstream-repos export PATH="$PATH:$CCI_SRC/sbin:$LKP_SRC/sbin:$LKP_SRC/bin" EOF +source /etc/os-release + +. $(dirname ${BASH_SOURCE[0]})/os/${ID} diff --git a/sparrow/3-code/os/debian b/sparrow/3-code/os/debian new file mode 100644 index 0000000..171b979 --- /dev/null +++ b/sparrow/3-code/os/debian @@ -0,0 +1,4 @@ +#!/bin/bash +cat >> /etc/default/docker <<EOF +DOCKER_OPTS="--dns 114.114.114.114" +EOF -- 2.23.0

2 1

[PATCH compass-ci 1/4] container: consider the situation that GITCACHE_HOST is undefined
by Hu Xuejiao 30 Sep '20

30 Sep '20

Error Msg: Step 5/12 : RUN [ -n "$GITCACCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig ---> Running in 52ffe9f8ecd1 The command '/bin/sh -c [ -n "$GITCACHE_HOST" ] && echo -e "[url\"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig' reton-zero code: 1 Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- container/crystal-shards/Dockerfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/crystal-shards/Dockerfile b/container/crystal-shards/Dockerfile index 0c5713c..bdc5b66 100644 --- a/container/crystal-shards/Dockerfile +++ b/container/crystal-shards/Dockerfile @@ -7,7 +7,7 @@ MAINTAINER chief <taxcom(a)tom.com> ARG GITCACHE_HOST ARG GITCACHE_PORT -RUN [ -n "$GITCACHE_HOST" ] && echo -e "[url \"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig +RUN [ -z "$GITCACHE_HOST" ] || echo -e "[url \"http://$GITCACHE_HOST:${GITCACHE_PORT:-5000}/\"]\n\tinsteadOf = https://" >> /etc/gitconfig copy shard.yml /usr/share/crystal/app/shard.yml -- 2.23.0

2 1

[PATCH compass-ci 1/2] feat(dnsmasq): allow the use of custom dnsmasq conf file
by Yu Chuan 29 Sep '20

29 Sep '20

The custom dnsmasq conf files should be placed under the follow dir: /etc/compass-ci/dnsmasq.d/ Signed-off-by: Yu Chuan <13186087857(a)163.com> --- container/dnsmasq/start | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/container/dnsmasq/start b/container/dnsmasq/start index 3cad801ec81c..8e0e05c953fb 100755 --- a/container/dnsmasq/start +++ b/container/dnsmasq/start @@ -8,6 +8,10 @@ docker_rm dnsmasq kill $(ps -ef| grep dnsmasq| grep libvirt| awk '{print $2}') 2> /dev/null +# load custom dnsmasq.conf +[ -d "/etc/compass-ci/dnsmasq.d/" ] && + cp /etc/compass-ci/dnsmasq.d/* ${PWD}/dnsmasq.d/ + cmd=( docker run --restart=always -- 2.23.0

4 5

[PATCH v2 compass-ci] sparrow: delete the same name install package
by Zhang Yu 29 Sep '20

29 Sep '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- sparrow/0-package/os/debian | 1 - 1 file changed, 1 deletion(-) diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 8301500..68423d7 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -12,9 +12,8 @@ pkgs=( make docker.io ruby-full ruby-dev - bridge-utils qemu dmidecode lockfile-progs bridge-utils -- 2.23.0

1 0

[PATCH compass-ci 4/4] sparrow: open the masquerade at public zone
by Hu Xuejiao 29 Sep '20

29 Sep '20

Signed-off-by: Hu XueJiao <huxuejiao1(a)huawei.com> --- sparrow/2-network/iptables | 1 + 1 file changed, 1 insertion(+) diff --git a/sparrow/2-network/iptables b/sparrow/2-network/iptables index 831c1a3..0af422e 100755 --- a/sparrow/2-network/iptables +++ b/sparrow/2-network/iptables @@ -32,3 +32,4 @@ iptables -t nat -A POSTROUTING -o $DOCKER0_IFACE -d $DOCKER0_SUBNET -j MASQUERAD firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$DOCKER0_SUBNET accept" firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=$BR0_SUBNET accept" firewall-cmd --zone=public --add-rich-rule="rule family=ipv4 source address=0.0.0.0/32 accept" +firewall-cmd --zone=public --add-masquerade -- 2.23.0

1 0

[PATCH compass-ci] add "host" field to job
by Zhang Yuhang 29 Sep '20

29 Sep '20

Signed-off-by: Zhang Yuhang <zhangyuhang25(a)huawei.com> --- src/lib/job.cr | 22 ++++++++++++++++++++++ src/lib/sched.cr | 1 + 2 files changed, 23 insertions(+) diff --git a/src/lib/job.cr b/src/lib/job.cr index 53a560c..5008638 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -28,6 +28,19 @@ module JobHelper return tbox_group end + def self.match_host(testbox : String) + # "xxx-xxx-123" remove "-123" + find = testbox.match(/(.*)-\d+$/) + if testbox[0, 3] == "vm-" + # "vm-xxx-xxx--xxx" remove "--xxx" + # "vm-xxx-2xx-xxx" remove "-xxx" + find = testbox.match(/(.*)--.+$/) || + testbox.match(/(.*-\d\w*)-[a-zA-Z]+$/) || + find + end + find ? find[1] : testbox + end + def self.service_path(path) temp_path = File.real_path(path) return temp_path.split("/srv")[-1] @@ -161,6 +174,7 @@ class Job private def set_result_root update_tbox_group_from_testbox # id must exists, need update tbox_group + update_host_from_testbox # id must exists, need update host date = Time.local.to_s("%F") self["result_root"] = "/result/#{suite}/#{tbox_group}/#{date}/#{id}" @@ -186,6 +200,14 @@ class Job end end + # if not assign host, set it to a match result from testbox + # if job special host, should we just set host=host + private def update_host_from_testbox + if self["host"] == "" + @hash["host"] = JSON::Any.new(JobHelper.match_host(testbox)) + end + end + def [](key : String) : String "#{@hash[key]?}" end diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 15f5c9e..4e71959 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -226,6 +226,7 @@ class Sched # add to job content when multi-test job["testbox"] = tbox_group job.update_tbox_group(tbox_group) + job.update_host(tbox_group) job["node_roles"] = config["roles"].as_a.join(" ") job["node_macs"] = config["macs"].as_a.join(" ") -- 2.23.0

1 0

help star 2020-09-25
by Wu Fengguang 29 Sep '20

29 Sep '20

Jihui is our help star, thanks to kind and helpful Jihui! wfg@crystal ~/todo% bin/help-star people/*/credit/2020-09-25.md 25.7 Shi Zhichao 21.9 Wu Fengguang 12.1 Wei Jihui 11.0 Zhang Yu 10.8 Yu Chuan 10.2 Du Kaitian 9.7 Lu Weitao 9.2 Wang Chenglong 8.6 Wu Zhende 7.7 Bai Jing 7.7 Wang Yong 7.7 Zhang Dewan 6.5 Sun Yukui 6.5 Cui Yili 6.1 Cao Xueliang 5.2 Li Yuanchao 5.1 Zhou Shengrui 4.7 Xu Xijian 4.5 Tong Qunfeng 4.5 Xiao Shenwei 4.3 Zhang Yale 3.9 Zhang Yuhang 3.7 Luan Shengde 3.0 Liu Shaofei 2.8 Liu Yinsi 2.4 Lu Kaiyi 1.6 Lin Jiaxin 1.3 Hu Xuejiao 1.0 Li Ping 0.9 Ren Wen 0.0 Ren Zhiwen 0.0 Sun Lijun 0.0 Liang Tao 0.0 Wang Shuai 0.0 Cui Xucui 0.0 Xie Zheng

1 0

review star 2020-09-29
by Wu Fengguang 29 Sep '20

29 Sep '20

Yuchuan is our review star, thanks Yuchuan! wfg@crystal ~/todo% bin/review-star Review email records in the last 7 days total review paragraph point people 401 212 425 75 Wu Fengguang 129 23 56 27 Tong Qunfeng 28 17 52 12 Shi Zhichao 49 22 43 24 Du Kaitian 67 14 41 10 Yu Chuan 28 18 27 7 Xu Xijian 41 12 21 7 Xiao Shenwei 24 14 20 5 Wei Jihui 21 11 20 6 Li Yuanchao 20 10 15 0 Zhang Yu 8 8 14 2 Sun Yukui 32 10 14 1 Liu Yinsi 54 5 11 3 Hu Xuejiao 33 4 10 1 Cao Xueliang 90 5 9 1 Luan Shengde 44 7 8 0 Wu Zhende 13 5 7 0 Xie Zheng 116 5 7 0 Ren Wen 13 7 7 0 Wang Yong 9 3 6 3 Zhou Shengrui 17 3 6 0 Liu Shaofei 69 1 5 1 Lu Weitao 67 1 5 3 Zhang Yuhang 20 3 3 1 Zhang Yale 29 1 3 1 Li Ping 30 1 2 6 Cui Yili 15 6 2 2 Lu Kaiyi 13 1 1 0 Lin Jiaxin 29 1 1 0 Wang Chenglong 1 0 0 0 Liang Tao 8 0 0 0 Zhang Dewan 22 3 0 1 Bai Jing

1 0

help star 2020-09-18
by Wu Fengguang 29 Sep '20

29 Sep '20

Jihui is our help star, thanks to kind and helpful Jihui! wfg@crystal ~/todo% bin/help-star people/*/credit/2020-09-18.md 32.6 Wu Fengguang 20.1 Shi Zhichao 18.1 Wei Jihui 17.6 Xu Xijian 15.1 Bai Jing 13.4 Yu Chuan 12.2 Zhang Dewan 11.8 Lu Weitao 10.4 Tong Qunfeng 9.8 Xiao Shenwei 9.6 Cui Yili 9.0 Liu Shaofei 8.5 Du Kaitian 8.4 Liu Yinsi 7.7 Zhang Yu 7.5 Wu Zhende 7.3 Wang Yong 7.1 Cao Xueliang 6.9 Sun Yukui 6.6 Hu Xuejiao 6.5 Wang Chenglong 6.1 Li Yuanchao 5.8 Zhou Shengrui 4.7 Lin Jiaxin 3.6 Liang Tao 3.5 Luan Shengde 3.1 Lu Kaiyi 2.5 Zhang Yale 1.4 Lu Shengde 1.2 Ren Wen 1.1 Zhang Yuhang 0.8 Li Ping 0.0 Ren Zhiwen 0.0 Sun Lijun 0.0 Wang Shuai 0.0 Cui Xucui 0.0 Xie Zheng

1 0

git count 2020-09-29
by Wu Fengguang 29 Sep '20

29 Sep '20

Renwen is code star of the week, thanks Renwen! wfg@crystal ~/todo% bin/git-count 11 40 Ren Wen 18 Wu Fengguang 12 Lu Weitao 11 Wu Zhende 10 Hu Xuejiao 8 Zhang Yuhang 7 Cao Xueliang 7 Liu Yinsi 7 Xiao Shenwei 6 Yu Chuan 5 Bai Jing 5 Zhang Yu 4 Xu Xijian 4 Wei Jihui 4 Cui Yili 4 Wang Yong 4 Li Yuanchao 3 Wang Chenglong 3 Luan Shengde 3 Lin Jiaxin 3 Xie Zheng 2 Zhang Yale 2 Li Ping 2 Lu Kaiyi 1 Du Kaitian 1 Liu Shaofei 1 Liang Tao 1 Zhang Dewan 0 chief 0 Ren Zhiwen 0 Sun Lijun 0 Sun Yukui 0 Shi Zhichao 0 Zhou Shengrui

1 0

[PATCH v2 compass-ci] sparrow/0-package: delete the repeate install package
by Zhang Yu 29 Sep '20

29 Sep '20

Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- sparrow/0-package/os/debian | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 8301500..ef90e50 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -13,11 +13,10 @@ pkgs=( docker.io ruby-full ruby-dev - bridge-utils qemu + bridge-utils dmidecode lockfile-progs - bridge-utils ) apt-get install -y "${pkgs[@]}" -- 2.23.0

3 2

[PATCH compass-ci] git_mirror: feat: reload fork_info when /upstream-repos update
by Li Yuanchao 29 Sep '20

29 Sep '20

Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index ad27880..88a94ac 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -147,6 +147,8 @@ class MirrorMain end def send_message(feedback_info) + feedback_info.merge!(@git_info[feedback_info[:git_repo]]) + feedback_info.delete(:cur_refs) message = feedback_info.to_json @message_queue.publish(message) end @@ -158,12 +160,12 @@ class MirrorMain @fork_stat[feedback_info[:git_repo]][:queued] = false return unless feedback_info[:possible_new_refs] + return reload_fork_info if feedback_info[:git_repo] == 'upstream-repos/upstream-repos' + new_refs = check_new_refs(feedback_info[:git_repo]) return if new_refs[:heads].empty? feedback_info[:new_refs] = new_refs - feedback_info.merge!(@git_info[feedback_info[:git_repo]]) - feedback_info.delete(:cur_refs) send_message(feedback_info) end @@ -219,4 +221,27 @@ class MirrorMain @git_info[git_repo][:cur_refs] = cur_refs return new_refs end + + def reload_fork_info + upstream_repos = 'upstream-repos/upstream-repos' + if @git_info[upstream_repos][:cur_refs].empty? + @git_info[upstream_repos][:cur_refs] = get_cur_refs(upstream_repos) + else + old_commit = @git_info[upstream_repos][:cur_refs][:heads]['refs/heads/master'] + new_refs = check_new_refs(upstream_repos) + new_commit = new_refs[:heads]['refs/heads/master'] + changed_files = %x(git -C /srv/git/#{upstream_repos}.git diff --name-only #{old_commit}...#{new_commit}) + reload(changed_files) + end + end + + def reload(file_list) + system("git -C #{REPO_DIR} pull") + file_list.each_line do |file| + next if File.basename(file) == '.ignore' + + repo_dir = "#{REPO_DIR}/#{file}" + load_repo_file(repo_dir, File.dirname(file), File.basename(file)) if File.file?(repo_dir) + end + end end -- 2.23.0

2 1

[PATCH v4 compass-ci] fix: sched: submit idle jobs
by Ren Wen 29 Sep '20

29 Sep '20

1) add field 'idle_job' for idle jobs, value is always 'true'. 2) add two fields 'FLUENTD_SERVER_*' for monitoring logs of virtual machines started by physical machine. Signed-off-by: Ren Wen <15991987063(a)163.com> --- src/lib/sched.cr | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/src/lib/sched.cr b/src/lib/sched.cr index 15f5c9e..309df04 100644 --- a/src/lib/sched.cr +++ b/src/lib/sched.cr @@ -468,9 +468,16 @@ class Sched def auto_submit_idle_job(tbox_group) full_path_patterns = "#{ENV["CCI_REPOS"]}/lab-#{ENV["lab"]}/allot/idle/#{tbox_group}/*.yaml" + extra_job_fields = [ + "idle_job=true", + "FLUENTD_SERVER_HOST=#{ENV["FLUENTD_SERVER_HOST"]}", + "FLUENTD_SERVER_PORT=#{ENV["FLUENTD_SERVER_PORT"]}", + ] + Jobfile::Operate.auto_submit_job( full_path_patterns, - "testbox: #{tbox_group}") if Dir.glob(full_path_patterns).size > 0 + "testbox: #{tbox_group}", + extra_job_fields) if Dir.glob(full_path_patterns).size > 0 end private def add_kernel_console_param(arch_tmp) @@ -520,7 +527,7 @@ class Sched "#{JobHelper.service_path("#{SRV_INITRD}/osimage/#{job.os_dir}/vmlinuz")}" else response += "kernel #{os_http_prefix}" + - "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" + "#{JobHelper.service_path("#{SRV_OS}/#{job.os_dir}/vmlinuz")}" end response += " user=lkp" response += " job=/lkp/scheduled/job.yaml RESULT_ROOT=/result/job rootovl ip=dhcp ro" -- 2.23.0

1 0

[PATCH compass-ci] check path of the upstream_repo under /srv/git/
by Li Ping 29 Sep '20

29 Sep '20

--- lib/git.rb | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/lib/git.rb b/lib/git.rb index db97c47..a181ec9 100644 --- a/lib/git.rb +++ b/lib/git.rb @@ -3,8 +3,34 @@ # wrap common git commands class GitCommit def initialize(repo, commit) - @git_prefix = "git -C /srv/git/#{repo}.git" + @repo = repo @commit = commit + @git_prefix = cd_repo_path + end + + def check_repo + fork_name = @repo.split('/')[0] + project = @repo.split('/')[1] + first_letter = project[0] + fork_name = 'linus' if project == 'linux' + path_a = find_path("/srv/git/#{first_letter}/#{project}/#{project}.git") + path_b = find_path("/srv/git/#{first_letter}/#{project}/#{fork_name}.git") + path_c = find_path("/srv/git/#{project}/#{fork_name}.git") + path_d = find_path("/srv/git/#{project}/#{project}.git") + path_a || path_b || path_c || path_d + end + + def find_path(path) + return false if Dir.glob(path).empty? + + Dir.glob(path).first + end + + def cd_repo_path + find_repo = check_repo + return unless find_repo + + "git -C #{find_repo}" end def author_name -- 2.23.0

5 13