- Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] etc/makepkg.conf: disable strip symbols
by Wei Jihui 02 Feb '21

02 Feb '21

[why] Strip symbols missing when exec cci-makepkg, file size is changed. [How] according to https://man.archlinux.org/man/makepkg.conf.5.en: Strip symbols from binaries and libraries. If you frequently use a debugger on programs or libraries, it may be helpful to disable this option. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- etc/makepkg.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/etc/makepkg.conf b/etc/makepkg.conf index 5d73399d..1ff5d558 100644 --- a/etc/makepkg.conf +++ b/etc/makepkg.conf @@ -83,7 +83,7 @@ BUILDENV=(!distcc color !ccache check !sign) #-- purge: Remove files specified by PURGE_TARGETS #-- debug: Add debugging flags as specified in DEBUG_* variables # -OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug) +OPTIONS=(!strip docs !libtool !staticlibs emptydirs zipman purge !debug) #-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512 INTEGRITY_CHECK=(md5) -- 2.23.0

2 1

[PATCH compass-ci] git_mirror.rb: merge default info when load upstream-repos
by Li Yuanchao 02 Feb '21

02 Feb '21

Load DEFAULTS files in upstream-repos. Merge the most close DEFAULTS file with git_info[git_repo]. The DEFAULTS files would keep common things like submit commands which used by many repos. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- lib/git_mirror.rb | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/lib/git_mirror.rb b/lib/git_mirror.rb index d089b1c..fa2e0cc 100644 --- a/lib/git_mirror.rb +++ b/lib/git_mirror.rb @@ -130,8 +130,8 @@ class MirrorMain defaults_file = "#{repodir}/DEFAULTS" return unless File.exist?(defaults_file) - project = repodir.delete_prefix("#{REPO_DIR}/") - @defaults[project] = YAML.safe_load(File.open(defaults_file)) + defaults_key = repodir == REPO_DIR ? 'default' : repodir.delete_prefix("#{REPO_DIR}/") + @defaults[defaults_key] = YAML.safe_load(File.open(defaults_file)) end def traverse_repodir(repodir) @@ -225,7 +225,7 @@ class MirrorMain git_repo = "#{project}/#{fork_name}" @git_info[git_repo] = YAML.safe_load(File.open(repodir)) @git_info[git_repo]['git_repo'] = git_repo - @git_info[git_repo].merge!(@defaults[project]) if @defaults[project] + @git_info[git_repo] = merge_defaults(git_repo) fork_stat_init(git_repo) @priority_queue.push git_repo, @priority @priority += 1 @@ -447,3 +447,16 @@ class MirrorMain return update_year_hash(new_refs_count, year, month, day) end end + +# main thread +class MirrorMain + def merge_defaults(git_repo) + defaults_key = File.dirname(git_repo) + while defaults_key != '.' + return @git_info[git_repo].merge!(@defaults[defaults_key]) if @defaults[defaults_key] + + defaults_key = File.dirname(defaults_key) + end + return @git_info[git_repo].merge!(@defaults['default']) if @defaults['default'] + end +end -- 2.23.0

1 0

[PATCH compass-ci 1/2] doc: add job/os_mount.md
by Yu Chuan 02 Feb '21

02 Feb '21

Add description of os_mount field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 62 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 doc/job/os_mount.md diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md new file mode 100644 index 000000000000..a4d832e07955 --- /dev/null +++ b/doc/job/os_mount.md @@ -0,0 +1,62 @@ +# Summary +========= + +`os_mount` defines the mount method of tbox's root partition. + +It has the following optional values: + - nfs + - cifs + - initramfs + - container + - local + +Usage example: + - submit iperf.yaml testbox=vm-2p8g os_mount=nfs + - submit iperf.yaml testbox=vm-2p8g os_mount=cifs + - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs + - submit iperf.yaml testbox=vm-2p8g os_mount=container + - submit iperf.yaml testbox=vm-2p8g os_mount=local + +# Optional Values +================= + +## Work flow when os_mount=local + +1. user submit job with os_mount: local + optional kernel cmdline params: # can be added in kernel_append_root field of job.yaml + - keep_last_root: keep the original data of disk what last job generated, don't wipe it + +2. scheduler return the ipxe_str to tbox + ``` + dhcp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/initrd.lkp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso/boot/vmlinuz + imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + boot + ``` + +3. boot + - check if `/dev/mapper/os-openeuler_aarch64_20.03_{timestamp}` exist # according the kernel cmdline + - if exist: + - if `keep_last_root` not in kernel cmdline params: + - re-create(lvremove and lvcreate) lvm rw snapshot /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if `keep_last_root` in kernel cmdline params: + - if /dev/mapper/os-openeuler_aarch64_20.03 exists: + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if /dev/mapper/os-openeuler_aarch64_20.03 not exists: + - generate /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03-{timestamp} + - boot from /dev/mapper/os-openeuler_aarch64_20.03 + - if not exist: + ``` + mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt + create /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} according rootfs_disk. # lvm size: 10G + mkdir /mnt1, mount /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} /mnt1 + rsync -az /mnt/. /mnt1/ + umount /mnt /mnt1 + lvchange -p r /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + create lvm rw snapshot /dev/mapper/os-openeuler_aarch64_20.03 from /dev/mapper/os-openeuler_aarch64_20.03_{timestamp} + boot from /dev/mapper/os-openeuler_aarch64_20.03 + ``` + +4. execute the job -- 2.23.0

3 4

[PATCH compass-ci] container/defconfig: fix directory name spell error
by Liu Yinsi 02 Feb '21

02 Feb '21

container/defconfig.rb: Dir.glob(... '/etc/compass-ci/accounts/*.yaml', ...) [why] two file defconfig.sh and defconfig.rb, all load /etc/compass-ci/accounts/*.yaml, so the directory name should be same. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/defconfig.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/defconfig.sh b/container/defconfig.sh index 4a5f5fa..54e65b5 100755 --- a/container/defconfig.sh +++ b/container/defconfig.sh @@ -10,7 +10,7 @@ load_cci_defaults() yaml_file=( /etc/compass-ci/defaults/*.yaml - /etc/compass-ci/account/*.yaml + /etc/compass-ci/accounts/*.yaml "$HOME"/.config/compass-ci/defaults/*.yaml ) for i in "${yaml_file[@]}" -- 2.23.0

1 0

[PATCH compass-ci 2/2] doc: add job/rootfs_disk.md
by Yu Chuan 02 Feb '21

02 Feb '21

Add description of rootfs_disk field in job.yaml. Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/rootfs_disk.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 doc/job/rootfs_disk.md diff --git a/doc/job/rootfs_disk.md b/doc/job/rootfs_disk.md new file mode 100644 index 000000000000..8236e24facb8 --- /dev/null +++ b/doc/job/rootfs_disk.md @@ -0,0 +1,25 @@ +# Summary +========= + +`rootfs_disk` defines a disk which storing multiple rootfs, used for jobs which need boot from disk. + +Each of these rootfs corresponds to an logical volume, such as follow: + /dev/mapper/ + . + ├── os-openeuler_aarch64_20.03 + ├── os-openeuler_aarch64_20.03_{timestamp} + ├── os-openeuler_aarch64_20.03_sp1 + ├── os-openeuler_aarch64_20.03_sp1_{timestamp} + └── ... + +Addition: +1. `/dev/mapper/os-{os}_{os_arch}_{os_version}_{timestamp}` + - is a readonly logical volume + - used for backup and rollback + +2. `/dev/mapper/os-{os}_{os_arch}_{os_version}` + - is a read write logical volume + - use for boot and run job + - defaultly, this logical volume will be wiped before every job run + +3. more info can refer os_mount.md -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] service/lifecycle: to process the machine that times out
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/lifecycle.cr | 51 ++++++++++++++++++++++++++++++++++++++ src/lifecycle/lifecycle.cr | 8 ++++++ 2 files changed, 59 insertions(+) diff --git a/src/lib/lifecycle.cr b/src/lib/lifecycle.cr index af5cd07..75a18ec 100644 --- a/src/lib/lifecycle.cr +++ b/src/lib/lifecycle.cr @@ -4,6 +4,7 @@ require "kemal" require "yaml" +require "./scheduler_api" require "./web_env" require "../scheduler/elasticsearch_client" @@ -14,6 +15,7 @@ class Lifecycle @es = Elasticsearch::Client.new @env = env @log = env.log.as(JSONLogger) + @scheduler_api = SchedulerAPI.new end def alive(version) @@ -36,4 +38,53 @@ class Lifecycle } @es.search("testbox", query) end + + def get_timeout_testbox(size) + # why use "now+8h" + # The time zone in the ES database is 0 + # add 8 hours to the current time and convert it to time zone 0 + query = { + "size" => size, + "query" =>{ + "range" => { + "deadline" => { + "lt" => "now+8h" + } + } + } + } + + results = @es.search("testbox", query) + results["hits"]["hits"] + end + + def process_timeout_testbox + size = @env.params.query["size"]? || 20 + results = get_timeout_testbox(size) + return results unless results.is_a?(JSON::Any) + + results.as_a.each do |result| + hostname = result["_id"].to_s + hash = result["_source"] + close_lifecycle(hostname, hash, "timeout") + end + end + + # 1.reboot testbox, do in the fature + # 2.update testbox info in es + # 3.close job + def close_lifecycle(hostname, hash, close_reason) + state = hash["state"]? == "rebooting" ? hash["state"] : "forcibly_rebooting" + testbox_update_info = { + "deadline" => nil, + "state" => state + } + @es.update_tbox(hostname, testbox_update_info) + + # If the value of state is rebooting, + # the job has been closed and don't need to close it again + return if state == "rebooting" + + @scheduler_api.close_job(hash["job_id"], close_reason).status_message + end end diff --git a/src/lifecycle/lifecycle.cr b/src/lifecycle/lifecycle.cr index 4154a27..13d36b7 100644 --- a/src/lifecycle/lifecycle.cr +++ b/src/lifecycle/lifecycle.cr @@ -29,4 +29,12 @@ module Cycle get "/get_running_testbox" do |env| env.lifecycle.get_running_testbox.to_json end + + # curl http://localhost:11311/process_timeout_testbox?size=10 + # size indicates the number of records to be processed at a time + get "/process_timeout_testbox" do |env| + env.lifecycle.process_timeout_testbox + + "done" + end end -- 2.23.0

1 0

[PATCH v2 compass-ci 1/2] lib/scheduler_api.cr: added parameter "job_state"
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/scheduler_api.cr | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/src/lib/scheduler_api.cr b/src/lib/scheduler_api.cr index b8dd514..0fdcd12 100644 --- a/src/lib/scheduler_api.cr +++ b/src/lib/scheduler_api.cr @@ -10,10 +10,14 @@ class SchedulerAPI @host = ENV.has_key?("SCHED_HOST") ? ENV["SCHED_HOST"] : "172.17.0.1" end - def close_job(job_id) + def close_job(job_id, job_state = nil) + url = "/~lkp/cgi-bin/lkp-post-run?job_id=#{job_id}" + url += "&job_state=#{job_state}" if job_state + client = HTTP::Client.new(@host, port: @port) - response = client.get("/~lkp/cgi-bin/lkp-post-run?job_id=#{job_id}") + response = client.get(url) client.close() + return response end end -- 2.23.0

1 0

[PATCH v2 compass-ci] lib/job.cr: add a new os_mount named local
by Wu Zhende 02 Feb '21

02 Feb '21

Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- src/lib/job.cr | 2 +- src/scheduler/kernel_params.cr | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/src/lib/job.cr b/src/lib/job.cr index 964c520..c013aa7 100644 --- a/src/lib/job.cr +++ b/src/lib/job.cr @@ -344,7 +344,7 @@ class Job end # defaults to the 1st value - VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container"] + VALID_OS_MOUNTS = ["initramfs", "nfs", "cifs", "container", "local"] private def set_os_mount if is_docker_job? diff --git a/src/scheduler/kernel_params.cr b/src/scheduler/kernel_params.cr index dea9215..ed64bea 100644 --- a/src/scheduler/kernel_params.cr +++ b/src/scheduler/kernel_params.cr @@ -18,6 +18,7 @@ class Job "cifs" => "root=cifs://#{OS_HTTP_HOST}#{os_real_path}" + ",guest,ro,hard,vers=1.0,noacl,nouser_xattr", "initramfs" => "rdinit=/sbin/init prompt_ramdisk=0", + "local" => "root=/dev/mapper/os-#{self.os}_#{self.os_arch}_#{self.os_version}", "container" => "", } -- 2.23.0

1 0

[PATCH v4 compass-ci] providers/libvirt: split domain.xml to some common templates
by Xiao Shenwei 02 Feb '21

02 Feb '21

[why] libvirt can boot a vm based on a xml file named domain.xml. it contains many elements like os, device, cpu. to meet a requirements of a single module or a combination of multiple modules. split the domain.xml may be better. here are some scenarios: case 1: the user provides a complete domain.xml file case 2: the user only provides a cpu.xml template case 3: the user provides cpu.xml and disk.xml or other templates Signed-off-by: Xiao Shenwei <xiaoshenwei96(a)163.com> --- providers/libvirt/templates/active.xml | 3 +++ providers/libvirt/templates/clock.xml | 5 +++++ providers/libvirt/templates/cpu.xml | 3 +++ providers/libvirt/templates/devices.xml | 3 +++ providers/libvirt/templates/disk.xml | 7 +++++++ providers/libvirt/templates/domain.xml | 10 ++++++++++ providers/libvirt/templates/emulator.xml | 1 + providers/libvirt/templates/interface.xml | 6 ++++++ providers/libvirt/templates/memory.xml | 1 + providers/libvirt/templates/name.xml | 1 + providers/libvirt/templates/options.yaml | 23 +++++++++++++++++++++++ providers/libvirt/templates/os.xml | 7 +++++++ providers/libvirt/templates/seclabel.xml | 4 ++++ providers/libvirt/templates/serial.xml | 4 ++++ 14 files changed, 78 insertions(+) create mode 100644 providers/libvirt/templates/active.xml create mode 100644 providers/libvirt/templates/clock.xml create mode 100644 providers/libvirt/templates/cpu.xml create mode 100644 providers/libvirt/templates/devices.xml create mode 100644 providers/libvirt/templates/disk.xml create mode 100644 providers/libvirt/templates/domain.xml create mode 100644 providers/libvirt/templates/emulator.xml create mode 100644 providers/libvirt/templates/interface.xml create mode 100644 providers/libvirt/templates/memory.xml create mode 100644 providers/libvirt/templates/name.xml create mode 100644 providers/libvirt/templates/options.yaml create mode 100644 providers/libvirt/templates/os.xml create mode 100644 providers/libvirt/templates/seclabel.xml create mode 100644 providers/libvirt/templates/serial.xml diff --git a/providers/libvirt/templates/active.xml b/providers/libvirt/templates/active.xml new file mode 100644 index 0000000..8ed7ee4 --- /dev/null +++ b/providers/libvirt/templates/active.xml @@ -0,0 +1,3 @@ +<on_poweroff>destroy</on_poweroff> +<on_reboot>destroy</on_reboot> +<on_crash>destroy</on_crash> diff --git a/providers/libvirt/templates/clock.xml b/providers/libvirt/templates/clock.xml new file mode 100644 index 0000000..5124f33 --- /dev/null +++ b/providers/libvirt/templates/clock.xml @@ -0,0 +1,5 @@ +<clock offset='localtime'> + <timer name='rtc' tickpolicy='catchup' track='guest'/> + <timer name='hpet' present='no'/> + <timer name='pit' tickpolicy='delay'/> +</clock> diff --git a/providers/libvirt/templates/cpu.xml b/providers/libvirt/templates/cpu.xml new file mode 100644 index 0000000..1f95dc6 --- /dev/null +++ b/providers/libvirt/templates/cpu.xml @@ -0,0 +1,3 @@ +<vcpu placement='static'><%= nr_cpu %></vcpu> +<cpu mode='host-passthrough' check='none'> +</cpu> diff --git a/providers/libvirt/templates/devices.xml b/providers/libvirt/templates/devices.xml new file mode 100644 index 0000000..83cd847 --- /dev/null +++ b/providers/libvirt/templates/devices.xml @@ -0,0 +1,3 @@ +<devices> + <graphics type='vnc' autoport='yes' listen='0.0.0.0'><graphics> +</devices> diff --git a/providers/libvirt/templates/disk.xml b/providers/libvirt/templates/disk.xml new file mode 100644 index 0000000..7aecef0 --- /dev/null +++ b/providers/libvirt/templates/disk.xml @@ -0,0 +1,7 @@ +<% if disk %> +<disk type='block' device='disk'> + <driver name='qemu' type='raw'/> + <source dev='<%= disk %>'/> + <target dev='vda' bus='virtio'/> +</disk> +<% end %> diff --git a/providers/libvirt/templates/domain.xml b/providers/libvirt/templates/domain.xml new file mode 100644 index 0000000..a3fdb63 --- /dev/null +++ b/providers/libvirt/templates/domain.xml @@ -0,0 +1,10 @@ +<?xml version="1.0" character="utf-8"?> +<domain type='kvm'> + <resource> + <partition>/machine</partition> + </resource> + <features> + <acpi/> + <gic version='3'/> + </features> +</domain> diff --git a/providers/libvirt/templates/emulator.xml b/providers/libvirt/templates/emulator.xml new file mode 100644 index 0000000..e915215 --- /dev/null +++ b/providers/libvirt/templates/emulator.xml @@ -0,0 +1 @@ +<emulator><%=qemu_path%></emulator> diff --git a/providers/libvirt/templates/interface.xml b/providers/libvirt/templates/interface.xml new file mode 100644 index 0000000..5d4c04f --- /dev/null +++ b/providers/libvirt/templates/interface.xml @@ -0,0 +1,6 @@ +<interface type='bridge'> + <source bridge='br0'/> + <mac address='<%= mac %>'/> + <target dev='vnet0'/> + <model type='virtio'/> +</interface> diff --git a/providers/libvirt/templates/memory.xml b/providers/libvirt/templates/memory.xml new file mode 100644 index 0000000..34aec49 --- /dev/null +++ b/providers/libvirt/templates/memory.xml @@ -0,0 +1 @@ +<memory unit='GiB'><%= memory.to_i %></memory> diff --git a/providers/libvirt/templates/name.xml b/providers/libvirt/templates/name.xml new file mode 100644 index 0000000..5dc98f4 --- /dev/null +++ b/providers/libvirt/templates/name.xml @@ -0,0 +1 @@ +<name><%= job_id %></name> diff --git a/providers/libvirt/templates/options.yaml b/providers/libvirt/templates/options.yaml new file mode 100644 index 0000000..c41ee29 --- /dev/null +++ b/providers/libvirt/templates/options.yaml @@ -0,0 +1,23 @@ +# Domain option: +# use templates/domain.xml if user not specified. +domain: domain + +# Default options: +# these elements must be replaced for each domain.xml. +default: + - name + - os + - memory + - serial + - interface + - active + - emulator + +# Replaceable options: +# these elements can be replaced by user configurations. +replaceable: + - devices + - cpu + - seclabel + - disk + - clock diff --git a/providers/libvirt/templates/os.xml b/providers/libvirt/templates/os.xml new file mode 100644 index 0000000..ee16126 --- /dev/null +++ b/providers/libvirt/templates/os.xml @@ -0,0 +1,7 @@ +<os> + <type arch='<%= arch %>' machine='virt'>hvm</type> + <loader readonly='yes' type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader> + <kernel><%= kernel %></kernel> + <initrd><%= initrd %></initrd> + <cmdline><%= cmdline %></cmdline> +</os> diff --git a/providers/libvirt/templates/seclabel.xml b/providers/libvirt/templates/seclabel.xml new file mode 100644 index 0000000..5c10a77 --- /dev/null +++ b/providers/libvirt/templates/seclabel.xml @@ -0,0 +1,4 @@ +<seclabel type='dynamic' model='dac' relabel='yes'> + <label>+0:+0</label> + <imagelabel>+0:+0</imagelabel> +</seclabel> diff --git a/providers/libvirt/templates/serial.xml b/providers/libvirt/templates/serial.xml new file mode 100644 index 0000000..2edc882 --- /dev/null +++ b/providers/libvirt/templates/serial.xml @@ -0,0 +1,4 @@ +<serial type='file'> + <source path="<%= log_file %>"/> + <target port="0"/> +</serial> -- 2.23.0

1 0

[PATCH v4 compass-ci 3/4] container/submit: submit file
by Luan Shengde 02 Feb '21

02 Feb '21

executable file for container: submit it can be run as the real command submit in the container. when you run it, it will calls the command 'submit' in the container. add ln for it to PATH, you can run it at anywhere. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/submit | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100755 container/submit/submit diff --git a/container/submit/submit b/container/submit/submit new file mode 100755 index 0000000..3f9539f --- /dev/null +++ b/container/submit/submit @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# This is a single-use container. +# The container will run if you run the 'submit' to submit jobs. +# After the command execute over, the container will been removed itself. +# Do not need to delete manually for it has been added option '--rm'. +date_suffix=$(date +%m%d-%H%M) +cmd=( + docker run + --rm + --name=submit-$USER-$data_suffix + -it + -v /etc/compass-ci:/etc/compass-ci:ro + -v $HOME/.config:/root/.config:ro + -v $HOME/.ssh:/root/.ssh:rw + submit + /root/lkp-tests/sbin/submit "$@" +) + +"${cmd[@]}" + -- 2.23.0

2 2

[PATCH v3 compass-ci 1/3] mail-robot: fix bug for parsing email
by Luan Shengde 02 Feb '21

02 Feb '21

[why] for non-gitee repos: we may sometimes can not get the author email from the http request feedback when checking the commit. users may still receive a fail email even if they send 'apply account' email correctly. [how] both gitee and non-gitee will clone the repo tree and check the commit: - check commit id exists - check commit author matches the email Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/parse-apply-account-email.rb | 42 ++++++++++--------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/container/mail-robot/lib/parse-apply-account-email.rb b/container/mail-robot/lib/parse-apply-account-email.rb index a41a89a..8baf7b7 100755 --- a/container/mail-robot/lib/parse-apply-account-email.rb +++ b/container/mail-robot/lib/parse-apply-account-email.rb @@ -6,7 +6,6 @@ require 'json' require 'mail' require 'rest-client' -require_relative 'gitee-commit-url-check' # used to parse mail_content for my_commit_url and my_ssh_pubkey # be called by AssignAccount when it needs to extract required data: @@ -91,29 +90,34 @@ class ParseApplyAccountEmail end def commit_url_availability(url, base_url) - hub_name = url.split('/')[2] - - # it requires authentication when sending request to get the commit information - # clone the repo and then validate the commit for the email address - if hub_name.eql? 'gitee.com' - gitee_commit(url, base_url) - else - non_gitee_commit(url) - end - end + repo_url = [base_url, 'git'].join('.') + repo_dir = repo_url.split('/')[-1] + commit_id = url.split('/')[-1] + + Dir.chdir '/tmp' + %x(/usr/bin/git clone --bare #{repo_url} #{repo_dir}) + + # get all commit IDs and check commit id exists + repo_commits = %x(git -C #{repo_dir} log --pretty=format:'%H').split(/\n/) + check_commit_exist(commit_id, repo_commits) - def gitee_commit(url, base_url) - my_gitee_commit = GiteeCommitUrlCheck.new(@my_info, url, base_url) - my_gitee_commit.gitee_commit_check + # get the auther's email for the commit + author_email = %x(/usr/bin/git -C #{repo_dir} show -s --format=%aE #{commit_id}).chomp + check_commit_email(author_email) + + FileUtils.rm_rf repo_dir end - def non_gitee_commit(url) - url_fdback = RestClient.get(url).body - email_index = url_fdback.index @my_info['my_email'] + def check_commit_exist(commit_id, repo_commits) + return if repo_commits.include? commit_id.chomp + + raise 'NO_COMMIT_ID' + end - return unless email_index.nil? + def check_commit_email(author_email) + return if author_email.eql? @my_info['my_email'].chomp - raise 'COMMIT_URL_NOT_AVAILABLE' + raise 'COMMIT_AUTHOR_ERROR' end def parse_pub_key -- 2.23.0

2 2

[PATCH lkp-tests] lib/job.rb: delete fields that need to be reloaded for reproduction task
by Li Ping 02 Feb '21

02 Feb '21

[why] load job from result's job yaml has key "my_email", like my_email: compass-ci-robot(a)qq.com my_name: compass-ci-robot lab: z9 reproduction task: my_email: xxxxx my_name: xxxx lab: xxx ==> check the field my_email fail each of these fields needs to be reset when submit job.yaml for reprodution task. however, these fields already exist in yaml file, can not to be reload, so delete these fields when submit reproduction job. Signed-off-by: Li Ping <1477412247(a)qq.com> --- lib/job.rb | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/lib/job.rb b/lib/job.rb index a03261537..2b122c01d 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -259,6 +259,11 @@ class Job hash.delete('override') end + if hash.has_key?('id') + d_keys = ["my_name", "my_email", "lab"] + hash.delete_if { |k| d_keys.include? k } + end + revise_hash(hash, load_include_yamls(@default_yamls), false) unless @default_yamls.empty? revise_hash(hash, load_include_yamls(@override_yamls), true) unless @override_yamls.empty? revise_hash(hash, @overrides, true) unless @overrides.empty? -- 2.23.0

1 0

[PATCH v3 compass-ci 2/3] mail-robot: optimize assign-account-fail-email
by Luan Shengde 02 Feb '21

02 Feb '21

add message for newly added error types. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- .../lib/assign-account-fail-eamil.rb | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/assign-account-fail-eamil.rb b/container/mail-robot/lib/assign-account-fail-eamil.rb index 242dfe1..3597159 100755 --- a/container/mail-robot/lib/assign-account-fail-eamil.rb +++ b/container/mail-robot/lib/assign-account-fail-eamil.rb @@ -33,17 +33,20 @@ def email_err_message(message) https://gitee.com/wu_fengguang/compass-ci/blob/master/doc/manual/test-oss-p… EMAIL_MESSAGE - when 'COMMIT_URL_NOT_AVAILABLE' + when 'COMMIT_AUTHOR_ERROR' err_message = <<~EMAIL_MESSAGE - We cannot confirm the commit url matches your email. + We cannot confirm the commit author matches your email. + Make sure it is truely submitted with your email. - Make sure the commit exists. - Make sure it is truely submitted with your email. - - Go to the commit history and find your commit. - Enter the commit and you will get your commit url. + Choose a commit that was submitted with your own email and try again. + EMAIL_MESSAGE + when 'NO_COMMIT_ID' + err_message = <<~EMAIL_MESSAGE + There is no such commit ID in the repo. - Try again after you have correctted the url + Please check the commit ID exists. + or + You have write the commit ID correctly. EMAIL_MESSAGE when 'NO_PUBKEY' err_message = <<~EMAIL_MESSAGE -- 2.23.0

2 2

[PATCH v4 lkp-tests] tests/cci-depends: fix generate symlink cgz file
by Wang Yong 02 Feb '21

02 Feb '21

[before] $ ll $LKP_SRC/distro/depends/sysbench-cpu sysbench-cpu -> sysbench $ tree /srv/initrd/deps/$os/$os_arch/$os_version/ . ├── sysbench_20200903.cgz ├── sysbench.cgz -> sysbench_20200903.cgz └── sysbench-cpu.cgz -> sysbench.cgz all cgz file under one directory, so we can generate symlink when depends file is a symlink [after] $ tree /srv/initrd/deps/$os/$os_arch/$os_version/sysbench-cpu . ├── sysbench-cpu_20210128.cgz └── sysbench-cpu.cgz -> sysbench-cpu_20210128.cgz now we upload cgz file through curl, it's not appropriate to generate symlink directly, now generate original cgz file and rename it Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 12 ++++++++++++ tests/cci-depends | 9 ++------- 2 files changed, 14 insertions(+), 7 deletions(-) diff --git a/distro/common b/distro/common index 1fd7df4e8..94d18f461 100755 --- a/distro/common +++ b/distro/common @@ -94,3 +94,15 @@ upload_to_target_dir() upload_one_curl "${dir}/${file}" "${dir}" >/dev/null } + +rename_cgz_file() +{ + [ "${BM_NAME}" == "${benchmark}" ] && return + + cd "$pack_to" && { + [ -f "${BM_NAME}".cgz ] || return + rm "${BM_NAME}".cgz + date=$(ls -F | grep -v "@" | grep -Eo "[0-9]{8}") + mv "${BM_NAME}"_"$date".cgz "${benchmark}"_"$date".cgz + } +} diff --git a/tests/cci-depends b/tests/cci-depends index 8ac1d7e58..4a7b78a41 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -38,13 +38,6 @@ umask 002 chown .lkp "$pack_to" } -check_shared_package "$benchmark" && { - ln -sf $BM_NAME.cgz $pack_to/$benchmark.cgz - echo "create symlink for shared package: $benchmark -> $BM_NAME" - - exit 0 -} - share_debian_package distro_install_depends cci-depends @@ -59,4 +52,6 @@ fixup_arch_install $pack_arch pack_benchmark_deps +rename_cgz_file + upload_to_target_dir ${pack_to} -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests/cci-depends: fix generate symlink cgz file
by Wang Yong 02 Feb '21

02 Feb '21

[before] $ ll $LKP_SRC/distro/depends/sysbench-cpu sysbench-cpu -> sysbench $ tree /srv/initrd/deps/$os/$os_arch/$os_version/ . ├── sysbench_20200903.cgz ├── sysbench.cgz -> sysbench_20200903.cgz └── sysbench-cpu.cgz -> sysbench.cgz all cgz file under one directory, so we can generate symlink when depends file is a symlink [after] $ tree /srv/initrd/deps/$os/$os_arch/$os_version/sysbench-cpu . ├── sysbench-cpu_20210128.cgz └── sysbench-cpu.cgz -> sysbench-cpu_20210128.cgz now we upload cgz file through curl, it's not appropriate to generate symlink directly, now generate original cgz file and rename it Signed-off-by: Wang Yong <wangyong0117(a)qq.com> --- distro/common | 11 +++++++++++ tests/cci-depends | 9 ++------- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/distro/common b/distro/common index 1fd7df4e8..f3a9c3071 100755 --- a/distro/common +++ b/distro/common @@ -94,3 +94,14 @@ upload_to_target_dir() upload_one_curl "${dir}/${file}" "${dir}" >/dev/null } + +rename_cgz_file() +{ + [ "${BM_NAME}" == "${benchmark}" ] && return + + cd "$pack_to" + [ -f "${BM_NAME}".cgz ] || return + rm "${BM_NAME}".cgz + date=$(ls -F | grep -v "@" | grep -Eo "[0-9]{8}") + mv "${BM_NAME}"_"$date".cgz "${benchmark}"_"$date".cgz +} diff --git a/tests/cci-depends b/tests/cci-depends index 8ac1d7e58..4a7b78a41 100755 --- a/tests/cci-depends +++ b/tests/cci-depends @@ -38,13 +38,6 @@ umask 002 chown .lkp "$pack_to" } -check_shared_package "$benchmark" && { - ln -sf $BM_NAME.cgz $pack_to/$benchmark.cgz - echo "create symlink for shared package: $benchmark -> $BM_NAME" - - exit 0 -} - share_debian_package distro_install_depends cci-depends @@ -59,4 +52,6 @@ fixup_arch_install $pack_arch pack_benchmark_deps +rename_cgz_file + upload_to_target_dir ${pack_to} -- 2.23.0

2 2

[PATCH compass-ci] fix spell error
by Bai Jing 02 Feb '21

02 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- doc/tutorial.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/tutorial.md b/doc/tutorial.md index bf72630..bb2951c 100644 --- a/doc/tutorial.md +++ b/doc/tutorial.md @@ -174,7 +174,7 @@ job yaml 是测试描述和执行的基本单元，以[YAML](http://yaml.org/YAM - 通过 qemu.img(qcow2格式)获取 rootfs (openEuler为例) - 1) 下载 openEule r官方网站提供的 qcow2 格式镜像 + 1) 下载 openEuler官方网站提供的 qcow2 格式镜像 ```bash wget https://repo.openeuler.org/openEuler-20.03-LTS/virtual_machine_img/aarch64/… ``` -- 2.23.0

1 0

[PATCH compass-ci] sbin/es-find order by start_time as default
by Lu Weitao 02 Feb '21

02 Feb '21

[Why] Many time, user would see the latest jobs [How] Set 'start_time' as default keyword for order with 'desc' Signed-off-by: Lu Weitao <luweitaobe(a)163.com> --- sbin/es-find | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sbin/es-find b/sbin/es-find index c57c02e..ed54955 100755 --- a/sbin/es-find +++ b/sbin/es-find @@ -72,7 +72,7 @@ def assign_range(value) range_value end -options = { nr_jobs: 10, desc_keyword: nil } +options = { nr_jobs: 10, desc_keyword: 'start_time' } range = nil opt_parser = OptionParser.new do |opts| -- 2.23.0

2 2

[PATCH v3 compass-ci] sparrow/install-server: add script to deploy compass-ci server
by Liu Yinsi 02 Feb '21

02 Feb '21

Classify compass-ci by functions: Client functions: submit job and boot testbox Server functions: run containers services Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 sparrow/install-server diff --git a/sparrow/install-server b/sparrow/install-server new file mode 100755 index 0000000..c532956 --- /dev/null +++ b/sparrow/install-server @@ -0,0 +1,20 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# For your reference only. +# It's better to run the below scripts step by step. + +[[ $CCI_SRC ]] || export CCI_SRC=$(dirname $(dirname $(realpath $0))) +cd $CCI_SRC/sparrow || exit + +0-package/common +1-storage/tiny +1-storage/permission +2-network/br0 +2-network/iptables +2-network/nfs +2-network/cifs +3-code/dev-env +. /etc/profile.d/compass.sh +4-docker/buildall run-only +7-systemd/systemd-setup -- 2.23.0

1 0

[PATCH v2 compass-ci] sparrow/install-server: add script to deploy compass-ci container services
by Liu Yinsi 02 Feb '21

02 Feb '21

[why] For user locally deploy compass-ci, some users' hosts can not access internet. [how] deploy all container services in the host machine, other functions: submit job and run qemu test, will deploy in other machine which can access internet by $LKP_SRC/setup/simply-ci. sparrow/install-tiny: one-click locally deploy compass-ci, do job test in locally compass-ci platform. sparrow/install-server: one-click locally deploy compass-ci server, just running containers services. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/install-server | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100755 sparrow/install-server diff --git a/sparrow/install-server b/sparrow/install-server new file mode 100755 index 0000000..c532956 --- /dev/null +++ b/sparrow/install-server @@ -0,0 +1,20 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. +# For your reference only. +# It's better to run the below scripts step by step. + +[[ $CCI_SRC ]] || export CCI_SRC=$(dirname $(dirname $(realpath $0))) +cd $CCI_SRC/sparrow || exit + +0-package/common +1-storage/tiny +1-storage/permission +2-network/br0 +2-network/iptables +2-network/nfs +2-network/cifs +3-code/dev-env +. /etc/profile.d/compass.sh +4-docker/buildall run-only +7-systemd/systemd-setup -- 2.23.0

2 2

[PATCH v5 compass-ci 3/3] replace log.sh
by Liu Yinsi 02 Feb '21

02 Feb '21

Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 2 +- container/result-cifs/start | 2 +- providers/my-qemu.sh | 4 +--- providers/qemu/kvm.sh | 3 +-- providers/qemu/pxe.sh | 4 +--- sparrow/4-docker/buildall | 6 +++--- 8 files changed, 10 insertions(+), 15 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 0b9601a..6a91b37 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-cifs/start b/container/os-cifs/start index 09ae3d2..85017cc 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/container/os-nfs/start b/container/os-nfs/start index 9539561..1f4ed42 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { log_error "kernel module nfs is missing (try: sudo modprobe nfs)" diff --git a/container/result-cifs/start b/container/result-cifs/start index de38ca2..7ce066a 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -3,7 +3,7 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. . $CCI_SRC/container/defconfig.sh -. $LKP_SRC/lib/log.sh +. $CCI_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { log_error "kernel module cifs is missing (try: sudo modprobe cifs)" diff --git a/providers/my-qemu.sh b/providers/my-qemu.sh index 113b3ef..3e3b2f4 100755 --- a/providers/my-qemu.sh +++ b/providers/my-qemu.sh @@ -2,9 +2,7 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh help() { diff --git a/providers/qemu/kvm.sh b/providers/qemu/kvm.sh index dd657cb..103e9e6 100755 --- a/providers/qemu/kvm.sh +++ b/providers/qemu/kvm.sh @@ -6,9 +6,8 @@ : ${nr_cpu:=1} : ${memory:=1G} -: ${LKP_SRC:="/c/lkp-tests"} -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh check_logfile() { diff --git a/providers/qemu/pxe.sh b/providers/qemu/pxe.sh index 0ba052c..3ab3c2b 100755 --- a/providers/qemu/pxe.sh +++ b/providers/qemu/pxe.sh @@ -4,9 +4,7 @@ # - nr_cpu # - memory -: ${LKP_SRC:="/c/lkp-tests"} - -source ${LKP_SRC}/lib/log.sh +source ${CCI_SRC}/lib/log.sh : ${nr_cpu:=1} : ${memory:=1G} diff --git a/sparrow/4-docker/buildall b/sparrow/4-docker/buildall index 3c0b7b0..727abb9 100755 --- a/sparrow/4-docker/buildall +++ b/sparrow/4-docker/buildall @@ -5,7 +5,7 @@ CONTAINER_PATH="$CCI_SRC/container" export action=$1 -[ -n "$LKP_SRC" ] && source "${LKP_SRC}/lib/log.sh" +source ${CCI_SRC}/lib/log.sh build_depends() { @@ -27,7 +27,7 @@ do_one() lockfile-create -q --use-pid --retry 100 --lock-name "$container_name".lock echo "$started_containers" | grep -w -q "$container_name" && return 0 - [ -n "$LKP_SRC" ] && log_info "start build $container." + log_info "start build $container." mkdir $tmpdir/$container_name 2>/dev/null && ( cd "$container" @@ -41,7 +41,7 @@ do_one() [ -x start ] && ./start [ "$container_name" == 'initrd-lkp' ] && ./run ) - [ -n "$LKP_SRC" ] && log_info "finish build $container." + log_info "finish build $container." started_containers+="$container_name" lockfile-remove --lock-name "$container_name".lock } -- 2.23.0

2 1

[PATCH v5 compass-ci 2/3] container: delete redundant log error info
by Liu Yinsi 02 Feb '21

02 Feb '21

[how] before: log_error "[ERROR] kernel module ..." after: log_error "kernel module ..." [why] log_error(): echo "$(date +'%F %T') [ERROR] -- $*" >&2 when call log_error(), already has complete log format, so delete redundant "[ERROR]" in parmas. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- container/initrd-cifs/start | 2 +- container/os-cifs/start | 2 +- container/os-nfs/start | 4 ++-- container/result-cifs/start | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/container/initrd-cifs/start b/container/initrd-cifs/start index 119d1b1..0b9601a 100755 --- a/container/initrd-cifs/start +++ b/container/initrd-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-cifs/start b/container/os-cifs/start index 47da260..09ae3d2 100755 --- a/container/os-cifs/start +++ b/container/os-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } diff --git a/container/os-nfs/start b/container/os-nfs/start index 8c08db8..9539561 100755 --- a/container/os-nfs/start +++ b/container/os-nfs/start @@ -6,11 +6,11 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^nfs\s" || { - log_error "[ERROR] kernel module nfs is missing (try: sudo modprobe nfs)" + log_error "kernel module nfs is missing (try: sudo modprobe nfs)" exit 1 } lsmod | grep -q "^nfsd\s" || { - log_error "[ERROR] kernel module nfsd is missing (try: sudo modprobe nfsd)" + log_error "kernel module nfsd is missing (try: sudo modprobe nfsd)" exit 1 } diff --git a/container/result-cifs/start b/container/result-cifs/start index 4f8c2b9..de38ca2 100755 --- a/container/result-cifs/start +++ b/container/result-cifs/start @@ -6,7 +6,7 @@ . $LKP_SRC/lib/log.sh lsmod | grep -q "^cifs\s" || { - log_error "[ERROR] kernel module cifs is missing (try: sudo modprobe cifs)" + log_error "kernel module cifs is missing (try: sudo modprobe cifs)" exit 1 } -- 2.23.0

2 1

[PATCH v5 compass-ci 1/3] lib: add log.sh
by Liu Yinsi 02 Feb '21

02 Feb '21

Records the run logs of the shell program Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- lib/log.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) create mode 100644 lib/log.sh diff --git a/lib/log.sh b/lib/log.sh new file mode 100644 index 0000000..626a6e4 --- /dev/null +++ b/lib/log.sh @@ -0,0 +1,23 @@ +#!/bin/bash +# SPDX-License-Identifier: MulanPSL-2.0+ +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +log_info() +{ + date +"[INFO] %F %T ${0##*/}: $*" +} + +log_debug() +{ + date +"[DEBUG] %F %T ${0##*/}: $*" +} + +log_warn() +{ + date +"[WARN] %F %T ${0##*/}: $*" >&2 +} + +log_error() +{ + date +"[ERROR] %F %T ${0##*/}: $*" >&2 +} -- 2.23.0

2 1

[PATCH v5 compass-ci] sbin: add script for download os rootfs
by Liu Yinsi 02 Feb '21

02 Feb '21

For locally deploy compass-ci, download os rootfs to run qemu test, the default download os rootfs is openeuler aarch64 20.03, user can also use this script to download other os rootfs if they need. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100755 sbin/download-rootfs diff --git a/sbin/download-rootfs b/sbin/download-rootfs new file mode 100755 index 0000000..e1d4091 --- /dev/null +++ b/sbin/download-rootfs @@ -0,0 +1,40 @@ +#!/bin/bash +# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. + +# download os rootfs. +# 1. download default os rootfs openeuler aarch64 20.03 +# example usage: +# ./download-rootfs +# 2. download os rootfs you want by add params $os, $os_arch, $os_version. +# example usage: +# ./download-rootfs debian aarch64 sid + +os=${1:-openeuler} +os_arch=${2:-aarch64} +os_version=${3:-20.03} + +. ${CCI_SRC}/container/defconfig.sh +. ${CCI_SRC}/lib/log.sh + +load_cci_defaults + +rootfs_dir=/srv/os/${os}/${os_arch}/${os_version} + +download_os_rootfs() +{ + mkdir -p $rootfs_dir + cd $rootfs_dir + + download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_PORT}/os/${os}/${os_arch}/${…" + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} + +download_os_rootfs -- 2.23.0

1 0

[PATCH compass-ci] service/lifecycle: change port
by Wu Zhende 02 Feb '21

02 Feb '21

Port 11311 has been used by email service Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/lifecycle/build | 2 +- container/lifecycle/start | 2 +- src/lifecycle/constants.cr | 2 +- src/lifecycle/lifecycle.cr | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/container/lifecycle/build b/container/lifecycle/build index ba68a85..bfb7490 100755 --- a/container/lifecycle/build +++ b/container/lifecycle/build @@ -9,7 +9,7 @@ require_relative '../defconfig.rb' names = Set.new ['LIFECYCLE_PORT'] defaults = relevant_defaults(names) -LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11311' +LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' CCI_SRC = ENV['CCI_SRC'] system "bash #{CCI_SRC}/container/compile ." diff --git a/container/lifecycle/start b/container/lifecycle/start index 7e67b3c..6e9b4fb 100755 --- a/container/lifecycle/start +++ b/container/lifecycle/start @@ -13,7 +13,7 @@ names = Set.new %w[ defaults = relevant_defaults(names) env = docker_env(defaults) -LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11311' +LIFECYCLE_PORT = defaults['LIFECYCLE_PORT'] || '11312' docker_rm "lifecycle-#{LIFECYCLE_PORT}" cmd = %W[ diff --git a/src/lifecycle/constants.cr b/src/lifecycle/constants.cr index a428251..137b7f9 100644 --- a/src/lifecycle/constants.cr +++ b/src/lifecycle/constants.cr @@ -1,4 +1,4 @@ # SPDX-License-Identifier: MulanPSL-2.0+ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. -LIFECYCLE_PORT = (ENV.has_key?("LIFECYCLE_PORT") ? ENV["LIFECYCLE_PORT"] : 11311).to_i32 +LIFECYCLE_PORT = (ENV.has_key?("LIFECYCLE_PORT") ? ENV["LIFECYCLE_PORT"] : 11312).to_i32 diff --git a/src/lifecycle/lifecycle.cr b/src/lifecycle/lifecycle.cr index 4154a27..da42fda 100644 --- a/src/lifecycle/lifecycle.cr +++ b/src/lifecycle/lifecycle.cr @@ -25,7 +25,7 @@ module Cycle end # find the testbox that are performing jobs - # curl http://localhost:11311/get_running_testbox?size=10&from=0 + # curl http://localhost:11312/get_running_testbox?size=10&from=0 get "/get_running_testbox" do |env| env.lifecycle.get_running_testbox.to_json end -- 2.23.0

1 0

[PATCH v4 compass-ci 4/4] container/submit: README file
by Luan Shengde 02 Feb '21

02 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/submit/README.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 container/submit/README.md diff --git a/container/submit/README.md b/container/submit/README.md new file mode 100644 index 0000000..6772961 --- /dev/null +++ b/container/submit/README.md @@ -0,0 +1,26 @@ +# Install lkp-tests and its depends in container. +# With docker container, will suit most of the linux OSes. +# Avoid much repetitive testing work to adapt various OSes. + +Prepare work: + apply account + config default yaml + generate ssh keys + +Installation: + git clone https://gitee.com/wu_fengguang/compass-ci.git + + cd compass-ci/container/submit + ./build + + Add soft link for executable file 'submit' to $PATH: + + ln -s $PWD/submit /usr/bin/submit + +Usage: + You can directly use the command 'submit' to submit jobs. + It's the same as you install the lkp-tests at your local host. + + Example: + + submit -c -m testbox=vm-2p8g borrow-1h.yaml -- 2.23.0

1 0