February 2021 - Compass-ci - mailweb.openeuler.org

[PATCH v3 compass-ci] doc: optimize os_mount.md
by Yu Chuan 23 Feb '21

23 Feb '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 263 ++++++++++++++++++++++++++++++-------------- 1 file changed, 179 insertions(+), 84 deletions(-) diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..0d0afd6469a8 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,94 +1,189 @@ -# Summary -========= +Summary +======= -`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type. -It has the following optional values: - - nfs - - cifs - - initramfs - - container - - local +The below types are supported: + - nfs + - cifs + - initramfs + - container + - local Usage example: - - submit iperf.yaml testbox=vm-2p8g os_mount=nfs - - submit iperf.yaml testbox=vm-2p8g os_mount=cifs - - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs - - submit iperf.yaml testbox=dc-8g os_mount=container - - submit iperf.yaml testbox=vm-2p8g os_mount=local - -# Optional Values -================= - -## Work flow when os_mount=local - -1. user submit job with os_mount: local - - optional {kernel_append_root}: - - save_root_partition: give an iconic string, then the root partition data used by this job will be retained. - and you can use the root partition data of this job in the futher job by {use_root_partition}. - - use_root_partition : give an iconic string, then the root partition data generated by the - pervious job which has the {save_root_partition} field will be used. - - root_partition_size: specify the size of root partition, *default is 10G*. - - - tips: - - value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended. - - value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume. + ``` + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs + submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs + submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container + submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local + ``` + +os_mount=local +============== + +`nfs`, `cifs`, `initramfs`, `container`, rootfs of them all are in memory. +So if you want to run your job in disk file system, you need to specify +`os_mount=local` in your job.yaml. + +BTW, the job's rootfs will be placed on a logical volume. + +## description + +### temporary rootfs data *Defaultly* + +If you only add `os_mount: local` in your job.yaml, the logical volume of rootfs +will be deleted and re-created in initrd stage before every job executes. + +The brief flow is as follow: + ``` + 1. boot and request scheduler for job. + 2. initrd stage: + - firstly, we need two logical volume: + ${src_lv} -- src logical volume: + - will never boot form this lv, just use it as a source data, and take a + snapshot of it to get the real boot logical volume. + - srv_lv=/dev/mapper/os-${os}_${os_arch}_${os_version}_${timestamp} + ${boot_lv} -- boot logical volume: + - will boot from this. + - boot_lv=/dev/mapper/os-${os}_${os_arch}_${os_version} + - if ${src_lv} not exists: create it, and rsync the rootfs from cluster nfs server. + - if ${boot_lv} exists: delete it + - create ${boot_lv} as the snapshot of ${src_lv}. + - switch root to ${boot_lv}. + 3. boot the rootfs and execute the job. + ``` + +### persistent rootfs data + +When you need to persist the rootfs data of a job, and use it in the subsequent job(s), +two fields in `kernel_append_root` will help you: `save_root_partition`, `use_root_partition`. + +The brief flow is as follow: + ``` + 1. boot and request scheduler for job. + 2. initrd stage: + - firstly, we need two logical volume: + ${src_lv} -- src logical volume: + - if have ${use_root_partition}, src_lv=/dev/mapper/os-${os}_${os_arch}_${os_version}_${use_root_partition} + - if no ${use_root_partition}, src_lv=/dev/mapper/os-${os}_${os_arch}_${os_version}_${timestamp} + ${boot_lv} -- boot logical volume: + - if have ${save_root_partition}, boot_lv=/dev/mapper/os-${os}_${os_arch}_${os_version}_${save_root_partition} + - if no ${save_root_partition}, boot_lv=/dev/mapper/os-${os}_${os_arch}_${os_version} + - if ${src_lv} not exists: + - if have ${use_root_partition}, exit 1. + - if no ${use_root_partition}, create ${src_lv}, and rsync the rootfs from cluster nfs server. + - if ${boot_lv} != ${src_lv}: + - if ${boot_lv} exists: delete it + - create ${boot_lv} as the snapshot of ${src_lv}. + - switch root to ${boot_lv}. + 3. boot the rootfs and execute the job. + ``` + +Demo usage: + ``` + - in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data of + job-20210218.yaml so that it can be used by the subsequent jobs. + Then you need add the follow field in your job-20210218.yaml: + kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218 + + - in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data of + job-20210218.yaml. + Then you need add the follow field in your job-20210219.yaml: + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 + ``` + +Notes: + - obviously, ${save_root_partition} and ${use_root_partition} must be the same, and unique. + - you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail. + +## work flow + +1. user submit job + job fields: + ``` + os: openeuler + os_arch: aarch64 + os_version: 20.03 + os_mount: local + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219 + ``` 2. scheduler return the ipxe_str to testbox - ``` - dhcp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} - boot - ``` + ``` + dhcp + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp + kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} + boot + ``` 3. dracut step of boot - ``` - if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { - # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 - - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 - cp -a /mnt/. /mnt1/ - umount /mnt /mnt1 - - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} - } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi - ``` + ``` + analysis_kernel_cmdline_params() + { + for i in $(cat /proc/cmdline) + do + [ "$i" =~ "rootfs_src=" ] && export ROOTFS_SRC=${i#rootfs_src=} + done + + [ -z "${ROOTFS_SRC}" ] && die "cannot find var in kernel cmdline params: rootfs_src" + + export TIMESTAMP=$(basename ${ROOTFS_SRC}) + } + + rsync_src_lv() + { + local src_lv=$1 + + lvdisplay ${src_lv} > /dev/null && return + + # create logical volume + lvcreate --size 10G --name $(basename ${src_lv}) os || exit + + # rsync nfsroot to ${src_lv} + mount -t nfs ${ROOTFS_SRC} /mnt + mkdir /mnt1 && mount ${src_lv} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change permission of ${src_lv} to readonly + lvchange --permission r ${src_lv} + } + + snapshot_boot_lv() + { + local src_lv=$1 + local boot_lv=$2 + + [ "$src_lv" == "$boot_lv" ] && return + + lvremove --force ${boot_lv} + lvcreate --size 10G --name $(basename ${boot_lv}) --snapshot ${src_lv} || exit + } + + main() + { + analysis_kernel_cmdline_params + + if [ -z "${use_root_partition}" ]; then + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${TIMESTAMP}" + rsync_src_lv ${src_lv} + else + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + lvdisplay ${src_lv} > /dev/null || exit + fi + + if [ -z "${save_root_partition}" ]; then + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03" + else + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + fi + + snapshot_boot_lv ${src_lv} ${boot_lv} + boot_from_boot_lv + } + + main() + ``` 4. execute the job -- 2.23.0

2 2

[PATCH v4 lkp-tests] stats/ansible_test: stats result for ansible migrating
by Wang Chenglong 23 Feb '21

23 Feb '21

[How] ansible_stats_openeuler < ${job_output} Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- stats/ansible_test | 135 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 124 insertions(+), 11 deletions(-) diff --git a/stats/ansible_test b/stats/ansible_test index 4c7d07b3e..bd60a3ebc 100755 --- a/stats/ansible_test +++ b/stats/ansible_test @@ -1,15 +1,128 @@ #!/usr/bin/env ruby -while (line = STDIN.gets) - case line.chomp! - when /ok=(\d+)\s+changed=(\d+)\s+unreachable=(\d+)\s+failed=(\d+)\s+skipped=(\d+)\s+rescued=(\d+)\s+ignored=(\d+)/ - puts "ansible_test.total.nr_ok: #{$1}" - puts "ansible_test.total.nr_changed: #{$2}" - puts "ansible_test.total.nr_failed: #{$4}" - puts "ansible_test.total.nr_skipped: #{$5}" - puts "ansible_test.total.nr_rescued: #{$6}" - puts "ansible_test.total.nr_ignored: #{$7}" - when /playbook_run_on_fail/ - puts line +require 'json' + +def parse_msg_1(ansible_failed_json) + case ansible_failed_json.chomp + + # When running some plug-in, the file was not found + when /No file was found when using (.+)\./ + puts "files.error.no-file-was-found-when-using.#{$1}: 1" + puts "files.error.no-file-was-found-when-using.#{$1}.message: #{@ansible_failed_info}" + + # {"changed": false, "cmd": "csf -v", "msg": "[Errno 2] No such file or directory: b'csf': b'csf'", "rc": 2} + when /No such file or directory: b (.+) :/ + puts "error.no-such-file-or-directory.#{$1}: 1" + puts "error.no-such-file-or-directory.#{$1}.message: #{@ansible_failed_info}" + + # The task includes an option with an undefined variable + when /: (.+) is undefined/ + puts "syntax.error.an-undefined-variable.#{$1}: 1" + puts "syntax.error.an-undefined-variable.#{$1}.message: #{@ansible_failed_info}" + + # {"changed": true, "cmd": ["packaging/installer/install-required-packages.sh", "--non-interactive", "netdata"], + # "msg": "non-zero return code", "rc": 1} + when /non-zero return code/ + puts "error.non-zero-return-code.#{ansible_failed_json}: 1" + puts "error.non-zero-return-code.#{ansible_failed_json}.message: #{@ansible_failed_info}" + + # Some service could not find the requested host + when /Could not find the requested service (.+): host/ + puts "request.error.could-not-find-service-host.#{$1}: 1" + puts "request.error.could-not-find-service-host.#{$1}.messgae: #{@ansible_failed_info}" + + # Unable to start some services + when /Unable to start service (.+):/ + puts "service.error.unable-to-start-service.#{$1}: 1" + puts "service.error.unable-to-start-service.#{$1}.message: #{@ansible_failed_info}" + end +end + +def parse_msg_2(ansible_failed_json) + case ansible_failed_json.chomp + + # {"changed": false, "msg": "Failed to download packages: Cannot download 7.10.2/filebeat-7.10.2-aarch64.rpm: + # All mirrors were tried", "results": []} + when /Failed to download packages: Cannot download (.+):/ + puts "download.error.failed-to-download-package.#{$1}: 1" + puts "download.error.failed-to-download-package.#{$1}.message: #{@ansible_failed_info}" + + # {"changed": false, "msg": "Failure downloading https://rpms.remirepo.net/enterprise/remi-release-20.rpm, + # HTTP Error 404: Not Found"} + when /Failure downloading (.+), / + puts "download.error.Failure-downloading.[#{$1}]: 1" + puts "download.error.Failure-downloading.[#{$1}]: #{@ansible_failed_info}" + + # Failed to download the yum repodata + # {"changed": false, "msg": "Failed to download metadata for repo 'docker-ce-stable': Cannot download repomd.xml", + # "rc": 1, "results": []} + when /Failed to download metadata for repo (.+) :/ + puts "download.error.failed-to-download-metadata.#{$1}: 1" + puts "download.error.failed-to-download-metadata.#{$1}.message: #{@ansible_failed_info}" + + # Some role in not sipported this system + # {"changed": false, "msg": "Distribution openEuler is not supported by this role!"} + when /(.+) is not supported/ + puts "system.error.not-supported.#{$1.gsub(/ /, '-')}: 1" + puts "system.error.not-supported.message: #{@ansible_failed_info}" + + # failed to create temporary content file: The read operation timed out. + when /The read operation timed out/ + puts 'connection.error.read-timed-out: 1' + puts "connection.error.read-timed-out.message: #{@ansible_failed_info}" end end + +def parse_failures(ansible_failed_json) + ansible_failed_json.each do |items| + case items.chomp + # Failed to install some of the specified packages. + # {"changed": false, "failures": ["No package erlang available."], "msg": + # "Failed to install some of the specified packages", "rc": 1, "results": []} + when /No package (.+) available/ + puts "yum.error.no-package-available.#{$1}: 1" + puts "yum.error.no-package-available.#{$1}.message: #{@ansible_failed_info}" + end + end +end + +def parse_message(ansible_failed_json) + case ansible_failed_json.chomp + + # Could not find or access ansible yaml + # {"ansible_facts": {}, "ansible_included_var_files": [], "changed": false, "message": + # "Could not find or access 'openEuler.yml' Searched in: + # /root/.ansible/roles/ansible-unzip/vars/openEuler.yml /root/.ansible/roles/ansible-unzip/openEuler.yml i + # /root/.ansible/roles/ansible-unzip/tasks/vars/openEuler.yml /root/.ansible/roles/ansible-unzip/tasks/openEuler.yml + # /root/.ansible/vars/openEuler.yml /root/.ansible/openEuler.yml on the Ansible Controller. + # If you are using a module and expect the file to exist on the remote, see the remote_src option"} + when /Could not find or access (.+) / + puts "yaml.error.not-find-or-access-yaml.#{$1}: 1" + puts "yaml.error.not-find-or-access-yaml.#{$1}.message: #{@ansible_failed_info}" + end +end + +def common_error_id(line) + line = line.chomp + line.gsub(/'/, ' ') + line.gsub(/{%/, ' ') + line.gsub(/%}/, ' ') + line.gsub(/{},/, '[],') + line.gsub(/\\n\\t/, ' ') + line +end + +while (line = STDIN.gets) + line = common_error_id(line) + next unless line =~ /(FAILED!|failed:).*({.*})/ + + @ansible_failed_info = $2 + next if @ansible_failed_info.empty? + + ansible_failed_json = JSON.load(@ansible_failed_info) + + parse_msg_1 ansible_failed_json['msg'] unless ansible_failed_json['msg'].nil? + parse_msg_2 ansible_failed_json['msg'] unless ansible_failed_json['msg'].nil? + parse_failures ansible_failed_json['failures'] unless ansible_failed_json['failures'].nil? + parse_message ansible_failed_json['message'] unless ansible_failed_json['message'].nil? +end -- 2.23.0

4 6

[PATCH lkp-tests] lib/job2sh.rb: delete secrets field when job2sh
by Cao Xueliang 23 Feb '21

23 Feb '21

We don't show the secrets field in job.sh file, so delete secrets field when job2sh. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/job2sh.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/job2sh.rb b/lib/job2sh.rb index 7f9cf562f..82ec7d249 100755 --- a/lib/job2sh.rb +++ b/lib/job2sh.rb @@ -318,6 +318,7 @@ class Job2sh < Job unless job job = (@jobx || @job).clone # a shallow copy so that delete_if won't impact @job job.delete_if { |key, val| parse_one([], key, val, :PASS_EXPORT_ENV) } + job.delete_if { |key, val| key == "secrets" } end @cur_func = :run_job -- 2.23.0

1 0

[PATCH compass-ci] sched: add the secrets field processing logic
by Cao Xueliang 23 Feb '21

23 Feb '21

You can add the secrets field which you don't want to make public but the SUT need it in job.yaml. secrets: my_token:xxx The scheduler will save the secrets value to the redis id2secrets table, then delete the field from job.yaml. When consume the job, the scheduler will read the value from redis, then create a secrets.yaml in job.cgz. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sbin/create-job-cpio.sh | 1 + src/scheduler/create_job_cpio.cr | 13 +++++++++++++ src/scheduler/submit_job.cr | 7 +++++++ 3 files changed, 21 insertions(+) diff --git a/sbin/create-job-cpio.sh b/sbin/create-job-cpio.sh index e18bc0d..afb316c 100755 --- a/sbin/create-job-cpio.sh +++ b/sbin/create-job-cpio.sh @@ -9,6 +9,7 @@ cd "$1" || exit install -m775 -D -t lkp/scheduled job.sh install -m664 -D -t lkp/scheduled job.yaml +[ -f secrets.yaml ] && install -m664 -D -t lkp/scheduled secrets.yaml find lkp | cpio --quiet -o -H newc | gzip > job.cgz diff --git a/src/scheduler/create_job_cpio.cr b/src/scheduler/create_job_cpio.cr index 51d79f7..5634f67 100644 --- a/src/scheduler/create_job_cpio.cr +++ b/src/scheduler/create_job_cpio.cr @@ -74,8 +74,21 @@ class Sched script_lines = JSON.parse(script_lines) end + def create_secrets_yaml(job_id, base_dir) + secrets = @redis.hash_get("id2secrets", job_id) + return nil unless secrets + + secrets_yaml = base_dir + "/#{job_id}/secrets.yaml" + prepare_dir(secrets_yaml) + + File.open(secrets_yaml, "w") do |file| + YAML.dump(JSON.parse(secrets), file) + end + end + def create_job_cpio(job_content : JSON::Any, base_dir : String) job_content = job_content.as_h + create_secrets_yaml(job_content["id"], base_dir) # put job2sh in an array if job_content.has_key?("job2sh") diff --git a/src/scheduler/submit_job.cr b/src/scheduler/submit_job.cr index 1837cb1..fe9efa1 100644 --- a/src/scheduler/submit_job.cr +++ b/src/scheduler/submit_job.cr @@ -144,8 +144,15 @@ class Sched JSON.parse(response[1].to_json)["id"].to_s if response[0] == 200 end + def save_secrets(job, job_id) + return nil unless job["secrets"]? + + @redis.hash_set("id2secrets", job_id, job["secrets"]?.to_json) + job.delete("secrets") + end # add job content to es and return a response def add_job(job, job_id) + save_secrets(job, job_id) job.update_id(job_id) @es.set_job_content(job) end -- 2.23.0

1 0

[PATCH lab-z9] add muti-qemu hosts and alter the multi-qemu.yaml
by Bai Jing 23 Feb '21

23 Feb '21

Signed-off-by: Bai Jing <799286817(a)qq.com> --- allot/idle/taishan200-2280-2s48p-256g--a63 | 1 + allot/idle/taishan200-2280-2s48p-256g--a64 | 1 + allot/idle/taishan200-2280-2s48p-256g--a65 | 1 + allot/idle/taishan200-2280-2s48p-256g--a66 | 1 + allot/idle/taishan200-2280-2s48p-256g--a67 | 1 + allot/idle/taishan200-2280-2s48p-256g--a68 | 1 + allot/idle/taishan200-2280-2s48p-256g--a69 | 1 + allot/idle/taishan200-2280-2s48p-256g--a70 | 1 + .../multi-qemu.yaml | 15 ++++++++++++--- 9 files changed, 20 insertions(+), 3 deletions(-) create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a63 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a64 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a65 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a66 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a67 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a68 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a69 create mode 120000 allot/idle/taishan200-2280-2s48p-256g--a70 diff --git a/allot/idle/taishan200-2280-2s48p-256g--a63 b/allot/idle/taishan200-2280-2s48p-256g--a63 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a63 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a64 b/allot/idle/taishan200-2280-2s48p-256g--a64 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a64 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a65 b/allot/idle/taishan200-2280-2s48p-256g--a65 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a65 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a66 b/allot/idle/taishan200-2280-2s48p-256g--a66 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a66 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a67 b/allot/idle/taishan200-2280-2s48p-256g--a67 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a67 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a68 b/allot/idle/taishan200-2280-2s48p-256g--a68 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a68 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a69 b/allot/idle/taishan200-2280-2s48p-256g--a69 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a69 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--a70 b/allot/idle/taishan200-2280-2s48p-256g--a70 new file mode 120000 index 0000000..3c3d0c3 --- /dev/null +++ b/allot/idle/taishan200-2280-2s48p-256g--a70 @@ -0,0 +1 @@ +taishan200-2280-2s48p-256g--multi-qemu \ No newline at end of file diff --git a/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml b/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml index ed55c08..87f684e 100644 --- a/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml +++ b/allot/idle/taishan200-2280-2s48p-256g--multi-qemu/multi-qemu.yaml @@ -12,14 +12,23 @@ fs: simplify-ci: multi-qemu-0: - nr_vm: 20 - tbox_group: vm-2p8g + nr_vm: 2 + tbox_group: vm-1p1g multi-qemu-1: nr_vm: 2 tbox_group: vm-2p4g multi-qemu-2: + nr_vm: 12 + tbox_group: vm-2p8g +multi-qemu-3: + nr_vm: 6 + tbox_group: vm-2p16g +multi-qemu-4: nr_vm: 2 - tbox_group: vm-1p1g + tbox_group: vm-2p32g +multi-qemu-5: + nr_vm: 2 + tbox_group: vm-4p64g sleep: 100d os: openeuler -- 2.23.0

1 0

[PATCH lkp-tests] setup/simplify-ci: git failed to exit directly
by Cui Yili 23 Feb '21

23 Feb '21

Signed-off-by: Cui Yili <2268260388(a)qq.com> --- setup/simplify-ci | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/setup/simplify-ci b/setup/simplify-ci index c8f8a5a53..5aede9cdb 100755 --- a/setup/simplify-ci +++ b/setup/simplify-ci @@ -14,7 +14,7 @@ git_ci() EOF fi - git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci + git clone https://gitee.com/wu_fengguang/compass-ci.git /c/compass-ci || return 1 } dev_env() @@ -68,7 +68,7 @@ run_service() main() { - git_ci + git_ci || return 1 install_env run_service } -- 2.23.0

1 0

[PATCH compass-ci] sbin/auto_submit: Adapting to new config style of DEFAULTS files
by Li Yuanchao 23 Feb '21

23 Feb '21

As the config style of DEFAULTS files in upstream-repos changes, the auto_submit need change to adapt to it. Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 53 ++++++++++++++++-------------------------------- 1 file changed, 18 insertions(+), 35 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index b408158..857af29 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -57,28 +57,13 @@ class AutoSubmit system(real_argvs.join(' ')) end - def get_branch_commands(submit_commands) - submit_commands.each do |element| - return element['branch_commands'] if element['branch_commands'] - end - return nil - end + def submit_specific_branch(submit_argv, newrefs_info, branches) + branches.each do |branch_name| + branch = "refs/heads/#{branch_name}" + next unless newrefs_info['new_refs']['heads'][branch] - def submit_specific_branch(submit_argv, newrefs_info, branch_commands) - return newrefs_info['new_refs']['heads'] if branch_commands.nil? - - non_specific_commits = {} - newrefs_info['new_refs']['heads'].each do |branch, commit_id| - branch_name = branch.delete_prefix('refs/heads/') - if branch_commands[branch_name].nil? - non_specific_commits[branch] = commit_id - next - end - argvs = Array.new(submit_argv) - argvs.push(branch_commands[branch_name]) - submit_one_job(argvs, newrefs_info['git_repo'], branch_name, commit_id) + submit_one_job(submit_argv, newrefs_info['git_repo'], branch_name, newrefs_info['new_refs']['heads'][branch]) end - return non_specific_commits end def submit_non_specific(non_specific_commits, git_repo, submit_argv) @@ -90,13 +75,13 @@ class AutoSubmit # Add support of different branch different job # newrefs_info['submit'] is like: - # ["submit command 1", - # "submit command 2", + # [ + # { "command" => "submit command 1", + # "branches" => [ "master", "next", ... ] + # }, # ..., - # { "branch_commands" => { "branch_1" => "submit command 1", - # ..., - # "branch_n" => "submit command n" - # } + # { "command" => "submit command n", + # "branches" => [ "branch_name_a", "branch_name_b", ... ] # } # ] def submit_job(newrefs_info) @@ -105,16 +90,14 @@ class AutoSubmit return unless newrefs_info['submit'] - branch_commands = get_branch_commands(newrefs_info['submit']) - non_specific_commits = submit_specific_branch(submit_argv, newrefs_info, branch_commands) - return if non_specific_commits.empty? - - newrefs_info['submit'].each do |argv_config| - next if argv_config.is_a? Hash - + newrefs_info['submit'].each do |element| argvs = Array.new(submit_argv) - argvs.push(argv_config) - submit_non_specific(non_specific_commits, newrefs_info['git_repo'], argvs) + argvs.push(element['command']) + if element['branches'] + submit_specific_branch(argvs, newrefs_info, element['branches']) + else + submit_non_specific(newrefs_info['new_refs']['heads'], newrefs_info['git_repo'], argvs) + end end end -- 2.23.0

1 0

[PATCH lkp-tests] tests/rpmbuild-pkg: redirect the sync_dest directory
by Li Ping 23 Feb '21

23 Feb '21

Signed-off-by: Li Ping <1477412247(a)qq.com> --- tests/rpmbuild-pkg | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/rpmbuild-pkg b/tests/rpmbuild-pkg index 9f455f0d0..38de16e66 100755 --- a/tests/rpmbuild-pkg +++ b/tests/rpmbuild-pkg @@ -6,6 +6,7 @@ [ -n "$rpm_repo" ] || die "rpm_repo is empty" package_name=${rpm_repo##*/} +[[ "$os_mount" = "cifs" ]] && os_mount="nfs" sync_dest="/initrd/rpmbuild-pkg/${os_mount}/${os}/${os_arch}/${os_version}/${package_name}" init_workspace() -- 2.23.0

1 0

[PATCH v2 lkp-tests] jobs/: add os-arch-version-mount.yaml for multi-dimension test
by Li Ping 23 Feb '21

23 Feb '21

1 1

[PATCH compass-ci] container/scheduler/start: fix read config
by Wei Jihui 23 Feb '21

23 Feb '21

scheduler service need read config from /etc/compass-ci/[service|scheduler], not only /etc/compass-ci/default. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- container/scheduler/start | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/container/scheduler/start b/container/scheduler/start index c32a7a8..4aaa71d 100755 --- a/container/scheduler/start +++ b/container/scheduler/start @@ -46,7 +46,6 @@ DEFAULT_CCI = "#{CCI_REPOS}/compass-ci" LAB = defaults['lab'] || 'z9' SCHED_PORT = defaults['SCHED_PORT'] SCHED_HOST = defaults['SCHED_HOST'] -DEFAULT_CONFIG_DIR = '/etc/compass-ci/defaults' docker_rm "s001-alpine-#{SCHED_PORT}" @@ -68,9 +67,9 @@ cmd = %W[ -e CCI_SRC=#{DEFAULT_CCI} -v #{ENV['LKP_SRC']}:#{DEFAULT_LKP} -v #{ENV['CCI_SRC']}:#{DEFAULT_CCI} - -v #{DEFAULT_CONFIG_DIR}:#{DEFAULT_CONFIG_DIR}:ro -v #{CCI_REPOS}/lab-#{LAB}:#{CCI_REPOS}/lab-#{LAB} -v /srv/cci/scheduler/alpine:/srv/cci/scheduler + -v /etc/compass-ci:/etc/compass-ci:ro -v /etc/localtime:/etc/localtime:ro -v /srv/result:/srv/result -v /srv/initrd:/srv/initrd:ro -- 2.23.0

1 0