- Compass-ci - mailweb.openeuler.org

[PATCH v2 compass-ci] doc: optimize os_mount.md
by Yu Chuan 22 Feb '21

22 Feb '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 207 +++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 70 deletions(-) diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..ebe8566b9443 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@ # Summary ========= -`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type. -It has the following optional values: +The below types are supported: - nfs - cifs - initramfs @@ -11,84 +11,151 @@ It has the following optional values: - local Usage example: - - submit iperf.yaml testbox=vm-2p8g os_mount=nfs - - submit iperf.yaml testbox=vm-2p8g os_mount=cifs - - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs - - submit iperf.yaml testbox=dc-8g os_mount=container - - submit iperf.yaml testbox=vm-2p8g os_mount=local - -# Optional Values -================= - -## Work flow when os_mount=local - -1. user submit job with os_mount: local - - optional {kernel_append_root}: - - save_root_partition: give an iconic string, then the root partition data used by this job will be retained. - and you can use the root partition data of this job in the futher job by {use_root_partition}. - - use_root_partition : give an iconic string, then the root partition data generated by the - pervious job which has the {save_root_partition} field will be used. - - root_partition_size: specify the size of root partition, *default is 10G*. + ``` + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs + submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs + submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container + submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local + ``` - - tips: - - value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended. - - value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume. +# os_mount=local +================ + +## description + +If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox. + +> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}` + +> optional vars in ${kernel_append_root}: + - ${save_root_partition}, ${use_root_partition} + - purpose: + - when os_mount=local, rootfs is stored in the logical volume on the disk, + but it will be wiped by default in the preprocessing action performed by each job. + just is initrd stage. + - so we need a solution to solve scenarios that rootfs data needs to be persisted. + - how to implement: + - design a unique string for your job, it will be used as a unique symbol for your job. + the rootfs data of your job will be persistently saved to the corresponding testbox. + - so, when you need to use the rootfs data of this job in subsequent jobs, you only need + to use this unique string to submit subsequent-job.yaml, and then the subsequent-job.yaml + will be executed based on the prsistent rootfs data. + - how to use: + - in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data of + this job-20210218 so that it can be used by the subsequent jobs. So you need add the + follow field in your job-20210218.yaml. + ``` + kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218 + ``` + - in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data of + job-20210218.yaml, So you need add the follow field in your job-20210219.yaml + ``` + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 + ``` + - description: + - ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique. + - you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail. + +## work flow + +1. user submit job + job fields: + ``` + os: openeuler + os_arch: aarch64 + os_version: 20.03 + os_mount: local + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219 + ``` 2. scheduler return the ipxe_str to testbox ``` dhcp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp + kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} boot ``` 3. dracut step of boot ``` - if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { - # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 - - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 - cp -a /mnt/. /mnt1/ - umount /mnt /mnt1 - - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} - } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi + rsync_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null && return + + # create logical volume + lvcreate --size 10G --name $(basename ${SRC_LV}) os || exit + + # rsync nfsroot to ${SRC_LV} + mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt + mkdir /mnt1 && mount ${SRC_LV} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change permission of ${SRC_LV} to readonly + lvchange --permission r ${SRC_LV} + } + + check_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null || exit + } + + snapshot_boot_lv() + { + [ "$SRC_LV" == "$BOOT_LV" ] && return + + lvremove --force ${BOOT_LV} + lvcreate --size 10G --name $(basename ${BOOT_LV}) --snapshot ${SRC_LV} || exit + } + + no_save_no_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + rsync_src_lv ${SRC_LV} + } + + only_save() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + rsync_src_lv ${SRC_LV} + } + + only_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + check_src_lv ${SRC_LV} + } + + save_and_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + check_src_lv ${SRC_LV} + } + + main() + { + [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use + + [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save + + [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use + + [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use + + snapshot_boot_lv + boot_from_lv + } + + main() ``` 4. execute the job -- 2.23.0

2 4

[PATCH compass-ci] doc: update document
by Luan Shengde 22 Feb '21

22 Feb '21

rename key word: my oss commit => my_oss_commit Signed-off-by: Luan Shengde <shdluan(a)163.com> --- doc/mail-robot.md | 4 ++-- doc/manual/apply-account.md | 6 +++--- doc/manual/apply-account.zh.md | 6 +++--- 3 files changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/mail-robot.md b/doc/mail-robot.md index 6158b65..4d38e78 100644 --- a/doc/mail-robot.md +++ b/doc/mail-robot.md @@ -17,9 +17,9 @@ send mail to: compass-ci-robot(a)qq.com subject: apply account mail content - my oss commit: commit_url + my_oss_commit: commit_url example: - my oss commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… + my_oss_commit: https://github.com/torvalds/aalinux/commit/7be74942f184fdfba34ddd19a0d995de… attahcment ssh pub_key diff --git a/doc/manual/apply-account.md b/doc/manual/apply-account.md index 974fe63..542251d 100644 --- a/doc/manual/apply-account.md +++ b/doc/manual/apply-account.md @@ -4,7 +4,7 @@ You'll need an account to submit jobs. The account is mainly for tracking your r We have limited pool of testboxes, so there have to be some usage control. We offer free accounts to OSS (Open Source Software) contributors where the OSS project is in our test coverage. -You can prove yourself as OSS contributor by providing "my oss commit" in the below email. +You can prove yourself as OSS contributor by providing "my_oss_commit" in the below email. We offer free accounts to our collaborators too. If you are one of our collaborators, you should know the collaboration channel to apply account. @@ -19,7 +19,7 @@ Email template: To: compass-ci-robot(a)qq.com Subject: apply account - my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + my_oss_commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} {{ attach your ssh pubkey file(s), e.g. ~/.ssh/id_rsa.pub }} @@ -29,7 +29,7 @@ Email template: - the email name should be in English or Chinese Pinyin, as: "David Rientjes" in the template. -- my oss commit: a git commit URL that has your name and email. +- my_oss_commit: a git commit URL that has your name and email. The git repo should be in our test coverage and registered in [upstream-repos](https://gitee.com/wu_fengguang/upstream-repos). If your contributed OSS project is not in upstream-repos, you may refer to diff --git a/doc/manual/apply-account.zh.md b/doc/manual/apply-account.zh.md index 5e6605d..849364f 100644 --- a/doc/manual/apply-account.zh.md +++ b/doc/manual/apply-account.zh.md @@ -4,7 +4,7 @@ 因为目前我们的测试机资源池有限，因此需要做一些使用限制。我们为平台测试所覆盖的开源软件（OSS）的贡献者提供了免费的账号。 -你可以通过在下面邮件中提供‘my oss commit’来验明正身您是一个开源软件（OSS）的贡献者。 +你可以通过在下面邮件中提供‘my_oss_commit’来验明正身您是一个开源软件（OSS）的贡献者。我们同时也为我们的合作者提供了免费的账号。这种情况需要提供用的邮件，名字，ssh公钥，以及申请测试机的目的。 @@ -18,7 +18,7 @@ 收件人: compass-ci-robot(a)qq.com 邮件标题: apply account - my oss commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} + my_oss_commit: {{ https://github.com/torvalds/linux/commit/7be74942f184fdfba34ddd19a0d995deb3… }} {{ 将您的ssh公钥添加为邮件附件，例如 ~/.ssh/id_rsa.pub }} @@ -26,7 +26,7 @@ - 使用您的信息替换{{ }}中的内容 - 邮件名应该使用英文或中文拼音，例如模板中的“David Rientjes” -- my oss commit: 使用您的邮件和名字提交的代码的地址。 +- my_oss_commit: 使用您的邮件和名字提交的代码的地址。您的仓库需要在平台测试覆盖的范围内，并且注册到： -- 2.23.0

1 0

[PATCH lkp-tests] distro/depends/sysbench-mysql
by Wei Jihui 22 Feb '21

22 Feb '21

1. when pack pkg/sysbench-mysql, it need mysql so add libmariadb-dev(mariadb-devel) at sysbench-mysql-dev 2. when exec test/sysbench-mysql, it will install mysql at client, no need install as depends, so delete libmariadb3 at sysbench-mysql Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- distro/depends/sysbench-mysql | 1 - distro/depends/sysbench-mysql-dev | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/distro/depends/sysbench-mysql b/distro/depends/sysbench-mysql index b0d33ffe..a248fd07 100644 --- a/distro/depends/sysbench-mysql +++ b/distro/depends/sysbench-mysql @@ -1,4 +1,3 @@ libnuma1 xfsprogs ethtool -libmariadb3 diff --git a/distro/depends/sysbench-mysql-dev b/distro/depends/sysbench-mysql-dev index 65633b33..992db26d 100644 --- a/distro/depends/sysbench-mysql-dev +++ b/distro/depends/sysbench-mysql-dev @@ -1,2 +1,3 @@ +libtool pkg-config -mysql-server +libmariadb-dev -- 2.23.0

1 0

[PATCH lkp-tests] lib/job.rb: fix fs parameter parse
by Wei Jihui 22 Feb '21

22 Feb '21

[input] fs: ext4 [error] this fs will parse error by lkp-tests/params/fs fs: fs: &1 fs: ext4 [why] job.rb will parse fs twice by lkp-test/params/fs: /(.*)/{ fs: $1} this will create two fs. [how] only parse all parameters once, that is only exec expand_params once. not at the first time is for job should split array params at the first time. Signed-off-by: Wei Jihui <weijihuiall(a)163.com> --- lib/job.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/job.rb b/lib/job.rb index eacb7ec9..7fc7cb5b 100755 --- a/lib/job.rb +++ b/lib/job.rb @@ -447,7 +447,7 @@ class Job job = deepcopy(@job) job['___'] = nil @jobx = job - if @job.include?(:expand_params) + if @job.include?(:expand_params) && !first_time expand_params @job = deepcopy(@jobx) @job.delete('___') -- 2.23.0

2 1

[PATCH v2 compass-ci 1/4] container/mail-robot: add handle forwarded my_name/my_email(s)
by Luan Shengde 22 Feb '21

22 Feb '21

[why] for forwarded email from account issuer, the email content will contain my_name/my_email(s). a forwarded email may contain multi my_name/my_email(s), just loop them. the forwarded email content may like: --- my_name: name_1 my_email: email_1 account_vm: true/false/yes/no --- my_name: name_2 my_email: email_2 --- the replied 'apply account' email will return login command for account VM server if account_vm has a value: yes/true add store lab to ES Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/lib/apply-account.rb | 65 ++++++++++++++++++++--- 1 file changed, 57 insertions(+), 8 deletions(-) diff --git a/container/mail-robot/lib/apply-account.rb b/container/mail-robot/lib/apply-account.rb index e8288d7..818f900 100755 --- a/container/mail-robot/lib/apply-account.rb +++ b/container/mail-robot/lib/apply-account.rb @@ -55,6 +55,9 @@ class ApplyAccount @es_host = @send_mail_host @es_port = ES_PORT + # email info file for account issuers. + @account_issuer = YAML.load_file '/etc/compass-ci/account-issuer.yaml' + @my_info = {} end @@ -63,11 +66,35 @@ class ApplyAccount # in order to successfully send email for failed parsing # firstly get my_email before execute parse_mail_content is needed @my_info['my_email'] = @mail_content.from[0] - parse_mail_content - apply_my_account - store_account_info - send_mail('') + # for the forwarded email, it contains my_email/my_name .etc. + # all emails from account issuer will be treated as forwarded emails. + # the mail_content may like: + # --- + # my_name: name_1 + # my_name: email_1 + # account_vm: true/false/yes/no + # --- + # my_name: name_2 + # my_email: email_2 + # --- + # the forwarded email allowed to contain multi my_email/my_name(s) + # in this case, we will loop them + if @account_issuer.include? @my_info['my_email'] + users_info = forward_users + users_info.each do |user_info| + # for forwarded email for multi user, avoid rezidual information from the last. + @my_info.clear + assign_account_vm = user_info.delete('account_vm') + @my_info.update user_info + + applying_account(assign_account_vm) + sleep 5 + end + else + parse_mail_content + applying_account(false) + end rescue StandardError => e puts e.message puts e.backtrace @@ -75,6 +102,24 @@ class ApplyAccount send_mail(e.message) end + def forward_users + forward_email_content = ParseApplyAccountEmail.new(@mail_content) + users_info = forward_email_content.extract_users + + users_info.clone.each_index do |i| + users_info[i]['my_ssh_pubkey'] = [] + users_info[i]['lab'] = ENV['lab'] + end + + users_info + end + + def applying_account(assign_account_vm) + account_info = apply_my_account + store_account_info + send_mail('', account_info, assign_account_vm) + end + def parse_mail_content parse_apply_account_email = ParseApplyAccountEmail.new(@mail_content) @@ -93,7 +138,9 @@ class ApplyAccount apply_info['my_token'] = my_account_es['my_uuid'] if my_account_es['my_token'].nil? apply_info.update my_account_es apply_info.update @my_info - apply_info['my_ssh_pubkey'] = (apply_info['my_ssh_pubkey'] + my_ssh_pubkey_new).uniq + if my_ssh_pubkey_new + apply_info['my_ssh_pubkey'] = (apply_info['my_ssh_pubkey'] + my_ssh_pubkey_new).uniq + end @my_info.update apply_info apply_info['is_update_account'] = true apply_info @@ -102,6 +149,8 @@ class ApplyAccount def apply_my_account my_account_es = read_my_account_es apply_info = {} + apply_info['enable_login'] = true + if my_account_es build_apply_info(apply_info, my_account_es) else @@ -109,7 +158,6 @@ class ApplyAccount @my_info['my_token'] = my_token apply_info.update @my_info end - apply_info['lab'] = ENV['lab'] apply_new_account(apply_info, my_account_es) end @@ -118,6 +166,7 @@ class ApplyAccount acct_info = apply_account.apply_jumper_account @my_info['my_login_name'] = acct_info['my_login_name'] unless my_account_es + acct_info end def store_account_info @@ -125,9 +174,9 @@ class ApplyAccount es.put_source_by_id(@my_info['my_email'], @my_info) end - def send_mail(error_message) + def send_mail(error_message, account_info, assign_account_vm) email_message = if error_message.empty? - build_apply_account_email(@my_info) + build_apply_account_email(@my_info, account_info, assign_account_vm) else build_apply_account_fail_email(@my_info, error_message) end -- 2.23.0

2 4

[PATCH compass-ci] doc: optimize os_mount.md
by Yu Chuan 21 Feb '21

21 Feb '21

Signed-off-by: Yu Chuan <13186087857(a)163.com> --- doc/job/os_mount.md | 181 +++++++++++++++++++++++++++++--------------- 1 file changed, 120 insertions(+), 61 deletions(-) diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..b06b27237c75 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@ # Summary ========= -`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type. -It has the following optional values: +The below types are supported: - nfs - cifs - initramfs @@ -11,84 +11,143 @@ It has the following optional values: - local Usage example: - - submit iperf.yaml testbox=vm-2p8g os_mount=nfs - - submit iperf.yaml testbox=vm-2p8g os_mount=cifs - - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs - - submit iperf.yaml testbox=dc-8g os_mount=container - - submit iperf.yaml testbox=vm-2p8g os_mount=local + - submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs + - submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs + - submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs + - submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container + - submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local -# Optional Values -================= +# os_mount=local +================ -## Work flow when os_mount=local +## description -1. user submit job with os_mount: local - - optional {kernel_append_root}: - - save_root_partition: give an iconic string, then the root partition data used by this job will be retained. - and you can use the root partition data of this job in the futher job by {use_root_partition}. - - use_root_partition : give an iconic string, then the root partition data generated by the - pervious job which has the {save_root_partition} field will be used. - - root_partition_size: specify the size of root partition, *default is 10G*. +If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox. - - tips: - - value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended. - - value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume. +> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}` + +> optional vars in ${kernel_append_root}: + - ${save_root_partition}, ${use_root_partition} + - you can save the rootfs data of any job, and use it in subsequent job. + - ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique. + - you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail. + - demo kernel_append_root field of job.yaml: + ``` + # demo 1. save the rootfs data of this job + kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210219 + + # demo 2. use the rootfs data of previous job + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 + + # demo 3. use the rootfs data of previous job, and save the rootfs data of this job + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219 + ``` + +## work flow + +1. user submit job + - job fields: + - os: openeuler + - os_arch: aarch64 + - os_version: 20.03 + - os_mount: local + - kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219 2. scheduler return the ipxe_str to testbox ``` dhcp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp + kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} boot ``` 3. dracut step of boot ``` - if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { + copy_nfsroot_to_src_lv() + { + src_lv=$1 + + lvdisplay ${src_lv} > /dev/null || { # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 + lvcreate -L 10G -n $(basename ${src_lv}) os || exit 1 - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 + # rsync nfsroot to ${src_lv} + mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt + mkdir /mnt1 && mount ${src_lv} /mnt1 cp -a /mnt/. /mnt1/ umount /mnt /mnt1 - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} + # change premission of ${src_lv} to readonly + lvchange -p r ${src_lv} } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi + } + + check_src_lv() + { + lvdisplay $1 > /dev/null || exit 1 + } + + clone_boot_lv_from_src_lv() + { + src_lv=$1 + boot_lv=$2 + + lvremove -f ${boot_lv} + lvcreate ${boot_lv} from ${src_lv} || exit 1 + } + + no_save_no_use() + { + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03" + + copy_nfsroot_to_src_lv ${src_lv} + clone_boot_lv_from_src_lv ${src_lv} ${boot_lv} + boot_from_lv ${boot_lv} + } + + only_save() + { + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + copy_nfsroot_to_src_lv ${src_lv} + clone_boot_lv_from_src_lv ${src_lv} ${boot_lv} + boot_from_lv ${boot_lv} + } + + only_use() + { + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03" + + check_src_lv ${src_lv} + clone_boot_lv_from_src_lv ${src_lv} ${boot_lv} + boot_from_lv ${boot_lv} + } + + save_and_use() + { + src_lv="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + boot_lv="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + check_src_lv ${src_lv} + clone_boot_lv_from_src_lv ${src_lv} ${boot_lv} + boot_from_lv ${boot_lv} + } + + main() + { + [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use + + [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save + + [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use + + [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use + } + + main() ``` 4. execute the job -- 2.23.0

2 10

[PATCH v2 compass-ci] sbin/build-my-info: fix no key: my_ssh_pubkey
by Luan Shengde 20 Feb '21

20 Feb '21

[why] it will read my_ssh_pubkey and check it when submit jobs. $ submit -m -c job.yaml -i ~/lkp-tests/jobs/ssh-on-fail.yaml submit_id=f87d2951-c226-4e51-a3e4-c71818a4c974 submit job.yaml failed, got job id=0, error: Missing hash key: "my_ssh_pubkey" [how] add initialize my_ssh_pubkey when building my_info. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- sbin/build-my-info.rb | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/build-my-info.rb b/sbin/build-my-info.rb index c7a7798..ecfcb42 100755 --- a/sbin/build-my-info.rb +++ b/sbin/build-my-info.rb @@ -12,6 +12,7 @@ def build_my_info(option) info_es = build_my_info.search_my_info info_es.update option info_es['my_token'] = %x(uuidgen).chomp if info_es['my_token'].nil? + info_es['my_ssh_pubkey'] = [] build_my_info.config_my_info(info_es) end -- 2.23.0

1 0

[PATCH v2 compass-ci] answerback-email: add store lab to ES
by Luan Shengde 20 Feb '21

20 Feb '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/assign-account/answerback-email.rb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/container/assign-account/answerback-email.rb b/container/assign-account/answerback-email.rb index 8dceee1..f573f0f 100755 --- a/container/assign-account/answerback-email.rb +++ b/container/assign-account/answerback-email.rb @@ -40,7 +40,8 @@ my_info = { 'my_commit_url' => nil, 'my_token' => nil, 'my_login_name' => nil, - 'my_ssh_pubkey' => [] + 'my_ssh_pubkey' => [], + 'lab' => LAB } # stdin_info is used to store infos added with option: @@ -179,7 +180,6 @@ def apply_account(my_info, conf_info) apply_info = {} apply_info.update my_info apply_info.update conf_info - apply_info['lab'] = LAB assign_account_url = "#{JUMPER_HOST}:#{JUMPER_PORT}/assign_account" account_info_str = RestClient.post assign_account_url, apply_info.to_json -- 2.23.0

1 0

[PATCH compass-ci] auto-submit: use the lib script to config the account
by Cao Xueliang 20 Feb '21

20 Feb '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sbin/auto_submit | 21 ++------------------- 1 file changed, 2 insertions(+), 19 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 6d29543..b408158 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -7,8 +7,7 @@ require 'bunny' require 'json' require 'yaml' require 'set' -require_relative '../lib/build_my_info_client' -require "#{ENV['CCI_SRC']}/container/defconfig" +require_relative '../lib/config_account' # receive message and auto submit job class AutoSubmit @@ -130,23 +129,7 @@ class AutoSubmit end end -def my_email - names = Set.new %w[auto-submit] - - defaults = relevant_defaults(names) - defaults['auto-submit']['my_email'] -end - -def config_yaml - build_my_info = BuildMyInfo.new(my_email) - - my_info = build_my_info.search_my_info - build_my_info.config_default_yaml(my_info) - build_my_info.config_lab_yaml(my_info) -end - -# fix verification problems when submit jobs by auto-submit service -config_yaml +config_yaml('auto-submit') auto_submitter = AutoSubmit.new auto_submitter.listen -- 2.23.0

1 0

[PATCH v2 compass-ci 4/4] ner/mail-robot: optimize start
by Luan Shengde 20 Feb '21

20 Feb '21

attach the directory instead of the single file. Signed-off-by: Luan Shengde <shdluan(a)163.com> --- container/mail-robot/start | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/mail-robot/start b/container/mail-robot/start index 3827e2c..bd2a62d 100755 --- a/container/mail-robot/start +++ b/container/mail-robot/start @@ -31,7 +31,7 @@ cmd = %w[ ] + env + %W[ -v /c/upstream-repos:/c/upstream-repos:ro -v #{defaults['MAILDIR']}:#{defaults['MAILDIR']}:rw - -v /etc/compass-ci/conf/sec_admins.yaml:/etc/compass-ci/conf/sec_admins.yaml:ro + -v /etc/compass-ci:/etc/compass-ci:ro -v #{ENV['CCI_SRC']}:/c/compass-ci:rw -w /c/compass-ci/container/mail-robot mail-robot -- 2.23.0

1 0