Signed-off-by: Yu Chuan 13186087857@163.com --- doc/job/os_mount.md | 207 +++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 70 deletions(-)
diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..ebe8566b9443 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@ # Summary =========
-`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type.
-It has the following optional values: +The below types are supported: - nfs - cifs - initramfs @@ -11,84 +11,151 @@ It has the following optional values: - local
Usage example: - - submit iperf.yaml testbox=vm-2p8g os_mount=nfs - - submit iperf.yaml testbox=vm-2p8g os_mount=cifs - - submit iperf.yaml testbox=vm-2p8g os_mount=initramfs - - submit iperf.yaml testbox=dc-8g os_mount=container - - submit iperf.yaml testbox=vm-2p8g os_mount=local - -# Optional Values -================= - -## Work flow when os_mount=local - -1. user submit job with os_mount: local - - optional {kernel_append_root}: - - save_root_partition: give an iconic string, then the root partition data used by this job will be retained. - and you can use the root partition data of this job in the futher job by {use_root_partition}. - - use_root_partition : give an iconic string, then the root partition data generated by the - pervious job which has the {save_root_partition} field will be used. - - root_partition_size: specify the size of root partition, *default is 10G*. + ``` + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs + submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs + submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs + submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container + submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local + ```
- - tips: - - value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended. - - value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume. +# os_mount=local +================ + +## description + +If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox. + +> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}` + +> optional vars in ${kernel_append_root}: + - ${save_root_partition}, ${use_root_partition} + - purpose: + - when os_mount=local, rootfs is stored in the logical volume on the disk, + but it will be wiped by default in the preprocessing action performed by each job. + just is initrd stage. + - so we need a solution to solve scenarios that rootfs data needs to be persisted. + - how to implement: + - design a unique string for your job, it will be used as a unique symbol for your job. + the rootfs data of your job will be persistently saved to the corresponding testbox. + - so, when you need to use the rootfs data of this job in subsequent jobs, you only need + to use this unique string to submit subsequent-job.yaml, and then the subsequent-job.yaml + will be executed based on the prsistent rootfs data. + - how to use: + - in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data of + this job-20210218 so that it can be used by the subsequent jobs. So you need add the + follow field in your job-20210218.yaml. + ``` + kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218 + ``` + - in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data of + job-20210218.yaml, So you need add the follow field in your job-20210219.yaml + ``` + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 + ``` + - description: + - ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique. + - you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail. + +## work flow + +1. user submit job + job fields: + ``` + os: openeuler + os_arch: aarch64 + os_version: 20.03 + os_mount: local + kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219 + ```
2. scheduler return the ipxe_str to testbox ``` dhcp - initrd http://$%7Bhttp_server_ip%7D:$%7Bhttp_server_port%7D/os/openeuler/aarch64/20... - initrd http://$%7Bhttp_server_ip%7D:$%7Bhttp_server_port%7D/os/openeuler/aarch64/20... - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + initrd http://$%7Bhttp_server_ip%7D:$%7Bhttp_server_port%7D/os/openeuler/aarch64/20... + kernel http://$%7Bhttp_server_ip%7D:$%7Bhttp_server_port%7D/os/openeuler/aarch64/20... root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} boot ```
3. dracut step of boot ``` - if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { - # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 - - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 - cp -a /mnt/. /mnt1/ - umount /mnt /mnt1 - - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} - } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi + rsync_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null && return + + # create logical volume + lvcreate --size 10G --name $(basename ${SRC_LV}) os || exit + + # rsync nfsroot to ${SRC_LV} + mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt + mkdir /mnt1 && mount ${SRC_LV} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change permission of ${SRC_LV} to readonly + lvchange --permission r ${SRC_LV} + } + + check_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null || exit + } + + snapshot_boot_lv() + { + [ "$SRC_LV" == "$BOOT_LV" ] && return + + lvremove --force ${BOOT_LV} + lvcreate --size 10G --name $(basename ${BOOT_LV}) --snapshot ${SRC_LV} || exit + } + + no_save_no_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + rsync_src_lv ${SRC_LV} + } + + only_save() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + rsync_src_lv ${SRC_LV} + } + + only_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + check_src_lv ${SRC_LV} + } + + save_and_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + check_src_lv ${SRC_LV} + } + + main() + { + [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use + + [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save + + [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use + + [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use + + snapshot_boot_lv + boot_from_lv + } + + main() ```
4. execute the job
It looks the below logic can be further improved to
if [ -n "$use_root_partition" ]; then src_lv_suffix=_$use_root_partition else src_lv_suffix=$timestamp fi
if [ -n "$save_root_partition" ]; then boot_lv_suffix=_$save_root_partition else boot_lv_suffix= fi
local base_path=/dev/mapper/os-openeuler_aarch64_20.03 SRC_LV="${base_path}${src_lv_suffix}" BOOT_LV="${base_path}${boot_lv_suffix}"
if [ -n "$use_root_partition" ]; then check_src_lv ${SRC_LV} else rsync_src_lv ${SRC_LV} fi
snapshot_boot_lv boot_from_lv
Thanks, Fengguang
- no_save_no_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"rsync_src_lv ${SRC_LV}- }
- only_save()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"rsync_src_lv ${SRC_LV}- }
- only_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"check_src_lv ${SRC_LV}- }
- save_and_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"check_src_lv ${SRC_LV}- }
- main()
- {
[ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use[ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save[ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use[ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_usesnapshot_boot_lvboot_from_lv- }
- main()
- execute the job
-- 2.23.0
On Mon, Feb 22, 2021 at 11:31:11AM +0800, Wu Fengguang wrote:
It looks the below logic can be further improved to
if [ -n "$use_root_partition" ]; then src_lv_suffix=_$use_root_partition else src_lv_suffix=$timestamp fi if [ -n "$save_root_partition" ]; then boot_lv_suffix=_$save_root_partition else boot_lv_suffix= fi local base_path=/dev/mapper/os-openeuler_aarch64_20.03 SRC_LV="${base_path}${src_lv_suffix}" BOOT_LV="${base_path}${boot_lv_suffix}" if [ -n "$use_root_partition" ]; then check_src_lv ${SRC_LV} else rsync_src_lv ${SRC_LV} fi snapshot_boot_lv boot_from_lv
Good!
-------- Thanks Yu Chuan
Thanks, Fengguang
- no_save_no_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"rsync_src_lv ${SRC_LV}- }
- only_save()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"rsync_src_lv ${SRC_LV}- }
- only_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"check_src_lv ${SRC_LV}- }
- save_and_use()
- {
export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"check_src_lv ${SRC_LV}- }
- main()
- {
[ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use[ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save[ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use[ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_usesnapshot_boot_lvboot_from_lv- }
- main()
- execute the job
-- 2.23.0
On Mon, Feb 22, 2021 at 11:00:38AM +0800, Yu Chuan wrote:
Signed-off-by: Yu Chuan 13186087857@163.com
doc/job/os_mount.md | 207 +++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 70 deletions(-)
diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..ebe8566b9443 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@
# Summary
-`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type.
-It has the following optional values: +The below types are supported:
- nfs
- cifs
- initramfs
@@ -11,84 +11,151 @@ It has the following optional values:
- local
Usage example:
- submit iperf.yaml testbox=vm-2p8g os_mount=nfs
- submit iperf.yaml testbox=vm-2p8g os_mount=cifs
- submit iperf.yaml testbox=vm-2p8g os_mount=initramfs
- submit iperf.yaml testbox=dc-8g os_mount=container
- submit iperf.yaml testbox=vm-2p8g os_mount=local
-# Optional Values
-## Work flow when os_mount=local
-1. user submit job with os_mount: local
- optional {kernel_append_root}:
- save_root_partition: give an iconic string, then the root partition data used by this job will be retained.
and you can use the root partition data of this job in the futher job by {use_root_partition}.
- use_root_partition : give an iconic string, then the root partition data generated by the
pervious job which has the {save_root_partition} field will be used.
- root_partition_size: specify the size of root partition, *default is 10G*.
- submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs
- submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs
- submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs
- submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container
- submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local
- tips:
- value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended.- value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume.+# os_mount=local +================
+## description
+If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox.
+> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}`
NFS, initramfs problem need local disk
## local mount
os_mount=local
temp snapshot _${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version} (job use it)
## persistent local mount
${save_root_partition}, ${use_root_partition}
_${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version}_${saveuse} (job use it)
Thanks, Fengguang
+> optional vars in ${kernel_append_root}:
- ${save_root_partition}, ${use_root_partition}
- purpose:
- when os_mount=local, rootfs is stored in the logical volume on the disk,but it will be wiped by default in the preprocessing action performed by each job.just is initrd stage.- so we need a solution to solve scenarios that rootfs data needs to be persisted.
- how to implement:
- design a unique string for your job, it will be used as a unique symbol for your job.the rootfs data of your job will be persistently saved to the corresponding testbox.- so, when you need to use the rootfs data of this job in subsequent jobs, you only needto use this unique string to submit subsequent-job.yaml, and then the subsequent-job.yamlwill be executed based on the prsistent rootfs data.- ### how to use:
in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data ofthis job-20210218 so that it can be used by the subsequent jobs. So you need add thefollow field in your job-20210218.yaml.
kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218
- in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data ofjob-20210218.yaml, So you need add the follow field in your job-20210219.yaml```kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218```
- description:
- ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique.- you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail.+## work flow
+1. user submit job
- job fields:
- os: openeuler
- os_arch: aarch64
- os_version: 20.03
- os_mount: local
- kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219
- scheduler return the ipxe_str to testbox
dhcp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp + kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} boot
- dracut step of boot
- if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { - # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 - - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 - cp -a /mnt/. /mnt1/ - umount /mnt /mnt1 - - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} - } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi + rsync_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null && return + + # create logical volume + lvcreate --size 10G --name $(basename ${SRC_LV}) os || exit + + # rsync nfsroot to ${SRC_LV} + mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt + mkdir /mnt1 && mount ${SRC_LV} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change permission of ${SRC_LV} to readonly + lvchange --permission r ${SRC_LV} + } + + check_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null || exit + } + + snapshot_boot_lv() + { + [ "$SRC_LV" == "$BOOT_LV" ] && return + + lvremove --force ${BOOT_LV} + lvcreate --size 10G --name $(basename ${BOOT_LV}) --snapshot ${SRC_LV} || exit + } + + no_save_no_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + rsync_src_lv ${SRC_LV} + } + + only_save() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + rsync_src_lv ${SRC_LV} + } + + only_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + check_src_lv ${SRC_LV} + } + + save_and_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + check_src_lv ${SRC_LV} + } + + main() + { + [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use + + [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save + + [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use + + [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use + + snapshot_boot_lv + boot_from_lv + } + + main()
- execute the job
-- 2.23.0
On Mon, Feb 22, 2021 at 02:54:28PM +0800, Wu Fengguang wrote:
On Mon, Feb 22, 2021 at 11:00:38AM +0800, Yu Chuan wrote:
Signed-off-by: Yu Chuan 13186087857@163.com
doc/job/os_mount.md | 207 +++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 70 deletions(-)
diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..ebe8566b9443 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@
# Summary
-`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type.
-It has the following optional values: +The below types are supported:
- nfs
- cifs
- initramfs
@@ -11,84 +11,151 @@ It has the following optional values:
- local
Usage example:
- submit iperf.yaml testbox=vm-2p8g os_mount=nfs
- submit iperf.yaml testbox=vm-2p8g os_mount=cifs
- submit iperf.yaml testbox=vm-2p8g os_mount=initramfs
- submit iperf.yaml testbox=dc-8g os_mount=container
- submit iperf.yaml testbox=vm-2p8g os_mount=local
-# Optional Values
-## Work flow when os_mount=local
-1. user submit job with os_mount: local
- optional {kernel_append_root}:
- save_root_partition: give an iconic string, then the root partition data used by this job will be retained.
and you can use the root partition data of this job in the futher job by {use_root_partition}.
- use_root_partition : give an iconic string, then the root partition data generated by the
pervious job which has the {save_root_partition} field will be used.
- root_partition_size: specify the size of root partition, *default is 10G*.
- submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs
- submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs
- submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs
- submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container
- submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local
- tips:
- value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended.- value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume.+# os_mount=local +================
+## description
+If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox.
+> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}`
NFS, initramfs problem need local disk
## local mount
os_mount=local
temp snapshot _${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version} (job use it)
## persistent local mount
${save_root_partition}, ${use_root_partition}
_${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version}_${saveuse} (job use it)
ok
-------- Thanks Yu Chuan
Thanks, Fengguang
+> optional vars in ${kernel_append_root}:
- ${save_root_partition}, ${use_root_partition}
- purpose:
- when os_mount=local, rootfs is stored in the logical volume on the disk,but it will be wiped by default in the preprocessing action performed by each job.just is initrd stage.- so we need a solution to solve scenarios that rootfs data needs to be persisted.
- how to implement:
- design a unique string for your job, it will be used as a unique symbol for your job.the rootfs data of your job will be persistently saved to the corresponding testbox.- so, when you need to use the rootfs data of this job in subsequent jobs, you only needto use this unique string to submit subsequent-job.yaml, and then the subsequent-job.yamlwill be executed based on the prsistent rootfs data.- ### how to use:
in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data ofthis job-20210218 so that it can be used by the subsequent jobs. So you need add thefollow field in your job-20210218.yaml.kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218
- in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data ofjob-20210218.yaml, So you need add the follow field in your job-20210219.yaml```kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218```
- description:
- ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique.- you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail.+## work flow
+1. user submit job
- job fields:
- os: openeuler
- os_arch: aarch64
- os_version: 20.03
- os_mount: local
- kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219
- scheduler return the ipxe_str to testbox
dhcp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp - initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz - imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root} + initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp + kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root} boot
- dracut step of boot
- if have {use_root_partition}; then - base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1 - - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - - if {save_root_partition} == {use_root_partition}; then - boot from /dev/mappaer/os-{boot_lv_name} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - else - boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition} - boot from /dev/mappaer/os-{boot_lv_name} - fi - - else: - base_lv_name=openeuler_aarch64_20.03_{timestamp} - lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || { - # create logical volume - lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1 - - # rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name} - mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt - mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1 - cp -a /mnt/. /mnt1/ - umount /mnt /mnt1 - - # change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly - lvchange -p r /dev/mapper/os-{base_lv_name} - } - - boot_lv_name=openeuler_aarch64_20.03 - if have {save_root_partition}; then - boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition} - fi - - lvremove -f /dev/mappaer/os-{boot_lv_name} - lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1 - boot from /dev/mappaer/os-{boot_lv_name} - fi + rsync_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null && return + + # create logical volume + lvcreate --size 10G --name $(basename ${SRC_LV}) os || exit + + # rsync nfsroot to ${SRC_LV} + mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt + mkdir /mnt1 && mount ${SRC_LV} /mnt1 + cp -a /mnt/. /mnt1/ + umount /mnt /mnt1 + + # change permission of ${SRC_LV} to readonly + lvchange --permission r ${SRC_LV} + } + + check_src_lv() + { + lvdisplay ${SRC_LV} > /dev/null || exit + } + + snapshot_boot_lv() + { + [ "$SRC_LV" == "$BOOT_LV" ] && return + + lvremove --force ${BOOT_LV} + lvcreate --size 10G --name $(basename ${BOOT_LV}) --snapshot ${SRC_LV} || exit + } + + no_save_no_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + rsync_src_lv ${SRC_LV} + } + + only_save() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + rsync_src_lv ${SRC_LV} + } + + only_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03" + + check_src_lv ${SRC_LV} + } + + save_and_use() + { + export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}" + export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}" + + check_src_lv ${SRC_LV} + } + + main() + { + [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use + + [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save + + [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use + + [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use + + snapshot_boot_lv + boot_from_lv + } + + main()
- execute the job
-- 2.23.0
-
Wu Fengguang -
Yu Chuan