On Mon, Feb 22, 2021 at 02:54:28PM +0800, Wu Fengguang wrote:

On Mon, Feb 22, 2021 at 11:00:38AM +0800, Yu Chuan wrote:

...

Signed-off-by: Yu Chuan 13186087857@163.com

doc/job/os_mount.md | 207 +++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 70 deletions(-)

diff --git a/doc/job/os_mount.md b/doc/job/os_mount.md index 226251a8bace..ebe8566b9443 100644 --- a/doc/job/os_mount.md +++ b/doc/job/os_mount.md @@ -1,9 +1,9 @@

# Summary

-`os_mount` defines the mount method of testbox's root partition. +`os_mount` defines the rootfs type.

-It has the following optional values: +The below types are supported:

• nfs
• cifs
• initramfs

@@ -11,84 +11,151 @@ It has the following optional values:

• local

Usage example:

• • submit iperf.yaml testbox=vm-2p8g os_mount=nfs
• • submit iperf.yaml testbox=vm-2p8g os_mount=cifs
• • submit iperf.yaml testbox=vm-2p8g os_mount=initramfs
• • submit iperf.yaml testbox=dc-8g os_mount=container
• • submit iperf.yaml testbox=vm-2p8g os_mount=local

-# Optional Values

-## Work flow when os_mount=local

-1. user submit job with os_mount: local

• • optional {kernel_append_root}:
• • save_root_partition: give an iconic string, then the root partition data used by this job will be retained.

•                       and you can use the root partition data of this job in the futher job by {use_root_partition}.
  

• • use_root_partition : give an iconic string, then the root partition data generated by the

•                       pervious job which has the {save_root_partition} field will be used.
  

• • root_partition_size: specify the size of root partition, *default is 10G*.

• submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=nfs
• submit -m -c borrow-1h.yaml testbox=vm-2p8g os_mount=cifs
• submit -m -c borrow-1h.yaml testbox=vm-2p16g os_mount=initramfs
• submit -m -c borrow-1h.yaml testbox=dc-8g os_mount=container
• submit -m -c borrow-1h.yaml testbox=taishan200-2280-2s48p-256g os_mount=local

• • tips:

•  - value of {save_root_partition} and {use_root_partition} need to be iconic enough, uuid is recommended.
  

•  - value of {save_root_partition} and {use_root_partition} will be the suffix of logical volume.
  

+# os_mount=local +================

+## description

+If you specifies os_mount=local, the rootfs type of your job will be a logical volume in the testbox.

+> logical volume format: `/dev/mapper/os-${os}_${os_arch}_${os_version}`

NFS, initramfs problem need local disk

## local mount

os_mount=local

temp snapshot _${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version} (job use it)

## persistent local mount

${save_root_partition}, ${use_root_partition}

_${timestamp} => /dev/mapper/os-${os}_${os_arch}_${os_version}_${saveuse} (job use it)

ok

-------- Thanks Yu Chuan

Thanks, Fengguang

...

+> optional vars in ${kernel_append_root}:

• • ${save_root_partition}, ${use_root_partition}
• • purpose:

•  - when os_mount=local, rootfs is stored in the logical volume on the disk,
  

•    but it will be wiped by default in the preprocessing action performed by each job.
  

•    just is initrd stage.
  

•  - so we need a solution to solve scenarios that rootfs data needs to be persisted.
  

• • how to implement:

•  - design a unique string for your job, it will be used as a unique symbol for your job.
  

•    the rootfs data of your job will be persistently saved to the corresponding testbox.
  

•  - so, when you need to use the rootfs data of this job in subsequent jobs, you only need
  

•    to use this unique string to submit subsequent-job.yaml, and then the subsequent-job.yaml
  

•    will be executed based on the prsistent rootfs data.
  

• ### how to use:

•  in 20210218, you submit a job-20210218.yaml, and you want to persist the rootfs data of
  

•    this job-20210218 so that it can be used by the subsequent jobs. So you need add the
  

•    follow field in your job-20210218.yaml.
  

   kernel_append_root: save_root_partition=zhangsan_local_for_iperf_20210218

...

•  - in 20210219, you submit a job-20210219.yaml, and you want to use the rootfs data of
  

•    job-20210218.yaml, So you need add the follow field in your job-20210219.yaml
  

•    ```
  

•    kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218
  

•    ```
  

• • description:

•  - ${save_root_partition} and ${use_root_partition} must be the same, and obviously, they need be unique.
  

•  - you must 'save_root_partition' firstly, then you can 'use_root_partition'. otherwise, your job will fail.
  

+## work flow

+1. user submit job

• job fields:
• os: openeuler
• os_arch: aarch64
• os_version: 20.03
• os_mount: local
• kernel_append_root: use_root_partition=zhangsan_local_for_iperf_20210218 save_root_partition=zhangsan_local_for_iperf_20210219

2. scheduler return the ipxe_str to testbox

dhcp
-  initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/initrd.lkp
-  initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/{timestamp}/boot/vmlinuz
-  imgargs vmlinux root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src={nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} initrd=initrd.lkp {kernel_append_root}
+  initrd http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/initrd.lkp
+  kernel http://${http_server_ip}:${http_server_port}/os/openeuler/aarch64/20.03-iso-snapshots/${timestamp}/boot/vmlinuz root=/dev/mapper/os-openeuler_aarch64_20.03 rootfs_src=${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} initrd=initrd.lkp ${kernel_append_root}
boot

3. dracut step of boot

-  if have {use_root_partition}; then
-	base_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition}
-	lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || exit 1
-
-	if have {save_root_partition}; then
-		boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition}
-
-		if {save_root_partition} == {use_root_partition}; then
-			boot from /dev/mappaer/os-{boot_lv_name}
-		fi
-
-		lvremove -f /dev/mappaer/os-{boot_lv_name}
-		lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1
-		boot from /dev/mappaer/os-{boot_lv_name}
-	else
-		boot_lv_name=openeuler_aarch64_20.03_{value_of_use_root_partition}
-		boot from /dev/mappaer/os-{boot_lv_name}
-	fi
-
-  else:
-	base_lv_name=openeuler_aarch64_20.03_{timestamp}
-	lvdisplay /dev/mappaer/os-{base_lv_name} > /dev/null || {
-		# create logical volume
-		lvcreate -L {root_partition_size} -n {base_lv_name} os || exit 1
-
-		# rsync nfsroot to lvm:/dev/mapper/os-{base_lv_name}
-		mount -t nfs {nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/{timestamp} /mnt
-		mkdir /mnt1 && mount /dev/mapper/os-{base_lv_name} /mnt1
-		cp -a /mnt/. /mnt1/
-		umount /mnt /mnt1
-
-		# change premission of lvm:/dev/mapper/os-{base_lv_name} to readonly
-		lvchange -p r /dev/mapper/os-{base_lv_name}
-	}
-
-	boot_lv_name=openeuler_aarch64_20.03
-	if have {save_root_partition}; then
-		boot_lv_name=openeuler_aarch64_20.03_{value_of_save_root_partition}
-	fi
-
-	lvremove -f /dev/mappaer/os-{boot_lv_name}
-	lvcreate /dev/mappaer/os-{boot_lv_name} from /dev/mappaer/os-{base_lv_name} || exit 1
-	boot from /dev/mappaer/os-{boot_lv_name}
-  fi
+  rsync_src_lv()
+  {
+        lvdisplay ${SRC_LV} > /dev/null && return
+
+        # create logical volume
+        lvcreate --size 10G --name $(basename ${SRC_LV}) os || exit
+
+        # rsync nfsroot to ${SRC_LV}
+        mount -t nfs ${nfs_server_ip}:os/openeuler/aarch64/20.03-iso-snapshots/${timestamp} /mnt
+        mkdir /mnt1 && mount ${SRC_LV} /mnt1
+        cp -a /mnt/. /mnt1/
+        umount /mnt /mnt1
+
+        # change permission of ${SRC_LV} to readonly
+        lvchange --permission r ${SRC_LV}
+  }
+
+  check_src_lv()
+  {
+        lvdisplay ${SRC_LV} > /dev/null || exit
+  }
+
+  snapshot_boot_lv()
+  {
+        [ "$SRC_LV" == "$BOOT_LV" ] && return
+
+        lvremove --force ${BOOT_LV}
+        lvcreate --size 10G --name $(basename ${BOOT_LV}) --snapshot ${SRC_LV} || exit
+  }
+
+  no_save_no_use()
+  {
+        export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"
+        export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"
+
+        rsync_src_lv ${SRC_LV}
+  }
+
+  only_save()
+  {
+        export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${timestamp}"
+        export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"
+
+        rsync_src_lv ${SRC_LV}
+  }
+
+  only_use()
+  {
+        export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"
+        export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03"
+
+        check_src_lv ${SRC_LV}
+  }
+
+  save_and_use()
+  {
+        export SRC_LV="/dev/mapper/os-openeuler_aarch64_20.03_${use_root_partition}"
+        export BOOT_LV="/dev/mapper/os-openeuler_aarch64_20.03_${save_root_partition}"
+
+        check_src_lv ${SRC_LV}
+  }
+
+  main()
+  {
+        [ -z ${use_root_partition} ] && [ -z ${save_root_partition} ] && no_save_no_use
+
+        [ -z ${use_root_partition} ] && [ -n ${save_root_partition} ] && only_save
+
+        [ -n ${use_root_partition} ] && [ -z ${save_root_partition} ] && only_use
+
+        [ -n ${use_root_partition} ] && [ -n ${save_root_partition} ] && save_and_use
+
+        snapshot_boot_lv
+        boot_from_lv
+  }
+
+  main()

4. execute the job

-- 2.23.0