[PATCH v2 lkp-tests] distro/archlinux: add archlinux distro file
add archlinux pack method Signed-off-by: Wang Yong <wangyong0117@qq.com> --- distro/archlinux | 104 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100755 distro/archlinux diff --git a/distro/archlinux b/distro/archlinux new file mode 100755 index 000000000..e6d7dce50 --- /dev/null +++ b/distro/archlinux @@ -0,0 +1,104 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +shopt -s nullglob + +. $LKP_SRC/lib/install.sh +. $LKP_SRC/distro/common +. $LKP_SRC/lib/debug.sh + +distro_install_depends() +{ + local script + local bm="$1" + local scripts=$(find $LKP_SRC/distro/depends/ -name "$bm" -o -name "${bm}.[0-9]") + + for script in $scripts + do + script=$(basename $script) + packages=$(get_dependency_packages $DISTRO $script) + + [ -z "$packages" ] && continue + + echo install packages for $script: $packages + + for pkg in $packages + do + pacman -Qqs ----noconfirm $pkg || pacman -Sy --noconfirm $pkg + done + done +} + +pack_benchmark() +{ + setup_proxy + + distro_install_depends lkp-dev + + # Process each benchmark + for BM_NAME in $benchmark + do + distro_install_depends $BM_NAME-dev || continue + echo $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ( + $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ) + done +} + +pacman_download() +{ + echo "pacman -Sw --noconfirm $*" + pacman -Sw --noconfirm "$@" +} + +pacman_pack() +{ + local target_dir=/opt/pkgs + local date=$(date +"%Y%m%d") + + local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz") + + [ "$downloaded_pkgs" ] || return + + mkdir -p $target_dir + + mv $downloaded_pkgs $target_dir + + find $target_dir | cpio --quiet -o -H newc --owner=root.root | gzip -n -9 >$pack_to/${BM_NAME}_$date.cgz + + ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + echo package installed to $pack_to/${BM_NAME}.cgz + + ls $target_dir/*.pkg.tar.gz > $pack_to/.${BM_NAME}.packages + + rm -rf $target_dir +} + +cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +} + +pack_benchmark_deps() +{ + setup_proxy + + for BM_NAME in $benchmark + do + + cleanup_downloaded_pkgs + + check_shared_package $BM_NAME + + packages=$(echo $(get_dependency_packages $DISTRO $BM_NAME)) + + if pacman_download $packages; then + pacman_pack + else + echo "failed to pack-deps $BM_NAME" >&2 + echo $? + fi + done +} -- 2.23.0
+ local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz")
+cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +}
The 2 find can be merged to a common function. btw what's the example find output? Can we limit the find depth and/or let the base dir more exact? Thanks, Fengguang
On Tue, Nov 17, 2020 at 10:22:23AM +0800, Wu Fengguang wrote:
+ local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz")
+cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +}
The 2 find can be merged to a common function.
Ok, i'll do it.
btw what's the example find output?
i'll add example next time.
Can we limit the find depth and/or let the base dir more exact?
OK. Thanks, Wang Yong
On Tue, Nov 17, 2020 at 10:22:23AM +0800, Wu Fengguang wrote:
+ local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz")
+cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +}
btw what's the example find output?
% pacman -Sw wget resolving dependencies... Packages (1) wget-1.20.3-3 Total Download Size: 0.63 MiB :: Proceed with download? [Y/n] y warning: no /var/cache/pacman/pkg/ cache exists, creating... :: Retrieving packages... wget-1.20.3-3-aa... 642.3 KiB 4.75 MiB/s 00:00 [######################] 100% (1/1) checking keys in keyring [######################] 100% (1/1) checking package integrity [######################] 100% % find /var/ -type f -name "*.pkg.tar.xz" /var/cache/pacman/pkg/wget-1.20.3-3-aarch64.pkg.tar.xz
Can we limit the find depth and/or let the base dir more exact?
/var => /var/cache/pacman/pkg/ change find dir to default cache dir Thanks, Wang Yong
On Tue, Nov 17, 2020 at 10:33:49AM +0800, Wang Yong wrote:
On Tue, Nov 17, 2020 at 10:22:23AM +0800, Wu Fengguang wrote:
+ local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz")
+cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +}
btw what's the example find output?
% pacman -Sw wget resolving dependencies...
Packages (1) wget-1.20.3-3
Total Download Size: 0.63 MiB
:: Proceed with download? [Y/n] y warning: no /var/cache/pacman/pkg/ cache exists, creating... :: Retrieving packages... wget-1.20.3-3-aa... 642.3 KiB 4.75 MiB/s 00:00 [######################] 100% (1/1) checking keys in keyring [######################] 100% (1/1) checking package integrity [######################] 100%
% find /var/ -type f -name "*.pkg.tar.xz" /var/cache/pacman/pkg/wget-1.20.3-3-aarch64.pkg.tar.xz
Can we limit the find depth and/or let the base dir more exact?
/var => /var/cache/pacman/pkg/
change find dir to default cache dir
Good. btw, .xz may not be the only suffix. Please check include other zip format. Thanks, Fengguang
On Tue, Nov 17, 2020 at 10:41:41AM +0800, Wu Fengguang wrote:
On Tue, Nov 17, 2020 at 10:33:49AM +0800, Wang Yong wrote:
On Tue, Nov 17, 2020 at 10:22:23AM +0800, Wu Fengguang wrote:
Can we limit the find depth and/or let the base dir more exact?
/var => /var/cache/pacman/pkg/
change find dir to default cache dir
Good. btw, .xz may not be the only suffix. Please check include other zip format.
Valid suffixes are .tar.gz, .tar.bz2, .tar.xz, .tar.zst, .tar.lzo, .tar.lrz, .tar.lz4, .tar.lz and .tar.Z, or simply .tar to disable compression entirely. $ cat /etc/makepkg.conf | grep COMPRESSION COMPRESSGZ=(gzip -c -f -n) COMPRESSBZ2=(bzip2 -c -f) COMPRESSXZ=(xz -c -z -) COMPRESSZST=(zstd -c -z -q -) COMPRESSLRZ=(lrzip -q) COMPRESSLZO=(lzop -q) COMPRESSZ=(compress -c -f) COMPRESSLZ4=(lz4 -q) COMPRESSLZ=(lzip -c -f) makepkg and packages from archlinux use .xz by default $ cat /etc/makepkg.conf | grep PKGEXT PKGEXT='.pkg.tar.xz' so i think use *.pkg.tar* is more general reference: https://wiki.archlinux.org/index.php/Arch_Build_System https://opentuna.cn/archlinuxarm/aarch64/ https://jlk.fjfi.cvut.cz/arch/manpages/man/makepkg.conf.5 Thanks, Wang Yong
On Tue, Nov 17, 2020 at 11:35:28AM +0800, Wang Yong wrote:
On Tue, Nov 17, 2020 at 10:41:41AM +0800, Wu Fengguang wrote:
On Tue, Nov 17, 2020 at 10:33:49AM +0800, Wang Yong wrote:
On Tue, Nov 17, 2020 at 10:22:23AM +0800, Wu Fengguang wrote:
Can we limit the find depth and/or let the base dir more exact?
/var => /var/cache/pacman/pkg/
change find dir to default cache dir
Good. btw, .xz may not be the only suffix. Please check include other zip format.
Valid suffixes are .tar.gz, .tar.bz2, .tar.xz, .tar.zst, .tar.lzo, .tar.lrz, .tar.lz4, .tar.lz and .tar.Z, or simply .tar to disable compression entirely.
$ cat /etc/makepkg.conf | grep COMPRESSION COMPRESSGZ=(gzip -c -f -n) COMPRESSBZ2=(bzip2 -c -f) COMPRESSXZ=(xz -c -z -) COMPRESSZST=(zstd -c -z -q -) COMPRESSLRZ=(lrzip -q) COMPRESSLZO=(lzop -q) COMPRESSZ=(compress -c -f) COMPRESSLZ4=(lz4 -q) COMPRESSLZ=(lzip -c -f)
makepkg and packages from archlinux use .xz by default
Not necessarily. Arch Linux现在开始使用 zstd 替代 xz 进行软件包压缩 https://www.jianshu.com/p/f92a066971be
$ cat /etc/makepkg.conf | grep PKGEXT PKGEXT='.pkg.tar.xz'
so i think use *.pkg.tar* is more general
OK. Thanks, Fengguang
reference: https://wiki.archlinux.org/index.php/Arch_Build_System https://opentuna.cn/archlinuxarm/aarch64/ https://jlk.fjfi.cvut.cz/arch/manpages/man/makepkg.conf.5
Thanks, Wang Yong
On Mon, Nov 16, 2020 at 08:24:05PM +0800, Wang Yong wrote:
add archlinux pack method
Signed-off-by: Wang Yong <wangyong0117@qq.com> --- distro/archlinux | 104 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100755 distro/archlinux
diff --git a/distro/archlinux b/distro/archlinux new file mode 100755 index 000000000..e6d7dce50 --- /dev/null +++ b/distro/archlinux @@ -0,0 +1,104 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 + +shopt -s nullglob + +. $LKP_SRC/lib/install.sh +. $LKP_SRC/distro/common +. $LKP_SRC/lib/debug.sh + +distro_install_depends() +{ + local script + local bm="$1" + local scripts=$(find $LKP_SRC/distro/depends/ -name "$bm" -o -name "${bm}.[0-9]") + + for script in $scripts + do + script=$(basename $script) + packages=$(get_dependency_packages $DISTRO $script) + + [ -z "$packages" ] && continue + + echo install packages for $script: $packages + + for pkg in $packages + do + pacman -Qqs ----noconfirm $pkg || pacman -Sy --noconfirm $pkg
^Is this the correct grammar?
+ done + done +} + +pack_benchmark() +{ + setup_proxy + + distro_install_depends lkp-dev + + # Process each benchmark + for BM_NAME in $benchmark + do + distro_install_depends $BM_NAME-dev || continue
Line changes
+ echo $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ( + $LKP_SRC/sbin/pack -d $DISTRO -f -c -s $PKG_MNT/$pack_to $BM_NAME + ) + done +} + +pacman_download() +{ + echo "pacman -Sw --noconfirm $*" + pacman -Sw --noconfirm "$@" +} + +pacman_pack() +{ + local target_dir=/opt/pkgs + local date=$(date +"%Y%m%d") + + local downloaded_pkgs=$(find /var/ -type f -name "*.pkg.tar.xz") + + [ "$downloaded_pkgs" ] || return + + mkdir -p $target_dir + + mv $downloaded_pkgs $target_dir + + find $target_dir | cpio --quiet -o -H newc --owner=root.root | gzip -n -9 >$pack_to/${BM_NAME}_$date.cgz + + ln -sf ${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return + chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return
echo package installed to $pack_to/${BM_NAME}.cgz echo packages installed to $pack_to/${BM_NAME}.cgz
+ + ls $target_dir/*.pkg.tar.gz > $pack_to/.${BM_NAME}.packages + + rm -rf $target_dir +} + +cleanup_downloaded_pkgs() +{ + find /var/ -type f -name "*.pkg.tar.xz" -exec rm -f -- '{}' \; +} + +pack_benchmark_deps() +{ + setup_proxy + + for BM_NAME in $benchmark + do + + cleanup_downloaded_pkgs
^One more line
+ check_shared_package $BM_NAME + + packages=$(echo $(get_dependency_packages $DISTRO $BM_NAME)) + + if pacman_download $packages; then + pacman_pack + else + echo "failed to pack-deps $BM_NAME" >&2
echo $? echo "exit $?" Thanks sunyukui
+ fi + done +} -- 2.23.0
On Tue, Nov 17, 2020 at 10:57:04AM +0800, Sun Yukui wrote:
On Mon, Nov 16, 2020 at 08:24:05PM +0800, Wang Yong wrote:
+ pacman -Qqs ----noconfirm $pkg || pacman -Sy --noconfirm $pkg
^Is this the correct grammar?
nice catch
+ distro_install_depends $BM_NAME-dev || continue
Line changes
it's aligned
+ chown .lkp $pack_to/${BM_NAME}_$date.cgz $pack_to/${BM_NAME}.cgz || return
echo package installed to $pack_to/${BM_NAME}.cgz echo packages installed to $pack_to/${BM_NAME}.cgz
all of this is package, so i think it's necessary to change it.
+ cleanup_downloaded_pkgs
^One more line
Ok.
+ echo "failed to pack-deps $BM_NAME" >&2
echo $? echo "exit $?" Thanks sunyukui
It's only test, already delete in newer patch Thanks, Wang Yong
participants (3)
-
Sun Yukui -
Wang Yong -
Wu Fengguang