- Compass-ci - mailweb.openeuler.org

[PATCH compass-ci] lib: change the file permission
by Zhang Yu 17 Dec '20

17 Dec '20

The file permission should be consistent with the other file in this directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- lib/mail-post.rb | 0 1 file changed, 0 insertions(+), 0 deletions(-) mode change 100755 => 100644 lib/mail-post.rb diff --git a/lib/mail-post.rb b/lib/mail-post.rb old mode 100755 new mode 100644 -- 2.23.0

1 0

[PATCH compass-ci] sparrow/0-package: install hostname
by Liu Yinsi 17 Dec '20

17 Dec '20

sparrow/3-code/dev-env will use hostname command, so add it. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/0-package/os/centos | 1 + sparrow/0-package/os/debian | 1 + sparrow/0-package/os/openEuler | 1 + 3 files changed, 3 insertions(+) diff --git a/sparrow/0-package/os/centos b/sparrow/0-package/os/centos index 6e3ed42..8df36c0 100755 --- a/sparrow/0-package/os/centos +++ b/sparrow/0-package/os/centos @@ -21,6 +21,7 @@ pkgs=( lockfile-progs busybox iproute + hostname ) yum install -y --skip-broken "${pkgs[@]}" diff --git a/sparrow/0-package/os/debian b/sparrow/0-package/os/debian index 8ee6d7e..6a58a0c 100755 --- a/sparrow/0-package/os/debian +++ b/sparrow/0-package/os/debian @@ -19,6 +19,7 @@ pkgs=( bridge-utils busybox iproute2 + hostname ) apt-get install -y --fix-missing "${pkgs[@]}" diff --git a/sparrow/0-package/os/openEuler b/sparrow/0-package/os/openEuler index 13e0875..a63964f 100755 --- a/sparrow/0-package/os/openEuler +++ b/sparrow/0-package/os/openEuler @@ -20,6 +20,7 @@ pkgs=( wget busybox iproute + hostname ) yum install -y --skip-broken "${pkgs[@]}" -- 2.23.0

1 0

[PATCH v3 lkp-tests] tests: fix error when running host-info in vm
by Zhang Yu 17 Dec '20

17 Dec '20

Error: Could not open device at /dev/ipmi0 or /dev/ipmi/0 or /dev/ipmidev/0: No such file or directory Signed-off-by: Zhang Yu <2134782174(a)qq.com> --- tests/host-info | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/tests/host-info b/tests/host-info index 0ab8e6c97..3984727b4 100755 --- a/tests/host-info +++ b/tests/host-info @@ -102,11 +102,9 @@ get_ipmi_ip() # 802.1q VLAN ID : Disabled # RMCP+ Cipher Suites : 0,1,2,3,17 # Cipher Suite Priv Max : XuuaXXXXXXXXXXX - is_virt && return - is_docker && return + [ -e /dev/ipmi0 ] || [ -e /dev/ipmi/0 ] || [ -e /dev/ipmidev/0 ] || return local ip=$(ipmitool lan print | awk 'NR==3{print $4}') [ -n "$ip" ] && echo "ipmi_ip: $ip" - } main() -- 2.23.0

1 0

[PATCH v2 compass-ci 2/2] sparrow/3-code: use defaults/* to be loaded as job defaults
by Liu Yinsi 16 Dec '20

16 Dec '20

before: /etc/compass-ci/account/autosubmit.yaml after: /etc/compass-ci/defaults/admin-account.yaml Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sparrow/3-code/dev-env | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index 750738f..7eb6166 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -41,8 +41,7 @@ LOCAL_ROBOT_EMAIL_ADDRESS: $LOCAL_ROBOT_EMAIL_ADDRESS MAILDIR: /srv/cci/Maildir/.compass-ci EOF -mkdir -p /etc/compass-ci/account -cat > /etc/compass-ci/account/autosubmit.yaml <<EOF +cat > /etc/compass-ci/defaults/admin-account.yaml <<EOF my_name: Auto Submit Robot my_email: autosubmit@localhost my_token: $(uuidgen) -- 2.23.0

2 1

[PATCH compass-ci] container/assign-account/get_account_info.rb: refactor sort for files
by Lu Kaiyi 16 Dec '20

16 Dec '20

files.sort make no difference for files itself Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- container/assign-account/get_account_info.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/assign-account/get_account_info.rb b/container/assign-account/get_account_info.rb index 9a7acda..0ce8cad 100755 --- a/container/assign-account/get_account_info.rb +++ b/container/assign-account/get_account_info.rb @@ -76,7 +76,7 @@ class AccountStorage end def build_account_name(available_dir, files) - files.sort + files.sort! account_info = [] account_info.push files[0] source_file = File.join(available_dir, files[0]) -- 2.23.0

2 1

[PATCH v5 compass-ci 3/3] lib/compare_data_format.rb: limit the number of table columns in a single row
by Lu Kaiyi 16 Dec '20

16 Dec '20

[example] compare -t compare_template.yaml in z9 [input] a compare_template.yaml like below: compare_metrics: - meminfo.Active - meminfo.Memused filter: suite: - fio-basic os_arch: - aarch64 - x86 compare_dimensions: - os: centos os_version: 7.6 - os: openeuler os_version: 20.03 x_params: - bs - test_size title: Hackbench Performance Testing unit: KB/s [output] +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | Hackbench Performance Testing (unit: KB/s, x_name: bs|test_size) | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | meminfo.Active | 4k|1G | 4k|10G | 4k|80G | 4k|100G | 16k|1G | 16k|10G | 16k|100G | 32k|1G | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | average openeuler 20.03 | 1051645.34 | 704958.00 | 706798.00 | 657669.27 | 1030484.85 | 979289.00 | 948576.00 | 1262591.17 | | standard_deviation openeuler 20.03 | 74.00 | 0.00 | 0.00 | 8.00 | 46.00 | 0.00 | 2.00 | 42.00 | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | meminfo.Active | 64k|1G | 64k|100G | 128k|1G | 128k|100G | 256k|1G | 256k|100G | 512k|1G | 512k|100G | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | average openeuler 20.03 | 1418731.53 | 1854231.33 | 2004321.76 | 2960171.75 | 2979326.50 | 4775340.00 | 5003555.53 | 5630845.75 | | standard_deviation openeuler 20.03 | 47.00 | 10.00 | 65.00 | 18.00 | 88.00 | 8.00 | 108.00 | 15.00 | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | meminfo.Active | 1024k|1G | 1024k|100G | 2048k|100G | 4096k|100G | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ | average openeuler 20.03 | 8974190.91 | 9641039.00 | 5480763.67 | 7302509.67 | | standard_deviation openeuler 20.03 | 118.00 | 36.00 | 86.00 | 67.00 | +------------------------------------+------------+------------+------------+------------+------------+------------+------------+------------+ ... Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare_data_format.rb | 41 +++++++++++++++++++++++++++++--------- 1 file changed, 32 insertions(+), 9 deletions(-) diff --git a/lib/compare_data_format.rb b/lib/compare_data_format.rb index c1f7023..0f6bdc0 100644 --- a/lib/compare_data_format.rb +++ b/lib/compare_data_format.rb @@ -72,19 +72,20 @@ end # format compare template results into a table format # class FormatTableData - def initialize(result_hash) + def initialize(result_hash, row_size = 8) @title = result_hash['title'] @tables = result_hash['tables'] @unit = result_hash['unit'] @x_name = result_hash['x_name'] + @row_size = row_size end def show_table @tables.each do |table_title, table| @tb = Terminal::Table.new set_table_title - set_field_names(table_title, table) - add_rows(table) + row_num = get_row_num(table) + split_data_column(table_title, table, row_num) set_align_column print_table end @@ -94,14 +95,36 @@ class FormatTableData @tb.title = "#{@title} (unit: #{@unit}, x_name: #{@x_name})" end - def set_field_names(table_title, table) + def get_row_num(table) + data_column_size = table['average']['source'][0].size + unless @row_size.positive? + warn('row size must be positive!') + exit + end + (data_column_size / @row_size.to_f).ceil + end + + def split_data_column(table_title, table, row_num) + row_num.times do |row| + starts = 1 + row * @row_size + ends = starts + @row_size + set_field_names(table_title, table, starts, ends) + add_rows(table, starts, ends) + break if row == row_num - 1 + + @tb.add_separator + @tb.add_separator + end + end + + def set_field_names(table_title, table, starts, ends) field_names = [table_title] - field_names.concat(table['average']['source'][0]) + field_names.concat(table['average']['source'][0][starts - 1...ends - 1]) @tb.add_row(field_names) @tb.add_separator end - def add_rows(table) + def add_rows(table, starts, ends) row_names = %w[average standard_deviation change] max_size = row_names.map(&:size).max row_names.each do |row_name| @@ -109,15 +132,15 @@ class FormatTableData dimensions_size = table[row_name]['dimensions'].size (1...dimensions_size).each do |index| - add_row(table, row_name, index, max_size) + add_row(table, row_name, index, max_size, starts, ends) end end end - def add_row(table, row_name, index, max_size) + def add_row(table, row_name, index, max_size, starts, ends) row = table[row_name]['source'][index] row_title = [row_name + ' ' * (max_size - row_name.size), row[0]].join(' ') - format_data_row = row[1..-1] + format_data_row = row[starts...ends] if row_name == 'change' format_data_row.map! { |data| format('%.1f%%', data) } else -- 2.23.0

2 1

[PATCH v5 compass-ci 2/3] add options for compare_by_template
by Lu Kaiyi 16 Dec '20

16 Dec '20

if want json format results, use command like below: compare -t compare_template.yaml --theme json otherwise, print table format results by default: compare -t compare_template.yaml Signed-off-by: Lu Kaiyi <2392863668(a)qq.com> --- lib/compare.rb | 4 ++-- lib/compare_matrixes.rb | 11 ++++++++--- sbin/compare | 2 +- 3 files changed, 11 insertions(+), 6 deletions(-) diff --git a/lib/compare.rb b/lib/compare.rb index f34e239..db264f8 100644 --- a/lib/compare.rb +++ b/lib/compare.rb @@ -96,11 +96,11 @@ end # unit: KB/s # -def compare_by_template(template) +def compare_by_template(template, options) template_params = load_template(template) groups_matrices = create_groups_matrices(template_params) compare_results = compare_metrics_values(groups_matrices) - show_compare_result(compare_results, template_params) + show_compare_result(compare_results, template_params, options) end def load_template(template) diff --git a/lib/compare_matrixes.rb b/lib/compare_matrixes.rb index 9bf676f..e8f52f1 100644 --- a/lib/compare_matrixes.rb +++ b/lib/compare_matrixes.rb @@ -508,11 +508,16 @@ def get_dimensions_combination(dimension_list) dims end -def show_compare_result(metrics_compare_results, template_params) +def show_compare_result(metrics_compare_results, template_params, options) + theme = options[:theme] formatter = FormatEchartData.new(metrics_compare_results, template_params) echart_results = formatter.format_for_echart - table_results = FormatTableData.new(echart_results) - table_results.show_table + if theme == 'json' + print JSON.pretty_generate(echart_results) + else + table_results = FormatTableData.new(echart_results) + table_results.show_table + end end # Format Fields diff --git a/sbin/compare b/sbin/compare index c13ec1a..e8d1ac4 100755 --- a/sbin/compare +++ b/sbin/compare @@ -67,7 +67,7 @@ opt_parser.parse!(argv) options = { theme: colorful } if colorful if template - compare_by_template(template) + compare_by_template(template, options) elsif is_group compare_group(argv, dimensions, options) else -- 2.23.0

2 1

[PATCH lkp-tests] tests/build-pkg: build different version software packages
by Liu Shaofei 16 Dec '20

16 Dec '20

[why] When building different version software, we need to pack out different *.cgz. The field of PKGBUILD_PKGVE can be appended job.yaml file, which specifies the version number. The parts of contents for job.yaml file is as follow: job.yaml: ... pkgbuild_repo: archlinux/packages//fcgi/trunk PKGBUILD_PKGVER: 2.4.2 ... [how] -rw-rw-r--. 1 lkp lkp 46K 2020-12-15 17:17 2.4.2.cgz lrwxrwxrwx. 1 lkp lkp 9 2020-12-15 17:17 latest.cgz -> 2.4.2.cgz Signed-off-by: Liu Shaofei <370072077(a)qq.com> --- tests/build-pkg | 25 ++++++++++--------------- 1 file changed, 10 insertions(+), 15 deletions(-) diff --git a/tests/build-pkg b/tests/build-pkg index aafc120ed..c3909d1b9 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -3,6 +3,7 @@ # - upstream_repo # - upstream_commit # - PKGBUILD_TAG +# - PKGBUILD_PKGVER # upstream_url # We'll use the below global env vars, but not treat them as parameter to avoid @@ -60,18 +61,6 @@ request_pkg() do get_pkgfile "${pkgrepo}" "$pkgfile" done - - # download PKGBUILD-$tag - benchmark="${upstream_repo##*/}" - benchmark="${benchmark%.git}" - [ -n "$PKGBUILD_TAG" ] && { - get_pkgfile "pkg/$benchmark" "PKGBUILD-$PKGBUILD_TAG" - [ -f "PKGBUILD-$PKGBUILD_TAG" ] || die "PKGBUILD-$PKGBUILD_TAG not exist" - [ -s "PKGBUILD-$PKGBUILD_TAG" ] || die "PKGBUILD-$PKGBUILD_TAG is empty" - } - - [ -f PKGBUILD ] || die "PKGBUILD not exist" - [ -s PKGBUILD ] || die "PKGBUILD is empty" } create_softlink() @@ -109,6 +98,7 @@ replace_source() build_source_pkg() { + [ -n "$PKGBUILD_PKGVER" ] && sed -i "s|^pkgver=.*|pkgver=${PKGBUILD_PKGVER}|g" PKGBUILD repo_dir="" source PKGBUILD [ -n "$upstream_repo" ] && [ -n "$upstream_url" ] && { @@ -122,11 +112,16 @@ build_source_pkg() } replace_source PKGBUILD.src - [ -n "$PKGBUILD_TAG" ] && replace_source "PKGBUILD-$PKGBUILD_TAG" - cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${upstream_commit}.cgz" + pkg_args="-A --check --skippgpcheck" + + [ -n ${PKGBUILD_PKGVER} ] && { + cgz_name="$PKG_MNT/${pack_to}/${pkgname}/${pkgver}.cgz" + pkg_args="${pkg_args} --skipchecksums --skipinteg" + } + PACMAN=true BUILDDIR=$TMP CARCH=$os_arch PKGEXT=.cgz CGZDEST="$cgz_name" STDERR_REDIR=y\ - $LKP_SRC/sbin/makepkg -A --check --skippgpcheck --config $LKP_SRC/etc/makepkg.conf + $LKP_SRC/sbin/makepkg ${pkg_args} --config $LKP_SRC/etc/makepkg.conf [ "$?" == 0 ] || exit 1 create_softlink -- 2.23.0

2 3

[PATCH lkp-tests 1/3] tests/build-pkg: fix format
by Lin Jiaxin 16 Dec '20

16 Dec '20

Signed-off-by: Lin Jiaxin <ljx.joe(a)qq.com> --- tests/build-pkg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/build-pkg b/tests/build-pkg index aafc120e..dbde3e54 100755 --- a/tests/build-pkg +++ b/tests/build-pkg @@ -2,8 +2,8 @@ # - pkgbuild_repo # - upstream_repo # - upstream_commit +# - upstream_url # - PKGBUILD_TAG -# upstream_url # We'll use the below global env vars, but not treat them as parameter to avoid # duplicates in result path like /openeuler-20.03-aarch64/openeuler-aarch64-cifs-2003/ -- 2.23.0

2 1

Re: [PATCH lkp-tests] lib/upload: change the key of upload results
by Cao Xueliang 16 Dec '20

16 Dec '20

On Wed, Dec 16, 2020 at 07:56:34PM +0800, Li Ping wrote: >On Wed, Dec 16, 2020 at 03:37:09PM +0800, Wei Jihui wrote: >>the key of upload results is changed from access_key to job id >> >>Signed-off-by: Wei Jihui <weijihuiall(a)163.com> >>--- >> lib/job-init.sh | 2 +- >> lib/upload.sh | 18 +++++++++--------- >> 2 files changed, 10 insertions(+), 10 deletions(-) >> >>diff --git a/lib/job-init.sh b/lib/job-init.sh >>index 40534af3..dc4a65d9 100755 >>--- a/lib/job-init.sh >>+++ b/lib/job-init.sh >>@@ -55,7 +55,7 @@ setup_result_service() >> >> supports_raw_upload && result_service=raw_upload && return >> >>- is_docker || [ -n "$access_key" ] || { >>+ is_docker || [ -n "$id" ] || { >> if [ "$os_mount" = 'nfs' ] || [ -z "$os_mount" ]; then >> supports_netfs 'nfs' && result_service=$RESULT_SERVER:/result && return >> elif [ "$os_mount" = 'cifs' ] || [ -z "$os_mount" ]; then >>diff --git a/lib/upload.sh b/lib/upload.sh >>index a5daceda..0115e350 100755 >>--- a/lib/upload.sh >>+++ b/lib/upload.sh >>@@ -78,9 +78,9 @@ upload_one_curl() >> ( >> cd $(dirname "$1") >> dir=$(basename "$1") >>- if [ -n "$access_key" ]; then >>- find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; >>- find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "ACCESSKEY=$access_key" \; >>+ if [ -n "$id" ]; then >>+ find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" --cookie "JOBID=$id" \; >>+ find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" --cookie "JOBID=$id" \; > > users can upload files or folders regardless > of whether the job id exists. what is the > function of determing whether the job id > exist? Get the id from the env var, when we run a job lkp will export all the job yaml as the env var. Thanks, Xueliang > > Thanks, > Ping > >> else >> find "$dir" -type d -exec curl -sSf -X MKCOL "$http_url/{}" \; >> find "$dir" -type f -size +0 -exec curl -sSf -T '{}' "$http_url/{}" \; >>@@ -88,8 +88,8 @@ upload_one_curl() >> ) >> else >> [ -s "$src" ] || return >>- if [ -n "$$access_key" ]; then >>- curl -sSf -T "$src" $http_url/ --cookie "ACCESSKEY=$access_key" >>+ if [ -n "$id" ]; then >>+ curl -sSf -T "$src" $http_url/ --cookie "JOBID=$id" >> else >> curl -sSf -T "$src" $http_url/ >> fi >>@@ -111,8 +111,8 @@ upload_files_curl() >> for dir in $(echo $target_directory | tr '/' ' ') >> do >> job_result_root=$job_result_root/$dir/ >>- if [ -n "$$access_key" ]; then >>- curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "ACCESSKEY=$access_key" >/dev/null >>+ if [ -n "$id" ]; then >>+ curl -sSf -X MKCOL "$http_host$job_result_root" --cookie "JOBID=$id" >/dev/null >> else >> curl -sSf -X MKCOL "$http_host$job_result_root" >/dev/null >> fi >>@@ -178,12 +178,12 @@ upload_files() >> upload_files_copy "$@" >> return >> } >>- if has_cmd rsync && has_rsync_server && [ -z "$access_key" ]; then >>+ if has_cmd rsync && has_rsync_server && [ -z "$id" ]; then >> upload_files_rsync "$@" >> return >> fi >> >>- if has_cmd lftp && [ -z "$access_key" ]; then >>+ if has_cmd lftp && [ -z "$id" ]; then >> upload_files_lftp "$@" >> return >> fi >>-- >>2.23.0 >>

1 0