April 2021 - Compass-ci - mailweb.openeuler.org

[PATCH lkp-tests] add job yaml file for building/testing mysql
by Luan Shengde 22 Apr '21

22 Apr '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- jobs/mysql-8-0-20.yaml | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 jobs/mysql-8-0-20.yaml diff --git a/jobs/mysql-8-0-20.yaml b/jobs/mysql-8-0-20.yaml new file mode 100644 index 000000000..4b0e97f6e --- /dev/null +++ b/jobs/mysql-8-0-20.yaml @@ -0,0 +1,36 @@ +suite: cci-makepkg +category: functional + +testbox: taishan200-2280-2s48p-256g--a18 + +os: openeuler +os_version: 20.03 +os_mount: initramfs + +mysql_user: root +mysql_host: localhost +mysql_port: 3306 +mysql_db: sysbench_test +mysql_password: mysql_password +data_dev: sdb + +cci-makepkg: + benchmark: mysql-8-0-20 + +mysql-8-0-20: + +sysbench-mysql: + oltp_test_mode: complex + max_requests: 0 + mysql_table_engine: innodb + oltp_tables_count: 1000 + oltp_table_size: 100000 + nr_threads: 256 + runtime: 60 + rand_type: special + rand_spec_pct: 100 + oltp_read_only: false + report_interval: 1 + events: 20000 + +sleep: 72000 -- 2.23.0

1 0

[PATCH lkp-tests 3/4] pkg/mysql-8-0-20: compile install mysql-8.0.20
by Luan Shengde 22 Apr '21

22 Apr '21

Signed-off-by: Luan Shengde <shdluan(a)163.com> --- pkg/mysql-8-0-20/PKGBUILD | 77 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 pkg/mysql-8-0-20/PKGBUILD diff --git a/pkg/mysql-8-0-20/PKGBUILD b/pkg/mysql-8-0-20/PKGBUILD new file mode 100644 index 000000000..d6b06f6f8 --- /dev/null +++ b/pkg/mysql-8-0-20/PKGBUILD @@ -0,0 +1,77 @@ +pkgname=mysql-server-mysql +pkgver=8.0.20 +gccdir=gcc +pkgrel=1 +arch=('aarch64' 'x86_64') +url="http://www.mysql.com" +license=('GPLv2') +source=("https://gitee.com/li-yancheng/my-sql_with_boostkit_patch.git" + "https://gitee.com/li-yancheng/gcc.git" + "my.cnf") +md5sums=('SKIP' 'SKIP' 'd90ba2c701507eeac43b06fcfa49c2c1') + +build() +{ + # build_gcc + build_mysql +} + +build_gcc() +{ + mkdir -p $HOME/rpmbuild/SOURCES/ + cd $srcdir/gcc && cp -r * $HOME/rpmbuild/SOURCES/ + cd $HOME/rpmbuild/SOURCES/ && { + rpmbuild -bp gcc.spec + } + cd $HOME/rpmbuild/BUILD/gcc-9.3.0 && { + ./configure --prefix=/usr/ \ + --enable-shared \ + --enable-threads=posix \ + --enable-checking=release \ + --with-system-zlib \ + --enable-__cxa_atexit \ + --disable-libunwind-exceptions \ + --enable-gnu-unique-object \ + --enable-linker-build-id \ + --with-linker-hash-style=gnu \ + --enable-languages=c,c++,objc,obj-c++,fortran,lto \ + --enable-plugin \ + --enable-initfini-array \ + --disable-libgcj \ + --without-isl \ + --without-cloog \ + --enable-gnu-indirect-function \ + --with-stage1-ldflags='-Wl,-z,relro,-z,now' \ + --with-boot-ldflags='-Wl,-z,relro,-z,now' \ + --with-multilib-list=lp64 + make -j96 + make install + } +} + +build_mysql() +{ + unset CPPFLAGS + unset CFLAGS + unset CXXFLAGS + cd $srcdir/my-sql_with_boostkit_patch/$pkgname-$pkgver + + CMAKE=cmake + grep -sqF "CentOS Linux release 7" /etc/centos-release && CMAKE=cmake3 + + mkdir -p build + cd build && { + $CMAKE .. -DBUILD_CONFIG=mysql_release -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql/data -DSYSCONFDIR=/etc -DWITH_BOOST=../boost/boost_1_70_0/ + make -j96 + } +} + +package() +{ + mkdir -p $pkgdir/etc + cp my.cnf $pkgdir/etc/ + + cd $srcdir/my-sql_with_boostkit_patch/$pkgname-$pkgver/build + # make DESTDIR=$pkgdir install + make install +} -- 2.23.0

2 2

[PATCH compass-ci] sbin/auto_submit: fix some errors
by Li Yuanchao 21 Apr '21

21 Apr '21

[errmsg] ./auto_submit:66:in `get_argvs_for_competition': wrong number of arguments (given 3, expected 4) (ArgumentError) ./auto_submit:74:in `get_argvs_for_competition': undefined method `chomp' for nil:NilClass (NoMethodError) Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- sbin/auto_submit | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/sbin/auto_submit b/sbin/auto_submit index 28e3715..dcf5b80 100755 --- a/sbin/auto_submit +++ b/sbin/auto_submit @@ -55,7 +55,7 @@ class AutoSubmit tag = %x(git -C #{mirror_dir} tag --points-at #{commit_id}) real_argvs.push("upstream_tag=#{tag}") unless tag.empty? if git_repo.include?('my-sql_with_boostkit_patch') - competi_argvs = get_argvs_for_competition(mirror_dir, branch, commit_id) + competi_argvs = get_argvs_for_competition(real_argvs, mirror_dir, branch, commit_id) real_argvs.push(competi_argvs) end real_argvs.push("upstream_commit=#{commit_id}") @@ -70,8 +70,10 @@ class AutoSubmit if committer_email.include?('noreply') emails = %x(git -C #{mirror_dir} log --format=%ae -2 #{commit_id}) - committer_email = emails.lines[0].chomp - author_email = emails.lines[1].chomp + if emails.lines.size == 2 + committer_email = emails.lines[0].chomp + author_email = emails.lines[1].chomp + end end base_commit = %x(git -C #{mirror_dir} merge-base master #{branch}).chomp -- 2.23.0

1 0

[PATCH compass-ci] container/auto-submit: add es authentication
by Li Yuanchao 21 Apr '21

21 Apr '21

For config_account Signed-off-by: Li Yuanchao <lyc163mail(a)163.com> --- container/auto-submit/start | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/container/auto-submit/start b/container/auto-submit/start index 79463f6..ad4ba9d 100755 --- a/container/auto-submit/start +++ b/container/auto-submit/start @@ -3,16 +3,26 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'set' require_relative '../defconfig.rb' +names = Set.new %w[ + ES_USER + ES_PASSWORD +] + +service_authentication = relevant_service_authentication(names) +env = docker_env(service_authentication) + DOCKER_CCI = '/c/compass-ci' DOCKER_LKP = '/c/lkp-tests' docker_rm "auto-submit" -cmd = %W[ +cmd = %w[ docker run --restart=always --name auto-submit +] + env + %W[ -u 1090:1090 -d -e CCI_SRC=#{DOCKER_CCI} -- 2.23.0

1 0

[PATCH compass-ci] sbin/download-rootfs: download osimage
by Liu Yinsi 21 Apr '21

21 Apr '21

the default os_mount=initramfs, so download osimage to run qemu kvm. Signed-off-by: Liu Yinsi <liuyinsi(a)163.com> --- sbin/download-rootfs | 38 ++++++++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 12 deletions(-) diff --git a/sbin/download-rootfs b/sbin/download-rootfs index 68a6803..d936dbe 100755 --- a/sbin/download-rootfs +++ b/sbin/download-rootfs @@ -19,22 +19,35 @@ os_version=${3:-20.03} load_cci_defaults rootfs_dir=/srv/os/${os}/${os_arch}/${os_version} +osimage_dir=/srv/initrd/osimage/${os}/${os_arch}/${os_version} -download_os_rootfs() -{ - mkdir -p $rootfs_dir - cd $rootfs_dir +mkdir -p $rootfs_dir $osimage_dir - download_url="http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" - wget $download_url +download_one() +{ + local download_url=$1 + wget $download_url + + [ "$?" -ne 0 ] && { + log_error "failed to download os rootfs: $download_url" + exit 1 + } +} - [ "$?" -ne 0 ] && { - log_error "failed to download os rootfs: $download_url" - exit 1 - } +download_os_rootfs() +{ + cd $rootfs_dir + download_one "http://api.compass-ci.openeuler.org:${SRV_HTTP_OS_PORT}/os/${os}/${os_arch}…" + + gzip -dc ${os_version}.cgz | cpio -id + rm -rf ${os_version}.cgz +} - gzip -dc ${os_version}.cgz | cpio -id - rm -rf ${os_version}.cgz +download_osimage() +{ + cd $osimage_dir + download_one "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/osimage/…" + download_one "http://api.compass-ci.openeuler.org:${SRV_HTTP_INITRD_PORT}/initrd/deps/nfs…" } is_virtual() @@ -43,3 +56,4 @@ is_virtual() } is_virtual || download_os_rootfs +is_virtual || download_osimage -- 2.23.0

1 0

[PATCH v2 compass-ci] qcow2rootfs: fix guestfish connection .xz file error
by Wang Chenglong 21 Apr '21

21 Apr '21

[Error] spawn /usr/bin/guestfish -i --ro -a /root/extract/qcow2-dir/openEuler-20.09.x86_64.qcow2.xz guestfish: no operating system was found on this disk after: spawn /usr/bin/guestfish -i --ro -a /root/extract/qcow2-dir/openEuler-20.09.x86_64.qcow2 Welcome to guestfish, the guest filesystem shell for editing virtual machine filesystems and disk images. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/qcow2rootfs/bin/main | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/qcow2rootfs/bin/main b/container/qcow2rootfs/bin/main index c7527b79..2d71c484 100755 --- a/container/qcow2rootfs/bin/main +++ b/container/qcow2rootfs/bin/main @@ -39,7 +39,7 @@ extract_rootfs() { echo "[$(date +'%D %T')] Converting .qcow2 to .tar.gz, please wait several minutes..." su - "$EXTRACT_USER" -c " export LIBGUESTFS_BACKEND=direct - $EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file + $EXTRACT_ROOT/bin/extract $EXTRACT_ROOT/qcow2-dir/$QCOW2_NAME $tar_out_file " echo "[$(date +'%D %T')] Depressing rootfs to $ROOTFS_DIR" -- 2.23.0

1 0

[PATCH compass-ci] qcow2rootfs: fix guestfish connection .xz file error
by Wang Chenglong 21 Apr '21

21 Apr '21

[Error] spawn /usr/bin/guestfish -i --ro -a /root/extract/qcow2-dir/openEuler-20.09.x86_64.qcow2.xz guestfish: no operating system was found on this disk after: spawn /usr/bin/guestfish -i --ro -a /root/extract/qcow2-dir/openEuler-20.09.x86_64.qcow2 Welcome to guestfish, the guest filesystem shell for editing virtual machine filesystems and disk images. Signed-off-by: Wang Chenglong <18509160991(a)163.com> --- container/qcow2rootfs/bin/main | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/container/qcow2rootfs/bin/main b/container/qcow2rootfs/bin/main index c7527b79..3bee1bf2 100755 --- a/container/qcow2rootfs/bin/main +++ b/container/qcow2rootfs/bin/main @@ -39,7 +39,7 @@ extract_rootfs() { echo "[$(date +'%D %T')] Converting .qcow2 to .tar.gz, please wait several minutes..." su - "$EXTRACT_USER" -c " export LIBGUESTFS_BACKEND=direct - $EXTRACT_ROOT/bin/extract $QCOW2_FILE $tar_out_file + $EXTRACT_ROOT/bin/extract /root/extract/qcow2-dir/$QCOW2_NAME $tar_out_file " echo "[$(date +'%D %T')] Depressing rootfs to $ROOTFS_DIR" -- 2.23.0

2 2

[PATCH compass-ci 1/3] service/logging-es: configuring the service account and password
by Wu Zhende 21 Apr '21

21 Apr '21

Improve security Signed-off-by: Wu Zhende <wuzhende666(a)163.com> --- container/defconfig.rb | 8 ++++++++ container/logging-es/Dockerfile | 7 +++++++ container/logging-es/build | 9 ++++++++- 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/container/defconfig.rb b/container/defconfig.rb index b121de8..ad94940 100755 --- a/container/defconfig.rb +++ b/container/defconfig.rb @@ -21,6 +21,14 @@ def relevant_defaults(names) cci_defaults.select { |k, _| names.include? k } end +def relevant_service_account(names) + hash = {} + Dir.glob(['/etc/compass-ci/passwd.yaml']).each do |file| + hash.update YAML.load_file(file) || {} + end + hash.select { |k, _| names.include? k } +end + def set_local_env hash = cci_defaults hash.map { |k, v| system "export #{k}=#{v}" } diff --git a/container/logging-es/Dockerfile b/container/logging-es/Dockerfile index a0acf94..f66dc35 100644 --- a/container/logging-es/Dockerfile +++ b/container/logging-es/Dockerfile @@ -5,6 +5,8 @@ ARG BASE_IMAGE FROM $BASE_IMAGE ARG MEMORY +ARG USER +ARG PASSWORD # docker image borrowed from hub.docker.com/r/gagara/elasticsearch-oss-arm64 @@ -16,6 +18,9 @@ RUN sed -i 's:#network.host\: _site_:network.host\: 0.0.0.0:' /usr/share/elastic sed -i '$a cluster.initial_master_nodes: ["node-1"]' /usr/share/elasticsearch/config/elasticsearch.yml && \ sed -i '$a indices.memory.index_buffer_size: 20%' /usr/share/elasticsearch/config/elasticsearch.yml && \ sed -i '$a thread_pool.write.queue_size: 2000' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a xpack.security.enabled: true' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a xpack.license.self_generated.type: basic' /usr/share/elasticsearch/config/elasticsearch.yml && \ + sed -i '$a xpack.security.transport.ssl.enabled: true' /usr/share/elasticsearch/config/elasticsearch.yml && \ sed -i "s/-Xms1g/-Xms${MEMORY}m/g" /usr/share/elasticsearch/config/jvm.options && \ sed -i "s/-Xmx1g/-Xmx${MEMORY}m/g" /usr/share/elasticsearch/config/jvm.options @@ -24,6 +29,8 @@ RUN mkdir /usr/share/elasticsearch/tmp && \ WORKDIR /usr/share/elasticsearch +RUN ./bin/elasticsearch-users useradd ${USER} -p ${PASSWORD} -r superuser + ENV PATH /usr/share/elasticsearch/bin:$PATH ENV ES_TMPDIR /usr/share/elasticsearch/tmp diff --git a/container/logging-es/build b/container/logging-es/build index b50830e..ff1f102 100755 --- a/container/logging-es/build +++ b/container/logging-es/build @@ -3,8 +3,15 @@ # Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. # frozen_string_literal: true +require 'set' require_relative '../defconfig.rb' +names = Set.new %w[ + LOGGING_ES_USER + LOGGING_ES_PASSWORD +] + +defaults = relevant_service_account(names) docker_skip_rebuild "logging-es:7.11.1" BASE_IMAGE_DICT = { @@ -16,4 +23,4 @@ BASE_IMAGE = BASE_IMAGE_DICT[%x(arch).chomp] available_memory = get_available_memory -system "docker build -t logging-es:7.11.1 --build-arg BASE_IMAGE=#{BASE_IMAGE} --build-arg MEMORY=#{available_memory} ." +system "docker build -t logging-es:7.11.1 --progress=plain --build-arg BASE_IMAGE=#{BASE_IMAGE} --build-arg MEMORY=#{available_memory} --build-arg USER=#{defaults['LOGGING_ES_USER']} --build-arg PASSWORD=#{defaults['LOGGING_ES_PASSWORD']} ." -- 2.23.0

2 1

[PATCH compass-ci] assist-result-client: fix git bisect email subject is the "["
by Cao Xueliang 21 Apr '21

21 Apr '21

Need use json parse the data, before return the response. Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- lib/assist_result_client.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/assist_result_client.rb b/lib/assist_result_client.rb index 592c841..2d2e60d 100644 --- a/lib/assist_result_client.rb +++ b/lib/assist_result_client.rb @@ -60,6 +60,6 @@ class AssistResult response = resource.post(Base64.encode64(data.to_json)) return nil unless response.code == 200 - return response.body + return JSON.parse(response.body) end end -- 2.23.0

1 0

[PATCH compass-ci] sparrow/dev-env: add the host/port to the job by scheduler
by Cao Xueliang 21 Apr '21

21 Apr '21

Signed-off-by: Cao Xueliang <caoxl78320(a)163.com> --- sparrow/3-code/dev-env | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sparrow/3-code/dev-env b/sparrow/3-code/dev-env index e02de35..a478fff 100755 --- a/sparrow/3-code/dev-env +++ b/sparrow/3-code/dev-env @@ -163,6 +163,11 @@ RESULT_WEBDAV_HOST: RESULT_WEBDAV_PORT: DOCKER_REGISTRY_HOST: DOCKER_REGISTRY_PORT: +GIT_MIRROR_HOST: +JUMPER_HOST: +JUMPER_PORT: +MONITOR_HOST: +MONITOR_PORT: EOF cat > /etc/profile.d/compass.sh <<'EOF' -- 2.23.0

1 0