- Compass-ci - mailweb.openeuler.org

Re: [todo] auto submit cci-makepkg for all tests
by Wu Fengguang 15 Feb '22

15 Feb '22

On Mon, Feb 14, 2022 at 04:34:20PM +0800, Cao Xueliang wrote: ># auto submit cci-makepkg before run test suite > >example: >- job.yaml >--- >'#! rpmbuild.yaml': >suite: rpmbuild >category: functional >rpmbuild: >repo_addr: https://vault.centos.org/6.10/os/Source/SPackages/ed-1.1-3.3.el6.src.rpm >arch: aarch64 >custom_repo_name: unstable/compatible/6 >mount_repo_addr: https://api.compass-ci.openeuler.org:20018/rpm/testing/openeuler-20.03-LTS-… >1/compat-centos6/standard_\$basearch/\$basearch >mount_repo_name: compatible-6 >docker_image: openeuler:20.03-LTS-SP1 >``` >- generate cci-makepkg.yaml ># detect $LKP_SRC/pkg/rpmbuild/PKGBUILD first ># detect /srv/initrd/pkg/container/openeuler/aarch64/20.03-LTS-SP1/rpmbuild/latest.cgz >``` >suite: cci-makepkg >category: functional > >cci-makepkg: > benchmark: rpmbuild # from rpmbuild.yaml >os: # from rpmbuild.yaml >os_arch: # from rpmbuild.yaml >os_mount: # from rpmbuild.yaml >``` Looks good. >- submit cci-makepkg.yaml & rpmbuild.yaml > - if cci-makepkg job success and exists latest.cgz then submit rpmbuild.yaml > - if cci-makepkg job not success then echo error & no need submit rpmbuild.yaml Just to make it clear: Here "submit cci-makepkg.yaml" should not mean "cci submit". The scheduler should - check and setup job depends - auto create and dispatch cci-makepkg.yaml - continue dispatch rpmbuild.yaml, when the above job succeeded Thanks, Fengguang

1 0

Re: [PATCH v5 compass-ci] data-api: add es search api
by Wei Jihui 15 Feb '22

15 Feb '22

On Mon, Feb 14, 2022 at 07:07:14PM +0800, Wei Jihui wrote: >From: Wei Jihui <weijihuiall(a)163.com> > pls ignore this patch. Thanks, Jihui >1. /data_api/es/:index/_search is es search by dsl >2. /data_api/es/:index/_search_by_sql is es search by sql > >Signed-off-by: Wei Jihui <weijihuiall(a)163.com> >--- > container/data-api/data-api | 36 +++++++----- > lib/es_client.rb | 13 ++-- > src/lib/data_api.rb | 93 +++++------------------------ > src/lib/data_api/es_data_api.rb | 101 ++++++++++++++++++++++++++++++++ > 4 files changed, 143 insertions(+), 100 deletions(-) > create mode 100644 src/lib/data_api/es_data_api.rb > >diff --git a/container/data-api/data-api b/container/data-api/data-api >index 2bd6e527..834ec702 100755 >--- a/container/data-api/data-api >+++ b/container/data-api/data-api >@@ -7,8 +7,6 @@ require 'sinatra' > > CCI_SRC = ENV['CCI_SRC'] || '/c/compass-ci' > require "#{CCI_SRC}/src/lib/data_api.rb" >-require "#{CCI_SRC}/src/lib/service_logs/service_logs.rb" >-require "#{CCI_SRC}/lib/es_query.rb" > > set :bind, '0.0.0.0' > set :port, 10005 >@@ -17,24 +15,32 @@ set :environment, :production > > # POST > # eg1: >-# curl -X POST localhost:10005/es_find -H 'Content-Type: application/json' -d '{ >+# curl -X POST localhost:10005/data_api/es/jobs/_search -H 'Content-Type: application/json' -d '{ > # "query": {"size":10, "sort": [{"start_time": {"order": "desc"}}]}, >-# "index": "jobs", >-# "my_token": "16132550-...", >-# "my_account": "auto-submit", >-# "query_type": "eql" # can be omitted in this scenario >+# "cci_credentials": { >+# "my_token": "16132550-...", >+# "my_account": "auto-submit" >+# } > # }' >-# eg2: >-# curl -X POST localhost:10005/es_find -H 'Content-Type: application/json' -d '{ >+# Response: >+# - es_result : JSON >+post '/data_api/es/:index/_search' do >+ request.body.rewind # in case someone already read it >+ es_search(params['index'], request.body.read) >+end >+ >+# POST >+# eg1: >+# curl -X POST localhost:10005/data_api/es/jobs/_search_by_sql -H 'Content-Type: application/json' -d '{ > # "query": "SELECT * FROM jobs WHERE ...", >-# "index": "jobs", >-# "my_token": "16132550-...", >-# "my_account": "auto-submit", >-# "query_type": "sql" # required >+# "cci_credentials": { >+# "my_token": "16132550-...", >+# "my_account": "auto-submit" >+# } > # }' > # Response: > # - es_result : JSON >-post '/data_api/es_find' do >+post '/data_api/es/:index/_search_by_sql' do > request.body.rewind # in case someone already read it >- es_find(request.body.read) >+ es_search_by_sql(params['index'], request.body.read) > end >diff --git a/lib/es_client.rb b/lib/es_client.rb >index 4ae1fc00..63e42d76 100644 >--- a/lib/es_client.rb >+++ b/lib/es_client.rb >@@ -6,9 +6,9 @@ require_relative 'es_query.rb' > # ------------------------------------------------------------------------------------------- > # put_source_by_id(source_id, source_content) > # - put a source to ES /<target>/_doc/<_id>, ingore the existence of source >-# >-# query_by_sql(query_sql) >-# - query es db by query_sql, query_sql is sql, like "SELECT * FROM JOBS WHERE ..." >+# >+# search_by_sql(search_sql) >+# - search es db by search_sql, search_sql is sql, like "SELECT * FROM JOBS WHERE ..." > # > # ------------------------------------------------------------------------------------------- > class ESClient < ESQuery >@@ -29,12 +29,11 @@ class ESClient < ESQuery > ) > end > >- # query es db by query_sql, query_sql is sql, like: >+ # search es db by search_sql, search_sql is sql, like: > # - "SELECT id, suite FROM JOBS" > # - "SELECT * FROM accounts WHERE my_account='test_user'" > # this plugin from: https://github.com/NLPchina/elasticsearch-sql >- def query_by_sql(query_sql) >- @client.perform_request('GET', '_nlpcn/sql', {}, query_sql) >+ def search_by_sql(search_sql) >+ @client.perform_request('GET', '_nlpcn/sql', {}, search_sql) > end >- > end >diff --git a/src/lib/data_api.rb b/src/lib/data_api.rb >index eeaf0f98..8aa4304b 100644 >--- a/src/lib/data_api.rb >+++ b/src/lib/data_api.rb >@@ -3,90 +3,27 @@ > # frozen_string_literal: true > > CCI_SRC ||= ENV['CCI_SRC'] || '/c/compass-ci' >-require 'json' >-require 'elasticsearch' >-require 'set' >-require_relative '../../lib/constants.rb' >-require_relative '../../lib/es_query.rb' >-require_relative '../../lib/es_client.rb' >-require_relative '../../lib/json_logger.rb' >+require "#{CCI_SRC}/lib/json_logger.rb" >+require "#{CCI_SRC}/src/lib/data_api/es_data_api.rb" > >-ES_ACCOUNTS = ESQuery.new(index: 'accounts') >-REQUIRED_TOKEN_INDEX = Set.new(['jobs']) >-UNOPEN_INDEX = Set.new(['accounts']) >-ES_QUERY_KEYWORD = Set.new(['term', 'match']) >- >-def es_find(params) >+def es_search(index, params) > begin >- result = query_es(params) >+ result = EsDataApi.search(index, params) > rescue StandardError => e >- log_error({ >- 'message' => e.message, >- 'error_message' => "query es error" >- }) >- return [500, headers.merge('Access-Control-Allow-Origin' => '*'), e.message] >+ error_msg = { 'error_msg' => e.message } >+ log_error(error_msg) >+ return [200, headers.merge('Access-Control-Allow-Origin' => '*'), error_msg.to_json] > end > [200, headers.merge('Access-Control-Allow-Origin' => '*'), result.to_json] > end > >-def query_es(params) >- request_body = JSON.parse(params) >- index = request_body['index'] || 'jobs' >- query = request_body['query'] || {'query' => {}} >- return "#{index} is not opened" if UNOPEN_INDEX.any? { |unopen_index| index.include?(unopen_index) } >- >- if REQUIRED_TOKEN_INDEX.any? { |req_index| index.include?(req_index) } >- my_account = request_body['my_account'] >- my_token = request_body['my_token'] >- return "missed my_account" unless my_account >- return "missed my_token" unless my_token >- return "user authentication failed" unless verify_user(my_account, my_token) >- >- query = build_account_query(query, my_account, request_body['query_type'], index) >- end >- >- if request_body['query_type'] == 'sql' >- es = ESClient.new(index: index) >- return es.query_by_sql(query).body >- else >- es = Elasticsearch::Client.new(hosts: ES_HOSTS) >- return es.search index: index + '*', body: query >- end >-end >- >-def verify_user(my_account, my_token) >- query = {} >- query['my_account'] = my_account if my_account >- query['my_token'] = my_token if my_token >- >- result = ES_ACCOUNTS.multi_field_query(query)['hits']['hits'] >- return nil if result.size == 0 >- >- result[0]['_source']['my_token'] == my_token >-end >- >-def build_account_query(query, my_account, query_type, index) >- if query_type == 'sql' >- user_limit = "my_account='#{my_account}'" >- if query =~ /where/i >- query = query.gsub(/FROM\s*\S+/i, "FROM #{index} ") >- query = query.gsub(/where/i, "WHERE #{user_limit} AND") >- else >- query = query.gsub(/from\s*\S+/i, "FROM #{index} WHERE #{user_limit}") >- end >- else >- query['query'] ||= {} >- query['query']['bool'] ||= {} >- query['query']['bool']['must'] ||= [] >- query['query']['bool']['must'] << {'term' => {'my_account' => my_account}} >- >- query['query'].each do |k, v| >- if ES_QUERY_KEYWORD.include?(k) >- query['query']['bool']['must'] << {k => v} >- query['query'].delete(k) >- end >- end >+def es_search_by_sql(index, params) >+ begin >+ result = EsDataApi.search_by_sql(index, params) >+ rescue StandardError => e >+ error_msg = { 'error_msg' => e.message } >+ log_error(error_msg) >+ return [200, headers.merge('Access-Control-Allow-Origin' => '*'), error_msg.to_json] > end >- >- query >+ [200, headers.merge('Access-Control-Allow-Origin' => '*'), result.to_json] > end >diff --git a/src/lib/data_api/es_data_api.rb b/src/lib/data_api/es_data_api.rb >new file mode 100644 >index 00000000..ed89b596 >--- /dev/null >+++ b/src/lib/data_api/es_data_api.rb >@@ -0,0 +1,101 @@ >+# SPDX-License-Identifier: MulanPSL-2.0+ >+# Copyright (c) 2020 Huawei Technologies Co., Ltd. All rights reserved. >+# frozen_string_literal: true >+ >+CCI_SRC ||= ENV['CCI_SRC'] || '/c/compass-ci' >+require 'json' >+require 'set' >+require "#{CCI_SRC}/lib/constants.rb" >+require "#{CCI_SRC}/lib/es_client.rb" >+ >+# this module is for es api >+# - search >+# - search_by_sql >+module EsDataApi >+ ES_ACCOUNTS = ESClient.new(index: 'accounts') >+ OPEN_INDEX = Set.new(['jobs']) >+ REQUIRED_TOKEN_INDEX = Set.new(['jobs']) >+ ES_QUERY_KEYWORD = Set.new(%w[term match]) >+ >+ def self.credentials_for_dsl(query, my_account) >+ query['query'] ||= {} >+ query['query']['bool'] ||= {} >+ query['query']['bool']['must'] ||= [] >+ query['query']['bool']['must'] << { 'term' => { 'my_account' => my_account } } >+ query = handle_dsl_query(query) >+ return query >+ end >+ >+ def self.handle_dsl_query(query) >+ query['query'].each do |k, v| >+ if ES_QUERY_KEYWORD.include?(k) >+ query['query']['bool']['must'] << { k => v } >+ query['query'].delete(k) >+ end >+ end >+ return query >+ end >+ >+ def self.check_my_account(request_body) >+ cci_credentials = request_body['cci_credentials'] || {} >+ my_account = cci_credentials['my_account'] >+ my_token = cci_credentials['my_token'] >+ raise 'user authentication failed, please check my_account and my_token.' unless verify_user(my_account, my_token) >+ >+ return my_account >+ end >+ >+ def self.search(index, params) >+ request_body = JSON.parse(params) >+ query = request_body['query'] || { 'query' => {} } >+ raise "#{index} is not opened for user query" unless OPEN_INDEX.include?(index) >+ >+ if REQUIRED_TOKEN_INDEX.include?(index) >+ my_account = check_my_account(request_body) >+ query = credentials_for_dsl(query, my_account) >+ end >+ es = Elasticsearch::Client.new(hosts: ES_HOSTS) >+ return es.search index: index + '*', body: query >+ end >+ >+ def self.credentials_for_sql(query, my_account, index) >+ user_limit = "my_account='#{my_account}'" >+ if query =~ /where/i >+ query = query.gsub(/FROM\s*\S+/i, "FROM #{index} ") >+ query = query.gsub(/where/i, "WHERE #{user_limit} AND") >+ else >+ query = query.gsub(/from\s*\S+/i, "FROM #{index} WHERE #{user_limit}") >+ end >+ return query >+ end >+ >+ def self.search_by_sql(index, params) >+ request_body = JSON.parse(params) >+ query = request_body['query'] >+ raise "#{index} is not opened for user query" unless OPEN_INDEX.include?(index) >+ >+ if REQUIRED_TOKEN_INDEX.include?(index) >+ my_account = check_my_account(request_body) >+ query = credentials_for_sql(query, my_account, index) >+ end >+ >+ es = ESClient.new(index: index) >+ p query >+ p 1 >+ return es.search_by_sql(query).body >+ end >+ >+ def self.verify_user(my_account, my_token) >+ raise 'missed my_account' unless my_account >+ raise 'missed my_token' unless my_token >+ >+ query = {} >+ query['my_account'] = my_account >+ query['my_token'] = my_token >+ >+ result = ES_ACCOUNTS.multi_field_query(query)['hits']['hits'] >+ return nil if result.empty? >+ >+ result[0]['_source']['my_token'] == my_token >+ end >+end >-- >2.23.0 >

1 0

Re: [PATCH v4 compass-ci] data-api: add es search api
by Wu Fengguang 15 Feb '22

15 Feb '22

On Mon, Feb 14, 2022 at 06:25:41PM +0800, weijihui wrote: >On Fri, Feb 11, 2022 at 06:51:34PM +0800, Wu Fengguang wrote: >>On Thu, Feb 10, 2022 at 09:25:37PM +0800, Wei Jihui wrote: >>>From: Wei Jihui <weijihuiall(a)163.com> >>> >>>1. /data_api/es/:index/_search is es search by dsl >> >>>2. /data_api/es/:index/_search_by_sql is es search by sql >> >>建议用 >> /data_api/_opendistro/_sql > >但是现在用的不是这个接口，用的是https://github.com/NLPchina/elasticsearch-sql 两个repo有继承性，基本是兼容的。要么也可以用NLPchina的接口，怎么也比现在的这个好: 2. /data_api/es/:index/_search_by_sql is es search by sql Thanks, Fengguang

2 1

Re: [PATCH v4 compass-ci] data-api: add es search api
by weijihui 14 Feb '22

14 Feb '22

On Fri, Feb 11, 2022 at 06:53:02PM +0800, Wu Fengguang wrote: >>+# "cci_authorization": { >>+# "my_token": "16132550-...", >>+# "my_account": "auto-submit" >>+# } > >>+# "cci_authorization": { >>+# "my_token": "16132550-...", >>+# "my_account": "auto-submit" >>+# } > >s/cci_authorization/cci_credentials/ > ok Thanks, Jihui >Thanks, >Fengguang

1 0

Re: [PATCH v4 compass-ci] data-api: add es search api
by weijihui 14 Feb '22

14 Feb '22

On Fri, Feb 11, 2022 at 06:51:34PM +0800, Wu Fengguang wrote: >On Thu, Feb 10, 2022 at 09:25:37PM +0800, Wei Jihui wrote: >>From: Wei Jihui <weijihuiall(a)163.com> >> >>1. /data_api/es/:index/_search is es search by dsl > >>2. /data_api/es/:index/_search_by_sql is es search by sql > >建议用 > /data_api/_opendistro/_sql 但是现在用的不是这个接口，用的是https://github.com/NLPchina/elasticsearch-sql Thanks, Jihui > >:index 有可能数据重复，此时不可信。 > >因为真正起作用的是 select 语句里的from， >必需解析后得到index，再基于它做认证。 > >Thanks, >Fengguang >

1 0

Re: [meeting] stats rules
by Wu Fengguang 14 Feb '22

14 Feb '22

># stats > > global stats pattern > > *.(log|message|element): > type: string - .log 出自下面的TODO - .log 命名不够清楚，是不是改成.logfile；类似的.message .element也可以改长一点 - .xxx 后缀有很小的可能与正常id混淆，改成#logfile#或者类似明显不同寻常的后缀，区分度较好，会比较安全和明显 Thanks, Fengguang --- [TODO] virttest stats TODO1 ===== (1/1) type_specific.io-github-autotest-libvirt.libvirt_usb_hotplug_device.attach_device.one_count.storage: FAIL: failed to attach device.\nDetail: command: '/bin/virsh attach-device avocado-vt-vm1 /tmp/lkp/xml_utils_temp_7xma5yyz.xml'\nexit_status: 1\nduration: 0.029788970947265625\ninterrupted: False\npid: 30074\nencoding: 'UTF-8'\nstdout: '\n'\nstderr: "error: Failed to a... (49.09 s) ==> libvirt_usb_hotplug_device.attach_device.one_count.storage.fail: 1 (for bisect) feature3.casexxx.fail: 1 TODO2 ===== for bisect report email feature3.casexxx.message => (1/1) type_specific.io-github-autotest-libvirt.libvirt_usb_hotplug_device.attach_device.one_count.storage: FAIL: failed to attach device.\nDetail: command: '/bin/virsh attach-device avocado-vt-vm1 /tmp/lkp/xml_utils_temp_7xma5yyz.xml'\nexit_status: 1\nduration: 0.029788970947265625\ninterrupted: False\npid: 30074\nencoding: 'UTF-8'\nstdout: '\n'\nstderr: "error: Failed to a... (49.09 s) TODO3 ===== for email/web report feature3.casexxx.log => job-2020-12-08T20.08-3bf7720/job.log (in result_root/) JOB LOG : /root/avocado/job-results/job-2020-12-08T20.08-3bf7720/job.log TODO4 ===== featureXXX.result.ok => featureXXX.nr_pass featureXXX.result.all => featureXXX.nr_total featureXXX.result.rate => featureXXX.pass_rate (perhaps not that useful, since the total pass_rate looks more useful, which can only be calculated by the framework across multi-jobs) RESULTS : PASS 0 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 => stats => libvirt_usb_hotplug_device.nr_pass: 0 (if 0, may not print this line) libvirt_usb_hotplug_device.nr_fail: 1 (for account report) libvirt_usb_hotplug_device.nr_error: 0 libvirt_usb_hotplug_device.nr_warn: 0 libvirt_usb_hotplug_device.nr_total: 1 Thanks, Fengguang

1 0

[meeting] stats rules
by Lu Weitao 14 Feb '22

14 Feb '22

# type fail 0/1 counter N perf 自然数，float text [string]/string datetime? epoch? number? is array or scalar? is kpi? is negative corelation? perf split to throughput, latency, power? # per-script definition etc/failure ~/lkp-tests/etc/event-counter-prefixes ~/lkp-tests/etc/independent-counter-prefixes 1|index-perf-all.yaml 2|index-perf.yaml 3|perf-metrics-patterns 4|perf-metrics-prefixes 5|perf-metrics-threshold.yaml are all global what if a user to add one single test case and script? best to only define it in a self maintained dir with yaml description tests/unixbench + tests/unixbench.yaml os_depends: ubuntu: unixbench input.params: # will be indexed by ES nr_task: description: #type: int # for lint and catch early typo at submit time unit: default: test: input.env: # won't be indexed by ES output.stats: score match by name by regex by prefix type: record: by regex start_time .*\.message result_root/stats.json result_root/record.json # stats global stats pattern *.(log|message|element): type: string script stats pattern buildinfo\..*: type: hash example proc-vmstat pattern nr_.*: type perf pg.*: counter xxx yyy zzz: counter etc/event-counter-prefixes:proc-vmstat.nr_promoted etc/event-counter-prefixes:proc-vmstat.nr_promote_fail etc/event-counter-prefixes:proc-vmstat.nr_promote_ratelimit etc/event-counter-prefixes:proc-vmstat.nr_promote_isolate_fail etc/event-counter-prefixes:proc-vmstat.allocstall etc/event-counter-prefixes:proc-vmstat.compact_ etc/event-counter-prefixes:proc-vmstat.htlb_buddy_alloc_ etc/event-counter-prefixes:proc-vmstat.kswapd_ etc/event-counter-prefixes:proc-vmstat.nr_dirtied etc/event-counter-prefixes:proc-vmstat.nr_vmscan_immediate_reclaim etc/event-counter-prefixes:proc-vmstat.nr_written etc/event-counter-prefixes:proc-vmstat.numa_ etc/event-counter-prefixes:proc-vmstat.pageoutrun etc/event-counter-prefixes:proc-vmstat.pg etc/event-counter-prefixes:proc-vmstat.pswp etc/event-counter-prefixes:proc-vmstat.slabs_scanned etc/event-counter-prefixes:proc-vmstat.thp_ etc/event-counter-prefixes:proc-vmstat.unevictable_pgs_ etc/event-counter-prefixes:proc-vmstat.zone_reclaim_failed problem: this is not clean namespace split log.xxx: xxx: solution1: move log out of stats? job.stats.buildinfo.hash from $(stats stdout), $script.yaml job.record.buildinfo.env from $script.record.yaml also job.stats.xxx.message: string/[] stats only has number types? # stats namespace change implement

1 0

[todo] auto submit cci-makepkg for all tests
by Cao Xueliang 14 Feb '22

14 Feb '22

# auto submit cci-makepkg before run test suite example: - job.yaml --- '#! rpmbuild.yaml': suite: rpmbuild category: functional rpmbuild: repo_addr: https://vault.centos.org/6.10/os/Source/SPackages/ed-1.1-3.3.el6.src.rpm arch: aarch64 custom_repo_name: unstable/compatible/6 mount_repo_addr: https://api.compass-ci.openeuler.org:20018/rpm/testing/openeuler-20.03-LTS-… 1/compat-centos6/standard_\$basearch/\$basearch mount_repo_name: compatible-6 docker_image: openeuler:20.03-LTS-SP1 ``` - generate cci-makepkg.yaml # detect $LKP_SRC/pkg/rpmbuild/PKGBUILD first # detect /srv/initrd/pkg/container/openeuler/aarch64/20.03-LTS-SP1/rpmbuild/latest.cgz ``` suite: cci-makepkg category: functional cci-makepkg: benchmark: rpmbuild # from rpmbuild.yaml os: # from rpmbuild.yaml os_arch: # from rpmbuild.yaml os_mount: # from rpmbuild.yaml ``` - submit cci-makepkg.yaml & rpmbuild.yaml - if cci-makepkg job success and exists latest.cgz then submit rpmbuild.yaml - if cci-makepkg job not success then echo error & no need submit rpmbuild.yaml

1 0

Re: [PATCH v3 compass-ci] data-api: add es search api
by Wu Fengguang 11 Feb '22

11 Feb '22

On Wed, Feb 09, 2022 at 09:23:20PM +0800, weijihui wrote: >On Wed, Feb 09, 2022 at 01:21:14PM +0800, Wu Fengguang wrote: >>On Tue, Feb 08, 2022 at 03:10:57PM +0800, Wei Jihui wrote: >>>From: Wei Jihui <weijihuiall(a)163.com> >>> >>>1. /data_api/es/:index/es_search is es search by dsl >> >>是这样的吧： >> >> es_search => _search > >yes > >> >>>2. /data_api/es/:index/es_search_by_sql is es search by sql >> >>我看你后面用了 >> >> >+# curl -X POST localhost:10005/data_api/es/jobs/_search_by_sql >> >>然后调用的是 >> >> >+ @client.perform_request('GET', '_nlpcn/sql', {}, search_sql) >> >>但我们应该最终调用这个吧: >> >> https://github.com/opendistro-for-elasticsearch/sql >> >> POST https://<host>:<port>/_opendistro/_sql >> { >> "query": "SELECT * FROM my-index LIMIT 50" >> } >> >>相应的API是不是相应的写成 >> >> /data_api/_opendistro/_sql > >这个接口需要安装的插件没有我们用的es版本 >目前用的这个接口 >_nlpcn/sql 可以计划下升级到 _opendistro 和与之匹配的ES。 API层面，可以先用 _opendistro 免得折腾。 >> >>>-# curl -X POST localhost:10005/es_find -H 'Content-Type: application/json' -d '{ >>>+# curl -X POST localhost:10005/data_api/es/jobs/_search -H 'Content-Type: application/json' -d '{ >>> # "query": {"size":10, "sort": [{"start_time": {"order": "desc"}}]}, >>>-# "index": "jobs", >>>-# "my_token": "16132550-...", >>>-# "my_account": "auto-submit", >>>-# "query_type": "eql" # can be omitted in this scenario >>>+# "cci_authorization": { >> >>这个貌似不错： >> >> credentials > >cci_credentials? Yes. Thanks, Fengguang

1 0

Re: [PATCH todo 1/2] add todo for docker improvement
by Luan Shengde 11 Feb '22

11 Feb '22

pls ignore this mail Thanks, Luan Shengde On Fri, Feb 11, 2022 at 05:05:52PM +0800, Luan Shengde wrote: >Signed-off-by: Luan Shengde <shdluan(a)163.com> >--- > .../todo/20220216-5d:docker-improvement.md | 35 +++++++++++++++++++ > 1 file changed, 35 insertions(+) > create mode 100644 people/LuanShengde/todo/20220216-5d:docker-improvement.md > >diff --git a/people/LuanShengde/todo/20220216-5d:docker-improvement.md b/people/LuanShengde/todo/20220216-5d:docker-improvement.md >new file mode 100644 >index 00000000..416c749d >--- /dev/null >+++ b/people/LuanShengde/todo/20220216-5d:docker-improvement.md >@@ -0,0 +1,35 @@ >+# docker improvement >+ >+## handle download failure and check dir besides /lkp and /opt >+ >+original: >+ curl_cmd: %x(curl -sS --create-dirs -o #{path}/#{name} #{url} && gzip -dc #{path}/#{name} | cpio -idu -D #{path}) >+ download cgz file and unpack it in one line >+ >+improvement: >+ curl_cmd: split download cgz file, check dir and unpack the cgz file into standalone lines. >+ system('wget -t 3 -T 20 -O #{path}/#{name} #{url} 2> /dev/null') >+ use system to call the linux command, it will retusn true/false >+ - true: execute success >+ - false: execute in failure >+ raise error if download in failure >+ >+ %x(gzip -dc path/cgz_file | cpio -idt) >+ use %x(command) can get the output of the command, extract the file list of the cgz file >+ check files from the file list and check if the file/dir is besides the /lkp or /opt >+ >+ if no files/dirs besides /lkp or /opt >+ %x(gzip -dc path/cgz_file | cpio -idu -D path) >+ >+ if exist files/dirs besides /lkp or /opt >+ save the cgz to opt >+ unpack the cgz file in container >+ >+ --- >+ maybe can ignore check the cgz file if it containers files/dirs besides /lkp or /opt >+ just unpack all the cgz file in the container >+ --- >+ >+## /usr/local/bin/busybox should be ro >+ >+## cancel mount unnecessary dir >-- >2.23.0 >

1 0

2024

2023

2022

2021

2020

Compass-ci