Commit Graph

61 Commits

Author SHA1 Message Date
吴晟 Wu Sheng de975c7e2b
Remove synchronous persistence mechanism in API level and ElasticSearch implementation (#7293)
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)

Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124

Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.

Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
2021-07-14 17:19:49 +08:00
kezhenxu94 d79094c178
Add pagination to event query (#7274) 2021-07-09 23:04:50 +08:00
Daming 022bf38116
Fix InfluxDB storage-plugin Metrics#multiGet issue (#7238) 2021-07-03 13:58:27 +08:00
kezhenxu94 528ee6de40
OAL supports generating metrics from events (#7134)
* OAL supports generating metrics from events

* Add new dashboard for event metrics

* Add some docs for event metrics
2021-06-19 20:20:37 +08:00
吴晟 Wu Sheng 5eb1021639
Initialize 8.7.0 iteration (#7078) 2021-06-08 07:41:09 +08:00
chen~ bb590daba9
Include event(s) to alarms. (#6888)
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-05-19 14:14:48 +08:00
liqiangz 7a53331367
Support alarm tags (#6820) 2021-04-25 17:22:43 +08:00
吴晟 Wu Sheng 17e820f156
Begin 8.6.0 iteration (#6721) 2021-04-09 23:01:00 +08:00
zhang-wei ba6d49ad91
Update the timestamp field type for LogQuery (#6335) 2021-02-07 12:09:51 +08:00
吴晟 Wu Sheng 5e8f1eb552
Support multiple implementations of StorageBuilder in different storage implementations - stage 2 (#6336) 2021-02-07 10:37:14 +08:00
吴晟 Wu Sheng fc23dabf37
Support multiple implementations of StorageBuilder in different storage implementations - stage 1. (#6334) 2021-02-06 21:22:58 +08:00
Zhenxu Ke b92eac2cd6
Add a new concept "Event" and its implementations to collect events (#6183) 2021-02-05 18:37:18 +08:00
吴晟 Wu Sheng cdfe6ab358
Begin the 8.5.0 iteration (#6302)
* [maven-release-plugin] prepare release v8.4.0

* [maven-release-plugin] prepare for next development iteration
2021-02-02 08:01:49 +08:00
Zhenxu Ke 5d47d4d29f
Remove state / isError and collect exception stacktrace in error logs (#6275) 2021-01-30 08:18:52 +08:00
zhang-wei 01d1eff6c7
Enhancement log-recerive e2e test (#6248) 2021-01-29 00:07:16 +08:00
zhang-wei 0bd8149596
LogQuery remove unused field (#6246) 2021-01-24 19:13:31 +08:00
Zhenxu Ke 98d0430b91
chore: set up license-eye to check license header (#6200) 2021-01-14 22:08:37 +08:00
zhang-wei 15452025c6
Log collecting and query implementation (#6120) 2021-01-14 09:49:11 +08:00
cuiweiwei e3fbb3f705
Fix some influxdb plugin bug (#6139) 2021-01-08 14:49:44 +08:00
Daming 6e48dca99a
Improvement Influxdb query performance (#6066) 2020-12-30 19:54:33 +08:00
songzhe 8899930082
fix influx query error #6056 (#6057)
* fix influx query error
* use InfluxConstants.NAME
2020-12-22 20:25:36 +08:00
Daming 8d33f0275e
chore: code polish (#6025) 2020-12-17 22:54:36 +08:00
cuiweiwei e792da1782
fix influxdb bug while setting influxdb connection response format as InfluxDB.ResponseFormat.JSON (#5946) 2020-12-07 22:56:03 +08:00
吴晟 Wu Sheng af19c72a7e
8.3.0 tagged, open for 8.3.0 iteration. (#5920)
* [maven-release-plugin] prepare release v8.3.0

* [maven-release-plugin] prepare for next development iteration
2020-11-29 11:01:20 +08:00
jianweizhang 9a61835cb7
Fix influxdb config resubmit (#5903) 2020-11-26 19:14:48 +08:00
吴晟 Wu Sheng c9202af1ee
Support group in the service traffic and service list query. (#5851)
* Support group in the service traffic and service list query.
2020-11-17 22:45:32 +08:00
Jared Tan ef1363d928
remove unused min date timebucket in jdbc delethistory logical. (#5845)
* remove unused min date timebucket in jdbc delethistory logical.

* update CHANGES.md
2020-11-15 19:28:38 +08:00
gerry 3a2706d104
Fix storage-influxdb-plugin serviceCode query (#5792)
* Fix storage-influxdb-plugin serviceCode query

* Update CHANGES.md

Add the changelog for the PR as I have time.

Co-authored-by: yuanguohua <yuanguohua@fin-shine.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-11-04 19:39:06 +08:00
kezhenxu94 f92a7bf897
Start next iteration 8.3.0 (#5719)
* [maven-release-plugin] prepare release v8.2.0

* [maven-release-plugin] prepare for next development iteration
2020-10-24 19:30:29 +08:00
xbkaishui d86570a88e
Health check for InfluxDB (#5637) 2020-10-13 16:59:42 +08:00
zhang-wei 7ef8b3e476
Support Browser protocol at OAP (#4228) 2020-08-31 15:46:37 +08:00
Gao Hongtao 68eba154c4
Add latest function (#5321)
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-08-15 17:28:32 +08:00
吴晟 Wu Sheng 7f7e96b088
[IMPORTANT] Query traces with tags as condition (#5270) 2020-08-11 07:24:40 +08:00
Gao Hongtao b0bb7cf3be
Query all of labels once selected labels condition is absent. (#5202) 2020-08-08 22:26:06 +08:00
吴晟 Wu Sheng 2f2840cbf5
Begin 8.2.0 iteration. (#5211)
* [maven-release-plugin] prepare release v8.1.0

* [maven-release-plugin] prepare for next development iteration
2020-07-31 20:15:27 +08:00
Daming 966403c0e0
missing time unit in query (#5210) 2020-07-31 16:49:19 +08:00
Daming d264f914d5
Fix timebucket conversion issue (#5014) 2020-07-02 22:05:03 +08:00
吴晟 Wu Sheng 5de0d2c6f6
Set version to 8.1.0 for next iteration. (#4916)
* [maven-release-plugin] prepare release v8.0.0

* [maven-release-plugin] prepare for next development iteration

* Remove all shardingsphere cases.
2020-06-12 23:44:42 +08:00
吴晟 Wu Sheng 0c1b8e4967
Add default templates for dashboard and topology. (#4876) 2020-06-09 08:24:12 +08:00
Daming 3f176bf50f
[hotfix]Fix UITemplate storage Influxdb implamented (#4849) 2020-06-02 16:05:21 +08:00
吴晟 Wu Sheng d094a0f7c0
Support UI template management. (#4843) 2020-06-01 23:06:14 +08:00
吴晟 Wu Sheng 6ed40b3395
Keep ID as ref ID in the readSampledRecords (#4822)
* Keep ID as ref ID in the readSampledRecords, as there is no meaningful logic entity, but based on the graphql, the client could request this.
2020-05-26 08:00:03 +08:00
Daming 544d419fc0
the number type casts error (#4788)
Co-authored-by: zhuanghaochao <zhuanghaochao@yy.com>
2020-05-14 21:13:22 +08:00
zhang-wei d4f3218dea
Support define multiple OAL configuration (#4748) 2020-05-08 23:53:49 +08:00
吴晟 Wu Sheng 7edcd5aa33
Change the model installation into the reactive module (#4761) 2020-05-08 21:21:03 +08:00
吴晟 Wu Sheng fac4ddc9e8
Fix two serious bugs. (#4741) 2020-05-05 12:55:14 +08:00
吴晟 Wu Sheng 071708a4a3
Support default value in labeled-value and heatmap query. (#4711)
* Support default value in labeled-value and heatmap query.

* Update a little document.
2020-04-25 21:24:15 +08:00
吴晟 Wu Sheng 991d083378
Support typeOfMetrics query (#4686)
* Add data type to the column definition to replace the simple isValue definition. And top N query is also requiring value column now.

* Support typeOfMetrics query.
2020-04-22 13:14:59 +08:00
吴晟 Wu Sheng 418bdff277
New metrics query protocol v2 (#4679)
1. Support the new query protocol, and the v1 query protocol is still supported 
2. All `numOfxxx`/`GlobalBrief` is only a mock now, no real number.
3. typeOfMetrics service is a mock only too, follow up PR will implement this.
2020-04-21 15:39:18 +08:00
Daming 28530cd79d
Upgrade the InfluxDB storage-plugin to protocol V3 (#4641)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-04-19 20:10:59 +08:00