383e32b44b
Fix the persistent session timeout mechanism bug. ( #7347 )
2021-07-22 05:50:19 +08:00
3a4ee08e54
Enhance persistent session timeout mechanism. ( #7334 )
...
Fix bug, the enhanced session could cache the metadata metrics(hot entity) forever. A new timeout mechanism is designed for avoiding this specific case.
Optimize this timeout mechanism, make it different for ES(one index per day) and non-ES storage implementation.
2021-07-20 07:50:27 +08:00
8df362b929
Adjust ElasticSearch index refresh period as INT(flushInterval * 2/3) ( #7310 )
...
Adjust index refresh period as INT(flushInterval * 2/3), it used to be as same as bulk flush period. At the edge case, in low traffic(traffic < bulkActions in the whole period), there is a possible case, 2 period bulks are included in one index refresh rebuild operation, which could cause version conflicts. And this case can't be fixed through core/persistentPeriod as the bulk fresh is not controlled by the persistent timer anymore.
This PR should avoid the following exception in the low load case, especially when bulkActions is set larger than the number of a metric type.
2021-07-16 12:10:35 +08:00
408bbf86e6
Fix a misused variable. ( #7309 )
2021-07-15 19:31:50 +08:00
c59ee90029
Performance: optimize IDs read of ElasticSearch storage options(6 and 7) ( #7307 )
...
* Performance: optimize IDs read of ElasticSearch storage options(6 and 7). Use the physical index rather than template alias name.
2021-07-15 18:06:11 +08:00
de975c7e2b
Remove synchronous persistence mechanism in API level and ElasticSearch implementation ( #7293 )
...
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)
Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124
Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.
Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
2021-07-14 17:19:49 +08:00
d79094c178
Add pagination to event query ( #7274 )
2021-07-09 23:04:50 +08:00
536b7d2328
Support connectTimeout and socketTimeout settings ( #7214 )
2021-07-01 08:20:59 +08:00
528ee6de40
OAL supports generating metrics from events ( #7134 )
...
* OAL supports generating metrics from events
* Add new dashboard for event metrics
* Add some docs for event metrics
2021-06-19 20:20:37 +08:00
5eb1021639
Initialize 8.7.0 iteration ( #7078 )
2021-06-08 07:41:09 +08:00
bb590daba9
Include event(s) to alarms. ( #6888 )
...
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-05-19 14:14:48 +08:00
7a53331367
Support alarm tags ( #6820 )
2021-04-25 17:22:43 +08:00
17e820f156
Begin 8.6.0 iteration ( #6721 )
2021-04-09 23:01:00 +08:00
2af77d4d03
make sync metrics concurrency ( #6642 )
...
* make sync metrics concurrency
* add changelog
* add changelog
* polish codes
* change default value
* remove unnecessary codes
Co-authored-by: Evan <evanljp@outlook.com>
Co-authored-by: Zhenxu Ke <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-04-02 11:33:46 +08:00
fd1d21021f
[Incompatible Enhancement]New index policy of ElasticSearch storage option ( #6499 )
2021-03-11 15:11:37 +08:00
ba6d49ad91
Update the timestamp field type for LogQuery ( #6335 )
2021-02-07 12:09:51 +08:00
5e8f1eb552
Support multiple implementations of StorageBuilder in different storage implementations - stage 2 ( #6336 )
2021-02-07 10:37:14 +08:00
fc23dabf37
Support multiple implementations of StorageBuilder in different storage implementations - stage 1. ( #6334 )
2021-02-06 21:22:58 +08:00
b92eac2cd6
Add a new concept "Event" and its implementations to collect events ( #6183 )
2021-02-05 18:37:18 +08:00
d4ea997523
chore: add @Override tag on some methods ( #6304 )
2021-02-03 09:46:57 +08:00
cdfe6ab358
Begin the 8.5.0 iteration ( #6302 )
...
* [maven-release-plugin] prepare release v8.4.0
* [maven-release-plugin] prepare for next development iteration
2021-02-02 08:01:49 +08:00
93165abc2a
Multiple exclude and include query conditions don't work as expected. ( #6295 )
2021-02-01 00:33:20 +08:00
5d47d4d29f
Remove state / isError and collect exception stacktrace in error logs ( #6275 )
2021-01-30 08:18:52 +08:00
e138d29778
add text analyzer for es ( #6249 )
2021-01-25 17:01:43 +08:00
0bd8149596
LogQuery remove unused field ( #6246 )
2021-01-24 19:13:31 +08:00
15452025c6
Log collecting and query implementation ( #6120 )
2021-01-14 09:49:11 +08:00
6e48dca99a
Improvement Influxdb query performance ( #6066 )
2020-12-30 19:54:33 +08:00
af19c72a7e
8.3.0 tagged, open for 8.3.0 iteration. ( #5920 )
...
* [maven-release-plugin] prepare release v8.3.0
* [maven-release-plugin] prepare for next development iteration
2020-11-29 11:01:20 +08:00
c9202af1ee
Support group in the service traffic and service list query. ( #5851 )
...
* Support group in the service traffic and service list query.
2020-11-17 22:45:32 +08:00
ef1363d928
remove unused min date timebucket in jdbc delethistory logical. ( #5845 )
...
* remove unused min date timebucket in jdbc delethistory logical.
* update CHANGES.md
2020-11-15 19:28:38 +08:00
f92a7bf897
Start next iteration 8.3.0 ( #5719 )
...
* [maven-release-plugin] prepare release v8.2.0
* [maven-release-plugin] prepare for next development iteration
2020-10-24 19:30:29 +08:00
287b8b9be5
Add syncBulkActions config to avoid the large amount of metrics data written ES in a single sync bulk request. ( #5699 )
2020-10-21 12:23:32 +08:00
a5f4052965
fix INEFFICIENT_KEYSET_ITERATOR ( #5612 )
...
* fix INEFFICIENT_KEYSET_ITERATOR
2020-10-06 21:44:23 +08:00
15b0068fdf
Add superdataset replicas number config ( #5424 )
...
* distinct the number of metrics or common record index with super dataset index.
2020-09-01 23:06:21 +08:00
7ef8b3e476
Support Browser protocol at OAP ( #4228 )
2020-08-31 15:46:37 +08:00
5bfac1b50f
Remove style-check exception for logger ( #5363 )
2020-08-21 18:26:25 +08:00
68eba154c4
Add latest function ( #5321 )
...
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-08-15 17:28:32 +08:00
197ef1e836
Improve Es trace basic query performance ( #5132 )
2020-08-15 08:24:37 +08:00
7f7e96b088
[IMPORTANT] Query traces with tags as condition ( #5270 )
2020-08-11 07:24:40 +08:00
2df3c683ba
Super Size Dataset record index es rolling step ( #5282 )
2020-08-10 14:49:11 +08:00
b0bb7cf3be
Query all of labels once selected labels condition is absent. ( #5202 )
2020-08-08 22:26:06 +08:00
2f2840cbf5
Begin 8.2.0 iteration. ( #5211 )
...
* [maven-release-plugin] prepare release v8.1.0
* [maven-release-plugin] prepare for next development iteration
2020-07-31 20:15:27 +08:00
2c676ead24
Add OAP configuration vocabulary documentation. ( #5149 )
2020-07-23 08:24:40 +08:00
3fdfdf3b83
Storage elasticsearch health check ( #5099 )
2020-07-19 23:37:13 +08:00
ce2469dbec
optimization MetricsEsDAO multiGet ( #5096 )
2020-07-14 22:47:37 +08:00
e8f7ae055c
Fix searchService bug in ES implementation . ( #5059 )
2020-07-09 00:01:30 +08:00
d305efcb52
In no-init mode, should only check the index existence. ( #4929 )
2020-06-16 08:06:02 +08:00
5de0d2c6f6
Set version to 8.1.0 for next iteration. ( #4916 )
...
* [maven-release-plugin] prepare release v8.0.0
* [maven-release-plugin] prepare for next development iteration
* Remove all shardingsphere cases.
2020-06-12 23:44:42 +08:00
b07b395691
Fix 4 high priority bugs ( #4913 )
2020-06-12 12:07:45 +08:00
d094a0f7c0
Support UI template management. ( #4843 )
2020-06-01 23:06:14 +08:00
Ax1an
吴晟 Wu Sheng
kezhenxu94
chen~
liqiangz
Evan
zhang-wei
Stephen Ni
Daming
Jared Tan
Ax1an
Kdump
Lu Jiajing
Gao Hongtao
denis