50688c1879
Fix issue#7527, add desc sort function in IBrowserLogQueryDAO of H2 and ES ( #7580 )
2021-08-26 22:36:24 +08:00
dc05539361
DataCarrier changes a `#consume` API to add properties as a parameter to initialize consumer ( #7452 )
2021-08-13 14:05:10 +08:00
6a241aed75
[Break Change] Remove endpoint name in backend log and browser log query ( #7444 )
...
* Remove page path in the browser log query condition.
* Remove endpoint name in the backend log query condition.
2021-08-12 09:44:19 +08:00
de6cae3429
[Break Change] Remove endpoint name in the trace query condition. ( #7438 )
...
* [Break Change] Remove endpoint name in the trace query condition.
* Fix a local UT error(MacOS only)
2021-08-11 16:27:10 +08:00
cc66254729
Remove statement, version, and endtime from segment entity. ( #7427 )
2021-08-10 00:05:06 +08:00
1c5e22a7be
Begin 8.8.0 iteration ( #7395 )
...
* [maven-release-plugin] prepare release v8.7.0
* [maven-release-plugin] prepare for next development iteration
2021-07-30 23:01:09 +08:00
ffc69dc6f3
ES storage support set order of index template. ( #7378 )
2021-07-27 20:13:31 +08:00
ea76335527
Avoid "select *" query ( #7372 )
2021-07-27 15:50:55 +08:00
383e32b44b
Fix the persistent session timeout mechanism bug. ( #7347 )
2021-07-22 05:50:19 +08:00
3a4ee08e54
Enhance persistent session timeout mechanism. ( #7334 )
...
Fix bug, the enhanced session could cache the metadata metrics(hot entity) forever. A new timeout mechanism is designed for avoiding this specific case.
Optimize this timeout mechanism, make it different for ES(one index per day) and non-ES storage implementation.
2021-07-20 07:50:27 +08:00
8df362b929
Adjust ElasticSearch index refresh period as INT(flushInterval * 2/3) ( #7310 )
...
Adjust index refresh period as INT(flushInterval * 2/3), it used to be as same as bulk flush period. At the edge case, in low traffic(traffic < bulkActions in the whole period), there is a possible case, 2 period bulks are included in one index refresh rebuild operation, which could cause version conflicts. And this case can't be fixed through core/persistentPeriod as the bulk fresh is not controlled by the persistent timer anymore.
This PR should avoid the following exception in the low load case, especially when bulkActions is set larger than the number of a metric type.
2021-07-16 12:10:35 +08:00
408bbf86e6
Fix a misused variable. ( #7309 )
2021-07-15 19:31:50 +08:00
c59ee90029
Performance: optimize IDs read of ElasticSearch storage options(6 and 7) ( #7307 )
...
* Performance: optimize IDs read of ElasticSearch storage options(6 and 7). Use the physical index rather than template alias name.
2021-07-15 18:06:11 +08:00
de975c7e2b
Remove synchronous persistence mechanism in API level and ElasticSearch implementation ( #7293 )
...
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)
Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124
Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.
Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
2021-07-14 17:19:49 +08:00
d79094c178
Add pagination to event query ( #7274 )
2021-07-09 23:04:50 +08:00
022bf38116
Fix InfluxDB storage-plugin Metrics#multiGet issue ( #7238 )
2021-07-03 13:58:27 +08:00
536b7d2328
Support connectTimeout and socketTimeout settings ( #7214 )
2021-07-01 08:20:59 +08:00
528ee6de40
OAL supports generating metrics from events ( #7134 )
...
* OAL supports generating metrics from events
* Add new dashboard for event metrics
* Add some docs for event metrics
2021-06-19 20:20:37 +08:00
402902f75c
Agent supports the collection of JVM arguments and jar dependencies information. ( #7069 )
2021-06-17 15:30:44 +08:00
5eb1021639
Initialize 8.7.0 iteration ( #7078 )
2021-06-08 07:41:09 +08:00
bb590daba9
Include event(s) to alarms. ( #6888 )
...
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-05-19 14:14:48 +08:00
3b15f8d132
feature: Envoy access log receiver supports TCP logs ( #6727 )
2021-05-07 07:50:03 +08:00
7a53331367
Support alarm tags ( #6820 )
2021-04-25 17:22:43 +08:00
1fb2b01b7e
fix: Receiver_zipkin missing ServiceInstanceID bug #6749 ( #6763 )
2021-04-19 11:55:43 +08:00
17e820f156
Begin 8.6.0 iteration ( #6721 )
2021-04-09 23:01:00 +08:00
58854af662
Fix slash root issues in es7 client ( #6707 )
...
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2021-04-08 11:19:00 +08:00
2af77d4d03
make sync metrics concurrency ( #6642 )
...
* make sync metrics concurrency
* add changelog
* add changelog
* polish codes
* change default value
* remove unnecessary codes
Co-authored-by: Evan <evanljp@outlook.com>
Co-authored-by: Zhenxu Ke <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-04-02 11:33:46 +08:00
bfef6d9120
Fix typos ( #6568 )
2021-03-18 09:58:47 +08:00
fd1d21021f
[Incompatible Enhancement]New index policy of ElasticSearch storage option ( #6499 )
2021-03-11 15:11:37 +08:00
0382447749
Fix tags store of log and trace on h2/mysql/pg storage ( #6505 )
2021-03-11 10:40:54 +08:00
e9196b7e20
remove meaningless code ( #6508 )
2021-03-08 12:37:10 +08:00
0e1d5c3767
Make Zipkin trace query available ( #6454 )
2021-02-27 14:50:54 +08:00
c45bb361ee
Storage plugin supports PostgreSQL ( #6345 )
2021-02-20 20:59:51 +08:00
1adea8a16e
Delete Es7DAO.java ( #6397 )
2021-02-19 07:52:27 +08:00
ba6d49ad91
Update the timestamp field type for LogQuery ( #6335 )
2021-02-07 12:09:51 +08:00
5e8f1eb552
Support multiple implementations of StorageBuilder in different storage implementations - stage 2 ( #6336 )
2021-02-07 10:37:14 +08:00
fc23dabf37
Support multiple implementations of StorageBuilder in different storage implementations - stage 1. ( #6334 )
2021-02-06 21:22:58 +08:00
b92eac2cd6
Add a new concept "Event" and its implementations to collect events ( #6183 )
2021-02-05 18:37:18 +08:00
d4ea997523
chore: add @Override tag on some methods ( #6304 )
2021-02-03 09:46:57 +08:00
cdfe6ab358
Begin the 8.5.0 iteration ( #6302 )
...
* [maven-release-plugin] prepare release v8.4.0
* [maven-release-plugin] prepare for next development iteration
2021-02-02 08:01:49 +08:00
93165abc2a
Multiple exclude and include query conditions don't work as expected. ( #6295 )
2021-02-01 00:33:20 +08:00
5d47d4d29f
Remove state / isError and collect exception stacktrace in error logs ( #6275 )
2021-01-30 08:18:52 +08:00
01d1eff6c7
Enhancement log-recerive e2e test ( #6248 )
2021-01-29 00:07:16 +08:00
e138d29778
add text analyzer for es ( #6249 )
2021-01-25 17:01:43 +08:00
0bd8149596
LogQuery remove unused field ( #6246 )
2021-01-24 19:13:31 +08:00
98d0430b91
chore: set up license-eye to check license header ( #6200 )
2021-01-14 22:08:37 +08:00
15452025c6
Log collecting and query implementation ( #6120 )
2021-01-14 09:49:11 +08:00
e3fbb3f705
Fix some influxdb plugin bug ( #6139 )
2021-01-08 14:49:44 +08:00
6e48dca99a
Improvement Influxdb query performance ( #6066 )
2020-12-30 19:54:33 +08:00
8899930082
fix influx query error #6056 ( #6057 )
...
* fix influx query error
* use InfluxConstants.NAME
2020-12-22 20:25:36 +08:00
8d33f0275e
chore: code polish ( #6025 )
2020-12-17 22:54:36 +08:00
957d549b68
[ISSUE 5541] Fix H2 in-memory database table missing issues ( #6008 )
...
* add ;DB_CLOSE_DELAY=-1 to the database URL
* Update CHANGES log
* add ;DB_CLOSE_DELAY=-1 to the H2 database URL of application.yml
Co-authored-by: zhanglei36 <zhanglei36@asiainfo.com>
2020-12-15 16:05:03 +08:00
e792da1782
fix influxdb bug while setting influxdb connection response format as InfluxDB.ResponseFormat.JSON ( #5946 )
2020-12-07 22:56:03 +08:00
af19c72a7e
8.3.0 tagged, open for 8.3.0 iteration. ( #5920 )
...
* [maven-release-plugin] prepare release v8.3.0
* [maven-release-plugin] prepare for next development iteration
2020-11-29 11:01:20 +08:00
94b205f23a
fix index not found on trace query when using es7 ( #5911 )
...
* fix index not found on trace query when using es7
* add changelog
Co-authored-by: Evan <evanljp@outlook.com>
2020-11-27 16:59:21 +08:00
9a61835cb7
Fix influxdb config resubmit ( #5903 )
2020-11-26 19:14:48 +08:00
b39c4306a8
Observing Istio control plane ( #5835 )
...
* Add Istio control plane otel MAL expressions and UI template
* Remove OpenCensus receiver, add OpenTelemetry receiver
* Transfer oc receiver to ot receiver
* Create indices on boot
* Document istio metrics and otel receiver
* Update CHANGES.md
* Fix potential NPE
* Add group to meter and cleanup istio mixer
* Update e2e case due to the change of meter metric name
* Update prometheus fetcher config, fix sql builder issue of h2
* Disable istio and so11y metrics by default.
* Fix micrometer testcase
2020-11-25 21:07:03 +08:00
8d164d3dd8
Add Tidb provider module. ( #5844 )
...
* fix transaction too large error when use TiDB as storage
* add CHANGES.md
* fix typo
* add tidb storage provider module.
* add e2e.
* revert ui submodule.
* fix license.
* Fix wrong health check test
* add tidb ttl e2e.
Co-authored-by: moonsphere <moonsphere@m.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-11-22 23:28:54 +08:00
c9202af1ee
Support group in the service traffic and service list query. ( #5851 )
...
* Support group in the service traffic and service list query.
2020-11-17 22:45:32 +08:00
ef1363d928
remove unused min date timebucket in jdbc delethistory logical. ( #5845 )
...
* remove unused min date timebucket in jdbc delethistory logical.
* update CHANGES.md
2020-11-15 19:28:38 +08:00
3a2706d104
Fix storage-influxdb-plugin serviceCode query ( #5792 )
...
* Fix storage-influxdb-plugin serviceCode query
* Update CHANGES.md
Add the changelog for the PR as I have time.
Co-authored-by: yuanguohua <yuanguohua@fin-shine.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-11-04 19:39:06 +08:00
9042ce6fe2
fix storage-jdbc isExists not set dbname ( #5786 )
...
* fix null catalog
* update CHANGES.md
2020-11-04 11:37:54 +08:00
a9d756cd53
Fix deadlock problem when using elasticsearch-client-7.0.0 ( #5775 )
...
* Update elasticsearch client 7.0.0 to elasticsearch client 7.5.0.
* Update CHANGES.md and LICENSE.
2020-11-03 22:01:54 +08:00
f92a7bf897
Start next iteration 8.3.0 ( #5719 )
...
* [maven-release-plugin] prepare release v8.2.0
* [maven-release-plugin] prepare for next development iteration
2020-10-24 19:30:29 +08:00
287b8b9be5
Add syncBulkActions config to avoid the large amount of metrics data written ES in a single sync bulk request. ( #5699 )
2020-10-21 12:23:32 +08:00
d86570a88e
Health check for InfluxDB ( #5637 )
2020-10-13 16:59:42 +08:00
a5f4052965
fix INEFFICIENT_KEYSET_ITERATOR ( #5612 )
...
* fix INEFFICIENT_KEYSET_ITERATOR
2020-10-06 21:44:23 +08:00
15b0068fdf
Add superdataset replicas number config ( #5424 )
...
* distinct the number of metrics or common record index with super dataset index.
2020-09-01 23:06:21 +08:00
7ef8b3e476
Support Browser protocol at OAP ( #4228 )
2020-08-31 15:46:37 +08:00
5bfac1b50f
Remove style-check exception for logger ( #5363 )
2020-08-21 18:26:25 +08:00
68eba154c4
Add latest function ( #5321 )
...
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-08-15 17:28:32 +08:00
197ef1e836
Improve Es trace basic query performance ( #5132 )
2020-08-15 08:24:37 +08:00
7f7e96b088
[IMPORTANT] Query traces with tags as condition ( #5270 )
2020-08-11 07:24:40 +08:00
2df3c683ba
Super Size Dataset record index es rolling step ( #5282 )
2020-08-10 14:49:11 +08:00
b0bb7cf3be
Query all of labels once selected labels condition is absent. ( #5202 )
2020-08-08 22:26:06 +08:00
2f2840cbf5
Begin 8.2.0 iteration. ( #5211 )
...
* [maven-release-plugin] prepare release v8.1.0
* [maven-release-plugin] prepare for next development iteration
2020-07-31 20:15:27 +08:00
966403c0e0
missing time unit in query ( #5210 )
2020-07-31 16:49:19 +08:00
2c676ead24
Add OAP configuration vocabulary documentation. ( #5149 )
2020-07-23 08:24:40 +08:00
3fdfdf3b83
Storage elasticsearch health check ( #5099 )
2020-07-19 23:37:13 +08:00
ce2469dbec
optimization MetricsEsDAO multiGet ( #5096 )
2020-07-14 22:47:37 +08:00
b6661b5164
Add health checker module ( #5046 )
2020-07-10 00:30:43 +08:00
e8f7ae055c
Fix searchService bug in ES implementation . ( #5059 )
2020-07-09 00:01:30 +08:00
d264f914d5
Fix timebucket conversion issue ( #5014 )
2020-07-02 22:05:03 +08:00
fb7912c6bd
fix fuzzy query sql injection ( #4970 )
2020-06-26 10:08:10 +08:00
d305efcb52
In no-init mode, should only check the index existence. ( #4929 )
2020-06-16 08:06:02 +08:00
5de0d2c6f6
Set version to 8.1.0 for next iteration. ( #4916 )
...
* [maven-release-plugin] prepare release v8.0.0
* [maven-release-plugin] prepare for next development iteration
* Remove all shardingsphere cases.
2020-06-12 23:44:42 +08:00
b07b395691
Fix 4 high priority bugs ( #4913 )
2020-06-12 12:07:45 +08:00
0c1b8e4967
Add default templates for dashboard and topology. ( #4876 )
2020-06-09 08:24:12 +08:00
3f176bf50f
[hotfix]Fix UITemplate storage Influxdb implamented ( #4849 )
2020-06-02 16:05:21 +08:00
d094a0f7c0
Support UI template management. ( #4843 )
2020-06-01 23:06:14 +08:00
6ed40b3395
Keep ID as ref ID in the readSampledRecords ( #4822 )
...
* Keep ID as ref ID in the readSampledRecords, as there is no meaningful logic entity, but based on the graphql, the client could request this.
2020-05-26 08:00:03 +08:00
ec175f5655
Miss the node type condition in the es searchServices query. ( #4821 )
2020-05-25 22:17:10 +08:00
1652e0c057
Add a SuperDataset tag for custom super size dataset shards config in es storage ( #4793 )
...
* add SuperDataset tag for add super size dataset shards config in es
2020-05-17 17:48:15 +08:00
cffcc58f8f
The NoneStreamEsDAO implementation doesn't use real index name to insert. ( #4796 )
2020-05-16 22:54:28 +08:00
544d419fc0
the number type casts error ( #4788 )
...
Co-authored-by: zhuanghaochao <zhuanghaochao@yy.com>
2020-05-14 21:13:22 +08:00
d4f3218dea
Support define multiple OAL configuration ( #4748 )
2020-05-08 23:53:49 +08:00
7edcd5aa33
Change the model installation into the reactive module ( #4761 )
2020-05-08 21:21:03 +08:00
1d5a0b0f1a
Format index name with namespace ( #4758 )
...
* Format index name with namespace
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Use another variable to compare index name
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-05-08 20:12:56 +08:00
fac4ddc9e8
Fix two serious bugs. ( #4741 )
2020-05-05 12:55:14 +08:00
7b071bd7f7
Fix nested query and type conversion in MySQL, remove unused codes ( #4738 )
2020-05-04 13:54:12 +08:00
刘威
吴晟 Wu Sheng
pkxiuluo
LEE
Ax1an
kezhenxu94
Daming
Ax1an
chen~
liqiangz
kl
Gao Hongtao
Evan
Jingguo Yao
hailin0
mantuliu
haoyann
Brandon Fergerson
zhang-wei
Stephen Ni
cuiweiwei
songzhe
Lei Zhang
jianweizhang
Jared Tan
gerry
zifeihan
Ax1an
xbkaishui
Kdump
Lu Jiajing
denis
yangy