Commit Graph

336 Commits

Author SHA1 Message Date
吴晟 Wu Sheng 91413a753c
Start 9.1.0-dev iteration (#8822) 2022-04-06 18:56:49 +08:00
吴晟 Wu Sheng faf6e2eb9e
Add `Error Prone Annotations` dependency to suppress warnings (#8774) 2022-04-01 00:57:34 +08:00
刘威 1554c087f4
Refactor IoTDB storage plugin and bump up iotdb-session to 0.12.5 (#8755) 2022-03-30 19:19:49 +08:00
mrproliu 2a8bf4deff
Support eBPF Profiling E2E (#8761) 2022-03-30 12:11:30 +08:00
mrproliu e4dac66e2f
Implement the eBPF profiling query and data collect protocol (#8696) 2022-03-20 19:38:16 +08:00
Kai 2bf7f0f4e7
Support ElasticSearch template mappings `properties parameters` and `_source` update (#8706)
Previous logic can't update the mappings `properties parameters` and `_source` only support adding a new property.

### Now
1. As the OAP core merges the index of metrics, properties would not be deleted still.
2. Would not update the current index to avoid conflict, a new index follows the new template to create.

**Warning**, if there are properties conflicts, same key with different values, they would override each other according to booting sequence. Unless we change booting process as merging modules before merging with existing templates and indices, we can't add check mechanism and warnings.
2022-03-19 00:55:47 +08:00
吴晟 Wu Sheng f956484e21
Add `Column.shardingKeyIdx` for column definition for BanyanDB (#8705)
* Add `Column.shardingKeyIdx` for column definition for BanyanDB.

```
Sharding key is used to group time series data per metric of one entity in one place (same sharding or same 
column for column-oriented database).
For example,
ServiceA's traffic gauge, service call per minute, includes following timestamp values, then it should be sharded by service ID
[ServiceA(encoded ID): 01-28 18:30 values-1, 01-28 18:31 values-2, 01-28 18:32 values-3, 01-28 18:32 values-4]

BanyanDB is the 1st storage implementation supporting this. It would make continuous time series metrics stored closely and compressed better.

NOTICE, this sharding concept is NOT just for splitting data into different database instances or physical files.
```

* Add `JD_PRESERVE_LINE_FEEDS=true` in official code style file. Then our comments don't have to use many blank lines to keep format.
2022-03-18 23:39:52 +08:00
吴晟 Wu Sheng a68f2d02fc
Remove the hard requirement of BASE64 encoding for the binary field (#8685)
* Remove the hard requirement of BASE64 encoding for the binary field, and resolved concerns from https://github.com/apache/skywalking/pull/8684#issuecomment-1068660880
2022-03-16 22:16:57 +08:00
Kai 91a82e056a
Add storage column attribute `indexOnly` (#8679)
* Add storage column attribute `indexOnly`, support ElasticSearch only index and not store some fields.
* Add `indexOnly=true` to `SegmentRecord.tags`, to reduce unnecessary storage.
2022-03-16 19:19:29 +08:00
吴晟 Wu Sheng 4def159162
Refactor the core Builder mechanism (#8684) 2022-03-16 08:22:18 +08:00
Kai 9c46d6f8d4
Fix es normal index couldn't apply mapping and update (#8661) 2022-03-11 10:24:17 +08:00
mrproliu 6bc2c3307a
Implement process metadata query protocol (#8649) 2022-03-10 17:08:03 +08:00
Kai f3743ee017
Introduce booster-ui and implement v9 template management protocol (#8637) 2022-03-09 08:15:55 +08:00
kezhenxu94 eb7529adb6
Support large service list query and replace deprecated API of Armeria (#8628) 2022-03-08 13:18:18 +08:00
kezhenxu94 68e40eb9fc
Support ElasticSearch 8 and add it into E2E tests (#8564) 2022-02-21 08:11:27 +08:00
kezhenxu94 eec05d8602
Set default connection timeout of ElasticSearch to 3000 milliseconds (#8560) 2022-02-19 14:08:40 +08:00
kezhenxu94 b33b78637b
Allow updating non-metrics templates when structure changed (#8549) 2022-02-16 16:42:29 +08:00
kezhenxu94 ba5e56f41a
Fix wrong position of arguments (#8548) 2022-02-16 15:13:17 +08:00
kezhenxu94 c49a82a6f5
Expose responseTimeout of ES client (#8531) 2022-02-10 14:46:40 +08:00
刘威 b0a875fe3f
Bump up iotdb-session to 0.12.4 and fix IoTDB storage option bug (#8484)
* Fix IoTDB Storage Option insert null index value.

* Set the default value of SW_STORAGE_IOTDB_SESSIONPOOL_SIZE to 8

* Bump up iotdb-session to 0.12.4

* Update doc
2022-01-30 09:02:39 +08:00
wankai123 17e8fa8ae5
Adjust e2e case for V9 core (#8429)
* Adjust e2e case for V9 core

* fix swctl install scrip

* fix cases of browser, iotdb

* fix instance serialize
2022-01-17 12:35:25 +08:00
Brandon Fergerson 14e74ad9f4
NPE fix for IndexController#getPhysicalTableName (#8397) 2022-01-09 11:56:27 +08:00
mrproliu e97b2d2165
Fix query sort metrics failure in H2 storage (#8393)
* Fix sort metrics failured in H2 storage

* Fix postgre DB
2022-01-06 15:11:05 +08:00
wankai123 8bcf22f3d8
Support OAP server v9 Core concept: Layer, backend side (#8379) 2022-01-04 16:16:24 +08:00
kezhenxu94 8fa4eb7d75
Upgrade H2 version to 2.0.202 to fix CVE-2021-23463 (#8332) 2021-12-23 21:37:00 +08:00
吴晟 Wu Sheng 1fdf3a8acf
Set version to 9.0.0-snapshot (#8242) 2021-12-06 10:04:20 +08:00
吴晟 Wu Sheng e7d6f89021
Set the version to 8.10.0, 8.9.0 release is in process. (#8233) 2021-12-02 18:54:21 +08:00
Daniel Qian e2d1ff435a
StorageModuleElasticsearchProvider doesn't watch on TrustStorePath (#8205)
Ref https://github.com/apache/skywalking/discussions/8204
2021-11-30 11:27:45 +08:00
mrproliu 1acba5d246
Remove `logback` dependencies in IoTDB plugin (#8202) 2021-11-29 18:02:54 +08:00
刘威 842b5d927e
Support Apache IoTDB as a storage option (#7766)
Support Apache IoTDB as a storage option, mostly refer to previous InfluxDB storage option.
* The Design of Apache IoTDB Storage Option, https://skywalking.apache.org/blog/2021-11-23-design-of-iotdb-storage-option/
2021-11-27 09:04:23 +08:00
刘威 6529c1d7ae
fix TopologyQuery.loadEndpointRelation bug in InfluxDB storage (#8187) 2021-11-26 23:18:16 +08:00
吴晟 Wu Sheng cef4f6dcac
A polish for comment. (#8178) 2021-11-24 12:56:15 +08:00
kezhenxu94 a2897db471
Fix the so11y latency of persistence execution latency not correct in ElasticSearch storage (#8161)
The ES persistence execution is now asynchronous and the execution
latency only counts the time to insert the requests into the bulk
processor, instead of the time after the requests are flushed into the
storage, this patch fixes that issue.

There is the same issue in prepare latency but that needs more changes
so I'll leave it to another pull request.
2021-11-20 16:51:54 +08:00
吴晟 Wu Sheng 2ea12934ce
Fix ElasticSearch implementation of `queryMetricsValues` and `readLabeledMetricsValues` doesn't fill default values (#8012)
* Fix ElasticSearch implementation of `queryMetricsValues` and `readLabeledMetricsValues` doesn't fill default values
* Change generated file path.
2021-10-27 08:27:09 +08:00
Kirs f7bcb76784
move the codes in apm-commons into server-library (#8003) 2021-10-26 20:31:36 +08:00
吴晟 Wu Sheng a68503b39f
Fix unexpected deleting due to TTL mechanism bug (#8000) 2021-10-25 22:55:42 +08:00
zhang-wei 929dc0e91f
support search browser service (#7975) 2021-10-21 15:22:43 +08:00
吴晟 Wu Sheng 55de557acf
Revert mis-change, which makes the endpoint dependency breaking. (#7829)
* Revert mis-change, which makes the endpoint dependency breaking.

* Update change log.
2021-09-29 09:06:29 +08:00
kezhenxu94 a4c68d30d8
test: fix that E2E is not run, also add ElasticSearch 7.15 to test matrix (#7826) 2021-09-28 21:57:19 +08:00
吴晟 Wu Sheng 8b681053df
Begin 8.9.0 iteration (#7801)
* [maven-release-plugin] prepare release v8.8.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-8.8.0.md

* Reset changelog for 8.9.0
2021-09-25 23:46:18 +08:00
吴晟 Wu Sheng c4b8c66174
Only report `bug warning` in precise conditions. (#7772)
* Only report `bug warning` in precise conditions.

* Add a test check for template existing.
2021-09-23 10:46:08 +08:00
kezhenxu94 ad842013f4
Replace `mget` with ids' query as `mget` doesn't apply on aliases (#7731) 2021-09-17 17:54:26 +08:00
nicolchen 224b6c6b3a
Improve the speed of writing TiDB by batching the SQL execution (#7691) 2021-09-17 00:05:18 +08:00
刘威 08990c9069
Fix H2EventQueryDAO doesn't sort data by Event.START_TIME and uses a wrong pagination query (#7720)
* Fix H2EventQueryDAO doesn't sort data by Event.START_TIME.

* Fix H2EventQueryDAO uses a wrong pagination query.
2021-09-15 19:58:41 +08:00
kezhenxu94 73b6c67344
Fix ser/deserializers of ES 6/7-7.8/7.8+, search query, aggregation, order, terms 2021-09-15 17:49:41 +08:00
kezhenxu94 b20f121e64
Fix that `sortMetrics` should only return `long` not `double` (#7698) 2021-09-12 12:31:42 +08:00
刘威 d06cb76eae
Fix ProfileThreadSnapshotQuery.queryProfiledSegments adopts a wrong sort function (#7696) 2021-09-11 22:55:56 +08:00
wankai123 4a344b01e9
Unified the config word `namespace` in the project (#7676) 2021-09-08 12:00:19 +08:00
kezhenxu94 26b81a2e59
Rebuilt ElasticSearch client on top of their REST API (#7634) 2021-09-06 15:21:35 +08:00
刘威 de59bf7aeb
Fix `H2MetadataQueryDAO.searchService` doesn't support auto grouping. [issue #7653] (#7654) 2021-09-05 13:56:00 +08:00