Commit Graph

188 Commits

Author SHA1 Message Date
吴晟 Wu Sheng 91413a753c
Start 9.1.0-dev iteration (#8822) 2022-04-06 18:56:49 +08:00
吴晟 Wu Sheng d86880ce7e
Upgrade depdencies (#8781)
* Upgrade OAP dependencies gson(2.9.0), guava(31.1), jackson(2.13.2), protobuf-java(3.18.4).
* Remove commons-pool and commons-dbcp from OAP dependencies(Not used before).
* Upgrade webapp dependencies gson(2.9.0), spring boot(2.6.5), jackson(2.13.2.2), spring cloud(2021.0.1).
2022-04-01 10:16:10 +08:00
吴晟 Wu Sheng faf6e2eb9e
Add `Error Prone Annotations` dependency to suppress warnings (#8774) 2022-04-01 00:57:34 +08:00
吴晟 Wu Sheng d96eed9145
Fix event type of export data, reduce redundancy ThreadLocal in MAL core. (#8754)
* Fix event type of export data is incorrect, it was `EventType.TOTAL` always.
* Reduce redundancy ThreadLocal in MAL core. Improve MAL performance.
* ExpressionDelegate public->private
2022-03-27 16:53:15 +08:00
Kai 2bf7f0f4e7
Support ElasticSearch template mappings `properties parameters` and `_source` update (#8706)
Previous logic can't update the mappings `properties parameters` and `_source` only support adding a new property.

### Now
1. As the OAP core merges the index of metrics, properties would not be deleted still.
2. Would not update the current index to avoid conflict, a new index follows the new template to create.

**Warning**, if there are properties conflicts, same key with different values, they would override each other according to booting sequence. Unless we change booting process as merging modules before merging with existing templates and indices, we can't add check mechanism and warnings.
2022-03-19 00:55:47 +08:00
kezhenxu94 bf03d9f67a
Improve CI with the GHA new feature (#8691) 2022-03-17 10:06:53 +08:00
Kai 91a82e056a
Add storage column attribute `indexOnly` (#8679)
* Add storage column attribute `indexOnly`, support ElasticSearch only index and not store some fields.
* Add `indexOnly=true` to `SegmentRecord.tags`, to reduce unnecessary storage.
2022-03-16 19:19:29 +08:00
kezhenxu94 289427990a
Migrate the HTTP server from Jetty to Armeria (#8673) 2022-03-15 13:38:42 +08:00
吴晟 Wu Sheng 08d481e5fc
Enhance DataCarrier#MultipleChannelsConsumer to add priority (#8664)
* Enhance DataCarrier#MultipleChannelsConsumer to add priority for the channels, which makes OAP server has a better
  performance to activate all analyzers on default.
* Activate `receiver-otel#enabledOcRules` receiver with `k8s-cluster,k8s-node,k8s-service,oap,vm` rules on default.
* Activate `satellite,spring-sleuth` for `agent-analyzer#meterAnalyzerActiveFiles`  on default.
* Activate `receiver-zabbix` receiver with `agent` rule on default.
2022-03-11 21:06:25 +08:00
kezhenxu94 b1d3f3b637
Expose more ElasticSearch health check related logs (#8652) 2022-03-10 19:19:44 +08:00
kezhenxu94 eb7529adb6
Support large service list query and replace deprecated API of Armeria (#8628) 2022-03-08 13:18:18 +08:00
kezhenxu94 68e40eb9fc
Support ElasticSearch 8 and add it into E2E tests (#8564) 2022-02-21 08:11:27 +08:00
kezhenxu94 c49a82a6f5
Expose responseTimeout of ES client (#8531) 2022-02-10 14:46:40 +08:00
吴晟 Wu Sheng 1fdf3a8acf
Set version to 9.0.0-snapshot (#8242) 2021-12-06 10:04:20 +08:00
吴晟 Wu Sheng e7d6f89021
Set the version to 8.10.0, 8.9.0 release is in process. (#8233) 2021-12-02 18:54:21 +08:00
kezhenxu94 a2897db471
Fix the so11y latency of persistence execution latency not correct in ElasticSearch storage (#8161)
The ES persistence execution is now asynchronous and the execution
latency only counts the time to insert the requests into the bulk
processor, instead of the time after the requests are flushed into the
storage, this patch fixes that issue.

There is the same issue in prepare latency but that needs more changes
so I'll leave it to another pull request.
2021-11-20 16:51:54 +08:00
Kirs 4284a57501
Add MicroBench module to make it easier for developers to write JMH Test (#7985) 2021-11-11 12:01:40 +08:00
Kirs f7bcb76784
move the codes in apm-commons into server-library (#8003) 2021-10-26 20:31:36 +08:00
kezhenxu94 508945f552
Upgrade Armeria to 1.12, upgrade OpenSearch test version to 1.1.0 (#7866) 2021-10-08 19:45:33 +08:00
kezhenxu94 a4c68d30d8
test: fix that E2E is not run, also add ElasticSearch 7.15 to test matrix (#7826) 2021-09-28 21:57:19 +08:00
吴晟 Wu Sheng 8b681053df
Begin 8.9.0 iteration (#7801)
* [maven-release-plugin] prepare release v8.8.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-8.8.0.md

* Reset changelog for 8.9.0
2021-09-25 23:46:18 +08:00
kezhenxu94 ad9d616297
Add `socketTimeout` back to the new implementation (#7798) 2021-09-24 22:35:56 +08:00
吴晟 Wu Sheng c4b8c66174
Only report `bug warning` in precise conditions. (#7772)
* Only report `bug warning` in precise conditions.

* Add a test check for template existing.
2021-09-23 10:46:08 +08:00
kezhenxu94 ad842013f4
Replace `mget` with ids' query as `mget` doesn't apply on aliases (#7731) 2021-09-17 17:54:26 +08:00
kezhenxu94 73b6c67344
Fix ser/deserializers of ES 6/7-7.8/7.8+, search query, aggregation, order, terms 2021-09-15 17:49:41 +08:00
kezhenxu94 3716219699
Fix that `Query` cannot be updated after it's set in `Search` (#7710) 2021-09-14 18:36:05 +08:00
kezhenxu94 f61dff9a9f
Merge `Dockerfile`s in apache/skywalking-docker into this codebase (#7681) 2021-09-09 12:37:50 +08:00
kezhenxu94 327d946978
Remove docker-maven-plugin in favor of testcontainers (#7664) 2021-09-06 21:04:14 +08:00
kezhenxu94 26b81a2e59
Rebuilt ElasticSearch client on top of their REST API (#7634) 2021-09-06 15:21:35 +08:00
Daming 0b67d29996
Support mTLS for gRPC channel (#7565) 2021-09-03 09:17:16 +08:00
吴晟 Wu Sheng 1c5e22a7be
Begin 8.8.0 iteration (#7395)
* [maven-release-plugin] prepare release v8.7.0

* [maven-release-plugin] prepare for next development iteration
2021-07-30 23:01:09 +08:00
pkxiuluo ffc69dc6f3
ES storage support set order of index template. (#7378) 2021-07-27 20:13:31 +08:00
吴晟 Wu Sheng de975c7e2b
Remove synchronous persistence mechanism in API level and ElasticSearch implementation (#7293)
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)

Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124

Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.

Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
2021-07-14 17:19:49 +08:00
Sergi Castro d15b67bd97
Allow configuring max request header size (#7249)
* Allow configuring max request header size

This allows configuring the HTTP max request header size from the jetty
server.
By default it uses 8192, the same jetty default.
2021-07-06 15:10:15 +08:00
Daming 5dd56a5ca4
Upgrade etcd cluster coordinator and dynamic configuration to v3.x (#7215) 2021-07-05 11:23:11 +08:00
吴晟 Wu Sheng 536b7d2328
Support connectTimeout and socketTimeout settings (#7214) 2021-07-01 08:20:59 +08:00
kezhenxu94 3536b3c1c4
Extract dependency management to a bom (#7136) 2021-06-20 20:57:26 +08:00
wankai123 19a068d708
Feature: support endpoint name grouping by OpenAPI definitions. (#7130) 2021-06-20 07:57:10 +08:00
吴晟 Wu Sheng 5eb1021639
Initialize 8.7.0 iteration (#7078) 2021-06-08 07:41:09 +08:00
Zhenxu 56f2e86a07
CVE: upgrade snakeyaml to prevent billion laughs attack in dynamic configuration. (#7071) 2021-06-05 14:41:46 +08:00
wankai123 3034963374
Fix Jetty HTTP `TRACE` issue, disable HTTP methods except `POST`. (#7052) 2021-06-01 16:37:56 +08:00
吴晟 Wu Sheng 17e820f156
Begin 8.6.0 iteration (#6721) 2021-04-09 23:01:00 +08:00
Gao Hongtao b8a040ea74
Append the root slash(/) to getIndex and getTemplate requests in ES client (#6663)
* Append the root slash(/) to getIndex and getTemplate requests

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Document this bug

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2021-04-01 14:26:57 +08:00
Gao Hongtao 7d06c210ae
Resolve envoy service name from a different position (#6654) 2021-03-31 12:09:18 +08:00
Jingguo Yao bfef6d9120
Fix typos (#6568) 2021-03-18 09:58:47 +08:00
YyWang 169640bd08
optimize moduleServiceLoader traversal logic (#6537)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-03-12 16:11:49 +08:00
Evan fd1d21021f
[Incompatible Enhancement]New index policy of ElasticSearch storage option (#6499) 2021-03-11 15:11:37 +08:00
Zhenxu Ke de6a731d72
Save Envoy http access logs when error occurs (#6322) 2021-03-02 22:40:30 +08:00
Zhenxu Ke 665a07cb0f
Clean up some warnings in maven build (#6426) 2021-02-24 07:05:56 +08:00
Zhenxu Ke b92eac2cd6
Add a new concept "Event" and its implementations to collect events (#6183) 2021-02-05 18:37:18 +08:00