Commit Graph

425 Commits

Author SHA1 Message Date
hailin0 ce45397c56
Fix kafka-reporter-plugin shade package conflict (#7463)
* Fix kafka-reporter-plugin shade package conflict

Don't shade lz4, snappy or zstd package because they call native library files.
2021-08-15 21:55:45 +08:00
wallezhang 4b66f1e0a7
fix: Fix MultiScopesAnalysisListener#setPublicAttrs log warn frequently (#7271)
* fix: Fix MultiScopesAnalysisListener#setPublicAttrs log warn frequently. #6562

1. Rename current STATUS_CODE to HTTP_RESPONSE_STATUS_CODE, real tag key should be http.status_code.
2. Add a new string tag RPC_RESPONSE_STATUS_CODE to hold all string type value tag. The key is `rpc.status_code`.
3. At the backend, keep responseCode for HTTP_RESPONSE_STATUS_CODE tag, but add @Deprecated. Also, add httpResponseStatusCode(a duplicate of responseCode) and rpcStatusCode(for RPC_RESPONSE_STATUS_CODE) fields for OAL.

* Modify `Tags.STATUS_CODE` field name to `Tags.HTTP_RESPONSE_STATUS_CODE` and type from `StringTag` to `IntegerTag`, add `Tags.RPC_RESPONSE_STATUS_CODE` field to hold rpc response code value.

* Add `rpcStatusCode` for `rpc.status_code` tag in the OAL objects. The `responseCode` field is marked as deprecated and replaced by `httpResponseStatusCode` field. 

* Tag `status_code` is still supported in the backend for forward compatbility. Will be removed one year later.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-08-14 20:17:46 +08:00
吴晟 Wu Sheng dc05539361
DataCarrier changes a `#consume` API to add properties as a parameter to initialize consumer (#7452) 2021-08-13 14:05:10 +08:00
probeyang 35dd15bfbf
Add component id and icon for Python falcon plugin (#7446) 2021-08-12 12:17:01 +08:00
Fine0830 18ae82f2d1
update changlog (#7445) 2021-08-12 09:58:58 +08:00
吴晟 Wu Sheng 6a241aed75
[Break Change] Remove endpoint name in backend log and browser log query (#7444)
* Remove page path in the browser log query condition.

* Remove endpoint name in the backend log query condition.
2021-08-12 09:44:19 +08:00
Jared Tan 375138887a
update jdk 11 in dockerfile and remove unused java_opts. (#7441)
* update jdk 11 and remove unused java_opts.
2021-08-11 19:27:25 +08:00
吴晟 Wu Sheng de6cae3429
[Break Change] Remove endpoint name in the trace query condition. (#7438)
* [Break Change] Remove endpoint name in the trace query condition.

* Fix a local UT error(MacOS only)
2021-08-11 16:27:10 +08:00
Kagaya 9a05ad4b42
Add component id for go-kratos & go-mysql (#7428) 2021-08-10 15:42:20 +08:00
Yihao Chen ac4330c0bc
Add a doc section for Python agent log reporter. (#7430) 2021-08-10 15:39:09 +08:00
kezhenxu94 85e22f5124
Fix issues in ES7.14 and add log cases to ES7.14 (#7429) 2021-08-10 15:36:53 +08:00
吴晟 Wu Sheng cc66254729
Remove statement, version, and endtime from segment entity. (#7427) 2021-08-10 00:05:06 +08:00
Halil Onur Çeçen bca62f63fa
Openapi-definitions file loader fix (#7417)
* excluded openapi-definitions resource folder

jar class loader comes before runtime config folder

* update CHANGES.md

* include openapi-definitions folder in bin packages.

Co-authored-by: Halil Onur Cecen <HalilOnur.Cecen@sahibinden.com>
2021-08-07 09:53:48 +08:00
wankai123 4e7d80fb88
Support Metric level function `serviceRelation` in `MAL` and binding envoy metrics into the topology. (#7399)
* Support Metric level function `serviceRelation` in `MAL`.
* Support envoy metrics binding into the topology.
* add istio args
* remove istio 1.7

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Zhenxu <kezhenxu94@apache.org>
2021-08-06 17:09:13 +08:00
Daming 9ecb9fd87c
add event http receiver (#7406) 2021-08-04 15:53:48 +08:00
Jared Tan 9b30d0edfd
fix not found error when refresh ui. (#7403)
* fix not found error when refresh ui.

* add CHANGES.md

* Update CHANGES.md

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-08-04 08:03:03 +08:00
吴晟 Wu Sheng 651f7f16ca
Fix CVE-2021-35515, CVE-2021-35516, CVE-2021-35517, CVE-2021-36090 (#7400) 2021-08-03 12:38:08 +08:00
ThisSeanZhang 0b418bf379
Support Multiple DNS period resolving mechanism (#7398) 2021-08-02 11:16:21 +08:00
吴晟 Wu Sheng 81f7be1315
Reset changelog for 8.8.0 (#7396) 2021-07-30 23:07:20 +08:00
Fine0830 b3f81acba3
sync ui (#7393) 2021-07-30 09:53:51 +08:00
wallezhang b74778f0f4
fix: fix spring gateway plugin npe (#7389) 2021-07-29 21:08:27 +08:00
Gao Hongtao 1f28d2036a
Add logging module to load log configuration dynamically (#7377)
* Add logging module to load log configuration dynamically

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2021-07-29 19:55:22 +08:00
kezhenxu94 65e8ab3ecc
Log rate limit sampler rename `qps` to `rpm` (#7384) 2021-07-29 18:11:07 +08:00
wuwen 85669fb40f
Fix `GRPCLogClientAppender` no context warning. (#7385) 2021-07-29 16:52:52 +08:00
pkxiuluo ffc69dc6f3
ES storage support set order of index template. (#7378) 2021-07-27 20:13:31 +08:00
LEE ea76335527
Avoid "select *" query (#7372) 2021-07-27 15:50:55 +08:00
Switch 68e699550d
Add Thread and ClassLoader Metrics for the self-observability and otel-oc-rules (#7367)
* Add Thread and ClassLoader Metrics for the self-observability

* update CHANGES.md

* update OC receiver MAL settings
2021-07-25 07:32:03 +08:00
Jared Tan 90de9f9515
add sending Envoy Metrics to OAP example within envoy 1.19 and bump up to Envoy V3 api. (#7366)
* bump up envoy metrics service sinks version. and update envoy.yaml config for demo.
2021-07-23 22:59:51 +08:00
ArjenDavid-sjtu 8820287f38
Improve okhttp plugin performance by optimizing Class.getDeclaredField(). (#7364)
Simple polish only.
2021-07-23 22:00:24 +08:00
Cool-Coding a7abb1f590
remove the logic of generated instance name in KafkaServiceManagementServiceClient prepare phase(#7357) (#7358) 2021-07-22 22:41:22 +08:00
吴伟杰 da9426dd6e
Upgrade agent plugin for ElasticJob 3.0.0 GA version (#7345)
* Upgrade agent plugin for ElasticJob 3.0.0 GA version.

* Remove 3.0.0 alpha support.
2021-07-22 15:30:00 +08:00
Ax1an 383e32b44b
Fix the persistent session timeout mechanism bug. (#7347) 2021-07-22 05:50:19 +08:00
Huang bf0e22d2c1
Fix IllegalArgumentException in PrometheusMetricConverter and NaN value samples in SampleFamily (#7349) 2021-07-21 23:14:49 +08:00
Ax1an 3704567b21
Fix possible version_conflict_engine_exception in bulk execution. (#7354) 2021-07-21 20:48:52 +08:00
吴晟 Wu Sheng bd23f263e6
Revert "Adjust the project structure of the alarm module" (#7342)
This reverts commit 62411964a4.
2021-07-20 19:47:22 +08:00
wankai123 a1c53bb866
doc: Add Self Observability service discovery (k8s) (#7339) 2021-07-20 18:02:51 +08:00
Jared Tan 62411964a4
update (#7340) 2021-07-20 16:27:26 +08:00
Cool-Coding 7654161721
Fix the problem of Kafka transport topics are created duplicated with and without namespace (#7326) (#7329) 2021-07-20 08:50:36 +08:00
吴晟 Wu Sheng 3a4ee08e54
Enhance persistent session timeout mechanism. (#7334)
Fix bug, the enhanced session could cache the metadata metrics(hot entity) forever. A new timeout mechanism is designed for avoiding this specific case.

Optimize this timeout mechanism, make it different for ES(one index per day) and non-ES storage implementation.
2021-07-20 07:50:27 +08:00
kezhenxu94 4003c4b777
Publish snapshot Docker images (#7337) 2021-07-19 22:19:11 +08:00
Cool-Coding 467a95b61d
Fix the problem of -meters metrics topic isn't created with namespace prefix (#7325) (#7330) 2021-07-19 14:58:24 +08:00
吴伟杰 06e589fab0
Remove plugins for ShardingSphere legacy and unstable versions (#7323)
Remove the following plugins due to they are legacy and unstable releases.
1. sharding-jdbc-1.5.x-plugin
2. sharding-sphere-4.x-rc3-plugin
3. sharding-sphere-4.x-plugin (This is for RC1 and RC2)

To people who want to use these removing versions, you still could copy the plugins from SkyWalking's 8.6.0 release to the '/plugins' folder to activate, unless SkyWalking has some core level APIs upgrade. This is very rare to happen.
2021-07-18 20:35:54 +08:00
yujiaxinlong aa4ee27d4a
fix gateway plugin async finish repeatedly when fallback configured (#5886)
When there is a fallback in webflux (like a hystrix fallback or a webflux fallback method that involves visiting a new URL). This interceptor would be invoked twice, one for origin request and another for fallback request. The ServerWebExchange for these two aren't exactly the same one, but the former one is a delegate inside of the latter one, thus, they share the same attributes.

The problem of the old code is that get the span from attributes within the execution of Mono would get span for the latter request twice. so just move this part of code out could solve the problem.

It seems impossible to make the mono chain work like expected. So add a check about whether the best matching pattern really matches the URL. This check code is how webflux generating the best match pattern,
2021-07-17 20:50:57 +08:00
Lu Jiajing cd029df360
Chore: polish methods naming for Spring Kafka (#7320) 2021-07-17 17:09:51 +08:00
吴晟 Wu Sheng a6afcac86e
Logically revert #6642 and partial #7153 to reduce unnecessary thread and concurrency process (#7318)
The key logic behinds all these is, metrics persistence is fully asynchronous.

* The core/maxSyncOperationNum setting(added in 8.5.0) is removed due to metrics persistence is fully asynchronous.
* The core/syncThreads setting(added in 8.5.0) is removed due to metrics persistence is fully asynchronous.
* Optimization: Concurrency mode of execution stage for metrics is removed(added in 8.5.0). The only concurrency of prepare stage is meaningful and kept.
* Remove the outside preparedRequest list initialization, worker instance could always build a suitable size list in the first place (Reduce Array.copy and GC load a little).
2021-07-17 16:03:50 +08:00
wankai123 9f3ff517d1
Support envoy cluster manager metrics (#7294) 2021-07-16 15:35:48 +08:00
吴晟 Wu Sheng 8df362b929
Adjust ElasticSearch index refresh period as INT(flushInterval * 2/3) (#7310)
Adjust index refresh period as INT(flushInterval * 2/3), it used to be as same as bulk flush period. At the edge case, in low traffic(traffic < bulkActions in the whole period), there is a possible case, 2 period bulks are included in one index refresh rebuild operation, which could cause version conflicts. And this case can't be fixed through core/persistentPeriod as the bulk fresh is not controlled by the persistent timer anymore.

This PR should avoid the following exception in the low load case, especially when bulkActions is set larger than the number of a metric type.
2021-07-16 12:10:35 +08:00
吴晟 Wu Sheng c59ee90029
Performance: optimize IDs read of ElasticSearch storage options(6 and 7) (#7307)
* Performance: optimize IDs read of ElasticSearch storage options(6 and 7). Use the physical index rather than template alias name.
2021-07-15 18:06:11 +08:00
吴晟 Wu Sheng 097a8b7231
Reduce the flush period of hour and day level metrics (#7299) 2021-07-15 07:36:12 +08:00
吴晟 Wu Sheng de975c7e2b
Remove synchronous persistence mechanism in API level and ElasticSearch implementation (#7293)
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)

Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124

Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.

Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
2021-07-14 17:19:49 +08:00