50a0bcbf73
Optimize L1 aggregation, enhance gRPC client between L1 and L2 aggregation. ( #7206 )
...
1.0Performance: Add L1 aggregation flush period, which reduces the CPU load and helps young GC.
2. Replace do not direct send after the first aggregation to reduce the network #6400 .
3. Enhance the DataCarrier to notify the consumer in no enqueue event in short term.
4. L1 aggregation flush period still works even no further metrics generated, powered by <3>
5. Fix gRPC remote client OOM. The concurrency control mechanism failed.
2021-06-30 19:14:20 +08:00
89549c0183
Fix the queue initial size is not set by max batch size accordingly ( #7204 )
2021-06-30 11:33:07 +08:00
16ebaade6a
Optimize IDs reading in the persistent worker. ( #7193 )
...
* Optimize IDs reading in the persistent worker.
2021-06-30 10:17:46 +08:00
f5b7c3e32d
Perf: cache metrics id and entity id ( #7201 )
...
There are a lot of calls to `Metrics.id()` and `ISource.getEntityId`, which calculates the id by manipulating strings in every single call, producing many garbage objects.
In this patch, I cache the id and only calculate the id if it's requested for the first time.
2021-06-29 22:48:51 +08:00
84a7fe30c8
Perf: cache regex pattern and result, optimize string concatenation ( #7199 )
...
The possible metrics names are relatively fixed in an OAP run, we previously always escape the metrics names, building a new regex Pattern from scratch, which is cpu-consuming.
In this patch, I cache the escaped metrics names, and if it missed, use a pre-compiled regex Pattern to escape the metrics name.
This patch also replace string concatenation `+` with `StringBuilder` in generated classes, which won't get optimized by Java compiler
This may reduce 1~1.5 vCPU under 20K RPS environment.
2021-06-29 17:46:42 +08:00
b8fadec033
fix slow db dynamic config bug ( #7192 )
2021-06-29 16:41:01 +08:00
a966eea35c
Support prepare and save metrics concurrency ( #7153 )
2021-06-29 14:34:29 +08:00
de1d0461cc
perf: trim useless metadata fields in ALS to improve performance ( #7191 )
...
These fields in `stateMap` metadata are not used by `metadata-service-mapping.yaml` now and there is low possibility that they may be used in real case, but, they cost a lot to deserialize, so we remove them to improve performance
From what I've tested, the analyzer can process 250k ALS logs before trimming the fields and can process 420k ALS logs after trimming the fields
2021-06-28 22:18:14 +08:00
d63f3ffcc1
revert #7091 ( #7189 )
2021-06-28 21:05:28 +08:00
e7bc81abbb
Perf: optimize Envoy ALS analyzer performance in high traffic load scenario (reduce ~1cpu in ~10k RPS) ( #7182 )
2021-06-27 17:56:59 +08:00
e95064d976
chore: set openAPI grouping CoreModuleConfig default value consistent with application.yaml, add re-benchmark result ( #7176 )
2021-06-26 00:08:14 +08:00
d8a5e8c15b
eliminate `now` error at a fixed rate schedule executor ( #7145 )
2021-06-24 09:24:48 +08:00
8a25ac6229
Python celery plugin ( #7159 )
2021-06-22 22:05:33 +08:00
1bc69370bf
feat: add neo4j-4.x-plugin ( #7099 )
2021-06-22 15:36:51 +08:00
3536b3c1c4
Extract dependency management to a bom ( #7136 )
2021-06-20 20:57:26 +08:00
19a068d708
Feature: support endpoint name grouping by OpenAPI definitions. ( #7130 )
2021-06-20 07:57:10 +08:00
528ee6de40
OAL supports generating metrics from events ( #7134 )
...
* OAL supports generating metrics from events
* Add new dashboard for event metrics
* Add some docs for event metrics
2021-06-19 20:20:37 +08:00
edf6fccf9c
Upgrade commons-lang3 to avoid potential NPE in some JDK versions ( #7133 )
2021-06-18 21:18:05 +08:00
402902f75c
Agent supports the collection of JVM arguments and jar dependencies information. ( #7069 )
2021-06-17 15:30:44 +08:00
76b3bece2b
add component id for python Psycopg -> PostgreSQL ( #7125 )
2021-06-17 06:30:52 +08:00
669fe1593c
Upgrade OAP dependencies ( #7119 )
...
* Introduce trivy to scan images
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Fix CVE
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Update licenses
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Remove log4j 1.x
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Update CHANGES.md
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
* Update LICENSE refer to webapp
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-06-16 12:14:39 +08:00
efcb396f73
Only count 5xx as error in Envoy ALS receiver ( #7111 )
2021-06-12 07:09:48 +08:00
7a3355f956
Use MAL to calculate JVM metrics, remove OAL dependency. ( #7091 )
2021-06-11 22:21:10 +08:00
553dd7c061
Disable spring sleuth analyzer by default ( #7106 )
2021-06-11 16:59:57 +08:00
ca155dee05
Upgrade several old dependencies ( #7090 )
2021-06-10 08:35:31 +08:00
5eb1021639
Initialize 8.7.0 iteration ( #7078 )
2021-06-08 07:41:09 +08:00
17abef4c59
polish debug logging avoid null value when segment be ignored. ( #7070 )
...
* polish debug logging avoid null value.
* update CHANGES.md.
2021-06-05 19:23:46 +08:00
56f2e86a07
CVE: upgrade snakeyaml to prevent billion laughs attack in dynamic configuration. ( #7071 )
2021-06-05 14:41:46 +08:00
3c4f2af158
change log REST endpoint with v3 prefixed ( #7062 )
2021-06-03 15:56:49 +08:00
38c182e4c0
Kafka transporter code polish ( #7032 )
2021-06-02 12:56:00 +08:00
3034963374
Fix Jetty HTTP `TRACE` issue, disable HTTP methods except `POST`. ( #7052 )
2021-06-01 16:37:56 +08:00
9e66bee7a0
Make metrics exporter still work even when storage layer failed ( #7041 )
2021-05-31 17:37:58 +08:00
bd38ec492e
Add HTTP implementation of logs reporting protocol ( #7038 )
2021-05-30 22:35:55 +08:00
f742142ca3
Add UI templates ( #7013 )
2021-05-28 14:35:27 +08:00
1c51550830
Remove inexisting config item and doc ( #7033 )
2021-05-28 08:14:04 +08:00
9701bd0944
Make the number of core worker in meter converter thread pool configurable ( #7027 )
2021-05-27 23:10:22 +08:00
68b918e3cd
Events can be configured as alarm source ( #7028 )
2021-05-27 18:04:21 +08:00
1a727585f9
Fix wrong service instance id in alarm query ( #7023 )
2021-05-27 12:05:54 +08:00
03294b81ea
Update MeterSystem.java ( #7019 )
2021-05-26 21:39:58 +08:00
379c4e1a46
Allow multiple definitions as fallback in `k8sServiceNameRule`. ( #7006 )
2021-05-25 17:33:07 +08:00
3d188e60f7
Fix counter misuse in the alarm core. Alarm can't be triggered in time. ( #7005 )
...
* Fix counter misuse in the alarm core
* Submit changelog
2021-05-25 16:37:21 +08:00
47fef5df22
Support native-json format log in kafka-fetcher-plugin ( #6952 )
2021-05-20 16:33:44 +08:00
6c90a954ed
Fetch further data only when they're requested ( #6976 )
2021-05-20 10:43:16 +08:00
bb590daba9
Include event(s) to alarms. ( #6888 )
...
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2021-05-19 14:14:48 +08:00
e7d6b5457b
Fix: `!=` is not supported in oal when parameters are numbers ( #6968 )
2021-05-19 09:10:04 +08:00
a1c5dc62a0
FIX: register Watcher ( #6956 )
2021-05-17 17:48:46 +08:00
d56b0b0d41
add Azure Functions http trigger component ID ( #6953 )
2021-05-16 07:21:06 +08:00
5e40a0998a
Fix issue 6945 ( #6948 )
2021-05-15 17:38:51 +08:00
85a993faf5
Allow multiple definitions as fallback in metadata-service-mapping.yaml file ( #6933 )
2021-05-13 06:53:05 +08:00
94adbf1c1f
Bump Kubernetes client-java version ( #6920 )
2021-05-10 05:07:09 +08:00
吴晟 Wu Sheng
kezhenxu94
zhyyu
Alvin
liqiangz
wankai123
Daming
Tomasz Pytel
wallezhang
Ax1an
Gao Hongtao
mrproliu
Jared Tan
Rei Shimizu
hailin0
chen~
Leon Yang
Joe