The ES persistence execution is now asynchronous and the execution
latency only counts the time to insert the requests into the bulk
processor, instead of the time after the requests are flushed into the
storage, this patch fixes that issue.
There is the same issue in prepare latency but that needs more changes
so I'll leave it to another pull request.
Optimize metrics of minute dimensionality persistence.
- The value of metrics, which has declaration of the default value and current value equals the default value logically, the whole row wouldn't be pushed into database.
- Add self-observability `metrics_persistence_skipped` metric to measure the number of skipped metrics.
- SLA(percent function), Apdex, AvgLong(MAL mostly) metrics are mostly optimized because of this. Such as a full-failing error(0% successful rate) and unhealthy(apdex=0) are not costing the storage.
- Hour/Day dimensionalities metrics are not changed, as they cover long duration.
- Fix `max` function in OAL doesn't support negative long.
* Support `!= null` in OAL engine.
* Add `Message Queue Consuming Count` metric for MQ consuming service and endpoint.
* Add `Message Queue Avg Consuming Latency` metric for MQ consuming service and endpoint.
* Document with a new menu in the `tracing` catalog is added
* Refactor the OAL compiler context to improve readability. Usually, this could not be included in the change log, but considering this kind of refactor could cause not-found bugs, Adding this as a hit if we face some related issues in the future.
* Fix wrong generated codes of hashCode and remoteHashCode methods for numeric fields.
Enhance OAL engine core
* Fix `funcParamExpression` and `literalExpression` can't be used in the same aggregation function.
* Support cast statement in the OAL core engine.
* Support `(str->long)` and `(long)` for string to long cast statement.
* Support `(str->int)` and `(int)` for string to int cast statement.
* Support Long literal number in the OAL core engine.
* Support literal `string` as parameter of aggregation function.
* Add `attributeExpression` and `attributeExpressionSegment` in the OAL grammar tree to support `map` type for the
attribute expression.
Besides added UTs, I locally tested these OAL scripts
```
mq_consume_latency = from((str->long)Service.tag["transmission.latency"]).longAvg();
mq_consume_latency = from((long)Service.tag["transmission.latency"]).longAvg();
mq_consume_latency = from((str->long)Service.tag["transmission.latency"]).filter((str->long)tag["transmission.latency"] > 0L).longAvg();
```
* [maven-release-plugin] prepare release v8.8.0
* [maven-release-plugin] prepare for next development iteration
* Create changes-8.8.0.md
* Reset changelog for 8.9.0
* Fix dynamic configuration watch implementation current value not null when the config is deleted.
* add changes.md
* fix LoggingConfigWatcher watch.vale, and when value = null ,config not reset.
* fix
* fix style
* Update oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/trace/UninstrumentedGatewaysConfig.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
* use Strings.nullToEmpty
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
* fix: Fix MultiScopesAnalysisListener#setPublicAttrs log warn frequently. #6562
1. Rename current STATUS_CODE to HTTP_RESPONSE_STATUS_CODE, real tag key should be http.status_code.
2. Add a new string tag RPC_RESPONSE_STATUS_CODE to hold all string type value tag. The key is `rpc.status_code`.
3. At the backend, keep responseCode for HTTP_RESPONSE_STATUS_CODE tag, but add @Deprecated. Also, add httpResponseStatusCode(a duplicate of responseCode) and rpcStatusCode(for RPC_RESPONSE_STATUS_CODE) fields for OAL.
* Modify `Tags.STATUS_CODE` field name to `Tags.HTTP_RESPONSE_STATUS_CODE` and type from `StringTag` to `IntegerTag`, add `Tags.RPC_RESPONSE_STATUS_CODE` field to hold rpc response code value.
* Add `rpcStatusCode` for `rpc.status_code` tag in the OAL objects. The `responseCode` field is marked as deprecated and replaced by `httpResponseStatusCode` field.
* Tag `status_code` is still supported in the backend for forward compatbility. Will be removed one year later.
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* excluded openapi-definitions resource folder
jar class loader comes before runtime config folder
* update CHANGES.md
* include openapi-definitions folder in bin packages.
Co-authored-by: Halil Onur Cecen <HalilOnur.Cecen@sahibinden.com>
* Support Metric level function `serviceRelation` in `MAL`.
* Support envoy metrics binding into the topology.
* add istio args
* remove istio 1.7
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Zhenxu <kezhenxu94@apache.org>
Fix bug, the enhanced session could cache the metadata metrics(hot entity) forever. A new timeout mechanism is designed for avoiding this specific case.
Optimize this timeout mechanism, make it different for ES(one index per day) and non-ES storage implementation.
The key logic behinds all these is, metrics persistence is fully asynchronous.
* The core/maxSyncOperationNum setting(added in 8.5.0) is removed due to metrics persistence is fully asynchronous.
* The core/syncThreads setting(added in 8.5.0) is removed due to metrics persistence is fully asynchronous.
* Optimization: Concurrency mode of execution stage for metrics is removed(added in 8.5.0). The only concurrency of prepare stage is meaningful and kept.
* Remove the outside preparedRequest list initialization, worker instance could always build a suitable size list in the first place (Reduce Array.copy and GC load a little).
Adjust index refresh period as INT(flushInterval * 2/3), it used to be as same as bulk flush period. At the edge case, in low traffic(traffic < bulkActions in the whole period), there is a possible case, 2 period bulks are included in one index refresh rebuild operation, which could cause version conflicts. And this case can't be fixed through core/persistentPeriod as the bulk fresh is not controlled by the persistent timer anymore.
This PR should avoid the following exception in the low load case, especially when bulkActions is set larger than the number of a metric type.
* Performance: remove the synchronous persistence mechanism from batch ElasticSearch DAO. Because the current enhanced persistent session mechanism, don't require the data queryable immediately after the insert and update anymore.
* Performance: share `flushInterval` setting for both metrics and record data, due to `synchronous persistence mechanism` removed. Record flush interval used to be hardcoded as 10s.
* Remove `syncBulkActions` in ElasticSearch storage option.
* Increase the default bulkActions(env, SW_STORAGE_ES_BULK_ACTIONS) to 5000(from 1000).
* Increase the flush interval of ElasticSearch indices to 15s(from 10s)
Add these 2 references. According to these, **(same _index, _type and _id) in same bulk will be in order**
1. https://github.com/elastic/elasticsearch/issues/50199
2. https://discuss.elastic.co/t/order-of--bulk-request-operations/98124
Notice, the order of different bulks is not guaranteed by the ElasticSearch cluster. We are going to have the risk of dirty write. But consider we set over 20s period between flush, and index flush period is 10, we should be safe.
Recommend 5000 bulk size and 15s flush interval only. The persistent period has been set to 25s.
Groovy naturally supports many dynamic features that we don't benefit for now but cost performance loss, in this patch we compile our Groovy-based DSL scripts statically to optimize performance.
* Allow configuring max request header size
This allows configuring the HTTP max request header size from the jetty
server.
By default it uses 8192, the same jetty default.
1.0Performance: Add L1 aggregation flush period, which reduces the CPU load and helps young GC.
2. Replace do not direct send after the first aggregation to reduce the network #6400.
3. Enhance the DataCarrier to notify the consumer in no enqueue event in short term.
4. L1 aggregation flush period still works even no further metrics generated, powered by <3>
5. Fix gRPC remote client OOM. The concurrency control mechanism failed.
There are a lot of calls to `Metrics.id()` and `ISource.getEntityId`, which calculates the id by manipulating strings in every single call, producing many garbage objects.
In this patch, I cache the id and only calculate the id if it's requested for the first time.
The possible metrics names are relatively fixed in an OAP run, we previously always escape the metrics names, building a new regex Pattern from scratch, which is cpu-consuming.
In this patch, I cache the escaped metrics names, and if it missed, use a pre-compiled regex Pattern to escape the metrics name.
This patch also replace string concatenation `+` with `StringBuilder` in generated classes, which won't get optimized by Java compiler
This may reduce 1~1.5 vCPU under 20K RPS environment.
These fields in `stateMap` metadata are not used by `metadata-service-mapping.yaml` now and there is low possibility that they may be used in real case, but, they cost a lot to deserialize, so we remove them to improve performance
From what I've tested, the analyzer can process 250k ALS logs before trimming the fields and can process 420k ALS logs after trimming the fields
刘威
吴晟 Wu Sheng
xuxiawei
liqiangz
kezhenxu94
mrproliu
Kirs
zhyyu
wankai123
zhang-wei
royal-dargon
xu1009
wallezhang
业余布道师
Jared Tan
nicolchen
CoderGang
honganan
Daming
YczYanchengzhe
HendSame
CharliePu
probeyang
Kagaya
Halil Onur Çeçen
Gao Hongtao
pkxiuluo
LEE
Switch
Ax1an
Huang
Cool-Coding
whl12345
Sergi Castro
Alvin
Tomasz Pytel
Ax1an