Commit Graph

93 Commits

Author SHA1 Message Date
Wan Kai 47ce2720b9
BanyanDB: Support custom `TopN pre-aggregation` rules configuration in file `bydb-topn.yml`. (#13319) 2025-06-17 12:49:11 +08:00
mrproliu ad61a58037
Added Ztunnel component in the Topology (#13251) 2025-05-17 17:32:16 +08:00
Wan Kai 7dccf96f30
BanyanDB: Support `@ShardingKey` for Measure tags and set to TopNAggregation group tag by default (#13180) 2025-04-18 14:39:21 +08:00
mrproliu 3c52eedd04
Support `labelAvg` function in the OAL engine (#12940) 2025-01-08 20:56:06 +08:00
kezhenxu94 b83213705b
Add `doc_values` for fields that need to be sorted or aggregated in ElasticSearch, and disable all others. (#12782) 2024-11-24 12:17:16 +08:00
Wan Kai 2ed6b58073
Support adding additional attr[0-5] for endpoint level metrics. (#12753) 2024-11-06 12:51:26 +08:00
Wan Kai 8b6cd1c241
Add more attrs for Service metrics (#12728) 2024-10-29 15:07:53 +08:00
mrproliu baa6fcdda8
Reduce Kubernetes metrics calculate in the MQE and Add global widget in Kubernetes Dashboard (#12726) 2024-10-26 19:18:44 +08:00
Wan Kai 80d3e2951d
Support adding additional attr[0-4] for service level metrics. (#12718) 2024-10-23 16:02:06 +08:00
kezhenxu94 a5d8777ee7
Use CI-friendly revision to set the project version (#12640) 2024-09-24 17:53:14 +08:00
吴晟 Wu Sheng 054d0e4ff9
Release 10.0.0 and move forward to 10.1.0 (#12203) 2024-05-10 09:19:56 +08:00
吴晟 Wu Sheng 497a7bdb02
Release 9.7 and start 10.0 iteration (#11607) 2023-11-28 23:49:44 -08:00
yswdqz 92af797296
[Feature] GraalVM Native Image support (Experimental). (#11365) 2023-10-30 21:23:53 +08:00
yswdqz a1822201ad
[Feature] Substitute classes to prepare for Graal native-image[stage 1]. (#11354) 2023-09-27 11:03:11 +08:00
吴晟 Wu Sheng a336b3cff4
Release 9.6.0 and begin 9.7.0 iteration (#11289) 2023-09-01 22:57:18 +08:00
吴晟 Wu Sheng 1f6ea1d744
Begin 9.6.0 iteration (#10935)
* [maven-release-plugin] prepare release v9.5.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-9.5.0.md

* Set changes.md for 9.6.0
2023-06-14 18:26:50 +08:00
Jiajing LU 06558fcf5a
Integrate BanyanDB server-side TopN (#10448) 2023-04-25 10:37:55 +08:00
吴晟 Wu Sheng 4874374ace
Tag 9.4.0 release, prepare for 9.5.0 iteration. (#10512) 2023-03-08 23:30:37 +08:00
kezhenxu94 383a5bbf7b
Migrate tests from junit 4 to junit 5 (#10382) 2023-02-13 18:54:32 +08:00
kezhenxu94 184c6ca2c4
Refactor `@Column` annotation (#10367)
* Swap `Column#name` and `ElasticSearch.Column#columnAlias`
* Rename `ElasticSearch.Column#columnAlias` to `ElasticSearch.Column#legacyName`
2023-02-09 22:47:12 +08:00
zhyyu 3663a4ac5f
fix ServiceMeshServiceDispatcher overwrite ServiceDispatcher debug file when open SW_OAL_ENGINE_DEBUG. (#10187) 2022-12-22 11:22:31 +08:00
吴晟 Wu Sheng 9a7bdc8eae
Refactor `StorageData#id` to the new StorageID object from a String type. (#10157)
* Refactor kernel id0()/id() methods to return StorageID object, rather than a built string.
2022-12-15 15:39:06 +08:00
吴晟 Wu Sheng 18008aa6b2
Release 9.3.0. Begin 9.4.0 iteration (#10077) 2022-12-02 21:19:13 +08:00
吴晟 Wu Sheng 6348f16b6b
Rename `BanyanDB.ShardingKey` to `BanyanDB.SeriesID` (#10037) 2022-11-26 13:06:53 +08:00
吴晟 Wu Sheng 6b60357324
Fix `BanyanDB.ShardingKey` annotation missed in the generated OAL metrics classes. (#10032) 2022-11-26 00:20:33 +08:00
吴晟 Wu Sheng 608a084350
Begin 9.3.0 iteration, 9.2.0 release is on vote (#9539)
* [maven-release-plugin] prepare release v9.2.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-9.2.0.md

* Update changes.md

* Update LICENSE
2022-08-31 00:31:04 +08:00
吴晟 Wu Sheng 8584ada7cc
Begin 9.2.0 iteration (#9191)
* [maven-release-plugin] prepare release v9.1.0

* [maven-release-plugin] prepare for next development iteration
2022-06-08 17:24:15 +08:00
吴晟 Wu Sheng b60855e1e2
Refactor the storage annotations. (#9075) 2022-05-17 12:44:26 +08:00
吴晟 Wu Sheng 6d9decddce
Support BanyanDB global index for entities (#8852)
* Move ElasticSearch and BanyanDB specific configurations out of column.
* Support BanyanDB global index for entities. Log and Segment record entities declare this new feature.
* Remove unnecessary analyzer settings in columns of templates. Many were added due to analyzer's default value.
* Move SQLDatabase(H2/MySQL/PostgreSQL) specific configurations out of column.
2022-04-12 08:21:44 +08:00
吴晟 Wu Sheng 91413a753c
Start 9.1.0-dev iteration (#8822) 2022-04-06 18:56:49 +08:00
吴晟 Wu Sheng faf6e2eb9e
Add `Error Prone Annotations` dependency to suppress warnings (#8774) 2022-04-01 00:57:34 +08:00
吴晟 Wu Sheng f956484e21
Add `Column.shardingKeyIdx` for column definition for BanyanDB (#8705)
* Add `Column.shardingKeyIdx` for column definition for BanyanDB.

```
Sharding key is used to group time series data per metric of one entity in one place (same sharding or same 
column for column-oriented database).
For example,
ServiceA's traffic gauge, service call per minute, includes following timestamp values, then it should be sharded by service ID
[ServiceA(encoded ID): 01-28 18:30 values-1, 01-28 18:31 values-2, 01-28 18:32 values-3, 01-28 18:32 values-4]

BanyanDB is the 1st storage implementation supporting this. It would make continuous time series metrics stored closely and compressed better.

NOTICE, this sharding concept is NOT just for splitting data into different database instances or physical files.
```

* Add `JD_PRESERVE_LINE_FEEDS=true` in official code style file. Then our comments don't have to use many blank lines to keep format.
2022-03-18 23:39:52 +08:00
吴晟 Wu Sheng 4def159162
Refactor the core Builder mechanism (#8684) 2022-03-16 08:22:18 +08:00
Jared Tan c1765b5177
[OAL] Keep the endpoint avg resp time meter name the same with others scope. (#8534)
* Keep the endpoint avg resp time meter name the same with others scope.

* fix e2e.

* update changes.

* Update CHANGES.md

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

* sync submodule.

* revert break changes

* fix UT

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2022-02-11 20:48:02 +08:00
吴晟 Wu Sheng 1fdf3a8acf
Set version to 9.0.0-snapshot (#8242) 2021-12-06 10:04:20 +08:00
吴晟 Wu Sheng e7d6f89021
Set the version to 8.10.0, 8.9.0 release is in process. (#8233) 2021-12-02 18:54:21 +08:00
Kirs f7bcb76784
move the codes in apm-commons into server-library (#8003) 2021-10-26 20:31:36 +08:00
吴晟 Wu Sheng e575701f85
Support JDK 16 and 17. (#8005) 2021-10-26 16:34:24 +08:00
吴晟 Wu Sheng fc0d0e1e7b
Message Queue performance and consuming latency monitoring (#7918)
* Support `!= null` in OAL engine.
* Add `Message Queue Consuming Count` metric for MQ consuming service and endpoint.
* Add `Message Queue Avg Consuming Latency` metric for MQ consuming service and endpoint.
* Document with a new menu in the `tracing` catalog is added
2021-10-13 20:20:36 +08:00
吴晟 Wu Sheng 720dc924e4
Refactor the OAL compiler context to improve readability (#7913)
* Refactor the OAL compiler context to improve readability. Usually, this could not be included in the change log, but considering this kind of refactor could cause not-found bugs, Adding this as a hit if we face some related issues in the future.
* Fix wrong generated codes of hashCode and remoteHashCode methods for numeric fields.
2021-10-12 19:04:28 +08:00
吴晟 Wu Sheng 7f7f604bed
Support `(str->long)` and `(str->int)` cast statements in OAL core engine. (#7907)
Enhance OAL engine core
* Fix `funcParamExpression` and `literalExpression` can't be used in the same aggregation function.
* Support cast statement in the OAL core engine.
* Support `(str->long)` and `(long)` for string to long cast statement.
* Support `(str->int)` and `(int)` for string to int cast statement.
* Support Long literal number in the OAL core engine.
* Support literal `string` as parameter of aggregation function.
* Add `attributeExpression` and `attributeExpressionSegment` in the OAL grammar tree to support `map` type for the
  attribute expression.

Besides added UTs, I locally tested these OAL scripts
```
mq_consume_latency = from((str->long)Service.tag["transmission.latency"]).longAvg();

mq_consume_latency = from((long)Service.tag["transmission.latency"]).longAvg();

mq_consume_latency = from((str->long)Service.tag["transmission.latency"]).filter((str->long)tag["transmission.latency"] > 0L).longAvg();
```
2021-10-12 14:33:23 +08:00
吴晟 Wu Sheng 9d8a774b6a
Enhance `get` generation mechanism of OAL engine, support map type of source's field. (#7904) 2021-10-11 21:04:06 +08:00
吴晟 Wu Sheng 8b681053df
Begin 8.9.0 iteration (#7801)
* [maven-release-plugin] prepare release v8.8.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-8.8.0.md

* Reset changelog for 8.9.0
2021-09-25 23:46:18 +08:00
YczYanchengzhe a3317a8238
Support for filter filtering of int type values (#7636) 2021-09-02 23:53:05 +08:00
吴晟 Wu Sheng 1c5e22a7be
Begin 8.8.0 iteration (#7395)
* [maven-release-plugin] prepare release v8.7.0

* [maven-release-plugin] prepare for next development iteration
2021-07-30 23:01:09 +08:00
kezhenxu94 f5b7c3e32d
Perf: cache metrics id and entity id (#7201)
There are a lot of calls to `Metrics.id()` and `ISource.getEntityId`, which calculates the id by manipulating strings in every single call, producing many garbage objects.

In this patch, I cache the id and only calculate the id if it's requested for the first time.
2021-06-29 22:48:51 +08:00
kezhenxu94 84a7fe30c8
Perf: cache regex pattern and result, optimize string concatenation (#7199)
The possible metrics names are relatively fixed in an OAP run, we previously always escape the metrics names, building a new regex Pattern from scratch, which is cpu-consuming.

In this patch, I cache the escaped metrics names, and if it missed, use a pre-compiled regex Pattern to escape the metrics name.

This patch also replace string concatenation `+` with `StringBuilder` in generated classes, which won't get optimized by Java compiler

This may reduce 1~1.5 vCPU under 20K RPS environment.
2021-06-29 17:46:42 +08:00
kezhenxu94 528ee6de40
OAL supports generating metrics from events (#7134)
* OAL supports generating metrics from events

* Add new dashboard for event metrics

* Add some docs for event metrics
2021-06-19 20:20:37 +08:00
吴晟 Wu Sheng 5eb1021639
Initialize 8.7.0 iteration (#7078) 2021-06-08 07:41:09 +08:00
吴晟 Wu Sheng 17e820f156
Begin 8.6.0 iteration (#6721) 2021-04-09 23:01:00 +08:00