Commit Graph

163 Commits

Author SHA1 Message Date
Wan Kai d7ee9a01d6
Support sharding MySQL database instances and tables. (#9689)
* [**Breaking Change**] Support sharding MySQL database instances and tables by [Shardingsphere-Proxy](https://shardingsphere.apache.org/document/current/en/overview/#shardingsphere-proxy).
  SQL-Database requires removing tables `log_tag/segment_tag/zipkin_query` before OAP starts, if bump up from previous releases.
2022-09-30 08:39:14 +08:00
吴晟 Wu Sheng f41b41ed21
Polish academy docs (#9707) 2022-09-29 10:12:17 +08:00
Wan Kai 4a7e8191dd
Unify query services and DAOs codes time range condition to `Duration`. (#9702) 2022-09-29 08:08:15 +08:00
kezhenxu94 fba069f4fb
Fix the wrong watch semantics in Kubernetes watchers (#9551) 2022-09-28 08:01:38 +08:00
吴晟 Wu Sheng 4f20059f7c
Fix words (#9694) 2022-09-27 16:32:57 +08:00
吴晟 Wu Sheng 2c265db6e8
Fix two kernel level bugs. (#9690)
* Fix meter functions `avgHistogram`, `avgHistogramPercentile`, `avgLabeled`, `sumHistogram` having data conflict when downsampling. FYI @jinho-son thanks for reporting.
* Do sorting `readLabeledMetricsValues` result forcedly in case the storage(database) doesn't return data consistent with the parameter list.
2022-09-27 16:05:17 +08:00
yswdqz f9db9ba2dc
Support monitoring PostgreSQL slow SQLs. (#9673) 2022-09-23 20:31:17 +08:00
kezhenxu94 9a674c95cb
Bump up the embedded `swctl` version in OAP Docker image (#9649) 2022-09-21 15:29:00 +08:00
Fine0830 a39f0308b6
Sync ui and add a document for adding menus to booster UI (#9653)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2022-09-21 12:37:14 +08:00
kezhenxu94 3e3a39cb66
Bump up snakeyaml to 1.32 (#9647) 2022-09-20 21:56:24 +08:00
吴晟 Wu Sheng 371bdab5ec
Remove `responseCode` fields of all OAL sources, as well as examples to avoid user's confusion. (#9638) 2022-09-20 10:27:26 +08:00
吴晟 Wu Sheng ebb42441ec
Add `metadata-uid` setup doc about Kubernetes coordinator (#9636) 2022-09-20 09:56:18 +08:00
pg.yang bbe0a7c4c2
Support analyzing cache related spans to provide metrics and slow commands for cache services from client side. (#9622)
* Support analyzing cache related spans to provide metrics and slow commands for cache services from client side
* Optimize virtual database, fix dynamic config watcher NPE when default value is null 
* [UI] Add virtual cache dashboard
2022-09-19 22:10:32 +08:00
吴晟 Wu Sheng c3e0b12648
[Critical] Remove physical index existing check and fix TTL activation bug (#9632)
* Remove physical index existing check

* Make sure instance list ordered in TTL processor to avoid TTL timer never runs.
2022-09-19 19:37:03 +08:00
yswdqz 0a85bc43b7
Support monitoring MySQL slow SQLs (#9614) 2022-09-16 23:44:57 +08:00
yswdqz d3727b7a9f
Enhance LAL to support convert LogData to DatabaseSlowStatement. (#9575) 2022-09-16 12:42:43 +08:00
Fine0830 f38ff09e6c
sync ui (#9611)
* Polish the endpoint list graph.
* Fix styles for an adaptive height.
* Fix setting up a new time range after clicking the refresh button.
* Enhance the process topology graph to support dragging nodes.
2022-09-16 07:40:26 +08:00
Wan Kai dd3a45e4e7
UI-template: Fix metrics calculation in `general-service/mesh-service/faas-function` top-list dashboard. (#9610)
* Fix metrics calculation in `general-service/mesh-service/faas-function` top-list dashboard

* changes.md
2022-09-16 07:33:14 +08:00
kezhenxu94 04c4fedf43
Adapt ElasticSearch 8.1+, migrate from removed APIs to recommended APIs (#9603) 2022-09-14 12:22:40 +08:00
yswdqz 12a18e6977
Add layer property to LAL script (#9593) 2022-09-12 22:16:07 +08:00
kezhenxu94 b3c7658a0e
webapp: migrate from Spring Web to Armeria (#9592) 2022-09-11 16:33:53 +08:00
Wan Kai 1cfca3ccc6
Fix `DurationUtils.convertToTimeBucket` missed verify date format (#9586) 2022-09-09 22:10:57 +08:00
吴晟 Wu Sheng 18d2463813
Bump up snakeyaml to 1.31 for fixing CVE-2022-25857 (#9567) 2022-09-06 11:20:01 +08:00
吴晟 Wu Sheng 5ad3782231
Use prepareStatement in H2SQLExecutor#getByIDs (#9561) 2022-09-05 09:25:27 +08:00
Lv Lifeng 8c683a0b67
Add component IDs for impala JDBC Java plugin (#9557) 2022-09-04 11:20:48 +08:00
吴晟 Wu Sheng 608a084350
Begin 9.3.0 iteration, 9.2.0 release is on vote (#9539)
* [maven-release-plugin] prepare release v9.2.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-9.2.0.md

* Update changes.md

* Update LICENSE
2022-08-31 00:31:04 +08:00
mrproliu 922f1b958d
Add Network Profiling for Service Mesh DP instance and K8s pod panels. (#9536) 2022-08-30 14:38:55 +08:00
mrproliu fbba5222d3
Add eBPF CPU Profiling documentation (#9535) 2022-08-29 11:46:35 +08:00
pg.yang 5e5ae7d6e0
Add component ID(132) for NAST java client plugin (#9533) 2022-08-27 13:19:41 +08:00
Fine0830 f8b89b78cd
sync ui (#9520) 2022-08-25 13:41:51 +08:00
Kirs ebe0078c34
Fix dependency license's NOTICE and binary jar included issues in the source release. (#9513) 2022-08-24 21:27:13 +08:00
Wan Kai 7df469a6e8
Fix JDBC TTL to delete additional tables data. (#9508) 2022-08-24 18:41:21 +08:00
pg.yang 1d930d0bb8
Update alarm doc introduce keys with data types of the configuration file. (#9514) 2022-08-24 07:31:30 +08:00
Fine0830 2942af80ef
sync ui (#9505) 2022-08-22 22:15:33 +08:00
kezhenxu94 c8a78149a6
Enable otlp OC handler by default and polish doc (#9499) 2022-08-22 16:09:59 +08:00
yswdqz 828669ce2a
Support PostgreSQL monitoring. (#9495) 2022-08-22 14:29:26 +08:00
kezhenxu94 f96b3eacf1
Adapt OpenTelemetry native metrics protocol (#9497) 2022-08-21 21:13:23 +08:00
kezhenxu94 17968b6690
Bump up kubernetes java client to fix cve (#9498) 2022-08-21 19:32:34 +08:00
kezhenxu94 e3d091ebd5
Add Kubernetes support module to share codes between modules and reduce calls to Kubernetes API server (#9493) 2022-08-20 15:24:44 +08:00
mrproliu 1b38dd9526
Add Spring Sleuth dashboard to general service instance template (#9478) 2022-08-18 08:15:42 +08:00
Wan Kai aa8ddd2587
Optimize TTL mechanism for Elasticsearch storage, skip executed indices in one TTL rotation. (#9473)
* Optimize TTL mechanism for Elasticsearch storage, skip executed indices in one TTL rotation

* remove inspect
2022-08-16 12:41:43 +08:00
mrproliu 9e07ad7e9f
Fix query history process data failure (#9471) 2022-08-15 18:17:52 +08:00
Brandon Fergerson d1f898419b
feat: support sending alarm to discord (#9469) 2022-08-14 14:58:44 +08:00
Wan Kai 6ab00031a0
Fix query services by serviceId error when Elasticsearch storage `SW_STORAGE_ES_QUERY_MAX_SIZE` > 10000 (#9468) 2022-08-12 20:05:54 +08:00
yswdqz aa6c5e9182
Support MySQL monitoring. (#9432) 2022-08-12 16:51:36 +08:00
pg.yang 29c4344bd3
Add component id for Micronaut HTTP plugin (#9455) 2022-08-10 22:49:12 +08:00
吴晟 Wu Sheng b15bbecb73
Add an explanation doc for alerting kernel mechanism (#9446) 2022-08-09 22:39:29 +08:00
Wan Kai 8a8070025a
Support Elasticsearch column alias for the compatibility between storage logicSharding model and no-logicSharding model. (#9442)
## New ElasticSearch storage option explanation in 9.2.0
Since v9.2.0, SkyWalking OAP provides 2 storage options for metrics/meter and records, 
system environment variable is (`SW_STORAGE_ES_LOGIC_SHARDING`):

### No-Sharding Model (OAP default setting, `SW_STORAGE_ES_LOGIC_SHARDING = false`)
1. OAP merges all metrics/meter and records(without super datasets, such as segments) indices into one physical 
index template `metrics-all` and `records-all`.
2. The logic index name would be present in columns `metric_table` and `record_table`.
3. If the logic column name has an alias (configured by `@ElasticSearch.Column()`), the alias would be the real physical column name.

### No-Sharding Model (`SW_STORAGE_ES_LOGIC_SHARDING = true `)
1. OAP shard metrics/meter indices into multi-physical indices as in the previous versions(one index template per metric/meter aggregation function).
2. Records and metrics without configuring aggregation function in `@MetricsFunction` and `@MeterFunction` would not be sharded.
3. The shard template name would be `metrics-aggregation function name` or `meter-aggregation function name` such as `metrics-count`,
and the logic index name would be present in column `metric_table`.
4. The OAP **would not** use the column alias, the logic column name would be the real physical column name.

**Notice**: 
Users still could choose to adjust ElasticSearch's shard number(`SW_STORAGE_ES_INDEX_SHARDS_NUMBER`) to scale out no matter the option.
2022-08-09 22:24:45 +08:00
吴晟 Wu Sheng 7f371ed565
[Critical] Fix a low performance issue of metrics persistent in ES (#9437)
[Critical] Fix a low-performance issue of metrics persistent in the ElasticSearch storage implementation. One single metric could have to wait for an unnecessary 7~10s(System Env Variable SW_STORAGE_ES_FLUSH_INTERVAL).

This is a solid and critical performance issue, from the self-observability metric, we could find one metric that would need 7-10s to execute the data flush, which is not expected.
2022-08-08 19:38:58 +08:00
Fine0830 fedc20991d
Sync ui: add a calculation to convert seconds to days (#9434) 2022-08-08 11:43:06 +08:00