Commit Graph

79 Commits

Author SHA1 Message Date
songzhendong b363b0ea4f
Add Nacos back into SkyWalking as an option(nacos-1.3.1) (#5137) 2020-07-21 23:30:06 +08:00
Evan ad47c9045b
update the kubernetes client verion to 8.0.0 (#4844) 2020-06-22 07:56:39 +08:00
Gao Hongtao 4f0f39ffcc
Ingest prometheus metrics through meter system (#4783) 2020-06-08 09:19:34 +08:00
songzhendong f3d907bf6d
Remove nacos (#4867)
* Add annotation(AlarmCore.start)

* Upgrade nacos version to 1.2.0

* Revert "Upgrade nacos version to 1.2.0"

This reverts commit 4d7b06f6d20ee09ad377ad6ef736aeab429314e0.

* Update nacos version to 1.2.0(Solve the security problem of the old version of fastsjon)

* Recovery annotation

* Recovery annotation

* Update nacos version to 1.2.0

* Update tls_key_generate.sh path

Update tls_key_generate.sh path

* Add annotation(AlarmCore.start)

* Revert "Upgrade nacos version to 1.2.0"

This reverts commit 4d7b06f6d20ee09ad377ad6ef736aeab429314e0.

* Update nacos version to 1.2.0(Solve the security problem of the old version of fastsjon)

* Recovery annotation

* Recovery annotation

* Update tls_key_generate.sh path

* Execution Authority

* Remove nacos

* remove nacos code

* remove nacos

remove nacos

* remove nacos

remove nacos

* remove nacos

* remove nacos

Co-authored-by: songzhendong <289505773@qq.com>
Co-authored-by: songzhendong <songzhendong@xiaomi.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-06-06 14:23:12 +08:00
mrproliu 8af10bed11
Support regex to include or exclude the names in alarm module (#4775)
* Support regex to include or exclude the names, requested through mail list, https://lists.apache.org/x/thread.html/rb6d91093776cf61880425449d3d25f9d560cd32363c0075de9a85d13@%3Cdev.skywalking.apache.org%3E

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-05-13 14:34:16 +08:00
mrproliu 749af3e04e
Alarm support multiple scope (#4769)
* Support service relation and database access alarm

* support service instance relation alarm

* support endpoint relation alarm

* document and format the name

* resolve issues

* add database access and endpoint relation rule example

* resolve code format

* remove unnecessary getter and setter package declare

* remove unnecessary fields

Co-authored-by: Mrproliu <mrproliu@lagou.com>
2020-05-11 00:28:10 +08:00
songzhendong 4368f82216
[CVE] Update Nacos 1.0.0 to Nacos 1.2.0(Solve the security problem of the old version of fastjson) (#4753)
* Update nacos version to 1.2.0(Solve the security problem of the old version of fastsjon)

Co-authored-by: songzhendong <289505773@qq.com>
Co-authored-by: songzhendong <songzhendong@xiaomi.com>
2020-05-07 14:51:21 +08:00
吴晟 Wu Sheng 950346c380
[CVE] Update Jetty version to fix its CVEs (#4642) 2020-04-13 14:27:19 +08:00
吴晟 Wu Sheng 44cae97c25
Set up restrict length rules for service, instance and endpoint (#4633)
* Set up restrict length rules for service, instance and endpoint

* Adjust two env names.
2020-04-10 17:59:21 +08:00
吴晟 Wu Sheng 6fe2041b47
Make 8.0.0 Core available. New protocol and register removed. (#4599)
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
2020-04-10 11:03:24 +08:00
吴晟 Wu Sheng 54fb6e7285
Set the endpoint name length to 150, to keep safe and avoid over-length. (#4595)
* Set the endpoint name length to 150, to keep safe and avoid over-length.

* Set endpointNameMaxLength in the application.yml

* Fix a mischange.

* Correct the endpoint name length control logic

Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-03-31 09:04:10 +08:00
wind2008hxy 203097446a
Set the trustStorePath default value to empty (#4579) 2020-03-27 12:53:59 +08:00
吴晟 Wu Sheng f1b2b298b1
Upgrade jackson-databind version to 2.9.10 (#4565) 2020-03-24 18:42:08 +08:00
kezhenxu94 69e3d80e24
Add selector property in application.yml (#4514)
### Motivation

 Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.

 ### Modification

 Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.

 ### Result

 - We can switch a configuration by environment variables or system property

 - Closes #4511
2020-03-15 23:28:14 +08:00
Gao Hongtao 78d14ebbd6
Enable OAP gRPC SSL transportation (#4470)
* Enable OAP gRPC SSL transportation

Porting to OpenSSL to enable SSL transportation. The server private
key is in the format of PCKS#8, certificates is x509 though.

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add netty-tcnative-boringssl-static 2.0.26 licnese

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Add SSL to grpc remote client and sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update documents

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix nits

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix test case

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add e2e-ssl and remove other test cases temporarily

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add TLS relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Decrease timeout

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Test config works

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* triger CI on oap-ssl branch

* Disable agent ca

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Switch to sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add agent log to console

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Upload log files

* Change backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert "Change backend"

This reverts commit 7a085711e3653d3bf642aa84f73650e22595e383.

* Update backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert e2e

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Ignore TSL relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Polish codes

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update certificates expires year to 2120

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-03-15 08:15:28 +08:00
吴晟 Wu Sheng e69391a207
Support Secrets Management File in the ElasticSearch 6/7 storage (#4493)
* Temp commit

* Support secretsManagementFile file.

* Update doc.

* 1. Support JKS/pass runtime change too.
2. Follow review.

* Fix format.

* Fix username/password/trustPass haven't been updated in the es client.

* Fix doc issue.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
2020-03-13 20:50:14 +08:00
吴晟 Wu Sheng d32450aa7d
Support name column(s) in the storage for 3rd party integration, like Kibana. Default OFF (#4439)
* Support dynamic column in the source.

* Support activeExtraModelColumns as a default OFF option.

* Fix words

* Update backend-setup.md

Fix format issue.
2020-03-04 14:44:54 +08:00
Daming 29da5738bc
Provide influxdb as a new storage plugin (#4239)
* provide influxdb as a new storage plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-19 18:00:57 +08:00
吴晟 Wu Sheng 334239388a
Support Daily Index Step feature in the OAP (#4368)
* Support Daily Index Step.

* Fix format.

* Fix mischanged ES.
2020-02-16 20:48:04 +08:00
吴晟 Wu Sheng 329f7e150b
Support Downsampling Data Packing feature in ES storage implementation (#4364)
* Support day/hour/minute metrics merging into one index. Reduce the number of index 50%.
2020-02-16 09:03:30 +08:00
kezhenxu94 5b255ba3e3
Set up more strict code styles and fix existing issues (#4337)
Motivation:

Review code styles with some bots automatically.

Modifications:

Set up ReviewDog in GitHub Action to review code style.

Add more check rules to checkstyle plugin.

Result:

Obvious code styles can be reviewed and commented automatically.
2020-02-11 17:10:49 +08:00
吴晟 Wu Sheng f9cf17be24
UI Sync & remove elementui license (#4227)
* Sync UI.

* Remove elementui LICENSE
2020-01-13 21:19:23 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
web-xiaxia 5fba91901f add Dynamic Configuration Consul AclToken (#4161)
* Update ConsulConfigurationCenterSettings.java

add aclTokne

* Update ConsulConfigurationWatcherRegister.java

add aclToken

* Update application.yml

add aclToken

* Update ConsulConfigurationCenterSettings.java

add aclToken

* Update application.yml

add aclToken

* add aclToken

* Replace with util

* Replace with util

* add consul aclToken
2020-01-02 20:05:16 +08:00
吴晟 Wu Sheng 568c2e53f0
Update NOTICE year (#4158) 2020-01-01 22:40:12 +08:00
mrproliu d38a8eaaa7 Provide thread monitor create task feature (#4145)
* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-31 20:49:45 +08:00
吴晟 Wu Sheng b570996006
Update JDK compiling level to 8, and remove zip dist. (#4119)
* Update JDK compiling level to 8, and remove zip dist.

* Exclude annotation and shade API.

* Fix more exclusion.

* 1. Update grava and gson 2. Fix compile issue. 3. Keep protoc and maven plugin in old version. Some incompatible of 3rd party grpc proto.

* Fix dependency conflicts, license and notice issue, and dependency check script issue

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-25 15:55:24 +08:00
吴晟 Wu Sheng 7c535f8612
Fix es7 dist not uploaded in the release process. (#4112) 2019-12-24 12:07:19 +08:00