Commit Graph

496 Commits

Author SHA1 Message Date
kezhenxu94 094eeb870f
Add alarm metrics OP >= and <=, and refactor the operator logics (#4221)
* Add alarm metrics OP >= and <=, and refactor the operator logics

* Add unit test for OP

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-13 00:50:37 +08:00
Jared Tan 1310719645
add token authentication between agent and oap receiver. (#4197)
* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 23:08:39 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
Jared Tan 7e9812e857
remove duplicate maven dependency. (#4215) 2020-01-11 11:01:18 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
aderm f181f7b483 delete unused code. (#4178) 2020-01-06 13:15:48 +08:00
web-xiaxia 5fba91901f add Dynamic Configuration Consul AclToken (#4161)
* Update ConsulConfigurationCenterSettings.java

add aclTokne

* Update ConsulConfigurationWatcherRegister.java

add aclToken

* Update application.yml

add aclToken

* Update ConsulConfigurationCenterSettings.java

add aclToken

* Update application.yml

add aclToken

* add aclToken

* Replace with util

* Replace with util

* add consul aclToken
2020-01-02 20:05:16 +08:00
吴晟 Wu Sheng 6c20f4a41b
Add service type from register and support browser as service type (#4157)
* Add service type from register and support browser as service type

* Add service type at agent side.

* Fix format.

* Fix a test.

* Fix formats.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-01 18:01:41 +08:00
mrproliu d38a8eaaa7 Provide thread monitor create task feature (#4145)
* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-31 20:49:45 +08:00
吴晟 Wu Sheng 1ee14fbbc8
Support customized instance name by a new UUID naming policy. (#4148) 2019-12-30 14:59:29 +08:00
Gao Hongtao 4211eff558 Fix os maven plugin issues for Eclipse m2e (#4144)
* Fix mave-os plugin issues for Eclipse m2e

* Add os-msven-plugin.version property
2019-12-29 13:49:27 +08:00
mrproliu 2fcf7c4649 provide config stream processor (#4133)
* provide config stream processor

* 1. rename config to none stream
2. add comments on none stream relate classes

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2019-12-27 14:58:15 +08:00
Weiyi Liu 9415f68127 Fix the wrong comments. (#4139) 2019-12-27 12:10:37 +08:00
aderm e25b1e6008 improve if find parent, quick exit (#4107) 2019-12-26 10:02:42 +08:00
aderm 8476316e96 refactor code. (#4120) 2019-12-25 23:35:35 +08:00
吴晟 Wu Sheng b570996006
Update JDK compiling level to 8, and remove zip dist. (#4119)
* Update JDK compiling level to 8, and remove zip dist.

* Exclude annotation and shade API.

* Fix more exclusion.

* 1. Update grava and gson 2. Fix compile issue. 3. Keep protoc and maven plugin in old version. Some incompatible of 3rd party grpc proto.

* Fix dependency conflicts, license and notice issue, and dependency check script issue

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-25 15:55:24 +08:00
吴晟 Wu Sheng d7f7799c80
Set the main branch to 7.0.0-SNAPSHOT (#4115)
* [maven-release-plugin] prepare release v6.6.0

* [maven-release-plugin] prepare for next development iteration
2019-12-24 16:37:13 +08:00
Gao Hongtao 4ed21e1c9d
Filter endpoint with an id == 0 (#4105) 2019-12-23 11:57:58 +08:00
zhang-wei 4c99480f70 Add relation metrics verification in e2e test (#4080) 2019-12-19 13:26:27 +08:00
Daming f9a849d6be fix typo (#4082) 2019-12-18 21:03:43 +08:00
Juan Pan(Trista) 0fd634092b Modify query SQL for MySQLDAO (#4047)
* Modify MySQLDAO and H2DAO
2019-12-17 23:36:22 +08:00
吴晟 Wu Sheng 5f420bf3ac
Fix service instance relationship ID build bug. (#4075)
* Fix service instance ID build bug.

* Add instance ids into the relation metrics.
2019-12-17 14:12:56 +08:00
kezhenxu94 3f48790c36 [Feature] Add Java agent plugin for Jdk Threading classes (#4067)
* Add Java agent plugin for Jdk Threading

* Fix expected data according to changes in master

* Update agent.config

* Fix failed plugin test
2019-12-17 00:12:14 +08:00
Jared Tan a6d0366c22 support min function. (#4072)
* support min function.

* fix logical bug and update test case.

* the same here.
2019-12-16 22:20:30 +08:00
吴晟 Wu Sheng 4e50132817
Remove the local/exit span register mechanism in Java agent scenario (#4059)
* Remove the local span and exit span register mechanism in Java agent scenario.

* Fix config, plugin, and document, as exit span is not required register anymore.

* Try to fix the test case expected data due to no register for exit span.

* Fix toolkit case.

* Remove exit span from plugin test framework document.

* Update protocol, FAQ, and CHANGELOG documents to highlight this change.

* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.

* Fix the snapshot and add inexistence operation name id.

* Fix several async cases.

* Fix undertow test case.

* Fix wrong error log name.

* Keep name consistent with 6.x concepts.

* Try to fix 3 async scenarios.

* Fix okhttp case.

* Fix Spring Async case.

* Fix another NPE

* Fix NPE when first span has no endpoint id when it is local or exit span.

* Still fill first operation name when no endpoint id.
2019-12-15 19:15:47 +08:00
Gao Hongtao 16d7782a5c
Remove client side endpoint register of service mesh (#4054) 2019-12-13 16:11:33 +08:00
kezhenxu94 eea5d027cc
Add Armeria Java agent plugin (#4024) 2019-12-11 15:57:45 +08:00
mrproliu cfb3379cd0 fix judge NPE when worker not found (#4039) 2019-12-10 23:22:56 +08:00
zhang-wei 3741f4e386 [Feature] Service instance dependency (#3978)
* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* test e2e

* add official_analysis.oal & update getServiceInstanceTopology

* test instance topo for single service

* fix gql

* fix verifyServiceInstances

* verifyServiceInstanceTopo

* fix verifyServiceInstanceTopo

* fix ServiceInstanceInventory#name

* fix ServiceInstanceInventory#name

* e2e test

* fix action

* fix provider port

* fix Instance Node Type

* re run

* fix expected-data

* add e2e mysql

* set @Test timeout

* fix ci

* increase timeout

* re check

* test ttl es7

* test ttl

* fix action

* merge group

* test ttl es7

* replace official_analysis.oal

* replace official_analysis.oal

* comments

* add TODO

* add TODO

* fix filed name
2019-12-09 08:23:21 +08:00
Gao Hongtao 34fd8637c0
Update envoy proto (#4013) 2019-12-06 14:41:54 +08:00
kezhenxu94 aa022ffa00 Remove unused ES7 config (#3998) 2019-12-04 00:20:10 +08:00
Gao Hongtao 87b7afcd64
Distribute "service-apdex-threshold.yml" (#3976)
* Update pom.xml

Exclude "service-apdex-threshold.yml" from jar

* Add service-apdex-threshold.yml to the config folder of release jar
2019-11-29 22:49:32 +08:00
Gao Hongtao 6475ab3d1d Add apdex function to OAL (#3855)
* Add apdex function to OAL

* Add empty line

* Setup config watcher

* Add identifier type to function parameter

* Add config test

* Update score algorithm

* Replace responseCode with status

* Add comments about apdex score algorithm

* Add docs

* Add e2e test case

* Update test case

* Fix disptch class generating error

* Update value name of apdex metric

* Tuning threshold

* Fix single tolerated point bug
2019-11-29 12:54:33 +08:00
Stalary b666dc7e11 MOD:alarm-settings.yml support exclude-names (#3949) 2019-11-28 22:25:08 +08:00
Jared Tan d581f2294f Support custom elasticsearch settings in index level. (#3913) 2019-11-21 22:30:59 +08:00
kezhenxu94 9b408e82d5 [Feature] Support ElasticSearch 7 as backend storage (#3870)
* [Feature] Officially support ElasticSearch 7 as backend storage

* Extract bootstrap module to be shared by multiple starters

* Add missing configuration file and distinguish different version in test

* Update known-oap-backend-dependencies-es7.txt

Update the dependency lib

* Fix missing adaptation to ElasticSearch 7

* Rename ElasticSearch 7 specific class with Es7 infix

* Fix miss-adapted ElasticSearch DAO API

* Add ES7 specific configuration and polish documentations

* Polish documentations

* Polish documentations
2019-11-21 09:26:23 +08:00
吴晟 Wu Sheng 0724cdb42d
Support ordered segment list in trace query (#3906)
* Sort the segment fragments

* Add comment.

* Remove unnecessary box codes.
2019-11-20 22:39:51 +08:00
Jared Tan 5d064ced5f make topN worker report period configurable. (#3892) 2019-11-19 17:26:30 +08:00
于玉桔 6ffd92792d Support spring-tx in optional plugin (#3857)
* plugin code

* NPE FIX
2019-11-19 10:05:01 +08:00
surechen 96a3e34243 Add componentId for apache servicecomb. (#3676)
* Add componentId for apache servicecomb.

* Edit according to comment

* add servicecomb

* Edit according to comment

* fix
2019-11-19 06:58:15 +08:00
Jared Tan 70f94d577c
make sql statement more semantic. (#3882)
* set no statement.

* update sql.

* update statement.

* revert threshold.
2019-11-18 21:30:25 +08:00
吴晟 Wu Sheng 2c4caedabb Update joda lib to fix UTC +13/+14 bug (#3876)
* Update joda lib to fix UTC 13/14 bug

* Change the dependencies file.
2019-11-17 23:18:28 +08:00
吴晟 Wu Sheng d7038fe67b
Refactor DataCarrier, support ArrayBlockingQueueBuffer as implementor (#3849)
* Refactor DataCarrier, support ArrayBlockingQueueBuffer as the implementation for blocking queue buffer.

* Fix style issue.

* Remove import.

* Remove uesless codes.
2019-11-15 23:33:40 +08:00
于玉桔 db0b7486a5 fix component-libraries.yml miss component:rest-high-level-client (#3859)
* fix component-libraries.yml miss component:rest-high-level-client

* Update component-libraries.yml

Add mapping.
2019-11-14 17:59:59 +08:00
aderm e11d20715d Add elasticsearch 6.x plugin & elasticsearch 6.x test scenario (#3803)
* Add elasticsearch 6.x plugin & elasticsearch 6.x test scenario
2019-11-13 13:55:28 +08:00
吴晟 Wu Sheng 3fbaec42dc
Move to 6.6.0 SNAPSHOT (#3839)
* [maven-release-plugin] prepare release v6.5.0

* [maven-release-plugin] prepare for next development iteration
2019-11-12 22:33:39 +08:00
zhang-wei c35c6f2a2e fix consul repeated invoker KVCache.newCache (#3833) 2019-11-11 22:26:21 +08:00
YuXiaoFeng 455f69ad9e Enhance EtcdCoordinator if disconnected from etcd server for a long time (#3782)
* Update EtcdCoordinator.java

when EtcdCoordinator cannot connect to etcd cluster for a long time, 
the information registered in etcd will expire and refresh() function will not work.
2019-11-05 13:51:29 +08:00
Jared Tan fbabcb1782 make query max window size configurable. (#3765)
* fix elasticsearch query data window size too large error.

* make query max window size configurable.
2019-11-01 16:02:06 +05:30
吴晟 Wu Sheng e9b404f1fa
Remove two max size 500 limit. (#3748) 2019-10-30 18:34:33 +05:30