Commit Graph

237 Commits

Author SHA1 Message Date
mrproliu 3252b6ed08
provide profile analyze empty interfaces (#4303)
* provide profile analyze empty interfaces

* fix comment error

* change comments

* update submodule

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-31 21:11:43 +08:00
吴晟 Wu Sheng 17e3fff83a
Refactor the storage to support `getSubsetOfMultipleLinearIntValues` (#4289) 2020-01-29 17:03:39 +08:00
kezhenxu94 75aefd34de Fix potential concurrent problems and tune performance (#4274) 2020-01-20 14:59:41 +08:00
mrproliu 90f3c4de55 Sniffer processing profile task and report status and snapshot (#4220)
* sniffer processing profile task and report status and snapshot

* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`

* resolve names

* change profile to single one thread run.

* 1. change to the ArrayList, because known the max size
2. rename issue resolved

* add profiling status enum

* change sniffer use full name issue

* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task

* remove unnecessary getter

* add test assert error message

* adding `AgentServiceRule`

* revert original assert

* remove unnecessary getter

* resolve issues

* reduce findService invoke

* resolve style error

* recheck profiling when change first span operatin name

* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting

* remove article link

* add `ProfileTask#maxSamplingCount` check

* resolve conflict (Downstream -> Commands)

* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump

* remove unused return val

* remove some `@param` and `@return`

* add profile task check result data bean

* change profiler slot to `AtomicReferenceArray`

* resolved java doc error

* fix doc error, remove meaningless descriptions

* resolve missed profile receiver on oap starter

* resolve method invoke error

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-18 18:38:03 +08:00
kezhenxu94 019af3e345
Drop support of protocol v1 (sw3) (#4244)
* Drop support of protocol v1 (sw3)

* Remove versioning logic further

* Remove more version5-only classes

* Commit missing files

* Update submodule and address review comment

* Fix failed tests
2020-01-17 20:33:52 +08:00
kezhenxu94 094eeb870f
Add alarm metrics OP >= and <=, and refactor the operator logics (#4221)
* Add alarm metrics OP >= and <=, and refactor the operator logics

* Add unit test for OP

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-13 00:50:37 +08:00
Jared Tan 1310719645
add token authentication between agent and oap receiver. (#4197)
* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 23:08:39 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
吴晟 Wu Sheng 6c20f4a41b
Add service type from register and support browser as service type (#4157)
* Add service type from register and support browser as service type

* Add service type at agent side.

* Fix format.

* Fix a test.

* Fix formats.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-01 18:01:41 +08:00
mrproliu d38a8eaaa7 Provide thread monitor create task feature (#4145)
* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-31 20:49:45 +08:00
mrproliu 2fcf7c4649 provide config stream processor (#4133)
* provide config stream processor

* 1. rename config to none stream
2. add comments on none stream relate classes

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2019-12-27 14:58:15 +08:00
aderm e25b1e6008 improve if find parent, quick exit (#4107) 2019-12-26 10:02:42 +08:00
aderm 8476316e96 refactor code. (#4120) 2019-12-25 23:35:35 +08:00
吴晟 Wu Sheng b570996006
Update JDK compiling level to 8, and remove zip dist. (#4119)
* Update JDK compiling level to 8, and remove zip dist.

* Exclude annotation and shade API.

* Fix more exclusion.

* 1. Update grava and gson 2. Fix compile issue. 3. Keep protoc and maven plugin in old version. Some incompatible of 3rd party grpc proto.

* Fix dependency conflicts, license and notice issue, and dependency check script issue

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-25 15:55:24 +08:00
zhang-wei 4c99480f70 Add relation metrics verification in e2e test (#4080) 2019-12-19 13:26:27 +08:00
Daming f9a849d6be fix typo (#4082) 2019-12-18 21:03:43 +08:00
吴晟 Wu Sheng 5f420bf3ac
Fix service instance relationship ID build bug. (#4075)
* Fix service instance ID build bug.

* Add instance ids into the relation metrics.
2019-12-17 14:12:56 +08:00
Jared Tan a6d0366c22 support min function. (#4072)
* support min function.

* fix logical bug and update test case.

* the same here.
2019-12-16 22:20:30 +08:00
吴晟 Wu Sheng 4e50132817
Remove the local/exit span register mechanism in Java agent scenario (#4059)
* Remove the local span and exit span register mechanism in Java agent scenario.

* Fix config, plugin, and document, as exit span is not required register anymore.

* Try to fix the test case expected data due to no register for exit span.

* Fix toolkit case.

* Remove exit span from plugin test framework document.

* Update protocol, FAQ, and CHANGELOG documents to highlight this change.

* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.

* Fix the snapshot and add inexistence operation name id.

* Fix several async cases.

* Fix undertow test case.

* Fix wrong error log name.

* Keep name consistent with 6.x concepts.

* Try to fix 3 async scenarios.

* Fix okhttp case.

* Fix Spring Async case.

* Fix another NPE

* Fix NPE when first span has no endpoint id when it is local or exit span.

* Still fill first operation name when no endpoint id.
2019-12-15 19:15:47 +08:00
mrproliu cfb3379cd0 fix judge NPE when worker not found (#4039) 2019-12-10 23:22:56 +08:00
zhang-wei 3741f4e386 [Feature] Service instance dependency (#3978)
* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* test e2e

* add official_analysis.oal & update getServiceInstanceTopology

* test instance topo for single service

* fix gql

* fix verifyServiceInstances

* verifyServiceInstanceTopo

* fix verifyServiceInstanceTopo

* fix ServiceInstanceInventory#name

* fix ServiceInstanceInventory#name

* e2e test

* fix action

* fix provider port

* fix Instance Node Type

* re run

* fix expected-data

* add e2e mysql

* set @Test timeout

* fix ci

* increase timeout

* re check

* test ttl es7

* test ttl

* fix action

* merge group

* test ttl es7

* replace official_analysis.oal

* replace official_analysis.oal

* comments

* add TODO

* add TODO

* fix filed name
2019-12-09 08:23:21 +08:00
Gao Hongtao 6475ab3d1d Add apdex function to OAL (#3855)
* Add apdex function to OAL

* Add empty line

* Setup config watcher

* Add identifier type to function parameter

* Add config test

* Update score algorithm

* Replace responseCode with status

* Add comments about apdex score algorithm

* Add docs

* Add e2e test case

* Update test case

* Fix disptch class generating error

* Update value name of apdex metric

* Tuning threshold

* Fix single tolerated point bug
2019-11-29 12:54:33 +08:00
吴晟 Wu Sheng 0724cdb42d
Support ordered segment list in trace query (#3906)
* Sort the segment fragments

* Add comment.

* Remove unnecessary box codes.
2019-11-20 22:39:51 +08:00
Jared Tan 5d064ced5f make topN worker report period configurable. (#3892) 2019-11-19 17:26:30 +08:00
吴晟 Wu Sheng d7038fe67b
Refactor DataCarrier, support ArrayBlockingQueueBuffer as implementor (#3849)
* Refactor DataCarrier, support ArrayBlockingQueueBuffer as the implementation for blocking queue buffer.

* Fix style issue.

* Remove import.

* Remove uesless codes.
2019-11-15 23:33:40 +08:00
吴晟 Wu Sheng 42d422e852
parameterize the cache size. (#3741) 2019-10-30 11:15:11 +05:30
Jared Tan 852f03d1db
remove unused time range query parameter. (#3730) 2019-10-28 22:21:54 +08:00
qiyang 69a28d4591 Add register persistent worker latency metrics #3671 (#3677)
* Add register persistent worker latency metrics  #3671

* remove never thrown exception on Time#close()
add panel to mesh-mode-grafana.json

* remove never thrown exception on Time#close()
add panel to mesh-mode-grafana.json
2019-10-24 15:22:38 +08:00
Jared Tan 2f3d009e63 Makes the scope of alarm message more semantic. (#3680)
* Makes the scope of alarm information more semantic.

* update md.

* update md.

* add ruleName in AlarmMsg.

* update md.

* update test case.

* fix coveralls.

* fix coveralls.

* fix ci.
2019-10-22 19:35:04 +03:00
Kdump c0be757f43 add graphql getServiceInstance instanceUUID field. (#3595)
* add graphql getServiceInstance instanceUUID field.

* add graphql getServiceInstance instanceUUID field.

* add graphql getServiceInstance instanceUUID field.

* sync query-protocol repo

* add instanceUUID field e2e

* add instanceUUID field e2e

* add instanceUUID field e2e

* add instanceUUID field e2e
2019-10-17 06:38:00 +03:00
Yanick.xia d68c0883ed Fix more resonable error (#3619)
* Fix #3402

* Fix unecessary blankline

* Optimize rebuild logic

* Fix checkstyle fail

* Add concurrency test

* Fix text mistakes

* Optimize refresh logic && new test for return clients never change by refresh logic

* Make unit test more clearness

* Add more unit test && fix a bug

* Rename variable && format by skywalking style
2019-10-16 19:33:54 +03:00
echooymxq 1b1e80b75a Optimize code (#3600) 2019-10-11 19:17:03 +08:00
吴晟 Wu Sheng 3cdc1a1928
Fix 5 places of using wrong Strings util (#3582)
* Exclude a dependency for test.

* Fix 5 places of using wrong Strings util.
2019-10-09 22:55:01 +08:00
Jared Tan 2db2638fc2 code cleanup: remove unnecessary and optimize the function's parameters. (#3577)
* code cleanup: remove unnecessary and optimize the function's parameters.

* fix code style.
2019-10-09 10:05:23 +08:00
kezhenxu94 114a74f35a
Fix some miscellaneous bugs (#3567) 2019-10-07 17:35:22 +08:00
吴晟 Wu Sheng 8ebf3aac04
Support timeout configuration in agent and backend. (#3491)
* Support timeout configuration in agent and backend.

* Fix CI

* no message
2019-09-20 15:02:28 +08:00
吴晟 Wu Sheng 6942ec8fcf
Language of instance could be NULL (#3485)
* Language of instance could be NULL

* Fix style check.

* Fix duplicated test cases.
2019-09-19 00:23:11 +08:00
Jared Tan 6c347a1169 agent support custom properties. (#3367)
* agent support custom properties.
2019-09-17 14:44:42 +08:00
吴晟 Wu Sheng 06c357da6c
Make sure the cluster register happens before streaming process. (#3471) 2019-09-17 08:53:02 +08:00
kezhenxu94 721f86485c Add TTL E2E test (#3437)
* Add TTL E2E test

* Add to Jenkins stage and minor bugfix

* Upgrade e2e container version

* Polish and minor fix

* Polish
2019-09-09 22:06:57 -07:00
吴晟 Wu Sheng 2a2679eefb
Fix Pxx calculate bug. (#3391) 2019-09-02 10:14:33 +08:00
kezhenxu94 6fd85c8303
Avoid unnecessary resetting when service mapping id is already 0 (#3383)
* Avoid unnecessary resetting when service mapping id is already 0

* Avoid as early as possible and minor optimization

* Remove unused import

* Polish
2019-08-30 23:54:54 +08:00
kezhenxu94 54614a0594 Support gateway without agent (#3308)
* Support Gateway without agent
2019-08-30 16:17:41 +08:00
吴晟 Wu Sheng dc0f6530cb
Fix TTL issue. (#3292)
* Fix TTL issue.

* Fix issue.

* Rename `getSecondTimeBucket` to `getRecordTimeBucket`.
2019-08-20 15:36:31 +08:00
kezhenxu94 db9759fcc2 Check license header strictly (#3274)
* Check license header strictly

* Ignore header template

* Include more files to check list
2019-08-16 14:37:44 +08:00
yantaowu 1bfda1dd65 Modify the thread pool of grpc server. (#3225)
* Modify the thread pool of grpc server.

* add license header

* make param confingable

* Add sharing and Jaeger config.

* Remove grpc thread config from yml
2019-08-09 09:14:55 +08:00
吴晟 Wu Sheng 0df76e4e38
Remove the scope-meta file (#3226)
* Remove the meta file
2019-08-07 17:35:12 +08:00
kezhenxu94 49f50a24a4 Support upgrade backend w/o rebooting agents (#3170)
* Support upgrade backend w/o rebooting agents
2019-08-02 09:25:17 +08:00
彭勇升 pengys 16b8236506 1. All the inventories add last_update_time attribute, delete the mapping_last_update_time attribute in service inventory. (#3168)
2. Fixed the service inventory and net address inventory update bugs.
2019-07-26 14:02:27 +08:00