Commit Graph

391 Commits

Author SHA1 Message Date
吴晟 Wu Sheng d63f2c9427
Support topology with multiple selected services (#4408)
* 1. Format codes

2. Add comments for storage interface of topology

3. Narrow the scope of secondary query to get the component name. Only query the nodes as client out of the given service Ids.

* Support topology with multiple selected services

* Sync the typo fix in the protocol
2020-02-24 23:01:13 +08:00
Daming 29da5738bc
Provide influxdb as a new storage plugin (#4239)
* provide influxdb as a new storage plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-19 18:00:57 +08:00
mrproliu 6185351688
fix profile analyze missed data (#4378) 2020-02-17 22:33:55 +08:00
吴晟 Wu Sheng 334239388a
Support Daily Index Step feature in the OAP (#4368)
* Support Daily Index Step.

* Fix format.

* Fix mischanged ES.
2020-02-16 20:48:04 +08:00
吴晟 Wu Sheng ae442e364f
More comments for important classes (#4361) 2020-02-14 17:03:50 +08:00
吴晟 Wu Sheng 6be3e9909a
Help people to read source codes of core module. (#4357)
* Add more comments in the core module, should help the source code readers a lot.
2020-02-13 21:11:05 +08:00
吴晟 Wu Sheng 0300048d92
Fill some comments for core. (#4356)
* Fill some comments for core.

* Fix a comment issue.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-02-13 11:27:17 +08:00
cui-liqiang c761f30e3b
Some code style and refactor (#4355)
* extract duplicated method

* change while loop to if since once rs.next() return false, it will never return true for the following calls

* simplify if() return true; return false pattern

* fix compilation

Signed-off-by: liqiang.clq <liqiang.clq@alibaba-inc.com>
2020-02-13 10:34:32 +08:00
吴晟 Wu Sheng 7c6016b18f
Make Profile entities could be disable by OAL (#4354)
* Make Profile entities could be disable by OAL
2020-02-12 23:28:27 +08:00
Jared Tan 7ff9a0cfd9
supprt gRPC alarm hook. (#4344)
* support gRPC hook.

* revert submodule commit.

* revert submodule commitid.

* finish logical.

* remove author anonatation.

* fix ci.

* revert submodule.

* update logical.

* update logical.

* update logical.

* fix

* fix

* fix empty settings.

* fix.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-12 16:20:20 +08:00
kezhenxu94 5b255ba3e3
Set up more strict code styles and fix existing issues (#4337)
Motivation:

Review code styles with some bots automatically.

Modifications:

Set up ReviewDog in GitHub Action to review code style.

Add more check rules to checkstyle plugin.

Result:

Obvious code styles can be reviewed and commented automatically.
2020-02-11 17:10:49 +08:00
mrproliu fa526e5227
Provide profile analyze query (#4335)
* provide profile analyze query

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-11 10:11:08 +08:00
mrproliu e01829aa57
Change profile stack element data structure (#4332)
* Change profile stack element to single level, not using tree
2020-02-09 16:37:45 +08:00
mrproliu e44bc36281
provide profiled segment list query (#4322)
* provide profiled segment query

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: AirTrioa <44222967+AirTrioa@users.noreply.github.com>
2020-02-06 21:02:08 +08:00
mrproliu 30ebe1caf9
provide thread stack analyzes (#4308)
* provide thread analyzes

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-04 16:24:05 +08:00
mrproliu 3252b6ed08
provide profile analyze empty interfaces (#4303)
* provide profile analyze empty interfaces

* fix comment error

* change comments

* update submodule

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-31 21:11:43 +08:00
Daming 9005d1e82c
Provide AVRO plugin (#4276)
* Provide Avro plugin
* provide testcase for avro-plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-30 17:49:43 +08:00
吴晟 Wu Sheng 17e3fff83a
Refactor the storage to support `getSubsetOfMultipleLinearIntValues` (#4289) 2020-01-29 17:03:39 +08:00
kezhenxu94 75aefd34de Fix potential concurrent problems and tune performance (#4274) 2020-01-20 14:59:41 +08:00
mrproliu 90f3c4de55 Sniffer processing profile task and report status and snapshot (#4220)
* sniffer processing profile task and report status and snapshot

* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`

* resolve names

* change profile to single one thread run.

* 1. change to the ArrayList, because known the max size
2. rename issue resolved

* add profiling status enum

* change sniffer use full name issue

* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task

* remove unnecessary getter

* add test assert error message

* adding `AgentServiceRule`

* revert original assert

* remove unnecessary getter

* resolve issues

* reduce findService invoke

* resolve style error

* recheck profiling when change first span operatin name

* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting

* remove article link

* add `ProfileTask#maxSamplingCount` check

* resolve conflict (Downstream -> Commands)

* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump

* remove unused return val

* remove some `@param` and `@return`

* add profile task check result data bean

* change profiler slot to `AtomicReferenceArray`

* resolved java doc error

* fix doc error, remove meaningless descriptions

* resolve missed profile receiver on oap starter

* resolve method invoke error

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-18 18:38:03 +08:00
kezhenxu94 019af3e345
Drop support of protocol v1 (sw3) (#4244)
* Drop support of protocol v1 (sw3)

* Remove versioning logic further

* Remove more version5-only classes

* Commit missing files

* Update submodule and address review comment

* Fix failed tests
2020-01-17 20:33:52 +08:00
kezhenxu94 094eeb870f
Add alarm metrics OP >= and <=, and refactor the operator logics (#4221)
* Add alarm metrics OP >= and <=, and refactor the operator logics

* Add unit test for OP

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-13 00:50:37 +08:00
Jared Tan 1310719645
add token authentication between agent and oap receiver. (#4197)
* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 23:08:39 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
Jared Tan 7e9812e857
remove duplicate maven dependency. (#4215) 2020-01-11 11:01:18 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
吴晟 Wu Sheng 6c20f4a41b
Add service type from register and support browser as service type (#4157)
* Add service type from register and support browser as service type

* Add service type at agent side.

* Fix format.

* Fix a test.

* Fix formats.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-01 18:01:41 +08:00
mrproliu d38a8eaaa7 Provide thread monitor create task feature (#4145)
* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-31 20:49:45 +08:00
Gao Hongtao 4211eff558 Fix os maven plugin issues for Eclipse m2e (#4144)
* Fix mave-os plugin issues for Eclipse m2e

* Add os-msven-plugin.version property
2019-12-29 13:49:27 +08:00
mrproliu 2fcf7c4649 provide config stream processor (#4133)
* provide config stream processor

* 1. rename config to none stream
2. add comments on none stream relate classes

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2019-12-27 14:58:15 +08:00
aderm e25b1e6008 improve if find parent, quick exit (#4107) 2019-12-26 10:02:42 +08:00
aderm 8476316e96 refactor code. (#4120) 2019-12-25 23:35:35 +08:00
吴晟 Wu Sheng b570996006
Update JDK compiling level to 8, and remove zip dist. (#4119)
* Update JDK compiling level to 8, and remove zip dist.

* Exclude annotation and shade API.

* Fix more exclusion.

* 1. Update grava and gson 2. Fix compile issue. 3. Keep protoc and maven plugin in old version. Some incompatible of 3rd party grpc proto.

* Fix dependency conflicts, license and notice issue, and dependency check script issue

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-25 15:55:24 +08:00
吴晟 Wu Sheng d7f7799c80
Set the main branch to 7.0.0-SNAPSHOT (#4115)
* [maven-release-plugin] prepare release v6.6.0

* [maven-release-plugin] prepare for next development iteration
2019-12-24 16:37:13 +08:00
zhang-wei 4c99480f70 Add relation metrics verification in e2e test (#4080) 2019-12-19 13:26:27 +08:00
Daming f9a849d6be fix typo (#4082) 2019-12-18 21:03:43 +08:00
吴晟 Wu Sheng 5f420bf3ac
Fix service instance relationship ID build bug. (#4075)
* Fix service instance ID build bug.

* Add instance ids into the relation metrics.
2019-12-17 14:12:56 +08:00
Jared Tan a6d0366c22 support min function. (#4072)
* support min function.

* fix logical bug and update test case.

* the same here.
2019-12-16 22:20:30 +08:00
吴晟 Wu Sheng 4e50132817
Remove the local/exit span register mechanism in Java agent scenario (#4059)
* Remove the local span and exit span register mechanism in Java agent scenario.

* Fix config, plugin, and document, as exit span is not required register anymore.

* Try to fix the test case expected data due to no register for exit span.

* Fix toolkit case.

* Remove exit span from plugin test framework document.

* Update protocol, FAQ, and CHANGELOG documents to highlight this change.

* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.

* Fix the snapshot and add inexistence operation name id.

* Fix several async cases.

* Fix undertow test case.

* Fix wrong error log name.

* Keep name consistent with 6.x concepts.

* Try to fix 3 async scenarios.

* Fix okhttp case.

* Fix Spring Async case.

* Fix another NPE

* Fix NPE when first span has no endpoint id when it is local or exit span.

* Still fill first operation name when no endpoint id.
2019-12-15 19:15:47 +08:00
kezhenxu94 eea5d027cc
Add Armeria Java agent plugin (#4024) 2019-12-11 15:57:45 +08:00
mrproliu cfb3379cd0 fix judge NPE when worker not found (#4039) 2019-12-10 23:22:56 +08:00
zhang-wei 3741f4e386 [Feature] Service instance dependency (#3978)
* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* dispatcher

* getServiceInstanceTopology implementation

* CacheUpdateTimer

* fix compilation failure

* fix compilation failure

* fix esDao

* Update dao list

* test e2e

* add official_analysis.oal & update getServiceInstanceTopology

* test instance topo for single service

* fix gql

* fix verifyServiceInstances

* verifyServiceInstanceTopo

* fix verifyServiceInstanceTopo

* fix ServiceInstanceInventory#name

* fix ServiceInstanceInventory#name

* e2e test

* fix action

* fix provider port

* fix Instance Node Type

* re run

* fix expected-data

* add e2e mysql

* set @Test timeout

* fix ci

* increase timeout

* re check

* test ttl es7

* test ttl

* fix action

* merge group

* test ttl es7

* replace official_analysis.oal

* replace official_analysis.oal

* comments

* add TODO

* add TODO

* fix filed name
2019-12-09 08:23:21 +08:00
Gao Hongtao 6475ab3d1d Add apdex function to OAL (#3855)
* Add apdex function to OAL

* Add empty line

* Setup config watcher

* Add identifier type to function parameter

* Add config test

* Update score algorithm

* Replace responseCode with status

* Add comments about apdex score algorithm

* Add docs

* Add e2e test case

* Update test case

* Fix disptch class generating error

* Update value name of apdex metric

* Tuning threshold

* Fix single tolerated point bug
2019-11-29 12:54:33 +08:00
吴晟 Wu Sheng 0724cdb42d
Support ordered segment list in trace query (#3906)
* Sort the segment fragments

* Add comment.

* Remove unnecessary box codes.
2019-11-20 22:39:51 +08:00
Jared Tan 5d064ced5f make topN worker report period configurable. (#3892) 2019-11-19 17:26:30 +08:00
于玉桔 6ffd92792d Support spring-tx in optional plugin (#3857)
* plugin code

* NPE FIX
2019-11-19 10:05:01 +08:00
吴晟 Wu Sheng d7038fe67b
Refactor DataCarrier, support ArrayBlockingQueueBuffer as implementor (#3849)
* Refactor DataCarrier, support ArrayBlockingQueueBuffer as the implementation for blocking queue buffer.

* Fix style issue.

* Remove import.

* Remove uesless codes.
2019-11-15 23:33:40 +08:00
于玉桔 db0b7486a5 fix component-libraries.yml miss component:rest-high-level-client (#3859)
* fix component-libraries.yml miss component:rest-high-level-client

* Update component-libraries.yml

Add mapping.
2019-11-14 17:59:59 +08:00
吴晟 Wu Sheng 3fbaec42dc
Move to 6.6.0 SNAPSHOT (#3839)
* [maven-release-plugin] prepare release v6.5.0

* [maven-release-plugin] prepare for next development iteration
2019-11-12 22:33:39 +08:00
吴晟 Wu Sheng 42d422e852
parameterize the cache size. (#3741) 2019-10-30 11:15:11 +05:30
Jared Tan 852f03d1db
remove unused time range query parameter. (#3730) 2019-10-28 22:21:54 +08:00
qiyang 69a28d4591 Add register persistent worker latency metrics #3671 (#3677)
* Add register persistent worker latency metrics  #3671

* remove never thrown exception on Time#close()
add panel to mesh-mode-grafana.json

* remove never thrown exception on Time#close()
add panel to mesh-mode-grafana.json
2019-10-24 15:22:38 +08:00
mrproliu c640b512cd Provide agent plugin netty-socketio (#3640)
* add plugin netty-socketio https://github.com/mrniko/netty-socketio

* delete create exit span interceptor.

* delete create exit span interceptor testcase.

* delete create exit span interceptor testcase.

* add constructor test unit

* change bind on random port in test case

* reverse upstream master whitespace

* resolve build_id param

* delete health check segment test case

* add ContextManager.isActive check before use activeSpan invoke

* remove unnecessary code and blank line
2019-10-23 13:06:17 +03:00
Jared Tan 2f3d009e63 Makes the scope of alarm message more semantic. (#3680)
* Makes the scope of alarm information more semantic.

* update md.

* update md.

* add ruleName in AlarmMsg.

* update md.

* update test case.

* fix coveralls.

* fix coveralls.

* fix ci.
2019-10-22 19:35:04 +03:00
Kdump c0be757f43 add graphql getServiceInstance instanceUUID field. (#3595)
* add graphql getServiceInstance instanceUUID field.

* add graphql getServiceInstance instanceUUID field.

* add graphql getServiceInstance instanceUUID field.

* sync query-protocol repo

* add instanceUUID field e2e

* add instanceUUID field e2e

* add instanceUUID field e2e

* add instanceUUID field e2e
2019-10-17 06:38:00 +03:00
Yanick.xia d68c0883ed Fix more resonable error (#3619)
* Fix #3402

* Fix unecessary blankline

* Optimize rebuild logic

* Fix checkstyle fail

* Add concurrency test

* Fix text mistakes

* Optimize refresh logic && new test for return clients never change by refresh logic

* Make unit test more clearness

* Add more unit test && fix a bug

* Rename variable && format by skywalking style
2019-10-16 19:33:54 +03:00
echooymxq 1b1e80b75a Optimize code (#3600) 2019-10-11 19:17:03 +08:00
mrproliu 58804174e1 Provide plugin for Ehcache 2.x (#3575)
* support ehcache 2.x
2019-10-10 22:39:37 +08:00
吴晟 Wu Sheng 3cdc1a1928
Fix 5 places of using wrong Strings util (#3582)
* Exclude a dependency for test.

* Fix 5 places of using wrong Strings util.
2019-10-09 22:55:01 +08:00
Jared Tan 2db2638fc2 code cleanup: remove unnecessary and optimize the function's parameters. (#3577)
* code cleanup: remove unnecessary and optimize the function's parameters.

* fix code style.
2019-10-09 10:05:23 +08:00
kezhenxu94 114a74f35a
Fix some miscellaneous bugs (#3567) 2019-10-07 17:35:22 +08:00
吴晟 Wu Sheng 8ebf3aac04
Support timeout configuration in agent and backend. (#3491)
* Support timeout configuration in agent and backend.

* Fix CI

* no message
2019-09-20 15:02:28 +08:00
吴晟 Wu Sheng 6942ec8fcf
Language of instance could be NULL (#3485)
* Language of instance could be NULL

* Fix style check.

* Fix duplicated test cases.
2019-09-19 00:23:11 +08:00
Louis Tsui 72f07b812a Add Light4j plugin (#3323)
* Add Light4J 1.6.x plug-in

* Add unit test for Light4J plug-in

* Add missing id to server-starter

* Remove mentions of version 1.6.x

Some local testing showed that the plugin successfully generated traces for a light4j project built with light-codegen v2.x. Therefore, there is no need for a separate plugin for v2.x and no distinction needs to be made with light4j v1.6.x.

* Update supported light4j versions

* Use ContextSnapshot to maintain trace

In a typical Light4j project, the ExceptionHandler#handleRequest is the asynchronous execution point. The flow is ExceptionHandler#handleRequest -> HttpServerExchange#dispatch -> ... -> ExceptionHandler#handleRequest. This means that handleRequest is being intercepted twice and each time a LocalSpan is created. Without handling thread propagation, the trace segment gets split up.

By saving a ContextSnapshot into the enhanced object (ExceptionHandler), we can determine if it is necessary to continue the segment.

* Update logic for span creation in trace

By default, the Light4J plugin now overrides the Undertow plugin's entry span with its own. This new entry span is created just before execution of the ExceptionHandler#handleRequest in the I/O thread. The entry span is generated here because it is considered the starting point of the Light4J handler chain.

There is now also the option to enable detailed span creation when tracing requests through Light4J. These local spans mark which middleware and business handlers are involved with handling the HTTP request to a Light4J service.

* Update skywalking version

* Update light4j version

* Fix LightInstrumentation javadoc

* Remove import violating checkstyle

This import was added from the link reference in the javadoc. The javadoc has now been changed to just reference the method name.

* Move light4j agent config property

* Sync UI
2019-09-18 21:43:17 +08:00
Jared Tan 6c347a1169 agent support custom properties. (#3367)
* agent support custom properties.
2019-09-17 14:44:42 +08:00
吴晟 Wu Sheng 06c357da6c
Make sure the cluster register happens before streaming process. (#3471) 2019-09-17 08:53:02 +08:00
stone.wlg bb0b3d369f provide cassandra java driver 3.x plugin (#3410)
* add cassandra java driver 3.x plugin
2019-09-12 16:14:48 -07:00
kezhenxu94 721f86485c Add TTL E2E test (#3437)
* Add TTL E2E test

* Add to Jenkins stage and minor bugfix

* Upgrade e2e container version

* Polish and minor fix

* Polish
2019-09-09 22:06:57 -07:00
吴晟 Wu Sheng ce20fe107d
Move to 6.5.0 (#3439)
* [maven-release-plugin] prepare release v6.4.0

* [maven-release-plugin] prepare for next development iteration
2019-09-08 16:24:21 -07:00
吴晟 Wu Sheng 2a2679eefb
Fix Pxx calculate bug. (#3391) 2019-09-02 10:14:33 +08:00
kezhenxu94 6fd85c8303
Avoid unnecessary resetting when service mapping id is already 0 (#3383)
* Avoid unnecessary resetting when service mapping id is already 0

* Avoid as early as possible and minor optimization

* Remove unused import

* Polish
2019-08-30 23:54:54 +08:00
kezhenxu94 54614a0594 Support gateway without agent (#3308)
* Support Gateway without agent
2019-08-30 16:17:41 +08:00
Andy Ai 8bf7e346ee Support play 2.x (#3207)
* Support play 2.x
2019-08-29 17:00:51 +08:00
吴晟 Wu Sheng dc0f6530cb
Fix TTL issue. (#3292)
* Fix TTL issue.

* Fix issue.

* Rename `getSecondTimeBucket` to `getRecordTimeBucket`.
2019-08-20 15:36:31 +08:00
kezhenxu94 db9759fcc2 Check license header strictly (#3274)
* Check license header strictly

* Ignore header template

* Include more files to check list
2019-08-16 14:37:44 +08:00
于玉桔 a0e816771c Webflux plugin optimization (#3243)
* webflux
2019-08-13 22:52:01 +08:00
于玉桔 73cf05c5b9 fix component-libraries typo (#3241) 2019-08-10 17:07:18 +08:00
yantaowu 1bfda1dd65 Modify the thread pool of grpc server. (#3225)
* Modify the thread pool of grpc server.

* add license header

* make param confingable

* Add sharing and Jaeger config.

* Remove grpc thread config from yml
2019-08-09 09:14:55 +08:00
吴晟 Wu Sheng 0df76e4e38
Remove the scope-meta file (#3226)
* Remove the meta file
2019-08-07 17:35:12 +08:00
吴晟 Wu Sheng b247dc979e
Move project master branch to 6.4.0-SNAPSHOT (#3220)
* [maven-release-plugin] prepare release v6.3.0

* [maven-release-plugin] prepare for next development iteration
2019-08-05 21:19:17 +08:00
kezhenxu94 49f50a24a4 Support upgrade backend w/o rebooting agents (#3170)
* Support upgrade backend w/o rebooting agents
2019-08-02 09:25:17 +08:00
彭勇升 pengys 16b8236506 1. All the inventories add last_update_time attribute, delete the mapping_last_update_time attribute in service inventory. (#3168)
2. Fixed the service inventory and net address inventory update bugs.
2019-07-26 14:02:27 +08:00
彭勇升 pengys f658d9eb4d Simplify the PxxMetrics and ThermodynamicMetrics to improve performance (#3162)
* Feature of database session

* Make it configurable.

* Change the metrics process flow.
before: metrics entrance -> aggregate worker -> remote worker -> trans worker -> minute, hour, day, month persistence worker -> storage
after: metrics entrance -> aggregate worker -> remote worker -> minute persistence worker ->  trans worker -> hour, day, month persistence worker -> storage

* IntKeyLongValueHashMap instead of IntKeyLongValueArray.

* Make the OAP server can't startup.

* Finish

* Rename the method and fixed some test case issues.

* Rename field.

* no message
2019-07-25 19:08:36 +08:00
彭勇升 pengys 8605f4ccc0
Change the metrics process flow. (#3157)
* Feature of database session

* Make it configurable.

* Change the metrics process flow.
before: metrics entrance -> aggregate worker -> remote worker -> trans worker -> minute, hour, day, month persistence worker -> storage
after: metrics entrance -> aggregate worker -> remote worker -> minute persistence worker ->  trans worker -> hour, day, month persistence worker -> storage
2019-07-24 18:18:20 +08:00
彭勇升 pengys 8d234a91e5 Feature of database session in OAP server. (#3147)
* Feature of database session

* Make it configurable.

* Make the OAP server can't startup.
2019-07-24 13:43:19 +08:00
吴晟 Wu Sheng f83dc64602
Fix a wrong comment. (#3134) 2019-07-21 18:38:19 +08:00
彭勇升 pengys 8306ad4531 Reduce the number of threads to improve performance. (#3133)
* Refactor Persistence worker.

* 1. Provide InsertRequest and UpdateRequest interface for prepare persistence.
2. Implement the ids query for H2 metrics DAO.

* Refactor worker framework

* Use queue to receive asynchronous batch request.

* Rename the Datacarrier thread name.

* Fixed some mistake.

* New mistake.
2019-07-21 15:51:24 +08:00
彭勇升 pengys 594994ffae 1. Provide InsertRequest and UpdateRequest interface for prepare persistence. (#3131)
2. Implement the ids query for H2 metrics DAO.
2019-07-21 13:56:12 +08:00
彭勇升 pengys 6de3e55280 Too many EndOfBatchContext instance created. (#3129) 2019-07-21 00:26:17 +08:00
吴晟 Wu Sheng d256fc348e OAP internal RemoteService protocol change and code refactor (#3128)
* Remove the worker id, and add worker name for remote handler only.

* Remote metrics and inventory classes mapping too.

* Refactor codes.
2019-07-20 23:47:17 +08:00
彭勇升 pengys 6338de392f Improve OAP server performance. (#3127)
* Improve OAP server performance.
2019-07-20 22:59:52 +08:00
彭勇升 pengys ed507b78ef
Dirty-Read in concurrency (#3117)
* Fixed the problem of dirty reading of the storage implementation of elasticsearch.

* Fixed a mistake.
2019-07-19 14:22:01 +08:00
吴晟 Wu Sheng 5b039e9747 All NEW OAL Runtime Engine (#3109)
* Change OAL generator tool to runtime.

Step 1. change project structure. API links and maven pom.

* Part of metrics class generation

* Metrcis class generated.

* Set up the basic structure of new OAL engine.

* Finish metrics generation.

* Support dispatcher generation.

* Format codes.

* Generate dispatcher all methods.

* Implement disable in hardcode.

* Clear up

* Fix compile startup.

* Update license and document of new OAL engine.

* Fix rat.

* Disable class output in default

* Copy from @kezhenxu94 's test PR, https://github.com/apache/skywalking/pull/3110

* Wait for the mount command to finish
2019-07-18 19:47:23 +08:00
彭勇升 pengys 54b01223de Aggregate worker will hold the metrics while there is no new metrics in. (#3100) 2019-07-17 13:59:54 +08:00
吴晟 Wu Sheng 34b63ae135
Make persistent period configurable. (#3070) 2019-07-14 16:13:32 +08:00
吴晟 Wu Sheng 8b2cfa2cd8 Try to fix MySQL TTL failure. (#3038) 2019-07-10 23:29:13 +08:00
吴晟 Wu Sheng ba24490166 Remove "IDs can't be null" in backend, as UI hasn't fixed this for a long time. (#3039) 2019-07-10 17:32:45 +08:00
Gao Hongtao 01b963c109 Adding deadline to gRPC client (#2987)
* Set up 10 seconds deadline after gRPC client sending
 * The duration of deadline contains three segments: connecting,
   request and response

For blocking stub, I just set deadline before invoke service. For
bi-streaming stub, I found all of them are used as a blocking style,
that after getting streaming response client stub just close current
streaming. Base on above reality, I pick the same way as blocking one.
2019-07-03 14:34:04 +08:00
于玉桔 48ffd09239 Spring @Async plugin support (#2902)
* spring_async_plugin

* fix

* fix

* fix

* fix

* format fix

* rename

* fix

* fix version

* Update SWCallable.java

* Update Supported-list.md

* Update Supported-list.md
2019-07-02 07:46:25 +08:00
吴晟 Wu Sheng 0225711e26
Check duplicate providers in bootstrap (#2976)
* Enhance module manager
2019-07-01 15:54:12 +08:00
吴晟 Wu Sheng 4faa950c14
Move master to 6.3.0 (#2968)
* [maven-release-plugin] prepare release v6.2.0

* [maven-release-plugin] prepare for next development iteration
2019-06-29 13:49:36 +08:00
彭勇升 pengys dea6cfb010 Change the statement column type from Keyword to Text. (#2944)
* Use ids query method instead of get method for metrics persistence.

#2933

* Query by alias name.

* #2943

Change the statement column type from Keyword to Text.
2019-06-25 17:55:12 +08:00
吴晟 Wu Sheng 85acc1b191
Revert "Introduce dead line settings from gRPC document. (#2917)" (#2936)
This reverts commit db8c45465b.
2019-06-24 00:08:59 +08:00
吴晟 Wu Sheng db8c45465b Introduce dead line settings from gRPC document. (#2917) 2019-06-21 00:53:31 +08:00
daming 6ff33d8816 Restore solrj-plugin (#2888)
* restore solrj-plugin

* restore solrj-plugin
2019-06-17 18:48:18 +08:00
彭勇升 pengys 70064f47cb Use the ids query method instead of get method. (#2887)
#2886
2019-06-17 12:05:21 +08:00
吴晟 Wu Sheng 0103d8ad28
Fix no stream register. (#2873)
* Fix no stream register.

* Make sure list order.

* Refactor codes.

* Fix wrong revert.
2019-06-14 11:31:51 +08:00
吴晟 Wu Sheng 0f9c27d481
Revert SolrJ plugin. (#2871)
* Revert SolrJ plugin.

* Remove an unexpected test case.

* Revert "Remove an unexpected test case."

This reverts commit 359990e18b0c0ae1a6e1005ff985bf237ec05619.

* Clear context.

* Fix a leak.
2019-06-14 09:24:58 +08:00
彭勇升 pengys 4e63331d35 #2860 (#2861)
Get the document by index name.
2019-06-12 18:56:11 +08:00
彭勇升 pengys 1c545a3708 Fixed a time series bug about downsampling setting in processors are different from the setting in receivers. (#2856)
#2851
2019-06-12 08:17:15 +08:00
吴晟 Wu Sheng 2197820577
Support exporter runs in increment and total modes (#2840)
* Make exporter interface better.

* Fix review.
2019-06-09 11:32:27 +08:00
daming 6ae517423c Provide plugin for Solr-7.x(client) (#2730)
* new branch for solrj-plugin

* rollback and add module solrj-plugin

* preparing to pr

* to resolve reviewers' suggestions

* remove unused code

* remove unused code

* remove unused comments

* To avoid NPE

* fix typo

* Change to JRE6

* fix pom.xml merge incorrectly. and java.net.URL instead of Regex

* Using RuntimeContext instead of ThreadLocal

* To reduce unnecessary tags

* add test cases

* fix validation fail

* add solrj-plugin into component-libraries & Supported-list

* to trace all patch to avoid recheck status of span

* remove unnecessary properites

* remove unnecessary tags

* Add the config to document of setup
2019-06-05 22:54:45 +08:00
彭勇升 pengys 1f7125a02e Remove storage annotation. (#2813) 2019-06-04 09:53:50 +08:00
吴晟 Wu Sheng 2e846c6078 Dynamic Configuration and Dynamic Configuration Service (#2810)
* Implement the base of config API module.

* Finish configuration module codebases w/o tests, revert some changes.

* Finish mock tests.

* Add new gRPC service for configuration.

* Make gRPC configuration sync provider works.

* Add database slow statement dynamic setting.

* Finish the dynamic configuration service.

* Fix version.
2019-06-03 23:11:52 +08:00
彭勇升 pengys 5036e83343 Good news of an important feature: time series implementation in Elasticsearch storage. (#2808)
* The new feature of Elasticsearch storage with the time series implementation.

* Time bucket util improve.

* Fixed some Gramma error.

* Update ttl.md

Polish English a little.

* Update ttl.md

Add more explicit description of TTL

* Update ttl.md

* Update StorageModuleElasticsearchConfig.java

Set otherMetricsDataTTL = 0 as default, make user more clear.

* Rename the timeSeriesAble to capableOfTimeSeries

* Fixed a test case failure.
2019-06-03 18:46:42 +08:00
彭勇升 pengys 0a696813c8 Time bucket util improve. (#2795) 2019-05-31 15:02:49 +08:00
彭勇升 pengys de46ee62d3
Make the step just to be the graphql java object. (#2792)
* Make the step just to be the graphql java object.

* Mistake.
2019-05-31 13:16:59 +08:00
FuCheng,Yan bc6f06f964 Support RESTEasy 3.x (#2665)
* feat(agent sdk): resteasy plugin

* fix(agent sdk): fix component define

* fix(agent sdk): fix file license

* fix(agent sdk): fix file license

* docs(agent sdk): add supported list

* fix(agent sdk): fix file license

* fix(agent sdk): fix response error tag

* fix(agent sdk): fix

* fix(agent sdk): adapter 3.5+

* fix(agent sdk): remove unreasonable async handling
2019-05-24 22:14:43 +08:00
吴晟 Wu Sheng 595498e787
New topology query protocol implementation (#2654)
* Make backend fits new topology query protocol and logic.

* Change topology line id.

* Change endpoint point id rule to match service relation id rule.

* Refactor util.

* Revert wrong refactor by IDEA.

* Revert some changes.

* Fix entity id bug.

* Fix endpoint topology query

* Update UI.

* UI license update.
2019-05-21 14:50:29 +08:00
于玉桔 98b495ebcc Spring cloud gateways 2.1.x plugin (#2517)
* spring cloud gateways plugin
2019-05-20 17:06:40 +08:00
彭勇升 pengys 4eb5def126 Manage models in one place. (#2695) 2019-05-17 00:33:29 +08:00
彭勇升 pengys 63fb961c0d New OAP annotations, they are friendly for streaming loader. (#2670)
* New stream annotation.
- Inventory annotation example:
@Stream(name = ServiceInventory.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_INVENTORY, storage = @Storage(builder = ServiceInventory.Builder.class, deleteHistory = false), kind = StreamKind.Inventory)

- Metrics annotation example:
@Stream(name = ServiceRelationClientSideMetrics.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_RELATION, storage = @Storage(builder = ServiceRelationClientSideMetrics.Builder.class), kind = StreamKind.Metrics)

- Record annotation example:
@Stream(name = SegmentRecord.INDEX_NAME, scopeId = DefaultScopeDefine.SEGMENT, storage = @Storage(builder = SegmentRecord.Builder.class), kind = StreamKind.Record)

- TopN annotation example:
@Stream(name = TopNDatabaseStatement.INDEX_NAME, scopeId = DefaultScopeDefine.DATABASE_SLOW_STATEMENT, storage = @Storage(builder = TopNDatabaseStatement.Builder.class), kind = StreamKind.TopN)

* no message

* Stream annotation finish.

* Make stream processors to be the singleton class.

* Fixed the compile errors.

* Modify the OAL module for this annotation refactor.

* Add apache license header.

* Duplicate import.

* Fixed check style error.
2019-05-15 22:42:38 +08:00
于玉桔 1b6bd5f81d study sharding sphere (#2679) 2019-05-15 19:12:09 +08:00
caoyixiong 91ffae2526 support Nacos cluster (#2640)
* Fix #2546

* Change the stackdepth type

* Fix bug

* Fix CI failed

* fix #2599

* fix ci

* add license

* add ActiveTag error methods

* fix

* fix

* nacos

* nacos

* nacos

* nacos

* fix ci

* support nacos cluster

* fix yml

* fix license

* fix ci

* remove optional

* update license

* update license and doc

* fix
2019-05-15 00:29:23 +08:00
彭勇升 pengys ed78dabe8b Rename metric and indicator to metrics. (#2643)
* Rename metric to metrics.

* Fixed test case execute failure issues.
2019-05-10 08:05:37 +08:00
吴晟 Wu Sheng 9b87086d72
Move to 6.2 (#2563)
* Move submodule plugin.

* Always auto-submodule

* [maven-release-plugin] prepare release v6.1.0

* [maven-release-plugin] prepare for next development iteration

* Change keys location.
2019-04-30 12:16:50 +08:00
Brandon Fergerson ce1c7aadfc Vert.x Core 3.x Plugin (#2386)
* vertx plugin impl
2019-04-16 23:46:49 +08:00
彭勇升 pengys 73e8853fa5 Provide elasticsearch template operation method. (#2474)
* Provide elasticsearch template operation method.

* Fixed compile failure issue.
2019-04-11 20:08:51 +08:00
吴晟 Wu Sheng df362b18f0
Query protocol and codebase for log type record (#2449)
* New query graphql protocol added.

* Finish a new metric, log(abstract) with HTTP Access Log.

* Submit the draft codes of logs.

* Remove @IDColumn

* Fix startup issue.

* Add http_access_log to disable list.

* Finish a wrong service implementation.

* Add trace id in log query condition and result.

* Fix style
2019-04-08 07:53:06 +08:00
吴晟 Wu Sheng ff6fb90d38
Add jaeger traces support (#2434)
* Make Jaeger receiver works in 12450 as default.

* Finish compile and codebase.

* Make codes runnable.

* Finish Jaeger query tests.

* Add document for jaeger receiver.

* Add readme.

* Fix a break link.

* Finish doc.
2019-04-01 20:52:59 -07:00
吴晟 Wu Sheng 601b472ef6 Support Backend acts as pure Zipkin collector (#2424)
* Codebase for zipkin span persistence.

* Fix missing fields in storage.

* Miss the latency field.

* Finish some tests.

* Fix wrong latency.

* Finish doc and reset application.yml

* Make the description more clear.
2019-03-31 01:36:36 +08:00
彭勇升 pengys 1fd5c5790a
Divide two static classes (#2411)
* 1. Remove static worker instances holder.
2. Remove static worker instance id generator.

* Fixed test case failure.
2019-03-30 15:37:19 +08:00
吴晟 Wu Sheng 9a09e96204 Support disable statement in OAL (#2402)
* Try to make disable work.

* Make disable works.

* Add disable document.
2019-03-26 23:34:31 +08:00
吴晟 Wu Sheng 975a06d163 Add catalog to ScopeDeclaration annotation. Make the alarm adapt to it. Now more metric could go through alarm, such as JVM, CLR. (#2391) 2019-03-21 18:38:32 +08:00
Ahoo Wang 3438151693 Sync SmartSql provider setting to all of component-libraries.yml (#2394)
* Add SmartSql provider components in component-libraries config

* sync SmartSql provider setting -> fix https://github.com/apache/incubator-skywalking/issues/2390
2019-03-20 23:06:13 -07:00
彭勇升 pengys 7419a59125 Supplement unexpected exception message. (#2389) 2019-03-20 18:02:16 -07:00
Lemon 7047c97e8b Add CLR_Receiver plugin (#2385) 2019-03-20 20:53:42 +08:00
peng-yongsheng 610a53784f fixed expressions error. 2019-03-17 00:21:20 +08:00
peng-yongsheng 940130e24a Added 4 expressions in OAL syntax: >, <, >=, <= 2019-03-17 00:15:51 +08:00
吴晟 Wu Sheng a3a85e520e Support Exporter in core (#2368)
* Rename ALarmSupported to WithMetadata

* Finish the base of exporter.

* Fix class comment.
2019-03-16 23:27:15 +08:00
吴晟 Wu Sheng 0951b9434b Make k8s-Coordinator works in role mode. (#2347)
* Make k8s-Coordinator back in role mode.

* Start query API server after the core started.

* Do lock when port has been intialized.

* Rename service.

* Fix text cases.

* Declare ConfigService.class in core module.

* Implement in an easier way.

* Envoy print whole metric, now in debug.

* Forcedly fix envoy timestamp issue.
2019-03-12 15:36:36 +07:00
彭勇升 pengys ab398f79c9 The role of OAP server. (#2344)
* The role of OAP server.
Mixed: Receive agent data, level 1 aggregate, level 2 aggregate
Receiver: Receive agent data, level 1 aggregate
Aggregator: level 2 aggregate

* Mistake, stadalone default.

* Update application.yml
2019-03-11 16:34:30 +08:00
吴晟 Wu Sheng 7c726905c3 If the node has no interaction, then it disappear in topology. (#2336) 2019-03-10 22:10:20 +08:00
吴晟 Wu Sheng c2a7e8e539
Bug fix. (#2333) 2019-03-08 16:31:42 +08:00
吴晟 Wu Sheng b4cb7e0bca
Envoy metric support and a bug fix (#2321)
* Extend envoy metric service, scope and OAL lexer/parser.

* Finish metric extension and oal.

* Make test env works.

* Finish test of envoy connection and memory metric.

* Change document.

* Finish docker compose test.
2019-03-07 17:05:46 +08:00
Gao Hongtao 5629bc33f3 Update RegisterPersistentWorker.java (#2307)
Fixed wrong parameter
2019-03-02 14:52:22 +08:00
Lemon 4a54c7687d Add clr metrics scope and oal script (#2297)
* Add CLR metric scope and oal_script

* Add max indicator

* Fix checkstyle
2019-03-02 09:36:48 +08:00
吴晟 Wu Sheng fb62025da8
Improve OAP extendibility (#2288)
* Make scope meta configurable.

* Fix test cases in ci.

* Change step 1

* Refactor step 2

* Other compile issue done. Prepare the generator changes.

* Revert some wrong changes to Istio receiver.

* Step 3, make compile pass.

* Try to make CI passed.

* Add a check.

* Fix generated code style

* Provide new and extendable dispatcher generator tool

* Fix startup.

* Can't guarantee the dispatchers are always existing. Put explicit comments at there to avoid confusion, since this is different with old version.

* Fix a H2 query for endpoint search.

* Fix a wrong mysql alarm query.

* Provide new document and `generate-tool-grammer` module.

* Add missing last section of document.

* Fix typo

* Relocate the @ScopeDeclaration annotation, make it more sense. Also remove the useless @SourceType annotation. And adjust document for these changes

* ScopeDeclaration can be used once for each class only, now.
2019-02-26 13:43:14 +08:00
Jared Tan c596d0ac82 add serviceInstanceId in Segment storage. (#2146)
* add serviceInstanceId in Segment storage.

* add serviceInstanceId in query condition.
2019-02-23 07:59:19 +08:00
吴晟 Wu Sheng 58cf5936e6 Improve OAP cluster performance (#2271)
* Refactor for cluster mode.

* Make send to L2 in certain cycle.
2019-02-21 16:33:07 +08:00