Commit Graph

239 Commits

Author SHA1 Message Date
吴晟 Wu Sheng 259a147682
Add miss doc about collecting parameters in the profiled traces. (#4552) 2020-03-23 09:41:02 +08:00
cngdkxw 2e9f001633
Add spring cloud feign 2.x plugin (#4526) 2020-03-20 20:28:30 +08:00
吴晟 Wu Sheng 1c7529a559
Add a miss link about apdex documentation. (#4543) 2020-03-20 08:24:35 +08:00
kezhenxu94 69e3d80e24
Add selector property in application.yml (#4514)
### Motivation

 Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.

 ### Modification

 Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.

 ### Result

 - We can switch a configuration by environment variables or system property

 - Closes #4511
2020-03-15 23:28:14 +08:00
Gao Hongtao 78d14ebbd6
Enable OAP gRPC SSL transportation (#4470)
* Enable OAP gRPC SSL transportation

Porting to OpenSSL to enable SSL transportation. The server private
key is in the format of PCKS#8, certificates is x509 though.

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add netty-tcnative-boringssl-static 2.0.26 licnese

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Add SSL to grpc remote client and sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update documents

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix nits

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix test case

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add e2e-ssl and remove other test cases temporarily

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add TLS relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Decrease timeout

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Test config works

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* triger CI on oap-ssl branch

* Disable agent ca

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Switch to sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add agent log to console

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Upload log files

* Change backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert "Change backend"

This reverts commit 7a085711e3653d3bf642aa84f73650e22595e383.

* Update backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert e2e

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Ignore TSL relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Polish codes

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update certificates expires year to 2120

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-03-15 08:15:28 +08:00
吴晟 Wu Sheng e69391a207
Support Secrets Management File in the ElasticSearch 6/7 storage (#4493)
* Temp commit

* Support secretsManagementFile file.

* Update doc.

* 1. Support JKS/pass runtime change too.
2. Follow review.

* Fix format.

* Fix username/password/trustPass haven't been updated in the es client.

* Fix doc issue.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
2020-03-13 20:50:14 +08:00
yoje ced5cb2828
Add finagle plugin (#4441)
* Add finagle plugin (#4433)

* fix failed checks for pr(#4441)
* Add licenses for some files
* Update Supported doc
* Add finagle to test component-libraries.yml

* Add finagle automatic tests into CI

* Add comments

* update comments

* Remove CompatibilityChecker

* Move query-protocol submodule to the right version

* Remove `enhancedInstance.setSkyWalkingDynamicField` in `ClientTracingFilterInterceptor`.

* Update expectedData.yaml of finagle tests

* Use lazy ContextCarrier injection.

* Add comments for `ContextCarrierHelper`

* Fix ContextHolderFactory to avoid context pollution

Co-authored-by: huangyongjie <huangyongjie@tigerbrokers>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-03-11 13:04:58 +08:00
kezhenxu94 9c974b80b4
Fix unclear doc and add missing variable (#4484)
* Revert "[BugFix] Docker InfluxDB cannot be enabled (#4483)"

This reverts commit 73d0802cd6.

* Fix unclear doc and add missing variable
2020-03-11 07:46:59 +08:00
吴晟 Wu Sheng d32450aa7d
Support name column(s) in the storage for 3rd party integration, like Kibana. Default OFF (#4439)
* Support dynamic column in the source.

* Support activeExtraModelColumns as a default OFF option.

* Fix words

* Update backend-setup.md

Fix format issue.
2020-03-04 14:44:54 +08:00
aderm 6d530c1d62
update doc. (#4403) 2020-02-24 00:30:13 +08:00
吴晟 Wu Sheng 4245334bac
Officially add LUA agent link in the main repo and document. (#4402) 2020-02-23 22:27:24 +08:00
Daming 29da5738bc
Provide influxdb as a new storage plugin (#4239)
* provide influxdb as a new storage plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-19 18:00:57 +08:00
吴晟 Wu Sheng 334239388a
Support Daily Index Step feature in the OAP (#4368)
* Support Daily Index Step.

* Fix format.

* Fix mischanged ES.
2020-02-16 20:48:04 +08:00
吴晟 Wu Sheng 329f7e150b
Support Downsampling Data Packing feature in ES storage implementation (#4364)
* Support day/hour/minute metrics merging into one index. Reduce the number of index 50%.
2020-02-16 09:03:30 +08:00
Trustin Lee 5fed153ee2
Add Armeria 0.98.0 to the supported version list (#4328)
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-02-15 12:27:03 +08:00
Jared Tan 7ff9a0cfd9
supprt gRPC alarm hook. (#4344)
* support gRPC hook.

* revert submodule commit.

* revert submodule commitid.

* finish logical.

* remove author anonatation.

* fix ci.

* revert submodule.

* update logical.

* update logical.

* update logical.

* fix

* fix

* fix empty settings.

* fix.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-12 16:20:20 +08:00
lxliuxuankb e5366c0921
Tag annotation supports returned expression (#4327)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-02-12 14:03:02 +08:00
aderm 8c96fd49dd Update elasticsearch-7.x agent support version. (#4340)
* test/plugin/scenarios/elasticsearch-7.x-scenario/support-version.list

* update elasticsearch-7.x agent support version.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-10 19:10:38 +08:00
吴晟 Wu Sheng 421ba88dbf
Provide the new version upgrade documentation/FAQ. (#4311)
* Provide the new version upgrade documentation/FAQ.

* Make the sentence more clear.
2020-02-04 17:40:12 +08:00
Daming 9005d1e82c
Provide AVRO plugin (#4276)
* Provide Avro plugin
* provide testcase for avro-plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-30 17:49:43 +08:00
吴晟 Wu Sheng 4072227e5b
Update backend-setup.md (#4271) 2020-01-19 17:43:12 +08:00
mrproliu 90f3c4de55 Sniffer processing profile task and report status and snapshot (#4220)
* sniffer processing profile task and report status and snapshot

* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`

* resolve names

* change profile to single one thread run.

* 1. change to the ArrayList, because known the max size
2. rename issue resolved

* add profiling status enum

* change sniffer use full name issue

* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task

* remove unnecessary getter

* add test assert error message

* adding `AgentServiceRule`

* revert original assert

* remove unnecessary getter

* resolve issues

* reduce findService invoke

* resolve style error

* recheck profiling when change first span operatin name

* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting

* remove article link

* add `ProfileTask#maxSamplingCount` check

* resolve conflict (Downstream -> Commands)

* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump

* remove unused return val

* remove some `@param` and `@return`

* add profile task check result data bean

* change profiler slot to `AtomicReferenceArray`

* resolved java doc error

* fix doc error, remove meaningless descriptions

* resolve missed profile receiver on oap starter

* resolve method invoke error

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-18 18:38:03 +08:00
kezhenxu94 019af3e345
Drop support of protocol v1 (sw3) (#4244)
* Drop support of protocol v1 (sw3)

* Remove versioning logic further

* Remove more version5-only classes

* Commit missing files

* Update submodule and address review comment

* Fix failed tests
2020-01-17 20:33:52 +08:00
kezhenxu94 094eeb870f
Add alarm metrics OP >= and <=, and refactor the operator logics (#4221)
* Add alarm metrics OP >= and <=, and refactor the operator logics

* Add unit test for OP

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-13 00:50:37 +08:00
Jared Tan 1310719645
add token authentication between agent and oap receiver. (#4197)
* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 23:08:39 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
dengliming 1881a040d4 Support plugin for mongodb v3.6.x (#4173)
* feat: add support plugin of mongodb v3.6.x

* update support-version.list

* update Supported-list.md

* update Supported-list.md

* Add witness class for v3.6

* update plugins-test.yaml

Co-authored-by: zhang-wei <pknfe@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
2020-01-07 14:25:19 +08:00
Kanro 4e03d5e6d7 Add kotlin coroutine plugin (#4165)
* Kotlin coroutine plugin write in Java

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-04 22:51:59 +08:00
aderm 6be84cfa78 update elasticsearch-7.x agent support version. (#4171)
* update elasticsearch-7.x agent support version.

* update supported-list.md doc & plugins-test.yaml
2020-01-04 19:56:35 +08:00
kezhenxu94 99100ed635
Split the `collect_http_params` as per plugins for better performance (#4170)
* Split the `collect_http_params` as per plugins for better performance

* Add missing change
2020-01-03 21:44:25 +08:00
geektcp 0f6b5c8def Fix dead link (#4167)
the quick start website cannot open
2020-01-03 16:11:02 +08:00
kezhenxu94 f52f7f466a
Support collecting http parameters (#4162)
Plugins of Tomcat, Spring MVC, Armeria are supported for now
2020-01-03 07:51:26 +08:00
web-xiaxia 5fba91901f add Dynamic Configuration Consul AclToken (#4161)
* Update ConsulConfigurationCenterSettings.java

add aclTokne

* Update ConsulConfigurationWatcherRegister.java

add aclToken

* Update application.yml

add aclToken

* Update ConsulConfigurationCenterSettings.java

add aclToken

* Update application.yml

add aclToken

* add aclToken

* Replace with util

* Replace with util

* add consul aclToken
2020-01-02 20:05:16 +08:00
mrproliu d38a8eaaa7 Provide thread monitor create task feature (#4145)
* provide thread monitor task service to create

* provide thread monitor task GraphQL implements

* 1. change thread monitor field names
2. fix `getThreadMonitorTaskList` query time bucket error

* provide config stream processor

* update submodule `query-protocol`

* resolve e2e-mysql error

* remove useless storage interface method.

* rename `ThreadMonitorTask` to `ProfileTask`

* change e2e-profile to the top level

* fix profile test analyze error

* remove StringUtil#isBlank.

* 1. remove create profile task duration unit
2. remove GraphQL getTask list duration field
3. add `profileTaskQueryMaxSize` in `storage` -> `elasticsearch(7)` configuration, default get 200 profile task

* provide e2e different storage tests

* 1. fix rat check
2. remove DurationUtils.java#toSecond
3. remove ProfileTaskQueryEs7DAO

* fix e2e code format error

* provide es6 and es7 storage e2e tests

* change e2e profile es module artifactId

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-31 20:49:45 +08:00
aderm f64f3c1433 [test/plugin] immigrate test activemq scenario. (#4151)
* [test/plugin] immigrate test activemq scenario.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2019-12-31 15:31:39 +08:00
kezhenxu94 51e74ca4c4
[Feature] Add tag annotation to allow tagging span with annotation (#4152)
* [Feature] Add tag annotation to allow tagging span with annotation

* Use repeatable annotation syntax and add docs and example codes

* Fix failed unit test
2019-12-31 12:11:06 +08:00
吴晟 Wu Sheng 1ee14fbbc8
Support customized instance name by a new UUID naming policy. (#4148) 2019-12-30 14:59:29 +08:00
Andy Ai 4aac3293ca [test/plugin] immigrate test play framework scenario (#4128)
* Add play-scenario

* Set play plugin as default

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
2019-12-26 22:42:56 +08:00
吴晟 Wu Sheng 25a677b9c7
Move Webflux plugin into default list. (#4129)
* Move Webflux plugin into default list.

* Fix a typo.
2019-12-25 22:08:14 +08:00
吴晟 Wu Sheng 3dde94ff66
Move lettuce into default plugin (#4127)
* Move lettuce into default plugin

* Fix maven pom
2019-12-25 18:53:49 +08:00
于玉桔 1e9f929f97 [test/plugin] immigrate test zookeeper scenario. (#4106)
* zookeeper test

* update expected data file

* fix review code

* fixed

* bugfix

* update version

* fix bug

* fix

Co-authored-by: zhang-wei <pknfe@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2019-12-25 17:25:09 +08:00
吴晟 Wu Sheng b570996006
Update JDK compiling level to 8, and remove zip dist. (#4119)
* Update JDK compiling level to 8, and remove zip dist.

* Exclude annotation and shade API.

* Fix more exclusion.

* 1. Update grava and gson 2. Fix compile issue. 3. Keep protoc and maven plugin in old version. Some incompatible of 3rd party grpc proto.

* Fix dependency conflicts, license and notice issue, and dependency check script issue

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2019-12-25 15:55:24 +08:00
zhang-wei ba192a9f13 [test/plugin] immigrate test servicecomb scenario (#4109)
* servicecomb scenario
2019-12-25 09:43:57 +08:00
kezhenxu94 bf6491f449
[Feature] Provide Java Agent for Apache HttpClient 3.x (#4111)
* Add Java Agent for Apache HttpClient 3.x

* Update Supported-list.md

* Update pom.xml
2019-12-24 23:20:44 +08:00
吴晟 Wu Sheng fa379d786c Update README.md (#4102) 2019-12-21 23:10:42 +08:00
cyejing bc77fda5c1 Enhance webflux plugin, related to Spring Gateway plugin too. (#3945)
* webflux-reactive

* add support version comment, modify gateway

* add test case, modify name

* add docs

* remove healthCheck
2019-12-17 08:58:57 +08:00
kezhenxu94 3f48790c36 [Feature] Add Java agent plugin for Jdk Threading classes (#4067)
* Add Java agent plugin for Jdk Threading

* Fix expected data according to changes in master

* Update agent.config

* Fix failed plugin test
2019-12-17 00:12:14 +08:00
吴晟 Wu Sheng 4e50132817
Remove the local/exit span register mechanism in Java agent scenario (#4059)
* Remove the local span and exit span register mechanism in Java agent scenario.

* Fix config, plugin, and document, as exit span is not required register anymore.

* Try to fix the test case expected data due to no register for exit span.

* Fix toolkit case.

* Remove exit span from plugin test framework document.

* Update protocol, FAQ, and CHANGELOG documents to highlight this change.

* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.

* Fix the snapshot and add inexistence operation name id.

* Fix several async cases.

* Fix undertow test case.

* Fix wrong error log name.

* Keep name consistent with 6.x concepts.

* Try to fix 3 async scenarios.

* Fix okhttp case.

* Fix Spring Async case.

* Fix another NPE

* Fix NPE when first span has no endpoint id when it is local or exit span.

* Still fill first operation name when no endpoint id.
2019-12-15 19:15:47 +08:00
Videl Smith daabc1f5b6 Documentation proofreading (#3999)
* Fixes: README.md

* Adds: new FAQ

See issues #3924, #3935, #3975

* Fixes: setup documentation

* Update README.md
2019-12-13 11:08:36 +08:00