Commit Graph

202 Commits

Author SHA1 Message Date
kezhenxu94 72f46f7499
Add missing `@Repeatable` to `@Tag` annotation (#4567)
* Add missing `@Repeatable` to `@Tag` annotation

* Remove some verified minor versions to speed up plugin tests
2020-03-25 09:58:16 +08:00
何延龙 954eede0ec
Add PHP module support to e2e test (#4547) 2020-03-23 23:16:28 +08:00
kezhenxu94 f51ba3a94d
[Chore] Fix license headers and rename E2E job name (#4551)
* Fix license headers and rename E2E job name

* Fix RAT plugin and enable both RAT and checkstyle plugin
2020-03-23 13:51:39 +08:00
mrproliu ace781ff49
Provide HTTP parameter in the profiling contet (#4546)
* Provide active HTTP parameter collection automatically in the profiling context

* Fix style check

* fix style error

* Add profile status, storage it into TracingContext. simplify to get profiling method

* Change the profile status field name in TracingContext

* Replace gone images

* resolve issues

* resole description issues

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-03-22 17:19:57 +08:00
mrproliu a9d34e3b61
Remove unnecessary parameter in the profile log query (#4549)
* Remove unnecessary parameter in the profile log query

* Replace gone images

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-03-22 12:49:43 +08:00
cngdkxw 2e9f001633
Add spring cloud feign 2.x plugin (#4526) 2020-03-20 20:28:30 +08:00
kezhenxu94 dcd66ee6e9
Rewrite the E2E test framework and tests (#4510)
Motivation:

According the feedback from the community, the e2e tests are hard for
them to understand and get them started, so this pull request aims to
rewrite the e2e tests with some improvements:

- debuggable locally, now you can run every individual case directly
inside the IDE, just like running a plain unit test.

- same functionalities of multiple counterparts share the same codes,
cases like different storages and cluster coordinators share the same
codes, easy to add more similar cases.

- the "end" now extends to the Doccker images, meaning that some bugs
like #4483 and #4471 can be more easily and early to be discovered.

- codes style and license headers are checked

Modifications:

- adopt jupeter (a.k.a junit 5), testcontainers as the new test
framework, all containers are orchestrated by docker-compose.yml,
well-known technology that developers should be familiar with.

- rewrite all test cases and verify them.

- reorganize the workflows in GitHub Actions.

Result:

- new e2e test should be easy to add.

- closes https://github.com/apache/skywalking/issues/4472
2020-03-16 23:45:14 +08:00
kezhenxu94 69e3d80e24
Add selector property in application.yml (#4514)
### Motivation

 Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.

 ### Modification

 Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.

 ### Result

 - We can switch a configuration by environment variables or system property

 - Closes #4511
2020-03-15 23:28:14 +08:00
Gao Hongtao 78d14ebbd6
Enable OAP gRPC SSL transportation (#4470)
* Enable OAP gRPC SSL transportation

Porting to OpenSSL to enable SSL transportation. The server private
key is in the format of PCKS#8, certificates is x509 though.

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add netty-tcnative-boringssl-static 2.0.26 licnese

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Update docs/en/setup/backend/grpc-ssl.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* Add SSL to grpc remote client and sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update documents

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix nits

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Fix test case

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add e2e-ssl and remove other test cases temporarily

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add TLS relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Decrease timeout

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Test config works

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* triger CI on oap-ssl branch

* Disable agent ca

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Switch to sharing server

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Add agent log to console

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Upload log files

* Change backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert "Change backend"

This reverts commit 7a085711e3653d3bf642aa84f73650e22595e383.

* Update backend

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Revert e2e

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Ignore TSL relevant files

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Polish codes

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Update certificates expires year to 2120

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-03-15 08:15:28 +08:00
cngdkxw d5efc97c80
RestTemplate async plugin ClassCastException (#4507)
* ClassCastException

* resttemplate 4.x scenario

* add support versions & fix expectedData format-

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-03-13 23:13:51 +08:00
Gao Hongtao fdb816c799
Update MiniKube for CI process (#4495)
* Update pre.sh

* add if func

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Reorder steps

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Upgrade minikube to 1.8.0

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-03-12 10:33:11 +08:00
yoje ced5cb2828
Add finagle plugin (#4441)
* Add finagle plugin (#4433)

* fix failed checks for pr(#4441)
* Add licenses for some files
* Update Supported doc
* Add finagle to test component-libraries.yml

* Add finagle automatic tests into CI

* Add comments

* update comments

* Remove CompatibilityChecker

* Move query-protocol submodule to the right version

* Remove `enhancedInstance.setSkyWalkingDynamicField` in `ClientTracingFilterInterceptor`.

* Update expectedData.yaml of finagle tests

* Use lazy ContextCarrier injection.

* Add comments for `ContextCarrierHelper`

* Fix ContextHolderFactory to avoid context pollution

Co-authored-by: huangyongjie <huangyongjie@tigerbrokers>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-03-11 13:04:58 +08:00
Daming da44d28547
Introduce Agent-Test-Tool suite (#4458)
* Introduce Agent-Test-Tool suite

* rename term
2020-03-08 09:15:41 +08:00
吴晟 Wu Sheng 9b8a8be27e
Update pom.xml (#4452)
Update the latest release commit to the e2e for release verification.
2020-03-06 22:37:08 +08:00
Jared Tan aa21765628
support http api for upstream trace. (#4399)
* support http api.

* e2e

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-03-06 21:07:39 +08:00
mrproliu 0df2d0a5e9
Provide profile exporter tool (#4416) 2020-03-04 20:28:14 +08:00
Jared Tan 85cc9c36ac
fix agent auth e2e not activated. (#4428)
* fix e2e miss.
2020-02-29 22:20:43 +08:00
mrproliu fa0b3df369
provide multiple time ranges to query profile analyze (#4417)
* profide multi time range to query profile analyze

* use method to avoid direct reference field
2020-02-26 10:51:38 +08:00
mrproliu 2ea341b890
Add missed profiled segment query (#4413)
* add profiled segment query

* fix query field error
2020-02-25 10:12:16 +08:00
zhang-wei 2ac2d2555a
[test/plugin] immigrate h2 & struts scenario (#4395)
* immigrate test code

* enhance mysql case healthCheck

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-23 20:34:44 +08:00
aderm 67b7f22a40
1.add es agent resthighlevelclient of ClusterClientInstrumentation.2.… (#4386)
* 1.add es agent resthighlevelclient of ClusterClientInstrumentation.2.fix agent test removeOnExit parse error.

* Update configuration.yml

* revert removeOnExit param.

Co-authored-by: Daming <zteny@foxmail.com>
2020-02-21 14:00:19 +08:00
mrproliu 0d4714564f
Merge branch 'master' into master 2020-02-19 18:18:31 +08:00
Daming 29da5738bc
Provide influxdb as a new storage plugin (#4239)
* provide influxdb as a new storage plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-19 18:00:57 +08:00
lsyf e57176d05e Fix ehcache:  add interceptor for Cache's private constructor and setName method 2020-02-18 22:33:33 +08:00
lsyf 74c1f7ef73 ehcache-2.x-scenario add EhcacheCloneInterceptor case 2020-02-18 13:07:16 +08:00
Daming ca6a5618c5
[test/plugin] remove the image on docker exit (#4374)
* remove image on docker exit

* polish
2020-02-17 17:10:37 +08:00
Trustin Lee 5fed153ee2
Add Armeria 0.98.0 to the supported version list (#4328)
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-02-15 12:27:03 +08:00
lxliuxuankb e5366c0921
Tag annotation supports returned expression (#4327)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-02-12 14:03:02 +08:00
kezhenxu94 5b255ba3e3
Set up more strict code styles and fix existing issues (#4337)
Motivation:

Review code styles with some bots automatically.

Modifications:

Set up ReviewDog in GitHub Action to review code style.

Add more check rules to checkstyle plugin.

Result:

Obvious code styles can be reviewed and commented automatically.
2020-02-11 17:10:49 +08:00
mrproliu fa526e5227
Provide profile analyze query (#4335)
* provide profile analyze query

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-11 10:11:08 +08:00
aderm 8c96fd49dd Update elasticsearch-7.x agent support version. (#4340)
* test/plugin/scenarios/elasticsearch-7.x-scenario/support-version.list

* update elasticsearch-7.x agent support version.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-10 19:10:38 +08:00
吴晟 Wu Sheng f29fc4bcb3
Set a shared submodule for e2e test project. (#4329) 2020-02-08 21:56:23 +08:00
mrproliu e44bc36281
provide profiled segment list query (#4322)
* provide profiled segment query

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: AirTrioa <44222967+AirTrioa@users.noreply.github.com>
2020-02-06 21:02:08 +08:00
John Niang 7cf79b1127
Move lombok dependency into root dependency manager for easier version management (#4309)
* Add lombok dependency into root pom.xml

* Update all depending lombok pom.xml(s)

* Change lombok version into latest(1.18.10)

* Revert lombok dependency config of test projects

* Remove redundant lombok dependency of submodule

* Move Lombok into dependencies node for applying globally
2020-02-03 10:44:50 +08:00
kezhenxu94 408a777d9d
Make traces verification loosely (#4306)
### Motivation:

Make E2E tests more stable.

### Modifications:

Verify the traces loosely as other tests, and remove the `verify` method which verifies traces strictly, as we have preparation that also generates traces.

### Result:

E2E tests are more stable.
2020-02-02 14:11:47 +08:00
jialong 5c6914fc9e
Make MVC plugin support inherited annotations from interface (#4278) 2020-02-01 16:18:22 +08:00
Kanro 97950807fa
Enhance gRPC plugin (#4177)
* Enhance gRPC plugin

Co-authored-by: Kanro <higan@live.cn>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: GuoDuanLZ <739537111@qq.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-30 20:12:48 +08:00
Daming 9005d1e82c
Provide AVRO plugin (#4276)
* Provide Avro plugin
* provide testcase for avro-plugin

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-30 17:49:43 +08:00
John Niang 0e105a49bf
Upgrade remainder java compiler versions to 1.8 (#4286)
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-29 14:19:22 +08:00
mrproliu 8e4c1e95d6 Merge e2e profile test project (#4268)
* merge e2e profile test project, Use `env` to distinguish different storage types

* remove maven child module

* change `env` -> `storage`

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-20 19:34:53 +08:00
mrproliu 90f3c4de55 Sniffer processing profile task and report status and snapshot (#4220)
* sniffer processing profile task and report status and snapshot

* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`

* resolve names

* change profile to single one thread run.

* 1. change to the ArrayList, because known the max size
2. rename issue resolved

* add profiling status enum

* change sniffer use full name issue

* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task

* remove unnecessary getter

* add test assert error message

* adding `AgentServiceRule`

* revert original assert

* remove unnecessary getter

* resolve issues

* reduce findService invoke

* resolve style error

* recheck profiling when change first span operatin name

* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting

* remove article link

* add `ProfileTask#maxSamplingCount` check

* resolve conflict (Downstream -> Commands)

* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump

* remove unused return val

* remove some `@param` and `@return`

* add profile task check result data bean

* change profiler slot to `AtomicReferenceArray`

* resolved java doc error

* fix doc error, remove meaningless descriptions

* resolve missed profile receiver on oap starter

* resolve method invoke error

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-18 18:38:03 +08:00
kezhenxu94 019af3e345
Drop support of protocol v1 (sw3) (#4244)
* Drop support of protocol v1 (sw3)

* Remove versioning logic further

* Remove more version5-only classes

* Commit missing files

* Update submodule and address review comment

* Fix failed tests
2020-01-17 20:33:52 +08:00
kezhenxu94 8e5e35f096
Fix failed E2E MySQL case (#4246) 2020-01-16 16:34:27 +08:00
Jared Tan 1310719645
add token authentication between agent and oap receiver. (#4197)
* add token authentication between agent and oap receiver.

* revert ui.

* fix code style.

* update docs.

* Update docs/en/setup/backend/backend-token-auth.md

Co-Authored-By: kezhenxu94 <kezhenxu94@apache.org>

* fix hits.

* filter

* revert yml.

* update logical.

* fix codestyle check.

* fix ci and add e2e test case.

* revert mistake commit.

* fix ci

* remove duplicate maven dependency.

* revert pom

* revert unchanged.

* update javadocs.

* update logical.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 23:08:39 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
dengliming 1881a040d4 Support plugin for mongodb v3.6.x (#4173)
* feat: add support plugin of mongodb v3.6.x

* update support-version.list

* update Supported-list.md

* update Supported-list.md

* Add witness class for v3.6

* update plugins-test.yaml

Co-authored-by: zhang-wei <pknfe@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
2020-01-07 14:25:19 +08:00
Kanro 4e03d5e6d7 Add kotlin coroutine plugin (#4165)
* Kotlin coroutine plugin write in Java

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-04 22:51:59 +08:00
aderm 6be84cfa78 update elasticsearch-7.x agent support version. (#4171)
* update elasticsearch-7.x agent support version.

* update supported-list.md doc & plugins-test.yaml
2020-01-04 19:56:35 +08:00
kezhenxu94 99100ed635
Split the `collect_http_params` as per plugins for better performance (#4170)
* Split the `collect_http_params` as per plugins for better performance

* Add missing change
2020-01-03 21:44:25 +08:00