* support multiple segments analyze
* Support multiple ways to analyze segment data
* Change to support single or list of the segment data, add the document
* fix document and change the lua agent commit id
Co-authored-by: Mrproliu <mrproliu@lagou.com>
* Add Mariadb JDBC plugin and provide tests
* Refactor component name
* Reformat plugin description and remove healthcheck span from the test
Co-authored-by: Efe Kaptan <Efe.Kaptan@webtrekk.com>
Co-authored-by: zhang-wei <pknfe@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* vertx web plugin re-impl attempt
* added license
* update test for new plugin structure
* got rid of registryItems
* remove unnecessary dependencies
* re-impl eventbus scenario
* more precise naming
* fix expected data
* removed unsupported versions and removed vertx caching (avoids permission error)
* rewrote to remove copied code
* Update plugins-test.3.yaml
* fixes issue of .vertx files remaining on v3.0.0
* only present in v3.0.0
* consistency
* add more supported versions
* track status code
* spelling fix
* track status code
* more correct naming
* reverts 213f5a69 (status code isn't sent at this point)
* made HttpServerResponseImplEndInstrumentation more strict based on version
added status code tracking
fixes issue not finishing async spans
* works 3.0.0,3.7.0
* increase compatibility
* 3.9.0 default
* versioned witness classes (advice from wu-sheng)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: zhang-wei <pknfe@outlook.com>
1. Re-balance the workload of plugintests
2. Restore testcases of MySQL, remove on #4641
3. Support user to runs plugin-test on debug mode. It will not remove the log files and actualData.yaml
4. remove paramater 'parallel_run_size' and disable parallel test.
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol.
2. New agent header protocol.
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
* Finish step one, source and entity changed.
* Step 2, finish the basic and core DAO changes.
* Step 3. Change all source codes of backend, and make project compiling successfully.
* Make startup successful and persistence works.
* Fix dead codes in the MySQLTableInstaller.
* Fix mischange.
* Fix MySQL storage bootstrap error.
* Rollback dependency change.
* Simply enable tailing logs from container no mater it's in CI or not
* Fix segment table issue.
* Fix MySQL Instanller.
* Support Influxdb implementation of new endpoint traffic.
* Set and modify timeout to 90mins
* Remove unnecessary entity in the EndpointTraffic.
* Fix wrong entity builder and make name more clear.
* Fix 2 equal methods bug.
* Fix InfluxDB identifier(name) conflict issue.
* Fix endpoint topology query, endpoint id should be changed as a string.
* Fix a missing change.
* Fix a wrong commit.
* Column name can't be changed, due to it affect the hybrid metadb(h2/mysql) query, change it explicitly with double quotation marks.
* Fix formats.
* Remove endpoint/operation register from the agent. and change the plugin tests(without local tests). operationId relation codes have been removed from agent too.
* Try to fix the hytrix case.
* Fix 3 cases.
* Try to fix cases.
* Fix 2.
* Fix one.
* Fix the serviceId type mis-change.
* Fix int query
* Fix hystrix case.
* Fix a case.
* Fix EndpointCallRelationDispatcher set the source/dest endpoint without id encoding.
* Update documentation and add 2 test cases.
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* fix#4539, add PHP to the query protocol
* Add PHP module support to e2e test
* Add PHP e2e test into the CI control file
* Remove networks aliases
* Add PHP module support to e2e test
* Add e2e.php.yaml
* Locked version
* Add 200000 microsecond sleep in E2E-PHP
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
* up ui
* up ui
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* const string witnesses
add some unit test
recommended to change ; to ,
* const string witnesses
add some unit test
recommended to change ; to ,
* up ES6.X test-plugin
* up es6.x test-plugin
* up es6.x test-plugin
* up es6.x test-plugin
* add DeleteIndexRequest
* up es6.x test-plugin
* remove TransportCaseController.java
* fix TransportActionNodeProxyExecuteMethodsInterceptorTest
Co-authored-by: yi.liang <yi.liang@zhangmen.cn>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: 梁懿 <liangyi@xforceplus.com>
Co-authored-by: aderm <394102339@qq.com>
* Provide active HTTP parameter collection automatically in the profiling context
* Fix style check
* fix style error
* Add profile status, storage it into TracingContext. simplify to get profiling method
* Change the profile status field name in TracingContext
* Replace gone images
* resolve issues
* resole description issues
Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Motivation:
According the feedback from the community, the e2e tests are hard for
them to understand and get them started, so this pull request aims to
rewrite the e2e tests with some improvements:
- debuggable locally, now you can run every individual case directly
inside the IDE, just like running a plain unit test.
- same functionalities of multiple counterparts share the same codes,
cases like different storages and cluster coordinators share the same
codes, easy to add more similar cases.
- the "end" now extends to the Doccker images, meaning that some bugs
like #4483 and #4471 can be more easily and early to be discovered.
- codes style and license headers are checked
Modifications:
- adopt jupeter (a.k.a junit 5), testcontainers as the new test
framework, all containers are orchestrated by docker-compose.yml,
well-known technology that developers should be familiar with.
- rewrite all test cases and verify them.
- reorganize the workflows in GitHub Actions.
Result:
- new e2e test should be easy to add.
- closes https://github.com/apache/skywalking/issues/4472
### Motivation
Ease the efforts of modifying configurations in automatic workflow and scripts, like Docker and helm chart.
### Modification
Add a `selector` property into `application.yml` to enable switching configuration simply by environment variables or system properties, and filter the unselected options, leaving only selected one.
### Result
- We can switch a configuration by environment variables or system property
- Closes#4511
Motivation:
Review code styles with some bots automatically.
Modifications:
Set up ReviewDog in GitHub Action to review code style.
Add more check rules to checkstyle plugin.
Result:
Obvious code styles can be reviewed and commented automatically.
* Add lombok dependency into root pom.xml
* Update all depending lombok pom.xml(s)
* Change lombok version into latest(1.18.10)
* Revert lombok dependency config of test projects
* Remove redundant lombok dependency of submodule
* Move Lombok into dependencies node for applying globally
### Motivation:
Make E2E tests more stable.
### Modifications:
Verify the traces loosely as other tests, and remove the `verify` method which verifies traces strictly, as we have preparation that also generates traces.
### Result:
E2E tests are more stable.
* sniffer processing profile task and report status and snapshot
* resolve testServiceDependencies test case error, use same register with `TraceSegmentServiceClient`
* resolve names
* change profile to single one thread run.
* 1. change to the ArrayList, because known the max size
2. rename issue resolved
* add profiling status enum
* change sniffer use full name issue
* 1. remove `prepareProfiling` method, build profiling status when construct `TracingContext`
2. add `TracingThreadListenerManager`, notify when tracing main thread finish
3. change ProfileThread start when process new profile task
* remove unnecessary getter
* add test assert error message
* adding `AgentServiceRule`
* revert original assert
* remove unnecessary getter
* resolve issues
* reduce findService invoke
* resolve style error
* recheck profiling when change first span operatin name
* resolve issues
1. remove `ContextManager#profilingRecheck`, only check on `TracingContext`
2. rename comments
3. resolve volatile array setting
* remove article link
* add `ProfileTask#maxSamplingCount` check
* resolve conflict (Downstream -> Commands)
* 1. change profilingSegmentSlots init on construct
2. if is profiling, recheck dont need to stop
3. total profiling count increment on first dump
* remove unused return val
* remove some `@param` and `@return`
* add profile task check result data bean
* change profiler slot to `AtomicReferenceArray`
* resolved java doc error
* fix doc error, remove meaningless descriptions
* resolve missed profile receiver on oap starter
* resolve method invoke error
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* Support new percentile func with new alarm and exporter for this new func.
* Fix e2e and OAL script
* Fix wrong column.
* Fix percentile bug and oal engine bug.
* Update query protocol and add percentile test case
* Support new query
* Adopt GraphQL requirement
* Fix wrong type cast.
* Fix query in H2 and ES.
* Fix docs and comments.
* Fix an e2e compile issue
* Fix javadoc issue and e2e test issue.
* Change CPM to Apdex in TTL test.
* Fix OAL for TTL e2e
* Add metrics query for service percentile.
* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap
* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.
* Fix graphql statement error in e2e.
* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.
* Fix test case format
* Remove generated code test.
* Fix failed e2e test
* Use avg resp time to apdex in the TTL test.
* ADD multiple linear metrics check for endpoint in e2e cluster.
* Support `-` to represent no threshold and doc of alarm about this.
* Move break to right place.
* Fix wrong break(s)
* Fix break and add a test case for multiple values alarm.
* Fix format.
* Add more doc for this new feature and GraphQL query protocol.
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* Provide profile task downstream to sniffer
* fix agent unit testcase issue
* add profile switch config on sniffer
* fix es error
* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error
* change timeFromStartMills use `<` to compare
* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package
* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task
* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only
* remove navigate time judge, fix comment wrong meaning
* [Feature] Add tag annotation to allow tagging span with annotation
* Use repeatable annotation syntax and add docs and example codes
* Fix failed unit test
* Add play-scenario
* Set play plugin as default
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
* lettuce test
* fix
* update version
* a new one group
* revert
* update expect data
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
* shardingsphere 3.x test
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* shardingsphere 3.x test refine
* Remove the local/exit span register mechanism
* for comments
* for comments
* Remove the local span and exit span register mechanism in Java agent scenario.
* Fix config, plugin, and document, as exit span is not required register anymore.
* Try to fix the test case expected data due to no register for exit span.
* Fix toolkit case.
* Remove exit span from plugin test framework document.
* Update protocol, FAQ, and CHANGELOG documents to highlight this change.
* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.
* Fix the snapshot and add inexistence operation name id.
* Fix several async cases.
* Fix undertow test case.
* Fix wrong error log name.
* Keep name consistent with 6.x concepts.
* Try to fix 3 async scenarios.
* Fix okhttp case.
* Fix Spring Async case.
* Fix another NPE
* Fix NPE when first span has no endpoint id when it is local or exit span.
* Still fill first operation name when no endpoint id.
* Add apdex function to OAL
* Add empty line
* Setup config watcher
* Add identifier type to function parameter
* Add config test
* Update score algorithm
* Replace responseCode with status
* Add comments about apdex score algorithm
* Add docs
* Add e2e test case
* Update test case
* Fix disptch class generating error
* Update value name of apdex metric
* Tuning threshold
* Fix single tolerated point bug
* [Feature] Officially support ElasticSearch 7 as backend storage
* Extract bootstrap module to be shared by multiple starters
* Add missing configuration file and distinguish different version in test
* Update known-oap-backend-dependencies-es7.txt
Update the dependency lib
* Fix missing adaptation to ElasticSearch 7
* Rename ElasticSearch 7 specific class with Es7 infix
* Fix miss-adapted ElasticSearch DAO API
* Add ES7 specific configuration and polish documentations
* Polish documentations
* Polish documentations
* Make istio/envoy ci test (#3697)
* for #3669
* Add Apache 2.0 header
* Fixed some irrational
* [WIP]Scripts which deploy Istio + OAP + ES to a standard k8s cluster (#3718)
* for #3669
* Add Apache 2.0 header
* Fixed some irrational
* create istio-deploy.sh
* modify notes
* istio version and namespace to parameterize
* Keep template files and tested successfully
* add sw istio adapter
* add a new key for isito adapter
* notes.txt
* add mixer template
* add sw deploy script
* bookinfo and cleanup scripts
* add timeout and set -e
* modify cleanup and request timeout of demo
* End blank line
* Add minikube to CI (#3836)
* Test
* Add github action files
* Add tunnel to provie loadbalancer
* upload logs
* Fixed some nits
* Intercept feign RequestTemplate resolve args method ,get the url without parameters
* fix ci warnings
* optimize import
* fix: miss path in feign target url like "http://localhost:8080/feign-scenario"
fix feign-scenario test use pathVariable
* Add testcase to DefaultHttpClientInterceptorTest
* Clean ThreadLocal
* User one ThreadLocal replace two. User try/final remove ThreadLocal .
* Code Optimize
* add SupplierWrapper to support java 1.8+
Signed-off-by: shixiang <xiang.shi001@bkjk.com>
* add SupplierWrapper to support java 1.8+
Signed-off-by: shixiang <xiang.shi001@bkjk.com>
* Update SupplierWrapper.java
add new line
* add supplier
* add supplier test
* add test
* add test yaml
* Update Application-toolkit-trace-cross-thread.md
Add how to use SupplierWrapper in Java 1.8+
* Not support copy all optional plugins
* fix
* Update test/plugin/run.sh
to correct the mistakes of grammar
Co-Authored-By: kezhenxu94 <kezhenxu94@163.com>
* fix deficient code, "/socket.io/" http segment declare
* add agent test and fix expectedData.yaml file
* add listener on tomcat start, work for reduce socket io server start time
change "/socket.io/" operation "http.method", cause get or post will invoke both
* resolve style check error
* change socket io server and client start to health check servlet
* if startClientAndWaitConnect invoke on multi-thread, it will only check client ref, no check connect state, fix it.
* feat: add support of mongodb v3.7.x or higher
* feat: remove mapping, using EnhancedInstance to mark remotePeer
* feat: add support of v3.6.x
* fix: disable support will 3.6.x, which will throw ClassCircularityError exception
* style: clean code
* fix support of 3.8.x~3.11.1, update test case and doc
* update stage name
* rename class
* rename stage
* add plugin netty-socketio https://github.com/mrniko/netty-socketio
* delete create exit span interceptor.
* delete create exit span interceptor testcase.
* delete create exit span interceptor testcase.
* add constructor test unit
* change bind on random port in test case
* reverse upstream master whitespace
* resolve build_id param
* delete health check segment test case
* add ContextManager.isActive check before use activeSpan invoke
* remove unnecessary code and blank line
* immigrate canal scenario
* add test cases report
* commit
* exclusion spring
* add canal stage
* del org.apache.logging.log4j dep
* del -Dskywalking.agent.service_name
* [CI] Make E2E tests run in multiple JDK versions
* Polish
* Use new e2e-container version number
* Install socat in reboot case
* Fix reboot case
* Polish
* Reorganize jobs
* immigrate spring async scenario
* fix rat
* Fix async not in effect
* immigrate spring scenario
* immigrate spring scenario
* fix
* immigrate spring scenario
* rollback
* fix log4j2.xml
* support docker-compose
* sync
* support docker-compose
* force rm container
* solrj-7.x-scenario
* fix license header
* fix actualData.yml
* verify Jenkins run on distribution
* test Jenkinsfile v2
* test Jenkinsfile v3, God bless
* test V4, God Bless
* test v5
* rollback
* update opertaion of post
The current latency of the test service is too slow, causing it's not counted and
the query result is 0, this patch increases the latency intentionally by sleeping
1 second
* Override ES settings in FAQ
* Resend traffic data when invalid
* Remove unsupported ES setting
* Use seperated buffer directory and remove unnecessary web app 2
e2e test setup. cc @hanahmily @peng-yongsheng @IanCao @zhaoyuguang @JaredTan95
# e2e test case
One single standalone Spring service, accessing H2 database. This service has been installed the agent.
OAP runs in single mode with H2 storage too.
The e2e test will verify the results through GraphQL query
1. Traces exist.
2. Service/Service Instance/Endpoint exist and have expected metrics.
3. Topology exists and match expected.
mrproliu
Stalary
Patrick Jiang(白泽)
Brandon Fergerson
kezhenxu94
caoyixiong
gongdewei
Gao Hongtao
dagmom
于玉桔
zhang-wei
吴晟 Wu Sheng
xiaoyu
Xin,Zhang
denis
何延龙
Daming
zifeihan
lipenghui
Efe Kaptan
Brandon Fergerson
Jared Tan
yi.liang
cngdkxw
yoje
aderm
lsyf
Trustin Lee
lxliuxuankb
John Niang
jialong
Kanro
dengliming
Zhang Yonglun
Andy Ai
cyejing
innerpeacez
Alan Lau
qiyang
viswaramamoorthy
sxzaihua
aderm
panjuan
scolia
xiaoy00
Kdump
kezhenxu94
ascrutae