Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol.
2. New agent header protocol.
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
* up ui
* up ui
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* fix elasticsearch-5.x-plugin on es6.x TransportClient error, Found class org.elasticsearch.common.transport.TransportAddress, but interface was expected
up elasticsearch-6.x-plugin support TransportClient
* const string witnesses
add some unit test
recommended to change ; to ,
* const string witnesses
add some unit test
recommended to change ; to ,
* up ES6.X test-plugin
* up es6.x test-plugin
* up es6.x test-plugin
* up es6.x test-plugin
* add DeleteIndexRequest
* up es6.x test-plugin
* remove TransportCaseController.java
* fix TransportActionNodeProxyExecuteMethodsInterceptorTest
Co-authored-by: yi.liang <yi.liang@zhangmen.cn>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: 梁懿 <liangyi@xforceplus.com>
Co-authored-by: aderm <394102339@qq.com>
* Provide active HTTP parameter collection automatically in the profiling context
* Fix style check
* fix style error
* Add profile status, storage it into TracingContext. simplify to get profiling method
* Change the profile status field name in TracingContext
* Replace gone images
* resolve issues
* resole description issues
Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Motivation:
Support Armeria 0.98.0 in its plugin.
Modifications:
Add an intercept point that is refactored in Armeria 0.98.
Result:
Now Armeria plugin supports 0.98.0, and 0.98.1
* extract duplicated method
* change while loop to if since once rs.next() return false, it will never return true for the following calls
* simplify if() return true; return false pattern
* fix compilation
Signed-off-by: liqiang.clq <liqiang.clq@alibaba-inc.com>
Motivation:
Review code styles with some bots automatically.
Modifications:
Set up ReviewDog in GitHub Action to review code style.
Add more check rules to checkstyle plugin.
Result:
Obvious code styles can be reviewed and commented automatically.
### Motivation:
Reduce footprint when tagging spans with the deprecated API: `AbstractSpan#tag(String, String)`.
### Modifications:
- Adopt [prototype pattern](https://en.wikipedia.org/wiki/Prototype_pattern) to create `Tag`, prevent from creating too many `StringTag` instances and `GC`ed.
- Replace `AbstractSpan#tag(String, String)` with `AbstractSpan#tag(AbstractTag, String)`.
### Result:
- Footprint is reduced.
* Add play-scenario
* Set play plugin as default
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Weiyi Liu <liuweiyi@cmss.chinamobile.com>
* Remove the local span and exit span register mechanism in Java agent scenario.
* Fix config, plugin, and document, as exit span is not required register anymore.
* Try to fix the test case expected data due to no register for exit span.
* Fix toolkit case.
* Remove exit span from plugin test framework document.
* Update protocol, FAQ, and CHANGELOG documents to highlight this change.
* Fix NPE when no entry span in the whole segment. Set endpoint to null in segment entity.
* Fix the snapshot and add inexistence operation name id.
* Fix several async cases.
* Fix undertow test case.
* Fix wrong error log name.
* Keep name consistent with 6.x concepts.
* Try to fix 3 async scenarios.
* Fix okhttp case.
* Fix Spring Async case.
* Fix another NPE
* Fix NPE when first span has no endpoint id when it is local or exit span.
* Still fill first operation name when no endpoint id.
* Intercept feign RequestTemplate resolve args method ,get the url without parameters
* fix ci warnings
* optimize import
* fix: miss path in feign target url like "http://localhost:8080/feign-scenario"
fix feign-scenario test use pathVariable
* Add testcase to DefaultHttpClientInterceptorTest
* Clean ThreadLocal
* User one ThreadLocal replace two. User try/final remove ThreadLocal .
* Code Optimize
* feat: add support of mongodb v3.7.x or higher
* feat: remove mapping, using EnhancedInstance to mark remotePeer
* feat: add support of v3.6.x
* fix: disable support will 3.6.x, which will throw ClassCircularityError exception
* style: clean code
* fix support of 3.8.x~3.11.1, update test case and doc
* update stage name
* rename class
* rename stage
* add plugin netty-socketio https://github.com/mrniko/netty-socketio
* delete create exit span interceptor.
* delete create exit span interceptor testcase.
* delete create exit span interceptor testcase.
* add constructor test unit
* change bind on random port in test case
* reverse upstream master whitespace
* resolve build_id param
* delete health check segment test case
* add ContextManager.isActive check before use activeSpan invoke
* remove unnecessary code and blank line
* Add pulsar apm plugin.
* Fix check style
* Fix pulsar consumer component define.
* Add pulsar to component-libraries.yml
* Fix error interceptor class.
* Add pulsar to agent support list.
* Add Pulsar to ComponentsDefine and component-libraries.yml
* Move create entry span log of consumer from after method to before method
* Fix send callback issue when exception cause.
* Fix test issues
* Move pulsar plugin to optional plugins
* Add none messages tests for interceptor of producer and consumer.
* Remove unused comments.
* Move pulsar plugin back to the apm-sdk-plugin
* Fix comments
* remove set startTime for entry span(default is set by System.currentTimeMillis)
* Fix comments
* Add Light4J 1.6.x plug-in
* Add unit test for Light4J plug-in
* Add missing id to server-starter
* Remove mentions of version 1.6.x
Some local testing showed that the plugin successfully generated traces for a light4j project built with light-codegen v2.x. Therefore, there is no need for a separate plugin for v2.x and no distinction needs to be made with light4j v1.6.x.
* Update supported light4j versions
* Use ContextSnapshot to maintain trace
In a typical Light4j project, the ExceptionHandler#handleRequest is the asynchronous execution point. The flow is ExceptionHandler#handleRequest -> HttpServerExchange#dispatch -> ... -> ExceptionHandler#handleRequest. This means that handleRequest is being intercepted twice and each time a LocalSpan is created. Without handling thread propagation, the trace segment gets split up.
By saving a ContextSnapshot into the enhanced object (ExceptionHandler), we can determine if it is necessary to continue the segment.
* Update logic for span creation in trace
By default, the Light4J plugin now overrides the Undertow plugin's entry span with its own. This new entry span is created just before execution of the ExceptionHandler#handleRequest in the I/O thread. The entry span is generated here because it is considered the starting point of the Light4J handler chain.
There is now also the option to enable detailed span creation when tracing requests through Light4J. These local spans mark which middleware and business handlers are involved with handling the HTTP request to a Light4J service.
* Update skywalking version
* Update light4j version
* Fix LightInstrumentation javadoc
* Remove import violating checkstyle
This import was added from the link reference in the javadoc. The javadoc has now been changed to just reference the method name.
* Move light4j agent config property
* Sync UI
* spring-cloud-gateway traceid does not transmit #3411
skywalking-version:6.5.0
spring-gateway-version:2.1.2
Error Description:
Customize the filter, traceid does not transmit.
"ServerWebExchangeDecorator" is a transport chain,His structure should be like this：
serverWebExchangeDecorator
-----(ServerWebExchangeDecorator)delegate
------(ServerWebExchangeDecorator)delegate
------.....
-----(DefaultServerWebExchange)delegate
In the current source code, there is no deep search, but only the next level. When there are multiple custom filters, you get an error.
Repair method:
Look for "delegate" of "ServerWebExchangeDecorator" recursively until "DefaultServerWebExchange"
* spring-boot-webflux traceid does not transmit #3411
spring-boot-starter-webflux-version:2.1.6
Error Description:
Customize the filter, traceid does not transmit.
"ServerWebExchangeDecorator" is a transport chain,His structure should be like this：
serverWebExchangeDecorator
-----(ServerWebExchangeDecorator)delegate
------(ServerWebExchangeDecorator)delegate
------.....
-----(DefaultServerWebExchange)delegate
In the current source code, there is no deep search, but only the next level. When there are multiple custom filters, you get an error.
Repair method:
Look for "delegate" of "ServerWebExchangeDecorator" recursively until "DefaultServerWebExchange"
* checkStyle
* ADD: add operationName length threshold
* MOD:move operationName threshold to Config, simplified code
* MOD:update agent set-up document, clean code
* MOD:add agent.operation_name_threshold conf prefix
* MOD: Modify compatibility of kafka plugin and expand operationName
* MOD: Delete localSpan layer, modify consumer operationName, delete producer key
* MOD: Modify operationName, reduce String operation
* FIX: Fix callback break, the new version of kafka callback is intercepted twice and needs to be judged
* FIX: Fix different problems with KafkaProducer constructor in old and new versions
* MOD: separate code
* Support op group name in agent config core.
* Provide core API for operation name grouping, rest template plugin update, and doc update.
* Fix doc
* Fix test case.
* Fix missing changes.
* add shuyun to powered-by.md (#3341)
* add shuyun to powered-by.md
* update powered-by.md
* Remove static method.
* POC: Intercept successfully.
* Refactor codes
* Build the new core
* Setup the new core.
* Push the javassist version to 3.23.2-GA, as it is the last version supports JDK 1.6 compile, according to https://github.com/jboss-javassist/javassist/blob/rel_3_23_2_ga/pom.xml#L149-L160
* Finish the all codes.
* Document updated for bootstrap instrumentation.
* Add a notice to document.
* Remove unexpected file.
* Set javassist version right. Add judge in core interceptors
* Fixe no log bridge implementation
* servlet 2.5
* method exist
* update MethodUtil
* Update MethodUtil.java
Add a comment to the new core util method. @candyleer I am concerning this method will be used widely even unnecessary.
* Revert SolrJ plugin.
* Remove an unexpected test case.
* Revert "Remove an unexpected test case."
This reverts commit 359990e18b0c0ae1a6e1005ff985bf237ec05619.
* Clear context.
* Fix a leak.
* new branch for solrj-plugin
* rollback and add module solrj-plugin
* preparing to pr
* to resolve reviewers' suggestions
* remove unused code
* remove unused code
* remove unused comments
* To avoid NPE
* fix typo
* Change to JRE6
* fix pom.xml merge incorrectly. and java.net.URL instead of Regex
* Using RuntimeContext instead of ThreadLocal
* To reduce unnecessary tags
* add test cases
* fix validation fail
* add solrj-plugin into component-libraries & Supported-list
* to trace all patch to avoid recheck status of span
* remove unnecessary properites
* remove unnecessary tags
* Add the config to document of setup
* merge
* merge
* Provide plugin for ShardingSphere (#2699)
* Provide plugin for ShardingSphere (#2699)
* Provide plugin for ShardingSphere (#2699)
* adjust package name (#2699)
* fix mvnw (#2699)
* revert files access permission from 644 to 755
* change the access permission from 644 to 755
* change files access permission from 644 to 755
* revert all no changes files
- found the unnecessary else block and remove it
- add test cover when AMQP.BasicProperties had null headers or headers Map had null value keys
issue #2450
2. Fixed the warning of duplicate declaration of plugin issue when running the mvn package command.
3. Fixed the warning of duplicate declaration of mysql connector jar when running the mvn package command, because of provided scope include test scope.
4. Fixed the warning of expression ${artifactId} is deprecated issue when running the mvn package command.
5. Fixed the warning of expression ${version} is deprecated issue when running the mvn package command.
* fix issue #2258
* fix#2097#1947
* remove Description and date tag
* split the diferent version to different plugins
* resolve conflict
* record all ips on multiple host mode of mysqlconnector-5x
* fix jdk 6.x
* optimise the code
* optimise code
* modify class name
* fix#2097#1947
* i do not think it is a good idea to move the Instrumentation to mysql-common,the better way is extract a public method to return the InteceptorPionts in mysql-common
* fix#2097#1947
* remove witnessClasses of subclasses
* Fixed an error that took a non-protocol uri string as a java.net.URL construct parameter.
* Fixed an error that took a non-protocol uri string as a java.net.URL construct parameter.
* Fixed checkstyle error.
* Update new protocol
* Support new protocol at agent side.
* Fix test case.
* Make backend supports new trace protocol
* Fix tests.
* Make all new services available in gRPC.
* Fix bugs.
* Finish new document for v2 trace protocol.
* Fix wrong version requirement of zk
* Fix trace query bug.
* Find a client side dictionary bug.
* Fix register bugs.
* Fix break doc links.
* Add some supports to sw6 header. Break many test cases because I turn sw3 default off. For sure, you could open in agent.conf. FYI @peng-yongsheng @ascrutae
* Fix CI and make user cases still work under v1 header.
* Support BASE64 in v2 header and make entryOperationName and parentOperationName optional in ContextCarrier or ContextSnapshot
* Fix CI.
* @peng-yongsheng This is backend related, please pay attention.
* Fix the carrier item order is incorrect (#1903)
* Lock oracle and resin plugins version.
* Move resin and oracle plugin to openskywalking
* Restore submodule active.
* Add another submodule active command.
* Remove the wrong test cases in release and fix some doc issues.
* [maven-release-plugin] prepare release v6.0.0-alpha
* [maven-release-plugin] prepare for next development iteration
* Fix some doc
* H2 storage code skeleton.
* Add batch dao for H2.
* Fix wrong refactor.
* Add codes of other dao. No metric dao, wait for @peng-yongsheng new PR.
* Change license of distribution
* Add topology query in H2.
* Add more daos.
* Add some daos.
* Commit all H2 daos. Wait @peng-yongsheng to submit new IAggregationQueryDAO interface.
* Fix some CIs
* Fix wrong refactor, which causes h2 plugin failure.
* Fix all codes of H2 storage. :) and CI should pass. No test.
* Fix CI and typo
* Fix connection closed.
* Fix register bug
* no message
* Fixed service inventory register bug.
* Fix a compile issue.
* Sync change
* Reset es as default storage.
* Fix rat
* Fix H2 compile issue.
* Fix bugs for H2 storage.
* Fix bug in mesh.
* 1. Fix Thermodynamic query bug
2. Fix service inventory query bug
3. Fix missing traceid in trace query bug
* Fix query bug.
* Submit reverse caused by prev PR.
* Fix duplicate spans.
* Use H2 as default storage implementor.
* Fix bug in topology query.
* Fix CI
* Fix rat
* Fix two query in H2 storage.
* Fix H2 query
* Make release running in H2 mode and develop mode running is ES mode in default
Using Mongo 2.14.3, we found that :" java. lang. ClassNotFoundException: Can't find MongoDBCollection Method Interceptor", must use "org.apache.skywalking.apm.plugin.mongodb.v2.MongoDBCollectionMethodInterceptor" to solve this problem.
* update 5.x grpc version form 1.10 to 1.14 and Solve the resulting guava and log4j version conflicts
* Specify guava version 20.0
* update parrent pom guava version
* [Agent] Provide plugin for Elasticsearch 5.x
Add [Agent] Elasticsearch transport client plugin
fix component name
add elasticsearch component
add elasticSearch component into component-libraries.yml
* update support list
* update PlainListenableActionFutureInterceptor.java
* fix set span attributes
* clear runtimeContext
* remove extra plugins
* Moves to Maven wrapper to stabilize the version in use
This will ensure those building will have similar experience due to
having a fixed maven version.
* Updates most plugin versions and removes redundant configuration
* Sets up build enforcement for JDK 1.8 until further work occurs
* #1447
Fixed the after method interceptor for the apache http client NPE error when the response is null cause by the doExecute method throw a exception.
* #1447
Fixed the after method interceptor for the apache http client NPE error when the response is null cause by the doExecute method throw a exception.
吴晟 Wu Sheng
yi.liang
mrproliu
cngdkxw
yoje
muyun12
aderm
lsyf
kezhenxu94
cui-liqiang
AirTrioa
jialong
Kanro
Daming
tzy1316106836
dengliming
Zhang Yonglun
Andy Ai
cyejing
louis
pop0505
zhang-wei
qiyang
viswaramamoorthy
于玉桔
aderm
scolia
echooymxq
lipenghui
Stalary
Louis Tsui
SummerOfServenteen
stone.wlg
hi-sb
qxo
刘新元 Liu XinYuan
honganan
li.can
于玉桔
caoyixiong
Xin,Zhang
kezhenxu94
FuCheng,Yan
zxbu
SataQiu
Jared Tan
Lemon
康智冬
Brandon Fergerson
Wilt
IluckySi
ksewen
彭勇升 pengys
Brandon Fergerson
dingshaocheng
刘正阳
dongbin
Xin,Zhang
XiaoFu
ScienJus
cheetah012
chenpengfei
survior
Fatih ERDEM
Adrian Cole
oatiz
XiaoFu