1. Support the new query protocol, and the v1 query protocol is still supported
2. All `numOfxxx`/`GlobalBrief` is only a mock now, no real number.
3. typeOfMetrics service is a mock only too, follow up PR will implement this.
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol.
2. New agent header protocol.
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
* Finish step one, source and entity changed.
* Step 2, finish the basic and core DAO changes.
* Step 3. Change all source codes of backend, and make project compiling successfully.
* Make startup successful and persistence works.
* Fix dead codes in the MySQLTableInstaller.
* Fix mischange.
* Fix MySQL storage bootstrap error.
* Rollback dependency change.
* Simply enable tailing logs from container no mater it's in CI or not
* Fix segment table issue.
* Fix MySQL Instanller.
* Support Influxdb implementation of new endpoint traffic.
* Set and modify timeout to 90mins
* Remove unnecessary entity in the EndpointTraffic.
* Fix wrong entity builder and make name more clear.
* Fix 2 equal methods bug.
* Fix InfluxDB identifier(name) conflict issue.
* Fix endpoint topology query, endpoint id should be changed as a string.
* Fix a missing change.
* Fix a wrong commit.
* Column name can't be changed, due to it affect the hybrid metadb(h2/mysql) query, change it explicitly with double quotation marks.
* Fix formats.
* Remove endpoint/operation register from the agent. and change the plugin tests(without local tests). operationId relation codes have been removed from agent too.
* Try to fix the hytrix case.
* Fix 3 cases.
* Try to fix cases.
* Fix 2.
* Fix one.
* Fix the serviceId type mis-change.
* Fix int query
* Fix hystrix case.
* Fix a case.
* Fix EndpointCallRelationDispatcher set the source/dest endpoint without id encoding.
* Update documentation and add 2 test cases.
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* 1. Format codes
2. Add comments for storage interface of topology
3. Narrow the scope of secondary query to get the component name. Only query the nodes as client out of the given service Ids.
* Support topology with multiple selected services
* Sync the typo fix in the protocol
* Support new percentile func with new alarm and exporter for this new func.
* Fix e2e and OAL script
* Fix wrong column.
* Fix percentile bug and oal engine bug.
* Update query protocol and add percentile test case
* Support new query
* Adopt GraphQL requirement
* Fix wrong type cast.
* Fix query in H2 and ES.
* Fix docs and comments.
* Fix an e2e compile issue
* Fix javadoc issue and e2e test issue.
* Change CPM to Apdex in TTL test.
* Fix OAL for TTL e2e
* Add metrics query for service percentile.
* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap
* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.
* Fix graphql statement error in e2e.
* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.
* Fix test case format
* Remove generated code test.
* Fix failed e2e test
* Use avg resp time to apdex in the TTL test.
* ADD multiple linear metrics check for endpoint in e2e cluster.
* Support `-` to represent no threshold and doc of alarm about this.
* Move break to right place.
* Fix wrong break(s)
* Fix break and add a test case for multiple values alarm.
* Fix format.
* Add more doc for this new feature and GraphQL query protocol.
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* Provide profile task downstream to sniffer
* fix agent unit testcase issue
* add profile switch config on sniffer
* fix es error
* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error
* change timeFromStartMills use `<` to compare
* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package
* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task
* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only
* remove navigate time judge, fix comment wrong meaning
* Add service type from register and support browser as service type
* Add service type at agent side.
* Fix format.
* Fix a test.
* Fix formats.
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
* Make backend fits new topology query protocol and logic.
* Change topology line id.
* Change endpoint point id rule to match service relation id rule.
* Refactor util.
* Revert wrong refactor by IDEA.
* Revert some changes.
* Fix entity id bug.
* Fix endpoint topology query
* Update UI.
* UI license update.
* New query graphql protocol added.
* Finish a new metric, log(abstract) with HTTP Access Log.
* Submit the draft codes of logs.
* Remove @IDColumn
* Fix startup issue.
* Add http_access_log to disable list.
* Finish a wrong service implementation.
* Add trace id in log query condition and result.
* Fix style
* The prototype of topN sql worker.
* Add scope and new manual dispatcher.
* no message
* Finish topN persistence codes. Not test yet. And query have not added.
* Finish the top n database statement persistent.
* Support different slow thresholds for different db types.
* Fix wrong db default threshold
* Finish new query protocol binding.
* Provide query empty implementation and sync ui.
* Finish all codes, hope it works :P
I will run the tests tonight.
* Fix
* Fix
* Fix a startup issue.
* Add time bucket to persistent.
* Fix wrong query result column name.
* Fix Database query.
* Fix checkstyle.
* Fix wrong order logic, and add a test case to verify, to fix https://github.com/apache/incubator-skywalking/pull/2239#discussion_r255948249
* Add Database source
* Add DatabaseSource source and database_* script for oal
* Enable Database_Indicator
* Add file license
* Rename DatabaseService to ClientDatabase. In the future, we may have real database server monitoring.
* Add query database from H2
* refactor
* update submodule
* refactor
* Fix style
* Fix: unknown tag error
* fix checkstyle
* H2 storage code skeleton.
* Add batch dao for H2.
* Fix wrong refactor.
* Add codes of other dao. No metric dao, wait for @peng-yongsheng new PR.
* Change license of distribution
* Add topology query in H2.
* Add more daos.
* Add some daos.
* Commit all H2 daos. Wait @peng-yongsheng to submit new IAggregationQueryDAO interface.
* Fix some CIs
* Fix wrong refactor, which causes h2 plugin failure.
* Fix all codes of H2 storage. :) and CI should pass. No test.
* Fix CI and typo
* Fix connection closed.
* Fix register bug
* no message
* Fixed service inventory register bug.
* Fix a compile issue.
* Sync change
* Reset es as default storage.
* Fix rat
* Fix H2 compile issue.
* Fix bugs for H2 storage.
* Fix bug in mesh.
* 1. Fix Thermodynamic query bug
2. Fix service inventory query bug
3. Fix missing traceid in trace query bug
* Fix query bug.
* Submit reverse caused by prev PR.
* Fix duplicate spans.
* Use H2 as default storage implementor.
* Fix bug in topology query.
* Fix CI
* Fix rat
* Fix two query in H2 storage.
* Fix H2 query
* Make release running in H2 mode and develop mode running is ES mode in default
* Manually trigger agent registration and write registration status to file
* Delete unused variables
* Code optimization, encapsulating some methods
* add clear DataCarrier function
* revert SkyWalkingAgent
* Optimize code and logic, including updating applicatin_id in unRegisterOperationNames
* Server end metadata loss notification
* Server end metadata loss notification
* fix file stream close and other problem
* revert to old
* fix some bug
Signed-off-by: Liu-XinYuan <879928098@qq.com>
* fix some bugs in agent reset
* add close inputStream and fileChannel and fix some bug
* fixed some bugs
* Replace v5 changes by a link to 5.x branch. New v6 changelog.
* Finish query doc and fix a missing field in alarm doc.
* Add link to official oal in alarm doc.
* Fix words.
* Fix some doc
* Update README.md
* Update README.md
* Update README.md
* Update README.md
zhang-wei
Zhenxu Ke
吴晟 Wu Sheng
Gao Hongtao
何延龙
mrproliu
Kdump
Lemon
刘新元 Liu XinYuan
彭勇升 pengys