Commit Graph

81 Commits

Author SHA1 Message Date
Jared Tan eb19715f99 set es storage namespace to lower case. (#2348) 2019-03-12 07:17:00 +08:00
吴晟 Wu Sheng fb62025da8
Improve OAP extendibility (#2288)
* Make scope meta configurable.

* Fix test cases in ci.

* Change step 1

* Refactor step 2

* Other compile issue done. Prepare the generator changes.

* Revert some wrong changes to Istio receiver.

* Step 3, make compile pass.

* Try to make CI passed.

* Add a check.

* Fix generated code style

* Provide new and extendable dispatcher generator tool

* Fix startup.

* Can't guarantee the dispatchers are always existing. Put explicit comments at there to avoid confusion, since this is different with old version.

* Fix a H2 query for endpoint search.

* Fix a wrong mysql alarm query.

* Provide new document and `generate-tool-grammer` module.

* Add missing last section of document.

* Fix typo

* Relocate the @ScopeDeclaration annotation, make it more sense. Also remove the useless @SourceType annotation. And adjust document for these changes

* ScopeDeclaration can be used once for each class only, now.
2019-02-26 13:43:14 +08:00
Jared Tan c596d0ac82 add serviceInstanceId in Segment storage. (#2146)
* add serviceInstanceId in Segment storage.

* add serviceInstanceId in query condition.
2019-02-23 07:59:19 +08:00
吴晟 Wu Sheng 4babd6ff73 Support Top sql (#2239)
* The prototype of topN sql worker.

* Add scope and new manual dispatcher.

* no message

* Finish topN persistence codes. Not test yet. And query have not added.

* Finish the top n database statement persistent.

* Support different slow thresholds for different db types.

* Fix wrong db default threshold

* Finish new query protocol binding.

* Provide query empty implementation and sync ui.

* Finish all codes, hope it works :P
I will run the tests tonight.

* Fix

* Fix

* Fix a startup issue.

* Add time bucket to persistent.

* Fix wrong query result column name.

* Fix Database query.

* Fix checkstyle.

* Fix wrong order logic, and add a test case to verify, to fix https://github.com/apache/incubator-skywalking/pull/2239#discussion_r255948249
2019-02-12 23:13:02 +08:00
Lemon f7ca26729d Database metrics provided. (#2025)
* Add Database source

* Add DatabaseSource source and database_* script for oal

* Enable Database_Indicator

* Add file license

* Rename DatabaseService to ClientDatabase. In the future, we may have real database server monitoring.

* Add query database from H2

* refactor

* update submodule

* refactor

* Fix style

* Fix: unknown tag error

* fix checkstyle
2019-01-27 08:27:34 +08:00
吴晟 Wu Sheng 2e0104e599 Register lock refactor (#2195)
* Refactor register and reduce the lock time.

* Refactor es id generation mechanism.

* Wrong commit place.
2019-01-23 23:21:36 +08:00
彭勇升 pengys 9810c5e308 Improve buffer reader speed. (#2188)
* 1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
4. Return the exchange flag after all the references and spans in one segment parsed to reduce the number of segment parse.
5. Put the not exchanged segment into a collection then try to exchange no more than 10 times because of the exchange is asynchronous.
6. Cache the segment object to avoid repeated deserialization.

#2185

* #2185
1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
2019-01-21 04:21:45 +08:00
吴晟 Wu Sheng d3155220c3
Fix H2 and MySQL endpoint dependency map miss upstream side and add changelogs (#2187)
* fix no language attr value in mesh (#2182)

* Fixed the inventory register lock invalid bug. (#2184)

* #2183

Merge commit into master.

* Make the h2 register lock same as mysql.

* Fix #2186
2019-01-20 18:14:18 +08:00
彭勇升 pengys 33f5cc19b5 Fixed the inventory register lock invalid bug. (#2184)
* #2183

Merge commit into master.

* Make the h2 register lock same as mysql.
2019-01-20 16:41:06 +08:00
Gao Hongtao 334dd03429 fix no language attr value in mesh (#2182) 2019-01-19 10:48:54 +08:00
吴晟 Wu Sheng 5a7eb5656a
GA tagged, move to 6.1 (#2180)
* [maven-release-plugin] prepare release v6.0.0-GA

* [maven-release-plugin] prepare for next development iteration
2019-01-18 20:04:23 +08:00
吴晟 Wu Sheng 18ed80bd95
Fix missing language field. (#2174) 2019-01-18 08:00:51 +08:00
吴晟 Wu Sheng 9cda28f054
Set H2 storage startup right. (#2173) 2019-01-18 00:14:04 +08:00
a198720 b1b5ee5103 fixed the index name to format name in logs (#2165)
* fixed  the index name to format name in logs

* modify the log level from info to debug
2019-01-16 16:34:59 +08:00
吴晟 Wu Sheng b5eb9e304d Support Service Instance Inventory properties and bug fixed (#2156)
* Refactor service instance inventory to properties based entity.

* Fix a bug and refactor properties update.
2019-01-13 22:07:52 +08:00
吴晟 Wu Sheng 6499d4f583
Implement Prometheus to oap backend telemetry (#2133)
* Finish the basic codes of telemetry.

* Add many telemetry in oap.

* Fix a test case.

* Finish all telemetry metric.

* Fix format.

* Make telemetry works.

* Support JVM metrics.

* Add doc for telemetry.

* Fix typo.

* Fix #2135
2019-01-08 23:20:13 +08:00
吴晟 Wu Sheng 95214aecf3
update version for next iteration. (#2084) 2018-12-23 09:14:14 +08:00
吴晟 Wu Sheng c592aea8db Refactor register API for making integration easier. (#2075)
* Refactor register API for making integration easier.

* Fix wrong refactor.
2018-12-21 22:32:06 +08:00
吴晟 Wu Sheng c7884694da
Move node type from network address to service level[Need storage module update] (#2051)
* Change srcLayer to nodeType. Add nodeType in service inventory entity.

* NodeType value refactor.

* Format codes.

* Fix compile issue.

* Fix bugs, and make query on service inventory instead of network inventory
2018-12-17 09:05:16 -08:00
彭勇升 pengys dfe2b08599 [OAP Server][Cluster Plugin] Provide consul cluster plugin (#2036)
* Provide consul cluster plugin.

* Change log and application config file.

* Add consul's config item into the docker's application.yml.

* Fixed bug.

* Add consul description into  readme.

* Add okhttp component license.
2018-12-12 08:14:41 -08:00
彭勇升 pengys 27c475f74c 1. Make storage and cluster using same name space. (#2014)
2. Zookeeper implementation of the cluster module using name space to support using same zookeeper cluster by different OAP clusters.
2018-12-06 21:03:00 +08:00
吴晟 Wu Sheng bd879fd2d0
Fix mapping update bug in H2, MySQL, TiDB storage (#1999)
* Service inventory mapping  doesn't update right.

* Remove codes.
2018-12-04 16:52:46 +08:00
吴晟 Wu Sheng 93a036499e Make sure all mapping data loaded. (#1992) 2018-12-02 23:09:32 +08:00
彭勇升 pengys 5777103a2b
Print exception stack into console or log files because of I found some NPE message in the log files. But I don't known where are they from without exception stack. (#1986) 2018-12-01 00:09:39 +08:00
Jared.Tan 9c642d50d8 fix https://github.com/apache/incubator-skywalking/pull/1921/. (#1974) 2018-11-29 12:39:58 +08:00
吴晟 Wu Sheng 215eab7428
Mysql style storage implementation (#1964)
Also fixed several bugs in this merge.
2018-11-28 22:59:02 +08:00
彭勇升 pengys 7c8a683c2f
gRPC client usage improve (#1946)
* Close the clients which are unreachable.
Remote client manager test case and comments.

* Test the GRPCRemote client.

* 1. Catch the throwable for onComplete method cause of it will throw exception when connection lost.
2. Check the gRPC channel state, send message when state is ready, wait 5 second when state is not ready.

Notice: gRPC channel getState with true parameter will trigger reconnect operation.

* gRPC client will reconnect to the server when network recorvered.

* Recovery application.yml

* Recovery proto module commit id.

* no message

* Fixed compile error.
2018-11-22 20:50:27 +08:00
吴晟 Wu Sheng c6ada8c98f
Support new v2 protocol and make concept consistently (#1937)
* Update new protocol

* Support new protocol at agent side.

* Fix test case.

* Make backend supports new trace protocol

* Fix tests.

* Make all new services available in gRPC.

* Fix bugs.

* Finish new document for v2 trace protocol.

* Fix wrong version requirement of zk

* Fix trace query bug.

* Find a client side dictionary bug.

* Fix register bugs.

* Fix break doc links.
2018-11-21 22:33:04 +08:00
彭勇升 pengys afd6164335 OAP server testing module. (#1931)
* OAP server testing module.

#1930

* #Missing delete code.
2018-11-18 20:22:56 +08:00
吴晟 Wu Sheng 6dd98b37bf Extend StorageEntity annotation to support source scope metadata (#1904)
* Add source attr into StorageEntity, and remove the this attr from InventoryType.

* Make RegisterLockInstaller works based on InventoryProcess registered info.
2018-11-12 14:19:16 +08:00
吴晟 Wu Sheng 72a39e1f28
Sync 6.0.0-alpha release and version up to beta. (#1883)
* Lock oracle and resin plugins version.

* Move resin and oracle plugin to openskywalking

* Restore submodule active.

* Add another submodule active command.

* Remove the wrong test cases in release and fix some doc issues.

* [maven-release-plugin] prepare release v6.0.0-alpha

* [maven-release-plugin] prepare for next development iteration

* Fix some doc
2018-11-06 15:45:15 +08:00
吴晟 Wu Sheng 4399c66278
Make metric value to long to avoid too big integer issue. (#1876)
* Make metric value to long to avoid too big integer issue.

* Update protocol.

* Add a new filter to service number.
2018-11-03 23:25:28 +08:00
彭勇升 pengys 884dd8c59e Add detect point to be a part of unique key of the endpoint id. (#1874) 2018-11-03 00:55:00 +08:00
吴晟 Wu Sheng 13f8825904 Bug fix (#1857)
* Fix mesh receiver doesn't update heartbeat.

* Fix bug of mesh heartbeat and rest receiver.

* Fix issue: opencensus is included in agent without shade. @tuohai666
2018-11-01 16:50:07 +08:00
吴晟 Wu Sheng b6241b2ad2
Optimize collector settings and indicator (#1841)
* Support maxConcurrentCallsPerConnection and maxMessageSize in grpcServer

* Make percent percent is 1/10000

* Fix test cases.

* Make service instance indicator dispatch based on server side only.

* Set init data right in AlarmMeta.

* Make endpoint query based on server side only.
2018-10-29 20:45:44 +08:00
吴晟 Wu Sheng 974192ecd8 Keep service instance register based on UUID, but change name to a meaning string. (#1836) 2018-10-29 09:28:33 +08:00
吴晟 Wu Sheng a9489b7067
H2 storage (#1712)
* H2 storage code skeleton.

* Add batch dao for H2.

* Fix wrong refactor.

* Add codes of other dao. No metric dao, wait for @peng-yongsheng new PR.

* Change license of distribution

* Add topology query in H2.

* Add more daos.

* Add some daos.

* Commit all H2 daos. Wait @peng-yongsheng to submit new IAggregationQueryDAO interface.

* Fix some CIs

* Fix wrong refactor, which causes h2 plugin failure.

* Fix all codes of H2 storage. :) and CI should pass. No test.

* Fix CI and typo

* Fix connection closed.

* Fix register bug

* no message

* Fixed service inventory register bug.

* Fix a compile issue.

* Sync change

* Reset es as default storage.

* Fix rat

* Fix H2 compile issue.

* Fix bugs for H2 storage.

* Fix bug in mesh.

* 1. Fix Thermodynamic query bug
2. Fix service inventory query bug
3. Fix missing traceid in trace query bug

* Fix query bug.

* Submit reverse caused by prev PR.

* Fix duplicate spans.

* Use H2 as default storage implementor.

* Fix bug in topology query.

* Fix CI

* Fix rat

* Fix two query in H2 storage.

* Fix H2 query

* Make release running in H2 mode and develop mode running is ES mode in default
2018-10-28 10:31:58 +08:00
彭勇升 pengys 9b3192fbac Set span layer into network inventory for distinguish what the network address is. MQ? DB? ... (#1824) 2018-10-26 22:57:05 +08:00
吴晟 Wu Sheng d9556c38fc
Make Mesh and Istio receivers ready (#1821)
* Make receiver more effective

* Add Istio test case and all source dispatch in

* Refactor mock data

* FIx rat.

* Support call component.

* Fix ThermodynamicIndicator bug.

* Fix test cases.

* Fix missing calculate in db merging.

* Add codes for debug.

* 1. Fixed elasticsearch bulk process not fresh bug. (#1819)

2. Fixed the bug of source register no queue but wait the end for batch tag.

* Remove debug log, and restore TTL timer for real scenarios.
2018-10-26 15:17:33 +08:00
彭勇升 pengys 24e958fec0 1. Fixed the bug of inventory equals method. (#1814)
2. Fixed the bug of agent mock register order.
3. Fixed the bug of all heatmap.
2018-10-25 21:23:20 +08:00
彭勇升 pengys 6c92fc5484 1. Delete service component indicator. (#1809)
2. Make service relation and endpoint relation to record component id.
3. Delete service mapping indicator.
4. Make service inventory to record mapping service id.
5. Refactor global topology query and service topology query.
2018-10-25 11:19:31 +08:00
彭勇升 pengys 7bb577a4e9 Provide auto delete history data service. (#1797) 2018-10-22 20:20:29 +08:00
彭勇升 pengys 4063397988 Trigger service heartbeat when received service instance heartbeat. (#1785)
Fixed some meta query bugs.
2018-10-17 22:44:21 +08:00
彭勇升 pengys 4e99e896c2
Set instance attributes into return value. (#1783)
* Set instance attributes into return value.

* Delete unused code.
2018-10-17 18:40:10 +08:00
彭勇升 pengys a12e8badb8 Provide new getEndpointInfo service in query protocol. (#1780) 2018-10-17 12:19:33 +08:00
彭勇升 pengys b367c36db9
Fixed the endpoint topology bug. (#1778)
* Delete the client side endpoint relation indicator.
Fixed the endpoint topology bug.

* Fixed package failure bug.
2018-10-17 11:28:16 +08:00
彭勇升 pengys b426390b79
Support metrics downsampling. (#1765)
* Support metrics downsampling.

* Fixed a check style error.
2018-10-15 15:27:38 +08:00
彭勇升 pengys a81b9920cd Set the function attribute for indicator and use AVG function to be the default value. (#1757) 2018-10-12 12:29:22 +08:00
彭勇升 pengys 966d5f54f1 Fixed the bugs, #1750, #1749, #1748, #1746, #1745 (#1755) 2018-10-11 19:37:57 +08:00
彭勇升 pengys 7c192b19d6
Alarm query and alarm persistence test. (#1753)
* Alarm query and alarm persistence test.

* Fixed the package error.
2018-10-11 15:55:04 +08:00