Commit Graph

284 Commits

Author SHA1 Message Date
Jared Tan c596d0ac82 add serviceInstanceId in Segment storage. (#2146)
* add serviceInstanceId in Segment storage.

* add serviceInstanceId in query condition.
2019-02-23 07:59:19 +08:00
LazyLei 8bacabf741 fix typo (#2274) 2019-02-22 07:32:36 +08:00
吴晟 Wu Sheng 58cf5936e6 Improve OAP cluster performance (#2271)
* Refactor for cluster mode.

* Make send to L2 in certain cycle.
2019-02-21 16:33:07 +08:00
吴晟 Wu Sheng c019535ee9
Fix mesh telemetry performance issue and adjust default thread number (#2261)
* Try adjustment.

* Remove heartbeat.

* Make heartbeat back to work. And default threadpool size more sense.

* Make L2 to less than before.

* Make instance heartbeat works.

* Try L1 aggregation thread = core * 2 * 2.
2019-02-20 08:29:17 +08:00
吴晟 Wu Sheng 479ee33244
Correct trace id in slow db statement (#2254)
* Correct trace id.

* Fix match.

* Add cover token to retry.

* Retry new test plugin.

* Reset to jacoco.

* Update jacoco version.
2019-02-19 13:02:54 +08:00
吴晟 Wu Sheng 333bcd48e0
Provide slow database rule document. (#2253) 2019-02-18 21:23:36 +08:00
吴晟 Wu Sheng 2f3ccf368c
Backend streaming thread model improvement (#2247)
[Performance Improvement]Backend streaming thread model improvement
2019-02-18 14:55:07 +08:00
于玉桔 0929ff3453 Support Zookeeper plugin (#2211)
* Zookeeper plugin
2019-02-15 22:13:44 +08:00
吴晟 Wu Sheng b86426c9a2 Add consumer pool to Datacarrier (#2245)
* Extend consume pool to data carrier.

* Change indicator aggregate worker and persistent worker to use default consumer pool.

* Fix CI.
2019-02-13 23:19:47 +08:00
吴晟 Wu Sheng 4c6429061a Support to register different host/port at cluster coordinator (#2244)
* Support to register different host/port at cluster coordinator. Only for Zookeeper and Consul

* Provide document for internalComHost and internalComPort

* Reset port to default.
2019-02-13 16:06:43 +08:00
吴晟 Wu Sheng 4babd6ff73 Support Top sql (#2239)
* The prototype of topN sql worker.

* Add scope and new manual dispatcher.

* no message

* Finish topN persistence codes. Not test yet. And query have not added.

* Finish the top n database statement persistent.

* Support different slow thresholds for different db types.

* Fix wrong db default threshold

* Finish new query protocol binding.

* Provide query empty implementation and sync ui.

* Finish all codes, hope it works :P
I will run the tests tonight.

* Fix

* Fix

* Fix a startup issue.

* Add time bucket to persistent.

* Fix wrong query result column name.

* Fix Database query.

* Fix checkstyle.

* Fix wrong order logic, and add a test case to verify, to fix https://github.com/apache/incubator-skywalking/pull/2239#discussion_r255948249
2019-02-12 23:13:02 +08:00
Lemon f7ca26729d Database metrics provided. (#2025)
* Add Database source

* Add DatabaseSource source and database_* script for oal

* Enable Database_Indicator

* Add file license

* Rename DatabaseService to ClientDatabase. In the future, we may have real database server monitoring.

* Add query database from H2

* refactor

* update submodule

* refactor

* Fix style

* Fix: unknown tag error

* fix checkstyle
2019-01-27 08:27:34 +08:00
于玉桔 a36394d9bc Support lettuce plugin (#2152)
* Support lettuce plugin

* lettuce plugin bugfix

* lettuce plugin fix license issue

* lettuce plugin fix cluster and config issue

* plugin lettuce compatible with low 5.0.x version

* lettuce plugin support version accurately describes on Supported-list.md

* plugin lettuce compatible with low 5.0.2 version

* lettuce plugin fix wrong batch method intercept point

* move lettuce to optional plugins

* fix ci issue

* fix ci issue

* Modify the description file

* Pom code optimization

* Optimization operation name

* Optimization operation name2
2019-01-26 13:09:03 +08:00
吴晟 Wu Sheng 2e0104e599 Register lock refactor (#2195)
* Refactor register and reduce the lock time.

* Refactor es id generation mechanism.

* Wrong commit place.
2019-01-23 23:21:36 +08:00
c704ee0f68 for issue#2196 (#2197)
for issue#2196
2019-01-23 18:55:38 +08:00
彭勇升 pengys 9810c5e308 Improve buffer reader speed. (#2188)
* 1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
4. Return the exchange flag after all the references and spans in one segment parsed to reduce the number of segment parse.
5. Put the not exchanged segment into a collection then try to exchange no more than 10 times because of the exchange is asynchronous.
6. Cache the segment object to avoid repeated deserialization.

#2185

* #2185
1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
2019-01-21 04:21:45 +08:00
吴晟 Wu Sheng d3155220c3
Fix H2 and MySQL endpoint dependency map miss upstream side and add changelogs (#2187)
* fix no language attr value in mesh (#2182)

* Fixed the inventory register lock invalid bug. (#2184)

* #2183

Merge commit into master.

* Make the h2 register lock same as mysql.

* Fix #2186
2019-01-20 18:14:18 +08:00
彭勇升 pengys 33f5cc19b5 Fixed the inventory register lock invalid bug. (#2184)
* #2183

Merge commit into master.

* Make the h2 register lock same as mysql.
2019-01-20 16:41:06 +08:00
Gao Hongtao 334dd03429 fix no language attr value in mesh (#2182) 2019-01-19 10:48:54 +08:00
吴晟 Wu Sheng 5a7eb5656a
GA tagged, move to 6.1 (#2180)
* [maven-release-plugin] prepare release v6.0.0-GA

* [maven-release-plugin] prepare for next development iteration
2019-01-18 20:04:23 +08:00
吴晟 Wu Sheng 18ed80bd95
Fix missing language field. (#2174) 2019-01-18 08:00:51 +08:00
吴晟 Wu Sheng 9cda28f054
Set H2 storage startup right. (#2173) 2019-01-18 00:14:04 +08:00
a198720 b1b5ee5103 fixed the index name to format name in logs (#2165)
* fixed  the index name to format name in logs

* modify the log level from info to debug
2019-01-16 16:34:59 +08:00
a3e03f6133 Un-exchanged endpoint name in ref can't find the id (#2162)
* Update ReferenceIdExchanger.java

Change comments of codes.
2019-01-15 20:38:05 +08:00
吴晟 Wu Sheng b5eb9e304d Support Service Instance Inventory properties and bug fixed (#2156)
* Refactor service instance inventory to properties based entity.

* Fix a bug and refactor properties update.
2019-01-13 22:07:52 +08:00
Jared Tan 84a5c8c546
fix log output. (#2153) 2019-01-12 21:42:52 +08:00
Yann.Xia f9fb69336e Fix unessential gson instance (#2151) 2019-01-12 12:01:11 +08:00
于玉桔 43b216aa92 Support redisson plugin (#2083)
* plugin support redisson
2019-01-11 22:32:09 +08:00
吴晟 Wu Sheng 9cc5e76d97
Support service inventory extension fields in json format (#2148)
* Add properties for service inventory

* Fix code bugs

* Provide develop doc for inventory extension.
2019-01-11 21:24:51 +08:00
于玉桔 20aa7f9ba1 Bugfix: logger information has not enough arguments (#2147) 2019-01-11 20:00:23 +08:00
吴晟 Wu Sheng 5f57e561c7 Provide grafana settings for telemetry (#2138)
* Change a label name for  sw instance to avoid confict.

* Fix a wrong metric name.

* Fix too many counter instances.

* Set uptime to second.

* Fix format.

* Fix format

* Remove two unnecessary counter.

* Provide grafana dashboard settings and remove unnecessary metric.

* Provide a simple UI brief doc. Fix #2137
2019-01-10 07:27:23 +08:00
吴晟 Wu Sheng 6499d4f583
Implement Prometheus to oap backend telemetry (#2133)
* Finish the basic codes of telemetry.

* Add many telemetry in oap.

* Fix a test case.

* Finish all telemetry metric.

* Fix format.

* Make telemetry works.

* Support JVM metrics.

* Add doc for telemetry.

* Fix typo.

* Fix #2135
2019-01-08 23:20:13 +08:00
吴晟 Wu Sheng aaa403ec00
Set telemetry module and APIs (#2123)
* Finish the APIs of telemetry module.

* Add prometheus license.

* Active default telemetry provider.
2019-01-07 09:48:53 +08:00
langyan1022 6a4e7ffc6c fix missing ojdbc component id in component-libraries.yml (#2122)
* add ojdbc component id
2019-01-04 20:56:38 +08:00
XiaoFu ee6841fc13 Support Gson Plugin (#2090)
* Support Gson plugin
2019-01-03 23:51:22 +08:00
吴晟 Wu Sheng 0a3af743b4
Fix wrong alarm text. (#2120) 2019-01-03 23:13:52 +08:00
吴晟 Wu Sheng ec391becad
Provide two start up modes and documents. (#2116) 2019-01-03 15:19:45 +08:00
吴晟 Wu Sheng 95214aecf3
update version for next iteration. (#2084) 2018-12-23 09:14:14 +08:00
吴晟 Wu Sheng c592aea8db Refactor register API for making integration easier. (#2075)
* Refactor register API for making integration easier.

* Fix wrong refactor.
2018-12-21 22:32:06 +08:00
XiaoFu addf5101e0 Support canal plugin (#2035)
* fix add Canal plugin

* fix add UnitTest

* fix   format pom.xml

* Delete CanalApplication.java

* fix support cluster url

* fix add Licensed

* fix bug

* fix
2018-12-20 21:07:31 +08:00
吴晟 Wu Sheng c7884694da
Move node type from network address to service level[Need storage module update] (#2051)
* Change srcLayer to nodeType. Add nodeType in service inventory entity.

* NodeType value refactor.

* Format codes.

* Fix compile issue.

* Fix bugs, and make query on service inventory instead of network inventory
2018-12-17 09:05:16 -08:00
ZhHong 0284293458 /instance/register osName, hostName (#2042)
osName and hostName values are reversed。
2018-12-13 07:58:02 -08:00
Jared Tan 2f37158e41
modify code style (#2041)
* modify code style

* ci
2018-12-13 21:59:17 +08:00
彭勇升 pengys dfe2b08599 [OAP Server][Cluster Plugin] Provide consul cluster plugin (#2036)
* Provide consul cluster plugin.

* Change log and application config file.

* Add consul's config item into the docker's application.yml.

* Fixed bug.

* Add consul description into  readme.

* Add okhttp component license.
2018-12-12 08:14:41 -08:00
彭勇升 pengys 7532583d90 Endpoint name need fuzzy query. (#2026) 2018-12-09 13:01:58 +08:00
彭勇升 pengys c12cfa5816 No need to dependency to elasticsearch. (#2024) 2018-12-09 11:27:16 +08:00
XiaoFu 45f44544eb Support rabbitmq plugin (#2000)
* Fix support rabbitmq plugin

* fix Unit test rename sw3 to sw6

* Update pom.xml

* fix adjust rabbitmq Unit test

* fix add rabbitmq component in docker component-libraries.yml

* Update RabbitMQProducerInterceptor.java

* Update RabbitMQProducerInterceptor.java
2018-12-07 21:20:10 +08:00
吴晟 Wu Sheng 40eb651238
OAL tool module established (#2015)
* Move oal tool back into main repo, and fix compile issue.

* Fix two bugs.

* Remove all generated codes.

* Make all oal generated codes out of source control.

* Make mvn compile and package work.

* Fix test cases.

* Make compile right and provide new documents.

* Remove an unnecessary tokens file.
2018-12-07 09:31:16 +08:00
彭勇升 pengys 27c475f74c 1. Make storage and cluster using same name space. (#2014)
2. Zookeeper implementation of the cluster module using name space to support using same zookeeper cluster by different OAP clusters.
2018-12-06 21:03:00 +08:00
吴晟 Wu Sheng 1587b057e1
Merge branch 'master' into clean 2018-12-06 14:40:56 +08:00
吴晟 Wu Sheng f2b158862c
[OAL refactor step 1]Refactor dispatcher manager (#2006)
* Remove the hard codes about DispatcherManager.

* Add comments.
2018-12-06 14:40:29 +08:00
Jared.Tan 1b1a9bf216 Remove unused Debug Switches 2018-12-06 11:00:43 +08:00
吴晟 Wu Sheng 5f5be77601
Fix the alarm being started too many times. (#2003)
* Fix the alarm being started too many times.

* Remove unnecessary interface.

* Fix wrong status changed
2018-12-05 20:56:08 +08:00
吴晟 Wu Sheng bd879fd2d0
Fix mapping update bug in H2, MySQL, TiDB storage (#1999)
* Service inventory mapping  doesn't update right.

* Remove codes.
2018-12-04 16:52:46 +08:00
吴晟 Wu Sheng fc79182c31
Make endpoint alarm setting not active in default (#1998) 2018-12-03 22:58:23 +08:00
Gao Hongtao 1fa3091984 Istio/namespace (#1994)
* Update submodule skywalking-ui

* Fix #1925 Add namespace to service name
2018-12-03 16:25:03 +08:00
Jared.Tan dc393b1c12 refactoring PropertyPlaceholderHelper class. (#1989) 2018-12-03 11:50:59 +08:00
吴晟 Wu Sheng 93a036499e Make sure all mapping data loaded. (#1992) 2018-12-02 23:09:32 +08:00
吴晟 Wu Sheng 44a85a4f5a Support service instance metric. (#1988) 2018-12-01 22:57:28 +08:00
吴晟 Wu Sheng 2d8f390351 Make endpoint and instance alarm active (#1987)
* Support alarm in service instance and endpoint.

* Fix alarm default settings and document.

* Fix wrong default setting

* Make service instance and endpoint scopes available in alarm core.

* Fix alarm bug.
2018-12-01 14:18:31 +08:00
彭勇升 pengys 5777103a2b
Print exception stack into console or log files because of I found some NPE message in the log files. But I don't known where are they from without exception stack. (#1986) 2018-12-01 00:09:39 +08:00
Jared.Tan 728033ce85 update application.yml for namespace. (#1975)
* Correct namespace description
2018-11-30 13:57:32 +08:00
彭勇升 pengys 76d2fa34e0 Parse segment binary with version 1 prototype by default. (#1980) 2018-11-30 13:15:04 +08:00
吴晟 Wu Sheng 5577f67de7 Support backend sampling (#1977)
* Support sampling trace at server side and keep metric right.

* Add a trace sampling document

* Fix wrong default value.

* Fix document issue.

* Fix assemble issue.

* Fix wrong settings and doc.
2018-11-29 23:33:07 +08:00
Jared.Tan 9c642d50d8 fix https://github.com/apache/incubator-skywalking/pull/1921/. (#1974) 2018-11-29 12:39:58 +08:00
吴晟 Wu Sheng 215eab7428
Mysql style storage implementation (#1964)
Also fixed several bugs in this merge.
2018-11-28 22:59:02 +08:00
Jared.Tan b79030765b feat:Active System.env Config In Both Collector And agent.config. (#1957)
* feat:Active System.env Config In Both Collector And agent.config.
1、moving related codes into apm-util module.
2、update Setting-override.md about how to use.

* resolve docs.

* typo

* finish test case.

* Update README.md
2018-11-26 00:12:01 +08:00
a198720 578fd70395 fixed bug: multiple formatIndexName in forceUpdate method (#1960) 2018-11-25 22:14:08 +08:00
彭勇升 pengys 983f7613b6 1. Close the output stream when switch to write next file. (#1958)
2. Close the input stream when switch to read next file.

#1665
2018-11-25 16:49:35 +08:00
Jared.Tan e697385b53 feat:support set config from system.envs. (#1919)
* support set config from system.envs.
2018-11-23 14:52:37 +08:00
彭勇升 pengys fd47fcb830 Remote clients selector error: / by zero (#1949)
* Fixed the bug of hash selector error cause of remote clients not build.

#1948

* Fixed a style check error.
2018-11-22 22:28:36 +08:00
彭勇升 pengys 7c8a683c2f
gRPC client usage improve (#1946)
* Close the clients which are unreachable.
Remote client manager test case and comments.

* Test the GRPCRemote client.

* 1. Catch the throwable for onComplete method cause of it will throw exception when connection lost.
2. Check the gRPC channel state, send message when state is ready, wait 5 second when state is not ready.

Notice: gRPC channel getState with true parameter will trigger reconnect operation.

* gRPC client will reconnect to the server when network recorvered.

* Recovery application.yml

* Recovery proto module commit id.

* no message

* Fixed compile error.
2018-11-22 20:50:27 +08:00
吴晟 Wu Sheng c6ada8c98f
Support new v2 protocol and make concept consistently (#1937)
* Update new protocol

* Support new protocol at agent side.

* Fix test case.

* Make backend supports new trace protocol

* Fix tests.

* Make all new services available in gRPC.

* Fix bugs.

* Finish new document for v2 trace protocol.

* Fix wrong version requirement of zk

* Fix trace query bug.

* Find a client side dictionary bug.

* Fix register bugs.

* Fix break doc links.
2018-11-21 22:33:04 +08:00
彭勇升 pengys 0ae26f8e6e Fixed a bug which the segment record index does not delete the history data because of the saving time bucket is a second format date, but the query delete is minute format date. (#1943)
#1942
2018-11-21 13:08:08 +08:00
吴晟 Wu Sheng 6b35113536 Restore Zipkin receiver based on new core (#1932)
* Change some codes to make zipkin receiver works, not done yet.

* @adriancole Zipkin receiver is coming back again.

* Fix document and fix for Zipkin receiver.

* Fix some bugs.

* Fix register bug.

* Fix transfer bug for new backend core.
2018-11-18 22:46:27 +08:00
彭勇升 pengys afd6164335 OAP server testing module. (#1931)
* OAP server testing module.

#1930

* #Missing delete code.
2018-11-18 20:22:56 +08:00
彭勇升 pengys e727de96b2 Upgrade gRPC library. (#1928)
* Upgrade gRPC library.

* License update.
2018-11-18 13:11:52 +08:00
彭勇升 pengys cb492abea1 Synchronous configuration section. (#1927)
#1915
2018-11-18 08:41:27 +08:00
彭勇升 pengys a4b2cd4df2 Make ZooKeeper Version Compatibility clear in document (#1916)
* ZooKeeper Version Compatibility

* Update backend-cluster.md

Reorganize words. cc @peng-yongsheng

* Update application.yml

* Update application.yml
2018-11-16 15:39:21 +08:00
吴晟 Wu Sheng 7188c9c776
Support init mode and document with it. (#1900) 2018-11-15 21:50:00 +08:00
吴晟 Wu Sheng 645a9a579d
New v2 header with header key `sw6` (#1894)
* Add some supports to sw6 header. Break many test cases because I turn sw3 default off. For sure, you could open in agent.conf.  FYI @peng-yongsheng @ascrutae

* Fix CI and make user cases still work under v1 header.

* Support BASE64 in v2 header and make entryOperationName and parentOperationName optional in ContextCarrier or ContextSnapshot

* Fix CI.

* @peng-yongsheng This is backend related, please pay attention.

* Fix the carrier item order is incorrect (#1903)
2018-11-12 17:42:17 +08:00
吴晟 Wu Sheng 6dd98b37bf Extend StorageEntity annotation to support source scope metadata (#1904)
* Add source attr into StorageEntity, and remove the this attr from InventoryType.

* Make RegisterLockInstaller works based on InventoryProcess registered info.
2018-11-12 14:19:16 +08:00
吴晟 Wu Sheng 83e0e5758f Make compare right. (#1899) 2018-11-09 22:23:50 +08:00
ScienJus 926911700e Fix logs with missing variables (#1884) 2018-11-06 18:32:02 +08:00
吴晟 Wu Sheng 72a39e1f28
Sync 6.0.0-alpha release and version up to beta. (#1883)
* Lock oracle and resin plugins version.

* Move resin and oracle plugin to openskywalking

* Restore submodule active.

* Add another submodule active command.

* Remove the wrong test cases in release and fix some doc issues.

* [maven-release-plugin] prepare release v6.0.0-alpha

* [maven-release-plugin] prepare for next development iteration

* Fix some doc
2018-11-06 15:45:15 +08:00
Jared.Tan 7537af0598 Fix the test case. (#1879) 2018-11-06 09:21:35 +08:00
吴晟 Wu Sheng 4399c66278
Make metric value to long to avoid too big integer issue. (#1876)
* Make metric value to long to avoid too big integer issue.

* Update protocol.

* Add a new filter to service number.
2018-11-03 23:25:28 +08:00
彭勇升 pengys 884dd8c59e Add detect point to be a part of unique key of the endpoint id. (#1874) 2018-11-03 00:55:00 +08:00
吴晟 Wu Sheng 76e4c5996d
avoid NPE. (#1869) 2018-11-02 15:00:33 +08:00
吴晟 Wu Sheng 75e4c67952 Endpoint register to client and server side based on detect point. (#1866) 2018-11-02 12:07:25 +08:00
彭勇升 pengys 6b1014c336 Fixed the collector OOM bug. (#1862)
* Fixed the bug of remote client not blocked when not received on complete message, it will carry the out of a memory exception.

* Sleep 10ms, not to sleep max 10ms.

* No more than 10 stream observers are allowed at the same time to send remote message. Otherwise block the remote queue.

* no message
2018-11-02 10:33:24 +08:00
吴晟 Wu Sheng 13f8825904 Bug fix (#1857)
* Fix mesh receiver doesn't update heartbeat.

* Fix bug of mesh heartbeat and rest receiver.

* Fix issue: opencensus is included in agent without shade. @tuohai666
2018-11-01 16:50:07 +08:00
吴晟 Wu Sheng 07e799e422 Fix bugs (#1854)
* Avoid service inventory object concurrency situation.

* Fix concurrency for minute persistence and others

* Fix document todo.

* Update license. Fix package with dirty agent plugin jar files.

* Fixed not combine error in persistence register worker.
2018-10-31 23:11:52 +08:00
Gao Hongtao 279c81e9fc
Fix k8s api connection leak (#1848)
* Update submodule skywalking-ui

* Fix k8s api bugs

* Fix api connection leak
2018-10-30 16:05:18 +08:00
吴晟 Wu Sheng dbf0444ec1
Fix NPE in protobuf and illegal query parameter. (#1847) 2018-10-30 12:17:17 +08:00
吴晟 Wu Sheng b6241b2ad2
Optimize collector settings and indicator (#1841)
* Support maxConcurrentCallsPerConnection and maxMessageSize in grpcServer

* Make percent percent is 1/10000

* Fix test cases.

* Make service instance indicator dispatch based on server side only.

* Set init data right in AlarmMeta.

* Make endpoint query based on server side only.
2018-10-29 20:45:44 +08:00
彭勇升 pengys 948ef6d1c4 Fixed the bugs of the OAP server running in cluster mode. (#1844)
* 1. Fixed the bug of the cluster node sync failure cause by curator instance cache is null.
2. Fixed the bug of the collection in remote data builder is not initialized.

* 1. Re-generate indicator.

* Fixed the bug of the collection in remote data builder is not initialized.

* Fixed the bug of the remote client created but not  initialize.
2018-10-29 17:31:30 +08:00
Gao Hongtao 90bb8c6c9d Fix k8s api bugs (#1842)
* Update submodule skywalking-ui

* Fix k8s api bugs
2018-10-29 16:02:20 +08:00
吴晟 Wu Sheng 974192ecd8 Keep service instance register based on UUID, but change name to a meaning string. (#1836) 2018-10-29 09:28:33 +08:00
吴晟 Wu Sheng 8cd327d8bb
Revert #1790 reset feature and provide default alarm settings. (#1833)
* Revert reset feature.

* Set default dist alarm settings and make dist package setting files right.

* Support empty list.
2018-10-28 22:28:21 +08:00