Commit Graph

142 Commits

Author SHA1 Message Date
吴晟 Wu Sheng cea04f3ca0 Wrong host name. (#2397) 2019-03-22 19:55:02 -07:00
Lemon 7047c97e8b Add CLR_Receiver plugin (#2385) 2019-03-20 20:53:42 +08:00
Lemon 9b055cc5d4 Update proto commit (#2383) 2019-03-19 20:37:19 +08:00
吴晟 Wu Sheng 0951b9434b Make k8s-Coordinator works in role mode. (#2347)
* Make k8s-Coordinator back in role mode.

* Start query API server after the core started.

* Do lock when port has been intialized.

* Rename service.

* Fix text cases.

* Declare ConfigService.class in core module.

* Implement in an easier way.

* Envoy print whole metric, now in debug.

* Forcedly fix envoy timestamp issue.
2019-03-12 15:36:36 +07:00
彭勇升 pengys 6d00642386 Separation the remote gRPC server from the receiver gRPC server. (#2335)
* #2327

Add a new module for share receiver server or to be a bridge of core server.

* Rename the module name.
2019-03-11 14:23:57 +08:00
吴晟 Wu Sheng 93347b2340
Envoy heartbeat update bug (#2334) 2019-03-08 17:41:26 +08:00
吴晟 Wu Sheng c2a7e8e539
Bug fix. (#2333) 2019-03-08 16:31:42 +08:00
吴晟 Wu Sheng b4cb7e0bca
Envoy metric support and a bug fix (#2321)
* Extend envoy metric service, scope and OAL lexer/parser.

* Finish metric extension and oal.

* Make test env works.

* Finish test of envoy connection and memory metric.

* Change document.

* Finish docker compose test.
2019-03-07 17:05:46 +08:00
Dhi Aurrahman b8a414edf5 Envoy metrics receiver plugin protos (#2304)
* Add envoy metrics service protos

This patch imports protos from envoyproxy/data-plane-api and
prometheus/client_model to enable metrics service service feature in
SkyWalking OAP.

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Add Envoy related docs

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Update the LICENSE to include new protos license

From:
- envoyproxy/data-plane-api
- prometheus/client_model

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Add a complete config.yaml example

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Add note on Envoy dynamic config

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Add protoc-gen-validate license

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Add samples

Signed-off-by: Dhi Aurrahman <dio@tetrate.io>

* Make CI and license right.

* Update README.md

@dhi typo :P

* Update README.md

Another typo :P

* Delete metrics.json

metrics.json is included in identify.json and not be linked in document.
2019-03-01 22:13:59 +08:00
Jared Tan c596d0ac82 add serviceInstanceId in Segment storage. (#2146)
* add serviceInstanceId in Segment storage.

* add serviceInstanceId in query condition.
2019-02-23 07:59:19 +08:00
吴晟 Wu Sheng c019535ee9
Fix mesh telemetry performance issue and adjust default thread number (#2261)
* Try adjustment.

* Remove heartbeat.

* Make heartbeat back to work. And default threadpool size more sense.

* Make L2 to less than before.

* Make instance heartbeat works.

* Try L1 aggregation thread = core * 2 * 2.
2019-02-20 08:29:17 +08:00
吴晟 Wu Sheng 479ee33244
Correct trace id in slow db statement (#2254)
* Correct trace id.

* Fix match.

* Add cover token to retry.

* Retry new test plugin.

* Reset to jacoco.

* Update jacoco version.
2019-02-19 13:02:54 +08:00
吴晟 Wu Sheng 333bcd48e0
Provide slow database rule document. (#2253) 2019-02-18 21:23:36 +08:00
吴晟 Wu Sheng 2f3ccf368c
Backend streaming thread model improvement (#2247)
[Performance Improvement]Backend streaming thread model improvement
2019-02-18 14:55:07 +08:00
吴晟 Wu Sheng 4babd6ff73 Support Top sql (#2239)
* The prototype of topN sql worker.

* Add scope and new manual dispatcher.

* no message

* Finish topN persistence codes. Not test yet. And query have not added.

* Finish the top n database statement persistent.

* Support different slow thresholds for different db types.

* Fix wrong db default threshold

* Finish new query protocol binding.

* Provide query empty implementation and sync ui.

* Finish all codes, hope it works :P
I will run the tests tonight.

* Fix

* Fix

* Fix a startup issue.

* Add time bucket to persistent.

* Fix wrong query result column name.

* Fix Database query.

* Fix checkstyle.

* Fix wrong order logic, and add a test case to verify, to fix https://github.com/apache/incubator-skywalking/pull/2239#discussion_r255948249
2019-02-12 23:13:02 +08:00
Lemon f7ca26729d Database metrics provided. (#2025)
* Add Database source

* Add DatabaseSource source and database_* script for oal

* Enable Database_Indicator

* Add file license

* Rename DatabaseService to ClientDatabase. In the future, we may have real database server monitoring.

* Add query database from H2

* refactor

* update submodule

* refactor

* Fix style

* Fix: unknown tag error

* fix checkstyle
2019-01-27 08:27:34 +08:00
c704ee0f68 for issue#2196 (#2197)
for issue#2196
2019-01-23 18:55:38 +08:00
彭勇升 pengys 9810c5e308 Improve buffer reader speed. (#2188)
* 1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
4. Return the exchange flag after all the references and spans in one segment parsed to reduce the number of segment parse.
5. Put the not exchanged segment into a collection then try to exchange no more than 10 times because of the exchange is asynchronous.
6. Cache the segment object to avoid repeated deserialization.

#2185

* #2185
1. Sleep 500 milliseconds after a batch re-call finish.
2. Re-create register lock index when the system property named debug is setting.
3. Get the register inventory before lock to avoid increment the sequence but not use.
2019-01-21 04:21:45 +08:00
彭勇升 pengys 33f5cc19b5 Fixed the inventory register lock invalid bug. (#2184)
* #2183

Merge commit into master.

* Make the h2 register lock same as mysql.
2019-01-20 16:41:06 +08:00
吴晟 Wu Sheng 5a7eb5656a
GA tagged, move to 6.1 (#2180)
* [maven-release-plugin] prepare release v6.0.0-GA

* [maven-release-plugin] prepare for next development iteration
2019-01-18 20:04:23 +08:00
a3e03f6133 Un-exchanged endpoint name in ref can't find the id (#2162)
* Update ReferenceIdExchanger.java

Change comments of codes.
2019-01-15 20:38:05 +08:00
吴晟 Wu Sheng b5eb9e304d Support Service Instance Inventory properties and bug fixed (#2156)
* Refactor service instance inventory to properties based entity.

* Fix a bug and refactor properties update.
2019-01-13 22:07:52 +08:00
吴晟 Wu Sheng 9cc5e76d97
Support service inventory extension fields in json format (#2148)
* Add properties for service inventory

* Fix code bugs

* Provide develop doc for inventory extension.
2019-01-11 21:24:51 +08:00
吴晟 Wu Sheng 5f57e561c7 Provide grafana settings for telemetry (#2138)
* Change a label name for  sw instance to avoid confict.

* Fix a wrong metric name.

* Fix too many counter instances.

* Set uptime to second.

* Fix format.

* Fix format

* Remove two unnecessary counter.

* Provide grafana dashboard settings and remove unnecessary metric.

* Provide a simple UI brief doc. Fix #2137
2019-01-10 07:27:23 +08:00
吴晟 Wu Sheng 6499d4f583
Implement Prometheus to oap backend telemetry (#2133)
* Finish the basic codes of telemetry.

* Add many telemetry in oap.

* Fix a test case.

* Finish all telemetry metric.

* Fix format.

* Make telemetry works.

* Support JVM metrics.

* Add doc for telemetry.

* Fix typo.

* Fix #2135
2019-01-08 23:20:13 +08:00
吴晟 Wu Sheng 95214aecf3
update version for next iteration. (#2084) 2018-12-23 09:14:14 +08:00
吴晟 Wu Sheng c7884694da
Move node type from network address to service level[Need storage module update] (#2051)
* Change srcLayer to nodeType. Add nodeType in service inventory entity.

* NodeType value refactor.

* Format codes.

* Fix compile issue.

* Fix bugs, and make query on service inventory instead of network inventory
2018-12-17 09:05:16 -08:00
ZhHong 0284293458 /instance/register osName, hostName (#2042)
osName and hostName values are reversed。
2018-12-13 07:58:02 -08:00
彭勇升 pengys dfe2b08599 [OAP Server][Cluster Plugin] Provide consul cluster plugin (#2036)
* Provide consul cluster plugin.

* Change log and application config file.

* Add consul's config item into the docker's application.yml.

* Fixed bug.

* Add consul description into  readme.

* Add okhttp component license.
2018-12-12 08:14:41 -08:00
Jared.Tan 1b1a9bf216 Remove unused Debug Switches 2018-12-06 11:00:43 +08:00
Gao Hongtao 1fa3091984 Istio/namespace (#1994)
* Update submodule skywalking-ui

* Fix #1925 Add namespace to service name
2018-12-03 16:25:03 +08:00
吴晟 Wu Sheng 5577f67de7 Support backend sampling (#1977)
* Support sampling trace at server side and keep metric right.

* Add a trace sampling document

* Fix wrong default value.

* Fix document issue.

* Fix assemble issue.

* Fix wrong settings and doc.
2018-11-29 23:33:07 +08:00
吴晟 Wu Sheng 215eab7428
Mysql style storage implementation (#1964)
Also fixed several bugs in this merge.
2018-11-28 22:59:02 +08:00
彭勇升 pengys 983f7613b6 1. Close the output stream when switch to write next file. (#1958)
2. Close the input stream when switch to read next file.

#1665
2018-11-25 16:49:35 +08:00
彭勇升 pengys 7c8a683c2f
gRPC client usage improve (#1946)
* Close the clients which are unreachable.
Remote client manager test case and comments.

* Test the GRPCRemote client.

* 1. Catch the throwable for onComplete method cause of it will throw exception when connection lost.
2. Check the gRPC channel state, send message when state is ready, wait 5 second when state is not ready.

Notice: gRPC channel getState with true parameter will trigger reconnect operation.

* gRPC client will reconnect to the server when network recorvered.

* Recovery application.yml

* Recovery proto module commit id.

* no message

* Fixed compile error.
2018-11-22 20:50:27 +08:00
吴晟 Wu Sheng c6ada8c98f
Support new v2 protocol and make concept consistently (#1937)
* Update new protocol

* Support new protocol at agent side.

* Fix test case.

* Make backend supports new trace protocol

* Fix tests.

* Make all new services available in gRPC.

* Fix bugs.

* Finish new document for v2 trace protocol.

* Fix wrong version requirement of zk

* Fix trace query bug.

* Find a client side dictionary bug.

* Fix register bugs.

* Fix break doc links.
2018-11-21 22:33:04 +08:00
彭勇升 pengys 0ae26f8e6e Fixed a bug which the segment record index does not delete the history data because of the saving time bucket is a second format date, but the query delete is minute format date. (#1943)
#1942
2018-11-21 13:08:08 +08:00
吴晟 Wu Sheng 6b35113536 Restore Zipkin receiver based on new core (#1932)
* Change some codes to make zipkin receiver works, not done yet.

* @adriancole Zipkin receiver is coming back again.

* Fix document and fix for Zipkin receiver.

* Fix some bugs.

* Fix register bug.

* Fix transfer bug for new backend core.
2018-11-18 22:46:27 +08:00
彭勇升 pengys afd6164335 OAP server testing module. (#1931)
* OAP server testing module.

#1930

* #Missing delete code.
2018-11-18 20:22:56 +08:00
吴晟 Wu Sheng 645a9a579d
New v2 header with header key `sw6` (#1894)
* Add some supports to sw6 header. Break many test cases because I turn sw3 default off. For sure, you could open in agent.conf.  FYI @peng-yongsheng @ascrutae

* Fix CI and make user cases still work under v1 header.

* Support BASE64 in v2 header and make entryOperationName and parentOperationName optional in ContextCarrier or ContextSnapshot

* Fix CI.

* @peng-yongsheng This is backend related, please pay attention.

* Fix the carrier item order is incorrect (#1903)
2018-11-12 17:42:17 +08:00
吴晟 Wu Sheng 72a39e1f28
Sync 6.0.0-alpha release and version up to beta. (#1883)
* Lock oracle and resin plugins version.

* Move resin and oracle plugin to openskywalking

* Restore submodule active.

* Add another submodule active command.

* Remove the wrong test cases in release and fix some doc issues.

* [maven-release-plugin] prepare release v6.0.0-alpha

* [maven-release-plugin] prepare for next development iteration

* Fix some doc
2018-11-06 15:45:15 +08:00
彭勇升 pengys 884dd8c59e Add detect point to be a part of unique key of the endpoint id. (#1874) 2018-11-03 00:55:00 +08:00
吴晟 Wu Sheng 76e4c5996d
avoid NPE. (#1869) 2018-11-02 15:00:33 +08:00
吴晟 Wu Sheng 75e4c67952 Endpoint register to client and server side based on detect point. (#1866) 2018-11-02 12:07:25 +08:00
吴晟 Wu Sheng 13f8825904 Bug fix (#1857)
* Fix mesh receiver doesn't update heartbeat.

* Fix bug of mesh heartbeat and rest receiver.

* Fix issue: opencensus is included in agent without shade. @tuohai666
2018-11-01 16:50:07 +08:00
吴晟 Wu Sheng b6241b2ad2
Optimize collector settings and indicator (#1841)
* Support maxConcurrentCallsPerConnection and maxMessageSize in grpcServer

* Make percent percent is 1/10000

* Fix test cases.

* Make service instance indicator dispatch based on server side only.

* Set init data right in AlarmMeta.

* Make endpoint query based on server side only.
2018-10-29 20:45:44 +08:00
吴晟 Wu Sheng 974192ecd8 Keep service instance register based on UUID, but change name to a meaning string. (#1836) 2018-10-29 09:28:33 +08:00
吴晟 Wu Sheng 8cd327d8bb
Revert #1790 reset feature and provide default alarm settings. (#1833)
* Revert reset feature.

* Set default dist alarm settings and make dist package setting files right.

* Support empty list.
2018-10-28 22:28:21 +08:00
吴晟 Wu Sheng a9489b7067
H2 storage (#1712)
* H2 storage code skeleton.

* Add batch dao for H2.

* Fix wrong refactor.

* Add codes of other dao. No metric dao, wait for @peng-yongsheng new PR.

* Change license of distribution

* Add topology query in H2.

* Add more daos.

* Add some daos.

* Commit all H2 daos. Wait @peng-yongsheng to submit new IAggregationQueryDAO interface.

* Fix some CIs

* Fix wrong refactor, which causes h2 plugin failure.

* Fix all codes of H2 storage. :) and CI should pass. No test.

* Fix CI and typo

* Fix connection closed.

* Fix register bug

* no message

* Fixed service inventory register bug.

* Fix a compile issue.

* Sync change

* Reset es as default storage.

* Fix rat

* Fix H2 compile issue.

* Fix bugs for H2 storage.

* Fix bug in mesh.

* 1. Fix Thermodynamic query bug
2. Fix service inventory query bug
3. Fix missing traceid in trace query bug

* Fix query bug.

* Submit reverse caused by prev PR.

* Fix duplicate spans.

* Use H2 as default storage implementor.

* Fix bug in topology query.

* Fix CI

* Fix rat

* Fix two query in H2 storage.

* Fix H2 query

* Make release running in H2 mode and develop mode running is ES mode in default
2018-10-28 10:31:58 +08:00
刘新元 Liu XinYuan b11d479337 Add service and instance reset function (#1790)
* Manually trigger agent registration and write registration status to file

* Delete unused variables

* Code optimization, encapsulating some methods

* add clear DataCarrier function

* revert SkyWalkingAgent

* Optimize code and logic, including updating applicatin_id in unRegisterOperationNames

* Server end metadata loss notification

* Server end metadata loss notification

* fix file stream close and other problem

* revert to old

* fix some bug

Signed-off-by: Liu-XinYuan <879928098@qq.com>

* fix some bugs in agent reset

* add close inputStream and fileChannel and fix some bug

* fixed some bugs
2018-10-27 19:58:15 +08:00
彭勇升 pengys 9b3192fbac Set span layer into network inventory for distinguish what the network address is. MQ? DB? ... (#1824) 2018-10-26 22:57:05 +08:00
吴晟 Wu Sheng d9556c38fc
Make Mesh and Istio receivers ready (#1821)
* Make receiver more effective

* Add Istio test case and all source dispatch in

* Refactor mock data

* FIx rat.

* Support call component.

* Fix ThermodynamicIndicator bug.

* Fix test cases.

* Fix missing calculate in db merging.

* Add codes for debug.

* 1. Fixed elasticsearch bulk process not fresh bug. (#1819)

2. Fixed the bug of source register no queue but wait the end for batch tag.

* Remove debug log, and restore TTL timer for real scenarios.
2018-10-26 15:17:33 +08:00
彭勇升 pengys 24e958fec0 1. Fixed the bug of inventory equals method. (#1814)
2. Fixed the bug of agent mock register order.
3. Fixed the bug of all heatmap.
2018-10-25 21:23:20 +08:00
彭勇升 pengys 6c92fc5484 1. Delete service component indicator. (#1809)
2. Make service relation and endpoint relation to record component id.
3. Delete service mapping indicator.
4. Make service inventory to record mapping service id.
5. Refactor global topology query and service topology query.
2018-10-25 11:19:31 +08:00
彭勇升 pengys 4063397988 Trigger service heartbeat when received service instance heartbeat. (#1785)
Fixed some meta query bugs.
2018-10-17 22:44:21 +08:00
彭勇升 pengys 064595eb81 Fixed the segment parse bug. Each use requires the creation of a new instance. (#1784) 2018-10-17 19:13:08 +08:00
彭勇升 pengys b367c36db9
Fixed the endpoint topology bug. (#1778)
* Delete the client side endpoint relation indicator.
Fixed the endpoint topology bug.

* Fixed package failure bug.
2018-10-17 11:28:16 +08:00
彭勇升 pengys 72f2b1cd78 Provide rest protocol for register and trace receiver. (#1774) 2018-10-16 17:01:49 +08:00
彭勇升 pengys c467e5c723 Fixed JVM handler error. Forgot to use break in switch case code. (#1766) 2018-10-15 16:02:51 +08:00
彭勇升 pengys 966d5f54f1 Fixed the bugs, #1750, #1749, #1748, #1746, #1745 (#1755) 2018-10-11 19:37:57 +08:00
彭勇升 pengys ef63bc63cb The heartbeat implementation for all the scopes. (#1742) 2018-10-10 16:25:50 +08:00
彭勇升 pengys a062fdae71 Fixed the bug from topology. (#1736)
* Fixed the bug from topology.

* no message
2018-10-09 20:33:04 +08:00
彭勇升 pengys 065a384f10 Trace buffer test success. (#1733) 2018-10-08 23:17:15 +08:00
吴晟 Wu Sheng 4cae884fc0
Add endpoint id to trace query. (#1722) 2018-09-28 19:22:15 +08:00
吴晟 Wu Sheng 813ea1ed39 Make DetectPoint in Mesh gRPC protocol, DetectPoint in Source, and SpanSrcType convert right. (#1714) 2018-09-28 11:52:31 +08:00
吴晟 Wu Sheng 2f43cb940e
Fix start when not config in provider and don't get module in prepare stage. (#1711) 2018-09-27 16:07:04 +08:00
吴晟 Wu Sheng 9d6bf6787b
Finish mesh source dispatch (#1687)
* Merge module and provider of mesh

* Finish the skeleton of mesh receiver.

* Finish mesh dispatcher.

* Fix missing timebucket
2018-09-26 08:20:55 +08:00
彭勇升 pengys 53b9e7f828 Implementation of aggregation query. (#1702) 2018-09-25 19:14:37 +08:00
彭勇升 pengys 0576466031 Implementation of metadata query. (#1686) 2018-09-21 11:47:40 +08:00
彭勇升 pengys 13fa30307a Add record persistence stream analysis. (#1683) 2018-09-19 15:21:56 +08:00
吴晟 Wu Sheng 7e23f49539
Generate all metric for official release, also new func (#1679)
* New generated indicators for real backend system.

* Support cpm and new indicators.

* Fix CI and change document.

* Remove an useless line.
2018-09-17 12:02:01 +08:00
吴晟 Wu Sheng 5a55b9ba1b Provide Pxx(p99,p95,p90,p75,p50) indicators and All dispatcher (#1654)
* Add 3 cases for existed indicators.

* Add Pxx indicator.

* Support All in trace analysis listener.

* Fix CI and add P99/95/90/75/50 to source ALL.
2018-09-11 16:20:22 +08:00
彭勇升 pengys bc571259b2
Streaming analysis topology test success. (#1646)
* Add generate indicator and dispatcher about service topology but not test.

* Delete call type from service relation.

* Streaming analysis topology test success.

* Fixed the check style error.
2018-09-09 15:30:43 +08:00
彭勇升 pengys 800a5353de Add generate indicator and dispatcher about service topology but not test. (#1638)
* Add generate indicator and dispatcher about service topology but not test.

* Delete call type from service relation.
2018-09-07 11:59:55 +08:00
刘新元 Liu XinYuan f0f4cde494 add the methods that are not in the graphql resolver and the classes … (#1634)
* add the methods that are not in the graphql resolver and the classes that are not in the type, and resolve the StackOverFlowOver that starts the report.

* add license

* update file header
2018-09-06 21:11:07 +08:00
吴晟 Wu Sheng 86452fd2b6 Fix all sources and sourceDispatchers. (#1620) 2018-09-04 17:36:44 +08:00
彭勇升 pengys d0e68872da SkyWalking's trace receiver implementation. (#1618)
* Buffer stream.

* Buffer file reader.

* Buffer library performance test and functional test successfully.

* Fixed the code merge mistake.

* SkyWalking's trace receiver implementation.

* SkyWalking's trace receiver implementation.

* Delete collector module.
2018-09-04 11:18:29 +08:00
peng-yongsheng ffece1617a Indicator analysis test successfully. 2018-08-20 11:01:33 +08:00
peng-yongsheng 4c04a599c4 All of the inventory register test success. 2018-08-18 14:14:28 +08:00
peng-yongsheng 3fa77d5c9d The core implementation of register and cache for scope inventory. 2018-08-18 03:06:24 +08:00
彭勇升 pengys 4ea4e04ecc
Service inventory register test success. (#1552)
* Register start up.

* Service inventory register test success.
2018-08-17 02:21:58 +08:00
Wu Sheng b0b7de9442 Fix CI. And add a new instance ping service. 2018-08-16 16:30:03 +08:00
Wu Sheng 550563aa8c Update protocol to 2.0 draft. 2018-08-16 15:54:09 +08:00
Wu Sheng 946376e2e5 Merge istio handler module and provider. Named as plugin directly. 2018-08-15 23:18:27 +08:00
Wu Sheng 451fcadd0b Add a debug log judge. 2018-08-15 23:00:20 +08:00
Gao Hongtao fc3b440ae9 Add handler to parse istio telemetry data. 2018-08-15 20:42:24 +08:00
Wu Sheng 38d88da5d1 Fix CI. 2018-08-09 21:17:25 +08:00
Wu Sheng 6ee67e6aa4 Create istio receiver module 2018-08-09 20:07:01 +08:00
彭勇升 pengys efe5299d03 Feature/oap/remote (#1505)
* Sample operator code.

* Indicator aggregator framework.

* Provide some annotation for OAL.

* Remote module.

* Register service.

* Add apache license header.

* Ignore comments when load definition files.
2018-07-30 09:54:36 +08:00
Wu Sheng 8c3fb27e0e Init codes of service mesh probe receiver. 2018-07-25 16:09:17 +08:00
Wu Sheng e7ce4d757c Add service mesh receiver. 2018-07-24 16:12:24 +08:00
彭勇升 pengys de3bc220e0 [OAP Server] Core framework (#1472)
* Receiver, aggregate, server finished.
2018-07-19 11:24:01 +08:00