Commit Graph

52 Commits

Author SHA1 Message Date
吴晟 Wu Sheng 6fe2041b47
Make 8.0.0 Core available. New protocol and register removed. (#4599)
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
2020-04-10 11:03:24 +08:00
吴晟 Wu Sheng fa9a44b750
Version number changed including a little documentation. (#4597) 2020-03-31 12:57:20 +08:00
吴晟 Wu Sheng 4474907fef
[Inventory Design Change] Remove endpoint register and endpoint inventory (#4570)
* Finish step one, source and entity changed.

* Step 2, finish the basic and core DAO changes.

* Step 3. Change all source codes of backend, and make project compiling successfully.

* Make startup successful and persistence works.

* Fix dead codes in the MySQLTableInstaller.

* Fix mischange.

* Fix MySQL storage bootstrap error.

* Rollback dependency change.

* Simply enable tailing logs from container no mater it's in CI or not

* Fix segment table issue.

* Fix MySQL Instanller.

* Support Influxdb implementation of new endpoint traffic.

* Set and modify timeout to 90mins

* Remove unnecessary entity in the EndpointTraffic.

* Fix wrong entity builder and make name more clear.

* Fix 2 equal methods bug.

* Fix InfluxDB identifier(name) conflict issue.

* Fix endpoint topology query, endpoint id should be changed as a string.

* Fix a missing change.

* Fix a wrong commit.

* Column name can't be changed, due to it affect the hybrid metadb(h2/mysql) query, change it explicitly with double quotation marks.

* Fix formats.

* Remove endpoint/operation register from the agent. and change the plugin tests(without local tests). operationId relation codes have been removed from agent too.

* Try to fix the hytrix case.

* Fix 3 cases.

* Try to fix cases.

* Fix 2.

* Fix one.

* Fix the serviceId type mis-change.

* Fix int query

* Fix hystrix case.

* Fix a case.

* Fix EndpointCallRelationDispatcher set the source/dest endpoint without id encoding.

* Update documentation and add 2 test cases.

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-03-28 22:51:41 +08:00
吴晟 Wu Sheng 61f98c81f0
Move the main branch to 7.1.0 iteration. (#4537)
* [maven-release-plugin] prepare release v7.0.0

* [maven-release-plugin] prepare for next development iteration
2020-03-19 10:38:41 +08:00
吴晟 Wu Sheng 334239388a
Support Daily Index Step feature in the OAP (#4368)
* Support Daily Index Step.

* Fix format.

* Fix mischanged ES.
2020-02-16 20:48:04 +08:00
Jared Tan 7ff9a0cfd9
supprt gRPC alarm hook. (#4344)
* support gRPC hook.

* revert submodule commit.

* revert submodule commitid.

* finish logical.

* remove author anonatation.

* fix ci.

* revert submodule.

* update logical.

* update logical.

* update logical.

* fix

* fix

* fix empty settings.

* fix.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-12 16:20:20 +08:00
kezhenxu94 5b255ba3e3
Set up more strict code styles and fix existing issues (#4337)
Motivation:

Review code styles with some bots automatically.

Modifications:

Set up ReviewDog in GitHub Action to review code style.

Add more check rules to checkstyle plugin.

Result:

Obvious code styles can be reviewed and commented automatically.
2020-02-11 17:10:49 +08:00
Gao Hongtao fc4402f5a4 Add trace log (#4339)
* Add trace log

 * Trace alarm rule reader
 * Trace the process of running alarm rule, especailly the moving of window
 * Add toString method to relevant class for tracing

* Remove unused importsC

* Revert "Remove unused importsC"

This reverts commit 395e1678

* Update query

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-02-10 15:53:43 +08:00
kezhenxu94 75aefd34de Fix potential concurrent problems and tune performance (#4274) 2020-01-20 14:59:41 +08:00
Liu Hao 72e6888ae3 Fix thread unsafe problem in server-alarm-plugin (#4230) (#4247)
* fix thread unsafe problem in server-alarm-plugin (#4230)

* remove redundant #moveTo

Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-20 09:57:35 +08:00
kezhenxu94 094eeb870f
Add alarm metrics OP >= and <=, and refactor the operator logics (#4221)
* Add alarm metrics OP >= and <=, and refactor the operator logics

* Add unit test for OP

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-01-13 00:50:37 +08:00
吴晟 Wu Sheng 9149de2a6e Support multiple linear values and merging p50/75/90/95/99 into percentile (#4214)
* Support new percentile func with new alarm and exporter for this new func.

* Fix e2e and OAL script

* Fix wrong column.

* Fix percentile bug and oal engine bug.

* Update query protocol and add percentile test case

* Support new query

* Adopt GraphQL requirement

* Fix wrong type cast.

* Fix query in H2 and ES.

* Fix docs and comments.

* Fix an e2e compile issue

* Fix javadoc issue and e2e test issue.

* Change CPM to Apdex in TTL test.

* Fix OAL for TTL e2e

* Add metrics query for service percentile.

* Fix OAL engine bug. Method deserialize is not working when more than two field types are IntKeyLongValueHashMap

* Support multiple IntKeyLongValueHashMap fields in remote. About serialize/deserialize methods.

* Fix graphql statement error in e2e.

* Fix serialize not working and add generated serialize/deserialize of percentile into test cases.

* Fix test case format

* Remove generated code test.

* Fix failed e2e test

* Use avg resp time to apdex in the TTL test.

* ADD multiple linear metrics check for endpoint in e2e cluster.

* Support `-` to represent no threshold and doc of alarm about this.

* Move break to right place.

* Fix wrong break(s)

* Fix break and add a test case for multiple values alarm.

* Fix format.

* Add more doc for this new feature and GraphQL query protocol.

Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
2020-01-12 22:10:50 +08:00
吴晟 Wu Sheng d7f7799c80
Set the main branch to 7.0.0-SNAPSHOT (#4115)
* [maven-release-plugin] prepare release v6.6.0

* [maven-release-plugin] prepare for next development iteration
2019-12-24 16:37:13 +08:00
Stalary b666dc7e11 MOD:alarm-settings.yml support exclude-names (#3949) 2019-11-28 22:25:08 +08:00
吴晟 Wu Sheng 3fbaec42dc
Move to 6.6.0 SNAPSHOT (#3839)
* [maven-release-plugin] prepare release v6.5.0

* [maven-release-plugin] prepare for next development iteration
2019-11-12 22:33:39 +08:00
Jared Tan 2f3d009e63 Makes the scope of alarm message more semantic. (#3680)
* Makes the scope of alarm information more semantic.

* update md.

* update md.

* add ruleName in AlarmMsg.

* update md.

* update test case.

* fix coveralls.

* fix coveralls.

* fix ci.
2019-10-22 19:35:04 +03:00
kezhenxu94 2801de752f Support dynamically configure alarm settings (#3557)
* Support dynamically configure alarm settings

* Update documentation
2019-10-05 22:47:01 +08:00
吴晟 Wu Sheng ce20fe107d
Move to 6.5.0 (#3439)
* [maven-release-plugin] prepare release v6.4.0

* [maven-release-plugin] prepare for next development iteration
2019-09-08 16:24:21 -07:00
kezhenxu94 db9759fcc2 Check license header strictly (#3274)
* Check license header strictly

* Ignore header template

* Include more files to check list
2019-08-16 14:37:44 +08:00
吴晟 Wu Sheng b247dc979e
Move project master branch to 6.4.0-SNAPSHOT (#3220)
* [maven-release-plugin] prepare release v6.3.0

* [maven-release-plugin] prepare for next development iteration
2019-08-05 21:19:17 +08:00
kezhenxu94 9d7649d907 Set charset in web hook and replace magic number (#2986) 2019-07-03 09:22:53 +08:00
吴晟 Wu Sheng 4faa950c14
Move master to 6.3.0 (#2968)
* [maven-release-plugin] prepare release v6.2.0

* [maven-release-plugin] prepare for next development iteration
2019-06-29 13:49:36 +08:00
彭勇升 pengys 63fb961c0d New OAP annotations, they are friendly for streaming loader. (#2670)
* New stream annotation.
- Inventory annotation example:
@Stream(name = ServiceInventory.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_INVENTORY, storage = @Storage(builder = ServiceInventory.Builder.class, deleteHistory = false), kind = StreamKind.Inventory)

- Metrics annotation example:
@Stream(name = ServiceRelationClientSideMetrics.INDEX_NAME, scopeId = DefaultScopeDefine.SERVICE_RELATION, storage = @Storage(builder = ServiceRelationClientSideMetrics.Builder.class), kind = StreamKind.Metrics)

- Record annotation example:
@Stream(name = SegmentRecord.INDEX_NAME, scopeId = DefaultScopeDefine.SEGMENT, storage = @Storage(builder = SegmentRecord.Builder.class), kind = StreamKind.Record)

- TopN annotation example:
@Stream(name = TopNDatabaseStatement.INDEX_NAME, scopeId = DefaultScopeDefine.DATABASE_SLOW_STATEMENT, storage = @Storage(builder = TopNDatabaseStatement.Builder.class), kind = StreamKind.TopN)

* no message

* Stream annotation finish.

* Make stream processors to be the singleton class.

* Fixed the compile errors.

* Modify the OAL module for this annotation refactor.

* Add apache license header.

* Duplicate import.

* Fixed check style error.
2019-05-15 22:42:38 +08:00
彭勇升 pengys ed78dabe8b Rename metric and indicator to metrics. (#2643)
* Rename metric to metrics.

* Fixed test case execute failure issues.
2019-05-10 08:05:37 +08:00
Ming Deng 4b19bd77cf add unit tests for alarm-plugin module (#2542) 2019-04-30 21:46:23 +08:00
吴晟 Wu Sheng 9b87086d72
Move to 6.2 (#2563)
* Move submodule plugin.

* Always auto-submodule

* [maven-release-plugin] prepare release v6.1.0

* [maven-release-plugin] prepare for next development iteration

* Change keys location.
2019-04-30 12:16:50 +08:00
吴晟 Wu Sheng 975a06d163 Add catalog to ScopeDeclaration annotation. Make the alarm adapt to it. Now more metric could go through alarm, such as JVM, CLR. (#2391) 2019-03-21 18:38:32 +08:00
吴晟 Wu Sheng fb62025da8
Improve OAP extendibility (#2288)
* Make scope meta configurable.

* Fix test cases in ci.

* Change step 1

* Refactor step 2

* Other compile issue done. Prepare the generator changes.

* Revert some wrong changes to Istio receiver.

* Step 3, make compile pass.

* Try to make CI passed.

* Add a check.

* Fix generated code style

* Provide new and extendable dispatcher generator tool

* Fix startup.

* Can't guarantee the dispatchers are always existing. Put explicit comments at there to avoid confusion, since this is different with old version.

* Fix a H2 query for endpoint search.

* Fix a wrong mysql alarm query.

* Provide new document and `generate-tool-grammer` module.

* Add missing last section of document.

* Fix typo

* Relocate the @ScopeDeclaration annotation, make it more sense. Also remove the useless @SourceType annotation. And adjust document for these changes

* ScopeDeclaration can be used once for each class only, now.
2019-02-26 13:43:14 +08:00
吴晟 Wu Sheng 5a7eb5656a
GA tagged, move to 6.1 (#2180)
* [maven-release-plugin] prepare release v6.0.0-GA

* [maven-release-plugin] prepare for next development iteration
2019-01-18 20:04:23 +08:00
吴晟 Wu Sheng 95214aecf3
update version for next iteration. (#2084) 2018-12-23 09:14:14 +08:00
吴晟 Wu Sheng 5f5be77601
Fix the alarm being started too many times. (#2003)
* Fix the alarm being started too many times.

* Remove unnecessary interface.

* Fix wrong status changed
2018-12-05 20:56:08 +08:00
吴晟 Wu Sheng 2d8f390351 Make endpoint and instance alarm active (#1987)
* Support alarm in service instance and endpoint.

* Fix alarm default settings and document.

* Fix wrong default setting

* Make service instance and endpoint scopes available in alarm core.

* Fix alarm bug.
2018-12-01 14:18:31 +08:00
吴晟 Wu Sheng 72a39e1f28
Sync 6.0.0-alpha release and version up to beta. (#1883)
* Lock oracle and resin plugins version.

* Move resin and oracle plugin to openskywalking

* Restore submodule active.

* Add another submodule active command.

* Remove the wrong test cases in release and fix some doc issues.

* [maven-release-plugin] prepare release v6.0.0-alpha

* [maven-release-plugin] prepare for next development iteration

* Fix some doc
2018-11-06 15:45:15 +08:00
Jared.Tan 7537af0598 Fix the test case. (#1879) 2018-11-06 09:21:35 +08:00
吴晟 Wu Sheng 8cd327d8bb
Revert #1790 reset feature and provide default alarm settings. (#1833)
* Revert reset feature.

* Set default dist alarm settings and make dist package setting files right.

* Support empty list.
2018-10-28 22:28:21 +08:00
彭勇升 pengys 7c192b19d6
Alarm query and alarm persistence test. (#1753)
* Alarm query and alarm persistence test.

* Fixed the package error.
2018-10-11 15:55:04 +08:00
吴晟 Wu Sheng 3f8af95df0 Alarm silence init wrong. (#1752) 2018-10-11 10:38:52 +08:00
彭勇升 pengys ef63bc63cb The heartbeat implementation for all the scopes. (#1742) 2018-10-10 16:25:50 +08:00
吴晟 Wu Sheng b643190005 Add alarm timestamp. (#1741) 2018-10-10 14:15:24 +08:00
吴晟 Wu Sheng 933ae20b1c Alarm module code core ready (#1644)
* Support alarm message.

* Fix document words.

* Support alarm hook.

* Fix javadoc issue.

* Sync submodule

* Sync submodule
2018-09-25 20:23:36 +08:00
吴晟 Wu Sheng fd83aa55af Support new indicator core. (#1669)
* Support new indicator core.

* Block 2 unaccessible methods.

* Add service call as manual indicator and refactor the core DIspatcherManager
2018-09-13 16:38:50 +08:00
吴晟 Wu Sheng 973ad9e35d
Support time bucket transfer for all indicators. (#1664)
* Add transfer time bucket methods.

* prepare to merge

* Support time bucket transfer.
2018-09-12 18:44:13 +08:00
彭勇升 pengys f75270b43a v6 startup successful. (#1625)
* Fixed the startup failure cause by query protocol.

* Fixed the bug of startup failure.
2018-09-04 21:41:13 +08:00
Wu Sheng 1f2a6f2277 Move count and silenceCount into window. 2018-09-03 16:42:18 +08:00
Wu Sheng 6d74e7829c Fix CI and release guide. 2018-08-31 22:57:19 +08:00
Wu Sheng fff7401283 Fix alarm issues. 2018-08-31 22:52:00 +08:00
Wu Sheng 4559c49e86 Change alarm notification to batch mode. 2018-08-31 20:01:01 +08:00
Wu Sheng b5378e757b Add optional test case for Alarm core. Wouldn't execute in CI. 2018-08-30 23:02:40 +08:00
Wu Sheng 15ed4e7ecd Fix CI 2018-08-30 17:19:35 +08:00
Wu Sheng 90c30d4728 Remove useless method. 2018-08-30 17:14:33 +08:00