Commit Graph

362 Commits

Author SHA1 Message Date
Gao Hongtao a49ce988b4
Support regular expression in labelFilter (#5174)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-07-27 10:45:41 +08:00
Jared Tan db9f8e6e00
Support Jetty server advanced configuration. (#5147) 2020-07-25 17:41:49 +08:00
Stalary 1877d0fcf4
Add jvm_thread metrics (#5129) 2020-07-22 09:05:32 +08:00
Ax1an 7b9cbe2177
Update NodeType.java (#5108)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-07-16 18:44:28 +08:00
Gao Hongtao b6661b5164
Add health checker module (#5046) 2020-07-10 00:30:43 +08:00
yazong 55ad655a5e
#5013 update @Column(columnName = ENTITY_ID, length = 512) private String entityId; (#5057) 2020-07-08 20:19:50 +08:00
yazong 2eefe98203
update (#5055) 2020-07-08 16:30:53 +08:00
Ax1an 3df2e8ddce
Fix no telemetry metrics when the oap role is receiver. (#5038) 2020-07-06 18:50:14 +08:00
Ax1an a3ae7b4dc6
Fix no data bug of oap self observability instance_metrics_second_aggregation metrics. (#5028) 2020-07-06 13:03:36 +08:00
Gao Hongtao 1654a0fbf0
Add opencensus reciever (#5029)
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-07-05 20:53:16 +08:00
Daming d264f914d5
Fix timebucket conversion issue (#5014) 2020-07-02 22:05:03 +08:00
吴晟 Wu Sheng 6103ca9256
Fix wrong validation. (#5005) 2020-07-01 19:06:12 +08:00
吴晟 Wu Sheng ea6be5b645
Fix endpoint dependency bug in MQ and uninstrumented proxy cases, and support endpoint dependency(v2 of endpoint topology case). (#4995) 2020-06-30 20:59:38 +08:00
吴晟 Wu Sheng a93f1b3c3c
Support `logic-endpoint` series tag(s). (#4985)
* Support `logic-endpoint` series tag(s).

* Enhance the plugin dev doc.

* Update Java-Plugin-Development-Guide.md

fix punctuation

Co-authored-by: caoyixiong <32847163+IanCao@users.noreply.github.com>
2020-06-30 00:07:53 +08:00
dagmom 08897ce5f8
New feature provided : new plugin for influxdb-java client (#4846) 2020-06-21 22:35:45 +08:00
于玉桔 c778a0d6c2
Support Quasar fiber tracing (#4951) 2020-06-20 22:30:03 +08:00
吴晟 Wu Sheng 5de0d2c6f6
Set version to 8.1.0 for next iteration. (#4916)
* [maven-release-plugin] prepare release v8.0.0

* [maven-release-plugin] prepare for next development iteration

* Remove all shardingsphere cases.
2020-06-12 23:44:42 +08:00
吴晟 Wu Sheng b07b395691
Fix 4 high priority bugs (#4913) 2020-06-12 12:07:45 +08:00
Gao Hongtao 5038f07094
Update prometheus fetcher for so11y (#4902) 2020-06-11 16:58:28 +08:00
Jared Tan 202d5711d4
fix NPE when fetch empty readHeatMap metrics. (#4891) 2020-06-09 21:22:12 +08:00
Gao Hongtao 4f0f39ffcc
Ingest prometheus metrics through meter system (#4783) 2020-06-08 09:19:34 +08:00
吴晟 Wu Sheng d094a0f7c0
Support UI template management. (#4843) 2020-06-01 23:06:14 +08:00
dvsv2 605d2449f2
ApdexMetrics Change type of `totalNum`、`sSum`,`tNum` from int to long (#4839) 2020-05-29 17:05:45 +08:00
吴晟 Wu Sheng fed62aa731
Fix #4813 (#4815) 2020-05-23 14:55:43 +08:00
吴晟 Wu Sheng a10e044932
Group Parameterized Endpoints (#4802)
* Group Parameterized Endpoints. Configure the grouping rules for parameterized endpoints, to improve the meaning of the metrics.
2020-05-20 22:50:22 +08:00
Evan 1652e0c057
Add a SuperDataset tag for custom super size dataset shards config in es storage (#4793)
* add SuperDataset tag for add super size dataset shards config in es
2020-05-17 17:48:15 +08:00
吴晟 Wu Sheng c5f0591e06
Fix the wrong sortN condition (#4784) 2020-05-14 11:36:32 +08:00
mrproliu 749af3e04e
Alarm support multiple scope (#4769)
* Support service relation and database access alarm

* support service instance relation alarm

* support endpoint relation alarm

* document and format the name

* resolve issues

* add database access and endpoint relation rule example

* resolve code format

* remove unnecessary getter and setter package declare

* remove unnecessary fields

Co-authored-by: Mrproliu <mrproliu@lagou.com>
2020-05-11 00:28:10 +08:00
吴晟 Wu Sheng 2880699860
Make OAL controlled by the receivers. (#4767)
* Split the official_analysis.oal into different parts

* Add Envoy OAL define.

* Fix license.
2020-05-09 21:39:19 +08:00
zhang-wei d4f3218dea
Support define multiple OAL configuration (#4748) 2020-05-08 23:53:49 +08:00
吴晟 Wu Sheng 7edcd5aa33
Change the model installation into the reactive module (#4761) 2020-05-08 21:21:03 +08:00
吴晟 Wu Sheng 377315edbb
Make sure the DB service could be generated. (#4757) 2020-05-08 11:14:16 +08:00
吴晟 Wu Sheng 07cc984d73
Polish metrics logic, fix service traffic bug (#4754)
1. Service Traffic should serialize and deserialize the time bucket, otherwise, there is a chance service_traffic-0 generated.
2. Make top N results more clear about the owner. Previously, the same endpoint names of different services look the same.
3. Sync UI.
4. Fix service traffic equal and hashcode method. The time bucket is not a part of the entity attribute.
2020-05-07 23:00:38 +08:00
Efe Kaptan d19ba41678
Mariadb JDBC plugin (#4740)
* Add Mariadb JDBC plugin and provide tests

* Refactor component name

* Reformat plugin description and remove healthcheck span from the test

Co-authored-by: Efe Kaptan <Efe.Kaptan@webtrekk.com>
Co-authored-by: zhang-wei <pknfe@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-05-07 20:54:30 +08:00
吴晟 Wu Sheng fac4ddc9e8
Fix two serious bugs. (#4741) 2020-05-05 12:55:14 +08:00
kezhenxu94 7b071bd7f7
Fix nested query and type conversion in MySQL, remove unused codes (#4738) 2020-05-04 13:54:12 +08:00
吴晟 Wu Sheng a2eddd699e
Fix the cache unclear bug. (#4739)
* Fix the cache unclear bug.

* Update LimitedSizeBufferedData.java

Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-05-04 10:46:11 +08:00
吴晟 Wu Sheng 6d389dc119
Optimize aggregation in L1 and L2. (#4735) 2020-04-30 16:27:57 +08:00
吴晟 Wu Sheng d97af96c85
New ReadWriteSafeCache. Rewrite the whole Window and Collection. Simplify codes (#4733)
* Rewrite the whole window and collection to new ReadWriteSafeCache.

* Remove redundant fields.
2020-04-29 22:23:21 +08:00
吴晟 Wu Sheng 217bb624f6
Support percentile function in the meter system. (#4725)
* Support percentile function in the meter system.

* Update document.

* Fix formats.
2020-04-29 08:43:26 +08:00
吴晟 Wu Sheng 7c84e77bf2
Support histogram function in Meter system. (#4719)
* Support histogram function in Meter system.
* Add histogram in the document.
* Support infinite-
2020-04-27 14:38:38 +08:00
吴晟 Wu Sheng 071708a4a3
Support default value in labeled-value and heatmap query. (#4711)
* Support default value in labeled-value and heatmap query.

* Update a little document.
2020-04-25 21:24:15 +08:00
吴晟 Wu Sheng b635e254c4
Provide Meter(metrics) system (#4694)
Here are the basic and simple usages of the MeterSystem APIs.

## Meter Creation
New meter could be created based on metrics name, function, scope and data type. 
1. Metrics name is still the storage entity name.
2. Functions are every similar with the OAL function, just accepting different input.
```java
final MeterSystem meterSystem = MeterSystem.meterSystem(getManager());
meterSystem.create("test_long_metrics", "avg", ScopeType.SERVICE, Long.class);
```
NOTICE, this creation should only be called in the `module#prepare` stage, otherwise, `Can't create new metrics anymore` exception will be raised after the **CORE** module `start` stage finished. You may find out, there is a chance `create` could be executed successfully in your own `module#start` stage, but it it just because of the sequence of provider loaded by the class loader, **no guarantee, so please don't do that**.

## Runtime Calculation
`AcceptableValue` is the object created at the runtime to accept new metrics value.
```java
         final MeterSystem service = getManager().find(CoreModule.NAME).provider().getService(MeterSystem.class);
        Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                final AcceptableValue<Long> value = service.buildMetrics("test_long_metrics", Long.class);
                value.accept(MeterEntity.newService("abc"), 5L);
                value.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
                service.doStreamingCalculation(value);
            }
        }, 2, 2, TimeUnit.SECONDS);
```

## Meter Functions
Right now, only `avg` function has been implemented. I submit this PR as soon as possible to get your feedback. I will add more functions.

## Notice
1. Make slow trace query available in the sampled record. Logically, they are the same thing. The UI doesn't need to concern about the trace as a special case.
2. Endpoint dependency will be removed from the dashboard. Because no query available for it. We will provide a new page for that in 8.1
3. Comparison page will be removed due to dashboard is powerful enough to replace it.
2020-04-24 23:33:13 +08:00
吴晟 Wu Sheng 991d083378
Support typeOfMetrics query (#4686)
* Add data type to the column definition to replace the simple isValue definition. And top N query is also requiring value column now.

* Support typeOfMetrics query.
2020-04-22 13:14:59 +08:00
mrproliu 7de392af65
fix cache update timer exception when disable profile in the backend (#4682)
* fix cache update timer exception when disable profile in the backend

* add comment

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-21 18:59:17 +08:00
吴晟 Wu Sheng 418bdff277
New metrics query protocol v2 (#4679)
1. Support the new query protocol, and the v1 query protocol is still supported 
2. All `numOfxxx`/`GlobalBrief` is only a mock now, no real number.
3. typeOfMetrics service is a mock only too, follow up PR will implement this.
2020-04-21 15:39:18 +08:00
kezhenxu94 c11ad038cc
Count the coverage in E2E tests and Plugin tests (#4651) 2020-04-15 23:09:05 +08:00
吴晟 Wu Sheng 44cae97c25
Set up restrict length rules for service, instance and endpoint (#4633)
* Set up restrict length rules for service, instance and endpoint

* Adjust two env names.
2020-04-10 17:59:21 +08:00
吴晟 Wu Sheng 6fe2041b47
Make 8.0.0 Core available. New protocol and register removed. (#4599)
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
2020-04-10 11:03:24 +08:00
吴晟 Wu Sheng fa9a44b750
Version number changed including a little documentation. (#4597) 2020-03-31 12:57:20 +08:00