Commit Graph

197 Commits

Author SHA1 Message Date
Evan fd1d21021f
[Incompatible Enhancement]New index policy of ElasticSearch storage option (#6499) 2021-03-11 15:11:37 +08:00
zhang-wei ba6d49ad91
Update the timestamp field type for LogQuery (#6335) 2021-02-07 12:09:51 +08:00
吴晟 Wu Sheng 5e8f1eb552
Support multiple implementations of StorageBuilder in different storage implementations - stage 2 (#6336) 2021-02-07 10:37:14 +08:00
吴晟 Wu Sheng fc23dabf37
Support multiple implementations of StorageBuilder in different storage implementations - stage 1. (#6334) 2021-02-06 21:22:58 +08:00
Zhenxu Ke b92eac2cd6
Add a new concept "Event" and its implementations to collect events (#6183) 2021-02-05 18:37:18 +08:00
Stephen Ni d4ea997523
chore: add @Override tag on some methods (#6304) 2021-02-03 09:46:57 +08:00
吴晟 Wu Sheng cdfe6ab358
Begin the 8.5.0 iteration (#6302)
* [maven-release-plugin] prepare release v8.4.0

* [maven-release-plugin] prepare for next development iteration
2021-02-02 08:01:49 +08:00
吴晟 Wu Sheng 93165abc2a
Multiple exclude and include query conditions don't work as expected. (#6295) 2021-02-01 00:33:20 +08:00
Zhenxu Ke 5d47d4d29f
Remove state / isError and collect exception stacktrace in error logs (#6275) 2021-01-30 08:18:52 +08:00
Evan e138d29778
add text analyzer for es (#6249) 2021-01-25 17:01:43 +08:00
zhang-wei 0bd8149596
LogQuery remove unused field (#6246) 2021-01-24 19:13:31 +08:00
zhang-wei 15452025c6
Log collecting and query implementation (#6120) 2021-01-14 09:49:11 +08:00
Daming 6e48dca99a
Improvement Influxdb query performance (#6066) 2020-12-30 19:54:33 +08:00
吴晟 Wu Sheng af19c72a7e
8.3.0 tagged, open for 8.3.0 iteration. (#5920)
* [maven-release-plugin] prepare release v8.3.0

* [maven-release-plugin] prepare for next development iteration
2020-11-29 11:01:20 +08:00
吴晟 Wu Sheng c9202af1ee
Support group in the service traffic and service list query. (#5851)
* Support group in the service traffic and service list query.
2020-11-17 22:45:32 +08:00
Jared Tan ef1363d928
remove unused min date timebucket in jdbc delethistory logical. (#5845)
* remove unused min date timebucket in jdbc delethistory logical.

* update CHANGES.md
2020-11-15 19:28:38 +08:00
kezhenxu94 f92a7bf897
Start next iteration 8.3.0 (#5719)
* [maven-release-plugin] prepare release v8.2.0

* [maven-release-plugin] prepare for next development iteration
2020-10-24 19:30:29 +08:00
Ax1an 287b8b9be5
Add syncBulkActions config to avoid the large amount of metrics data written ES in a single sync bulk request. (#5699) 2020-10-21 12:23:32 +08:00
Kdump a5f4052965
fix INEFFICIENT_KEYSET_ITERATOR (#5612)
* fix INEFFICIENT_KEYSET_ITERATOR
2020-10-06 21:44:23 +08:00
Evan 15b0068fdf
Add superdataset replicas number config (#5424)
* distinct the number of metrics or common record index with super dataset index.
2020-09-01 23:06:21 +08:00
zhang-wei 7ef8b3e476
Support Browser protocol at OAP (#4228) 2020-08-31 15:46:37 +08:00
Lu Jiajing 5bfac1b50f
Remove style-check exception for logger (#5363) 2020-08-21 18:26:25 +08:00
Gao Hongtao 68eba154c4
Add latest function (#5321)
Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-08-15 17:28:32 +08:00
Evan 197ef1e836
Improve Es trace basic query performance (#5132) 2020-08-15 08:24:37 +08:00
吴晟 Wu Sheng 7f7e96b088
[IMPORTANT] Query traces with tags as condition (#5270) 2020-08-11 07:24:40 +08:00
Evan 2df3c683ba
Super Size Dataset record index es rolling step (#5282) 2020-08-10 14:49:11 +08:00
Gao Hongtao b0bb7cf3be
Query all of labels once selected labels condition is absent. (#5202) 2020-08-08 22:26:06 +08:00
吴晟 Wu Sheng 2f2840cbf5
Begin 8.2.0 iteration. (#5211)
* [maven-release-plugin] prepare release v8.1.0

* [maven-release-plugin] prepare for next development iteration
2020-07-31 20:15:27 +08:00
吴晟 Wu Sheng 2c676ead24
Add OAP configuration vocabulary documentation. (#5149) 2020-07-23 08:24:40 +08:00
Gao Hongtao 3fdfdf3b83
Storage elasticsearch health check (#5099) 2020-07-19 23:37:13 +08:00
denis ce2469dbec
optimization MetricsEsDAO multiGet (#5096) 2020-07-14 22:47:37 +08:00
吴晟 Wu Sheng e8f7ae055c
Fix searchService bug in ES implementation . (#5059) 2020-07-09 00:01:30 +08:00
吴晟 Wu Sheng d305efcb52
In no-init mode, should only check the index existence. (#4929) 2020-06-16 08:06:02 +08:00
吴晟 Wu Sheng 5de0d2c6f6
Set version to 8.1.0 for next iteration. (#4916)
* [maven-release-plugin] prepare release v8.0.0

* [maven-release-plugin] prepare for next development iteration

* Remove all shardingsphere cases.
2020-06-12 23:44:42 +08:00
吴晟 Wu Sheng b07b395691
Fix 4 high priority bugs (#4913) 2020-06-12 12:07:45 +08:00
吴晟 Wu Sheng d094a0f7c0
Support UI template management. (#4843) 2020-06-01 23:06:14 +08:00
吴晟 Wu Sheng 6ed40b3395
Keep ID as ref ID in the readSampledRecords (#4822)
* Keep ID as ref ID in the readSampledRecords, as there is no meaningful logic entity, but based on the graphql, the client could request this.
2020-05-26 08:00:03 +08:00
吴晟 Wu Sheng ec175f5655
Miss the node type condition in the es searchServices query. (#4821) 2020-05-25 22:17:10 +08:00
Evan 1652e0c057
Add a SuperDataset tag for custom super size dataset shards config in es storage (#4793)
* add SuperDataset tag for add super size dataset shards config in es
2020-05-17 17:48:15 +08:00
吴晟 Wu Sheng cffcc58f8f
The NoneStreamEsDAO implementation doesn't use real index name to insert. (#4796) 2020-05-16 22:54:28 +08:00
吴晟 Wu Sheng 7edcd5aa33
Change the model installation into the reactive module (#4761) 2020-05-08 21:21:03 +08:00
Gao Hongtao 1d5a0b0f1a
Format index name with namespace (#4758)
* Format index name with namespace

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

* Use another variable to compare index name

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-05-08 20:12:56 +08:00
吴晟 Wu Sheng fac4ddc9e8
Fix two serious bugs. (#4741) 2020-05-05 12:55:14 +08:00
吴晟 Wu Sheng 217bb624f6
Support percentile function in the meter system. (#4725)
* Support percentile function in the meter system.

* Update document.

* Fix formats.
2020-04-29 08:43:26 +08:00
吴晟 Wu Sheng 071708a4a3
Support default value in labeled-value and heatmap query. (#4711)
* Support default value in labeled-value and heatmap query.

* Update a little document.
2020-04-25 21:24:15 +08:00
吴晟 Wu Sheng b635e254c4
Provide Meter(metrics) system (#4694)
Here are the basic and simple usages of the MeterSystem APIs.

## Meter Creation
New meter could be created based on metrics name, function, scope and data type. 
1. Metrics name is still the storage entity name.
2. Functions are every similar with the OAL function, just accepting different input.
```java
final MeterSystem meterSystem = MeterSystem.meterSystem(getManager());
meterSystem.create("test_long_metrics", "avg", ScopeType.SERVICE, Long.class);
```
NOTICE, this creation should only be called in the `module#prepare` stage, otherwise, `Can't create new metrics anymore` exception will be raised after the **CORE** module `start` stage finished. You may find out, there is a chance `create` could be executed successfully in your own `module#start` stage, but it it just because of the sequence of provider loaded by the class loader, **no guarantee, so please don't do that**.

## Runtime Calculation
`AcceptableValue` is the object created at the runtime to accept new metrics value.
```java
         final MeterSystem service = getManager().find(CoreModule.NAME).provider().getService(MeterSystem.class);
        Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                final AcceptableValue<Long> value = service.buildMetrics("test_long_metrics", Long.class);
                value.accept(MeterEntity.newService("abc"), 5L);
                value.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
                service.doStreamingCalculation(value);
            }
        }, 2, 2, TimeUnit.SECONDS);
```

## Meter Functions
Right now, only `avg` function has been implemented. I submit this PR as soon as possible to get your feedback. I will add more functions.

## Notice
1. Make slow trace query available in the sampled record. Logically, they are the same thing. The UI doesn't need to concern about the trace as a special case.
2. Endpoint dependency will be removed from the dashboard. Because no query available for it. We will provide a new page for that in 8.1
3. Comparison page will be removed due to dashboard is powerful enough to replace it.
2020-04-24 23:33:13 +08:00
吴晟 Wu Sheng 991d083378
Support typeOfMetrics query (#4686)
* Add data type to the column definition to replace the simple isValue definition. And top N query is also requiring value column now.

* Support typeOfMetrics query.
2020-04-22 13:14:59 +08:00
吴晟 Wu Sheng 418bdff277
New metrics query protocol v2 (#4679)
1. Support the new query protocol, and the v1 query protocol is still supported 
2. All `numOfxxx`/`GlobalBrief` is only a mock now, no real number.
3. typeOfMetrics service is a mock only too, follow up PR will implement this.
2020-04-21 15:39:18 +08:00
Gao Hongtao 12703fa54e
Keep latest index always present. (#4655)
* Keep latest index always present.
  * Avoid TTL timer to remove latest index
  * Create a latest index even without any data.

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-04-15 15:08:38 +08:00
吴晟 Wu Sheng 31385159c2
Support to use empty string to override existing config. (#4635) 2020-04-11 13:05:57 +08:00