Commit Graph

17 Commits

Author SHA1 Message Date
吴晟 Wu Sheng b635e254c4
Provide Meter(metrics) system (#4694)
Here are the basic and simple usages of the MeterSystem APIs.

## Meter Creation
New meter could be created based on metrics name, function, scope and data type. 
1. Metrics name is still the storage entity name.
2. Functions are every similar with the OAL function, just accepting different input.
```java
final MeterSystem meterSystem = MeterSystem.meterSystem(getManager());
meterSystem.create("test_long_metrics", "avg", ScopeType.SERVICE, Long.class);
```
NOTICE, this creation should only be called in the `module#prepare` stage, otherwise, `Can't create new metrics anymore` exception will be raised after the **CORE** module `start` stage finished. You may find out, there is a chance `create` could be executed successfully in your own `module#start` stage, but it it just because of the sequence of provider loaded by the class loader, **no guarantee, so please don't do that**.

## Runtime Calculation
`AcceptableValue` is the object created at the runtime to accept new metrics value.
```java
         final MeterSystem service = getManager().find(CoreModule.NAME).provider().getService(MeterSystem.class);
        Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                final AcceptableValue<Long> value = service.buildMetrics("test_long_metrics", Long.class);
                value.accept(MeterEntity.newService("abc"), 5L);
                value.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
                service.doStreamingCalculation(value);
            }
        }, 2, 2, TimeUnit.SECONDS);
```

## Meter Functions
Right now, only `avg` function has been implemented. I submit this PR as soon as possible to get your feedback. I will add more functions.

## Notice
1. Make slow trace query available in the sampled record. Logically, they are the same thing. The UI doesn't need to concern about the trace as a special case.
2. Endpoint dependency will be removed from the dashboard. Because no query available for it. We will provide a new page for that in 8.1
3. Comparison page will be removed due to dashboard is powerful enough to replace it.
2020-04-24 23:33:13 +08:00
吴晟 Wu Sheng 6fe2041b47
Make 8.0.0 Core available. New protocol and register removed. (#4599)
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
2020-04-10 11:03:24 +08:00
mrproliu 4cf0600f35 Provide profile task downstream to sniffer (#4172)
* Provide profile task downstream to sniffer

* fix agent unit testcase issue

* add profile switch config on sniffer

* fix es error

* 1. add @DefaultImplementor on the sniffer profile task service
2. change ProfileTaskExecutionService#PROFILE_TASK_READY_SCHEDULE to final and remove volatile
2. fix style error

* change timeFromStartMills use `<` to compare

* 1. add `maxSamplingCount` to profile task
2. make profile task limit to the common package

* 1. change `agent.active_profile` to `profile.active` and make true on default
2. add `maxSamplingCount` in profile task
3. use `createTime` to check has new command list
4. add task re-check before process profile task

* 1. add `profile-receiver` document
2. change `ProfileTaskExecutionService` use single schedule thread pool
3. cache dont need fetch data when no data, use auto-fresh mechanism only

* remove navigate time judge, fix comment wrong meaning
2020-01-09 09:35:39 +08:00
panjuan 9dccace951 Is it a typo or not? (#3537) 2019-09-27 21:49:59 +08:00
吴晟 Wu Sheng 3ca7f7b73c
Support ALS and observe service mesh without Mixer (#2460)
* Add ALS proto and receiver in envoy
2019-05-17 15:29:38 +08:00
彭勇升 pengys 3a31812854 Recover the module name of envoy-metric. (#2645) 2019-05-10 12:02:17 +08:00
彭勇升 pengys ed78dabe8b Rename metric and indicator to metrics. (#2643)
* Rename metric to metrics.

* Fixed test case execute failure issues.
2019-05-10 08:05:37 +08:00
吴晟 Wu Sheng ff6fb90d38
Add jaeger traces support (#2434)
* Make Jaeger receiver works in 12450 as default.

* Finish compile and codebase.

* Make codes runnable.

* Finish Jaeger query tests.

* Add document for jaeger receiver.

* Add readme.

* Fix a break link.

* Finish doc.
2019-04-01 20:52:59 -07:00
吴晟 Wu Sheng 601b472ef6 Support Backend acts as pure Zipkin collector (#2424)
* Codebase for zipkin span persistence.

* Fix missing fields in storage.

* Miss the latency field.

* Finish some tests.

* Fix wrong latency.

* Finish doc and reset application.yml

* Make the description more clear.
2019-03-31 01:36:36 +08:00
吴晟 Wu Sheng 499638775f
New architecture graph, new documents. (#2345) 2019-03-11 18:12:59 +08:00
吴晟 Wu Sheng b4cb7e0bca
Envoy metric support and a bug fix (#2321)
* Extend envoy metric service, scope and OAL lexer/parser.

* Finish metric extension and oal.

* Make test env works.

* Finish test of envoy connection and memory metric.

* Change document.

* Finish docker compose test.
2019-03-07 17:05:46 +08:00
吴晟 Wu Sheng 5577f67de7 Support backend sampling (#1977)
* Support sampling trace at server side and keep metric right.

* Add a trace sampling document

* Fix wrong default value.

* Fix document issue.

* Fix assemble issue.

* Fix wrong settings and doc.
2018-11-29 23:33:07 +08:00
吴晟 Wu Sheng 6b35113536 Restore Zipkin receiver based on new core (#1932)
* Change some codes to make zipkin receiver works, not done yet.

* @adriancole Zipkin receiver is coming back again.

* Fix document and fix for Zipkin receiver.

* Fix some bugs.

* Fix register bug.

* Fix transfer bug for new backend core.
2018-11-18 22:46:27 +08:00
吴晟 Wu Sheng 2dd4f54957
Add some missing documents for v6 (#1830)
* Add more documents for v6.

* Make default setting consistent .
2018-10-28 21:45:58 +08:00
吴晟 Wu Sheng 64d886f197
Add document of contributions and some guides (#1672)
* Change oal doc.

* Remove typo

* Remove document WIP tag.

* Fix few words.

* no message

* Adjust contribution guides.
2018-09-13 21:01:36 +08:00
Wu Sheng f2811800b2 Add more ducoments. 2018-09-03 15:46:04 +08:00
Wu Sheng f80b031250 Add many new chapters of v6 document. Make the document more clear. 2018-09-02 23:11:56 +08:00