Commit Graph

5635 Commits

Author SHA1 Message Date
kezhenxu94 7b071bd7f7
Fix nested query and type conversion in MySQL, remove unused codes (#4738) 2020-05-04 13:54:12 +08:00
吴晟 Wu Sheng a2eddd699e
Fix the cache unclear bug. (#4739)
* Fix the cache unclear bug.

* Update LimitedSizeBufferedData.java

Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-05-04 10:46:11 +08:00
zifeihan c33f583806
fix to get 'peer' in SingleServerConfig (#4737)
* fix to get 'peer' in SingleServerConfig, this method to throw exception in 'org.apache.skywalking.apm.agent.core.context.TracingContext#createExitSpan' because 'peer' is null. in sw 6 version, this also one of  problem to throw  'NoSuchElementException'  in 'org.apache.skywalking.apm.agent.core.context.ContextManager#capture'

* Polishing

Co-authored-by: gui.cao <gui.cao@zhangmen.com>
Co-authored-by: hanfei <zifeihan@foxmail.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-05-03 18:19:55 +08:00
Brandon Fergerson 4fc57a6663
Increase Vert.x Compatability (#4736)
* vertx web plugin re-impl attempt

* added license

* update test for new plugin structure

* got rid of registryItems

* remove unnecessary dependencies

* re-impl eventbus scenario

* more precise naming

* fix expected data

* removed unsupported versions and removed vertx caching (avoids permission error)

* rewrote to remove copied code

* Update plugins-test.3.yaml

* fixes issue of .vertx files remaining on v3.0.0

* only present in v3.0.0

* consistency

* add more supported versions

* track status code

* spelling fix

* track status code

* more correct naming

* reverts 213f5a69 (status code isn't sent at this point)

* made HttpServerResponseImplEndInstrumentation more strict based on version
added status code tracking
fixes issue not finishing async spans

* works 3.0.0,3.7.0

* increase compatibility

* 3.9.0 default

* versioned witness classes (advice from wu-sheng)

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: zhang-wei <pknfe@outlook.com>
2020-05-01 17:06:46 +08:00
吴晟 Wu Sheng 6d389dc119
Optimize aggregation in L1 and L2. (#4735) 2020-04-30 16:27:57 +08:00
吴晟 Wu Sheng d97af96c85
New ReadWriteSafeCache. Rewrite the whole Window and Collection. Simplify codes (#4733)
* Rewrite the whole window and collection to new ReadWriteSafeCache.

* Remove redundant fields.
2020-04-29 22:23:21 +08:00
Jared Tan d51c788113
support user to customize an endpoint (#4716)
* support user to customize an endpoint with toolkit.

* fix code style.

* add md.

* Update docs/en/setup/service-agent/java-agent/Application-toolkit-trace.md

Co-Authored-By: 吴晟 Wu Sheng <wu.sheng@foxmail.com>

* add test case.

* fix expecteddata.tml.

* fix data.

* fix expecteddata.tml.

* remove operationId in expectedData.yaml.

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-29 19:39:52 +08:00
吴晟 Wu Sheng 217bb624f6
Support percentile function in the meter system. (#4725)
* Support percentile function in the meter system.

* Update document.

* Fix formats.
2020-04-29 08:43:26 +08:00
kezhenxu94 4a8f86f817
Use the built-in ability to check out submodules (#4728) 2020-04-28 20:32:56 +08:00
吴晟 Wu Sheng 7c84e77bf2
Support histogram function in Meter system. (#4719)
* Support histogram function in Meter system.
* Add histogram in the document.
* Support infinite-
2020-04-27 14:38:38 +08:00
innerpeacez 3bc6cb270c
k8s doc https://github.com/apache/skywalking-kubernetes (#4715)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-26 12:38:20 +08:00
吴晟 Wu Sheng f4864597c2
Fix an old query compatible bug. (#4714) 2020-04-26 10:58:36 +08:00
Daming e8be63465c
More strict PluginTests expected data verifier (#4713)
* More strict PluginTests expected data verifier

* remove unnecessary file

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-25 23:35:47 +08:00
yoje b6cc9a59d8
Fix npe in afterMethod/handleMethodException of kafka/finagle plugins (#4712)
* Fix npe in afterMethod/handleMethodException of kafka/finagle plugins
2020-04-25 22:27:45 +08:00
吴晟 Wu Sheng 071708a4a3
Support default value in labeled-value and heatmap query. (#4711)
* Support default value in labeled-value and heatmap query.

* Update a little document.
2020-04-25 21:24:15 +08:00
J-Cod3r 2b41aea639
log4j2 support AsyncAppender print tid (#4670)
* log4j2 support AsyncAppender print tid
2020-04-25 17:35:22 +08:00
zhang-wei 0babc1a6ff
Modification Plugin-test.md (#4706)
* modification doc

* modification doc

* Update docs/en/guides/Plugin-test.md

* Update docs/en/guides/Plugin-test.md

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Brandon Fergerson <fergerson92@gmail.com>
2020-04-25 14:51:09 +08:00
qiyang 4acf0ebc95
fix: IgnoredTraceContext continue (#4709) (#4710)
Co-authored-by: 亓杨 <qiyang@enmonster.com>
2020-04-25 09:17:44 +08:00
吴晟 Wu Sheng b635e254c4
Provide Meter(metrics) system (#4694)
Here are the basic and simple usages of the MeterSystem APIs.

## Meter Creation
New meter could be created based on metrics name, function, scope and data type. 
1. Metrics name is still the storage entity name.
2. Functions are every similar with the OAL function, just accepting different input.
```java
final MeterSystem meterSystem = MeterSystem.meterSystem(getManager());
meterSystem.create("test_long_metrics", "avg", ScopeType.SERVICE, Long.class);
```
NOTICE, this creation should only be called in the `module#prepare` stage, otherwise, `Can't create new metrics anymore` exception will be raised after the **CORE** module `start` stage finished. You may find out, there is a chance `create` could be executed successfully in your own `module#start` stage, but it it just because of the sequence of provider loaded by the class loader, **no guarantee, so please don't do that**.

## Runtime Calculation
`AcceptableValue` is the object created at the runtime to accept new metrics value.
```java
         final MeterSystem service = getManager().find(CoreModule.NAME).provider().getService(MeterSystem.class);
        Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                final AcceptableValue<Long> value = service.buildMetrics("test_long_metrics", Long.class);
                value.accept(MeterEntity.newService("abc"), 5L);
                value.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
                service.doStreamingCalculation(value);
            }
        }, 2, 2, TimeUnit.SECONDS);
```

## Meter Functions
Right now, only `avg` function has been implemented. I submit this PR as soon as possible to get your feedback. I will add more functions.

## Notice
1. Make slow trace query available in the sampled record. Logically, they are the same thing. The UI doesn't need to concern about the trace as a special case.
2. Endpoint dependency will be removed from the dashboard. Because no query available for it. We will provide a new page for that in 8.1
3. Comparison page will be removed due to dashboard is powerful enough to replace it.
2020-04-24 23:33:13 +08:00
Brandon Fergerson 776d508018
Vertx plugin test re-implementation (#4624)
* vertx web plugin re-impl attempt
2020-04-24 19:49:03 +08:00
Jared Tan 15f1e048e9
fix skywalking refister-receiver artifactId name (#4697)
* fix  skywalking refister-receiver artifactId

* update more.
2020-04-23 19:07:58 +08:00
Weiyi Liu cf7482a79c
Fix typo. (#4693) 2020-04-23 11:25:40 +08:00
Daming 44f1b4a4ff
[test/plugin] Enhance PluginTest (#4677)
1. Re-balance the workload of plugintests
2. Restore testcases of MySQL, remove on #4641
3. Support user to runs plugin-test on debug mode. It will not remove the log files and actualData.yaml
4. remove paramater 'parallel_run_size' and disable parallel test.
2020-04-22 20:43:17 +08:00
mrproliu 4e821415ea
Fix profile recheck error (#4690) 2020-04-22 19:25:56 +08:00
Jared Tan e28e2f48b4
ignored trace. (#4691) 2020-04-22 17:06:36 +08:00
吴晟 Wu Sheng 991d083378
Support typeOfMetrics query (#4686)
* Add data type to the column definition to replace the simple isValue definition. And top N query is also requiring value column now.

* Support typeOfMetrics query.
2020-04-22 13:14:59 +08:00
mrproliu 51ce763eb9
sync ui (#4685)
Co-authored-by: Mrproliu <mrproliu@lagou.com>
2020-04-22 08:38:41 +08:00
mrproliu 7de392af65
fix cache update timer exception when disable profile in the backend (#4682)
* fix cache update timer exception when disable profile in the backend

* add comment

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-21 18:59:17 +08:00
mrproliu 7d1fdd3ea8
Support v3 extension header (#4666)
* support v3 extension header

Co-authored-by: Mrproliu <mrproliu@lagou.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-21 17:09:02 +08:00
吴晟 Wu Sheng 418bdff277
New metrics query protocol v2 (#4679)
1. Support the new query protocol, and the v1 query protocol is still supported 
2. All `numOfxxx`/`GlobalBrief` is only a mock now, no real number.
3. typeOfMetrics service is a mock only too, follow up PR will implement this.
2020-04-21 15:39:18 +08:00
kezhenxu94 8e6a527f78
Add Java versions 9~14 to E2E tests (#4674) 2020-04-21 13:45:32 +08:00
Daming 28530cd79d
Upgrade the InfluxDB storage-plugin to protocol V3 (#4641)
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: kezhenxu94 <kezhenxu94@163.com>
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-04-19 20:10:59 +08:00
kezhenxu94 21bb638d50
[Test] Remove all images after E2E tests and disable ES threshold (#4671) 2020-04-19 16:54:16 +08:00
kezhenxu94 c11ad038cc
Count the coverage in E2E tests and Plugin tests (#4651) 2020-04-15 23:09:05 +08:00
吴晟 Wu Sheng 181bf13daf
Update README.md (#4656) 2020-04-15 16:23:05 +08:00
Gao Hongtao 12703fa54e
Keep latest index always present. (#4655)
* Keep latest index always present.
  * Avoid TTL timer to remove latest index
  * Create a latest index even without any data.

Signed-off-by: Gao Hongtao <hanahmily@gmail.com>
2020-04-15 15:08:38 +08:00
zhang-wei ccb65ff27f
Add `java` -> `go2sky` -> `java` e2e test case, and adapt v3 protocol (#4647)
* go2sky e2e

Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2020-04-15 10:01:11 +08:00
Jared Tan ecd7d99799
fix profile main function file link. (#4650) 2020-04-14 19:57:19 +08:00
mrproliu 171620bfa8
Add Nginx correlation e2e test (#4646) 2020-04-14 09:32:54 +08:00
songzhendong b3b8aaea6b
Add Tomcat 9 's Java agent deployment instructions(Linux) (#4645) 2020-04-13 18:12:54 +08:00
吴晟 Wu Sheng 950346c380
[CVE] Update Jetty version to fix its CVEs (#4642) 2020-04-13 14:27:19 +08:00
kezhenxu94 00e8f2d886
Fix bug that the agent heart beats too frequently (#4640)
* Fix bug that the agent heart beats too frequently

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2020-04-12 20:48:44 +08:00
吴晟 Wu Sheng 4ce2e9e873
Fix security issue of the metrics query (#4639) 2020-04-12 15:44:48 +08:00
mrproliu 5646dfbb23
fix wrong correlation header name document (#4638)
Co-authored-by: Mrproliu <mrproliu@lagou.com>
2020-04-12 07:51:11 +08:00
mrproliu 1601cc107c
Support `java` -> `nginx` -> `java` e2e test, and v3 protocol (#4637) 2020-04-11 22:31:28 +08:00
吴晟 Wu Sheng 31385159c2
Support to use empty string to override existing config. (#4635) 2020-04-11 13:05:57 +08:00
吴晟 Wu Sheng 44cae97c25
Set up restrict length rules for service, instance and endpoint (#4633)
* Set up restrict length rules for service, instance and endpoint

* Adjust two env names.
2020-04-10 17:59:21 +08:00
吴晟 Wu Sheng 6fe2041b47
Make 8.0.0 Core available. New protocol and register removed. (#4599)
Here are the list of changes we made in this pull request. Top 3 are the principle changes, others are following these principles.
1. New agent and mesh report protocol. 
2. New agent header protocol. 
3. Service register, instance register and network address register have been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days now.
11. Simplify the TTL to metrics and record. And the unit has been formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been removed due to no register.
13. New service id, instance id and endpoint id rules, including service relation, instance relation and endpoint relation id rules.
14. Java agent support `keep tracing` mode, meaning, agent generating tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. Telemetry of Grafana config has been merged into one.
19. Documentation updates.
20. [TBD] InfluxDB storage implementation is not available, need @dmsolr to fix later, in order to reduce the master change block by this PR.
2020-04-10 11:03:24 +08:00
Brandon Fergerson 50a5416117
Update Plugin-test.md (#4623) 2020-04-08 20:22:04 +08:00
Brandon Fergerson 778b86e1de
Update ConsumeDriver.java (#4610)
spelling fix
2020-04-06 07:17:31 +08:00