kazusa
/
skywalking-java
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

revert #7091 (#7189)

This commit is contained in:
liqiangz 2021-06-28 21:05:28 +08:00 committed by GitHub
parent 273bc5bc0b
commit d63f3ffcc1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
21 changed files with 652 additions and 237 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGES.md
View File

@ -19,7 +19,6 @@ Release Notes.
#### OAP-Backend #### OAP-Backend
* Disable Spring sleuth meter analyzer by default. * Disable Spring sleuth meter analyzer by default.
* Use MAL to calculate JVM metrics, remove OAL dependency.
* Only count 5xx as error in Envoy ALS receiver. * Only count 5xx as error in Envoy ALS receiver.
* Upgrade apollo core caused by CVE-2020-15170. * Upgrade apollo core caused by CVE-2020-15170.
* Upgrade kubernetes client caused by CVE-2020-28052. * Upgrade kubernetes client caused by CVE-2020-28052.

1
apm-dist-es7/src/main/assembly/binary-es7.xml
View File

@ -67,7 +67,6 @@
<include>ui-initialized-templates/*</include> <include>ui-initialized-templates/*</include>
<include>lal/*</include> <include>lal/*</include>
<include>log-mal-rules/*</include> <include>log-mal-rules/*</include>
<include>jvm-metrics-rules/*.yaml</include>
</includes> </includes>
<outputDirectory>config</outputDirectory> <outputDirectory>config</outputDirectory>
</fileSet> </fileSet>

1
apm-dist/src/main/assembly/binary.xml
View File

@ -67,7 +67,6 @@
<include>ui-initialized-templates/*</include> <include>ui-initialized-templates/*</include>
<include>lal/*</include> <include>lal/*</include>
<include>log-mal-rules/*</include> <include>log-mal-rules/*</include>
<include>jvm-metrics-rules/*.yaml</include>
</includes> </includes>
<outputDirectory>config</outputDirectory> <outputDirectory>config</outputDirectory>
</fileSet> </fileSet>

56
docs/en/concepts-and-designs/scope-definitions.md
View File

@ -52,6 +52,62 @@ This calculates the metrics data from each request of the service instance.
| tcpInfo.receivedBytes | The received bytes of the TCP traffic, if this request is a TCP call. | | long | | tcpInfo.receivedBytes | The received bytes of the TCP traffic, if this request is a TCP call. | | long |
| tcpInfo.sentBytes | The sent bytes of the TCP traffic, if this request is a TCP call. | | long | | tcpInfo.sentBytes | The sent bytes of the TCP traffic, if this request is a TCP call. | | long |
#### Secondary scopes of `ServiceInstance`
This calculates the metrics data if the service instance is a JVM and collects through javaagent.
1. SCOPE `ServiceInstanceJVMCPU`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
| name | The name of the service instance, such as `ip:port@Service Name`. **Note**: Currently, the native agent uses `uuid@ipv4` as the instance name, which does not assist in setting up a filter in aggregation. | | string|
| serviceName | The name of the service. | | string |
| usePercent | The percentage of CPU time spent.| | double|
2. SCOPE `ServiceInstanceJVMMemory`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
| name | The name of the service instance, such as `ip:port@Service Name`. **Note**: Currently, the native agent uses `uuid@ipv4` as the instance name, which does not assist in setting up a filter in aggregation. | | string|
| serviceName | The name of the service. | | string |
| heapStatus | Indicates whether the metric has a heap property or not. | | bool |
| init | See the JVM documentation. | | long |
| max | See the JVM documentation. | | long |
| used | See the JVM documentation. | | long |
| committed | See the JVM documentation. | | long |
3. SCOPE `ServiceInstanceJVMMemoryPool`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
| name | The name of the service instance, such as `ip:port@Service Name`. **Note**: Currently, the native agent uses `uuid@ipv4` as the instance name, which does not assist in setting up a filter in aggregation. | | string|
| serviceName | The name of the service. | | string |
| poolType | The type may be CODE_CACHE_USAGE, NEWGEN_USAGE, OLDGEN_USAGE, SURVIVOR_USAGE, PERMGEN_USAGE, or METASPACE_USAGE based on different versions of JVM. | | enum |
| init | See the JVM documentation. | | long |
| max | See the JVM documentation. | | long |
| used | See the JVM documentation. | | long |
| committed | See the JVM documentation. | | long |
4. SCOPE `ServiceInstanceJVMGC`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
| name | The name of the service instance, such as `ip:port@Service Name`. **Note**: Currently, the native agent uses `uuid@ipv4` as the instance name, which does not assist in setting up a filter in aggregation. | | string|
| serviceName | The name of the service. | | string |
| phrase | Includes both NEW and OLD. | | Enum |
| time | The time spent in GC. | | long |
| count | The count in GC operations. | | long |
5. SCOPE `ServiceInstanceJVMThread`
| Name | Remarks | Group Key | Type |
|---|---|---|---|
| name | The name of the service instance, such as `ip:port@Service Name`. **Note**: Currently, the native agent uses `uuid@ipv4` as the instance name, which does not assist in setting up a filter in aggregation. | | string|
| serviceName | The name of the service. | | string |
| liveCount | The current number of live threads. | | int |
| daemonCount | The current number of daemon threads. | | int |
| peakCount | The current number of peak threads. | | int |
### SCOPE `Endpoint` ### SCOPE `Endpoint`
This calculates the metrics data from each request of the endpoint in the service. This calculates the metrics data from each request of the endpoint in the service.

285
oap-server/analyzer/agent-analyzer/src/main/java/org/apache/skywalking/oap/server/analyzer/provider/jvm/JVMSourceDispatcher.java
View File

@ -18,167 +18,182 @@
package org.apache.skywalking.oap.server.analyzer.provider.jvm; package org.apache.skywalking.oap.server.analyzer.provider.jvm;
import java.util.Arrays;
import java.util.Collection;
import java.util.List; import java.util.List;
import java.util.stream.Collectors;
import java.util.Collections;
import com.google.common.collect.ImmutableMap;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.apm.network.common.v3.CPU;
import org.apache.skywalking.apm.network.language.agent.v3.GC; import org.apache.skywalking.apm.network.language.agent.v3.GC;
import org.apache.skywalking.apm.network.language.agent.v3.JVMMetric; import org.apache.skywalking.apm.network.language.agent.v3.JVMMetric;
import org.apache.skywalking.apm.network.language.agent.v3.Memory; import org.apache.skywalking.apm.network.language.agent.v3.Memory;
import org.apache.skywalking.apm.network.language.agent.v3.MemoryPool; import org.apache.skywalking.apm.network.language.agent.v3.MemoryPool;
import org.apache.skywalking.apm.network.language.agent.v3.Thread; import org.apache.skywalking.apm.network.language.agent.v3.Thread;
import org.apache.skywalking.oap.meter.analyzer.MetricConvert;
import org.apache.skywalking.oap.meter.analyzer.dsl.Sample;
import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamily;
import org.apache.skywalking.oap.meter.analyzer.dsl.SampleFamilyBuilder;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
import org.apache.skywalking.oap.server.analyzer.provider.meter.process.SampleBuilder;
import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem; import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.analysis.NodeType;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.source.GCPhrase;
import org.apache.skywalking.oap.server.core.source.MemoryPoolType;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMCPU;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMGC;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMMemory;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMMemoryPool;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMThread;
import org.apache.skywalking.oap.server.core.source.SourceReceiver;
import org.apache.skywalking.oap.server.library.module.ModuleManager; import org.apache.skywalking.oap.server.library.module.ModuleManager;
@Slf4j @Slf4j
public class JVMSourceDispatcher { public class JVMSourceDispatcher {
private final SourceReceiver sourceReceiver;
private final List<MetricConvert> metricConverts; public JVMSourceDispatcher(ModuleManager moduleManager) {
this.sourceReceiver = moduleManager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
public JVMSourceDispatcher(ModuleManager moduleManager, List<Rule> rules) {
this.metricConverts = rules.stream()
.map(it -> new MetricConvert(it, moduleManager.find(CoreModule.NAME).provider().getService(MeterSystem.class)))
.collect(Collectors.toList());
} }
public void sendMetric(String service, String serviceInstance, JVMMetric jvmMetric) { public void sendMetric(String service, String serviceInstance, JVMMetric metrics) {
List<Sample> cpuSamples = Collections.singletonList(parseCpuData(service, serviceInstance, jvmMetric)); long minuteTimeBucket = TimeBucket.getMinuteTimeBucket(metrics.getTime());
List<Sample> memorySamples = parseMemoryData(service, serviceInstance, jvmMetric);
List<Sample> memoryPoolSamples = parseMemoryPollData(service, serviceInstance, jvmMetric);
List<Sample> gcCountSamples = parseGcCountData(service, serviceInstance, jvmMetric);
List<Sample> gcTimeSamples = parseGcTimeData(service, serviceInstance, jvmMetric);
List<Sample> threadSamples = parseThreadData(service, serviceInstance, jvmMetric);
ImmutableMap<String, SampleFamily> sampleFamilies = ImmutableMap.<String, SampleFamily>builder() final String serviceId = IDManager.ServiceID.buildId(service, NodeType.Normal);
.put("sw_jvm_gc_time", SampleFamilyBuilder.newBuilder(gcTimeSamples.toArray(new Sample[0])).build()) final String serviceInstanceId = IDManager.ServiceInstanceID.buildId(serviceId, serviceInstance);
.put("sw_jvm_gc_count", SampleFamilyBuilder.newBuilder(gcCountSamples.toArray(new Sample[0])).build())
.put("sw_jvm_cpu", SampleFamilyBuilder.newBuilder(cpuSamples.toArray(new Sample[0])).build())
.put("sw_jvm_thread", SampleFamilyBuilder.newBuilder(threadSamples.toArray(new Sample[0])).build())
.put("sw_jvm_memory", SampleFamilyBuilder.newBuilder(memorySamples.toArray(new Sample[0])).build())
.put("sw_jvm_memory_poll", SampleFamilyBuilder.newBuilder(memoryPoolSamples.toArray(new Sample[0])).build())
.build();
metricConverts.forEach(metricConvert -> metricConvert.toMeter(sampleFamilies)); this.sendToCpuMetricProcess(
service, serviceId, serviceInstance, serviceInstanceId, minuteTimeBucket, metrics.getCpu());
this.sendToMemoryMetricProcess(
service, serviceId, serviceInstance, serviceInstanceId, minuteTimeBucket, metrics.getMemoryList());
this.sendToMemoryPoolMetricProcess(
service, serviceId, serviceInstance, serviceInstanceId, minuteTimeBucket, metrics.getMemoryPoolList());
this.sendToGCMetricProcess(
service, serviceId, serviceInstance, serviceInstanceId, minuteTimeBucket, metrics.getGcList());
this.sendToThreadMetricProcess(
service, serviceId, serviceInstance, serviceInstanceId, minuteTimeBucket, metrics.getThread());
} }
private List<Sample> parseThreadData(String service, String serviceInstance, JVMMetric jvmMetric) { private void sendToCpuMetricProcess(String service,
Thread thread = jvmMetric.getThread(); String serviceId,
return Arrays.asList( String serviceInstance,
buildThreadSample(thread.getDaemonCount(), "daemon", service, serviceInstance, jvmMetric.getTime()), String serviceInstanceId,
buildThreadSample(thread.getLiveCount(), "live", service, serviceInstance, jvmMetric.getTime()), long timeBucket,
buildThreadSample(thread.getPeakCount(), "peak", service, serviceInstance, jvmMetric.getTime()) CPU cpu) {
); ServiceInstanceJVMCPU serviceInstanceJVMCPU = new ServiceInstanceJVMCPU();
serviceInstanceJVMCPU.setId(serviceInstanceId);
serviceInstanceJVMCPU.setName(serviceInstance);
serviceInstanceJVMCPU.setServiceId(serviceId);
serviceInstanceJVMCPU.setServiceName(service);
// If the cpu usage percent is less than 1, will set to 1
double adjustedCpuUsagePercent = Math.max(cpu.getUsagePercent(), 1.0);
serviceInstanceJVMCPU.setUsePercent(adjustedCpuUsagePercent);
serviceInstanceJVMCPU.setTimeBucket(timeBucket);
sourceReceiver.receive(serviceInstanceJVMCPU);
} }
private List<Sample> parseGcCountData(String service, String serviceInstance, JVMMetric jvmMetric) { private void sendToGCMetricProcess(String service,
return jvmMetric.getGcList().stream().map(gc -> String serviceId,
buildGcSample(gc, gc.getCount(), "sw_jvm_gc_count", service, serviceInstance, jvmMetric.getTime()) String serviceInstance,
).collect(Collectors.toList()); String serviceInstanceId,
long timeBucket,
List<GC> gcs) {
gcs.forEach(gc -> {
ServiceInstanceJVMGC serviceInstanceJVMGC = new ServiceInstanceJVMGC();
serviceInstanceJVMGC.setId(serviceInstanceId);
serviceInstanceJVMGC.setName(serviceInstance);
serviceInstanceJVMGC.setServiceId(serviceId);
serviceInstanceJVMGC.setServiceName(service);
switch (gc.getPhrase()) {
case NEW:
serviceInstanceJVMGC.setPhrase(GCPhrase.NEW);
break;
case OLD:
serviceInstanceJVMGC.setPhrase(GCPhrase.OLD);
break;
}
serviceInstanceJVMGC.setTime(gc.getTime());
serviceInstanceJVMGC.setCount(gc.getCount());
serviceInstanceJVMGC.setTimeBucket(timeBucket);
sourceReceiver.receive(serviceInstanceJVMGC);
});
} }
private List<Sample> parseGcTimeData(String service, String serviceInstance, JVMMetric jvmMetric) { private void sendToMemoryMetricProcess(String service,
return jvmMetric.getGcList().stream().map(gc -> String serviceId,
buildGcSample(gc, gc.getTime(), "sw_jvm_gc_time", service, serviceInstance, jvmMetric.getTime()) String serviceInstance,
).collect(Collectors.toList()); String serviceInstanceId,
long timeBucket,
List<Memory> memories) {
memories.forEach(memory -> {
ServiceInstanceJVMMemory serviceInstanceJVMMemory = new ServiceInstanceJVMMemory();
serviceInstanceJVMMemory.setId(serviceInstanceId);
serviceInstanceJVMMemory.setName(serviceInstance);
serviceInstanceJVMMemory.setServiceId(serviceId);
serviceInstanceJVMMemory.setServiceName(service);
serviceInstanceJVMMemory.setHeapStatus(memory.getIsHeap());
serviceInstanceJVMMemory.setInit(memory.getInit());
serviceInstanceJVMMemory.setMax(memory.getMax());
serviceInstanceJVMMemory.setUsed(memory.getUsed());
serviceInstanceJVMMemory.setCommitted(memory.getCommitted());
serviceInstanceJVMMemory.setTimeBucket(timeBucket);
sourceReceiver.receive(serviceInstanceJVMMemory);
});
} }
private Sample parseCpuData(String service, String serviceInstance, JVMMetric jvmMetric) { private void sendToMemoryPoolMetricProcess(String service,
SampleBuilder.SampleBuilderBuilder sampleBuilderBuilder = SampleBuilder.builder(); String serviceId,
double adjustedCpuUsagePercent = Math.max(jvmMetric.getCpu().getUsagePercent(), 1.0); String serviceInstance,
sampleBuilderBuilder.name("sw_jvm_cpu"); String serviceInstanceId,
sampleBuilderBuilder.value(adjustedCpuUsagePercent); long timeBucket,
sampleBuilderBuilder.labels(ImmutableMap.<String, String>builder().build()); List<MemoryPool> memoryPools) {
return sampleBuilderBuilder.build().build(service, serviceInstance, jvmMetric.getTime());
memoryPools.forEach(memoryPool -> {
ServiceInstanceJVMMemoryPool serviceInstanceJVMMemoryPool = new ServiceInstanceJVMMemoryPool();
serviceInstanceJVMMemoryPool.setId(serviceInstanceId);
serviceInstanceJVMMemoryPool.setName(serviceInstance);
serviceInstanceJVMMemoryPool.setServiceId(serviceId);
serviceInstanceJVMMemoryPool.setServiceName(service);
switch (memoryPool.getType()) {
case NEWGEN_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.NEWGEN_USAGE);
break;
case OLDGEN_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.OLDGEN_USAGE);
break;
case PERMGEN_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.PERMGEN_USAGE);
break;
case SURVIVOR_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.SURVIVOR_USAGE);
break;
case METASPACE_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.METASPACE_USAGE);
break;
case CODE_CACHE_USAGE:
serviceInstanceJVMMemoryPool.setPoolType(MemoryPoolType.CODE_CACHE_USAGE);
break;
}
serviceInstanceJVMMemoryPool.setInit(memoryPool.getInit());
serviceInstanceJVMMemoryPool.setMax(memoryPool.getMax());
serviceInstanceJVMMemoryPool.setUsed(memoryPool.getUsed());
serviceInstanceJVMMemoryPool.setCommitted(memoryPool.getCommitted());
serviceInstanceJVMMemoryPool.setTimeBucket(timeBucket);
sourceReceiver.receive(serviceInstanceJVMMemoryPool);
});
} }
private List<Sample> parseMemoryData(String service, String serviceInstance, JVMMetric jvmMetric) { private void sendToThreadMetricProcess(String service,
return jvmMetric.getMemoryList().stream().map(memory -> Arrays.asList( String serviceId,
buildMemorySample(memory, memory.getInit(), "init", service, serviceInstance, jvmMetric.getTime()), String serviceInstance,
buildMemorySample(memory, memory.getMax(), "max", service, serviceInstance, jvmMetric.getTime()), String serviceInstanceId,
buildMemorySample(memory, memory.getCommitted(), "committed", service, serviceInstance, jvmMetric.getTime()), long timeBucket,
buildMemorySample(memory, memory.getUsed(), "used", service, serviceInstance, jvmMetric.getTime()) Thread thread) {
)).flatMap(Collection::stream).collect(Collectors.toList()); ServiceInstanceJVMThread serviceInstanceJVMThread = new ServiceInstanceJVMThread();
} serviceInstanceJVMThread.setId(serviceInstanceId);
serviceInstanceJVMThread.setName(serviceInstance);
private List<Sample> parseMemoryPollData(String service, String serviceInstance, JVMMetric jvmMetric) { serviceInstanceJVMThread.setServiceId(serviceId);
return jvmMetric.getMemoryPoolList().stream().map(memoryPool -> Arrays.asList( serviceInstanceJVMThread.setServiceName(service);
buildMemoryPoolSample(memoryPool, memoryPool.getInit(), "init", service, serviceInstance, jvmMetric.getTime()), serviceInstanceJVMThread.setLiveCount(thread.getLiveCount());
buildMemoryPoolSample(memoryPool, memoryPool.getMax(), "max", service, serviceInstance, jvmMetric.getTime()), serviceInstanceJVMThread.setDaemonCount(thread.getDaemonCount());
buildMemoryPoolSample(memoryPool, memoryPool.getCommitted(), "committed", service, serviceInstance, jvmMetric.getTime()), serviceInstanceJVMThread.setPeakCount(thread.getPeakCount());
buildMemoryPoolSample(memoryPool, memoryPool.getUsed(), "used", service, serviceInstance, jvmMetric.getTime()) serviceInstanceJVMThread.setTimeBucket(timeBucket);
)).flatMap(Collection::stream).collect(Collectors.toList()); sourceReceiver.receive(serviceInstanceJVMThread);
}
private Sample buildGcSample(GC gc, long value, String name, String service, String serviceInstance, long time) {
SampleBuilder.SampleBuilderBuilder sampleBuilderBuilder = SampleBuilder.builder();
sampleBuilderBuilder.name(name);
sampleBuilderBuilder.value(value);
switch (gc.getPhrase()) {
case NEW:
sampleBuilderBuilder.labels(ImmutableMap.of("gc_phrase", "new"));
break;
case OLD:
sampleBuilderBuilder.labels(ImmutableMap.of("gc_phrase", "old"));
break;
default:
}
return sampleBuilderBuilder.build().build(service, serviceInstance, time);
}
private Sample buildThreadSample(long value, String threadType, String service, String serviceInstance, long time) {
SampleBuilder.SampleBuilderBuilder sampleBuilderBuilder = SampleBuilder.builder();
sampleBuilderBuilder.name("sw_jvm_thread");
sampleBuilderBuilder.value(value);
sampleBuilderBuilder.labels(ImmutableMap.of("thread_type", threadType));
return sampleBuilderBuilder.build().build(service, serviceInstance, time);
}
private Sample buildMemorySample(Memory memory, long value, String memoryType, String service, String serviceInstance, long time) {
SampleBuilder.SampleBuilderBuilder sampleBuilderBuilder = SampleBuilder.builder();
sampleBuilderBuilder.name("sw_jvm_memory");
sampleBuilderBuilder.labels(ImmutableMap.of("heap_status", String.valueOf(memory.getIsHeap()), "memory_type", memoryType));
sampleBuilderBuilder.value(value);
return sampleBuilderBuilder.build().build(service, serviceInstance, time);
}
private Sample buildMemoryPoolSample(MemoryPool memoryPool, long value, String memoryType, String service, String serviceInstance, long time) {
SampleBuilder.SampleBuilderBuilder sampleBuilderBuilder = SampleBuilder.builder();
sampleBuilderBuilder.name("sw_jvm_memory_poll");
sampleBuilderBuilder.value(value);
String pollType = "poll_type";
String memoryTypeKey = "memory_type";
switch (memoryPool.getType()) {
case NEWGEN_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "memoryTypeKey", memoryTypeKey, memoryType));
break;
case OLDGEN_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "oldgenUsage", memoryTypeKey, memoryType));
break;
case PERMGEN_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "permgenUsage", memoryTypeKey, memoryType));
break;
case SURVIVOR_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "survivorUsage", memoryTypeKey, memoryType));
break;
case METASPACE_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "metaspaceUsage", memoryTypeKey, memoryType));
break;
case CODE_CACHE_USAGE:
sampleBuilderBuilder.labels(ImmutableMap.of(pollType, "codeCacheUsage", memoryTypeKey, memoryType));
break;
default:
}
return sampleBuilderBuilder.build().build(service, serviceInstance, time);
} }
} }

5
oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALLexer.g4
View File

@ -34,6 +34,11 @@ SRC_ENDPOINT: 'Endpoint';
SRC_SERVICE_RELATION: 'ServiceRelation'; SRC_SERVICE_RELATION: 'ServiceRelation';
SRC_SERVICE_INSTANCE_RELATION: 'ServiceInstanceRelation'; SRC_SERVICE_INSTANCE_RELATION: 'ServiceInstanceRelation';
SRC_ENDPOINT_RELATION: 'EndpointRelation'; SRC_ENDPOINT_RELATION: 'EndpointRelation';
SRC_SERVICE_INSTANCE_JVM_CPU: 'ServiceInstanceJVMCPU';
SRC_SERVICE_INSTANCE_JVM_MEMORY: 'ServiceInstanceJVMMemory';
SRC_SERVICE_INSTANCE_JVM_MEMORY_POOL: 'ServiceInstanceJVMMemoryPool';
SRC_SERVICE_INSTANCE_JVM_GC: 'ServiceInstanceJVMGC';
SRC_SERVICE_INSTANCE_JVM_THREAD: 'ServiceInstanceJVMThread';
SRC_DATABASE_ACCESS: 'DatabaseAccess'; SRC_DATABASE_ACCESS: 'DatabaseAccess';
SRC_SERVICE_INSTANCE_CLR_CPU: 'ServiceInstanceCLRCPU'; SRC_SERVICE_INSTANCE_CLR_CPU: 'ServiceInstanceCLRCPU';
SRC_SERVICE_INSTANCE_CLR_GC: 'ServiceInstanceCLRGC'; SRC_SERVICE_INSTANCE_CLR_GC: 'ServiceInstanceCLRGC';

1
oap-server/oal-grammar/src/main/antlr4/org/apache/skywalking/oal/rt/grammar/OALParser.g4
View File

@ -53,6 +53,7 @@ source
: SRC_ALL | SRC_SERVICE | SRC_DATABASE_ACCESS | SRC_SERVICE_INSTANCE | SRC_ENDPOINT | : SRC_ALL | SRC_SERVICE | SRC_DATABASE_ACCESS | SRC_SERVICE_INSTANCE | SRC_ENDPOINT |
SRC_SERVICE_RELATION | SRC_SERVICE_INSTANCE_RELATION | SRC_ENDPOINT_RELATION | SRC_SERVICE_RELATION | SRC_SERVICE_INSTANCE_RELATION | SRC_ENDPOINT_RELATION |
SRC_SERVICE_INSTANCE_CLR_CPU | SRC_SERVICE_INSTANCE_CLR_GC | SRC_SERVICE_INSTANCE_CLR_THREAD | SRC_SERVICE_INSTANCE_CLR_CPU | SRC_SERVICE_INSTANCE_CLR_GC | SRC_SERVICE_INSTANCE_CLR_THREAD |
SRC_SERVICE_INSTANCE_JVM_CPU | SRC_SERVICE_INSTANCE_JVM_MEMORY | SRC_SERVICE_INSTANCE_JVM_MEMORY_POOL | SRC_SERVICE_INSTANCE_JVM_GC | SRC_SERVICE_INSTANCE_JVM_THREAD |// JVM source of service instance
SRC_ENVOY_INSTANCE_METRIC | SRC_ENVOY_INSTANCE_METRIC |
SRC_BROWSER_APP_PERF | SRC_BROWSER_APP_PAGE_PERF | SRC_BROWSER_APP_SINGLE_VERSION_PERF | SRC_BROWSER_APP_PERF | SRC_BROWSER_APP_PAGE_PERF | SRC_BROWSER_APP_SINGLE_VERSION_PERF |
SRC_BROWSER_APP_TRAFFIC | SRC_BROWSER_APP_PAGE_TRAFFIC | SRC_BROWSER_APP_SINGLE_VERSION_TRAFFIC | SRC_BROWSER_APP_TRAFFIC | SRC_BROWSER_APP_PAGE_TRAFFIC | SRC_BROWSER_APP_SINGLE_VERSION_TRAFFIC |

1
oap-server/server-bootstrap/pom.xml
View File

@ -285,7 +285,6 @@
<exclude>zabbix-rules/</exclude> <exclude>zabbix-rules/</exclude>
<exclude>lal/</exclude> <exclude>lal/</exclude>
<exclude>log-mal-rules/</exclude> <exclude>log-mal-rules/</exclude>
<exclude>jvm-metrics-rules/</exclude>
</excludes> </excludes>
</configuration> </configuration>
</plugin> </plugin>

61
oap-server/server-bootstrap/src/main/resources/jvm-metrics-rules/jvm.yaml
View File

@ -1,61 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
expSuffix: instance(['service'], ['instance'])
metricPrefix: instance_jvm
metricsRules:
- name: cpu
exp: sw_jvm_cpu
- name: memory_heap
exp: sw_jvm_memory.tagEqual('memory_type', 'used', 'heap_status', 'true').avg(['service', 'instance'])
- name: memory_noheap
exp: sw_jvm_memory.tagEqual('memory_type', 'used', 'heap_status', 'false').avg(['service', 'instance'])
- name: memory_heap_max
exp: sw_jvm_memory.tagEqual('memory_type', 'max', 'heap_status', 'true').avg(['service', 'instance'])
- name: memory_noheap_max
exp: sw_jvm_memory.tagEqual('memory_type', 'max', 'heap_status', 'false').avg(['service', 'instance'])
- name: young_gc_time
exp: sw_jvm_gc_time.tagEqual('gc_phrase', 'new').sum(['service', 'instance'])
- name: old_gc_time
exp: sw_jvm_gc_time.tagEqual('gc_phrase', 'old').sum(['service', 'instance'])
- name: young_gc_count
exp: sw_jvm_gc_count.tagEqual('gc_phrase', 'new').sum(['service', 'instance'])
- name: old_gc_count
exp: sw_jvm_gc_count.tagEqual('gc_phrase', 'old').sum(['service', 'instance'])
- name: thread_live_count
exp: sw_jvm_thread.tagEqual('thread_type', 'live').avg(['service', 'instance'])
- name: thread_daemon_count
exp: sw_jvm_thread.tagEqual('thread_type', 'daemon').avg(['service', 'instance'])
- name: thread_peak_count
exp: sw_jvm_thread.tagEqual('thread_type', 'peak').avg(['service', 'instance'])

31
oap-server/server-bootstrap/src/main/resources/oal/java-agent.oal Normal file
View File

@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
// JVM instance metrics
instance_jvm_cpu = from(ServiceInstanceJVMCPU.usePercent).doubleAvg();
instance_jvm_memory_heap = from(ServiceInstanceJVMMemory.used).filter(heapStatus == true).longAvg();
instance_jvm_memory_noheap = from(ServiceInstanceJVMMemory.used).filter(heapStatus == false).longAvg();
instance_jvm_memory_heap_max = from(ServiceInstanceJVMMemory.max).filter(heapStatus == true).longAvg();
instance_jvm_memory_noheap_max = from(ServiceInstanceJVMMemory.max).filter(heapStatus == false).longAvg();
instance_jvm_young_gc_time = from(ServiceInstanceJVMGC.time).filter(phrase == GCPhrase.NEW).sum();
instance_jvm_old_gc_time = from(ServiceInstanceJVMGC.time).filter(phrase == GCPhrase.OLD).sum();
instance_jvm_young_gc_count = from(ServiceInstanceJVMGC.count).filter(phrase == GCPhrase.NEW).sum();
instance_jvm_old_gc_count = from(ServiceInstanceJVMGC.count).filter(phrase == GCPhrase.OLD).sum();
instance_jvm_thread_live_count = from(ServiceInstanceJVMThread.liveCount).longAvg();
instance_jvm_thread_daemon_count = from(ServiceInstanceJVMThread.daemonCount).longAvg();
instance_jvm_thread_peak_count = from(ServiceInstanceJVMThread.peakCount).longAvg();

58
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/ServiceInstanceJVMCPU.java Normal file
View File

@ -0,0 +1,58 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.source;
import lombok.Getter;
import lombok.Setter;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_CATALOG_NAME;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_JVM_CPU;
@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_CPU, name = "ServiceInstanceJVMCPU", catalog = SERVICE_INSTANCE_CATALOG_NAME)
@ScopeDefaultColumn.VirtualColumnDefinition(fieldName = "entityId", columnName = "entity_id", isID = true, type = String.class)
public class ServiceInstanceJVMCPU extends Source {
@Override
public int scope() {
return DefaultScopeDefine.SERVICE_INSTANCE_JVM_CPU;
}
@Override
public String getEntityId() {
return String.valueOf(id);
}
@Getter
@Setter
private String id;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "name", requireDynamicActive = true)
private String name;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_name", requireDynamicActive = true)
private String serviceName;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_id")
private String serviceId;
@Getter
@Setter
private double usePercent;
}

64
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/ServiceInstanceJVMGC.java Normal file
View File

@ -0,0 +1,64 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.source;
import lombok.Getter;
import lombok.Setter;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_CATALOG_NAME;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_JVM_GC;
@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_GC, name = "ServiceInstanceJVMGC", catalog = SERVICE_INSTANCE_CATALOG_NAME)
@ScopeDefaultColumn.VirtualColumnDefinition(fieldName = "entityId", columnName = "entity_id", isID = true, type = String.class)
public class ServiceInstanceJVMGC extends Source {
@Override
public int scope() {
return DefaultScopeDefine.SERVICE_INSTANCE_JVM_GC;
}
@Override
public String getEntityId() {
return String.valueOf(id);
}
@Getter
@Setter
private String id;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "name", requireDynamicActive = true)
private String name;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_name", requireDynamicActive = true)
private String serviceName;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_id")
private String serviceId;
@Getter
@Setter
private GCPhrase phrase;
@Getter
@Setter
private long time;
@Getter
@Setter
private long count;
}

70
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/ServiceInstanceJVMMemory.java Normal file
View File

@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.source;
import lombok.Getter;
import lombok.Setter;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_CATALOG_NAME;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_JVM_MEMORY;
@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_MEMORY, name = "ServiceInstanceJVMMemory", catalog = SERVICE_INSTANCE_CATALOG_NAME)
@ScopeDefaultColumn.VirtualColumnDefinition(fieldName = "entityId", columnName = "entity_id", isID = true, type = String.class)
public class ServiceInstanceJVMMemory extends Source {
@Override
public int scope() {
return DefaultScopeDefine.SERVICE_INSTANCE_JVM_MEMORY;
}
@Override
public String getEntityId() {
return String.valueOf(id);
}
@Getter
@Setter
private String id;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "name", requireDynamicActive = true)
private String name;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_name", requireDynamicActive = true)
private String serviceName;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_id")
private String serviceId;
@Getter
@Setter
private boolean heapStatus;
@Getter
@Setter
private long init;
@Getter
@Setter
private long max;
@Getter
@Setter
private long used;
@Getter
@Setter
private long committed;
}

70
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/ServiceInstanceJVMMemoryPool.java Normal file
View File

@ -0,0 +1,70 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.source;
import lombok.Getter;
import lombok.Setter;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_CATALOG_NAME;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_JVM_MEMORY_POOL;
@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_MEMORY_POOL, name = "ServiceInstanceJVMMemoryPool", catalog = SERVICE_INSTANCE_CATALOG_NAME)
@ScopeDefaultColumn.VirtualColumnDefinition(fieldName = "entityId", columnName = "entity_id", isID = true, type = String.class)
public class ServiceInstanceJVMMemoryPool extends Source {
@Override
public int scope() {
return DefaultScopeDefine.SERVICE_INSTANCE_JVM_MEMORY_POOL;
}
@Override
public String getEntityId() {
return String.valueOf(id);
}
@Getter
@Setter
private String id;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "name", requireDynamicActive = true)
private String name;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_name", requireDynamicActive = true)
private String serviceName;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_id")
private String serviceId;
@Getter
@Setter
private MemoryPoolType poolType;
@Getter
@Setter
private long init;
@Getter
@Setter
private long max;
@Getter
@Setter
private long used;
@Getter
@Setter
private long committed;
}

64
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/source/ServiceInstanceJVMThread.java Normal file
View File

@ -0,0 +1,64 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.core.source;
import lombok.Getter;
import lombok.Setter;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_CATALOG_NAME;
import static org.apache.skywalking.oap.server.core.source.DefaultScopeDefine.SERVICE_INSTANCE_JVM_THREAD;
@ScopeDeclaration(id = SERVICE_INSTANCE_JVM_THREAD, name = "ServiceInstanceJVMThread", catalog = SERVICE_INSTANCE_CATALOG_NAME)
@ScopeDefaultColumn.VirtualColumnDefinition(fieldName = "entityId", columnName = "entity_id", isID = true, type = String.class)
public class ServiceInstanceJVMThread extends Source {
@Override
public int scope() {
return SERVICE_INSTANCE_JVM_THREAD;
}
@Override
public String getEntityId() {
return String.valueOf(id);
}
@Getter
@Setter
private String id;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "name", requireDynamicActive = true)
private String name;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_name", requireDynamicActive = true)
private String serviceName;
@Getter
@Setter
@ScopeDefaultColumn.DefinedByField(columnName = "service_id")
private String serviceId;
@Getter
@Setter
private long liveCount;
@Getter
@Setter
private long daemonCount;
@Getter
@Setter
private long peakCount;
}

12
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/KafkaFetcherProvider.java
View File

@ -20,8 +20,6 @@ package org.apache.skywalking.oap.server.analyzer.agent.kafka.provider;
import lombok.extern.slf4j.Slf4j; import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.log.analyzer.module.LogAnalyzerModule; import org.apache.skywalking.oap.log.analyzer.module.LogAnalyzerModule;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.KafkaFetcherHandlerRegister; import org.apache.skywalking.oap.server.analyzer.agent.kafka.KafkaFetcherHandlerRegister;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig; import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherModule; import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherModule;
@ -41,9 +39,6 @@ import org.apache.skywalking.oap.server.library.module.ModuleStartException;
import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException; import org.apache.skywalking.oap.server.library.module.ServiceNotProvidedException;
import org.apache.skywalking.oap.server.telemetry.TelemetryModule; import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
import java.util.Collections;
import java.util.List;
@Slf4j @Slf4j
public class KafkaFetcherProvider extends ModuleProvider { public class KafkaFetcherProvider extends ModuleProvider {
private KafkaFetcherHandlerRegister handlerRegister; private KafkaFetcherHandlerRegister handlerRegister;
@ -74,9 +69,8 @@ public class KafkaFetcherProvider extends ModuleProvider {
} }
@Override @Override
public void start() throws ServiceNotProvidedException, ModuleStartException { public void start() throws ServiceNotProvidedException {
List<Rule> rules = Rules.loadRules("jvm-metrics-rules", Collections.singletonList("jvm")); handlerRegister.register(new JVMMetricsHandler(getManager(), config));
handlerRegister.register(new JVMMetricsHandler(getManager(), config, rules));
handlerRegister.register(new ServiceManagementHandler(getManager(), config)); handlerRegister.register(new ServiceManagementHandler(getManager(), config));
handlerRegister.register(new TraceSegmentHandler(getManager(), config)); handlerRegister.register(new TraceSegmentHandler(getManager(), config));
handlerRegister.register(new ProfileTaskHandler(getManager(), config)); handlerRegister.register(new ProfileTaskHandler(getManager(), config));
@ -108,4 +102,4 @@ public class KafkaFetcherProvider extends ModuleProvider {
}; };
} }
} }

13
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/handler/JVMMetricsHandler.java
View File

@ -22,7 +22,6 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.kafka.clients.consumer.ConsumerRecord; import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.common.utils.Bytes; import org.apache.kafka.common.utils.Bytes;
import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricCollection; import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricCollection;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig; import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig;
import org.apache.skywalking.oap.server.analyzer.provider.jvm.JVMSourceDispatcher; import org.apache.skywalking.oap.server.analyzer.provider.jvm.JVMSourceDispatcher;
import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModule;
@ -35,8 +34,6 @@ import org.apache.skywalking.oap.server.telemetry.api.HistogramMetrics.Timer;
import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator; import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
import org.apache.skywalking.oap.server.telemetry.api.MetricsTag; import org.apache.skywalking.oap.server.telemetry.api.MetricsTag;
import java.util.List;
/** /**
* A handler deserializes the message of JVM Metrics and pushes it to downstream. * A handler deserializes the message of JVM Metrics and pushes it to downstream.
*/ */
@ -50,12 +47,12 @@ public class JVMMetricsHandler extends AbstractKafkaHandler {
private final HistogramMetrics histogramBatch; private final HistogramMetrics histogramBatch;
private final CounterMetrics errorCounter; private final CounterMetrics errorCounter;
public JVMMetricsHandler(ModuleManager manager, KafkaFetcherConfig config, List<Rule> rules) { public JVMMetricsHandler(ModuleManager manager, KafkaFetcherConfig config) {
super(manager, config); super(manager, config);
this.jvmSourceDispatcher = new JVMSourceDispatcher(manager, rules); this.jvmSourceDispatcher = new JVMSourceDispatcher(manager);
this.namingLengthControl = manager.find(CoreModule.NAME) this.namingLengthControl = manager.find(CoreModule.NAME)
.provider() .provider()
.getService(NamingControl.class); .getService(NamingControl.class);
MetricsCreator metricsCreator = manager.find(TelemetryModule.NAME) MetricsCreator metricsCreator = manager.find(TelemetryModule.NAME)
.provider() .provider()
.getService(MetricsCreator.class); .getService(MetricsCreator.class);
@ -112,4 +109,4 @@ public class JVMMetricsHandler extends AbstractKafkaHandler {
protected String getPlainTopic() { protected String getPlainTopic() {
return config.getTopicNameOfMetrics(); return config.getTopicNameOfMetrics();
} }
} }

31
oap-server/server-fetcher-plugin/kafka-fetcher-plugin/src/test/java/org/apache/skywalking/oap/server/analyzer/agent/kafka/provider/handler/JVMMetricsHandlerTest.java
View File

@ -19,8 +19,7 @@
package org.apache.skywalking.oap.server.analyzer.agent.kafka.provider.handler; package org.apache.skywalking.oap.server.analyzer.agent.kafka.provider.handler;
import com.google.common.collect.Lists; import com.google.common.collect.Lists;
import java.util.List;
import java.util.ArrayList;
import org.apache.kafka.clients.consumer.ConsumerRecord; import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.common.utils.Bytes; import org.apache.kafka.common.utils.Bytes;
import org.apache.skywalking.apm.network.common.v3.CPU; import org.apache.skywalking.apm.network.common.v3.CPU;
@ -32,6 +31,12 @@ import org.apache.skywalking.apm.network.language.agent.v3.MemoryPool;
import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.NamingControl; import org.apache.skywalking.oap.server.core.config.NamingControl;
import org.apache.skywalking.oap.server.core.config.group.EndpointNameGrouping; import org.apache.skywalking.oap.server.core.config.group.EndpointNameGrouping;
import org.apache.skywalking.oap.server.core.source.ISource;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMCPU;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMGC;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMMemory;
import org.apache.skywalking.oap.server.core.source.ServiceInstanceJVMMemoryPool;
import org.apache.skywalking.oap.server.core.source.SourceReceiver;
import org.apache.skywalking.oap.server.library.module.ModuleManager; import org.apache.skywalking.oap.server.library.module.ModuleManager;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig; import org.apache.skywalking.oap.server.analyzer.agent.kafka.module.KafkaFetcherConfig;
import org.apache.skywalking.oap.server.analyzer.agent.kafka.mock.MockModuleManager; import org.apache.skywalking.oap.server.analyzer.agent.kafka.mock.MockModuleManager;
@ -41,8 +46,11 @@ import org.apache.skywalking.oap.server.telemetry.api.MetricsCreator;
import org.apache.skywalking.oap.server.telemetry.none.MetricsCreatorNoop; import org.apache.skywalking.oap.server.telemetry.none.MetricsCreatorNoop;
import org.junit.Assert; import org.junit.Assert;
import org.junit.Before; import org.junit.Before;
import org.junit.ClassRule;
import org.junit.Test; import org.junit.Test;
import static org.hamcrest.CoreMatchers.is;
public class JVMMetricsHandlerTest { public class JVMMetricsHandlerTest {
private static final String TOPIC_NAME = "skywalking-metrics"; private static final String TOPIC_NAME = "skywalking-metrics";
private JVMMetricsHandler handler = null; private JVMMetricsHandler handler = null;
@ -50,6 +58,20 @@ public class JVMMetricsHandlerTest {
private ModuleManager manager; private ModuleManager manager;
@ClassRule
public static SourceReceiverRule SOURCE_RECEIVER = new SourceReceiverRule() {
@Override
protected void verify(final List<ISource> sourceList) throws Throwable {
Assert.assertTrue(sourceList.get(0) instanceof ServiceInstanceJVMCPU);
ServiceInstanceJVMCPU serviceInstanceJVMCPU = (ServiceInstanceJVMCPU) sourceList.get(0);
Assert.assertThat(serviceInstanceJVMCPU.getUsePercent(), is(1.0));
Assert.assertTrue(sourceList.get(1) instanceof ServiceInstanceJVMMemory);
Assert.assertTrue(sourceList.get(2) instanceof ServiceInstanceJVMMemoryPool);
Assert.assertTrue(sourceList.get(3) instanceof ServiceInstanceJVMGC);
}
};
@Before @Before
public void setup() { public void setup() {
manager = new MockModuleManager() { manager = new MockModuleManager() {
@ -60,6 +82,7 @@ public class JVMMetricsHandlerTest {
protected void register() { protected void register() {
registerServiceImplementation(NamingControl.class, new NamingControl( registerServiceImplementation(NamingControl.class, new NamingControl(
512, 512, 512, new EndpointNameGrouping())); 512, 512, 512, new EndpointNameGrouping()));
registerServiceImplementation(SourceReceiver.class, SOURCE_RECEIVER);
} }
}); });
register(TelemetryModule.NAME, () -> new MockModuleProvider() { register(TelemetryModule.NAME, () -> new MockModuleProvider() {
@ -70,7 +93,7 @@ public class JVMMetricsHandlerTest {
}); });
} }
}; };
handler = new JVMMetricsHandler(manager, config, new ArrayList<>()); handler = new JVMMetricsHandler(manager, config);
} }
@Test @Test
@ -97,4 +120,4 @@ public class JVMMetricsHandlerTest {
handler.handle(new ConsumerRecord<>(TOPIC_NAME, 0, 0, "", Bytes.wrap(metrics.toByteArray()))); handler.handle(new ConsumerRecord<>(TOPIC_NAME, 0, 0, "", Bytes.wrap(metrics.toByteArray())));
} }
} }

16
oap-server/server-receiver-plugin/skywalking-jvm-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/jvm/provider/JVMModuleProvider.java
View File

@ -18,9 +18,8 @@
package org.apache.skywalking.oap.server.receiver.jvm.provider; package org.apache.skywalking.oap.server.receiver.jvm.provider;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rules;
import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.oal.rt.OALEngineLoaderService;
import org.apache.skywalking.oap.server.core.server.GRPCHandlerRegister; import org.apache.skywalking.oap.server.core.server.GRPCHandlerRegister;
import org.apache.skywalking.oap.server.library.module.ModuleConfig; import org.apache.skywalking.oap.server.library.module.ModuleConfig;
import org.apache.skywalking.oap.server.library.module.ModuleDefine; import org.apache.skywalking.oap.server.library.module.ModuleDefine;
@ -31,9 +30,6 @@ import org.apache.skywalking.oap.server.receiver.jvm.provider.handler.JVMMetricR
import org.apache.skywalking.oap.server.receiver.jvm.provider.handler.JVMMetricReportServiceHandlerCompat; import org.apache.skywalking.oap.server.receiver.jvm.provider.handler.JVMMetricReportServiceHandlerCompat;
import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule; import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
import java.util.Collections;
import java.util.List;
public class JVMModuleProvider extends ModuleProvider { public class JVMModuleProvider extends ModuleProvider {
@Override @Override
@ -57,12 +53,16 @@ public class JVMModuleProvider extends ModuleProvider {
@Override @Override
public void start() throws ModuleStartException { public void start() throws ModuleStartException {
List<Rule> rules = Rules.loadRules("jvm-metrics-rules", Collections.singletonList("jvm")); // load official analysis
JVMMetricReportServiceHandler jvmMetricReportServiceHandler = new JVMMetricReportServiceHandler(getManager(), rules); getManager().find(CoreModule.NAME)
.provider()
.getService(OALEngineLoaderService.class)
.load(JVMOALDefine.INSTANCE);
GRPCHandlerRegister grpcHandlerRegister = getManager().find(SharingServerModule.NAME) GRPCHandlerRegister grpcHandlerRegister = getManager().find(SharingServerModule.NAME)
.provider() .provider()
.getService(GRPCHandlerRegister.class); .getService(GRPCHandlerRegister.class);
JVMMetricReportServiceHandler jvmMetricReportServiceHandler = new JVMMetricReportServiceHandler(getManager());
grpcHandlerRegister.addHandler(jvmMetricReportServiceHandler); grpcHandlerRegister.addHandler(jvmMetricReportServiceHandler);
grpcHandlerRegister.addHandler(new JVMMetricReportServiceHandlerCompat(jvmMetricReportServiceHandler)); grpcHandlerRegister.addHandler(new JVMMetricReportServiceHandlerCompat(jvmMetricReportServiceHandler));
} }
@ -79,4 +79,4 @@ public class JVMModuleProvider extends ModuleProvider {
SharingServerModule.NAME SharingServerModule.NAME
}; };
} }
} }

35
oap-server/server-receiver-plugin/skywalking-jvm-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/jvm/provider/JVMOALDefine.java Normal file
View File

@ -0,0 +1,35 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.server.receiver.jvm.provider;
import org.apache.skywalking.oap.server.core.oal.rt.OALDefine;
/**
* JVM OAl script includes the metrics related to JVM only.
*/
public class JVMOALDefine extends OALDefine {
public static final JVMOALDefine INSTANCE = new JVMOALDefine();
private JVMOALDefine() {
super(
"oal/java-agent.oal",
"org.apache.skywalking.oap.server.core.source"
);
}
}

13
oap-server/server-receiver-plugin/skywalking-jvm-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/jvm/provider/handler/JVMMetricReportServiceHandler.java
View File

@ -23,23 +23,19 @@ import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.apm.network.common.v3.Commands; import org.apache.skywalking.apm.network.common.v3.Commands;
import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricCollection; import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricCollection;
import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricReportServiceGrpc; import org.apache.skywalking.apm.network.language.agent.v3.JVMMetricReportServiceGrpc;
import org.apache.skywalking.oap.meter.analyzer.prometheus.rule.Rule;
import org.apache.skywalking.oap.server.analyzer.provider.jvm.JVMSourceDispatcher; import org.apache.skywalking.oap.server.analyzer.provider.jvm.JVMSourceDispatcher;
import org.apache.skywalking.oap.server.core.CoreModule; import org.apache.skywalking.oap.server.core.CoreModule;
import org.apache.skywalking.oap.server.core.config.NamingControl; import org.apache.skywalking.oap.server.core.config.NamingControl;
import org.apache.skywalking.oap.server.library.module.ModuleManager; import org.apache.skywalking.oap.server.library.module.ModuleManager;
import org.apache.skywalking.oap.server.library.server.grpc.GRPCHandler; import org.apache.skywalking.oap.server.library.server.grpc.GRPCHandler;
import java.util.List;
@Slf4j @Slf4j
public class JVMMetricReportServiceHandler extends JVMMetricReportServiceGrpc.JVMMetricReportServiceImplBase implements GRPCHandler { public class JVMMetricReportServiceHandler extends JVMMetricReportServiceGrpc.JVMMetricReportServiceImplBase implements GRPCHandler {
private final JVMSourceDispatcher jvmSourceDispatcher;
private final NamingControl namingControl; private final NamingControl namingControl;
private final JVMSourceDispatcher jvmSourceDispatcher; public JVMMetricReportServiceHandler(ModuleManager moduleManager) {
this.jvmSourceDispatcher = new JVMSourceDispatcher(moduleManager);
public JVMMetricReportServiceHandler(ModuleManager moduleManager, List<Rule> rules) {
this.jvmSourceDispatcher = new JVMSourceDispatcher(moduleManager, rules);
this.namingControl = moduleManager.find(CoreModule.NAME) this.namingControl = moduleManager.find(CoreModule.NAME)
.provider() .provider()
.getService(NamingControl.class); .getService(NamingControl.class);
@ -65,4 +61,5 @@ public class JVMMetricReportServiceHandler extends JVMMetricReportServiceGrpc.JV
responseObserver.onNext(Commands.newBuilder().build()); responseObserver.onNext(Commands.newBuilder().build());
responseObserver.onCompleted(); responseObserver.onCompleted();
} }
}
}