kazusa
/
skywalking-java
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Support Metric level function `serviceRelation` in `MAL` and binding envoy metrics into the topology. (#7399) 

* Support Metric level function `serviceRelation` in `MAL`.
* Support envoy metrics binding into the topology.
* add istio args
* remove istio 1.7

Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: Jared Tan <jian.tan@daocloud.io>
Co-authored-by: Zhenxu <kezhenxu94@apache.org>
This commit is contained in:
wankai123 2021-08-06 17:09:13 +08:00 committed by GitHub
parent b903bf17a0
commit 4e7d80fb88
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
26 changed files with 840 additions and 168 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
.github/workflows/e2e.istio.yaml vendored
View File

@ -184,7 +184,7 @@ jobs:
strategy:
fail-fast: true
matrix:
istio_version: [1.7.1, 1.8.2, 1.9.1]
istio_version: [1.8.2, 1.9.1]
env:
ISTIO_VERSION: ${{ matrix.istio_version }}
steps:
@ -231,6 +231,14 @@ jobs:
bash ${SCRIPTS_DIR}/istio.sh \
--set profile=demo \
--set meshConfig.defaultConfig.envoyMetricsService.address=skywalking-oap.istio-system:11800 \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[0]=.*membership_healthy.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[1]=.*upstream_cx_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[2]=.*upstream_cx_total.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[3]=.*upstream_rq_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[4]=.*upstream_rq_total.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[5]=.*upstream_rq_pending_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[6]=.*lb_healthy_panic.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[7]=.*upstream_cx_none_healthy.*' \
--set values.telemetry.v2.enabled=false # disable the metadata-exchange extension intentionally to make sure metrics service doesn't rely on it
- name: Install SkyWalking
@ -262,7 +270,10 @@ jobs:
- name: Deploy demo services
if: env.SKIP_CI != 'true'
run: bash ${SCRIPTS_DIR}/demo.sh
run: |
bash ${SCRIPTS_DIR}/demo.sh
kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/networking/destination-rule-all.yaml
kubectl apply -f https://raw.githubusercontent.com/istio/istio/$ISTIO_VERSION/samples/bookinfo/networking/virtual-service-all-v1.yaml
- name: Cluster Info
if: ${{ failure() }}

2
CHANGES.md
View File

@ -17,6 +17,8 @@ Release Notes.
1.21.
* kubernetes java client upgrade from 12.0.1 to 13.0.0
* Add `event` http receiver
* Support Metric level function `serviceRelation` in `MAL`.
* Support envoy metrics binding into the topology.
#### UI

4
docs/en/concepts-and-designs/mal.md
View File

@ -236,13 +236,15 @@ last_server_state_sync_time_in_seconds.tagEqual('production', 'catalog').downsam
## Metric level function
There are three levels in metric: service, instance and endpoint. They extract level relevant labels from metric labels, then informs the meter-system the level to which this metric belongs.
They extract level relevant labels from metric labels, then informs the meter-system the level to which this metric belongs.
- `servcie([svc_label1, svc_label2...])` extracts service level labels from the array argument.
- `instance([svc_label1, svc_label2...], [ins_label1, ins_label2...])` extracts service level labels from the first array argument,
extracts instance level labels from the second array argument.
- `endpoint([svc_label1, svc_label2...], [ep_label1, ep_label2...])` extracts service level labels from the first array argument,
extracts endpoint level labels from the second array argument.
- `serviceRelation(DetectPoint, [source_svc_label1...], [dest_svc_label1...])` DetectPoint including `DetectPoint.CLIENT` and `DetectPoint.SERVER`,
extracts `sourceService` labels from the first array argument, extracts `destService` labels from the second array argument.
## More Examples

23
docs/en/setup/envoy/metrics_service_setting.md
View File

@ -72,7 +72,7 @@ Under this circumstance, emitting the metrics to SyWalking is as simple as addin
```shell
istioctl install -y \
--set profile=demo `# replace the profile as per your need` \
--set profile=demo # replace the profile as per your need \
--set meshConfig.defaultConfig.envoyMetricsService.address=<skywalking.address.port.11800> \ # replace <skywalking.address.port.11800> with your actual SkyWalking OAP address
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[0]=.*'
```
@ -81,11 +81,30 @@ If you already have Istio installed, you can use the following command to apply
```shell
istioctl manifest install -y \
--set profile=demo `# replace the profile as per your need` \
--set profile=demo # replace the profile as per your need \
--set meshConfig.defaultConfig.envoyMetricsService.address=<skywalking.address.port.11800> \ # replace <skywalking.address.port.11800> with your actual SkyWalking OAP address
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[0]=.*'
```
Note:
`proxyStatsMatcher` only supported by `Istio 1.8+`.
We recommend using `inclusionRegexps` to reserve the specific metrics which need to analyze that can reduce Memory and CPU overhead.
For example, OAP used these metrics:
```shell
istioctl manifest install -y \
--set profile=demo # replace the profile as per your need \
--set meshConfig.defaultConfig.envoyMetricsService.address=<skywalking.address.port.11800> \ # replace <skywalking.address.port.11800> with your actual SkyWalking OAP address
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[0]=.*membership_healthy.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[1]=.*upstream_cx_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[2]=.*upstream_cx_total.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[3]=.*upstream_rq_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[4]=.*upstream_rq_total.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[5]=.*upstream_rq_pending_active.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[6]=.*lb_healthy_panic.*' \
--set 'meshConfig.defaultConfig.proxyStatsMatcher.inclusionRegexps[7]=.*upstream_cx_none_healthy.*'
```
# Metrics data
Some Envoy statistics are listed in this [list](https://www.envoyproxy.io/docs/envoy/v1.17.0/configuration/upstream/cluster_manager/cluster_stats#config-cluster-manager-cluster-stats). A sample data that contains identifier can be found [here](identify.json), while the metrics only can be observed [here](metrics.json).

52
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/Analyzer.java
View File

@ -39,6 +39,8 @@ import org.apache.skywalking.oap.server.core.analysis.NodeType;
import org.apache.skywalking.oap.server.core.analysis.TimeBucket;
import org.apache.skywalking.oap.server.core.analysis.manual.endpoint.EndpointTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.instance.InstanceTraffic;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationClientSideMetrics;
import org.apache.skywalking.oap.server.core.analysis.manual.relation.service.ServiceRelationServerSideMetrics;
import org.apache.skywalking.oap.server.core.analysis.manual.service.ServiceTraffic;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterEntity;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterSystem;
@ -240,11 +242,23 @@ public class Analyzer {
}
private void generateTraffic(MeterEntity entity) {
ServiceTraffic s = new ServiceTraffic();
s.setName(requireNonNull(entity.getServiceName()));
s.setNodeType(NodeType.Normal);
s.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
MetricsStreamProcessor.getInstance().in(s);
if (entity.getDetectPoint() != null) {
switch (entity.getDetectPoint()) {
case SERVER:
entity.setServiceName(entity.getDestServiceName());
toService(requireNonNull(entity.getDestServiceName()));
serverSide(entity);
break;
case CLIENT:
entity.setServiceName(entity.getSourceServiceName());
toService(requireNonNull(entity.getSourceServiceName()));
clientSide(entity);
break;
}
} else {
toService(requireNonNull(entity.getServiceName()));
}
if (!com.google.common.base.Strings.isNullOrEmpty(entity.getInstanceName())) {
InstanceTraffic instanceTraffic = new InstanceTraffic();
instanceTraffic.setName(entity.getInstanceName());
@ -261,4 +275,32 @@ public class Analyzer {
MetricsStreamProcessor.getInstance().in(endpointTraffic);
}
}
private void toService(String serviceName) {
ServiceTraffic s = new ServiceTraffic();
s.setName(requireNonNull(serviceName));
s.setNodeType(NodeType.Normal);
s.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
MetricsStreamProcessor.getInstance().in(s);
}
private void serverSide(MeterEntity entity) {
ServiceRelationServerSideMetrics metrics = new ServiceRelationServerSideMetrics();
metrics.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
metrics.setSourceServiceId(entity.sourceServiceId());
metrics.setDestServiceId(entity.destServiceId());
metrics.setComponentId(0);
metrics.setEntityId(entity.id());
MetricsStreamProcessor.getInstance().in(metrics);
}
private void clientSide(MeterEntity entity) {
ServiceRelationClientSideMetrics metrics = new ServiceRelationClientSideMetrics();
metrics.setTimeBucket(TimeBucket.getMinuteTimeBucket(System.currentTimeMillis()));
metrics.setSourceServiceId(entity.sourceServiceId());
metrics.setDestServiceId(entity.destServiceId());
metrics.setComponentId(0);
metrics.setEntityId(entity.id());
MetricsStreamProcessor.getInstance().in(metrics);
}
}

2
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/DSL.java
View File

@ -22,6 +22,7 @@ import groovy.lang.Binding;
import groovy.lang.GroovyShell;
import groovy.util.DelegatingScript;
import org.apache.skywalking.oap.meter.analyzer.dsl.tagOpt.K8sRetagType;
import org.apache.skywalking.oap.server.core.source.DetectPoint;
import org.codehaus.groovy.control.CompilerConfiguration;
import org.codehaus.groovy.control.customizers.ImportCustomizer;
@ -41,6 +42,7 @@ public final class DSL {
cc.setScriptBaseClass(DelegatingScript.class.getName());
ImportCustomizer icz = new ImportCustomizer();
icz.addImport("K8sRetagType", K8sRetagType.class.getName());
icz.addImport("DetectPoint", DetectPoint.class.getName());
cc.addCompilationCustomizers(icz);
GroovyShell sh = new GroovyShell(new Binding(), cc);
DelegatingScript script = (DelegatingScript) sh.parse(expression);

5
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/EntityDescription/EndpointEntityDescription.java
View File

@ -38,9 +38,4 @@ public class EndpointEntityDescription implements EntityDescription {
public List<String> getLabelKeys() {
return Stream.concat(this.serviceKeys.stream(), this.endpointKeys.stream()).collect(Collectors.toList());
}
@Override
public List<String> getInstanceKeys() {
throw new UnsupportedOperationException("Unsupported Operation of getInstanceKeys() " + this.toString());
}
}

6
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/EntityDescription/EntityDescription.java
View File

@ -25,10 +25,4 @@ public interface EntityDescription {
ScopeType getScopeType();
List<String> getLabelKeys();
List<String> getServiceKeys();
List<String> getInstanceKeys();
List<String> getEndpointKeys();
}

5
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/EntityDescription/InstanceEntityDescription.java
View File

@ -38,9 +38,4 @@ public class InstanceEntityDescription implements EntityDescription {
public List<String> getLabelKeys() {
return Stream.concat(this.serviceKeys.stream(), this.instanceKeys.stream()).collect(Collectors.toList());
}
@Override
public List<String> getEndpointKeys() {
throw new UnsupportedOperationException("Unsupported Operation of getEndpointKeys() " + this.toString());
}
}

10
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/EntityDescription/ServiceEntityDescription.java
View File

@ -35,14 +35,4 @@ public class ServiceEntityDescription implements EntityDescription {
public List<String> getLabelKeys() {
return serviceKeys;
}
@Override
public List<String> getInstanceKeys() {
throw new UnsupportedOperationException("Unsupported Operation of getInstanceKeys() " + this.toString());
}
@Override
public List<String> getEndpointKeys() {
throw new UnsupportedOperationException("Unsupported Operation of getEndpointKeys() " + this.toString());
}
}

43
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/EntityDescription/ServiceRelationEntityDescription.java Normal file
View File

@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription;
import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.ToString;
import org.apache.skywalking.oap.server.core.analysis.meter.ScopeType;
import org.apache.skywalking.oap.server.core.source.DetectPoint;
@Getter
@RequiredArgsConstructor
@ToString
public class ServiceRelationEntityDescription implements EntityDescription {
private final ScopeType scopeType = ScopeType.SERVICE_RELATION;
private final List<String> sourceServiceKeys;
private final List<String> destServiceKeys;
private final DetectPoint detectPoint;
@Override
public List<String> getLabelKeys() {
return Stream.concat(this.sourceServiceKeys.stream(), this.destServiceKeys.stream()).collect(Collectors.toList());
}
}

108
oap-server/analyzer/meter-analyzer/src/main/java/org/apache/skywalking/oap/meter/analyzer/dsl/SampleFamily.java
View File

@ -38,6 +38,7 @@ import org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription.EndpointEn
import org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription.EntityDescription;
import org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription.InstanceEntityDescription;
import org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription.ServiceEntityDescription;
import org.apache.skywalking.oap.meter.analyzer.dsl.EntityDescription.ServiceRelationEntityDescription;
import org.apache.skywalking.oap.meter.analyzer.dsl.tagOpt.K8sRetagType;
import org.apache.skywalking.oap.server.core.UnexpectedException;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterEntity;
@ -56,6 +57,7 @@ import java.util.function.DoubleBinaryOperator;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import org.apache.skywalking.oap.server.core.source.DetectPoint;
import static com.google.common.collect.ImmutableMap.toImmutableMap;
import static java.util.function.UnaryOperator.identity;
@ -213,7 +215,8 @@ public class SampleFamily {
}
if (by == null) {
double result = Arrays.stream(samples).mapToDouble(Sample::getValue).average().orElse(0.0D);
return SampleFamily.build(this.context, InternalOps.newSample(samples[0].name, ImmutableMap.of(), samples[0].timestamp, result));
return SampleFamily.build(
this.context, InternalOps.newSample(samples[0].name, ImmutableMap.of(), samples[0].timestamp, result));
}
return SampleFamily.build(
@ -238,7 +241,8 @@ public class SampleFamily {
}
if (by == null) {
double result = Arrays.stream(samples).mapToDouble(s -> s.value).reduce(aggregator).orElse(0.0D);
return SampleFamily.build(this.context, InternalOps.newSample(samples[0].name, ImmutableMap.of(), samples[0].timestamp, result));
return SampleFamily.build(
this.context, InternalOps.newSample(samples[0].name, ImmutableMap.of(), samples[0].timestamp, result));
}
return SampleFamily.build(
this.context,
@ -320,15 +324,19 @@ public class SampleFamily {
Object r = c.call(arg);
return sample.toBuilder()
.labels(
ImmutableMap.copyOf(Optional.ofNullable((r instanceof Map) ? (Map<String, String>) r : null)
.orElse(arg)))
ImmutableMap.copyOf(
Optional.ofNullable((r instanceof Map) ? (Map<String, String>) r : null)
.orElse(arg)))
.build();
}).toArray(Sample[]::new)
);
}
/* k8s retags*/
public SampleFamily retagByK8sMeta(String newLabelName, K8sRetagType type, String existingLabelName, String namespaceLabelName) {
public SampleFamily retagByK8sMeta(String newLabelName,
K8sRetagType type,
String existingLabelName,
String namespaceLabelName) {
Preconditions.checkArgument(!Strings.isNullOrEmpty(newLabelName));
Preconditions.checkArgument(!Strings.isNullOrEmpty(existingLabelName));
Preconditions.checkArgument(!Strings.isNullOrEmpty(namespaceLabelName));
@ -337,7 +345,8 @@ public class SampleFamily {
return EMPTY;
}
return SampleFamily.build(this.context, type.execute(samples, newLabelName, existingLabelName, namespaceLabelName));
return SampleFamily.build(
this.context, type.execute(samples, newLabelName, existingLabelName, namespaceLabelName));
}
public SampleFamily histogram() {
@ -368,8 +377,17 @@ public class SampleFamily {
double r = this.context.histogramType == HistogramType.ORDINARY ? s.value : s.value - pre.get();
pre.set(s.value);
ImmutableMap<String, String> ll = ImmutableMap.<String, String>builder()
.putAll(Maps.filterKeys(s.labels, key -> !Objects.equals(key, le)))
.put("le", String.valueOf((long) ((Double.parseDouble(this.context.histogramType == HistogramType.ORDINARY ? s.labels.get(le) : preLe.get())) * scale))).build();
.putAll(Maps.filterKeys(s.labels,
key -> !Objects.equals(
key, le)
))
.put(
"le", String.valueOf(
(long) ((Double.parseDouble(
this.context.histogramType == HistogramType.ORDINARY ? s.labels
.get(
le) : preLe.get())) * scale)))
.build();
preLe.set(s.labels.get(le));
return InternalOps.newSample(s.name, ll, s.timestamp, r);
})
@ -381,7 +399,8 @@ public class SampleFamily {
Preconditions.checkArgument(percentiles.size() > 0);
int[] p = percentiles.stream().mapToInt(i -> i).toArray();
ExpressionParsingContext.get().ifPresent(ctx -> {
Preconditions.checkState(ctx.isHistogram, "histogram() should be invoked before invoking histogram_percentile()");
Preconditions.checkState(
ctx.isHistogram, "histogram() should be invoked before invoking histogram_percentile()");
ctx.percentiles = p;
});
return this;
@ -402,7 +421,6 @@ public class SampleFamily {
public SampleFamily instance(List<String> serviceKeys, List<String> instanceKeys) {
Preconditions.checkArgument(serviceKeys.size() > 0);
Preconditions.checkArgument(instanceKeys.size() > 0);
ExpressionParsingContext.get().ifPresent(ctx -> ctx.scopeType = ScopeType.SERVICE_INSTANCE);
ExpressionParsingContext.get().ifPresent(ctx -> {
ctx.scopeType = ScopeType.SERVICE_INSTANCE;
ctx.scopeLabels.addAll(serviceKeys);
@ -417,7 +435,6 @@ public class SampleFamily {
public SampleFamily endpoint(List<String> serviceKeys, List<String> endpointKeys) {
Preconditions.checkArgument(serviceKeys.size() > 0);
Preconditions.checkArgument(endpointKeys.size() > 0);
ExpressionParsingContext.get().ifPresent(ctx -> ctx.scopeType = ScopeType.ENDPOINT);
ExpressionParsingContext.get().ifPresent(ctx -> {
ctx.scopeType = ScopeType.ENDPOINT;
ctx.scopeLabels.addAll(serviceKeys);
@ -429,10 +446,26 @@ public class SampleFamily {
return createMeterSamples(new EndpointEntityDescription(serviceKeys, endpointKeys));
}
public SampleFamily serviceRelation(DetectPoint detectPoint, List<String> sourceServiceKeys, List<String> destServiceKeys) {
Preconditions.checkArgument(sourceServiceKeys.size() > 0);
Preconditions.checkArgument(destServiceKeys.size() > 0);
ExpressionParsingContext.get().ifPresent(ctx -> {
ctx.scopeType = ScopeType.SERVICE_RELATION;
ctx.scopeLabels.addAll(sourceServiceKeys);
ctx.scopeLabels.addAll(destServiceKeys);
});
if (this == EMPTY) {
return EMPTY;
}
return createMeterSamples(new ServiceRelationEntityDescription(sourceServiceKeys, destServiceKeys, detectPoint));
}
private SampleFamily createMeterSamples(EntityDescription entityDescription) {
Map<MeterEntity, Sample[]> meterSamples = new HashMap<>();
Arrays.stream(samples)
.collect(groupingBy(it -> InternalOps.getLabels(entityDescription.getLabelKeys(), it), mapping(identity(), toList())))
.collect(groupingBy(it -> InternalOps.getLabels(entityDescription.getLabelKeys(), it),
mapping(identity(), toList())
))
.forEach((labels, samples) -> {
MeterEntity meterEntity = InternalOps.buildMeterEntity(samples, entityDescription);
meterSamples.put(meterEntity, InternalOps.left(samples, entityDescription.getLabelKeys()));
@ -452,7 +485,10 @@ public class SampleFamily {
Sample[] ss = Arrays.stream(samples)
.filter(sample -> ll.entrySet()
.stream()
.allMatch(entry -> op.apply(sample.labels.getOrDefault(entry.getKey(), ""), entry.getValue())))
.allMatch(
entry -> op.apply(sample.labels.getOrDefault(entry.getKey(), ""),
entry.getValue()
)))
.toArray(Sample[]::new);
return ss.length > 0 ? SampleFamily.build(this.context, ss) : EMPTY;
}
@ -480,7 +516,10 @@ public class SampleFamily {
Sample[] ss = Arrays.stream(samples)
.flatMap(cs -> io.vavr.collection.Stream.of(another.samples)
.find(as -> cs.labels.equals(as.labels))
.map(as -> cs.toBuilder().value(transform.apply(cs.value, as.value)))
.map(as -> cs.toBuilder()
.value(transform.apply(cs.value,
as.value
)))
.map(Sample.SampleBuilder::build)
.toJavaStream())
.toArray(Sample[]::new);
@ -506,9 +545,9 @@ public class SampleFamily {
static RunningContext instance() {
return RunningContext.builder()
.histogramType(HistogramType.CUMULATIVE)
.defaultHistogramBucketUnit(TimeUnit.SECONDS)
.build();
.histogramType(HistogramType.CUMULATIVE)
.defaultHistogramBucketUnit(TimeUnit.SECONDS)
.build();
}
private Map<MeterEntity, Sample[]> meterSamples = new HashMap<>();
@ -523,8 +562,10 @@ public class SampleFamily {
private static Sample[] left(List<Sample> samples, List<String> labelKeys) {
return samples.stream().map(s -> {
ImmutableMap<String, String> ll = ImmutableMap.<String, String>builder()
.putAll(Maps.filterKeys(s.labels, key -> !labelKeys.contains(key)))
.build();
.putAll(Maps.filterKeys(s.labels,
key -> !labelKeys.contains(key)
))
.build();
return s.toBuilder().labels(ll).build();
}).toArray(Sample[]::new);
}
@ -540,22 +581,37 @@ public class SampleFamily {
EntityDescription entityDescription) {
switch (entityDescription.getScopeType()) {
case SERVICE:
return MeterEntity.newService(InternalOps.dim(samples, entityDescription.getServiceKeys()));
ServiceEntityDescription serviceEntityDescription = (ServiceEntityDescription) entityDescription;
return MeterEntity.newService(InternalOps.dim(samples, serviceEntityDescription.getServiceKeys()));
case SERVICE_INSTANCE:
InstanceEntityDescription instanceEntityDescription = (InstanceEntityDescription) entityDescription;
return MeterEntity.newServiceInstance(
InternalOps.dim(samples, entityDescription.getServiceKeys()),
InternalOps.dim(samples, entityDescription.getInstanceKeys())
InternalOps.dim(samples, instanceEntityDescription.getServiceKeys()),
InternalOps.dim(samples, instanceEntityDescription.getInstanceKeys())
);
case ENDPOINT:
EndpointEntityDescription endpointEntityDescription = (EndpointEntityDescription) entityDescription;
return MeterEntity.newEndpoint(
InternalOps.dim(samples, entityDescription.getServiceKeys()),
InternalOps.dim(samples, entityDescription.getEndpointKeys())
InternalOps.dim(samples, endpointEntityDescription.getServiceKeys()),
InternalOps.dim(samples, endpointEntityDescription.getEndpointKeys())
);
default: throw new UnexpectedException("Unexpected scope type of entityDescription " + entityDescription.toString());
case SERVICE_RELATION:
ServiceRelationEntityDescription serviceRelationEntityDescription = (ServiceRelationEntityDescription) entityDescription;
return MeterEntity.newServiceRelation(
InternalOps.dim(samples, serviceRelationEntityDescription.getSourceServiceKeys()),
InternalOps.dim(samples, serviceRelationEntityDescription.getDestServiceKeys()),
serviceRelationEntityDescription.getDetectPoint()
);
default:
throw new UnexpectedException(
"Unexpected scope type of entityDescription " + entityDescription.toString());
}
}
private static Sample newSample(String name, ImmutableMap<String, String> labels, long timestamp, double newValue) {
private static Sample newSample(String name,
ImmutableMap<String, String> labels,
long timestamp,
double newValue) {
return Sample.builder()
.value(newValue)
.labels(labels)

320
oap-server/analyzer/meter-analyzer/src/test/java/org/apache/skywalking/oap/meter/analyzer/dsl/ScopeTest.java
View File

@ -25,6 +25,7 @@ import java.util.HashMap;
import java.util.Map;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.oap.server.core.analysis.meter.MeterEntity;
import org.apache.skywalking.oap.server.core.source.DetectPoint;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@ -60,10 +61,26 @@ public class ScopeTest {
"sum_service",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['idc']).service(['idc'])",
false,
@ -84,10 +101,26 @@ public class ScopeTest {
"sum_service_labels",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['region', 'idc']).service(['idc'])",
false,
@ -96,13 +129,24 @@ public class ScopeTest {
put(
MeterEntity.newService("t1"),
new Sample[] {
Sample.builder().labels(of("region", "")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("region", "us")).value(150).name("http_success_request").build()
Sample.builder()
.labels(of("region", ""))
.value(50)
.name("http_success_request").build(),
Sample.builder()
.labels(of("region", "us"))
.value(150)
.name("http_success_request").build()
}
);
put(
MeterEntity.newService("t3"),
new Sample[] {Sample.builder().labels(of("region", "cn")).value(54).name("http_success_request").build()}
new Sample[] {
Sample.builder()
.labels(of("region", "cn"))
.value(54)
.name("http_success_request").build()
}
);
}
}
@ -111,10 +155,26 @@ public class ScopeTest {
"sum_service_m",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['idc', 'region']).service(['idc' , 'region'])",
false,
@ -139,10 +199,26 @@ public class ScopeTest {
"sum_service_endpiont",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['region', 'idc']).endpoint(['idc'] , ['region'])",
false,
@ -168,10 +244,26 @@ public class ScopeTest {
"sum_service_endpiont_labels",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['region', 'idc' , 'instance']).endpoint(['idc'] , ['region'])",
false,
@ -180,20 +272,37 @@ public class ScopeTest {
put(
MeterEntity.newEndpoint("t1", "us"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(100).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(100)
.name("http_success_request").build()
}
);
put(
MeterEntity.newEndpoint("t3", "cn"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(51)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(3)
.name("http_success_request").build()
}
);
put(
MeterEntity.newEndpoint("t1", ""),
new Sample[] {Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build()}
new Sample[] {
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build()
}
);
}
}
@ -202,8 +311,16 @@ public class ScopeTest {
"sum_service_endpiont_labels_m",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "product")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "catalog")).value(50).name("http_success_request").build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "product"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "catalog"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "catalog", "instance", "10.0.0.1"))
.value(100)
@ -222,20 +339,37 @@ public class ScopeTest {
put(
MeterEntity.newEndpoint("t1", "us.catalog"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(100).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(100)
.name("http_success_request").build()
}
);
put(
MeterEntity.newEndpoint("t3", "cn.product"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(51)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(3)
.name("http_success_request").build()
}
);
put(
MeterEntity.newEndpoint("t1", ""),
new Sample[] {Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build()}
new Sample[] {
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build()
}
);
}
}
@ -244,10 +378,26 @@ public class ScopeTest {
"sum_service_instance",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['region', 'idc']).instance(['idc'] , ['region'])",
false,
@ -272,10 +422,26 @@ public class ScopeTest {
"sum_service_instance_labels",
of("http_success_request", SampleFamilyBuilder.newBuilder(
Sample.builder().labels(of("idc", "t1")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "svc", "catalog")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "svc", "product")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1")).value(100).name("http_success_request").build(),
Sample.builder().labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "svc", "catalog"))
.value(51)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "svc", "product"))
.value(50)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t1", "region", "us", "instance", "10.0.0.1"))
.value(100)
.name("http_success_request")
.build(),
Sample.builder()
.labels(of("idc", "t3", "region", "cn", "instance", "10.0.0.1"))
.value(3)
.name("http_success_request")
.build()
).build()),
"http_success_request.sum(['region', 'idc' , 'instance']).instance(['idc'] , ['region'])",
false,
@ -284,25 +450,81 @@ public class ScopeTest {
put(
MeterEntity.newServiceInstance("t1", "us"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(100).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(100)
.name("http_success_request").build()
}
);
put(
MeterEntity.newServiceInstance("t3", "cn"),
new Sample[] {
Sample.builder().labels(of("instance", "")).value(51).name("http_success_request").build(),
Sample.builder().labels(of("instance", "10.0.0.1")).value(3).name("http_success_request").build()
Sample.builder()
.labels(of("instance", ""))
.value(51)
.name("http_success_request").build(),
Sample.builder()
.labels(of("instance", "10.0.0.1"))
.value(3)
.name("http_success_request").build()
}
);
put(
MeterEntity.newServiceInstance("t1", ""),
new Sample[] {Sample.builder().labels(of("instance", "")).value(50).name("http_success_request").build()}
new Sample[] {
Sample.builder()
.labels(of("instance", ""))
.value(50)
.name("http_success_request").build()
}
);
}
}
},
});
{
"sum_service_relation",
of("envoy_cluster_metrics_up_cx_active", SampleFamilyBuilder.newBuilder(
Sample.builder()
.labels(of("app", "productpage", "cluster_name", "details"))
.value(11)
.name("envoy_cluster_metrics_up_cx_active")
.build(),
Sample.builder()
.labels(of("app", "productpage", "cluster_name", "reviews"))
.value(16)
.name("envoy_cluster_metrics_up_cx_active")
.build()
).build()),
"envoy_cluster_metrics_up_cx_active.sum(['app' ,'cluster_name']).serviceRelation(DetectPoint.CLIENT, ['app'], ['cluster_name'])",
false,
new HashMap<MeterEntity, Sample[]>() {
{
put(
MeterEntity.newServiceRelation("productpage", "details", DetectPoint.CLIENT),
new Sample[] {
Sample.builder()
.labels(of())
.value(11)
.name("envoy_cluster_metrics_up_cx_active").build()
}
);
put(
MeterEntity.newServiceRelation("productpage", "reviews", DetectPoint.CLIENT),
new Sample[] {
Sample.builder()
.labels(of())
.value(16)
.name("envoy_cluster_metrics_up_cx_active").build()
}
);
}
}
}
});
}
@Test

48
oap-server/server-bootstrap/src/main/resources/envoy-metrics-rules/envoy-svc-relation.yaml Normal file
View File

@ -0,0 +1,48 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
expSuffix: serviceRelation(DetectPoint.CLIENT, ['app'], ['cluster_name'])
metricPrefix: envoy_sr
metricsRules:
- name: cluster_up_cx_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_active').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app' ,'cluster_name'])
- name: cluster_up_cx_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_total').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app' , 'cluster_name']).increase('PT1M')
- name: cluster_up_rq_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_active').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app', 'cluster_name'])
- name: cluster_up_rq_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_total').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app' , 'cluster_name']).increase('PT1M')
- name: cluster_up_rq_pending_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_pending_active').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app' , 'cluster_name'])
- name: cluster_lb_healthy_panic_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+lb_healthy_panic').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app', 'cluster_name']).increase('PT1M')
- name: cluster_up_cx_none_healthy_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_none_healthy').tagMatch('metrics_name' , 'cluster.outbound.+').sum(['app', 'cluster_name']).increase('PT1M')

16
oap-server/server-bootstrap/src/main/resources/envoy-metrics-rules/envoy.yaml
View File

@ -61,17 +61,17 @@ metricsRules:
# envoy_cluster_metrics
- name: cluster_membership_healthy
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+membership_healthy').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').tagNotMatch('cluster_name' , '.+kube-system').sum(['app', 'instance' , 'cluster_name'])
- name: cluster_upstream_cx_active
- name: cluster_up_cx_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_active').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name'])
- name: cluster_upstream_cx_increase
- name: cluster_up_cx_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_total').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name']).increase('PT1M')
- name: cluster_upstream_rq_active
- name: cluster_up_rq_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_active').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name'])
- name: cluster_upstream_rq_increase
- name: cluster_up_rq_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_total').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name']).increase('PT1M')
- name: cluster_upstream_rq_pending_active
- name: cluster_up_rq_pending_active
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_rq_pending_active').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name'])
- name: cluster_lb_healthy_panic_increase
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+lb_healthy_panic').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name'])
- name: cluster_upstream_cx_none_healthy_increase
- name: cluster_lb_healthy_panic_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+lb_healthy_panic').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name']).increase('PT1M')
- name: cluster_up_cx_none_healthy_incr
exp: envoy_cluster_metrics.tagMatch('metrics_name' , '.+upstream_cx_none_healthy').tagMatch('metrics_name' , 'cluster.outbound.+|cluster.inbound.+').sum(['app', 'instance' , 'cluster_name']).increase('PT1M')

2
oap-server/server-bootstrap/src/main/resources/log4j2.xml
View File

@ -37,6 +37,8 @@
<logger name="org.apache.skywalking.oap.server.core.analysis.worker" level="DEBUG" />
<logger name="org.apache.skywalking.oap.server.core.remote.client" level="DEBUG"/>
<logger name="org.apache.skywalking.oap.server.library.buffer" level="INFO"/>
<logger name="org.apache.skywalking.oap.server.receiver.envoy.MetricServiceGRPCHandler" level="INFO"/>
<logger name="org.apache.skywalking.oap.meter.analyzer.prometheus.PrometheusMetricConverter" level="INFO"/>
<Root level="DEBUG">
<AppenderRef ref="Console"/>
</Root>

2
oap-server/server-bootstrap/src/main/resources/metadata-service-mapping.yaml
View File

@ -13,5 +13,5 @@
# See the License for the specific language governing permissions and
# limitations under the License.
serviceName: ${LABELS."service.istio.io/canonical-revision"}.${LABELS."service.istio.io/canonical-name",LABELS."app.kubernetes.io/name",LABELS.app}.${NAMESPACE}
serviceName: ${LABELS."service.istio.io/canonical-name",LABELS."app.kubernetes.io/name",LABELS.app}
serviceInstanceName: ${NAME}

34
oap-server/server-bootstrap/src/main/resources/ui-initialized-templates/istio-dp.yml
View File

@ -86,58 +86,58 @@ templates:
},
{
"width": "3",
"title": "Upstream CX Active",
"title": "Upstream Connection Active",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_cx_active",
"metricName": "envoy_cluster_up_cx_active",
"queryMetricType": "readLabeledMetricsValues",
"chartType": "ChartLine"
},
{
"width": "3",
"title": "Upstream CX Increase",
"title": "Upstream Connection Increase",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_cx_increase",
"metricName": "envoy_cluster_up_cx_incr",
"queryMetricType": "readLabeledMetricsValues",
"unit": "pm",
"unit": "Per Minute",
"chartType": "ChartLine"
},
{
"width": "3",
"title": "Upstream RQ Active",
"title": "Upstream Request Active",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_rq_active",
"metricName": "envoy_cluster_up_rq_active",
"queryMetricType": "readLabeledMetricsValues",
"chartType": "ChartLine"
},
{
"width": "3",
"title": "Upstream RQ Increase",
"title": "Upstream Request Increase",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_rq_increase",
"metricName": "envoy_cluster_up_rq_incr",
"queryMetricType": "readLabeledMetricsValues",
"unit": "pm",
"unit": "Per Minute",
"chartType": "ChartLine"
},
{
"width": "3",
"title": "Upstream RQ Pending Active",
"title": "Upstream Request Pending Active",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_rq_pending_active",
"metricName": "envoy_cluster_up_rq_pending_active",
"queryMetricType": "readLabeledMetricsValues",
"chartType": "ChartLine"
},
@ -148,21 +148,21 @@ templates:
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_lb_healthy_panic_increase",
"metricName": "envoy_cluster_lb_healthy_panic_incr",
"queryMetricType": "readLabeledMetricsValues",
"unit": "pm",
"unit": "Per Minute",
"chartType": "ChartLine"
},
{
"width": "3",
"title": "Upstream CX None Healthy Increase",
"title": "Upstream Connection None Healthy Increase",
"height": 350,
"entityType": "ServiceInstance",
"independentSelector": false,
"metricType": "LABELED_VALUE",
"metricName": "envoy_cluster_upstream_cx_none_healthy_increase",
"metricName": "envoy_cluster_up_cx_none_healthy_incr",
"queryMetricType": "readLabeledMetricsValues",
"unit": "pm",
"unit": "Per Minute",
"chartType": "ChartLine"
}
]

80
oap-server/server-bootstrap/src/main/resources/ui-initialized-templates/topology-service-relation.yml
View File

@ -243,6 +243,86 @@ templates:
"unit":"PPM - Packets Per Minutes"
}
]
},
"envoy_metrics":{
"server":[
],
"client":[
{
"width":12,
"title":"Upstream Connection Active",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_cx_active",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine"
},
{
"width":12,
"title":"Upstream Connection Increase",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_cx_incr",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine",
"unit":"Per Minute"
},
{
"width":12,
"title":"Upstream Request Active",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_rq_active",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine"
},
{
"width":12,
"title":"Upstream Request Increase",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_rq_incr",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine",
"unit":"Per Minute"
},
{
"width":12,
"title":"Upstream Request Pending Active",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_rq_pending_active",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine"
},
{
"width":12,
"title":"LB Healthy Panic Increase",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_lb_healthy_panic_incr",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine",
"unit":"Per Minute"
},
{
"width":12,
"title":"Upstream Connection None Healthy Increase",
"height":"200",
"entityType":"ServiceRelation",
"metricType":"REGULAR_VALUE",
"metricName":"envoy_sr_cluster_up_cx_none_healthy_incr",
"queryMetricType":"readMetricsValues",
"chartType":"ChartLine",
"unit":"Per Minute"
}
]
}
}
# Activated as the TOPOLOGY_INSTANCE type, makes this templates added into the UI page automatically.

7
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/manual/relation/service/ServiceRelationClientSideMetrics.java
View File

@ -68,6 +68,13 @@ public class ServiceRelationClientSideMetrics extends Metrics {
@Override
public boolean combine(Metrics metrics) {
ServiceRelationClientSideMetrics serviceRelationClientSideMetrics = (ServiceRelationClientSideMetrics) metrics;
if (this.getComponentId() == 0 && serviceRelationClientSideMetrics.getComponentId() != 0) {
this.componentId = serviceRelationClientSideMetrics.getComponentId();
}
if (this.getTimeBucket() > metrics.getTimeBucket()) {
this.setTimeBucket(metrics.getTimeBucket());
}
return true;
}

56
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/meter/MeterEntity.java
View File

@ -18,11 +18,14 @@
package org.apache.skywalking.oap.server.core.analysis.meter;
import lombok.Builder;
import lombok.EqualsAndHashCode;
import lombok.Getter;
import lombok.Setter;
import lombok.ToString;
import org.apache.skywalking.oap.server.core.UnexpectedException;
import org.apache.skywalking.oap.server.core.analysis.IDManager;
import org.apache.skywalking.oap.server.core.source.DetectPoint;
/**
* MeterEntity represents the entity in the meter system.
@ -30,21 +33,16 @@ import org.apache.skywalking.oap.server.core.analysis.IDManager;
@EqualsAndHashCode
@ToString
@Getter
@Builder(toBuilder = true)
public class MeterEntity {
private ScopeType scopeType;
@Setter
private String serviceName;
private String instanceName;
private String endpointName;
private MeterEntity(final ScopeType scopeType,
final String serviceName,
final String instanceName,
final String endpointName) {
this.scopeType = scopeType;
this.serviceName = serviceName;
this.instanceName = instanceName;
this.endpointName = endpointName;
}
private String sourceServiceName;
private String destServiceName;
private DetectPoint detectPoint;
public String id() {
switch (scopeType) {
@ -56,6 +54,11 @@ public class MeterEntity {
IDManager.ServiceID.buildId(serviceName, true), instanceName);
case ENDPOINT:
return IDManager.EndpointID.buildId(IDManager.ServiceID.buildId(serviceName, true), endpointName);
case SERVICE_RELATION:
return IDManager.ServiceID.buildRelationId(new IDManager.ServiceID.ServiceRelationDefine(
sourceServiceId(),
destServiceId()
));
default:
throw new UnexpectedException("Unexpected scope type of entity " + this.toString());
}
@ -65,24 +68,51 @@ public class MeterEntity {
return IDManager.ServiceID.buildId(serviceName, true);
}
public String sourceServiceId() {
return IDManager.ServiceID.buildId(sourceServiceName, true);
}
public String destServiceId() {
return IDManager.ServiceID.buildId(destServiceName, true);
}
/**
* Create a service level meter entity.
*/
public static MeterEntity newService(String serviceName) {
return new MeterEntity(ScopeType.SERVICE, serviceName, null, null);
return MeterEntity.builder().scopeType(ScopeType.SERVICE).serviceName(serviceName).build();
}
/**
* Create a service instance level meter entity.
*/
public static MeterEntity newServiceInstance(String serviceName, String serviceInstance) {
return new MeterEntity(ScopeType.SERVICE_INSTANCE, serviceName, serviceInstance, null);
return MeterEntity.builder()
.scopeType(ScopeType.SERVICE_INSTANCE)
.serviceName(serviceName)
.instanceName(serviceInstance)
.build();
}
/**
* Create an endpoint level meter entity.
*/
public static MeterEntity newEndpoint(String serviceName, String endpointName) {
return new MeterEntity(ScopeType.ENDPOINT, serviceName, null, endpointName);
return MeterEntity.builder()
.scopeType(ScopeType.ENDPOINT)
.serviceName(serviceName)
.endpointName(endpointName)
.build();
}
public static MeterEntity newServiceRelation(String sourceServiceName,
String destServiceName,
DetectPoint detectPoint) {
return MeterEntity.builder()
.scopeType(ScopeType.SERVICE_RELATION)
.sourceServiceName(sourceServiceName)
.destServiceName(destServiceName)
.detectPoint(detectPoint)
.build();
}
}

3
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/meter/ScopeType.java
View File

@ -24,7 +24,8 @@ import org.apache.skywalking.oap.server.core.source.DefaultScopeDefine;
public enum ScopeType {
SERVICE(DefaultScopeDefine.SERVICE),
SERVICE_INSTANCE(DefaultScopeDefine.SERVICE_INSTANCE),
ENDPOINT(DefaultScopeDefine.ENDPOINT);
ENDPOINT(DefaultScopeDefine.ENDPOINT),
SERVICE_RELATION(DefaultScopeDefine.SERVICE_RELATION);
@Getter
private final int scopeId;

2
oap-server/server-receiver-plugin/envoy-metrics-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/envoy/EnvoyMetricReceiverConfig.java
View File

@ -57,7 +57,7 @@ public class EnvoyMetricReceiverConfig extends ModuleConfig {
}
public List<Rule> rules() throws ModuleStartException {
return Rules.loadRules("envoy-metrics-rules", Collections.singletonList("envoy"));
return Rules.loadRules("envoy-metrics-rules", Arrays.asList("envoy", "envoy-svc-relation"));
}
public ServiceMetaInfoFactory serviceMetaInfoFactory() {

49
test/e2e/e2e-data/src/main/java/org/apache/skywalking/e2e/metrics/MetricsQuery.java
View File

@ -78,7 +78,7 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
public static String INSTANCE_JVM_THREAD_RUNNABLE_STATE_THREAD_COUNT = "instance_jvm_thread_runnable_state_thread_count";
public static String INSTANCE_JVM_CLASS_LOADED_CLASS_COUNT = "instance_jvm_class_loaded_class_count";
public static String INSTANCE_JVM_CLASS_TOTAL_LOADED_CLASS_COUNT = "instance_jvm_class_total_loaded_class_count";
public static String [] ALL_INSTANCE_JVM_METRICS = {
public static String[] ALL_INSTANCE_JVM_METRICS = {
INSTANCE_JVM_CLASS_TOTAL_LOADED_CLASS_COUNT,
INSTANCE_JVM_CLASS_LOADED_CLASS_COUNT,
INSTANCE_JVM_THREAD_RUNNABLE_STATE_THREAD_COUNT,
@ -88,16 +88,16 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
INSTANCE_JVM_MEMORY_NOHEAP,
INSTANCE_JVM_MEMORY_HEAP_MAX,
INSTANCE_JVM_MEMORY_HEAP,
};
};
public static String [] ALL_INSTANCE_JVM_METRICS_COMPAT = {
public static String[] ALL_INSTANCE_JVM_METRICS_COMPAT = {
INSTANCE_JVM_THREAD_LIVE_COUNT,
INSTANCE_JVM_THREAD_DAEMON_COUNT,
INSTANCE_JVM_THREAD_PEAK_COUNT,
INSTANCE_JVM_MEMORY_NOHEAP,
INSTANCE_JVM_MEMORY_HEAP_MAX,
INSTANCE_JVM_MEMORY_HEAP,
};
};
public static String SERVICE_RELATION_CLIENT_CPM = "service_relation_client_cpm";
public static String SERVICE_RELATION_SERVER_CPM = "service_relation_server_cpm";
@ -110,6 +110,9 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
public static String[] ALL_SERVICE_RELATION_CLIENT_METRICS = {
SERVICE_RELATION_CLIENT_CPM
};
public static String[] ENVOY_METRICS_SERVICE_RELATION_CLIENT_METRICS = {
"envoy_sr_cluster_up_cx_active"
};
public static String[] ALL_SERVICE_RELATION_SERVER_METRICS = {
SERVICE_RELATION_SERVER_CPM
@ -161,11 +164,18 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
"envoy_memory_allocated_max",
"envoy_memory_physical_size",
"envoy_memory_physical_size_max",
"envoy_total_connections_used",
"envoy_worker_threads",
"envoy_worker_threads_max"
};
public static Map<String, List<String>> ALL_ENVOY_LABELED_METRICS =
ImmutableMap.<String, List<String>>builder()
.put(
"envoy_cluster_membership_healthy",
Arrays.asList("e2e::details")
)
.build();
public static String[] ALL_SO11Y_LABELED_METRICS = {
// Nothing to check for now.
};
@ -182,10 +192,19 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
};
public static Map<String, List<String>> SIMPLE_ZABBIX_METERS = ImmutableMap.<String, List<String>>builder()
.put("meter_agent_system_cpu_util", Arrays.asList("idle"))
.put("meter_agent_vm_memory_size", Arrays.asList("total"))
.put("meter_agent_vfs_fs_size", Arrays.asList("/-total"))
.build();
.put(
"meter_agent_system_cpu_util",
Arrays.asList("idle")
)
.put(
"meter_agent_vm_memory_size",
Arrays.asList("total")
)
.put(
"meter_agent_vfs_fs_size",
Arrays.asList("/-total")
)
.build();
public static String[] SIMPLE_PROM_VM_METERS = {
"meter_vm_memory_used",
@ -201,8 +220,14 @@ public class MetricsQuery extends AbstractQuery<MetricsQuery> {
};
public static Map<String, List<String>> SIMPLE_PROM_VM_LABELED_METERS = ImmutableMap.<String, List<String>>builder()
.put("meter_vm_cpu_average_used", Arrays.asList("idle"))
.put("meter_vm_filesystem_percentage", Arrays.asList("/etc/hosts"))
.build();
.put(
"meter_vm_cpu_average_used",
Arrays.asList("idle")
)
.put(
"meter_vm_filesystem_percentage",
Arrays.asList("/etc/hosts")
)
.build();
}

75
test/e2e/e2e-test/src/test/java/org/apache/skywalking/e2e/mesh/MetricsServiceE2E.java
View File

@ -21,6 +21,7 @@ package org.apache.skywalking.e2e.mesh;
import com.google.common.base.Strings;
import java.net.URL;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;
import lombok.extern.slf4j.Slf4j;
@ -28,7 +29,9 @@ import org.apache.skywalking.e2e.base.SkyWalkingTestAdapter;
import org.apache.skywalking.e2e.base.TrafficController;
import org.apache.skywalking.e2e.common.HostAndPort;
import org.apache.skywalking.e2e.metrics.AtLeastOneOfMetricsMatcher;
import org.apache.skywalking.e2e.metrics.Metrics;
import org.apache.skywalking.e2e.metrics.MetricsValueMatcher;
import org.apache.skywalking.e2e.metrics.ReadLabeledMetricsQuery;
import org.apache.skywalking.e2e.metrics.ReadMetrics;
import org.apache.skywalking.e2e.metrics.ReadMetricsQuery;
import org.apache.skywalking.e2e.retryable.RetryableTest;
@ -39,10 +42,17 @@ import org.apache.skywalking.e2e.service.instance.Instance;
import org.apache.skywalking.e2e.service.instance.Instances;
import org.apache.skywalking.e2e.service.instance.InstancesMatcher;
import org.apache.skywalking.e2e.service.instance.InstancesQuery;
import org.apache.skywalking.e2e.topo.Call;
import org.apache.skywalking.e2e.topo.TopoMatcher;
import org.apache.skywalking.e2e.topo.TopoQuery;
import org.apache.skywalking.e2e.topo.Topology;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.TestInstance;
import static org.apache.skywalking.e2e.metrics.MetricsMatcher.verifyMetrics;
import static org.apache.skywalking.e2e.metrics.MetricsQuery.ALL_ENVOY_LABELED_METRICS;
import static org.apache.skywalking.e2e.metrics.MetricsQuery.ALL_ENVOY_LINER_METRICS;
import static org.apache.skywalking.e2e.metrics.MetricsQuery.ENVOY_METRICS_SERVICE_RELATION_CLIENT_METRICS;
import static org.apache.skywalking.e2e.utils.Times.now;
import static org.apache.skywalking.e2e.utils.Yamls.exists;
import static org.apache.skywalking.e2e.utils.Yamls.load;
@ -50,9 +60,11 @@ import static org.apache.skywalking.e2e.utils.Yamls.load;
@Slf4j
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
public class MetricsServiceE2E extends SkyWalkingTestAdapter {
private final String swWebappHost = Optional.ofNullable(Strings.emptyToNull(System.getenv("WEBAPP_HOST"))).orElse("127.0.0.1");
private final String swWebappHost = Optional.ofNullable(Strings.emptyToNull(System.getenv("WEBAPP_HOST")))
.orElse("127.0.0.1");
private final String swWebappPort = Optional.ofNullable(Strings.emptyToNull(System.getenv("WEBAPP_PORT"))).orElse("12800");
private final String swWebappPort = Optional.ofNullable(Strings.emptyToNull(System.getenv("WEBAPP_PORT")))
.orElse("12800");
protected HostAndPort swWebappHostPort = HostAndPort.builder()
.host(swWebappHost)
@ -65,8 +77,10 @@ public class MetricsServiceE2E extends SkyWalkingTestAdapter {
queryClient(swWebappHostPort);
String gatewayHost = Strings.isNullOrEmpty(System.getenv("GATEWAY_HOST")) ? "127.0.0.1" : System.getenv("GATEWAY_HOST");
String gatewayPort = Strings.isNullOrEmpty(System.getenv("GATEWAY_PORT")) ? "80" : System.getenv("GATEWAY_PORT");
String gatewayHost = Strings.isNullOrEmpty(System.getenv("GATEWAY_HOST")) ? "127.0.0.1" : System.getenv(
"GATEWAY_HOST");
String gatewayPort = Strings.isNullOrEmpty(System.getenv("GATEWAY_PORT")) ? "80" : System.getenv(
"GATEWAY_PORT");
HostAndPort serviceHostPort = HostAndPort.builder()
.host(gatewayHost)
@ -124,6 +138,59 @@ public class MetricsServiceE2E extends SkyWalkingTestAdapter {
instanceRespTimeMatcher.setValue(greaterThanZero);
instanceRespTimeMatcher.verify(instanceMetrics.getValues());
}
for (Map.Entry<String, List<String>> entry : ALL_ENVOY_LABELED_METRICS.entrySet()) {
String metricsName = entry.getKey();
List<String> labels = entry.getValue();
LOGGER.info("verifying envoy labeledMetrics: {}", metricsName);
List<ReadMetrics> labeledMetrics = graphql.readLabeledMetrics(
new ReadLabeledMetricsQuery().stepByMinute()
.metricsName(metricsName)
.serviceName(service.getLabel())
.scope("ServiceInstance")
.instanceName(instance.getLabel())
.labels(labels)
);
LOGGER.info("envoy labeledMetrics: {}", labeledMetrics);
Metrics allValues = new Metrics();
for (ReadMetrics readMetrics : labeledMetrics) {
allValues.getValues().addAll(readMetrics.getValues().getValues());
}
final AtLeastOneOfMetricsMatcher instanceRespTimeMatcher = new AtLeastOneOfMetricsMatcher();
final MetricsValueMatcher greaterThanZero = new MetricsValueMatcher();
greaterThanZero.setValue("gt 0");
instanceRespTimeMatcher.setValue(greaterThanZero);
instanceRespTimeMatcher.verify(allValues);
}
}
}
}
@RetryableTest
void topology() throws Exception {
LOGGER.info("topology starts {} {}", graphql, startTime);
final Topology topology = graphql.topo(new TopoQuery().stepByMinute().start(startTime.minusDays(1)).end(now()));
LOGGER.info("topology: {}", topology);
load("expected/metricsservice/topo.yml").as(TopoMatcher.class).verify(topology);
verifyServiceRelationMetrics(topology.getCalls());
}
private void verifyServiceRelationMetrics(final List<Call> calls) throws Exception {
verifyRelationMetrics(calls, ENVOY_METRICS_SERVICE_RELATION_CLIENT_METRICS);
}
private void verifyRelationMetrics(final List<Call> calls,
final String[] relationClientMetrics) throws Exception {
for (Call call : calls) {
for (String detectPoint : call.getDetectPoints()) {
for (String metricName : relationClientMetrics) {
verifyMetrics(graphql, metricName, call.getId(), startTime);
}
}
}
}

39
test/e2e/e2e-test/src/test/resources/expected/metricsservice/topo.yml Normal file
View File

@ -0,0 +1,39 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
nodes:
- id: not null
name: e2e::reviews
type: null
isReal: true
- id: not null
name: e2e::productpage
isReal: true
- id: not null
name: e2e::details
type: null
isReal: true
calls:
- id: not null
source: ZTJlOjpwcm9kdWN0cGFnZQ==.1
target: ZTJlOjpyZXZpZXdz.1
detectPoints:
- CLIENT
- id: not null
source: ZTJlOjpwcm9kdWN0cGFnZQ==.1
target: ZTJlOjpkZXRhaWxz.1
detectPoints:
- CLIENT