diff --git a/.travis.yml b/.travis.yml index 40ba9d4d5..540b9e8f0 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,8 +6,7 @@ services: language: java install: - jdk_switcher use oraclejdk8 - - mvn clean install + - mvn clean install --quiet jacoco:report coveralls:report after_success: - - mvn clean test jacoco:report coveralls:report - bash ./travis/push_image.sh diff --git a/README.md b/README.md index 31e607ea9..4943df313 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ Sky Walking | [中文](README_ZH.md) This project adheres to the Contributor Covenant [code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to wu.sheng@foxmail.com. # Screenshots -- Topological graph of application clusters based on dubbox and [motan](https://github.com/weibocom/motan). +- Discovery topological graph of application clusters automatically. - Trace query. diff --git a/README_ZH.md b/README_ZH.md index f5b9c8c25..3d92f9357 100644 --- a/README_ZH.md +++ b/README_ZH.md @@ -19,7 +19,7 @@ Sky Walking | [English](README.md) * 使用 [**@Trace**](docs/cn/Application-toolkit-trace-CN.md) 标注追踪业务方法 * 将 traceId 集成到 log4j, log4j2 或 logback这些日志组件中 * 纯Java后端Collector实现，提供RESTful和gRPC接口。兼容接受其他语言探针发送数据 - * [如何将探针的Metric和Trace数据上传到Collector？]() + * [如何将探针的Metric和Trace数据上传到Collector？](/docs/cn/How-to-communicate-with-the-collector-CN.md) * UI工程请查看 [skywalking-ui](https://github.com/OpenSkywalking/skywalking-ui) * 中文QQ群：392443393 @@ -33,7 +33,7 @@ Sky Walking | [English](README.md) This project adheres to the Contributor Covenant [code of conduct](CODE_OF_CONDUCT.md). By participating, you are expected to uphold this code. Please report unacceptable behavior to wu.sheng@foxmail.com. # Screenshots -- 追踪基于 dubbox 和 [motan](https://github.com/weibocom/motan)的分布式系统，生成的拓扑截图 +- 分布式系统拓扑图自动发现 - 调用链查询 diff --git a/apm-application-toolkit/apm-toolkit-log4j-1.x/pom.xml b/apm-application-toolkit/apm-toolkit-log4j-1.x/pom.xml index a8e36e3eb..1da25be41 100644 --- a/apm-application-toolkit/apm-toolkit-log4j-1.x/pom.xml +++ b/apm-application-toolkit/apm-toolkit-log4j-1.x/pom.xml @@ -23,7 +23,7 @@ apm-application-toolkit org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -49,6 +49,7 @@ attach-sources + none jar diff --git a/apm-application-toolkit/apm-toolkit-log4j-2.x/pom.xml b/apm-application-toolkit/apm-toolkit-log4j-2.x/pom.xml index 27b2839b4..86e16fa3b 100644 --- a/apm-application-toolkit/apm-toolkit-log4j-2.x/pom.xml +++ b/apm-application-toolkit/apm-toolkit-log4j-2.x/pom.xml @@ -23,7 +23,7 @@ apm-application-toolkit org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -49,6 +49,7 @@ attach-sources + none jar diff --git a/apm-application-toolkit/apm-toolkit-logback-1.x/pom.xml b/apm-application-toolkit/apm-toolkit-logback-1.x/pom.xml index 3cc15452b..94da54eb3 100644 --- a/apm-application-toolkit/apm-toolkit-logback-1.x/pom.xml +++ b/apm-application-toolkit/apm-toolkit-logback-1.x/pom.xml @@ -23,7 +23,7 @@ apm-application-toolkit org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -49,6 +49,7 @@ attach-sources + none jar diff --git a/apm-application-toolkit/apm-toolkit-opentracing/pom.xml b/apm-application-toolkit/apm-toolkit-opentracing/pom.xml index 825d2522e..3d69a33cd 100644 --- a/apm-application-toolkit/apm-toolkit-opentracing/pom.xml +++ b/apm-application-toolkit/apm-toolkit-opentracing/pom.xml @@ -23,7 +23,7 @@ apm-application-toolkit org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-application-toolkit/apm-toolkit-trace/pom.xml b/apm-application-toolkit/apm-toolkit-trace/pom.xml index 36ef9349c..024864a3e 100644 --- a/apm-application-toolkit/apm-toolkit-trace/pom.xml +++ b/apm-application-toolkit/apm-toolkit-trace/pom.xml @@ -22,7 +22,7 @@ apm-application-toolkit org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -41,6 +41,7 @@ attach-sources + none jar diff --git a/apm-application-toolkit/pom.xml b/apm-application-toolkit/pom.xml index ca827e440..b545efacb 100644 --- a/apm-application-toolkit/pom.xml +++ b/apm-application-toolkit/pom.xml @@ -21,7 +21,7 @@ apm org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 apm-application-toolkit diff --git a/apm-collector/apm-collector-agent/collector-agent-define/pom.xml b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/pom.xml similarity index 87% rename from apm-collector/apm-collector-agent/collector-agent-define/pom.xml rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/pom.xml index 1bc2c4eee..b5504230c 100644 --- a/apm-collector/apm-collector-agent/collector-agent-define/pom.xml +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/pom.xml @@ -21,13 +21,13 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-agent + apm-collector-agent-grpc org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 - collector-agent-define + collector-agent-grpc-define jar - + \ No newline at end of file diff --git a/apm-collector/apm-collector-agent/collector-agent-define/src/main/java/org/skywalking/apm/collector/agent/AgentModule.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentGRPCModule.java similarity index 86% rename from apm-collector/apm-collector-agent/collector-agent-define/src/main/java/org/skywalking/apm/collector/agent/AgentModule.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentGRPCModule.java index 202a5554b..7e574016a 100644 --- a/apm-collector/apm-collector-agent/collector-agent-define/src/main/java/org/skywalking/apm/collector/agent/AgentModule.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentGRPCModule.java @@ -16,16 +16,16 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.agent; +package org.skywalking.apm.collector.agent.grpc; import org.skywalking.apm.collector.core.module.Module; /** * @author peng-yongsheng */ -public class AgentModule extends Module { +public class AgentGRPCModule extends Module { - public static final String NAME = "agent"; + public static final String NAME = "agent_gRPC"; @Override public String name() { return NAME; diff --git a/apm-collector/apm-collector-stream/collector-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module similarity index 92% rename from apm-collector/apm-collector-stream/collector-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module index 468c08d65..d6fe0707c 100644 --- a/apm-collector/apm-collector-stream/collector-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module @@ -16,4 +16,4 @@ # Project repository: https://github.com/OpenSkywalking/skywalking # -org.skywalking.apm.collector.stream.StreamModule \ No newline at end of file +org.skywalking.apm.collector.agent.grpc.AgentGRPCModule \ No newline at end of file diff --git a/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/pom.xml b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/pom.xml new file mode 100644 index 000000000..da92a7319 --- /dev/null +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/pom.xml @@ -0,0 +1,60 @@ + + + + + + apm-collector-agent-grpc + org.skywalking + 3.3.0-2017 + + 4.0.0 + + collector-agent-grpc-provider + jar + + + + org.skywalking + collector-agent-grpc-define + ${project.version} + + + org.skywalking + collector-grpc-manager-define + ${project.version} + + + org.skywalking + collector-cluster-define + ${project.version} + + + org.skywalking + collector-naming-define + ${project.version} + + + org.skywalking + collector-agent-stream-define + ${project.version} + + + diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java similarity index 85% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java index 863016d12..d45d38228 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCProvider.java @@ -19,7 +19,6 @@ package org.skywalking.apm.collector.agent.grpc; import java.util.Properties; -import org.skywalking.apm.collector.agent.AgentModule; import org.skywalking.apm.collector.agent.grpc.handler.ApplicationRegisterServiceHandler; import org.skywalking.apm.collector.agent.grpc.handler.InstanceDiscoveryServiceHandler; import org.skywalking.apm.collector.agent.grpc.handler.JVMMetricsServiceHandler; @@ -27,8 +26,7 @@ import org.skywalking.apm.collector.agent.grpc.handler.ServiceNameDiscoveryServi import org.skywalking.apm.collector.agent.grpc.handler.TraceSegmentServiceHandler; import org.skywalking.apm.collector.agent.grpc.handler.naming.AgentGRPCNamingHandler; import org.skywalking.apm.collector.agent.grpc.handler.naming.AgentGRPCNamingListener; -import org.skywalking.apm.collector.agent.stream.AgentStreamSingleton; -import org.skywalking.apm.collector.cache.CacheModule; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; import org.skywalking.apm.collector.cluster.ClusterModule; import org.skywalking.apm.collector.cluster.service.ModuleListenerService; import org.skywalking.apm.collector.cluster.service.ModuleRegisterService; @@ -39,10 +37,7 @@ import org.skywalking.apm.collector.grpc.manager.GRPCManagerModule; import org.skywalking.apm.collector.grpc.manager.service.GRPCManagerService; import org.skywalking.apm.collector.naming.NamingModule; import org.skywalking.apm.collector.naming.service.NamingHandlerRegisterService; -import org.skywalking.apm.collector.remote.RemoteModule; import org.skywalking.apm.collector.server.Server; -import org.skywalking.apm.collector.storage.StorageModule; -import org.skywalking.apm.collector.stream.StreamModule; /** * @author peng-yongsheng @@ -58,7 +53,7 @@ public class AgentModuleGRPCProvider extends ModuleProvider { } @Override public Class module() { - return AgentModule.class; + return AgentGRPCModule.class; } @Override public void prepare(Properties config) throws ServiceNotProvidedException { @@ -70,7 +65,7 @@ public class AgentModuleGRPCProvider extends ModuleProvider { Integer port = (Integer)config.get(PORT); ModuleRegisterService moduleRegisterService = getManager().find(ClusterModule.NAME).getService(ModuleRegisterService.class); - moduleRegisterService.register(AgentModule.NAME, this.name(), new AgentModuleGRPCRegistration(host, port)); + moduleRegisterService.register(AgentGRPCModule.NAME, this.name(), new AgentModuleGRPCRegistration(host, port)); AgentGRPCNamingListener namingListener = new AgentGRPCNamingListener(); ModuleListenerService moduleListenerService = getManager().find(ClusterModule.NAME).getService(ModuleListenerService.class); @@ -82,7 +77,6 @@ public class AgentModuleGRPCProvider extends ModuleProvider { GRPCManagerService managerService = getManager().find(GRPCManagerModule.NAME).getService(GRPCManagerService.class); Server gRPCServer = managerService.createIfAbsent(host, port); - AgentStreamSingleton.createInstanceIfAbsent(getManager()); addHandlers(gRPCServer); } @@ -91,14 +85,14 @@ public class AgentModuleGRPCProvider extends ModuleProvider { } @Override public String[] requiredModules() { - return new String[] {ClusterModule.NAME, NamingModule.NAME, StorageModule.NAME, GRPCManagerModule.NAME, CacheModule.NAME, RemoteModule.NAME, StreamModule.NAME}; + return new String[] {ClusterModule.NAME, NamingModule.NAME, GRPCManagerModule.NAME, AgentStreamModule.NAME}; } private void addHandlers(Server gRPCServer) { gRPCServer.addHandler(new ApplicationRegisterServiceHandler(getManager())); gRPCServer.addHandler(new InstanceDiscoveryServiceHandler(getManager())); gRPCServer.addHandler(new ServiceNameDiscoveryServiceHandler(getManager())); - gRPCServer.addHandler(new JVMMetricsServiceHandler()); + gRPCServer.addHandler(new JVMMetricsServiceHandler(getManager())); gRPCServer.addHandler(new TraceSegmentServiceHandler(getManager())); } } diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCRegistration.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCRegistration.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCRegistration.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/AgentModuleGRPCRegistration.java diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java similarity index 88% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java index 1008f22c8..d3dcbff20 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandler.java @@ -20,7 +20,8 @@ package org.skywalking.apm.collector.agent.grpc.handler; import com.google.protobuf.ProtocolStringList; import io.grpc.stub.StreamObserver; -import org.skywalking.apm.collector.agent.stream.worker.register.ApplicationIDService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IApplicationIDService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.grpc.GRPCHandler; import org.skywalking.apm.network.proto.Application; @@ -37,10 +38,10 @@ public class ApplicationRegisterServiceHandler extends ApplicationRegisterServic private final Logger logger = LoggerFactory.getLogger(ApplicationRegisterServiceHandler.class); - private final ApplicationIDService applicationIDService; + private final IApplicationIDService applicationIDService; public ApplicationRegisterServiceHandler(ModuleManager moduleManager) { - applicationIDService = new ApplicationIDService(moduleManager); + applicationIDService = moduleManager.find(AgentStreamModule.NAME).getService(IApplicationIDService.class); } @Override public void register(Application request, StreamObserver responseObserver) { diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java similarity index 91% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java index 456c1cd26..0f5197a3c 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/InstanceDiscoveryServiceHandler.java @@ -21,7 +21,8 @@ package org.skywalking.apm.collector.agent.grpc.handler; import com.google.gson.JsonArray; import com.google.gson.JsonObject; import io.grpc.stub.StreamObserver; -import org.skywalking.apm.collector.agent.stream.worker.register.InstanceIDService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IInstanceIDService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.server.grpc.GRPCHandler; @@ -41,10 +42,10 @@ public class InstanceDiscoveryServiceHandler extends InstanceDiscoveryServiceGrp private final Logger logger = LoggerFactory.getLogger(InstanceDiscoveryServiceHandler.class); - private final InstanceIDService instanceIDService; + private final IInstanceIDService instanceIDService; public InstanceDiscoveryServiceHandler(ModuleManager moduleManager) { - this.instanceIDService = new InstanceIDService(moduleManager); + this.instanceIDService = moduleManager.find(AgentStreamModule.NAME).getService(IInstanceIDService.class); } @Override diff --git a/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java new file mode 100644 index 000000000..b3f16f2c6 --- /dev/null +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java @@ -0,0 +1,101 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.grpc.handler; + +import io.grpc.stub.StreamObserver; +import java.util.List; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.jvm.ICpuMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IGCMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IInstanceHeartBeatService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryPoolMetricService; +import org.skywalking.apm.collector.core.module.ModuleManager; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; +import org.skywalking.apm.collector.server.grpc.GRPCHandler; +import org.skywalking.apm.network.proto.CPU; +import org.skywalking.apm.network.proto.Downstream; +import org.skywalking.apm.network.proto.GC; +import org.skywalking.apm.network.proto.JVMMetrics; +import org.skywalking.apm.network.proto.JVMMetricsServiceGrpc; +import org.skywalking.apm.network.proto.Memory; +import org.skywalking.apm.network.proto.MemoryPool; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class JVMMetricsServiceHandler extends JVMMetricsServiceGrpc.JVMMetricsServiceImplBase implements GRPCHandler { + + private final Logger logger = LoggerFactory.getLogger(JVMMetricsServiceHandler.class); + + private final ICpuMetricService cpuMetricService; + private final IGCMetricService gcMetricService; + private final IMemoryMetricService memoryMetricService; + private final IMemoryPoolMetricService memoryPoolMetricService; + private final IInstanceHeartBeatService instanceHeartBeatService; + + public JVMMetricsServiceHandler(ModuleManager moduleManager) { + this.cpuMetricService = moduleManager.find(AgentStreamModule.NAME).getService(ICpuMetricService.class); + this.gcMetricService = moduleManager.find(AgentStreamModule.NAME).getService(IGCMetricService.class); + this.memoryMetricService = moduleManager.find(AgentStreamModule.NAME).getService(IMemoryMetricService.class); + this.memoryPoolMetricService = moduleManager.find(AgentStreamModule.NAME).getService(IMemoryPoolMetricService.class); + this.instanceHeartBeatService = moduleManager.find(AgentStreamModule.NAME).getService(IInstanceHeartBeatService.class); + } + + @Override public void collect(JVMMetrics request, StreamObserver responseObserver) { + int instanceId = request.getApplicationInstanceId(); + logger.debug("receive the jvm metric from application instance, id: {}", instanceId); + + request.getMetricsList().forEach(metric -> { + long time = TimeBucketUtils.INSTANCE.getSecondTimeBucket(metric.getTime()); + sendToInstanceHeartBeatService(instanceId, metric.getTime()); + sendToCpuMetricService(instanceId, time, metric.getCpu()); + sendToMemoryMetricService(instanceId, time, metric.getMemoryList()); + sendToMemoryPoolMetricService(instanceId, time, metric.getMemoryPoolList()); + sendToGCMetricService(instanceId, time, metric.getGcList()); + }); + + responseObserver.onNext(Downstream.newBuilder().build()); + responseObserver.onCompleted(); + } + + private void sendToInstanceHeartBeatService(int instanceId, long heartBeatTime) { + instanceHeartBeatService.send(instanceId, heartBeatTime); + } + + private void sendToMemoryMetricService(int instanceId, long timeBucket, List memories) { + memories.forEach(memory -> memoryMetricService.send(instanceId, timeBucket, memory.getIsHeap(), memory.getInit(), memory.getMax(), memory.getUsed(), memory.getCommitted())); + } + + private void sendToMemoryPoolMetricService(int instanceId, long timeBucket, + List memoryPools) { + + memoryPools.forEach(memoryPool -> memoryPoolMetricService.send(instanceId, timeBucket, memoryPool.getType().getNumber(), memoryPool.getInit(), memoryPool.getMax(), memoryPool.getUsed(), memoryPool.getCommited())); + } + + private void sendToCpuMetricService(int instanceId, long timeBucket, CPU cpu) { + cpuMetricService.send(instanceId, timeBucket, cpu.getUsagePercent()); + } + + private void sendToGCMetricService(int instanceId, long timeBucket, List gcs) { + gcs.forEach(gc -> gcMetricService.send(instanceId, timeBucket, gc.getPhraseValue(), gc.getCount(), gc.getTime())); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java similarity index 89% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java index 4b84db6c3..2fe84ea73 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/ServiceNameDiscoveryServiceHandler.java @@ -20,7 +20,8 @@ package org.skywalking.apm.collector.agent.grpc.handler; import io.grpc.stub.StreamObserver; import java.util.List; -import org.skywalking.apm.collector.agent.stream.worker.register.ServiceNameService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IServiceNameService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.grpc.GRPCHandler; import org.skywalking.apm.network.proto.ServiceNameCollection; @@ -38,10 +39,10 @@ public class ServiceNameDiscoveryServiceHandler extends ServiceNameDiscoveryServ private final Logger logger = LoggerFactory.getLogger(ServiceNameDiscoveryServiceHandler.class); - private final ServiceNameService serviceNameService; + private final IServiceNameService serviceNameService; public ServiceNameDiscoveryServiceHandler(ModuleManager moduleManager) { - this.serviceNameService = new ServiceNameService(moduleManager); + this.serviceNameService = moduleManager.find(AgentStreamModule.NAME).getService(IServiceNameService.class); } @Override public void discovery(ServiceNameCollection request, diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java similarity index 84% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java index 4c6b80a6b..0dbcfd5e4 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/TraceSegmentServiceHandler.java @@ -19,7 +19,8 @@ package org.skywalking.apm.collector.agent.grpc.handler; import io.grpc.stub.StreamObserver; -import org.skywalking.apm.collector.agent.stream.parser.SegmentParse; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.trace.ITraceSegmentService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.grpc.GRPCHandler; import org.skywalking.apm.network.proto.Downstream; @@ -35,18 +36,17 @@ public class TraceSegmentServiceHandler extends TraceSegmentServiceGrpc.TraceSeg private final Logger logger = LoggerFactory.getLogger(TraceSegmentServiceHandler.class); - private final ModuleManager moduleManager; + private final ITraceSegmentService traceSegmentService; public TraceSegmentServiceHandler(ModuleManager moduleManager) { - this.moduleManager = moduleManager; + this.traceSegmentService = moduleManager.find(AgentStreamModule.NAME).getService(ITraceSegmentService.class); } @Override public StreamObserver collect(StreamObserver responseObserver) { return new StreamObserver() { @Override public void onNext(UpstreamSegment segment) { logger.debug("receive segment"); - SegmentParse segmentParse = new SegmentParse(moduleManager); - segmentParse.parse(segment, SegmentParse.Source.Agent); + traceSegmentService.send(segment); } @Override public void onError(Throwable throwable) { diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java similarity index 96% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java index dda852e44..e729074ee 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingHandler.java @@ -37,8 +37,7 @@ public class AgentGRPCNamingHandler extends JettyHandler { } @Override public String pathSpec() { -// return "/agent/gRPC"; - return "/agentstream/grpc"; + return "/agent/gRPC"; } @Override protected JsonElement doGet(HttpServletRequest req) throws ArgumentsParseException { diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java similarity index 88% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java index b75a32d33..0d0d190c6 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java +++ b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/naming/AgentGRPCNamingListener.java @@ -18,7 +18,7 @@ package org.skywalking.apm.collector.agent.grpc.handler.naming; -import org.skywalking.apm.collector.agent.AgentModule; +import org.skywalking.apm.collector.agent.grpc.AgentGRPCModule; import org.skywalking.apm.collector.agent.grpc.AgentModuleGRPCProvider; import org.skywalking.apm.collector.cluster.ClusterModuleListener; @@ -27,7 +27,7 @@ import org.skywalking.apm.collector.cluster.ClusterModuleListener; */ public class AgentGRPCNamingListener extends ClusterModuleListener { - public static final String PATH = "/" + AgentModule.NAME + "/" + AgentModuleGRPCProvider.NAME; + public static final String PATH = "/" + AgentGRPCModule.NAME + "/" + AgentModuleGRPCProvider.NAME; @Override public String path() { return PATH; diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/test/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandlerTestCase.java b/apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/test/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandlerTestCase.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/test/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandlerTestCase.java rename to apm-collector/apm-collector-agent-grpc/collector-agent-grpc-provider/src/test/java/org/skywalking/apm/collector/agent/grpc/handler/ApplicationRegisterServiceHandlerTestCase.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/pom.xml b/apm-collector/apm-collector-agent-grpc/pom.xml similarity index 74% rename from apm-collector/apm-collector-agent/collector-agent-stream/pom.xml rename to apm-collector/apm-collector-agent-grpc/pom.xml index da8ed0465..c57e53acd 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/pom.xml +++ b/apm-collector/apm-collector-agent-grpc/pom.xml @@ -21,25 +21,24 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-agent + apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 - collector-agent-stream - jar + apm-collector-agent-grpc + pom + + collector-agent-grpc-define + collector-agent-grpc-provider + org.skywalking - collector-stream-provider - ${project.version} - - - org.skywalking - collector-cache-define + apm-collector-core ${project.version} - + \ No newline at end of file diff --git a/apm-collector/apm-collector-stream/collector-stream-define/pom.xml b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/pom.xml similarity index 87% rename from apm-collector/apm-collector-stream/collector-stream-define/pom.xml rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/pom.xml index 083972267..f8f848969 100644 --- a/apm-collector/apm-collector-stream/collector-stream-define/pom.xml +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/pom.xml @@ -21,12 +21,13 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-stream + apm-collector-agent-jetty org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 - collector-stream-define + collector-agent-jetty-define jar + \ No newline at end of file diff --git a/apm-collector/apm-collector-stream/collector-stream-define/src/main/java/org/skywalking/apm/collector/stream/StreamModule.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentJettyModule.java similarity index 86% rename from apm-collector/apm-collector-stream/collector-stream-define/src/main/java/org/skywalking/apm/collector/stream/StreamModule.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentJettyModule.java index 29ae5467c..be74cdfc1 100644 --- a/apm-collector/apm-collector-stream/collector-stream-define/src/main/java/org/skywalking/apm/collector/stream/StreamModule.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentJettyModule.java @@ -16,16 +16,16 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.stream; +package org.skywalking.apm.collector.agent.jetty; import org.skywalking.apm.collector.core.module.Module; /** * @author peng-yongsheng */ -public class StreamModule extends Module { +public class AgentJettyModule extends Module { - public static final String NAME = "stream"; + public static final String NAME = "agent_jetty"; @Override public String name() { return NAME; diff --git a/apm-collector/apm-collector-agent/collector-agent-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module similarity index 92% rename from apm-collector/apm-collector-agent/collector-agent-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module index 53a054ae3..6bd8395e8 100644 --- a/apm-collector/apm-collector-agent/collector-agent-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module @@ -16,4 +16,4 @@ # Project repository: https://github.com/OpenSkywalking/skywalking # -org.skywalking.apm.collector.agent.AgentModule \ No newline at end of file +org.skywalking.apm.collector.agent.jetty.AgentJettyModule \ No newline at end of file diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/pom.xml b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/pom.xml similarity index 73% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/pom.xml rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/pom.xml index f8a854702..03bac8d60 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/pom.xml +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/pom.xml @@ -21,9 +21,9 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-agent + apm-collector-agent-jetty org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -33,12 +33,7 @@ org.skywalking - collector-agent-define - ${project.version} - - - org.skywalking - collector-agent-stream + collector-agent-jetty-define ${project.version} @@ -46,5 +41,20 @@ collector-jetty-manager-define ${project.version} + + org.skywalking + collector-cluster-define + ${project.version} + + + org.skywalking + collector-naming-define + ${project.version} + + + org.skywalking + collector-agent-stream-define + ${project.version} + diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java similarity index 90% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java index 75d911b44..ce317e3e3 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyProvider.java @@ -19,14 +19,13 @@ package org.skywalking.apm.collector.agent.jetty; import java.util.Properties; -import org.skywalking.apm.collector.agent.AgentModule; import org.skywalking.apm.collector.agent.jetty.handler.ApplicationRegisterServletHandler; import org.skywalking.apm.collector.agent.jetty.handler.InstanceDiscoveryServletHandler; import org.skywalking.apm.collector.agent.jetty.handler.ServiceNameDiscoveryServiceHandler; import org.skywalking.apm.collector.agent.jetty.handler.TraceSegmentServletHandler; import org.skywalking.apm.collector.agent.jetty.handler.naming.AgentJettyNamingHandler; import org.skywalking.apm.collector.agent.jetty.handler.naming.AgentJettyNamingListener; -import org.skywalking.apm.collector.cache.CacheModule; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; import org.skywalking.apm.collector.cluster.ClusterModule; import org.skywalking.apm.collector.cluster.service.ModuleListenerService; import org.skywalking.apm.collector.cluster.service.ModuleRegisterService; @@ -38,8 +37,6 @@ import org.skywalking.apm.collector.jetty.manager.service.JettyManagerService; import org.skywalking.apm.collector.naming.NamingModule; import org.skywalking.apm.collector.naming.service.NamingHandlerRegisterService; import org.skywalking.apm.collector.server.Server; -import org.skywalking.apm.collector.storage.StorageModule; -import org.skywalking.apm.collector.stream.StreamModule; /** * @author peng-yongsheng @@ -56,7 +53,7 @@ public class AgentModuleJettyProvider extends ModuleProvider { } @Override public Class module() { - return AgentModule.class; + return AgentJettyModule.class; } @Override public void prepare(Properties config) throws ServiceNotProvidedException { @@ -69,7 +66,7 @@ public class AgentModuleJettyProvider extends ModuleProvider { String contextPath = config.getProperty(CONTEXT_PATH); ModuleRegisterService moduleRegisterService = getManager().find(ClusterModule.NAME).getService(ModuleRegisterService.class); - moduleRegisterService.register(AgentModule.NAME, this.name(), new AgentModuleJettyRegistration(host, port, contextPath)); + moduleRegisterService.register(AgentJettyModule.NAME, this.name(), new AgentModuleJettyRegistration(host, port, contextPath)); AgentJettyNamingListener namingListener = new AgentJettyNamingListener(); ModuleListenerService moduleListenerService = getManager().find(ClusterModule.NAME).getService(ModuleListenerService.class); @@ -88,7 +85,7 @@ public class AgentModuleJettyProvider extends ModuleProvider { } @Override public String[] requiredModules() { - return new String[] {ClusterModule.NAME, NamingModule.NAME, StorageModule.NAME, JettyManagerModule.NAME, CacheModule.NAME, StreamModule.NAME}; + return new String[] {ClusterModule.NAME, NamingModule.NAME, JettyManagerModule.NAME, AgentStreamModule.NAME}; } private void addHandlers(Server jettyServer) { diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyRegistration.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyRegistration.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyRegistration.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/AgentModuleJettyRegistration.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java similarity index 89% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java index 5fb5db4b6..5d90e79fd 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterServletHandler.java @@ -24,7 +24,8 @@ import com.google.gson.JsonElement; import com.google.gson.JsonObject; import java.io.IOException; import javax.servlet.http.HttpServletRequest; -import org.skywalking.apm.collector.agent.stream.worker.register.ApplicationIDService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IApplicationIDService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.jetty.ArgumentsParseException; import org.skywalking.apm.collector.server.jetty.JettyHandler; @@ -38,13 +39,13 @@ public class ApplicationRegisterServletHandler extends JettyHandler { private final Logger logger = LoggerFactory.getLogger(ApplicationRegisterServletHandler.class); - private final ApplicationIDService applicationIDService; + private final IApplicationIDService applicationIDService; private Gson gson = new Gson(); private static final String APPLICATION_CODE = "c"; private static final String APPLICATION_ID = "i"; public ApplicationRegisterServletHandler(ModuleManager moduleManager) { - this.applicationIDService = new ApplicationIDService(moduleManager); + this.applicationIDService = moduleManager.find(AgentStreamModule.NAME).getService(IApplicationIDService.class); } @Override public String pathSpec() { diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java similarity index 88% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java index 658d7d419..17627398b 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceDiscoveryServletHandler.java @@ -23,7 +23,8 @@ import com.google.gson.JsonElement; import com.google.gson.JsonObject; import java.io.IOException; import javax.servlet.http.HttpServletRequest; -import org.skywalking.apm.collector.agent.stream.worker.register.InstanceIDService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IInstanceIDService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.jetty.ArgumentsParseException; import org.skywalking.apm.collector.server.jetty.JettyHandler; @@ -37,8 +38,8 @@ public class InstanceDiscoveryServletHandler extends JettyHandler { private final Logger logger = LoggerFactory.getLogger(InstanceDiscoveryServletHandler.class); - private final InstanceIDService instanceIDService; - private Gson gson = new Gson(); + private final IInstanceIDService instanceIDService; + private final Gson gson = new Gson(); private static final String APPLICATION_ID = "ai"; private static final String AGENT_UUID = "au"; @@ -47,7 +48,7 @@ public class InstanceDiscoveryServletHandler extends JettyHandler { private static final String OS_INFO = "oi"; public InstanceDiscoveryServletHandler(ModuleManager moduleManager) { - this.instanceIDService = new InstanceIDService(moduleManager); + this.instanceIDService = moduleManager.find(AgentStreamModule.NAME).getService(IInstanceIDService.class); } @Override public String pathSpec() { diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java similarity index 88% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java index b842e17c9..7e7b224f5 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameDiscoveryServiceHandler.java @@ -24,7 +24,8 @@ import com.google.gson.JsonElement; import com.google.gson.JsonObject; import java.io.IOException; import javax.servlet.http.HttpServletRequest; -import org.skywalking.apm.collector.agent.stream.worker.register.ServiceNameService; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.register.IServiceNameService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.jetty.ArgumentsParseException; import org.skywalking.apm.collector.server.jetty.JettyHandler; @@ -38,8 +39,8 @@ public class ServiceNameDiscoveryServiceHandler extends JettyHandler { private final Logger logger = LoggerFactory.getLogger(ServiceNameDiscoveryServiceHandler.class); - private final ServiceNameService serviceNameService; - private Gson gson = new Gson(); + private final IServiceNameService serviceNameService; + private final Gson gson = new Gson(); private static final String APPLICATION_ID = "ai"; private static final String SERVICE_NAME = "sn"; @@ -47,7 +48,7 @@ public class ServiceNameDiscoveryServiceHandler extends JettyHandler { private static final String ELEMENT = "el"; public ServiceNameDiscoveryServiceHandler(ModuleManager moduleManager) { - this.serviceNameService = new ServiceNameService(moduleManager); + this.serviceNameService = moduleManager.find(AgentStreamModule.NAME).getService(IServiceNameService.class); } @Override public String pathSpec() { diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java similarity index 86% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java index 6c873bd77..c880674ff 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/TraceSegmentServletHandler.java @@ -25,7 +25,8 @@ import java.io.IOException; import javax.servlet.http.HttpServletRequest; import org.skywalking.apm.collector.agent.jetty.handler.reader.TraceSegment; import org.skywalking.apm.collector.agent.jetty.handler.reader.TraceSegmentJsonReader; -import org.skywalking.apm.collector.agent.stream.parser.SegmentParse; +import org.skywalking.apm.collector.agent.stream.AgentStreamModule; +import org.skywalking.apm.collector.agent.stream.service.trace.ITraceSegmentService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.server.jetty.ArgumentsParseException; import org.skywalking.apm.collector.server.jetty.JettyHandler; @@ -39,10 +40,10 @@ public class TraceSegmentServletHandler extends JettyHandler { private final Logger logger = LoggerFactory.getLogger(TraceSegmentServletHandler.class); - private final ModuleManager moduleManager; + private final ITraceSegmentService traceSegmentService; public TraceSegmentServletHandler(ModuleManager moduleManager) { - this.moduleManager = moduleManager; + this.traceSegmentService = moduleManager.find(AgentStreamModule.NAME).getService(ITraceSegmentService.class); } @Override public String pathSpec() { @@ -71,9 +72,8 @@ public class TraceSegmentServletHandler extends JettyHandler { reader.beginArray(); while (reader.hasNext()) { - SegmentParse segmentParse = new SegmentParse(moduleManager); TraceSegment traceSegment = jsonReader.read(reader); - segmentParse.parse(traceSegment.getUpstreamSegment(), SegmentParse.Source.Agent); + traceSegmentService.send(traceSegment.getUpstreamSegment()); } reader.endArray(); } diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingHandler.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingHandler.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingHandler.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingHandler.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java similarity index 87% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java index 1dad304a9..e9592cd12 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/naming/AgentJettyNamingListener.java @@ -18,7 +18,7 @@ package org.skywalking.apm.collector.agent.jetty.handler.naming; -import org.skywalking.apm.collector.agent.AgentModule; +import org.skywalking.apm.collector.agent.jetty.AgentJettyModule; import org.skywalking.apm.collector.agent.jetty.AgentModuleJettyProvider; import org.skywalking.apm.collector.cluster.ClusterModuleListener; @@ -27,7 +27,7 @@ import org.skywalking.apm.collector.cluster.ClusterModuleListener; */ public class AgentJettyNamingListener extends ClusterModuleListener { - public static final String PATH = "/" + AgentModule.NAME + "/" + AgentModuleJettyProvider.NAME; + public static final String PATH = "/" + AgentJettyModule.NAME + "/" + AgentModuleJettyProvider.NAME; @Override public String path() { return PATH; diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/KeyWithStringValueJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/KeyWithStringValueJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/KeyWithStringValueJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/KeyWithStringValueJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/LogJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/LogJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/LogJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/LogJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/ReferenceJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/ReferenceJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/ReferenceJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/ReferenceJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SegmentJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SegmentJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SegmentJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SegmentJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SpanJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SpanJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SpanJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/SpanJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/StreamJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/StreamJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/StreamJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/StreamJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegment.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegment.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegment.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegment.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegmentJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegmentJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegmentJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/TraceSegmentJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/UniqueIdJsonReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/UniqueIdJsonReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/UniqueIdJsonReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/java/org/skywalking/apm/collector/agent/jetty/handler/reader/UniqueIdJsonReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterPost.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterPost.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterPost.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ApplicationRegisterPost.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/HttpClientTools.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/HttpClientTools.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/HttpClientTools.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/HttpClientTools.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceRegisterPost.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceRegisterPost.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceRegisterPost.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/InstanceRegisterPost.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/JsonFileReader.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/JsonFileReader.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/JsonFileReader.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/JsonFileReader.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java similarity index 88% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java index 29b3efc31..297eab2a2 100644 --- a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java +++ b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/SegmentPost.java @@ -40,8 +40,11 @@ public class SegmentPost { serviceNameRegisterPost.send("json/servicename-register-provider.json"); JsonElement provider = JsonFileReader.INSTANCE.read("json/dubbox-provider.json"); - HttpClientTools.INSTANCE.post("http://localhost:12800/segments", provider.toString()); JsonElement consumer = JsonFileReader.INSTANCE.read("json/dubbox-consumer.json"); - HttpClientTools.INSTANCE.post("http://localhost:12800/segments", consumer.toString()); + + for (int i = 0; i < 1000; i++) { + HttpClientTools.INSTANCE.post("http://localhost:12800/segments", provider.toString()); + HttpClientTools.INSTANCE.post("http://localhost:12800/segments", consumer.toString()); + } } } diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameRegisterPost.java b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameRegisterPost.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameRegisterPost.java rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/java/org/skywalking/apm/collector/agent/jetty/handler/ServiceNameRegisterPost.java diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/application-register-consumer.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/application-register-consumer.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/application-register-consumer.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/application-register-consumer.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/application-register-provider.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/application-register-provider.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/application-register-provider.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/application-register-provider.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/dubbox-consumer.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/dubbox-consumer.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/dubbox-consumer.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/dubbox-consumer.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/dubbox-provider.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/dubbox-provider.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/dubbox-provider.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/dubbox-provider.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/instance-register-consumer.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/instance-register-consumer.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/instance-register-consumer.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/instance-register-consumer.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/instance-register-provider.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/instance-register-provider.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/instance-register-provider.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/instance-register-provider.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/servicename-register-consumer.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/servicename-register-consumer.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/servicename-register-consumer.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/servicename-register-consumer.json diff --git a/apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/servicename-register-provider.json b/apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/servicename-register-provider.json similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-jetty-provider/src/test/resources/json/servicename-register-provider.json rename to apm-collector/apm-collector-agent-jetty/collector-agent-jetty-provider/src/test/resources/json/servicename-register-provider.json diff --git a/apm-collector/apm-collector-agent-jetty/pom.xml b/apm-collector/apm-collector-agent-jetty/pom.xml new file mode 100644 index 000000000..79454b956 --- /dev/null +++ b/apm-collector/apm-collector-agent-jetty/pom.xml @@ -0,0 +1,44 @@ + + + + + + apm-collector + org.skywalking + 3.3.0-2017 + + 4.0.0 + + apm-collector-agent-jetty + pom + + collector-agent-jetty-define + collector-agent-jetty-provider + + + + + org.skywalking + apm-collector-core + ${project.version} + + + \ No newline at end of file diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/pom.xml b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/pom.xml similarity index 85% rename from apm-collector/apm-collector-stream/collector-stream-provider/pom.xml rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-define/pom.xml index 2cb289d8f..fb4fd771a 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/pom.xml +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/pom.xml @@ -21,20 +21,20 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-stream + apm-collector-agent-stream org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 - collector-stream-provider + collector-agent-stream-define jar org.skywalking - collector-stream-define + apm-network ${project.version} - + \ No newline at end of file diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModule.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModule.java new file mode 100644 index 000000000..01e960c24 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModule.java @@ -0,0 +1,68 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream; + +import java.util.ArrayList; +import java.util.List; +import org.skywalking.apm.collector.agent.stream.service.jvm.ICpuMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IGCMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IInstanceHeartBeatService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryPoolMetricService; +import org.skywalking.apm.collector.agent.stream.service.register.IApplicationIDService; +import org.skywalking.apm.collector.agent.stream.service.register.IInstanceIDService; +import org.skywalking.apm.collector.agent.stream.service.register.IServiceNameService; +import org.skywalking.apm.collector.agent.stream.service.trace.ITraceSegmentService; +import org.skywalking.apm.collector.core.module.Module; + +/** + * @author peng-yongsheng + */ +public class AgentStreamModule extends Module { + + public static final String NAME = "agent_stream"; + + @Override public String name() { + return NAME; + } + + @Override public Class[] services() { + List classes = new ArrayList<>(); + + addRegisterService(classes); + addJVMService(classes); + classes.add(ITraceSegmentService.class); + + return classes.toArray(new Class[] {}); + } + + private void addRegisterService(List classes) { + classes.add(IApplicationIDService.class); + classes.add(IInstanceIDService.class); + classes.add(IServiceNameService.class); + } + + private void addJVMService(List classes) { + classes.add(ICpuMetricService.class); + classes.add(IGCMetricService.class); + classes.add(IMemoryMetricService.class); + classes.add(IMemoryPoolMetricService.class); + classes.add(IInstanceHeartBeatService.class); + } +} diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/ICpuMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/ICpuMetricService.java new file mode 100644 index 000000000..659f881ea --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/ICpuMetricService.java @@ -0,0 +1,28 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.jvm; + +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface ICpuMetricService extends Service { + void send(int instanceId, long timeBucket, double usagePercent); +} diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IGCMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IGCMetricService.java new file mode 100644 index 000000000..62038953d --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IGCMetricService.java @@ -0,0 +1,28 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.jvm; + +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface IGCMetricService extends Service { + void send(int instanceId, long timeBucket, int phraseValue, long count, long time); +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferConfig.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IInstanceHeartBeatService.java similarity index 76% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferConfig.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IInstanceHeartBeatService.java index b27b7290f..53c487f7c 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferConfig.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IInstanceHeartBeatService.java @@ -16,13 +16,13 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.agent.stream.buffer; +package org.skywalking.apm.collector.agent.stream.service.jvm; -import org.skywalking.apm.collector.core.config.SystemConfig; +import org.skywalking.apm.collector.core.module.Service; /** * @author peng-yongsheng */ -public class SegmentBufferConfig { - public static String BUFFER_PATH = SystemConfig.DATA_PATH + "/buffer/"; +public interface IInstanceHeartBeatService extends Service { + void send(int instanceId, long heartBeatTime); } diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryMetricService.java new file mode 100644 index 000000000..917ac2900 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryMetricService.java @@ -0,0 +1,28 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.jvm; + +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface IMemoryMetricService extends Service { + void send(int instanceId, long timeBucket, boolean isHeap, long init, long max, long used, long commited); +} diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryPoolMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryPoolMetricService.java new file mode 100644 index 000000000..88766a509 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/jvm/IMemoryPoolMetricService.java @@ -0,0 +1,28 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.jvm; + +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface IMemoryPoolMetricService extends Service { + void send(int instanceId, long timeBucket, int poolType, long init, long max, long used, long commited); +} diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteServerService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IApplicationIDService.java similarity index 82% rename from apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteServerService.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IApplicationIDService.java index 17b6febf3..20e755370 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteServerService.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IApplicationIDService.java @@ -16,13 +16,13 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.remote.service; +package org.skywalking.apm.collector.agent.stream.service.register; import org.skywalking.apm.collector.core.module.Service; /** * @author peng-yongsheng */ -public interface RemoteServerService extends Service { - void registerReceiver(DataReceiver receiver); +public interface IApplicationIDService extends Service { + int getOrCreate(String applicationCode); } diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IInstanceIDService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IInstanceIDService.java new file mode 100644 index 000000000..efb3542fa --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IInstanceIDService.java @@ -0,0 +1,30 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.register; + +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface IInstanceIDService extends Service { + int getOrCreate(int applicationId, String agentUUID, long registerTime, String osInfo); + + void recover(int instanceId, int applicationId, long registerTime, String osInfo); +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IServiceNameService.java similarity index 75% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IServiceNameService.java index 6cc6f7724..3dc3c5d16 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/register/IServiceNameService.java @@ -16,12 +16,13 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.agent.stream.buffer; +package org.skywalking.apm.collector.agent.stream.service.register; + +import org.skywalking.apm.collector.core.module.Service; /** * @author peng-yongsheng */ -public class BufferFileConfig { - public static int BUFFER_OFFSET_MAX_FILE_SIZE = 10 * 1024 * 1024; - public static int BUFFER_SEGMENT_MAX_FILE_SIZE = 10 * 1024 * 1024; +public interface IServiceNameService extends Service { + int getOrCreate(int applicationId, String serviceName); } diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/trace/ITraceSegmentService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/trace/ITraceSegmentService.java new file mode 100644 index 000000000..581e62319 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/java/org/skywalking/apm/collector/agent/stream/service/trace/ITraceSegmentService.java @@ -0,0 +1,29 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.service.trace; + +import org.skywalking.apm.collector.core.module.Service; +import org.skywalking.apm.network.proto.UpstreamSegment; + +/** + * @author peng-yongsheng + */ +public interface ITraceSegmentService extends Service { + void send(UpstreamSegment segment); +} diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module new file mode 100644 index 000000000..2016bceae --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-define/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.Module @@ -0,0 +1,19 @@ +# +# Copyright 2017, OpenSkywalking Organization All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# Project repository: https://github.com/OpenSkywalking/skywalking +# + +org.skywalking.apm.collector.agent.stream.AgentStreamModule \ No newline at end of file diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/pom.xml b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/pom.xml similarity index 74% rename from apm-collector/apm-collector-agent/collector-agent-grpc-provider/pom.xml rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/pom.xml index 1aaf8da1f..132b29e68 100644 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/pom.xml +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/pom.xml @@ -21,30 +21,35 @@ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> - apm-collector-agent + apm-collector-agent-stream org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 - collector-agent-grpc-provider + collector-agent-stream-provider jar org.skywalking - collector-agent-define + collector-agent-stream-define ${project.version} org.skywalking - collector-agent-stream + collector-storage-define ${project.version} org.skywalking - collector-grpc-manager-define + collector-cache-define + ${project.version} + + + org.skywalking + apm-collector-stream ${project.version} - + \ No newline at end of file diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamSingleton.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamBootStartup.java similarity index 85% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamSingleton.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamBootStartup.java index 91afecb5a..7757ad1b9 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamSingleton.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamBootStartup.java @@ -22,34 +22,23 @@ import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; import org.skywalking.apm.collector.agent.stream.graph.RegisterStreamGraph; import org.skywalking.apm.collector.agent.stream.graph.TraceStreamGraph; import org.skywalking.apm.collector.core.module.ModuleManager; -import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.stream.timer.PersistenceTimer; import org.skywalking.apm.collector.stream.worker.base.WorkerCreateListener; /** * @author peng-yongsheng */ -public class AgentStreamSingleton { - - private static AgentStreamSingleton INSTANCE; +public class AgentStreamBootStartup { private final ModuleManager moduleManager; private final WorkerCreateListener workerCreateListener; - private AgentStreamSingleton(ModuleManager moduleManager) { + public AgentStreamBootStartup(ModuleManager moduleManager) { this.moduleManager = moduleManager; this.workerCreateListener = new WorkerCreateListener(); - this.create(); } - public static synchronized AgentStreamSingleton createInstanceIfAbsent(ModuleManager moduleManager) { - if (ObjectUtils.isEmpty(INSTANCE)) { - INSTANCE = new AgentStreamSingleton(moduleManager); - } - return INSTANCE; - } - - private void create() { + public void start() { createJVMGraph(); createRegisterGraph(); createTraceGraph(); diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModuleProvider.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModuleProvider.java new file mode 100644 index 000000000..c8c0d6e78 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/AgentStreamModuleProvider.java @@ -0,0 +1,96 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream; + +import java.util.Properties; +import org.skywalking.apm.collector.agent.stream.buffer.BufferFileConfig; +import org.skywalking.apm.collector.agent.stream.service.jvm.ICpuMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IGCMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IInstanceHeartBeatService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryMetricService; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryPoolMetricService; +import org.skywalking.apm.collector.agent.stream.service.register.IApplicationIDService; +import org.skywalking.apm.collector.agent.stream.service.register.IInstanceIDService; +import org.skywalking.apm.collector.agent.stream.service.register.IServiceNameService; +import org.skywalking.apm.collector.agent.stream.service.trace.ITraceSegmentService; +import org.skywalking.apm.collector.agent.stream.worker.AgentStreamRemoteDataRegister; +import org.skywalking.apm.collector.agent.stream.worker.jvm.CpuMetricService; +import org.skywalking.apm.collector.agent.stream.worker.jvm.GCMetricService; +import org.skywalking.apm.collector.agent.stream.worker.jvm.InstanceHeartBeatService; +import org.skywalking.apm.collector.agent.stream.worker.jvm.MemoryMetricService; +import org.skywalking.apm.collector.agent.stream.worker.jvm.MemoryPoolMetricService; +import org.skywalking.apm.collector.agent.stream.worker.register.ApplicationIDService; +import org.skywalking.apm.collector.agent.stream.worker.register.InstanceIDService; +import org.skywalking.apm.collector.agent.stream.worker.register.ServiceNameService; +import org.skywalking.apm.collector.agent.stream.worker.trace.TraceSegmentService; +import org.skywalking.apm.collector.cache.CacheModule; +import org.skywalking.apm.collector.core.module.Module; +import org.skywalking.apm.collector.core.module.ModuleProvider; +import org.skywalking.apm.collector.core.module.ServiceNotProvidedException; +import org.skywalking.apm.collector.remote.RemoteModule; +import org.skywalking.apm.collector.remote.service.RemoteDataRegisterService; +import org.skywalking.apm.collector.storage.StorageModule; + +/** + * @author peng-yongsheng + */ +public class AgentStreamModuleProvider extends ModuleProvider { + + @Override public String name() { + return "default"; + } + + @Override public Class module() { + return AgentStreamModule.class; + } + + @Override public void prepare(Properties config) throws ServiceNotProvidedException { + this.registerServiceImplementation(IApplicationIDService.class, new ApplicationIDService(getManager())); + this.registerServiceImplementation(IInstanceIDService.class, new InstanceIDService(getManager())); + this.registerServiceImplementation(IServiceNameService.class, new ServiceNameService(getManager())); + + this.registerServiceImplementation(ICpuMetricService.class, new CpuMetricService()); + this.registerServiceImplementation(IGCMetricService.class, new GCMetricService()); + this.registerServiceImplementation(IMemoryMetricService.class, new MemoryMetricService()); + this.registerServiceImplementation(IMemoryPoolMetricService.class, new MemoryPoolMetricService()); + this.registerServiceImplementation(IInstanceHeartBeatService.class, new InstanceHeartBeatService()); + + this.registerServiceImplementation(ITraceSegmentService.class, new TraceSegmentService(getManager())); + + BufferFileConfig.Parser parser = new BufferFileConfig.Parser(); + parser.parse(config); + } + + @Override public void start(Properties config) throws ServiceNotProvidedException { + RemoteDataRegisterService remoteDataRegisterService = getManager().find(RemoteModule.NAME).getService(RemoteDataRegisterService.class); + AgentStreamRemoteDataRegister agentStreamRemoteDataRegister = new AgentStreamRemoteDataRegister(remoteDataRegisterService); + agentStreamRemoteDataRegister.register(); + + AgentStreamBootStartup bootStartup = new AgentStreamBootStartup(getManager()); + bootStartup.start(); + } + + @Override public void notifyAfterCompleted() throws ServiceNotProvidedException { + + } + + @Override public String[] requiredModules() { + return new String[] {StorageModule.NAME, CacheModule.NAME}; + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/IdAutoIncrement.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/IdAutoIncrement.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/IdAutoIncrement.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/IdAutoIncrement.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java new file mode 100644 index 000000000..812048b72 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/BufferFileConfig.java @@ -0,0 +1,85 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.buffer; + +import java.util.Properties; + +/** + * @author peng-yongsheng + */ +public class BufferFileConfig { + static int BUFFER_OFFSET_MAX_FILE_SIZE = 10 * 1024 * 1024; + static int BUFFER_SEGMENT_MAX_FILE_SIZE = 10 * 1024 * 1024; + static String BUFFER_PATH = "../buffer/"; + + private static final String BUFFER_PATH_KEY = "buffer_file_path"; + private static final String BUFFER_OFFSET_MAX_FILE_SIZE_KEY = "buffer_offset_max_file_size"; + private static final String BUFFER_SEGMENT_MAX_FILE_SIZE_KEY = "buffer_segment_max_file_size"; + + public static class Parser { + + public void parse(Properties config) { + if (config.containsKey(BUFFER_PATH_KEY)) { + BUFFER_PATH = config.getProperty(BUFFER_PATH_KEY); + } + + if (config.containsKey(BUFFER_OFFSET_MAX_FILE_SIZE_KEY)) { + String sizeStr = config.getProperty(BUFFER_OFFSET_MAX_FILE_SIZE_KEY).toUpperCase(); + if (sizeStr.endsWith("K")) { + int size = Integer.parseInt(sizeStr.replace("K", "")); + BUFFER_OFFSET_MAX_FILE_SIZE = size * 1024; + } else if (sizeStr.endsWith("KB")) { + int size = Integer.parseInt(sizeStr.replace("KB", "")); + BUFFER_OFFSET_MAX_FILE_SIZE = size * 1024; + } else if (sizeStr.endsWith("M")) { + int size = Integer.parseInt(sizeStr.replace("M", "")); + BUFFER_OFFSET_MAX_FILE_SIZE = size * 1024 * 1024; + } else if (sizeStr.endsWith("MB")) { + int size = Integer.parseInt(sizeStr.replace("MB", "")); + BUFFER_OFFSET_MAX_FILE_SIZE = size * 1024 * 1024; + } else { + BUFFER_OFFSET_MAX_FILE_SIZE = 10 * 1024 * 1024; + } + } else { + BUFFER_OFFSET_MAX_FILE_SIZE = 10 * 1024 * 1024; + } + + if (config.containsKey(BUFFER_SEGMENT_MAX_FILE_SIZE_KEY)) { + String sizeStr = config.getProperty(BUFFER_SEGMENT_MAX_FILE_SIZE_KEY).toUpperCase(); + if (sizeStr.endsWith("K")) { + int size = Integer.parseInt(sizeStr.replace("K", "")); + BUFFER_SEGMENT_MAX_FILE_SIZE = size * 1024; + } else if (sizeStr.endsWith("KB")) { + int size = Integer.parseInt(sizeStr.replace("KB", "")); + BUFFER_SEGMENT_MAX_FILE_SIZE = size * 1024; + } else if (sizeStr.endsWith("M")) { + int size = Integer.parseInt(sizeStr.replace("M", "")); + BUFFER_SEGMENT_MAX_FILE_SIZE = size * 1024 * 1024; + } else if (sizeStr.endsWith("MB")) { + int size = Integer.parseInt(sizeStr.replace("MB", "")); + BUFFER_SEGMENT_MAX_FILE_SIZE = size * 1024 * 1024; + } else { + BUFFER_SEGMENT_MAX_FILE_SIZE = 1024 * 1024; + } + } else { + BUFFER_SEGMENT_MAX_FILE_SIZE = 1024 * 1024; + } + } + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/Offset.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/Offset.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/Offset.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/Offset.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java similarity index 94% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java index 68bce1581..a78aa1ac4 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/OffsetManager.java @@ -49,7 +49,7 @@ public enum OffsetManager { public synchronized void initialize() throws IOException { if (!initialized) { this.offset = new Offset(); - File dataPath = new File(SegmentBufferConfig.BUFFER_PATH); + File dataPath = new File(BufferFileConfig.BUFFER_PATH); if (dataPath.mkdirs()) { createOffsetFile(); } else { @@ -70,14 +70,14 @@ public enum OffsetManager { offset.deserialize(offsetRecord); initialized = true; - Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(() -> flush(), 10, 3, TimeUnit.SECONDS); + Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(this::flush, 10, 3, TimeUnit.SECONDS); } } private void createOffsetFile() throws IOException { String timeBucket = String.valueOf(TimeBucketUtils.INSTANCE.getSecondTimeBucket(System.currentTimeMillis())); String offsetFileName = OFFSET_FILE_PREFIX + "_" + timeBucket + "." + Const.FILE_SUFFIX; - offsetFile = new File(SegmentBufferConfig.BUFFER_PATH + offsetFileName); + offsetFile = new File(BufferFileConfig.BUFFER_PATH + offsetFileName); this.offset.getWriteOffset().setWriteFileName(Const.EMPTY_STRING); this.offset.getWriteOffset().setWriteFileOffset(0); this.offset.getReadOffset().setReadFileName(Const.EMPTY_STRING); @@ -99,7 +99,7 @@ public enum OffsetManager { private void nextFile() { String timeBucket = String.valueOf(TimeBucketUtils.INSTANCE.getSecondTimeBucket(System.currentTimeMillis())); String offsetFileName = OFFSET_FILE_PREFIX + "_" + timeBucket + "." + Const.FILE_SUFFIX; - File newOffsetFile = new File(SegmentBufferConfig.BUFFER_PATH + offsetFileName); + File newOffsetFile = new File(BufferFileConfig.BUFFER_PATH + offsetFileName); offsetFile.delete(); offsetFile = newOffsetFile; this.flush(); diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java similarity index 92% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java index 2e2a3d86c..2d3a87807 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferManager.java @@ -44,14 +44,14 @@ public enum SegmentBufferManager { logger.info("segment buffer initialize"); try { OffsetManager.INSTANCE.initialize(); - if (new File(SegmentBufferConfig.BUFFER_PATH).mkdirs()) { + if (new File(BufferFileConfig.BUFFER_PATH).mkdirs()) { newDataFile(); } else { String writeFileName = OffsetManager.INSTANCE.getWriteFileName(); if (StringUtils.isNotEmpty(writeFileName)) { - File dataFile = new File(SegmentBufferConfig.BUFFER_PATH + writeFileName); + File dataFile = new File(BufferFileConfig.BUFFER_PATH + writeFileName); if (dataFile.exists()) { - outputStream = new FileOutputStream(new File(SegmentBufferConfig.BUFFER_PATH + writeFileName), true); + outputStream = new FileOutputStream(new File(BufferFileConfig.BUFFER_PATH + writeFileName), true); } else { newDataFile(); } @@ -83,7 +83,7 @@ public enum SegmentBufferManager { logger.debug("create new segment buffer file"); String timeBucket = String.valueOf(TimeBucketUtils.INSTANCE.getSecondTimeBucket(System.currentTimeMillis())); String writeFileName = DATA_FILE_PREFIX + "_" + timeBucket + "." + Const.FILE_SUFFIX; - File dataFile = new File(SegmentBufferConfig.BUFFER_PATH + writeFileName); + File dataFile = new File(BufferFileConfig.BUFFER_PATH + writeFileName); dataFile.createNewFile(); OffsetManager.INSTANCE.setWriteOffset(writeFileName, 0); try { @@ -98,6 +98,5 @@ public enum SegmentBufferManager { } public synchronized void flush() { - } } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java similarity index 94% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java index 148b33d9b..0126686cb 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/buffer/SegmentBufferReader.java @@ -53,7 +53,7 @@ public enum SegmentBufferReader { private void preRead() { String readFileName = OffsetManager.INSTANCE.getReadFileName(); if (StringUtils.isNotEmpty(readFileName)) { - File readFile = new File(SegmentBufferConfig.BUFFER_PATH + readFileName); + File readFile = new File(BufferFileConfig.BUFFER_PATH + readFileName); if (readFile.exists()) { deleteTheDataFilesBeforeReadFile(readFileName); long readFileOffset = OffsetManager.INSTANCE.getReadFileOffset(); @@ -69,7 +69,7 @@ public enum SegmentBufferReader { } private void deleteTheDataFilesBeforeReadFile(String readFileName) { - File[] dataFiles = new File(SegmentBufferConfig.BUFFER_PATH).listFiles(new PrefixFileNameFilter()); + File[] dataFiles = new File(BufferFileConfig.BUFFER_PATH).listFiles(new PrefixFileNameFilter()); long readFileCreateTime = getFileCreateTime(readFileName); for (File dataFile : dataFiles) { @@ -90,7 +90,7 @@ public enum SegmentBufferReader { private void readEarliestCreateDataFile() { String readFileName = OffsetManager.INSTANCE.getReadFileName(); - File[] dataFiles = new File(SegmentBufferConfig.BUFFER_PATH).listFiles(new PrefixFileNameFilter()); + File[] dataFiles = new File(BufferFileConfig.BUFFER_PATH).listFiles(new PrefixFileNameFilter()); if (CollectionUtils.isNotEmpty(dataFiles)) { if (dataFiles[0].getName().equals(readFileName)) { diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/JvmMetricStreamGraph.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/JvmMetricStreamGraph.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/JvmMetricStreamGraph.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/JvmMetricStreamGraph.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/RegisterStreamGraph.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/RegisterStreamGraph.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/RegisterStreamGraph.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/RegisterStreamGraph.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/TraceStreamGraph.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/TraceStreamGraph.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/graph/TraceStreamGraph.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/graph/TraceStreamGraph.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/EntrySpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/EntrySpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/EntrySpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/EntrySpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/ExitSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/ExitSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/ExitSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/ExitSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/FirstSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/FirstSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/FirstSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/FirstSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/GlobalTraceIdsListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/GlobalTraceIdsListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/GlobalTraceIdsListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/GlobalTraceIdsListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/LocalSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/LocalSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/LocalSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/LocalSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/RefsListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/RefsListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/RefsListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/RefsListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java similarity index 99% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java index 668987fc7..97d518eb0 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SegmentParse.java @@ -163,7 +163,7 @@ public class SegmentParse { } private void writeToBufferFile(String id, UpstreamSegment upstreamSegment) { - logger.debug("send to segment buffer write worker, id: {}", id); + logger.debug("push to segment buffer write worker, id: {}", id); SegmentStandardization standardization = new SegmentStandardization(id); standardization.setUpstreamSegment(upstreamSegment); Graph graph = GraphManager.INSTANCE.createIfAbsent(TraceStreamGraph.SEGMENT_STANDARDIZATION_GRAPH_ID, SegmentStandardization.class); diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/SpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/IdExchanger.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/IdExchanger.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/IdExchanger.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/IdExchanger.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceDecorator.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceDecorator.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceDecorator.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceDecorator.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceIdExchanger.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceIdExchanger.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceIdExchanger.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/ReferenceIdExchanger.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentDecorator.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentDecorator.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentDecorator.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentDecorator.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardization.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardization.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardization.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardization.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardizationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardizationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardizationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SegmentStandardizationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanDecorator.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanDecorator.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanDecorator.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanDecorator.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanIdExchanger.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanIdExchanger.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanIdExchanger.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/SpanIdExchanger.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/StandardBuilder.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/StandardBuilder.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/StandardBuilder.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/parser/standardization/StandardBuilder.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/util/FileUtils.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/util/FileUtils.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/util/FileUtils.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/util/FileUtils.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/AgentStreamRemoteDataRegister.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/AgentStreamRemoteDataRegister.java new file mode 100644 index 000000000..65eee6eed --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/AgentStreamRemoteDataRegister.java @@ -0,0 +1,53 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker; + +import org.skywalking.apm.collector.remote.service.RemoteDataRegisterService; +import org.skywalking.apm.collector.storage.table.node.NodeComponent; +import org.skywalking.apm.collector.storage.table.node.NodeMapping; +import org.skywalking.apm.collector.storage.table.noderef.NodeReference; +import org.skywalking.apm.collector.storage.table.register.Application; +import org.skywalking.apm.collector.storage.table.register.Instance; +import org.skywalking.apm.collector.storage.table.register.ServiceName; +import org.skywalking.apm.collector.storage.table.service.ServiceEntry; +import org.skywalking.apm.collector.storage.table.serviceref.ServiceReference; + +/** + * @author peng-yongsheng + */ +public class AgentStreamRemoteDataRegister { + + private final RemoteDataRegisterService remoteDataRegisterService; + + public AgentStreamRemoteDataRegister(RemoteDataRegisterService remoteDataRegisterService) { + this.remoteDataRegisterService = remoteDataRegisterService; + } + + public void register() { + remoteDataRegisterService.register(Application.class, Application::new); + remoteDataRegisterService.register(Instance.class, Instance::new); + remoteDataRegisterService.register(ServiceName.class, ServiceName::new); + + remoteDataRegisterService.register(NodeComponent.class, NodeComponent::new); + remoteDataRegisterService.register(NodeMapping.class, NodeMapping::new); + remoteDataRegisterService.register(NodeReference.class, NodeReference::new); + remoteDataRegisterService.register(ServiceEntry.class, ServiceEntry::new); + remoteDataRegisterService.register(ServiceReference.class, ServiceReference::new); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricService.java new file mode 100644 index 000000000..1bbb03bd0 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/CpuMetricService.java @@ -0,0 +1,56 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.jvm; + +import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.jvm.ICpuMetricService; +import org.skywalking.apm.collector.core.graph.Graph; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; +import org.skywalking.apm.collector.storage.table.jvm.CpuMetric; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class CpuMetricService implements ICpuMetricService { + + private final Logger logger = LoggerFactory.getLogger(CpuMetricService.class); + + private Graph cpuMetricGraph; + + private Graph getCpuMetricGraph() { + if (ObjectUtils.isEmpty(cpuMetricGraph)) { + cpuMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.CPU_METRIC_GRAPH_ID, CpuMetric.class); + } + return cpuMetricGraph; + } + + @Override public void send(int instanceId, long timeBucket, double usagePercent) { + CpuMetric cpuMetric = new CpuMetric(timeBucket + Const.ID_SPLIT + instanceId); + cpuMetric.setInstanceId(instanceId); + cpuMetric.setUsagePercent(usagePercent); + cpuMetric.setTimeBucket(timeBucket); + + logger.debug("push to cpu metric graph, id: {}", cpuMetric.getId()); + getCpuMetricGraph().start(cpuMetric); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricService.java new file mode 100644 index 000000000..e2ae3a348 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/GCMetricService.java @@ -0,0 +1,58 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.jvm; + +import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.jvm.IGCMetricService; +import org.skywalking.apm.collector.core.graph.Graph; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; +import org.skywalking.apm.collector.storage.table.jvm.GCMetric; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class GCMetricService implements IGCMetricService { + + private final Logger logger = LoggerFactory.getLogger(GCMetricService.class); + + private Graph gcMetricGraph; + + private Graph getGcMetricGraph() { + if (ObjectUtils.isEmpty(gcMetricGraph)) { + gcMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.GC_METRIC_GRAPH_ID, GCMetric.class); + } + return gcMetricGraph; + } + + @Override public void send(int instanceId, long timeBucket, int phraseValue, long count, long time) { + GCMetric gcMetric = new GCMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(phraseValue)); + gcMetric.setInstanceId(instanceId); + gcMetric.setPhrase(phraseValue); + gcMetric.setCount(count); + gcMetric.setTime(time); + gcMetric.setTimeBucket(timeBucket); + + logger.debug("push to gc metric graph, id: {}", gcMetric.getId()); + getGcMetricGraph().start(gcMetric); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstHeartBeatPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstHeartBeatPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstHeartBeatPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstHeartBeatPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstanceHeartBeatService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstanceHeartBeatService.java new file mode 100644 index 000000000..8ef9c3506 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/InstanceHeartBeatService.java @@ -0,0 +1,55 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.jvm; + +import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.jvm.IInstanceHeartBeatService; +import org.skywalking.apm.collector.core.graph.Graph; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.util.ObjectUtils; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; +import org.skywalking.apm.collector.storage.table.register.Instance; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class InstanceHeartBeatService implements IInstanceHeartBeatService { + + private final Logger logger = LoggerFactory.getLogger(InstanceHeartBeatService.class); + + private Graph heartBeatGraph; + + private Graph getHeartBeatGraph() { + if (ObjectUtils.isEmpty(heartBeatGraph)) { + this.heartBeatGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.INST_HEART_BEAT_GRAPH_ID, Instance.class); + } + return heartBeatGraph; + } + + @Override public void send(int instanceId, long heartBeatTime) { + Instance instance = new Instance(String.valueOf(instanceId)); + instance.setHeartBeatTime(TimeBucketUtils.INSTANCE.getSecondTimeBucket(heartBeatTime)); + instance.setInstanceId(instanceId); + + logger.debug("push to instance heart beat persistence worker, id: {}", instance.getId()); + getHeartBeatGraph().start(instance); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricService.java new file mode 100644 index 000000000..32d5e8f2e --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryMetricService.java @@ -0,0 +1,61 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.jvm; + +import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryMetricService; +import org.skywalking.apm.collector.core.graph.Graph; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; +import org.skywalking.apm.collector.storage.table.jvm.MemoryMetric; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class MemoryMetricService implements IMemoryMetricService { + + private final Logger logger = LoggerFactory.getLogger(MemoryMetricService.class); + + private Graph memoryMetricGraph; + + private Graph getMemoryMetricGraph() { + if (ObjectUtils.isEmpty(memoryMetricGraph)) { + this.memoryMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.MEMORY_METRIC_GRAPH_ID, MemoryMetric.class); + } + return memoryMetricGraph; + } + + @Override + public void send(int instanceId, long timeBucket, boolean isHeap, long init, long max, long used, long commited) { + MemoryMetric memoryMetric = new MemoryMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(isHeap)); + memoryMetric.setInstanceId(instanceId); + memoryMetric.setIsHeap(isHeap); + memoryMetric.setInit(init); + memoryMetric.setMax(max); + memoryMetric.setUsed(used); + memoryMetric.setCommitted(commited); + memoryMetric.setTimeBucket(timeBucket); + + logger.debug("push to memory metric graph, id: {}", memoryMetric.getId()); + getMemoryMetricGraph().start(memoryMetric); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricService.java new file mode 100644 index 000000000..8e4aed00a --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/jvm/MemoryPoolMetricService.java @@ -0,0 +1,61 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.jvm; + +import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.jvm.IMemoryPoolMetricService; +import org.skywalking.apm.collector.core.graph.Graph; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; +import org.skywalking.apm.collector.storage.table.jvm.MemoryPoolMetric; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class MemoryPoolMetricService implements IMemoryPoolMetricService { + + private final Logger logger = LoggerFactory.getLogger(MemoryPoolMetricService.class); + + private Graph memoryPoolMetricGraph; + + private Graph getMemoryPoolMetricGraph() { + if (ObjectUtils.isEmpty(memoryPoolMetricGraph)) { + this.memoryPoolMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.MEMORY_POOL_METRIC_GRAPH_ID, MemoryPoolMetric.class); + } + return memoryPoolMetricGraph; + } + + @Override + public void send(int instanceId, long timeBucket, int poolType, long init, long max, long used, long commited) { + MemoryPoolMetric memoryPoolMetric = new MemoryPoolMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(poolType)); + memoryPoolMetric.setInstanceId(instanceId); + memoryPoolMetric.setPoolType(poolType); + memoryPoolMetric.setInit(init); + memoryPoolMetric.setMax(max); + memoryPoolMetric.setUsed(used); + memoryPoolMetric.setCommitted(commited); + memoryPoolMetric.setTimeBucket(timeBucket); + + logger.debug("push to memory pool metric graph, id: {}", memoryPoolMetric.getId()); + getMemoryPoolMetricGraph().start(memoryPoolMetric); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java similarity index 62% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java index 1595a8cf0..42aa307ba 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationIDService.java @@ -19,11 +19,13 @@ package org.skywalking.apm.collector.agent.stream.worker.register; import org.skywalking.apm.collector.agent.stream.graph.RegisterStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.register.IApplicationIDService; import org.skywalking.apm.collector.cache.CacheModule; import org.skywalking.apm.collector.cache.service.ApplicationCacheService; import org.skywalking.apm.collector.core.graph.Graph; import org.skywalking.apm.collector.core.graph.GraphManager; import org.skywalking.apm.collector.core.module.ModuleManager; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.storage.table.register.Application; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -31,28 +33,41 @@ import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ -public class ApplicationIDService { +public class ApplicationIDService implements IApplicationIDService { private final Logger logger = LoggerFactory.getLogger(ApplicationIDService.class); private final ModuleManager moduleManager; - private final Graph applicationRegisterGraph; + private ApplicationCacheService applicationCacheService; + private Graph applicationRegisterGraph; public ApplicationIDService(ModuleManager moduleManager) { this.moduleManager = moduleManager; - this.applicationRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.APPLICATION_REGISTER_GRAPH_ID, Application.class); + } + + private Graph getApplicationRegisterGraph() { + if (ObjectUtils.isEmpty(applicationRegisterGraph)) { + this.applicationRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.APPLICATION_REGISTER_GRAPH_ID, Application.class); + } + return this.applicationRegisterGraph; + } + + private ApplicationCacheService getApplicationCacheService() { + if (ObjectUtils.isEmpty(applicationCacheService)) { + this.applicationCacheService = moduleManager.find(CacheModule.NAME).getService(ApplicationCacheService.class); + } + return applicationCacheService; } public int getOrCreate(String applicationCode) { - ApplicationCacheService service = moduleManager.find(CacheModule.NAME).getService(ApplicationCacheService.class); - int applicationId = service.get(applicationCode); + int applicationId = getApplicationCacheService().get(applicationCode); if (applicationId == 0) { Application application = new Application(applicationCode); application.setApplicationCode(applicationCode); application.setApplicationId(0); - applicationRegisterGraph.start(application); + getApplicationRegisterGraph().start(application); } return applicationId; } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterRemoteWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterRemoteWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterSerialWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterSerialWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterSerialWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/ApplicationRegisterSerialWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java similarity index 67% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java index 831b9dcbf..4f1c94231 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceIDService.java @@ -19,11 +19,13 @@ package org.skywalking.apm.collector.agent.stream.worker.register; import org.skywalking.apm.collector.agent.stream.graph.RegisterStreamGraph; +import org.skywalking.apm.collector.agent.stream.service.register.IInstanceIDService; import org.skywalking.apm.collector.cache.CacheModule; import org.skywalking.apm.collector.cache.service.InstanceCacheService; import org.skywalking.apm.collector.core.graph.Graph; import org.skywalking.apm.collector.core.graph.GraphManager; import org.skywalking.apm.collector.core.module.ModuleManager; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.storage.StorageModule; import org.skywalking.apm.collector.storage.dao.IInstanceRegisterDAO; import org.skywalking.apm.collector.storage.table.register.Instance; @@ -33,24 +35,43 @@ import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ -public class InstanceIDService { +public class InstanceIDService implements IInstanceIDService { private final Logger logger = LoggerFactory.getLogger(InstanceIDService.class); private final ModuleManager moduleManager; - private final Graph instanceRegisterGraph; - private final IInstanceRegisterDAO instanceRegisterDAO; + private InstanceCacheService instanceCacheService; + private Graph instanceRegisterGraph; + private IInstanceRegisterDAO instanceRegisterDAO; public InstanceIDService(ModuleManager moduleManager) { this.moduleManager = moduleManager; - this.instanceRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.INSTANCE_REGISTER_GRAPH_ID, Instance.class); - this.instanceRegisterDAO = moduleManager.find(StorageModule.NAME).getService(IInstanceRegisterDAO.class); + } + + private InstanceCacheService getInstanceCacheService() { + if (ObjectUtils.isEmpty(instanceCacheService)) { + instanceCacheService = moduleManager.find(CacheModule.NAME).getService(InstanceCacheService.class); + } + return instanceCacheService; + } + + private Graph getInstanceRegisterGraph() { + if (ObjectUtils.isEmpty(instanceRegisterGraph)) { + this.instanceRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.INSTANCE_REGISTER_GRAPH_ID, Instance.class); + } + return instanceRegisterGraph; + } + + private IInstanceRegisterDAO getInstanceRegisterDAO() { + if (ObjectUtils.isEmpty(instanceRegisterDAO)) { + instanceRegisterDAO = moduleManager.find(StorageModule.NAME).getService(IInstanceRegisterDAO.class); + } + return instanceRegisterDAO; } public int getOrCreate(int applicationId, String agentUUID, long registerTime, String osInfo) { logger.debug("get or create instance id, application id: {}, agentUUID: {}, registerTime: {}, osInfo: {}", applicationId, agentUUID, registerTime, osInfo); - InstanceCacheService service = moduleManager.find(CacheModule.NAME).getService(InstanceCacheService.class); - int instanceId = service.getInstanceId(applicationId, agentUUID); + int instanceId = getInstanceCacheService().getInstanceId(applicationId, agentUUID); if (instanceId == 0) { Instance instance = new Instance("0"); @@ -61,14 +82,13 @@ public class InstanceIDService { instance.setInstanceId(0); instance.setOsInfo(osInfo); - instanceRegisterGraph.start(instance); + getInstanceRegisterGraph().start(instance); } return instanceId; } public void recover(int instanceId, int applicationId, long registerTime, String osInfo) { logger.debug("instance recover, instance id: {}, application id: {}, register time: {}", instanceId, applicationId, registerTime); - Instance instance = new Instance(String.valueOf(instanceId)); instance.setApplicationId(applicationId); instance.setAgentUUID(""); @@ -76,6 +96,7 @@ public class InstanceIDService { instance.setHeartBeatTime(registerTime); instance.setInstanceId(instanceId); instance.setOsInfo(osInfo); - instanceRegisterDAO.save(instance); + + getInstanceRegisterDAO().save(instance); } } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java similarity index 97% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java index f36fae2ba..07ba0fcc9 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/register/InstanceRegisterRemoteWorker.java @@ -39,7 +39,7 @@ public class InstanceRegisterRemoteWorker extends AbstractRemoteWorker serviceNameRegisterGraph; + private ServiceIdCacheService serviceIdCacheService; + private Graph serviceNameRegisterGraph; public ServiceNameService(ModuleManager moduleManager) { this.moduleManager = moduleManager; - this.serviceNameRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.SERVICE_NAME_REGISTER_GRAPH_ID, ServiceName.class); + } + + private ServiceIdCacheService getServiceIdCacheService() { + if (ObjectUtils.isEmpty(serviceIdCacheService)) { + serviceIdCacheService = moduleManager.find(CacheModule.NAME).getService(ServiceIdCacheService.class); + } + return serviceIdCacheService; + } + + private Graph getServiceNameRegisterGraph() { + if (ObjectUtils.isEmpty(serviceNameRegisterGraph)) { + this.serviceNameRegisterGraph = GraphManager.INSTANCE.createIfAbsent(RegisterStreamGraph.SERVICE_NAME_REGISTER_GRAPH_ID, ServiceName.class); + } + return serviceNameRegisterGraph; } public int getOrCreate(int applicationId, String serviceName) { - ServiceIdCacheService idCacheService = moduleManager.find(CacheModule.NAME).getService(ServiceIdCacheService.class); - int serviceId = idCacheService.get(applicationId, serviceName); + int serviceId = getServiceIdCacheService().get(applicationId, serviceName); if (serviceId == 0) { ServiceName service = new ServiceName("0"); @@ -53,7 +68,7 @@ public class ServiceNameService { service.setServiceName(serviceName); service.setServiceId(0); - serviceNameRegisterGraph.start(service); + getServiceNameRegisterGraph().start(service); } return serviceId; } diff --git a/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/TraceSegmentService.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/TraceSegmentService.java new file mode 100644 index 000000000..86832dde8 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/TraceSegmentService.java @@ -0,0 +1,41 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.agent.stream.worker.trace; + +import org.skywalking.apm.collector.agent.stream.parser.SegmentParse; +import org.skywalking.apm.collector.agent.stream.service.trace.ITraceSegmentService; +import org.skywalking.apm.collector.core.module.ModuleManager; +import org.skywalking.apm.network.proto.UpstreamSegment; + +/** + * @author peng-yongsheng + */ +public class TraceSegmentService implements ITraceSegmentService { + + private final ModuleManager moduleManager; + + public TraceSegmentService(ModuleManager moduleManager) { + this.moduleManager = moduleManager; + } + + public void send(UpstreamSegment segment) { + SegmentParse segmentParse = new SegmentParse(moduleManager); + segmentParse.parse(segment, SegmentParse.Source.Agent); + } +} diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTracePersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTracePersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTracePersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTracePersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTraceSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTraceSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTraceSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/global/GlobalTraceSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformancePersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformancePersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformancePersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformancePersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformanceSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformanceSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformanceSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/instance/InstPerformanceSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentAggregationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentAggregationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentAggregationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentAggregationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java similarity index 97% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java index 732a4e16d..d1b86441c 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentRemoteWorker.java @@ -31,7 +31,7 @@ import org.skywalking.apm.collector.stream.worker.base.WorkerException; */ public class NodeComponentRemoteWorker extends AbstractRemoteWorker { - public NodeComponentRemoteWorker(ModuleManager moduleManager) { + NodeComponentRemoteWorker(ModuleManager moduleManager) { super(moduleManager); } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeComponentSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingAggregationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingAggregationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingAggregationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingAggregationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java similarity index 97% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java index 6f8b8fc4e..56a4f8734 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingRemoteWorker.java @@ -31,7 +31,7 @@ import org.skywalking.apm.collector.stream.worker.base.WorkerException; */ public class NodeMappingRemoteWorker extends AbstractRemoteWorker { - public NodeMappingRemoteWorker(ModuleManager moduleManager) { + NodeMappingRemoteWorker(ModuleManager moduleManager) { super(moduleManager); } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java similarity index 98% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java index d1ac5bfaf..e42275d98 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/node/NodeMappingSpanListener.java @@ -67,7 +67,7 @@ public class NodeMappingSpanListener implements RefsListener, FirstSpanListener for (NodeMapping nodeMapping : nodeMappings) { nodeMapping.setId(timeBucket + Const.ID_SPLIT + nodeMapping.getId()); nodeMapping.setTimeBucket(timeBucket); - logger.debug("send to node mapping aggregation worker, id: {}", nodeMapping.getId()); + logger.debug("push to node mapping aggregation worker, id: {}", nodeMapping.getId()); graph.start(nodeMapping); } } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceAggregationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceAggregationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceAggregationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceAggregationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferencePersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferencePersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferencePersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferencePersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java similarity index 97% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java index f12a7a1e7..a06dcc133 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceRemoteWorker.java @@ -31,7 +31,7 @@ import org.skywalking.apm.collector.stream.worker.base.WorkerException; */ public class NodeReferenceRemoteWorker extends AbstractRemoteWorker { - public NodeReferenceRemoteWorker(ModuleManager moduleManager) { + NodeReferenceRemoteWorker(ModuleManager moduleManager) { super(moduleManager); } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/noderef/NodeReferenceSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostSpanListener.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentCostSpanListener.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/segment/SegmentPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryAggregationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryAggregationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryAggregationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryAggregationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryPersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryPersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryPersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryPersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryRemoteWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntryRemoteWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java similarity index 98% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java index 0312bbf6d..1540fcf89 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/service/ServiceEntrySpanListener.java @@ -84,7 +84,7 @@ public class ServiceEntrySpanListener implements RefsListener, FirstSpanListener serviceEntry.setRegisterTime(timeBucket); serviceEntry.setNewestTime(timeBucket); - logger.debug("send to service entry aggregation worker, id: {}", serviceEntry.getId()); + logger.debug("push to service entry aggregation worker, id: {}", serviceEntry.getId()); Graph graph = GraphManager.INSTANCE.createIfAbsent(TraceStreamGraph.SERVICE_ENTRY_GRAPH_ID, ServiceEntry.class); graph.start(serviceEntry); } diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceAggregationWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceAggregationWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceAggregationWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceAggregationWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferencePersistenceWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferencePersistenceWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferencePersistenceWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferencePersistenceWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceRemoteWorker.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceRemoteWorker.java similarity index 100% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceRemoteWorker.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceRemoteWorker.java diff --git a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java similarity index 98% rename from apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java index 45aa2a1e3..9c8256d74 100644 --- a/apm-collector/apm-collector-agent/collector-agent-stream/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/java/org/skywalking/apm/collector/agent/stream/worker/trace/serviceref/ServiceReferenceSpanListener.java @@ -130,7 +130,7 @@ public class ServiceReferenceSpanListener implements FirstSpanListener, EntrySpa serviceReference.setId(idBuilder.toString()); serviceReference.setTimeBucket(timeBucket); - logger.debug("send to service reference aggregation worker, id: {}", serviceReference.getId()); + logger.debug("push to service reference aggregation worker, id: {}", serviceReference.getId()); Graph graph = GraphManager.INSTANCE.createIfAbsent(TraceStreamGraph.SERVICE_REFERENCE_GRAPH_ID, ServiceReference.class); graph.start(serviceReference); diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider similarity index 91% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider rename to apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider index ee323473e..134461a3b 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider +++ b/apm-collector/apm-collector-agent-stream/collector-agent-stream-provider/src/main/resources/META-INF/services/org.skywalking.apm.collector.core.module.ModuleProvider @@ -16,4 +16,4 @@ # Project repository: https://github.com/OpenSkywalking/skywalking # -org.skywalking.apm.collector.stream.StreamModuleProvider \ No newline at end of file +org.skywalking.apm.collector.agent.stream.AgentStreamModuleProvider \ No newline at end of file diff --git a/apm-collector/apm-collector-agent-stream/pom.xml b/apm-collector/apm-collector-agent-stream/pom.xml new file mode 100644 index 000000000..f1a564bf1 --- /dev/null +++ b/apm-collector/apm-collector-agent-stream/pom.xml @@ -0,0 +1,44 @@ + + + + + + apm-collector + org.skywalking + 3.3.0-2017 + + 4.0.0 + + apm-collector-agent-stream + pom + + collector-agent-stream-define + collector-agent-stream-provider + + + + + org.skywalking + apm-collector-core + ${project.version} + + + \ No newline at end of file diff --git a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java b/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java deleted file mode 100644 index e1d86f431..000000000 --- a/apm-collector/apm-collector-agent/collector-agent-grpc-provider/src/main/java/org/skywalking/apm/collector/agent/grpc/handler/JVMMetricsServiceHandler.java +++ /dev/null @@ -1,148 +0,0 @@ -/* - * Copyright 2017, OpenSkywalking Organization All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - * Project repository: https://github.com/OpenSkywalking/skywalking - */ - -package org.skywalking.apm.collector.agent.grpc.handler; - -import io.grpc.stub.StreamObserver; -import java.util.List; -import org.skywalking.apm.collector.agent.stream.graph.JvmMetricStreamGraph; -import org.skywalking.apm.collector.core.graph.Graph; -import org.skywalking.apm.collector.core.graph.GraphManager; -import org.skywalking.apm.collector.core.util.Const; -import org.skywalking.apm.collector.core.util.TimeBucketUtils; -import org.skywalking.apm.collector.server.grpc.GRPCHandler; -import org.skywalking.apm.collector.storage.table.jvm.CpuMetric; -import org.skywalking.apm.collector.storage.table.jvm.GCMetric; -import org.skywalking.apm.collector.storage.table.jvm.MemoryMetric; -import org.skywalking.apm.collector.storage.table.jvm.MemoryPoolMetric; -import org.skywalking.apm.collector.storage.table.register.Instance; -import org.skywalking.apm.network.proto.CPU; -import org.skywalking.apm.network.proto.Downstream; -import org.skywalking.apm.network.proto.GC; -import org.skywalking.apm.network.proto.JVMMetrics; -import org.skywalking.apm.network.proto.JVMMetricsServiceGrpc; -import org.skywalking.apm.network.proto.Memory; -import org.skywalking.apm.network.proto.MemoryPool; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * @author peng-yongsheng - */ -public class JVMMetricsServiceHandler extends JVMMetricsServiceGrpc.JVMMetricsServiceImplBase implements GRPCHandler { - - private final Logger logger = LoggerFactory.getLogger(JVMMetricsServiceHandler.class); - - private final Graph memoryMetricGraph; - private final Graph memoryPoolMetricGraph; - private final Graph gcMetricGraph; - private final Graph cpuMetricGraph; - private final Graph heartBeatGraph; - - public JVMMetricsServiceHandler() { - memoryMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.MEMORY_METRIC_GRAPH_ID, MemoryMetric.class); - memoryPoolMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.MEMORY_POOL_METRIC_GRAPH_ID, MemoryPoolMetric.class); - gcMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.GC_METRIC_GRAPH_ID, GCMetric.class); - cpuMetricGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.CPU_METRIC_GRAPH_ID, CpuMetric.class); - heartBeatGraph = GraphManager.INSTANCE.createIfAbsent(JvmMetricStreamGraph.INST_HEART_BEAT_GRAPH_ID, Instance.class); - } - - @Override public void collect(JVMMetrics request, StreamObserver responseObserver) { - int instanceId = request.getApplicationInstanceId(); - logger.debug("receive the jvm metric from application instance, id: {}", instanceId); - - request.getMetricsList().forEach(metric -> { - long time = TimeBucketUtils.INSTANCE.getSecondTimeBucket(metric.getTime()); - sendToInstanceHeartBeatPersistenceWorker(instanceId, metric.getTime()); - sendToCpuMetricPersistenceWorker(instanceId, time, metric.getCpu()); - sendToMemoryMetricPersistenceWorker(instanceId, time, metric.getMemoryList()); - sendToMemoryPoolMetricPersistenceWorker(instanceId, time, metric.getMemoryPoolList()); - sendToGCMetricPersistenceWorker(instanceId, time, metric.getGcList()); - }); - - responseObserver.onNext(Downstream.newBuilder().build()); - responseObserver.onCompleted(); - } - - private void sendToInstanceHeartBeatPersistenceWorker(int instanceId, long heartBeatTime) { - Instance instance = new Instance(String.valueOf(instanceId)); - instance.setHeartBeatTime(TimeBucketUtils.INSTANCE.getSecondTimeBucket(heartBeatTime)); - instance.setInstanceId(instanceId); - - logger.debug("send to instance heart beat persistence worker, id: {}", instance.getId()); - heartBeatGraph.start(instance); - } - - private void sendToCpuMetricPersistenceWorker(int instanceId, long timeBucket, CPU cpu) { - CpuMetric cpuMetric = new CpuMetric(timeBucket + Const.ID_SPLIT + instanceId); - cpuMetric.setInstanceId(instanceId); - cpuMetric.setUsagePercent(cpu.getUsagePercent()); - cpuMetric.setTimeBucket(timeBucket); - - logger.debug("send to cpu metric graph, id: {}", cpuMetric.getId()); - cpuMetricGraph.start(cpuMetric); - } - - private void sendToMemoryMetricPersistenceWorker(int instanceId, long timeBucket, List memories) { - memories.forEach(memory -> { - MemoryMetric memoryMetric = new MemoryMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(memory.getIsHeap())); - memoryMetric.setInstanceId(instanceId); - memoryMetric.setIsHeap(memory.getIsHeap()); - memoryMetric.setInit(memory.getInit()); - memoryMetric.setMax(memory.getMax()); - memoryMetric.setUsed(memory.getUsed()); - memoryMetric.setCommitted(memory.getCommitted()); - memoryMetric.setTimeBucket(timeBucket); - - logger.debug("send to memory metric graph, id: {}", memoryMetric.getId()); - memoryMetricGraph.start(memoryMetric); - }); - } - - private void sendToMemoryPoolMetricPersistenceWorker(int instanceId, long timeBucket, - List memoryPools) { - - memoryPools.forEach(memoryPool -> { - MemoryPoolMetric memoryPoolMetric = new MemoryPoolMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(memoryPool.getType().getNumber())); - memoryPoolMetric.setInstanceId(instanceId); - memoryPoolMetric.setPoolType(memoryPool.getType().getNumber()); - memoryPoolMetric.setInit(memoryPool.getInit()); - memoryPoolMetric.setMax(memoryPool.getMax()); - memoryPoolMetric.setUsed(memoryPool.getUsed()); - memoryPoolMetric.setCommitted(memoryPool.getCommited()); - memoryPoolMetric.setTimeBucket(timeBucket); - - logger.debug("send to memory pool metric graph, id: {}", memoryPoolMetric.getId()); - memoryPoolMetricGraph.start(memoryPoolMetric); - }); - } - - private void sendToGCMetricPersistenceWorker(int instanceId, long timeBucket, List gcs) { - gcs.forEach(gc -> { - GCMetric gcMetric = new GCMetric(timeBucket + Const.ID_SPLIT + instanceId + Const.ID_SPLIT + String.valueOf(gc.getPhraseValue())); - gcMetric.setInstanceId(instanceId); - gcMetric.setPhrase(gc.getPhraseValue()); - gcMetric.setCount(gc.getCount()); - gcMetric.setTime(gc.getTime()); - gcMetric.setTimeBucket(timeBucket); - - logger.debug("send to gc metric graph, id: {}", gcMetric.getId()); - gcMetricGraph.start(gcMetric); - }); - } -} diff --git a/apm-collector/apm-collector-agent/pom.xml b/apm-collector/apm-collector-agent/pom.xml deleted file mode 100644 index 84cc91273..000000000 --- a/apm-collector/apm-collector-agent/pom.xml +++ /dev/null @@ -1,38 +0,0 @@ - - - - apm-collector - org.skywalking - 3.2.5-2017 - - 4.0.0 - - apm-collector-agent - pom - - collector-agent-define - collector-agent-jetty-provider - collector-agent-grpc-provider - collector-agent-stream - - - - - org.skywalking - apm-collector-core - ${project.version} - - - org.skywalking - collector-cluster-define - ${project.version} - - - org.skywalking - collector-naming-define - ${project.version} - - - diff --git a/apm-collector/apm-collector-baseline/collector-baseline-computing-define/pom.xml b/apm-collector/apm-collector-baseline/collector-baseline-computing-define/pom.xml index cfeca1c63..956d41523 100644 --- a/apm-collector/apm-collector-baseline/collector-baseline-computing-define/pom.xml +++ b/apm-collector/apm-collector-baseline/collector-baseline-computing-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-baseline org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-baseline/collector-baseline-computing-provider/pom.xml b/apm-collector/apm-collector-baseline/collector-baseline-computing-provider/pom.xml index 8b2b26cad..ad9b9d61e 100644 --- a/apm-collector/apm-collector-baseline/collector-baseline-computing-provider/pom.xml +++ b/apm-collector/apm-collector-baseline/collector-baseline-computing-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-baseline org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-baseline/collector-baseline-scheduler-define/pom.xml b/apm-collector/apm-collector-baseline/collector-baseline-scheduler-define/pom.xml index 3c50590ef..5a0cc49bf 100644 --- a/apm-collector/apm-collector-baseline/collector-baseline-scheduler-define/pom.xml +++ b/apm-collector/apm-collector-baseline/collector-baseline-scheduler-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-baseline org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-baseline/pom.xml b/apm-collector/apm-collector-baseline/pom.xml index afbfa74fc..7c9c05140 100644 --- a/apm-collector/apm-collector-baseline/pom.xml +++ b/apm-collector/apm-collector-baseline/pom.xml @@ -23,7 +23,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-boot/bin/collectorService.bat b/apm-collector/apm-collector-boot/bin/collectorService.bat new file mode 100644 index 000000000..08660014c --- /dev/null +++ b/apm-collector/apm-collector-boot/bin/collectorService.bat @@ -0,0 +1,21 @@ +@echo off + +setlocal +set COLLECTOR_PROCESS_TITLE=Skywalking-Collector +set COLLECTOR_HOME=%~dp0%.. +set COLLECTOR_OPTS="-Xms256M -Xmx512M -Dcollector.logDir=%COLLECTOR_HOME%\logs" + +set CLASSPATH=%COLLECTOR_HOME%\config;.; +set CLASSPATH=%COLLECTOR_HOME%\libs\*;%CLASSPATH% + +if defined JAVA_HOME ( + set _EXECJAVA="%JAVA_HOME:"=%"\bin\java +) + +if not defined JAVA_HOME ( + echo "JAVA_HOME not set." + set _EXECJAVA=java +) + +start "%COLLECTOR_PROCESS_TITLE%" %_EXECJAVA% "%COLLECTOR_OPTS%" -cp "%CLASSPATH%" org.skywalking.apm.collector.boot.CollectorBootStartUp +endlocal diff --git a/apm-collector/apm-collector-boot/bin/collectorService.sh b/apm-collector/apm-collector-boot/bin/collectorService.sh new file mode 100644 index 000000000..05e1f17d0 --- /dev/null +++ b/apm-collector/apm-collector-boot/bin/collectorService.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env sh + +PRG="$0" +PRGDIR=`dirname "$PRG"` +[ -z "$COLLECTOR_HOME" ] && COLLECTOR_HOME=`cd "$PRGDIR/.." >/dev/null; pwd` + +COLLECT_LOG_DIR="${COLLECTOR_HOME}/logs" +JAVA_OPTS=" -Xms256M -Xmx512M" + +if [ ! -d "${COLLECTOR_HOME}/logs" ]; then + mkdir -p "${COLLECT_LOG_DIR}" +fi + +_RUNJAVA=${JAVA_HOME}/bin/java +[ -z "$JAVA_HOME" ] && _RUNJAVA=java + +CLASSPATH="$COLLECTOR_HOME/config:$CLASSPATH" +for i in "$COLLECTOR_HOME"/libs/*.jar +do + CLASSPATH="$i:$CLASSPATH" +done + +COLLECTOR_OPTIONS=" -Dcollector.logDir=${COLLECT_LOG_DIR}" + +eval exec "\"$_RUNJAVA\" ${JAVA_OPTS} ${COLLECTOR_OPTIONS} -classpath $CLASSPATH org.skywalking.apm.collector.boot.CollectorBootStartUp \ + 2>${COLLECT_LOG_DIR}/collector.log 1> /dev/null &" + +if [ $? -eq 0 ]; then + sleep 1 + echo "Skywalking Web started successfully!" +else + echo "Skywalking Web started failure!" + exit 1 +fi diff --git a/apm-collector/apm-collector-boot/bin/startup.bat b/apm-collector/apm-collector-boot/bin/startup.bat new file mode 100644 index 000000000..d1f9f401d --- /dev/null +++ b/apm-collector/apm-collector-boot/bin/startup.bat @@ -0,0 +1,5 @@ +@echo off + +setlocal +call "%~dp0"\collectorService.bat start +endlocal diff --git a/apm-collector/apm-collector-boot/bin/startup.sh b/apm-collector/apm-collector-boot/bin/startup.sh new file mode 100644 index 000000000..d8f441aa7 --- /dev/null +++ b/apm-collector/apm-collector-boot/bin/startup.sh @@ -0,0 +1,7 @@ +#!/usr/bin/env sh + +PRG="$0" +PRGDIR=`dirname "$PRG"` +EXECUTABLE=collectorService.sh + +exec "$PRGDIR"/"$EXECUTABLE" start diff --git a/apm-collector/apm-collector-boot/docker/Dockerfile b/apm-collector/apm-collector-boot/docker/Dockerfile new file mode 100644 index 000000000..bfc1dcad9 --- /dev/null +++ b/apm-collector/apm-collector-boot/docker/Dockerfile @@ -0,0 +1,29 @@ +FROM openjdk:8u111-jdk + +ENV ZK_ADDRESSES=127.0.0.1:2181 \ + ES_CLUSTER_NAME=CollectorDBCluster \ + ES_ADDRESSES=localhost:9300 \ + BIND_HOST=localhost \ + NAMING_BIND_HOST=localhost \ + NAMING_BIND_PORT=10800 \ + REMOTE_BIND_PORT=11800 \ + AGENT_GRPC_BIND_PORT=11800 \ + AGENT_JETTY_BIND_HOST=localhost \ + AGENT_JETTY_BIND_PORT=12800 \ + UI_JETTY_BIND_PORT=12800 \ + UI_JETTY_BIND_HOST=localhost + +ADD skywalking-collector.tar.gz /usr/local +COPY collectorService.sh /usr/local/skywalking-collector/bin +COPY log4j2.xml /usr/local/skywalking-collector/config +COPY application.yml /usr/local/skywalking-collector/config +COPY docker-entrypoint.sh / + +RUN chmod +x /usr/local/skywalking-collector/bin/collectorService.sh && chmod +x /docker-entrypoint.sh + +EXPOSE 10800 +EXPOSE 11800 +EXPOSE 12800 + +ENTRYPOINT ["/docker-entrypoint.sh"] +CMD ["/usr/local/skywalking-collector/bin/collectorService.sh"] diff --git a/apm-collector/apm-collector-boot/docker/application.yml b/apm-collector/apm-collector-boot/docker/application.yml new file mode 100644 index 000000000..d3b4f3c74 --- /dev/null +++ b/apm-collector/apm-collector-boot/docker/application.yml @@ -0,0 +1,40 @@ +cluster: + zookeeper: + hostPort: {ZK_ADDRESSES} + sessionTimeout: 100000 +naming: + jetty: + host: {NAMING_BIND_HOST} + port: {NAMING_BIND_PORT} + context_path: / +remote: + gRPC: + host: {BIND_HOST} + port: {REMOTE_BIND_PORT} +agent_gRPC: + gRPC: + host: {BIND_HOST} + port: {AGENT_GRPC_BIND_PORT} +agent_jetty: + jetty: + host: {AGENT_JETTY_BIND_HOST} + port: {AGENT_JETTY_BIND_PORT} + context_path: / +agent_stream: + default: + buffer_file_path: ../buffer/ + buffer_offset_max_file_size: 10M + buffer_segment_max_file_size: 500M +ui: + jetty: + host: {UI_JETTY_BIND_HOST} + port: {UI_JETTY_BIND_PORT} + context_path: / +storage: + elasticsearch: + cluster_name: {ES_CLUSTER_NAME} + cluster_transport_sniffer: true + cluster_nodes: {ES_ADDRESSES} + index_shards_number: 2 + index_replicas_number: 0 + ttl: 7 diff --git a/apm-collector/apm-collector-boot/docker/collectorService.sh b/apm-collector/apm-collector-boot/docker/collectorService.sh new file mode 100644 index 000000000..919015eee --- /dev/null +++ b/apm-collector/apm-collector-boot/docker/collectorService.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env sh + +PRG="$0" +PRGDIR=`dirname "$PRG"` +[ -z "$COLLECTOR_HOME" ] && COLLECTOR_HOME=`cd "$PRGDIR/.." >/dev/null; pwd` + +COLLECT_LOG_DIR="${COLLECTOR_HOME}/logs" +JAVA_OPTS=" -Xms256M -Xmx512M" + +if [ ! -d "${COLLECTOR_HOME}/logs" ]; then + mkdir -p "${COLLECT_LOG_DIR}" +fi + +_RUNJAVA=${JAVA_HOME}/bin/java +[ -z "$JAVA_HOME" ] && _RUNJAVA=java + +CLASSPATH="$COLLECTOR_HOME/config:$CLASSPATH" +for i in "$COLLECTOR_HOME"/libs/*.jar +do + CLASSPATH="$i:$CLASSPATH" +done + +${_RUNJAVA} ${JAVA_OPTS} -classpath $CLASSPATH org.skywalking.apm.collector.boot.CollectorBootStartUp diff --git a/apm-collector/apm-collector-boot/docker/docker-entrypoint.sh b/apm-collector/apm-collector-boot/docker/docker-entrypoint.sh new file mode 100644 index 000000000..ab9fe8e7e --- /dev/null +++ b/apm-collector/apm-collector-boot/docker/docker-entrypoint.sh @@ -0,0 +1,40 @@ +#!/bin/sh + +echo "replace {ZK_ADDRESSES} to ${ZK_ADDRESSES}" +eval sed -i -e 's/\{ZK_ADDRESSES\}/${ZK_ADDRESSES}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {ES_CLUSTER_NAME} to ${ES_CLUSTER_NAME}" +eval sed -i -e 's/\{ES_CLUSTER_NAME\}/${ES_CLUSTER_NAME}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {ES_ADDRESSES} to ${ES_ADDRESSES}" +eval sed -i -e 's/\{ES_ADDRESSES\}/${ES_ADDRESSES}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {BIND_HOST} to ${BIND_HOST}" +eval sed -i -e 's/\{BIND_HOST\}/${BIND_HOST}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {NAMING_BIND_HOST} to ${NAMING_BIND_HOST}" +eval sed -i -e 's/\{NAMING_BIND_HOST\}/${NAMING_BIND_HOST}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {NAMING_BIND_PORT} to ${NAMING_BIND_PORT}" +eval sed -i -e 's/\{NAMING_BIND_PORT\}/${NAMING_BIND_PORT}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {REMOTE_BIND_PORT} to ${REMOTE_BIND_PORT}" +eval sed -i -e 's/\{REMOTE_BIND_PORT\}/${REMOTE_BIND_PORT}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {AGENT_GRPC_BIND_PORT} to ${AGENT_GRPC_BIND_PORT}" +eval sed -i -e 's/\{AGENT_GRPC_BIND_PORT\}/${AGENT_GRPC_BIND_PORT}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {AGENT_JETTY_BIND_HOST} to ${AGENT_JETTY_BIND_HOST}" +eval sed -i -e 's/\{AGENT_JETTY_BIND_HOST\}/${AGENT_JETTY_BIND_HOST}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {AGENT_JETTY_BIND_PORT} to ${AGENT_JETTY_BIND_PORT}" +eval sed -i -e 's/\{AGENT_JETTY_BIND_PORT\}/${AGENT_JETTY_BIND_PORT}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {UI_JETTY_BIND_PORT} to ${UI_JETTY_BIND_PORT}" +eval sed -i -e 's/\{UI_JETTY_BIND_PORT\}/${UI_JETTY_BIND_PORT}/' /usr/local/skywalking-collector/config/application.yml + +echo "replace {UI_JETTY_BIND_HOST} to ${UI_JETTY_BIND_HOST}" +eval sed -i -e 's/\{UI_JETTY_BIND_HOST\}/${UI_JETTY_BIND_HOST}/' /usr/local/skywalking-collector/config/application.yml + + +exec "$@" diff --git a/apm-collector/apm-collector-boot/docker/log4j2.xml b/apm-collector/apm-collector-boot/docker/log4j2.xml new file mode 100644 index 000000000..e6915ec9a --- /dev/null +++ b/apm-collector/apm-collector-boot/docker/log4j2.xml @@ -0,0 +1,36 @@ + + + + + + + + + + + + + + + + + + + + diff --git a/apm-collector/apm-collector-boot/pom.xml b/apm-collector/apm-collector-boot/pom.xml index b9873cc2c..caec74d40 100644 --- a/apm-collector/apm-collector-boot/pom.xml +++ b/apm-collector/apm-collector-boot/pom.xml @@ -23,13 +23,18 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 apm-collector-boot jar + + skywalking/skywalking-collector + ${version} + + org.yaml @@ -79,18 +84,27 @@ ${project.version} - + org.skywalking collector-agent-grpc-provider ${project.version} + + org.skywalking collector-agent-jetty-provider ${project.version} - + + + + org.skywalking + collector-agent-stream-provider + ${project.version} + + org.skywalking @@ -129,13 +143,6 @@ ${project.version} - - - org.skywalking - collector-stream-provider - ${project.version} - - org.skywalking @@ -144,4 +151,76 @@ - \ No newline at end of file + + + skywalking-collector + + + maven-compiler-plugin + + ${compiler.version} + ${compiler.version} + ${project.build.sourceEncoding} + + + + org.apache.maven.plugins + maven-resources-plugin + 2.4.3 + + ${project.build.sourceEncoding} + + + + org.apache.maven.plugins + maven-jar-plugin + 2.3.2 + + + application.yml + log4j2.xml + + + + + org.apache.maven.plugins + maven-assembly-plugin + + + assembly + package + + single + + + + src/main/assembly/assembly.xml + + ${project.basedir}/../../packages + + + + + + com.spotify + docker-maven-plugin + ${docker.plugin.version} + + false + ${docker.image.name} + + ${docker.image.version} + + ${project.basedir}/docker + + + / + ${project.basedir}/../../packages + ${build.finalName}.tar.gz + + + + + + + diff --git a/apm-collector/apm-collector-boot/src/main/assembly/assembly.xml b/apm-collector/apm-collector-boot/src/main/assembly/assembly.xml new file mode 100644 index 000000000..683048fbf --- /dev/null +++ b/apm-collector/apm-collector-boot/src/main/assembly/assembly.xml @@ -0,0 +1,67 @@ + + + + + + zip + tar.gz + + + + /libs + runtime + + + + + ${project.basedir}/bin + /bin + + *.sh + *.bat + + 0755 + + + src/main/resources + /config + + application.yml + + + + src/main/assembly + /config + + log4j2.xml + + + + src/main/resources + + application.yml + log4j2.xml + + /config + + + diff --git a/apm-collector/apm-collector-boot/src/main/assembly/log4j2.xml b/apm-collector/apm-collector-boot/src/main/assembly/log4j2.xml new file mode 100644 index 000000000..ffb1ef68e --- /dev/null +++ b/apm-collector/apm-collector-boot/src/main/assembly/log4j2.xml @@ -0,0 +1,46 @@ + + + + + + ${sys:collector.logDir} + + + + + %d - %c -%-4r [%t] %-5p %x - %m%n + + + + + + + + + + + + + + + + + + diff --git a/apm-collector/apm-collector-boot/src/main/resources/application.yml b/apm-collector/apm-collector-boot/src/main/resources/application.yml index 254f84d23..2ba018472 100644 --- a/apm-collector/apm-collector-boot/src/main/resources/application.yml +++ b/apm-collector/apm-collector-boot/src/main/resources/application.yml @@ -7,15 +7,22 @@ naming: host: localhost port: 10800 context_path: / -agent: +remote: gRPC: host: localhost port: 11800 +agent_gRPC: + gRPC: + host: localhost + port: 11800 +agent_jetty: jetty: host: localhost port: 12800 context_path: / - config: +agent_stream: + default: + buffer_file_path: ../buffer/ buffer_offset_max_file_size: 10M buffer_segment_max_file_size: 500M ui: @@ -23,10 +30,11 @@ ui: host: localhost port: 12800 context_path: / -storage: - elasticsearch: - cluster_name: CollectorDBCluster - cluster_transport_sniffer: true - cluster_nodes: localhost:9300 - index_shards_number: 2 - index_replicas_number: 0 \ No newline at end of file +#storage: +# elasticsearch: +# cluster_name: CollectorDBCluster +# cluster_transport_sniffer: true +# cluster_nodes: localhost:9300 +# index_shards_number: 2 +# index_replicas_number: 0 +# ttl: 7 \ No newline at end of file diff --git a/apm-collector/apm-collector-boot/src/main/resources/log4j2.xml b/apm-collector/apm-collector-boot/src/main/resources/log4j2.xml index db54a6402..e6915ec9a 100644 --- a/apm-collector/apm-collector-boot/src/main/resources/log4j2.xml +++ b/apm-collector/apm-collector-boot/src/main/resources/log4j2.xml @@ -17,7 +17,7 @@ ~ Project repository: https://github.com/OpenSkywalking/skywalking --> - + @@ -29,7 +29,7 @@ - + diff --git a/apm-collector/apm-collector-cache/collector-cache-define/pom.xml b/apm-collector/apm-collector-cache/collector-cache-define/pom.xml index 76c0be88f..a8747780d 100644 --- a/apm-collector/apm-collector-cache/collector-cache-define/pom.xml +++ b/apm-collector/apm-collector-cache/collector-cache-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-cache org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cache/collector-cache-guava-provider/pom.xml b/apm-collector/apm-collector-cache/collector-cache-guava-provider/pom.xml index dda14c71e..b6233486e 100644 --- a/apm-collector/apm-collector-cache/collector-cache-guava-provider/pom.xml +++ b/apm-collector/apm-collector-cache/collector-cache-guava-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-cache org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -36,5 +36,10 @@ collector-cache-define ${project.version} + + com.google.guava + guava + 19.0 + \ No newline at end of file diff --git a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ApplicationCacheGuavaService.java b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ApplicationCacheGuavaService.java index 80770f160..743390be9 100644 --- a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ApplicationCacheGuavaService.java +++ b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ApplicationCacheGuavaService.java @@ -23,6 +23,7 @@ import com.google.common.cache.CacheBuilder; import org.skywalking.apm.collector.cache.service.ApplicationCacheService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.core.util.StringUtils; import org.skywalking.apm.collector.storage.StorageModule; import org.skywalking.apm.collector.storage.dao.IApplicationCacheDAO; @@ -38,22 +39,30 @@ public class ApplicationCacheGuavaService implements ApplicationCacheService { private final Cache codeCache = CacheBuilder.newBuilder().initialCapacity(100).maximumSize(1000).build(); - private final IApplicationCacheDAO applicationCacheDAO; + private final ModuleManager moduleManager; + private IApplicationCacheDAO applicationCacheDAO; public ApplicationCacheGuavaService(ModuleManager moduleManager) { - this.applicationCacheDAO = moduleManager.find(StorageModule.NAME).getService(IApplicationCacheDAO.class); + this.moduleManager = moduleManager; + } + + private IApplicationCacheDAO getApplicationCacheDAO() { + if (ObjectUtils.isEmpty(applicationCacheDAO)) { + this.applicationCacheDAO = moduleManager.find(StorageModule.NAME).getService(IApplicationCacheDAO.class); + } + return this.applicationCacheDAO; } public int get(String applicationCode) { int applicationId = 0; try { - applicationId = codeCache.get(applicationCode, () -> applicationCacheDAO.getApplicationId(applicationCode)); + applicationId = codeCache.get(applicationCode, () -> getApplicationCacheDAO().getApplicationId(applicationCode)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (applicationId == 0) { - applicationId = applicationCacheDAO.getApplicationId(applicationCode); + applicationId = getApplicationCacheDAO().getApplicationId(applicationCode); if (applicationId != 0) { codeCache.put(applicationCode, applicationId); } @@ -66,13 +75,13 @@ public class ApplicationCacheGuavaService implements ApplicationCacheService { public String get(int applicationId) { String applicationCode = Const.EMPTY_STRING; try { - applicationCode = idCache.get(applicationId, () -> applicationCacheDAO.getApplicationCode(applicationId)); + applicationCode = idCache.get(applicationId, () -> getApplicationCacheDAO().getApplicationCode(applicationId)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (StringUtils.isEmpty(applicationCode)) { - applicationCode = applicationCacheDAO.getApplicationCode(applicationId); + applicationCode = getApplicationCacheDAO().getApplicationCode(applicationId); if (StringUtils.isNotEmpty(applicationCode)) { codeCache.put(applicationCode, applicationId); } diff --git a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/InstanceCacheGuavaService.java b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/InstanceCacheGuavaService.java index 330b3b9a1..0f2e9a27e 100644 --- a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/InstanceCacheGuavaService.java +++ b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/InstanceCacheGuavaService.java @@ -23,6 +23,7 @@ import com.google.common.cache.CacheBuilder; import org.skywalking.apm.collector.cache.service.InstanceCacheService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.storage.StorageModule; import org.skywalking.apm.collector.storage.dao.IInstanceCacheDAO; import org.slf4j.Logger; @@ -39,23 +40,31 @@ public class InstanceCacheGuavaService implements InstanceCacheService { private final Cache stringCache = CacheBuilder.newBuilder().initialCapacity(100).maximumSize(5000).build(); - private final IInstanceCacheDAO instanceCacheDAO; + private final ModuleManager moduleManager; + private IInstanceCacheDAO instanceCacheDAO; public InstanceCacheGuavaService(ModuleManager moduleManager) { - this.instanceCacheDAO = moduleManager.find(StorageModule.NAME).getService(IInstanceCacheDAO.class); + this.moduleManager = moduleManager; + } + + private IInstanceCacheDAO getInstanceCacheDAO() { + if (ObjectUtils.isEmpty(instanceCacheDAO)) { + this.instanceCacheDAO = moduleManager.find(StorageModule.NAME).getService(IInstanceCacheDAO.class); + } + return this.instanceCacheDAO; } public int get(int applicationInstanceId) { int applicationId = 0; try { - applicationId = integerCache.get(applicationInstanceId, () -> instanceCacheDAO.getApplicationId(applicationInstanceId)); + applicationId = integerCache.get(applicationInstanceId, () -> getInstanceCacheDAO().getApplicationId(applicationInstanceId)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (applicationId == 0) { - applicationId = instanceCacheDAO.getApplicationId(applicationInstanceId); + applicationId = getInstanceCacheDAO().getApplicationId(applicationInstanceId); if (applicationId != 0) { integerCache.put(applicationInstanceId, applicationId); } @@ -68,13 +77,13 @@ public class InstanceCacheGuavaService implements InstanceCacheService { int instanceId = 0; try { - instanceId = stringCache.get(key, () -> instanceCacheDAO.getInstanceId(applicationId, agentUUID)); + instanceId = stringCache.get(key, () -> getInstanceCacheDAO().getInstanceId(applicationId, agentUUID)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (instanceId == 0) { - instanceId = instanceCacheDAO.getInstanceId(applicationId, agentUUID); + instanceId = getInstanceCacheDAO().getInstanceId(applicationId, agentUUID); if (applicationId != 0) { stringCache.put(key, instanceId); } diff --git a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceIdCacheGuavaService.java b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceIdCacheGuavaService.java index eb59ec750..85d6aa5aa 100644 --- a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceIdCacheGuavaService.java +++ b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceIdCacheGuavaService.java @@ -23,6 +23,7 @@ import com.google.common.cache.CacheBuilder; import org.skywalking.apm.collector.cache.service.ServiceIdCacheService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.storage.StorageModule; import org.skywalking.apm.collector.storage.dao.IServiceNameCacheDAO; import org.slf4j.Logger; @@ -37,22 +38,30 @@ public class ServiceIdCacheGuavaService implements ServiceIdCacheService { private final Cache serviceIdCache = CacheBuilder.newBuilder().maximumSize(1000).build(); - private final IServiceNameCacheDAO serviceNameCacheDAO; + private final ModuleManager moduleManager; + private IServiceNameCacheDAO serviceNameCacheDAO; public ServiceIdCacheGuavaService(ModuleManager moduleManager) { - this.serviceNameCacheDAO = moduleManager.find(StorageModule.NAME).getService(IServiceNameCacheDAO.class); + this.moduleManager = moduleManager; + } + + private IServiceNameCacheDAO getServiceNameCacheDAO() { + if (ObjectUtils.isEmpty(serviceNameCacheDAO)) { + this.serviceNameCacheDAO = moduleManager.find(StorageModule.NAME).getService(IServiceNameCacheDAO.class); + } + return this.serviceNameCacheDAO; } public int get(int applicationId, String serviceName) { int serviceId = 0; try { - serviceId = serviceIdCache.get(applicationId + Const.ID_SPLIT + serviceName, () -> serviceNameCacheDAO.getServiceId(applicationId, serviceName)); + serviceId = serviceIdCache.get(applicationId + Const.ID_SPLIT + serviceName, () -> getServiceNameCacheDAO().getServiceId(applicationId, serviceName)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (serviceId == 0) { - serviceId = serviceNameCacheDAO.getServiceId(applicationId, serviceName); + serviceId = getServiceNameCacheDAO().getServiceId(applicationId, serviceName); if (serviceId != 0) { serviceIdCache.put(applicationId + Const.ID_SPLIT + serviceName, serviceId); } diff --git a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceNameCacheGuavaService.java b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceNameCacheGuavaService.java index 2b235ef9d..55b9d7792 100644 --- a/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceNameCacheGuavaService.java +++ b/apm-collector/apm-collector-cache/collector-cache-guava-provider/src/main/java/org/skywalking/apm/collector/cache/guava/service/ServiceNameCacheGuavaService.java @@ -23,6 +23,7 @@ import com.google.common.cache.CacheBuilder; import org.skywalking.apm.collector.cache.service.ServiceNameCacheService; import org.skywalking.apm.collector.core.module.ModuleManager; import org.skywalking.apm.collector.core.util.Const; +import org.skywalking.apm.collector.core.util.ObjectUtils; import org.skywalking.apm.collector.core.util.StringUtils; import org.skywalking.apm.collector.storage.StorageModule; import org.skywalking.apm.collector.storage.dao.IServiceNameCacheDAO; @@ -38,22 +39,30 @@ public class ServiceNameCacheGuavaService implements ServiceNameCacheService { private final Cache serviceNameCache = CacheBuilder.newBuilder().maximumSize(10000).build(); - private final IServiceNameCacheDAO serviceNameCacheDAO; + private final ModuleManager moduleManager; + private IServiceNameCacheDAO serviceNameCacheDAO; public ServiceNameCacheGuavaService(ModuleManager moduleManager) { - this.serviceNameCacheDAO = moduleManager.find(StorageModule.NAME).getService(IServiceNameCacheDAO.class); + this.moduleManager = moduleManager; + } + + private IServiceNameCacheDAO getServiceNameCacheDAO() { + if (ObjectUtils.isEmpty(serviceNameCacheDAO)) { + this.serviceNameCacheDAO = moduleManager.find(StorageModule.NAME).getService(IServiceNameCacheDAO.class); + } + return this.serviceNameCacheDAO; } public String get(int serviceId) { String serviceName = Const.EMPTY_STRING; try { - serviceName = serviceNameCache.get(serviceId, () -> serviceNameCacheDAO.getServiceName(serviceId)); + serviceName = serviceNameCache.get(serviceId, () -> getServiceNameCacheDAO().getServiceName(serviceId)); } catch (Throwable e) { logger.error(e.getMessage(), e); } if (StringUtils.isEmpty(serviceName)) { - serviceName = serviceNameCacheDAO.getServiceName(serviceId); + serviceName = getServiceNameCacheDAO().getServiceName(serviceId); if (StringUtils.isNotEmpty(serviceName)) { serviceNameCache.put(serviceId, serviceName); } diff --git a/apm-collector/apm-collector-cache/pom.xml b/apm-collector/apm-collector-cache/pom.xml index ed9b7b174..2f544b299 100644 --- a/apm-collector/apm-collector-cache/pom.xml +++ b/apm-collector/apm-collector-cache/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cluster/collector-cluster-define/pom.xml b/apm-collector/apm-collector-cluster/collector-cluster-define/pom.xml index c7e86c9f8..6aae47ecb 100644 --- a/apm-collector/apm-collector-cluster/collector-cluster-define/pom.xml +++ b/apm-collector/apm-collector-cluster/collector-cluster-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-cluster org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cluster/collector-cluster-redis-provider/pom.xml b/apm-collector/apm-collector-cluster/collector-cluster-redis-provider/pom.xml index b6ced5bdf..93f498265 100644 --- a/apm-collector/apm-collector-cluster/collector-cluster-redis-provider/pom.xml +++ b/apm-collector/apm-collector-cluster/collector-cluster-redis-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-cluster org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cluster/collector-cluster-standalone-provider/pom.xml b/apm-collector/apm-collector-cluster/collector-cluster-standalone-provider/pom.xml index 886c74d65..a20b3a6f0 100644 --- a/apm-collector/apm-collector-cluster/collector-cluster-standalone-provider/pom.xml +++ b/apm-collector/apm-collector-cluster/collector-cluster-standalone-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-cluster org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cluster/collector-cluster-zookeeper-provider/pom.xml b/apm-collector/apm-collector-cluster/collector-cluster-zookeeper-provider/pom.xml index 5a734bf16..23a1bb836 100644 --- a/apm-collector/apm-collector-cluster/collector-cluster-zookeeper-provider/pom.xml +++ b/apm-collector/apm-collector-cluster/collector-cluster-zookeeper-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-cluster org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-cluster/pom.xml b/apm-collector/apm-collector-cluster/pom.xml index 0f1a2cee7..9675ace2c 100644 --- a/apm-collector/apm-collector-cluster/pom.xml +++ b/apm-collector/apm-collector-cluster/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-component/client-component/pom.xml b/apm-collector/apm-collector-component/client-component/pom.xml index 8d1c5e84f..93c322fdf 100644 --- a/apm-collector/apm-collector-component/client-component/pom.xml +++ b/apm-collector/apm-collector-component/client-component/pom.xml @@ -5,7 +5,7 @@ apm-collector-component org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -32,6 +32,34 @@ snakeyaml org.yaml + + netty-common + io.netty + + + netty-transport + io.netty + + + netty-codec + io.netty + + + netty-codec-http + io.netty + + + netty-buffer + io.netty + + + netty-handler + io.netty + + + netty-resolver + io.netty + @@ -50,9 +78,15 @@ - io.grpc - grpc-core - 1.4.0 + org.skywalking + apm-network + ${project.version} + + + guava + com.google.guava + + diff --git a/apm-collector/apm-collector-component/client-component/src/main/java/org/skywalking/apm/collector/client/elasticsearch/ElasticSearchClient.java b/apm-collector/apm-collector-component/client-component/src/main/java/org/skywalking/apm/collector/client/elasticsearch/ElasticSearchClient.java index fa5df5e9a..f4843c7cd 100644 --- a/apm-collector/apm-collector-component/client-component/src/main/java/org/skywalking/apm/collector/client/elasticsearch/ElasticSearchClient.java +++ b/apm-collector/apm-collector-component/client-component/src/main/java/org/skywalking/apm/collector/client/elasticsearch/ElasticSearchClient.java @@ -37,6 +37,8 @@ import org.elasticsearch.client.IndicesAdminClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.transport.InetSocketTransportAddress; import org.elasticsearch.common.xcontent.XContentBuilder; +import org.elasticsearch.index.reindex.DeleteByQueryAction; +import org.elasticsearch.index.reindex.DeleteByQueryRequestBuilder; import org.elasticsearch.transport.client.PreBuiltTransportClient; import org.skywalking.apm.collector.client.Client; import org.skywalking.apm.collector.client.ClientException; @@ -146,6 +148,10 @@ public class ElasticSearchClient implements Client { return client.prepareGet(indexName, "type", id); } + public DeleteByQueryRequestBuilder prepareDelete() { + return DeleteByQueryAction.INSTANCE.newRequestBuilder(client); + } + public MultiGetRequestBuilder prepareMultiGet() { return client.prepareMultiGet(); } diff --git a/apm-collector/apm-collector-component/pom.xml b/apm-collector/apm-collector-component/pom.xml index 538c82fcf..c68c18766 100644 --- a/apm-collector/apm-collector-component/pom.xml +++ b/apm-collector/apm-collector-component/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-component/server-component/pom.xml b/apm-collector/apm-collector-component/server-component/pom.xml index 817f6385b..0ff3d1fcf 100644 --- a/apm-collector/apm-collector-component/server-component/pom.xml +++ b/apm-collector/apm-collector-component/server-component/pom.xml @@ -5,7 +5,7 @@ apm-collector-component org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -21,6 +21,12 @@ org.skywalking apm-network ${project.version} + + + guava + com.google.guava + + org.eclipse.jetty diff --git a/apm-collector/apm-collector-core/pom.xml b/apm-collector/apm-collector-core/pom.xml index d43d1fb6a..99b596a91 100644 --- a/apm-collector/apm-collector-core/pom.xml +++ b/apm-collector/apm-collector-core/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Collection.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Collection.java new file mode 100644 index 000000000..fe2003221 --- /dev/null +++ b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Collection.java @@ -0,0 +1,43 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.core.cache; + +/** + * @author peng-yongsheng + */ +public interface Collection { + + void reading(); + + boolean isReading(); + + void writing(); + + boolean isWriting(); + + void clear(); + + int size(); + + void finishReading(); + + void finishWriting(); + + Data collection(); +} diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/Window.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Window.java similarity index 74% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/Window.java rename to apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Window.java index a1b53450a..90e62e824 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/Window.java +++ b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/cache/Window.java @@ -16,28 +16,30 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.collector.stream.worker.impl.data; +package org.skywalking.apm.collector.core.cache; import java.util.concurrent.atomic.AtomicInteger; /** * @author peng-yongsheng */ -public abstract class Window { +public abstract class Window { private AtomicInteger windowSwitch = new AtomicInteger(0); - private DataCollection pointer; + private WINDOW_COLLECTION pointer; - private DataCollection windowDataA; - private DataCollection windowDataB; + private WINDOW_COLLECTION windowDataA; + private WINDOW_COLLECTION windowDataB; - public Window() { - windowDataA = new DataCollection(); - windowDataB = new DataCollection(); - pointer = windowDataA; + protected Window() { + this.windowDataA = collectionInstance(); + this.windowDataB = collectionInstance(); + this.pointer = windowDataA; } + public abstract WINDOW_COLLECTION collectionInstance(); + public boolean trySwitchPointer() { return windowSwitch.incrementAndGet() == 1 && !getLast().isReading(); } @@ -55,7 +57,7 @@ public abstract class Window { getLast().reading(); } - protected DataCollection getCurrentAndWriting() { + protected WINDOW_COLLECTION getCurrentAndWriting() { if (pointer == windowDataA) { windowDataA.writing(); return windowDataA; @@ -65,11 +67,11 @@ public abstract class Window { } } - protected DataCollection getCurrent() { + protected WINDOW_COLLECTION getCurrent() { return pointer; } - public DataCollection getLast() { + public WINDOW_COLLECTION getLast() { if (pointer == windowDataA) { return windowDataB; } else { diff --git a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/config/SystemConfig.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/config/SystemConfig.java deleted file mode 100644 index ec787f684..000000000 --- a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/config/SystemConfig.java +++ /dev/null @@ -1,26 +0,0 @@ -/* - * Copyright 2017, OpenSkywalking Organization All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - * Project repository: https://github.com/OpenSkywalking/skywalking - */ - -package org.skywalking.apm.collector.core.config; - -/** - * @author peng-yongsheng - */ -public class SystemConfig { - public static String DATA_PATH = "../data"; -} diff --git a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/data/Data.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/data/Data.java index 6f6beca17..cb0eb02df 100644 --- a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/data/Data.java +++ b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/data/Data.java @@ -183,6 +183,7 @@ public abstract class Data extends EndOfBatchQueueMessage { for (Boolean dataBoolean : dataBooleans) { dataStr.append(dataBoolean).append(","); } + dataStr.append("]"); return dataStr.toString(); } } diff --git a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/module/BootstrapFlow.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/module/BootstrapFlow.java index 9cd994750..63a5ba5f3 100644 --- a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/module/BootstrapFlow.java +++ b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/module/BootstrapFlow.java @@ -103,10 +103,12 @@ public class BootstrapFlow { if (isAllRequiredModuleStarted) { startupSequence.add(provider); allProviders.remove(i); + i--; } } else { startupSequence.add(provider); allProviders.remove(i); + i--; } } diff --git a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/util/CollectionUtils.java b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/util/CollectionUtils.java index a0dd89bdb..6946b7acd 100644 --- a/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/util/CollectionUtils.java +++ b/apm-collector/apm-collector-core/src/main/java/org/skywalking/apm/collector/core/util/CollectionUtils.java @@ -20,6 +20,7 @@ package org.skywalking.apm.collector.core.util; import java.util.List; import java.util.Map; +import java.util.Set; /** * @author peng-yongsheng @@ -34,10 +35,18 @@ public class CollectionUtils { return list == null || list.size() == 0; } + public static boolean isEmpty(Set set) { + return set == null || set.size() == 0; + } + public static boolean isNotEmpty(List list) { return !isEmpty(list); } + public static boolean isNotEmpty(Set set) { + return !isEmpty(set); + } + public static boolean isNotEmpty(Map map) { return !isEmpty(map); } diff --git a/apm-collector/apm-collector-core/src/main/resources/application-default.yml b/apm-collector/apm-collector-core/src/main/resources/application-default.yml index a77838189..3fb433933 100644 --- a/apm-collector/apm-collector-core/src/main/resources/application-default.yml +++ b/apm-collector/apm-collector-core/src/main/resources/application-default.yml @@ -6,15 +6,29 @@ cache: guava: queue: disruptor: -remote: - gRPC: - host: localhost - port: 11800 naming: jetty: host: localhost port: 10800 context_path: / +remote: + gRPC: + host: localhost + port: 11800 +agent_gRPC: + gRPC: + host: localhost + port: 11800 +agent_jetty: + jetty: + host: localhost + port: 12800 + context_path: / +agent_stream: + default: + buffer_file_path: ../buffer/ + buffer_offset_max_file_size: 10M + buffer_segment_max_file_size: 500M ui: jetty: host: localhost @@ -24,8 +38,6 @@ jetty_manager: jetty: gRPC_manager: gRPC: -stream: - worker: storage: h2: url: jdbc:h2:~/memorydb diff --git a/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-define/pom.xml b/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-define/pom.xml index c43f6aa9b..204658b86 100644 --- a/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-define/pom.xml +++ b/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-grpc-manager org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-provider/pom.xml b/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-provider/pom.xml index f4782a2d1..16e82e897 100644 --- a/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-provider/pom.xml +++ b/apm-collector/apm-collector-grpc-manager/collector-grpc-manager-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-grpc-manager org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-grpc-manager/pom.xml b/apm-collector/apm-collector-grpc-manager/pom.xml index 564557929..8f49f859b 100644 --- a/apm-collector/apm-collector-grpc-manager/pom.xml +++ b/apm-collector/apm-collector-grpc-manager/pom.xml @@ -23,7 +23,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-instrument/pom.xml b/apm-collector/apm-collector-instrument/pom.xml index 968d4fd40..5ef52b736 100644 --- a/apm-collector/apm-collector-instrument/pom.xml +++ b/apm-collector/apm-collector-instrument/pom.xml @@ -23,7 +23,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-define/pom.xml b/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-define/pom.xml index 2e0baae79..943717b19 100644 --- a/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-define/pom.xml +++ b/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-jetty-manager org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-provider/pom.xml b/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-provider/pom.xml index 42c70b869..bdc4e882a 100644 --- a/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-provider/pom.xml +++ b/apm-collector/apm-collector-jetty-manager/collector-jetty-manager-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-jetty-manager org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-jetty-manager/pom.xml b/apm-collector/apm-collector-jetty-manager/pom.xml index 2663829e3..0e731a269 100644 --- a/apm-collector/apm-collector-jetty-manager/pom.xml +++ b/apm-collector/apm-collector-jetty-manager/pom.xml @@ -23,7 +23,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-naming/collector-naming-define/pom.xml b/apm-collector/apm-collector-naming/collector-naming-define/pom.xml index 6685a0fa7..6ec82d3fc 100644 --- a/apm-collector/apm-collector-naming/collector-naming-define/pom.xml +++ b/apm-collector/apm-collector-naming/collector-naming-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-naming org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-naming/collector-naming-jetty-provider/pom.xml b/apm-collector/apm-collector-naming/collector-naming-jetty-provider/pom.xml index cc44483dd..14cbc73c8 100644 --- a/apm-collector/apm-collector-naming/collector-naming-jetty-provider/pom.xml +++ b/apm-collector/apm-collector-naming/collector-naming-jetty-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-naming org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-naming/pom.xml b/apm-collector/apm-collector-naming/pom.xml index 97db76736..cf466c859 100644 --- a/apm-collector/apm-collector-naming/pom.xml +++ b/apm-collector/apm-collector-naming/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-queue/collector-queue-datacarrier-provider/pom.xml b/apm-collector/apm-collector-queue/collector-queue-datacarrier-provider/pom.xml index 421c03b26..8c51bcff3 100644 --- a/apm-collector/apm-collector-queue/collector-queue-datacarrier-provider/pom.xml +++ b/apm-collector/apm-collector-queue/collector-queue-datacarrier-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-queue org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-queue/collector-queue-define/pom.xml b/apm-collector/apm-collector-queue/collector-queue-define/pom.xml index 288e558c0..4a78d2312 100644 --- a/apm-collector/apm-collector-queue/collector-queue-define/pom.xml +++ b/apm-collector/apm-collector-queue/collector-queue-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-queue org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-queue/collector-queue-disruptor-provider/pom.xml b/apm-collector/apm-collector-queue/collector-queue-disruptor-provider/pom.xml index 320151790..e162ac72e 100644 --- a/apm-collector/apm-collector-queue/collector-queue-disruptor-provider/pom.xml +++ b/apm-collector/apm-collector-queue/collector-queue-disruptor-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-queue org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-queue/pom.xml b/apm-collector/apm-collector-queue/pom.xml index 0d9f8876c..5e1c53d62 100644 --- a/apm-collector/apm-collector-queue/pom.xml +++ b/apm-collector/apm-collector-queue/pom.xml @@ -23,7 +23,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-remote/collector-remote-define/pom.xml b/apm-collector/apm-collector-remote/collector-remote-define/pom.xml index b1005665b..5a5c1b73f 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/pom.xml +++ b/apm-collector/apm-collector-remote/collector-remote-define/pom.xml @@ -5,7 +5,7 @@ apm-collector-remote org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteException.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteException.java new file mode 100644 index 000000000..3c9339ec8 --- /dev/null +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteException.java @@ -0,0 +1,35 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.remote; + +import org.skywalking.apm.collector.core.CollectorException; + +/** + * @author peng-yongsheng + */ +public abstract class RemoteException extends CollectorException { + + public RemoteException(String message) { + super(message); + } + + public RemoteException(String message, Throwable cause) { + super(message, cause); + } +} diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteModule.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteModule.java index b3e6fd673..e91b96574 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteModule.java +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/RemoteModule.java @@ -19,8 +19,8 @@ package org.skywalking.apm.collector.remote; import org.skywalking.apm.collector.core.module.Module; +import org.skywalking.apm.collector.remote.service.RemoteDataRegisterService; import org.skywalking.apm.collector.remote.service.RemoteSenderService; -import org.skywalking.apm.collector.remote.service.RemoteServerService; /** * @author peng-yongsheng @@ -34,6 +34,6 @@ public class RemoteModule extends Module { } @Override public Class[] services() { - return new Class[] {RemoteServerService.class, RemoteSenderService.class}; + return new Class[] {RemoteSenderService.class, RemoteDataRegisterService.class}; } } diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/CommonRemoteDataRegisterService.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/CommonRemoteDataRegisterService.java new file mode 100644 index 000000000..6b13da96a --- /dev/null +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/CommonRemoteDataRegisterService.java @@ -0,0 +1,71 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.remote.service; + +import java.util.HashMap; +import java.util.Map; +import org.skywalking.apm.collector.core.data.Data; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * @author peng-yongsheng + */ +public class CommonRemoteDataRegisterService implements RemoteDataRegisterService, RemoteDataIDGetter, RemoteDataInstanceCreatorGetter { + + private final Logger logger = LoggerFactory.getLogger(CommonRemoteDataRegisterService.class); + + private Integer id; + private final Map, Integer> dataClassMapping; + private final Map dataInstanceCreatorMapping; + + public CommonRemoteDataRegisterService() { + this.id = 1; + this.dataClassMapping = new HashMap<>(); + this.dataInstanceCreatorMapping = new HashMap<>(); + } + + @Override public void register(Class dataClass, RemoteDataInstanceCreator instanceCreator) { + if (!dataClassMapping.containsKey(dataClass)) { + dataClassMapping.put(dataClass, this.id); + dataInstanceCreatorMapping.put(this.id, instanceCreator); + this.id++; + } else { + logger.warn("The data class {} was registered.", dataClass.getName()); + } + } + + @Override + public Integer getRemoteDataId(Class dataClass) throws RemoteDataMappingIdNotFoundException { + if (dataClassMapping.containsKey(dataClass)) { + return dataClassMapping.get(dataClass); + } else { + throw new RemoteDataMappingIdNotFoundException("Could not found the id of remote data class " + dataClass.getName()); + } + } + + @Override public RemoteDataInstanceCreator getInstanceCreator( + Integer remoteDataId) throws RemoteDataInstanceCreatorNotFoundException { + if (dataInstanceCreatorMapping.containsKey(remoteDataId)) { + return dataInstanceCreatorMapping.get(remoteDataId); + } else { + throw new RemoteDataInstanceCreatorNotFoundException("Could not found the instance creator of remote data id " + remoteDataId); + } + } +} diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClient.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClient.java index 8771cd7f6..150e1d9e9 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClient.java +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClient.java @@ -26,7 +26,7 @@ import org.skywalking.apm.collector.core.data.Data; public interface RemoteClient extends Comparable { String getAddress(); - void send(int graphId, int nodeId, Data data); + void push(int graphId, int nodeId, Data data); boolean equals(String address); } diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClientService.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClientService.java index 109e9320f..c3e493da5 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClientService.java +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteClientService.java @@ -24,5 +24,5 @@ import org.skywalking.apm.collector.core.module.Service; * @author peng-yongsheng */ public interface RemoteClientService extends Service { - RemoteClient create(String host, int port); + RemoteClient create(String host, int port, int channelSize, int bufferSize); } diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiver.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataIDGetter.java similarity index 85% rename from apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiver.java rename to apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataIDGetter.java index 7baeb3478..04912f3eb 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiver.java +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataIDGetter.java @@ -23,8 +23,6 @@ import org.skywalking.apm.collector.core.data.Data; /** * @author peng-yongsheng */ -public interface DataReceiver { - Output output(int graphId, int nodeId); - - void receive(Output data); +public interface RemoteDataIDGetter { + Integer getRemoteDataId(Class dataClass) throws RemoteDataMappingIdNotFoundException; } diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorGetter.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorGetter.java new file mode 100644 index 000000000..f27e626ec --- /dev/null +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorGetter.java @@ -0,0 +1,27 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.remote.service; + +/** + * @author peng-yongsheng + */ +public interface RemoteDataInstanceCreatorGetter { + RemoteDataRegisterService.RemoteDataInstanceCreator getInstanceCreator( + Integer remoteDataId) throws RemoteDataInstanceCreatorNotFoundException; +} diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiverRegisterListener.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorNotFoundException.java similarity index 75% rename from apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiverRegisterListener.java rename to apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorNotFoundException.java index dcc9b4e3e..4a3f418ff 100644 --- a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/DataReceiverRegisterListener.java +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataInstanceCreatorNotFoundException.java @@ -18,18 +18,13 @@ package org.skywalking.apm.collector.remote.service; +import org.skywalking.apm.collector.remote.RemoteException; + /** * @author peng-yongsheng */ -public class DataReceiverRegisterListener { - - private DataReceiver dataReceiver; - - public DataReceiver getDataReceiver() { - return dataReceiver; - } - - public void setDataReceiver(DataReceiver dataReceiver) { - this.dataReceiver = dataReceiver; +public class RemoteDataInstanceCreatorNotFoundException extends RemoteException { + public RemoteDataInstanceCreatorNotFoundException(String message) { + super(message); } } diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataMappingIdNotFoundException.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataMappingIdNotFoundException.java new file mode 100644 index 000000000..ea1c82932 --- /dev/null +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataMappingIdNotFoundException.java @@ -0,0 +1,30 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.remote.service; + +import org.skywalking.apm.collector.remote.RemoteException; + +/** + * @author peng-yongsheng + */ +public class RemoteDataMappingIdNotFoundException extends RemoteException { + public RemoteDataMappingIdNotFoundException(String message) { + super(message); + } +} diff --git a/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataRegisterService.java b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataRegisterService.java new file mode 100644 index 000000000..e6dfe74bb --- /dev/null +++ b/apm-collector/apm-collector-remote/collector-remote-define/src/main/java/org/skywalking/apm/collector/remote/service/RemoteDataRegisterService.java @@ -0,0 +1,33 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.remote.service; + +import org.skywalking.apm.collector.core.data.Data; +import org.skywalking.apm.collector.core.module.Service; + +/** + * @author peng-yongsheng + */ +public interface RemoteDataRegisterService extends Service { + void register(Class dataClass, RemoteDataInstanceCreator instanceCreator); + + interface RemoteDataInstanceCreator { + RemoteData createInstance(String id); + } +} diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/pom.xml b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/pom.xml index 6fb3137e1..9131201cb 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/pom.xml +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-remote org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -41,6 +41,11 @@ collector-grpc-manager-define ${project.version} + + org.skywalking + apm-datacarrier + ${project.version} + diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/RemoteModuleGRPCProvider.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/RemoteModuleGRPCProvider.java index 3493cd7f4..01c815637 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/RemoteModuleGRPCProvider.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/RemoteModuleGRPCProvider.java @@ -12,10 +12,9 @@ import org.skywalking.apm.collector.grpc.manager.service.GRPCManagerService; import org.skywalking.apm.collector.remote.RemoteModule; import org.skywalking.apm.collector.remote.grpc.handler.RemoteCommonServiceHandler; import org.skywalking.apm.collector.remote.grpc.service.GRPCRemoteSenderService; -import org.skywalking.apm.collector.remote.grpc.service.GRPCRemoteServerService; -import org.skywalking.apm.collector.remote.service.DataReceiverRegisterListener; +import org.skywalking.apm.collector.remote.service.CommonRemoteDataRegisterService; +import org.skywalking.apm.collector.remote.service.RemoteDataRegisterService; import org.skywalking.apm.collector.remote.service.RemoteSenderService; -import org.skywalking.apm.collector.remote.service.RemoteServerService; import org.skywalking.apm.collector.server.Server; /** @@ -24,10 +23,14 @@ import org.skywalking.apm.collector.server.Server; public class RemoteModuleGRPCProvider extends ModuleProvider { public static final String NAME = "gRPC"; + private static final String HOST = "host"; private static final String PORT = "port"; - private final DataReceiverRegisterListener listener = new DataReceiverRegisterListener(); + private static final String CHANNEL_SIZE = "channel_size"; + private static final String BUFFER_SIZE = "buffer_size"; + private GRPCRemoteSenderService remoteSenderService; + private CommonRemoteDataRegisterService remoteDataRegisterService; @Override public String name() { return NAME; @@ -40,10 +43,13 @@ public class RemoteModuleGRPCProvider extends ModuleProvider { @Override public void prepare(Properties config) throws ServiceNotProvidedException { String host = config.getProperty(HOST); Integer port = (Integer)config.get(PORT); + Integer channelSize = (Integer)config.getOrDefault(CHANNEL_SIZE, 5); + Integer bufferSize = (Integer)config.getOrDefault(BUFFER_SIZE, 1000); - remoteSenderService = new GRPCRemoteSenderService(host, port); - this.registerServiceImplementation(RemoteServerService.class, new GRPCRemoteServerService(listener)); + remoteDataRegisterService = new CommonRemoteDataRegisterService(); + remoteSenderService = new GRPCRemoteSenderService(host, port, channelSize, bufferSize, remoteDataRegisterService); this.registerServiceImplementation(RemoteSenderService.class, remoteSenderService); + this.registerServiceImplementation(RemoteDataRegisterService.class, remoteDataRegisterService); } @Override public void start(Properties config) throws ServiceNotProvidedException { @@ -52,7 +58,7 @@ public class RemoteModuleGRPCProvider extends ModuleProvider { GRPCManagerService managerService = getManager().find(GRPCManagerModule.NAME).getService(GRPCManagerService.class); Server gRPCServer = managerService.createIfAbsent(host, port); - gRPCServer.addHandler(new RemoteCommonServiceHandler(listener)); + gRPCServer.addHandler(new RemoteCommonServiceHandler(remoteDataRegisterService)); ModuleRegisterService moduleRegisterService = getManager().find(ClusterModule.NAME).getService(ModuleRegisterService.class); moduleRegisterService.register(RemoteModule.NAME, this.name(), new RemoteModuleGRPCRegistration(host, port)); diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/handler/RemoteCommonServiceHandler.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/handler/RemoteCommonServiceHandler.java index e1a5f47fc..06c8ac65d 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/handler/RemoteCommonServiceHandler.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/handler/RemoteCommonServiceHandler.java @@ -20,12 +20,16 @@ package org.skywalking.apm.collector.remote.grpc.handler; import io.grpc.stub.StreamObserver; import org.skywalking.apm.collector.core.data.Data; +import org.skywalking.apm.collector.core.graph.GraphManager; +import org.skywalking.apm.collector.core.graph.Next; +import org.skywalking.apm.collector.core.util.Const; import org.skywalking.apm.collector.remote.grpc.proto.Empty; import org.skywalking.apm.collector.remote.grpc.proto.RemoteCommonServiceGrpc; import org.skywalking.apm.collector.remote.grpc.proto.RemoteData; import org.skywalking.apm.collector.remote.grpc.proto.RemoteMessage; import org.skywalking.apm.collector.remote.grpc.service.GRPCRemoteDeserializeService; -import org.skywalking.apm.collector.remote.service.DataReceiverRegisterListener; +import org.skywalking.apm.collector.remote.service.RemoteDataInstanceCreatorGetter; +import org.skywalking.apm.collector.remote.service.RemoteDataInstanceCreatorNotFoundException; import org.skywalking.apm.collector.server.grpc.GRPCHandler; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -37,24 +41,31 @@ public class RemoteCommonServiceHandler extends RemoteCommonServiceGrpc.RemoteCo private final Logger logger = LoggerFactory.getLogger(RemoteCommonServiceHandler.class); - private final DataReceiverRegisterListener listener; + private final RemoteDataInstanceCreatorGetter instanceCreatorGetter; private final GRPCRemoteDeserializeService service; - public RemoteCommonServiceHandler(DataReceiverRegisterListener listener) { - this.listener = listener; + public RemoteCommonServiceHandler(RemoteDataInstanceCreatorGetter instanceCreatorGetter) { + this.instanceCreatorGetter = instanceCreatorGetter; this.service = new GRPCRemoteDeserializeService(); } + @SuppressWarnings("unchecked") @Override public StreamObserver call(StreamObserver responseObserver) { return new StreamObserver() { @Override public void onNext(RemoteMessage message) { int graphId = message.getGraphId(); int nodeId = message.getNodeId(); + int remoteDataId = message.getRemoteDataId(); RemoteData remoteData = message.getRemoteData(); - Data output = listener.getDataReceiver().output(graphId, nodeId); - service.deserialize(remoteData, output); - listener.getDataReceiver().receive(output); + try { + Data output = instanceCreatorGetter.getInstanceCreator(remoteDataId).createInstance(Const.EMPTY_STRING); + service.deserialize(remoteData, output); + Next next = GraphManager.INSTANCE.findGraph(graphId).toFinder().findNext(nodeId); + next.execute(output); + } catch (RemoteDataInstanceCreatorNotFoundException e) { + logger.error(e.getMessage(), e); + } } @Override public void onError(Throwable throwable) { @@ -62,6 +73,7 @@ public class RemoteCommonServiceHandler extends RemoteCommonServiceGrpc.RemoteCo } @Override public void onCompleted() { + responseObserver.onNext(Empty.newBuilder().build()); responseObserver.onCompleted(); } }; diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClient.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClient.java index 979e4c71e..f69e8763c 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClient.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClient.java @@ -19,36 +19,146 @@ package org.skywalking.apm.collector.remote.grpc.service; import io.grpc.stub.StreamObserver; +import java.util.List; +import org.skywalking.apm.collector.client.grpc.GRPCClient; import org.skywalking.apm.collector.core.data.Data; +import org.skywalking.apm.collector.remote.grpc.proto.Empty; +import org.skywalking.apm.collector.remote.grpc.proto.RemoteCommonServiceGrpc; import org.skywalking.apm.collector.remote.grpc.proto.RemoteMessage; import org.skywalking.apm.collector.remote.service.RemoteClient; +import org.skywalking.apm.collector.remote.service.RemoteDataIDGetter; +import org.skywalking.apm.collector.remote.service.RemoteDataMappingIdNotFoundException; +import org.skywalking.apm.commons.datacarrier.DataCarrier; +import org.skywalking.apm.commons.datacarrier.buffer.BufferStrategy; +import org.skywalking.apm.commons.datacarrier.consumer.IConsumer; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class GRPCRemoteClient implements RemoteClient { - private final GRPCRemoteSerializeService service; - private final StreamObserver streamObserver; - private final String address; + private final Logger logger = LoggerFactory.getLogger(GRPCRemoteClient.class); - public GRPCRemoteClient(String host, int port, StreamObserver streamObserver) { - this.address = host + ":" + String.valueOf(port); - this.streamObserver = streamObserver; + private final GRPCRemoteSerializeService service; + private final GRPCClient client; + private final DataCarrier carrier; + private final String address; + private final RemoteDataIDGetter remoteDataIDGetter; + + GRPCRemoteClient(GRPCClient client, RemoteDataIDGetter remoteDataIDGetter, int channelSize, int bufferSize) { + this.address = client.toString(); + this.client = client; this.service = new GRPCRemoteSerializeService(); + this.remoteDataIDGetter = remoteDataIDGetter; + this.carrier = new DataCarrier<>(channelSize, bufferSize); + this.carrier.setBufferStrategy(BufferStrategy.BLOCKING); + this.carrier.consume(new RemoteMessageConsumer(), 1); } @Override public final String getAddress() { return this.address; } - @Override public void send(int graphId, int nodeId, Data data) { - RemoteMessage.Builder builder = RemoteMessage.newBuilder(); - builder.setGraphId(graphId); - builder.setNodeId(nodeId); - builder.setRemoteData(service.serialize(data)); + @Override public void push(int graphId, int nodeId, Data data) { + try { + Integer remoteDataId = remoteDataIDGetter.getRemoteDataId(data.getClass()); + RemoteMessage.Builder builder = RemoteMessage.newBuilder(); + builder.setGraphId(graphId); + builder.setNodeId(nodeId); + builder.setRemoteDataId(remoteDataId); + builder.setRemoteData(service.serialize(data)); - streamObserver.onNext(builder.build()); + this.carrier.produce(builder.build()); + logger.debug("put remote message into queue, id: {}", data.getId()); + } catch (RemoteDataMappingIdNotFoundException e) { + logger.error(e.getMessage(), e); + } + } + + class RemoteMessageConsumer implements IConsumer { + @Override public void init() { + } + + @Override public void consume(List remoteMessages) { + StreamObserver streamObserver = createStreamObserver(); + for (RemoteMessage remoteMessage : remoteMessages) { + streamObserver.onNext(remoteMessage); + } + streamObserver.onCompleted(); + } + + @Override public void onError(List remoteMessages, Throwable t) { + logger.error(t.getMessage(), t); + } + + @Override public void onExit() { + } + } + + private StreamObserver createStreamObserver() { + RemoteCommonServiceGrpc.RemoteCommonServiceStub stub = RemoteCommonServiceGrpc.newStub(client.getChannel()); + + StreamStatus status = new StreamStatus(false); + return stub.call(new StreamObserver() { + @Override public void onNext(Empty empty) { + } + + @Override public void onError(Throwable throwable) { + logger.error(throwable.getMessage(), throwable); + } + + @Override public void onCompleted() { + status.finished(); + } + }); + } + + class StreamStatus { + + private final Logger logger = LoggerFactory.getLogger(StreamStatus.class); + + private volatile boolean status; + + StreamStatus(boolean status) { + this.status = status; + } + + public boolean isFinish() { + return status; + } + + void finished() { + this.status = true; + } + + /** + * @param maxTimeout max wait time, milliseconds. + */ + public void wait4Finish(long maxTimeout) { + long time = 0; + while (!status) { + if (time > maxTimeout) { + break; + } + try2Sleep(5); + time += 5; + } + } + + /** + * Try to sleep, and ignore the {@link InterruptedException} + * + * @param millis the length of time to sleep in milliseconds + */ + private void try2Sleep(long millis) { + try { + Thread.sleep(millis); + } catch (InterruptedException e) { + logger.error(e.getMessage(), e); + } + } } @Override public boolean equals(String address) { diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClientService.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClientService.java index e98a1f103..69eddb461 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClientService.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteClientService.java @@ -18,14 +18,11 @@ package org.skywalking.apm.collector.remote.grpc.service; -import io.grpc.stub.StreamObserver; import org.skywalking.apm.collector.client.ClientException; import org.skywalking.apm.collector.client.grpc.GRPCClient; -import org.skywalking.apm.collector.remote.grpc.proto.Empty; -import org.skywalking.apm.collector.remote.grpc.proto.RemoteCommonServiceGrpc; -import org.skywalking.apm.collector.remote.grpc.proto.RemoteMessage; import org.skywalking.apm.collector.remote.service.RemoteClient; import org.skywalking.apm.collector.remote.service.RemoteClientService; +import org.skywalking.apm.collector.remote.service.RemoteDataIDGetter; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -36,74 +33,19 @@ public class GRPCRemoteClientService implements RemoteClientService { private final Logger logger = LoggerFactory.getLogger(GRPCRemoteClientService.class); - @Override public RemoteClient create(String host, int port) { + private final RemoteDataIDGetter remoteDataIDGetter; + + GRPCRemoteClientService(RemoteDataIDGetter remoteDataIDGetter) { + this.remoteDataIDGetter = remoteDataIDGetter; + } + + @Override public RemoteClient create(String host, int port, int channelSize, int bufferSize) { GRPCClient client = new GRPCClient(host, port); try { client.initialize(); } catch (ClientException e) { - e.printStackTrace(); - } - RemoteCommonServiceGrpc.RemoteCommonServiceStub stub = RemoteCommonServiceGrpc.newStub(client.getChannel()); - StreamObserver streamObserver = createStreamObserver(stub); - return new GRPCRemoteClient(host, port, streamObserver); - } - - private StreamObserver createStreamObserver(RemoteCommonServiceGrpc.RemoteCommonServiceStub stub) { - StreamStatus status = new StreamStatus(false); - return stub.call(new StreamObserver() { - @Override public void onNext(Empty empty) { - } - - @Override public void onError(Throwable throwable) { - logger.error(throwable.getMessage(), throwable); - } - - @Override public void onCompleted() { - status.finished(); - } - }); - } - - class StreamStatus { - private volatile boolean status; - - public StreamStatus(boolean status) { - this.status = status; - } - - public boolean isFinish() { - return status; - } - - public void finished() { - this.status = true; - } - - /** - * @param maxTimeout max wait time, milliseconds. - */ - public void wait4Finish(long maxTimeout) { - long time = 0; - while (!status) { - if (time > maxTimeout) { - break; - } - try2Sleep(5); - time += 5; - } - } - - /** - * Try to sleep, and ignore the {@link InterruptedException} - * - * @param millis the length of time to sleep in milliseconds - */ - private void try2Sleep(long millis) { - try { - Thread.sleep(millis); - } catch (InterruptedException e) { - - } + logger.error(e.getMessage(), e); } + return new GRPCRemoteClient(client, remoteDataIDGetter, channelSize, bufferSize); } } diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteDeserializeService.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteDeserializeService.java index 5ffbb5421..1c6d39b10 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteDeserializeService.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteDeserializeService.java @@ -43,8 +43,5 @@ public class GRPCRemoteDeserializeService implements RemoteDeserializeService remoteClients; private final String selfAddress; private final HashCodeSelector hashCodeSelector; private final ForeverFirstSelector foreverFirstSelector; private final RollingSelector rollingSelector; + private final int channelSize; + private final int bufferSize; @Override public Mode send(int graphId, int nodeId, Data data, Selector selector) { RemoteClient remoteClient; @@ -66,18 +69,21 @@ public class GRPCRemoteSenderService extends ClusterModuleListener implements Re if (remoteClient.equals(selfAddress)) { return Mode.Local; } else { - remoteClient.send(graphId, nodeId, data); + remoteClient.push(graphId, nodeId, data); return Mode.Remote; } } - public GRPCRemoteSenderService(String host, int port) { - this.service = new GRPCRemoteClientService(); + public GRPCRemoteSenderService(String host, int port, int channelSize, int bufferSize, + RemoteDataIDGetter remoteDataIDGetter) { + this.service = new GRPCRemoteClientService(remoteDataIDGetter); this.remoteClients = new ArrayList<>(); this.selfAddress = host + ":" + String.valueOf(port); this.hashCodeSelector = new HashCodeSelector(); this.foreverFirstSelector = new ForeverFirstSelector(); this.rollingSelector = new RollingSelector(); + this.channelSize = channelSize; + this.bufferSize = bufferSize; } @Override public String path() { @@ -90,7 +96,7 @@ public class GRPCRemoteSenderService extends ClusterModuleListener implements Re String host = serverAddress.split(":")[0]; int port = Integer.parseInt(serverAddress.split(":")[1]); - RemoteClient remoteClient = service.create(host, port); + RemoteClient remoteClient = service.create(host, port, channelSize, bufferSize); newRemoteClients.add(remoteClient); Collections.sort(newRemoteClients); diff --git a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteSerializeService.java b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteSerializeService.java index dbe08251b..374e76696 100644 --- a/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteSerializeService.java +++ b/apm-collector/apm-collector-remote/collector-remote-grpc-provider/src/main/java/org/skywalking/apm/collector/remote/grpc/service/GRPCRemoteSerializeService.java @@ -18,7 +18,6 @@ package org.skywalking.apm.collector.remote.grpc.service; -import com.google.protobuf.ByteString; import org.skywalking.apm.collector.core.data.Data; import org.skywalking.apm.collector.remote.grpc.proto.RemoteData; import org.skywalking.apm.collector.remote.service.RemoteSerializeService; @@ -31,22 +30,22 @@ public class GRPCRemoteSerializeService implements RemoteSerializeService apm-collector-remote org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-remote/pom.xml b/apm-collector/apm-collector-remote/pom.xml index a64e3491d..f125137d7 100644 --- a/apm-collector/apm-collector-remote/pom.xml +++ b/apm-collector/apm-collector-remote/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-storage/collector-storage-define/pom.xml b/apm-collector/apm-collector-storage/collector-storage-define/pom.xml index 79fd23dea..3411fc80a 100644 --- a/apm-collector/apm-collector-storage/collector-storage-define/pom.xml +++ b/apm-collector/apm-collector-storage/collector-storage-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-storage org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-storage/collector-storage-define/src/main/java/org/skywalking/apm/collector/storage/base/dao/IPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-define/src/main/java/org/skywalking/apm/collector/storage/base/dao/IPersistenceDAO.java index 8219b5c99..071ecf3ad 100644 --- a/apm-collector/apm-collector-storage/collector-storage-define/src/main/java/org/skywalking/apm/collector/storage/base/dao/IPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-define/src/main/java/org/skywalking/apm/collector/storage/base/dao/IPersistenceDAO.java @@ -29,4 +29,6 @@ public interface IPersistenceDAO extends Insert prepareBatchInsert(DataImpl data); Update prepareBatchUpdate(DataImpl data); + + void deleteHistory(Long startTimestamp, Long endTimestamp); } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/pom.xml b/apm-collector/apm-collector-storage/collector-storage-es-provider/pom.xml index 1585a938a..d4ab6ac60 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/pom.xml +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-storage org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -36,5 +36,10 @@ collector-storage-define ${project.version} + + org.skywalking + collector-cluster-define + ${project.version} + diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/DataTTLKeeperTimer.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/DataTTLKeeperTimer.java new file mode 100644 index 000000000..f33f65290 --- /dev/null +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/DataTTLKeeperTimer.java @@ -0,0 +1,118 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.storage.es; + +import java.util.Calendar; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeUnit; +import org.skywalking.apm.collector.core.module.ModuleManager; +import org.skywalking.apm.collector.storage.StorageModule; +import org.skywalking.apm.collector.storage.dao.ICpuMetricPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IGCMetricPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IGlobalTracePersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IInstPerformancePersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IMemoryMetricPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IMemoryPoolMetricPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.INodeComponentPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.INodeMappingPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.INodeReferencePersistenceDAO; +import org.skywalking.apm.collector.storage.dao.ISegmentCostPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.ISegmentPersistenceDAO; +import org.skywalking.apm.collector.storage.dao.IServiceReferencePersistenceDAO; + +/** + * @author peng-yongsheng + */ +public class DataTTLKeeperTimer { + + private final ModuleManager moduleManager; + private final StorageModuleEsNamingListener namingListener; + private final String selfAddress; + private final int daysBefore; + + public DataTTLKeeperTimer(ModuleManager moduleManager, + StorageModuleEsNamingListener namingListener, String selfAddress, int daysBefore) { + this.moduleManager = moduleManager; + this.namingListener = namingListener; + this.selfAddress = selfAddress; + this.daysBefore = daysBefore; + } + + public void start() { + Executors.newSingleThreadScheduledExecutor().scheduleAtFixedRate(this::delete, 1, 8, TimeUnit.HOURS); + } + + private void delete() { + Calendar calendar = Calendar.getInstance(); + calendar.setTimeInMillis(System.currentTimeMillis()); + calendar.set(Calendar.DAY_OF_MONTH, -daysBefore); + calendar.set(Calendar.HOUR_OF_DAY, 0); + calendar.set(Calendar.MINUTE, 0); + calendar.set(Calendar.SECOND, 0); + + long startTimestamp = calendar.getTimeInMillis(); + + calendar.set(Calendar.MINUTE, 59); + calendar.set(Calendar.SECOND, 59); + long endTimestamp = calendar.getTimeInMillis(); + + deleteJVMRelatedData(startTimestamp, endTimestamp); + deleteTraceRelatedData(startTimestamp, endTimestamp); + } + + private void deleteJVMRelatedData(long startTimestamp, long endTimestamp) { + ICpuMetricPersistenceDAO cpuMetricPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(ICpuMetricPersistenceDAO.class); + cpuMetricPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + IGCMetricPersistenceDAO gcMetricPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IGCMetricPersistenceDAO.class); + gcMetricPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + IMemoryMetricPersistenceDAO memoryMetricPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IMemoryMetricPersistenceDAO.class); + memoryMetricPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + IMemoryPoolMetricPersistenceDAO memoryPoolMetricPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IMemoryPoolMetricPersistenceDAO.class); + memoryPoolMetricPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + } + + private void deleteTraceRelatedData(long startTimestamp, long endTimestamp) { + IGlobalTracePersistenceDAO globalTracePersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IGlobalTracePersistenceDAO.class); + globalTracePersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + IInstPerformancePersistenceDAO instPerformancePersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IInstPerformancePersistenceDAO.class); + instPerformancePersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + INodeComponentPersistenceDAO nodeComponentPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(INodeComponentPersistenceDAO.class); + nodeComponentPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + INodeMappingPersistenceDAO nodeMappingPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(INodeMappingPersistenceDAO.class); + nodeMappingPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + INodeReferencePersistenceDAO nodeReferencePersistenceDAO = moduleManager.find(StorageModule.NAME).getService(INodeReferencePersistenceDAO.class); + nodeReferencePersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + ISegmentCostPersistenceDAO segmentCostPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(ISegmentCostPersistenceDAO.class); + segmentCostPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + ISegmentPersistenceDAO segmentPersistenceDAO = moduleManager.find(StorageModule.NAME).getService(ISegmentPersistenceDAO.class); + segmentPersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + + IServiceReferencePersistenceDAO serviceReferencePersistenceDAO = moduleManager.find(StorageModule.NAME).getService(IServiceReferencePersistenceDAO.class); + serviceReferencePersistenceDAO.deleteHistory(startTimestamp, endTimestamp); + } +} diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsNamingListener.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsNamingListener.java new file mode 100644 index 000000000..eb8558c4a --- /dev/null +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsNamingListener.java @@ -0,0 +1,42 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.storage.es; + +import org.skywalking.apm.collector.cluster.ClusterModuleListener; +import org.skywalking.apm.collector.storage.StorageModule; + +/** + * @author peng-yongsheng + */ +public class StorageModuleEsNamingListener extends ClusterModuleListener { + + public static final String PATH = "/" + StorageModule.NAME + "/" + StorageModuleEsProvider.NAME; + + @Override public String path() { + return PATH; + } + + @Override public void serverJoinNotify(String serverAddress) { + + } + + @Override public void serverQuitNotify(String serverAddress) { + + } +} diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsProvider.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsProvider.java index 43e0e7c0e..55115a1bc 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsProvider.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsProvider.java @@ -19,8 +19,12 @@ package org.skywalking.apm.collector.storage.es; import java.util.Properties; +import java.util.UUID; import org.skywalking.apm.collector.client.ClientException; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.cluster.ClusterModule; +import org.skywalking.apm.collector.cluster.service.ModuleListenerService; +import org.skywalking.apm.collector.cluster.service.ModuleRegisterService; import org.skywalking.apm.collector.core.module.Module; import org.skywalking.apm.collector.core.module.ModuleProvider; import org.skywalking.apm.collector.core.module.ServiceNotProvidedException; @@ -107,16 +111,19 @@ public class StorageModuleEsProvider extends ModuleProvider { private final Logger logger = LoggerFactory.getLogger(StorageModuleEsProvider.class); + public static final String NAME = "elasticsearch"; private static final String CLUSTER_NAME = "cluster_name"; private static final String CLUSTER_TRANSPORT_SNIFFER = "cluster_transport_sniffer"; private static final String CLUSTER_NODES = "cluster_nodes"; private static final String INDEX_SHARDS_NUMBER = "index_shards_number"; private static final String INDEX_REPLICAS_NUMBER = "index_replicas_number"; + private static final String TIME_TO_LIVE_OF_DATA = "ttl"; private ElasticSearchClient elasticSearchClient; + private DataTTLKeeperTimer deleteTimer; @Override public String name() { - return "elasticsearch"; + return NAME; } @Override public Class module() { @@ -147,14 +154,25 @@ public class StorageModuleEsProvider extends ModuleProvider { } catch (ClientException | StorageException e) { logger.error(e.getMessage(), e); } + + String uuId = UUID.randomUUID().toString(); + ModuleRegisterService moduleRegisterService = getManager().find(ClusterModule.NAME).getService(ModuleRegisterService.class); + moduleRegisterService.register(StorageModule.NAME, this.name(), new StorageModuleEsRegistration(uuId, 0)); + + StorageModuleEsNamingListener namingListener = new StorageModuleEsNamingListener(); + ModuleListenerService moduleListenerService = getManager().find(ClusterModule.NAME).getService(ModuleListenerService.class); + moduleListenerService.addListener(namingListener); + + Integer beforeDay = (Integer)config.getOrDefault(TIME_TO_LIVE_OF_DATA, 3); + deleteTimer = new DataTTLKeeperTimer(getManager(), namingListener, uuId + 0, beforeDay); } @Override public void notifyAfterCompleted() throws ServiceNotProvidedException { - + deleteTimer.start(); } @Override public String[] requiredModules() { - return new String[0]; + return new String[] {ClusterModule.NAME}; } private void registerCacheDAO() throws ServiceNotProvidedException { diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsRegistration.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsRegistration.java new file mode 100644 index 000000000..4ba3dc4d9 --- /dev/null +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/StorageModuleEsRegistration.java @@ -0,0 +1,40 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.collector.storage.es; + +import org.skywalking.apm.collector.cluster.ModuleRegistration; +import org.skywalking.apm.collector.core.util.Const; + +/** + * @author peng-yongsheng + */ +public class StorageModuleEsRegistration extends ModuleRegistration { + + private final String virtualHost; + private final int virtualPort; + + StorageModuleEsRegistration(String virtualHost, int virtualPort) { + this.virtualHost = virtualHost; + this.virtualPort = virtualPort; + } + + @Override public Value buildValue() { + return new Value(this.virtualHost, virtualPort, Const.EMPTY_STRING); + } +} diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/CpuMetricEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/CpuMetricEsPersistenceDAO.java index 18b43116f..3d2ffb891 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/CpuMetricEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/CpuMetricEsPersistenceDAO.java @@ -22,7 +22,10 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.ICpuMetricPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.jvm.CpuMetric; @@ -58,4 +61,16 @@ public class CpuMetricEsPersistenceDAO extends EsDAO implements ICpuMetricPersis @Override public UpdateRequestBuilder prepareBatchUpdate(CpuMetric cpuMetric) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(CpuMetricTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(CpuMetricTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, CpuMetricTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GCMetricEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GCMetricEsPersistenceDAO.java index 643480bd3..9cd87a535 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GCMetricEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GCMetricEsPersistenceDAO.java @@ -22,17 +22,24 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IGCMetricPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.jvm.GCMetric; import org.skywalking.apm.collector.storage.table.jvm.GCMetricTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class GCMetricEsPersistenceDAO extends EsDAO implements IGCMetricPersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(GCMetricEsPersistenceDAO.class); + public GCMetricEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -55,4 +62,16 @@ public class GCMetricEsPersistenceDAO extends EsDAO implements IGCMetricPersiste @Override public UpdateRequestBuilder prepareBatchUpdate(GCMetric gcMetric) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(GCMetricTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(GCMetricTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, GCMetricTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GlobalTraceEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GlobalTraceEsPersistenceDAO.java index 27c4aa3bb..412b40cdc 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GlobalTraceEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/GlobalTraceEsPersistenceDAO.java @@ -22,8 +22,11 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; import org.skywalking.apm.collector.core.UnexpectedException; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IGlobalTracePersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.global.GlobalTrace; @@ -58,4 +61,16 @@ public class GlobalTraceEsPersistenceDAO extends EsDAO implements IGlobalTracePe logger.debug("global trace source: {}", source.toString()); return getClient().prepareIndex(GlobalTraceTable.TABLE, data.getId()).setSource(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(GlobalTraceTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(GlobalTraceTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, GlobalTraceTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstPerformanceEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstPerformanceEsPersistenceDAO.java index 59aa39a69..06e811adf 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstPerformanceEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstPerformanceEsPersistenceDAO.java @@ -23,7 +23,10 @@ import java.util.Map; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IInstPerformancePersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.instance.InstPerformance; @@ -80,4 +83,16 @@ public class InstPerformanceEsPersistenceDAO extends EsDAO implements IInstPerfo return getClient().prepareUpdate(InstPerformanceTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(InstPerformanceTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(InstPerformanceTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, InstPerformanceTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstanceHeartBeatEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstanceHeartBeatEsPersistenceDAO.java index 1cd0756c3..1fe3b15dc 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstanceHeartBeatEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/InstanceHeartBeatEsPersistenceDAO.java @@ -67,4 +67,7 @@ public class InstanceHeartBeatEsPersistenceDAO extends EsDAO implements IInstanc source.put(InstanceTable.COLUMN_HEARTBEAT_TIME, data.getHeartBeatTime()); return getClient().prepareUpdate(InstanceTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryMetricEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryMetricEsPersistenceDAO.java index 9e81860f9..252f50c33 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryMetricEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryMetricEsPersistenceDAO.java @@ -22,17 +22,24 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IMemoryMetricPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.jvm.MemoryMetric; import org.skywalking.apm.collector.storage.table.jvm.MemoryMetricTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class MemoryMetricEsPersistenceDAO extends EsDAO implements IMemoryMetricPersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(MemoryMetricEsPersistenceDAO.class); + public MemoryMetricEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -57,4 +64,16 @@ public class MemoryMetricEsPersistenceDAO extends EsDAO implements IMemoryMetric @Override public UpdateRequestBuilder prepareBatchUpdate(MemoryMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(MemoryMetricTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(MemoryMetricTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, MemoryMetricTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryPoolMetricEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryPoolMetricEsPersistenceDAO.java index aae64b469..15d5f909d 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryPoolMetricEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/MemoryPoolMetricEsPersistenceDAO.java @@ -22,17 +22,24 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IMemoryPoolMetricPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.jvm.MemoryPoolMetric; import org.skywalking.apm.collector.storage.table.jvm.MemoryPoolMetricTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class MemoryPoolMetricEsPersistenceDAO extends EsDAO implements IMemoryPoolMetricPersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(MemoryPoolMetricEsPersistenceDAO.class); + public MemoryPoolMetricEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -57,4 +64,16 @@ public class MemoryPoolMetricEsPersistenceDAO extends EsDAO implements IMemoryPo @Override public UpdateRequestBuilder prepareBatchUpdate(MemoryPoolMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getSecondTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(MemoryPoolMetricTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(MemoryPoolMetricTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, MemoryPoolMetricTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeComponentEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeComponentEsPersistenceDAO.java index 5dad91001..d1fde779c 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeComponentEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeComponentEsPersistenceDAO.java @@ -23,17 +23,24 @@ import java.util.Map; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.INodeComponentPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.node.NodeComponent; import org.skywalking.apm.collector.storage.table.node.NodeComponentTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class NodeComponentEsPersistenceDAO extends EsDAO implements INodeComponentPersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(NodeComponentEsPersistenceDAO.class); + public NodeComponentEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -69,4 +76,16 @@ public class NodeComponentEsPersistenceDAO extends EsDAO implements INodeCompone return getClient().prepareUpdate(NodeComponentTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(NodeComponentTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(NodeComponentTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, NodeComponentTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeMappingEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeMappingEsPersistenceDAO.java index 0372faadb..08d341339 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeMappingEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeMappingEsPersistenceDAO.java @@ -23,17 +23,24 @@ import java.util.Map; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.INodeMappingPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.node.NodeMapping; import org.skywalking.apm.collector.storage.table.node.NodeMappingTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class NodeMappingEsPersistenceDAO extends EsDAO implements INodeMappingPersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(NodeMappingEsPersistenceDAO.class); + public NodeMappingEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -68,4 +75,16 @@ public class NodeMappingEsPersistenceDAO extends EsDAO implements INodeMappingPe source.put(NodeMappingTable.COLUMN_TIME_BUCKET, data.getTimeBucket()); return getClient().prepareUpdate(NodeMappingTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(NodeMappingTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(NodeMappingTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, NodeMappingTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeReferenceEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeReferenceEsPersistenceDAO.java index c58f94061..784acf513 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeReferenceEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/NodeReferenceEsPersistenceDAO.java @@ -23,17 +23,24 @@ import java.util.Map; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.INodeReferencePersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.noderef.NodeReference; import org.skywalking.apm.collector.storage.table.noderef.NodeReferenceTable; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; /** * @author peng-yongsheng */ public class NodeReferenceEsPersistenceDAO extends EsDAO implements INodeReferencePersistenceDAO { + private final Logger logger = LoggerFactory.getLogger(NodeReferenceEsPersistenceDAO.class); + public NodeReferenceEsPersistenceDAO(ElasticSearchClient client) { super(client); } @@ -87,4 +94,16 @@ public class NodeReferenceEsPersistenceDAO extends EsDAO implements INodeReferen return getClient().prepareUpdate(NodeReferenceTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(NodeReferenceTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(NodeReferenceTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, NodeReferenceTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentCostEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentCostEsPersistenceDAO.java index 8e1b0eeac..56f2ce801 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentCostEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentCostEsPersistenceDAO.java @@ -22,7 +22,10 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.ISegmentCostPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.segment.SegmentCost; @@ -63,4 +66,16 @@ public class SegmentCostEsPersistenceDAO extends EsDAO implements ISegmentCostPe logger.debug("segment cost source: {}", source.toString()); return getClient().prepareIndex(SegmentCostTable.TABLE, data.getId()).setSource(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(SegmentCostTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(SegmentCostTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, SegmentCostTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentEsPersistenceDAO.java index aed49c510..ea08369d6 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/SegmentEsPersistenceDAO.java @@ -23,7 +23,10 @@ import java.util.HashMap; import java.util.Map; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.ISegmentPersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.segment.Segment; @@ -53,7 +56,20 @@ public class SegmentEsPersistenceDAO extends EsDAO implements ISegmentPersistenc @Override public IndexRequestBuilder prepareBatchInsert(Segment data) { Map source = new HashMap<>(); source.put(SegmentTable.COLUMN_DATA_BINARY, new String(Base64.getEncoder().encode(data.getDataBinary()))); + source.put(SegmentTable.COLUMN_TIME_BUCKET, data.getTimeBucket()); logger.debug("segment source: {}", source.toString()); return getClient().prepareIndex(SegmentTable.TABLE, data.getId()).setSource(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(SegmentTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(SegmentTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, SegmentTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceEntryEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceEntryEsPersistenceDAO.java index a4f16d612..e04c367d2 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceEntryEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceEntryEsPersistenceDAO.java @@ -74,4 +74,7 @@ public class ServiceEntryEsPersistenceDAO extends EsDAO implements IServiceEntry return getClient().prepareUpdate(ServiceEntryTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceReferenceEsPersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceReferenceEsPersistenceDAO.java index ff96008b4..f39f3ca44 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceReferenceEsPersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/dao/ServiceReferenceEsPersistenceDAO.java @@ -23,7 +23,10 @@ import java.util.Map; import org.elasticsearch.action.get.GetResponse; import org.elasticsearch.action.index.IndexRequestBuilder; import org.elasticsearch.action.update.UpdateRequestBuilder; +import org.elasticsearch.index.query.QueryBuilders; +import org.elasticsearch.index.reindex.BulkByScrollResponse; import org.skywalking.apm.collector.client.elasticsearch.ElasticSearchClient; +import org.skywalking.apm.collector.core.util.TimeBucketUtils; import org.skywalking.apm.collector.storage.dao.IServiceReferencePersistenceDAO; import org.skywalking.apm.collector.storage.es.base.dao.EsDAO; import org.skywalking.apm.collector.storage.table.serviceref.ServiceReference; @@ -97,4 +100,16 @@ public class ServiceReferenceEsPersistenceDAO extends EsDAO implements IServiceR return getClient().prepareUpdate(ServiceReferenceTable.TABLE, data.getId()).setDoc(source); } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + long startTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(startTimestamp); + long endTimeBucket = TimeBucketUtils.INSTANCE.getMinuteTimeBucket(endTimestamp); + BulkByScrollResponse response = getClient().prepareDelete() + .filter(QueryBuilders.rangeQuery(ServiceReferenceTable.COLUMN_TIME_BUCKET).gte(startTimeBucket).lte(endTimeBucket)) + .source(ServiceReferenceTable.TABLE) + .get(); + + long deleted = response.getDeleted(); + logger.info("Delete {} rows history from {} index.", deleted, ServiceReferenceTable.TABLE); + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/define/SegmentEsTableDefine.java b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/define/SegmentEsTableDefine.java index c5885869a..787cca045 100644 --- a/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/define/SegmentEsTableDefine.java +++ b/apm-collector/apm-collector-storage/collector-storage-es-provider/src/main/java/org/skywalking/apm/collector/storage/es/define/SegmentEsTableDefine.java @@ -37,5 +37,6 @@ public class SegmentEsTableDefine extends ElasticSearchTableDefine { @Override public void initialize() { addColumn(new ElasticSearchColumnDefine(SegmentTable.COLUMN_DATA_BINARY, ElasticSearchColumnDefine.Type.Binary.name())); + addColumn(new ElasticSearchColumnDefine(SegmentTable.COLUMN_TIME_BUCKET, ElasticSearchColumnDefine.Type.Long.name())); } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/pom.xml b/apm-collector/apm-collector-storage/collector-storage-h2-provider/pom.xml index 63e5f706c..6b9093bb5 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/pom.xml +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-storage org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/CpuMetricH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/CpuMetricH2PersistenceDAO.java index 4d146d192..5faa595ee 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/CpuMetricH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/CpuMetricH2PersistenceDAO.java @@ -63,4 +63,7 @@ public class CpuMetricH2PersistenceDAO extends H2DAO implements ICpuMetricPersis @Override public H2SqlEntity prepareBatchUpdate(CpuMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GCMetricH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GCMetricH2PersistenceDAO.java index 9cca8ddaa..81616f249 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GCMetricH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GCMetricH2PersistenceDAO.java @@ -60,4 +60,7 @@ public class GCMetricH2PersistenceDAO extends H2DAO implements IGCMetricPersiste @Override public H2SqlEntity prepareBatchUpdate(GCMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GlobalTraceH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GlobalTraceH2PersistenceDAO.java index 805670fca..837226ae5 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GlobalTraceH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/GlobalTraceH2PersistenceDAO.java @@ -64,4 +64,7 @@ public class GlobalTraceH2PersistenceDAO extends H2DAO implements IGlobalTracePe entity.setParams(source.values().toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstPerformanceH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstPerformanceH2PersistenceDAO.java index a42773530..f3aabc4e3 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstPerformanceH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstPerformanceH2PersistenceDAO.java @@ -97,4 +97,7 @@ public class InstPerformanceH2PersistenceDAO extends H2DAO implements IInstPerfo entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstanceHeartBeatH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstanceHeartBeatH2PersistenceDAO.java index fd8e3982e..db94d82a7 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstanceHeartBeatH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/InstanceHeartBeatH2PersistenceDAO.java @@ -81,4 +81,7 @@ public class InstanceHeartBeatH2PersistenceDAO extends H2DAO implements IInstanc entity.setParams(params.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryMetricH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryMetricH2PersistenceDAO.java index 0b28d6ba0..404343462 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryMetricH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryMetricH2PersistenceDAO.java @@ -62,4 +62,7 @@ public class MemoryMetricH2PersistenceDAO extends H2DAO implements IMemoryMetric @Override public H2SqlEntity prepareBatchUpdate(MemoryMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryPoolMetricH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryPoolMetricH2PersistenceDAO.java index c5fe14eeb..aea2d72fc 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryPoolMetricH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/MemoryPoolMetricH2PersistenceDAO.java @@ -62,4 +62,7 @@ public class MemoryPoolMetricH2PersistenceDAO extends H2DAO implements IMemoryPo @Override public H2SqlEntity prepareBatchUpdate(MemoryPoolMetric data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeComponentH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeComponentH2PersistenceDAO.java index 50f059840..15c33d0b7 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeComponentH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeComponentH2PersistenceDAO.java @@ -94,4 +94,7 @@ public class NodeComponentH2PersistenceDAO extends H2DAO implements INodeCompone entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeMappingH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeMappingH2PersistenceDAO.java index 7d170c4a8..bb8ee9fb3 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeMappingH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeMappingH2PersistenceDAO.java @@ -92,4 +92,7 @@ public class NodeMappingH2PersistenceDAO extends H2DAO implements INodeMappingPe entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeReferenceH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeReferenceH2PersistenceDAO.java index d690aebb3..c5cc58d16 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeReferenceH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/NodeReferenceH2PersistenceDAO.java @@ -110,4 +110,7 @@ public class NodeReferenceH2PersistenceDAO extends H2DAO implements INodeReferen entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentCostH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentCostH2PersistenceDAO.java index eae890307..31939c078 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentCostH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentCostH2PersistenceDAO.java @@ -69,4 +69,7 @@ public class SegmentCostH2PersistenceDAO extends H2DAO implements ISegmentCostPe @Override public H2SqlEntity prepareBatchUpdate(SegmentCost data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentH2PersistenceDAO.java index 4f84b1c61..4d207ca4e 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/SegmentH2PersistenceDAO.java @@ -50,6 +50,7 @@ public class SegmentH2PersistenceDAO extends H2DAO implements ISegmentPersistenc H2SqlEntity entity = new H2SqlEntity(); source.put(SegmentTable.COLUMN_ID, data.getId()); source.put(SegmentTable.COLUMN_DATA_BINARY, data.getDataBinary()); + source.put(SegmentTable.COLUMN_TIME_BUCKET, data.getTimeBucket()); logger.debug("segment source: {}", source.toString()); String sql = SqlBuilder.buildBatchInsertSql(SegmentTable.TABLE, source.keySet()); @@ -61,4 +62,7 @@ public class SegmentH2PersistenceDAO extends H2DAO implements ISegmentPersistenc @Override public H2SqlEntity prepareBatchUpdate(Segment data) { return null; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceEntryH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceEntryH2PersistenceDAO.java index d74abfda5..da6311fd8 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceEntryH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceEntryH2PersistenceDAO.java @@ -97,4 +97,7 @@ public class ServiceEntryH2PersistenceDAO extends H2DAO implements IServiceEntry entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceReferenceH2PersistenceDAO.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceReferenceH2PersistenceDAO.java index 9b18bb5aa..bdcd189fb 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceReferenceH2PersistenceDAO.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/dao/ServiceReferenceH2PersistenceDAO.java @@ -120,4 +120,7 @@ public class ServiceReferenceH2PersistenceDAO extends H2DAO implements IServiceR entity.setParams(values.toArray(new Object[0])); return entity; } + + @Override public void deleteHistory(Long startTimestamp, Long endTimestamp) { + } } diff --git a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/define/SegmentH2TableDefine.java b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/define/SegmentH2TableDefine.java index 50001f547..aa096647c 100644 --- a/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/define/SegmentH2TableDefine.java +++ b/apm-collector/apm-collector-storage/collector-storage-h2-provider/src/main/java/org/skywalking/apm/collector/storage/h2/define/SegmentH2TableDefine.java @@ -34,5 +34,6 @@ public class SegmentH2TableDefine extends H2TableDefine { @Override public void initialize() { addColumn(new H2ColumnDefine(SegmentTable.COLUMN_ID, H2ColumnDefine.Type.Varchar.name())); addColumn(new H2ColumnDefine(SegmentTable.COLUMN_DATA_BINARY, H2ColumnDefine.Type.BINARY.name())); + addColumn(new H2ColumnDefine(SegmentTable.COLUMN_TIME_BUCKET, H2ColumnDefine.Type.Bigint.name())); } } diff --git a/apm-collector/apm-collector-storage/pom.xml b/apm-collector/apm-collector-storage/pom.xml index 509515956..2bc215c7e 100644 --- a/apm-collector/apm-collector-storage/pom.xml +++ b/apm-collector/apm-collector-storage/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/StreamModuleProvider.java b/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/StreamModuleProvider.java deleted file mode 100644 index 0e73dc8d2..000000000 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/StreamModuleProvider.java +++ /dev/null @@ -1,52 +0,0 @@ -/* - * Copyright 2017, OpenSkywalking Organization All rights reserved. - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - * Project repository: https://github.com/OpenSkywalking/skywalking - */ - -package org.skywalking.apm.collector.stream; - -import java.util.Properties; -import org.skywalking.apm.collector.core.module.Module; -import org.skywalking.apm.collector.core.module.ModuleProvider; -import org.skywalking.apm.collector.core.module.ServiceNotProvidedException; - -/** - * @author peng-yongsheng - */ -public class StreamModuleProvider extends ModuleProvider { - - @Override public String name() { - return "worker"; - } - - @Override public Class module() { - return StreamModule.class; - } - - @Override public void prepare(Properties config) throws ServiceNotProvidedException { - } - - @Override public void start(Properties config) throws ServiceNotProvidedException { - } - - @Override public void notifyAfterCompleted() throws ServiceNotProvidedException { - - } - - @Override public String[] requiredModules() { - return new String[] {}; - } -} diff --git a/apm-collector/apm-collector-stream/pom.xml b/apm-collector/apm-collector-stream/pom.xml index b3eb5c6d1..41859a26d 100644 --- a/apm-collector/apm-collector-stream/pom.xml +++ b/apm-collector/apm-collector-stream/pom.xml @@ -23,16 +23,12 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 apm-collector-stream - pom - - collector-stream-define - collector-stream-provider - + jar diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/timer/PersistenceTimer.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/timer/PersistenceTimer.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/timer/PersistenceTimer.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/timer/PersistenceTimer.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorker.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorker.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorker.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorker.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorkerProvider.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorkerProvider.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorkerProvider.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractLocalAsyncWorkerProvider.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java similarity index 79% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java index 4718a3c29..272ababa0 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java +++ b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/AbstractRemoteWorker.java @@ -38,17 +38,4 @@ public abstract class AbstractRemoteWorker extends Wo private final RemoteSenderService remoteSenderService; private final int graphId; - public RemoteWorkerRef(AbstractRemoteWorker remoteWorker, RemoteSenderService remoteSenderService, + RemoteWorkerRef(AbstractRemoteWorker remoteWorker, RemoteSenderService remoteSenderService, int graphId) { super(remoteWorker); this.remoteWorker = remoteWorker; diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/UsedRoleNameException.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/UsedRoleNameException.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/UsedRoleNameException.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/UsedRoleNameException.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerCreateListener.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerCreateListener.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerCreateListener.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerCreateListener.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerException.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerException.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerException.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerException.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerInvokeException.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerInvokeException.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerInvokeException.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerInvokeException.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerNotFoundException.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerNotFoundException.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerNotFoundException.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerNotFoundException.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerRef.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerRef.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerRef.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/base/WorkerRef.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java similarity index 94% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java index 2da917796..31e00ffe4 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java +++ b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/AggregationWorker.java @@ -63,7 +63,7 @@ public abstract class AggregationWorker throw new WorkerException(e.getMessage(), e); } } - dataCache.getLast().asMap().forEach((String id, Data data) -> { + dataCache.getLast().collection().forEach((String id, Data data) -> { logger.debug(data.toString()); onNext((OUTPUT)data); }); @@ -73,7 +73,7 @@ public abstract class AggregationWorker private void aggregate(INPUT message) { dataCache.writing(); if (dataCache.containsKey(message.getId())) { - message.mergeData(dataCache.get(message.getId())); + dataCache.get(message.getId()).mergeData(message); } else { dataCache.put(message.getId(), message); } diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/FlushAndSwitch.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/FlushAndSwitch.java similarity index 100% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/FlushAndSwitch.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/FlushAndSwitch.java diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java similarity index 97% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java index c6e8063cd..33248e73c 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java +++ b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/PersistenceWorker.java @@ -86,8 +86,8 @@ public abstract class PersistenceWorker } } - if (dataCache.getLast().asMap() != null) { - batchCollection = prepareBatch(dataCache.getLast().asMap()); + if (dataCache.getLast().collection() != null) { + batchCollection = prepareBatch(dataCache.getLast().collection()); } } finally { dataCache.finishReadingLast(); @@ -104,7 +104,7 @@ public abstract class PersistenceWorker if (ObjectUtils.isNotEmpty(dbData)) { dbData.mergeData(data); try { - updateBatchCollection.add(persistenceDAO().prepareBatchUpdate(data)); + updateBatchCollection.add(persistenceDAO().prepareBatchUpdate(dbData)); } catch (Throwable t) { logger.error(t.getMessage(), t); } diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java similarity index 87% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java index b8877b084..855f2952d 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java +++ b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCache.java @@ -18,15 +18,20 @@ package org.skywalking.apm.collector.stream.worker.impl.data; +import org.skywalking.apm.collector.core.cache.Window; import org.skywalking.apm.collector.core.data.Data; /** * @author peng-yongsheng */ -public class DataCache extends Window { +public class DataCache extends Window { private DataCollection lockedDataCollection; + @Override public DataCollection collectionInstance() { + return new DataCollection(); + } + public boolean containsKey(String id) { return lockedDataCollection.containsKey(id); } diff --git a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java similarity index 79% rename from apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java rename to apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java index eff0e468c..843ac385d 100644 --- a/apm-collector/apm-collector-stream/collector-stream-provider/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java +++ b/apm-collector/apm-collector-stream/src/main/java/org/skywalking/apm/collector/stream/worker/impl/data/DataCollection.java @@ -20,12 +20,13 @@ package org.skywalking.apm.collector.stream.worker.impl.data; import java.util.Map; import java.util.concurrent.ConcurrentHashMap; +import org.skywalking.apm.collector.core.cache.Collection; import org.skywalking.apm.collector.core.data.Data; /** * @author peng-yongsheng */ -public class DataCollection { +public class DataCollection implements Collection> { private Map data; private volatile boolean writing; private volatile boolean reading; @@ -40,23 +41,23 @@ public class DataCollection { writing = false; } - public void writing() { + @Override public void writing() { writing = true; } - public boolean isWriting() { + @Override public boolean isWriting() { return writing; } - public void finishReading() { + @Override public void finishReading() { reading = false; } - public void reading() { + @Override public void reading() { reading = true; } - public boolean isReading() { + @Override public boolean isReading() { return reading; } @@ -72,15 +73,15 @@ public class DataCollection { return data.get(key); } - public int size() { + @Override public int size() { return data.size(); } - public void clear() { + @Override public void clear() { data.clear(); } - public Map asMap() { + public Map collection() { return data; } } diff --git a/apm-collector/apm-collector-ui/collector-ui-define/pom.xml b/apm-collector/apm-collector-ui/collector-ui-define/pom.xml index 2125963df..3de7600b9 100644 --- a/apm-collector/apm-collector-ui/collector-ui-define/pom.xml +++ b/apm-collector/apm-collector-ui/collector-ui-define/pom.xml @@ -23,7 +23,7 @@ apm-collector-ui org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-ui/collector-ui-jetty-provider/pom.xml b/apm-collector/apm-collector-ui/collector-ui-jetty-provider/pom.xml index 6a0a7e5ea..1480ae870 100644 --- a/apm-collector/apm-collector-ui/collector-ui-jetty-provider/pom.xml +++ b/apm-collector/apm-collector-ui/collector-ui-jetty-provider/pom.xml @@ -23,7 +23,7 @@ apm-collector-ui org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/apm-collector-ui/pom.xml b/apm-collector/apm-collector-ui/pom.xml index 28784f1b2..ccdeed092 100644 --- a/apm-collector/apm-collector-ui/pom.xml +++ b/apm-collector/apm-collector-ui/pom.xml @@ -5,7 +5,7 @@ apm-collector org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-collector/pom.xml b/apm-collector/pom.xml index d34f3f702..35ecc7398 100644 --- a/apm-collector/pom.xml +++ b/apm-collector/pom.xml @@ -23,7 +23,7 @@ apm org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -33,7 +33,6 @@ apm-collector-boot apm-collector-ui - apm-collector-agent apm-collector-core apm-collector-component apm-collector-cluster @@ -47,6 +46,9 @@ apm-collector-queue apm-collector-instrument apm-collector-baseline + apm-collector-agent-grpc + apm-collector-agent-jetty + apm-collector-agent-stream @@ -68,11 +70,6 @@ log4j-core 2.9.0 - - com.google.guava - guava - 22.0 - org.apache.logging.log4j log4j-slf4j-impl diff --git a/apm-commons/apm-datacarrier/pom.xml b/apm-commons/apm-datacarrier/pom.xml index 3aba8dbf5..92f452dd5 100644 --- a/apm-commons/apm-datacarrier/pom.xml +++ b/apm-commons/apm-datacarrier/pom.xml @@ -23,7 +23,7 @@ apm-commons org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-commons/apm-datacarrier/src/main/java/org/skywalking/apm/commons/datacarrier/consumer/ConsumerThread.java b/apm-commons/apm-datacarrier/src/main/java/org/skywalking/apm/commons/datacarrier/consumer/ConsumerThread.java index 17473a4d5..231bafa95 100644 --- a/apm-commons/apm-datacarrier/src/main/java/org/skywalking/apm/commons/datacarrier/consumer/ConsumerThread.java +++ b/apm-commons/apm-datacarrier/src/main/java/org/skywalking/apm/commons/datacarrier/consumer/ConsumerThread.java @@ -91,12 +91,14 @@ public class ConsumerThread extends Thread { consumeList.add(element); } hasData = true; - } - try { - consumer.consume(consumeList); - } catch (Throwable t) { - consumer.onError(consumeList, t); + + if (consumeList.size() > 0) { + try { + consumer.consume(consumeList); + } catch (Throwable t) { + consumer.onError(consumeList, t); + } } return hasData; } diff --git a/apm-commons/apm-util/pom.xml b/apm-commons/apm-util/pom.xml index 7d5e31a99..6dda94a57 100644 --- a/apm-commons/apm-util/pom.xml +++ b/apm-commons/apm-util/pom.xml @@ -21,7 +21,7 @@ apm-commons org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-commons/pom.xml b/apm-commons/pom.xml index 38b7765fa..424427c72 100644 --- a/apm-commons/pom.xml +++ b/apm-commons/pom.xml @@ -21,7 +21,7 @@ apm org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-network/pom.xml b/apm-network/pom.xml index 34d12090f..7e3c3cb33 100644 --- a/apm-network/pom.xml +++ b/apm-network/pom.xml @@ -23,7 +23,7 @@ apm org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -31,8 +31,7 @@ UTF-8 - 1.4.0 - 4.1.12.Final + 1.7.0 1.6 @@ -41,16 +40,6 @@ io.grpc grpc-netty ${grpc.version} - - - io.netty - netty-codec-http2 - - - io.netty - netty-handler-proxy - - io.grpc @@ -62,16 +51,6 @@ grpc-stub ${grpc.version} - - io.netty - netty-codec-http2 - ${netty.version} - - - io.netty - netty-handler-proxy - ${netty.version} - @@ -104,7 +83,7 @@ com.google.protobuf:protoc:3.3.0:exe:${os.detected.classifier} grpc-java - io.grpc:protoc-gen-grpc-java:1.4.0:exe:${os.detected.classifier} + io.grpc:protoc-gen-grpc-java:1.7.0:exe:${os.detected.classifier} diff --git a/apm-sniffer/apm-agent-core/pom.xml b/apm-sniffer/apm-agent-core/pom.xml index 095cede21..6f6bdcdec 100644 --- a/apm-sniffer/apm-agent-core/pom.xml +++ b/apm-sniffer/apm-agent-core/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sniffer - 3.2.5-2017 + 3.3.0-2017 apm-agent-core @@ -35,7 +35,7 @@ UTF-8 9.4.2.v20170220 - 1.4.0 + 1.7.0 1.7.6 org.skywalking.apm.dependencies @@ -105,7 +105,7 @@ io.grpc grpc-testing - 1.4.0 + ${grpc.version} mockito-core diff --git a/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/conf/Config.java b/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/conf/Config.java index f2903d530..2d6a755be 100644 --- a/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/conf/Config.java +++ b/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/conf/Config.java @@ -52,6 +52,12 @@ public class Config { * memory cost estimated. */ public static int SPAN_LIMIT_PER_SEGMENT = 300; + + /** + * If true, skywalking agent will save all instrumented classes files in `/debugging` folder. + * Skywalking team may ask for these files in order to resolve compatible problem. + */ + public static boolean IS_OPEN_DEBUGGING_CLASS = false; } public static class Collector { @@ -76,7 +82,7 @@ public class Config { /** * Collector service discovery REST service name */ - public static String DISCOVERY_SERVICE_NAME = "/agentstream/grpc"; + public static String DISCOVERY_SERVICE_NAME = "/agent/gRPC"; } public static class Jvm { diff --git a/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/remote/DiscoveryRestServiceClient.java b/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/remote/DiscoveryRestServiceClient.java index 72d814461..6890adc36 100644 --- a/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/remote/DiscoveryRestServiceClient.java +++ b/apm-sniffer/apm-agent-core/src/main/java/org/skywalking/apm/agent/core/remote/DiscoveryRestServiceClient.java @@ -141,21 +141,12 @@ public class DiscoveryRestServiceClient implements Runnable { */ private void findBackupServer() { selectedServer++; - if (selectedServer == serverList.length) { + if (selectedServer >= serverList.length) { selectedServer = 0; } - } - - /** - * Try to sleep, and ignore the {@link InterruptedException} - * - * @param millis the length of time to sleep in milliseconds - */ - private void try2Sleep(long millis) { - try { - Thread.sleep(millis); - } catch (InterruptedException e) { + if (serverList.length == 0) { + selectedServer = -1; } } } diff --git a/apm-sniffer/apm-agent/pom.xml b/apm-sniffer/apm-agent/pom.xml index 1ac320a2a..d62b3673c 100644 --- a/apm-sniffer/apm-agent/pom.xml +++ b/apm-sniffer/apm-agent/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sniffer - 3.2.5-2017 + 3.3.0-2017 apm-agent diff --git a/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/InstrumentDebuggingClass.java b/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/InstrumentDebuggingClass.java new file mode 100644 index 000000000..bd5afa8a8 --- /dev/null +++ b/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/InstrumentDebuggingClass.java @@ -0,0 +1,71 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.agent; + +import java.io.File; +import java.io.IOException; +import net.bytebuddy.description.type.TypeDescription; +import net.bytebuddy.dynamic.DynamicType; +import org.skywalking.apm.agent.core.boot.AgentPackageNotFoundException; +import org.skywalking.apm.agent.core.boot.AgentPackagePath; +import org.skywalking.apm.agent.core.conf.Config; +import org.skywalking.apm.agent.core.logging.api.ILog; +import org.skywalking.apm.agent.core.logging.api.LogManager; + +/** + * @author wu-sheng + */ +public enum InstrumentDebuggingClass { + INSTANCE; + + private static final ILog logger = LogManager.getLogger(InstrumentDebuggingClass.class); + private File debuggingClassesRootPath; + + public void log(TypeDescription typeDescription, DynamicType dynamicType) { + if (!Config.Agent.IS_OPEN_DEBUGGING_CLASS) { + return; + } + + /** + * try to do I/O things in synchronized way, to avoid unexpected situations. + */ + synchronized (INSTANCE) { + try { + if (debuggingClassesRootPath == null) { + try { + debuggingClassesRootPath = new File(AgentPackagePath.getPath(), "/debugging"); + if (!debuggingClassesRootPath.exists()) { + debuggingClassesRootPath.mkdir(); + } + } catch (AgentPackageNotFoundException e) { + logger.error(e, "Can't find the root path for creating /debugging folder."); + } + } + + try { + dynamicType.saveIn(debuggingClassesRootPath); + } catch (IOException e) { + logger.error(e, "Can't save class {} to file." + typeDescription.getActualName()); + } + } catch (Throwable t) { + logger.error(t, "Save debugging classes fail."); + } + } + } +} diff --git a/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/SkyWalkingAgent.java b/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/SkyWalkingAgent.java index 766191c2e..7a75978e5 100644 --- a/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/SkyWalkingAgent.java +++ b/apm-sniffer/apm-agent/src/main/java/org/skywalking/apm/agent/SkyWalkingAgent.java @@ -18,6 +18,8 @@ package org.skywalking.apm.agent; +import java.lang.instrument.Instrumentation; +import java.util.List; import net.bytebuddy.agent.builder.AgentBuilder; import net.bytebuddy.description.type.TypeDescription; import net.bytebuddy.dynamic.DynamicType; @@ -26,10 +28,11 @@ import org.skywalking.apm.agent.core.boot.ServiceManager; import org.skywalking.apm.agent.core.conf.SnifferConfigInitializer; import org.skywalking.apm.agent.core.logging.api.ILog; import org.skywalking.apm.agent.core.logging.api.LogManager; -import org.skywalking.apm.agent.core.plugin.*; - -import java.lang.instrument.Instrumentation; -import java.util.List; +import org.skywalking.apm.agent.core.plugin.AbstractClassEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.EnhanceContext; +import org.skywalking.apm.agent.core.plugin.PluginBootstrap; +import org.skywalking.apm.agent.core.plugin.PluginException; +import org.skywalking.apm.agent.core.plugin.PluginFinder; /** * The main entrance of sky-waking agent, @@ -103,6 +106,8 @@ public class SkyWalkingAgent { if (logger.isDebugEnable()) { logger.debug("On Transformation class {}.", typeDescription.getName()); } + + InstrumentDebuggingClass.INSTANCE.log(typeDescription, dynamicType); } @Override @@ -113,7 +118,7 @@ public class SkyWalkingAgent { @Override public void onError(String typeName, ClassLoader classLoader, JavaModule module, boolean loaded, Throwable throwable) { - logger.error("Failed to enhance class " + typeName, throwable); + logger.error("Enhance class " + typeName + " error.", throwable); } @Override diff --git a/apm-sniffer/apm-sdk-plugin/dubbo-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/dubbo-plugin/pom.xml index 6bb9a9b69..b7596e805 100644 --- a/apm-sniffer/apm-sdk-plugin/dubbo-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/dubbo-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -148,6 +148,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/feign-default-http-9.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/feign-default-http-9.x-plugin/pom.xml index 667576fa6..a888b8121 100644 --- a/apm-sniffer/apm-sdk-plugin/feign-default-http-9.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/feign-default-http-9.x-plugin/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/grpc-1.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/grpc-1.x-plugin/pom.xml index fac98b707..daa0c5df4 100644 --- a/apm-sniffer/apm-sdk-plugin/grpc-1.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/grpc-1.x-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 apm-grpc-1.x-plugin diff --git a/apm-sniffer/apm-sdk-plugin/h2-1.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/h2-1.x-plugin/pom.xml index 88eba4802..d8930b18e 100755 --- a/apm-sniffer/apm-sdk-plugin/h2-1.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/h2-1.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -64,6 +64,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/httpClient-4.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/httpClient-4.x-plugin/pom.xml index a498f9aa5..415900642 100644 --- a/apm-sniffer/apm-sdk-plugin/httpClient-4.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/httpClient-4.x-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 apm-httpClient-4.x-plugin diff --git a/apm-sniffer/apm-sdk-plugin/jdbc-commons/pom.xml b/apm-sniffer/apm-sdk-plugin/jdbc-commons/pom.xml index 3c6c8c6a9..bb30f7e7c 100755 --- a/apm-sniffer/apm-sdk-plugin/jdbc-commons/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/jdbc-commons/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -58,6 +58,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/StatementEnhanceInfos.java b/apm-sniffer/apm-sdk-plugin/jdbc-commons/src/main/java/org/skywalking/apm/plugin/jdbc/define/StatementEnhanceInfos.java similarity index 96% rename from apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/StatementEnhanceInfos.java rename to apm-sniffer/apm-sdk-plugin/jdbc-commons/src/main/java/org/skywalking/apm/plugin/jdbc/define/StatementEnhanceInfos.java index 4a6b8fbc4..739c63e77 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/StatementEnhanceInfos.java +++ b/apm-sniffer/apm-sdk-plugin/jdbc-commons/src/main/java/org/skywalking/apm/plugin/jdbc/define/StatementEnhanceInfos.java @@ -16,7 +16,7 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.skywalking.apm.plugin.jdbc.mysql.define; +package org.skywalking.apm.plugin.jdbc.define; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; diff --git a/apm-sniffer/apm-sdk-plugin/jedis-2.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/jedis-2.x-plugin/pom.xml index 2fbe5ad10..a9009abaa 100644 --- a/apm-sniffer/apm-sdk-plugin/jedis-2.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/jedis-2.x-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 apm-jedis-2.x-plugin diff --git a/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-client-9.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-client-9.x-plugin/pom.xml index 36b5d56ec..ed18ab4a3 100644 --- a/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-client-9.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-client-9.x-plugin/pom.xml @@ -21,7 +21,7 @@ jetty-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-server-9.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-server-9.x-plugin/pom.xml index b53cda873..916c70d4e 100644 --- a/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-server-9.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/jetty-plugin/jetty-server-9.x-plugin/pom.xml @@ -21,7 +21,7 @@ jetty-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/jetty-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/jetty-plugin/pom.xml index b15dd8c6f..29ecd7e1e 100644 --- a/apm-sniffer/apm-sdk-plugin/jetty-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/jetty-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 jetty-plugins diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/pom.xml new file mode 100644 index 000000000..fc9c7519a --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/pom.xml @@ -0,0 +1,72 @@ + + + + + + apm-sdk-plugin + org.skywalking + 3.3.0-2017 + + 4.0.0 + + apm-mongodb-2.x-plugin + jar + + + UTF-8 + + + + + org.mongodb + mongo-java-driver + 2.14.2 + provided + + + org.mongodb + bson + 2.14.2 + provided + + + + + + + org.apache.maven.plugins + maven-source-plugin + + + + attach-sources + none + + jar + + + + + + + \ No newline at end of file diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptor.java b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptor.java new file mode 100644 index 000000000..376d29c90 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptor.java @@ -0,0 +1,102 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.mongodb.v2; + +import com.mongodb.AggregationOutput; +import com.mongodb.CommandResult; +import com.mongodb.DB; +import com.mongodb.ServerAddress; +import com.mongodb.WriteResult; +import java.lang.reflect.Method; +import java.util.List; +import org.skywalking.apm.agent.core.context.ContextCarrier; +import org.skywalking.apm.agent.core.context.ContextManager; +import org.skywalking.apm.agent.core.context.tag.Tags; +import org.skywalking.apm.agent.core.context.trace.AbstractSpan; +import org.skywalking.apm.agent.core.context.trace.SpanLayer; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceConstructorInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.skywalking.apm.network.trace.component.ComponentsDefine; + +/** + * {@link MongoDBCollectionMethodInterceptor} intercepts constructor of {@link com.mongodb.DBCollection}or {@link + * com.mongodb.DBCollectionImpl} recording the ServerAddress and creating the exit span. + * + * @author liyuntao + */ + +public class MongoDBCollectionMethodInterceptor implements InstanceMethodsAroundInterceptor, InstanceConstructorInterceptor { + + private static final String DB_TYPE = "MongoDB"; + + private static final String MONGO_DB_OP_PREFIX = "MongoDB/"; + + @Override public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, MethodInterceptResult result) throws Throwable { + + String remotePeer = (String)objInst.getSkyWalkingDynamicField(); + String opertaion = method.getName(); + AbstractSpan span = ContextManager.createExitSpan(MONGO_DB_OP_PREFIX + opertaion, new ContextCarrier(), remotePeer); + span.setComponent(ComponentsDefine.MONGODB); + Tags.DB_TYPE.set(span, DB_TYPE); + SpanLayer.asDB(span); + + } + + @Override public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Object ret) throws Throwable { + AbstractSpan activeSpan = ContextManager.activeSpan(); + CommandResult cresult = null; + if (ret instanceof WriteResult) { + WriteResult wresult = (WriteResult)ret; + cresult = wresult.getCachedLastError(); + } else if (ret instanceof AggregationOutput) { + AggregationOutput aresult = (AggregationOutput)ret; + cresult = aresult.getCommandResult(); + } + if (null != cresult && !cresult.ok()) { + activeSpan.log(cresult.getException()); + } + ContextManager.stopSpan(); + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + AbstractSpan activeSpan = ContextManager.activeSpan(); + activeSpan.errorOccurred(); + activeSpan.log(t); + } + + @Override + public void onConstruct(EnhancedInstance objInst, Object[] allArguments) { + List servers = null; + DB db = (DB)allArguments[0]; + servers = db.getMongo().getAllAddress(); + StringBuilder peers = new StringBuilder(); + for (ServerAddress address : servers) { + peers.append(address.getHost() + ":" + address.getPort() + ";"); + } + + objInst.setSkyWalkingDynamicField(peers.subSequence(0, peers.length() - 1).toString()); + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/InterceptPoint.java b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/InterceptPoint.java new file mode 100644 index 000000000..01ddb6081 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/InterceptPoint.java @@ -0,0 +1,38 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.mongodb.v2.define; + +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; + +/** + * @auther liyuntao + */ +public abstract class InterceptPoint implements InstanceMethodsInterceptPoint { + private static final String MONGDB_METHOD_INTERCET_CLASS = "org.skywalking.apm.plugin.mongodb.v2.MongoDBCollectionMethodInterceptor"; + + @Override + public String getMethodsInterceptor() { + return MONGDB_METHOD_INTERCET_CLASS; + } + + @Override + public boolean isOverrideArgs() { + return false; + } +} diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionImplInstrumentation.java b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionImplInstrumentation.java new file mode 100644 index 000000000..47213d615 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionImplInstrumentation.java @@ -0,0 +1,140 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.mongodb.v2.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassInstanceMethodsEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.any; +import static net.bytebuddy.matcher.ElementMatchers.named; +import static net.bytebuddy.matcher.ElementMatchers.takesArguments; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link MongoDBCollectionImplInstrumentation} define that the MongoDB Java Driver 2.13.x-2.14.x plugin intercepts the + * following methods in the {@link com.mongodb.DBCollectionImpl}class: + * 1. find
+ * 2. insert
+ * 3. insertImpl
+ * 4. update
+ * 5. updateImpl
+ * 6. remove
+ * 7. createIndex
+ * + * @author liyuntao + */ +public class MongoDBCollectionImplInstrumentation extends ClassInstanceMethodsEnhancePluginDefine { + + private static final String ENHANCE_CLASS = "com.mongodb.DBCollectionImpl"; + + private static final String MONGDB_METHOD_INTERCET_CLASS = "org.skywalking.apm.plugin.mongodb.v2.MongoDBCollectionMethodInterceptor"; + + @Override + protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[] { + new ConstructorInterceptPoint() { + @Override + public ElementMatcher getConstructorMatcher() { + return any(); + } + + @Override + public String getConstructorInterceptor() { + return MONGDB_METHOD_INTERCET_CLASS; + } + } + }; + } + + @Override + protected InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("find").and(takesArguments(9)); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("insert").and(takesArguments(4)); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("insertImpl"); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("update"); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("updateImpl"); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("remove").and(takesArguments(4)); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("createIndex"); + } + + }, + + }; + } + + @Override + protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } + + @Override + protected String[] witnessClasses() { + /** + * @see {@link com.mongodb.tools.ConnectionPoolStat} + */ + return new String[] { + "com.mongodb.tools.ConnectionPoolStat" + }; + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionInstrumentation.java b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionInstrumentation.java new file mode 100644 index 000000000..70da4a3b5 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/java/org/skywalking/apm/plugin/mongodb/v2/define/MongoDBCollectionInstrumentation.java @@ -0,0 +1,173 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.mongodb.v2.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassInstanceMethodsEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.any; +import static net.bytebuddy.matcher.ElementMatchers.named; +import static net.bytebuddy.matcher.ElementMatchers.takesArguments; +import static org.skywalking.apm.agent.core.plugin.bytebuddy.ArgumentTypeNameMatch.takesArgumentWithType; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link MongoDBCollectionInstrumentation} define that the MongoDB Java Driver 2.13.x-2.14.x plugin intercepts the + * following methods in the {@link com.mongodb.DBCollection}class: + * 1. aggregate
+ * 2. findAndModify
+ * 3. getCount + *
+ * 4. drop
+ * 5. dropIndexes
+ * 6. rename
+ * 7. group
+ * 8. distinct
+ * 9. mapReduce
+ * + * @author liyuntao + */ +public class MongoDBCollectionInstrumentation extends ClassInstanceMethodsEnhancePluginDefine { + + private static final String ENHANCE_CLASS = "com.mongodb.DBCollection"; + + private static final String MONGDB_METHOD_INTERCET_CLASS = "org.skywalking.apm.plugin.mongodb.v2.MongoDBCollectionMethodInterceptor"; + + @Override + protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[] { + new ConstructorInterceptPoint() { + @Override + public ElementMatcher getConstructorMatcher() { + return any(); + } + + @Override + public String getConstructorInterceptor() { + return MONGDB_METHOD_INTERCET_CLASS; + } + } + }; + } + + @Override + protected InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("aggregate").and(takesArgumentWithType(1, "com.mongodb.ReadPreference")); + } + }, + + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("findAndModify").and(takesArguments(9)); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("getCount").and(takesArgumentWithType(6, "java.util.concurrent.TimeUnit")); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("drop"); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("dropIndexes"); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("rename").and(takesArgumentWithType(1, "boolean")); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("group").and(takesArgumentWithType(1, "boolean")); + } + + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("group").and(takesArgumentWithType(1, "com.mongodb.DBObject")); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("distinct").and(takesArgumentWithType(2, "com.mongodb.ReadPreference")); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("mapReduce").and(takesArgumentWithType(0, "com.mongodb.MapReduceCommand")); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("mapReduce").and(takesArgumentWithType(0, "com.mongodb.DBObject")); + } + }, + new InterceptPoint() { + @Override + public ElementMatcher getMethodsMatcher() { + return named("explainAggregate"); + } + }, + + }; + } + + @Override + protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } + + @Override + protected String[] witnessClasses() { + /** + * @see {@link com.mongodb.tools.ConnectionPoolStat} + */ + return new String[] { + "com.mongodb.tools.ConnectionPoolStat" + }; + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/resources/skywalking-plugin.def b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/resources/skywalking-plugin.def new file mode 100644 index 000000000..fb8a07eab --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/main/resources/skywalking-plugin.def @@ -0,0 +1,2 @@ +mongodb-2.x=org.skywalking.apm.plugin.mongodb.v2.define.MongoDBCollectionInstrumentation +mongodb-2.x=org.skywalking.apm.plugin.mongodb.v2.define.MongoDBCollectionImplInstrumentation diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/test/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptorTest.java b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/test/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptorTest.java new file mode 100644 index 000000000..28f0805a4 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/mongodb-2.x-plugin/src/test/java/org/skywalking/apm/plugin/mongodb/v2/MongoDBCollectionMethodInterceptorTest.java @@ -0,0 +1,125 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.mongodb.v2; + +import com.mongodb.DBCollection; +import com.mongodb.DBObject; +import java.lang.reflect.Method; +import java.util.List; +import org.hamcrest.MatcherAssert; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.powermock.modules.junit4.PowerMockRunner; +import org.powermock.modules.junit4.PowerMockRunnerDelegate; +import org.skywalking.apm.agent.core.conf.Config; +import org.skywalking.apm.agent.core.context.trace.AbstractTracingSpan; +import org.skywalking.apm.agent.core.context.trace.LogDataEntity; +import org.skywalking.apm.agent.core.context.trace.SpanLayer; +import org.skywalking.apm.agent.core.context.trace.TraceSegment; +import org.skywalking.apm.agent.core.context.util.KeyValuePair; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.test.helper.SegmentHelper; +import org.skywalking.apm.agent.test.helper.SpanHelper; +import org.skywalking.apm.agent.test.tools.AgentServiceRule; +import org.skywalking.apm.agent.test.tools.SegmentStorage; +import org.skywalking.apm.agent.test.tools.SegmentStoragePoint; +import org.skywalking.apm.agent.test.tools.TracingSegmentRunner; + +import static org.hamcrest.CoreMatchers.is; +import static org.junit.Assert.assertThat; +import static org.mockito.Mockito.when; +import static org.skywalking.apm.agent.test.tools.SpanAssert.assertException; + +@RunWith(PowerMockRunner.class) +@PowerMockRunnerDelegate(TracingSegmentRunner.class) +public class MongoDBCollectionMethodInterceptorTest { + + @SegmentStoragePoint + private SegmentStorage segmentStorage; + + @Rule + public AgentServiceRule serviceRule = new AgentServiceRule(); + + private MongoDBCollectionMethodInterceptor interceptor; + + @Mock + private EnhancedInstance enhancedInstance; + + private Object[] arguments = new Object[3]; + private Class[] argumentTypes; + + @SuppressWarnings({"rawtypes", "unchecked"}) + @Before + public void setUp() throws Exception { + + interceptor = new MongoDBCollectionMethodInterceptor(); + + Config.Plugin.MongoDB.TRACE_PARAM = true; + + when(enhancedInstance.getSkyWalkingDynamicField()).thenReturn("127.0.0.1:27017"); + + } + + @Test + public void testIntercept() throws Throwable { + interceptor.beforeMethod(enhancedInstance, getExecuteMethod(), null, null, null); + interceptor.afterMethod(enhancedInstance, getExecuteMethod(), null, null, null); + + MatcherAssert.assertThat(segmentStorage.getTraceSegments().size(), is(1)); + TraceSegment traceSegment = segmentStorage.getTraceSegments().get(0); + List spans = SegmentHelper.getSpans(traceSegment); + assertMongoSpan(spans.get(0)); + } + + @Test + public void testInterceptWithException() throws Throwable { + interceptor.beforeMethod(enhancedInstance, getExecuteMethod(), null, null, null); + interceptor.handleMethodException(enhancedInstance, getExecuteMethod(), null, null, new RuntimeException()); + interceptor.afterMethod(enhancedInstance, getExecuteMethod(), null, null, null); + + MatcherAssert.assertThat(segmentStorage.getTraceSegments().size(), is(1)); + TraceSegment traceSegment = segmentStorage.getTraceSegments().get(0); + List spans = SegmentHelper.getSpans(traceSegment); + assertMongoSpan(spans.get(0)); + List logDataEntities = SpanHelper.getLogs(spans.get(0)); + assertThat(logDataEntities.size(), is(1)); + assertException(logDataEntities.get(0), RuntimeException.class); + } + + private void assertMongoSpan(AbstractTracingSpan span) { + assertThat(span.getOperationName(), is("MongoDB/insert")); + assertThat(SpanHelper.getComponentId(span), is(9)); + List tags = SpanHelper.getTags(span); + assertThat(tags.get(0).getValue(), is("MongoDB")); + assertThat(span.isExit(), is(true)); + assertThat(SpanHelper.getLayer(span), is(SpanLayer.DB)); + } + + private Method getExecuteMethod() { + try { + return DBCollection.class.getMethod("insert", DBObject[].class); + } catch (NoSuchMethodException e) { + return null; + } + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/mongodb-3.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/mongodb-3.x-plugin/pom.xml index 9fa2527e6..a5a4bf6c7 100644 --- a/apm-sniffer/apm-sdk-plugin/mongodb-3.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/mongodb-3.x-plugin/pom.xml @@ -22,7 +22,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 apm-mongodb-3.x-plugin @@ -54,6 +54,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/motan-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/motan-plugin/pom.xml index 19617e0b1..40b3e67c2 100644 --- a/apm-sniffer/apm-sdk-plugin/motan-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/motan-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/pom.xml index c65b4c70e..ef20bcc36 100755 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -70,6 +70,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateCallableStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateCallableStatementInterceptor.java index 5aaa176ab..cf07f9d83 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateCallableStatementInterceptor.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateCallableStatementInterceptor.java @@ -22,7 +22,7 @@ import java.lang.reflect.Method; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; -import org.skywalking.apm.plugin.jdbc.mysql.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; /** diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreatePreparedStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreatePreparedStatementInterceptor.java index ca4c7deed..2f41e3a5f 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreatePreparedStatementInterceptor.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreatePreparedStatementInterceptor.java @@ -22,7 +22,7 @@ import java.lang.reflect.Method; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; -import org.skywalking.apm.plugin.jdbc.mysql.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; /** diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateStatementInterceptor.java index 564254b5c..490c1d9c1 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateStatementInterceptor.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/CreateStatementInterceptor.java @@ -22,7 +22,7 @@ import java.lang.reflect.Method; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; -import org.skywalking.apm.plugin.jdbc.mysql.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; /** diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptor.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptor.java index 779ae38b0..015a3da30 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptor.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptor.java @@ -26,7 +26,7 @@ import org.skywalking.apm.agent.core.context.trace.SpanLayer; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; -import org.skywalking.apm.plugin.jdbc.mysql.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; /** diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/ConnectionInstrumentation.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/ConnectionInstrumentation.java index 3ddded5bb..f90c97dfe 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/ConnectionInstrumentation.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/mysql/define/ConnectionInstrumentation.java @@ -115,6 +115,6 @@ public class ConnectionInstrumentation extends ClassInstanceMethodsEnhancePlugin } @Override protected ClassMatch enhanceClass() { - return byMultiClassMatch(ENHANCE_CLASS, "com.mysql.cj.jdbc.ConnectionImpl"); + return byMultiClassMatch(ENHANCE_CLASS, "com.mysql.cj.jdbc.ConnectionImpl", "com.mysql.jdbc.Connection"); } } diff --git a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/test/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptorTest.java b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/test/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptorTest.java index 8c8425c6f..86efbe777 100644 --- a/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/test/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptorTest.java +++ b/apm-sniffer/apm-sdk-plugin/mysql-5.x-plugin/src/test/java/org/skywalking/apm/plugin/jdbc/mysql/StatementExecuteMethodsInterceptorTest.java @@ -37,7 +37,7 @@ import org.skywalking.apm.agent.test.tools.SegmentStoragePoint; import org.skywalking.apm.agent.test.tools.SpanAssert; import org.skywalking.apm.agent.test.tools.TracingSegmentRunner; import org.skywalking.apm.network.trace.component.ComponentsDefine; -import org.skywalking.apm.plugin.jdbc.mysql.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; import static org.hamcrest.CoreMatchers.is; diff --git a/apm-sniffer/apm-sdk-plugin/nutz-plugins/http-1.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/nutz-plugins/http-1.x-plugin/pom.xml index 37f2a7c1c..4af95a27c 100644 --- a/apm-sniffer/apm-sdk-plugin/nutz-plugins/http-1.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/nutz-plugins/http-1.x-plugin/pom.xml @@ -21,7 +21,7 @@ nutz-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/nutz-plugins/mvc-annotation-1.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/nutz-plugins/mvc-annotation-1.x-plugin/pom.xml index a89635fd4..4ad215257 100644 --- a/apm-sniffer/apm-sdk-plugin/nutz-plugins/mvc-annotation-1.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/nutz-plugins/mvc-annotation-1.x-plugin/pom.xml @@ -21,7 +21,7 @@ nutz-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/nutz-plugins/pom.xml b/apm-sniffer/apm-sdk-plugin/nutz-plugins/pom.xml index 526d24dd3..55f1c8d5e 100644 --- a/apm-sniffer/apm-sdk-plugin/nutz-plugins/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/nutz-plugins/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 nutz-plugins diff --git a/apm-sniffer/apm-sdk-plugin/okhttp-3.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/okhttp-3.x-plugin/pom.xml index b396e8b94..640e7896b 100644 --- a/apm-sniffer/apm-sdk-plugin/okhttp-3.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/okhttp-3.x-plugin/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/oracle-10.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/oracle-10.x-plugin/pom.xml index e255f4b02..b3a766b9b 100755 --- a/apm-sniffer/apm-sdk-plugin/oracle-10.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/oracle-10.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -64,6 +64,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/pom.xml index b4587d7a1..edfd655fb 100644 --- a/apm-sniffer/apm-sdk-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sniffer - 3.2.5-2017 + 3.3.0-2017 apm-sdk-plugin @@ -53,6 +53,7 @@ postgresql-8.x-plugin oracle-10.x-plugin elastic-job-2.x-plugin + mongodb-2.x-plugin pom @@ -143,6 +144,7 @@ attach-sources + none jar @@ -165,9 +167,12 @@ - - + + diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/pom.xml index 0264d6121..afb53be0a 100755 --- a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -64,6 +64,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateCallableStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateCallableStatementInterceptor.java new file mode 100644 index 000000000..60f4d1512 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateCallableStatementInterceptor.java @@ -0,0 +1,54 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql; + +import java.lang.reflect.Method; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; + +/** + * {@link CreateStatementInterceptor} intercepts the {@link org.postgresql.jdbc.PgConnection#prepareCall} method in + * {@link org.postgresql.jdbc.PgConnection}, {@link org.postgresql.jdbc.PgConnection} or {@link org.postgresql.jdbc3.Jdbc3Connection} class. + * + * @author zhangxin + */ +public class CreateCallableStatementInterceptor implements InstanceMethodsAroundInterceptor { + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + if (ret instanceof EnhancedInstance) { + ((EnhancedInstance)ret).setSkyWalkingDynamicField(new StatementEnhanceInfos((ConnectionInfo)objInst.getSkyWalkingDynamicField(), (String)allArguments[0], "CallableStatement")); + } + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreatePreparedStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreatePreparedStatementInterceptor.java new file mode 100644 index 000000000..ea8972fc6 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreatePreparedStatementInterceptor.java @@ -0,0 +1,54 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql; + +import java.lang.reflect.Method; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; + +/** + * {@link CreatePreparedStatementInterceptor} intercepts the {@link org.postgresql.jdbc.PgConnection#prepareStatement} method in + * {@link org.postgresql.jdbc.PgConnection}, {@link org.postgresql.jdbc.PgConnection} or {@link org.postgresql.jdbc3.Jdbc3Connection} class. + * + * @author zhangxin + */ +public class CreatePreparedStatementInterceptor implements InstanceMethodsAroundInterceptor { + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + if (ret instanceof EnhancedInstance) { + ((EnhancedInstance)ret).setSkyWalkingDynamicField(new StatementEnhanceInfos((ConnectionInfo)objInst.getSkyWalkingDynamicField(), (String)allArguments[0], "PreparedStatement")); + } + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateStatementInterceptor.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateStatementInterceptor.java new file mode 100644 index 000000000..c26f7acf1 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/CreateStatementInterceptor.java @@ -0,0 +1,57 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql; + +import java.lang.reflect.Method; +import org.postgresql.jdbc.PgConnection; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; + +/** + * {@link CreateStatementInterceptor} intercepts the {@link PgConnection#createStatement} method in + * {@link org.postgresql.jdbc.PgConnection}, {@link org.postgresql.jdbc.PgConnection} or {@link org.postgresql.jdbc3.Jdbc3Connection} class. + * + * @author zhangxin + */ +public class CreateStatementInterceptor implements InstanceMethodsAroundInterceptor { + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + + if (ret instanceof EnhancedInstance) { + ((EnhancedInstance)ret).setSkyWalkingDynamicField(new StatementEnhanceInfos((ConnectionInfo)objInst.getSkyWalkingDynamicField(), "", "CallableStatement")); + } + + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/StatementExecuteMethodsInterceptor.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/StatementExecuteMethodsInterceptor.java new file mode 100644 index 000000000..aab57d395 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/StatementExecuteMethodsInterceptor.java @@ -0,0 +1,75 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql; + +import java.lang.reflect.Method; +import org.skywalking.apm.agent.core.context.ContextManager; +import org.skywalking.apm.agent.core.context.tag.Tags; +import org.skywalking.apm.agent.core.context.trace.AbstractSpan; +import org.skywalking.apm.agent.core.context.trace.SpanLayer; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.skywalking.apm.plugin.jdbc.define.StatementEnhanceInfos; +import org.skywalking.apm.plugin.jdbc.trace.ConnectionInfo; + +/** + * {@link StatementExecuteMethodsInterceptor} create the exit span when the client call the interceptor methods. + * + * @author zhangxin + */ +public class StatementExecuteMethodsInterceptor implements InstanceMethodsAroundInterceptor { + @Override + public final void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + StatementEnhanceInfos cacheObject = (StatementEnhanceInfos)objInst.getSkyWalkingDynamicField(); + ConnectionInfo connectInfo = cacheObject.getConnectionInfo(); + AbstractSpan span = ContextManager.createExitSpan(buildOperationName(connectInfo, method.getName(), cacheObject.getStatementName()), connectInfo.getDatabasePeer()); + Tags.DB_TYPE.set(span, "sql"); + Tags.DB_INSTANCE.set(span, connectInfo.getDatabaseName()); + Tags.DB_STATEMENT.set(span, cacheObject.getSql()); + span.setComponent(connectInfo.getComponent()); + + SpanLayer.asDB(span); + } + + @Override + public final Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, + Object ret) throws Throwable { + StatementEnhanceInfos cacheObject = (StatementEnhanceInfos)objInst.getSkyWalkingDynamicField(); + if (cacheObject.getConnectionInfo() != null) { + ContextManager.stopSpan(); + } + return ret; + } + + @Override public final void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + StatementEnhanceInfos cacheObject = (StatementEnhanceInfos)objInst.getSkyWalkingDynamicField(); + if (cacheObject.getConnectionInfo() != null) { + ContextManager.activeSpan().errorOccurred().log(t); + } + } + + private String buildOperationName(ConnectionInfo connectionInfo, String methodName, String statementName) { + return connectionInfo.getDBType() + "/JDBI/" + statementName + "/" + methodName; + } +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/AbstractJdbc2StatementInstrumentation.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/AbstractJdbc2StatementInstrumentation.java new file mode 100644 index 000000000..b46f90cb8 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/AbstractJdbc2StatementInstrumentation.java @@ -0,0 +1,85 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassInstanceMethodsEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.named; +import static net.bytebuddy.matcher.ElementMatchers.takesArguments; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link AbstractJdbc2StatementInstrumentation} intercept the following methods that the class which extend {@link + * org.postgresql.jdbc2.AbstractJdbc2Statement} by {@link org.skywalking.apm.plugin.jdbc.postgresql.StatementExecuteMethodsInterceptor}.
+ * 1. the execute with non parameter + * 2. the execute with one parameter + * 3. the executeBatch + * 4. the executeQuery with non parameter + * 5. the executeQuery with one parameter + * 6. the executeUpdate with non parameter + * 7. the executeUpdate with one parameter + * 8. the addBatch with non parameter + * 9. the addBatch with one parameter + * + * @author zhangxin + */ +public class AbstractJdbc2StatementInstrumentation extends ClassInstanceMethodsEnhancePluginDefine { + + private static final String ENHANCE_CLASS = "org.postgresql.jdbc2.AbstractJdbc2Statement"; + private static final String INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.jdbc.postgresql.StatementExecuteMethodsInterceptor"; + + @Override protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[0]; + } + + @Override protected InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + new InstanceMethodsInterceptPoint() { + @Override public ElementMatcher getMethodsMatcher() { + return named("execute").and(takesArguments(0)) + .or(named("execute").and(takesArguments(1))) + .or(named("executeBatch")) + .or(named("executeQuery").and(takesArguments(0))) + .or(named("executeQuery").and(takesArguments(1))) + .or(named("executeUpdate").and(takesArguments(0))) + .or(named("executeUpdate").and(takesArguments(1))) + .or(named("addBatch").and(takesArguments(1))) + .or(named("addBatch").and(takesArguments(0))); + } + + @Override public String getMethodsInterceptor() { + return INTERCEPTOR_CLASS; + } + + @Override public boolean isOverrideArgs() { + return false; + } + } + }; + } + + @Override protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/ConnectionInstrumentation.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/ConnectionInstrumentation.java index df520a9e8..8c670cc15 100644 --- a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/ConnectionInstrumentation.java +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/ConnectionInstrumentation.java @@ -31,15 +31,15 @@ import static org.skywalking.apm.agent.core.plugin.bytebuddy.ArgumentTypeNameMat import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; import static org.skywalking.apm.plugin.jdbc.define.Constants.CLOSE_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.COMMIT_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.RELEASE_SAVE_POINT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.ROLLBACK_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.SERVICE_METHOD_INTERCEPT_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_STATEMENT_INTERCEPTOR_CLASS; /** * {@link ConnectionInstrumentation} intercept the following methods that the class which extend {@link @@ -74,7 +74,7 @@ public class ConnectionInstrumentation extends ClassInstanceMethodsEnhancePlugin } @Override public String getMethodsInterceptor() { - return PREPARE_STATEMENT_INTERCEPT_CLASS; + return CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -100,7 +100,7 @@ public class ConnectionInstrumentation extends ClassInstanceMethodsEnhancePlugin } @Override public String getMethodsInterceptor() { - return PREPARE_CALL_INTERCEPT_CLASS; + return CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -113,7 +113,7 @@ public class ConnectionInstrumentation extends ClassInstanceMethodsEnhancePlugin } @Override public String getMethodsInterceptor() { - return CREATE_STATEMENT_INTERCEPT_CLASS; + return CREATE_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Constants.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Constants.java new file mode 100644 index 000000000..78d3e7055 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Constants.java @@ -0,0 +1,31 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.jdbc.postgresql.define; + +/** + * Interceptor class name constant variable + * + * @author zhangxin + */ +public class Constants { + public static final String CREATE_STATEMENT_INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.jdbc.postgresql.CreateStatementInterceptor"; + public static final String CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.jdbc.postgresql.CreatePreparedStatementInterceptor"; + public static final String CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.jdbc.postgresql.CreateCallableStatementInterceptor"; + +} diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc3ConnectionInstrumentation.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc3ConnectionInstrumentation.java index 191d8607b..2c26d3436 100644 --- a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc3ConnectionInstrumentation.java +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc3ConnectionInstrumentation.java @@ -30,15 +30,15 @@ import static net.bytebuddy.matcher.ElementMatchers.takesArguments; import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; import static org.skywalking.apm.plugin.jdbc.define.Constants.CLOSE_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.COMMIT_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.RELEASE_SAVE_POINT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.ROLLBACK_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.SERVICE_METHOD_INTERCEPT_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_STATEMENT_INTERCEPTOR_CLASS; /** * {@link Jdbc3ConnectionInstrumentation} intercept the following methods that the class which extend {@link @@ -69,7 +69,7 @@ public class Jdbc3ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP } @Override public String getMethodsInterceptor() { - return PREPARE_STATEMENT_INTERCEPT_CLASS; + return CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -82,7 +82,7 @@ public class Jdbc3ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP } @Override public String getMethodsInterceptor() { - return PREPARE_CALL_INTERCEPT_CLASS; + return CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -95,7 +95,7 @@ public class Jdbc3ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP } @Override public String getMethodsInterceptor() { - return CREATE_STATEMENT_INTERCEPT_CLASS; + return CREATE_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -104,7 +104,8 @@ public class Jdbc3ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP }, new InstanceMethodsInterceptPoint() { @Override public ElementMatcher getMethodsMatcher() { - return named(COMMIT_METHOD_NAME).or(named(ROLLBACK_METHOD_NAME)).or(named(CLOSE_METHOD_NAME)).or(named(RELEASE_SAVE_POINT_METHOD_NAME)); + return named(COMMIT_METHOD_NAME).or(named(ROLLBACK_METHOD_NAME)).or(named(CLOSE_METHOD_NAME)) + .or(named(RELEASE_SAVE_POINT_METHOD_NAME)); } @Override public String getMethodsInterceptor() { diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc4ConnectionInstrumentation.java b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc4ConnectionInstrumentation.java index 7be5b62f6..c3507bc43 100644 --- a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc4ConnectionInstrumentation.java +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/java/org/skywalking/apm/plugin/jdbc/postgresql/define/Jdbc4ConnectionInstrumentation.java @@ -32,13 +32,13 @@ import static org.skywalking.apm.plugin.jdbc.define.Constants.CLOSE_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.COMMIT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.CREATE_STATEMENT_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_CALL_METHOD_NAME; -import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_INTERCEPT_CLASS; import static org.skywalking.apm.plugin.jdbc.define.Constants.PREPARE_STATEMENT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.RELEASE_SAVE_POINT_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.ROLLBACK_METHOD_NAME; import static org.skywalking.apm.plugin.jdbc.define.Constants.SERVICE_METHOD_INTERCEPT_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; +import static org.skywalking.apm.plugin.jdbc.postgresql.define.Constants.CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; /** * {@link Jdbc4ConnectionInstrumentation} intercept the following methods that the class which extend {@link @@ -69,7 +69,7 @@ public class Jdbc4ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP } @Override public String getMethodsInterceptor() { - return PREPARE_STATEMENT_INTERCEPT_CLASS; + return CREATE_PREPARED_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { @@ -82,7 +82,7 @@ public class Jdbc4ConnectionInstrumentation extends ClassInstanceMethodsEnhanceP } @Override public String getMethodsInterceptor() { - return PREPARE_CALL_INTERCEPT_CLASS; + return CREATE_CALLABLE_STATEMENT_INTERCEPTOR_CLASS; } @Override public boolean isOverrideArgs() { diff --git a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/resources/skywalking-plugin.def b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/resources/skywalking-plugin.def index 3fe0ad09b..b069baff8 100644 --- a/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/resources/skywalking-plugin.def +++ b/apm-sniffer/apm-sdk-plugin/postgresql-8.x-plugin/src/main/resources/skywalking-plugin.def @@ -2,3 +2,4 @@ postgresql-8.x=org.skywalking.apm.plugin.jdbc.postgresql.define.DriverInstrument postgresql-8.x=org.skywalking.apm.plugin.jdbc.postgresql.define.Jdbc3ConnectionInstrumentation postgresql-8.x=org.skywalking.apm.plugin.jdbc.postgresql.define.Jdbc4ConnectionInstrumentation postgresql-8.x=org.skywalking.apm.plugin.jdbc.postgresql.define.ConnectionInstrumentation +postgresql-8.x=org.skywalking.apm.plugin.jdbc.postgresql.define.AbstractJdbc2StatementInstrumentation diff --git a/apm-sniffer/apm-sdk-plugin/resin-3.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/resin-3.x-plugin/pom.xml index c701ba984..27c229d94 100644 --- a/apm-sniffer/apm-sdk-plugin/resin-3.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/resin-3.x-plugin/pom.xml @@ -25,7 +25,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 apm-resin-3.x-plugin @@ -63,6 +63,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/resin-4.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/resin-4.x-plugin/pom.xml index a09075b5a..6b96dd0dd 100644 --- a/apm-sniffer/apm-sdk-plugin/resin-4.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/resin-4.x-plugin/pom.xml @@ -25,7 +25,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 apm-resin-4.x-plugin @@ -63,6 +63,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/sharding-jdbc-1.5.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/sharding-jdbc-1.5.x-plugin/pom.xml index 73c4c5ed1..0694f35b7 100644 --- a/apm-sniffer/apm-sdk-plugin/sharding-jdbc-1.5.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/sharding-jdbc-1.5.x-plugin/pom.xml @@ -23,7 +23,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -66,6 +66,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/concurrent-util-4.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/concurrent-util-4.x-plugin/pom.xml index 4deaa9e5f..ff4aa2760 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/concurrent-util-4.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/concurrent-util-4.x-plugin/pom.xml @@ -21,7 +21,7 @@ spring-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/pom.xml new file mode 100644 index 000000000..8a54de602 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/pom.xml @@ -0,0 +1,42 @@ + + + + + + spring-plugins + org.skywalking + 3.3.0-2017 + + 4.0.0 + + apm-spring-core-patch + core-patch + + + + org.springframework + spring-aop + 3.2.9.RELEASE + provided + + + + diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/AutowiredAnnotationProcessorInterceptor.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/AutowiredAnnotationProcessorInterceptor.java new file mode 100644 index 000000000..3a6b48cf6 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/AutowiredAnnotationProcessorInterceptor.java @@ -0,0 +1,109 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.patch; + +import java.lang.reflect.Constructor; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.concurrent.ConcurrentHashMap; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceConstructorInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; + +/** + * {@link AutowiredAnnotationProcessorInterceptor} return the correct constructor when the bean class is enhanced by + * skywalking. + * + * @author zhangxin + */ +public class AutowiredAnnotationProcessorInterceptor implements InstanceMethodsAroundInterceptor, InstanceConstructorInterceptor { + + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + Class beanClass = (Class)allArguments[0]; + if (EnhancedInstance.class.isAssignableFrom(beanClass)) { + Map, Constructor[]> candidateConstructorsCache = (Map, Constructor[]>)objInst.getSkyWalkingDynamicField(); + + Constructor[] candidateConstructors = candidateConstructorsCache.get(beanClass); + if (candidateConstructors == null) { + Constructor[] returnCandidateConstructors = (Constructor[])ret; + + /** + * The return for the method {@link org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor#determineCandidateConstructors(Class, String) + * contains three cases: + * 1. Constructors with annotation {@link org.springframework.beans.factory.annotation.Autowired}. + * 2. The bean class only has one constructor with parameters. + * 3. The bean has constructor without parameters. + * + * because of the manipulate mechanism generates another private constructor in the enhance class, all the class that constrcutor enhance by skywalking + * cannot go to case two, and it will go to case three. case one is not affected in the current manipulate mechanism situation. + * + * The interceptor fill out the private constructor when the class is enhanced by skywalking, and check if the remainder constructors size is equals one, + * if yes, return the constructor. or return constructor without parameters. + * + * @see org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor#determineCandidateConstructors(Class, String) + */ + if (returnCandidateConstructors == null) { + Constructor[] rawConstructor = beanClass.getDeclaredConstructors(); + List> candidateRawConstructors = new ArrayList>(); + for (Constructor constructor : rawConstructor) { + if (!Modifier.isPrivate(constructor.getModifiers())) { + candidateRawConstructors.add(constructor); + } + } + + if (candidateRawConstructors.size() == 1 && candidateRawConstructors.get(0).getParameterTypes().length > 0) { + candidateConstructors = new Constructor[] {candidateRawConstructors.get(0)}; + } else { + candidateConstructors = new Constructor[0]; + } + + } else { + candidateConstructors = returnCandidateConstructors; + } + + candidateConstructorsCache.put(beanClass, candidateConstructors); + } + + return candidateConstructors.length > 0 ? candidateConstructors : null; + } + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } + + @Override public void onConstruct(EnhancedInstance objInst, Object[] allArguments) { + Map, Constructor[]> candidateConstructorsCache = new ConcurrentHashMap, Constructor[]>(20); + objInst.setSkyWalkingDynamicField(candidateConstructorsCache); + } +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptor.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptor.java new file mode 100644 index 000000000..1a25564f4 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptor.java @@ -0,0 +1,56 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.patch; + +import java.lang.reflect.Method; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.springframework.aop.framework.AdvisedSupport; + +/** + * {@link CreateAopProxyInterceptor} check that the bean has been implement {@link EnhancedInstance}.

+ * if yes, true will be returned. + * + * @author zhang xin + */ +public class CreateAopProxyInterceptor implements InstanceMethodsAroundInterceptor { + + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + AdvisedSupport advisedSupport = (AdvisedSupport)allArguments[0]; + + if (EnhancedInstance.class.isAssignableFrom(advisedSupport.getTargetClass())) { + return true; + } + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AopProxyFactoryInstrumentation.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AopProxyFactoryInstrumentation.java new file mode 100644 index 000000000..ce6c87fce --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AopProxyFactoryInstrumentation.java @@ -0,0 +1,70 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.patch.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassInstanceMethodsEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.named; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link AopProxyFactoryInstrumentation} indicate that spring core patch plugin intercepts the {@link + * org.springframework.aop.framework.DefaultAopProxyFactory#hasNoUserSuppliedProxyInterfaces} method by using {@link + * org.skywalking.apm.plugin.spring.patch.CreateAopProxyInterceptor} class. + * + * @author zhangxin + */ +public class AopProxyFactoryInstrumentation extends ClassInstanceMethodsEnhancePluginDefine { + + private static final String ENHANCE_CLASS = "org.springframework.aop.framework.DefaultAopProxyFactory"; + public static final String ENHANCE_METHOD = "hasNoUserSuppliedProxyInterfaces"; + public static final String INTERCEPT_CLASS = "org.skywalking.apm.plugin.spring.patch.CreateAopProxyInterceptor"; + + @Override protected final ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[0]; + } + + @Override protected final InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + new InstanceMethodsInterceptPoint() { + @Override public ElementMatcher getMethodsMatcher() { + return named(ENHANCE_METHOD); + } + + @Override public String getMethodsInterceptor() { + return INTERCEPT_CLASS; + } + + @Override public boolean isOverrideArgs() { + return false; + } + } + }; + } + + @Override protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AutowiredAnnotationProcessorInstrumentation.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AutowiredAnnotationProcessorInstrumentation.java new file mode 100644 index 000000000..6c26653dc --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/java/org/skywalking/apm/plugin/spring/patch/define/AutowiredAnnotationProcessorInstrumentation.java @@ -0,0 +1,79 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.patch.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.ClassInstanceMethodsEnhancePluginDefine; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.any; +import static net.bytebuddy.matcher.ElementMatchers.named; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link AutowiredAnnotationProcessorInstrumentation} indicates a spring core class patch for making sure the + * determineCandidateConstructors method in the class {@link org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor} + * works in spring designed ways + * + * @author zhang xin + */ +public class AutowiredAnnotationProcessorInstrumentation extends ClassInstanceMethodsEnhancePluginDefine { + private static final String ENHANCE_CLASS = "org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor"; + private static final String ENHANCE_METHOD = "determineCandidateConstructors"; + private static final String INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.spring.patch.AutowiredAnnotationProcessorInterceptor"; + + @Override protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[] { + new ConstructorInterceptPoint() { + @Override public ElementMatcher getConstructorMatcher() { + return any(); + } + + @Override public String getConstructorInterceptor() { + return INTERCEPTOR_CLASS; + } + } + }; + } + + @Override protected InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + new InstanceMethodsInterceptPoint() { + @Override public ElementMatcher getMethodsMatcher() { + return named(ENHANCE_METHOD); + } + + @Override public String getMethodsInterceptor() { + return INTERCEPTOR_CLASS; + } + + @Override public boolean isOverrideArgs() { + return false; + } + } + }; + } + + @Override protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/resources/skywalking-plugin.def b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/resources/skywalking-plugin.def new file mode 100644 index 000000000..2d8504014 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/main/resources/skywalking-plugin.def @@ -0,0 +1,2 @@ +spring-core-patch=org.skywalking.apm.plugin.spring.patch.define.AopProxyFactoryInstrumentation +spring-core-patch=org.skywalking.apm.plugin.spring.patch.define.AutowiredAnnotationProcessorInstrumentation diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/test/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptorTest.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/test/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptorTest.java new file mode 100644 index 000000000..e372866da --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/core-patch/src/test/java/org/skywalking/apm/plugin/spring/patch/CreateAopProxyInterceptorTest.java @@ -0,0 +1,73 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.patch; + +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.runners.MockitoJUnitRunner; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.springframework.aop.framework.AdvisedSupport; + +import static org.hamcrest.core.Is.is; +import static org.junit.Assert.assertThat; +import static org.mockito.Mockito.doReturn; + +@RunWith(MockitoJUnitRunner.class) +public class CreateAopProxyInterceptorTest { + + private CreateAopProxyInterceptor interceptor; + + @Mock + private EnhancedInstance enhancedInstance; + + @Mock + private AdvisedSupport advisedSupport; + + @Before + public void setUp() { + interceptor = new CreateAopProxyInterceptor(); + + } + + @Test + public void testInterceptNormalObject() throws Throwable { + doReturn(Object.class).when(advisedSupport).getTargetClass(); + assertThat(false, is(interceptor.afterMethod(enhancedInstance, null, new Object[] {advisedSupport}, new Class[] {Object.class}, false))); + } + + @Test + public void testInterceptEnhanceInstanceObject() throws Throwable { + doReturn(MockClass.class).when(advisedSupport).getTargetClass(); + assertThat(true, is(interceptor.afterMethod(enhancedInstance, null, new Object[] {advisedSupport}, new Class[] {Object.class}, false))); + } + + private class MockClass implements EnhancedInstance { + + @Override public Object getSkyWalkingDynamicField() { + return null; + } + + @Override public void setSkyWalkingDynamicField(Object value) { + + } + } + +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/pom.xml index a825bea25..4d13789f7 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/pom.xml @@ -21,7 +21,7 @@ spring-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/HandlerMethodInvokerInterceptor.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/HandlerMethodInvokerInterceptor.java new file mode 100644 index 000000000..52dd64a32 --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/HandlerMethodInvokerInterceptor.java @@ -0,0 +1,53 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.mvc.v3; + +import java.lang.reflect.Method; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; +import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.springframework.web.context.request.NativeWebRequest; + +/** + * {@link HandlerMethodInvokerInterceptor} pass the {@link NativeWebRequest} object into the {@link + * org.springframework.stereotype.Controller} object. + * + * @author zhangxin + */ +public class HandlerMethodInvokerInterceptor implements InstanceMethodsAroundInterceptor { + @Override + public void beforeMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + MethodInterceptResult result) throws Throwable { + Object handler = allArguments[1]; + if (handler instanceof EnhancedInstance) { + ((EnhanceRequireObjectCache)((EnhancedInstance)handler).getSkyWalkingDynamicField()).setNativeWebRequest((NativeWebRequest)allArguments[2]); + } + } + + @Override + public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, + Object ret) throws Throwable { + return ret; + } + + @Override public void handleMethodException(EnhancedInstance objInst, Method method, Object[] allArguments, + Class[] argumentsTypes, Throwable t) { + + } +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/define/HandlerMethodInvokerInstrumentation.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/define/HandlerMethodInvokerInstrumentation.java new file mode 100644 index 000000000..b708c4c6d --- /dev/null +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/mvc/v3/define/HandlerMethodInvokerInstrumentation.java @@ -0,0 +1,66 @@ +/* + * Copyright 2017, OpenSkywalking Organization All rights reserved. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + * Project repository: https://github.com/OpenSkywalking/skywalking + */ + +package org.skywalking.apm.plugin.spring.mvc.v3.define; + +import net.bytebuddy.description.method.MethodDescription; +import net.bytebuddy.matcher.ElementMatcher; +import org.skywalking.apm.agent.core.plugin.interceptor.ConstructorInterceptPoint; +import org.skywalking.apm.agent.core.plugin.interceptor.InstanceMethodsInterceptPoint; +import org.skywalking.apm.agent.core.plugin.match.ClassMatch; + +import static net.bytebuddy.matcher.ElementMatchers.named; +import static org.skywalking.apm.agent.core.plugin.match.NameMatch.byName; + +/** + * {@link HandlerMethodInvokerInstrumentation} intercept the invokeHandlerMethod method in the + * org.springframework.web.bind.annotation.support.HandlerMethodInvoker class. + * + * @author zhangxin + */ +public class HandlerMethodInvokerInstrumentation extends AbstractSpring3Instrumentation { + private static final String ENHANCE_CLASS = "org.springframework.web.bind.annotation.support.HandlerMethodInvoker"; + private static final String ENHANCE_METHOD = "invokeHandlerMethod"; + private static final String INTERCEPTOR_CLASS = "org.skywalking.apm.plugin.spring.mvc.v3.HandlerMethodInvokerInterceptor"; + + @Override protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { + return new ConstructorInterceptPoint[0]; + } + + @Override protected InstanceMethodsInterceptPoint[] getInstanceMethodsInterceptPoints() { + return new InstanceMethodsInterceptPoint[] { + new InstanceMethodsInterceptPoint() { + @Override public ElementMatcher getMethodsMatcher() { + return named(ENHANCE_METHOD); + } + + @Override public String getMethodsInterceptor() { + return INTERCEPTOR_CLASS; + } + + @Override public boolean isOverrideArgs() { + return false; + } + } + }; + } + + @Override protected ClassMatch enhanceClass() { + return byName(ENHANCE_CLASS); + } +} diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/resources/skywalking-plugin.def b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/resources/skywalking-plugin.def index 86b7b7658..8f4010b67 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/resources/skywalking-plugin.def +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-3.x-plugin/src/main/resources/skywalking-plugin.def @@ -1,3 +1,4 @@ spring-mvc-annotation-3.x=org.skywalking.apm.plugin.spring.mvc.v3.define.ControllerInstrumentation spring-mvc-annotation-3.x=org.skywalking.apm.plugin.spring.mvc.v3.define.HandlerMethodInstrumentation spring-mvc-annotation-3.x=org.skywalking.apm.plugin.spring.mvc.v3.define.InvocableHandlerInstrumentation +spring-mvc-annotation-3.x=org.skywalking.apm.plugin.spring.mvc.v3.define.HandlerMethodInvokerInstrumentation diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-4.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-4.x-plugin/pom.xml index 84e9e348e..90f78f650 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-4.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/mvc-annotation-4.x-plugin/pom.xml @@ -21,7 +21,7 @@ spring-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/pom.xml index c3236975d..d6f934a5b 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 spring-plugins @@ -34,6 +34,7 @@ mvc-annotation-4.x-plugin spring-cloud mvc-annotation-3.x-plugin + core-patch pom diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/pom.xml index 0fc33ea23..53a8eeeba 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/pom.xml @@ -21,7 +21,7 @@ spring-plugins org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/springframework/http/client/RestRequestInterceptor.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/RestRequestInterceptor.java similarity index 91% rename from apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/springframework/http/client/RestRequestInterceptor.java rename to apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/RestRequestInterceptor.java index 66fec4888..fc4e47a72 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/springframework/http/client/RestRequestInterceptor.java +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/RestRequestInterceptor.java @@ -16,7 +16,7 @@ * Project repository: https://github.com/OpenSkywalking/skywalking */ -package org.springframework.http.client; +package org.skywalking.apm.plugin.spring.resttemplate.async; import java.lang.reflect.Method; import org.skywalking.apm.agent.core.context.CarrierItem; @@ -24,6 +24,7 @@ import org.skywalking.apm.agent.core.context.ContextCarrier; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.EnhancedInstance; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.InstanceMethodsAroundInterceptor; import org.skywalking.apm.agent.core.plugin.interceptor.enhance.MethodInterceptResult; +import org.springframework.http.client.AsyncClientHttpRequest; public class RestRequestInterceptor implements InstanceMethodsAroundInterceptor { @@ -36,7 +37,7 @@ public class RestRequestInterceptor implements InstanceMethodsAroundInterceptor @Override public Object afterMethod(EnhancedInstance objInst, Method method, Object[] allArguments, Class[] argumentsTypes, Object ret) throws Throwable { - AbstractAsyncClientHttpRequest clientHttpRequest = (AbstractAsyncClientHttpRequest)ret; + AsyncClientHttpRequest clientHttpRequest = (AsyncClientHttpRequest)ret; if (ret != null) { Object[] cacheValues = (Object[])objInst.getSkyWalkingDynamicField(); ContextCarrier contextCarrier = (ContextCarrier)cacheValues[1]; diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/define/RestTemplateInstrumentation.java b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/define/RestTemplateInstrumentation.java index 6365a0d40..540076d12 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/define/RestTemplateInstrumentation.java +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/resttemplate-4.x-plugin/src/main/java/org/skywalking/apm/plugin/spring/resttemplate/async/define/RestTemplateInstrumentation.java @@ -47,7 +47,7 @@ public class RestTemplateInstrumentation extends ClassInstanceMethodsEnhancePlug private static final String DO_EXECUTE_METHOD_NAME = "doExecute"; private static final String DO_EXECUTE_INTERCEPTOR = "org.skywalking.apm.plugin.spring.resttemplate.async.RestExecuteInterceptor"; private static final String CREATE_REQUEST_METHOD_NAME = "createAsyncRequest"; - private static final String CREATE_REQUEST_INTERCEPTOR = "org.springframework.http.client.RestRequestInterceptor"; + private static final String CREATE_REQUEST_INTERCEPTOR = "org.skywalking.apm.plugin.spring.resttemplate.async.RestRequestInterceptor"; @Override protected ConstructorInterceptPoint[] getConstructorsInterceptPoints() { diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/pom.xml index 160775d98..dd289f4ef 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/pom.xml @@ -24,7 +24,7 @@ org.skywalking spring-cloud - 3.2.5-2017 + 3.3.0-2017 netflix-plugins diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/spring-cloud-feign-1.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/spring-cloud-feign-1.x-plugin/pom.xml index 8b3201a78..232969584 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/spring-cloud-feign-1.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/netflix-plugins/spring-cloud-feign-1.x-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking netflix-plugins - 3.2.5-2017 + 3.3.0-2017 apm-spring-cloud-feign-1.x-plugin diff --git a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/pom.xml b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/pom.xml index 719f93f27..9ba620471 100644 --- a/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spring-plugins/spring-cloud/pom.xml @@ -24,7 +24,7 @@ org.skywalking spring-plugins - 3.2.5-2017 + 3.3.0-2017 spring-cloud diff --git a/apm-sniffer/apm-sdk-plugin/spymemcached-2.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/spymemcached-2.x-plugin/pom.xml index 344cdb98d..730981614 100644 --- a/apm-sniffer/apm-sdk-plugin/spymemcached-2.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/spymemcached-2.x-plugin/pom.xml @@ -24,7 +24,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 apm-spymemcached-2.x-plugin diff --git a/apm-sniffer/apm-sdk-plugin/struts2-2.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/struts2-2.x-plugin/pom.xml index 3b4395617..cd1015318 100644 --- a/apm-sniffer/apm-sdk-plugin/struts2-2.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/struts2-2.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 @@ -65,6 +65,7 @@ attach-sources + none jar diff --git a/apm-sniffer/apm-sdk-plugin/tomcat-7.x-8.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/tomcat-7.x-8.x-plugin/pom.xml index fde90fa12..8452dee90 100644 --- a/apm-sniffer/apm-sdk-plugin/tomcat-7.x-8.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/tomcat-7.x-8.x-plugin/pom.xml @@ -21,7 +21,7 @@ apm-sdk-plugin org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-sdk-plugin/xmemcached-2.x-plugin/pom.xml b/apm-sniffer/apm-sdk-plugin/xmemcached-2.x-plugin/pom.xml index 8bec52852..4e43bdb77 100644 --- a/apm-sniffer/apm-sdk-plugin/xmemcached-2.x-plugin/pom.xml +++ b/apm-sniffer/apm-sdk-plugin/xmemcached-2.x-plugin/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sdk-plugin - 3.2.5-2017 + 3.3.0-2017 apm-xmemcached-2.x-plugin diff --git a/apm-sniffer/apm-test-tools/pom.xml b/apm-sniffer/apm-test-tools/pom.xml index 4f94de169..a8438029a 100644 --- a/apm-sniffer/apm-test-tools/pom.xml +++ b/apm-sniffer/apm-test-tools/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm-sniffer - 3.2.5-2017 + 3.3.0-2017 apm-test-tools diff --git a/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-1.x-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-1.x-activation/pom.xml index 63a462ce0..734621e65 100644 --- a/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-1.x-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-1.x-activation/pom.xml @@ -23,7 +23,7 @@ apm-toolkit-activation org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-2.x-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-2.x-activation/pom.xml index 8e97571d8..88cab4baf 100644 --- a/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-2.x-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/apm-toolkit-log4j-2.x-activation/pom.xml @@ -23,7 +23,7 @@ apm-toolkit-activation org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-toolkit-activation/apm-toolkit-logback-1.x-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/apm-toolkit-logback-1.x-activation/pom.xml index 40c06e275..a7ba9cd07 100644 --- a/apm-sniffer/apm-toolkit-activation/apm-toolkit-logback-1.x-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/apm-toolkit-logback-1.x-activation/pom.xml @@ -23,7 +23,7 @@ apm-toolkit-activation org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-toolkit-activation/apm-toolkit-opentracing-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/apm-toolkit-opentracing-activation/pom.xml index f6f7bac38..550a18432 100644 --- a/apm-sniffer/apm-toolkit-activation/apm-toolkit-opentracing-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/apm-toolkit-opentracing-activation/pom.xml @@ -23,7 +23,7 @@ apm-toolkit-activation org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-toolkit-activation/apm-toolkit-trace-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/apm-toolkit-trace-activation/pom.xml index 0eadd0921..4e1a3854f 100644 --- a/apm-sniffer/apm-toolkit-activation/apm-toolkit-trace-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/apm-toolkit-trace-activation/pom.xml @@ -23,7 +23,7 @@ apm-toolkit-activation org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/apm-sniffer/apm-toolkit-activation/pom.xml b/apm-sniffer/apm-toolkit-activation/pom.xml index 2aa9ab40e..eac10ee37 100644 --- a/apm-sniffer/apm-toolkit-activation/pom.xml +++ b/apm-sniffer/apm-toolkit-activation/pom.xml @@ -23,7 +23,7 @@ apm-sniffer org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 pom diff --git a/apm-sniffer/config/agent.config b/apm-sniffer/config/agent.config index d85abb433..8616a4b04 100644 --- a/apm-sniffer/config/agent.config +++ b/apm-sniffer/config/agent.config @@ -12,6 +12,10 @@ agent.application_code=Your_ApplicationName # Ignore the segments if their operation names start with these suffix. # agent.ignore_suffix=.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg +# If true, skywalking agent will save all instrumented classes files in `/debugging` folder. +# Skywalking team may ask for these files in order to resolve compatible problem. +# agent.is_open_debugging_class = true + # Server addresses. # Mapping to `agent_server/jetty/port` in `config/application.yml` of Collector. # Examples： diff --git a/apm-sniffer/pom.xml b/apm-sniffer/pom.xml index 9653cf8ea..bdefb2570 100644 --- a/apm-sniffer/pom.xml +++ b/apm-sniffer/pom.xml @@ -23,7 +23,7 @@ apm org.skywalking - 3.2.5-2017 + 3.3.0-2017 4.0.0 diff --git a/docker-compose.yml b/docker-compose.yml index 233cc714b..243f5fa2b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,7 +1,7 @@ version: '2.1' services: skywalking-webui: - image: skywalking/skywalking-ui:3.2.5-2017 + image: skywalking/skywalking-ui:3.3.0-2017 expose: - "8080" ports: @@ -15,7 +15,7 @@ services: - COLLECTOR_SERVERS=skywalking-collector:10800 skywalking-collector: - image: skywalking/skywalking-collector:3.2.5-2017 + image: skywalking/skywalking-collector:3.3.0-2017 expose: - "10800" - "11800" @@ -36,10 +36,11 @@ services: - ZK_ADDRESSES=zookeeper-server:2181 - ES_ADDRESSES=es-server:9300 - BIND_HOST=0.0.0.0 - - UI_BIND_HOST=skywalking-collector - - CLUSTER_BIND_HOST=0.0.0.0 + - AGENT_JETTY_BIND_HOST=skywalking-collector + - NAMING_BIND_HOST=skywalking-collector + - UI_JETTY_BIND_HOST=skywalking-collector healthcheck: - test: ["CMD", "curl", "-f", "http://localhost:10800/agentstream/grpc"] + test: ["CMD", "curl", "-f", "http://skywalking-collector:10800/agent/gRPC"] interval: 10s timeout: 10s retries: 5 diff --git a/docs/README.md b/docs/README.md index f71ad71cc..900c41c81 100644 --- a/docs/README.md +++ b/docs/README.md @@ -24,4 +24,3 @@ * [Skywalking 3 Cross Process Propagation Headers Protocol, v1.0](en/Skywalking-3-Cross-Process-Propagation-Headers-Protocol.md) * FAQ - \ No newline at end of file diff --git a/docs/README_ZH.md b/docs/README_ZH.md index 920c225ed..205ae0ddb 100644 --- a/docs/README_ZH.md +++ b/docs/README_ZH.md @@ -29,4 +29,4 @@ * [探针与Collector间网络协议，v3.2+](cn/How-to-communicate-with-the-collector-CN.md) * FAQ * [Trace查询有数据，但是没有拓扑图和JVM数据?](cn/FAQ/Why-have-traces-no-others-CN.md) - * [加载探针，Console被GRPC日志刷屏](cn/FAQ/Too-many-gRPC-logs-CN.md) \ No newline at end of file + * [加载探针，Console被GRPC日志刷屏](cn/FAQ/Too-many-gRPC-logs-CN.md) diff --git a/docs/Supported-list.md b/docs/Supported-list.md index d429b25c9..da388b973 100644 --- a/docs/Supported-list.md +++ b/docs/Supported-list.md @@ -29,7 +29,7 @@ * NoSQL * Redis * [Jedis](https://github.com/xetorthio/jedis) 2.8 - * [MongoDB Java Driver](https://github.com/mongodb/mongo-java-driver) 3.4+ + * [MongoDB Java Driver](https://github.com/mongodb/mongo-java-driver) 2.13-2.14,3.3+ * Memcached Client * [Spymemcached](https://github.com/couchbase/spymemcached) 2.x * [Xmemcached](https://github.com/killme2008/xmemcached) 2.x diff --git a/docs/cn/Deploy-collector-in-cluster-mode-CN.md b/docs/cn/Deploy-collector-in-cluster-mode-CN.md index d62ae2f05..2fba44ea5 100644 --- a/docs/cn/Deploy-collector-in-cluster-mode-CN.md +++ b/docs/cn/Deploy-collector-in-cluster-mode-CN.md @@ -23,49 +23,58 @@ thread_pool.bulk.queue_size: 1000 ### 部署collector 1. 解压安装包`tar -xvf skywalking-collector.tar.gz`，windows用户可以选择zip包 -1. 运行`bin/startup.sh`启动。windows用户为.bat文件。 +2. 设置Collector集群模式 + +集群模式主要依赖Zookeeper的注册和应用发现能力。所以，你只需要调整 `config/application.yml`中的host和port配置，使用实际IP和端口，代替默认配置。 +其次，将storage的注释取消，并修改为Elasticsearch集群的节点地址信息。 + - `config/application.yml` ``` cluster: -# Zookeeper地址配置 +# 配置zookeeper集群信息 zookeeper: hostPort: localhost:2181 sessionTimeout: 100000 -# agent_server, agent_stream, ui, collector_inside中配置的IP都是Collector所使用的IP地址 -agent_server: +naming: +# 配置探针使用的host和port jetty: host: localhost - # The port used port: 10800 context_path: / -agent_stream: - grpc: +remote: + gRPC: host: localhost port: 11800 +agent_gRPC: + gRPC: + host: localhost + port: 11800 +agent_jetty: jetty: host: localhost port: 12800 context_path: / +agent_stream: + default: + buffer_file_path: ../buffer/ + buffer_offset_max_file_size: 10M + buffer_segment_max_file_size: 500M ui: jetty: host: localhost port: 12800 context_path: / -collector_inside: - grpc: - host: localhost - port: 11800 +# 配置 Elasticsearch 集群连接信息 storage: elasticsearch: cluster_name: CollectorDBCluster cluster_transport_sniffer: true - # Elastic Search地址信息 cluster_nodes: localhost:9300 index_shards_number: 2 index_replicas_number: 0 + ttl: 7 ``` -## Collector集群模式启动 -集群模式主要依赖Zookeeper的注册和应用发现能力。所以，你只需要调整 `config/application.yml`中，agent_server, agent_stream, ui, collector_inside这些配置项的ip信息，使用真实的IP地址或者hostname，Collector就会使用集群模式运行。 -其次，将elasticsearch的注释取消，并修改集群的节点地址信息。 + +3. 运行`bin/startup.sh`启动。windows用户为.bat文件。 diff --git a/docs/cn/Deploy-collector-in-standalone-mode-CN.md b/docs/cn/Deploy-collector-in-standalone-mode-CN.md index 358a74dd1..7eca7f8ac 100644 --- a/docs/cn/Deploy-collector-in-standalone-mode-CN.md +++ b/docs/cn/Deploy-collector-in-standalone-mode-CN.md @@ -1,5 +1,5 @@ -## 用途说明 -单机模式使用本地H2数据库，不支持集群部署。主要用于：预览、功能测试、演示和低压力系统。 +# 用途说明 +单机模式默认使用本地H2数据库，不支持集群部署。主要用于：预览、功能测试、演示和低压力系统。如果使用单机collector用于非演示环境，你可选择使用Elasticsearch作为存储实现。 ## 所需的第三方软件 - JDK8+ @@ -10,52 +10,32 @@ ## Quick Start Collector单机模拟启动简单，提供和集群模式相同的功能，单机模式下除端口被占用的情况下，直接启动即可。 -### 部署collector +## 部署collector 1. 解压安装包`tar -xvf skywalking-collector.tar.gz`，windows用户可以选择zip包 1. 运行`bin/startup.sh`启动。windows用户为.bat文件。 -- `config/application.yml` -``` -# 单机模式下无需配置集群相关信息 -#cluster: -# zookeeper: -# hostPort: localhost:2181 -# sessionTimeout: 100000 -# agent_server, agent_stream, ui, collector_inside中配置的IP都是Collector所使用的IP地址 -agent_server: - jetty: - host: localhost - # The port used - port: 10800 - context_path: / -agent_stream: - grpc: - host: localhost - port: 11800 - jetty: - host: localhost - port: 12800 - context_path: / -ui: - jetty: - host: localhost - port: 12800 - context_path: / -collector_inside: - grpc: - host: localhost - port: 11800 - +## 使用Elastic Search代替H2存储 +- 在单机模式下除了支持内置的H2数据库运行，也支持其他的存储（当前已支持的ElasticSearch 5.3），取消Storage相关配置节的注释，并修改配置。 +```yaml #storage: # elasticsearch: # cluster_name: CollectorDBCluster # cluster_transport_sniffer: true -# Elastic Search地址信息 # cluster_nodes: localhost:9300 # index_shards_number: 2 # index_replicas_number: 0 ``` -## 使用Elastic Search代替H2存储 -由于H2数据库性能的局限性，我们在单机模式下，也支持使用ElasticSearch 5.3作为存储。你需要安装对应的ElasticSearch 5.3，并取消 -- 在单机模式下除了支持内置的H2数据库运行，也支持其他的存储（当前已支持的ElasticSearch 5.3以及将会支持的ShardJdbc），安装storage注释，修改配置信息即可。 +## 部署Elasticsearch +- 修改`elasticsearch.yml`文件 + - 设置 `cluster.name: CollectorDBCluster`。此名称需要和collector配置文件一致。 + - 设置 `node.name: anyname`, 可以设置为任意名字，如Elasticsearch为集群模式，则每个节点名称需要不同。 + - 增加如下配置 + +``` +# ES监听的ip地址 +network.host: 0.0.0.0 +thread_pool.bulk.queue_size: 1000 +``` + +- 启动Elasticsearch diff --git a/docs/cn/Deploy-skywalking-agent-CN.md b/docs/cn/Deploy-skywalking-agent-CN.md index d95db9c7a..976c0e503 100644 --- a/docs/cn/Deploy-skywalking-agent-CN.md +++ b/docs/cn/Deploy-skywalking-agent-CN.md @@ -38,6 +38,10 @@ agent.application_code=Your_ApplicationName # 默认配置如下 # agent.ignore_suffix=.jpg,.jpeg,.js,.css,.png,.bmp,.gif,.ico,.mp3,.mp4,.html,.svg +# 探针调试开关，如果设置为true，探针会将所有操作字节码的类输出到/debugging目录下 +# skywalking团队可能在调试，需要此文件 +# agent.is_open_debugging_class = true + # 对应Collector的config/application.yml配置文件中 agent_server/jetty/port 配置内容 # 例如： # 单节点配置：SERVERS="127.0.0.1:8080" diff --git a/docs/en/Deploy-collector-in-cluster-mode.md b/docs/en/Deploy-collector-in-cluster-mode.md index 4ff29a0a1..bea5ca0b5 100644 --- a/docs/en/Deploy-collector-in-cluster-mode.md +++ b/docs/en/Deploy-collector-in-cluster-mode.md @@ -12,7 +12,7 @@ - Set `cluster.name: CollectorDBCluster` - Set `node.name: anyname`, this name can be any, it based on Elasticsearch. - Add the following configurations to - + ``` # The ip used for listening network.host: 0.0.0.0 @@ -21,53 +21,58 @@ thread_pool.bulk.queue_size: 1000 - Start Elasticsearch -## Single Node Mode Collector -Single Node collector is easy to deploy, and provides same features as cluster mode. You can use almost all default config to run in this mode. And attention, all the default configs of single node mode, depend on running the collector, traced application, ElasticSearch and Zookeeper in the same machine. - ### Deploy collector servers 1. Run `tar -xvf skywalking-collector.tar.gz` -1. Run `bin/startup.sh` +2. Config collector in cluster mode. + +Cluster mode depends on Zookeeper register and application discovery capabilities. So, you just need to adjust the IP config items in `config/application.yml`. Change IP and port configs of agent_server, agent_stream, ui, collector_inside, replace them to the real ip or hostname which you want to use for cluster. - `config/application.yml` ``` cluster: - # The address of Zookeeper +# The Zookeeper cluster for collector cluster management. zookeeper: hostPort: localhost:2181 sessionTimeout: 100000 -# IPs in agent_server, agent_stream, ui, collector_inside are addresses of Collector -agent_server: +naming: +# Host and port used for agent config jetty: host: localhost - # The port used port: 10800 context_path: / -agent_stream: - grpc: +remote: + gRPC: host: localhost port: 11800 +agent_gRPC: + gRPC: + host: localhost + port: 11800 +agent_jetty: jetty: host: localhost port: 12800 context_path: / +agent_stream: + default: + buffer_file_path: ../buffer/ + buffer_offset_max_file_size: 10M + buffer_segment_max_file_size: 500M ui: jetty: host: localhost port: 12800 context_path: / -collector_inside: - grpc: - host: localhost - port: 11800 +# Config Elasticsearch cluster connection info. storage: elasticsearch: cluster_name: CollectorDBCluster cluster_transport_sniffer: true - # The address of Elastic Search cluster_nodes: localhost:9300 index_shards_number: 2 index_replicas_number: 0 + ttl: 7 ``` -## Cluster Mode Collector -Cluster mode depends on Zookeeper register and application discovery capabilities. So, you just need to adjust the IP config items in `config/application.yml`. Change IP and port configs of agent_server, agent_stream, ui, collector_inside, replace them to the real ip or hostname which you want to use for cluster. + +3. Run `bin/startup.sh` diff --git a/docs/en/Deploy-collector-in-standalone-mode.md b/docs/en/Deploy-collector-in-standalone-mode.md index 9a53fdaa4..375374e1c 100644 --- a/docs/en/Deploy-collector-in-standalone-mode.md +++ b/docs/en/Deploy-collector-in-standalone-mode.md @@ -1,13 +1,15 @@ # Usage scenario -Standalong mode collector means don't support cluster. It uses H2 as storage layer implementation, suggest that use only for preview, test, demonstration, low throughputs and small scale system. +Default standalong mode collector means don't support cluster. It uses H2 as storage layer implementation, suggest that use only for preview, test, demonstration, low throughputs and small scale system. -# Requirements +If you are using skywalking in a low throughputs monitoring scenario, and don't want to deploy cluster, at least, swith the storage implementation from H2 to Elasticsearch. + +## Requirements * JDK 8+ -# Download +## Download * [Releases](https://github.com/OpenSkywalking/skywalking/releases) -# Quick start +## Quick start You can simplely tar/unzip and startup if ports 10800, 11800, 12800 are free. - `tar -xvf skywalking-collector.tar.gz` in Linux, or unzip in windows. @@ -15,6 +17,19 @@ You can simplely tar/unzip and startup if ports 10800, 11800, 12800 are free. You should keep the `config/application.yml` as default. -# Use Elastic Search instead of H2 as storage layer implementation -Even in standalone mode, collector can run with Elastic Search as storage. If so, uncomment the `storage` section in `application.yml`, set the config right. +## Use Elastic Search instead of H2 as storage layer implementation +Even in standalone mode, collector can run with Elastic Search as storage. If so, uncomment the `storage` section in `application.yml`, set the config right. The default configs fit for collector and Elasticsearch both running in same machine, and not cluster. +## Deploy Elasticsearch server +- Modify `elasticsearch.yml` + - Set `cluster.name: CollectorDBCluster` + - Set `node.name: anyname`, this name can be any, it based on Elasticsearch. + - Add the following configurations to + +``` +# The ip used for listening +network.host: 0.0.0.0 +thread_pool.bulk.queue_size: 1000 +``` + +- Start Elasticsearch diff --git a/pom.xml b/pom.xml index e357174de..da91cc79b 100644 --- a/pom.xml +++ b/pom.xml @@ -23,7 +23,7 @@ org.skywalking apm - 3.2.5-2017 + 3.3.0-2017 @@ -252,6 +252,7 @@ attach-sources + none jar