232 lines
7.2 KiB
Markdown
232 lines
7.2 KiB
Markdown
# Trace Data Protocol
|
||
Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend.
|
||
|
||
## Abstract
|
||
This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to
|
||
uplink data to the SkyWalking backend.
|
||
|
||
- Discovery service provided by HTTP only
|
||
- Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both.
|
||
|
||
### Version
|
||
v1.1
|
||
|
||
#### gRPC proto files
|
||
[gRPC proto files](https://github.com/apache/incubator-skywalking-data-collect-protocol/tree/v1.1.1)
|
||
|
||
## Collector discovery service
|
||
### Abstract
|
||
**Collector discovery service should be the first service after agent started**
|
||
Through this service, get the gRPC service list. The agent can choose any one of them for uplink data. Recommend to
|
||
acquire the list periodically.
|
||
|
||
### HTTP GET
|
||
- Input
|
||
GET：http://collectorIp:port/agent/gRPC
|
||
|
||
- Output
|
||
JSON Array. Each element in the array ia a valid gRPC service address.
|
||
```json
|
||
["ip address1:port1","ip address2:port2","ip address3:port3"]
|
||
```
|
||
|
||
## Application Register Service
|
||
### Abstract
|
||
Register Application Code to the backend, and receive an integer represents the application.
|
||
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/ApplicationRegisterService.proto)
|
||
|
||
- applicationCode is the config in your `agent.config`.
|
||
- The return id is **ApplicationId** as the value in `KeyWithIntegerValue`, which will be used in further data uplink.
|
||
|
||
## Discovery Service
|
||
|
||
### Register Instance Service
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L29)
|
||
|
||
- agentUUID generated by agent, should be unique. Stay same before reboot, at least.
|
||
- **ApplicationInstanceId** will be used in further data uplink.
|
||
|
||
### Heart beat service
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L32)
|
||
|
||
- Recommend to report heart beat every 20-60 seconds.
|
||
- Java agent don't use this, because JVM metrics upstream replace the capabilities of this.
|
||
|
||
## Service Name Discovery Service
|
||
### Abstract
|
||
Replace the literal String service(operation) name by an id(integer)
|
||
|
||
[gRPC service define](.https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L70)
|
||
|
||
- Optional service, reduce the network cost but use more memory as a buffer mapping.
|
||
|
||
## Network Address Register Service
|
||
### Abstract
|
||
Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc.
|
||
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/NetworkAddressRegisterService.proto)
|
||
|
||
- Optional service, reduce the network cost but use more memory as a buffer mapping.
|
||
|
||
## JVM Metrics Service
|
||
### Abstract
|
||
Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every second.
|
||
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/JVMMetricsService.proto)
|
||
|
||
## Trace Segment Service
|
||
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/TraceSegmentService.proto)
|
||
|
||
- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
|
||
- Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md)
|
||
- Id and name both exist, please use id if possible.
|
||
- operationNameId/operationName
|
||
- networkAddress/networkAddressId
|
||
- entryServiceName/entryServiceId
|
||
- parentServiceName/parentServiceId
|
||
- peerId/peer
|
||
- componentIds are defined in backend, [here](../../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)
|
||
|
||
# HTTP JSON Services
|
||
All HTTP Services match the gRPC services, just adjust use short keys.
|
||
|
||
## Instance Register Service
|
||
- http://ip:port/instance/register(default: localhost:12800)
|
||
|
||
Input:
|
||
```
|
||
{
|
||
ai: x, #applicationId
|
||
au: "", #agentUUID
|
||
rt: x, #registerTime
|
||
oi: "", #osinfo
|
||
}
|
||
```
|
||
|
||
Output:
|
||
```
|
||
{
|
||
ai: x, #applicationId
|
||
ii: x, #applicationInstanceId
|
||
}
|
||
```
|
||
|
||
## Heart beat service
|
||
- http://ip:port/instance/heartbeat(default: localhost:12800)
|
||
|
||
Input:
|
||
```
|
||
{
|
||
"ii": x, #applicationInstanceId
|
||
"ht": x #heartbeatTime, java timestamp format
|
||
}
|
||
```
|
||
|
||
Output: empty
|
||
|
||
## Service Name Discovery
|
||
- http://ip:port/servicename/discovery(default: localhost:12800)
|
||
|
||
Input：
|
||
```
|
||
{
|
||
ai: x, #applicationId
|
||
sn: "", #serviceName
|
||
st: x, #srcSpanType
|
||
}
|
||
```
|
||
|
||
Output：
|
||
```
|
||
{
|
||
si: x, #osinfo
|
||
el: { #element
|
||
ai: x, #applicationId
|
||
sn: "", #serviceName
|
||
st: x, #srcSpanType
|
||
}
|
||
}
|
||
```
|
||
|
||
## Trace Segment Service
|
||
Input：
|
||
```
|
||
[
|
||
{
|
||
"gt": [[230150, 185809, 24040000]],
|
||
"sg": { //TraceSegmentObject
|
||
"ts": [137150, 185809, 48780000],
|
||
"ai": 2, //applicationId
|
||
"ii": 3, //applicationInstanceId
|
||
"ss": [ //SpanObject
|
||
{
|
||
"si": 0, //spanId
|
||
"tv": 0, //SpanType
|
||
"lv": 2, //SpanLayer
|
||
"ps": -1, //parentSpanId
|
||
"st": 1501858094726, //startTime
|
||
"et": 1501858096804, //endTime
|
||
"ci": 3, //componentId
|
||
"cn": "", //component
|
||
"oi": 0, //operationNameId
|
||
"on": "org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()", //operationName
|
||
"pi": 0, //peerId
|
||
"pn": "", //peer
|
||
"ie": false, //isError
|
||
"rs": [ //TraceSegmentReference
|
||
{
|
||
"pts": [230150, 185809, 24040000], //parentTraceSegmentId
|
||
"pii": 2, //parentApplicationInstanceId
|
||
"psp": 1, //parentSpanId
|
||
"psi": 0, //parentServiceId
|
||
"psn": "/dubbox-case/case/dubbox-rest", //parentServiceName
|
||
"ni": 0, //networkAddressId
|
||
"nn": "172.25.0.4:20880", //networkAddress
|
||
"eii": 2, //entryApplicationInstanceId
|
||
"esi": 0, //entryServiceId
|
||
"esn": "/dubbox-case/case/dubbox-rest", //entryServiceName
|
||
"rv": 0 //RefTypeValue
|
||
}
|
||
],
|
||
"to": [ //KeyWithStringValue
|
||
{
|
||
"k": "url", //key
|
||
"v": "rest://172.25.0.4:20880/org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()" //value
|
||
},
|
||
{
|
||
"k": "http.method",
|
||
"v": "GET"
|
||
}
|
||
],
|
||
"lo": [{
|
||
"ti": 1501858094726,
|
||
"ld": [{
|
||
"k": "NullPointException",
|
||
"v": "Error Stack"
|
||
}]
|
||
}]
|
||
},
|
||
{
|
||
"si": 1,
|
||
"tv": 1,
|
||
"lv": 1,
|
||
"ps": 0,
|
||
"st": 1501858094726,
|
||
"et": 1501858095804,
|
||
"ci": 9,
|
||
"cn": "",
|
||
"oi": 0,
|
||
"on": "mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]",
|
||
"pi": 0,
|
||
"pn": "localhost:27017",
|
||
"ie": false,
|
||
"to": [],
|
||
"lo": []
|
||
}
|
||
]
|
||
}
|
||
}
|
||
]
|
||
```
|