skywalking-java/docs/en/protocols/Trace-Data-Protocol.md

Trace Data Protocol

Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend.

Abstract

This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to uplink data to the SkyWalking backend.

• Discovery service provided by HTTP only
• Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both.

Version

v1.1

gRPC proto files

gRPC proto files

Collector discovery service

Abstract

Collector discovery service should be the first service after agent started Through this service, get the gRPC service list. The agent can choose any one of them for uplink data. Recommend to acquire the list periodically.

HTTP GET

• Input GET：http://collectorIp:port/agent/gRPC

• Output JSON Array. Each element in the array ia a valid gRPC service address.

["ip address1:port1","ip address2:port2","ip address3:port3"]

Application Register Service

Abstract

Register Application Code to the backend, and receive an integer represents the application.

gRPC service define

• applicationCode is the config in your agent.config.
• The return id is ApplicationId as the value in KeyWithIntegerValue, which will be used in further data uplink.

Discovery Service

Register Instance Service

gRPC service define

• agentUUID generated by agent, should be unique. Stay same before reboot, at least.
• ApplicationInstanceId will be used in further data uplink.

Heart beat service

gRPC service define

• Recommend to report heart beat every 20-60 seconds.
• Java agent don't use this, because JVM metrics upstream replace the capabilities of this.

Service Name Discovery Service

Abstract

Replace the literal String service(operation) name by an id(integer)

gRPC service define

• Optional service, reduce the network cost but use more memory as a buffer mapping.

Network Address Register Service

Abstract

Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc.

gRPC service define

• Optional service, reduce the network cost but use more memory as a buffer mapping.

JVM Metrics Service

Abstract

Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every second.

gRPC service define

Trace Segment Service

gRPC service define

• UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
• Span data please refs to Plugin Development Guide
• Id and name both exist, please use id if possible.
  • operationNameId/operationName
  • networkAddress/networkAddressId
  • entryServiceName/entryServiceId
  • parentServiceName/parentServiceId
  • peerId/peer
• componentIds are defined in backend, here

HTTP JSON Services

All HTTP Services match the gRPC services, just adjust use short keys.

Instance Register Service

• http://ip:port/instance/register(default: localhost:12800)

Input:

{
    ai: x, #applicationId
    au: "", #agentUUID
    rt: x, #registerTime
    oi: "", #osinfo
}

Output:

{
    ai: x, #applicationId
    ii: x, #applicationInstanceId
}

Heart beat service

• http://ip:port/instance/heartbeat(default: localhost:12800)

Input:

{
    "ii": x, #applicationInstanceId
    "ht": x #heartbeatTime, java timestamp format
}

Output: empty

Service Name Discovery

• http://ip:port/servicename/discovery(default: localhost:12800)

Input：

{
    ai: x, #applicationId
    sn: "", #serviceName
    st: x, #srcSpanType
}

Output：

{
    si: x, #osinfo
    el: { #element
        ai: x, #applicationId
        sn: "", #serviceName
        st: x, #srcSpanType
    }
}

Trace Segment Service

Input：

[
  {
    "gt": [[230150, 185809, 24040000]], 
    "sg": { //TraceSegmentObject 
      "ts": [137150, 185809, 48780000], 
      "ai": 2, //applicationId
      "ii": 3, //applicationInstanceId
      "ss": [ //SpanObject
        {
          "si": 0, //spanId
          "tv": 0, //SpanType
          "lv": 2, //SpanLayer
          "ps": -1, //parentSpanId
          "st": 1501858094726, //startTime
          "et": 1501858096804, //endTime
          "ci": 3, //componentId
          "cn": "", //component
          "oi": 0, //operationNameId
          "on": "org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()", //operationName
          "pi": 0, //peerId
          "pn": "", //peer
          "ie": false, //isError
          "rs": [ //TraceSegmentReference
            {
              "pts": [230150, 185809, 24040000], //parentTraceSegmentId
              "pii": 2, //parentApplicationInstanceId
              "psp": 1, //parentSpanId
              "psi": 0, //parentServiceId
              "psn": "/dubbox-case/case/dubbox-rest", //parentServiceName
              "ni": 0,  //networkAddressId
              "nn": "172.25.0.4:20880", //networkAddress
              "eii": 2, //entryApplicationInstanceId
              "esi": 0, //entryServiceId
              "esn": "/dubbox-case/case/dubbox-rest", //entryServiceName
              "rv": 0 //RefTypeValue
            }
          ],
          "to": [ //KeyWithStringValue
            {
              "k": "url", //key
              "v": "rest://172.25.0.4:20880/org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()" //value
            },
            {
              "k": "http.method",
              "v": "GET"
            }
          ],
          "lo": [{
                "ti": 1501858094726,
                "ld": [{ 
                        "k": "NullPointException",
                        "v": "Error Stack"
                    }]
           }]
        },
        {
          "si": 1,
          "tv": 1,
          "lv": 1,
          "ps": 0,
          "st": 1501858094726,
          "et": 1501858095804,
          "ci": 9,
          "cn": "",
          "oi": 0,
          "on": "mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]",
          "pi": 0,
          "pn": "localhost:27017",
          "ie": false,
          "to": [],
          "lo": []
        }
      ]
    }
  }
]