skywalking-java/docs/en/protocols/Trace-Data-Protocol-v1.md

Trace Data Protocol

Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend.

Abstract

This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to uplink data to the SkyWalking backend.

• Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both.

Version

v1

gRPC proto files

gRPC proto files

Trace Segment Service

gRPC service define

• UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
• Span data please refs to Plugin Development Guide
• Id and name both exist, please use id if possible.
  • operationNameId/endpointName
  • networkAddress/networkAddressId
  • entryServiceName/entryServiceId
  • parentServiceName/parentServiceId
  • peerId/peer
• componentIds are defined in backend, here

HTTP format:

Input：

[
  {
    "gt": [[230150, 185809, 24040000]], 
    "sg": { //TraceSegmentObject 
      "ts": [137150, 185809, 48780000], 
      "ai": 2, //serviceId
      "ii": 3, //applicationInstanceId
      "ss": [ //SpanObject
        {
          "si": 0, //spanId
          "tv": 0, //SpanType
          "lv": 2, //SpanLayer
          "ps": -1, //parentSpanId
          "st": 1501858094726, //startTime
          "et": 1501858096804, //endTime
          "ci": 3, //componentId
          "cn": "", //component
          "oi": 0, //operationNameId
          "on": "org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()", //endpointName
          "pi": 0, //peerId
          "pn": "", //peer
          "ie": false, //isError
          "rs": [ //TraceSegmentReference
            {
              "pts": [230150, 185809, 24040000], //parentTraceSegmentId
              "pii": 2, //parentServiceInstanceId
              "psp": 1, //parentSpanId
              "psi": 0, //parentServiceId
              "psn": "/dubbox-case/case/dubbox-rest", //parentServiceName
              "ni": 0,  //networkAddressId
              "nn": "172.25.0.4:20880", //networkAddress
              "eii": 2, //entryServiceInstanceId
              "esi": 0, //entryServiceId
              "esn": "/dubbox-case/case/dubbox-rest", //entryServiceName
              "rv": 0 //RefTypeValue
            }
          ],
          "to": [ //KeyWithStringValue
            {
              "k": "url", //key
              "v": "rest://172.25.0.4:20880/org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()" //value
            },
            {
              "k": "http.method",
              "v": "GET"
            }
          ],
          "lo": [{
                "ti": 1501858094726,
                "ld": [{ 
                        "k": "NullPointException",
                        "v": "Error Stack"
                    }]
           }]
        },
        {
          "si": 1,
          "tv": 1,
          "lv": 1,
          "ps": 0,
          "st": 1501858094726,
          "et": 1501858095804,
          "ci": 9,
          "cn": "",
          "oi": 0,
          "on": "mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]",
          "pi": 0,
          "pn": "localhost:27017",
          "ie": false,
          "to": [],
          "lo": []
        }
      ]
    }
  }
]

Deprecated services

Deprecated service(s) are the gRPC services SkyWalking used before. In SkyWalking v6, in order to match the common concepts in CloudNative world, these services are deprecated. Although there services are still supported at this moment, but it will keep in supported at least before the end of 2019.

Application Register Service

Deprecated service

Abstract

Register Application Code to the backend, and receive an integer represents the application.

gRPC service define

• applicationCode is the config in your agent.config.
• The return id is ApplicationId as the value in KeyWithIntegerValue, which will be used in further data uplink.

Discovery Services

Deprecated services

Register Instance Service

gRPC service define

• agentUUID generated by agent, should be unique. Stay same before reboot, at least.
• ApplicationInstanceId will be used in further data uplink.

HTTP format http://ip:port/instance/register(default: localhost:12800)

Input:

{
    ai: x, #serviceId
    au: "", #agentUUID
    rt: x, #registerTime
    oi: "", #osinfo
}

Output:

{
    ai: x, #serviceId
    ii: x, #applicationInstanceId
}

Heart beat service

gRPC service define

• Recommend to report heart beat every 20-60 seconds.
• Java agent don't use this, because JVM metrics upstream replace the capabilities of this.

HTTP format http://ip:port/instance/heartbeat(default: localhost:12800)

Input:

{
    "ii": x, #applicationInstanceId
    "ht": x #heartbeatTime, java timestamp format
}

Service Name Discovery Service

Deprecated services

Abstract

Replace the literal String service(operation) name by an id(integer)

gRPC service define

• Optional service, reduce the network cost but use more memory as a buffer mapping.

HTTP format http://ip:port/servicename/discovery(default: localhost:12800)

Input：

{
    ai: x, #serviceId
    sn: "", #serviceName
    st: x, #srcSpanType
}

Output：

{
    si: x, #osinfo
    el: { #element
        ai: x, #serviceId
        sn: "", #serviceName
        st: x, #srcSpanType
    }
}

Network Address Register Service

Abstract

Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc.

gRPC service define

• Optional service, reduce the network cost but use more memory as a buffer mapping.