Update docs for new data-collecting and query protocols (#9809)

docs/en/changes/changes.md

@@ -98,5 +98,7 @@
 * Add re-post for blog `Scaling with Apache SkyWalking` in the academy list.
 * Add re-post for blog `Diagnose Service Mesh Network Performance with eBPF` in the academy list.
 * Add **Security Notice** doc.
+* Add new docs for `Report Span Attached Events` data collecting protocol.
+* Add new docs for `Record` query protocol
 
 All issues and pull requests are [here](https://github.com/apache/skywalking/milestone/149?closed=1)

docs/en/concepts-and-designs/manual-sdk.md

@@ -1,5 +1,6 @@
 # Manual instrument SDK
 Our incredible community has contributed to the manual instrument SDK.
+- [Rust](https://github.com/apache/skywalking-rust). Rust SDK follows the SkyWalking format.
 - [Go2Sky](https://github.com/SkyAPM/go2sky). Go SDK follows the SkyWalking format.
 - [C++](https://github.com/SkyAPM/cpp2sky). C++ SDK follows the SkyWalking format. 
 

docs/en/protocols/README.md

@@ -13,7 +13,7 @@ Please read SkyWalking language agents documentation to see whether it is suppor
 
 - **Trace Data Protocol** is an out-of-wire data format. Agent/SDK uses this to send traces to SkyWalking OAP server.
 
-[SkyWalking Trace Data Protocol v3](Trace-Data-Protocol-v3.md) defines the communication method and format between the agent and backend.
+[SkyWalking Trace Data Protocol v3.1](Trace-Data-Protocol-v3.md) defines the communication method and format between the agent and backend.
 
 ### Logging
 - **Log Data Protocol** is an out-of-wire data format. Agent/SDK and collector use this to send logs into SkyWalking OAP server.

docs/en/protocols/Trace-Data-Protocol-v3.md

@@ -1,17 +1,16 @@
-# Trace Data Protocol v3
+# Trace Data Protocol v3.1
 Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend. 
 
-## Overview
 Trace data protocol is defined and provided in [gRPC format](https://github.com/apache/skywalking-data-collect-protocol), and implemented in [HTTP 1.1](HTTP-API-Protocol.md).
 
-### Report service instance status
+## Report service instance status
 1. Service Instance Properties 
 Service instance contains more information than just a name. In order for the agent to report service instance status, use `ManagementService#reportInstanceProperties` service to provide a string-key/string-value pair list as the parameter. The `language` of target instance must be provided as the minimum requirement.
 
 2. Service Ping
 Service instance should keep alive with the backend. The agent should set a scheduler using `ManagementService#keepAlive` service every minute.
 
-### Send trace and metrics
+## Send trace and JVM metrics
 After you have the service ID and service instance ID ready, you could send traces and metrics. Now we
 have 
 1. `TraceSegmentReportService#collect` for the SkyWalking native trace format
@@ -40,7 +39,7 @@ See [Cross Process Propagation Headers Protocol v3](Skywalking-Cross-Process-Pro
 
 4. `Span#skipAnalysis` may be TRUE, if this span doesn't require backend analysis.
 
-### Protocol Definition
+## Trace Report Protocol
 ```protobuf
 // The segment is a collection of spans. It includes all collected spans in a simple one request context, such as a HTTP request process.
 //
@@ -220,3 +219,70 @@ message SegmentCollection {
     repeated SegmentObject segments = 1;
 }
 ```
+
+## Report Span Attached Events
+Besides in-process agents, there are other out-of-process agent, such as ebpf agent, could report additional information
+as attached events for the relative spans.
+
+`SpanAttachedEventReportService#collect` for attached event reporting.
+
+```protobuf
+////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+// ebpf agent(SkyWalking Rover) collects extra information from the OS(Linux Only) level to attach on the traced span.
+// Since v3.1
+////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
+service SpanAttachedEventReportService {
+    // Collect SpanAttachedEvent to the OAP server in the streaming mode.
+    rpc collect (stream SpanAttachedEvent) returns (Commands) {
+    }
+}
+
+// SpanAttachedEvent represents an attached event for a traced RPC.
+//
+// When an RPC is being traced by the in-process language agent, a span would be reported by the client-side agent.
+// And the rover would be aware of this RPC due to the existing tracing header.
+// Then, the rover agent collects extra information from the OS level to provide assistance information to diagnose network performance.
+message SpanAttachedEvent {
+    // The nanosecond timestamp of the event's start time.
+    // Notice, most unit of timestamp in SkyWalking is milliseconds, but NANO-SECOND is required here.
+    // Because the attached event happens in the OS syscall level, most of them are executed rapidly.
+    Instant startTime = 1;
+    // The official event name.
+    // For example, the event name is a method signature from syscall stack.
+    string event = 2;
+    // [Optional] The nanosecond timestamp of the event's end time.
+    Instant endTime = 3;
+    // The tags for this event includes some extra OS level information,
+    // such as
+    // 1. net_device used for this exit span.
+    // 2. network L7 protocol
+    repeated KeyStringValuePair tags = 4;
+    // The summary of statistics during this event.
+    // Each statistic provides a name(metric name) to represent the name, and an int64/long as the value.
+    repeated KeyIntValuePair summary = 5;
+    // Refer to a trace context decoded from `sw8` header through network, such as HTTP header, MQ metadata
+    // https://skywalking.apache.org/docs/main/next/en/protocols/skywalking-cross-process-propagation-headers-protocol-v3/#standard-header-item
+    SpanReference traceContext = 6;
+
+    message SpanReference {
+        SpanReferenceType type = 1;
+        // [Optional] A string id represents the whole trace.
+        string traceId = 2;
+        // A unique id represents this segment. Other segments could use this id to reference as a child segment.
+        // [Optional] when this span reference
+        string traceSegmentId = 3;
+        // If type == SKYWALKING
+        // The number id of the span. Should be unique in the whole segment.
+        // Starting at 0
+        //
+        // If type == ZIPKIN
+        // The type of span ID is string.
+        string spanId = 4;
+    }
+
+    enum SpanReferenceType {
+        SKYWALKING = 0;
+        ZIPKIN = 1;
+    }
+}
+```

docs/en/protocols/query-protocol.md

@@ -80,6 +80,7 @@ extend type Query {
     readLabeledMetricsValues(condition: MetricsCondition!, labels: [String!]!, duration: Duration!): [MetricsValues!]!
     # Heatmap is bucket based value statistic result.
     readHeatMap(condition: MetricsCondition!, duration: Duration!): HeatMap
+    # Deprecated since 9.3.0, replaced by readRecords defined in record.graphqls
     # Read the sampled records
     # TopNCondition#scope is not required.
     readSampledRecords(condition: TopNCondition!, duration: Duration!): [SelectedRecord!]!
@@ -127,6 +128,18 @@ extend type Query {
 }
 ```
 
+### Record
+Record is a general and abstract type for collected raw data.
+In the observability, traces and logs have specific and well-defined meanings, meanwhile, the general records represent other
+collected records. Such as sampled slow SQL statement, HTTP request raw data(request/response header/body)
+
+```graphql
+extend type Query {
+    # Query collected records with given metric name and parent entity conditions, and return in the requested order.
+    readRecords(condition: RecordCondition!, duration: Duration!): [Record!]!
+}
+```
+
 ### Logs
 ```graphql
 extend type Query {