kazusa
/
skywalking
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Finish the new document structure.

This commit is contained in:
Wu Sheng 2018-07-08 21:04:50 -07:00
parent d2753037bb
commit f121730ede
43 changed files with 267 additions and 1108 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
README.md
View File

@ -29,28 +29,6 @@ microservices, cloud native and container-based (Docker, Kubernetes, Mesos) arch
<img src="https://skywalkingtest.github.io/page-resources/6_overview.png"/>
SkyWalking provides multiple probes to collect data from different sources.
- SkyWalking native instrument agents or SDKs
- Auto instrument agents
* Java. Included in official release.
* [.NET Core](https://github.com/OpenSkywalking/skywalking-netcore)
* [Node.js](https://github.com/OpenSkywalking/skywalking-nodejs)
- SDKs
* OpenCensus with SkyWalking exporter.
- Service mesh control panel or data panel probes.
- Istio mixer adaptor.
- Linkerd.
- 3rd party data tracing format.
- Zipkin JSON, Thrift, Protobuf v1 and v2 formats, powered by [OpenZipkin](https://github.com/openzipkin/zipkin) libs
- Jaeger in [Zipkin Thrift or JSON v1/v2 formats](https://github.com/jaegertracing/jaeger#backwards-compatibility-with-zipkin).
# Recommend and typical usages
1. All service are under SkyWalking native agents or SDKs monitoring.
2. Service mesh probes.
3. Service mesh probes + distributed tracing(SkyWalking's or 3rd-party's).
# Document
- [Documents in English](docs/README.md)

89
docs/README.md
View File

@ -1,66 +1,27 @@
## Documents
# Welcome
**SkyWalking 6 documents and codes are still on building, Status: WIP. Of course, we welcome everyone to join us.**
* [SkyWalking Platform Overview](en/OAP/README.md)
* Use application agents
* Use service mesh probes
* Deploy OAP backend
We will begin to build our new documents for our new 6 version. Below it is 5.x document, wait for reformat.
----------------------------
* Getting Started
* [Quick start](en/Quick-start.md)
* [Install javaagent](en/Deploy-skywalking-agent.md)
* [Deploy backend in cluster mode](en/Deploy-backend-in-cluster-mode.md)
* [Supported middlewares, frameworks and libraries](Supported-list.md)
* [How to disable plugins?](en/How-to-disable-plugin.md)
* [Optional plugins](en/Optional-plugins.md)
* [Trace Spring beans](en/agent-optional-plugins/Spring-bean-plugins.md)
* [Trace Oracle and Resin](en/agent-optional-plugins/Oracle-Resin-plugins.md)
* [[**Incubating**] Filter traces through custom services](../apm-sniffer/optional-plugins/trace-ignore-plugin/README.md)
* Advanced Features
* [Override settings through System.properties](en/Setting-override.md)
* [Direct uplink and disable naming discovery](en/Direct-uplink.md)
* [Open TLS](en/TLS.md)
* [Namespace Isolation](en/Namespace.md)
* [Token Authentication](en/Token-auth.md)
* [Add your own component library settings in collector](en/Component-libraries-extend.md)
* Incubating Features
* [Why are some features in **Incubating**?](en/Incubating/Abstract.md)
* [Use Sharding JDBC as storage implementor](en/Use-ShardingJDBC-as-storage-implementor.md)
* [Receive Zipkin span data format](../apm-collector/apm-collector-thirdparty-receiver/receiver-zipkin/docs/README.md)
* Application Toolkit
* [Overview](en/Applicaton-toolkit.md)
* [Use SkyWalking OpenTracing compatible tracer](en/Opentracing.md)
* Integration with log frameworks
* [log4j](en/Application-toolkit-log4j-1.x.md)
* [log4j2](en/Application-toolkit-log4j-2.x.md)
* [logback](en/Application-toolkit-logback-1.x.md)
* [Trace by SkyWalking manual APIs](en/Application-toolkit-trace.md)
* [Trace across threads](en/Application-toolkit-trace-cross-thread.md)
* Testing
* [Plugin Test](https://github.com/SkywalkingTest/agent-integration-test-report)
* [Java Agent Performance Test](https://skywalkingtest.github.io/Agent-Benchmarks/)
* Development Guides
* [How to build project](en/How-to-build.md)
* [Plugin development guide](en/Plugin-Development-Guide.md)
* [Collector table description](en/Collector-Table-Description.md)
* Protocol
* [Cross Process Propagation Headers Protocol, v1.0](en/Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md)
* [SkyWalking Trace Data Protocol](en/Trace-Data-Protocol.md)
* [Release Guide](en/How-to-release.md)
* [Roadmap](ROADMAP.md)
* Resources provided by community
* [Public speakings](https://github.com/OpenSkywalking/Community#public-speakings)
* [Videos](https://github.com/OpenSkywalking/Community#videos)
* [Articles](https://github.com/OpenSkywalking/Community#articles)
* FAQ
* [Why only traces in UI?](en/FAQ/Why-have-traces-no-others.md)
* [Too many GRPC logs in the console](en/FAQ/Too-many-gRPC-logs.md)
* [The trace doesn't continue in kafka consumer side](en/FAQ/kafka-plugin.md)
* [Agent or collector version upgrade](en/FAQ/Upgrade.md)
* [Protoc plugin fails in maven build](en/FAQ/Protoc-Plugin-Fails-When-Build.md)
* [EnhanceRequireObjectCache class cast exception](en/FAQ/EnhanceRequireObjectCache-Cast-Exception.md)
* [Required items could not be found, when import project into Eclipse](en/FAQ/Import-Project-Eclipse-RequireItems-Exception.md)
From here you can learn all about **SkyWalking**’s architecture, how to deploy and use SkyWalking and
resources related to the project.
- [Concepts and Designs](en/concepts-and-designs/README.md). Concepts and designs explain the most important core ideas about
SkyWalking. You can learn from here if you want to understand what is going on under our cool features and visualization.
- [Setup](en/setup/README.md). Setup contains guides for installing SkyWalking in different scenarios. As a platform, it provides
several ways to provide observability, including monitoring and alarm of course.
- [Guides](en/guides/README.md). Guides are for PPMC, committer or new contributor. At here, you can know how to develop, release
and what is ongoing of the community.
- [Protocols](en/protocols/README.md). Protocols show the communication ways between agents/probes and backend. Anyone, interested
in uplink telemetry data, definitely should read this.
- [FAQs](en/FAQ/README.md). Include a manifest, including already known setup problems, secondary developments experiments. When
you are facing a problem, check here first.
In addition, you might find these links interesting:
- The latest and old releases are all available in [Apache SkyWalking release page](http://skywalking.apache.org/downloads/) and [CHANGELOG](../CHANGES.md).
- You can find the speaking schedules at Conf, online videos and articles about SkyWalking in [Community resource catalog](https://github.com/OpenSkywalking/Community).
Also, if you have some related to us, welcome to submit a pull request to add.
- We’re always looking for help improving our documentation and codes, so please don’t hesitate to [file an issue](https://github.com/apache/incubator-skywalking/issues/new)
if you see some problem.
Or better yet, submit your own contributions to help make them better.

572
docs/en/Collector-Table-Description.md
View File

@ -1,572 +0,0 @@
# Collector Table Description
This document describes the usage of tables and their columns, based on elasticsearch storage implementation.
## Metric table time bucket
### Date format
- second: `yyyyMMddHHmmss`
- minute: `yyyyMMddHHmm`
- hour: `yyyyMMddHH`
- day: `yyyyMMdd`
- month: `yyyyMM`
## Tables of Register related
### Application
- Table name: application
- Get or create a database record by "application_code".
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value same as application_id
application_code | ac | Keyword | The application name, see `agent.config`
application_id | ai | Integer | Auto increment, is a signed integer
is_address | ia | Integer | Is a boolean data. True(1), False(0)
address_id | ni | Integer | A foreign key reference by network_address table
- Column `is_address`
- `false`. A real application, which has a custom `application_code`. At the same time, the `address_id` column value must to be 0.
- `true`. A conjunction application based on IP address. `address_id` is registered in `network_address` table.
### Instance
- Table name: instance
- Create a instance by `application_id` and `agent_uuid`
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value same as instance_id
application_id | ai | Integer | Owner application id
application_code | ac | Text | Owner application code
agent_uuid | iau | Keyword | Uniquely identifies each server monitored by agent
register_time | irt | Long | First register time
instance_id | ii | Integer | Auto increment, is a unsigned integer
heartbeat_time | iht | Long | Represent server is alive
os_info | ioi | Text | A Json data.
is_address | iia | Integer | Is a boolean data. True(1), False(0)
address_id | ni | Integer | A foreign key reference by network_address table
- Column `os_info`
- For example: {"osName":"MacOS X","hostName":peng-yongsheng","processId":1000,"ipv4s":["10.0.0.1","10.0.0.2"]}
- Column `heartbeat_time`
- Updated by agent heart beat [1]
- Updated by JVM metric data [2]
- Updated by trace segment data. [3]
- Priority: [1] > [2] > [3]
### NetworkAddress
- Table name: network_address
- Create a network address record by "network_address" and "span_layer".
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value same as address_id
address_id | ni | Integer | Auto increment, is a signed integer
network_address | na | Keyword | Host name or IP address
src_span_layer | ssl | Integer | Register by client or server side
server_type | st | Integer | Such as component id, used for topology.
### ServiceName
- Table name: service_name
- Create a service record by "service_name_keyword", "application_id" and "src_span_type".
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value same as service_id
service_id | si | Integer | Auto increment, is a signed integer
service_name | sn | Text | Operation name, used for fuzzy matching
service_name_keyword | snk | Keyword | Operation name, used for full matching
application_id | ai | Integer | Owner application id
src_span_type | sst | Integer | Register from client or server side based on `src_span_type`
- See `src_span_type` in [protocol doc](Trace-Data-Protocol.md#network-address-register-service)
## Table of Trace Metric related
### ApplicationComponent
- Table name: application_component_`TimeDimension`
- TimeDimension contains minute, hour, day, month
- It is primarily used for the view of node type in application topology.
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `application_id`_`component_id`
component_id | ci | Integer | [Component id](../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)
application_id | ai | Integer | Owner application id
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationMapping
- Table name: application_mapping_`TimeDimension`
- TimeDimension contains minute, hour, day, month
- For example: Application A calls Application B, collector generates the following two metrics::
* From the caller's trace data: A application -> B application's IP address (Topology will use this metric when B application is not monitored by agent)
* From the callee's trace data: A application -> B application (Topology will use this metric when B application is monitored by agent)
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `application_id`_`mapping_application_id`
application_id | ai | Integer | Registered at server side.
mapping_application_id | mai | Integer | Registered at client side with the server's IP address.
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationMetric
- Table name: application_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `application_id`_`source_value`
application_id | ai | Integer | Owner application id
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
satisfied_count | a1 | Long | [The formula](../../apm-collector/apm-collector-core/src/main/java/org/apache/skywalking/apm/collector/core/util/ApdexThresholdUtils.java)
tolerating_count | a2 | Long |
frustrated_count | a3 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationReferenceMetric
- Table name: application_reference_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `front_application_id`_`behind_application_id`_`source_value`
front_application_id | fai | Integer |
behind_application_id | bai | Integer |
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
satisfied_count | a1 | Long | [The formula](../../apm-collector/apm-collector-core/src/main/java/org/apache/skywalking/apm/collector/core/util/ApdexThresholdUtils.java)
tolerating_count | a2 | Long |
frustrated_count | a3 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceMapping
- Table name: instance_mapping_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `instance_id`_`address_id`
application_id | ai | Integer | The `instance_id`'s owner application id.
instance_id | ii | Integer | Registered at server side.
address_id | ni | Integer | Registered at client side with the server's IP address.
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceMetric
- Table name: instance_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `instance_id`_`source_value`
application_id | ai | Integer |
instance_id | ii | Integer |
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceReferenceMetric
- Table name: instance_reference_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `front_instance_id`_`behind_instance_id`_`source_value`
front_application_id | fai | Integer |
behind_application_id | bai | Integer |
front_instance_id | fii | Integer |
behind_instance_id | bii | Integer |
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceMetric
- Table name: service_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `service_id`_`source_value`
application_id | ai | Integer |
instance_id | ii | Integer |
service_id | si | Integer |
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceReferenceMetric
- Table name: service_reference_metric_`TimeDimension`
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `time_bucket`_`metric_id`
metric_id | mi | Keyword | `front_service_id`_`behind_service_id`_`source_value`
front_application_id | fai | Integer |
front_instance_id | fii | Integer |
front_service_id | fsi | Integer |
behind_application_id | bai | Integer |
behind_instance_id | bii | Integer |
behind_service_id | bsi | Integer |
source_value | sv | Integer | Caller(0), Callee(1)
transaction_calls | t1 | Long | The total number of calls based on `time_bucket`
transaction_error_calls | t2 | Long | The total number of error calls, sums values aggregate by `time_bucket`
transaction_duration_sum | t3 | Long | The total duration of calls based on `time_bucket`
transaction_error_duration_sum | t4 | Long | The total duration of error calls based on `time_bucket`
transaction_average_duration | t5 | Long | The average duration of all calls, used for order by this column in database.
business_transaction_calls | b1 | Long |
business_transaction_error_calls | b2 | Long |
business_transaction_duration_sum | b3 | Long |
business_transaction_error_duration_sum | b4 | Long |
business_transaction_average_duration | b5 | Long |
mq_transaction_calls | m1 | Long |
mq_transaction_error_calls | m2 | Long |
mq_transaction_duration_sum | m3 | Long |
mq_transaction_error_duration_sum | m4 | Long |
mq_transaction_average_duration | m5 | Long |
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### GlobalTrace
- Table name: global_trace
- The relationship between trace id and segment id is many to many.
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `global_trace_id`_`segment_id`
segment_id | sgi | Keyword |
trace_id | ti | Keyword |
time_bucket | tb | Long | Second date format, [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### SegmentDuration
- Table name: segment_duration
- The relationship between trace id and segment id is many to many.
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, `global_trace_id`_`segment_id`
segment_id | sgi | Keyword |
trace_id | ti | Keyword |
application_id | ai | Integer | Owner of the segment
service_name | sn | Text | The entry span's operation name in this segment
duration | ddt | Long | The cost duration of this segment
start_time | dst | Long |
end_time | det | Long |
is_error | die | Long | Is a boolean data. True(1), False(0)
time_bucket | tb | Long | Second date format, [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### Segment
- Table name: segment
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value same as `segment_id`
data_binary | sdb | Binary | The protobuf segment binary encode by base64 scheme.
time_bucket | tb | Long | Second date format, [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
## Tables of JVM Metric related
### CpuMetric
- Table name: cpu_metric_`TimeDimension`
- TimeDimension contains second, minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`metric_id`
metric_id | mi | Keyword | the value is `instance_id`
instance_id | ii | Integer | Owner instance id
usage_percent | up | Double | Cpu usage percent, sums values aggregate by `time_bucket`
times | t | Long | The records received times in this time bucket
time_bucket | tb | Long | [Date format](Collector-Table-Description.md#Metric-table-time-bucket)
### GCMetric
- Table name: gc_metric_`TimeDimension`
- TimeDimension contains second, minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`metric_id`
metric_id | mi | Keyword | the value is `instance_id`_`phrase`
instance_id | ii | Integer | Owner instance id
phrase | p | Integer | [GCPhrase](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/master/JVMMetricsService.proto#L80-L83)
count | c | Long | GC count, sums values aggregate by `time_bucket`
times | t | Long | The records received times in this time bucket
duration | d | Long | GC duration count, sums values aggregate by `time_bucket`
time_bucket | tb | Long | [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### MemoryMetric
- Table name: memory_metric_`TimeDimension`
- TimeDimension contains second, minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`metric_id`
metric_id | mi | Keyword | the value is `instance_id`_`is_heap`
instance_id | ii | Integer | Owner instance id
is_heap | ih | Integer | Is a boolean data. True(1), False(0)
init | init | Long | Sums values aggregate by `time_bucket`
max | max | Long | Sums values aggregate by `time_bucket`
used | used | Long | Sums values aggregate by `time_bucket`
committed | cd | Long | Sums values aggregate by `time_bucket`
times | t | Long | The records received times in this time bucket
time_bucket | tb | Long | [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### MemoryPoolMetric
- Table name: memory_pool_metric_`TimeDimension`
- TimeDimension contains second, minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`metric_id`
metric_id | mi | Keyword | the value is `instance_id`_`pool_type`
instance_id | ii | Integer | Owner instance id
pool_type | pt | Integer | Owner instance id
init | init | Long | Sums values aggregate by `time_bucket`
max | max | Long | Sums values aggregate by `time_bucket`
used | used | Long | Sums values aggregate by `time_bucket`
committed | ct | Long | Sums values aggregate by `time_bucket`
times | t | Long | The records received times in this time bucket
time_bucket | tb | Long | [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
## Table of Alarm metric related
### ApplicationAlarm
- Table name: application_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`application_id`
application_id | ai | Integer | Owner application id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationAlarmList
- Table name: application_alarm_list
- TimeDimension contains minute, hour, day, month
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`metric_id`
metric_id | mi | Keyword | `source_value`_`alarm_type`_`application_id`
alarm_content | aac | Text |
application_id | ai | Integer | Owner application id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
time_bucket | tb | Long | [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationReferenceAlarm
- Table name: application_reference_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`application_id`
front_application_id | fai | Integer | Owner application id
behind_application_id | bai | Integer | Owner application id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ApplicationReferenceAlarmList
- Table name: application_reference_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`application_id`
front_application_id | fai | Integer | Owner application id
behind_application_id | bai | Integer | Owner application id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
time_bucket | tb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceAlarm
- Table name: application_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`instance_id`
application_id | ai | Integer | Owner application id
instance_id | ii | Integer | Owner instance id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceAlarmList
- Table name: instance_alarm_list
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`instance_id`
application_id | ai | Integer | Owner application id
instance_id | ii | Integer | Owner instance id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
time_bucket | tb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceReferenceAlarm
- Table name: instance_reference_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`instance_id`
front_application_id | fai | Integer | Owner application id
front_instance_id | fii | Integer | Owner instance id
behind_application_id | bai | Integer | Owner instance id
behind_instance_id | bii | Integer | Owner instance id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### InstanceReferenceAlarmList
- Table name: instance_reference_alarm_list
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`instance_id`
front_application_id | fai | Integer | Owner application id
front_instance_id | fii | Integer | Owner instance id
behind_application_id | bai | Integer | Owner instance id
behind_instance_id | bii | Integer | Owner instance id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
time_bucket | tb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceAlarm
- Table name: service_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`service_id`
application_id | ai | Integer | Owner application id
instance_id | ii | Integer | Owner instance id
service_id | si | Integer | Owner service id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceAlarmList
- Table name: service_alarm_list
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`service_id`
application_id | ai | Integer | Owner application id
instance_id | ii | Integer | Owner instance id
service_id | si | Integer | Owner instance id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
time_bucket | tb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceReferenceAlarm
- Table name: service_reference_alarm
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `source_value`_`alarm_type`_`service_id`
front_application_id | fai | Integer | Owner application id
front_instance_id | fii | Integer | Owner instance id
front_service_id | fsi | Integer | Owner service id
behind_application_id | bai | Integer | Owner service id
behind_instance_id | bii | Integer | Owner service id
behind_service_id | bsi | Integer | Owner service id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
last_time_bucket | ltb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)
### ServiceReferenceAlarmList
- Table name: service_reference_alarm_list
Column Name | Short Name | Data Type | Description
----------- | ---------- | --------- | ---------
_id | _id | Keyword | primary key, es speciality, the value is `time_bucket`_`source_value`_`alarm_type`_`service_id`
front_application_id | fai | Integer | Owner application id
front_instance_id | fii | Integer | Owner instance id
front_service_id | fsi | Integer | Owner service id
behind_application_id | bai | Integer | Owner service id
behind_instance_id | bii | Integer | Owner service id
behind_service_id | bsi | Integer | Owner service id
source_value | sv | Integer | Caller(0), Callee(1)
alarm_type | aat | Integer | ERROR_RATE(0), SLOW_RTT(1)
alarm_content | aac | Text |
time_bucket | tb | Long | Second date format. [A formatted date](Collector-Table-Description.md#Metric-table-time-bucket)

61
docs/en/Component-libraries-extend.md
View File

@ -1,61 +0,0 @@
# Component library settings
Component library settings are about your own or 3rd part libraries used in monitored application.
In agent or SDK, no matter library name collected as ID or String(literally, e.g. SpringMVC), collector
formats data in ID for better performance and less storage requirements.
Also, collector conjectures the remote service based on the component library, such as:
the component library is MySQL Driver library, then the remote service should be MySQL Server.
For those two reasons, collector require two parts of settings in this file:
1. Component Library id, name and languages.
1. Remote server mapping, based on local library.
**All component names and IDs must be defined in this file.**
## Component Library id
Define all component libraries' names and IDs, used in monitored application.
This is a both-way mapping, agent or SDK could use the value(ID) to represent the component name in uplink data.
- Name: the component name used in agent and UI
- id: Unique ID. All IDs are reserved, once it is released.
- languages: Program languages may use this component. Multi languages should be separated by `,`
### ID rules
- Java and multi languages shared: (0, 3000]
- .NET Platform reserved: (3000, 4000]
- Node.js Platform reserved: (4000, 5000]
- Go reserved: (5000, 6000]
- PHP reserved: (6000, 7000]
Example
```yaml
Tomcat:
id: 1
languages: Java
HttpClient:
id: 2
languages: Java,C#,Node.js
Dubbo:
id: 3
languages: Java
H2:
id: 4
languages: Java
```
## Remote server mapping
Remote server will be conjectured by the local component. The mappings are based on Component library names.
- Key: client component library name
- Value: server component name
```yaml
Component-Server-Mappings:
Jedis: Redis
StackExchange.Redis: Redis
SqlClient: SqlServer
Npgsql: PostgreSQL
MySqlConnector: Mysql
EntityFrameworkCore.InMemory: InMemoryDatabase
```

149
docs/en/Deploy-backend-in-cluster-mode.md
View File

@ -1,149 +0,0 @@
## Requirements
- JDK 6+（instruments application can run in jdk6）
- JDK8 ( SkyWalking collector and SkyWalking WebUI )
- Elasticsearch 5.x, cluster mode or not
- Zookeeper 3.4.10
- OS time(include time zone) of Applications under monitoring, collectors, UIs and your local machine are all correct and same.
## Download released version
- Go to [released page](http://skywalking.apache.org/downloads/)
## Deploy Zookeeper
Zookeeper is used for collector coordination. Only required if you need more than one collector instances.
Add Zookeeper cluster info in each collector `application.yml`
```yml
cluster:
# The Zookeeper cluster for collector cluster management.
zookeeper:
# multiple instances should be separated by comma.
hostPort: localhost:2181
sessionTimeout: 100000
```
## Deploy ElasticSearch server
ElasticSearch is using for storage all traces, metrics and alarms.
- Modify `elasticsearch.yml`
- Set `cluster.name: CollectorDBCluster`
- Set `node.name: anyname`, this name can be any, it based on Elasticsearch.
- Add the following configurations
```
# The ip used for listening
network.host: 0.0.0.0
thread_pool.bulk.queue_size: 1000
```
See ElasticSearch Official documents to understand how to deploy cluster(Recommend).
- Start Elasticsearch
### Set collector
There are five types of connection for SkyWalking cluster, related to collector settings.
1. Agent to collectors by using HTTP, named as `naming`.
1. Agent to collectors by using gRPC, named as `agent_gRPC`.
1. Collector to collector by using gRPC, named as `remote`.
1. UI to collector by using HTTP, named as `ui`. Don't need to change in most cases.
1. Optional connection: Agent to collector by using HTTP, named as `agent_jetty`.
The following segment of `application.yml` shows you the detail of each settings.
- `config/application.yml`
```
cluster:
# The Zookeeper cluster for collector cluster management.
zookeeper:
hostPort: localhost:2181
sessionTimeout: 100000
naming:
# Host and port used for agent config
jetty:
# OS real network IP(binding required), for agent to find collector cluster. agent --(HTTP)--> collector
host: localhost
port: 10800
contextPath: /
remote:
gRPC:
# OS real network IP(binding required), for collector node to communicate with each other in cluster. collectorN --(gRPC) --> collectorM
host: localhost
port: 11800
agent_gRPC:
gRPC:
# OS real network IP(binding required), for agent to uplink data(trace/metrics) to collector. agent--(gRPC)--> collector
host: localhost
port: 11800
agent_jetty:
jetty:
# OS real network IP(binding required), for agent to uplink data(trace/metrics) to collector through HTTP. agent--(HTTP)--> collector
# SkyWalking native Java/.Net/node.js agents don't use this.
# Open this for other implementor.
host: localhost
port: 12800
contextPath: /
analysis_register:
default:
analysis_jvm:
default:
analysis_segment_parser:
default:
bufferFilePath: ../buffer/
bufferOffsetMaxFileSize: 10M
bufferSegmentMaxFileSize: 500M
ui:
jetty:
# OS real network IP(binding required), for UI to query from collector.
host: localhost
port: 12800
contextPath: /
# Config Elasticsearch cluster connection info.
storage:
elasticsearch:
clusterName: CollectorDBCluster
clusterTransportSniffer: true
clusterNodes: localhost:9300
indexShardsNumber: 2
indexReplicasNumber: 0
highPerformanceMode: true
# Set an expired for metric/trace data. After the timeout has expired, the metric/trace data will be deleted automatically.
traceDataTTL: 90 # Unit is minute
minuteMetricDataTTL: 45 # Unit is minute
hourMetricDataTTL: 36 # Unit is hour
dayMetricDataTTL: 45 # Unit is day
monthMetricDataTTL: 18 # Unit is month
configuration:
default:
# namespace: xxxxx
# alarm threshold
applicationApdexThreshold: 2000
serviceErrorRateThreshold: 10.00
serviceAverageResponseTimeThreshold: 2000
instanceErrorRateThreshold: 10.00
instanceAverageResponseTimeThreshold: 2000
applicationErrorRateThreshold: 10.00
applicationAverageResponseTimeThreshold: 2000
# thermodynamic
thermodynamicResponseTimeStep: 50
thermodynamicCountOfResponseTimeSteps: 40
```
### Set UI
The config items of UI is saved in `webapp/webapp.yml`.
Change `collector.ribbon.listOfServers` by following the description, with matching `naming.jetty`.
| Config | Description |
|----------------------------------|------------------------------------------------------------------------------------------------------|
| `server.port` | Port to listen on. Default: 8080 |
| `collector.ribbon.listOfServers` | Address to access collector naming service.(Consist with `naming.jetty` in `config/application.yml`). Multiple collector addresses are split by ',' |
| `collector.path` | Collector query uri. Default: /graphql |
| `collector.ribbon.ReadTimeout` | Query timeout. Default: 10 seconds |
| `security.user.*` | Login username/password. Default: admin/admin |
### Start up collector node
1. Run `bin/startup.sh` to start collector and UI together.
2. (Run if don't use 1 to start up)Run `bin/collectorService.sh` when you only want to start up collector.
3. (Run if don't use 1 to start up)Run `bin/webappService.sh` when you only want to start up UI.

42
docs/en/Deploy-backend-in-standalone-mode.md
View File

@ -1,42 +0,0 @@
# Usage scenario
Default standalone mode collector means don't support cluster. It uses H2 as storage layer implementation, suggest that use only for preview, test, demonstration, low throughputs and small scale system.
If you are using SkyWalking in a low throughputs monitoring scenario, and don't want to deploy cluster, at least, switch the storage implementation from H2 to Elasticsearch.
**H2 storage implementation is not provided in 5.0.0 for now, so you must deploy ElasticSearch before try to start backend. Welcome to contribute.**
## Requirements
- JDK 6+（instruments application can run in jdk6）
- JDK8 ( SkyWalking collector and SkyWalking WebUI )
- Elasticsearch 5.x, cluster mode or not
## Download
* [Releases](http://skywalking.apache.org/downloads/)
## Quick start
You can simply tar/unzip and startup if ports 8080, 10800, 11800, 12800 are free.
- Deploy ElasticSearch.
- `tar -xvf skywalking-dist.tar.gz` in Linux, or unzip in windows.
- run `bin/startup.sh` or `bin/startup.bat`
You should keep the `config/application.yml` as default.
- NOTICE: **In 5.0.0, startup.sh will start two processes, collector and UI, and UI uses 127.0.0.1:10800 as default.**
## Use Elastic Search instead of H2 as storage layer implementation
Even in standalone mode, collector can run with Elastic Search as storage. If so, uncomment the `storage` section in `application.yml`, set the config right. The default configs fit for collector and Elasticsearch both running in same machine, and not cluster.
## Deploy Elasticsearch server
- Modify `elasticsearch.yml`
- Set `cluster.name: CollectorDBCluster`
- Set `node.name: anyname`, this name can be any, it based on Elasticsearch.
- Add the following configurations
```
# The ip used for listening
network.host: 0.0.0.0
thread_pool.bulk.queue_size: 1000
```
- Start Elasticsearch

48
docs/en/Deploy-skywalking-agent.md
View File

@ -1,48 +0,0 @@
## Download skywalking agent release version
- Go to [release page](http://skywalking.apache.org/downloads/)
## Install skywalking javaagent
1. Copy the agent package to anywhere you like. The logs, plugins and config are all included in the package.
2. Add -javaagent:/path/to/skywalking-agent/skywalking-agent.jar to VM argument.
New agent package looks like this：
```
+-- skywalking-agent
+-- activations
apm-toolkit-log4j-1.x-activation.jar
apm-toolkit-log4j-2.x-activation.jar
apm-toolkit-logback-1.x-activation.jar
...
+-- config
agent.config
+-- plugins
apm-dubbo-plugin.jar
apm-feign-default-http-9.x.jar
apm-httpClient-4.x-plugin.jar
.....
skywalking-agent.jar
```
- Start your application.
# Advanced features
- All plugins are in `/plugins` folder. The plugin jar is active when it is in there. Remove the plugin jar, it disabled.
- The default logging output folder is `/logs`.
# Install javaagent FAQs
- Linux Tomcat 7, Tomcat 8
Change the first line of `tomcat/bin/catalina.sh`.
```shell
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/path/to/skywalking-agent/skywalking-agent.jar"; export CATALINA_OPTS
```
- Windows Tomcat 7, Tomcat 8
Change the first line of `tomcat/bin/catalina.bat`.
```shell
set "CATALINA_OPTS=-javaagent:/path/to/skywalking-agent/skywalking-agent.jar"
```
- JAR file
Add `-javaagent` argument to command line in which you start your app. And make sure to add it before the `-jar` argument. eg:
```shell
java -javaagent:/path/to/skywalking-agent/skywalking-agent.jar -jar yourApp.jar
```

13
docs/en/FAQ/README.md Normal file
View File

@ -0,0 +1,13 @@
# FAQs
These are known and common FAQs. We welcome you to contribute yours.
## Compiling
* [Protoc plugin fails in maven build](Protoc-Plugin-Fails-When-Build.md)
* [Required items could not be found, when import project into Eclipse](Import-Project-Eclipse-RequireItems-Exception.md)
## Runtime
* [Why only traces in UI?](Why-have-traces-no-others.md)
* [Too many GRPC logs in the console](Too-many-gRPC-logs.md)
* [The trace doesn't continue in kafka consumer side](kafka-plugin.md)
* [Agent or collector version upgrade](Upgrade.md)
* [EnhanceRequireObjectCache class cast exception](EnhanceRequireObjectCache-Cast-Exception.md)

13
docs/en/Incubating/Abstract.md
View File

@ -1,13 +0,0 @@
# Incubating Features
Incubating features are developed or contributed from SkyWalking community,
but it is still a prototype or preview. At the same time, they need or their contributors
want to be included in the primary repository.
The PMC and committer team will evaluate the feature, make sure it stays in optional,
has compatible with current version.
List some typical incubating features
1. New storage implementor in collector, such as: ElasticSearch HTTP, MySQL...
1. New module provided in collector.
1. New optional plugins in agent.
1. New features/services in collector, as the integration.

8
docs/en/Optional-plugins.md
View File

@ -1,8 +0,0 @@
# Optional plugins
Optional plugins could be provided by source codes or in `optional-plugins` folder under agent.
For using these plugins, you need to compile source codes by yourself, or copy the certain plugins to `/plugins`.

16
docs/en/concepts-and-designs/README.md Normal file
View File

@ -0,0 +1,16 @@
# Concepts and Designs
Concepts and Designs help you to learn and understand the SkyWalking and the landscape.
- What is SkyWalking?
- [Overview](overview.md). Provides a high-level description and introduction, including the problems it solves.
- [Project Goals](project-goals.md). Provides the goals, which SkyWalking is trying to focus and provide features about them.
- Probe
- [Introduction](probe-introduction.md). Lead readers to understand what the probe is, how many different probes existed and
why need them.
- [Service auto instrument agent](service-agent.md). Introduce what the auto instrument agents do and which languages does
SkyWalking already support.
- [Manual instrument SDK](manual-sdk.md). Introduce the role of the manual instrument SDKs in SkyWalking ecosystem.
- Observability Analysis Platform Backend
- [Overview](oap-overview.md). Provides a high level introduction about the OAP backend.
- [Observability Analysis Language](oal.md). Introduces the core languages, which is designed for aggregation behaviour definition.
- [Query in OAP](query-oap.md). Introduces how to query data from backend, whether a simple metric data or topology map, even trace or log.

135
docs/en/OAP/README.md → docs/en/concepts-and-designs/oal.md
View File

@ -1,38 +1,4 @@
# Observability Analysis Platform
OAP(Observability Analysis Platform) is a new concept, which starts in SkyWalking 6.x. OAP replaces the
old SkyWalking whole backend. The capabilities of the platform are following.
## OAP capabilities
<img src="https://skywalkingtest.github.io/page-resources/6_overview.png"/>
In SkyWalking 6 series, OAP accepts data from more sources, which belongs two groups: **Tracing** and **Metric**.
- **Tracing**. Including, SkyWalking native data formats. Zipkin v1,v2 data formats and Jaeger data formats.
- **Metric**. SkyWalking integrates with Service Mesh platforms, such as Istio, Envoy, Linkerd, to provide observability from data panel
or control panel. Also, SkyWalking native agents can run in metric mode, which highly improve the
performance.
At the same time by using any integration solution provided, such as SkyWalking log plugin or toolkits,
SkyWalking provides visualization integration for binding tracing and logging together by using the
trace id and span id.
As usual, all services provided by gRPC and HTTP protocol to make integration easier for unsupported ecosystem.
## Tracing in OAP
Tracing in OAP has two ways to process.
1. Traditional way in SkyWalking 5 series. Format tracing data in SkyWalking trace segment and span formats,
even for Zipkin data format. The AOP analysis the segments to get metrics, and push the metric data into
the streaming aggregation.
1. Consider tracing as some kinds of logging only. Just provide save and visualization capabilities for trace.
## Metric in OAP
Metric in OAP is totally new feature in 6 series. Build observability for a distributed system based on metric of connected nodes.
No tracing data is required.
Metric data are aggregated inside AOP cluster in streaming mode. See below about [Observability Analysis Language](#observability-analysis-language),
which provides the easy way to do aggregation and analysis in script style.
### Observability Analysis Language
# Observability Analysis Language
Provide OAL(Observability Analysis Language) to analysis incoming data in streaming mode.
OAL focuses on metric in Service, Service Instance and Endpoint. Because of that, the language is easy to
@ -41,7 +7,7 @@ learn and use.
Considering performance, reading and debugging, OAL is defined as a compile language.
The OAL scrips will be compiled to normal Java codes in package stage.
#### Grammar
## Grammar
Scripts should be named as `*.oal`
```
@ -50,10 +16,10 @@ METRIC_NAME = from(SCOPE.(* | [FIELD][,FIELD ...]))
.FUNCTION([PARAM][, PARAM ...])
```
#### Scope
## Scope
**SCOPE** in (`All`, `Service`, `ServiceInstance`, `Endpoint`, `ServiceRelation`, `ServiceInstanceRelation`, `EndpointRelation`).
#### Field
## Field
By using Aggregation Function, the requests will group by time and **Group Key(s)** in each scope.
- SCOPE `All`
@ -169,14 +135,14 @@ including auto instrument agents(like Java, .NET), OpenCensus SkyWalking exporte
| type | Represent the type of each request. Such as: Database, HTTP, RPC, gRPC. | | enum |
| detectPoint | Represent where is the relation detected. Values: client, server, proxy. | yes | enum|
#### Filter
## Filter
Use filter to build the conditions for the value of fields, by using field name and expression.
The expressions support to link by `and`, `or` and `(...)`.
The OPs support `=`, `!=`, `>`, `<`, `in (v1, v2, ...`, `like "%..."`, with type detection based of field type. Trigger compile
or code generation error if incompatible.
#### Aggregation Function
## Aggregation Function
The default functions are provided by SkyWalking OAP core, and could implement more.
Provided functions
@ -189,15 +155,15 @@ Provided functions
- `histogram(start, step)`. Group the given value by the given step, begin with the start value.
- `sum()`. The sum number of selected by filter. No type requirement.
#### Metric name
## Metric name
The metric name for storage implementor, alarm and query modules. The type inference supported by core.
#### Group
## Group
All metric data will be grouped by Scope.ID and min-level TimeBucket.
- In `Endpoint` scope, the Scope.ID = Endpoint id (the unique id based on service and its Endpoint)
#### Examples
## Examples
```
// Caculate p99 of both Endpoint1 and Endpoint2
Endpoint_p99 = from(Endpoint.latency).filter(name in ("Endpoint1", "Endpoint2")).summary(0.99)
@ -223,85 +189,4 @@ Endpoint_500 = from(Endpoint.*).filter(responseCode like "5%").percent()
// Caculate the sum of calls for each service.
EndpointCalls = from(Endpoint.*).sum()
```
## Query in OAP
Query is the core feature of OAP for visualization and other higher system. The query matches the metric type.
There are two types of query provided.
1. Hard codes query implementor
1. Metric style query of implementor
### Hard codes
Hard codes query implementor, is for complex logic query, such as: topology map, dependency map, which
most likely relate to mapping mechanism of the node relationship.
Even so, hard codes implementors are based on metric style query too, just need extra codes to assemble the
results.
### Metric style query
Metric style query is based on the given scope and metric name in oal scripts.
Metric style query provided in two ways
- GraphQL way. UI uses this directly, and assembles the pages.
- API way. Most for `Hard codes query implementor` to do extra works.
#### Grammar
```
Metric.Scope(SCOPE).Func(METRIC_NAME [, PARAM ...])
```
#### Scope
**SCOPE** in (`All`, `Service`, `ServiceInst`, `Endpoint`, `ServiceRelation`, `ServiceInstRelation`, `EndpointRelation`).
#### Metric name
Metric name is defined in oal script. Such as **EndpointCalls** is the name defined by `EndpointCalls = from(Endpoint.*).sum()`.
#### Metric Query Function
Metric Query Functions match the Aggregation Function in most cases, but include some order or filter features.
Try to keep the name as same as the aggregation functions.
Provided functions
- `top`
- `trend`
- `histogram`
- `sum`
#### Example
For `avg` aggregate func, `top` match it, also with parameter[1] of result size and parameter[2] of order
```
# for Service_avg = from(Service.latency).avg()
Metric.Scope("Service").topn("Service_avg", 10, "desc")
```
## Project structure overview
This overview shows maven modules AOP provided.
```
- SkyWalking Project
- apm-commons
- ...
- apm-oap
- oap-receiver
- receiver-skywalking
- receiver-zipkin
- ...
- oap-discovery
- discovery-naming
- discovery-zookeeper
- discovery-standalone
- ...
- oap-register
- register-skywalking
- ...
- oap-analysis
- analysis-trace
- analysis-metric
- analysis-log
- oap-web
- oap-libs
- cache-lib
- remote-lib
- storage-lib
- client-lib
- server-lib
```
```

33
docs/en/concepts-and-designs/oap-overview.md Normal file
View File

@ -0,0 +1,33 @@
# Observability Analysis Platform
OAP(Observability Analysis Platform) is a new concept, which starts in SkyWalking 6.x. OAP replaces the
old SkyWalking whole backend. The capabilities of the platform are following.
## OAP capabilities
<img src="https://skywalkingtest.github.io/page-resources/6_overview.png"/>
In SkyWalking 6 series, OAP accepts data from more sources, which belongs two groups: **Tracing** and **Metric**.
- **Tracing**. Including, SkyWalking native data formats. Zipkin v1,v2 data formats and Jaeger data formats.
- **Metric**. SkyWalking integrates with Service Mesh platforms, such as Istio, Envoy, Linkerd, to provide observability from data panel
or control panel. Also, SkyWalking native agents can run in metric mode, which highly improve the
performance.
At the same time by using any integration solution provided, such as SkyWalking log plugin or toolkits,
SkyWalking provides visualization integration for binding tracing and logging together by using the
trace id and span id.
As usual, all services provided by gRPC and HTTP protocol to make integration easier for unsupported ecosystem.
## Tracing in OAP
Tracing in OAP has two ways to process.
1. Traditional way in SkyWalking 5 series. Format tracing data in SkyWalking trace segment and span formats,
even for Zipkin data format. The AOP analysis the segments to get metrics, and push the metric data into
the streaming aggregation.
1. Consider tracing as some kinds of logging only. Just provide save and visualization capabilities for trace.
## Metric in OAP
Metric in OAP is totally new feature in 6 series. Build observability for a distributed system based on metric of connected nodes.
No tracing data is required.
Metric data are aggregated inside AOP cluster in streaming mode. See about [Observability Analysis Language](oal.md),
which provides the easy way to do aggregation and analysis in script style.

0
docs/en/concepts-and-designs/overview.md Normal file
View File

48
docs/en/concepts-and-designs/query-oap.md Normal file
View File

@ -0,0 +1,48 @@
# Query in OAP
Query is the core feature of OAP for visualization and other higher system. The query matches the metric type.
There are two types of query provided.
1. Hard codes query implementor
1. Metric style query of implementor
## Hard codes
Hard codes query implementor, is for complex logic query, such as: topology map, dependency map, which
most likely relate to mapping mechanism of the node relationship.
Even so, hard codes implementors are based on metric style query too, just need extra codes to assemble the
results.
## Metric style query
Metric style query is based on the given scope and metric name in oal scripts.
Metric style query provided in two ways
- GraphQL way. UI uses this directly, and assembles the pages.
- API way. Most for `Hard codes query implementor` to do extra works.
### Grammar
```
Metric.Scope(SCOPE).Func(METRIC_NAME [, PARAM ...])
```
### Scope
**SCOPE** in (`All`, `Service`, `ServiceInst`, `Endpoint`, `ServiceRelation`, `ServiceInstRelation`, `EndpointRelation`).
### Metric name
Metric name is defined in oal script. Such as **EndpointCalls** is the name defined by `EndpointCalls = from(Endpoint.*).sum()`.
### Metric Query Function
Metric Query Functions match the Aggregation Function in most cases, but include some order or filter features.
Try to keep the name as same as the aggregation functions.
Provided functions
- `top`
- `trend`
- `histogram`
- `sum`
### Example
For `avg` aggregate func, `top` match it, also with parameter[1] of result size and parameter[2] of order
```
# for Service_avg = from(Service.latency).avg()
Metric.Scope("Service").topn("Service_avg", 10, "desc")
```

0
docs/en/How-to-release.md → docs/en/guides/How-to-release.md
View File

8
docs/en/guides/README.md Normal file
View File

@ -0,0 +1,8 @@
# Guides
Guides help everyone developer, including PPMC member, committer and contributor, to understand the project structure.
Also learn to build the project, even to release the official Apache version(If you have been accepted as the formal committer).
- [Compiling Guide](../How-to-build.md). Teaches developer how to build the project in local.
- [Apache Release Guide](How-to-release.md). Apache license allows everyone to redistribute if you keep our licenses and NOTICE
in your redistribution. This document introduces to the committer team about doing official Apache version release, to avoid
breaking any Apache rule.

4
docs/en/protocols/README.md Normal file
View File

@ -0,0 +1,4 @@
# Protocols
* [Cross Process Propagation Headers Protocol, v1.0](Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md)
* [SkyWalking Trace Data Protocol](Trace-Data-Protocol.md)

0
docs/en/Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md → docs/en/protocols/Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md
View File

4
docs/en/Trace-Data-Protocol.md → docs/en/protocols/Trace-Data-Protocol.md
View File

@ -79,14 +79,14 @@ Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every s
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/TraceSegmentService.proto)
- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
- Span data please refs to [Plugin Development Guide](Plugin-Development-Guide.md)
- Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md)
- Id and name both exist, please use id if possible.
- operationNameId/operationName
- networkAddress/networkAddressId
- entryServiceName/entryServiceId
- parentServiceName/parentServiceId
- peerId/peer
- componentIds are defined in backend, [here](../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)
- componentIds are defined in backend, [here](../../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)
# HTTP JSON Services
All HTTP Services match the gRPC services, just adjust use short keys.

12
docs/en/setup/README.md Normal file
View File

@ -0,0 +1,12 @@
# Setup
Setup based on which kind of probes are you going to use. If you don't understand, please read [Concepts and Designs](../concepts-and-designs/README.md) first.
- Language agents in Service
- Install agents
- [Java agent](service-agent/java-agent/README.md). Introduce how to install java agent to your application, without change any codes.
- [.NET Core agent](https://github.com/OpenSkywalking/skywalking-netcore). See .NET Core agent project document for more details.
- [Node.js agent](https://github.com/OpenSkywalking/skywalking-nodejs). See Node.js server side agent project document for more details.
- [Setup backend](service-agent/backend-setup.md). Set the backend for language agents scenario.
- On Service Mesh
- Istio
- [SkyWalking on Istio](istio/README.md). Introduce how to use Istio Mixer SkyWalking Adapter to work with SkyWalking.

0
docs/en/setup/service-agent/backend-setup.md Normal file
View File

0
docs/en/Application-toolkit-log4j-1.x.md → docs/en/setup/service-agent/java-agent/Application-toolkit-log4j-1.x.md
View File

0
docs/en/Application-toolkit-log4j-2.x.md → docs/en/setup/service-agent/java-agent/Application-toolkit-log4j-2.x.md
View File

0
docs/en/Application-toolkit-logback-1.x.md → docs/en/setup/service-agent/java-agent/Application-toolkit-logback-1.x.md
View File

0
docs/en/Application-toolkit-trace-cross-thread.md → docs/en/setup/service-agent/java-agent/Application-toolkit-trace-cross-thread.md
View File

0
docs/en/Application-toolkit-trace.md → docs/en/setup/service-agent/java-agent/Application-toolkit-trace.md
View File

0
docs/en/Applicaton-toolkit.md → docs/en/setup/service-agent/java-agent/Applicaton-toolkit.md
View File

0
docs/en/Direct-uplink.md → docs/en/setup/service-agent/java-agent/Direct-uplink.md
View File

0
docs/en/How-to-disable-plugin.md → docs/en/setup/service-agent/java-agent/How-to-disable-plugin.md
View File

2
docs/en/Namespace.md → docs/en/setup/service-agent/java-agent/Namespace.md
View File

@ -20,7 +20,7 @@ Namespace is proposal from this.It is used for tracing and monitoring isolation.
The default value of `agent.namespace` is empty.
**Influence**
The default header key of SkyWalking is `sw3`, more in this [document](Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md).
The default header key of SkyWalking is `sw3`, more in this [document](../../../protocols/Skywalking-Cross-Process-Propagation-Headers-Protocol-v1.md).
After `agent.namespace` set, the key changes to `namespace:sw3`.
The across process propagation chain breaks, when the two sides are using different namespace.

0
docs/en/Opentracing.md → docs/en/setup/service-agent/java-agent/Opentracing.md
View File

0
docs/en/Plugin-Development-Guide.md → docs/en/setup/service-agent/java-agent/Plugin-Development-Guide.md
View File

0
docs/en/Quick-start.md → docs/en/setup/service-agent/java-agent/Quick-start.md
View File

77
docs/en/setup/service-agent/java-agent/README.md Normal file
View File

@ -0,0 +1,77 @@
# Setup java agent
1. Find `agent` folder in SkyWalking
2. Add `-javaagent:/path/to/skywalking-agent/skywalking-agent.jar` to VM argument.
New agent package looks like this：
```
+-- skywalking-agent
+-- activations
apm-toolkit-log4j-1.x-activation.jar
apm-toolkit-log4j-2.x-activation.jar
apm-toolkit-logback-1.x-activation.jar
...
+-- config
agent.config
+-- plugins
apm-dubbo-plugin.jar
apm-feign-default-http-9.x.jar
apm-httpClient-4.x-plugin.jar
.....
skywalking-agent.jar
```
- Start your application.
## Advanced features
- All plugins are in `/plugins` folder. The plugin jar is active when it is in there. Remove the plugin jar, it disabled.
- The default logging output folder is `/logs`.
## Install javaagent FAQs
- Linux Tomcat 7, Tomcat 8
Change the first line of `tomcat/bin/catalina.sh`.
```shell
CATALINA_OPTS="$CATALINA_OPTS -javaagent:/path/to/skywalking-agent/skywalking-agent.jar"; export CATALINA_OPTS
```
- Windows Tomcat 7, Tomcat 8
Change the first line of `tomcat/bin/catalina.bat`.
```shell
set "CATALINA_OPTS=-javaagent:/path/to/skywalking-agent/skywalking-agent.jar"
```
- JAR file
Add `-javaagent` argument to command line in which you start your app. And make sure to add it before the `-jar` argument. eg:
```shell
java -javaagent:/path/to/skywalking-agent/skywalking-agent.jar -jar yourApp.jar
```
## Supported middlewares, frameworks and libraries
See [supported list](Supported-list.md).
## Plugins
Java agent plugins are all pluggable. Optional plugins could be provided by source codes or in `optional-plugins` folder under agent.
For using these plugins, you need to compile source codes by yourself, or copy the certain plugins to `/plugins`.
Now, we have the following known plugins.
* [Trace Spring beans](agent-optional-plugins/Spring-bean-plugins.md)
* [Trace Oracle and Resin](agent-optional-plugins/Oracle-Resin-plugins.md)
* [Filter traces through custom services](agent-optional-plugins/trace-ignore-plugin.md)
## Advanced Features
* [Override settings through System.properties](en/java-agent/Setting-override.md)
* [Direct uplink and disable naming discovery](en/java-agent/Direct-uplink.md)
* [Open TLS](en/java-agent/TLS.md)
* [Namespace Isolation](en/java-agent/Namespace.md)
* [Token Authentication](en/java-agent/Token-auth.md)
* Application Toolkit
* [Overview](en/java-agent/Applicaton-toolkit.md)
* [Use SkyWalking OpenTracing compatible tracer](en/java-agent/Opentracing.md)
* Integration with log frameworks
* [log4j](en/java-agent/Application-toolkit-log4j-1.x.md)
* [log4j2](en/java-agent/Application-toolkit-log4j-2.x.md)
* [logback](en/java-agent/Application-toolkit-logback-1.x.md)
* [Trace by SkyWalking manual APIs](en/java-agent/Application-toolkit-trace.md)
* [Trace across threads](en/java-agent/Application-toolkit-trace-cross-thread.md)
# Test
If you are interested in plugin compatible tests or agent performance, see the following reports.
* [Plugin Test](https://github.com/SkywalkingTest/agent-integration-test-report)
* [Java Agent Performance Test](https://skywalkingtest.github.io/Agent-Benchmarks/)

0
docs/en/Setting-override.md → docs/en/setup/service-agent/java-agent/Setting-override.md
View File

0
docs/Supported-list.md → docs/en/setup/service-agent/java-agent/Supported-list.md
View File

2
docs/en/TLS.md → docs/en/setup/service-agent/java-agent/TLS.md
View File

@ -18,7 +18,7 @@ So we didn't support TLS in naming service of HTTP service.
## Authentication Mode
Only support **no mutual auth**.
- Use this [script](../../tools/TLS/tls_key_generate.sh) if you are not familiar with how to generate key files.
- Use this [script](../../../tools/TLS/tls_key_generate.sh) if you are not familiar with how to generate key files.
- Find `ca.crt`, and use it at client side
- Find `server.crt` and `server.pem`. Use them at server side.

0
docs/en/Token-auth.md → docs/en/setup/service-agent/java-agent/Token-auth.md
View File

0
docs/en/Use-ShardingJDBC-as-storage-implementor.md → docs/en/setup/service-agent/java-agent/Use-ShardingJDBC-as-storage-implementor.md
View File

0
docs/en/agent-optional-plugins/Oracle-Resin-plugins.md → docs/en/setup/service-agent/java-agent/agent-optional-plugins/Oracle-Resin-plugins.md
View File

0
docs/en/agent-optional-plugins/Spring-bean-plugins.md → docs/en/setup/service-agent/java-agent/agent-optional-plugins/Spring-bean-plugins.md
View File

17
docs/en/setup/service-agent/java-agent/agent-optional-plugins/trace-ignore-plugin.md Normal file
View File

@ -0,0 +1,17 @@
## Support custom trace ignore
Here is an optional plugin `apm-trace-ignore-plugin`
## Introduce
- The purpose of this plugin is to filter custom services which are expected to ignore from the tracing system.
- You can set up multiple URL path patterns, The services matches these patterns means won't be traced and analysis by agent and collector.
- The current matching rule follows `Ant Path` match style , like `/path/*`, `/path/**`, `/path/?`.
- Copy `apm-trace-ignore-plugin-x.jar` to `agent/plugins`, restarting the `agent` can effect the plugin.
## How to configure
There are two ways to configure ignore patterns. Settings through system env has higher priority.
1. Set through the system environment variable,you need to add `skywalking.trace.ignore_path` to the system variables, the value is the path that you need to ignore, multiple paths should be separated by `,`
2. Copy`/agent/optional-plugins/apm-trace-ignore-plugin/apm-trace-ignore-plugin.config` to `/agent/config/` dir, and add rules to filter traces
```
trace.ignore_path=/your/path/1/**,/your/path/2/**
```