kazusa
/
skywalking
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Support RocketMQ monitoring (#11758) 

* Support RocketMQ monitoring
* Add swip3 - RocketMQ monitoring
* Add RocketMQ e2e cases

---------

Co-authored-by: peachisai <497533062@qq.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: 邵一鸣 <shaoym02@vanke.com>
Co-authored-by: Wan Kai <wankai123@foxmail.com>
This commit is contained in:
peachisai 2024-02-18 07:17:20 +08:00 committed by GitHub
parent b775e7f547
commit 48c766ee01
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
30 changed files with 2031 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/skywalking.yaml vendored
View File

@ -650,6 +650,8 @@ jobs:
config: test/e2e-v2/cases/mqe/e2e.yaml config: test/e2e-v2/cases/mqe/e2e.yaml
- name: Pulsar and BookKeeper - name: Pulsar and BookKeeper
config: test/e2e-v2/cases/pulsar/e2e.yaml config: test/e2e-v2/cases/pulsar/e2e.yaml
- name: RocketMQ
config: test/e2e-v2/cases/rocketmq/e2e.yaml
- name: UI Menu BanyanDB - name: UI Menu BanyanDB
config: test/e2e-v2/cases/menu/banyandb/e2e.yaml config: test/e2e-v2/cases/menu/banyandb/e2e.yaml

4
docs/en/changes/changes.md
View File

@ -46,6 +46,10 @@
- APISIX -> K8S_SERVICE - APISIX -> K8S_SERVICE
- GENERAL -> APISIX - GENERAL -> APISIX
* Add Golang as a supported language for RocketMQ. * Add Golang as a supported language for RocketMQ.
* Support Apache RocketMQ server monitoring.
* Add Service Hierarchy auto matching layer relationships (upper -> lower) as following:
- ROCKETMQ -> K8S_SERVICE
- VIRTUAL_MQ -> ROCKETMQ
#### UI #### UI

46
docs/en/concepts-and-designs/service-hierarchy.md
View File

@ -11,20 +11,22 @@ There 2 ways to detect the connections:
If you want to customize it according to your own needs, please refer to [Service Hierarchy Configuration](service-hierarchy-configuration.md). If you want to customize it according to your own needs, please refer to [Service Hierarchy Configuration](service-hierarchy-configuration.md).
### Automatically Matching ### Automatically Matching
| Upper layer | Lower layer | Matching rule | | Upper layer | Lower layer | Matching rule |
|-------------------|-------------|-------------------------------------------------------------------| |------------------|-------------|-------------------------------------------------------------------|
| MESH | MESH_DP | [MESH On MESH_DP](#mesh-on-mesh_dp) | | MESH | MESH_DP | [MESH On MESH_DP](#mesh-on-mesh_dp) |
| MESH | K8S_SERVICE | [MESH On K8S_SERVICE](#mesh-on-k8s_service) | | MESH | K8S_SERVICE | [MESH On K8S_SERVICE](#mesh-on-k8s_service) |
| MESH_DP | K8S_SERVICE | [MESH_DP On K8S_SERVICE](#mesh_dp-on-k8s_service) | | MESH_DP | K8S_SERVICE | [MESH_DP On K8S_SERVICE](#mesh_dp-on-k8s_service) |
| GENERAL | K8S_SERVICE | [GENERAL On K8S_SERVICE](#general-on-k8s_service) | | GENERAL | K8S_SERVICE | [GENERAL On K8S_SERVICE](#general-on-k8s_service) |
| GENERAL | APISIX | [GENERAL On APISIX](#general-on-apisix) | | GENERAL | APISIX | [GENERAL On APISIX](#general-on-apisix) |
| MYSQL | K8S_SERVICE | [MYSQL On K8S_SERVICE](#mysql-on-k8s_service) | | MYSQL | K8S_SERVICE | [MYSQL On K8S_SERVICE](#mysql-on-k8s_service) |
| POSTGRESQL | K8S_SERVICE | [POSTGRESQL On K8S_SERVICE](#postgresql-on-k8s_service) | | POSTGRESQL | K8S_SERVICE | [POSTGRESQL On K8S_SERVICE](#postgresql-on-k8s_service) |
| SO11Y_OAP | K8S_SERVICE | [SO11Y_OAP On K8S_SERVICE](#so11y_oap-on-k8s_service) | | SO11Y_OAP | K8S_SERVICE | [SO11Y_OAP On K8S_SERVICE](#so11y_oap-on-k8s_service) |
| VIRTUAL_DATABASE | MYSQL | [VIRTUAL_DATABASE On MYSQL](#virtual_database-on-mysql) | | VIRTUAL_DATABASE | MYSQL | [VIRTUAL_DATABASE On MYSQL](#virtual_database-on-mysql) |
| VIRTUAL_DATABASE | POSTGRESQL | [VIRTUAL_DATABASE On POSTGRESQL](#virtual_database-on-postgresql) | | VIRTUAL_DATABASE | POSTGRESQL | [VIRTUAL_DATABASE On POSTGRESQL](#virtual_database-on-postgresql) |
| NGINX | K8S_SERVICE | [NGINX On K8S_SERVICE](#nginx-on-k8s_service) | | NGINX | K8S_SERVICE | [NGINX On K8S_SERVICE](#nginx-on-k8s_service) |
| APISIX | K8S_SERVICE | [APISIX On K8S_SERVICE](#apisix-on-k8s_service) | | APISIX | K8S_SERVICE | [APISIX On K8S_SERVICE](#apisix-on-k8s_service) |
| ROCKETMQ | K8S_SERVICE | [ROCKETMQ On K8S_SERVICE](#rocketmq-on-k8s_service) |
| VIRTUAL_MQ | ROCKETMQ | [VIRTUAL_MQ On K8S_SERVICE](#virtual_mq-on-rocketmq) |
- The following sections will describe the **default matching rules** in detail and use the `upper-layer On lower-layer` format. - The following sections will describe the **default matching rules** in detail and use the `upper-layer On lower-layer` format.
- The example service name are based on SkyWalking [Showcase](https://github.com/apache/skywalking-showcase) default deployment. - The example service name are based on SkyWalking [Showcase](https://github.com/apache/skywalking-showcase) default deployment.
@ -126,6 +128,22 @@ If you want to customize it according to your own needs, please refer to [Servic
- APISIX.service.name: `APISIX::frontend.sample-services` - APISIX.service.name: `APISIX::frontend.sample-services`
- K8S_SERVICE.service.name: `skywalking-showcase::frontend.sample-services` - K8S_SERVICE.service.name: `skywalking-showcase::frontend.sample-services`
#### ROCKETMQ On K8S_SERVICE
- Rule name: `short-name`
- Groovy script: `{ (u, l) -> u.shortName == l.shortName }`
- Description: ROCKETMQ.service.shortName == K8S_SERVICE.service.shortName
- Matched Example:
- ROCKETMQ.service.name: `rocketmq::rocketmq.skywalking-showcase`
- K8S_SERVICE.service.name: `skywalking-showcase::rocketmq.skywalking-showcase`
#### VIRTUAL_MQ On ROCKETMQ
- Rule name: `lower-short-name-with-fqdn`
- Groovy script: `{ (u, l) -> u.shortName.substring(0, u.shortName.lastIndexOf(':')) == l.shortName.concat('.svc.cluster.local') }`
- Description: VIRTUAL_MQ.service.shortName remove port == ROCKETMQ.service.shortName with fqdn suffix
- Matched Example:
- VIRTUAL_MQ.service.name: `rocketmq.skywalking-showcase.svc.cluster.local:9876`
- ROCKETMQ.service.name: `rocketmq::rocketmq.skywalking-showcase`
### Build Through Specific Agents ### Build Through Specific Agents
Use agent tech involved(such as eBPF) and deployment tools(such as operator and agent injector) detect the service hierarchy relations. Use agent tech involved(such as eBPF) and deployment tools(such as operator and agent injector) detect the service hierarchy relations.

73
docs/en/setup/backend/backend-rocketmq-monitoring.md Normal file
View File

@ -0,0 +1,73 @@
# RocketMQ monitoring
SkyWalking leverages rocketmq-exporter for collecting metrics data from RocketMQ. It leverages OpenTelemetry
Collector to transfer the metrics to
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
## Data flow
1. The `rocketmq-exporter` (https://github.com/apache/rocketmq-exporter?tab=readme-ov-file#readme) collects metrics data from RocketMQ, The RocketMQ version is required to be 4.3.2+.
2. OpenTelemetry Collector fetches metrics from rocketmq-exporter via Prometheus Receiver and pushes metrics to
SkyWalking OAP Server via OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to
filter/calculate/aggregate and store the results.
## Setup
1. Setup [rocketmq-exporter](https://github.com/apache/rocketmq-exporter).
2. Set up [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/#docker). The example for OpenTelemetry Collector configuration, refer
to [here](../../../../test/e2e-v2/cases/rocketmq/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
## RocketMQ Monitoring
RocketMQ monitoring provides multidimensional metrics monitoring of RocketMQ Exporter as `Layer: RocketMQ` `Service` in
the OAP. In each cluster, the broker is represented as `Instance` and the topic is represented as `Endpoint`.
### RocketMQ Cluster Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|--------------------------------------------------------------------------|------------------|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
### RocketMQ Broker Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|----------------------------------------------------|-------------------|
| Produce TPS | Msg/sec | meter_rocketmq_broker_produce_tps | The number of broker produces messages per second. | RocketMQ Exporter |
| Consume QPS | Msg/sec | meter_rocketmq_broker_consume_qps | The number of broker consumes messages per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_broker_producer_message_size | The max size of the messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_broker_consumer_message_size | The max size of the messages consumed per second. | RocketMQ Exporter |
### RocketMQ Topic Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
## Customizations
You can customize your own metrics/expression/dashboard panel.
The metrics definition and expression rules are found
in `otel-rules/rocketmq/rocketmq-cluster.yaml, otel-rules/rocketmq/rocketmq-broker.yaml, otel-rules/rocketmq/rocketmq-topic.yaml`.
The RocketMQ dashboard panel configurations are found in `ui-initialized-templates/rocketmq`.

56
docs/en/swip/SWIP-3.md
View File

@ -11,21 +11,23 @@ Provide cluster, broker, and topic dimensions monitoring.
### RocketMQ Cluster Supported Metrics ### RocketMQ Cluster Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source | | Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|--------------------------------------------------------------------------|------------------| |--------------------------------------|------------|-------------------------------------------------------------|---------------------------------------------------------------------------|------------------|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter | | Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter | | Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter | | Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter | | Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter | | Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter | | Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter | | Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter | | Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter | | Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter | | Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter | | CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter | | Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter | | Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
| Topic Count | Count | meter_rocketmq_cluster_topic_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_cluster_broker_count | The number of brokers that received messages from the producer. | RocketMQ Exporter |
### RocketMQ Broker Supported Metrics ### RocketMQ Broker Supported Metrics
@ -38,17 +40,19 @@ Provide cluster, broker, and topic dimensions monitoring.
### RocketMQ Topic Supported Metrics ### RocketMQ Topic Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source | | Monitoring Panel | Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------| |---------------------------|-----------|------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter | | Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter | | Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter | | Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter | | Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter | | Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter | | Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter | | Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter | | Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter | | Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
| Consumer Group_Count | Count | meter_rocketmq_topic_consumer_group_count | The number of consumer groups. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_topic_broker_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
## Imported Dependencies libs and their licenses. ## Imported Dependencies libs and their licenses.
No new dependency. No new dependency.

2
docs/menu.yml
View File

@ -260,6 +260,8 @@ catalog:
path: "/en/setup/backend/backend-kafka-monitoring" path: "/en/setup/backend/backend-kafka-monitoring"
- name: "Pulsar" - name: "Pulsar"
path: "/en/setup/backend/backend-pulsar-monitoring" path: "/en/setup/backend/backend-pulsar-monitoring"
- name: "RocketMQ"
path: "/en/setup/backend/backend-rocketmq-monitoring"
- name: "Self Observability" - name: "Self Observability"
catalog: catalog:
- name: "OAP self telemetry" - name: "OAP self telemetry"

7
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java
View File

@ -213,7 +213,12 @@ public enum Layer {
/** /**
* Nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server. * Nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server.
*/ */
NGINX(34, true); NGINX(34, true),
/**
* A cloud native messaging and streaming platform, making it simple to build event-driven applications.
*/
ROCKETMQ(35, true);
private final int value; private final int value;
/** /**

1
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
View File

@ -72,6 +72,7 @@ public class UITemplateInitializer {
Layer.PULSAR.name(), Layer.PULSAR.name(),
Layer.BOOKKEEPER.name(), Layer.BOOKKEEPER.name(),
Layer.NGINX.name(), Layer.NGINX.name(),
Layer.ROCKETMQ.name(),
"custom" "custom"
}; };
private final UITemplateManagementService uiTemplateManagementService; private final UITemplateManagementService uiTemplateManagementService;

2
oap-server/server-starter/src/main/resources/application.yml
View File

@ -353,7 +353,7 @@ receiver-otel:
selector: ${SW_OTEL_RECEIVER:default} selector: ${SW_OTEL_RECEIVER:default}
default: default:
enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"otlp-metrics,otlp-logs"} enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"otlp-metrics,otlp-logs"}
enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*"} enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*,rocketmq/*"}
receiver-zipkin: receiver-zipkin:
selector: ${SW_RECEIVER_ZIPKIN:-} selector: ${SW_RECEIVER_ZIPKIN:-}

8
oap-server/server-starter/src/main/resources/hierarchy-definition.yml
View File

@ -45,10 +45,16 @@ hierarchy:
SO11Y_OAP: SO11Y_OAP:
K8S_SERVICE: short-name K8S_SERVICE: short-name
ROCKETMQ:
K8S_SERVICE: short-name
VIRTUAL_DATABASE: VIRTUAL_DATABASE:
MYSQL: lower-short-name-with-fqdn MYSQL: lower-short-name-with-fqdn
POSTGRESQL: lower-short-name-with-fqdn POSTGRESQL: lower-short-name-with-fqdn
VIRTUAL_MQ:
ROCKETMQ: lower-short-name-with-fqdn
# Use Groovy script to define the matching rules, the input parameters are the upper service(u) and the lower service(l) and the return value is a boolean, # Use Groovy script to define the matching rules, the input parameters are the upper service(u) and the lower service(l) and the return value is a boolean,
# which are used to match the relation between the upper service(u) and the lower service(l) on the different layers. # which are used to match the relation between the upper service(u) and the lower service(l) on the different layers.
auto-matching-rules: auto-matching-rules:
@ -70,11 +76,13 @@ layer-levels:
GENERAL: 3 GENERAL: 3
SO11Y_OAP: 3 SO11Y_OAP: 3
VIRTUAL_DATABASE: 3 VIRTUAL_DATABASE: 3
VIRTUAL_MQ: 3
MYSQL: 2 MYSQL: 2
POSTGRESQL: 2 POSTGRESQL: 2
APISIX: 2 APISIX: 2
NGINX: 2 NGINX: 2
ROCKETMQ: 2
MESH_DP: 1 MESH_DP: 1

46
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-broker.yaml Normal file
View File

@ -0,0 +1,46 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).instance(['cluster'], ['broker'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_broker
metricsRules:
- name: produce_tps
exp: rocketmq_broker_tps.sum(['cluster', 'broker'])
- name: consume_qps
exp: rocketmq_broker_qps.sum(['cluster','broker'])
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster','broker']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster','broker']).downsampling(MAX)

81
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-cluster.yaml Normal file
View File

@ -0,0 +1,81 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).service(['cluster'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_cluster
metricsRules:
- name: messages_produced_today
exp: rocketmq_brokeruntime_msg_put_total_today_now.sum((['cluster']))-rocketmq_brokeruntime_msg_puttotal_yesterdaymorning.sum(['cluster'])
- name: messages_consumed_today
exp: rocketmq_brokeruntime_msg_gettotal_today_now.sum(['cluster'])-rocketmq_brokeruntime_msg_gettotal_yesterdaymorning.sum(['cluster'])
- name: total_producer_tps
exp: rocketmq_producer_tps.sum(['cluster'])
- name: total_consumer_tps
exp: rocketmq_consumer_tps.sum(['cluster'])
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster']).downsampling(MAX)
- name: messages_produced_until_yesterday
exp: rocketmq_brokeruntime_msg_puttotal_yesterdaymorning.sum(['cluster'])
- name: messages_consumed_until_yesterday
exp: rocketmq_brokeruntime_msg_gettotal_yesterdaymorning.sum(['cluster'])
- name: max_consumer_latency
exp: rocketmq_group_get_latency_by_storetime.max(['cluster','broker','topic','group'])
- name: max_commitLog_disk_ratio
exp: rocketmq_brokeruntime_commitlog_disk_ratio.max(['cluster','brokerIP'])*100
- name: commitLog_disk_ratio
exp: rocketmq_brokeruntime_commitlog_disk_ratio.sum(['cluster','brokerIP'])*100
- name: pull_threadPool_queue_head_wait_time
exp: rocketmq_brokeruntime_pull_threadpoolqueue_headwait_timemills.sum(['cluster','brokerIP'])
- name: send_threadPool_queue_head_wait_time
exp: rocketmq_brokeruntime_send_threadpoolqueue_headwait_timemills.sum(['cluster','brokerIP'])
- name: topic_count
exp: rocketmq_producer_offset.count(['cluster','topic'])
- name: broker_count
exp: rocketmq_producer_offset.count(['cluster','broker'])

70
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-topic.yaml Normal file
View File

@ -0,0 +1,70 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).endpoint(['cluster'], ['topic'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_topic
metricsRules:
- name: max_producer_message_size
exp: rocketmq_producer_message_size.max(['cluster','topic'])
- name: max_consumer_message_size
exp: rocketmq_consumer_message_size.max(['cluster','topic','group'])
- name: consumer_latency
exp: rocketmq_group_get_latency_by_storetime.sum(['cluster','topic','group'])
- name: producer_tps
exp: rocketmq_producer_tps.sum(['cluster','topic'])
- name: consumer_group_tps
exp: rocketmq_consumer_tps.sum(['cluster','topic'])
- name: producer_offset
exp: rocketmq_producer_offset.sum(['cluster','topic']).downsampling(MAX)
- name: consumer_group_offset
exp: rocketmq_consumer_offset.sum(['cluster','topic','group']).downsampling(MAX)
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster','topic']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster','topic','group']).downsampling(MAX)
- name: consumer_group_count
exp: rocketmq_consumer_offset.count(['cluster','topic','group'])
- name: broker_count
exp: rocketmq_producer_offset.count(['cluster','topic','broker'])

5
oap-server/server-starter/src/main/resources/ui-initialized-templates/menu.yaml
View File

@ -208,6 +208,11 @@ menus:
description: Provide pulsar monitoring through OpenTelemetry's Prometheus Receiver. description: Provide pulsar monitoring through OpenTelemetry's Prometheus Receiver.
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-pulsar-monitoring/ documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-pulsar-monitoring/
i18nKey: mq_pulsar i18nKey: mq_pulsar
- title: RocketMQ
layer: ROCKETMQ
description: Provide RocketMQ monitoring through OpenTelemetry's Prometheus Receiver.
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-rocketmq-monitoring/
i18nKey: mq_rocketmq
- title: Self Observability - title: Self Observability
icon: self_observability icon: self_observability
description: Self Observability provides the observabilities for running components and servers from the SkyWalking ecosystem. description: Self Observability provides the observabilities for running components and servers from the SkyWalking ecosystem.

142
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-broker.json Normal file
View File

@ -0,0 +1,142 @@
[
{
"id": "RocketMQ-Broker",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 12,
"h": 14,
"i": "8",
"type": "Widget",
"id": "8",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Produce Tps",
"tips": "The number of broker produces messages per second",
"name": "produce qps"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"meter_rocketmq_broker_produce_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
]
},
{
"x": 12,
"y": 0,
"w": 12,
"h": 14,
"i": "9",
"type": "Widget",
"id": "9",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_consume_qps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Consume Qps",
"tips": "The number of broker consumes messages per second",
"name": "consume qps"
}
},
{
"x": 0,
"y": 14,
"w": 12,
"h": 13,
"i": "13",
"type": "Widget",
"id": "13",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "producer_message_size_per_second",
"title": "Producer Message Size(MB)",
"tips": "The size of the messages produced per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 14,
"w": 12,
"h": 13,
"i": "14",
"type": "Widget",
"id": "14",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer message size",
"title": "Consumer Message Size(MB)",
"tips": "The size of the messages consumed per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
}
],
"layer": "ROCKETMQ",
"entity": "ServiceInstance",
"name": "RocketMQ-Broker",
"id": "RocketMQ-Broker"
}
}
]

569
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-cluster.json Normal file
View File

@ -0,0 +1,569 @@
[
{
"id": "RocketMQ-Cluster",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 24,
"h": 48,
"i": "0",
"type": "Tab",
"children": [
{
"name": "Overview",
"children": [
{
"x": 12,
"y": 0,
"w": 6,
"h": 6,
"i": "8",
"type": "Widget",
"id": "0-0-8",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"title": "Messages Produced Today",
"tips": "The number of the cluster messages produced today",
"name": "messages produced today"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"latest(meter_rocketmq_cluster_messages_produced_today)"
],
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 6,
"i": "9",
"type": "Widget",
"id": "0-0-9",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"title": "Messages Consumed Today",
"tips": "The number of the cluster messages consumed today",
"name": "messages consumed today"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"latest(meter_rocketmq_cluster_messages_consumed_today)"
],
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 0,
"y": 20,
"w": 12,
"h": 14,
"i": "10",
"type": "Widget",
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Total Producer Tps",
"tips": "The number of messages produced per second",
"name": "total producer TPS"
},
"expressions": [
"meter_rocketmq_cluster_total_producer_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"metricMode": "Expression",
"id": "0-0-10",
"moved": false,
"metrics": [],
"metricTypes": [],
"metricConfig": []
},
{
"x": 12,
"y": 20,
"w": 12,
"h": 14,
"i": "11",
"type": "Widget",
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Total Consume Tps",
"tips": "The number of messages consumed per second",
"name": "total consumer TPS"
},
"expressions": [
"meter_rocketmq_cluster_total_consumer_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"metricMode": "Expression",
"id": "0-0-11",
"moved": false,
"metrics": [],
"metricTypes": [],
"metricConfig": []
},
{
"x": 0,
"y": 48,
"w": 12,
"h": 14,
"i": "12",
"type": "Widget",
"id": "0-0-12",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Bar",
"showBackground": true
},
"widget": {
"name": "producer message size",
"title": "Producer Message Size(MB)",
"tips": "The size of a message produced per second"
}
},
{
"x": 12,
"y": 48,
"w": 12,
"h": 14,
"i": "13",
"type": "Widget",
"id": "0-0-13",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Bar",
"showBackground": true
},
"widget": {
"name": "consumer message size",
"title": "Consumer Message Size(MB)",
"tips": "The size of the consumed message per second"
}
},
{
"x": 0,
"y": 0,
"w": 6,
"h": 6,
"i": "16",
"type": "Widget",
"id": "0-0-16",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_cluster_messages_produced_until_yesterday)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "messages produced until yesterday",
"title": "Messages Produced Until Yesterday",
"tips": "The total number of messages put until 12 o'clock last night"
}
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 6,
"i": "17",
"type": "Widget",
"id": "0-0-17",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_cluster_messages_consumed_until_yesterday)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "messages consumed until yesterday",
"title": "Messages Consumed Until Yesterday",
"tips": "The total number of messages read until 12 o'clock last night"
}
},
{
"x": 6,
"y": 6,
"w": 6,
"h": 7,
"i": "18",
"type": "Widget",
"id": "0-0-18",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "ms"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_max_consumer_latency)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"name": "max consumer latency",
"title": "Max Consumer Latency",
"tips": "The max number of consumer latency"
}
},
{
"x": 0,
"y": 6,
"w": 6,
"h": 7,
"i": "19",
"type": "Widget",
"id": "0-0-19",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "%"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_max_commitLog_disk_ratio)"
],
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"name": "max commitLog disk ratio",
"title": "Max CommitLog Disk Ratio",
"tips": "The max utilization ratio of the commit log disk"
},
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 12,
"y": 6,
"w": 12,
"h": 14,
"i": "21",
"type": "Widget",
"id": "0-0-21",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Area",
"opacity": 0.4,
"showXAxis": true,
"showYAxis": true
},
"expressions": [
"meter_rocketmq_cluster_commitLog_disk_ratio"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "commitLog disk ratio",
"title": "CommitLog Disk Ratio(%)",
"tips": "The utilization ratio of commit log disk per brokerIp"
}
},
{
"x": 0,
"y": 34,
"w": 12,
"h": 14,
"i": "22",
"type": "Widget",
"id": "0-0-22",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"name": "pull threadPool queue head wait time",
"title": "Pull ThreadPool Queue Head Wait Time(ms)",
"tips": "The wait time in milliseconds for pulling threadPool queue"
}
},
{
"x": 12,
"y": 34,
"w": 12,
"h": 14,
"i": "23",
"type": "Widget",
"id": "0-0-23",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_send_threadPool_queue_head_wait_time"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "send threadPool queue head wait time",
"title": "Send ThreadPool Queue Head Wait Time(ms)",
"tips": "The wait time in milliseconds for sending threadPool queue"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 13,
"w": 6,
"h": 7,
"i": "24",
"type": "Widget",
"id": "0-0-24",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_topic_count)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "topic count",
"title": "Topic Count",
"tips": "The number of topics that received messages from the producer"
},
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 6,
"y": 13,
"w": 6,
"h": 7,
"i": "25",
"type": "Widget",
"id": "0-0-25",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_broker_count)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "broker count",
"title": "Broker Count",
"tips": "The number of brokers that received messages from the producer"
},
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
}
}
]
},
{
"name": "Broker",
"children": [
{
"x": 0,
"y": 0,
"w": 24,
"h": 30,
"i": "0",
"type": "Widget",
"graph": {
"type": "InstanceList",
"dashboardName": "RocketMQ-Broker",
"fontSize": 12
},
"metrics": [],
"metricTypes": [],
"metricConfig": [],
"id": "0-1-0"
}
]
},
{
"name": "Topic",
"children": [
{
"x": 0,
"y": 16,
"w": 24,
"h": 48,
"i": "0",
"type": "Widget",
"graph": {
"type": "EndpointList",
"dashboardName": "RocketMQ-Topic",
"fontSize": 12
},
"metrics": [],
"metricTypes": [],
"metricConfig": [],
"id": "0-2-0"
}
]
}
],
"id": "0",
"activedTabIndex": 0,
"moved": false
}
],
"layer": "ROCKETMQ",
"entity": "Service",
"name": "RocketMQ-Cluster",
"id": "RocketMQ-Cluster",
"isRoot": false,
"isDefault": true,
"expressions": [
"avg(meter_rocketmq_cluster_total_producer_tps)",
"avg(meter_rocketmq_cluster_total_consumer_tps)",
"avg(meter_rocketmq_cluster_consumer_message_size)/1024/1024",
"avg(meter_rocketmq_cluster_producer_message_size)/1024/1024"
],
"expressionsConfig": [
{
"unit": "messages / s",
"label": "Producer TPS"
},
{
"unit": "messages / s",
"label": "Consumer TPS"
},
{
"unit": "MB / s",
"label": "Producer Message Size"
},
{
"unit": "MB / s",
"label": "Consumer Message Size"
}
]
}
}
]

41
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-root.json Normal file
View File

@ -0,0 +1,41 @@
[{
"id": "RocketMQ-Root",
"configuration": {
"children": [{
"x": 0,
"y": 3,
"w": 24,
"h": 29,
"i": "0",
"type": "Widget",
"graph": {
"type": "ServiceList",
"dashboardName": "RocketMQ-Cluster",
"fontSize": 12,
"showXAxis": false,
"showYAxis": false,
"showGroup": true
}
}, {
"x": 0,
"y": 0,
"w": 24,
"h": 3,
"i": "1",
"type": "Text",
"graph": {
"fontColor": "theme",
"backgroundColor": "theme",
"content": "Provide RocketMQ monitoring through OpenTelemetry's Prometheus Receiver",
"fontSize": 14,
"textAlign": "left",
"url": "https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-Rocketmq-monitoring/"
}
}],
"id": "Rocketmq-Root",
"layer": "ROCKETMQ",
"entity": "All",
"name": "Rocketmq-Root",
"isRoot": true
}
}]

391
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-topic.json Normal file
View File

@ -0,0 +1,391 @@
[
{
"id": "RocketMQ-Topic",
"configuration": {
"children": [
{
"x": 0,
"y": 48,
"w": 12,
"h": 15,
"i": "0",
"type": "Widget",
"id": "0",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Bar",
"showBackground": true
},
"expressions": [
"meter_rocketmq_topic_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"title": "Producer Message Size(MB)",
"tips": "The size of messages produced per second",
"name": "producer_message_size"
}
},
{
"x": 12,
"y": 48,
"w": 12,
"h": 15,
"i": "1",
"type": "Widget",
"id": "1",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Bar",
"showBackground": true
},
"expressions": [
"meter_rocketmq_topic_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer_message_size",
"title": "Consumer Message Size(MB)",
"tips": "The size of messages consumed per second "
}
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 9,
"i": "2",
"type": "Widget",
"id": "2",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"metricConfig": [
{
"unit": "MB"
}
],
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_max_consumer_message_size)/1024/1024"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "max_consumer_message_size",
"title": "Max Consumer Message Size",
"tips": "The maximum number of messages consumed"
},
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 0,
"y": 0,
"w": 6,
"h": 9,
"i": "3",
"type": "Widget",
"id": "3",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "MB"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_max_producer_message_size)/1024/1024"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "max_producer_message_size",
"title": "Max Producer Message Size",
"tips": "The maximum number of messages produced"
},
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 12,
"y": 35,
"w": 12,
"h": 13,
"i": "5",
"type": "Widget",
"id": "5",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_latency/1000"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer_latency",
"title": "Consumer Latency(s)",
"tips": "Consumption delay time of a consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 22,
"w": 12,
"h": 13,
"i": "6",
"type": "Widget",
"id": "6",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_producer_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Producer Offset",
"name": "producer_offset",
"tips": "The progress of a topic's production message"
}
},
{
"x": 12,
"y": 22,
"w": 12,
"h": 13,
"i": "7",
"type": "Widget",
"id": "7",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_group_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"title": "Consumer Group Offset",
"name": "consumer_group_offset",
"tips": "The progress of a topic's consumption message per consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 9,
"w": 12,
"h": 13,
"i": "8",
"type": "Widget",
"id": "8",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_group_tps"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"title": "Consumer Group Tps",
"name": "consumer_group_tps",
"tips": "The number of messages consumed per second per consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 9,
"w": 12,
"h": 13,
"i": "9",
"type": "Widget",
"id": "9",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_producer_tps"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"title": "Producer Tps",
"name": "producer_tps",
"tips": "The number of messages produced per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 0,
"w": 6,
"h": 9,
"i": "10",
"type": "Widget",
"id": "10",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_consumer_group_count)"
],
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
},
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "consumer group count",
"title": "Consumer Group Count",
"tips": "The number of consumer groups"
}
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 9,
"i": "11",
"type": "Widget",
"id": "11",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_topic_broker_count)"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"name": "broker count",
"title": "Broker Count",
"tips": "The number of topics that received messages from the producer"
}
},
{
"x": 0,
"y": 35,
"w": 12,
"h": 13,
"i": "12",
"type": "Widget",
"id": "12",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"expressions": [
"meter_rocketmq_topic_producer_offset-meter_rocketmq_topic_consumer_group_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "backlogged messages",
"title": "Backlogged Messages",
"tips": "The number of backlogged messages in each group"
}
}
],
"layer": "ROCKETMQ",
"entity": "Endpoint",
"name": "RocketMQ-Topic",
"id": "RocketMQ-Topic"
}
}
]

51
oap-server/server-starter/src/main/resources/ui-initialized-templates/virtual_mq/virtual-mq-service.json
View File

@ -12,7 +12,7 @@
"i": "1", "i": "1",
"type": "Widget", "type": "Widget",
"widget": { "widget": {
"title": "Consume Traffic (calls / min)" "title": "Consumer Traffic (calls / min)"
}, },
"graph": { "graph": {
"type": "Line", "type": "Line",
@ -32,7 +32,7 @@
"metricConfig": [ "metricConfig": [
{ {
"unit": "calls / min", "unit": "calls / min",
"label": "Consume Traffic" "label": "Consumer Traffic"
} }
], ],
"associate": [ "associate": [
@ -64,7 +64,7 @@
"i": "2", "i": "2",
"type": "Widget", "type": "Widget",
"widget": { "widget": {
"title": "Consume Access Successful Rate (%)", "title": "Consumer Access Successful Rate (%)",
"name": "Successful_Rate" "name": "Successful_Rate"
}, },
"graph": { "graph": {
@ -221,7 +221,7 @@
"i": "5", "i": "5",
"type": "Widget", "type": "Widget",
"widget": { "widget": {
"title": "Produce Traffic (calls / min)" "title": "Producer Traffic (calls / min)"
}, },
"graph": { "graph": {
"type": "Line", "type": "Line",
@ -241,7 +241,7 @@
"metricConfig": [ "metricConfig": [
{ {
"unit": "calls / min", "unit": "calls / min",
"label": "Produce Traffic" "label": "Producer Traffic"
} }
], ],
"associate": [ "associate": [
@ -273,7 +273,7 @@
"i": "6", "i": "6",
"type": "Widget", "type": "Widget",
"widget": { "widget": {
"title": "Produce Access Successful Rate (%)", "title": "Producer Access Successful Rate (%)",
"name": "Successful_Rate" "name": "Successful_Rate"
}, },
"graph": { "graph": {
@ -355,19 +355,19 @@
"label": "Transmission Latency" "label": "Transmission Latency"
}, },
{ {
"label": "Consume Successful Rate", "label": "Consumer Successful Rate",
"unit": "%" "unit": "%"
}, },
{ {
"label": "Consume Traffic", "label": "Consumer Traffic",
"unit": "calls / min" "unit": "calls / min"
}, },
{ {
"label": "Produce Successful Rate", "label": "Producer Successful Rate",
"unit": "%" "unit": "%"
}, },
{ {
"label": "Produce Traffic", "label": "Producer Traffic",
"unit": "calls / min" "unit": "calls / min"
} }
] ]
@ -378,7 +378,36 @@
"name": "Virtual-MQ-Service", "name": "Virtual-MQ-Service",
"id": "Virtual-MQ-Service", "id": "Virtual-MQ-Service",
"isRoot": false, "isRoot": false,
"isDefault": true "isDefault": true,
"expressions": [
"avg(mq_service_consume_latency)",
"avg(mq_endpoint_produce_sla)/100",
"avg(mq_endpoint_consume_sla)/100",
"avg(mq_service_produce_cpm)",
"avg(mq_service_consume_cpm)"
],
"expressionsConfig": [
{
"unit": "ms",
"label": "Consumer Latency"
},
{
"unit": "%",
"label": "Producer Successful Rate"
},
{
"unit": "%",
"label": "Consumer Successful Rate"
},
{
"unit": "calls / min",
"label": "Producer Traffic"
},
{
"unit": "calls / min",
"label": "Consumer Traffic"
}
]
} }
} }
] ]

117
test/e2e-v2/cases/rocketmq/docker-compose.yml Normal file
View File

@ -0,0 +1,117 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: "3"
services:
oap:
extends:
file: ../../script/docker-compose/base-compose.yml
service: oap
ports:
- "12800:12800"
networks:
- e2e
rocketmq-namesrv-1:
image: apache/rocketmq:4.9.4
hostname: rocketmq-namesrv-1
ports:
- "9876:9876"
environment:
TZ: Asia/Shanghai
command: sh mqnamesrv
healthcheck:
test: [ "CMD", "nc", "-nz", "127.0.0.1", "9876" ]
interval: 15s
timeout: 10s
retries: 120
networks:
- e2e
rocketmq-broker-1:
image: apache/rocketmq:4.9.4
hostname: rocketmq-broker-1
ports:
- "10911:10911"
- "10909:10909"
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh mqbroker
healthcheck:
test: [ "CMD", "nc", "-nz", "127.0.0.1", "10911" ]
interval: 15s
timeout: 10s
retries: 120
depends_on:
rocketmq-namesrv-1:
condition: service_healthy
networks:
- e2e
rocketmq-producer-perf-test:
image: apache/rocketmq:4.9.4
hostname: rocketmq-producer-perf-test
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh tools.sh org.apache.rocketmq.example.quickstart.Producer
depends_on:
rocketmq-broker-1:
condition: service_healthy
networks:
- e2e
rocketmq-consumer-perf-test:
image: apache/rocketmq:4.9.4
hostname: rocketmq-consumer-perf-test
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh tools.sh org.apache.rocketmq.example.quickstart.Consumer
depends_on:
- rocketmq-producer-perf-test
networks:
- e2e
rocketmq-exporter:
image: apache/rocketmq-exporter:0.0.2
hostname: rocketmq-exporter
ports:
- "5557:5557"
environment:
TZ: Asia/Shanghai
rocketmq.config.namesrvAddr: "rocketmq-namesrv-1:9876"
depends_on:
- rocketmq-consumer-perf-test
networks:
- e2e
otel-collector:
image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION}
networks:
- e2e
command: [ "--config=/etc/otel-collector-config.yaml" ]
volumes:
- ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
expose:
- 55678
depends_on:
oap:
condition: service_healthy
networks:
e2e:

37
test/e2e-v2/cases/rocketmq/e2e.yaml Normal file
View File

@ -0,0 +1,37 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
setup:
env: compose
file: docker-compose.yml
timeout: 20m
init-system-environment: ../../script/env
steps:
- name: set PATH
command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
- name: install yq
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
- name: install swctl
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
verify:
retry:
count: 60
interval: 3s
cases:
- includes:
- ./rocketmq-cases.yaml

19
test/e2e-v2/cases/rocketmq/expected/endpoint.yml Normal file
View File

@ -0,0 +1,19 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains .}}
- id: {{ b64enc "rocketmq::rocketmq-cluster" }}.1_{{ b64enc "TopicTest" }}
name: TopicTest
{{- end}}

22
test/e2e-v2/cases/rocketmq/expected/instance.yml Normal file
View File

@ -0,0 +1,22 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ notEmpty .id }}
name: rocketmq-broker-1
instanceuuid: {{ notEmpty .instanceuuid }}
attributes: []
language: UNKNOWN
{{- end }}

35
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-empty-label.yml Normal file
View File

@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
{{- contains .metric.labels }}
- key: _
value: ""
{{- end}}
values:
{ { - contains .values } }
- id: { { notEmpty .id } }
value: null
traceid: null
- id: { { notEmpty .id } }
value: { { .value } }
traceid: null
{ { - end } }
{{- end}}
error: null

33
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-group-label.yml Normal file
View File

@ -0,0 +1,33 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
- key: _
value: please_rename_unique_group_name_4
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

35
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-label.yml Normal file
View File

@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
{{- contains .metric.labels }}
- key: _
value: {{ .value }}
{{- end}}
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

31
test/e2e-v2/cases/rocketmq/expected/metrics-has-value.yml Normal file
View File

@ -0,0 +1,31 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels: []
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

24
test/e2e-v2/cases/rocketmq/expected/service.yml Normal file
View File

@ -0,0 +1,24 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ b64enc "rocketmq::rocketmq-cluster" }}.1
name: rocketmq::rocketmq-cluster
group: rocketmq
shortname: rocketmq-cluster
layers:
- ROCKETMQ
normal: true
{{- end }}

45
test/e2e-v2/cases/rocketmq/otel-collector-config.yaml Normal file
View File

@ -0,0 +1,45 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
receivers:
prometheus:
config:
scrape_configs:
- job_name: "rocketmq-monitoring"
scrape_interval: 30s
static_configs:
- targets: ['rocketmq-exporter:5557']
relabel_configs:
- source_labels: [ ]
target_label: cluster
replacement: rocketmq-cluster
exporters:
otlp:
endpoint: oap:11800
tls:
insecure: true
processors:
batch:
service:
pipelines:
metrics:
receivers:
- prometheus
processors:
- batch
exporters:
- otlp

89
test/e2e-v2/cases/rocketmq/rocketmq-cases.yaml Normal file
View File

@ -0,0 +1,89 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
cases:
# service cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
expected: expected/service.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_produced_today --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_consumed_today --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_total_producer_tps --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_total_consumer_tps --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_producer_message_size --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_consumer_message_size --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_produced_until_yesterday --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_consumed_until_yesterday --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_max_consumer_latency --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_max_commitLog_disk_ratio --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_commitLog_disk_ratio --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_send_threadPool_queue_head_wait_time --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_topic_count --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-empty-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_broker_count --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-empty-label.yml
# instance cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance ls --service-name=rocketmq::rocketmq-cluster
expected: expected/instance.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_produce_tps --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_consume_qps --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_producer_message_size --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_consumer_message_size --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
# endpoint cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql endpoint ls --service-name=rocketmq::rocketmq-cluster
expected: expected/endpoint.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_max_producer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_max_consumer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_latency --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_tps --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_tps --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_offset --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_offset --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_count --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-empty-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_broker_count --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-empty-label.yml