kazusa
/
skywalking
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Support RocketMQ monitoring (#11758) 

* Support RocketMQ monitoring
* Add swip3 - RocketMQ monitoring
* Add RocketMQ e2e cases

---------

Co-authored-by: peachisai <497533062@qq.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
Co-authored-by: 邵一鸣 <shaoym02@vanke.com>
Co-authored-by: Wan Kai <wankai123@foxmail.com>
This commit is contained in:
peachisai 2024-02-18 07:17:20 +08:00 committed by GitHub
parent b775e7f547
commit 48c766ee01
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
30 changed files with 2031 additions and 53 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.github/workflows/skywalking.yaml vendored
View File

@ -650,6 +650,8 @@ jobs:
config: test/e2e-v2/cases/mqe/e2e.yaml
- name: Pulsar and BookKeeper
config: test/e2e-v2/cases/pulsar/e2e.yaml
- name: RocketMQ
config: test/e2e-v2/cases/rocketmq/e2e.yaml
- name: UI Menu BanyanDB
config: test/e2e-v2/cases/menu/banyandb/e2e.yaml

4
docs/en/changes/changes.md
View File

@ -46,6 +46,10 @@
- APISIX -> K8S_SERVICE
- GENERAL -> APISIX
* Add Golang as a supported language for RocketMQ.
* Support Apache RocketMQ server monitoring.
* Add Service Hierarchy auto matching layer relationships (upper -> lower) as following:
- ROCKETMQ -> K8S_SERVICE
- VIRTUAL_MQ -> ROCKETMQ
#### UI

46
docs/en/concepts-and-designs/service-hierarchy.md
View File

@ -11,20 +11,22 @@ There 2 ways to detect the connections:
If you want to customize it according to your own needs, please refer to [Service Hierarchy Configuration](service-hierarchy-configuration.md).
### Automatically Matching
| Upper layer | Lower layer | Matching rule |
|-------------------|-------------|-------------------------------------------------------------------|
| MESH | MESH_DP | [MESH On MESH_DP](#mesh-on-mesh_dp) |
| MESH | K8S_SERVICE | [MESH On K8S_SERVICE](#mesh-on-k8s_service) |
| MESH_DP | K8S_SERVICE | [MESH_DP On K8S_SERVICE](#mesh_dp-on-k8s_service) |
| GENERAL | K8S_SERVICE | [GENERAL On K8S_SERVICE](#general-on-k8s_service) |
| GENERAL | APISIX | [GENERAL On APISIX](#general-on-apisix) |
| MYSQL | K8S_SERVICE | [MYSQL On K8S_SERVICE](#mysql-on-k8s_service) |
| POSTGRESQL | K8S_SERVICE | [POSTGRESQL On K8S_SERVICE](#postgresql-on-k8s_service) |
| SO11Y_OAP | K8S_SERVICE | [SO11Y_OAP On K8S_SERVICE](#so11y_oap-on-k8s_service) |
| VIRTUAL_DATABASE | MYSQL | [VIRTUAL_DATABASE On MYSQL](#virtual_database-on-mysql) |
| VIRTUAL_DATABASE | POSTGRESQL | [VIRTUAL_DATABASE On POSTGRESQL](#virtual_database-on-postgresql) |
| NGINX | K8S_SERVICE | [NGINX On K8S_SERVICE](#nginx-on-k8s_service) |
| APISIX | K8S_SERVICE | [APISIX On K8S_SERVICE](#apisix-on-k8s_service) |
| Upper layer | Lower layer | Matching rule |
|------------------|-------------|-------------------------------------------------------------------|
| MESH | MESH_DP | [MESH On MESH_DP](#mesh-on-mesh_dp) |
| MESH | K8S_SERVICE | [MESH On K8S_SERVICE](#mesh-on-k8s_service) |
| MESH_DP | K8S_SERVICE | [MESH_DP On K8S_SERVICE](#mesh_dp-on-k8s_service) |
| GENERAL | K8S_SERVICE | [GENERAL On K8S_SERVICE](#general-on-k8s_service) |
| GENERAL | APISIX | [GENERAL On APISIX](#general-on-apisix) |
| MYSQL | K8S_SERVICE | [MYSQL On K8S_SERVICE](#mysql-on-k8s_service) |
| POSTGRESQL | K8S_SERVICE | [POSTGRESQL On K8S_SERVICE](#postgresql-on-k8s_service) |
| SO11Y_OAP | K8S_SERVICE | [SO11Y_OAP On K8S_SERVICE](#so11y_oap-on-k8s_service) |
| VIRTUAL_DATABASE | MYSQL | [VIRTUAL_DATABASE On MYSQL](#virtual_database-on-mysql) |
| VIRTUAL_DATABASE | POSTGRESQL | [VIRTUAL_DATABASE On POSTGRESQL](#virtual_database-on-postgresql) |
| NGINX | K8S_SERVICE | [NGINX On K8S_SERVICE](#nginx-on-k8s_service) |
| APISIX | K8S_SERVICE | [APISIX On K8S_SERVICE](#apisix-on-k8s_service) |
| ROCKETMQ | K8S_SERVICE | [ROCKETMQ On K8S_SERVICE](#rocketmq-on-k8s_service) |
| VIRTUAL_MQ | ROCKETMQ | [VIRTUAL_MQ On K8S_SERVICE](#virtual_mq-on-rocketmq) |
- The following sections will describe the **default matching rules** in detail and use the `upper-layer On lower-layer` format.
- The example service name are based on SkyWalking [Showcase](https://github.com/apache/skywalking-showcase) default deployment.
@ -126,6 +128,22 @@ If you want to customize it according to your own needs, please refer to [Servic
- APISIX.service.name: `APISIX::frontend.sample-services`
- K8S_SERVICE.service.name: `skywalking-showcase::frontend.sample-services`
#### ROCKETMQ On K8S_SERVICE
- Rule name: `short-name`
- Groovy script: `{ (u, l) -> u.shortName == l.shortName }`
- Description: ROCKETMQ.service.shortName == K8S_SERVICE.service.shortName
- Matched Example:
- ROCKETMQ.service.name: `rocketmq::rocketmq.skywalking-showcase`
- K8S_SERVICE.service.name: `skywalking-showcase::rocketmq.skywalking-showcase`
#### VIRTUAL_MQ On ROCKETMQ
- Rule name: `lower-short-name-with-fqdn`
- Groovy script: `{ (u, l) -> u.shortName.substring(0, u.shortName.lastIndexOf(':')) == l.shortName.concat('.svc.cluster.local') }`
- Description: VIRTUAL_MQ.service.shortName remove port == ROCKETMQ.service.shortName with fqdn suffix
- Matched Example:
- VIRTUAL_MQ.service.name: `rocketmq.skywalking-showcase.svc.cluster.local:9876`
- ROCKETMQ.service.name: `rocketmq::rocketmq.skywalking-showcase`
### Build Through Specific Agents
Use agent tech involved(such as eBPF) and deployment tools(such as operator and agent injector) detect the service hierarchy relations.

73
docs/en/setup/backend/backend-rocketmq-monitoring.md Normal file
View File

@ -0,0 +1,73 @@
# RocketMQ monitoring
SkyWalking leverages rocketmq-exporter for collecting metrics data from RocketMQ. It leverages OpenTelemetry
Collector to transfer the metrics to
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
## Data flow
1. The `rocketmq-exporter` (https://github.com/apache/rocketmq-exporter?tab=readme-ov-file#readme) collects metrics data from RocketMQ, The RocketMQ version is required to be 4.3.2+.
2. OpenTelemetry Collector fetches metrics from rocketmq-exporter via Prometheus Receiver and pushes metrics to
SkyWalking OAP Server via OpenTelemetry gRPC exporter.
3. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to
filter/calculate/aggregate and store the results.
## Setup
1. Setup [rocketmq-exporter](https://github.com/apache/rocketmq-exporter).
2. Set up [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/getting-started/#docker). The example for OpenTelemetry Collector configuration, refer
to [here](../../../../test/e2e-v2/cases/rocketmq/otel-collector-config.yaml).
3. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
## RocketMQ Monitoring
RocketMQ monitoring provides multidimensional metrics monitoring of RocketMQ Exporter as `Layer: RocketMQ` `Service` in
the OAP. In each cluster, the broker is represented as `Instance` and the topic is represented as `Endpoint`.
### RocketMQ Cluster Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|--------------------------------------------------------------------------|------------------|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
### RocketMQ Broker Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|----------------------------------------------------|-------------------|
| Produce TPS | Msg/sec | meter_rocketmq_broker_produce_tps | The number of broker produces messages per second. | RocketMQ Exporter |
| Consume QPS | Msg/sec | meter_rocketmq_broker_consume_qps | The number of broker consumes messages per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_broker_producer_message_size | The max size of the messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_broker_consumer_message_size | The max size of the messages consumed per second. | RocketMQ Exporter |
### RocketMQ Topic Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
## Customizations
You can customize your own metrics/expression/dashboard panel.
The metrics definition and expression rules are found
in `otel-rules/rocketmq/rocketmq-cluster.yaml, otel-rules/rocketmq/rocketmq-broker.yaml, otel-rules/rocketmq/rocketmq-topic.yaml`.
The RocketMQ dashboard panel configurations are found in `ui-initialized-templates/rocketmq`.

56
docs/en/swip/SWIP-3.md
View File

@ -11,21 +11,23 @@ Provide cluster, broker, and topic dimensions monitoring.
### RocketMQ Cluster Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|--------------------------------------------------------------------------|------------------|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------|------------|-------------------------------------------------------------|---------------------------------------------------------------------------|------------------|
| Messages Produced Today | Count | meter_rocketmq_cluster_messages_produced_today | The number of cluster messages produced today. | RocketMQ Exporter |
| Messages Consumed Today | Count | meter_rocketmq_cluster_messages_consumed_today | The number of cluster messages consumed today. | RocketMQ Exporter |
| Total Producer Tps | Msg/sec | meter_rocketmq_cluster_total_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Total Consume Tps | Msg/sec | meter_rocketmq_cluster_total_consumer_tps | The number of messages consumed per second. | RocketMQ Exporter |
| Producer Message Size | Bytes/sec | meter_rocketmq_cluster_producer_message_size | The max size of a message produced per second. | RocketMQ Exporter |
| Consumer Message Size | Bytes/sec | meter_rocketmq_cluster_consumer_message_size | The max size of the consumed message per second. | RocketMQ Exporter |
| Messages Produced Until Yesterday | Count | meter_rocketmq_cluster_messages_produced_until_yesterday | The total number of messages put until 12 o'clock last night. | RocketMQ Exporter |
| Messages Consumed Until Yesterday | Count | meter_rocketmq_cluster_messages_consumed_until_yesterday | The total number of messages read until 12 o'clock last night. | RocketMQ Exporter |
| Max Consumer Latency | ms | meter_rocketmq_cluster_max_consumer_latency | The max number of consumer latency. | RocketMQ Exporter |
| Max CommitLog Disk Ratio | % | meter_rocketmq_cluster_max_commitLog_disk_ratio | The max utilization ratio of the commit log disk. | RocketMQ Exporter |
| CommitLog Disk Ratio | % | meter_rocketmq_cluster_commitLog_disk_ratio | The utilization ratio of the commit log disk per broker IP. | RocketMQ Exporter |
| Pull ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time | The wait time in milliseconds for pulling threadPool queue per broker IP. | RocketMQ Exporter |
| Send ThreadPool Queue Head Wait Time | ms | meter_rocketmq_cluster_send_threadPool_queue_head_wait_time | The wait time in milliseconds for sending threadPool queue per broker IP. | RocketMQ Exporter |
| Topic Count | Count | meter_rocketmq_cluster_topic_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_cluster_broker_count | The number of brokers that received messages from the producer. | RocketMQ Exporter |
### RocketMQ Broker Supported Metrics
@ -38,17 +40,19 @@ Provide cluster, broker, and topic dimensions monitoring.
### RocketMQ Topic Supported Metrics
| Monitoring Panel |Unit | Metric Name | Description | Data Source |
|--------------------------------------------|------------|-------------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
| Monitoring Panel | Unit | Metric Name | Description | Data Source |
|---------------------------|-----------|------------------------------------------------------------------|-----------------------------------------------------------------------|-------------------|
| Max Producer Message Size | Byte | meter_rocketmq_topic_max_producer_message_size | The maximum number of messages produced. | RocketMQ Exporter |
| Max Consumer Message Size | Byte | meter_rocketmq_topic_max_consumer_message_size | The maximum number of messages consumed. | RocketMQ Exporter |
| Consumer Latency | ms | meter_rocketmq_topic_consumer_latency | Consumption delay time of a consumer group. | RocketMQ Exporter |
| Producer Tps | Msg/sec | meter_rocketmq_topic_producer_tps | The number of messages produced per second. | RocketMQ Exporter |
| Consumer Group Tps | Msg/sec | meter_rocketmq_topic_consumer_group_tps | The number of messages consumed per second per consumer group. | RocketMQ Exporter |
| Producer Offset | Count | meter_rocketmq_topic_producer_offset | The max progress of a topic's production message. | RocketMQ Exporter |
| Consumer Group Offset | Count | meter_rocketmq_topic_consumer_group_offset | The max progress of a topic's consumption message per consumer group. | RocketMQ Exporter |
| Producer Message Size | Byte/sec | meter_rocketmq_topic_producer_message_size | The max size of messages produced per second. | RocketMQ Exporter |
| Consumer Message Size | Byte/sec | meter_rocketmq_topic_consumer_message_size | The max size of messages consumed per second. | RocketMQ Exporter |
| Consumer Group_Count | Count | meter_rocketmq_topic_consumer_group_count | The number of consumer groups. | RocketMQ Exporter |
| Broker Count | Count | meter_rocketmq_topic_broker_count | The number of topics that received messages from the producer. | RocketMQ Exporter |
## Imported Dependencies libs and their licenses.
No new dependency.

2
docs/menu.yml
View File

@ -260,6 +260,8 @@ catalog:
path: "/en/setup/backend/backend-kafka-monitoring"
- name: "Pulsar"
path: "/en/setup/backend/backend-pulsar-monitoring"
- name: "RocketMQ"
path: "/en/setup/backend/backend-rocketmq-monitoring"
- name: "Self Observability"
catalog:
- name: "OAP self telemetry"

7
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java
View File

@ -213,7 +213,12 @@ public enum Layer {
/**
* Nginx is an HTTP and reverse proxy server, a mail proxy server, and a generic TCP/UDP proxy server.
*/
NGINX(34, true);
NGINX(34, true),
/**
* A cloud native messaging and streaming platform, making it simple to build event-driven applications.
*/
ROCKETMQ(35, true);
private final int value;
/**

1
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
View File

@ -72,6 +72,7 @@ public class UITemplateInitializer {
Layer.PULSAR.name(),
Layer.BOOKKEEPER.name(),
Layer.NGINX.name(),
Layer.ROCKETMQ.name(),
"custom"
};
private final UITemplateManagementService uiTemplateManagementService;

2
oap-server/server-starter/src/main/resources/application.yml
View File

@ -353,7 +353,7 @@ receiver-otel:
selector: ${SW_OTEL_RECEIVER:default}
default:
enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"otlp-metrics,otlp-logs"}
enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*"}
enabledOtelMetricsRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_METRICS_RULES:"apisix,nginx/*,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*,aws-dynamodb/*,aws-gateway/*,redis/*,elasticsearch/*,rabbitmq/*,mongodb/*,kafka/*,pulsar/*,bookkeeper/*,rocketmq/*"}
receiver-zipkin:
selector: ${SW_RECEIVER_ZIPKIN:-}

8
oap-server/server-starter/src/main/resources/hierarchy-definition.yml
View File

@ -45,10 +45,16 @@ hierarchy:
SO11Y_OAP:
K8S_SERVICE: short-name
ROCKETMQ:
K8S_SERVICE: short-name
VIRTUAL_DATABASE:
MYSQL: lower-short-name-with-fqdn
POSTGRESQL: lower-short-name-with-fqdn
VIRTUAL_MQ:
ROCKETMQ: lower-short-name-with-fqdn
# Use Groovy script to define the matching rules, the input parameters are the upper service(u) and the lower service(l) and the return value is a boolean,
# which are used to match the relation between the upper service(u) and the lower service(l) on the different layers.
auto-matching-rules:
@ -70,11 +76,13 @@ layer-levels:
GENERAL: 3
SO11Y_OAP: 3
VIRTUAL_DATABASE: 3
VIRTUAL_MQ: 3
MYSQL: 2
POSTGRESQL: 2
APISIX: 2
NGINX: 2
ROCKETMQ: 2
MESH_DP: 1

46
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-broker.yaml Normal file
View File

@ -0,0 +1,46 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).instance(['cluster'], ['broker'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_broker
metricsRules:
- name: produce_tps
exp: rocketmq_broker_tps.sum(['cluster', 'broker'])
- name: consume_qps
exp: rocketmq_broker_qps.sum(['cluster','broker'])
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster','broker']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster','broker']).downsampling(MAX)

81
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-cluster.yaml Normal file
View File

@ -0,0 +1,81 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).service(['cluster'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_cluster
metricsRules:
- name: messages_produced_today
exp: rocketmq_brokeruntime_msg_put_total_today_now.sum((['cluster']))-rocketmq_brokeruntime_msg_puttotal_yesterdaymorning.sum(['cluster'])
- name: messages_consumed_today
exp: rocketmq_brokeruntime_msg_gettotal_today_now.sum(['cluster'])-rocketmq_brokeruntime_msg_gettotal_yesterdaymorning.sum(['cluster'])
- name: total_producer_tps
exp: rocketmq_producer_tps.sum(['cluster'])
- name: total_consumer_tps
exp: rocketmq_consumer_tps.sum(['cluster'])
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster']).downsampling(MAX)
- name: messages_produced_until_yesterday
exp: rocketmq_brokeruntime_msg_puttotal_yesterdaymorning.sum(['cluster'])
- name: messages_consumed_until_yesterday
exp: rocketmq_brokeruntime_msg_gettotal_yesterdaymorning.sum(['cluster'])
- name: max_consumer_latency
exp: rocketmq_group_get_latency_by_storetime.max(['cluster','broker','topic','group'])
- name: max_commitLog_disk_ratio
exp: rocketmq_brokeruntime_commitlog_disk_ratio.max(['cluster','brokerIP'])*100
- name: commitLog_disk_ratio
exp: rocketmq_brokeruntime_commitlog_disk_ratio.sum(['cluster','brokerIP'])*100
- name: pull_threadPool_queue_head_wait_time
exp: rocketmq_brokeruntime_pull_threadpoolqueue_headwait_timemills.sum(['cluster','brokerIP'])
- name: send_threadPool_queue_head_wait_time
exp: rocketmq_brokeruntime_send_threadpoolqueue_headwait_timemills.sum(['cluster','brokerIP'])
- name: topic_count
exp: rocketmq_producer_offset.count(['cluster','topic'])
- name: broker_count
exp: rocketmq_producer_offset.count(['cluster','broker'])

70
oap-server/server-starter/src/main/resources/otel-rules/rocketmq/rocketmq-topic.yaml Normal file
View File

@ -0,0 +1,70 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'rocketmq-monitoring' }" # The OpenTelemetry job name
expSuffix: tag({tags -> tags.cluster = 'rocketmq::' + tags.cluster}).endpoint(['cluster'], ['topic'], Layer.ROCKETMQ)
metricPrefix: meter_rocketmq_topic
metricsRules:
- name: max_producer_message_size
exp: rocketmq_producer_message_size.max(['cluster','topic'])
- name: max_consumer_message_size
exp: rocketmq_consumer_message_size.max(['cluster','topic','group'])
- name: consumer_latency
exp: rocketmq_group_get_latency_by_storetime.sum(['cluster','topic','group'])
- name: producer_tps
exp: rocketmq_producer_tps.sum(['cluster','topic'])
- name: consumer_group_tps
exp: rocketmq_consumer_tps.sum(['cluster','topic'])
- name: producer_offset
exp: rocketmq_producer_offset.sum(['cluster','topic']).downsampling(MAX)
- name: consumer_group_offset
exp: rocketmq_consumer_offset.sum(['cluster','topic','group']).downsampling(MAX)
- name: producer_message_size
exp: rocketmq_producer_message_size.sum(['cluster','topic']).downsampling(MAX)
- name: consumer_message_size
exp: rocketmq_consumer_message_size.sum(['cluster','topic','group']).downsampling(MAX)
- name: consumer_group_count
exp: rocketmq_consumer_offset.count(['cluster','topic','group'])
- name: broker_count
exp: rocketmq_producer_offset.count(['cluster','topic','broker'])

5
oap-server/server-starter/src/main/resources/ui-initialized-templates/menu.yaml
View File

@ -208,6 +208,11 @@ menus:
description: Provide pulsar monitoring through OpenTelemetry's Prometheus Receiver.
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-pulsar-monitoring/
i18nKey: mq_pulsar
- title: RocketMQ
layer: ROCKETMQ
description: Provide RocketMQ monitoring through OpenTelemetry's Prometheus Receiver.
documentLink: https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-rocketmq-monitoring/
i18nKey: mq_rocketmq
- title: Self Observability
icon: self_observability
description: Self Observability provides the observabilities for running components and servers from the SkyWalking ecosystem.

142
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-broker.json Normal file
View File

@ -0,0 +1,142 @@
[
{
"id": "RocketMQ-Broker",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 12,
"h": 14,
"i": "8",
"type": "Widget",
"id": "8",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Produce Tps",
"tips": "The number of broker produces messages per second",
"name": "produce qps"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"meter_rocketmq_broker_produce_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
]
},
{
"x": 12,
"y": 0,
"w": 12,
"h": 14,
"i": "9",
"type": "Widget",
"id": "9",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_consume_qps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Consume Qps",
"tips": "The number of broker consumes messages per second",
"name": "consume qps"
}
},
{
"x": 0,
"y": 14,
"w": 12,
"h": 13,
"i": "13",
"type": "Widget",
"id": "13",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "producer_message_size_per_second",
"title": "Producer Message Size(MB)",
"tips": "The size of the messages produced per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 14,
"w": 12,
"h": 13,
"i": "14",
"type": "Widget",
"id": "14",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_broker_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer message size",
"title": "Consumer Message Size(MB)",
"tips": "The size of the messages consumed per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
}
],
"layer": "ROCKETMQ",
"entity": "ServiceInstance",
"name": "RocketMQ-Broker",
"id": "RocketMQ-Broker"
}
}
]

569
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-cluster.json Normal file
View File

@ -0,0 +1,569 @@
[
{
"id": "RocketMQ-Cluster",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 24,
"h": 48,
"i": "0",
"type": "Tab",
"children": [
{
"name": "Overview",
"children": [
{
"x": 12,
"y": 0,
"w": 6,
"h": 6,
"i": "8",
"type": "Widget",
"id": "0-0-8",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"title": "Messages Produced Today",
"tips": "The number of the cluster messages produced today",
"name": "messages produced today"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"latest(meter_rocketmq_cluster_messages_produced_today)"
],
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 6,
"i": "9",
"type": "Widget",
"id": "0-0-9",
"metricTypes": [],
"metrics": [],
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"title": "Messages Consumed Today",
"tips": "The number of the cluster messages consumed today",
"name": "messages consumed today"
},
"metricMode": "Expression",
"metricConfig": [],
"expressions": [
"latest(meter_rocketmq_cluster_messages_consumed_today)"
],
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 0,
"y": 20,
"w": 12,
"h": 14,
"i": "10",
"type": "Widget",
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Total Producer Tps",
"tips": "The number of messages produced per second",
"name": "total producer TPS"
},
"expressions": [
"meter_rocketmq_cluster_total_producer_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"metricMode": "Expression",
"id": "0-0-10",
"moved": false,
"metrics": [],
"metricTypes": [],
"metricConfig": []
},
{
"x": 12,
"y": 20,
"w": 12,
"h": 14,
"i": "11",
"type": "Widget",
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Total Consume Tps",
"tips": "The number of messages consumed per second",
"name": "total consumer TPS"
},
"expressions": [
"meter_rocketmq_cluster_total_consumer_tps"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"metricMode": "Expression",
"id": "0-0-11",
"moved": false,
"metrics": [],
"metricTypes": [],
"metricConfig": []
},
{
"x": 0,
"y": 48,
"w": 12,
"h": 14,
"i": "12",
"type": "Widget",
"id": "0-0-12",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Bar",
"showBackground": true
},
"widget": {
"name": "producer message size",
"title": "Producer Message Size(MB)",
"tips": "The size of a message produced per second"
}
},
{
"x": 12,
"y": 48,
"w": 12,
"h": 14,
"i": "13",
"type": "Widget",
"id": "0-0-13",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Bar",
"showBackground": true
},
"widget": {
"name": "consumer message size",
"title": "Consumer Message Size(MB)",
"tips": "The size of the consumed message per second"
}
},
{
"x": 0,
"y": 0,
"w": 6,
"h": 6,
"i": "16",
"type": "Widget",
"id": "0-0-16",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_cluster_messages_produced_until_yesterday)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "messages produced until yesterday",
"title": "Messages Produced Until Yesterday",
"tips": "The total number of messages put until 12 o'clock last night"
}
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 6,
"i": "17",
"type": "Widget",
"id": "0-0-17",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_cluster_messages_consumed_until_yesterday)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "messages consumed until yesterday",
"title": "Messages Consumed Until Yesterday",
"tips": "The total number of messages read until 12 o'clock last night"
}
},
{
"x": 6,
"y": 6,
"w": 6,
"h": 7,
"i": "18",
"type": "Widget",
"id": "0-0-18",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "ms"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_max_consumer_latency)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"name": "max consumer latency",
"title": "Max Consumer Latency",
"tips": "The max number of consumer latency"
}
},
{
"x": 0,
"y": 6,
"w": 6,
"h": 7,
"i": "19",
"type": "Widget",
"id": "0-0-19",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "%"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_max_commitLog_disk_ratio)"
],
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
},
"widget": {
"name": "max commitLog disk ratio",
"title": "Max CommitLog Disk Ratio",
"tips": "The max utilization ratio of the commit log disk"
},
"typesOfMQE": [
"SINGLE_VALUE"
]
},
{
"x": 12,
"y": 6,
"w": 12,
"h": 14,
"i": "21",
"type": "Widget",
"id": "0-0-21",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Area",
"opacity": 0.4,
"showXAxis": true,
"showYAxis": true
},
"expressions": [
"meter_rocketmq_cluster_commitLog_disk_ratio"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "commitLog disk ratio",
"title": "CommitLog Disk Ratio(%)",
"tips": "The utilization ratio of commit log disk per brokerIp"
}
},
{
"x": 0,
"y": 34,
"w": 12,
"h": 14,
"i": "22",
"type": "Widget",
"id": "0-0-22",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"name": "pull threadPool queue head wait time",
"title": "Pull ThreadPool Queue Head Wait Time(ms)",
"tips": "The wait time in milliseconds for pulling threadPool queue"
}
},
{
"x": 12,
"y": 34,
"w": 12,
"h": 14,
"i": "23",
"type": "Widget",
"id": "0-0-23",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_cluster_send_threadPool_queue_head_wait_time"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "send threadPool queue head wait time",
"title": "Send ThreadPool Queue Head Wait Time(ms)",
"tips": "The wait time in milliseconds for sending threadPool queue"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 13,
"w": 6,
"h": 7,
"i": "24",
"type": "Widget",
"id": "0-0-24",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_topic_count)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "topic count",
"title": "Topic Count",
"tips": "The number of topics that received messages from the producer"
},
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 6,
"y": 13,
"w": 6,
"h": 7,
"i": "25",
"type": "Widget",
"id": "0-0-25",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_cluster_broker_count)"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "broker count",
"title": "Broker Count",
"tips": "The number of brokers that received messages from the producer"
},
"graph": {
"type": "Card",
"fontSize": 26,
"textAlign": "center",
"showUnit": true
}
}
]
},
{
"name": "Broker",
"children": [
{
"x": 0,
"y": 0,
"w": 24,
"h": 30,
"i": "0",
"type": "Widget",
"graph": {
"type": "InstanceList",
"dashboardName": "RocketMQ-Broker",
"fontSize": 12
},
"metrics": [],
"metricTypes": [],
"metricConfig": [],
"id": "0-1-0"
}
]
},
{
"name": "Topic",
"children": [
{
"x": 0,
"y": 16,
"w": 24,
"h": 48,
"i": "0",
"type": "Widget",
"graph": {
"type": "EndpointList",
"dashboardName": "RocketMQ-Topic",
"fontSize": 12
},
"metrics": [],
"metricTypes": [],
"metricConfig": [],
"id": "0-2-0"
}
]
}
],
"id": "0",
"activedTabIndex": 0,
"moved": false
}
],
"layer": "ROCKETMQ",
"entity": "Service",
"name": "RocketMQ-Cluster",
"id": "RocketMQ-Cluster",
"isRoot": false,
"isDefault": true,
"expressions": [
"avg(meter_rocketmq_cluster_total_producer_tps)",
"avg(meter_rocketmq_cluster_total_consumer_tps)",
"avg(meter_rocketmq_cluster_consumer_message_size)/1024/1024",
"avg(meter_rocketmq_cluster_producer_message_size)/1024/1024"
],
"expressionsConfig": [
{
"unit": "messages / s",
"label": "Producer TPS"
},
{
"unit": "messages / s",
"label": "Consumer TPS"
},
{
"unit": "MB / s",
"label": "Producer Message Size"
},
{
"unit": "MB / s",
"label": "Consumer Message Size"
}
]
}
}
]

41
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-root.json Normal file
View File

@ -0,0 +1,41 @@
[{
"id": "RocketMQ-Root",
"configuration": {
"children": [{
"x": 0,
"y": 3,
"w": 24,
"h": 29,
"i": "0",
"type": "Widget",
"graph": {
"type": "ServiceList",
"dashboardName": "RocketMQ-Cluster",
"fontSize": 12,
"showXAxis": false,
"showYAxis": false,
"showGroup": true
}
}, {
"x": 0,
"y": 0,
"w": 24,
"h": 3,
"i": "1",
"type": "Text",
"graph": {
"fontColor": "theme",
"backgroundColor": "theme",
"content": "Provide RocketMQ monitoring through OpenTelemetry's Prometheus Receiver",
"fontSize": 14,
"textAlign": "left",
"url": "https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-Rocketmq-monitoring/"
}
}],
"id": "Rocketmq-Root",
"layer": "ROCKETMQ",
"entity": "All",
"name": "Rocketmq-Root",
"isRoot": true
}
}]

391
oap-server/server-starter/src/main/resources/ui-initialized-templates/rocketmq/rocketmq-topic.json Normal file
View File

@ -0,0 +1,391 @@
[
{
"id": "RocketMQ-Topic",
"configuration": {
"children": [
{
"x": 0,
"y": 48,
"w": 12,
"h": 15,
"i": "0",
"type": "Widget",
"id": "0",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Bar",
"showBackground": true
},
"expressions": [
"meter_rocketmq_topic_producer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"title": "Producer Message Size(MB)",
"tips": "The size of messages produced per second",
"name": "producer_message_size"
}
},
{
"x": 12,
"y": 48,
"w": 12,
"h": 15,
"i": "1",
"type": "Widget",
"id": "1",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Bar",
"showBackground": true
},
"expressions": [
"meter_rocketmq_topic_consumer_message_size/1024/1024"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer_message_size",
"title": "Consumer Message Size(MB)",
"tips": "The size of messages consumed per second "
}
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 9,
"i": "2",
"type": "Widget",
"id": "2",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"metricConfig": [
{
"unit": "MB"
}
],
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_max_consumer_message_size)/1024/1024"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "max_consumer_message_size",
"title": "Max Consumer Message Size",
"tips": "The maximum number of messages consumed"
},
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 0,
"y": 0,
"w": 6,
"h": 9,
"i": "3",
"type": "Widget",
"id": "3",
"metricTypes": [],
"metrics": [],
"metricConfig": [
{
"unit": "MB"
}
],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_max_producer_message_size)/1024/1024"
],
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "max_producer_message_size",
"title": "Max Producer Message Size",
"tips": "The maximum number of messages produced"
},
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
}
},
{
"x": 12,
"y": 35,
"w": 12,
"h": 13,
"i": "5",
"type": "Widget",
"id": "5",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_latency/1000"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "consumer_latency",
"title": "Consumer Latency(s)",
"tips": "Consumption delay time of a consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 22,
"w": 12,
"h": 13,
"i": "6",
"type": "Widget",
"id": "6",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_producer_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"widget": {
"title": "Producer Offset",
"name": "producer_offset",
"tips": "The progress of a topic's production message"
}
},
{
"x": 12,
"y": 22,
"w": 12,
"h": 13,
"i": "7",
"type": "Widget",
"id": "7",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_group_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"title": "Consumer Group Offset",
"name": "consumer_group_offset",
"tips": "The progress of a topic's consumption message per consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 9,
"w": 12,
"h": 13,
"i": "8",
"type": "Widget",
"id": "8",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_consumer_group_tps"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"title": "Consumer Group Tps",
"name": "consumer_group_tps",
"tips": "The number of messages consumed per second per consumer group"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 0,
"y": 9,
"w": 12,
"h": 13,
"i": "9",
"type": "Widget",
"id": "9",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"meter_rocketmq_topic_producer_tps"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"title": "Producer Tps",
"name": "producer_tps",
"tips": "The number of messages produced per second"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
}
},
{
"x": 12,
"y": 0,
"w": 6,
"h": 9,
"i": "10",
"type": "Widget",
"id": "10",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"expressions": [
"latest(meter_rocketmq_topic_consumer_group_count)"
],
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
},
"typesOfMQE": [
"SINGLE_VALUE"
],
"widget": {
"name": "consumer group count",
"title": "Consumer Group Count",
"tips": "The number of consumer groups"
}
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 9,
"i": "11",
"type": "Widget",
"id": "11",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Card",
"fontSize": 40,
"textAlign": "center",
"showUnit": true
},
"expressions": [
"latest(meter_rocketmq_topic_broker_count)"
],
"typesOfMQE": [
"UNKNOWN"
],
"widget": {
"name": "broker count",
"title": "Broker Count",
"tips": "The number of topics that received messages from the producer"
}
},
{
"x": 0,
"y": 35,
"w": 12,
"h": 13,
"i": "12",
"type": "Widget",
"id": "12",
"metricTypes": [],
"metrics": [],
"metricMode": "Expression",
"moved": false,
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": true,
"showXAxis": true,
"showYAxis": true
},
"expressions": [
"meter_rocketmq_topic_producer_offset-meter_rocketmq_topic_consumer_group_offset"
],
"typesOfMQE": [
"TIME_SERIES_VALUES"
],
"widget": {
"name": "backlogged messages",
"title": "Backlogged Messages",
"tips": "The number of backlogged messages in each group"
}
}
],
"layer": "ROCKETMQ",
"entity": "Endpoint",
"name": "RocketMQ-Topic",
"id": "RocketMQ-Topic"
}
}
]

51
oap-server/server-starter/src/main/resources/ui-initialized-templates/virtual_mq/virtual-mq-service.json
View File

@ -12,7 +12,7 @@
"i": "1",
"type": "Widget",
"widget": {
"title": "Consume Traffic (calls / min)"
"title": "Consumer Traffic (calls / min)"
},
"graph": {
"type": "Line",
@ -32,7 +32,7 @@
"metricConfig": [
{
"unit": "calls / min",
"label": "Consume Traffic"
"label": "Consumer Traffic"
}
],
"associate": [
@ -64,7 +64,7 @@
"i": "2",
"type": "Widget",
"widget": {
"title": "Consume Access Successful Rate (%)",
"title": "Consumer Access Successful Rate (%)",
"name": "Successful_Rate"
},
"graph": {
@ -221,7 +221,7 @@
"i": "5",
"type": "Widget",
"widget": {
"title": "Produce Traffic (calls / min)"
"title": "Producer Traffic (calls / min)"
},
"graph": {
"type": "Line",
@ -241,7 +241,7 @@
"metricConfig": [
{
"unit": "calls / min",
"label": "Produce Traffic"
"label": "Producer Traffic"
}
],
"associate": [
@ -273,7 +273,7 @@
"i": "6",
"type": "Widget",
"widget": {
"title": "Produce Access Successful Rate (%)",
"title": "Producer Access Successful Rate (%)",
"name": "Successful_Rate"
},
"graph": {
@ -355,19 +355,19 @@
"label": "Transmission Latency"
},
{
"label": "Consume Successful Rate",
"label": "Consumer Successful Rate",
"unit": "%"
},
{
"label": "Consume Traffic",
"label": "Consumer Traffic",
"unit": "calls / min"
},
{
"label": "Produce Successful Rate",
"label": "Producer Successful Rate",
"unit": "%"
},
{
"label": "Produce Traffic",
"label": "Producer Traffic",
"unit": "calls / min"
}
]
@ -378,7 +378,36 @@
"name": "Virtual-MQ-Service",
"id": "Virtual-MQ-Service",
"isRoot": false,
"isDefault": true
"isDefault": true,
"expressions": [
"avg(mq_service_consume_latency)",
"avg(mq_endpoint_produce_sla)/100",
"avg(mq_endpoint_consume_sla)/100",
"avg(mq_service_produce_cpm)",
"avg(mq_service_consume_cpm)"
],
"expressionsConfig": [
{
"unit": "ms",
"label": "Consumer Latency"
},
{
"unit": "%",
"label": "Producer Successful Rate"
},
{
"unit": "%",
"label": "Consumer Successful Rate"
},
{
"unit": "calls / min",
"label": "Producer Traffic"
},
{
"unit": "calls / min",
"label": "Consumer Traffic"
}
]
}
}
]

117
test/e2e-v2/cases/rocketmq/docker-compose.yml Normal file
View File

@ -0,0 +1,117 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: "3"
services:
oap:
extends:
file: ../../script/docker-compose/base-compose.yml
service: oap
ports:
- "12800:12800"
networks:
- e2e
rocketmq-namesrv-1:
image: apache/rocketmq:4.9.4
hostname: rocketmq-namesrv-1
ports:
- "9876:9876"
environment:
TZ: Asia/Shanghai
command: sh mqnamesrv
healthcheck:
test: [ "CMD", "nc", "-nz", "127.0.0.1", "9876" ]
interval: 15s
timeout: 10s
retries: 120
networks:
- e2e
rocketmq-broker-1:
image: apache/rocketmq:4.9.4
hostname: rocketmq-broker-1
ports:
- "10911:10911"
- "10909:10909"
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh mqbroker
healthcheck:
test: [ "CMD", "nc", "-nz", "127.0.0.1", "10911" ]
interval: 15s
timeout: 10s
retries: 120
depends_on:
rocketmq-namesrv-1:
condition: service_healthy
networks:
- e2e
rocketmq-producer-perf-test:
image: apache/rocketmq:4.9.4
hostname: rocketmq-producer-perf-test
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh tools.sh org.apache.rocketmq.example.quickstart.Producer
depends_on:
rocketmq-broker-1:
condition: service_healthy
networks:
- e2e
rocketmq-consumer-perf-test:
image: apache/rocketmq:4.9.4
hostname: rocketmq-consumer-perf-test
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "rocketmq-namesrv-1:9876"
command: sh tools.sh org.apache.rocketmq.example.quickstart.Consumer
depends_on:
- rocketmq-producer-perf-test
networks:
- e2e
rocketmq-exporter:
image: apache/rocketmq-exporter:0.0.2
hostname: rocketmq-exporter
ports:
- "5557:5557"
environment:
TZ: Asia/Shanghai
rocketmq.config.namesrvAddr: "rocketmq-namesrv-1:9876"
depends_on:
- rocketmq-consumer-perf-test
networks:
- e2e
otel-collector:
image: otel/opentelemetry-collector:${OTEL_COLLECTOR_VERSION}
networks:
- e2e
command: [ "--config=/etc/otel-collector-config.yaml" ]
volumes:
- ./otel-collector-config.yaml:/etc/otel-collector-config.yaml
expose:
- 55678
depends_on:
oap:
condition: service_healthy
networks:
e2e:

37
test/e2e-v2/cases/rocketmq/e2e.yaml Normal file
View File

@ -0,0 +1,37 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
setup:
env: compose
file: docker-compose.yml
timeout: 20m
init-system-environment: ../../script/env
steps:
- name: set PATH
command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
- name: install yq
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
- name: install swctl
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
verify:
retry:
count: 60
interval: 3s
cases:
- includes:
- ./rocketmq-cases.yaml

19
test/e2e-v2/cases/rocketmq/expected/endpoint.yml Normal file
View File

@ -0,0 +1,19 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains .}}
- id: {{ b64enc "rocketmq::rocketmq-cluster" }}.1_{{ b64enc "TopicTest" }}
name: TopicTest
{{- end}}

22
test/e2e-v2/cases/rocketmq/expected/instance.yml Normal file
View File

@ -0,0 +1,22 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ notEmpty .id }}
name: rocketmq-broker-1
instanceuuid: {{ notEmpty .instanceuuid }}
attributes: []
language: UNKNOWN
{{- end }}

35
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-empty-label.yml Normal file
View File

@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
{{- contains .metric.labels }}
- key: _
value: ""
{{- end}}
values:
{ { - contains .values } }
- id: { { notEmpty .id } }
value: null
traceid: null
- id: { { notEmpty .id } }
value: { { .value } }
traceid: null
{ { - end } }
{{- end}}
error: null

33
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-group-label.yml Normal file
View File

@ -0,0 +1,33 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
- key: _
value: please_rename_unique_group_name_4
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

35
test/e2e-v2/cases/rocketmq/expected/metrics-has-value-label.yml Normal file
View File

@ -0,0 +1,35 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels:
{{- contains .metric.labels }}
- key: _
value: {{ .value }}
{{- end}}
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

31
test/e2e-v2/cases/rocketmq/expected/metrics-has-value.yml Normal file
View File

@ -0,0 +1,31 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
type: TIME_SERIES_VALUES
results:
{{- contains .results }}
- metric:
labels: []
values:
{{- contains .values }}
- id: {{ notEmpty .id }}
value: {{ .value }}
traceid: null
- id: {{ notEmpty .id }}
value: null
traceid: null
{{- end}}
{{- end}}
error: null

24
test/e2e-v2/cases/rocketmq/expected/service.yml Normal file
View File

@ -0,0 +1,24 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ b64enc "rocketmq::rocketmq-cluster" }}.1
name: rocketmq::rocketmq-cluster
group: rocketmq
shortname: rocketmq-cluster
layers:
- ROCKETMQ
normal: true
{{- end }}

45
test/e2e-v2/cases/rocketmq/otel-collector-config.yaml Normal file
View File

@ -0,0 +1,45 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
receivers:
prometheus:
config:
scrape_configs:
- job_name: "rocketmq-monitoring"
scrape_interval: 30s
static_configs:
- targets: ['rocketmq-exporter:5557']
relabel_configs:
- source_labels: [ ]
target_label: cluster
replacement: rocketmq-cluster
exporters:
otlp:
endpoint: oap:11800
tls:
insecure: true
processors:
batch:
service:
pipelines:
metrics:
receivers:
- prometheus
processors:
- batch
exporters:
- otlp

89
test/e2e-v2/cases/rocketmq/rocketmq-cases.yaml Normal file
View File

@ -0,0 +1,89 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
cases:
# service cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
expected: expected/service.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_produced_today --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_consumed_today --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_total_producer_tps --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_total_consumer_tps --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_producer_message_size --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_consumer_message_size --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_produced_until_yesterday --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_messages_consumed_until_yesterday --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_max_consumer_latency --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_max_commitLog_disk_ratio --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_commitLog_disk_ratio --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_pull_threadPool_queue_head_wait_time --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_send_threadPool_queue_head_wait_time --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_topic_count --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-empty-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_cluster_broker_count --service-name=rocketmq::rocketmq-cluster
expected: expected/metrics-has-value-empty-label.yml
# instance cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance ls --service-name=rocketmq::rocketmq-cluster
expected: expected/instance.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_produce_tps --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_consume_qps --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_producer_message_size --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_broker_consumer_message_size --service-name=rocketmq::rocketmq-cluster --instance-name=rocketmq-broker-1
expected: expected/metrics-has-value.yml
# endpoint cases
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql endpoint ls --service-name=rocketmq::rocketmq-cluster
expected: expected/endpoint.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_max_producer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_max_consumer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_latency --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_tps --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_tps --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_offset --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_offset --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_producer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_message_size --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-group-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_consumer_group_count --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-empty-label.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics exec --expression=meter_rocketmq_topic_broker_count --service-name=rocketmq::rocketmq-cluster --endpoint-name=TopicTest
expected: expected/metrics-has-value-empty-label.yml