Commit Graph

446 Commits

Author SHA1 Message Date
mrproliu 31e0ecacdb
Add OTLP Tracing support as a Zipkin trace input (#11287) 2023-09-01 17:12:58 +08:00
kezhenxu94 572c195928
Do not start server and Kafka consumer in init mode (#11278) 2023-08-29 22:24:09 +08:00
kezhenxu94 16f940b5ce
Check results in ALS as per downstream/upstream instead of per log (#11265) 2023-08-27 12:20:35 +08:00
kezhenxu94 f63764cd33
Fix the wrong default value of `k8sServiceNameRule` if it's not explicitly set (#11200) 2023-08-09 15:01:46 +08:00
Brandon Fergerson c5c268b653
refactor: rename recevier to receiver (#11001) 2023-06-27 08:51:35 +08:00
kezhenxu94 a23f905ce0
Add Istio ServiceEntry registry to resolve unknown IPs in ALS (#10936) 2023-06-15 20:11:19 +08:00
吴晟 Wu Sheng 1f6ea1d744
Begin 9.6.0 iteration (#10935)
* [maven-release-plugin] prepare release v9.5.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-9.5.0.md

* Set changes.md for 9.6.0
2023-06-14 18:26:50 +08:00
mrproliu 43668b7655
Add cause message field when query continuous profiling task (#10919) 2023-06-13 11:51:56 +08:00
Marc Navarro 5dd91181a1
Consider 4xx as client side error for envoy ALS based metrics (#10905) 2023-06-13 10:41:21 +08:00
kezhenxu94 1726040a04
Migrate from kubernetes-client/java to fabric8 client (#10917) 2023-06-13 10:00:23 +08:00
mrproliu a7bb50e63e
Adapt the continuous profiling task query GraphQL (#10878) 2023-06-02 22:14:40 +08:00
kezhenxu94 fc2eced32c
Add OpenTelemetry log protocol support (#10808) 2023-05-18 11:46:42 +08:00
Wan Kai 505d55a1c8
Fix potential NPE in Zipkin receiver when the Span is missing some fields. (#10791) 2023-05-11 21:29:42 +08:00
kezhenxu94 d874c211f6
Clean up OpenCensus related codes and docs (#10755) 2023-05-04 11:13:17 +08:00
pg.yang 8c6bc1b4c5
Support no-proxy mode for aws-firehose receiver (#10684)
* upgrade armeria to 1.23.1

* Support no-proxy mode for aws-firehose receiver
2023-04-15 18:54:47 +08:00
pg.yang b7f456d29f
Support Amazon API Gateway monitoring (#10645) 2023-04-05 12:19:46 +08:00
innerpeacez 8604ac36ce
Delete duplicated dependency (#10640) 2023-04-05 06:14:33 +08:00
mrproliu 7dcdb279ef
Support continuous profiling feature (#10473) 2023-03-20 20:26:42 +08:00
吴晟 Wu Sheng bfa1f7e8cf
Support to bind TLS status as a part of component for service topology. (#10522)
* Support to bind TLS status as a part of component for service topology.

* Fix component ID priority bug.

* Fix wrong priority for process relation.

* Add a UT for topology build

* Fix component ID of topology overlap due to storage layer bugs.
2023-03-10 14:31:47 +08:00
吴晟 Wu Sheng 4874374ace
Tag 9.4.0 release, prepare for 9.5.0 iteration. (#10512) 2023-03-08 23:30:37 +08:00
kezhenxu94 96b6690e24
Bump up snakeyaml to 2.0 (#10495) 2023-03-07 18:15:54 +08:00
pg.yang 4274b6ec5e
Polish aws-firehose-receiver to support setting accessKey (#10484) 2023-03-05 00:36:11 +08:00
叶梦飞 f93a98d4c7
Adapt otel exponential histogram data (#10449) 2023-02-28 12:16:22 +08:00
叶梦飞 155d231111
Support process Sum metrics with AGGREGATION_TEMPORALITY_DELTA case (#10410) 2023-02-20 07:18:36 +08:00
吴晟 Wu Sheng eda83dca93
Remove TLS support from firehose receiver (#10409)
* Remove TLS support from firehose receiver

Notice, no HTTPS/TLS setup support. By following AWS Firehose request, it uses [proxy request](https://en.wikipedia.org/wiki/Proxy_server#Web_proxy_servers) (`https://...` instead of `/aws/firehose/metrics`), there must be a proxy(Nginx, Envoy, etc.).

* Update docs.
2023-02-19 23:25:59 +08:00
kezhenxu94 383a5bbf7b
Migrate tests from junit 4 to junit 5 (#10382) 2023-02-13 18:54:32 +08:00
kezhenxu94 974cecb67f
Add missing tcp throughput metrics (#10356) 2023-02-07 18:38:54 +08:00
pg.yang b991cfd125
Polish aws-firehose-receiver to adapt existing OTEL proto (#10343) 2023-02-05 17:47:48 +08:00
pg.yang 29556a0d00
Add aws-firehose-receiver to support collecting AWS CloudWatch metric(OpenTelemetry format) (#10300) 2023-01-24 14:04:27 +08:00
pg.yang e360ca3628
Remove handler scan in otel receiver, manual initialization instead (#10303) 2023-01-23 16:10:23 +08:00
kezhenxu94 4df62b6210
Fix TCP service instances are lack of instance properties like `pod` and `namespace`, which causes Pod log not to work for TCP workloads (#10252) 2023-01-09 21:08:21 +08:00
kezhenxu94 210c374976
Fix memory leak in Zipkin API (#10126) 2022-12-07 22:25:03 +08:00
吴晟 Wu Sheng 18008aa6b2
Release 9.3.0. Begin 9.4.0 iteration (#10077) 2022-12-02 21:19:13 +08:00
mrproliu 2ac671cc1b
Fix wrong time bucket on attached event (#10049) 2022-11-30 10:22:52 +08:00
Wan Kai 6a75de5ece
Add `@BanyanDB.TimestampColumn` to identify `which column in Record` is providing the timestamp(milliseconds) for BanyanDB (#10019) 2022-11-24 21:20:36 +08:00
Marc Navarro 281be8e3c6
Fix java docs that linked to old classes that doesn't exist anymore (#9989) 2022-11-19 07:27:27 +08:00
mrproliu 3122697e3a
Support span attached event concept (#9916) 2022-11-09 11:28:42 +08:00
pg.yang f38aad5fa5
Optimize MQ Topology analysis. (#9911)
Use entry span's peer from the consumer side as source service when no producer instrumentation(no cross-process reference).
2022-11-06 17:24:26 +08:00
soander 42f3396b64
Support the telegraf receiver plugin module (#9620)
* The telegraf receiver plugin module development.

* Refactored code of converting telegraf data and fixed some errors.

* Support Telegraf receiver plugin module.

* Add telegraf-receiver.md, update changes.md and vm-monitoring.md.

* Rename YAML file and change code style.

* Change receiver-telegraf of application.yml.

* The receiver-telegraf e2e test.

* The telegraf receiver e2e test.

* Fix an issue and delete redundant configs.

* Add Unit Test about converting Telegraf metrics.

* Adjust Unit Test about converting Telegraf metrics.

* Add License Header to Unit Test and change binary.xml.

* Change module provider's config initialization mechanism.

* Exclude the telegraf-rules in server-starter pom.xml.

* Fix telegraf e2e test issues.

* Change telegraf e2e test.

* Fix issues

* Fix issues.

* Add Sample convert Unit Test.

* Change vm.yaml, related documents and fix some issues.

* Change backend-vm-monitoring.md.

* Fix SampleConvertTest checkstyle issue.

* Change vm.yaml swap MAL.

* Update menu.yml and binary.xml.

* Update Telegraf Unit test.

* Delete telegraf config package, use meter.analyzer.prometheus package to load config file.

* Reorder telegraf metrics in menu.yml.

* Change e2e, vm, config, linux-service and vm.md.

* Update telegraf.conf file.

* Change url of telegraf.conf file.

* Update e2e.yaml, conf file, menu.yml and delete useless code of provider.

* Update grouping sampleFamily by timestamp and name, and add new UTs.

* Update vm-monitoring.md and .asf.yaml.

Co-authored-by: Superskyyy (ONLINE) <Superskyyy@outlook.com>
Co-authored-by: 吴晟 Wu Sheng <wu.sheng@foxmail.com>
2022-11-05 08:08:20 +08:00
yswdqz e497efd245
Change the way of loading MAL rules (#9822) 2022-11-01 15:11:04 +08:00
kezhenxu94 fb2ac0e7f9
Split TCP related service mesh metrics protobuffer definition (#9851) 2022-10-28 09:14:38 +08:00
Marc Navarro 7f9f792229
Do not log grpc envoy receiver client-side streaming cancellation as errors (#9843) 2022-10-26 07:32:34 +08:00
吴晟 Wu Sheng 9d3982db74
Add new config initialization mechanism of module provider to replace existing (#9828) 2022-10-22 19:07:53 +08:00
Wan Kai d98775790a
Support export `Trace` and `Log` through Kafka. (#9817) 2022-10-21 08:04:18 +08:00
kezhenxu94 5abe6ceb1f
Improve Kubernetes Pod lister with field selector (#9795) 2022-10-18 13:02:07 +08:00
kezhenxu94 956a9ec973
Lazy load the Kubernetes metadata and switch from event-driven to polling (#9747) 2022-10-11 23:38:00 +08:00
Wan Kai f1e0558ce3
Limit the max length of trace and log tag's `key=value` (#9741) 2022-10-08 16:12:48 +08:00
吴晟 Wu Sheng ba84b8af3e
Support skywalking-client-js report empty `service version` and `page path` (#9735) 2022-10-07 21:09:24 +08:00
pg.yang bbe0a7c4c2
Support analyzing cache related spans to provide metrics and slow commands for cache services from client side. (#9622)
* Support analyzing cache related spans to provide metrics and slow commands for cache services from client side
* Optimize virtual database, fix dynamic config watcher NPE when default value is null 
* [UI] Add virtual cache dashboard
2022-09-19 22:10:32 +08:00
吴晟 Wu Sheng 608a084350
Begin 9.3.0 iteration, 9.2.0 release is on vote (#9539)
* [maven-release-plugin] prepare release v9.2.0

* [maven-release-plugin] prepare for next development iteration

* Create changes-9.2.0.md

* Update changes.md

* Update LICENSE
2022-08-31 00:31:04 +08:00
kezhenxu94 c8a78149a6
Enable otlp OC handler by default and polish doc (#9499) 2022-08-22 16:09:59 +08:00
kezhenxu94 f96b3eacf1
Adapt OpenTelemetry native metrics protocol (#9497) 2022-08-21 21:13:23 +08:00
kezhenxu94 e3d091ebd5
Add Kubernetes support module to share codes between modules and reduce calls to Kubernetes API server (#9493) 2022-08-20 15:24:44 +08:00
mrproliu b8cf95d165
Add `SUM_PER_MIN`(sumHistogramPercentile) downsampling in MAL. Add eBPF Network Profiling E2E (#9377) 2022-07-28 10:00:36 +08:00
mrproliu 5ea7c0d778
Support eBPF Network Profiling (#9337) 2022-07-13 22:41:22 +08:00
mrproliu 7bb8c69c67
Add `forEach`, `processRelation` function to MAL Expression, and add `expPrefix` and `initExp` in MAL config (#9299) 2022-07-04 20:59:34 +08:00
Wan Kai e346b618ea
Support Zipkin kafka collector. (#9252) 2022-06-21 20:40:43 +08:00
kezhenxu94 735ee00b99
Mesh: fix only last rule works when multiple rules are defined in metadata-service-mapping.yaml (#9205) 2022-06-11 21:06:11 +08:00
Wan Kai 43d65775f0
Add more entities for Zipkin to improve performance (#9198) 2022-06-10 20:04:11 +08:00
吴晟 Wu Sheng 8584ada7cc
Begin 9.2.0 iteration (#9191)
* [maven-release-plugin] prepare release v9.1.0

* [maven-release-plugin] prepare for next development iteration
2022-06-08 17:24:15 +08:00
Wan Kai f0aef53d49
Support Zipkin traces collect and zipkin traces query API. (#9177) 2022-06-08 07:40:36 +08:00
kezhenxu94 8d34c17871
Add on demand logs support to service mesh (#9171) 2022-06-06 13:16:55 +08:00
mrproliu 47509715f5
Support OFF CPU eBPF Profiling (#9107) 2022-05-20 14:46:16 +08:00
mrproliu 91b94cfc10
Adapt the process procotol update (#9069) 2022-05-16 16:30:56 +08:00
mrproliu 9494fb2545
Remove layer from process and remove duration from eBPF Profiling scheduler query (#9034) 2022-05-11 17:15:09 +08:00
Wan Kai 2e7cf5db10
Remove `Layer` concept from `ServiceInstance`. (#9029) 2022-05-11 09:59:33 +08:00
kezhenxu94 7344714ee9
Remove unnecessary `onCompleted` on gRPC error (#9027) 2022-05-10 19:37:03 +08:00
kezhenxu94 59c63d90e8
Add Envoy internal cost metrics (#9020) 2022-05-10 12:53:38 +08:00
Wan Kai 26d8c36e3a
VM monitoring adapt the latest version `0.50.0`of the `opentelemetry-collector` (#9013) 2022-05-09 16:54:05 +08:00
mrproliu 1285f5b492
Fix saving Process Traffic error when cluster mode (#9009) 2022-05-08 14:51:15 +08:00
Wan Kai 6c6ef24664
Support autocomplete tags in logs query. (#8999) 2022-05-06 20:18:45 +08:00
mrproliu 48a5b8cc59
Add the support eBPF profiling field into the process entity (#8976) 2022-04-30 17:31:31 +08:00
Superskyyy 211efc0562
Further remove jetty references and config (#8974) 2022-04-29 14:32:44 +08:00
Wan Kai 12ba7aa533
Support autocomplete tags in traces query (backend side), Replace all configurations **_JETTY_** to **_REST_** (#8971)
* Support autocomplete tags in traces query (backend side)

* [Breaking Change] Replace configuration `**_JETTY_**` to `**_REST_**`.
2022-04-29 12:52:45 +08:00
mrproliu a93667184f
Support receive Kubernetes process from the Rover (#8960) 2022-04-27 19:20:50 +08:00
Wan Kai 70560e3b2d
HTTPServer support the handler register with allowed HTTP methods (#8917) 2022-04-21 21:16:51 +08:00
Hoshea Jiang 8247bc3c71
Add layer field to event (#8881) 2022-04-19 16:49:52 +08:00
mrproliu db3c1da10d
Update the eBPF Profiling task as service level (#8840) 2022-04-14 07:30:40 +08:00
Wan Kai f3478b187d
Fix Zipkin receiver: Decode spans error, missing `Layer` for V9 and wrong time bucket for generate Service and Endpoint. (#8848) 2022-04-11 21:34:03 +08:00
kezhenxu94 e38f25e28b
[CI] set up Java explicitly and use the built-in cache mechanism (#8851) 2022-04-11 18:55:39 +08:00
kezhenxu94 118ba3b569
Fix wrong example in doc, sync Python E2E version and adjust E2E (#8751)
* Fix wrong example in doc, sync Python E2E version and adjust E2E
2022-04-07 13:02:21 +08:00
吴晟 Wu Sheng 91413a753c
Start 9.1.0-dev iteration (#8822) 2022-04-06 18:56:49 +08:00
zhang-wei bfdbd55285
Add template of OpenFunction (#8789) 2022-04-02 18:03:43 +08:00
吴晟 Wu Sheng faf6e2eb9e
Add `Error Prone Annotations` dependency to suppress warnings (#8774) 2022-04-01 00:57:34 +08:00
吴晟 Wu Sheng d96eed9145
Fix event type of export data, reduce redundancy ThreadLocal in MAL core. (#8754)
* Fix event type of export data is incorrect, it was `EventType.TOTAL` always.
* Reduce redundancy ThreadLocal in MAL core. Improve MAL performance.
* ExpressionDelegate public->private
2022-03-27 16:53:15 +08:00
吴晟 Wu Sheng d90e2e6ef6
Enhance endpoint dependency analysis to support cross threads cases. (#8747)
* [**Breaking Change**] Remove **Deprecated responseCode** from sources, including Service, ServiceInstance, Endpoint
* Enhance endpoint dependency analysis to support cross threads cases. Refactor span analysis code structures.
* Remove `isNotNormal` service requirement when use alias to merge service topology from client side. All RPCs' peer
  services from client side are always normal services. This cause the topology is not merged correctly.

-----------

In our showcase and Java agent scenarios, there are several cross-thread tracing cases, we were not generating endpoint dependency if ref exists in local and exit span. In this PR, I am adding this missing part back.
Also, in case the downstream endpoint of the dependency is another local/exit span's operation name, which was not generating endpoint traffic before, I add traffic generation of both sides' endpoints of dependency.
Another important point that drives me to fix this is, in the new Booster UI, metrics of endpoint dependency are shown in a separate dashboard, which requires the endpoints of both sides existing in metadata, otherwise, nothing shows up.

For e2e, updated the Java agent to the latest.
2022-03-26 20:00:12 +08:00
Kai 0dfb93a2d1
Insure span analyze isNormal and Layer consistent. (#8724) 2022-03-22 23:48:00 +08:00
mrproliu e4dac66e2f
Implement the eBPF profiling query and data collect protocol (#8696) 2022-03-20 19:38:16 +08:00
kezhenxu94 4c7ac43d05
Polish Zipkin HTTP receiver with Armeria APIs (#8680) 2022-03-15 17:11:59 +08:00
kezhenxu94 289427990a
Migrate the HTTP server from Jetty to Armeria (#8673) 2022-03-15 13:38:42 +08:00
吴晟 Wu Sheng d1f5b58f5c
[Breaking Change] Deprecate `All` from OAL source. (#8656)
* Add source `event` generated metrics to SERVICE_CATALOG_NAME catalog.
* [Breaking Change] Deprecate `All` from OAL source.
* [Breaking Change] Remove `SRC_ALL: 'All'` from OAL grammar tree.
* Remove `all_heatmap` and `all_percentile` metrics.
2022-03-10 22:36:22 +08:00
mrproliu f06c220e50
Introduce the entity of Process Type (#8617) 2022-03-03 20:16:02 +08:00
Kai ff1103ec1d
Unify module name `receiver_zipkin` to `receiver-zipkin`, remove `receiver_jaeger` from `application.yaml`. (#8616) 2022-03-02 22:05:02 +08:00
Kirs 057fdf553e
Use the plugin version defined by the Apache maven parent (#8613) 2022-03-02 09:57:11 +08:00
kezhenxu94 ffba719e2e
Add a new sampler (`possibility`) in LAL (#8591) 2022-02-25 20:44:00 +08:00
Kai f390b14404
Fix Zipkin receiver wrong condition for decoding `gzip` (#8592) 2022-02-25 18:58:13 +08:00
kezhenxu94 810d840027
Sync proto buffers files from upstream Envoy (#8434)
Related https://github.com/envoyproxy/envoy/pull/18955
2022-01-17 10:51:08 +08:00
wankai123 ed35f4e845
Set layer to mesh relation. (#8424) 2022-01-13 16:35:24 +08:00
wankai123 8bcf22f3d8
Support OAP server v9 Core concept: Layer, backend side (#8379) 2022-01-04 16:16:24 +08:00
吴晟 Wu Sheng 8a322e8559
Fix `Can't split endpoint id into 2 parts` bug for endpoint ID in the mesh's TCP case. (#8327) 2021-12-23 09:56:33 +08:00