Monitoring Kafka Debezium Connector metrics using Prometheus

Описание

Debezium - Stream changes from your database.
Monitor Apache Kafka Debezium Connector using Prometheus in Docker.

The Debezium PostgreSQL connector provides two types of metrics that are in addition to the built-in support for JMX metrics that Zookeeper, Kafka, and Kafka Connect provide.

Snapshot metrics provide information about connector operation while performing a snapshot.

Streaming metrics provide information about connector operation when the connector is capturing changes and streaming change event records.

Debezium monitoring documentation provides details for how to expose these metrics by using JMX.

Snapshot metrics
The MBean is debezium.postgres:type=connector-metrics,context=snapshot,server=postgresql.server.name.

Snapshot metrics are not exposed unless a snapshot operation is active, or if a snapshot has occurred since the last connector start.

Streaming metrics
The MBean is debezium.postgres:type=connector-metrics,context=streaming,server=postgresql.server.name.

Source code: https://github.com/waqasdilawar/CDC_mechanism_using_Kafka_Postgres_Docker_elasticsearch