Prometheus Integration

Overview

Prometheus serves as the primary metrics collection and storage system for Gluesync. This integration provides real-time monitoring capabilities with configurable retention and storage options.

For a complete list of available metrics and their descriptions, see the Metrics Reference.

Default Configuration

Gluesync provides a pre-configured Prometheus setup through Docker Compose:

services:
  prometheus:
    image: prom/prometheus
    restart: 'unless-stopped'
    volumes:
      - ./prometheus-gs2.yml:/etc/prometheus/prometheus.yml
      - ./data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.retention.size=10GB'
      - '--storage.tsdb.retention.time=7d'
    ports:
      - 9090:9090

Configuration Options

Core Settings

Setting Default Description

retention.size

10GB

Maximum storage size for metrics data. Increase for larger deployments or longer history.

retention.time

7d

Duration to retain metrics. Adjust based on your compliance and analysis needs.

scrape_interval

15s

Frequency at which Prometheus samples Core Hub’s current metric values.

ports

9090

Prometheus web interface and API port. Can be modified for security or port conflicts.

The scrape_interval setting controls Prometheus sampling frequency; it does not control how often Core Hub updates its internal measurements. For example:

  • A 15s scrape interval stores one sample approximately every 15 seconds

  • A 5s scrape interval provides finer dashboard resolution but increases scrape and storage load

  • A 30s scrape interval reduces load but can hide short-lived changes

Choose an interval that balances your monitoring needs with system performance.

Security Considerations

HTTPS Configuration

To enable HTTPS for secure metrics collection:

command:
  - '--web.config.file=/etc/prometheus/web-config.yml'

volumes:
  - ./web-config.yml:/etc/prometheus/web-config.yml
  - ./certificates:/etc/prometheus/certs

Example web-config.yml:

tls_server_config:
  cert_file: /etc/prometheus/certs/prometheus.crt
  key_file: /etc/prometheus/certs/prometheus.key

Core Hub Address Configuration

To modify the Core Hub scrape target:

  1. Edit prometheus-gs2.yml:

    scrape_configs:
  - job_name: 'gluesync'
    static_configs:
      - targets: ['Core Hub:8080']  # Default
      # - targets: ['custom-address:port']

Available Metrics

Core Hub 2.2.11 exposes:

  • Per-entity replication snapshots, moving averages, and cumulative row/byte totals

  • Per-entity source lag and cache lag

  • CDC checkpoint and entity-freshness metrics when the corresponding readers and monitors are active

  • Smart-alert configuration, including entity-heartbeat watchdog settings

  • Live database connection gauges and conditional HikariCP pool metrics

  • Host CPU, RAM, uptime, network, and interrupt metrics

  • Standard JVM memory, garbage-collection, buffer, and thread metrics

  • Ktor request concurrency, duration, count, and status metrics

Some families are conditional and do not appear until the corresponding feature has registered data. For a complete list, wire types, labels, units, and safe query guidance, see the Metrics Reference.

Performance Tuning

Storage Optimization

For high-throughput environments:

command:
  - '--storage.tsdb.retention.size=200GB'  # Increased storage
  - '--storage.tsdb.retention.time=30d'   # Extended retention
  - '--storage.tsdb.wal-compression'      # Compress write-ahead log
  - '--query.max-samples=100000000'       # Larger query limit

Scrape Configuration

Adjust scrape intervals based on your needs:

global:
  scrape_interval: 15s     # Default collection interval
  scrape_timeout: 10s      # Timeout for each scrape
  evaluation_interval: 15s  # Rule evaluation interval
For production environments, consider increasing scrape_interval to reduce system load, or decreasing it for more granular metrics.

The official Core Hub Grafana dashboard refreshes every 5 seconds. A refresh faster than the Prometheus scrape interval can redraw the same sample several times; set both values deliberately when you need finer-grained diagnosis.

Troubleshooting

Common Issues

  • Connection Refused: Verify Core Hub address and port configuration

  • Data Storage Full: Adjust retention settings or increase storage capacity

  • High Memory Usage: Consider reducing retention period or increasing query timeout

Monitoring Prometheus Itself

Enable self-monitoring:

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']