Validator

Overview

Validator compares one replicated entity with the table or collection it is written to. It reports rows that exist only on the source, rows that exist only on the target, rows whose values differ, and schema differences. From the same screen you can write the source values back to the target (reconcile) or leave the differences in place and alert on them.

Open it from the Control Plane navigation, from an entity’s overflow menu (Validate), or from the entity detail Inspect tables actions. Those links open /data-comparison with pipelineId and entityId already selected.

Validator compares one entity at a time. A pipeline-wide check is a sequence of entity runs, which is how Chronos schedules it.

What a run compares

Each run walks the source and the target in the same key order and records:

Difference Meaning

MISSING_IN_TARGET

The key exists on the source and not on the target. Reconciliation inserts the source row.

MISSING_IN_SOURCE

The key exists on the target and not on the source. Reconciliation deletes the target row.

ROW_MISMATCH

The key exists on both sides and at least one compared column differs. Reconciliation updates the target from the source.

SCHEMA_MISMATCH

A column cannot be compared (missing, or a type that cannot be aligned). Schema differences are counted and are not rewritten as row fixes.

A finished run is one of:

Status Meaning

PENDING

Accepted, not started.

RUNNING

Scanning.

COMPLETED

Finished with no differences.

COMPLETED_WITH_WARNINGS

Finished, and at least one difference was stored.

FAILED

The run stopped on an error. errorCode and errorMessage are on the summary.

CANCELLED

Stopped before completion. Cancelling does not emit a terminal validation event.

Run it from the Control Plane

  1. Open Validator and pick the pipeline and entity. The source agent, target agent, and primary key are taken from the entity.

  2. Optionally narrow the compared columns, add a WHERE filter per side, or set ordering expressions when the key type needs an explicit cast.

  3. Start the run. The summary shows rows scanned and the four difference counters.

  4. Open a difference to see the key and the column values. Mismatched cells are marked on the row.

  5. Select differences and reconcile them, or reconcile the whole run. Counters update as fixes are applied.

Reconcile writes to the target. It does not change the source.

REST API

All routes are on Core Hub. The caller’s role still applies (view, view differences, reconcile, cancel).

Start a run

POST /data-comparison/runs

The smallest body is the pipeline and the entity:

{
  "pipelineId": "pipeline-id",
  "entityId": "entity-id"
}

When sourceAgentId, targetAgentId, or comparisonKey is omitted or blank, Core Hub fills them from the entity: the source and target agents registered on that entity, and the primary-key columns of the source entity. The Control Plane sends those fields explicitly. Automation (Chronos, MCP) can omit them.

Optional fields:

  • columns — non-key columns to compare. Default is the mapped columns.

  • columnMapping — source column name to target column name, when the names differ.

  • options — fetchSize (default 10000, max 50000), maxRuntimeSeconds (default 3600, max 86400), maxEmittedDifferences (default 10000, max 50000), countOnly, string and timestamp comparison modes, and reconciliation toggles (reconcileInserts, reconcileDeletes, reconcileUpdates).

  • queryOptions — whereConditionSource, whereConditionTarget, orderBySource, orderByTarget. Ordering values are comma-separated SQL expressions, one per key column, with no ASC, DESC, or NULLS.

The response is { "jobId", "status": "PENDING", "startedAt" }.

Read a run

  • GET /data-comparison/runs — list runs. Optional entityId query parameter.

  • GET /data-comparison/runs/{jobId} — one summary: status, row counts, missingInSourceCount, missingInTargetCount, rowMismatchesCount, schemaDifferencesCount, duplicateKeysCount, timestamps, errorCode, errorMessage.

  • GET /data-comparison/runs/{jobId}/differences?limit=&offset= — one page of differences. Each record has id, differenceType, keyJson, and columnDiffsJson.

  • DELETE /data-comparison/runs/{jobId} — cancel a PENDING or RUNNING run, or delete a finished run.

Reconcile

POST /data-comparison/runs/{jobId}/reconciliation

{ "reconcileAll": true }

reconcileAll fixes every remaining difference of that run. Core Hub pages the stored differences itself. Use this from Chronos or any client that should not walk difference ids.

To fix a selection instead:

{
  "differenceIds": ["diff-id"],
  "columns": ["status"]
}

comparisonKey may be omitted. The key saved with the run is used when it is present. columns limits which mismatched columns are written; omitted means every compared column.

The response is insertedCount, deletedCount, updatedCount, and skippedCount. A difference is removed only after its write succeeds. A skipped row stays in the run.

Platform events

Every run emits CloudEvents that webhook subscriptions, email notifications, and in-app notifications can select. The source of the event is the entity.

Event Type Severity When

DATA_VALIDATION_STARTED

gluesync.validation.started

INFO

The run was accepted.

DATA_VALIDATION_COMPLETED

gluesync.validation.completed

INFO

Status COMPLETED. Source and target match.

DATA_VALIDATION_DIFFERENCES_FOUND

gluesync.validation.differences-found

WARNING

Status COMPLETED_WITH_WARNINGS.

DATA_VALIDATION_FAILED

gluesync.validation.failed

CRITICAL

Status FAILED. The message includes errorMessage when Core Hub has one.

DATA_VALIDATION_RECONCILED

gluesync.validation.reconciled

INFO, or WARNING when skippedCount is greater than zero

A reconcile call inserted, deleted, updated, or skipped rows.

Terminal event data contains jobId, status, sourceRowsCount, targetRowsCount, sourceRowsScanned, targetRowsScanned, missingInSourceCount, missingInTargetCount, rowMismatchesCount, schemaDifferencesCount, duplicateKeysCount, totalDifferences, startedAt, and, when present, endedAt, errorCode, and errorMessage.

The reconciled event data contains jobId, insertedCount, deletedCount, updatedCount, skippedCount, and reconcileAll.

Subscribe to these types from Settings → Notifications or a webhook configuration. The event catalog is served by Core Hub (get_webhook_event_types on the MCP server lists the same names).

A Chronos job that validates with reconciliation turned off fails the job and stores the difference breakdown as the job error. That failure is what Chronos sends on its own webhook and email path. The platform events above are emitted by Core Hub whether or not Chronos is the caller. See Schedule validation.

MCP tools

The embedded MCP server exposes six Validator tools. They call the REST routes above with the caller’s token.

Tool Category What it does

trigger_data_comparison

WRITE

Starts a run. pipeline_id and entity_id are required. source_agent_id, target_agent_id, and comparison_key are optional.

list_data_comparison_runs

READ

Lists runs, optionally filtered by entity_id.

get_data_comparison_run

READ

Returns one run summary by job_id.

list_data_comparison_differences

READ

Returns one page of differences (limit, offset).

reconcile_data_comparison_differences

WRITE

Reconciles difference_ids, or every remaining difference when reconcile_all is true.

cancel_data_comparison_run

WRITE

Cancels a running job or deletes a finished run.

These six tools are part of the 125-tool built-in catalog. See Core Hub MCP server.