Schedule validation

Chronos can validate selected entities on a schedule, from a Core Hub platform event, from an incoming webhook, or as a step in a chained event. The Control Plane action is Validate. The API task type is entity_validate.

The same form appears on Scheduler → Schedules (main action and chained events) and Scheduler → Events (one action per event). Saving the form stores the action. It does not start a comparison.

What runs, and where

Chronos does not compare rows itself. For each selected entity it calls Core Hub:

  1. POST /data-comparison/runs with pipelineId and entityId. Core Hub resolves the source agent, the target agent, and the primary key. See Validator.

  2. Chronos polls GET /data-comparison/runs/{jobId} until the run is terminal.

  3. When the run found differences and Reconcile differences is off, the Chronos job fails. The error text is the breakdown (only in source, only in target, mismatching rows, schema differences) plus the note that reconciliation is disabled. That text is stored as the job’s last_error_message and is what Chronos sends on its webhook and email failure path.

  4. When Reconcile differences is on, Chronos calls POST /data-comparison/runs/{jobId}/reconciliation with { "reconcileAll": true }. The job succeeds when every difference was written. It fails when Core Hub skipped a row or reported schema differences it could not rewrite.

Entities of one job run one after another. The first failure fails the job. Later entities in that fire are not started.

Core Hub also emits gluesync.validation.* platform events for the run itself, including when Chronos is the caller. Subscribe to those separately if you want the validation payload in addition to the Chronos job-failure notification. See Validator platform events.

Target: pipeline and entities

A Validate action is entity-scoped. The form hides groups. At least one entity is required. group_ids are ignored.

Every action needs:

  • a pipeline

  • one or more entities on that pipeline

Reconcile differences

The toggle is stored as validation_reconcile (boolean, default false).

Toggle Chronos behavior when the run finds differences

Off (Report only)

The job fails. The error lists how many rows were only in the source, only in the target, mismatching, or a schema difference. Nothing is written to the target.

On (Reconcile)

Chronos asks Core Hub to reconcile every remaining difference. The job fails only when something could not be reconciled (skipped or schema differences that were not rewritten).

A run that finishes COMPLETED (no differences) succeeds in both modes. A run that finishes FAILED or CANCELLED fails the Chronos job in both modes.

API fields

On a scheduled job, a chained event, and a trigger-flow event:

{
  "task_type": "entity_validate",
  "pipeline_id": "pipeline-id",
  "entity_ids": ["entity-id"],
  "validation_reconcile": false
}

entity_ids must contain at least one id when task_type is entity_validate. validation_reconcile omitted or null is stored as false.

The internal Chronos route is POST /api/pipelines/{pipeline_id}/validate with body { "entity_ids": ["…"], "validation_reconcile": false }. The crontab runner uses the same route.

Timeouts

Comparison runs are longer than a play or pause. Two environment variables on the Chronos process bound the wait:

Variable Default Meaning

SCHEDULER_VALIDATION_TIMEOUT_SECONDS

3600

How long Chronos waits for one entity run to reach a terminal status.

SCHEDULER_VALIDATION_POLL_SECONDS

2

Delay between GET /data-comparison/runs/{jobId} polls.

The HTTP call from the scheduler to its own API uses validation timeout × entity count + 30 seconds, so a job with several entities is not cut off at the default internal timeout.

Chained events and trigger flows

A chained event with action Validate uses the same entity picker, the same toggle, and the same entity_validate task type. validation_reconcile is stored on the chained event row.

A trigger-flow event is the same shape. Platform-event and webhook triggers do not wait for a separate Chronos completion event for this task type: the HTTP response of the validation call is the result. A failed entity fails that event and the failure message is the difference breakdown when reconciliation is off.

Failure text

When reconciliation is off and the run found differences, the message looks like:

Validation of entity <id> found N difference(s): X only in source, Y only in target, Z mismatching rows, W schema differences. Reconciliation is disabled for this job.

Several entities that fail in one request are joined with ` | `.