Build your first pipeline

This guide follows the current Control Plane workflow: configure a source, configure one or more targets, name the pipeline, add entities, and start replication.

Before you start

You need:

  • A running Gluesync deployment. If you do not have one, start with Get Gluesync and Install with the kit.

  • A Control Plane account that can create pipelines and entities.

  • Network access and credentials for the source and target systems.

  • The source-side prerequisites from the relevant connector guide. CDC often requires database configuration beyond a normal login.

1. Create the pipeline

  1. Sign in to the Control Plane.

  2. Open Pipelines.

  3. Select Pipeline (the button with the plus icon).

The pipeline wizard has three stages: Setup source, Setup target(s), and Ready.

2. Set up the source

  1. Select the source technology. If several capture methods are available, select the flavor that matches your database setup.

  2. Enter the connection and agent settings.

  3. Open Advanced settings only when the connector guide or your environment requires an override.

  4. Select Test and continue.

The test must pass before the wizard moves on. For the required database permissions and CDC setup, use the connector-specific Source setup guide.

3. Set up the targets

  1. Select a target technology.

  2. Enter its connection settings.

  3. Select Test and continue.

  4. Add another target if the same source data must be delivered to more than one system.

Gluesync can deploy a connector during this workflow when the deployment includes Modules Manager. If no compatible connector is available, see Configure connectors.

4. Name the pipeline

On Ready, enter a recognizable pipeline name and select Finish. Use a name that identifies the data flow, for example orders-postgres-to-snowflake.

The pipeline opens with no entities.

5. Add entities

  1. Select Entities.

  2. In Objects Browser, choose the source tables or collections to replicate.

  3. Review the proposed target names and field mappings.

  4. Create the entities.

An entity is one source object and its target mapping. See Select data in Objects Browser for bulk selection and Fields Editor for mapping changes.

6. Start replication

  1. Select one or more entities.

  2. Choose the action that matches the goal:

    • Start CDC continuously captures new changes. You can include an initial snapshot when prompted.

    • Snapshot copies the current data without leaving CDC running.

    • Pause stops an active entity without deleting its configuration.

  3. Review the run settings and confirm.

  4. Watch the entity status and snapshot progress.

Do not use a second snapshot to recover blindly from an error. Read Pipeline states and error recovery and Snapshot tasks first.

Verify the result

  • Confirm that the entity becomes active and its row count increases.

  • Compare representative records in the source and target.

  • Check the entity’s end-to-end latency and last update timestamp.

  • Review the Notifications Hub for warnings.

Put the pipeline into service

  1. Configure smart alerts.

  2. Take a configuration backup.

  3. Record the connector permissions and any advanced settings used.

  4. Follow Move to production before using the pipeline for production data.