Data Filtering

Data filtering enables granular control over which data gets synchronized, allowing you to define specific criteria so that only rows matching your rules are replicated to the target.

On-the-fly data filtering feature

You can access this feature from the Objects Browser via the Settings dialog. The Settings dialog includes an on-the-fly data filtering feature that enables users to define inclusion rules for incoming data feeds.

Filters are inclusive: a row is replicated to the target only if it matches all defined conditions.

Snapshot: only matching rows are copied to the target.
CDC: if a row no longer matches the filter (e.g. a field value changes), the corresponding operation is converted to a DELETE on the target, keeping the target consistent with the filter at all times.

Filters do not consume resources at source datastore level; they are applied at the Gluesync platform level, after the data is fetched from the source and before it is written to the target.

How to use

Here’s a step-by-step guide to setting up data filtering:

Setup entity

First, choose tables/collections and set up columns for a new entity to replicate

From the Objects Browser, open the Settings dialog and navigate to the dedicated section named Filter out incoming data. This section allows you to define filtering rules.

Select columns for filtering

Define filtering operators

For each selected column, you can choose one of the following operators:

Operator

Description

=

Equal to

>

Greater than

<

Less than

>=

Greater than or equal to

⇐

Less than or equal to

!=

Not equal to

in

Value exists in a list

Not in

Value does not exist in a list

Is null

Value is null

Not null

Value is not null

Regex

Use regular expressions like Dan.* to filter data similarly to a SQL LIKE Dan% clause

Specify rule values

In the field values section, define the specific rule to apply. For example, if you choose the > operator, enter the value that the data must exceed.

Example use case

Scenario: You want to replicate only records where the ID column is greater than 20.

Steps:

Select the ID column from the dropdown.
Choose the > operator.
Enter 20 in the field values section.

With this filter active:

Snapshot: only rows with ID > 20 are copied to the target table.
CDC: changes to rows with ID > 20 are replicated normally. If an existing row is updated so that its ID becomes ≤ 20, that row is deleted from the target.

The label "Filter out incoming data" in the UI refers to filtering out rows that do not match your condition — not filtering out the rows that do match. Think of it as: "keep only what passes the filter".

Another example: equality filter

Scenario: You want to replicate only orders where ORDER_STATUS = 1.

Select the ORDER_STATUS column.
Choose the = operator.
Enter 1 in the field values section.

With this filter:

Rows with ORDER_STATUS = 1 are replicated to the target.
If a row’s ORDER_STATUS changes from 1 to 2, Gluesync sends a DELETE to the target for that row.
New rows inserted with ORDER_STATUS = 2 are ignored entirely.