Before & After Images

Before & after images provide a comprehensive view of data changes by capturing the state of records both before and after modifications, enabling precise synchronization and data validation.

Overview

Before & after images support is a powerful feature that enhances data synchronization accuracy and efficiency. This feature is available for both RDBMS and NoSQL agents, with optimized implementations for each type of database system.

Implementation by database type

Database type	Features
RDBMS Agents	* Field-level comparison at source agent level * Only changed columns are transmitted to target * Updates on non-monitored columns are automatically skipped * Rows with identical before & after values are excluded from replication
NoSQL Agents	* Row-level comparison for efficient synchronization * Automatic skipping of unchanged records * Bandwidth and processing optimization * Ongoing feature improvements and extensions

Database type

Features

RDBMS Agents

* Field-level comparison at source agent level * Only changed columns are transmitted to target * Updates on non-monitored columns are automatically skipped * Rows with identical before & after values are excluded from replication

NoSQL Agents

* Row-level comparison for efficient synchronization * Automatic skipping of unchanged records * Bandwidth and processing optimization * Ongoing feature improvements and extensions

Key benefits

Bandwidth optimization: Only transmit data that has actually changed
Processing efficiency: Skip unnecessary updates when values remain unchanged
Precise change tracking: Field-level granularity for RDBMS systems
Resource conservation: Reduce processing overhead by filtering out unchanged data

Performance impact

Benefit	Description
Reduced network traffic	By transmitting only changed columns, network bandwidth usage is significantly reduced
Lower processing overhead	Skipping unchanged rows eliminates unnecessary processing at both source and target
Optimized storage usage	Efficient handling of data changes reduces storage requirements for change tracking
Improved synchronization speed	Focusing only on actual changes leads to faster overall synchronization

Benefit

Description

Reduced network traffic

By transmitting only changed columns, network bandwidth usage is significantly reduced

Lower processing overhead

Skipping unchanged rows eliminates unnecessary processing at both source and target

Optimized storage usage

Efficient handling of data changes reduces storage requirements for change tracking

Improved synchronization speed

Focusing only on actual changes leads to faster overall synchronization

Setup

While the Control plane UI determines the best option for your specific database automatically, you are required to check for its specific availability in your source database.

To further help you setting the before & after images for your database, we have implemented a set of instructions for each database type.

Check for the following section named Working with Before & After images in the source agent documentation for a guide on how to enable it.

Using Rest APIs

To setup the before & after images for a specific agent, you can use the following Rest API snippet:

curl --location --request PUT 'http://core-hub-ip-address/pipelines/{pipelineId}/entities/' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '[
  {
    "entity": {
      "entityId": "cb64f725",
      "entityName": "dbo.DRIVERS",
      "agentEntities": [
        {
          "type": "SingleTable",
          "entityId": "cb64f725",
          "entityName": "dbo.DRIVERS",
          "agentEntityId": "0b7d244d",
          "entityType": {
            "type": "Source",
            "maxFetchItemsCountPerIteration": 1000,
            "maxTransactionMessageKbSize": 1024,
            "pollingIntervalMilliseconds": 100,
            "unchangedDataFilterType": "ENTIRE_ROW"  // Can be set to ENTIRE_ROW, FIELDS or DISABLED
          },
          "agentId": "92f6e98d",
          "tablesProperties": {
            "dbo.DRIVERS": {}
          },
          "table": {
            "name": "DRIVERS",
            "schema": "dbo"
          },
          "columns": [
            {"name": "ID", "alias": "ID", "type": "int"},
            {"name": "FIRST_NAME", "alias": "FIRST_NAME", "type": "varchar"}
          ],
          "keys": [
            {"name": "ID", "alias": "ID", "type": "int"}
          ]
        },
        {
          "type": "NoSqlEntity",
          "entityId": "cb64f725",
          "entityName": "dbo.DRIVERS",
          "agentEntityId": "9dcc0fad",
          "entityType": {
            "type": "Target",
            "skipDeletion": false
          },
          "agentId": "a021dd8e",
          "customProperties": {},
          "tablesProperties": {
            "dbo.DRIVERS": {}
          },
          "entityObject": {
            "scope": "dbo",
            "collection": "DRIVERS"
          },
          "columns": [
            {"name": "ID", "alias": "ID", "type": "int"},
            {"name": "FIRST_NAME", "alias": "FIRST_NAME", "type": "string"}
          ],
          "keys": [
            {"name": "ID", "type": "int"}
          ]
        }
      ]
    }
  }
]'

Accepted params for the unchangedDataFilterType are:

ENTIRE_ROW: Filters data only if the entire row is unchanged;
FIELDS: Filters data based on changed columns/fields values;
DISABLED: Disables the functionality.