Entities: a key concept

Entities are a key concept in the Gluesync world. Here we’re guiding you on the core areas that need to be covered when learning how Gluesync models data to reflect your expected data model.

Virtual entities

What is a virtual entity? A virtual entity in Gluesync is a feature that enables our replication engine to understand and map any given source object to a JSON model (and vice versa) that will be the result of the execution of the given filter/query/mapping against your source database.

Entities are not a new concept for those who come from the relational database world.

Where are Legacy entities?

Legacy entities have been deprecated from version 1.5. If you want to learn more about the reason why we dropped the support for Legacy entities please take a look at the changelog page of version 1.5.

Applications

Before jumping on how you can configure Gluesync with your entities we’d like to provide a bit of context around the level of application of what you’ll learn here following.

The entities you will declare in your configuration have a broad context of application they are shared across each of the layers that compose the Gluesync architecture, by stating that we mean that each entity will apply in:

every snapshot operation you intend to run;
every incoming data changes feed that will hit Gluesync source connectors.

Basically whatever you define at source entities level maps the entire journey of your data when passing through the defined data pipeline.

Entity types

Gluesync offers different types of Entities, each of which has a specific functionality and business logic triggered by the defined statements you’re configuring via the Gluesync config.json file.

Following in the next section we’re covering the fundamentals of each type.

Full table

This entity is the basic entity that comes with Gluesync. The full table takes the whole content of a source entity (collection/table/set, …) and moves it to the target database without applying any additional transformation on top of the sourced data other than inserting in the destination model a key that identifies the source model called type where inside you’ll find your custom entity name given during the configuration.

To define at the configuration level a Full table entity you can follow the example presented here below:

"MyCustomEntityName": {
    "table": "CUSTOMERS",
    "schema": "MYSCHEMA",
    "type": "customers",
    "scope": "myscope"
},

Where:

table: points Gluesync to the source table/collection where it needs to source data from;
schema: points Gluesync to the source schema/scope/set where your table belongs from;
type: every modeled document stored on the target database will have a key called type with the given custom value;
scope (optional): will be used as source/target scope in NoSQL databases supporting this kind of data tenancy, like Couchbase for example.

Mapping

Referred also using Field skipping and/or Field renaming, similar to Full table it helps you with a data modeling feature that skips and/or renames column / key names. Given the names of your columns/keys mapped with the respective new names it will:

skip all the non-declared column / key entries;
rename key/column entries matching the desired new values.

To make use of that feature your entity declaration has to follow the example presented here below:

"MyCustomEntityName": {
    "mapping": {
        "id": "DRIVERID",
        "name": "DRIVERNAME",
        "surname": "DRIVERSURNAME"
    },
    "table": "DRIVERS",
    "schema": "MYSCHEMA",
    "type": "drivers",
    "scope": "myscope"
}

Where:

table: points Gluesync to the source table/collection where it needs to source data from;
schema: points Gluesync to the source schema/scope/set where your table belongs from;
type: every modeled document stored on the target database will have a key called type with the given custom value;
scope (optional): will be used as source/target scope in NoSQL databases supporting this kind of data tenancy, like Couchbase for example.

Query

Also known as SQL query data modeling is an important feature announced along with version 1.3. To learn deeply how this entity works we have a full coverage made available at this link.

In order to make use of this powerful entity you will be required to declare:

query: a key that tells the engine to treat this object for SQL query DM;
- SELECT …: the query statement to be executed by Gluesync’s query engine against your source database.

"MyCustomEntityName": {
    "query": "SELECT ID, latitude, longitude FROM LOCATIONS INNER JOIN ... WHERE ...",
    "type": "orders",
    "scope": "myscope"
}

Where:

type: every modeled document stored on the target database will have a key called type with the given custom value;
scope (optional): will be used as source/target scope in NoSQL databases supporting this kind of data tenancy, like Couchbase for example.

You don’t need to declare here table and schema since these two keys are automatically discovered by the query engine.

Data modeling

Also known as Advanced data modeling is also one of the latest announced features, it came with version 1.4. It is well covered in 2 specific sections of this documentation, respectively for RDBMS data sources and for NoSQL data sources. Even if they do not share many differences we have differentiated them for further needs.

We invite you to check the respective documentation pages to learn more about how to define a data modeling entity that can then help you achieve your business goals.

In order to make use of this powerful entity you will be required to declare:

dataModeling: a key that tells the engine to treat this object for Advanced DM;
- […]: the meta definition statement to be executed by Gluesync’s query engine against your source database.

To declare a Data modeling entity you can follow this example here below:

"MyCustomEntityName": {
    "dataModeling": [...],
    "type": "orders",
    "scope": "myscope"
}

Where:

type: every modeled document stored on the target database will have a key called type with the given custom value;
scope (optional): will be used as source/target scope in NoSQL databases supporting this kind of data tenancy, like Couchbase for example.

You don’t need to declare here table and schema since these two keys are automatically discovered by the query engine.