Entity import file reference

Applied

Sets the entity setting from the value.

Checked

Applies the value when it can. A value the import cannot apply is reported: it blocks the row when leaving it out would change the replicated data, and is a warning otherwise.

Ignored, with a warning

Does not apply the key. The finding says where to set it after the import.

Not applied, with an information finding

Does not apply the key, which has no effect in this case, and says so with an information finding.

Blocks the row

The row cannot be added. Either the row is Unsupported and only a change to the file fixes it, or it is an Error that you can fix under the row. See Fix a row before adding it.

Replaced by the pipeline

Uses the pipeline’s own value instead, with an information finding.

Silent

Skips the key without a finding. These are keys the Automator export writes as metadata.

name

text

Applied: the target table or collection. Absent, it has the source table’s name. The name is kept as written, dots included, with one exception: when name differs from the source table and its last dotted part is the key of the entry, the prefix is dropped, so name: FILELIB.SECMSTPM under the key SECMSTPM gives SECMSTPM.

targetSchema

text

Applied: the target schema of this table. It wins over the schema’s target.

entityName

text

Applied: the name of the entity. Absent, the entity is named <source schema>.<source table>. An entity name the pipeline already has makes the row Already exists. Two tables of the file that give one entity name are an error.

sourceTableName

text

Applied: the source table that an entry whose key is not a table name configures. When no key names that table itself, the first such entry configures its entity, named as usual. Every further entry for the same table, typically a key of the form <table>@@<n> such as orders@@2, configures another entity of it, named <target table> - <source schema>.<source table> unless entityName says otherwise. A @@ key without sourceTableName is an error. Under a key that is itself the name of a table, a sourceTableName that names another table is ignored, with a warning.

keys

list of column names

Applied: the source keys of the entity, as in Logical keys. Absent, the source table’s primary key is used. When the source table has a primary key, the entity uses all of it: keys may list some or all of its columns, and a column outside the primary key is an error. On a source table without a primary key, whose target table has one, the keys are the source columns that feed the target’s primary key, and keys that say otherwise are an error.

targetKeys

list of column names

Applied on relational targets. On a target table with a primary key, targetKeys must name that key, or the row is an error that you can fix under it. On an existing target table without a primary key, they mark the key columns as logical keys: a name that is not a column of the table is an error that you can fix under the row, and a marked column fed by a single source column makes that source column a key too, with an information finding. On a table the import creates, they become its primary key. On a non-relational target they are not applied, with an information finding: the keys follow the source’s.

columns

list of column rows

Applied: the column mapping. Absent, every source column is mapped under its own name. Present, only the listed columns are copied. See Columns and relations.

targetOnlyColumns

list of {name, type, dataLength, numericPrecision, numericScale, isNullable}

Applied to a table the import creates: extra columns that no source column feeds. Every entry needs all six properties; an entry without them blocks the row. On an existing target table the key is ignored, with a warning.

whereClause

text

Applied as the source table’s WHERE clause, when the source connector supports one. Otherwise it is ignored, with a warning. See Apply SQL WHERE clauses at source database level.

filter

{clauses: [{column, operation, value}]}

Checked: the entity’s data filter. See Filters.

snapshotDeleteFilter

as filter

Checked: the rows deleted from the target before a snapshot. See Filters and Snapshot delete filter by target.

documentKey

{prefix, suffix, separator, keys}, or one column name

Checked. On a NoSQL target it becomes the entity’s custom document key: prefix and suffix default to empty text, separator to -, and documentKey: <column> means {keys: [<column>]}. A document key without key columns, or on a column the entity does not copy, is an error on the row. On other targets the key has no effect, with an information finding.

groupId, group

text

Applied: the entity’s group. groupId is read first; when it is empty, group is read. See Groups and Chains.

chainId

text or number

Applied: the table becomes a member of a chain, one chain per chainId value and schema. A chain reads its settings in its own way: see Settings of a chain and Chains.

customProperties

{source: {…​}, target: {…​}}

Checked: the entity settings. See Custom properties.

snapshotWriteMethod

UPSERT or INSERT

Checked. UPSERT is silent. Any other value is not applied, with a warning: choose how the snapshot writes when you start the entity.

orderIndex

number

Ignored, with a warning: the entity keeps the default order.

schedules

list

Ignored, with a warning: the import does not create the entity’s schedules. Add them to the entity in Schedules and events after the import.

unlockedSchema

flag

true blocks the row: the row is Unsupported. false is silent. See Unlock Schema.

PostgreSQL

Applies the filter. Regex and IsNotNull are errors that you can fix under the row: the database cannot run them, and the snapshot would fail before copying anything.

SQL Server

Applies the filter. Regex, IsNull, IsNotNull, In and NotIn are errors that you can fix under the row.

Other relational targets, including CockroachDB, YugabyteDB, Supabase, CedarDB and Greenplum

Apply the filter. Regex, IsNotNull, In and NotIn are errors that you can fix under the row.

Databricks, Elasticsearch, Redis Streams, Fabric Open Mirroring

Error: these targets ignore the delete filter and empty the whole target before the snapshot. Remove the filter under the row; to keep the other rows, keep the existing rows when you start the snapshot.

Snowflake, Amazon Redshift, Vertica, Informix, GridGain

Apply the filter, as other relational targets do. When the pipeline’s source is NoSQL, they empty the whole target instead, and the filter is an error.

BigQuery, DynamoDB, Couchbase, MongoDB, Cosmos DB, Aerospike, RavenDB, Redis, Kafka, Google Pub/Sub, Solace, Amazon S3, Azure Data Lake, Google Cloud Storage, Universal File Store

Not applied, with an information finding: the connector does not run delete filters, so the rows the filter matches are not deleted before the snapshot.

Other targets, such as CouchDB, HBase and Courier

Not checked: the import gives no finding.

snapshotWritingConcurrency absent

Sends 1.

Sends 2, the New Entity page’s default; 1 for a chain, as the Automator does.

Column names that differ by case only

Compares keys with the column names ignoring case, and matches a field function’s target column ignoring case.

Keeps id and ID apart everywhere. A name in another case matches only when a single column fits; otherwise the row reports it.

Types of a created table

Maps the source type to the target’s default type for the same Gluesync data type; uses the default text type only for a source type it cannot map.

Keeps the source type when the target declares it.

MAX lengths on a created table

Creates varchar(max) as varchar(1024) when one table of the file feeds the target table.

Keeps MAX on a type of varying length.

Default length on a created table

Gives 1024 to a text or binary column with no length, which some targets refuse for char or binary.

Gives 1024, capped at the longest the target allows. Both tools keep a length the file states.

NOT NULL on a column nothing writes

Creates it NOT NULL when the file says isNullable: false, and by default for a targetOnlyColumns entry.

Creates it nullable.

columns starting with a bare name

Ignores the list and copies every column.

Copies only the listed columns, with a warning.

keys with the primary key plus other columns

Accepts the list, with a warning.

Refuses it: Core Hub refuses it too.

targetOnlyColumns with missing properties

Creates the column of the new table with defaults: varchar, length 1024, NOT NULL.

Blocks the row until every property is given.

Unknown operations in allowedOperations

Passes them on.

Accepts any case, and blocks the row on an unknown name.

partitions count

Any whole number from 1 up; 10 when absent.

1 to 10; a column alone means 10. A count above 10 is a warning, and the entity is saved without partitions.

udf in a schema’s customProperties

Applies it to every table of the schema.

Blocks every table of the schema without a udf of its own: an entity links its own UDF. The finding asks you to move udf under the table.

Write policies (changedColumnsPolicy, nullValuePolicy, absentFieldPolicy, columnWritePolicies)

Passes them on to the entity.

The row is Unsupported unless the value is Core Hub’s default.

Tables whose names start with sys

Skips them, even when the whitelist names them.

Skips them only when there is no whitelist.

A schema’s custom next to tables.custom

Reads tables.custom only.

Reads both; tables.custom wins.

CustomFn with no Custom Field Functions published

Warns.

Error on the row.

Custom Field Functions in a ZIP

Replace the pipeline’s.

Never replace an installed collection.

Filter clause type

Uses it as the column type when the column is not found.

Does not read it.

Filter operation aliases (<, lt, LessThan, eq, >= and others) and an operator key

Reads them as Core Hub’s operation names.

Refuses them: write operation with Core Hub’s names. See Filters.

partitions written on the table, outside customProperties

Reads it, as it reads customProperties.source.partitions.

Ignores it, with an unknown-key warning. Write it under customProperties.source.

Chains from a source without chain support

Does not check the source.

The chain is Unsupported. See Chains.

Schedules

Creates them while scheduling is on, which is the default.

Not created: see Limitations.

unlockedSchema: true

Unlocks the target schema of the entity when the table has a udf; without one, it skips the table with an error.

The row is Unsupported.

snapshotWriteMethod

Not applied when entities are created; the Bootstrapper uses it when it starts the snapshot.

Not applied, with a warning when it is not UPSERT: choose how the snapshot writes when you start the entity.