Data Type Mapping

Overview

Gluesync does not treat every source and target database as if they shared the same type system. Instead, it reads the source value, normalizes it into a Gluesync data family, and then writes the value using the target connector’s supported representation.

In practice, that means:

  • Some values move across unchanged, such as strings, integers, booleans, and byte arrays.

  • Some values are normalized before they are written, especially dates and timestamps.

  • Some source types collapse into a broader target type, for example Oracle NUMBER values becoming big_decimal internally.

  • If the target cannot represent a value safely, the Fields Editor flags it and you must adjust the mapping or add a transformation.

Use the Fields Editor to inspect the inferred target type for each field, and Unlock Schema when you need to change the target type itself.

What happens across different type systems

When source and target systems do not share their native types, Gluesync keeps the best compatible representation it can:

  • Relational to document targets: database columns are converted into values the document store can serialize. For example, Oracle or SQL Server text and numeric types become string or numeric values in MongoDB, while binary columns stay binary.

  • Relational to event/log targets: when the target is Kafka or another streaming sink, Gluesync preserves the value and serializes temporal fields into ISO-8601 strings so downstream consumers can read them consistently.

  • Structured to file targets: when the target is CSV or JSON-based output, values are flattened into a representation that the file format can store.

The rule of thumb is simple: Gluesync prefers a native type when the target supports it, and a safe serialized form when it does not.

General coercion rules

Strings

  • Text-like source types typically stay as strings.

  • Oracle CHAR, VARCHAR2, CLOB, JSON, and similar text columns map to Gluesync string values.

  • SQL Server VARCHAR, NVARCHAR, TEXT, NTEXT, XML, and similar text columns also map to strings.

  • UUID-like values are often handled as strings unless the target connector has a dedicated native type.

Numbers

  • Small integers, integers, and longs are preserved in the closest numeric family the target supports.

  • Fixed-point values such as Oracle NUMBER, SQL Server DECIMAL/NUMERIC, and MONEY are mapped to big_decimal so precision is not lost at the source-normalization stage.

  • Floating-point values remain floating point, but they can still lose precision if the target expects a narrower numeric type.

Booleans

  • Boolean source values remain booleans when the target supports them.

  • If a target has no native boolean type, the value is usually written in the closest format that target accepts.

Dates and times

  • Date-only values stay date-only when the target supports them.

  • Time-only values stay time-only when the target supports them.

  • Timestamp values are normalized into date-time objects, and timezone-aware values are preserved only when the target connector supports offsets.

  • If the target only accepts text, Gluesync serializes date/time values as ISO-8601 strings.

Binary data

  • BLOB, RAW, VARBINARY, IMAGE, and similar binary fields map to byte arrays.

  • If the target is file- or document-based, the binary value is written in the format that target can store directly.

Nulls

  • null stays null.

  • If the target column is required or the target cannot represent the value family, the mismatch is surfaced as a validation problem before deployment.

Example scenarios

Oracle to MongoDB

Oracle is a good example of a source with rich numeric and date semantics:

  • NUMBER, NUMERIC, and DECIMAL columns are treated as precise decimal values.

  • DATE and TIMESTAMP columns become date-time values without inventing a fake timezone.

  • TIMESTAMP WITH TIME ZONE and TIMESTAMP WITH LOCAL TIME ZONE keep their offset-aware shape.

  • BLOB, RAW, and LONG RAW become binary values that MongoDB can store.

On the MongoDB side, Gluesync writes:

  • numbers as numeric BSON values

  • strings as strings

  • binaries as binary

  • date/time values as BSON dates or serialized temporal values depending on the field shape

If you need timezone-preserving behavior, prefer an offset-aware source type and verify the target field is defined accordingly.

MSSQL to Kafka

Kafka is not a SQL database, so the important question is how the payload will be serialized for consumers.

  • SQL Server fixed-point numerics such as DECIMAL and NUMERIC stay as precise decimal values.

  • Binary columns stay as byte arrays.

  • Date/time fields are formatted as ISO-8601 text when written to the Kafka payload.

  • String, numeric, and boolean values are passed through in their native shape when possible.

If a downstream consumer expects JSON, Avro, Protobuf, or plain strings, make sure the serializer and schema match the target mapping. Kafka will faithfully carry the mapped value, but the consumer still has to know how to interpret it.

Common gotchas

  • Precision loss: Oracle NUMBER(38) or SQL Server DECIMAL values can exceed what downstream systems or client libraries want to store in floating-point fields. Keep them as big_decimal unless you deliberately want rounding.

  • Timezone drift: date-only and local date-time values do not carry timezone information. If the target stores UTC timestamps, confirm whether your source value should be treated as local time or as an instant.

  • Binary payload size: BLOB and CLOB-style values can be large. Check target limits, message size limits, and serializer limits before enabling high-volume replication.

  • Document targets are stricter than they look: MongoDB and similar targets still need a concrete field shape. Arrays, documents, and regex-like values must match the target field definition.

  • Text is not always a safe fallback: serializing a value to string can help with compatibility, but it also pushes parsing responsibility downstream.

Inspecting and overriding mappings

The Fields Editor is where you review the inferred mapping for each field.

From there you can:

  • inspect the source and target field list side by side

  • see warnings when the engine expects a safe cast

  • see errors when the target type is incompatible with the source value

  • rename target fields if the downstream contract needs a different name

  • unlock the schema if the target type itself needs to change

  • attach a UDF when you need custom parsing, normalization, or casting logic

If you are unsure what a field will become, check the target-side type shown in the Fields Editor first. That is the clearest indicator of what Gluesync will attempt to write.

Practical tips

  • Check the target type before you deploy, especially for decimal, timestamp, and binary fields.

  • Treat warnings as a signal to verify behavior, not as a guarantee that the conversion is harmless.

  • Use a UDF when the value needs parsing or reshaping, not when you only need a different target column type.

  • Unlock the schema when the target column definition is the real problem.

  • Re-test any mapping that crosses a type family boundary, such as text to date, number to string, or local time to timestamp.

  • For large binary or text columns, validate the target size limits and the end-to-end serialization path.

See also