Search docs.molo17.com

    Custom Field Functions

    Custom Field Functions let you write Java functions once and reuse them throughout a pipeline. A function can fill one target column from several source columns, be called by any entity or UDF in the pipeline, and be tested before it processes replicated data.

    How-to guides

    For a step-by-step walkthrough tested on a running Gluesync, see the how-to guides Write and test a UDF for common conversions and Choose a key strategy: physical, logical, RRN or computed.

    Overview

    Field functions provide ready-made conversions, but they are limited to the transformations in the catalog. A UDF gives you full access to a record, but belongs to one entity.

    Custom Field Functions cover the space between them. They provide pipeline-level custom logic that can be shared by many entities. When the logic changes, you update it once instead of maintaining a copy for every entity.

    Custom Field Functions editor showing the Java code editor and Try a function panel
    Figure 1. Custom Field Functions editor with a Java class and the test panel

    A typical example

    Your source stores a date in one column and a time in another, while the target expects one timestamp in the plant’s timezone. Many entities need the same rule.

    Write the function once:

    package com.molo17.gluesync.collections;
    
    import java.time.*;
    import com.molo17.gluesync.commons.function.GsFunction;
    
    public class Format {
    
        @GsFunction(
            name = "compose_timestamp",
            description = "A date column and a time column into one timestamp"
        )
        public static OffsetDateTime composeTimestamp(
                LocalDate date,
                LocalTime time,
                String zone) {
            if (date == null) {
                return null;
            }
            LocalTime effective = (time == null) ? LocalTime.MIDNIGHT : time;
            return date.atTime(effective)
                    .atZone(ZoneId.of(zone))
                    .toOffsetDateTime();
        }
    }

    On each entity, bind the target column to compose_timestamp and choose the source columns or fixed values that feed its parameters. Different entities can use the same function with different inputs and timezones.

    When to use what

    Use When Example

    Field function

    The transformation is already in the catalog.

    Format a date as text, trim a string, or mask an email.

    Custom Field Function

    You need your own logic for one target column, particularly when the logic is reused or takes several inputs.

    Combine several source columns into one target column.

    UDF

    You need to work on the whole record: add or remove fields, skip an operation, or react to the operation type.

    Turn a delete into a soft delete, filter records, or enrich several columns at once.

    A UDF can call a Custom Field Function like any other Java method. Keep shared logic in Custom Field Functions and entity-specific record handling in the UDF.

    Key capabilities

    • Write once, reuse throughout the pipeline: every entity and UDF in the pipeline can call the same function.

    • Several columns in, one column out: a function can accept as many parameters as needed.

    • Safe publishing: Gluesync recompiles every pipeline UDF and re-checks every bound column before accepting a new version.

    • Versioning: each successful publication creates a version that can be restored.

    • Testing: run an installed function on sample values before binding it to a column.

    • Failure protection: repeatedly failing functions are suspended so they cannot fail on every record indefinitely.

    Authoring functions

    One set per pipeline

    Custom Field Functions belong to a pipeline, not to an individual entity. A pipeline has one set of Custom Field Functions, which can contain multiple Java classes. The classes are compiled together and can call one another.

    Exporting a function

    A class can contain supporting code, but only functions explicitly exported with @GsFunction can be selected in the Fields Editor or called as exported functions. An exported method must be public static:

    @GsFunction(name = "shout", description = "Uppercases a value")
    public static String shout(String value) {
        return value == null ? null : value.toUpperCase();
    }
    Rule Why

    The method must be public static

    The function is shared by every entity in the pipeline and can be called concurrently.

    name must be unique

    This is the name shown in the Fields Editor and used by UDFs.

    description is optional but recommended

    It helps users choose the correct function.

    If name is omitted, Gluesync uses the Java method name.

    Working with several classes

    Use New class to add a Java file. All classes are compiled together, while only annotated functions are exported.

    • A file is named after its declared class.

    • Two classes cannot have the same name.

    • The backend package is com.molo17.gluesync.collections. Keep the generated package declaration unless your deployment requires a different package.

    Logging

    Functions can write to the Core Hub log:

    import org.slf4j.Logger;
    import com.molo17.gluesync.commons.function.GsLog;
    
    public class Format {
    
        private static final Logger LOG = GsLog.logger("Format");
    
        @GsFunction(name = "compose_timestamp")
        public static String composeTimestamp(String date, String time) {
            if (date == null) {
                LOG.warn("no date to compose from");
                return null;
            }
            return date + "T" + time;
        }
    }

    Log lines are attributed to the pipeline, agent, and entity that produced them. See Logging.

    Publishing and installing

    Publishing and installing are separate steps:

    Step What it does

    Compile and publish

    Checks the code and its consumers, then stores a new version. The running pipeline is not changed.

    Install

    Makes the published version the version executed by the pipeline.

    In the editor, the Compile action starts publication. Installing stops the pipeline’s entities, replaces the executable code, and starts the entities again. Gluesync reports which entities were restarted. If the pipeline is idle, no running replication is interrupted.

    Publishing alone does not change pipeline behavior. Until you select Install, the pipeline continues to run the previously installed version.

    Publication checks

    Gluesync publishes only when all of these checks pass:

    1. All Custom Field Function classes compile.

    2. Every UDF in the pipeline recompiles against the proposed version.

    3. Every target column bound to a Custom Field Function remains valid.

    The UDF check catches removed functions and incompatible signatures before they can fail for every record. The column check validates configuration that the Java compiler cannot inspect, such as parameter counts and target types.

    On a pipeline with many UDFs, publication can take minutes. Progress remains available if you leave the page and return. Only one publication can run for a pipeline at a time.

    If publication is refused

    Nothing changes when a publication is refused. The response identifies the affected code or configuration:

    Reported as Meaning

    Compilation errors

    The Java code does not compile. Gluesync reports the file, line, and column.

    UDFs that no longer compile

    A UDF calls code that the proposed version removed or changed incompatibly.

    Columns that would break

    A bound target column no longer matches its function, for example because the parameter count changed.

    Versions and rollback

    Every successful publication creates a numbered version containing both the compiled result and its source code. The most recent versions are retained.

    From the versions panel you can:

    • Roll back to a previous version, restoring its code to the editor and making it the published version.

    • Delete a version that is no longer needed.

    A rollback runs the same compatibility checks as a publication. It can be refused if, for example, a newer UDF depends on a function absent from the older version. The installed version cannot be deleted.

    After a rollback, install the restored version when you are ready for the pipeline to execute it.

    Configuring a target column

    Before you start

    The pipeline must have a published and installed Custom Field Functions version. If it has never been installed, its functions do not appear in the Fields Editor.

    1. Open the replication task in the Objects Browser and go to the Fields tab.

    2. Locate the target column to fill. If it does not exist, add it with Add field where the target lets you add fields, or add the column to the target table first. See Adding fields.

    3. Open Select Expression and choose the required Custom Field Function.

    4. Bind each parameter to a source column or fixed value.

    5. Save the entity.

    Only functions whose declared result type fits the target column are offered.

    Composing a column from several source columns

    To fill one target column from several source columns, draw an arrow from each source column to the target column in the Fields Editor. On such an N:1 target:

    • The picker offers only the functions with one parameter per source column, and their arguments are bound to those source columns in their source order.

    • Add a custom field function opens the editor with a new function already declared on those source columns, named after the target column. Once you publish and install it, it is bound to the target column.

    Binding arguments

    Arguments are bound by position: the first argument feeds the first parameter, the second argument feeds the second parameter, and so on. The function signature determines the number of arguments.

    Type Meaning

    Source column

    The value held by that column in the current record. If the column is absent from the change, the function receives null.

    Fixed value

    A constant used for every record, such as a timezone or country code.

    A fixed value reaches the function as a String, exactly as entered; it is not converted.

    For a non-String parameter, bind a source column that supplies the correct type. A fixed value such as 2026-09-05 does not satisfy a LocalDate parameter.

    Missing and previous values

    A source change does not always contain every column. Handle null explicitly:

    @GsFunction(name = "compose_timestamp")
    public static String composeTimestamp(String date, String time) {
        if (date == null) {
            return null;
        }
        return date + "T" + (time == null ? "00:00:00" : time);
    }

    On databases that provide before-images, Gluesync also composes the previous target value from the previous values of the bound columns. If none of those columns has a previous value, as with an insert, the composed previous value is null and the function is not called. See Before and after images.

    Column validation

    Gluesync validates a binding when you save the entity and whenever a new Custom Field Functions version is published.

    Check What it means

    Function and parameter count

    The selected function exists and the configured argument count matches its signature.

    Result type

    The declared return type fits the target column. Compatible types in the same family are accepted, such as int for a bigint column.

    Writable result

    A function returning void or an unsupported type such as java.math.BigInteger cannot fill a column. Use a supported type such as BigDecimal.

    Not a key column

    A Custom Field Function cannot fill a primary-key column because a calculated key cannot be reconstructed reliably for updates and deletes.

    Testing a function

    Select an exported function in Try a function, enter one sample value for each parameter, and run it to inspect the result. The test uses the same execution path as replication.

    Testing requires an installed version because tests run against the code the pipeline can execute. After publishing a new version, install it before testing that version.

    Sample text is parsed according to each declared parameter type:

    Parameter type Sample

    String

    The text itself

    short, int, long

    42

    float, double, BigDecimal

    1.5

    boolean

    true or false

    LocalDate

    2026-09-05

    LocalTime

    10:15:00

    LocalDateTime

    2026-09-05T10:15:00

    OffsetTime

    10:15:00+02:00

    OffsetDateTime

    2026-09-05T10:15:00+02:00

    byte[]

    A base64-encoded value

    Empty

    null

    If a parameter type cannot be entered as text, test the function from a UDF with a value of that type.

    Monitoring

    For each called function, Gluesync records failure counts, unusually slow calls, and whether the function is suspended.

    A function is suspended automatically after repeated failures. While suspended, it is not called and the target columns it composes are left untouched. Fix the function, publish a new version, and install it to restore processing.

    Suspensions are also reported in the Notifications and alerts. Function log messages are available in the Core Hub logs.

    Removing Custom Field Functions

    You can remove Custom Field Functions from a pipeline only when no target column uses them. If bindings remain, Gluesync refuses the removal and lists each entity, column, and function to update first.

    Permissions

    Action Who can do it

    View code and versions

    Anyone who can view the pipeline.

    Publish, install, roll back, delete versions, or remove all Custom Field Functions

    Users with permission to manage pipeline-level custom functions. The MONITOR role does not receive this permission because the change can affect every entity in the pipeline.

    Test a function or bind it to a column

    Users who can modify an entity.

    REST APIs

    The REST API retains function-collection in endpoint paths for backend compatibility:

    Operation Endpoint

    Read Custom Field Functions

    GET /pipelines/{pipelineId}/config/function-collection

    Publish a new version

    PUT /pipelines/{pipelineId}/config/function-collection

    Follow a publication

    GET /pipelines/{pipelineId}/config/function-collection/publications/{jobId}

    List versions

    GET /pipelines/{pipelineId}/config/function-collection/versions

    Roll back to a version

    POST /pipelines/{pipelineId}/config/function-collection/versions/{version}/rollback

    Install the published version

    POST /pipelines/{pipelineId}/config/function-collection/install

    Test a function

    POST /pipelines/{pipelineId}/config/function-collection/test

    Delete a version

    DELETE /pipelines/{pipelineId}/config/function-collection/versions/{version}

    Remove all Custom Field Functions

    DELETE /pipelines/{pipelineId}/config/function-collection

    Publishing and rollback are asynchronous. Each request returns a job that you follow until completion. See Core Hub APIs for authentication and Open APIs for request and response schemas.

    Troubleshooting

    What you see What to check

    No custom functions in the expression list

    The pipeline has no installed version. Publish and install one first.

    The required function is not offered

    Its declared result type does not fit the target column.

    Saving is refused because the column is part of a key

    Use a non-key or target-only column. Custom Field Functions cannot fill key columns.

    Publication lists affected entities and columns

    The proposed version changed or removed a function used by those bindings. Preserve the signature or update the bindings first.

    A fixed value fails at runtime

    Fixed values are strings and are not converted. Bind a source column for a non-string parameter.

    A target column stops being written

    Check whether the function was suspended after repeated failures. Review logs and notifications, then publish and install a fix.

    Limitations and considerations

    • Functions are written in Java.

    • Custom Field Functions are scoped to one pipeline and are not shared between pipelines.

    • Only the most recent versions are retained.

    • Installing a version restarts the pipeline’s entities.

    • Publication can take several minutes because every UDF and bound column is checked.