Query Forge module
Overview
Query Forge turns Gluesync Core Hub into a single, federated SQL endpoint. Where Query Studio lets you query one agent from inside the Gluesync portal, Query Forge lets an external JDBC client — DBeaver, DataGrip, Tableau, a BI dashboard, a script — run one query that spans several of the Core Hub’s SQL-capable agents at once, joining tables that live behind different agents.
It works on the same unit of access as Query Studio: each SQL-capable agent (a pipeline + agent) becomes a schema you can query. Everything Query Forge asks a source database to run goes through that agent’s existing Query Studio execute endpoint, so no customer data path changes — the query engine only sees the rows you are already permitted to read.
|
Query Forge federates the agents of a Core Hub; it does not introduce a new data channel. Reads reuse the same permissioned Query Studio path. |
How it works
JDBC client (jdbc:gluesync:// driver)
-> Query Forge endpoint
-> Core Hub (per-user auth, SQL-capable agent discovery)
-> Query Studio execute, per pipeline + agent
-> your databases
-
Each SQL-capable agent you can access becomes a schema in the JDBC connection.
-
Each table behind that agent becomes a table under its schema.
-
Query Forge plans each statement centrally. The part of it that touches only one agent’s tables runs on that source database as one SQL statement; only what the source cannot run, such as a join across agents, runs inside Query Forge. See How Query Forge runs a query.
Connecting
Driver
Query Forge ships its own JDBC driver. Download it from the Query Forge setup screen in the Control Plane, or from gluesync-jdbc-latest.jar. Add the JAR to your SQL client’s driver classpath, then use the connection URL:
jdbc:gluesync://<host>:<port>
Internally the driver speaks Apache Avatica; you only ever deal with the jdbc:gluesync: URL. SQL clients name the driver Gluesync JDBC driver with its own version, and the server Gluesync Query Forge with the Core Hub version. DBeaver’s connection test, for example, shows both.
Driver versions and updates
The driver has its own version, separate from Core Hub. Each release is listed in the JDBC driver changelog.
When Core Hub can reach api.backoffice.molo17.com, the driver download on the setup screen shows the latest version and what changed in it. If you have at least one Query Forge token, the Query Forge toolbar also shows a red badge when a newer driver is out. Open it to download that version, then choose Mark as updated: the badge stays hidden until the next release. The setting is per user.
Without internet access, Core Hub shows no version or badge, and the setup screen links to the latest JAR.
Core Hub asks the catalog at most every 6 hours, and after a failed request waits 15 minutes before trying again, so a Core Hub without internet access never slows Query Forge down. To reach the catalog through a mirror or a proxy, set GLUESYNC_BACKOFFICE_API_BASE_URL on Core Hub (default https://api.backoffice.molo17.com). The same information is available from GET /query-forge/driver and, for Spark, the get_query_forge_driver_info MCP tool.
Authentication
You always authenticate as yourself against Core Hub: every query runs with your identity and permissions, and you only see the SQL-capable agents your account may query. There is no shared service token.
Query Forge accepts only Personal API Tokens (gsp_…), whether your account is local or signs in through SSO / OIDC. Create one under Settings → Account → Personal API Tokens. Password login and Control Plane session tokens are refused with SQLState 28000, and so is a token that is expired or revoked.
Pass the token in either of two ways:
-
user = your Core Hub username, password = your Personal API Token (
gsp_…). This suits SQL clients that only show user and password fields. The username only identifies you in the logs; the token decides what you can query. -
a
tokenconnection property set to your Personal API Token.
When a token is limited to certain pipelines and agents, the connection shows only those agents.
|
When you reach Core Hub through Gluesync Connect, the same endpoint and the same authentication apply — Connect simply proxies to the Core Hub API. There is no separate configuration for local vs. Connect access. |
Example: DBeaver
-
Register a new driver pointing at the
gluesync-unified-jdbcJAR; the driver class is handled automatically by thejdbc:gluesync:URL. -
Create a connection with URL
jdbc:gluesync://corehub.example.com:8765. -
Authenticate: put your Core Hub username in Username and your Personal API Token (
gsp_…) in Password, or add atokendriver property. -
Connect. The schema browser shows one schema per SQL-capable agent you can access.
Connection properties
Besides user, password, and token, Query Forge reads these connection properties. Property names are not case-sensitive.
| Property | Unit | Effect |
|---|---|---|
|
Seconds |
Statement timeout for this connection; |
|
Rows |
Scan cap for this connection, higher or lower than the server’s default of 1,000,000, up to a ceiling of 10,000,000 by default. |
|
Milliseconds |
How long the client waits for the server’s answer to one request. This is a client-side Avatica setting, not a Query Forge one. See Client response timeout for long queries. |
A value that is not a whole number of zero or more, or a maxScanRows above the ceiling (or 0 while a ceiling applies), refuses the connection with SQLState 22023, so a typo never runs queries under limits you did not ask for.
You can set these properties in three ways:
-
As driver properties. In DBeaver, add them on the Driver properties tab of the connection.
-
In a
jdbc:gluesync:URL, as;name=valueoptions, for examplejdbc:gluesync://corehub.example.com:8765;maxScanRows=5000000;queryTimeout=600. The Gluesync driver sendsmaxScanRowsandqueryTimeoutto the server as connection properties; every other URL option,http_response_timeoutincluded, stays on the client. An option in the URL overrides a driver property of the same name. -
With a plain Avatica driver, in the query string of the server URL:
jdbc:avatica:remote:url=http://corehub.example.com:8765/?maxScanRows=5000000;serialization=PROTOBUF. A driver property of the same name wins over the query string. Do not putmaxScanRowsorqueryTimeoutas a;name=valueoption of an Avatica URL: the Avatica client keeps those options on the client and never sends them to Query Forge.
Writing federated SQL
Reference each agent by its schema name, then a table:
SELECT c.name, COUNT(o.id) AS order_count, SUM(o.total) AS revenue
FROM crm.customers c
JOIN sales.orders o ON o.customer_id = c.id
GROUP BY c.name
ORDER BY c.name;
Schema naming:
-
A pipeline with a single SQL-capable agent is exposed under the pipeline name (for example,
sales.orders). -
A pipeline with multiple SQL-capable agents disambiguates by agent (for example,
sales__pg.orders).
Identifiers are case-insensitive, matching the friendliest behavior for BI tools.
Ask AI from Query Forge
Query Forge exposes the reserved gluesync.ai virtual table alongside your database catalogs. It turns a natural-language question into one regular JDBC result row containing:
-
answer— the model’s (or Spark agent’s) plain-language result; -
proposed_sql— an optional read-only SQL proposal, never executed automatically; -
schema citations;
-
a conversation ID for follow-up questions;
-
the question, plus the pipeline, database-agent, provider, and optional Spark-agent IDs the request actually used.
Before using it, a Super admin must configure at least one provider under Settings → LLM providers. OpenAI, Anthropic, Azure OpenAI, and local Ollama models are supported. See AI models and LLM providers.
Ask a question without IDs
SELECT *
FROM gluesync.ai
WHERE question = 'retrieve customer data';
Example result:
answer |
proposed_sql |
citations |
conversation_id |
question |
pipeline_id |
agent_id |
provider_id |
spark_agent |
|---|---|---|---|---|---|---|---|---|
I found a |
|
|
|
|
|
|
|
|
This example omits pipeline_id and agent_id and names nothing Core Hub can resolve, so it runs with empty grounding and reports NULL IDs.
You do not need pipeline_id or agent_id for a grounded answer. When the question names a pipeline, agent, schema, or table the caller can see, Core Hub grounds the answer on the strongest unique match and returns the resolved IDs in the row:
SELECT answer, proposed_sql, pipeline_id, agent_id
FROM gluesync.ai
WHERE question = 'in the crm-sync pipeline, list the columns of the dbo.CUSTOMERS table';
Two equally strong matches are ambiguous and fail with SQLSTATE 22023. Only then is it useful to pin the scope with both IDs together — they are optional and never required for a normal ask. See name grounding from the question.
Choose a provider
Use provider_id to select a configured cloud or Ollama provider:
SELECT answer, proposed_sql
FROM gluesync.ai
WHERE question = 'find duplicate customer email addresses'
AND provider_id = 'provider-ollama-local';
Call a Spark agent
spark_agent selects an AI Studio Spark agent by id or unique name. Query Forge applies that agent’s system prompt and only the overlap of its tool allow-list with this path’s read-only MCP / Query Studio tools. Write, DDL, DML, and administrative tools stay blocked even if the Spark agent allows them in chat.
Do not confuse spark_agent with agent_id. agent_id is a database agent used for schema grounding. spark_agent is the AI Studio agent that answers the question.
The Spark agent’s plain result is the answer column. proposed_sql remains an optional SQL draft for review and is never executed:
SELECT answer, proposed_sql, spark_agent
FROM gluesync.ai
WHERE question = 'summarize replication lag'
AND spark_agent = 'Ops helper';
Ask a follow-up
Continue the exchange by passing the returned conversation_id:
SELECT answer, proposed_sql, conversation_id
FROM gluesync.ai
WHERE question = 'limit that query to active customers'
AND conversation_id = 'b5acd3d2-7c9d-4100-8296-25b10132ebe0';
|
|
For the complete column contract, accepted predicates, prepared-statement example, SQLSTATE errors, and safety behavior, see AI as SQL virtual catalog.
How Query Forge runs a query
Query Forge splits every statement in two. The part that touches only the tables of one agent runs on that source database as a single SQL statement, in the source’s own dialect. The rest runs inside Query Forge, over the rows the source returns. The more of a query the source runs, the fewer rows cross the network and the less Query Forge has to hold.
What runs on the source database
On the agents listed in Which agents run the full query, the source database runs:
-
Filters: comparisons (
=,<>,<,<=,>,>=), ranges,INlists of up to 1,000 values,LIKE,IS NULL/IS NOT NULL, and theirAND/OR/NOTcombinations. -
Projections: only the columns and expressions the query needs, including arithmetic,
CASE,COALESCE,NULLIF, casts, and the functions in Functions run on the source. -
Aggregates:
COUNT,SUM,MIN,MAX, andAVG, withGROUP BYandDISTINCT. The source computes anAVGas aSUMand aCOUNT; Query Forge divides them. The grouping keys and the aggregated values can be expressions, for exampleSUM(quantity * unit_price * (1 - COALESCE(discount, 0)))grouped byEXTRACT(YEAR FROM sale_date). -
Sorting and paging:
ORDER BY,LIMIT, andOFFSET. -
Joins between tables of the same agent.
-
UNION ALLof queries on the same agent. (UNION, which removes duplicates, runs inside Query Forge.)
A filter is split when only some of its AND conditions can run on the source. Those run on the source, and the others run inside Query Forge over the rows that come back. For example, WHERE employee_id = 5 AND POSITION('X' IN last_name) = 0 reads one row from the source, then Query Forge applies the POSITION condition. An OR runs on the source whole, or not at all.
A COUNT(*) over a table of any size is therefore one statement on the source that returns one row.
Functions run on the source
Query Forge sends a function to the source only where the source computes it exactly as Query Forge would, and only over a column type the source reads the same way. Anything whose result could differ stays inside Query Forge. A function over a masked column is never sent to the source (see PII masking in Query Forge).
The matrix depends on the source family:
| Construct | PostgreSQL, YugabyteDB | SQL Server | Oracle, MySQL, MariaDB, SingleStore, Db2 LUW, Snowflake, Vertica, Redshift, CockroachDB |
|---|---|---|---|
|
Source |
Source ( |
Source |
|
Source |
Source |
Redshift and Db2 LUW only |
|
Source |
Source |
Source |
|
Source |
Source |
|
|
Source |
Source |
Local |
|
Source |
Source |
Local |
|
Source |
Source |
Local |
|
Source |
Source |
Source |
|
Source (not over |
Source (not over |
Local |
|
Source (not over |
Local |
Local |
Casts between numbers that change no value (widening an integer, an integer to a decimal that holds it, a decimal to one with as many or more digits), and any number to floating point |
Source |
Source |
Source |
Casts of a timestamp to a date, a date to a timestamp, an integer to text |
Source |
Source |
Local |
These always run inside Query Forge, on every source:
-
UPPERandLOWER. Query Forge maps case with full Unicode rules (UPPER('straße')is'STRASSE'), while PostgreSQL and SQL Server map one character at a time ('STRAẞE','STRAßE'). -
SUBSTRINGandCHAR_LENGTHon SQL Server, which counts UTF-16 units (an emoji is two) except on columns with a supplementary-character (_SC) collation;POSITIONandOVERLAYeverywhere. -
EXTRACT(SECOND …),FLOOR(x TO WEEK), andROUNDof a floating-point value. -
Casts that can drop digits: a decimal or floating-point value to an integer, an integer to a narrower integer, a decimal to a smaller scale. Also casts of a date or decimal to text, and of text to anything.
-
Division of decimals, including the division that completes an
AVG: PostgreSQL, SQL Server, and Query Forge would each produce different digits, so Query Forge always computes it (see [numeric-results]). -
Functions over a date or timestamp with a time zone (
timestamptz,datetimeoffset).
|
|
|
|
|
|
What runs inside Query Forge
These parts of a query run inside Query Forge, over rows read from the sources:
-
Joins between tables of different agents. Each side of the join still has its own filters, projections, and expressions run on its source first, an aggregate over the join is pre-aggregated on each side where possible (see Aggregates over joins across agents), and a selective side’s keys filter the other side on its source (see Joins across agents with a selective side).
-
Functions not in Functions run on the source for that source, and window functions such as
ROW_NUMBER() OVER (…). -
Prepared-statement parameters (
?). A condition such asWHERE id = ?is applied inside Query Forge. Write the value as a literal when the table is large. -
Text literals with characters outside ASCII, such as
'München'or'Москва'. They work, but a condition that uses one is applied in Query Forge. -
TIMEliterals, and, on MySQL, MariaDB, and SingleStore, text literals containing a backslash. -
INlists of more than 1,000 values. -
UNION(withoutALL) and grouping sets such asROLLUPandCUBE. -
Anything that computes with, renames, aggregates, groups, sorts, or joins on a masked column. See PII masking in Query Forge.
When a query needs any of these, Query Forge reads the rows the rest of the query selects and finishes the work itself. Those reads count against the scan cap.
Aggregates over joins across agents
An aggregate over a join of tables on different agents is first aggregated on each side. Each side is grouped on its own source by the join keys and the grouping columns it holds, with COUNT, SUM, MIN, and MAX split into sub-totals (AVG as a SUM and a COUNT). Query Forge then joins the sub-totals and combines them, so the result is the same as aggregating the joined rows. A side that is unique on the join keys, such as a dimension joined on its primary key, is read as it is. Filters and expressions that use only one side’s columns run on that side first.
For example, revenue by store region over a 10-million-row fact table on PostgreSQL and its store dimension on SQL Server becomes one aggregate on PostgreSQL, grouped by store, and one read of the stores. The fact table is never paged into Query Forge.
The pre-aggregated side returns one row per distinct combination of join keys and grouping values it holds. A matrix such as category by year, over a fact table joined to products, groups the fact table by product and year, which can still be many rows and take several calls.
LEFT and RIGHT joins across agents are pre-aggregated too: the side the join pads with NULL`s is aggregated on its source by the join keys, and the sub-totals are combined so that a preserved row with no match still counts once in `COUNT(*), as 0 in COUNT(column), and as NULL in SUM, MIN, and MAX. For example, dim_store LEFT JOIN sales_fact … GROUP BY region becomes one aggregate on the fact table’s source. This needs an ON clause made of column equalities (a condition on one side alone, such as AND f.sale_id <= 500, is first moved into that side), and no SUM or COUNT of a column of the preserved side.
These aggregates are not pre-aggregated, and run inside Query Forge over each side’s rows:
-
FULLouter joins across agents; -
outer joins whose
ONclause is not only column equalities, andSUMorCOUNTof a preserved column over an outer join; -
COUNT(DISTINCT …)and filtered aggregates (… FILTER (WHERE …)) over a join across agents; -
expressions that mix columns of both sides inside an aggregate.
Joins within one agent are not rewritten: they run on that source whole.
The order of the tables in FROM does not matter. Before pre-aggregating, Query Forge regroups the inner joins of a query that crosses agents so that the tables of one agent that a join condition connects are joined together on that agent. For example, fact (PostgreSQL) JOIN store (SQL Server) JOIN customer (PostgreSQL) runs as fact JOIN customer on PostgreSQL, pre-aggregated, then joined with the stores in Query Forge, exactly as if the query had listed the tables in that order. The groups of different agents are then joined in the order of their first table, each connected by a condition where there is one; Query Forge never adds a cross join the query did not write. Outer joins, and joins inside a subquery or under an aggregate, keep their place. Reordering inner joins changes no result.
Joins across agents with a selective side
A join across agents that is not pre-aggregated (a detail query, or an aggregate from the list above), or whose pre-aggregated sides are still large, is reduced by keys. When one side has few rows, Query Forge reads it first and sends its join keys to the other side’s source as IN lists, so that source returns only the matching rows. For example, returns_fact r JOIN sales_fact f ON r.sale_id = f.sale_id WHERE r.return_id <= 2000 reads the 2,000 returns, then asks the sales table’s source only for their sales, instead of paging 10 million sales into Query Forge.
-
Which joins: equi-joins on integer columns (an
INTEGERkey against aBIGINTone included, and composite keys): inner joins (either side’s keys),LEFTandRIGHTjoins (the preserved side’s keys; its rows with no match come out padded withNULL`s), and `IN,EXISTS,NOT EXISTS, andNOT INsubqueries. The side to filter must be one Query Forge sends to a source; the key side can be anything, even aUNIONcomputed in Query Forge. A join on an integer key and a decimal key is reduced by its integer key, and the decimal key is compared on each pair. -
Extra conditions: a filter that uses only the side being filtered goes to its source with the keys:
NOT EXISTS (SELECT 1 FROM sales_fact f WHERE f.sale_id = r.sale_id AND f.quantity > 5)sendsWHERE quantity > 5 AND sale_id IN (…). A condition that mixes both sides, such asAND f.quantity > r.quantity, is checked on each pair of rows the keys bring together. -
When: the decision is made while the query runs, because Query Forge does not know table sizes in advance. It reads the candidate sides, a filtered one first, until one of them ends within
GLUESYNC_JDBC_JOIN_REDUCTION_MAX_KEYSrows (50,000 by default). That side’s keys then filter the other side. While deciding, Query Forge holds at most that many rows of the other side; if the other side ends within them, it is joined as it is. -
Batches: each query to the other source sends at most
GLUESYNC_JDBC_JOIN_REDUCTION_BATCH_SIZEkeys (1,000 by default, Oracle’s limit for anINlist; a composite key counts each of its values). -
Never sent: text keys, because each source compares text by its own collation and padding, and keys on columns that are masked for you.
-
Semantics: keys compare as numbers, a
NULLkey matches nothing, and duplicate keys on either side multiply as in the join. The result is the same as without the reduction.
The two settings apply to the whole Core Hub; a connection cannot override them. See Configuration reference for administrators.
When both sides are large
When neither side of an integer-key join has at most GLUESYNC_JDBC_JOIN_REDUCTION_MAX_KEYS rows, Query Forge still avoids holding them where it can:
-
Both sides unique on one integer key (a primary key, or the group key of an aggregate): a merge join. Both sides are read in key order, a page at a time, and nothing is held in memory. When one side’s next key is more than a page ahead, the other side skips straight to it.
-
One side unique on its key: the other side is read whole, and the unique side is read in key order, only between the first and last key the other side needs.
-
Neither side unique: Calcite’s hash join, over every row of both sides.
Every row read still counts against the scan cap, so a join of 2 million returns with 10 million sales stops with 54000 at the default cap of 1,000,000. Raise maxScanRows on the connection for such a join (10,000,000 is within the default ceiling); a merge join holds none of those rows in memory.
Range (band) joins
A join without an equality, whose condition bounds one integer column of a side from below and from above by columns of the other side, is a band join: for example f.sale_id >= r.sale_id AND f.sale_id < r.sale_id + 3, or p.k BETWEEN r.lo AND r.hi. A bound can be a column plus or minus a constant. It applies to inner and LEFT joins and to semi-joins. The other side is read whole, and the bounded side is read once:
-
when it is unique on the bounded column, in key order over just the ranges the other side’s rows ask for;
-
otherwise, by one query over at most 64 ranges covering those rows (overlapping ranges merged, the smallest gaps closed), then joined in Query Forge.
The whole condition is checked on each pair. A NULL bound matches nothing, and a LEFT join pads that row.
Joins that still read every row of both sides into Query Forge, with no reduction, merge, or range read, and stop at the scan cap unless maxScanRows is raised:
-
joins where both sides have more rows than
GLUESYNC_JDBC_JOIN_REDUCTION_MAX_KEYSand neither is unique on one integer key; -
FULLouter joins across agents; -
joins on text keys;
-
band joins whose bounds are expressions other than a column plus or minus a constant, and joins whose condition is neither equalities nor a band.
Correlated subqueries
Query Forge rewrites correlated subqueries into joins before it plans them, so they benefit from the join handling above. The rewrite is exact around NULL`s: `EXISTS and NOT EXISTS with an OR … IS NULL condition, a correlated NOT IN whose values or subquery contain NULL`s or return no rows, and a correlated scalar `COUNT(*) that finds no rows (it returns 0) are all exact.
Two shapes are refused with SQLState 0A000:
-
a subquery correlated with a query two or more levels up, that is, one that refers to a table outside the query directly around it. For example,
EXISTS (… f.store_id = s.store_id AND EXISTS (… r.sale_id = f.sale_id AND r.store_id = s.store_id))refers tosfrom two levels down. A subquery correlated with the query just around it runs at any depth; -
a correlated
NOT INin aSELECTlist or under anOR. In aWHEREorHAVINGcondition it is supported.
Any other query the planner cannot handle is also refused with 0A000 and the message "Query Forge cannot plan this query".
Which agents run the full query
The source runs everything in What runs on the source database on these agents:
-
PostgreSQL, YugabyteDB, and CockroachDB;
-
Amazon Redshift;
-
Microsoft SQL Server (2012 and later);
-
MySQL, MariaDB, and SingleStore;
-
Oracle (12c and later);
-
IBM Db2 LUW;
-
Snowflake;
-
Vertica.
On every other agent (for example MongoDB, IBM i, Informix, Sybase ASE, SAP HANA, or Google BigQuery), the source runs only the column projection and simple filters: comparisons with a literal, LIKE, IS NULL / IS NOT NULL, and their AND / OR / NOT combinations. Functions, aggregates, sorting, LIMIT, IN lists, and joins run inside Query Forge, so every read from those agents counts against the scan cap. The one exception is a MongoDB COUNT(*), which MongoDB counts itself; MongoDB also has a tighter read limit of its own. See MongoDB tables.
Results follow the source database
Work that runs on the source follows that database’s rules, as the same query would if you ran it there directly. A GROUP BY, DISTINCT, join, comparison, MIN / MAX, or ORDER BY on text uses the source’s collation (case-insensitive by default on SQL Server, for example). The same operation run inside Query Forge compares text by Unicode code point.
Numbers, on the other hand, come out the same whichever source or plan computes them:
-
SUMof an integer column returns aBIGINT, so a large sum does not overflow on SQL Server or Db2. -
A decimal quotient is always computed inside Query Forge, exactly to 12 digits after the point (or to the dividend’s own scale when that is larger) and rounded half up, so it is identical whichever source or plan the query uses:
137.02 / 3is45.673333333333, and1234.567 / 7is176.366714285714. Its type isDECIMAL(38, 12), orDECIMAL(38, s)for a dividend with a larger scales. -
Every quotient fits that type. A quotient whose integer part needs more digits than the type leaves (26 for
DECIMAL(38, 12)) fails with SQLState22003instead of losing digits after the point. -
AVGof an integer or decimal column follows the same rule. The source sums and counts, and Query Forge divides, also when it pre-aggregates a join across agents. SQL Server on its own would return5for the average of 5 and 6. -
Values are never padded with zeros: the average of 1 and 2 is
1.5, not1.500000000000. -
For a specific precision, cast the result, for example
CAST(SUM(amount) / SUM(quantity) AS DECIMAL(18, 4)). -
Every
DECIMALresult is rounded to its type’s scale, whichever plan computes it. -
A computed
DECIMALhas at most 38 digits. -
Decimal and floating-point values compare by value wherever Query Forge matches them itself: joins,
INandEXISTS,GROUP BYandDISTINCT(alsoROLLUP,CUBE, andGROUPING SETS),COUNT,SUM, andAVGofDISTINCTvalues, andUNION,INTERSECT, andEXCEPT. So1.5matches1.50,-0.0matches0.0, and, as on PostgreSQL,NaNmatchesNaN. The values you read keep the scale their source gave them; a group shows one of its values. -
A decimal constant takes its declared type’s scale, rounded half up, before the query is planned:
CAST(1.555 AS DECIMAL(5, 2))is1.56. -
A cast to a decimal is sent to the source with all its digits, up to what that database holds: 38 digits, with at most 30 after the point on MySQL, MariaDB, and SingleStore, 37 on Snowflake and Redshift, and 31 digits on Db2. A cast the database cannot hold is computed in Query Forge.
A decimal column has the precision and scale its source declares, up to 38 digits. A decimal declared without a precision (PostgreSQL’s plain numeric), or with more than 38 digits, is a DECIMAL(38, 18): its values are rounded half up to 18 digits after the point, and the result set metadata reports precision 38 and scale 18. A value that its column type cannot hold fails the query with SQLState 22018 and a message naming the column.
Text labels of different lengths, such as the results of CASE … THEN 'low' ELSE 'medium' END, are returned without trailing padding.
DATE, TIME, and TIMESTAMP columns arrive as native JDBC date and time values. A TIMESTAMP column reads the wall-clock time the database holds. A time-zone-aware column that Query Forge exposes as TIMESTAMP reads in Core Hub’s time zone.
How results reach your client
Query Forge streams results. It reads from the source in pages and passes rows on to the client while it reads.
-
Each call to a source returns at most 10,000 rows by default: the smaller of
GLUESYNC_JDBC_MAX_ROWSand the agents' Query Studio cap,GLUESYNC_QUERY_STUDIO_MAX_ROWS(see Configuration reference for administrators). -
A pushed query is read in a fixed order, so no row is read twice: its own
ORDER BYfirst, then the primary key, or, when the table has no usable key, every column the source can sort. A result that fits in one page takes one call. -
The client receives rows in frames of at most 1,000 rows and fetches the next frame as it reads.
Each page runs the query again on the source. When a result is ordered only by one unique key of an integer or exact numeric type, and the query has no LIMIT or OFFSET of its own, every page after the first asks for the rows after the last key read (WHERE key > …), which the source finds through its index. Otherwise the source skips the rows before each page (OFFSET), so the deep pages of a very large export, or of a large pre-aggregated result, cost the source more than the first ones.
|
When a result has no usable key and includes a column the source cannot sort (LOB, XML, JSON, or spatial columns), the page order may not be total. Rows whose sortable values are identical can then repeat or go missing across pages. Add an |
Write fast queries from BI tools
How fast a query runs depends on where it runs, not on how large the table is. Use this table to tell the two cases apart.
| Query shape | Where it runs | At scale |
|---|---|---|
|
Source |
Fast at any size: only the result rows come back. Check Functions run on the source for the functions your source runs. |
Filter, then |
Source |
Fast at any size, given an index the source can use. |
Join of tables on the same agent |
Source |
As fast as the same join run on the source. |
Inner join across agents with |
Tables of the same agent joined on it, each side pre-aggregated on its source, the sub-totals combined in Query Forge |
Fast when each side has few distinct join keys and grouping values. Only the sub-totals cross the network. See [join-order]. |
|
MongoDB |
Fast at any size. |
Filtered rows and columns from one table (an extract or export) |
Source, streamed in pages |
Not capped. Time grows with the number of rows returned. |
|
The padded side pre-aggregated on its source, the sub-totals combined in Query Forge |
Fast when the padded side has few distinct join keys. See Aggregates over joins across agents. |
Join across agents on integer keys where one side is selective (at most 50,000 rows), including fact-to-fact joins such as returns to sales, and |
The selective side read first, its keys sent to the other source in batches |
Fast at any size of the other side: only matching rows are read. See Joins across agents with a selective side. |
Join across agents where both sides are large and unique on one integer key (for example a fact table joined with another table’s sub-totals by the same key) |
Merge join: both sides read in key order, page by page |
Streams without holding rows, but every row read counts against the scan cap: raise |
Band join across agents ( |
The bounded side read once, over the ranges the other side asks for |
Fast when the ranges are selective. See Range (band) joins. |
Join across agents where both sides are large and neither is unique on an integer key, on text keys, or |
Each side on its source, the join in Query Forge |
Every row each side returns is read into Query Forge. Filter each side as much as you can. |
|
Inside Query Forge |
Every row each side returns is read into Query Forge. |
A function over a masked column, a function the source does not run, a |
Inside Query Forge |
The rows the rest of the query selects are read into Query Forge. |
Any function, aggregate, sort, or |
Inside Query Forge |
All the rows the filters select are read into Query Forge. |
Queries that read rows into Query Forge stop at the scan cap instead of growing without bound. If a dashboard hits the cap, rewrite the query so the source does more of the work: add a filter the source can run, aggregate over an inner join instead of joining detail rows, or replace a parameter with a literal. If the query really needs the rows, raise the cap for that connection (see Scan cap).
Limits
Query Forge bounds the work one statement can do, so that a single query cannot exhaust Core Hub’s memory or keep a source busy indefinitely.
| Limit | Default | What it bounds | When exceeded |
|---|---|---|---|
1,000,000 rows |
Rows Query Forge reads from one table scan whose rows it keeps. A connection can raise or lower it. |
SQLState |
|
Page size |
10,000 rows |
Rows one call to a source returns: the smaller of |
Query Forge reads the next page. |
Client frame |
1,000 rows |
Rows sent to the client in one frame. |
The client fetches the next frame. |
Call timeout |
60 seconds |
Time one call to a source may take. A longer |
SQLState |
None |
Time the server spends on one statement. |
SQLState |
|
32 running, 256 waiting |
JDBC requests the gateway works on at once. |
SQLState |
|
Source queries per agent |
8 |
Page queries one agent runs at once, for all connections. |
The query waits for a free slot. |
Request size |
16 MiB |
Size of one JDBC request from the client. |
SQLState |
10,000 rows |
Rows one MongoDB table read may return. A |
The query fails without partial rows. |
|
10 minutes with the Gluesync driver, 3 minutes with a plain Avatica client |
Time the client waits for the server’s answer to one request. |
The statement fails with SQLState |
Administrators change these defaults with the environment variables in Configuration reference for administrators.
Scan cap
The scan cap limits how many rows Query Forge reads from a single table scan when it has to keep those rows to finish the query. This is the case for:
-
a scan that feeds a join across agents;
-
a scan whose rows Query Forge sorts, aggregates, or de-duplicates itself;
-
every scan of an agent without full pushdown (see Which agents run the full query).
A pushed query whose rows stream straight to the client, with only filters, projections, and LIMIT above it, is not capped. A large export therefore runs to the end.
The default cap is 1,000,000 rows per scan (GLUESYNC_JDBC_MAX_SCAN_ROWS). A query that needs more fails with SQLState 54000 and a message that names the table and the agent. To fix it, let the source do more of the work (see Write fast queries from BI tools), or raise the cap for the connection.
Each connection can set its own cap with the maxScanRows connection property (see Connection properties), higher or lower than the server’s default, without any administrator action, up to a ceiling. A low cap suits a shared dashboard connection that should fail fast; a high one suits an analyst who knowingly runs a large join across agents.
The ceiling is ten times the server’s default cap: 10,000,000 rows with the default of 1,000,000. When the default cap is 0 (no cap), there is no default ceiling either. An administrator can set the ceiling lower or higher with GLUESYNC_JDBC_MAX_SCAN_ROWS_CEILING:
-
a connection that asks for more than the ceiling is refused at connect with SQLState
22023and a message that names the ceiling, rather than given a lower cap than it asked for; -
maxScanRows=0(no cap) is refused with22023while a ceiling applies. SettingGLUESYNC_JDBC_MAX_SCAN_ROWS_CEILING=0removes the ceiling, and is the only way to let a connection run without a cap; -
a connection that does not set
maxScanRowsgets the server’s default cap, or the ceiling if an administrator set it lower than the default.
Memory use of a raised cap
The rows a scan keeps stay on Core Hub’s Java heap until the statement ends, at roughly 50 to 100 bytes per value. A million rows of five columns take a few hundred MB. One statement can keep several scans (one per side of a join across agents), and several connections run at once.
The default ceiling lets any user raise their own cap to 10,000,000 rows without asking. A join across agents or a sort that keeps 10 million rows can take 1 to 2 GB of Core Hub heap for one query, in the same JVM that runs your pipelines. On a Core Hub with a small heap, lower GLUESYNC_JDBC_MAX_SCAN_ROWS_CEILING to what the heap can spare.
Raising maxScanRows, or removing the cap, therefore lets one query use heap that Core Hub needs for replication. Before you raise it:
-
size Core Hub’s heap (
-Xmx) for the largest scans you expect, multiplied by the queries that run at the same time; -
lower
GLUESYNC_JDBC_MAX_SCAN_ROWS_CEILINGif the default ceiling of 10,000,000 rows does not fit the heap, and remove the ceiling (0) only where you trust every user with it; -
watch the JVM heap panel of the Core Hub Grafana dashboard while the large queries run.
Streamed results, whose rows go straight to the client, are never held. A pre-aggregated side of a join across agents holds one row per group.
Statement timeout
The statement timeout limits how long the server spends on one statement. It counts only the time Query Forge spends serving the statement’s execute and fetch requests, including the source calls they make. The time your client spends between fetches, for example while you scroll a result grid, does not count.
There is no statement timeout by default; only the 60-second timeout of each source call applies. Set one in either of two ways:
-
for every connection, with
GLUESYNC_JDBC_STATEMENT_TIMEOUT_MS(administrators); -
for one connection, with the
queryTimeoutconnection property, in seconds (0means none).
When both are set, the shorter one applies: queryTimeout never extends the server’s statement timeout. A statement that runs out of time fails with SQLState HYT00, and Query Forge stops its query on the source.
queryTimeout also gives each call to a source more time. A single pushed aggregate over a large table can need more than the default 60 seconds. When queryTimeout is longer than GLUESYNC_JDBC_QUERY_TIMEOUT_MS, each source call may run as long as queryTimeout, up to GLUESYNC_JDBC_MAX_QUERY_TIMEOUT_MS (5 minutes by default). The agents stop a query after 5 minutes in any case, unless GLUESYNC_QUERY_STUDIO_MAX_TIMEOUT_MS is set higher on the agents. To let a source call run for, say, 15 minutes, an administrator raises both, and the client sets queryTimeout=900.
|
|
Concurrent requests
Query Forge works on up to 32 JDBC requests at once, and up to 256 more wait for one of them to finish. Further requests are refused at once with the SQL error "Query Forge is busy" (SQLState 53000). Try again shortly.
Closing a statement or a connection is never refused or queued, so you can always stop a running query by closing its connection.
Each agent runs at most 8 page queries at a time for the whole gateway. Further queries for that agent wait for a free slot, so one busy BI tool cannot flood a source database.
|
A refused request reaches the client as HTTP status |
MongoDB tables
Query Forge reads a MongoDB table in a single call of at most 10,000 rows, after the filters the source can run. If the query needs more rows than that call returns, it fails rather than return a partial result. A query that is complete within those rows, for example one with a LIMIT or a selective filter, runs normally.
A COUNT(*) over one collection, without GROUP BY, is counted by MongoDB instead, so it works on a collection of any size. Its filters must be ones the collection’s read runs on MongoDB: comparisons of a field with a literal of its type, IS NULL / IS NOT NULL, LIKE, and their AND / OR / NOT combinations. With any other filter, a GROUP BY, another aggregate, or a join, the count runs inside Query Forge over the 10,000-row read.
Cancel queries and set timeouts in BI tools
Query Forge uses the Apache Avatica protocol between the driver and the server. That protocol has no cancel request, so some client controls do not reach the server.
These do not stop a query on the server:
-
The Stop or Cancel button of DBeaver and other SQL clients, which call
Statement.cancel(). -
Statement.setQueryTimeout()and the client’s own query timeout setting. -
Closing the statement while it runs: the client holds the statement until the execution ends, so the close waits too.
These do stop it:
-
Close the connection. In DBeaver, select Disconnect. Query Forge stops the connection’s scans and cancels its queries on the source databases. Only your queries are cancelled: other users' queries on the same agent keep running.
-
Set the
queryTimeoutconnection property. The server enforces it. See Statement timeout. -
Ask an administrator for a server statement timeout (
GLUESYNC_JDBC_STATEMENT_TIMEOUT_MS), which applies to every connection.
Closing a streamed result between fetches, for example closing a result tab after the first rows arrived, also stops that result’s read on the source.
On agents that cannot cancel a single query, such as MongoDB, Google BigQuery, and Couchbase, a call already running on the source finishes on its own, within the call timeout. Query Forge does not start another one and never stops other users' queries on that agent.
A statement that was running when its connection closed fails with SQLState 57014.
Client response timeout for long queries
The Avatica protocol answers each request whole. Query Forge cannot send the client anything, not even a keep-alive, before a statement’s first rows exist, and an aggregate or a sort has none until it has read all its input. The client therefore waits for one response for the whole time such a statement runs, and gives up after its HTTP response timeout, http_response_timeout (milliseconds):
-
The Gluesync JDBC driver (
jdbc:gluesync:URL) waits 10 minutes by default, or the connection’squeryTimeoutplus one minute when that is longer. -
A plain Avatica client (
jdbc:avatica:remote:URL) waits 3 minutes.
When the client gives up first, the query fails while the server may still be running it: with the Gluesync driver as a SQLException with SQLState 08006 (its cause is a SocketTimeoutException), with a plain Avatica client as a SocketTimeoutException with no SQLState.
The Gluesync driver’s longer default is a trade-off: if Core Hub or the network hangs, a client thread (a DBeaver editor, a BI tool’s worker) also waits up to 10 minutes before it fails. Set a shorter http_response_timeout on connections that only run quick queries.
For BI queries that run longer:
-
Set
queryTimeoutto the time the query may take, in seconds (see Statement timeout). With the Gluesync driver, the response timeout follows it. -
Or set
http_response_timeoutexplicitly, in milliseconds. In DBeaver, open Edit Connection → Driver properties, add a user propertyhttp_response_timeoutwith a value such as1800000(30 minutes), and reconnect. A value you set is never changed by the driver.
See schema changes on an open connection
Tables created, dropped, or altered in a source show up on a connection that is already open.
-
Refresh in your client. When you refresh the connection or a schema in your SQL client (for example, Refresh in DBeaver’s Database Navigator), the client asks for the schemas, tables, and columns again. Query Forge re-discovers them from the sources for that request, so the change shows at once. The burst of calls one refresh sends, within 2 seconds, shares a single discovery.
-
Without a refresh, queries trust what was discovered for up to 60 seconds (
GLUESYNC_JDBC_METADATA_TTL_MS). After that, the next query re-discovers it.
A query never waits for another query’s re-discovery when an earlier result is available. If re-discovery fails, Query Forge keeps serving the previous result and tries again after 30 seconds.
PII masking in Query Forge
Query Forge reads through Query Studio, so the PII discovery and masking rules apply to its results. JDBC clients have no PII visible toggle: while platform-wide masking is on, results stay masked for every role. Query Forge also makes sure that pushing work to the source does not bypass masking.
-
Masked columns stay on a short leash. Query Studio masks a column it classifies as personal data when the column comes back under its own name. Query Forge therefore lets the source handle a masked column only as itself: read under its own name, compared with a literal of the same type, or counted. Anything else that uses a masked column (renaming it, computing with it, aggregating, grouping, sorting, or joining on it) runs inside Query Forge, over the masked values. The result is the same as a plain read would give. When Core Hub cannot say which columns are masked, the table is only read, and everything else runs inside Query Forge.
-
Masking is a display control, not an access boundary. It hides values in results. A user who may query a table can still infer masked values with filters, for example
WHERE email LIKE 'a%'orWHERE first_name = 'Ada', as in Query Studio. To keep data away from a user, deny the table or schema with a Query Studio data-access policy. -
Classify columns that are masked only by sampling. Query Forge knows in advance only the columns classified by name or by a stored verdict. A column that Query Studio masks only because it recognised the values in a result is not known beforehand, so a value computed from it on the source, such as
MAX(notes)ornotes AS n, can come back unmasked. Run Scan table for PII or Mark as PII in Query Studio so that a verdict is stored for such columns. -
Row order can reveal rank order. A result with no unmasked key is paged in an order that includes its masked columns, so that pages do not repeat or skip rows. The order of the rows can then reveal the rank order of masked values. This reveals less than the literal comparisons and
LIKEfilters above already do.
Security notes
-
Per-user scope: you only see the SQL-capable agents your account may query; a rejected token or bad credentials fail the connection immediately.
-
Read-only: queries run against agents through Query Studio’s read-only path.
-
PII masking: results are masked as in Query Studio. See PII masking in Query Forge.
-
Stored connection details: to re-open connections after a restart, Core Hub keeps only each connection’s credentials, catalog, and its
maxScanRowsandqueryTimeoutsettings, whatever else a client sends. The Gluesync driver sends no other URL option to the server. -
Transport: place the endpoint behind TLS (a reverse proxy, or the platform’s own TLS termination) so credentials and result sets are encrypted in transit.
Troubleshooting
Every error Query Forge answers carries a meaningful SQLState, never 00000. When a query fails on a source database, Query Forge does not yet pass on the source’s own SQLState: it infers one from the source’s error message, for example 22003 or 42S02, and the message itself still shows the source’s error. With the Gluesync JDBC driver, every error reaches the client as a SQLException that carries this SQLState: a connection that is refused (an invalid token, a maxScanRows above the ceiling, a busy gateway), a failed statement, and an error that arrives while the client fetches more rows, such as the scan cap reached after the first rows were shown. When the client cannot reach Query Forge at all, the SQLState is 08006. A plain Avatica client reports some of these as unchecked runtime exceptions instead.
Query Forge logs every error it answers in the Core Hub log at WARN level, with the request type, the connection and statement ids, the SQLState, and the whole chain of causes. It never logs connection properties, credentials, or tokens. The cause can include the failing SQL text with its literal values, as Query Studio’s own log already does. When a client shows only a short message, check the Core Hub log for the full cause.
| SQLState | Meaning | What to do |
|---|---|---|
|
The query needed more rows from one table than the scan cap allows (the message names the table, the agent, and |
Let the source do more of the work: add a filter it can run, aggregate over an inner join instead of joining detail rows, or replace a |
|
The JDBC request was larger than Query Forge accepts (the message names |
Shorten the statement, for example a very long |
|
Query Forge is busy: 32 requests are running and 256 are waiting. |
Try again shortly. If it happens often, look for BI tools or dashboards that send many queries at once, and spread their refreshes. |
|
The statement timeout, or the timeout of one source call (60 seconds unless |
Make the query cheaper for the source (see Write fast queries from BI tools), or raise |
|
The connection was refused: |
Fix the value in the driver properties or the URL. Use a |
|
The statement was cancelled because its connection was closed. |
Expected after you close a connection to stop a query. Reconnect to run it again. |
|
A value ( |
Check the column’s type and data on the source, or the cast in the query. Core Hub’s log has the full cause. |
|
Division by zero ( |
Guard the divisor ( |
|
The statement uses a feature Query Forge does not support: a subquery correlated two or more levels up, a correlated |
Rewrite the query without it; the message says which. For example, correlate the inner subquery with its direct parent only, or move a correlated |
|
The connection or statement is no longer known to Query Forge, for example after a long idle period. |
Reconnect. |
|
The Gluesync driver could not reach Query Forge, the client stopped waiting for the answer to a statement (its response timeout ran out), or a call from Query Forge to Core Hub or to an agent failed in transit. |
Check the host and port in the URL, that Query Forge is enabled on Core Hub ( |
|
Core Hub rejected the token, or the password is not a Personal API Token ( |
Check that the Personal API Token is valid and not expired or revoked, and that it is the value you pass as the password or the |
|
Core Hub denied the query: your account, or a token limited to certain pipelines and agents, may not read that data. |
Ask a |
|
The statement has a syntax error, or failed validation. |
Fix the SQL. The message, and the Core Hub log, name the problem. |
|
The statement names a table ( |
Check the schema, table, and column names, and refresh the connection if the table is new (see See schema changes on an open connection). Check that your account may query that agent. |
|
Any other error. |
Check the Core Hub log for the full cause. |
These problems have no SQLState of their own:
| Problem | What to do |
|---|---|
The Stop button does not stop a query. |
Close the connection instead, and set the |
A new or changed table does not show in the client. |
Refresh the connection or the schema in the client. See See schema changes on an open connection. |
A MongoDB query fails with a message about the pushdown cap and partial rows. |
The query needs more than the 10,000 rows one MongoDB read returns. Add a filter or a |
A long query fails in the client with |
The client stopped waiting for the server’s answer. Set |
An HTTP proxy or monitor reports HTTP 500 from the Query Forge port. |
Check the client for SQLState |
Configuration reference for administrators
Query Forge runs inside Core Hub and starts with it. Configure it with these Core Hub environment variables. All of them are optional; restart Core Hub after you change one.
| Variable | Default | Description |
|---|---|---|
|
|
Set to |
|
|
Port the JDBC endpoint listens on. Clients connect to |
|
|
Most rows asked for in one call to a source. A page asks for the smaller of this value and |
|
|
Timeout of one call to a source, in milliseconds. A connection whose |
|
|
Longest per-call timeout, in milliseconds, that a connection’s |
|
|
Default scan cap, for connections that do not set |
|
10 × |
Highest |
|
|
Statement timeout, in milliseconds, counted over the server’s time only. |
|
|
Largest JDBC request Query Forge accepts. A larger one is refused with SQLState |
|
|
Most rows a side of a join across agents may have for its keys to filter the other side on its source. Query Forge keeps up to this many rows of each side in memory while it decides. |
|
|
Keys sent in one query of a reduced join, from |
|
|
How long queries trust discovered schemas, tables, and columns before re-discovering them, in milliseconds. A client’s Refresh re-discovers them at once. See See schema changes on an open connection. |
These two are global Query Studio settings, set on the agents of JDBC databases, where the Query Studio executors run (MongoDB and other non-JDBC agents keep their own limits). They apply to every Query Studio execution on that agent, not only to Query Forge: the Query Studio grid in the Control Plane, MCP execute_sql, and the AI helpers get the same per-call row cap and timeout. A higher row cap also means more rows held in the agent’s memory for each call. Restart the agent after you change one.
| Variable | Default | Description |
|---|---|---|
|
|
Most rows one Query Studio execution returns, and so the size of a Query Forge page. A higher value reads a large result, such as a big pre-aggregated side of a join, in fewer calls, each holding more rows in memory on the agent and on Core Hub. Set the same value on Core Hub too: Query Forge sizes the pages it asks for from Core Hub’s value, and an agent that returns fewer rows makes it ask for fewer. |
|
|
Longest a Query Studio execution may run, in milliseconds, whatever the caller asks. Raise it together with |
These limits are fixed: 32 concurrent JDBC requests with 256 waiting, 8 page queries per agent, and client frames of 1,000 rows.
Limitations
-
Federation performance depends on how much of a query the sources can run. A join across agents with no selective side (both sides above 50,000 rows) and no side unique on an integer key, a join on text keys, a
FULLouter join across agents, or a query on an agent without full pushdown reads the selected rows into Query Forge and stops at the scan cap. See Write fast queries from BI tools. -
The Stop button of SQL clients does not reach the server. See Cancel queries and set timeouts in BI tools.
-
A subquery correlated two or more levels up, and a correlated
NOT INin aSELECTlist or under anOR, are refused with SQLState0A000. See Correlated subqueries. -
gluesync.aiaccepts equality predicates only onquestion,conversation_id,pipeline_id,agent_id,provider_id, andspark_agent. Onlyquestionis required.pipeline_idandagent_idare optional; if you use them, supply both together.
See also
-
Query Studio — the in-portal SQL workbench Query Forge builds on.
-
Query pipeline data with Query Studio: how-to guide for the workbench that Query Forge builds on.
-
PII discovery and masking — how columns are classified and masked.
-
AI as SQL virtual catalog — detailed
gluesync.aiSQL contract and examples. -
AI models and LLM providers — configure cloud or local Ollama models.