HTTP Source and HTTP Target agent changelog

Initial release: Introduced the HTTP agent (courier-agent), the first agent of Courier, the REST editor of the Core Hub web UI. As HTTP Target it delivers every captured change as one HTTP call toward a REST API described in Courier (collections, endpoints, bindings); as HTTP Source it serves Courier inbound endpoints and turns every authenticated request into a pipeline row.

Courier outbound definitions: base URL and authentication (None, Basic, Bearer, API key in header or query) on the collection, with secrets encrypted at rest and masked in every response; a base URL cannot carry user:pass@. Verb, path template, headers, body type and typed bindings (PATH, QUERY, HEADER, BODY) on the endpoint. Header values are secrets too: encrypted at rest, shown as ********, and bound to the origin of the base URL they were entered for; moving a collection to another origin while its endpoints hold such values needs a confirmation (?force=true).

Request rendering: path and query values URL-encoded, JSON values escaped and typed (NUMBER, BOOLEAN), numbers in plain notation with every digit, binary values in Base64, arrays as JSON arrays, unresolved bindings as null; form-encoded bodies carry the BODY bindings only. A binding takes its default only when the row does not carry its column, and a NULL in a required binding fails the write; a delete keeps the defaults of its PATH and QUERY bindings only. The collection’s credentials and the endpoint’s header values are sent to the origin of its base URL only.

Operation forwarding: the row operation (Insert, Update, Delete) is sent in the body operation field and available as the {__operation} placeholder.

Idempotency key: every row request carries an Idempotency-Key, a SHA-256 of the change’s source position, table, operation, values and before-images, the same each time Core Hub hands the change over again, after a retry or a resume. A DELETE answered 404 or 410 counts as delivered.

Safe outbound calls: loopback, link-local, cloud metadata and multicast addresses (NAT64 included) and Core Hub’s own port are refused unless GS_COURIER_ALLOW_LOCAL_EGRESS=true; the client connects to the addresses checked; redirects are never followed; failures are redacted of credentials, query values and user:pass@. The Send test request button and a history of the latest 50 results of each endpoint, with credentials and endpoint headers shown as ••••.

Pipeline integration: collections, endpoints and bindings exposed through discovery as schemas, tables and columns; column renaming through the entity mapping; endpoint lookup scoped by collection.

Write protocol: every transaction stored as consumed or failed with its metrics; a non-2xx answer or a transport error fails the write and is retried from the checkpoint.

Embedded deployment: the agent runs inside Core Hub and reads the Courier definitions from it directly, with no token of its own.

Bootstrapper support: the courier block of the target agent configuration creates or updates the Courier definitions before the agent is configured.

HTTP Source: the agent can be assigned to a pipeline as its source. It serves Courier inbound write endpoints with bindings (pipeline endpoints) on its own listener (port 8090 by default), turns every authenticated request into one row, queues it on the local disk, forces it to the disk and answers 202 with the row’s sequence; the pipeline delivers the rows to any target, at least once and in order per endpoint. See Source setup.

Courier inbound pipeline endpoints: CREATE, UPDATE and DELETE endpoints with typed bindings (PATH, QUERY, HEADER, BODY, nested body fields with dotted names), key bindings (synthetic _seq key when there is none), an optional operation binding (Insert, Update, Delete), JSON or form-encoded bodies, validation on save and a Preview the row panel. A collection has a one-segment base path; a POST of a collection or endpoint whose id is already stored is refused with 409. While Core Hub masks personal data, a request carrying a value shaped like a mask is refused with 400, so a client cannot write back the masked value a READ answered.

Inbound authentication: Personal API Token with the HTTPS agent permission scope (gluesync.https_agent), checked against the pipeline and HTTP Source agent and against the role of its owner (a Manager or a Super Admin may write), with a per-collection token allowlist and blocklist. Core Hub saves a collection only while an active token can call it. Verdicts are cached on the agent for at most a minute and never past the token’s expiry, and cleared on revocation, on a user change and on every change made in Courier.

Backpressure: limits on body size, requests in flight, request time and body speed, unconfirmed queue size per endpoint and in total, and free disk, answered with 408, 413 or 503 and Retry-After; configurable listener threads, accept queue, acceptors, bind address and TLS keystore. Without a keystore the listener serves plain HTTP on loopback only, unless inboundAllowPlainHttp is set.

Observability: http_source_requests_total{endpoint,status}, queue lag and read checkpoint metrics, refusals summed up in one log line per minute.