Search docs.molo17.com

    Redis glyph screenshot Redis agent

    Target Redis

    For a tested walkthrough with an ACL user, a snapshot and CDC, read Set up a Redis target for snapshots and CDC.

    Prerequisites

    To have Gluesync working on your Redis instance you will need to have:

    • A Redis server that Core Hub reaches without TLS;

    • A password for the Redis default user, unless your Redis has no password;

    • Permission for the default user to run the commands HSET, HSETNX and DEL on the keys that Gluesync writes, and the command SELECT when the database index is not 0.

    The agent authenticates with the password only. Redis signs it in as the default user. The Username field is required in the form, but the agent ignores its value.

    For example, this command gives the default user the minimum commands. Replace the password and the key pattern:

    redis-cli ACL SETUSER default reset on '>change-me' '~public::*' +hset +hsetnx +del

    How the data is written

    The agent writes each source row as a Redis hash. An insert or an update runs HSET (and HSETNX for unchanged columns), and a delete runs DEL on the key.

    The key is the default document key, <scope>::<collection>::<key values>, or the custom prefix and separator that you set in the entity. The hash holds only the columns that you map. Here is an example of a hash that Gluesync wrote:

    127.0.0.1:6379> HGETALL public::customers::1
     1) "id"
     2) "1"
     3) "name"
     4) "Name-1"
     5) "surname"
     6) "Surname-1"
     7) "address"
     8) "1 Integration Test Street"
     9) "gender"
    10) "Male"
    11) "phone"
    12) "+39-555-0001"
    13) "email"
    14) "customer-1@example.com"

    Libraries for many programming languages convert a hash directly into a native object. Here is an example in Python:

    In [6]: r.hgetall("public::customers::1")
    Out[6]:
    {'id': '1',
     'name': 'Name-1',
     'surname': 'Surname-1',
     'address': '1 Integration Test Street',
     'gender': 'Male',
     'phone': '+39-555-0001',
     'email': 'customer-1@example.com'}

    The hash is automatically converted into a Python dictionary.

    Keep these facts in mind:

    • Redis stores every value as a string.

    • The agent writes a null column as an empty string. The field stays in the hash.

    • The agent writes bytea and array columns as the text of a Java object, for example [B@de32bc9. This text carries no data. Exclude these columns in the Field mapping dialog.

    • A truncate on the source removes no keys. The agent logs a warning and skips it. Delete the keys yourself with DEL or UNLINK.

    • The agent has no bulk load.

    • This release has no working TLS connection to Redis. A Redis server that accepts only TLS resets the connection.

    • The Control Plane connects to a single Redis host. It does not expose Redis Cluster or additional hosts.

    • Every connection failure shows the same message, Unable to connect to <host>/<unresolved>:<port>. Check the password, the database index, the SELECT permission and the network.

    Setup via Web UI

    • Hostname / IP Address: Hostname or IP Address of your Redis server;

    • Port: Optional, defaults to 6379;

    • Database name/index: Number of the logical database, 0 by default;

    • Username: Required by the form. Enter default. The agent ignores the value;

    • Password: Password of the default user;

    • Disable auth: Switch on only when your Redis has no password.

    Specific configuration

    This agent has no specific configuration properties.

    Setup via Rest APIs

    Here following an example of calling the Core Hub’s Rest API via curl to setup the connection for this Agent.

    Connect the agent

    curl --location --request PUT 'http://core-hub-ip-address/pipelines/{pipelineId}/agents/{agentId}/config/credentials' \
    --header 'Content-Type: application/json' \
    --header 'Authorization: ••••••' \
    --data '{
            "hostCredentials": {
            "connectionName": "myAgentNickName",
            "host": "host-address",
            "port": 6379,
            "databaseName": "0",
            "username": "default",
            "password": "my-password",
            "disableAuth": false
          }
    }'