Installation steps

Gluesync NoSQL to NoSQL for AWS S3

Prerequisites

In order to have Gluesync working on your AWS S3 instance you will need to have:

an S3 bucket;
valid user credentials with permissions: arn:aws:iam::aws:policy/AmazonS3FullAccess;
IAM access key obtained through AWS IAM console;
IAM secret key obtained through AWS IAM console;
name of the region where the buckets belogs to.

Basic configuration example

This module can be customized by using a configuration file, in JSON format. The file name to use must be specified as parameter when launching the app, with the -f or --file tokens. The file should be composed by union of common configuration file (see here Installation steps) and source specific configuration:

{
  ...
  // (source section omitted)
  ...
  "targetHost": "eu-central-1",
  "targetName": "myBucketName",
  "targetUsername": "IAMCredentialAccessKeyID",
  "targetPassword": "IAMCredentialSecret",
  ...

  "awsS3": {
    "entitiesMetadata": {
      "articles": [ "id", "article_name" ]
    },

    "allowObjectsDeletion": false
  },
}

The parameters that refer to a target are the parameters that describe the connection to the AWS S3 bucket.

targetHost: the name of the region where your bucket belongs to;
targetName: the name of the target bucket as represented inside your AWS S3 console (do not use the arn);
targetUsername: IAM Credential access key ID created under your AWS IAM console;
targetPassword: IAM Credential secret created under your AWS IAM console.

Other configuration parameters are the following:

AWS S3 specific configurations are listed under the awsS3 property:

timeoutSeconds (optional): defaults to 60. Number of seconds to set as timeout for operations involving communication with the destination;
allowObjectsDeletion (optional): defaults to true. If set to false tells Gluesync to skip any action that involves data deletion committed into the source database: Gluesync will propagate only events that involve Insert(s) and Update(s).
entitiesMetadata (optional): defaults to null. Key-value object that describes the binding of values per each table-fields that you would like to set as metadata values belonging to each stored object in the S3 bucket. Where key is the table (entity) name and as a value a list (array) of strings representing the column names.