How to convert DBMoto’s metadata.xml to Gluesync Bootstrapper template

This guide explains how to convert an existing DBMoto metadata.xml file into a Gluesync Bootstrapper-ready template. This process helps migrate your database schema and mappings from DBMoto to Gluesync.

Prerequisites

Before starting the conversion process, ensure you have:

  • A valid DBMoto metadata.xml file

  • Python 3.8 or higher installed

  • The Gluesync Bootstrapper tool installed

  • Gluesync already set and configured (at least the pipeline to be used for the conversion)

Conversion process

Step 1: Prepare your environment

  • Clone the Gluesync Bootstrapper repository:

git clone https://gitlab.com/molo17-public/gluesync/gluesync-bootstrapper.git
cd gluesync-bootstrapper

  • Install the required dependencies (if any are specified in the repository)

  • Create a working directory for the conversion process:

mkdir -p gluesync-conversion

Step 2: Convert metadata.xml

  • Place your metadata.xml file in the working directory

  • Run the conversion script:

Basic conversion:

python3 parse_dbmoto_metadata_xml.py /path/to/your/dbmoto_export.xml

Advanced options:

python3 parse_dbmoto_metadata_xml.py /path/to/your/dbmoto_export.xml \
  --output-dir ./output_configs \
  --template ./custom-template.yaml

Refer to the [Gluesync Bootstrapper repository](https://gitlab.com/molo17-public/gluesync/gluesync-bootstrapper) for the most up-to-date instructions and additional options.

The script will:

  • Parse your metadata.xml file

  • Convert the schema definitions to Gluesync format

  • Generate a YAML template compatible with Gluesync Bootstrapper

Step 3: Review and adjust the template

After conversion, review the generated gluesync_template.yaml file. Pay attention to:

  • Table mappings and their relationships

  • Column definitions and data types

  • Any custom transformations or business rules

  • Connection settings and credentials

Make any necessary adjustments to ensure the template meets your requirements.

Step 4: Get authentication token

Before using the Gluesync Bootstrapper, you need to obtain a valid authentication token from the Core Hub API. You can get this token in two ways:

  1. Via Core Hub Web UI: Log in to the Core Hub Web UI and copy the token from your browser’s cookies or user profile settings.

  2. Via HTTP request: Use curl or any HTTP client to request a token:

curl -X POST https://<corehub-url>/authentication/login -H "Content-Type: application/json" -d '{"username":"<username>","password":"<password>"}'

The response will contain the authentication token that you’ll use in the --token parameter.

Step 5: Use the template with Gluesync Bootstrapper

Once your template is ready and you have your authentication token, use it with the Gluesync Bootstrapper:

python3 create_all_entities.py --pipeline <pipeline_id> --source-schema <source_schema> --target-schema <target_schema> --source-type <source_agent_type> --target-type <target_agent_type> --yaml-file <path_to_yaml_config> --token <auth_token> [--skip-errors] [--chunk-size <number>]

This command will:

  • Create the necessary database schema

  • Set up table mappings

  • Configure data transformations

  • Prepare the environment for data synchronization

Best practices

Data validation

  • Always validate the converted template in a test environment first

  • Verify that all required fields and relationships are preserved

  • Test with a small subset of data before full migration

Performance considerations

  • Optimize batch sizes for large datasets

  • Consider partitioning strategies for very large tables

  • Monitor performance during initial synchronization

Security

While the conversion tool does not store/convert database credentials from DBMoto metadata.xml file, it is recommended to: * Never store credentials in the template file * Use secure connection strings * Implement proper access controls

Documentation

  • Document any custom transformations

  • Maintain a version history of template changes

  • Keep track of any manual adjustments made during conversion

Troubleshooting

Common issues

  • Missing table definitions

  • Incorrect data type mappings

Solution steps

  1. Check the conversion logs for errors

  2. Verify source database connectivity

  3. Compare converted schema with source schema

  4. Adjust batch sizes and timeouts

  5. Consult the Gluesync bootstrapper documentation or reach out to Gluesync support for further assistance

Additional resources

Next steps

After successful conversion and testing:

  1. Deploy the converted template to your production environment

  2. Configure monitoring and alerting

  3. Set up regular maintenance tasks

  4. Document the migration process for future reference

This guide provides a comprehensive overview of converting DBMoto metadata.xml to Gluesync bootstrapper. For more detailed information about specific features or advanced configurations, refer to the Gluesync bootstrapper documentation.