Connection Pool management
Connection pooling is a crucial aspect of database performance optimization in Gluesync. It helps manage and reuse database connections efficiently, reducing the overhead of creating and destroying connections for each database operation.
|
This guide provides detailed information specifically about connection pool sizing and optimization. For general Gluesync configuration recommendations, see our Best Practices Guide. |
Default configuration
The connection pool setting in Gluesync is configured in the Advanced Settings section of each agent, under the "Max Connection Count" option.
It comes with the following default settings:
-
Maximum of 100 connections per agent (configurable via "Max Connection Count")
-
One connection per entity (table/mapping) by default
-
Automatic connection grouping for deployments with >100 entities
Best practices for connection pool management
For small to medium deployments (≤100 entities)
For deployments with less than 100 entities:
The default settings are usually any good but you can always adjust them to better suit your needs:
-
In a resource-constrained environment, you can reduce the number of connections to save resources (CPU load);
-
Monitor database performance and adjust the max connection count if necessary.
For large deployments (>100 entities)
When managing more than 100 entities, consider:
-
Be wise on the max connection count setting: the highe the number the more resources you’ll need (compute power at source database);
-
Adjusting max concurrency settings based on:
-
Source database connection limits
-
Available system resources
-
Performance requirements
To learn more about connection pooling and its impact on performance, refer to the HikariCP documentation, where HikariCP is the connection pool engine used by Gluesync.
Connection Pool sizing considerations
|
More connections don’t always mean better performance. GS has a default of 100 connections, which satisfies 99% of use cases. |
-
Connection Pool Sizing Considerations: The number of connections should be carefully calculated based on several factors:
-
Number of simultaneous tables receiving changes
-
Compute power at the source database
-
Allowed delay in data fetching between event generation and processing
-
-
Connection Pool Sizing Guidelines:
-
Start with default pool size (100 connections)
-
For low-change environments (few changes per hour in small tables):
-
Recommended pool size: 20-25 connections
-
Recommended polling interval: 100ms
-
This configuration allows for quick table hydration (sub-second)
-
-
Increase pool size if:
-
You see connection timeouts
-
Database performance is good
-
System resources are available
-
-
Decrease pool size if:
-
Database load is too high
-
System resources are constrained
-
-
-
Monitor database load:
-
Watch for connection timeouts
-
Track database CPU and memory usage
-
Adjust pool size based on monitoring data
-
-
Connection Grouping:
-
Use when source system has connection limits
-
Benefits include:
-
Reduced connection overhead
-
Better resource utilization
-
Improved stability
-
-
Group related entities together to maximize efficiency
-
Monitoring and troubleshooting
-
Key Metrics to Monitor:
-
Active connections
-
Connection wait times
-
Database response times
-
System resource usage
-
-
Common Issues and Solutions:
-
Connection timeouts: Increase pool size or adjust connection timeout settings
-
High database load: Reduce pool size or implement connection grouping
-
Resource constraints: Optimize pool settings based on available resources
-