Data Replication Tips and Tricks
In my work as Director of Enablement for HVR, I help customers get up and running with HVR software. Because HVR is extremely flexible and offers so many configuration options, part of my mission is to teach customers how to best implement and configure HVR as well as take advantage of HVR capabilities for their use case. This blog is a compendium of five of my best data replication tips-and-tricks for helping customers make the most of HVR in any environment.
Tip #1: Adding new sources and targets.
The HVR architecture makes it easy to add new sources and replicate data between them.
Simply install the HVR Hub on a dedicated server separate from the servers that run your data sources and targets. Having the Hub in the middle tier gives you a centralized, logical view of sources and targets across your environment that makes it easy to design, deploy and monitor all replication tasks.
To add a new source, you can simply install an HVR agent and start a listener process. Once you define the connection properties to that new location, you are ready to start replicating data.
Tip #2: Loading your data while capturing changes
When initially loading your data into HVR, the best practice is to start an incremental data capture, and then run your initial load. HVR automatically tracks when the change capture was started, when the initial load began, and when the initial load was completed. HVR also handles all the appropriate resiliency to ensure that all data comes over as part of the initial refresh. Once the initial load is finished, the HVR integrate function processes all the captured changes that have queued up, automatically synchronizing the source and target.
Without HVR, DBAs would have to keep source transaction logs available until the initial load is complete, which can mean altering their retention policy. For example, say the data retention policy is to keep only two days’ worth of data and archive the data after that period. If the initial load takes five days, the DBA would have to change the retention policy to ensure she doesn’t lose the extra three days’ worth of data. With HVR, DBAs can maintain their existing retention policies regardless of how long the initial load runs.
Tip #3: Performing initial data loads faster
HVR provides the Refresh feature to allow users to initially load data directly from source tables. To do the initial materialization of tables, users simply create target tables based on the source layouts and then move the data from the source tables to the target. HVR provides built-in performance options to help reduce the time it takes to initially load the tables. For example, you can ask HVR to run jobs in parallel, either by table or location. For large tables, you can ask HVR to slice the data into ranges of data for improved parallelism.
Behind the scenes, HVR further improves initial load performance by using the native bulk load capabilities from the database vendor, which typically offer the most efficient way to load the data without requiring HVR users to configure utilities or write scripts.
Tip #4: Improving data capture and replication performance
If you have a centralized HVR Hub and want higher throughput and lower latency for near real-time data replication, place an HVR agent on your local source server. The agent will capture incremental changes from the local transaction logs as they occur, which means you’re pulling data almost directly from the cache. Since HVR only sends highly compressed messages over the network, it eliminates the need for disk I/O or local disk storage requirements. Not only does this save on resources, but it also saves on cost.
Additionally, because HVR stores only incremental changes, it has a minimal impact on the network, which further accelerates throughput.
Tip #5: Capturing data from the cloud
Although HVR has minimal impact on production systems, in some cases DBAs need to have zero impact on them. Zero impact is often necessary when the DBA wants to capture data from the cloud and the cloud service provider won’t allow them to install an agent on the cloud infrastructure.
In this case, DBAs can employ a remote or archival capture topology in which HVR runs directly off the Hub without an agent listening on the source system. While there is a small performance penalty, this architecture allows the DBA to capture that data from the cloud and replicate it to wherever necessary.
Use HVR more effectively today
HVR offers considerable flexibility in meeting the demands of individual customer use cases. These tips offer fundamental principles that will help you get started using your HVR solution more quickly and effectively. For more information, you may contact me: education (at) hvr-software.com or by clicking the contact us button.