Change Data Capture
Change data capture (CDC) is how HVR replicates data changes in real-time. As its name implies, CDC identifies changes and can then synchronize incremental changes with another system or store an audit trail of changes.
CDC comes in multiple flavors, including trigger based and log-based. HVR gives you the option to use both types. However, log-based CDC is superior since it can be applied to all possible scenarios including systems with extremely high transaction volumes. Our product includes log-based CDC for all supported relational database sources.
The biggest benefits of log-based CDC include:
- Minimal Impact: Log-based CDC has less impact on the database because it reads directly from the logs without directly impacting the transaction. In contrast, trigger-based CDC creates triggers on tables that require change data capture, and firing these slows down transactions.
- Fast performance: HVR directly reads the logs on the file system allowing highly efficient change data capture, supporting large volumes of data.
- More flexibility: Log-based capture supports more data operations such as truncates, and enables support for DDL capture.
Additional Resources on Change Data Capture
BLOG: Change Data Capture for Real-Time BI. Using Change Data Capture to Augment your ELT/ETL Solutions
Reporting on near real-time data is a must in today’s landscape. To stay competitive, companies are now implementing more of an operational BI strategy for day-to-day tactical decision making to increase profits and react faster to changing attitudes. This cannot be accomplished by ETL tools on its own.
BLOG Excerpt: A Guide to Selecting Replication Software
“To meet the need for real-time data access without imposing a major performance overhead on the systems originating the data to be accessed, a new approach is needed. This approach is often referred to as log-based replication. In this approach, instead of reading the operational databases that contain the raw information, the log-files that underlie this data are monitored and, when changes are detected, those changes are replicated (copied) to other systems, for example, analytic systems.
Log-based replication therefore provides a way to deliver the data needed to power real-time analytics without the performance problems and complexity issues associated with traditional approaches. Because these real-time analytics are so much in demand for many business and government organizations, many software vendors ranging from industry giants such as Dell and Oracle to specialty vendors such as HVR have developed solutions that use this approach.”
Whitepaper: Achieve Greater Business Agility with Real-Time Analytics. Why ETL Data Integration Approaches are Insufficient and How Log-Based CDC Enables Continual Integration.