Exploring Oracle GoldenGate Alternatives
Data replication, specifically, data replication using log-based Change Data Capture, has been a data integration option for organizations for over a decade. It is a solution that serves several traditional and modern use cases: distributing load and offloading reporting, consolidating data (e.g. in an Operational Data Store (ODS) or Data Warehouse), feeding a data lake, high availability (for databases and/or application schemas), or delivering data for streaming analytics (e.g. using Kafka).
What is Oracle GoldenGate?
Oracle acquired GoldenGate Software in 2009. GoldenGate Software’s main product at the time was called GoldenGate and delivered heterogeneous, real-time data replication. Oracle now calls this product Oracle GoldenGate, available either as a separately-licensed database option for the Oracle Database (on top of Oracle Enterprise Edition Database) or as a separately-licensed product for a non-Oracle Database.
Since the acquisition, a lot of effort has been put into embedding GoldenGate into the Oracle Database. With Oracle Database 19c, Oracle GoldenGate supersedes and displaces Oracle Streams, replacing a data replication capability that used to be included with the Enterprise Edition of the Oracle Database.
For non-Oracle Databases, Oracle tried its best to maintain support but had to make changes for Microsoft SQL Server and the Teradata Database (see Oracle – GoldenGate Statement of Direction for more details. New Big Data platforms have been added to Oracle GoldenGate in the separately-licensed GoldenGate for Big Data offering (in addition to the statement of direction linked above, see Oracle’s Technology Price List), increasing the license fees for organizations wanting to explore modern cloud platforms such as AWS S3 or Snowflake.
Data Replication Requirements
Most data replication use cases need an initial data load to get started. With that, to get data replication up and running, you generally require:
- Table creation, based on table definitions on the source, for heterogeneous environments. Data type mapping is particularly important to avoid data loss during replication.
- Initial data load, aligned with data replication, so that there is no overlap in data getting processed, and also no data is lost.
- Log-based Change Data Capture (CDC) to minimize the impact of CDC on the source system transactions, and to get maximum throughput with close to real-time access to data changes. Performance is important so CDC can keep up with the change volume generated on the source.
- Continuous integration/apply into the target(s). Knowing that different database technologies have different strengths, data integration must be optimized for maximum throughput, so that in a data consolidation use case the data changes coming from multiple high volume data sources can flow into a single target in near real-time.
- Data compare capabilities to validate source and target are in sync, in a heterogeneous setup.
- A graphical user interface to productively define data replication.
- Monitoring and alerting capabilities to have the tools to address data replication issues when they arise without having to watch the environments closely.
Considering this list of requirements, why would you consider an Oracle GoldenGate alternative?
Evaluating Oracle GoldenGate
Oracle GoldenGate covers requirement 3 on the list for a variety of database types, as well as number 4. Oracle can of course cover items 1 through 7 but you must look beyond the core Oracle GoldenGate offering to address all requirements:
- Start with Oracle Data Integrator (ODI), an Extract, Transform and Load (ETL) tool, to address requirements 1 and 2.
- Use Oracle GoldenGate for various database sources and targets as needed, and Oracle GoldenGate for Big Data as needed for a big data target to cover requirements 3 and 4.
- Add Oracle GoldenGate Foundation Suite, consisting of Oracle Veridata to cover requirement 5, Oracle GoldenGate Studio for number 6, and Oracle GoldenGate Management Pack (to plug into Oracle Enterprise Manager) to address requirement 7.
Not only do you have to license multiple products, but you also have to install a dizzying amount of software, and get it all working together. Let’s compare this process to an Oracle GoldenGate alternative.
HVR as the GoldenGate Alternative
How does HVR fare on the list of 7 requirements?
1. With HVR you typically discover/explore table definitions from the source database’s data dictionary. HVR retrieves key metadata from the data dictionary including schema, table and column names, data types and lengths, optionality, primary key information, as well as character set encoding. This information is the basis for mapping to the target, with HVR automatically taking care of mapping data types to compatible, loss-less data types on the target. Table definitions can be created on the target as part of HVR Refresh to perform a one-time data load.
2. The initial data load is covered by HVR Refresh. It uses the mapping rules from the channel, the group of tables to be replicated, indicating any transformations between source and target like changing schemas, table names, introducing extra columns or eliminating columns, etc.
3. HVR supports log-based CDC on many transaction processing databases, including Oracle, SQL Server, all flavors of DB2, PostgreSQL, MySQL, MariaDB, Ingres, and SAP HANA. Several HVR customers generate over 1 TB of transaction log changes per day on a single source, with HVR CDC keeping up with the load.
4. Integration into targets is optimized for maximum throughput, leveraging native bulk load capabilities and an approach called burst to use micro-batches for updating the target tables. Columnar and/or Massively Parallel Processing (MPP) databases like Snowflake, Teradata, Redshift, Greenplum, and others benefit greatly from using this approach. Of course, the integration uses the same transformations HVR Refresh uses for the table creation and initial data load.
5. HVR includes data validation, HVR Compare, as part of its core offering. Data validation again uses the same transformation rules defined in the channel definition and is available for targets in a heterogeneous setup including file-system-based data lakes.
6. The HVR software installation includes a Graphical User Interface (GUI) with a context-sensitive help function.
7. GUI-based monitoring is included, with historical time-series statistics available through the browser as part of HVR Insights. The Insights component also provides a topology chart, showing a single page overview of all data replication channels with high-level status information. Configurable alerts can be sent out as emails, Slack messages or integrated into another monitoring environment, including Oracle Enterprise Manager, through industry-standard SNMP.
The HVR download is less than 100 MB in size, and all of the software installs (untars) in a single top-level directory.
Additional Considerations for a GoldenGate Alternative
Beyond having a complete feature set to address your replication scenarios, there are additional questions to consider when reviewing solutions.
- What source and target databases/stores do you plan to use and are these supported?
- What capabilities are provided?
- What is the effort/cost to manage and maintain the solution?
- Where is your organization in adopting the Cloud, and what providers are being considered? How much does the license cost?
To date, HVR’s single technology focus continues to be data replication. To be able to best serve our clients, most of our employees have a rich background in data management, specifically data replication. Many of us have worked at other companies providing data replication technologies. I personally met several of my current colleagues for the first time when I joined GoldenGate Software in April 2008.
Is HVR a viable GoldenGate alternative? This is up to you to decide, and you should not just take our word for it. To help you decide we created an eBook, listing a set of additional considerations. The eBook also highlights a selection of customers who, despite having access to Oracle GoldenGate, chose HVR for some of their projects, indicating the reasons why.