Mark Van de Wiel HVR_live_demo_data_integration

SAP Data Integration

How to Unlock Your SAP Data, Continuously and in Real-Time

integrating data from SAPSAP is a popular platform for organizations large and small. Available since 1973, it’s used for everything from Collaborative Lifecycle Management (CLM) to supply chain management to industrial manufacturing and more. But SAP is rarely the only system companies have in place. While an organization might employ SAP as its ERP, it probably also runs a host of other applications, such as Salesforce, ServiceNow, and homegrown systems. As organizations of all sizes come to understand the value of analyzing data in a timely manner, they build real-time analytical systems, data warehouses and, perhaps, a cloud-based data lake running on technologies like Amazon S3 or Azure Data Lake Storage.

Given that most organizations have multiple systems in place, it’s important to be able to consolidate data from SAP in real-time with data from these other systems.

Challenges of Sharing SAP Data

The problem is that getting data out of SAP isn’t easy. For starters, SAP uses the proprietary ABAP language. While many consultants are available to help, this remains an impediment. Second, ABAP is batch only. Yet in our work with customers, we see the need for real-time or near real-time data loads. Third, SAP ERP Central Component (ECC), currently deployed by most organizations running SAP, stores some of the application’s most important data in Cluster and Pool Tables. It can be difficult to get data out of Cluster and Pool Tables because the data is stored in a compressed and encoded format. Using that data requires decompression/decoding. Now it’s true that as SAP migrates its ERP suite over to SAP HANA®, it’s doing away with SAP Cluster and Pool Tables, so this is a temporary problem. Nonetheless, given ECC’s pervasiveness, it will be available for another decade or longer.

Finally, as an SAP customer, you’ll need to migrate to SAP HANA sooner or later, and you probably want to continue to feed data lakes and other downstream analytical systems.

So, if you want to integrate data across all the data stores in your enterprise and you run SAP, you need a solution to unlock the data from Cluster and Pool Tables today, and SAP HANA tomorrow.

How Does HVR Give You Access to Your SAP Data?

HVR provides the underlying infrastructure that moves changes out of the source SAP and into both of these solutions. HVR also offers its own embedded SAP transform capability to decompress/decode the SAP Cluster and Pool Tables as part of a regular replication setup. With this solution, HVR can unlock data residing in SAP running on any one of the supported technologies (Oracle, SQL Server, DB2) and deliver it into any of the supported target technologies, decoding Cluster and Pool data as part of the setup. HVR also supports SAP HANA as a source for log-based Change Data Capture.

How does HVR’s Infrastructure Work for SAP Data Integration?

HVR offers log-based Change Data Capture to move data from your SAP ECC database into target technologies, including relational databases such as SQL Server, Teradata, Oracle, DB2, and SAP HANA, as well as streaming analytical systems like Kafka and big data technologies like Amazon S3. Our solution captures raw data as it is written to the source database transaction logs. This is true In the case of SAP and we capture the encoded data from cluster and pool tables. This solution does not require any ABAP code to extract the data from SAP—we simply capture the data from the transaction logs in real-time, with minimal impact on the SAP application.

HVR_distributed_architecture_data_integrationNext, the transform phase de-clusters or de-pools the data on the fly. Because SAP also has many Transparent tables where data does not need to be de-clustered or depooled, we only perform that operation as needed—not for every table. After the transformation, we can deliver SAP data to HVR supported targets (Oracle, SQL Server, DB2, SAP HANA, Snowflake, Amazon Aurora, Amazon S3, Kafka, HDFS, Hive, HBase and more), as well as include data from other sources (Oracle, industry-specific ERPs and much more). You can also apply filters that deliver a subset of data to select targets. As the data is moved from source to target, HVR also monitors the process, providing you with stats that let you see data flows and how many rows are moving, and alerts you if something goes wrong. Our Compare function lets you see whether data is in sync between the source and the target.

Find Out More

SAP is widely used. But it’s rarely the only system in most environments. Many organizations want to combine all their data. But they run into roadblocks in the form of Cluster and Pool tables in SAP ECC and earlier environments. HVR lets you capture and transform all your SAP data in real time so you can have data in a format you can use without putting additional load on your SAP application servers.

For more information on how HVR can help you unlock your SAP data, contact us.

About Mark

Mark Van de Wiel is the CTO for HVR. He has a strong background in data replication as well as real-time Business Intelligence and analytics.

Test drive
Contact us