Mark Van de Wiel

The Real-Time Data Reality

Today’s world can truly be described as a real-time world. From 24×7 news to social media to stock trading to traffic updates, everyone expects the latest information to be available on demand on their smartphone.

With this real-time access to data in our personal lives, it is not surprising that similar expectations are now being set for access to business information. Businesses and government entities are increasingly demanding real-time analytics to optimize customer service, marketing campaigns, logistics, inventory, cash flow, human capital, and public safety.

In response to this demand, there has been an explosion in the number of business intelligence tools that enable business analysts and users to analyze and visualize their data in real-time to meet these needs. But there is a problem with this picture. Unlike news, social media and financial trading systems that were developed to be real-time, most business information systems were developed to process data on a periodic, not real-time, basis.

“We’ve got the data and tools we need, but we can’t get access to our data quickly enough to make good decisions”.

What’s more, the explosion of data volumes and data types means that even the best designed business information systems are struggling to provide the level of data access that business users are demanding. For today’s business manager, the business intelligence problem has morphed from “we don’t have the data and tools we need to make good decisions” to “we’ve got the data and tools we need, but we can’t get access to our data quickly enough to make good decisions”.

The traditional approach to providing the data to power business intelligence has been to extract data from source systems (e.g. ERP, CRM, databases powering custom applications, etc.), to store this in data warehouses and allow users to run reports and queries against this data. With data that is updated on a periodic basis or when a specific event occurs, this approach can work well. However, data warehouses typically do not support data being updated in real-time or streamed from external sources such as social media feeds.

To meet the need for real-time data access without imposing a major performance overhead on the systems originating the data to be accessed, a new approach is needed. This approach is often referred to as log-based replication. In this approach, instead of reading the operational databases that contain the raw information, the log-files that underlie this data are monitored and, when changes are detected, those changes are replicated (copied) to other systems, for example, analytic systems.

Log-Based Replication (CDC)

Log-based replication, therefore, provides a way to deliver the data needed to power real-time analytics without the performance problems and complexity issues associated with traditional approaches. Because such real-time analytics are so much in demand for many business and government organizations, many software vendors ranging from industry giants such as Dell and Oracle to specialty vendors such as Attunity and HVR have developed solutions that use this approach.

how do you choose which of these replication solutions is right for your business?

As a business user faced with finding a way to get data from your source systems to your analytics tools, how do you choose which of these replication solutions is right for your business? Do you go with a vendor whose name you recognize? Do you go with a solution chosen by IT that may or may not fit your business needs? Or, do you look at the specialty vendors?

Checklist for Selecting a Replication Software Provider

There is no right or wrong answer to these questions. However, as one of the specialty vendors focused on solving this problem, HVR would like to suggest a few things for you to consider when making this decision.

  1. Does the solution you are considering access all of the data types you need to support your analytics?
  2. Can this solution scale to support the volumes of data that you need without the solution becoming complex?
  3. How long will it take to implement the solution and for you to see business results?
  4. Will you need dedicated or highly trained resources to manage the solution?
  5. Will the solution tie you to a software vendor who may have an agenda that favors, for example, their own database products or hardware over competitive products?
  6. How good and how accessible is the vendor’s support?
  7. How easy is the vendor to work with?

As we said, there is no right or wrong answer but, with real-time analytics becoming one of the key drivers of today’s business, it is a decision that deserves careful consideration. At HVR, we’ve built our reputation and business helping organizations from leading industrial groups to government organizations such as the U.S. Coast Guard to transportation organizations such as Lufthansa, to on-line retailers such as ASOS,  make the right decision for their unique real-time data replication needs and we’re here to help you do the same.

Want to see HVR in action — join us for a live demo!

About Mark

Mark Van de Wiel is the CTO for HVR. He has a strong background in data replication as well as real-time Business Intelligence and analytics.

Test drive
Contact us