Metrics for Statistics

Last updated on Jun 25, 2021

This section lists and describes the metrics used by HVR for tracking the replication performance. By using these metrics the performance of replication is represented graphically in Statistics and Topology for easier tracking of performance.

The unit of measurement is different for each metrics, following are the units and its description:

Unit

Description

%

Percent. When used for compression this means ratio of bytes left after compression. So if 100 bytes are compressed by 70% then 30 bytes remain.

bytes

changes

Changes of tables affected by insert, update or delete statements. Updates are sometimes moved as 2 rows (before-update and after-update).

cycles

A capture or integrate cycle.

files

int

Integer number

lines

Number of message lines written to log file. A single line could be an error message which mentions multiple failed changes

rows

Rows of tables affected by insert, update or delete statements.

rows/sec

Measures the average of 'rows' or 'changes' replicated per second.

runs

Number of times a job has been run.

secs

Seconds

string

time

Timestamp

transaction

Unit 'transaction' means a group of changes terminated by a commit (not just a changed row).

The metrics gathered by the hvrstats job is divided into the following groups: 

In the following tables,

  • the metric names highlighted using green color indicate the basic/important metrics that are gathered by the hvrstats job when action Scheduling /StatsMetrics=LITE is defined. Also note that this is the default metric gathering mode since HVR 5.7.0/17 and 5.7.5/11.
  • the column Source indicates location from where the hvrstats job gathers the metric information - router file or HVR log file.

Latency Metrics

Metric Name

Unit

Source

Remarks

Capture Latency Min

secs

Log file

This is the minimum latency value as reported by the capture job. This value is taken from the log message of the capture job. Therefore a value is only available when the capture job captures some rows. In older HVR versions this metric was called Minimum Capture Latency.

Capture Latency Max

secs

Log file

This is the maximum latency value as reported by the capture job. This value is taken from the log message of the capture job. Therefore a value is only available when the capture job captures some rows. In older HVR versions this metric was called Maximum Capture Latency.

Integrate Latency Min

secs

Log file

This is the minimum latency value as reported by the integrate job. This value is taken from the log message of the integrate job. Therefore a value is only available when the integrate job integrates some rows. In older HVR versions this metric was called Minimum Integrate Latency.

Integrate Latency Max

secs

Log file

This is the maximum latency value as reported by the integrate job. This value is taken from the log message of the integrate job. Therefore a value is only available when the integrate job integrates some rows. In older HVR versions this metric was called Maximum Integrate Latency.

Capture Rewind Interval

secs

Router file

 

Captured Row Counts Metrics

Metric Name

Unit

Source

Remarks

Captured Inserts

rows

Log file

Number of captured inserts.

Captured Updates

rows

Log file

Number of captured updates. Updates are captured as 2 rows not 1, unless Capture /NoBeforeUpdate is defined. This means captured inserts + updates + deletes can be less than captured rows.

Captured Deletes

rows

Log file

Number of captured deletes.

Captured ChangesrowsLog fileNumber of all captured changes. That is captured inserts + updates + deletes. Note that captured changes can be less than captured rows.
Captured Changes BackdatedrowsLog fileA 'backdated' version of 'Captured Changes'. Backdated means latency in the message is used to assign the value to an time earlier than the message's timestamp. For example if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Changes' for 16:55 and also to 'Captured Changes Backdated' for 16:25. Visual comparison of a back dated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible.

Captured Rows

rows

Log file

Sometimes changes (e.g. updates) are captured as 2 rows not 1. So captured inserts + updates + deletes can be less than captured rows

Captured Rows Backdated

rows

Log file

A 'backdated' version of 'Captured Rows'. Backdated means latency in the message is used to assign the value to an time earlier than the message's timestamp. For example if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Rows' for 16:55 and also to 'Captured Rows Backdated' for 16:25. Visual comparison of a back dated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible. In older HVR versions this metric was called DBMS Log Rows.

Captured Skipped Rows

rows

Log file

Changes are skipped by HVR 'controls', e.g. after an on-line refresh.

Augmented Rows

rows

Log file

This counts situations where capture job performs DB query to fetch extra column value(s) to 'augment' other column values read from DBMS logging, These operations are relatively slow (DB query needed).

SAP Augment Selects

rows

Log file

Only occurs when SapXForm engine is used. Counts situations where capture job performs DB query to fetch extra rows to 'augment' SAP cluster rows read from DBMS logging, These operations are relatively slow (DB query needed).

Captured DDL Statements

rows

Log file

Measures number of DDL statements processed by action AdaptDDL. These can be followed by on-line refreshes which can cause row skipping. In older HVR versions this metric was called Captured DDL.

Integrated Change Counts Metrics

Metric Name

Unit

Source

Remarks

Integrated Inserts

changes

Log file

Number of integrated inserts.

Integrated Updates

changes

Log file

Number of integrated updates. Although updates can be captured as 2 rows not 1, this metric measures the updated changes, not underlying rows.

Integrated Deletes

changes

Log file

Number of integrated deletes.

Integrated Changes

changes

Log file

Number of all integrated changes. That is integrated inserts + updates + deletes. Sometimes changes (e.g. updates) are captured as 2 rows not 1. Such an update would count as 1 here (not 2). Therefore the number of changes can be less than the number of rows. This count does NOT include DDL changes (or rows refreshed due to DDL changes).

Integrated Skipped Changes

changes

Log file

Changes are skipped by HVR 'controls', e.g. during an on-line refresh or if a job is restarted after some interrupt.

Changes Coalesced Away

changes

Log file

Integrate /Burst and /Coalesce will both squeeze consecutive operations on same row (e.g. insert + 3 updates) into a single change.

Failed Inserts Saved

changes

Log file

Failed inserts written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined. possibly because row already existed.

Failed Updates Saved

changes

Log file

Failed updates written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because row did not exist.

Failed Deletes Saved

changes

Log file

Failed delete written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because row did not exist.

Failed Changes Saved

changes

Log file

Total number of changes ( inserts + updates + deletes) which failed and were written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because row did not exist.

Collision Changes Discarded

changes

Log file

Change which was discarded due to collision detection (action CollisionDetect) which decided change was older then current row in target DB. In older HVR versions this metric was called Discarded Changes.

Empty Updates DiscardedchangesLog fileUpdates produced by the source database, which were discarded because no replicated column was changed.

Transactions Metrics

Metric Name

Unit

Source

Remarks

Captured Transactions

transaction

Log file

This metric counts number of COMMITs of transactions which contain changes to replicated tables. Note that its interesting to see if a system is dominated by large (e.g. 1000 row/commit) transactions, but comparing this filed with 'Captured Rows' only shows an average.

Captured Transactions Backdated

transaction

Log file

A 'backdated' version of 'Captured Transactions'. Backdated means latency in the message is used to assign the value to an time earlier than the message's timestamp. For example if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Transactions' for 16:55 and also to 'Captured Transactions Backdated' for 16:25. Visual comparison of a back dated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible. In older HVR versions this metric was called Transactions Logged in DBMS .

Integrated Transactions

transaction

Log file

Integrate will often bundle lots of smaller 'Captured Transaction' values into fewer 'Integrated Transactions' for speed. This metric measure these bundled commits, not the original ones.

Durations Metrics

Metric Name

Unit

Source

Remarks

Capture Duration Total

secs

Log file

In older HVR versions this metric was called Total Capture Duration .

Capture Duration Max

secs

Log file

In older HVR versions this metric was called Maximum Capture Duration .

Capture Duration Average

secs

Log file

In older HVR versions this metric was called Average Capture Duration .

Integrate Duration Total

secs

Log file

In older HVR versions this metric was called Total Integrate Duration .

Integrate Duration Max

secs

Log file

In older HVR versions this metric was called Maximum Integrate Duration .

Integrate Duration Average

secs

Log file

In older HVR versions this metric was called Average Integrate Duration .

Speed Metrics

Metric Name

Unit

Source

Remarks

Capture Speed Max

rows/sec

Log file

This is maximum speed reached by during this time period. For a 'cl*' value its the max speed for any job; if this is aggregated In older HVR versions this metric was called Maximum Capture Change Speed .

Capture Speed Average

rows/sec

Log file

In older HVR versions this metric was called Average Capture Change Speed .

Integrate Speed Max

rows/sec

Log file

In older HVR versions this metric was called Maximum Integrate Change Speed .

Integrate Speed Average

rows/sec

Log file

In older HVR versions this metric was called Average Integrate Change Speed .

Cycles Metrics

Metric Name

Unit

Source

Remarks

Capture Cycles

cycles

Log file

Count of capture cycles whose start occurred during this time period. This does not include 'sub-cycles' or 'silent cycles', but does include 'empty cycles'. A 'sub-cycle' happens when a busy capture job emits a block of changes but has not yet caught up to the 'top'. It can be recognized as an extra 'Scanning' message which is not preceded by a 'Cycle X' line'. A 'silent-cycle' is when a capture job sees no change activity and decides to progress its 'capture state' files but without writing an line in the log (above every 10 secs). A 'empty-cycle' is when a capture job sees no change activity and does write line in the log (above every 10 mins).

Integrate Cycles

cycles

Log file

Count of integrate cycles whose start occurred during this time period. Note that the integrate activity may fall into a subsequent period.

Byte I/O Metrics

Metric Name

Unit

Source

Remarks

Routed Bytes Written

bytes

Log file

Amount of compressed bytes HVR routed from the capture machine to the hub machine. In older HVR versions this metric was called Routed Bytes .

Routed Bytes Written Uncompressed

bytes

Log file

Amount of uncompressed bytes HVR routed from the capture machine to the hub machine. These routed bytes refer to HVR's representation of routed rows in memory. This is different from the DBMS's 'storage size' (DBMS's storage of that row on disk) For example, if a table has a varchar(100) column containing 'Hello World' which HVR manages to compress that down to 3 bytes. HVR's memory representation of varchar(100) is 103 bytes, whereas the DBMS storage is 13 bytes. In older HVR versions this metric was called Routed Bytes (uncompressed) .

Captured File Size

bytes

Log file

In a file to database or file to file replication channel, this counts the total size of the captured files. In older HVR versions this metric was called Captured File Size .

Capture DbmsLog Bytes

bytes

Log file

Amount of bytes HVR captured from the database's log file. In older HVR versions this metric was called Capture Log Bytes Read .

Capture DbmsLog Bytes Backdated

bytes

Log file

A 'backdated' version of 'Capture DbmsLog Bytes'. Backdated means latency from HVR's log message is used to assign the value to an time earlier than the message's timestamp. So this reflects when the databse wrote the bytes, not when HVR captured them. In older HVR versions this metric was called DBMS Log Bytes Written .

Compression Metrics

Metric Name

Unit

Source

Remarks

Compression Ratio Max

%

Log file

When transporting table rows, HVR reports its 'memory compression' ratio; the number of compressed bytes transmitted over network compared with HVR's representation of that row in memory (see 'Routed Bytes Written' and 'Routed Bytes Written Uncompressed'). This different from the DBMS's 'storage compression ratio' (number of compressed bytes transmitted compared with DBMS's storage of that row on disk. Example: Say a table has a varchar(100) column containing 'Hello World' which Hvr manages to compress that down to 3 bytes. Hvr's memory representation of varchar(100) is 103 bytes, wheres the DBMS storage is 13 bytes. In this case Hvr's 'memory compression' ration is 97% (1-(3/103)) whereas the storage compression ratio would be 77% (1-(13/103)). In older HVR versions this metric was called Maximum Compression Ratio .

Compression Ratio Total
Log file
Compression Ratio Count
Log file

Compression Ratio Average

%

Log file

In older HVR versions this metric was called Average Compression Ratio .

Replicated Files Metrics

Metric Name

Unit

Source

Remarks

Captured Files

files

Log file

In a file to database or file to file replication channel, this counts the number of captured files.

Integrated Files

files

Log file

In a file integration this counts the number of integrated files.

Failed Files Saved

files

Log file

 

Errors/Warnings Metrics

Metric Name

Unit

Source

Remarks

Errors

lines

Log file

Counts number of errors (lines matching F_J*) in the job logfile. Such an error line could affect multiple rows, or could affect one row and be repeated lots of times (so it counts as multiple 'errors'). In older HVR versions this metric was called Number of Errors .

^Errors

string

Log file

Annotation: Text of the most recent error line as additional information for metric Errors. In older HVR versions this metric was called Last Error .

Errors F_J*

lines

Log file

Groups the appearing errors by up to the 2 most significant error numbers. Example: Say error F_JA1234 happened. Then the metric 'Errors F_JA1234' will increase by 1. If the errors F_JA1234; F_JB1234: The previous error occured ...; F_JC1234: The previous error occured ...; appear then 'Errors F_JA1234 F_JB1234' will increase by 1.

^Errors F_J*

string

Log file

Annotation: Groups the appearing errors messages by up to the 2 most significant error numbers. Example: Say error F_JA1234 happened. Then the metric '^Errors F_JA1234' will hold the message of F_JA1234. If the errors F_JA1234; F_JB1234: The previous error occured ...; F_JC1234: The previous error occured ...; appear then '^Errors F_JA1234 F_JB1234' will hold the messages of F_JA1234 and F_JB1234.

Warnings

lines

Log file

Counts number of warnings (lines matching W_J*) in the job logfile. In older HVR versions this metric was called Number of Warnings .

^Warnings

string

Log file

Annotation: Most recent warning line. In older HVR versions this metric was called Last Warning .

Warnings W_J*

lines

Log file

Groups the appearing warning by the warning number. Example: Say warning W_JA1234 happened. Then the metric 'Warnings W_JA1234' will increase by 1.

^Warnings W_J*

string

Log file

Annotation: Groups the appearing warning messages by the warning number. Example: Say warning W_JA1234 happened. Then the metric '^Warnings W_JA1234' will hold the message of W_JA1234.

Refresh and Compare Runs Metrics

Metric Name

Unit

Source

Remarks

Bulk Compare Job Runs

runs

Log file

 

Bulk Compare Table Runs

runs

Log file

 

Bulk Refresh Job Runs

runs

Log file

 

Bulk Refresh Table Runs

runs

Log file

 

Row-wise Compare Job Runs

runs

Log file

 

Row-wise Compare Table Runs

runs

Log file

 

Row-wise Refresh Job Runs

runs

Log file

 

Row-wise Refresh Table Runs

runs

Log file

 

Refresh and Compare Job Duration Metrics

Metric Name

Unit

Source

Remarks

Bulk Compare Job Duration Min

secs

Log file

In older HVR versions this metric was called Bulk Compare Job Dur Min .

Bulk Compare Job Duration Max

secs

Log file

In older HVR versions this metric was called Bulk Compare Job Dur Max .

Bulk Compare Job Duration Average

secs

Log file

In older HVR versions this metric was called Bulk Compare Job Dur Avg .

Row-wise Compare Job Duration Min

secs

Log file

In older HVR versions this metric was called Row-wise Compare Job Dur Min .

Row-wise Compare Job Duration Max

secs

Log file

In older HVR versions this metric was called Row-wise Compare Job Dur Max .

Row-wise Compare Job Duration Average

secs

Log file

In older HVR versions this metric was called Row-wise Compare Job Dur Avg .

Bulk Refresh Job Duration Min

secs

Log file

In older HVR versions this metric was called Bulk Refresh Job Dur Min .

Bulk Refresh Job Duration Max

secs

Log file

In older HVR versions this metric was called Bulk Refresh Job Dur Max .

Bulk Refresh Job Duration Average

secs

Log file

In older HVR versions this metric was called Bulk Refresh Job Dur Avg .

Row-wise Refresh Job Duration Min

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Job Dur Min .

Row-wise Refresh Job Duration Max

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Job Dur Max .

Row-wise Refresh Job Duration Average

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Job Dur Avg .

Refresh and Compare Table Duration Metrics

Metric Name

Unit

Source

Remarks

Bulk Compare Table Duration Min

secs

Log file

In older HVR versions this metric was called Bulk Compare Table Dur Min .

Bulk Compare Table Duration Max

secs

Log file

In older HVR versions this metric was called Bulk Compare Table Dur Max .

Bulk Compare Table Duration Average

secs

Log file

In older HVR versions this metric was called Bulk Compare Table Dur Avg .

Row-wise Compare Table Duration Min

secs

Log file

In older HVR versions this metric was called Row-wise Compare Table Dur Min .

Row-wise Compare Table Duration Max

secs

Log file

In older HVR versions this metric was called Row-wise Compare Table Dur Max .

Row-wise Compare Table Duration Average

secs

Log file

In older HVR versions this metric was called Row-wise Compare Table Dur Avg .

Bulk Refresh Table Duration Min

secs

Log file

In older HVR versions this metric was called Bulk Refresh Table Dur Min .

Bulk Refresh Table Duration Max

secs

Log file

In older HVR versions this metric was called Bulk Refresh Table Dur Max .

Bulk Refresh Table Duration Average

secs

Log file

In older HVR versions this metric was called Bulk Refresh Table Dur Avg .

Row-wise Refresh Table Duration Min

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Table Dur Min .

Row-wise Refresh Table Duration Max

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Table Dur Max .

Row-wise Refresh Table Duration Average

secs

Log file

In older HVR versions this metric was called Row-wise Refresh Table Dur Avg .

Refresh and Compare Rows Metrics

Metric Name

Unit

Source

Remarks

Bulk Compare Rows

rows

Log file

 

Bulk Compare Packed Rows

rows

Log file

 

Row-wise Compare Rows

rows

Log file

 

Row-wise Compare Packed Rows

rows

Log file

 

Bulk Refresh Rows

rows

Log file

 

Bulk Refresh Packed Rows

rows

Log file

 

Row-wise Refresh Rows

rows

Log file

 

Row-wise Refresh Packed Rows

rows

Log file

 

Compare Differences Inserts

rows

Log file

 

Compare Differences Updates

rows

Log file

 

Compare Differences Deletes

rows

Log file

 

Refresh Differences Inserts

rows

Log file

 

Refresh Differences Updates

rows

Log file

 

Refresh Differences Deletes

rows

Log file

 

Router Rows Metrics

Metric Name

Unit

Source

Remarks

Capture Router Rows

rows

Router file

 

Integrate Router Rows

rows

Router file

 

Router Bytes Metrics

Metric Name

Unit

Source

Remarks

Capture Router Bytes

bytes

Router file

 

Integrate Router Bytes

bytes

Router file

 

Router Files Metrics

Metric Name

Unit

Source

Remarks

Capture Router Files

files

Router file

 

Integrate Router Files

files

Router file

 

Router Timestamps Metrics

Metric Name

Unit

Source

Remarks

Capture Rewind Time

time

Router file

 

Capture Emit Time

time

Router file

 

Capture Last Cycle Time

time

Router file

 

Capture Router Timestamp

time

Router file

 

Integrate Router Timestamp

time

Router file

 

Enroll File TimestamptimeRouter file