Edit on GitHub
Metrics for Statisticslink
This section lists and describes the metrics used by HVR for tracking the replication performance. By using these metrics the performance of replication is represented graphically in Statistics and Topology for easier tracking of performance.
In the following tables,
- the metric names highlighted in green color indicate the essential metrics that are gathered by the hvrstats job when action Scheduling /StatsMetrics=LITE is defined. Also, note that this is the default metric gathering mode since HVR 5.7.0/17 and 5.7.5/11.
- the column Source indicates the location from where the hvrstats job gathers the metric information - HVR router file or HVR log file.
The metrics gathered by the hvrstats job are divided into the following groups:
Latency Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Latency Min | secs | log file | This is the minimum latency value as reported by the capture job. This value is taken from the log message of the capture job. Therefore a value is only available when the capture job captures some rows. In older HVR versions this metric was called Minimum Capture Latency. |
Capture Latency Max | secs | log file | This is the maximum latency value as reported by the capture job. This value is taken from the log message of the capture job. Therefore a value is only available when the capture job captures some rows. In older HVR versions this metric was called Maximum Capture Latency. |
Integrate Latency Min | secs | log file | This is the minimum latency value as reported by the integrate job. This value is taken from the log message of the integrate job. Therefore a value is only available when the integrate job integrates some rows. In older HVR versions this metric was called Minimum Integrate Latency. |
Integrate Latency Max | secs | log file | This is the maximum latency value as reported by the integrate job. This value is taken from the log message of the integrate job. Therefore a value is only available when the integrate job integrates some rows. In older HVR versions this metric was called Maximum Integrate Latency. |
Capture Rewind Interval | secs | router file | Time interval describing how far the capture job has to rewind back after a restart to safely capture all transactions that have not been emitted yet. This interval can grow if there are long-running transactions. |
Captured Row Counts Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Captured Inserts | rows | log file | Number of captured inserts. |
Captured Updates | rows | log file | Number of captured updates. Updates are captured as 2 rows, not 1 unless Capture /NoBeforeUpdate is defined. This means captured inserts + updates + deletes can be less than captured rows. |
Captured Deletes | rows | log file | Number of captured deletes. |
Captured ChangesSince v5.5.5/6 | rows | log file | Number of all captured changes. That is captured inserts + updates + deletes. Note that captured changes can be less than captured rows. |
Captured Changes Backdated | rows | log file | A 'backdated' version of 'Captured Changes'. Backdated means latency in the message is used to assign the value to a time earlier than the message's timestamp. For example, if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Changes' for 16:55 and also to 'Captured Changes Backdated' for 16:25. Visual comparison of a back-dated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible. |
Captured Rows | rows | log file | Sometimes changes (e.g. updates) are captured as 2 rows, not 1. So captured inserts + updates + deletes can be less than captured rows |
Captured Rows Backdated | rows | log file | A 'backdated' version of 'Captured Rows'. Backdated means latency in the message is used to assign the value to a time earlier than the message's timestamp. For example, if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Rows' for 16:55 and also to 'Captured Rows Backdated' for 16:25. Visual comparison of a back-dated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible. In older HVR versions this metric was called DBMS Log Rows. |
Captured Skipped Rows | rows | log file | Changes are skipped by HVR 'controls', e.g. after an online refresh. |
Augmented Rows | rows | log file | This counts situations where the capture job performs a database query to fetch extra column value(s) to 'augment' other column values read from DBMS logging, These operations are relatively slow (DB query needed). |
SAP Augment Selects | rows | log file | Only occurs when the SapXForm engine is used. Counts situations where capture job performs DB query to fetch extra rows to 'augment' SAP cluster rows read from DBMS logging, These operations are relatively slow (DB query needed). |
Captured DDL Statements | rows | log file | Measures the number of DDL statements processed by action AdaptDDL. These can be followed by online refreshes which can cause row skipping. In older HVR versions this metric was called Captured DDL. |
Integrated Change Counts Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Integrated Inserts | changes | log file | Number of integrated inserts. |
Integrated Updates | changes | log file | Number of integrated updates. Although updates can be captured as 2 rows not 1, this metric measures the updated changes, not underlying rows. |
Integrated Deletes | changes | log file | Number of integrated deletes. |
Integrated Changes | changes | log file | Number of all integrated changes. That is integrated inserts + updates + deletes. Sometimes changes (e.g. updates) are captured as 2 rows, not 1. Such an update would count as 1 here (not 2). Therefore the number of changes can be less than the number of rows. This count does NOT include DDL changes (or rows refreshed due to DDL changes). |
Integrated Skipped Changes | changes | log file | Changes are skipped by HVR 'controls', e.g. during an online refresh or if a job is restarted after some interruption. |
Changes Coalesced Away | changes | log file | Integrate/Burst and /Coalesce will both squeeze consecutive operations on the same row (e.g. insert + 3 updates) into a single change. |
Failed Inserts Saved | changes | log file | Failed inserts written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined. possibly because the row already existed. |
Failed Updates Saved | changes | log file | Failed updates written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because the row did not exist. |
Failed Deletes Saved | changes | log file | Failed delete written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because row did not exist. |
Failed Changes Saved | changes | log file | Total number of changes ( inserts + updates + deletes) that failed and were written to <tbl>__f table after errors when Integrate /OnErrorSaveFailed is defined, possibly because row did not exist. |
Collision Changes Discarded | changes | log file | Change which was discarded due to collision detection (action CollisionDetect) which decided the change was older than the current row in target DB. In older HVR versions this metric was called Discarded Changes. |
Empty Updates Discarded | changes | log file | Updates produced by the source database that were discarded because no replicated column was changed. |
Transactions Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Captured Transactions | transaction | log file | This metric counts the number of COMMITs of transactions that contain changes to replicated tables. Note that it's interesting to see if a system is dominated by large (e.g. 1000 row/commit) transactions, but comparing this field with 'Captured Rows' only shows an average. |
Captured Transactions Backdated | transaction | log file | A 'backdated' version of 'Captured Transactions'. Backdated means latency in the message is used to assign the value to a time earlier than the message's timestamp. For example, if a message 'Scanned 100 changes from 30 mins ago' has timestamp '16:55:00' then 100 is both added to 'Captured Transactions' for 16:55 and also to 'Captured Transactions Backdated' for 16:25. Visual comparison of a backdated metric to its regularly dated one can display bottleneck pattern; the area under both graph lines should be identical (the total amount of work done) but if the backdated graph shows a peak which quickly subsides and the regularly dated line shows a smaller rise which subsided more slowly, then a bottleneck is visible. In older HVR versions this metric was called Transactions Logged in DBMS. |
Integrated Transactions | transaction | log file | Integrate will often bundle lots of smaller 'Captured Transaction' values into fewer 'Integrated Transactions' for speed. This metric measures these bundled commits, not the original ones. |
Durations Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Duration Total | secs | log file | Total amount of time that capture cycles took. In older HVR versions this metric was called Total Capture Duration. |
Capture Duration Max | secs | log file | Duration of the longest capture cycle. In older HVR versions this metric was called Maximum Capture Duration. |
Capture Duration Average | secs | log file | Average duration of a capture cycle. In older HVR versions this metric was called Average Capture Duration. |
Integrate Duration Total | secs | log file | Total amount of time that integrate cycles took. In older HVR versions this metric was called Total Integrate Duration. |
Integrate Duration Max | secs | log file | Duration of the longest integrate cycle. In older HVR versions this metric was called Maximum Integrate Duration. |
Integrate Duration Average | secs | log file | Average duration of an integrate cycle. In older HVR versions this metric was called Average Integrate Duration. |
Speed Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Speed Max | rows/sec | log file | Maximum speed that capture cycle reached. In older HVR versions this metric was called Maximum Capture Change Speed. |
Capture Speed Average | rows/sec | log file | Average speed of capture cycles. In older HVR versions this metric was called Average Capture Change Speed. |
Integrate Speed Max | rows/sec | log file | Maximum speed that integrate cycle had through all phases. In older HVR versions this metric was called Maximum Integrate Change Speed. |
Integrate Speed Average | rows/sec | log file | Average speed that burst integrate cycle had through all phases. In older HVR versions this metric was called Average Integrate Change Speed. |
Cycles Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Cycles | cycles | log file | Count of capture cycles whose start occurred during this time period. This does not include 'sub-cycles' or 'silent cycles', but does include 'empty cycles'. A 'sub-cycle' happens when a busy capture job emits a block of changes but has not yet caught up to the 'top'. It can be recognized as an extra 'Scanning' message which is not preceded by a 'Cycle X' line'. A 'silent cycle' is when a capture job sees no change activity and decides to progress its 'capture state' files but without writing a line in the log (above every 10 secs). An 'empty cycle' is when a capture job sees no change activity and does write a line in the log (above every 10 mins).met |
Integrate Cycles | cycles | log file | Count of integrate cycles whose start occurred during this time period. Note that the integrate activity may fall into a subsequent period. |
Byte I/O Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Routed Bytes Written | bytes | log file | Amount of compressed bytes HVR routed from the capture machine to the hub machine. In older HVR versions this metric was called Routed Bytes. |
Routed Bytes Written Uncompressed | bytes | log file | Amount of uncompressed bytes HVR routed from the capture machine to the hub machine. These routed bytes refer to HVR's representation of routed rows in memory. This is different from the DBMS's 'storage size' (DBMS's storage of that row on disk) For example, if a table has a varchar(100) column containing 'Hello World', HVR manages to compress that down to 3 bytes. HVR's memory representation of varchar(100) is 103 bytes, whereas the DBMS storage is 13 bytes. In older HVR versions this metric was called Routed Bytes (uncompressed). |
Captured File Size | bytes | log file | In a file-to-database or file-to-file replication channel, this metric counts the total size of the captured files. In older HVR versions this metric was called Captured File Size. |
Capture DbmsLog Bytes | bytes | log file | Amount of bytes HVR captured from the database's log file. In older HVR versions this metric was called Capture Log Bytes Read. |
Capture DbmsLog Bytes Backdated | bytes | log file | A 'backdated' version of 'Capture DbmsLog Bytes'. Backdated means latency from HVR's log message is used to assign the value to a time earlier than the message's timestamp. So this reflects when the database wrote the bytes, not when HVR captured them. In older HVR versions this metric was called DBMS Log Bytes Written. |
Compression Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Compression Ratio Max | % | log file | When transporting table rows, HVR reports its 'memory compression' ratio; the number of compressed bytes transmitted over the network compared with HVR's representation of that row in memory (see 'Routed Bytes Written' and 'Routed Bytes Written Uncompressed'). This is different from the DBMS's 'storage compression ratio' (number of compressed bytes transmitted compared with DBMS's storage of that row on disk. For example, if a table has a varchar(100) column containing 'Hello World', HVR manages to compress that down to 3 bytes. the HVR's memory representation of varchar(100) is 103 bytes, whereas the DBMS storage is 13 bytes. In this case, the HVR's 'memory compression' ratio is 97% (1-(3/103)) whereas the storage compression ratio would be 77% (1-(13/103)). In older HVR versions this metric was called Maximum Compression Ratio. |
Compression Ratio Average | % | log file | In older HVR versions this metric was called Average Compression Ratio. |
Replicated Files Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Captured Files | files | log file | In a file-to-database or file-to-file replication channel, this metric counts the number of captured files. |
Integrated Files | files | log file | In a file integration, this metric counts the number of integrated files. |
Failed Files Saved | files | log file | Number of failed files saved. |
Errors/Warnings Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Errors | lines | log file | Counts the number of errors (lines matching F_J*) in the job log file. Such an error line could affect multiple rows or could affect one row and be repeated lots of times (so it counts as multiple 'errors'). In older HVR versions this metric was called Number of Errors. |
^Errors | string | log file | Annotation: Text of the most recent error line as additional information for metric Errors. In older HVR versions this metric was called Last Error. |
Errors F_J* | lines | log file | Groups the appearing errors by up to the 2 most significant error numbers. Example: Say error F_JA1234 happened. Then the metric 'Errors F_JA1234' will increase by 1. If the errors F_JA1234; F_JB1234: The previous error occurred ...; F_JC1234: The previous error occurred ...; appear then 'Errors F_JA1234 F_JB1234' will increase by 1. |
^Errors F_J* | string | log file | Annotation: Groups the appearing error messages by up to the 2 most significant error numbers. Example: Say error F_JA1234 happened. Then the metric '^Errors F_JA1234' will hold the message of F_JA1234. If the errors F_JA1234; F_JB1234: The previous error occurred ...; F_JC1234: The previous error occurred ...; appear then '^Errors F_JA1234 F_JB1234' will hold the messages of F_JA1234 and F_JB1234. |
Warnings | lines | log file | Counts the number of warnings (lines matching W_J*) in the job log file. In older HVR versions this metric was called Number of Warnings. |
^Warnings | string | log file | Annotation: Most recent warning line. In older HVR versions this metric was called Last Warning. |
Warnings W_J* | lines | log file | Groups the appearing warning by the warning number. Example: Say warning W_JA1234 happened. Then the metric 'Warnings W_JA1234' will increase by 1. |
^Warnings W_J* | string | log file | Annotation: Groups the appearing warning messages by the warning number. Example: Say warning W_JA1234 happened. Then the metric '^Warnings W_JA1234' will hold the message of W_JA1234. |
Refresh and Compare Runs Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Bulk Compare Job Runs | runs | log file | Number of times a scheduled compare job ran. |
Bulk Compare Table Runs | runs | log file | Number of times a particular table was compared. |
Bulk Refresh Job Runs | runs | log file | Number of times a scheduled refresh job ran. |
Bulk Refresh Table Runs | runs | log file | Number of times a particular table was refreshed. |
Row-wise Compare Job Runs | runs | log file | Number of times a scheduled row-wise compare job ran. |
Row-wise Compare Table Runs | runs | log file | Number of times a particular table was compared. |
Row-wise Refresh Job Runs | runs | log file | Number of times a scheduled refresh job ran. |
Row-wise Refresh Table Runs | runs | log file | Number of times a particular table was refreshed. |
Refresh and Compare Job Duration Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Bulk Compare Job Duration Min | secs | log file | Minimum duration a whole compare job took to run. In older HVR versions this metric was called Bulk Compare Job Dur Min. |
Bulk Compare Job Duration Max | secs | log file | Maximum duration a whole compare job took to run. In older HVR versions this metric was called Bulk Compare Job Dur Max. |
Bulk Compare Job Duration Average | secs | log file | Average duration a whole compare job took to run. In older HVR versions this metric was called Bulk Compare Job Dur Avg. |
Row-wise Compare Job Duration Min | secs | log file | Minimum duration a whole compare job took to run. In older HVR versions this metric was called Row-wise Compare Job Dur Min. |
Row-wise Compare Job Duration Max | secs | log file | Maximum duration a whole compare job took to run. In older HVR versions this metric was called Row-wise Compare Job Dur Max. |
Row-wise Compare Job Duration Average | secs | log file | Average duration a whole compare job took to run. In older HVR versions this metric was called Row-wise Compare Job Dur Avg. |
Bulk Refresh Job Duration Min | secs | log file | Minimum duration a whole refresh job took to run. In older HVR versions this metric was called Bulk Refresh Job Dur Min. |
Bulk Refresh Job Duration Max | secs | log file | Maximum duration a whole refresh job took to run. In older HVR versions this metric was called Bulk Refresh Job Dur Max. |
Bulk Refresh Job Duration Average | secs | log file | Average duration a whole refresh job took to run. In older HVR versions this metric was called Bulk Refresh Job Dur Avg. |
Row-wise Refresh Job Duration Min | secs | log file | Minimum duration a whole refresh job took to run. In older HVR versions this metric was called Row-wise Refresh Job Dur Min. |
Row-wise Refresh Job Duration Max | secs | log file | Maximum duration a whole refresh job took to run. In older HVR versions this metric was called Row-wise Refresh Job Dur Max. |
Row-wise Refresh Job Duration Average | secs | log file | Average duration a whole refresh job took to run. In older HVR versions this metric was called Row-wise Refresh Job Dur Avg. |
Refresh and Compare Table Duration Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Bulk Compare Table Duration Min | secs | log file | Minimum duration a compare job took to compare a specific table. In older HVR versions this metric was called Bulk Compare Table Dur Min. |
Bulk Compare Table Duration Max | secs | log file | Maximum duration a compare job took to compare a specific table. In older HVR versions this metric was called Bulk Compare Table Dur Max. |
Bulk Compare Table Duration Average | secs | log file | Average duration a compare job took to compare a specific table. In older HVR versions this metric was called Bulk Compare Table Dur Avg. |
Row-wise Compare Table Duration Min | secs | log file | Minimum duration a compare job took to compare a specific table. In older HVR versions this metric was called Row-wise Compare Table Dur Min. |
Row-wise Compare Table Duration Max | secs | log file | Maximum duration a compare job took to compare a specific table. In older HVR versions this metric was called Row-wise Compare Table Dur Max. |
Row-wise Compare Table Duration Average | secs | log file | Average duration a compare job took to compare a specific table. In older HVR versions this metric was called Row-wise Compare Table Dur Avg. |
Bulk Refresh Table Duration Min | secs | log file | Minimum duration a refresh job took to compare a specific table. In older HVR versions this metric was called Bulk Refresh Table Dur Min. |
Bulk Refresh Table Duration Max | secs | log file | Maximum duration a refresh job took to compare a specific table. In older HVR versions this metric was called Bulk Refresh Table Dur Max. |
Bulk Refresh Table Duration Average | secs | log file | Average duration a refresh job took to compare a specific table. In older HVR versions this metric was called Bulk Refresh Table Dur Avg. |
Row-wise Refresh Table Duration Min | secs | log file | Minimum duration a refresh job took to compare a specific table. In older HVR versions this metric was called Row-wise Refresh Table Dur Min. |
Row-wise Refresh Table Duration Max | secs | log file | Maximum duration a refresh job took to compare a specific table. In older HVR versions this metric was called Row-wise Refresh Table Dur Max. |
Row-wise Refresh Table Duration Average | secs | log file | Average duration a refresh job took to compare a specific table. In older HVR versions this metric was called Row-wise Refresh Table Dur Avg. |
Refresh and Compare Rows Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Bulk Compare Rows | rows | log file | Number of rows handled in bulk compare job. |
Bulk Compare Packed Rows | rows | log file | Number of packed rows handled in bulk compare job (only relevant if channel is doing unpacking with either SapUnpack or SapXForm). |
Row-wise Compare Rows | rows | log file | Number of rows handled in row-wise compare job. |
Row-wise Compare Packed Rows | rows | log file | Number of packed rows handled in row-wise compare job (only relevant if channel is doing unpacking with either SapUnpack or SapXForm). |
Bulk Refresh Rows | rows | log file | Number of rows handled in bulk refresh job. |
Bulk Refresh Packed Rows | rows | log file | Number of packed rows handled in bulk refresh job (only relevant if channel is doing unpacking with either SapUnpack or SapXForm). |
Row-wise Refresh Rows | rows | log file | Number of rows handled in row-wise refresh job. |
Row-wise Refresh Packed Rows | rows | log file | Number of packed rows handled in row-wise refresh job (only relevant if channel is doing unpacking with either SapUnpack or SapXForm) |
Compare Differences Inserts | rows | log file | Number of different rows to be inserted into a target location found during compare. |
Compare Differences Updates | rows | log file | Number of different rows to be updated on a target location found during compare. |
Compare Differences Deletes | rows | log file | Number of different rows to be deleted on a target location found during compare. |
Refresh Differences Inserts | rows | log file | Number of different rows inserted into a target location during a row-wise refresh. |
Refresh Differences Updates | rows | log file | Number of different rows updated on a target location during a row-wise refresh. |
Refresh Differences Deletes | rows | log file | Number of different rows deleted on a target location during a row-wise refresh. |
Router Rows Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Router Rows | rows | router file | Number of rows that are queued up for integration in router transaction files. This counts rows for all integrate locations that integrate changes captured from the capture location of this measurement. |
Integrate Router Rows | rows | router file | Number of rows that are queued up for integration in router transaction files. This counts rows for all capture locations that capture changes that are integrated into the location of this measurement. |
Router Bytes Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Router Bytes | bytes | router file | Amount of bytes that are queued up for integration in router transaction files. This counts bytes for all integrate locations that integrate changes captured from the capture location of this measurement. |
Integrate Router Bytes | bytes | router file | Amount of bytes that are queued up for integration in router transaction files. This counts bytes for all capture locations that capture changes that are integrated into the location of this measurement. |
Router Files Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Router Files | files | router file | Number of router transaction files that are queued up for integration. This counts files for all integrate locations that integrate changes captured from the capture location of this measurement. |
Integrate Router Files | files | router file | Number of router transaction files that are queued up for integration. This counts files for all capture locations that capture changes that are integrated into the location of this measurement. |
Router Timestamps Metricslink
Metric Name | Unit | Source | Remarks |
---|---|---|---|
Capture Rewind Time | time | router file | Timestamp that capture has to rewind back to after a restart to safely capture all transactions that have not been emitted yet. |
Capture Emit Time | time | router file | Timestamp of last emitted change. |
Capture Last Cycle Time | time | router file | Timestamp when the last capture cycle finished. |
Capture Router Timestamp | time | router file | Earliest creation timestamp of router transaction files. This checks files for all integrate locations that integrate changes captured from the capture location of this measurement. |
Integrate Router Timestamp | time | router file | Earliest creation timestamp of router transaction files. This checks files for all capture locations that capture changes that are integrated into the location of this measurement. |
Enroll File Timestamp | time | router file | Timestamp when the last enrollment was performed. |
Description of Unitslink
The unit of measurement is different for each metric, following are the units and its description:
Unit | Description |
---|---|
% | Percent. When used for compression this means the ratio of bytes left after compression. So if 100 bytes are compressed by 70% then 30 bytes remain. |
bytes | Amount of bytes. |
changes | Changes of tables affected by insert, update, or delete statements. Updates are sometimes moved as 2 rows (before-update and after-update). |
cycles | A capture or integrate cycle. |
files | Number of files. |
int | Integer number |
lines | The number of message lines written to the log file. A single line could be an error message which mentions multiple failed changes |
rows | Rows of tables affected by insert, update, or delete statements. |
rows/sec | Measures the average of 'rows' or 'changes' replicated per second. |
runs | The number of times a job has been run. |
secs | Seconds |
string | String data type. |
time | Timestamp |
transaction | Unit 'transaction' means a group of changes terminated by a commit (not just a changed row). |