Location Properties

Last updated on Dec 21, 2021

This section lists and describes the location properties.

A location property specifies the characteristics/attributes of a location in HVR. This can include location connection parameters, location/database type, database version, method of capture, etc.

A property that is automatically discovered by HVR when it connects to a database/location is called discovered property. A user cannot specify/input value into a discovered property.

PropertyArgumentDescriptionLocation Type

ABFS_Account

account

Name of the Azure Data Lake Storage Gen2 storage account.

This property is required for connecting HVR to an ADLS Gen2 location.

Azure Data Lake Storage (Gen 2)

ABFS_Authentication_Method

method

Authentication method for connecting HVR to Azure Data Lake Storage (ADLS) Gen2 server.

Available options for method are:

  • SHARED_KEY
  • CLIENT_CREDS
  • USER_PASS
  • MSI
  • REFRESH_TOKEN

For more information about these authentication methods, see section Authentication Methods in Azure Data Lake Storage Requirements.

Azure Data Lake Storage (Gen 2)

ABFS_Container

container

Name of the container available within the Azure Data Lake Storage Gen2 storage account (defined in ABFS_Account).

Azure Data Lake Storage (Gen 2)

ADL_Authentication_Method

method

Authentication method for connecting HVR to Azure Data Lake Storage Gen1 server.

Available options for method are:

  • CLIENT_CREDS
  • MSI
  • REFRESH_TOKEN

For more information about these authentication methods, see section Authentication Methods in Azure Data Lake Storage Requirements.

[Azure Data Lake Storage (Gen 1)]

Agent_Client_Kerberos_Keytab

keytabfileDirectory path to the Kerberos keytab file that contains a security key for identifying the hub to the agent during authentication (when connecting hub to the agent). If defined, this keytab file will be used instead of the operating system defaults.

Agent_Client_Kerberos_Principal

principalKerberos principal name for identifying the hub to the agent during authentication (when connecting hub to the agent). If defined, this principal name will be used instead of the operating system defaults.

Agent_Client_Kerberos_Ticket_Cache

ticketcachefileDirectory path to the Kerberos ticket cache file for identifying the hub to the agent during authentication (when connecting hub to the agent). If defined, this ticket cache file will be used instead of the operating system defaults.

Agent_Host

host

Hostname or IP-address of the server on which the HVR agent is installed/running.

All

Agent_HVR_CONFIG


This is a discovered property that stores the directory path of HVR_CONFIG for HVR agent.All

Agent_HVR_HOME


This is a discovered property that stores the directory path of HVR_HOME for HVR agent.All

Agent_Operating_System


This is a discovered property that stores the name of the operating system on which HVR agent is installed/running.All

Agent_Oracle_RAC_Port

port

Port number of the Oracle RAC database available on the remote server.

[Oracle]

Agent_Oracle_RAC_Service

service


Service name of the Oracle RAC database available on the remote server.

Example: HVR1900

[Oracle]

Agent_Password

password

Password for the HVR agent (defined in Agent_User).

All

Agent_Platform


This is a discovered property that stores the name of the HVR platform (e.g, linux_glibc2.12-x64-64bit, windows-x64-64bit) used for installing the HVR agent.All

Agent_Port

port

TCP/IP port number of the HVR agent. This is used for connecting HVR Hub to the HVR Agent.

For Oracle RAC connection, this is the TCP/IP port number of the HVR agent on the RAC nodes.

All

Agent_Server_Kerberos_Principal

principalUser specified Kerberos principal name for identifying the agent to the hub during authentication (when connecting hub to the agent).

Agent_Server_Public_Certificate

base64

The SSL public certificate file for the HVR agent.

This property is discovered on first connection to the agent and verified for all future connections.

All

Agent_User

username

Username for the HVR agent. This property is used for connecting HVR Hub to the HVR Agent.

All

Agent_Version


This is a discovered property that stores the HVR version of the agent installation.All

Archive_Log_Format

format

Describes the filename format (template) of the transaction log archive files stored in the directory specified by the Archive_Log_Path property.

The list of supported format variables and the default format string are database-specific.

For Oracle,

 Oracle

This property accepts the following format variables:

  • %d - match numbers (zero or more decimal digits). Numbers matched using this variable are ignored by HVR.
  • %r or %R - resetlogs ID
  • %s or %S - log sequence number
  • %t or %T - thread number
  • %z or %Z - match alphanumeric characters. Characters matched using this variable are ignored by HVR.

Wildcard character * is not supported.

For more information about the format variables, refer to the article LOG_ARCHIVE_FORMAT in Oracle documentation.

When this location property is not defined, then by default HVR will query the database for Oracle's initialization property LOG_ARCHIVE_FORMAT.

For SQL Server,

 SQL Server

This property accepts the following format variables:

  • %d - database name
  • %Y - year (up to 4 digit decimal integer)
  • %M - month (up to 2 digit decimal integer)
  • %D - day (up to 2 digit decimal integer)
  • %h - hours (up to 2 digit decimal integer)
  • %m - minutes (up to 2 digit decimal integer)
  • %s - seconds (up to 2 digit decimal integer)
  • %n - file sequence number (up to 64 bit decimal integer)
  • %% - matches %
  • * - wildcard, matches zero or more characters
  • HVR uses the %Y%M%D%h%m%s and %n values to sort and processes the log backup files in the correct (chronological) order.
  • The combinations of the %Y, %M, %D and %h, %m, %s values are expected to form valid date and time values, however no validation is performed.
  • Any value that is missing from the format string is considered to be 0.
  • When sorting the files comparison is done in the following order: %Y, %M, %D, %h, %m, %s, %n.

This property has no default format value and must be specified if Archive_Log_Path property is defined.

For SAP HANA,

 SAP HANA

This property accepts the following format variables:

  • %v - log volume ID
  • %p - log partition ID
  • %s - start sequence number
  • %e - end sequence number
  • %t - start timestamp (in milliseconds since UNIX epoch)
  • %% - matches %
  • - wildcard, matches zero or more characters

The %s%e and %t format variables are mandatory.

This property is optional. When this property is not defined, the default format value is log_backup_%v_%p_%s_%e.%t.

For Sybase ASE,

 Sybase ASE

This property accepts the following format variables:

  • * - wildcard, matches zero or more characters
  • ? - matches any single character

This property is optional. When this property is not defined, by default HVR will scan all files available in the Archive_Log_Path.

[Oracle] [SAP HANA] [SQL Server]

Archive_Log_Path

dir

HVR will search for the transaction log archives in the specified directory (path) dir.

The behavior of this property is database-specific.

For Oracle,

 Oracle

HVR will search for the log archives in the specified directory dir in addition to the primary Oracle archive directory.

If the Capture_Method is set to ARCHIVE_ONLY then HVR will search for the log archives in the directory dir only. Any process could be copying log archive files to this directory; the Oracle archiver (if another LOG_ARCHIVE_DEST_N is defined), RMAN, hvrlogrelease or a simple shell script.

It should be ensured that the files in this directory are purged periodically, otherwise the directory will fill up.

For SQL Server,

 SQL Server

HVR normally locates the transaction log backup files by querying the backup history table in the msdb database. Specifying this property tells HVR to search for the log backup files in the specified directory dir instead.

If this property is defined, then Archive_Log_Format must also be defined.

For SAP HANA,

 SAP HANA

HVR will search for the log backups in the specified directory dir instead of the default log backup location for the source database.

For Sybase ASE,

 Sybase ASE

HVR will search for the transaction log backups in the specified directory dir.

[Oracle] [SAP HANA] [SQL Server]

AWS_Access_Key_Id

keyid

Access key ID of IAM user for connecting HVR to Amazon S3.

This property is used together with AWS_Secret_Access_Key when connecting HVR to Amazon S3 using IAM User Access Keys.

For more information about Access Keys, refer to Understanding and Getting Your Security Credentials in AWS documentation.

#Amazon credentials

AWS_IAM_Role

role

AWS IAM role name for connecting HVR to Amazon S3.

This property is used when connecting HVR to Amazon S3 using AWS Identity and Access Management (IAM) Role. This property may be used only if the HVR remote agent or the HVR Hub is running inside the AWS network on an EC2 instance and the AWS IAM role specified here should be attached to this EC2 instance. When a role is used, HVR obtains temporary Access Keys Pair from the EC2 server. For more information about IAM Role, refer to IAM Roles in AWS documentation.

#Amazon credentials

AWS_Secret_Access_Key

key

Secret access key of IAM user for connecting HVR to Amazon S3.

This property is used together with AWS_Access_Key_Id when connecting HVR to Amazon S3 using IAM User Access Keys.

#Amazon credentials

Azure_OAuth2_Client_Id

id

Client ID (or application ID) used to obtain Azure AD access token.

This property is required only if the authentication method (ABFS_Authentication_Method or ADL_Authentication_Method) is set to CLIENT_CREDS or REFRESH_TOKEN.

#Azure credentials

Azure_OAuth2_Client_Secret

key

Secret key of the Azure_OAuth2_Client_Id.

This property is required only if the authentication method (ABFS_Authentication_Method or ADL_Authentication_Method) is set to CLIENT_CREDS.

#Azure credentials

Azure_OAuth2_Endpoint

url

URL used for obtaining bearer token with credential token.

This property is required only if the authentication method (ABFS_Authentication_Method or ADL_Authentication_Method) is set to CLIENT_CREDS.

#Azure credentials

Azure_OAuth2_MSI_Port

port

Port number for the REST endpoint of the token service exposed to localhost by the identity extension in the Azure VM. The default value for this property is 50342.

This property is required only if the authentication method (ADL_Authentication_Method) is set to MSI.

#Azure credentials

Azure_OAuth2_MSI_Tenant

url

URL for the REST endpoint of the token service exposed to localhost by the identity extension in the Azure VM.

For Azure Data Lake Storage, this property is required only if the authentication method (ABFS_Authentication_Method) is set to MSI.

#Azure credentials

Azure_OAuth2_Password

password

Password for Azure_OAuth2_User.

#Azure credentials

Azure_OAuth2_Refresh_Token

path

Directory path to the text file containing the refresh token.

This property is required only if the authentication method (ABFS_Authentication_Method or ADL_Authentication_Method) is set to REFRESH_TOKEN.

#Azure credentials

Azure_OAuth2_User

user

Username for the OAuth 2.0 authentication.

This property is required only if the authentication method (ABFS_Authentication_Method) is set to USER_PASS.

#Azure credentials

Azure_Shared_Secret_Key

account

Access key of the Azure storage account.

For Azure Data Lake Storage, this property is required only if the authentication method (ABFS_Authentication_Method) is set to SHARED_KEY.

#Azure credentials

BigQuery_Region


Geographic location of the dataset. For more information about dataset locations, refer to Dataset Locations in BigQuery Documentation.

Examples: US, europe-west4, us-west4


Capture_Checkpoint_Frequency

secs

Checkpointing frequency in seconds for long running transactions, so the capture job can recover quickly when it restarts. Value secs is the interval (in seconds) at which the capture job creates checkpoints.

Thedefaultcheckpoint frequency is 300 seconds (5 minutes). Value 0 means no checkpoints are written.

Without checkpoints, capture jobs must rewind back to the start of the oldest open transaction, which can take a long time and may require access too many old DBMS log files (e.g. archive files).

The checkpoints are written into HVR_CONFIG/capckp/hub/chn directory. If a transaction continues to make changes for a long period then successive checkpoints will not rewrite its same changes each time; instead the checkpoint will only write new changes for that transaction; for older changes it will reuse files written by earlier checkpoints.

Checkpoints are written only for long-running transactions. For example, if the checkpoint frequency is each 5 minutes but users always do an SQL commit within 4 minutes then checkpoints will never be written. However, if users keep transactions open for 10 minutes, then those transactions will be saved but shorter-lived ones in the same period will not.

The frequency with which capture checkpoints are written is relative to the capture jobs own clock, but it decides whether a transaction has been running long enough to be checkpointed by comparing the timestamps in its DBMS logging records. As a consequence, the maximum (worst-case) time that an interrupted capture job would need to recover (rewind back over all its open transactions) is its checkpoint frequency plus the amount of time it takes to reread the amount of changes that the DBMS can write in that period of time.

When a capture job is recovering it will only use checkpoints which were written before the 'capture cycle' was completed. This means that very frequent capture checkpointing (say every 10 seconds) is wasteful and will not speed up capture job recovery time.

This property is supported only for certain location types. For the list of supported location types, see Log-based capture checkpointing in Capabilities.

[Db2 Database] [Ingres] [Oracle] [SQL Server] [Azure SQL Database]

Capture_Checkpoint_Retention

secs

Retains capture checkpoint files up to the specified period secs (in seconds).

The retained checkpoint files are saved in HVR_CONFIG/capckpretain/hub/channel/location directory.

Depending on the storage location defined in Capture_Checkpoint_Storage, this directory can be located either on the capture location or hub.

[Db2 Database] [Ingres] [Oracle] [SQL Server] [Azure SQL Database]

Capture_Checkpoint_Storage

stor

Storage location of capture checkpoint files for quick capture recovery.

Available options for storare:

  • LOCATIONdefault: Checkpoint files are saved in a directory on capture location.
  • HUB: Checkpoint files are saved in a directory on hub server.
    Writing checkpoints on the hub is more expensive because extra data must be sent across the network. When capturing changes from an Oracle RAC, the checkpoint files should be stored on the hub server because the directory on the remote location where the capture job would otherwise write checkpoints may not be shared inside the RAC cluster, so it may not be available when the capture job restarts.

For both the storage locations, the checkpoint files are saved in HVR_CONFIG/capckp/ directory.

When the capture job is restarted and if it cannot find the most recent checkpoint files (perhaps the contents of that directory have been lost during a failover) then it will write a warning and rewind back to the start of the oldest open transaction.

[Db2 Database] [Ingres] [Oracle] [SQL Server] [Azure SQL Database]

Capture_Method

method

Method of reading/capturing changes from the DBMS log file.

This property is supported only for location types from which HVR can capture changes. For the list of supported location types, see Capture changes from location in Capabilities.

Valid values for method are:

  • DIRECTdefault: Reads transaction log records directly from the DBMS log file using file I/O. This method is generally faster and more efficient than the SQL mode. The DIRECT log read method requires that HVR agent is installed on the source database server.

    For SQL Server, this capture method requires Windows Administrator privileges, and reduced permission models are not supported.

    This capture method is supported only for certain location types. For the list of supported location types, see Direct access to logs on a file system in Capabilities.
  • SQL: Reads transaction log records using a special SQL function. The advantage of this method is that it reads change data over an SQL connection and does not require an HVR agent to be installed on the source database server. The disadvantages of the SQL method is that it is slower than the DIRECT method and exposes additional load on the source database. 

    For SQL Server, this capture method supports reduced permission models but it may require incomplete row augmenting.

    This capture method is supported only for certain location types. For the list of supported location types, see Access to logs using SQL interface in Capabilities.
  • LOGMINER: Reads data using Oracle's LogMiner. The advantage of this method is that it reads change data over an SQL connection and does not require an HVR agent to be installed on the source database server. The disadvantages of the LOGMINER method is that it is slower than the DIRECT method and exposes additional load on the source database.
    This capture method is supported only for Oracle.
  • ARCHIVE_ONLY: Reads data from the archived redo files in directory defined using Archive_Log_Path property and do not read anything from online redo files or the 'primary' archive destination. This allows the HVR process to reside on a different server than the Oracle DBMS or SQL Server and read changes from files that are sent to it by some remote file copy mechanism (e.g. FTP). The capture job still needs an SQL connection to the database for accessing dictionary tables, but this can be a regular connection.

    Replication in this capture method can have longer delays in comparison with the 'online' mode.

    For Oracle, to control the delays it is possible to force Oracle to issue an archive once per predefined period of time.

    For Oracle RAC systems, delays are defined by the slowest or the least busy node. This is because archives from all threads have to be merged by SCNs in order to generate replicated data flow.

    This capture method is supported only for certain location types. For the list of supported location types, see Capture from Archive log files only in Capabilities.

  • DB_TRIGGER: Capture changes through DBMS triggers generated by HVR, instead of using log-based capture.

    In HVR UI, when creating a location, the option Database Triggers is displayed only if HVR detects database triggers in the connected database. This option can also be manually enabled by executing the following command:

    hvruserconfig user_name Show_Deprecated_Features=true

    This capture method is supported only for certain location types. For the list of supported location types, see Trigger-based capture in Capabilities.

#L4 Properties

<<Loc Types??>>

Capture_Method_Unavailable


This is a discovered property that stores information whether the location supports a specific capture method.


Case_Sensitive_Names

true

Normally, HVR converts the DBMS table names to lowercase and treats table and column names as case insensitive.

If set to true, DBMS table names and column names are treated case sensitive by HVR.

Defining this property allows the replication of tables with mixed case names or tables whose names do not match the DBMS case convention. For example, normally an Oracle table name is held in uppercase internally (e.g. MYTAB), so this property is needed to replicate a table named mytab or MyTab.

This property is supported only for certain location types. For the list of supported location types, see Treat DBMS table names and columns case sensitive in Capabilities.

Columns with duplicate names with different cases are not supported within the same table: e.g., column1 and COLUMN1.

[Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [Ingres] [Actian Vector] [MySQL] [MariaDB] [Aurora MySQL] [Oracle] [PostgreSQL] [Aurora PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Azure SQL Data Warehouse] [Azure SQL Database] [Snowflake] [Teradata]

Class

class

This property specifies class of the database. For example, oracle, sqlserver, etc.

All

Class_Flavor


This is a discovered property that stores the flavor of the specific database Class. The combination of Class and Class_Flavor forms the location type.

Example: For Azure Database, the Class is sqlserver and Class_Flavor is azure.

All

Class_Version


This is a discovered property that stores the version of the database Class.

All

Database_Char_Encoding


This is a discovered property that stores the character encoding of the database (defined in Database_Name).

#Database generic

Database_Client_Private_Key

path

Directory path where the .pem file containing the client's SSL private key is located.

This property is required for enabling two way SSL.

Defining this property along with Database_Public_Certificate, Database_Client_Public_Certificate, and Database_Client_Private_Key_Password will enable two way SSL, which means, HVR will authenticate the Hive server by validating the SSL certificate shared by the Hive server.

[ACID Hive]

Database_Client_Private_Key_Password

password

Password of the client's SSL private key specified in Database_Client_Private_Key.

This property is required for enabling two way SSL.

Defining this property along with Database_Public_Certificate, Database_Client_Public_Certificate, and Database_Client_Private_Key will enable two way SSL, which means, HVR will authenticate the Hive server by validating the SSL certificate shared by the Hive server.

[ACID Hive]

Database_Client_Public_Certificate

path

Directory path where the .pem file containing the client's SSL public certificate is located.

This property is required for enabling two way SSL.

Defining this property along with Database_Public_Certificate, Database_Client_Private_Key, and Database_Client_Private_Key_Password will enable two way SSL, which means, HVR will authenticate the Hive server by validating the SSL certificate shared by the Hive server.

[ACID Hive]

Database_Default_Schema


This is a discovered property that stores the name of the default schema in the database (Database_Name).#Database generic

Database_Host

host

Hostname or IP-address of the server on which the database is running.

For Db2 for i, this is the hostname or IP-address of the Db2 for i system.

For Hive ACID, this is the hostname or IP address of the Hive server.
When Hive_Service_Discovery_Mode is ZooKeeper, specify the list of ZooKeeper servers in following format [ZK_Host1]:[ZK_Port1],[ZK_Host2]:[ZK_Port2], where [ZK_Host] is the IP address or hostname of the ZooKeeper server and [ZK_Port] is the TCP port that the ZooKeeper server uses to listen for client connections.

[ACID Hive] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 for i] [Greenplum] [MariaDB] [MySQL] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

Database_Name

dbname

Name of the database.

For Db2 for i, this is the named database in Db2 for i. It could be on another (independent) auxiliary storage pool (IASP). The user profile's default setting will be used when no value is specified. Specifying *SYSBAS will connect a user to the SYSBAS database.


For Db2 for z/OS, the following formats are supported for this property:

  • alias for the database in Db2 for z/OS
  • database:server:port
    • database is the name of the database in Db2 for z/OS
    • server can either be a hostname or an IP-address of the database server
    • port is the TCP/IP port used for connecting to the database server

For BigQuery, this is the name of the dataset in Google BigQuery.

For HANA, this is the name of the specific database in a multiple-container environment.

For Hive ACID, this is the name of the database schema to use when a schema is not explicitly specified in a query.

[Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [SAP HANA] [ACID Hive] [Ingres] [Actian Vector] [MySQL] [MariaDB] [Aurora MySQL] [PostgreSQL] [Aurora PostgreSQL] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database]

Database_Nchar_Encoding


This is a discovered property that stores the national character encoding of the database (defined in Database_Name).

#Database generic

Database_Password

password

Password for the Database_User.

[ACID Hive] [Actian Vector] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [MariaDB] [MySQL] [Oracle] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

Database_Port

port

Port number on which the database (defined in Database_Host) server is expecting connections.

For Hive ACID, this property is required only if Hive_Service_Discovery_Mode is set to NONE.

[ACID Hive] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Greenplum] [MariaDB] [MySQL] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake]

Database_Public_Certificate

path

Directory path where the .pem file containing the server's public SSL certificate signed by a trusted CA is located.

Defining this property will enable (one way) SSL, which means, HVR will authenticate the Hive server by validating the SSL certificate shared by the Hive server.

This property is also required for enabling two way SSL.

For enabling two way SSL, this property must be defined along with Database_Client_Public_Certificate, Database_Client_Private_Key, and Database_Client_Private_Key_Password.


Database_Schema

schema

Name of the default schema to be used for this connection.


[Snowflake]

[ACID Hive]

Database_User

user

Username for connecting HVR to the database (defined in Database_Name).

For Azure SQL Database, this is the user name and host name of the Azure SQL Database server. The format to be used is username@hostname.

For Hive ACID, this is the username for connecting HVR to Hive server.

This property can be used only if Hive_Authentication_Method is set to USER or USER_PASS.

For Sybase ASE, this property can be used only if the Sybase_Authentication_Method is set to USER_PASS.

For Teradata, this is the username for connecting HVR to the Teradata Node.


[ACID Hive] [Actian Vector] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [Ingres] [MariaDB] [MySQL] [Oracle] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

DB2i_Log_Journal


Capture from specified DB2 for i journal. Both the schema (library) of the journal and the journal name should be specified (separated by a dot). This parameter is mandatory for DB2 for i. All tables in a channel should use the same journal. Use different channels for tables associated with different journals. If this parameter is defined for any table, then it affects all tables captured from that location.


DB2i_Log_Journal_Schema


Capture from journal with specified journal (SCHEMA)

DB2i_Log_Journal_SysSeq


Capture from journal using *SYSSEQ. This parameter requires /LogJournal.

Db2_DB2INSTANCE

instance
Name of the Db2 instance.

[Db2 Database] [Db2 for z/OS]

Db2_INSTHOME

path
Directory path of the Db2 installation.

[Db2 Database] [Db2 for z/OS]

Description

descriptionDescription for location created in HVR.All

File_Host

host

Hostname or IP-address of the server on which the file server is running.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Password

password

Password for the File_User.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Path

path

Directory path where the files are replicated to or captured from.

For Amazon S3, this is the directory path in the S3 BUCKET where the files are replicated to or captured from.

For Azure Blob Storage, this is the directory path in the container (defined in WASB_Container) where the files are replicated to or captured from.

For Azure Data Lake Storage Gen1, this is the directory path where the files are replicated to or captured from.

For Azure Data Lake Storage Gen2, this is the directory path in container (defined in ABFS_Container) where the files are replicated to or captured from.

For Google Cloud Storage, this is the directory path in the Google Cloud Storage BUCKET where the files are replicated to or captured from.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Port

port

Port number on which the file server (defined in File_Host) is expecting connections.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Proxy_Host

host

Host name of the proxy server used for connecting HVR to the file server (defined in File_Host).

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Proxy_Password

password

Password for the File_Proxy_User.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Proxy_Port

port

Port number for the proxy server (defined in File_Proxy_Host) used for connecting HVR to the file server (File_Host).

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Proxy_Scheme

protocol

Protocol for the proxy server (defined in File_Proxy_Host) used for connecting HVR to the file server (defined in File_Host).

Available options for protocol are:

  • HTTP
  • SOCKS4
  • SOCKS4A
  • SOCKS5
  • SOCKS5H

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Proxy_User

username

Username for the proxy server (defined in File_Proxy_Host) used for connecting HVR to the file server (defined in File_Host).


#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_Scheme

protocol

Protocol for connecting HVR to the file server (defined in File_Host).

The options available/supported for protocol are location type specific.

Available options for Amazon S3:

 Amazon S3
  • S3S
  • S3
 Azure Blob Storage

Available options for Azure Blob Storage:

  • WASBS
  • WASB

Available options for Azure Data Lake Storage:

 Azure Data Lake Storage
  • ABFSS (applicable only for Azure DLS Gen2)
  • ABFS (applicable only for Azure DLS Gen2)
  • ADL (applicable only for Azure DLS Gen1)

Available options for File/FTP/SFTP are:

 File/FTP/SFTP
  • FTP

  • FTPS

  • SFTP

Available options for Google Cloud Storage:

 Google Cloud Storage
  • GSS
  • GS

Available options for Sharepoint / WebDAV are:

 Sharepoint / WebDAV
  • WEBDAVS
  • WEBDAV
[ACID Hive] [Greenplum] [MySQL] [MariaDB] [Aurora MySQL] [Redshift] [SAP HANA] [Snowflake]

File_State_Directory

path

Directory path for internal state files used by HVR during file replication.

By default these files are created in sub-directory _hvr_state which is created inside the file location top directory.

If path is relative (e.g. ../work), then the path used is relative to the file location's top directory. The state directory can either be defined to be a path inside the location's top directory or put outside this top directory. If the state directory is on the same file system as the file location's top directory, then HVR integrates file move operations will be 'atomic', so users will not be able to see the file partially written. Defining this property on a SharePoint/WebDAV integrate location ensures that the SharePoint version history is preserved.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_State_Directory_Is_Local

true

If set to true, the directory specified in File_State_Directory is stored on the local drive of the file location's server.

If this property is not set to true or enabled, then by default the internal state files are stored in file location. For example, in Amazon S3, by default the state directory is stored in the S3 bucket.

#File locations, and other file-related storage settings (e.g. Staging_Directory)

File_User

username

Username for connecting HVR to the file server (defined in File_Host).

#File locations, and other file-related storage settings (e.g. Staging_Directory)

FTP_Encryption_Type

type

Encryption type used for connecting HVR to the file server (defined in File_Host). This is applicable only if File_Scheme is set to FTP.

Available options for type are:

  • SSL
  • TLS
#unencrypted FTP

GCloud_Authentication_Method

method

Authentication method for connecting HVR to the Google Cloud server (Cloud Storage and Google BigQuery).

The options available/supported for method are location type specific.

Available options for Google Cloud Storage:

 Google Cloud Storage

Available options for Google BigQuery:

 Google BigQuery
  • OAUTH_ENV: Authentication (OAuth) using the credentials (service account key) fetched from the environment variable GOOGLE_APPLICATION_CREDENTIALS. For more information about setting this environment variable, refer to the Google Cloud documentation.
  • OAUTH_FILE: Authentication (OAuth) using the credentials supplied in the service account key file (GCloud_OAuth_File). For more information about creating the service account key file, refer to the Google Cloud documentation.
#Google Cloud Credentials

GCloud_Email

email

Service account email for connecting HVR to the Google BigQuery server.


GCloud_OAuth_Env

true

If set to true, enables OAuth 2.0 protocol based authentication for connecting HVR to the Google Cloud Storage.

This method connects using the credentials fetched from the environment variable GOOGLE_APPLICATION_CREDENTIALS. For more information about configuring this environment variable, see Getting Started with Authentication in Google Cloud Storage documentation.


GCloud_OAuth_File

path

Directory path for the service account key file (JSON) used in OAuth 2.0 protocol based authentication.

This property is required only if GCloud_Authentication_Method is set to OAUTH_FILE.

#Google Cloud Credentials

GCloud_Project

id

ID of the google cloud project. For more information about google cloud projects, refer to Creating and Managing Projects in BigQuery Documentation.

#Google BigQuery

GS_Bucket

bucket

Name or IP address of the Google Cloud Storage bucket.

#Google Cloud Storage

GS_Bucket_Region

region

Geographic location of the dataset. For more information about dataset locations, refer to Dataset Locations in BigQuery Documentation.


GS_HMAC_Access_Key_Id

id

The HMAC access ID of the service account.

This property is required only if GCloud_Authentication_Method is set to HMAC when connecting HVR to the Google Cloud Storage.

#Google Cloud Storage

GS_HMAC_Secret_Access_Key

key

The HMAC secret of the service account.

This property is required only if GCloud_Authentication_Method is set to HMAC when connecting HVR to the Google Cloud Storage.

#Google Cloud Storage

GS_Storage_Integration

name

Integration name of the google cloud storage.


HDFS_Kerberos_Credential_Cache

path

Directory path for the Kerberos ticket cache file.

It is not required to define this property if keytab file is used for authentication or if Kerberos is not used on the Hadoop cluster.

For more information about using Kerberos authentication, see HDFS Authentication and Kerberos.

#[HDFS]

HDFS_Namenode

host

Hostname of the HDFS NameNode.

#[HDFS]

Hive_Authentication_Method

method

Authentication method for connecting HVR to Hive Server 2.

This property is required only if Hive_Server_Type is set to 2.

Available options for method are:

  • NONE
  • USER
  • USER_PASS
  • KERBEROS
  • HDINSIGHT
[Acid Hive]

Hive_HTTP_Path

url

The partial URL corresponding to the Hive server.

This property is required only if Hive_Thrift_Transport is set to HTTP.

Acid Hive

Hive_Kerberos_Host

host

Fully Qualified Domain Name (FQDN) of the Hive server host. This is the host part of Kerberos principal of the Hive server. For example, if the principal is "hive/[email protected]" then "example.host" should be specified here. 

The value for this property may be set to _HOST to use the Hive server hostname as the domain name for Kerberos authentication.

If Hive_Service_Discovery_Mode is set to NONE, then the driver uses the value specified in the Host connection attribute.
If Hive_Service_Discovery_Mode is set to ZOOKEEPER, then the driver uses the Hive Server 2 host name returned by the ZooKeeper.

This property is required only if Hive_Authentication_Method is set to Kerberos.

[Acid Hive]

Hive_Kerberos_Realm

realm

Realm of the Hive Server 2 host.

It is not required to specify any value in this property if the realm of the Hive Server 2 host is defined as the default realm in Kerberos configuration.

This property is required only if Hive_Authentication_Method is set to Kerberos.

[Acid Hive]

Hive_Kerberos_Service

name

Kerberos service principal name of the Hive server. This is the service name part of Kerberos principal of the Hive server. For example, if the principal is hive/[email protected] then "hive" should be specified here. 

This property is required only if Hive_Authentication_Method is set to Kerberos.

[Acid Hive]

Hive_Server_Type

type

Type of the Hive server.

Available options for type are:

  • 1: HVR will connect to Hive Server 1 instance.
  • 2: HVR will connect to Hive Server 2 instance.
[Acid Hive]

Hive_Service_Discovery_Mode

mode

Mode for connecting HVR to Hive Server 2.

This property is required only if Hive_Server_Type is set to 2.

Available options for mode are:

  • NONE: HVR connects to Hive Server 2 without using the ZooKeeper service.
  • ZOOKEEPER: HVR discovers Hive Server 2 services using the ZooKeeper service.
[Acid Hive]

Hive_Thrift_Transport

protocol

Transport protocol to use in the Thrift layer.

This property is required only if Hive_Server_Type is set to 2.


Available options for protocol are:


For information about determining which Thrift transport protocols your Hive server supports, refer to HiveServer2 Overview and Setting Up HiveServer2 sections in Hive documentation.

Acid Hive

Hive_Zookeeper_Namespace

namespace

Namespace on ZooKeeper under which Hive Server 2 nodes are added.

This property is required only if Hive_Service_Discovery_Mode is set to ZooKeeper.

[Acid Hive]

Ingres_II_SYSTEM

path

Directory path where the Actian Vector or Ingres database is installed.

[Ingres] [Actian Vector]

Intermediate_Directory

path

Directory path for storing 'intermediate files' that are generated during compare. Intermediate files are generated while performing direct file or online compare.

If this property is not defined, then by default the intermediate files are stored in integratedir/_hvr_intermediate directory.

The integratedir is the replication Directory the New Location screen while creating a file location.

#Temporary / Staging directories

Intermediate_Directory_Is_Local

true

If set to true, the directory specified in Intermediate_Directory is stored on the local drive of the file location's server.

If not set to true, then by default the intermediate files are stored in file location. For example, in Amazon S3, by default the intermediate directory is stored in the S3 bucket.

#Temporary / Staging directories

Kafka_Authentication_Method

method

Authentication method for connecting HVR to Kafka server (Broker). 

Available options for method are:

  • NONE
  • USER_PASS
  • KERBEROS

On Linux, to use either of the Kafka authentication methods (USER_PASS or KERBEROS), HVR requires the library libsasl2.so.2 to be installed. For more information, see section Installation Dependencies in Apache Kafka Requirements.

[Kafka]

Kafka_Brokers

list:host, ports

Hostname or IP address of the Kafka broker server(s) along with the TCP port that the Kafka server uses to listen for client connections. The default port is 9092.

[Kafka]

Kafka_Default_Topic

topic

Kafka topic to which the messages are written.

[Kafka]

Kafka_Kerberos_Client_Principal

host

Full Kerberos principal of the client connecting to the Kafka server. This property definition is required only on Linux/Unix.

This property is required only if Kafka_Authentication_Method is set to KERBEROS.

[Kafka]

Kafka_Kerberos_Keytab

path

Directory path where the Kerberos keytab file containing key for the Kafka_Kerberos_Client_Principal is located.

This property is required only if Kafka_Authentication_Method is set to KERBEROS.

[Kafka]

Kafka_Kerberos_Service

name

Kerberos Service Principal Name (SPN) of the Kafka server.

This property is required only if Kafka_Authentication_Method is set to KERBEROS.

[Kafka]

Kafka_Message_Bundling

mode

Number of messages written (bundled) into single Kafka message. Regardless of the file format chosen, each Kafka message contains one row by default.

Available options for mode are:

  • ROW: Each Kafka message contains a single row; this mode does not support bundling of multiple rows into a single message. Note that this mode causes a key-update to be sent as multiple Kafka messages (first a 'before update' with hvr_op 3, and then an 'after update' with hvr_op 2).
  • CHANGE: Each Kafka message is a bundle containing two rows (a 'before update' and an 'after update' row) whereas messages for other changes (e.g. insert and delete) contain just one row. During refresh there is no concept of changes, so each row is put into a single message. Therefore in that situation, this behavior is the same as mode ROW.
  • TRANSACTION: During replication, each message contains all rows in the original captured transaction. An exception is if the message size looks like it will exceed the bundling threshold (see property Kafka_Message_Bundling_Threshold). During refresh, all changes are treated as if they are from a single capture transaction so this mode behaves the same as bundling mode THRESHOLD.
  • THRESHOLD: Each Kafka message is bundled with rows until it exceeds the message bundling threshold (see property Kafka_Message_Bundling_Threshold).

Note that Confluent's Kafka Connect only allows certain message formats and does not allow any message bundling, therefore Kafka_Message_Bundling must either be undefined or set to ROW. Bundled messages simply consist of the contents of several single-row messages concatenated together.

When the mode is set to TRANSACTION or THRESHOLD and if the name of the Kafka topic contains an expression such as {hvr_tbl_name} then the rows of different tables will not be bundled into the same message.

[Kafka]

Kafka_Message_Bundling_Threshold

threshold

Threshold (in bytes) for bundling rows in a Kafka message. Rows continue to be bundled into the same message until this threshold is exceeded, after which, the message is sent and new rows are bundled into the next message.

The default value is 800,000 bytes.

This property may be defined only if Kafka_Message_Bundling is set to TRANSACTION or THRESHOLD.

By default, the maximum size of a Kafka message is 1,000,000 bytes; HVR will not send a message exceeding this size and will instead give a fatal error.

[Kafka]

Kafka_Message_Compress

algorithm

HVR will configure the Kafka transport protocol to compress message sets transmitted to Kafka broker using one of the available algorithms. The compression allows to decrease the network latency and save disk space on the Kafka broker. Each message set can contain more than one Kafka message. For more information, see section Kafka Message Bundling and Size in Apache Kafka Requirements.

Available options for the algorithm are:

LZ4 compression is not supported on the Windows platform if Kafka broker version is less than 1.0.

[Kafka]

Kafka_Schema_Registry

url

URL (http:// or https://) of the schema registry to use Confluent compatible messages in Avro format.

If the basic authentication is configured for the schema registry, then the login credentials (username and password) must be specified in the URL. The format is http[s]://user:password@schemaregistry_url:port

Example:`https://myuser:mypassword@abcd-efgh.eu-central-1.aws.confluent.cloud

For more information, see section Kafka Message Format in Apache Kafka Requirements.

[Kafka]

Kafka_Schema_Registry_Format


Format of the Kafka message. For more information, see section Kafka Message Format in Apache Kafka Requirements.

Available options are:

  • AVRO
  • JSON

Key_Only_Trigger_Tables

true

If set to true, then write only the key columns into the capture table to improve the performance of trigger-based capture (when Capture_Method is DB_TRIGGER).

The non-key columns are extracted using an outer join from the capture table to the replicated table. Internally HVR uses the same outer join technique to capture changes to long columns (e.g. long varchar). This is necessary because DBMS rules/triggers do not support long data types.

The disadvantage of this technique is that 'transient' column values can sometimes be replicated, for example, if a delete happens just after the toggle has changed, then the outer join could produce a NULL for a column which never had that value.

This property requires Capture_Method set to DB_TRIGGER.

[Db2 Database] [Ingres] [Oracle] [SQL Server] [Azure SQL Database]

Log_Truncater

method

Specify who advances the SQL Server/Sybase ASE transaction log truncation point (truncates the log).

Valid values for method are database specific:

 SQL Server
  • CAP_JOBdefault: This method is used to indicate that the capture job regularly calls sp_repldone to unconditionally release the hold of the truncation point for replication. When this option is selected and Activate Replication (hvractivate) is run, depending on the value of the property Supplemental_Logging, HVR will also drop/disable the SQL Server agent jobs that are created when CDC tables are created through the CDC stored procedures and the agent jobs related to data distribution. As a result, the additional impact of auxiliary database objects to enable supplemental logging is minimized. For example, the CDC tables are not populated (and the space allocation increased) because the agent job to do this is dropped. Multiple Capture jobs can be set up on a single database with option CAP_JOB selected. However, note that if no Capture job is running with the CDC tables and/or Articles in place, the transaction log will grow because the truncation point for replication is not released. Do not set this option if there is another data replication solution or the database uses CDC tables.
  • CAP_JOB_RETAIN: This method should be used when capturing from a SQL Server database with the recovery mode set to Simple Recovery. The capture job moves the truncation point of the transaction log forward by calling the stored procedure sp_repldone at the end of each sub-cycle. Only part of the transaction log that has already been processed (captured) is marked for truncation (this is different from the CAP_JOB mode, where all records in the transaction log are marked for truncation, including those that have not been captured yet). This value is not compatible with multi-capture and does not allow for coexistence with a third party replication solution. This setting will also result in SQL Server's agent jobs being dropped/disabled, so the transaction log will grow when the capture job is not running and CDC tables and/or Articles are still in place. Do not set this option if another data replication solution is in place or CDC tables are used in the database.
  • LOGRELEASE_TASK: This method should be used if a separate job/task is created to release the truncation point for replication. For example, schedule a separate SQL Server Agent job to unconditionally call sp_repldone at an interval. Choosing the option LOGRELEASE_TASK will also result in SQL Server's agent jobs being dropped/disabled. However, as long as the scheduled log release task runs, the truncation point for replication is released, even if the capture job(s) is(are) not running. This option should only be used in conjunction with another replication or CDC solution if the log release task that is scheduled is aware of the requirements of the other solution.
  • NATIVE_DBMS_AGENT: This method should be used if native replication and/or CDC tables are used on the database. With this option, HVR will not drop/disable the native SQL Server agent jobs that are created when CDC tables and/or Articles are created. HVR will also not interfere with the release of the truncation point for replication. If CDC tables are used to enable supplemental logging it may cause I/O overhead (SQL Server jobs copy each change to a CDC table, which no-one uses).
 Sybase ASE
  • CAP_JOB_RETAIN: This method ensures Sybase transactions are always available for HVR. Log device usage will grow if the capture job is idle. Not using truncation points allows multiple systems to capture from the same database, but the user must ensure that older transaction files are kept available as transaction dumps until they are replicated.

[SQL Server] [Azure Data Warehouse] [Azure SQL Database]
[Sybase ASE]

Log_Truncater_Unavailable


This is a discovered property that stores information regarding HVR's support for log truncation.

[SQL Server] [Azure Data Warehouse] [Azure SQL Database]

NetWeaver_Native_DB_Dictionaries

trueIf set to true, HVR will query the native database dictionaries instead of the SAP dictionaries. When this property is defined, you cannot select/add the SAP Cluster and Pool tables to channel.[NetWeaver]

ODBC_DM_Lib_Path

path

Directory path where the ODBC Driver Manager Library is installed. This property is applicable only for Linux/Unix operating system.

For a default installation, the ODBC Driver Manager Library is available at /usr/lib64 and does not need to be specified. However, when UnixODBC is installed in for example /opt/unixodbc the value for this field would be /opt/unixodbc/lib.


[Unix]
[Db2 for i] [Greenplum] [SAP HANA] [ACID Hive] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database] [Teradata]

ODBC_Driver

odbcdriver
Name of the user defined (installed) ODBC driver used for connecting HVR to the database.

[ACID Hive] [Azure Data Warehouse] [Azure SQL Database] [Db2 for i] [Greenplum] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

ODBC_Inst

path

Directory path where the odbcinst.ini file is located. This property is applicable only for Linux/Unix operating system.

[Unix]
[Greenplum] [Teradata]

ODBC_Sysini

path

Directory path where the odbc.ini and odbcinst.ini files are located. This property is applicable only for Linux/Unix operating system.

For a default installation, these files are available at /etc directory and do not need to be specified using this property. However, when UnixODBC is installed in for example /opt/unixodbc the value for this field would be /opt/unixodbc/etc.

For Azure SQL Database, the odbcinst.ini file should contain information about the Azure SQL Database ODBC Driver under the heading [ODBC Driver version for SQL Server].

For Db2 for i, the odbcinst.ini file should contain information about the IBM i Access Client Solutions ODBC Driver under the heading [IBM i Access ODBC Driver 64-bit].

For Redshift, the odbcinst.ini file should contain information about the Amazon Redshift ODBC Driver under the heading [Amazon Redshift (x64)].

For SAP HANA, the odbcinst.ini file should contain information about the HANA ODBC Driver under heading [HDBODBC] or [HDBODBC32].

For Snowflake, the odbcinst.ini file should contain information about the Snowflake ODBC Driver under the heading [SnowflakeDSIIDriver].

[Db2 for i] [SAP HANA] [ACID Hive] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database]

Oracle_ASM_Home

path

Directoy path where the Oracle ASM instance is installed. In Linux/Unix, by default, this is located in /etc/oratab file.

This property is only relevant for a source Oracle with redo and/or archive files in ASM and Capture_Method is DIRECT.

The value of this property explicitly sets the system identifier (SID) for the ASM instance. Typically the value is +ASM or +ASM[instance_number], but in some cases it may be +asm in lowercase. HVR can automatically assess what it should be.

To find the value for the SID on a Linux/Unix environment, execute the following command:

ps -ef |grep pmon
This command returns the pmon (process monitor) processes for all instances running on the server, including the ASM instance(s), that always starts with +

[Oracle]

Oracle_ASM_Password

password

Password for Oracle_ASM_User.


[Oracle]

Oracle_ASM_TNS

connstring

Connection string for connecting HVR to Oracle's Automatic Storage Management (ASM) using Transparent Network Substrate (TNS).

The format for the connection string is host:port/service_name.

[Oracle]

Oracle_ASM_User

username

Username for connecting to Oracle ASM instance. This user must have sysasm privileges.

[Oracle]

Oracle_Container


This is a discovered property that stores information whether the Oracle database is Root Container or Pluggable Database (PDB).

[Oracle]

Oracle_Container_Root_Password

passwordPassword for the Oracle_Container_Root_User.[Oracle]

Oracle_Container_Root_SID

identifier

Unique name identifier of the Oracle root container.

[Oracle]

Oracle_Container_Root_TNS

connstring

Connection string for connecting HVR to Oracle root container using Transparent Network Substrate (TNS).

The format for the connection string is host:port/service_name.

[Oracle]

Oracle_Container_Root_User

usernameUsername for connecting to Oracle root container.[Oracle]

Oracle_Dataguard_Primary_Password

passwordPassword for Oracle_Dataguard_Primary_User.[Oracle]

Oracle_Dataguard_Primary_TNS

connstring

Connection string for connecting HVR to the Oracle data guard primary database using Transparent Network Substrate (TNS).

The format for the connection string is host:port/service_name.

[Oracle]

Oracle_Dataguard_Primary_User

usernameUsername for connecting HVR to the primary database. [Oracle]

Oracle_Home

path

Directory path where Oracle is installed.

[Oracle]

Oracle_NLS_LANG


This is a discovered property that stores the value of Oralce's NLS_LANG parameter used for connecting to Oracle database.[Oracle]

Oracle_SID

identifier

Unique name identifier of the Oracle instance/database.


[Oracle]

Oracle_TDE_Wallet_Password

passwordPassword for the Oracle TDE wallet.[Oracle]

Oracle_TNS

connstring

Connection string for connecting to the Oracle database using TNS (Transparent Network Substrate).

The format for the connection string is host:port/service_name.

Alternatively, you can add the connection details into the clients tnsnames.ora file and use that net service name in this field. This method requires easy connect enabled


[Oracle]

PostgreSQL_Pglib

path

Directory path of the library (lib) directory in the PostgreSQL installation. This property can be left empty to use the system default path.

Example: /postgres/935/lib

[PostgreSQL] [Aurora PostgreSQL]

PostgreSQL_XLog

path

Directory path containing the current PostgreSQL xlog files.

[PostgreSQL] [Aurora PostgreSQL]

S3_Bucket

bucket

Name or IP address of the Amazon S3 bucket.

#[Amazon S3]

S3_Bucket_Region


This is a discovered property that stores the region of the S3 bucket for the connected location.

S3_Encryption_KMS_Access_Key_Id

keyid

If client-side encryption using a CMK stored in AWS KMS is enabled (S3_Encryption_KMS_Customer_Master_Key_Id without S3_Encryption_SSE_KMS), this specifies the AWS access key id when querying KMS. By default, the credentials of the S3 connection is used.

#[Amazon S3]

S3_Encryption_KMS_Customer_Master_Key_Id

keyid

If S3_Encryption_SSE_KMS is defined, this specifies the KMS CMK ID which is used for the server-side encryption. Otherwise, it enables client-side encryption using a CMK stored in AWS KMS. For client-side encryption, each object is encrypted with a unique AES256 data key obtained from KMS. This data key is stored alongside the S3 object.

#[Amazon S3]

S3_Encryption_KMS_IAM_Role

role

If client-side encryption using a CMK stored in AWS KMS is enabled (S3_Encryption_KMS_Customer_Master_Key_Id without S3_Encryption_SSE_KMS), this specifies the IAM role when querying KMS. By default, the credentials of the S3 connection is used.

#[Amazon S3]

S3_Encryption_KMS_Region

region

If client-side encryption using a CMK stored in AWS KMS is enabled (S3_Encryption_KMS_Customer_Master_Key_Id without S3_Encryption_SSE_KMS), this specifies the KMS region when querying KMS. By default, the region of the S3 connection is used.

#[Amazon S3]

S3_Encryption_KMS_Secret_Access_Key

key

If client-side encryption using a CMK stored in AWS KMS is enabled (S3_Encryption_KMS_Customer_Master_Key_Id without S3_Encryption_SSE_KMS), this specifies the AWS secret access key when querying KMS. By default, the credentials of the S3 connection is used.

#[Amazon S3]

S3_Encryption_Master_Symmetric_Key

key

Enable client-side encryption using a master symmetric key for AES. Each object is encrypted with a unique AES256 data key. This data key is encrypted using AES256 with the specified master symmetric key and then stored alongside the S3 object.

#[Amazon S3]

S3_Encryption_Materials_Description

desc

Provides optional encryption materials description which is stored alongside the S3 object. If used with KMS, the value must be a JSON object containing only string values.

#[Amazon S3]

S3_Encryption_SSE

true

If set to true, enables server-side encryption with Amazon S3 managed keys.

#[Amazon S3]

S3_Encryption_SSE_KMS

true

If set to true, enables server-side encryption with customer master keys (CMKs) stored in AWS key management service (KMS). If S3_Encryption_KMS_Customer_Master_Key_Id is not defined, a KMS managed CMK is used.

#[Amazon S3]

SAP_Client

clientid

Three digit (000-999) identifier of the SAP client, which is sent to an AS ABAP upon logon.

[SAP NetWeaver]

SAP_Database_Owner


This is a discovered property that stores information about the database schema that contain the SAP data. This property is discovered when creating or modifying a SAP NetWeaver location.
When SAP dictionaries are used, HVR will add only SAP tables from the database to the channel.


SAP_Instance_Number

number

Two digit number (00-97) of the SAP instance within its host.

[SAP NetWeaver]

SAP_Source_Schema

schemaName of the database schema that contain the SAP data. Defining this property enables the SAP table explore and the  SAP unpack feature. If this property is defined, the SAP dictionaries are used, HVR will add only SAP tables from the database to the channel.

SAP_NetWeaver_RFC_Library_Path

path

Directory path containing the SAP NetWeaver RFC SDK library files.

For more information about the NetWeaver RFC SDK library file location, see section Install NetWeaver RFC SDK Libraries in SAP NetWeaver Requirements.

[SAP NetWeaver]

Salesforce_Bulk_API

true

If set to true, use Salesforce Bulk API instead of the SOAP interface.

This is more efficient for large volumes of data, because less round-trips are used across the network. A potential disadvantage is that some Salesforce.com licenses limit the number of bulk API operations per day.

If this property is defined for any table, then it affects all tables captured from that location.

[Salesforce]

Salesforce_Dataloader

path

Directory path where the Salesforce dataloader.jar file is located.

HVR uses Salesforce Dataloader tool for connecting to the Salesforce location. 

[Salesforce]

Salesforce_Endpoint

url

Complete URL for connecting HVR to Salesforce.

[Salesforce]

Salesforce_Serial_Mode

true

If set to true, force serial mode instead of parallel processing for Bulk API.

The default is parallel processing, but enabling Salesforce_Serial_Mode can be used to avoid some problems inside Salesforce.com.

If this property is defined for any table, then it affects all tables captured from that location.

[Salesforce]

Service_Password

password

Password for the Salesforce Service_User.

[Salesforce]

Service_User

username

Username for connecting HVR to Salesforce.

[Salesforce]

Snowflake_Role

role

Name of the Snowflake role.

[Snowflake]

Snowflake_Warehouse

warehouse

Name of the Snowflake warehouse.

[Snowflake]

SqlServer_Native_Replicator_Connection



If set to true, disables the firing of database triggers, foreign key constraints and check constraints during integration if those objects were defined with not for replication. This is done by connecting to the database with the SQL Server replication connection capability. When this property is defined, the database connection string format in SqlServer_Server must be server_name,port_number, the other two formats of the connection string are not supported. This port_number must be configured in the Network Configuration section of the SQL Server Configuration Manager.

When this property is defined, encryption of the ODBC connection is not supported.

[SQL Server]

SqlServer_Server

server

Name of the server (host) on which SQL Server is running and the Port number or SQL Server instance name.

The following formats are supported for this property:

  • server_name : Specify only server name and HVR will automatically use the default port to connect to the server on which SQL Server is running.
    Example: myserver
  • server_name,port_number : Specify server name and port number separated by a comma (,) to connect to the server on which SQL Server is running.
    This format is required when using custom port for connection or when SqlServer_Native_Replicator_Connection is defined.
    Example: myserver,1435
  • server_name\server_instance_name : Specify server name and server instance name separated by a backslash (\) to connect to the server on which SQL Server is running.
    This format is not supported on Linux.
    Example: myserver\HVR6048

For Azure SQL Database and Azure Synapse Analytics, this is the fully qualified domain name (FQDN) of the Azure SQL Database and Azure Synapse Analytics server respectively.

[SQL Server] [Azure Data Warehouse] [Azure SQL Database]

Staging_Directory

path

Directory path for bulk load staging files. For certain databases (ACID Hive, Redshift, and Snowflake), HVR splits large amount data into multiple staging files, to optimize performance.

This property is supported only for certain location classes. For the list of supported location classes, see Bulk load requires a staging area in Capabilities.

For Hive ACID, this should be an S3 or HDFS location.For Greenplum and HANA, this should be a local directory on the server where HVR connects to the DBMS.

For MariaDB or MySQL, when direct loading by the MySQL/MariaDB server option is used, this should be a directory local to the MySQL/MariaDB server on which the HVR user has write access from the server that HVR uses to connect to the DBMS. And when initial loading by the MySQL/MariaDB client option is used, this should be a local directory on the server where HVR connects to the DBMS.

For Redshift and Snowflake, this should be an S3 location.

[ACID Hive] [Greenplum] [MySQL] [MariaDB] [Aurora MySQL] [Redshift] [SAP HANA] [Snowflake]

Staging_Directory_Database

path

Directory path for the bulk load staging files visible from the database. This property should point to the same files as Staging_Directory.

This property requires Staging_Directory.

This property is supported only for certain location classes. For the list of supported location classes, see Bulk load requires a staging area in Capabilities.

For Greenplum, this should either be a local directory on the Greenplum head-node or it should be a URL pointing to Staging_Directory, for example a path starting with gpfdist: or gpfdists:.

For HANA, this should be a local directory on the HANA server which is configured for importing data by HANA.

For Hive ACID, this should be the S3 or HDFS location that is used for Staging_Directory.

For MariaDB or MySQL, when direct loading by the MySQL/MariaDB server option is used, this should be the directory from which the MySQL/MariaDB server should load the files. And when initial loading by the MySQL/MariaDB client option is used, this should be left empty.

For Redshift and Snowflake, this should be the S3 location that is used for Staging_Directory.

[ACID Hive] [Greenplum] [MySQL] [MariaDB] [Aurora MySQL] [Redshift] [SAP HANA] [Snowflake]

Staging_Directory_Is_Local

true

If set to true, the directory specified in Staging_Directory_Database is stored on the local drive of the file location's server.

If this property is not set to true or enabled, then by default the bulk load staging files are stored in the bucket or container available in the file location. For example, in Amazon S3, by default the staging directory is stored in the S3 bucket.

#Temporary / Staging directories

Stream_Client_Private_Key

path

Directory path where the .pem file containing the client's SSL private key is located.

[Kafka]

Stream_Client_Private_Key_Password

password

Password of the private key file that is specified in Stream_Client_Private_Key.

[Kafka]

Stream_Client_Public_Certificate

path

Directory path where the .pem file containing the client's SSL public certificate is located.

[Kafka]

Stream_Password

password

Password of the Stream_User.

[Kafka]

Stream_Public_Certificate

pathDirectory path where the file containing public certificate of Kafka server is located.[Kafka]

Stream_User

username

Username for connecting HVR to the Kafka server.

This property is required only if Kafka_Authentication_Method is set to USER_PASS.

[Kafka]

Supplemental_Logging

method

Specify what action should be performed to enable supplemental logging for tables.

Supplemental logging should be enabled for HVR to perform log-based capture of updates. For more details see, section Supplemental Logging in SQL Server Requirements.

Valid values for method are:

  • CDCTAB_ARTICLE (default): Enable supplemental logging of updates by creating a CDC table for the source table.
    If it is not possible to create the CDC table, then HVR will create replication articles instead. And, if it is not possible to create the replication articles also, then an error will be displayed.
  • ARTICLE_CDCTAB: Enable supplemental logging of updates by creating SQL Server transactional replication articles if the source table has a primary key.
    If it is not possible to create replication articles or if the source table does not have a primary key, then HVR will create CDC table instead. And, if it is not possible to create the CDC table also, then an error will be displayed.
  • EXISTING_CDCTAB_ARTICLE: Enable supplemental logging of updates by using the existing CDC table or replication article for the source table. If neither the CDC tables nor the replication article exists, then HVR will create a CDC table for the source table.
    If it is not possible to create the CDC table, then HVR will create replication articles instead. And, if it is not possible to create replication articles also, then an error will be displayed.
  • EXISTING_ARTICLE_CDCTAB: Enable supplemental logging of updates by using the existing CDC table or replication articles for the source table. If neither the CDC tables nor the replication article exist, then HVR will create replication articles for the source table.
    If it is not possible to create replication articles, then HVR will create a CDC table instead. And, if it is not possible to create the CDC table also, then an error will be displayed.
[SQL Server] [Azure Data Warehouse] [Azure SQL Database]

Supplemental_Logging_Unavailable



This is a discovered property that stores information whether the database supports supplemental logging.

Sybase

path

Directory path where the Sybase ASE database is installed.


Sybase_Authentication_Method

method

Authentication method for connecting HVR to Sybase ASE server.

Available options for method are:

  • USER_PASS
  • KERBEROS

For more information about using KERBEROS authentication, see section Kerberos Authentication in Sybase ASE Requirements.


Sybase_CT_Library

path

Directory path where the Sybase Open Client (CT library) is installed.


Sybase_Kerberos_Keytab


Directory path where the Kerberos keytab file is located. This keytab file contains the security key for the Database_User.

This property is required only if Sybase_Authentication_Method is set to KERBEROS.


Sybase_Kerberos_Server_Principal


The Kerberos Service Principal Name (SPN) of the Sybase ASE server.

This property is required only if Sybase_Authentication_Method is set to KERBEROS.


Sybase_Kerberos_Security_Mechanism


Name of the security mechanism that performs security services for this connection. Security mechanism names are defined in the Sybase libtcl.cfg configuration file.

If this property is not defined, the default mechanism defined in the libtcl.cfg file will be used.

This property is required only if Sybase_Authentication_Method is set to KERBEROS.


Sybase_Kerberos_Security_Services


Kerberos security mechanism services. It only defines how the connection behaves.

This property is required only if Sybase_Authentication_Method is set to KERBEROS.

Defining this property is optional.

Available options:

  • Mutual Client/Server Authentication: Both HVR and the Sybase server are required to authenticate themselves.
  • Encrypted Connection: Enables encrypted connection between HVR and the Sybase server.
  • Data Integrity Checking: Enables data integrity checking.
  • Replay Transmission Detection: Enables data replay detection.
  • Data Out-Of-Sequence Detection: Enables out-of-sequence detection.
  • Data Origin Verification: Enables data origin stamping service.
  • Channel Binding: Enables channel binding.

Teradata_TPT_Lib_Path

path

Directory path where the Teradata TPT Library is installed.

Example: /opt/teradata/client/16.10/odbc_64/lib

[Teradata]

Trigger_Quick_Toggle

true

If set to true, allows end user transactions to avoid lock on toggle table.

The toggle table is changed by HVR during trigger-based capture. Normally all changes from user transactions before a toggle is put into one set of capture tables and changes from after a toggle are put in the other set. This ensures that transactions are not split. If an end user transaction is running when HVR changes the toggle then HVR must wait, and if other end user transactions start then they must wait behind HVR.

Defining this property allows other transactions to avoid waiting, but the consequence is that their changes can be split across both sets of capture tables. During integration these changes will be applied in separate transactions; in between these transactions the target database is not consistent.

For Ingres, variable ING_SET must be defined to force readlock=nolock on the quick toggle table. For example,

$ ingsetenv ING_SET 'set lockmode on hvr_qtogmychn where readlock=nolock'

This property requires Capture_Method set to DB_TRIGGER.
[Ingres] [Oracle]

Trigger_Toggle_Frequency

secs
Instruct HVR's trigger-based capture jobs to wait for a fixed interval secs (in seconds) before toggling and reselecting capture tables.

This property requires Trigger_Quick_Toggle.

If this property is not defined, the trigger-based capture job dynamically waits for a capture trigger to raise a database alert. This requires the Oracle's DBMS_ALERT package. For more information about this package, refer to the Oracle documentation. Raising and waiting for database alerts is an unnecessary overhead if the capture database is very busy.
[Db2 Database] [Ingres] [Oracle] [SQL Server] [Azure SQL Database]

View_Class

class

Class of the database (defined in View_Database_Name) that is used for providing an SQL based view on the file location. For example, Hive External Tables.

All

View_Class_Flavor


This is a discovered property that stores the flavor of the database (defined in View_Database_Name).

All

View_Class_Version


This is a discovered property that stores the version of the database (defined in View_Database_Name). HVR stores and uses this number internally to determine which Hive functionality should HVR attempt to use.

For example, if value 121 is stored in this property it indicates Hive version 1.2.1.

All

View_Database_Char_Encoding


This is a discovered property that stores the character encoding of the database (defined in View_Database_Name).

#Database generic

View_Database_Client_Private_Key

path

Directory path where the .pem file containing the client's SSL private key is located.

[ACID Hive]

View_Database_Client_Private_Key_Password

passwordPassword for the private key file specified in View_Database_Client_Private_Key.[ACID Hive]

View_Database_Client_Public_Certificate

path

Directory path where the .pem file containing the client's SSL public certificate is located.

[ACID Hive]

View_Database_Default_Schema

schema

This is a discovered property that stores the name of the default schema in the database (defined in View_Database_Name).

#Database generic

View_Database_Host

host

The hostname or IP-address of the server on which the database (defined in View_Database_Name) is running.

[ACID Hive] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 for i] [Greenplum] [MariaDB] [MySQL] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

View_Database_Name

name

Name of the database used for an SQL based view on the file location.

[Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [SAP HANA] [ACID Hive] [Ingres] [Actian Vector] [MySQL] [MariaDB] [Aurora MySQL] [PostgreSQL] [Aurora PostgreSQL] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database]

View_Database_Nchar_Encoding

charset

This is a discovered property that stores the national character encoding of the database (defined in View_Database_Name).

#Database generic

View_Database_Password

password

Password for the View_Database_User.

[ACID Hive] [Actian Vector] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [MariaDB] [MySQL] [Oracle] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

View_Database_Port

port

Port number for the database (defined in View_Database_Name).

[ACID Hive] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Greenplum] [MariaDB] [MySQL] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake]

View_Database_Public_Certificate

pathDirectory path where the .pem file containing the server's public SSL certificate signed by a trusted CA is located.

View_Database_User

user

Username for connecting to the database (defined in View_Database_Name).

[ACID Hive] [Actian Vector] [Aurora MySQL] [Aurora PostgreSQL] [Azure Data Warehouse] [Azure SQL Database] [Db2 Database] [Db2 for i] [Db2 for z/OS] [Greenplum] [Ingres] [MariaDB] [MySQL] [Oracle] [PostgreSQL] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

View_Hive_Authentication_Method

method

Authentication method for connecting HVR to Hive Server 2 instance.

This property is required only if View_Hive_Server_Type is set to Hive Server 2.

Available options for method are:

  • NONE

  • KERBEROS

  • USER

  • USER_PASS

  • HDINSIGHT

[Amazon S3], [Azure DLS], [HDFS]

View_Hive_HTTP_Path

url

The partial URL corresponding to the Hive server.

This property is required only if View_Hive_Thrift_Transport is set to HTTP.

Amazon S3, Azure DLS, HDFS

View_Hive_Kerberos_Host

name

Fully Qualified Domain Name (FQDN) of the Hive Server 2 host. The value of Host can be set to _HOST to use the Hive server hostname as the domain name for Kerberos authentication.

If Hive_Service_Discovery_Mode is disabled, then the driver uses the value specified in the Host connection attribute.
If Hive_Service_Discovery_Mode is set to ZooKeeper, then the driver uses the Hive Server 2 host name returned by ZooKeeper.

This property is required only if View_Hive_Authentication_Method is set to Kerberos.

[Amazon S3], [Azure DLS], [HDFS]

View_Hive_Kerberos_Realm

realm

Realm of the Hive Server 2 host.

It is not required to specify any value in this property if the realm of the Hive Server 2 host is defined as the default realm in Kerberos configuration.

This property is required only if View_Hive_Authentication_Method is set to Kerberos.

[Amazon S3], [Azure DLS], [HDFS]

View_Hive_Kerberos_Service

name

Kerberos service principal name of the Hive server.

This property is required only if View_Hive_Authentication_Method is set to Kerberos.

[Amazon S3], [Azure DLS], [HDFS]

View_Hive_Server_Type

type

Type of the Hive server to which HVR will be connected.

Available options for type are:

  • Hive Server 1: HVR connects to a Hive Server 1 instance.
  • Hive Server 2: HVR connects to a Hive Server 2 instance.
[Amazon S3], [Azure DLS], [HDFS]

View_Hive_Service_Discovery_Mode

mode

Mode for connecting to Hive.

This property is required only if View_Hive_Server_Type is set to Hive Server 2.

Available options for mode are:

  • NONE: HVR connects to Hive server without using the ZooKeeper service.
  • ZOOKEEPER: HVR discovers Hive Server 2 services using the ZooKeeper service.
[Amazon S3], [Azure DLS], [HDFS]

View_Hive_Thrift_Transport

protocol

Transport protocol to use in the Thrift layer.

This property is required only if View_Hive_Server_Type is set to Hive Server 2.

Available options for protocol are:

For information about determining which Thrift transport protocols your Hive server supports, refer to HiveServer2 Overview and Setting Up HiveServer2 sections in Hive documentation.

Amazon S3, Azure DLS, HDFS

View_Hive_Zookeeper_Namespace

namespace

Namespace on ZooKeeper under which Hive Server 2 nodes are added.

This property is required only if View_Hive_Service_Discovery_Mode is ZooKeeper.

[Amazon S3], [Azure DLS], [HDFS]

View_ODBC_DM_Lib_Path

path

Directory path where the ODBC Driver Manager Library is installed. This property is applicable only for Linux/Unix operating system.

For a default installation, the ODBC Driver Manager Library is available at /usr/lib64 and does not need to be specified. However, when UnixODBC is installed in for example /opt/unixodbc the value for this field would be /opt/unixodbc/lib.

[Db2 for i] [Greenplum] [SAP HANA] [ACID Hive] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database] [Teradata]

View_ODBC_Driver

drivername

User defined (installed) ODBC driver for connecting HVR to the database.

[ACID Hive] [Azure Data Warehouse] [Azure SQL Database] [Db2 for i] [Greenplum] [Redshift] [SAP HANA] [SQL Server] [Snowflake] [Teradata]

View_ODBC_Sysini

path

Directory path where the odbc.ini and odbcinst.ini files are located. This property is applicable only for Linux/Unix operating system.

For a default installation, these files are available at /etc and do not need to be specified. However, when UnixODBC is installed in for example /opt/unixodbc the value for this property would be /opt/unixodbc/etc.

  • For Db2i, the odbcinst.ini file should contain information about the IBM i Access Client Solutions ODBC Driver under the heading [IBM i Access ODBC Driver 64-bit].
  • For Redshift, the odbcinst.ini file should contain information about the Amazon Redshift ODBC Driver under the heading [Amazon Redshift (x64)].
  • For SAP HANA, the odbcinst.ini file should contain information about the HANA ODBC Driver under heading [HDBODBC] or [HDBODBC32].
  • For Snowflake, the odbcinst.ini file should contain information about the Snowflake ODBC Driver under the heading [SnowflakeDSIIDriver].
  • For Azure SQL Database, the odbcinst.ini file should contain information about the Snowflake ODBC Driver under the heading [ODBC Driver version for SQL Server].
[Db2 for i] [SAP HANA] [ACID Hive] [Redshift] [Snowflake] [SQL Server] [Azure Data Warehouse] [Azure SQL Database]

WASB_Account

account

Name of the Azure Blob Storage account.

#Azure Storage Blobs

WASB_Container

container

Name of the container available within the Azure Blob Storage account.

#Azure Storage Blobs