How To: Use /MaxFileSize parameter to split the incoming file into number of files on the Target in csv format

Viewing 1 post (of 1 total)
  • Author
    Posts
  • #18506
    Pallavik
    Keymaster

    How To:  Use /MaxFileSize parameter to split the incoming file into number of files on the Target in csv format

    Scenario:

    This parameter allows users to bundle the incoming rows in a file. The rows are bundled into the same file until after the specified threshold is reached. After this is done the file is sent to the target and HVR starts writing data to the new file.

    Some of the use case can be that the customer wants the smaller size of the files so that it is easier to manage the files.

    Pre-requisites:

    1) A channel f_rep already present in HVR hub

    2) In this particular example we will look at csv file as the Target

    Steps:

    Step 1: Create the channel to replicate a table from oracle database to a file in csv file format. The channel will look like below

    Step 2: Add Integrate Action MaxFileSize to split the files in bytes. For example in this case I will specify the MaxFileSize to be 1000 bytes which will create files on the target with size 1000 bytes

    Step 3: Perform HVR Initialize to make this action take effect to the channel.

    Step 4: Perform HVR Refresh. This option will create number of files on the target with size 10KB and a little over.

    Please note that for efficiency reasons HVR decides to start writing new file depending on the length of previous row, not the current row. This means that the actual size of the may slightly exceed the specified value.

     

    If you look at the properties of each file, it is evident that it is approximately 10KB

    Step 5: Start HVR Integrate Job. This will create the files with size of approximately 10KB each on the Target.

    Notes

    1) With integration, if the incoming data is less than MaxFileSize and the integration cycle completes then a file less than MaxFileSize is created on the Target. Its only when the incoming data is more than
    MaxFileSize, this parameter comes into effect.

    2) Similar behavior is for refresh operation that the last file created will be more often than not less than MaxFileSize

Viewing 1 post (of 1 total)
  • The forum ‘How to’ is closed to new topics and replies.

© 2020 HVR

Test drive Contact us