Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: How to append all results in a file while selecting execute for every input row ?

  1. #1
    Join Date
    Dec 2013
    Posts
    3

    Post How to append all results in a file while selecting execute for every input row ?

    Hi all,

    My question is how to append each loop result to the same output file while selecting "execute for every input row" option in a sub-job ?

    Here is my scenario : some informations (filename, path, rowscount etc...) related to CSV files would be collected and passed to
    Pentaho Report Designer to generate reports. for simplicity's sake, the temporary output step is plein text file and the process described as below:

    [start]--[set-variable]--[get_File_Names_rows]--[get_information_for_each_file](loop) --[end]

    I totally followed the tutorial here, which explains how to retrieve all files in a single "get File Names" step and loop each one to do some stuff in a sub-job.
    The only difference is that I want every output row to be written in the same file NOT separated file.

    Please find attached the created jobs, I just put an output text file at the last step called "generate_entities_log", but apparently it receives only the last record of the loop.

    --------------------------------------------------------------------------------

    The alternative I've made so far without success:

    1, generate separate file from each loop and try to merge all these files (20 CSV files) together, I've checked "merge join" step, as well as posts in forum but got no idea how to make it.
    2, use 'modified javascript value' step to receive all rows from each loop, and then write to output file all at once. If javascript step is put within the loop it receive only each loop record,
    while outside the loop it receive only the last loop result.

    Any suggestions or technical hints would be appreciated.

    Thanks in advance for your help!

    PS: Please modify the extension of the attached zip file to .7z, to execute the attached job, you have to defined in kettle.properties file a variable so that Kettle would know where to find the config.properties file, in which data file path are parametrized. the related tutorial is here.



    Shaokun
    Attached Files Attached Files
    Last edited by Chibi; 12-30-2013 at 12:20 PM.
    pdi-ce-4.4.0-stable
    prd-ce-3.9.1-GA 16112
    java-sdk-1.7.0_40
    win7

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Did you ever notice the Text File Output option "Append" on the Content tab?
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Dec 2013
    Posts
    3

    Default

    Hi Marabu,

    Thanks for your reply, it works like a charm !

    Shaokun
    pdi-ce-4.4.0-stable
    prd-ce-3.9.1-GA 16112
    java-sdk-1.7.0_40
    win7

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.