Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Transforming one file at a time

  1. #1
    Join Date
    Nov 2013
    Posts
    6

    Default Transforming one file at a time

    Hi,

    I have created a transformation to fetch a csv file and convert it into a desired format by picking data from specific rows (e.g. 1st and 3rd row).

    The transformation works fine with one file; however while dealing with multiple files the transformation starts transforming consecutive files as well for each step. Due to this the system is not able to pick up correct data for file 2 onwards and gives an error.

    I would like to know if there's any way where I can control the execution of multiple files and have the Transformation only process one file at a time i.e. first process all steps for file 1 and then repeat all steps for file 2 onwards.

    FYI: I am using text file input and regular expressions to pick up multiple files.

    Any help would be much appreciated.

    I am having issues while uploading a screenshot, will try again later and upload it soon.

    Thanks in advance.

    Cheers.

    Kunal

  2. #2
    Join Date
    Nov 2013
    Posts
    382

    Default

    Create a job to get the list of files. Copy rows to result as final step. Now call your transformation passing the fields as parameters, so you transformation uses the job "output" as input and processes each file on its own.

  3. #3
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Quote Originally Posted by DepButi View Post
    Now call your transformation passing the fields as parameters, so you transformation uses the job "output" as input and processes each file on its own.
    It might help to enable row processing.

    Name:  per-row.png
Views: 375
Size:  21.6 KB
    So long, and thanks for all the fish.

  4. #4
    Join Date
    Nov 2013
    Posts
    6

    Default

    Thanks guys for help. Really appreciate it.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.