Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: CSV File Input --> Accept filename from previous step inconsistent

  1. #1

    Default CSV File Input --> Accept filename from previous step inconsistent

    Hello experts,
    I am using a Job having Add Files to result step, passing the file name to the subsequent transformations.
    In that transformation we use GetFilesFromResult to get the filename, which get passed to the CSV File Input step.
    So the observation is :
    1> Initially, the CSV File Input Step was configured to read a file from a fixed file, known path. No input was passed to this step.
    2> The step was later changed to accept the filename from the above mentioned GetFilesFromResult step.
    3> The CSV file input failed with the error - Cannot find filename.
    4> We want to know why the CSV File input step initializes without waiting for the results from the previous step.
    5> We added a fresh CSV File Input Step, we directly configured it for accepting filename from the previous step.
    6> This step works fine.

    The 2 instances of the same CSV File Input show different behaviours. Is this inconsistent or we misunderstood its functioning? Please clarify.
    Added the snapshot of the transformation for ready reference.

    -Thanks and Regards
    Yogesh
    Attached Images Attached Images  

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    I rarely use step "CSV Input" nowadays, Text File Input can be conceived as an improved step in comparison.
    I suggest you enable TFI option "Accept filenames from prevoius step" and see wher it gets you.
    So long, and thanks for all the fish.

  3. #3

    Default

    We faced an issue in TFI where it failed for a record which has an CRLF in a column value and hence switched to CSV.
    Attached Images Attached Images  

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    If you want to stick with step CSV Input, I wonder how you manage to provide a dynamic filename given the design of your transformation.
    CSV-Input doesn't support filenames from a field, so you're left with Kettle variables, but those must be defined outside of your transformation.
    So long, and thanks for all the fish.

  5. #5

    Default

    If you have a step attached to CSVInput the way we had in our transformation for example we have an WriteLog step (any step will do) then it allows you to substitute dynamic filenames.

    -Thanks
    Yogesh

  6. #6
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    You are right, I keep forgetting, never use CSV Input as I said.
    If you want me to have a look at it, prepare a transformation that I can run on my machine.
    If you are confident you found a bug, you can open a Jira case right away.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.