Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Wait For File: need to wait for multiple files

  1. #1

    Default Wait For File: need to wait for multiple files

    Is there a way to use the Wait For File module (maybe multiple copies) so that processing would not proceed until all files were present?
    Right now I need to wait for 20 files to come in and then move to the next step. If I don't have the complete fileset then that is a failure and we need to abort. How can I deal with this in PDI?

    Gerry

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    - 20 job entries
    - javascript
    - some combination of jobs using variables
    - shelling out

    Regards,
    Sven

  3. #3
    Join Date
    Jan 2008
    Posts
    22

    Default

    Quote Originally Posted by sboden View Post
    - 20 job entries
    - javascript
    - some combination of jobs using variables
    - shelling out

    Regards,
    Sven
    Any sample for this ?

    Regards,

    Thunderbird

  4. #4
    DEinspanjer Guest

    Default

    Quote Originally Posted by thunderbird View Post
    Any sample for this ?
    Your case sounds a bit tedious for someone to write up a sample by their own good graces. I'm sure if Matt were to respond to this thread, he'd suggest that you buy a subscription then someone might be able to do so.

    Even then, your stated case is a bit too vague for even better ideas.

    Are the twenty files known ahead of time? If so, you could probably pass them in to a single job entry configured to translate the row to arguments and to execute for each input row. That job would have a wait for in it.

    The end effect would be that it would wait for the first file to be seen then it would jump to looking for the next file, etc. If these files likely appear all at once, you would see it run very quickly through the 19 other wait jobs.

    hth.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.