Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Step "Wait for file" with different File name

  1. #1
    Join Date
    Jun 2017
    Posts
    29

    Question Step "Wait for file" with different File name

    Hi!

    I get a file every day to use in my PDI solution, however, it comes with a specific name "CompanyName_LoadBI_YYYYmmDD.HHMM.zip".
    Because date and time are variables, I can not put a specific name in "Wait for file" -> "File name".
    Remembering that there are not multiple files, it will always be the same file once a day almost always at the same time, but with different date / time.


    I tried to do the following in the "File name" field:
    C:\Users\Company\upload\.*CompanyName_LoadBI.*\.zip$


    But to no avail.
    Can you help me?


    Thanks!
    Last edited by lucas.turco; 01-15-2019 at 01:19 PM.

  2. #2
    Join Date
    Apr 2008
    Posts
    4,663

    Default

    What about using a transformation to:
    1) Get File Name
    2) Set Variable

    Then connect that to a "Wait for File" using the variable that you just set...

  3. #3
    Join Date
    Jun 2017
    Posts
    29

    Default

    Hi gutlez! Thanks for answering.


    I do not think that would work that way.
    How will I know if the file has arrived or not to execute the transformation with "Get File Name"? This would be the "Wait for file". But to use "Wait for file" I need the filename..


    My mind exploded. x.x

    EDIT: I want to use the step "Wait for file" to wait for my client to send the file to perform load.
    Sometimes it delays sending, returning on failure in my job. I would like to use the "Wait for file" to remedy this problem.

    Quote Originally Posted by gutlez View Post
    What about using a transformation to:
    1) Get File Name
    2) Set Variable

    Then connect that to a "Wait for File" using the variable that you just set...
    Last edited by lucas.turco; 01-15-2019 at 02:21 PM.

  4. #4
    Join Date
    Apr 2008
    Posts
    4,663

    Default

    Ok...

    So, what about:
    Job:
    1) Start
    2) Transformation
    2.1) Get File Names (Note: Only requires Directory and RegEx to run)
    2.2) Set Variable (Filename)
    3) Simple Evaluation (Variable Filename is not null)
    3.1 (Fail) -> Wait 5s -> 2) Transformation
    3.2 (Success) -> Wait for File (Filename = Variable Filename) (Ensure that it has finished writing!)
    4) Remainder of Job Steps


    However, this setup does have a known issue with potential "memory leaks"
    The correct design would be to kick the overall job off every 5 seconds or so, and do a job:
    Job:
    1) Start
    2) Transformation
    2.1) Get File Names (Note: Only requires Directory and RegEx to run)
    2.2) Set Variable (Filename)
    3) Simple Evaluation (Variable Filename is not null)
    3.1 (Fail) -> Write to Log ("$Date $Time : Nothing to process...") -> "Success" (Yes, there's a step called Success)
    3.2 (Success) -> Wait for File (Filename = Variable Filename) (Ensure that it has finished writing!)
    4) Remainder of Job Steps
    Last edited by gutlez; 01-15-2019 at 08:57 PM.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.