Hitachi Vantara Pentaho Community Forums
Results 1 to 13 of 13

Thread: JOb - Flie Exists - Wildcard

  1. #1

    Default JOb - Flie Exists - Wildcard

    Hi guys,

    I need to monitor if text files are located in a specific folder. I have tried several ways to do it, but I have not reached a valid conclusion yet.

    My questions is if the file exists job step, can accept wildcards to monitor a specific folder or type of archives. I have a FTP job that constantly is dl text archives, the next part of the flow needs to know if there are text archives to process them, In case that there are no text archives, it must abort. That is the main idea.

    Any advice will be appreciated.

    Regards,

    Ben

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    Can't accept wildcards for the moment. Nothing planned to build it in yet.

    Regards,
    Sven

  3. #3

    Default

    Instead of the job-level File Exist step, you can take a look at the 'Get File Names' step in transformations.

    Just need to be careful with periods (.) when defining your wildcards -- normal java regexp.

    example wildcard: file_.*\.txt

  4. #4

    Default JOb - Flie Exists - Wildcard Reply

    Thanks dhartford,

    I am using Get File Names Step, But the issue here is when executing the transformation from a Job, this step executes correclty even if it finds archives or not. With this in mind I try to have something that can give me an error if there are no files present, so I can abort the job, in that case.

    I think something can be done using JavaScript But I have little knowledge on this subject. The Idea here is how to force an error when there are no files.

    Any advice will be appreciated.

    Regards,

    Ben

    Quote Originally Posted by dhartford View Post
    Instead of the job-level File Exist step, you can take a look at the 'Get File Names' step in transformations.

    Just need to be careful with periods (.) when defining your wildcards -- normal java regexp.

    example wildcard: file_.*\.txt

  5. #5
    Join Date
    Jul 2007
    Posts
    247

    Default

    Design a job that runs a transform.

    In this transform, get all filenames using the GetFileNames step. Then count all your rows by using the Group By step. Copy your result to another job by using the Copy Rows to Result step.

    In the job you can evaluate the result from the transform. See this tip for an example how to evaluate a job parameter...

    Regards,
    Ben

  6. #6
    Join Date
    Apr 2008
    Posts
    20

    Default Updated link?

    Can you supply an updated link for this? Very usefull for me.

  7. #7
    Join Date
    May 2006
    Posts
    4,882

    Default

    It's also in your local PDI installation under doc/English/tips

    Regards,
    Sven

  8. #8
    Join Date
    Apr 2008
    Posts
    20

    Default Evaluate

    Hi Sven

    The only problem with this, is the versions... the evaluate parameter.. I can not find this functionality in 3.0.2?

  9. #9
    Join Date
    May 2006
    Posts
    4,882

    Default

    That\s not what you asked originally ... in v2.5 that job entry was a special case of the javascript one... in v3 just use javascript.

    Regards,
    Sven

  10. #10
    Join Date
    Apr 2008
    Posts
    20

    Default bugging.. :-)

    Just to bug you... how would you go about it?

    I am not yet very proficient in JavaScript and have only worked with Spoon this week.. I have worked with ETL before, and very enthusiastic about this product, but will appreciate very much if you could spare a little help.

  11. #11

    Default Folder is empty

    Hi,

    You have since > PDI 3.0 (i guess) a job entry called "Folder is empty".
    (please see attached picture).

    rgds

    Samatar
    Last edited by shassan2; 09-02-2008 at 04:33 PM.

  12. #12
    Join Date
    Apr 2008
    Posts
    20

    Default ... :-s

    I have version 3.0.2 build 538, but can not find this anywhere... is it in the newest version?

  13. #13

    Default

    Sure :-)

    Samatar

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.