Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Getting One file with date on the FTP

  1. #1

    Default Getting One file with date on the FTP

    Hi there !

    I wish to get a file from a FTP where there is 10 files named similarly, with the date at the end. Example:

    file_20100826.csv
    file_20100825.csv
    file_20100824.csv
    ...

    how could I get only the last one ? with env variables ?

    Thank you for your help

    Rom's

  2. #2

    Default

    Roms

    Is the date of the file known? e.g today's date. If so you can get a date from a Get System Info step, using a transformation similar to the tutorial example:

    http://wiki.pentaho.com/display/EAI/...ng+Hello+World

    You'll need to use a Select values step to convert the meta data of the date to the string format you need using the pattern "yyyyMMdd".

    Then you can use this variable in your Get a File with FTP step as a wildcard ${YOUR_DATE}

    Another idea - why not download all the files (if there are only 10) and use the "Don't overide files" flag with "If File exists=Skip" so that you don't re-download them each time the job runs. Then use "add filename to result" to send the filename(s) to the transformations that need them.
    Tony, Cloud2Land.com

    Get the latest snapshot Pentaho Data Integration (Kettle) builds from SVN on my blog!
    http://www.cloud2land.com/

  3. #3

    Default

    Hello Tony,

    thank you for your reply,

    so I have to create a transformation to have the date, right ? I can't have it directly into the kjb ?

    Thank's for your help

    Quote Originally Posted by Tony.Cloud2Land View Post
    Roms

    Is the date of the file known? e.g today's date. If so you can get a date from a Get System Info step, using a transformation similar to the tutorial example:

    http://wiki.pentaho.com/display/EAI/...ng+Hello+World

    You'll need to use a Select values step to convert the meta data of the date to the string format you need using the pattern "yyyyMMdd".

    Then you can use this variable in your Get a File with FTP step as a wildcard ${YOUR_DATE}

    Another idea - why not download all the files (if there are only 10) and use the "Don't overide files" flag with "If File exists=Skip" so that you don't re-download them each time the job runs. Then use "add filename to result" to send the filename(s) to the transformations that need them.

  4. #4

    Default

    I haven't replied to the second part because we have to have the possibiliy to re-launch the script if needed in the day.

    Thank you again

  5. #5

    Default

    up ?

  6. #6

    Default

    Quote Originally Posted by roms View Post
    I haven't replied to the second part because we have to have the possibiliy to re-launch the script if needed in the day.
    I'm not sure what you mean, do you mean that you wish to run the job several times a day and that you *don't* want to download a new/changed file?

    so I have to create a transformation to have the date, right ? I can't have it directly into the kjb ?
    Well the transformation could return a date to the job by means of a variable which can then be used by the FTP component in the job. Attached is a demo job and transformation to explain.
    get_date.ktr
    ftp.kjb

    Here is another suggestion for you - have you considered passing the date of filename as a parameter to the job on the command line?
    Tony, Cloud2Land.com

    Get the latest snapshot Pentaho Data Integration (Kettle) builds from SVN on my blog!
    http://www.cloud2land.com/

  7. #7

    Default

    Hello,

    thank's for replying. I just want to download the file of the day.

    The parameter could be a good solution for the server version. But for the interface version, is it possible to prompt and ask it (so that users don't forget...) ?

    Thank you again
    Quote Originally Posted by Tony.Cloud2Land View Post
    I'm not sure what you mean, do you mean that you wish to run the job several times a day and that you *don't* want to download a new/changed file?



    Well the transformation could return a date to the job by means of a variable which can then be used by the FTP component in the job. Attached is a demo job and transformation to explain.
    get_date.ktr
    ftp.kjb

    Here is another suggestion for you - have you considered passing the date of filename as a parameter to the job on the command line?

  8. #8

    Default

    is it possible to prompt and ask it (so that users don't forget...) ?
    Well you could design a job to prompt the user if they have not set a parameter/variable by using the "Display Msgbox Info" job entry. But there is no built-in GUI functionality to prompt for user input at run-time.
    Tony, Cloud2Land.com

    Get the latest snapshot Pentaho Data Integration (Kettle) builds from SVN on my blog!
    http://www.cloud2land.com/

  9. #9
    Join Date
    Jul 2014
    Posts
    5

    Default

    Thx mate, awesome post! You saved my day

  10. #10

    Default

    Hi ,

    I have same question like Rom's . Could you please help ?

    Thanks in advance.

    Vishnu

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.