Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Retrieving file on server.

  1. #1
    Join Date
    Aug 2013
    Posts
    16

    Default Retrieving file on server.

    Hello all!
    I am having some issues trying to access a file to read with my transformation. The ETL is located on a server and I need to retrieve a file that is loaded daily unto the system. Previously, I have been able to just use ${Internal.Transformation.Filename.Directory} and the file name with a dynamic variable(the date) however this file is located on the server but in a different location.

    For example,
    If the transformations are located on C:\Users\A\Transformations and the txt files are located on C:\DAILYLOAD\EXTRACTS

    Can I directly link to this address in the text file input or would it be more beneficial to use a "Get a file with FTP" step? I would like to keep the files in the original location but if need be, I can transfer to a temp folder in the same location of the transformation.


    What would be the best way to dynamically adjust the file name(date) to make sure only the most updated file is being used?
    If C:\DAILYLOAD\EXTRACTS have the files
    TEST1_20140603
    TEST2_20140603
    TEST1_20140604
    TEST2_20140604

    Is there some slick way to pull the most updated Test1 and Test2? Or should I use variable substitution?
    -----------

    Thank you so much for all your help. I hope to one day be able to contribute back to this wonderful forum soon.

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Given a folder containing some versioned files, you want to pick the youngest version of each file?
    The version is a timestamp (yyyyMMdd) that's separated from the file's basename by an underscore?

    Name:  GFN-2.jpg
Views: 55
Size:  10.7 KB


    1. GFN picks up certain files from a folder specified by a variable
    2. RE splits the filenames at the last underscore into basename and version
    3. SR sorts by basename to prepare the grouping
    4. GB finds the latest version (MAX) for each basename
    5. FR only passes the filenames matching the latest version
    6. TFI processes the passed filenames
    Attached Files Attached Files
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.