Hitachi Vantara Pentaho Community Forums
Page 1 of 2 12 LastLast
Results 1 to 10 of 14

Thread: Compare list of files on FTP

  1. #1
    Join Date
    Mar 2015
    Posts
    8

    Default Compare list of files on FTP

    Hello all,

    I've been trying to search for this, but haven't found anything, so I'm hoping someone here can help me. I have an FTP server where a number of files get dropped in every day. I'm trying to create a job/transformation that checks this folder and makes sure that an expected list of files are there. The only way I've been able to do it is to download them all and then compare them to my list - is there a simpler way of doing this? Can I somehow retrieve the list of files on the FTP?

    Thank you for your help.

  2. #2
    Join Date
    Mar 2015
    Posts
    8

    Default

    Does anyone have any suggestions?

  3. #3
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    Hi.
    I think that you can GET FILENAME from an FTP folder using a step in a Job (not a transformation).
    Once you get that list you can compare it with your own.

    Another option would be to get a list of files using a different product (i tend to use cUrl) and download them as csv - you can then compare this list with yours.
    -- Mick --

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    In my Kettle version step Get-File-Names does retrieve all the filenames in a specified folder just fine.

    Name:  LIST.jpg
Views: 235
Size:  39.6 KB
    So long, and thanks for all the fish.

  5. #5
    Join Date
    Mar 2015
    Posts
    8

    Default

    @marabu: thank you for that, it got me a step closer in the right. any idea on how to do it if I need a user/pass to logon? I tried ftp://userass@url, but that didn't work...

  6. #6
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    I don't think authentication via URL is a problem here.
    Didn't you get error messages in your Kettle log?
    What about proxy usage?
    So long, and thanks for all the fish.

  7. #7
    Join Date
    Mar 2015
    Posts
    8

    Default

    The only thing I get is "Sorry, no rows found to be previewed." No other error message...

  8. #8
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    That's the text from the message dialog box during preview mode.
    Run the transformation and watch the messages in the log window, instead.
    BTW: If no step is marked red during preview, technically there wasn't an error.
    So long, and thanks for all the fish.

  9. #9
    Join Date
    Mar 2015
    Posts
    8

    Default

    Getting this error: org.apache.commons.vfs.FileNotFolderException: Could not list the contents of [removed] because it is not a folder.
    Tried with and without slash at the end, still the same error. The folder definitely exists as I can access it through FTP client or via Chrome.

  10. #10
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    Wish I could see what you're doing.
    So long, and thanks for all the fish.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.