Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Get a file with FTP - using job parameters

  1. #1

    Default Get a file with FTP - using job parameters

    I tried to create a job that gets a file by FTP. I ran it under Spoon and filled in the job parameters used in the "Files" tab of the FTP step. Nothing was retrieved and no feedback shows even with logging set to debugging mode. Then I tried running Wireshark to view communication with the FTP server. What I thought was strange was that the FTP step quit before issuing a GET or MGET command. It ran a LIST command then a TYPE A to set the transfer type to ASCII then issued a QUIT command.

    I suppose the step internally checks the output of the LIST command and only issues a GET for anything matching the regular expression on the "Files" tab. Any recommendations on how I can debug my regular expression? If I need to run Spoon under Eclipse how would I start?

    Thanks for reading...

  2. #2

    Default

    I am working on this issue again. I have one job using FTP successfully. The best way I found to troubleshoot the regular expression is to print it to the log with the "Write to Log" step. It is not perfect but at least the variables get interpolated.

    Now I have one job that just freezes in the FTP GET step. It gets to the point of starting to download the file with an FTP GET command.

    Running wireshark again it appears that the Spoon process is freezing and not reading the TCP buffer.
    The TCP Window Size goes down to zero and eventually the server stops sending.

    Any ideas? Please save me from manually getting these files by FTP every two weeks.
    Last edited by britebyte; 03-29-2011 at 04:43 PM. Reason: Typo

  3. #3
    Join Date
    Mar 2010
    Posts
    159

    Default

    You can test regexes by messing with a Regex Evaluation step.

    I'm not sure what your ftp connection issue is, that part sounds strange. I would suggest checking or unchecking the Use active FTP connection checkbox, but I would have thought that the LIST command would also fail if an active/passive problem.

    Regards,
    Jeremy

  4. #4

    Default

    Jbeal, Thanks for taking a look.

    I figured this one out finally. It turns out my output folder "did not exist" because the of the way I was trying to use some of the usual regex characters in the output folder field. That seems to have caused it to freeze. If this is always the case even with a simple test case I should file a jira report.

  5. #5
    Join Date
    Mar 2011
    Posts
    4

    Default

    cpcc
    Thanks for posting your eperiences. I am getting similar results using SSH2 GET. The results say "Destination folder (move to) [] can not be found!" but I have browsed to the folder when entering it and I know it exists.
    What were the regex characters causing your problems and how did you overcome it?
    BernieF


    Quote Originally Posted by cpcc View Post
    Jbeal, Thanks for taking a look.

    I figured this one out finally. It turns out my output folder "did not exist" because the of the way I was trying to use some of the usual regex characters in the output folder field. That seems to have caused it to freeze. If this is always the case even with a simple test case I should file a jira report.

  6. #6

    Default

    Hi BernieF,

    I was using a backslash to escape special characters as I was still thinking the output folder field was a regular expression. I guess the output folder field interpolates variables but is not a regular expression so this was being interpreted as an extra folder level (Windows).

  7. #7
    Join Date
    Mar 2011
    Posts
    4

    Default

    Thanks for that.

    I am on the verge of giving up with SSH2 GET. It creates folders but can't put the files in it.

    I'm switching to SFTP which I have managed to get working

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.