Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Processing flat files

  1. #1
    Join Date
    Mar 2009
    Posts
    29

    Default Processing Unstructured Data (Flat Files)

    Hi Experts,

    Looking for a requirement to read unstructured data basically from flat files and process that data in a format (csv format) so that it gets loaded in server database.

    Also wondering other option, thru kettle can we get these remote files copied to a local server and then perform processing?

    Is it possible to remotely execute such requirement, I mean ETL job is running on a server and it connects to a remote server where flat files are stored.

    Please let me know your comments and also correct my understanding.

    Regards,
    Neuron
    Last edited by neuron; 11-16-2011 at 07:16 AM.

  2. #2
    Join Date
    Feb 2008
    Posts
    107

    Default

    Yes, you can do this. Kettle uses VFS (http://commons.apache.org/vfs/) which you can use to directly refer to files on remote servers.

    Also look at the "File Transfer" job steps.

  3. #3
    Join Date
    Mar 2009
    Posts
    29

    Default

    Thanks Buachaille!

    Regarding kettle using VFS, is there any sample to refer files on remote servers? That will definitely help.

    Regards,
    Neuron

  4. #4
    Join Date
    Mar 2009
    Posts
    29

    Default

    Any Pointers ??

    Regards,
    Neuron

  5. #5
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Just pick the protocol, for example: http://remoteserver/some/folder/file.txt

    or: sftp://remoteserver/some/folder/file.txt

    etc.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.