Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: ETL question

  1. #1
    Join Date
    Aug 2008
    Posts
    4

    Default ETL question

    hi there,

    i hope somebody can halp me

    i have a folder that countaines 100000 small files, file names are "344124.rd.txt", the set of integers at the beginig of a file name is a bit of information that i need, inside the file there is a content that i need to import into table that would look like "int, text",int beeing a set of integers from file name.
    is there a way in Pentaho that i could do that without manulay adding every file name into file text step from pentaho?


    regards,
    Dado

  2. #2

    Default

    On the text file wildcard step specify something like .*\.rd.txt$ to pick up all .rd.txt text files and specify a folder and add that and it will pick up all text files in that folder.

    Tom
    This is a signature.... everyone gets it.

    Join the Unofficial Pentaho IRC channel on freenode.
    Server: chat.freenode.net Channel: ##pentaho

    Please try and make an effort and search the wiki and forums before posting!
    Checkout the Saiku, the future of Open Source Interactive OLAP(http://analytical-labs.com)

    http://mattlittle.files.wordpress.co...-bananaman.jpg

  3. #3
    Join Date
    Aug 2008
    Posts
    4

    Default

    hi bugg_tb,

    thanks for your replay, i tried it and it semas to work

    but the content of the file is just text and entire content needs to be loaded in on text fieldl in DB, when i try to do that cattle just hangs, i tell the node that there is no separator and it just hangs...do you maybe know how would i do that? tell the node to insert the entire text into DB?

    is there a way taht could insert the entire file name int a filed in DB?


    thanks a lot!!!!!

  4. #4
    Join Date
    Aug 2008
    Posts
    4

    Default

    Quote Originally Posted by dado112233 View Post
    hi bugg_tb,

    thanks for your replay, i tried it and it semas to work

    but the content of the file is just text and entire content needs to be loaded in on text fieldl in DB, when i try to do that cattle just hangs, i tell the node that there is no separator and it just hangs...do you maybe know how would i do that? tell the node to insert the entire text into DB?

    is there a way taht could insert the entire file name int a filed in DB?


    thanks a lot!!!!!

    i have found the option to insert the file name so thats sorted...but soon as i set the separator to nothing it just hangs and doesnt do anything....could somebody please help?

  5. #5
    Join Date
    Aug 2008
    Posts
    4

    Default

    Quote Originally Posted by dado112233 View Post
    i have found the option to insert the file name so thats sorted...but soon as i set the separator to nothing it just hangs and doesnt do anything....could somebody please help?

    have managed to setup some things but i get out of memory and it just crashes.....

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.