Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Load files from folder to database using ETL

  1. #1
    Join Date
    Aug 2013
    Posts
    139

    Default Load files from folder to database using ETL

    Hello,

    I am New to PDI i have to read the files present in a folder and insert data to mysql table and move those files to different location this has to run every hour or when ever files present i that folder, any one help me to build this transformation or job with you suggestion to do it in a proper way

    PDI 4.4, Files .csv DB:Mysql OS:windows
    Thank U For Your Time
    Suresh

  2. #2
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    You can use the following steps in a transformation:
    Text File input - read text file
    Select Values - rename fields, change format if needed
    table output - upload data to a table

    Then you can wrap this transformation into a job and use:
    move files

    To run every hour use your os scheduler with a batch file
    -- Mick --

  3. #3
    Join Date
    Aug 2013
    Posts
    139

    Default

    Thanks Mick,

    I was done with two parts could u plz elaborate on scheduling, like i have to publish this to puc or how i make this job automatized say run every hour ..

    if we double click on start it shows some scheduling option as well what does those mean...
    Thank U For Your Time
    Suresh

  4. #4
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Try Pan Documentation or Kitchen Documentation under the heading scheduling
    **THIS IS A SIGNATURE - IT GETS POSTED ON (ALMOST) EVERY POST**
    I'm no expert.
    Take my comments at your own risk.

    PDI user since PDI 3.1
    PDI on Windows 7 & Linux

    Please keep in mind (and this may not apply to this thread):
    No forum member is going to do your work for you. We will help you sort out how to do a specific part of the work, as best we can, in the timelines that our work will allow us.
    Signature Updated: 2014-06-30

  5. #5
    Join Date
    Aug 2013
    Posts
    139

    Default

    Thanks gultez,

    I have written a bat file and scheduled it using task scheduler, i was just curious to know which scheduler is better in performance and monitoring

    task scheduler or use quartz scheduler from pentaho user console using Xaction for .ktr,.kjb files
    Thank U For Your Time
    Suresh

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Unfortunately, I don't have that kind of information for Quartz / Pentaho User Console, since my PDI runs using cron on a system that doesn't have BI Server installed.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.