Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: CSV to MySQL Bulk Loading

  1. #1
    Join Date
    Mar 2014
    Posts
    1

    Default CSV to MySQL Bulk Loading

    Hi,

    I am trying to Load CSV files in to MYSQL using MYSQL Bulk Loader using Kettle, for 5 Million records it is almost taking 20 minutes to load. Is there is anyway I can reduce the timings to load, right now I have "Change Number of Copies to Start" as 1 . I am using Text File Input and MYSQL Bulk Loader. I also followed the tips in http://julienhofstede.blogspot.nl/20...owssecond.html but sill no improvement. Can anyone of you please give some suggestions as how to improve the performance.

  2. #2

    Default

    There is a csvjdbc driver I read about recently. Or you could try to use the MySQL load Data infile Syntax but of course that is Used Outsider kettle. You could Split your file in multiple logical Parts and load them in parallel.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.