Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Adivice on MS Access performance

  1. #1
    Join Date
    Dec 2009
    Posts
    332

    Default Adivice on MS Access performance

    Please do not ask why, but we have a total of 5 million records in 6 MS Access databases that we pull once a week into Oracle. The simple transform uses an Access input and a table output. The transform takes about an hour (Windows XP Spoon UI to linux server over local 1GPS network connection).

    My questions regarding this are:

    1. Is an hour about as fast as I should hope for?
    2. To avoid the network issues, can linux open an MS Access database via Pentaho PDI?
    3. The Core2Duo windows XP client is running at 100% of a single CPU total. Can Pentaho use both CPUs for this process? (RAM, network and IO are all underutilized as well.)
    4. Are there any tricks to increase the performance of the MS Access database?

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You can use the "Access Input" steps on any platform, including Linux.
    I'm assuming that parsing such a database uses a lot of CPU power. Kettle will run happily on multiple CPUs but reading a single database in parallel is not possible. Reading multiple files in parallel should be easy to do however.

  3. #3
    Join Date
    Dec 2009
    Posts
    332

    Default Thank you!

    The script went from 1500 rps to 5100 rps by moving the MS Access files onto the linux server and processing them there. Thanks a ton.

    Gotta love this product

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.