Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Performance Issue

  1. #1
    Join Date
    Feb 2007
    Posts
    13

    Default Performance Issue

    Hi,
    I am making transformation from multiple source table to one destination table where data is about 20,00,000 rows. For source table I have one table having near about 20,00,000 rows and other source table having 2 to 3K rows. By using joins I prepared one query for transformation to one destination table. I am using MySQL source and destination database. Source machine having 4 GB RAM with MySQL database and destination machine having 2GB RAM with MySQL database.
    This transformation is giving me performance problem. My transformation was running near about 15 hrs, still single record also not transferred. Finally I tranfer only 500 records on separate table and try with that. But still same problem.
    Is this the problem of MysQL? If I use MsSQL server as source database and MySQL as destination, will this problem resolve?

    Thanks in advance

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Already answered by private mail:

    This looks like a MySQL problem. You are doing a complex (or incorrect) join probably. The LIMIT clause is not having any effect in that case.
    This effect is happening not just on MySQL but on all databases when you for example forget a join key or when indexes are missing.

    HTH,

    Matt


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.