Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Infobright Loading

  1. #1
    Join Date
    Aug 2008
    Posts
    18

    Default Infobright Loading

    I have created a transformation which contains 15 unique steps which simply takes data from 15 different tables (using table Input) and loads this into infobright.

    I attach the ktr for further clarifications.

    The problem i am facing is that the transformation run fine loading all the data in each of the 15 tables in Infobright however, its leaving some connections open which means the Transformation is not stopping and its in a run mode.

    Any ways to come around this problem will be highly appreciated.
    Last edited by dutyfree; 12-24-2009 at 05:28 AM.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I have no idea what you mean. I'm sure the database connections get closed.
    Perhaps this is an ICE/IEE limitation?

  3. #3

    Default

    Hi,

    I am the developer of the Infobright output step.

    Can you confirm that all tables for output are unique? You cannot load the same table in parallel at the same time.

    Please do the following test:
    1) Run the transformation until the end. It "hangs"
    2) Log on to the mysql> prompt in Infobright
    3) do "show full processlist" command

    Post the processlist output here and I'll take a look.

    Thanks
    Geoffrey

  4. #4

    Default

    Hi,

    Are there any errors in your Kettle log file? I did a test and found that it is very easy to run out of memory when launching multiple loads in the same transformation.

    Each Infobright loader launches as a separate process, and consumes the amount of memory specified in the LoaderMainHeapSize parameter in your brighthouse.ini file. This can quickly add up.

    I recommend splitting out your loads into 3 separate transformations, that load 5 tables each, and run them sequentially in a Kettle job.

    Also, do not run more loads in parallel than the number of CPU cores on your server.

    Also, make sure you are using the latest release of Kettle, which contains the Infobright core library version 2.4. (under libext/infobright-core-2.4.jar).

    Good luck,
    Geoffrey

  5. #5
    Join Date
    Jul 2011
    Posts
    4

    Default

    where is your ktr attach

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.