Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: Memory usage in Report Designer (report preview)

  1. #1
    Join Date
    Jun 2007
    Posts
    260

    Default Memory usage in Report Designer (report preview)

    I tried to run a preview of a report based on a query that joined about 8 tables and returned about 18000 rows.

    Each time the Report Designer chocked on heap space errors and I was forced to destroy the process.

    Deployed and running on the platform yielded similar results, despite me fiddling with the memory configuration.

    However, I ran the exact query in Kettle and it executed and wrote the results to an Excel spreadsheet in a few minutes.

    Why is it that the Report Designer
    (and probably the reporting engine) will not run the query to completion but a tool like Kettle will,

    Regards
    Pradesh

  2. #2

    Default

    Thomas has commented on this before: http://forums.pentaho.org/showthread.php?t=56586

    If you don't need layouting and pagination, then the reporting engine is doing a ton of work that you're not utilizing. Kettle is much better suited for straight data transfers as it doesn't have to build the entire model in memory and output it as a single unit like the reporting engine has to for some output formats. Using the reporting engine as an ETL tool is a bit like using a screwdriver as a hammer... it'll get you there, but your performance might be subpar.

  3. #3
    Join Date
    Jun 2007
    Posts
    260

    Default

    Thank for your input.

    On a point of clarification, I'm not using the reporting engine as an ETL tool. Actually I ended up using an ETL tool to produce a report.

    Nice link, and I see from researching quickly that my database (MySQL) is supported, so it's a step in the right direction.

    So it seems like the entire resultset is being held in memory until the query finishes. If I reduce the number of columns SELECTed in the query my problems disappear at some stage. This may be compounded by the fact that there are some text field types in the result set. Perhaps the result set is not trimming the data (Just guessing here).

  4. #4
    Join Date
    Mar 2003
    Posts
    8,085

    Default

    The Report-Designer 1.7 itself holds all data in memory, as suspected. On the platform, you can turn on "live-resultsets" to use the "scrollable resultset" feature of your database to keep the memory usage low. With scrollable resultsets, the database only provides a small window to the results and pages the data in and out as we traverse over the data.

    Rd-2.0 enables this mode automatically, so in future versions you wont have to kill the process.
    Get the latest news and tips and tricks for Pentaho Reporting at the Pentaho Reporting Blog.

  5. #5
    Join Date
    Jun 2007
    Posts
    260

    Default

    Thanks for the input Thomas, I'll try the scrollable resultset.

    Good news as well. PRD 2 will be worth looking out for .

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.