Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Carte service is down

  1. #1
    Join Date
    Sep 2017
    Posts
    13

    Exclamation Carte service is down

    Hi,
    I have a few questions regards to Pentaho configurations:
    1. I see there are -Xms, -Xmx and -MaxPermSize in Spoon.bat/Spoon.sh. I understand that Xms and Xmx is minimum and maximum memory allocation to Spoon. But I am not sure what MaxPermSize is and what it doing. We have set the xms and MaxPermSize to 8GB and xmx to 32GB. Does this help to boost Spoon's performance? We have 256GB in the server and we split to 4 Spoon services, allocate 32GB each.


    2. Carte service is down when Pentaho process large files (10-12million records text files) without any indication and we would need to restart the carte service. How can we improve this? Is Carte responsible to invoke Spoon to run transformations?


    3. How can we store the details of the log file into database?


    4. The service crashed after processed a large file and not able to process the next file, and we needed to restart the services again. I believe that is because the memory is not fully being release after processed the large file. How can we flush/release the machine memory if the memory has hit the maximum after processed the large file?


    5. How can we improve the Spoon performance if the file is too large (10-12million records text files). The duration taken now is around 4 hours. How can we shorten by 40-50% of the duration?


    Pentaho exact version: 4.0.1


    Appreciate your help on my queries.
    Best Regards,
    Kar Ho

  2. #2
    Join Date
    Nov 2009
    Posts
    688

    Default

    1. The MaxPermSize is the maximum size for the permanent generation heap, a heap that holds the byte code of classes and is kept separated from the object heap containing the actual instances.

    3. In the transformation/job properties you can point the logging to a database table

    5. To improve performance: There can be a lot of reasons why it is so slow. Try to determine the step which causing that and try to fix that. You also work with quite an old version. Newer versions can also speed things up



  3. #3
    Join Date
    Sep 2017
    Posts
    13

    Default

    Quote Originally Posted by johanhammink View Post
    1. The MaxPermSize is the maximum size for the permanent generation heap, a heap that holds the byte code of classes and is kept separated from the object heap containing the actual instances.

    3. In the transformation/job properties you can point the logging to a database table

    5. To improve performance: There can be a lot of reasons why it is so slow. Try to determine the step which causing that and try to fix that. You also work with quite an old version. Newer versions can also speed things up


    Hi Johan,

    Thanks for the reply.

    For (1), does it means that we set the MaxPermSize the lower the better? I have set minimum to 8GB, maximum to 32GB and MaxPermSize to 8GB. Looks like I need to adjust it.

    For (3), I can push all the logs to a db table now. Thank you.
    For (5), I will try to work on it. Appreciate on the explanation.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.