Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: Kitchen performance local vs. remote

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1
    Join Date
    Jun 2014
    Posts
    27

    Default Kitchen performance local vs. remote

    Hello all, I am running a job the reads a .txt file with 1.5 million records, performs several transformations, and writes records to various tables in PostgreSQL. My local machine has 8GB of memory and kitchen.bat contains PENTAHO_DI_JAVA_OPTIONS=-Xmx16384m. The job runs in 54 minutes. When I run the job remotely, on a Linux server with 16GB of memory, using kitchen.sh with JAVAMAXMEM="16384", the job runs in 4 hours and 42 minutes. All other things being equal, like network speed and access, I would expect the remote server to process the job in at most, the same amount of time as my local machine. What other configuration values can I check to determine where the bottleneck lies?

    Thank you in advance for your response.

  2. #2
    Join Date
    Jul 2009
    Posts
    476

    Default

    I wouldn't set JAVAMAXMEM to the entire amount of memory on the machine, because other OS processes need memory, too, and you could be swapping a lot of pages between memory and disk, which will slow you way down.

    I can't tell from your post if your Postgres database is on the same or different box as PDI. If it's on the same box in dev (your PC) and on a different box in prod (the Linux server), then I would expect it to run more slowly in prod, particularly if you have steps that require round trips between the PDI server and the database server. You can mitigate the network latency by using caching in your lookup steps, inserting rows with the PostgreSQL Bulk Loader, and doing other things to minimize network round trips.

  3. #3
    Join Date
    Jun 2014
    Posts
    27

    Default

    Thanks for the response, Rob. I have tried changing the memory setting to various levels and 16GB offers the best response time. I am connecting to the same Postgres server from both environments (it is remote to both servers, but in the same domain as both), I am using the PostgreSQL Bulk Loader. Since the settings in the transforms are the same in both places, I would expect the processing time to be roughly the same. What other differences could exist which would account for a more than 4 times processing speed?

  4. #4
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    "call the job remotely"
    Care to expand on that a little?

    Are you copying the KJB/KTR to the remote linux box and running it via SSH and command line?
    Are you using carte and telling spoon to run it on the other machine?

    Why are you setting the memory in two different ways? Wouldn't it make more sense to set it the same way in both places?

  5. #5
    Join Date
    Jun 2014
    Posts
    27

    Default

    Yes, I am copying the job, transform, and data files from my local to the remote machine so they are all run as close as possible to the environment. I am running from the command line using kitchen. I have configured the kitchen.bat file on my local Windows machine and I am using the command kitchen.bat /file:"ETL.kjb" and the kitchen.sh file on the remote Linux machine using the command sh kitchen.sh -file="ETL.kjb"
    Last edited by tjumps; 10-06-2014 at 03:02 PM.

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    So, if your transform has the file paths directly written in them (can't say for sure without the KJB/KTR), then it's possible that your server is trying to pull the data from your workstation. That would easily cause the 4x issue that you're seeing.

  7. #7
    Join Date
    Jun 2014
    Posts
    27

    Default

    I don't think that is the case because I am specifying the path to the file on the current machine. I didn't include that path for simplicity's sake, sorry for the confusion!

  8. #8
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Generally, I've found Linux ETL to be faster, but again, without a complete picture, we're all shooting in the dark here.

  9. #9
    Join Date
    Jul 2009
    Posts
    476

    Default

    Here are a few troubleshooting ideas:

    * Ping the database server from both your PC and the Linux ETL server, and compare the reported round-trip times. Even if everything is in the same domain, there could be network hops and bottlenecks in between systems.
    * Look at "top" and/or "sar" on your Linux ETL server while it is running. Are you maxed out on CPU, or maybe I/O? Server hardware isn't always as fast as desktop hardware, particularly when the desktop is newer than the server.
    * Review the PDI job log file to see if one particular transformation is responsible for most of the job's slowdown, and focus on what that transformation is doing.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.