Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Potential causes of all jobs (10+ unrelated) -- all suddenly running much slower?

  1. #1
    Join Date
    Sep 2014
    Posts
    175

    Default Potential causes of all jobs (10+ unrelated) -- all suddenly running much slower?

    Hi there-

    I've had various PDI scripts running since early 2015 -- they are unrelated except they are all run by a master job in PDI (version 5.4.0.1), on the same remote computer, and to the same SQL Server data warehouse.

    For the last two nights, they've all been running about 20-30x slower.

    What are some potential tests I can do to determine what is causing this across the board?

    Moreover, at least for now (around noon as opposed to midnight when these scripts typically run, along with other processes) -- it seems instead of taking about 30x longer (whoof) --- they are only now taking 2x as long, making it harder to find the source.


    I can test the SQL database but I have a hunch that isn't it.

    What else can be going on here? The remote computer is shared - but I find it highly unlikely a change was made after Friday night (it ran just fine then).

    Maybe it was connectivity issues (database is on a network with PDI ETL machine) -- but yeah, I'm not sure. Maybe I need more granularity in performance of each step.

  2. #2
    Join Date
    Aug 2011
    Posts
    360

    Default

    We had same problem with DI server 5.4.0.1, everything works fine for one or two days
    than everythings gets really slow, even a generate row with a dummy step.
    I suspect some ressource not being released that slow all down.
    After weeks of analysis we couldn't find a solution, so we decided to restart the tomcat server everyday at 20h,
    before the nightly batchs.
    This is a poor work arround, but everything's ok since then.
    Now migrating to version 7.0 so i hope it will work better!

  3. #3
    Join Date
    Sep 2014
    Posts
    175

    Default

    For future reference, it wasn't a PDI issue. The computer running the PDI scripts is a shared one with another team using their own ETL scripts, and the CPU was running at a constant 99% ... due to unrelated tasks.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.