US and Worldwide: +1 (866) 660-7555
Results 1 to 5 of 5

Thread: Repository Connection Performance

  1. #1
    Join Date
    May 2007
    Posts
    3

    Default Repository Connection Performance

    Run CE 4.3.0 on Windoze with a repository on SQL Server 2008.

    Opening and saving transformations local on the PDI server are very fast, however, remotely over a VPN connection takes 30+ seconds per transformation. It is a cable connection and is generally quick for other uses. In past cases when I've had a DI Server the saving performed reasonably well so I'm guessing the issue is latency as PDI makes a lot of DB transactions when opening/saving the transformations.

    My plan B was to go with a File Based repository, but I haven't seen a work around for PDI-4958.

    My questions:
    1) Are there settings like connection pooling I should use to improve performance with the repository? Recommendations?
    2) Is there a way to address the Slave Server Exists error in setting up a file based repo?
    3) Are others not seeing slow repo performance?

    Any feedback would be appreciated.

    Thanks,
    Todd

  2. #2

    Question

    I'm having the same performance issues with a mysql repository over a vpn.

    I also tried using a file based repo, and hit the same bug you linked too.

    I'm at a loss as to what to try next.

  3. #3
    Join Date
    Nov 1999
    Posts
    9,535

    Default

    If you're on 4.3.0, I'm afraid there's nothing you can do about it as it's already heavily optimized for the DB repository.
    Perhaps it would have been better to use a version control system with a local file repository.
    Matt Casters, Chief Data Integration
    Pentaho, Open Source Business Intelligence
    http://www.pentaho.org -- mcasters@pentaho.org

    Author of the book Pentaho Kettle Solutions by Wiley. Also available as e-Book and on the Kindle reading applications (iPhone, iPad, Android, Kindle devices, ...)

    Join us on IRC server Freenode.net, channel ##pentaho

  4. #4

    Default

    Thanks for the idea. Unfortunately I started with the local file repository, and hit PDI-4959 :-(

  5. #5
    Join Date
    Nov 1999
    Posts
    9,535

    Default

    Mind you, it doesn't really count as a local repository if you're putting the data on a slow network folder.
    Another think I can't understand at all was one person that created a file repository on all possible branches and tags of a subversion tree.
    With hundreds of tags and branches and a hundred transformations and jobs all of a sudden you're looking at tens of thousands of entries.

    I don't mind anyone doing these things as long as they don't come complaining about "bad performance" :-)
    Matt Casters, Chief Data Integration
    Pentaho, Open Source Business Intelligence
    http://www.pentaho.org -- mcasters@pentaho.org

    Author of the book Pentaho Kettle Solutions by Wiley. Also available as e-Book and on the Kindle reading applications (iPhone, iPad, Android, Kindle devices, ...)

    Join us on IRC server Freenode.net, channel ##pentaho

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •