Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: Latency connecting to MySQL db

  1. #1

    Default Latency connecting to MySQL db

    I have quite a strange problem.

    When I run spoon from my Windoze desktop, there is roughly 5-10 seconds of latency each time it opens a connection to MySQL. So when running a transformation that reads from, say, three DBs and writes to one, we have about 20-40 seconds of sit-there-and-hang. Likewise, any action that involves talking to the db (select fields, merely opening a Filter step, etc) sees several seconds of latency. Once the connection is open, the transfer speed is normal. No relevant messages ever appear in the log.

    I do not believe this is a problem with the MySQL instance or the network, because the command line MySQL client (running atop Cygwin) does not exhibit any unusual latency. Nor does Spoon exhibit latency when it is run remotely from the same Solaris box that hosts the MySQL instance.

    Any ideas?

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    It's probably a problem with the network trying to do a reverse name services lookup and timing out (or something).
    The command line client doesn't go over the network AFAICT.

    Try it by changing the hostname of your connection to localhost or even better 127.0.0.1
    If that's not it, I wouldn't know what it could be. Never had the issue myself on MySQL. It's a very fast connecting, a LOT faster than most other databases.

  3. #3

    Default

    The command line client is capable of connecting to a local db via unix socket; but in this case, it is connecting over TCP/IP to a remote db, just as Kettle is doing. I tried disabling the MySQL server's hostname lookup feature, but that did not help. Also tried telling Kettle my database's IP address rather than hostname; no luck there, either.

    So I am still left with the guess that this has something to do with the difference in how the MySQL client connects to the DB versus how Kettle connects. What version of the JDBC MySQL driver is Kettle using?

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    For the moment the one PDI is using is MySql Connector 3.1.14 (in 2.5.0-RC1), but you can change it yourself in the libext/JDBC directory by removing the one there and replacing it by the mysql driver of your choice. YMMV, at least 1 newer version of the driver was tried and rolled back because of too many problems.

    Regards,
    Sven

  5. #5

    Default

    Using PDI 2.4.0, I swapped out Connector 3.1.14 for Connector 5.0.5. The new driver did not cause any problems, but it also did not have any noticeable effect on the latency issue.
    Last edited by jmcvetta; 04-19-2007 at 10:49 AM.

  6. #6
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Trust me, 5.0.5 will give you problems.

  7. #7

    Default

    I'll take your word for it, Matt -- I only tested it by running two fairly simple transformations.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.