Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: writing to Sybase IQ tables

  1. #1
    Join Date
    Oct 2006
    Posts
    24

    Default writing to Sybase IQ tables

    Does anybody have any experience with inserting into Sybase IQ tables.

    I'm seeing horrible data transfer speeds compared to running the same transformations in oracle. I'm of the opinion at this point that it's either some sort
    of Sybase setting, or the JDBC driver is trash....

    Any opinions or ideas...I'm at a loss....

  2. #2
    Join Date
    Oct 2006
    Posts
    24

    Default jdbc driver error

    More of an update:

    I have connection success using a generic jdbc driver, but when using the sybase
    one...my test connection works, but when i try to explore the db i get this error

    -------error

    java.lang.reflect.InvocationTargetException: Problem encountered getting information from the database: be.ibridge.kettle.core.exception.KettleDatabaseException:
    Unable to retrieve database information because of an error
    Invalid state, the Statement object is closed.

    at be.ibridge.kettle.core.dialog.GetDatabaseInfoProgressDialog$1.run(GetDatabaseInfoProgressDialog.java:71)
    at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:113)
    Caused by: be.ibridge.kettle.core.exception.KettleDatabaseException:
    Unable to retrieve database information because of an error
    Invalid state, the Statement object is closed.

    at be.ibridge.kettle.core.database.DatabaseMetaInformation.getData(DatabaseMetaInformation.java:270)
    at be.ibridge.kettle.core.dialog.GetDatabaseInfoProgressDialog$1.run(GetDatabaseInfoProgressDialog.java:67)
    ... 1 more
    Caused by: java.sql.SQLException: Invalid state, the Statement object is closed.
    at net.sourceforge.jtds.jdbc.JtdsStatement.checkOpen(JtdsStatement.java:219)
    at net.sourceforge.jtds.jdbc.JtdsStatement.getConnection(JtdsStatement.java:1161)
    at net.sourceforge.jtds.jdbc.JtdsResultSet.getConnection(JtdsResultSet.java:421)
    at net.sourceforge.jtds.jdbc.JtdsResultSet.close(JtdsResultSet.java:482)
    at be.ibridge.kettle.core.database.DatabaseMetaInformation.getData(DatabaseMetaInformation.java:239)
    ... 2 more

  3. #3
    Join Date
    May 2006
    Posts
    4,882

    Default

    Well... Sybase JDBC driver/server is kind of an odd duck.... a very odd one. If the database code was not specifically written for Sybase (or keeping Sybase in mind) you get into all kinds of problems with it (mostly transaction based). So using a generic driver to work with Sybase is also not an option.

    For your speed problems, some more hints:
    - replace the table output by a dummy step (or a file output step) and you will see the maximum output speed you will be able to achieve in your current job.
    - Is your target database co-located with your execution. If not, the IO can cause a lot of slowdown.

    Regards,
    Sven

  4. #4
    Join Date
    Dec 2006
    Posts
    2

    Default

    Due to the architecture of Sybase IQ, row based transactions are very slow and inefficient. You need to perform bulk operations either a bulk load for inserts or set based update for updates.

    You will not get the same performance for transaction based operations as you would in Oracle.

    regards,
    Peter

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.