Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Use Batch update for inserts, DB2 is it really doing Batch updates

  1. #1

    Default Use Batch update for inserts, DB2 is it really doing Batch updates

    Hi,

    I am trying to copy Table from one Database of the Operational system to the Staging area DB2 Database.

    I have a simple Kettle transformation, “Table Import” to a “Table Output”

    Table Output-> Nr of copies: 10
    Use Batch update for inserts: Checked

    I am connected with the JDBC driver to the DB2 output database, if I checked the Feature List I see the Feature 46, “supports batch updates” is set to N(No).

    Will it still be using “Use Batch update for inserts” even thou the Feature List is checked at no, or how can I change the Feature List property?

    The current performance is at 6760 lines/s, looking for ways to increase it.

    Thanks for the help,
    Thorsten

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    In the code there is a comment on WHY it was disabled:

    Code:
    // DB2 support batches but big decimals and binary data is
    // broken, so for the moment batches are not done in DB2.
    return false;
    I can't remember doing this myself but in any case my guess is we should make this an option in the Advanced section as well so that you can make the call yourself.

  3. #3

    Default

    Hi Matt Casters,

    Thanks for pointing that out to me.
    I enabled it and compiled PDI, with it enabled. Fully aware that it will not work for big decimals and binary data. Just to see the performance gain, I am now getting 8162 lines/s a 20% improvement over the 6760 lines/s without BatchUpdate Enabled.

    Will see if I can optimise it further, as it is still not quite the result we where hoping for.

    Thanks again,
    Thorsten

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    It's come up here before that DB2 can be quite slow. However, it should be possible to create a bulk loader for DB2 to speed things up for straight data loads. Especially if you can do it in a streaming fashion, either over named pipes or stdio. Since I'm no DB2 expert, I'll take a backseat on either problem :-)

    Cheers,
    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.