Insert / Update Not Committing (despite saying it did)

[meconlin]

My Setup
--------------------------
PDI version 4.1.0
Linux
MySQL 5.1
Java 1.6

My Issue
------------------------------------------------------------------

Short Desc:
Insert / Update finished successfully, but no data written?

Long Desc:
I have a Kettle job that does an Insert/Update.
The job runs, shows data in the stream (I logged to check), and logs "All transactions of database connection... where committed at the end of the transformation"

HOWEVER

The MySQL binlog show no inserts or updates.
The next job in my chain can not find any of those items inserted above.

This problem is occasional, and only occurs a few times a day or even every other day.

Any insights would be appreciated.

[codek]

uncheck "use batch updates" and try again. if that fixes it then upgrade to pdi 4.3.0 where it's fixed.
(although, the bug i'm thinking of only applied to the "update" step, not the "insert/update" step.. )

[meconlin]

Will do and report back soon.

In your experience have you seen this bug be this inconsistent (failing 2-5 times a day out of 48 runs) ?
If it failed everytime I would be much happier.

[meconlin]

We have rev'd up to PDI 4.2 and the problem still occurs.
The "Insert / Update " block does not have a "use batch updates" setting, so that as a fix was not available to me.

I am running ~100 tranformations and most of them depend on insert/update, can this block be trusted?

[gutlez]

I remember an odd-case bug, which might be the one codek is mentioning.

Try setting the commit size to 1, and run one transformation (that has been failing).
It will run *VERY* slowly.

If this has almost all the rows, then it is the bug that I am thinking of, and you need to upgrade to a very recent (eg. 4.3.0 ) version of PDI.

[meconlin]

Yes, I am on 4.3.... that 4.2 was a typo.
I will try commit = 1

[MattCasters]

Are the transformations in question running "database transactional"?
I've recently fixed a race-condition that might apply if it's the case.

[meconlin]

It is in fact marked database transactional.
Would unchecking this prevent the bug?

If I understand that setting correctly, I can turn it off in this job with no real effect because I have only on setp that writes/updates to the db.

[MattCasters]

yes, it would.

[meconlin]

SOLVED

Unchecking "make this transaction database transactional" has worked well. All commits appear to be taking now.
Has this bug been fixed in an available release?

I am doing nothing but linear inserts with this job, so a failure with a non rollback doesn't worry me here, however in other situations this could be a big problem.

Anyway, Thanks for all the help!