PDA

View Full Version : MongoDB output Bug



bilal.berjawi
06-07-2012, 05:21 AM
Hi,
I've an CSV file and i want to ETL its content to an Mongodb database.
In the attached file you can find the Transformation(.ktr), CSV file, and the mongodb database
the transformation consist to read from the CSV file, transform data using row denormalizer and update documents in Mongodb database.
if the update is set only to upsert all will work fine,
but if I use the modifier update, only one value will be update for some records (and always its the value in the last column).
try to copy the database 'PPR' to your mongodb database repository and run the mongod server
then open then transformation (be sure of all configurations) and run it, then go to the mongodb collection called 'dep' in the 'PPR' db
check the value in the 'PPRt' embedded document where id_dep='23Dep' or id_dep='01Dep'
compare checked value to the CSV file. you find that only one value is loaded into mongodb (the last value),but in reality there's two values for each dep
I've tried to put another type of data, always got same problem, even if the source is a table input,even I eliminate the row denormalizer.

Note that I'm working on windows 7 32bit, using mongodb v2.1.1

8850

Mark
06-10-2012, 04:01 AM
Hi,

Thanks for the bug report. I'll look into it this week.

Cheers,
Mark.

troostw
06-13-2012, 11:57 AM
I experienced the same bug, on 64bit Ubuntu.
As a workaround, I tried to chain two MongoDBOutput step together, though that did not work. It appears that MongoDbOutput does not write the rows to the next step.

Mark
06-13-2012, 04:38 PM
I have a fix for this that should get pulled into the master branch for our big data plugin today.

http://jira.pentaho.com/browse/PDI-7982

I'll post again when the fix is available and you will be able to get it from our CI server.

Cheers,
Mark.

jganoff
06-15-2012, 01:52 PM
Try the latest build here. The 1.3 version of the Pentaho Big Data Plugin is compatible with PDI 4.3.0+.

pentaho-big-data-plugin-1.3-SNAPSHOT.zip (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.zip)
pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz)

It builds continuously here: http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/
Code resides in Git here: https://github.com/pentaho/big-data-plugin

bilal.berjawi
06-21-2012, 04:02 AM
Try the latest build here. The 1.3 version of the Pentaho Big Data Plugin is compatible with PDI 4.3.0+.

pentaho-big-data-plugin-1.3-SNAPSHOT.zip (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.zip)
pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz (http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/lastSuccessfulBuild/artifact/dist/pentaho-big-data-plugin-1.3-SNAPSHOT.tar.gz)

It builds continuously here: http://ci.pentaho.com/job/BRANCH_pentaho-big-data-plugin/
Code resides in Git here: https://github.com/pentaho/big-data-plugin


Thanks, this work perfectly.
Problem resolved.