Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Mongo input help - join mongo collections

  1. #1
    Join Date
    Aug 2011
    Posts
    236

    Default Mongo input help - join mongo collections

    Hi,

    I am pretty new to the mongo input step in PDI. I have the following mongo query :-

    db.locations.find().forEach(function(l) {
    for (var i=0; i<l.sourceUrls.length; i++) {
    if (!l.sourceUrls[i].isCompetitor) {
    var c = db.collectors.findOne({url: l.sourceUrls[i].url});
    if (c) {
    if (c.numFailures > 1) {
    var csv = l.code + ',' + l.name + ',' + l.sourceUrls[i].url + ',' + c.numFailures;
    printjson(csv);
    }
    }
    }
    }
    });

    but I don't see how I can get this to work in the mongodb input step. It have multiple joins to collections.

    I have tried various versions of quoting but I find the documentation on what to quote lacking.


    The output fields (l.code + ',' + l.name + ',' + l.sourceUrls[i].url + ',' + c.numFailures) should go into the 'field expression' - is this correct?

    It looks like the current Mongo input component cannot handle arrays - is this correct?

    Any guidance would really help me out.

    Thanks,
    Last edited by tnewman; 02-26-2013 at 05:39 PM.
    PDI 8.0.0
    MySQL - 5.6.27
    Redshift - 1.0.1485
    PostgreSQL 8.0.2
    OS - Ubuntu 10.04.2

  2. #2
    Join Date
    Aug 2006
    Posts
    1,741

    Default

    Hi,

    The MongoDB input step does not handle joins between collections. Furthermore, it looks like the Java driver for MongoDB does not support the forEach() cursor method:

    http://api.mongodb.org/java/2.10.1/c.../DBCursor.html

    Nor is forEach possible via an operator to find().

    The version of MongoDB input that will be available in PDI 5.0 has support for extracting from arrays and unwinding arrays. It allows you to specify field paths in much the same way as you can in the JSON step. It also has support for executing aggregation pipeline queries. Neither of these are going to help you with joins though. However, the new array unwinding plus PDI's join steps might accomplish what you want.

    Cheers,
    Mark.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.