Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: New Mongo Input component question - Mongo in general

  1. #1
    Join Date
    Aug 2011
    Posts
    236

    Unhappy New Mongo Input component question - Mongo in general

    Hi,

    I was taking a look at the new mongo input (PDI 5.0.1) and from what I had read it was supposed to make it easier to get a list of fields within the Mongo record.

    I only seem to be able to output a single JSON field. This does not really solve anything for me because I either have to type in the field list in the JSON input or run it through a script.

    I can connect to my Mongo source just fine and I have a query expression :-

    { $query : { $or : [{ rating.addedTimestamp : { $gte : 1385253746 }},{ rating.updatedTimestamp : { $gte : 1385253746 }} ]}}

    Am I missing something? The documentation suggests it should be able to read the schema.

    Are my expectations too high?

    There just seems to be no tool that makes reporting from Mongo easy - not being able do joins is a big issue for me. If someone knows of a good tutorial - please let me know. The pentaho example are too simplistic and report aggregate values. We still have great need to report details.

    Thanks,
    PDI 8.0.0
    MySQL - 5.6.27
    Redshift - 1.0.1485
    PostgreSQL 8.0.2
    OS - Ubuntu 10.04.2

  2. #2
    Join Date
    Oct 2013
    Posts
    28

    Default

    Here's what I'm doing until I can get a better grip on Mongo; it's a kludge but it solves my immediate needs for the time being.

    • Set up and expose an API to the data via Node.js (export orthogonal data structures to simplify the Kettle transforms)
    • Daily job:
      • Consume the API with Kettle using (primarily) the HTTP Client, JSON Input and Modified Java Script Value steps; transform the data into normalized structures.
      • Empty then store the normalized data in SQL tables

    • Build report and/or dashboard queries against the SQL data instead of Mongo


    More information / clarity would definitely be welcome.
    Last edited by psema4; 11-28-2013 at 11:08 AM. Reason: fix list

  3. #3
    Join Date
    Aug 2011
    Posts
    236

    Default

    Hi,

    Thanks for the reply. I basically extract/load Mongo to MySQL too, but this is not scale-able. I was hoping to use the new functionality to make it easier to make my reports easier to produce. I do not have direct input into the structure of the mongo collections which is why I need to do joins. It's easy if the data you need is all in one collection.

    It just seems Mongo is great for developers but not so great for reporting from.

    Would be interested to find out other folks thoughts on this subject - Am I missing the point?

    Thanks,
    PDI 8.0.0
    MySQL - 5.6.27
    Redshift - 1.0.1485
    PostgreSQL 8.0.2
    OS - Ubuntu 10.04.2

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.