US and Worldwide: +1 (866) 660-7555
Results 1 to 4 of 4

Thread: MongoDB as the main source to do ETL

  1. #1
    Join Date
    Mar 2017
    Posts
    10

    Question MongoDB as the main source to do ETL

    Hi All, I am pretty new to Pentaho and I just decided to use Pentaho as a tool to help me build the ETL solutions.

    Now, I am facing couple of problems:
    1. How do we lookup/join collections of mongoDB in pentaho.?
    2. How do we validate an empty object in pentaho?

    I have been searching it for 2 days, and it appears to me that I haven't found something useful for me., such as:
    https://www.safaribooksonline.com/li...3/ch08s05.html
    http://know-be-eye.blogspot.com.au/2...n-pentaho.html


    Also, I am not an advance developer, instead I am only a BA who may not have a lot of knowledge of coding.

    Lastly, (haven't researched yet), we will do the ETL not only from mongoDB database, but also integrate it with some data from REST, and perhaps mySQL. Is that possible to do this in PDI?
    Last edited by hnandarusdy; 03-24-2017 at 12:53 AM.

  2. #2

    Default

    You have searched for two days and not found the MongoDB input step in PDI?

    http://wiki.pentaho.com/display/EAI/MongoDB+Input

  3. #3
    Join Date
    Mar 2017
    Posts
    10

    Default

    I knew about mongoDB input. However, I was not be able to LOOKUP from another collection.

    For example:
    Collection A. {orderId:123, name:Herman}
    Collection B: {orderId:123, product:'abc'}

    I want to have in the output: {orderId:123, name:Herman, product:'abc'}

    Anyway I found the way to use it using sream lookup.



    I currently try to search how to validate of empty object.
    Why I need this? because I will do data cleansing for my ETL. In our collection currently, I have an attribute called, orderValue. And this attribute:
    1. has a string data type on it, ex: "123"
    2. has a number data type, ex: 123
    3. empty object. ex: {}

    This inconsistent was because of our team mistake that didn't realise it saved as a string/number etc. So, what I am planning to do is to convert all of them to number.

  4. #4
    Join Date
    Mar 2017
    Posts
    10

    Default

    Don't worry. I sorted it out. Thanks.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •