Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: `Unconnected` Database Lookup?

  1. #1

    Question `Unconnected` Database Lookup?

    Greetings!

    I am using Kettle to update our data-mart every night. Our fact table is updated via a transformation that uses multiple database lookup steps to grab foreign key values before insert().

    I am wondering if there is a way to make a database lookup `unconnected` (i.e. not part of the transformation stream, yet in the transformation) and call it as a variable (e.g in a Javascript step).

    Something like:
    Code:
    if 
    dblookup.v_lkp_fieldname==input_step.fieldname 
    then
    dblookup.v_lkp_fieldname 
    else 
    input_step.fieldname
    In my mind, it would present a few advantages:
    a) it would return only one value
    b) it would be re-usable as it could be called multiple times
    c) it would not process every row but return whatever value an expression sets as `true`... or `false`

    I know another ETL tool (starts with "I...") is doing this already but I don't care. The suits at my company are beginning to see the benefit and power of Kettle and I want seize that opportunity to show them what it can do without spending millions of $ . If nothing else, I hope this capability is in the works.

    Is this feasible at all? I just began experimenting with it today...

    I welcome your input. Thanks, Al.
    Last edited by acbonnemaison; 12-03-2008 at 07:42 PM.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I am wondering if there is a way to make a database lookup `unconnected` (i.e. not part of the transformation stream, yet in the transformation) and call it as a variable (e.g in a Javascript step).
    That sounds like a great way to make it very confusing.
    Can't you simply apply a filter and split the stream: for a certain condition do the lookup, otherwise add a default value?

    Matt

  3. #3

    Default

    Quote Originally Posted by MattCasters View Post
    Can't you simply apply a filter and split the stream: for a certain condition do the lookup, otherwise add a default value?
    You are right. I can do that.

    I did not explain myself clearly...I am inquiring about ways to make a database lookup re-usable within a transformation numerous times.

    Within a transformation, I know I can copy a database lookup step N times and it works great but it has be connected to the stream.

    I wonder if there's a way to have a `standalone` database lookup step within a transformation and call it like an expression.

    Just being curious. Nothing else.

    Al.

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Well, I guess you can use a Mapping (sub-transformation) to do it.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.