Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Stream lookup ...

  1. #1

    Default Stream lookup ...

    When can we expect "Stream lookup" to be able to compare fields with an comparator like "Database lookup" does ..?

    And why was this not implemented from the beginning ..?
    Any catch with the functionality ..?

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default RE: Stream lookup ...

    Hi Thomas,

    >When can we expect "Stream lookup" to be able to compare fields with an comparator like "Database lookup" does ..?

    Please keep an eye on trackers 1500 and 1501.
    If I need to make a guess, in a month or two, maybe sooner.

    >why was this not implemented from the beginning ..?

    There can only be one answer to this question and many others alike: because it was not needed before. Don't get me wrong, I'm happy that Kettle is being used in a lot of roles like data migration, application integration, message parsing, etc. However, the principle role for which it was designed is data warehousing & ETL.
    More on-topic, in this case, I used a hashtable to drive the lookups as this is the fasted way to look up values. It's as simple as that, no need to get worked up about it ;-)

    >Any catch with the functionality ..?

    Until tracker 1500 gets implemented, it runs in-memory. You might need to give Kettle more memory if you want to use it on large data sets.

    Hope this clarifies things for you...

    Best regards,
    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.