Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Stream look up - How to

  1. #1

    Default Stream look up - How to

    Hi,

    I have lookup values and codes as

    Code-Value
    AA-1
    B-2
    CCC-3

    Database codes
    AA
    AA1
    AA123
    B
    B123
    1B123
    CCC
    1_CCC123

    First three should map to AA, Next three should map to B and the last set should map to CCC.

    How to achieve this. Whether I can do pattern matching in stream lookup. I have close to 1500 values to looked up and morethan 500,000 records.

    Please guide me.

    Thanks and Regards,
    Subbu S

  2. #2
    Join Date
    Nov 2008
    Posts
    143

    Default

    If you mean the Stream Lookup Step, it only works for exact values.
    A possible way would be using a User Defined Java Expression to create a new column which values are a copy of your database codes but with numbers and other characters that are not letters removed. After that you can match them with the Code from your Code-Value thing.

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    The "Join Rows" step can generate all combinations for you. After that you can define complex or fuzzy matching algorithms in a step like JavaScript, User Defined Java Class and so on.

  4. #4
    Join Date
    Feb 2009
    Posts
    296

    Default

    I'd try to strip both stream down to the values that should match. Maybe remove everything except the digits?
    After that use the stream lookup.
    Fabian,
    doing ETL with his hands bound on his back

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.