Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Loading facts

  1. #1

    Default Loading facts

    When loading a fact table, facts are coupled to dkey 0 for dimensions where no match is found. Is there an easy way of handling these records? I mean, besides an SQL node where I check on these dkeys = 0? Is there a standard handling of these records in PDI?

    E.g. in Cognos these records are written to a .rej file. Is there something similar in PDI?

    Thanks.

  2. #2
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Not quite sure I follow what you are getting at...

    You have a source flow that is going through a Dim Lookup / Update, and it gets back a Dimension Key of 0 for No Match?
    If that's the case, then filter the flow so that those records that have a Key = 0 go through an alternate flow, going to a Text File Out with your .rej Filename.
    **THIS IS A SIGNATURE - IT GETS POSTED ON (ALMOST) EVERY POST**
    I'm no expert.
    Take my comments at your own risk.

    PDI user since PDI 3.1
    PDI on Windows 7 & Linux

    Please keep in mind (and this may not apply to this thread):
    No forum member is going to do your work for you. We will help you sort out how to do a specific part of the work, as best we can, in the timelines that our work will allow us.
    Signature Updated: 2014-06-30

  3. #3
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    I think a data warehouse should represent your source systems as closely as possible. If, for example, there are orders without a product, you can list those in the data warehouse using the unknown (0) entries in the dimensions.
    If you can list the data quality problems in the source systems, the data owners can fix them, creating a nice feedback loop between the source systems and the data warehouse.

    If you choose to try and fix these problems you can end up in a situation where your data warehouse project fails simply because you're trying to do someone else's job.

    Just my €0.02,

    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.