Hitachi Vantara Pentaho Community Forums
Results 1 to 7 of 7

Thread: metaDataInjection in a subtrans

  1. #1

    Default metaDataInjection in a subtrans

    Hi,

    i'm hoping on the power of the forum for this:

    From a /locationA/TransformationA.ktr, i use the step Mapping(sub-transformation) to start another trans /locationB/TransformationB.ktr. In the subtrans there is a metaDataInjection step. This step calls an transformation from the following location: ${internal.transformation.filename.directory}/TransformationC.ktr. The location of Transformation C is /locationB/TransformationC.ktr.

    When i start TransformationB, all works fine.
    When i start TransformationA i get an error message from the metaDataInjection step, that i can't find the KTR file from location: /LocationA/TransformationC.ktr.

    Is this normal behaviour? Why does it seem to find the transformation used for injection in the folder from where i start the main transformation?

    i use kettle 5.4.



  2. #2

    Default

    ow, forgot to mention that when i change the path of the metaDataInjection step to /locationB/TransformationC.ktr ​(the full path) everything works fine.

  3. #3
    Join Date
    Oct 2008
    Posts
    25

    Default

    Hi,

    Calling a transformation from another transformation using the sub-transformation step causes all steps in the sub-transformation to behave as if thery were part of the main transformation. This also applies to the system variables and is not specific for the meta-data injection step. This can be solved by using the transformation executor step in PDI 5.4. See the attached example (subtrans.zip) which shows the difference in behavior between both approaches in the log output when execution main/tr_main.ktr.
    Hope this helps
    Attached Files Attached Files

  4. #4

    Default

    Thank you for your reply!

    It did help in understanding WHY this is happening. But i'm making a date converter, which needs the input stream. The input stream is dynamically build, so the field names in the stream are variable. That's why i can't use a Transformation Executor. A sub transformation will pass the stream because it is part of the stream .

    I'm trying now to create a parameter which holds the full path to the inject KTR.

  5. #5

    Default

    We can't use a variable either as a injectFile, because the metaDataInject step initialises on startup of the mainJob, which is in a total different directory.

    I've shared 2 KTR in the zip:
    There are notes in the KTR which documents the KTR.

    - dateConverter.ktr ==> called from another transformation with a submapping step.
    - injectConverter.ktr ==> called from dateConverter.ktr with metaDataInjection.

    We use the above ktr's as a library, so that more projects can use this, by calling the dateConverter.ktr with a submapping step. Since the parent ktr's are unaware of the library directory we cannot set the KTR there.

    I hope someone is able to help me here.

    dateConverter.zip

  6. #6

    Default

    little update: we resolved it by using a "global" variable which points to the library directory. Since it is a global variable, we set it at the beginning of the job and the metaDataInjection step can be called with it. I.e. : ${global.lib.dir}/dateConverter/etl/injectFile.ktr.

    It's not the best option to use global variables in a library component (the component itself should run fine without it), but i think it's the only possible solution if you want to use a stream in a library.
    Happy to hear it if anyone thinks (or knows) differently.

  7. #7
    Join Date
    Aug 2011
    Posts
    360

    Default

    Hi,

    We also have some transformations/jobs that can used by any projects.
    We have a directory (in repository) called "shared" which is the root of all shared objects.
    The variable to this dir is defined in the kettle.properties.

    This seems to be the only way to do it. But it has the advantage to identify componants that are reused, that should be
    changed with care.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.