Hitachi Vantara Pentaho Community Forums
Results 1 to 11 of 11

Thread: Mapping Input with include unspecified fields checked not working?

  1. #1

    Default Mapping Input with include unspecified fields checked not working?

    Hi,

    I am trying to call a subtransformation with a specified input and need to retain all the fields that were prior to calling the sub transformation so I am checking the box for keeping the unspecified fields. I get an exception while verifying the transformation

    java.lang.reflect.InvocationTargetException: Unable to get fields for step [Call Upsert Category1 XForm] :
    Unable to connect find mapped value with name 'segment'.

    at org.pentaho.di.ui.spoon.dialog.SearchFieldsProgressDialog.run(SearchFieldsProgressDialog.java:73)
    at org.eclipse.jface.operation.ModalContext$ModalContextThread.run(ModalContext.java:113)
    Caused by: org.pentaho.di.core.exception.KettleStepException:
    Unable to connect find mapped value with name 'segment'.

    at org.pentaho.di.trans.steps.mappinginput.MappingInputMeta.getFields(MappingInputMeta.java:261)
    at org.pentaho.di.trans.TransMeta.getThisStepFields(TransMeta.java:1759)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1604)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)
    at org.pentaho.di.trans.TransMeta.getStepFields(TransMeta.java:1581)

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    This could very well be. If you don't specify segment as an expected field, you will get this error.
    If you do expect the field you should specified it in the Mapping Input step.
    All that sounds pretty obvious but it simply means that the non-specified fields are only known at runtime.
    Any other behavior would be pointless.

  3. #3

    Default

    So whats the conclusion?

    I am reading in a fixed file with bunch of fields. I am calling transformation with only subset of those fields required/specified for that transformation. And obviously after the call to the transformation I need to process the other fields that were in the original fixed file and may be call other transformation which work on other subset of fields.

    Is this possible? Did you mean to say, just ignore the exception for verfication and continue with the running of the transformation and things will be ok?

    Surprizingly I have a small test transformation with 1. Generate Rows 2. Call this sub transformation 3. Dummy step. In this test transformation even if I dont check include unspecified fields, Show output fields on the call sub transformation step does show the fields from the input as well the ones generated by the sub transformation itself. But the same thing doesnt work with my production transformation where the input is the fixed file.

    Quote Originally Posted by MattCasters View Post
    This could very well be. If you don't specify segment as an expected field, you will get this error.
    If you do expect the field you should specified it in the Mapping Input step.
    All that sounds pretty obvious but it simply means that the non-specified fields are only known at runtime.
    Any other behavior would be pointless.

  4. #4
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Ritesh, all I'm saying that if you are using an input field in a sub-transformation, you need to specify it in the "Mapping Input" step.Since the rest of your comments are surprisingly unclear to me I'll leave it at that.If you think you found a bug, please do us a favor and file a bug report with a sample set of transformations attached.

  5. #5

    Default

    Thanks for the response.

    Ok I could reproduce the problem in Mapping samples. If you take a look at the sample mapping transformation (Kettle\samples\mapping\Mapping - simple mapping.ktr) and check the box for Include Unspecified fields, ordered by name in the Mapping input step and preview/run the other sample which uses this transformation (Kettle\samples\mapping\Mapping - use simple mapping.ktr) you will see the error.

    May be I am misunderstanding how that checkbox works.

    If you do show output fields on call sub transformation, you will see all the fields, even the unspecified ones getting passed through correctly. But in my sub transformation, I only get the fields produced in the sub transformation, not including the input fields. I am attaching both the wrapper transformation and sub transformation for the reference.

    Also interesting is, if you add a step Add Constants in between the Generate Rows and Calling the sub transformation and hit preview in the 2nd sample, you get no rows! Can you please explain why? Add Constants doesnt pass the rows?DblFullFeedTransform.ktr

    CategoryUpsert.ktr

  6. #6
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    These are the sort of transformations we can't do anything with since we can't reproduce anything with.
    Unless you're willing to put your databases on the internet and give us access to it of-course.

    Anyway, there is nothing inherently wrong with the mapping as far as can see.

  7. #7

    Default

    Well you can reproduce the error with the samples that already come with PDI - as I mentioned earlier.

    Also, why would inserting add constant step not work after the input step and before the call sub transformation step?

  8. #8

    Default

    Just to complete the thread for other ppl.

    This is a bug and here is JIRA
    http://jira.pentaho.com/browse/PDI-1846

  9. #9
    DEinspanjer Guest

    Default

    The problem appears to be that if a field is renamed via the input panel, and you configure the mapping input step to include unnamed fields, the getFields() method doesn't honor the checkbox for renaming the fields back.

    I am able to reproduce the problem with showing the output fields. Note that the preview works properly which leads me to believe this is strictly a design time problem.

  10. #10

    Default

    I am not too sure its just design time problem. If you had steps following - atleast in my production version of the transformation I did - I couldnt do anything with those steps since it wont see those fields at all. You could still call this design time I guess, but basically you cant proceed further completing the transformation. Havent tried adding steps following the Mapping step in the sample myself.


    Quote Originally Posted by DEinspanjer View Post
    The problem appears to be that if a field is renamed via the input panel, and you configure the mapping input step to include unnamed fields, the getFields() method doesn't honor the checkbox for renaming the fields back.

    I am able to reproduce the problem with showing the output fields. Note that the preview works properly which leads me to believe this is strictly a design time problem.

  11. #11
    DEinspanjer Guest

    Default

    I agree with you that it can be a serious problem if you get into this state because many steps require getting the prior fields in order to be designed properly.

    For the time being, you can work around the problem by not renaming those fields in the mapping step. While designing, change the mapping to use the same names as your transformation and delete the field renaming. You should then be able to finish designing and then if you need to put the renaming back in, do that when you are ready to test run.

    I know it isn't a great solution, but hopefully someone (maybe me! will look at the bug and figure out a fix in time for the next version.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.