Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Create multiple (multilevel) XML files in 1 transformation

  1. #1

    Default Create multiple (multilevel) XML files in 1 transformation

    Hello all!

    I have a problem that I don't get sorted out somehow. We load and update huge amounts of data about persons, like basic information, addresses and so on. So, whenever such an info-bit has changed, we need to create a small XML-message and send it to a queue.
    So, getting some basic information, there's a 2nd level with an indefinite amount of statuses. So I want to join the basic information and the statuses to 1 XML and then send it. The problem I have is that I seemingly can't do this for many persons in 1 transformation.
    There are 2 examples in the transformation attached but both don't work. The first one has all the persons in the upper and all the statuses in the lower part and joins them. The problem is that not only a certain person's statuses are joined but all in the lower part. In the second example I tried to get both XML parts into 1 row and join them, without a need to match, but it still joins all statuses onto the person.
    So, basically what I need is to query 50 persons and 50 statuses and then join them together to 50 XML-messages on the condition that person.person_id = status.person_id. All that in 1 transformation. Can I do that?
    Thanks a lot for your help!
    Attached Files Attached Files
    Last edited by d.hommrich; 04-08-2016 at 05:46 AM. Reason: got logged out

  2. #2

    Default

    Hello again!

    I found the functionality that Mappings provide and it's actually a possible solution, I think. So, I now have 2 transformations, one that queries the data and one that assemblies the XML. Thing is, I need the Mapping to be executed row by row or "Single Threaded", as it's called in Kettle. So, a first attempt with Mapping and Simple Mapping didn't work, it gave me the same result as before, but invoking the Single Threader should do it. Should, since it's not running the same transformation that worked just fine before (see attachments). Here's the error-message I get:
    Code:
    2016/04/11 14:35:20 - XML Join.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Error initializing step [XML Join]2016/04/11 14:35:20 - t_assemble_person_basic_information_xml - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Step [XML Join.0] failed to initialize!
    2016/04/11 14:35:20 - Single Threader.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Unable to load the mapping transformation because of an error : org.pentaho.di.core.exception.KettleException: 
    2016/04/11 14:35:20 - Single Threader.0 - We failed to initialize at least one step.  Execution can not begin!
    2016/04/11 14:35:20 - Single Threader.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : org.pentaho.di.core.exception.KettleException: 
    2016/04/11 14:35:20 - Single Threader.0 - We failed to initialize at least one step.  Execution can not begin!
    2016/04/11 14:35:20 - Single Threader.0 - 
    2016/04/11 14:35:20 - Single Threader.0 - 
    2016/04/11 14:35:20 - Single Threader.0 -     at org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1149)
    2016/04/11 14:35:20 - Single Threader.0 -     at org.pentaho.di.trans.steps.singlethreader.SingleThreader.prepareMappingExecution(SingleThreader.java:223)
    2016/04/11 14:35:20 - Single Threader.0 -     at org.pentaho.di.trans.steps.singlethreader.SingleThreader.init(SingleThreader.java:315)
    2016/04/11 14:35:20 - Single Threader.0 -     at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:69)
    2016/04/11 14:35:20 - Single Threader.0 -     at java.lang.Thread.run(Unknown Source)
    2016/04/11 14:35:20 - Single Threader.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Error initializing step [Single Threader]
    2016/04/11 14:35:20 - t_send_person_basic_information_to_queue - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) : Step [Single Threader.0] failed to initialize!
    Why is that, why craps the XML Join out here but not when using the other 2 Mapping-steps?
    Attached Files Attached Files

  3. #3

    Default

    After some discussions and even talking to PDI Support it seems that the wanted functionality is not supported. XML-Join expects only 1 row to be supplied as target which means you only get 1 XML document out as a result of the transformation. The solution is to change cardinality - instead of querying all changed persons in the transformation, you execute it once per person and assemble the XML. Another solution (possibly with better performance) is to omit the XML functionality in PDI and instead build the XML using Strings, e.g. in SQL or whatever suits you best.
    That as a short summary. If anyone is interested in details, feel free to comment here.

    EDIT: The problem that XML Join doen't work in a Single Threader when only sending 1 row to the underlying mapping was not solved, this might be a bug.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.