Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: infoStreams information is delivered later than i need it in processRows.

  1. #1
    begunrom Guest

    Default infoStreams information is delivered later than i need it in processRows.

    In my processRows, i am reading the InfoStreams, because i want to get
    information about the Input steps (4) connected to this step. What i
    see is that data.InputRowMeta[j]==null with j variable. Sometimes j=3,
    meaning that 3 input steps out of 4 delivered information. If i put a
    "Thread.sleep(1000);" right before "if (first) " introducing a delay,
    everything works fine.

    What measures can i take to be assured that i can readout a complete
    infoStreams ?
    How does the exchange of metadata between steps work?

    public boolean processRow(StepMetaInterface smi, StepDataInterface
    sdi) throws KettleException
    {
    meta=(BMECATMeta)smi;
    data=(BMECATData)sdi;


    if (first)
    {
    List<StreamInterface> infoStreams =
    meta.getStepIOMeta().getInfoStreams();
    for (int j=0;j<meta.getnrOfFieldGroups();j++)
    {
    data.InputRowSets[j] =
    findInputRowSet(infoStreams.get(j).getStepname());
    if (data.InputRowSets[j]==null) {
    throw new KettleException( BaseMessages.getString(PKG,
    "BMECAT.Exception.UnableToFindSpecifiedStep",
    infoStreams.get(j).getStepname()) );
    }
    data.InputRowMeta[j]=data.InputRowSets[j].getRowMeta();
    if (data.InputRowMeta[j]==null) {
    throw new KettleException( BaseMessages.getString(PKG,
    "BMECAT.Exception.UnableToFindStepMeta",
    infoStreams.get(j).getStepname()) );
    }
    }
    }

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

  2. #2
    Matt Casters Guest

    Default Re: infoStreams information is delivered later than i need it in processRows.

    The row metadata is only guaranteed to be provided when you have actually
    read a row.
    For example, if no row is read, there is no row metadata specified for the
    row set.
    As such, you need to read at least one row before you get the metadata.
    getRowFrom(rowset) will do the trick.

    If you have no input row and you still want to know about the input row
    metadata you need to calculate the row layout statically with
    TransMeta.getStepFields(sourceStep)

    Good luck,

    Matt


    2011/9/14 begunrom <gunter.rombauts (AT) skynet (DOT) be>

    > In my processRows, i am reading the InfoStreams, because i want to get
    > information about the Input steps (4) connected to this step. What i
    > see is that data.InputRowMeta[j]==null with j variable. Sometimes j=3,
    > meaning that 3 input steps out of 4 delivered information. If i put a
    > "Thread.sleep(1000);" right before "if (first) " introducing a delay,
    > everything works fine.
    >
    > What measures can i take to be assured that i can readout a complete
    > infoStreams ?
    > How does the exchange of metadata between steps work?
    >
    > public boolean processRow(StepMetaInterface smi, StepDataInterface
    > sdi) throws KettleException
    > {
    > meta=(BMECATMeta)smi;
    > data=(BMECATData)sdi;
    >
    >
    > if (first)
    > {
    > List<StreamInterface> infoStreams =
    > meta.getStepIOMeta().getInfoStreams();
    > for (int j=0;j<meta.getnrOfFieldGroups();j++)
    > {
    > data.InputRowSets[j] =
    > findInputRowSet(infoStreams.get(j).getStepname());
    > if (data.InputRowSets[j]==null) {
    > throw new KettleException(
    > BaseMessages.getString(PKG,
    > "BMECAT.Exception.UnableToFindSpecifiedStep",
    > infoStreams.get(j).getStepname()) );
    > }
    >
    > data.InputRowMeta[j]=data.InputRowSets[j].getRowMeta();
    > if (data.InputRowMeta[j]==null) {
    > throw new KettleException(
    > BaseMessages.getString(PKG,
    > "BMECAT.Exception.UnableToFindStepMeta",
    > infoStreams.get(j).getStepname()) );
    > }
    > }
    > }
    >
    > --
    > You received this message because you are subscribed to the Google Groups
    > "kettle-developers" group.
    > To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    > To unsubscribe from this group, send email to
    > kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    > For more options, visit this group at
    > http://groups.google.com/group/kettle-developers?hl=en.
    >
    >



    --
    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration, Kettle founder, Author of Pentaho Kettle
    Solutions<http://www.amazon.com/Pentaho-Kettle-Solutions-Building-Integration/dp/0470635177>
    (Wiley <http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0470635177.html>)
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence

    --
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.