Java Plugin: different output between Preview and Run of a transformation

[danielh]

Hey, I developing a java plugin which should create phonetic codes from strings. The plugin read the inputstream, for example two columns with surname and name, these string were transformed with different phonetic algorithms and for each algorithm and column there are will generated a new column with the codes.
Now, when I do a preview of the transformation in pentaho kettle everything works great and the source columns and the new generated columns were written in the output textfile. But when I run the transformation normally I got just the two source columns in the output textfile but not the new generated columns.
I haven't any clue whats going wrong.
Are there two different routines/classes used? I really hope somebody can help me.

Here is my Step Java Class.

PhoneticStep.java

Kind regards
Daniel

[Mark.James.Cloud2Land]

Hi Daniel,

Just so I'm clear your plugin takes an arbitrary number of fields and creates a new phonetic field for every one? and it can apply multiple algorithms? so if you select 3 algorithms you'd have 4 x <original number of fields> in the output

From a quick scan it doesn't look like you're adding the new fields (columns) meta values to data.outputRowMeta (used in putRow to pass your data to the next step)

in your if(first) block try something like:

...
if (temp_check_alg[1]==true) {
    ArrayList<String> col_sound = createColumnsNames("Koelner Phonetik");
    for (String newFieldName : col_sound)
        data.outputRowMeta.addValueMeta(new ValueMeta(newFieldName, ValueMeta.TYPE_STRING))
...

[danielh]

Hey Mark,
thx for your solution. Yeah you're right, you can choose multiple algorithms and for each you get a new column. I tried your solution, but it is still the same problem. I have no idea, why it runs well in debug-mode but not the normal way. I can assume the columns are created correctly. What else can lead to this problem?

Regards
Daniel

[Mark.James.Cloud2Land]

Hi,

I'm guessing you have your plugin connected direct to a text output step? that step will need to know which fields to expect if you right click it and "show input fields" is it correct? similarly for your step -> show output fields. FYI, the PDI preview does actually run the transformation so you can't be far off.

[danielh]

Your guess is right. The output fields of my plugin show the two source fields and a field called phonetic algorithm (the actual name of step). So the new columns are created. Here is a screenshot:
screen01.jpg

Why is a field with name of the stepname created but not the neccessary output fields?
Regards,
Daniel

[Mark.James.Cloud2Land]

I've had a read of your step code and I can't see where you're adding the new ValueMetas to data.outputRowMeta?? if you add the following before putRow(...) what do you get?

log.logBasic("output meta len = "+data.outputRowMeta.size());
log.logBasic("output data len = "+outputRow.length);

[danielh]

When I run the transformation I got this output:
2011/07/08 14:25:19 - phonetic transformation.0 - output meta len = 2
2011/07/08 14:25:19 - phonetic transformation.0 - output data len = 2

BUT when I run the preview I got this:
2011/07/08 14:27:21 - phonetic transformation.0 - output meta len = 6 (2 Inputfields with 2 algorithm = 6 outputfields)
2011/07/08 14:27:21 - phonetic transformation.0 - output data len = 6

this is the actual Stepcode

I try your solution at line 55
PhoneticStep.java

I have no clue why there is still a difference?

Regards,
Daniel

[Mark.James.Cloud2Land]

Have you implemented preview in your dialog? if so can you attach that too. also can you export your trans as a ktr and attach that, possibly something's not saving correctly in the meta. it is a strange problem though.

[danielh]

I didn't implement a preview in my dialog. You can just give a stepname and choose from different algorithms.

test_trans.ktr
PhoneticStepMeta.java
PhoneticStepDialog.java

Iam really glad, that you try to figure out my problem.

Regards,
Daniel

[Mark.James.Cloud2Land]

Hi,

Well I'm not sure why preview works but I see why execute doesn't, it's reading from the saved version of your meta data (eg. a repository entry) and your meta class isn't saving all the data you're configuring with your dialog. have a look at the code for a org.pentaho.di.trans.step.<anything>Meta class, you need to implement getXML, readData, readRep and saveRep so that the information you're setting in the dialog is actually saved in the repository/ktr file.