Review requests

Printable View

08-08-2009, 07:56 AM
Phillip Cole

Review requests

Hi All,

I've been committing simple changes that I'm confident with and that do
not affect functionality. I'm now down to the last few that I'm not so
confident with. If you have time, please can you do a quick
review/approval of the following:

http://jira.pentaho.com/browse/PDI-771 : Preserve relative path when
browsing/selecting a transformation/job.
- Added an environmentReplace() function within Variables.java. This
substitutes a string with the variables that were used. This means that
all users of Variables.java now require this environmentReplace()
function. Is this the correct way of doing this? Will it break any
external users?

http://jira.pentaho.com/browse/PDI-2081 : Multiple copies of a step need
sometimes data movement copy instead of distribution
- Are you happy that this is implemented via colouring the suspect hop
line red?

http://jira.pentaho.com/browse/PDI-2257 : Get files from result gives
ArrayIndexOutOfBoundsException thrown when cloning rows.
- I've got a fix for this, but the fix prevents the "show output fields"
from working on the "Get Files from Result" step. Is there a better way
of doing this?

http://jira.pentaho.com/browse/PDI-2546 : Sequential job steps should be
numbered.
- Is this the correct UI suggestion? See
http://jira.pentaho.com/secure/attac...nshot-test.png

Kind Regards

Phil

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-09-2009, 01:21 AM
Matt Casters

Re: Review requests

On Saturday 08 August 2009 13:56:50 Phillip Cole wrote:
> http://jira.pentaho.com/browse/PDI-2081 : Multiple copies of a step need
> sometimes data movement copy instead of distribution
> - Are you happy that this is implemented via colouring the suspect hop
> line red?

Let's not go back to using red for that color as it is not an error condition.
Some time back we agreed that more visual icons, mini-connectors, mouse-over helps, etc were needed, not another color coding scheme.

Cheers,
Matt

Matt Casters <mcasters (AT) pentaho (DOT) org>
Chief Data Integration
Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho : The Commercial Open Source Alternative for Business Intelligence

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-09-2009, 04:41 AM
Matt Casters

Re: Review requests

On Saturday 08 August 2009 13:56:50 Phillip Cole wrote:
> http://jira.pentaho.com/browse/PDI-2546 : Sequential job steps should be
> numbered.
> - Is this the correct UI suggestion? See
> http://jira.pentaho.com/secure/attac...nshot-test.png

I have mixed feelings about this one...
On the one hand, Sven is right in suggesting that putting job entries in parallel when you should put them in sequence could be considered incorrect use of a job.

On the other hand, we do allow things to be drawn out that way. In general I don't like to limit options for people.
However, that leads to the annoying situation that you can see the order in which the job entries are executed without having the possibility to change that order.
Adding these options would seem a bit messy although I guess it wouldn't be overly dramatic to do. So you would need 2 extra menu options for the hop (in JobGraph) : execute earlier, execute later.

Looking a bit at this from above I think that the whole problem comes from the fact that people don't have a legend for the hop/dashed-hop (serial/parallel execution).
As such, I would be more inclined to add little icons over the hops with a mouse-over tooltip explanation in very much the same way as we did for the transformation hops.

Cheers,
Matt

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-09-2009, 04:53 AM
Matt Casters

Re: Review requests

On Saturday 08 August 2009 13:56:50 Phillip Cole wrote:
> http://jira.pentaho.com/browse/PDI-771 : Preserve relative path when
> browsing/selecting a transformation/job.
> - Added an environmentReplace() function within Variables.java. This
> substitutes a string with the variables that were used. This means that
> all users of Variables.java now require this environmentReplace()
> function. Is this the correct way of doing this? Will it break any
> external users?

Hi Phillip,

These are pretty dramatic changes for what is essentially a relative path calculation.
What Daniel seems to request is that if you have a Browse button on a dialog to browse for a file and the field already contains a variable, these things happen:
- You are taken to the location that is specified (example, ${FOO}/bar.txt --> browse in ${FOO} == /foo/ )
- When you pick a file, you get a location that is converted to a relative path to the specified variable. (example /foo/bar/file.txt --> ${FOO}/bar/file.txt)

I guess it would be more efficient to create a new method for the relative path browse function somewhere and allow re-use. You could even subclass the File browse dialog for that I guess. (probably not that easy with SWT)

Anyway, let's keep it simple. I don't exactly know why these changes to Variables and VariableSpace are needed in this specific case.

>+ //FIXME: I don't understand all this synchronized stuff.

Since multiple threads operate at the same time, set variables/use variables in multiple threads, it's best to be cautious.

>+ // FIXME: Are replaceWindows and replaceHex required?

Yes, I don't think a lot of folks use the %%Windows%% variable notation, but there's always someone out there that does :-)
replaceHex() is needed as it allows you to specify special characters not allowed in XML for example.

Matt Casters <mcasters (AT) pentaho (DOT) org>
Chief Data Integration
Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho : The Commercial Open Source Alternative for Business Intelligence

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-09-2009, 04:57 AM
Matt Casters

Re: Review requests

On Saturday 08 August 2009 13:56:50 Phillip Cole wrote:
> http://jira.pentaho.com/browse/PDI-2257 : Get files from result gives
> ArrayIndexOutOfBoundsException thrown when cloning rows.
> - I've got a fix for this, but the fix prevents the "show output fields"
> from working on the "Get Files from Result" step. Is there a better way
> of doing this?

This sort of error usually happens if the data that is sent out doesn't correspond to the metadata.
For example, 10 fields were specified in metadata and only 9 data fields are sent out, etc.
Verifying both the data and especially the metadata is key in this case.

The metadata is typically defined in the getFields() method of the ~Meta classes.

Hitting space while mousing over a step reveals the output fields. Do a preview and verify if the output corresponds to the output fields.

Cheers,
Matt

Matt Casters <mcasters (AT) pentaho (DOT) org>
Chief Data Integration
Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho : The Commercial Open Source Alternative for Business Intelligence

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-09-2009, 05:39 PM
Phil Cole

Re: Review requests

Matt,

Thanks for all the reviews. I'll work through them and see what I can come
up with.

--
Phil

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---
08-10-2009, 08:17 AM
Jens Bleuel

List of Kettle Plugins: Updated Wiki page

Hi all,

after some more links to new plugins (the latest is the Teradata bulk
loader), I decided to update and reorganize the following page a bit:
http://wiki.pentaho.com/display/EAI/...ation+Plug-Ins

I hope the new listing is a bit more "eye-friendly" and I categorized
the plugins a bit more and ordered them alphabetical within each category.

For everyone who contributed a plugin and is listed at this page: please
double check, if all is correct with your plugin and the links (I wonder
a bit why we have differences and specifications for 3.0 and 3.1 - in
this case I would like the contributor to check, if we can change this
to 3.x or 3.0, 3.x).

If you have a plugin somewhere laying around that is not listed here,
just add it to the Wiki page or contact Doug or me.

Thanks a lot & keep on hacking,
Jens

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---