I am developing a set of plugins to interface with a new data
platform. I've got it working in a linear fashion. However I want to
run some of the tasks in parallel or multiple paths/threads. I see
you can run multiple paths but rejoining them and having data passed
to the merge step seems to be an issue. I am using the prevResult and
returning the Result in the execute function to carry my data between
steps. The problem the merge/join is just called by the thread that
finishes first, is there a way to have some type of wait loop that I
can merge the data from all the previous steps going into the merge
step.

I'm looking at using a static variable to enter a waiting loop that
would block all other calls until all the data is available, each
additional call to this step would, based on this static variable, go
into a merge function that would merge its data into a static variable
and then once the count has reached the number of paths continue.
With this I need to know a way to write a split step that can some how
detect the number of exiting paths, is this possible?

There has to be a better way but I don't see a construct to do it.

I know this doesn't quite fit in with Spoon's existing infrastructure
but I've been tasked with doing this.

Thanks,
Joseph

--
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com.
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) g...oups (DOT) com.
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en.