Dear Kettle friends,

The original 4.0 schedule was kinda crazy. With all the new stuff on the agenda, specifically related to all kinds of Enterprise Edition stuff, we have been struggling to finalise all the API changes.
In any case, lots of work has been going on in the background related to these 4 topics:

- Repository interface & plugins
- Logging API changes
- Step metadata API changes

All 3 topics are related to metadata and API. Since these changes cause the API to break vs 3.x we decided to take a bit of extra time in planning. This to make sure we take the bulk of everything we want to do with us for the next couple of years.
We wouldn't want to recompile all our plugins in a year or so.

Anyway, as far as the repository plugin architecture is concerned, most of the changes are known and out there.
What is less known is that I'm about to commit a new logging architecture to PDI trunk. Logging issues have crept up to the top of the agenda at a remarkable pace :-)
I guess that simply means that there are no other problems left <cough>.

The architecture will use logging channels. A logging channel basically has a type (Job, Trans, Step, GUI, TransMeta, StepMeta, etc) and an ID.
Before you use a logging channel, you register it and you will get an ID from the registry. The registry will keep the lineage of the object in question.

For example, a step logging channel registers itself and the hierarchy (parent transformation, parent job, parent job, ..., root job) is extracted.
This lineage information is stored in the registry.

When we log a line, we no longer log the name of the step, but use the log channel ID. This way we can always see in retrospect where a log line came from.
From which step, transformation, job, etc.

Code wise the change is not that dramatic, the bulk of all log commands appear in the steps and job entries.
For example, this log line from TableInput:

if (log.isDetailed()) logDetailed(toString(), "Reading from step [" + meta.getLookupStepname() + "]");

Nothing changes to this line. However the functionality is different.
The command in the background is now:


log is no longer LogWriter but LogChannelInterface.
In the constructor of BaseStep we call

log = new LogChannel(this);

This actually extracts the hierarchy and registers with the log registry (singleton).

In other places of the code, the change has been less trivial but in general I think it's really worth it to do this.
For example, what we capture in the hierarchy is the repository ID of the object. As such, it will be possible to later cross reference log lines with repository objects, files, etc.

In the background, everything is still running on Log4j. We just don't log String objects any more. Rather we pass LogMessage objects around.

Anyway, if you have any suggestions for more new possibilities rock, let us know.
I should be able to commit these changes to trunk in the 2nd part of this week. As you can imagine, this impacts most of the code in trunk :-)

All the best,

Matt Casters <mcasters (AT) pentaho (DOT) org>
Chief Data Integration
Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho : The Commercial Open Source Alternative for Business Intelligence

You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at