Dear Kettle friends,

As mentioned earlier, I'm doing a major cleanup and refactoring of the existing repository code.
Today and tomorrow I'll be committing large blocks of code-changes so do me (and yourself) a favor by not modifying the existing repository code in the mean time. (TIA!)
Also make sure to do an SVN update once in a while so you don't get in trouble.
There are no changes anticipated to the i18n files.

Here are the changes:
- Move everything into the Repository class, this includes constructors of JobMeta, TransMeta, etc.
- Move everything non-essential out of Repository into a Repository*Delegate class (RepositoryJobDelegate, RepositoryTransDelegate)
- Convert Repository into an interface

As you can imagine, this cleanup takes a bit of time as the Repository code managed to get intertwined with all the other code over the course of the last 6 years.

For example, JobMeta now includes a method :

public boolean showReplaceWarning(Repository rep);

It is used to verify if the job already exists in the repository.

We are changing this method (only used in one place) to a method in Repository:

public boolean existsJobMeta(JobMeta);

Once Repository is an interface, it will become a lot easier to create alternative versions of the current Repository, for example :
- A repository that uses XML files for Jobs, Transformations, Connections, slave servers, etc
- A repository that uses XML files stores in a Pentaho solutions repository
- A repository that stores information in CWM
- A repository that uses a Content management system (CMIS), including version control, access control, etc.

That last example is obviously the goal of this complete exercise. That being said, it will be *really* good to have a cleaner codebase as well.
With the approach above we think we can make the current installations of the Kettle repository work as before and still provide a seamless upgrade path.

There are a lot of loose ends that need to be tied up with respect to this subject, including management of the relationships between objects (transformations using a database for example). However, I'm confident we'll be able to get there.

Since we are changing the API of JobMeta, TransMeta and a lot of other classes, I propose to set the version number of PDI to 4.0 right after these changes.

Note: It is still our intention to make the existing (step/jobentry/partitioning) plug-ins work on the next release as well.
The consequence of using the old repository code is however, that the generated XML through the alternative repositories is different from the standard Kettle XML.
Because of this, it might be "easier" or more convenient to break compatibility since it's just the removal of a few methods from StepMetaInterface.
That being said, in that case there should be a well-tested migration utility for the existing Kettle repositories.

As usual, your opinion is valued on these subjects, let us know what you think before we go further in that respect.

All the best,

Matt Casters
Pentaho, Chief Data Integration
mcasters (AT) pentaho (DOT) org

You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
For more options, visit this group at