PDA

View Full Version : approach to shared files in projects



aphillips
09-11-2008, 09:05 PM
Folks,

Some of you may be aware that we are in the process of standardizing all Pentaho projects on a common build file, aptly named common_build.xml (see this thread for more details http://forums.pentaho.org/showthread.php?t=64111). In short, each project will have a build.xml that imports this common_build.xml. Obviously, in order for all projects to behave the same, they need to be importing the same exact common_build.xml. This is the heart of the forum question, what is the best approach to shared files?

Here are the criteria for the solution:
1. Ease of use by developers (both Pentaho and Community members)
2. Uncomplicated process when the file needs to change
3. Preserve the file's history in a single place
4. Maintain multiple versions of the file (to accomodate branches)

Just to get the conversation going, here are some
**Possible approaches**:
1. use svn externals to physically host the file in one svn tree but logically link it to all projects
2. have all projects require the presence of a separate "build" project (separate svn repository)
3. physically commit the common file to all projects and develop a checkin utility that allows checkin to a single place and commits the file out to all projects automatically

Thanks in advance for your input,
Aaron

Taqua
09-12-2008, 03:54 AM
1. only works if you are in the same repository, right? Kettle and reporting are in a own repository.
2. Explain *that* to an unexpecting user who wants nothing more than a easy build process and who does not want to study arcane magic to get his custom jar.

So sad but true:
3. I dont see an alternative to that manual copying. At least this is what I ended up using in the reporting-projects. There I have a "ant" directory that contains the ant-lib file with all the definitions. The main-ant file then simply imports that file and in most cases simply uses the tasks from the lib, while I'm still able to customize/extend the build-process when needed.

But speaking out of experience: The common-ant file *must* have a big, BIIIIG section telling what version it is (without using external tools, just by looking into it). In the past, I had too much fun trying to keep all common-ant files in sync, until I added such a boiler-plate.

mlowery
09-12-2008, 09:17 AM
Maven POMs (an XML file that describes your project) can have a parent from which they inherit settings. Usually you reference a parent POM as a file relative to the current POM. But you can also reference a POM via HTTP. While this alone doesn't pass the airplane test (i.e. disconnected from network) you could pair this approach with one that caches the file locally and uses the cached version if no network is available.

aphillips
09-12-2008, 09:56 AM
But you can also reference a POM via HTTP
The reason I didn't mention this approach is it doesn't pass requirement #4 (Maintain multiple versions of the file) at least not in a straightforward manner. Also like you mentioned there is the network issue, but I think the #4 issue is the bigger one. I can't think of an elegant solution to that problem.

aphillips
09-12-2008, 10:48 AM
3. I dont see an alternative to that manual copying
So my concerns with approach 3 are developers will be discouraged from making improvements to the common build file due to the shear effort required to deliver that to other projects. Worse yet, as you probably have encountered, projects will begin to evolve their own flavor of the common build file, which would make it uncommon and totally defeat the point. Is there a way to solve this? The only thing I can think of is have some kind of central place to commit changes to the common build file and that central place has a mechanism to deliver the changes automatically to all projects.

mlowery
09-15-2008, 10:50 AM
Here's how Maven solves the parent issue:



<parent>
<artifactId>pentaho-parent</artifactId>
<groupId>org.pentaho</groupId>
<version>2.0.0-SNAPSHOT</version>
</parent>


The pom file for the pentaho-parent project is located in the repository just like all other projects' pom files. Could we put the common-build.xml in the repository and have it get versioned like other artifacts? Using artifactory will also provide us with an upload interface.

aphillips
09-16-2008, 11:54 AM
Could we put the common-build.xml in the repository and have it get versioned like other artifacts?
This presents a cart-before-the-horse scenario. How would you extract the common build file without have an IVY aware build process? The common build itself is the IVY-aware component.