Short term road ahead?

[Sven Boden]

Matt,

Just a few questions:

1) The SVN version is currently still 2.5.0 . Is the same going to
happen as before. Move 2.5.0 as a copy in a different place and
continue development in the current trunk?
If it's going to be the same what will be the SVN place to make
"critical patches" to 2.5.0 (not that we used such patching a lot
before).

2) Is the code freeze gone, can we start implementing CRQs/changes
again? Business as usual or something else?

3) Any ideas on a short road map to v3.0.0 from your side, besides the
"new engine"? Can we "vote" for other CRQs as well e.g.?

Best regards,
Sven


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

[Matt Casters]

Hi Sven,

I think it's a great idea to put together a list of all the things that need
to be done.
As far as myself is concerned, we should go to 3.0.0 directly. I branched
2.5.0 so if there are critical patches we can copy that branch to a 2.5.1.
working copy and do a maintenance release later.

The code freeze is absolutely gone and all seems to be well for 2.5.0. I
think taking the time to deliver it was well worth it.
BTW, Pentaho is going to do the announcement / bragging / marketing May 14th
at the TDWI World Conference in Boston
(http://www.tdwi.org/education/confer...007/index.aspx)
Let's hope it finally attracts some attention from analysts there. Analysts
these days seem to do their utmost best to ignore Open Source BI, let's see
how long they can keep doing that :-)

So yes, let's put together a roadmap for version 3 that is very ambitious.
Throw in anything you like and then we can vote on what we need the most.

It would not surprise me at all if building this new version would keep us
busy until the end of this year or even longer.
If we can release earlier, that's OK too, but I don't want to have anyone
set a deadline on this one.

All the best,

Matt

-----Original Message-----
From: kettle-developers (AT) googlegroups (DOT) com
[mailto:kettle-developers (AT) googlegroups (DOT) com] On Behalf Of Sven Boden
Sent: Saturday, May 05, 2007 3:07 PM
To: kettle-developers
Subject: Short term road ahead?



Matt,

Just a few questions:

1) The SVN version is currently still 2.5.0 . Is the same going to happen as
before. Move 2.5.0 as a copy in a different place and continue development
in the current trunk?
If it's going to be the same what will be the SVN place to make "critical
patches" to 2.5.0 (not that we used such patching a lot before).

2) Is the code freeze gone, can we start implementing CRQs/changes again?
Business as usual or something else?

3) Any ideas on a short road map to v3.0.0 from your side, besides the "new
engine"? Can we "vote" for other CRQs as well e.g.?

Best regards,
Sven





--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

[Sven Boden]

I've rummaged a bit through the CRQ's and BUGs...

Qua features we miss in PDI and which fails us some short-listings is
probably a cobol reader (CRQ #4436) and being able to read IMS
databases (CRQ #4448).
IMS should be pretty easy to implement with the right JDBC driver (but
I don't have access for the moment). Implementing a good cobol reader
can be a big task, anyway the cobolreaders included in most other ETL
products suck bigtime.

For a further list (except for the first, more or less in order):
- Icon refresh?
- Your 3.0 engine refresh, if it's faster and doesn't break the whole
thing <grin>
- Avoid repository/job corruption by making job entry names unique
(CRQ #4991)
- Provide categories for jobs in a same way as for transformations
(CRQ #4647)
- Make kettle variables work in all cases (bug #5392 e.g.)
- Bug #4965: Filter step does not consider to be "" equal to NULL
- Extend carte: jobs, timed execution... so it becomes something of a
headless spoon. Where people can logon to and execute things, or look
at log files, ... maybe make it able to publish jobs as webservices.
- Include version of step/job entry per instance. Version number is 1
number (number 1 can be forgotten). Include version of saving version
at transformation level as well (CRQ #2384 e.g.)
- Support for Oracle RAC connections (CRQ #4403), this should be
smallish.
- CRQ #4988: include "include" filter on text file input step. This
allows for a more easy way of splitting multi-level files.
- CRQ #4839: upgrade javascript support + include E4X
- CRQ #4362: access LDAP (CRQ #1961)
- DB2 bulkloader?
- CRQ #5446: LDAP authentication

Personal planning SB (for now)
-------------------------------
- CRQ #5556: Extend Blocking Step do optionally pass all rows through
- CRQ #4620: Concatenate step
- CRQ #5524: Case-Sensitive UniqueRows cannot rely on output of
SortRows + optimize sort rows
maybe also make unique rows optionally non case
sensitive.
- More unit testcases
- Gold plate/optimize/cleanup some steps.

And what I personally would also like is a small execution engine for
PDI that takes care of high avalability which some extra things....
I'll try something later on the year outside of the kettle trunk (I
currently don't see a way to put it in Carte anymore ;-)).
Requirements for me:
- Per project I want to put 1 line in a HA crontab. That executable
should do scheduling for the project itself of jobs. It should also
take care of automatic restarting of jobs on fail-over, execute
missing job executions, ...
- Everything in the engine should be file based so I can put the
project itself in SVN
- People being able to adhoc restart jobs and to watch log files
(something like carted does now)

Regards,
Sven


--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---

[Matt Casters]

All these are simple implementation, almost trivial to the point of being
boring, except for the Icon refresh and the new engine.
Obviously I'm not forgetting to exaggerate here. <cough>

Wes Brown at Pentaho apparently already has a number of icons ready, I'll
try to get a few from him to showcase 'm.
He wants to make all icons in 4 states:
- Normal
- Mouse-over
- Selected
- Disabled

I would like to add to the list a debugger for transformations:
- Pauze/Resume transformations
- row backtracking
- single stepping
- breakpoints using filters

All the best,

Matt


-----Original Message-----
From: kettle-developers (AT) googlegroups (DOT) com
[mailto:kettle-developers (AT) googlegroups (DOT) com] On Behalf Of Sven Boden
Sent: Saturday, May 05, 2007 10:21 PM
To: kettle-developers
Subject: Re: Short term road ahead?



I've rummaged a bit through the CRQ's and BUGs...

Qua features we miss in PDI and which fails us some short-listings is
probably a cobol reader (CRQ #4436) and being able to read IMS databases
(CRQ #4448).
IMS should be pretty easy to implement with the right JDBC driver (but I
don't have access for the moment). Implementing a good cobol reader can be a
big task, anyway the cobolreaders included in most other ETL products suck
bigtime.

For a further list (except for the first, more or less in order):
- Icon refresh?
- Your 3.0 engine refresh, if it's faster and doesn't break the whole thing
<grin>
- Avoid repository/job corruption by making job entry names unique (CRQ
#4991)
- Provide categories for jobs in a same way as for transformations (CRQ
#4647)
- Make kettle variables work in all cases (bug #5392 e.g.)
- Bug #4965: Filter step does not consider to be "" equal to NULL
- Extend carte: jobs, timed execution... so it becomes something of a
headless spoon. Where people can logon to and execute things, or look at log
files, ... maybe make it able to publish jobs as webservices.
- Include version of step/job entry per instance. Version number is 1 number
(number 1 can be forgotten). Include version of saving version at
transformation level as well (CRQ #2384 e.g.)
- Support for Oracle RAC connections (CRQ #4403), this should be smallish.
- CRQ #4988: include "include" filter on text file input step. This allows
for a more easy way of splitting multi-level files.
- CRQ #4839: upgrade javascript support + include E4X
- CRQ #4362: access LDAP (CRQ #1961)
- DB2 bulkloader?
- CRQ #5446: LDAP authentication

Personal planning SB (for now)
-------------------------------
- CRQ #5556: Extend Blocking Step do optionally pass all rows through
- CRQ #4620: Concatenate step
- CRQ #5524: Case-Sensitive UniqueRows cannot rely on output of SortRows +
optimize sort rows
maybe also make unique rows optionally non case
sensitive.
- More unit testcases
- Gold plate/optimize/cleanup some steps.

And what I personally would also like is a small execution engine for PDI
that takes care of high avalability which some extra things....
I'll try something later on the year outside of the kettle trunk (I
currently don't see a way to put it in Carte anymore ;-)).
Requirements for me:
- Per project I want to put 1 line in a HA crontab. That executable should
do scheduling for the project itself of jobs. It should also take care of
automatic restarting of jobs on fail-over, execute missing job executions,
....
- Everything in the engine should be file based so I can put the project
itself in SVN
- People being able to adhoc restart jobs and to watch log files (something
like carted does now)

Regards,
Sven





--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
To unsubscribe from this group, send email to kettle-developers-unsubscribe (AT) g...oups (DOT) com
For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
-~----------~----~----~----~------~----~------~--~---