Hitachi Vantara Pentaho Community Forums
Results 1 to 14 of 14

Thread: Kettle 4 - Exposing step metadata in Spoon

  1. #1
    Matt Casters Guest

    Default Kettle 4 - Exposing step metadata in Spoon

    Hi Kettle devs,

    Some time ago we discussed on the EE customer forum how we could improve the user experience in Spoon wrt the transformations and jobs.
    A few recommendations from back then were implemented in the form of the new transformation hop icons.

    For the rest, we need to change the StepMetaInterface.
    This is represented in PDI-2766. Feel free to add comments with respect to the API over there.

    Since we can't wait forever until we release 4.0 and I don't want to release 5.0 in another 6 months, I got started on this work.
    Early feedback is the best, so here is the first stab at it.

    http://www.youtube.com/watch?v=EJHyjwy6-sc

    The idea is that when you mouse over a step, you get indicators as to how you can attach the step to other steps.
    In the sample, the "CSV Input" step accepts input (filenames) and produces output (rows of CSV data).

    You can click on the "output" icon to connect to another step.
    So you no longer need to SHIFT-click drag or drag the middle mouse button.
    In fact you no longer need to drag the mouse anymore either (even though you still can).

    Next on the agenda is exposing the type of input/output that is possible for each step.
    Please keep in mind that the positioning of the mini-icons, the icons themselves are all up for discussion.
    I just put these in to get a feel for the API that is needed to get this show on the road.

    For the info steps I'm thinking about showing the possibilities at the top of the step.
    The target steps are harder since they are always driven by the step itself.
    Take for example the "Switch/Case" step. That one allows N target output steps.
    There is a different step-specific condition for each target. Unless the conditions are lined up, you can't really allow the user to pick a target.

    Please let us know what you think and help us come up with a cool solution for these things.
    I'll make sure it gets in 4.0 somewhere this week. ;-)

    Cheers,
    Matt


    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  2. #2
    Daniel E Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    I believe for the general input and output targets,an icon of a funnel
    with an arrow going into the top or out of the bottom might be pretty
    clear.

    For a named input like the stream input step, maybe the input funnel,
    but with the blue circle i that we use on the hop to distinguish it?

    Same for an error output. The output funnel with the red circle x.


    Switch/Case certainly is the hardest. The best solution I can think
    of at the moment is fairly fancy, I don't know how complex it would be
    to implement. If you could have an output funnel with an ellipsis
    (...), then when the user clicked it, maybe we could expand to a set
    of named output funnel icons, each one with the string from in the
    condition field of the step displayed next to it.

    Same thing applies to a true/false filter. You could display two
    named output funnel icons, one with "true" and one with "false".


    I really like the demo video Matt, it feels very comfortable to me.

    On Oct 12, 3:55

  3. #3
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Hi,

    I continued a bit further along the same path. Have a look at this video:

    http://www.youtube.com/watch?v=h1sib2DGSfM

    The mini-icons themselves are things we can easily change.
    What I did was the following: if you mouse over an icon, you will see all input and output "stream" possibilities... for about 3 seconds and then they disappear again.
    For example, at the end of the short video I waited for 3 seconds before making up my mind and the mini-icons went away.

    If a step like "Stream Lookup" has a special "info" stream, you will see an (i) icon at the top of the step.
    If a step supports error handling, you will see an error icon at the top too.

    Then it's simply a matter of connecting the dots.

    I've been simplifying the StepMetaInterface and StepIOMetaInterface to make this happen.
    The downside is that I have to change steps like "Merge Rows", "Merge Join", etc.
    I guess when I'll be done it will be better :-)
    In the mean time things should still work as it should.

    I'll try to tackle "If" and "Switch/Case" later to see where we could end up.
    For now, play with revision 11027 and let me know what you think.
    Thanks for all the feedback!

    Cheers,
    Matt

    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence



    On Monday 12 October 2009 22:34:12 Daniel E wrote:
    >
    > I believe for the general input and output targets,an icon of a funnel
    > with an arrow going into the top or out of the bottom might be pretty
    > clear.
    >
    > For a named input like the stream input step, maybe the input funnel,
    > but with the blue circle i that we use on the hop to distinguish it?
    >
    > Same for an error output. The output funnel with the red circle x.
    >
    >
    > Switch/Case certainly is the hardest. The best solution I can think
    > of at the moment is fairly fancy, I don't know how complex it would be
    > to implement. If you could have an output funnel with an ellipsis
    > (...), then when the user clicked it, maybe we could expand to a set
    > of named output funnel icons, each one with the string from in the
    > condition field of the step displayed next to it.
    >
    > Same thing applies to a true/false filter. You could display two
    > named output funnel icons, one with "true" and one with "false".
    >
    >
    > I really like the demo video Matt, it feels very comfortable to me.
    >
    > On Oct 12, 3:55 pm, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > Hi Kettle devs,
    > >
    > > Some time ago we discussed on the EE customer forum how we could improve the user experience in Spoon wrt the transformations and jobs.
    > > A few recommendations from back then were implemented in the form of the new transformation hop icons.
    > >
    > > For the rest, we need to change the StepMetaInterface.
    > > This is represented in PDI-2766. Feel free to add comments with respect to the API over there.
    > >
    > > Since we can't wait forever until we release 4.0 and I don't want to release 5.0 in another 6 months, I got started on this work.
    > > Early feedback is the best, so here is the first stab at it.
    > >
    > > http://www.youtube.com/watch?v=EJHyjwy6-sc
    > >
    > > The idea is that when you mouse over a step, you get indicators as to how you can attach the step to other steps.
    > > In the sample, the "CSV Input" step accepts input (filenames) and produces output (rows of CSV data).
    > >
    > > You can click on the "output" icon to connect to another step.
    > > So you no longer need to SHIFT-click drag or drag the middle mouse button.
    > > In fact you no longer need to drag the mouse anymore either (even though you still can).
    > >
    > > Next on the agenda is exposing the type of input/output that is possible for each step.
    > > Please keep in mind that the positioning of the mini-icons, the icons themselves are all up for discussion.
    > > I just put these in to get a feel for the API that is needed to get this show on the road.
    > >
    > > For the info steps I'm thinking about showing the possibilities at the top of the step.
    > > The target steps are harder since they are always driven by the step itself.
    > > Take for example the "Switch/Case" step. That one allows N target output steps.
    > > There is a different step-specific condition for each target. Unless the conditions are lined up, you can't really allow the user to pick a target.
    > >
    > > Please let us know what you think and help us come up with a cool solution for these things.
    > > I'll make sure it gets in 4.0 somewhere this week. ;-)
    > >
    > > Cheers,
    > > Matt
    > >
    > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > Chief Data Integration
    > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > Pentaho : The Commercial Open Source Alternative for Business Intelligence
    > >

    >
    >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  4. #4
    Daniel E Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Matt,

    This is excellent work. Like you said, the icons can be easily
    changed (I still vote for the funnels though!

    The "connect-the-dots" approach looks and feels very natural to me. I
    think this is going to be a great improvement to usability.
    What do you think should happen if the user tries to connect more than
    one step to a named connection like a stream lookup or an error
    output?

    On Oct 14, 8:57

  5. #5
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    > This is excellent work. Like you said, the icons can be easily
    > changed (I still vote for the funnels though!


    Feel free to send me an icon if you have one. We can drop it in and see how it looks.

    > What do you think should happen if the user tries to connect more than
    > one step to a named connection like a stream lookup or an error
    > output?


    Actually, that happens more often simply by mistake and I tested this.
    What happens is that internally there is only one spot reserved for the Info hop, so you can't allocate 2.
    So the other hop simply replaces the first and all is well.
    In the specific case of "Stream Lookup", data and lookup are switched.

    I think that's pretty much the way it should be.

    Cheers,
    Matt

    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence



    On Wednesday 14 October 2009 16:10:41 Daniel E wrote:
    >
    > Matt,
    >
    > This is excellent work. Like you said, the icons can be easily
    > changed (I still vote for the funnels though!
    >
    > The "connect-the-dots" approach looks and feels very natural to me. I
    > think this is going to be a great improvement to usability.
    > What do you think should happen if the user tries to connect more than
    > one step to a named connection like a stream lookup or an error
    > output?
    >
    > On Oct 14, 8:57 am, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > Hi,
    > >
    > > I continued a bit further along the same path. Have a look at this video:
    > >
    > > http://www.youtube.com/watch?v=h1sib2DGSfM
    > >
    > > The mini-icons themselves are things we can easily change.
    > > What I did was the following: if you mouse over an icon, you will see all input and output "stream" possibilities... for about 3 seconds and then they disappear again.
    > > For example, at the end of the short video I waited for 3 seconds before making up my mind and the mini-icons went away.
    > >
    > > If a step like "Stream Lookup" has a special "info" stream, you will see an (i) icon at the top of the step.
    > > If a step supports error handling, you will see an error icon at the top too.
    > >
    > > Then it's simply a matter of connecting the dots.
    > >
    > > I've been simplifying the StepMetaInterface and StepIOMetaInterface to make this happen.
    > > The downside is that I have to change steps like "Merge Rows", "Merge Join", etc.
    > > I guess when I'll be done it will be better :-)
    > > In the mean time things should still work as it should.
    > >
    > > I'll try to tackle "If" and "Switch/Case" later to see where we could end up.
    > > For now, play with revision 11027 and let me know what you think.
    > > Thanks for all the feedback!
    > >
    > > Cheers,
    > > Matt
    > >
    > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > Chief Data Integration
    > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > Pentaho : The Commercial Open Source Alternative for Business Intelligence
    > >
    > > On Monday 12 October 2009 22:34:12 Daniel E wrote:
    > >
    > >
    > >
    > > > I believe for the general input and output targets,an icon of a funnel
    > > > with an arrow going into the top or out of the bottom might be pretty
    > > > clear.

    > >
    > > > For a named input like the stream input step, maybe the input funnel,
    > > > but with the blue circle i that we use on the hop to distinguish it?

    > >
    > > > Same for an error output. The output funnel with the red circle x.

    > >
    > > > Switch/Case certainly is the hardest. The best solution I can think
    > > > of at the moment is fairly fancy, I don't know how complex it would be
    > > > to implement. If you could have an output funnel with an ellipsis
    > > > (...), then when the user clicked it, maybe we could expand to a set
    > > > of named output funnel icons, each one with the string from in the
    > > > condition field of the step displayed next to it.

    > >
    > > > Same thing applies to a true/false filter. You could display two
    > > > named output funnel icons, one with "true" and one with "false".

    > >
    > > > I really like the demo video Matt, it feels very comfortable to me.

    > >
    > > > On Oct 12, 3:55 pm, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > > > Hi Kettle devs,

    > >
    > > > > Some time ago we discussed on the EE customer forum how we could improve the user experience in Spoon wrt the transformations and jobs.
    > > > > A few recommendations from back then were implemented in the form of the new transformation hop icons.

    > >
    > > > > For the rest, we need to change the StepMetaInterface.
    > > > > This is represented in PDI-2766. Feel free to add comments with respect to the API over there.

    > >
    > > > > Since we can't wait forever until we release 4.0 and I don't want to release 5.0 in another 6 months, I got started on this work.
    > > > > Early feedback is the best, so here is the first stab at it.

    > >
    > > > >http://www.youtube.com/watch?v=EJHyjwy6-sc

    > >
    > > > > The idea is that when you mouse over a step, you get indicators as to how you can attach the step to other steps.
    > > > > In the sample, the "CSV Input" step accepts input (filenames) and produces output (rows of CSV data).

    > >
    > > > > You can click on the "output" icon to connect to another step.
    > > > > So you no longer need to SHIFT-click drag or drag the middle mouse button.
    > > > > In fact you no longer need to drag the mouse anymore either (even though you still can).

    > >
    > > > > Next on the agenda is exposing the type of input/output that is possible for each step.
    > > > > Please keep in mind that the positioning of the mini-icons, the icons themselves are all up for discussion.
    > > > > I just put these in to get a feel for the API that is needed to get this show on the road.

    > >
    > > > > For the info steps I'm thinking about showing the possibilities at the top of the step.
    > > > > The target steps are harder since they are always driven by the step itself.
    > > > > Take for example the "Switch/Case" step. That one allows N target output steps.
    > > > > There is a different step-specific condition for each target. Unless the conditions are lined up, you can't really allow the user to pick a target.

    > >
    > > > > Please let us know what you think and help us come up with a cool solution for these things.
    > > > > I'll make sure it gets in 4.0 somewhere this week. ;-)

    > >
    > > > > Cheers,
    > > > > Matt

    > >
    > > > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > > > Chief Data Integration
    > > > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > > > Pentaho : The Commercial Open Source Alternative for Business Intelligence

    > >
    > >
    > >

    >
    >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  6. #6
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    A more complex info-stream example from the "Data Validator" step.
    That one is capable of reading an info-stream of data per validation.

    http://www.youtube.com/watch?v=1sDevb3kJ6g

    So the number of info-steps is unknown up-front.

    Cheers,
    Matt

    On Wednesday 14 October 2009 16:10:41 Daniel E wrote:
    >
    > Matt,
    >
    > This is excellent work. Like you said, the icons can be easily
    > changed (I still vote for the funnels though!
    >
    > The "connect-the-dots" approach looks and feels very natural to me. I
    > think this is going to be a great improvement to usability.
    > What do you think should happen if the user tries to connect more than
    > one step to a named connection like a stream lookup or an error
    > output?
    >
    > On Oct 14, 8:57 am, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > Hi,
    > >
    > > I continued a bit further along the same path. Have a look at this video:
    > >
    > > http://www.youtube.com/watch?v=h1sib2DGSfM
    > >
    > > The mini-icons themselves are things we can easily change.
    > > What I did was the following: if you mouse over an icon, you will see all input and output "stream" possibilities... for about 3 seconds and then they disappear again.
    > > For example, at the end of the short video I waited for 3 seconds before making up my mind and the mini-icons went away.
    > >
    > > If a step like "Stream Lookup" has a special "info" stream, you will see an (i) icon at the top of the step.
    > > If a step supports error handling, you will see an error icon at the top too.
    > >
    > > Then it's simply a matter of connecting the dots.
    > >
    > > I've been simplifying the StepMetaInterface and StepIOMetaInterface to make this happen.
    > > The downside is that I have to change steps like "Merge Rows", "Merge Join", etc.
    > > I guess when I'll be done it will be better :-)
    > > In the mean time things should still work as it should.
    > >
    > > I'll try to tackle "If" and "Switch/Case" later to see where we could end up.
    > > For now, play with revision 11027 and let me know what you think.
    > > Thanks for all the feedback!
    > >
    > > Cheers,
    > > Matt
    > >
    > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > Chief Data Integration
    > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > Pentaho : The Commercial Open Source Alternative for Business Intelligence
    > >
    > > On Monday 12 October 2009 22:34:12 Daniel E wrote:
    > >
    > >
    > >
    > > > I believe for the general input and output targets,an icon of a funnel
    > > > with an arrow going into the top or out of the bottom might be pretty
    > > > clear.

    > >
    > > > For a named input like the stream input step, maybe the input funnel,
    > > > but with the blue circle i that we use on the hop to distinguish it?

    > >
    > > > Same for an error output. The output funnel with the red circle x.

    > >
    > > > Switch/Case certainly is the hardest. The best solution I can think
    > > > of at the moment is fairly fancy, I don't know how complex it would be
    > > > to implement. If you could have an output funnel with an ellipsis
    > > > (...), then when the user clicked it, maybe we could expand to a set
    > > > of named output funnel icons, each one with the string from in the
    > > > condition field of the step displayed next to it.

    > >
    > > > Same thing applies to a true/false filter. You could display two
    > > > named output funnel icons, one with "true" and one with "false".

    > >
    > > > I really like the demo video Matt, it feels very comfortable to me.

    > >
    > > > On Oct 12, 3:55 pm, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > > > Hi Kettle devs,

    > >
    > > > > Some time ago we discussed on the EE customer forum how we could improve the user experience in Spoon wrt the transformations and jobs.
    > > > > A few recommendations from back then were implemented in the form of the new transformation hop icons.

    > >
    > > > > For the rest, we need to change the StepMetaInterface.
    > > > > This is represented in PDI-2766. Feel free to add comments with respect to the API over there.

    > >
    > > > > Since we can't wait forever until we release 4.0 and I don't want to release 5.0 in another 6 months, I got started on this work.
    > > > > Early feedback is the best, so here is the first stab at it.

    > >
    > > > >http://www.youtube.com/watch?v=EJHyjwy6-sc

    > >
    > > > > The idea is that when you mouse over a step, you get indicators as to how you can attach the step to other steps.
    > > > > In the sample, the "CSV Input" step accepts input (filenames) and produces output (rows of CSV data).

    > >
    > > > > You can click on the "output" icon to connect to another step.
    > > > > So you no longer need to SHIFT-click drag or drag the middle mouse button.
    > > > > In fact you no longer need to drag the mouse anymore either (even though you still can).

    > >
    > > > > Next on the agenda is exposing the type of input/output that is possible for each step.
    > > > > Please keep in mind that the positioning of the mini-icons, the icons themselves are all up for discussion.
    > > > > I just put these in to get a feel for the API that is needed to get this show on the road.

    > >
    > > > > For the info steps I'm thinking about showing the possibilities at the top of the step.
    > > > > The target steps are harder since they are always driven by the step itself.
    > > > > Take for example the "Switch/Case" step. That one allows N target output steps.
    > > > > There is a different step-specific condition for each target. Unless the conditions are lined up, you can't really allow the user to pick a target.

    > >
    > > > > Please let us know what you think and help us come up with a cool solution for these things.
    > > > > I'll make sure it gets in 4.0 somewhere this week. ;-)

    > >
    > > > > Cheers,
    > > > > Matt

    > >
    > > > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > > > Chief Data Integration
    > > > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > > > Pentaho : The Commercial Open Source Alternative for Business Intelligence

    > >
    > >
    > >

    >
    >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  7. #7
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    And an attempt to get "Target" hops working:

    http://www.youtube.com/watch?v=ZaYz9Hqggcg

    Cheers,
    Matt


    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  8. #8
    Roland Bouman Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Looks great Matt, kudos!!

    On Wed, Oct 14, 2009 at 8:22 PM, Matt Casters <mcasters (AT) pentaho (DOT) org> wrote:
    >
    >
    > And an attempt to get "Target" hops working:
    >
    > http://www.youtube.com/watch?v=ZaYz9Hqggcg
    >
    > Cheers,
    > Matt
    >
    >
    > >

    >




    --
    Roland Bouman
    http://rpbouman.blogspot.com/

    Author of "Pentaho Solutions: Business Intelligence and Data
    Warehousing with Pentaho and MySQL",
    http://www.wiley.com/WileyCDA/WileyT...470484322.html

    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  9. #9
    Paul Stoellberger Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Yeah, looks great!!!

    If i can make a wish, I'd really like to see the read / write row
    count next to each input/output hop on runtime or after finished run.
    would look really nice and i don't think its that hard to do, but I'm
    just guessing.

    Looking forward to Kettle 4 ;-)

    -paul

    On 14 Oct 2009, at 20:36, Roland Bouman wrote:

    >
    > Looks great Matt, kudos!!
    >
    > On Wed, Oct 14, 2009 at 8:22 PM, Matt Casters <mcasters (AT) pentaho (DOT) org>
    > wrote:
    >>
    >>
    >> And an attempt to get "Target" hops working:
    >>
    >> http://www.youtube.com/watch?v=ZaYz9Hqggcg
    >>
    >> Cheers,
    >> Matt
    >>
    >>
    >>>

    >>

    >
    >
    >
    > --
    > Roland Bouman
    > http://rpbouman.blogspot.com/
    >
    > Author of "Pentaho Solutions: Business Intelligence and Data
    > Warehousing with Pentaho and MySQL",
    > http://www.wiley.com/WileyCDA/WileyT...470484322.html
    >
    > >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

  10. #10
    samatar hassan Guest

    Default Re : Kettle 4 - Exposing step metadata in Spoon

    Great :-)

    Paul have a nice wish (available in another ETL tool).

    Is it hard to show also in runtime which job entry (s) is currently running?
    Some like of highlight.

    Samatar




    ________________________________
    De : Paul Stoellberger <paul.stoellberger (AT) aschauer-edv (DOT) at>

  11. #11
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Should be all pretty simple. Create a timer that refreshes every X seconds or when a job entry switches location.
    Then hit the canvas with a refresh like in TransGraph.asyncRedraw().

    In the transformations we have to look out that we don't do stupid things where we slow down execution so much that the "bling bling things" become unusable. (like in the "other" ETL tool)

    Cheers,
    Matt

    Matt Casters <mcasters (AT) pentaho (DOT) org>
    Chief Data Integration
    Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    Pentaho : The Commercial Open Source Alternative for Business Intelligence



    On Wednesday 14 October 2009 23:00:24 samatar hassan wrote:[color=blue]
    > Great :-)
    >
    > Paul have a nice wish (available in another ETL tool).
    >
    > Is it hard to show also in runtime which job entry (s) is currently running?
    > Some like of highlight.
    >
    > Samatar
    >
    >
    >
    >
    > ________________________________
    > De : Paul Stoellberger <paul.stoellberger (AT) aschauer-edv (DOT) at>
    >

  12. #12
    samatar hassan Guest

    Default Re : Kettle 4 - Exposing step metadata in Spoon

    Nice to hear,

    transformations got a lot of visual improvements and job entries deserve the same thing :-)
    I guess one important

  13. #13
    Benjamin Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Hi Matt,

    first of all, excellent work! This makes designing a lot easier!

    Some more suggestions:

    What about steps which need two input to run (Merge Join, Merge Rows,
    Append Streams, etc.)? The behavior of the steps is different
    depending on which stream is treated as first or second input (e.g.
    the append stream result will be different if one marks a stream as
    head or tail). It would be nice to have the option to connect a stream
    as either first or second input and to have another icon which allows
    easy switching (maybe a icon with a doubled arrow or something like
    that?)

    Just my 2 cents... I'm really looking forward to see 4.0 becoming GA.


    Thanks again,
    all the best!

    Ben

    On 14 Okt., 14:57, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:[color=blue]
    > Hi,
    >
    > I continued a bit further along the same path.

  14. #14
    Matt Casters Guest

    Default Re: Kettle 4 - Exposing step metadata in Spoon

    Hi Ben,

    That's exactly what video 3 shows you:

    http://www.youtube.com/watch?v=1sDevb3kJ6g

    For each info stream (inputs) you get a separate icon (for now).
    In the case of the "Data Validator" step, it's not fixed to 1 or 2 since you can have any number of validations in there.

    In the API I made provisions for the descriptions of the info streams.
    In the case of Merge Join for example it says:

    * Left hand side stream of the join
    * Right hand side stream of the join

    I do agree that the tooltips that are given are not sufficiently clear
    At this moment I'm thinking about overlaying the complete step with a grid or something.
    You can do nice things with alpha channels fading in and out information, etc.

    But again, the UI part is not that important, the UI engineers in Orlando should take a look at it.
    They are Good(tm) at this sort of thing.

    I'm confident that the API is in there and that the system is usable, that's enough for me at the moment.

    As for 4.0GA, we're still a long way from that point. In fact, looking at the internal plans it seems like we've only just begun.
    There is a strong desire from the side of my employer to include all sorts of new features and parts of the rest of the stack in 4.0 (CE & EE).

    Cheers,
    Matt


    On Thursday 15 October 2009 13:49:30 Benjamin wrote:
    >
    > Hi Matt,
    >
    > first of all, excellent work! This makes designing a lot easier!
    >
    > Some more suggestions:
    >
    > What about steps which need two input to run (Merge Join, Merge Rows,
    > Append Streams, etc.)? The behavior of the steps is different
    > depending on which stream is treated as first or second input (e.g.
    > the append stream result will be different if one marks a stream as
    > head or tail). It would be nice to have the option to connect a stream
    > as either first or second input and to have another icon which allows
    > easy switching (maybe a icon with a doubled arrow or something like
    > that?)
    >
    > Just my 2 cents... I'm really looking forward to see 4.0 becoming GA.
    >
    >
    > Thanks again,
    > all the best!
    >
    > Ben
    >
    > On 14 Okt., 14:57, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > Hi,
    > >
    > > I continued a bit further along the same path. Have a look at this video:
    > >
    > > http://www.youtube.com/watch?v=h1sib2DGSfM
    > >
    > > The mini-icons themselves are things we can easily change.
    > > What I did was the following: if you mouse over an icon, you will see all input and output "stream" possibilities... for about 3 seconds and then they disappear again.
    > > For example, at the end of the short video I waited for 3 seconds before making up my mind and the mini-icons went away.
    > >
    > > If a step like "Stream Lookup" has a special "info" stream, you will see an (i) icon at the top of the step.
    > > If a step supports error handling, you will see an error icon at the top too.
    > >
    > > Then it's simply a matter of connecting the dots.
    > >
    > > I've been simplifying the StepMetaInterface and StepIOMetaInterface to make this happen.
    > > The downside is that I have to change steps like "Merge Rows", "Merge Join", etc.
    > > I guess when I'll be done it will be better :-)
    > > In the mean time things should still work as it should.
    > >
    > > I'll try to tackle "If" and "Switch/Case" later to see where we could end up.
    > > For now, play with revision 11027 and let me know what you think.
    > > Thanks for all the feedback!
    > >
    > > Cheers,
    > > Matt
    > >
    > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > Chief Data Integration
    > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > Pentaho : The Commercial Open Source Alternative for Business Intelligence
    > >
    > > On Monday 12 October 2009 22:34:12 Daniel E wrote:
    > >
    > >
    > >
    > > > I believe for the general input and output targets,an icon of a funnel
    > > > with an arrow going into the top or out of the bottom might be pretty
    > > > clear.

    > >
    > > > For a named input like the stream input step, maybe the input funnel,
    > > > but with the blue circle i that we use on the hop to distinguish it?

    > >
    > > > Same for an error output. The output funnel with the red circle x.

    > >
    > > > Switch/Case certainly is the hardest. The best solution I can think
    > > > of at the moment is fairly fancy, I don't know how complex it would be
    > > > to implement. If you could have an output funnel with an ellipsis
    > > > (...), then when the user clicked it, maybe we could expand to a set
    > > > of named output funnel icons, each one with the string from in the
    > > > condition field of the step displayed next to it.

    > >
    > > > Same thing applies to a true/false filter. You could display two
    > > > named output funnel icons, one with "true" and one with "false".

    > >
    > > > I really like the demo video Matt, it feels very comfortable to me.

    > >
    > > > On Oct 12, 3:55 pm, Matt Casters <mcast... (AT) pentaho (DOT) org> wrote:
    > > > > Hi Kettle devs,

    > >
    > > > > Some time ago we discussed on the EE customer forum how we could improve the user experience in Spoon wrt the transformations and jobs.
    > > > > A few recommendations from back then were implemented in the form of the new transformation hop icons.

    > >
    > > > > For the rest, we need to change the StepMetaInterface.
    > > > > This is represented in PDI-2766. Feel free to add comments with respect to the API over there.

    > >
    > > > > Since we can't wait forever until we release 4.0 and I don't want to release 5.0 in another 6 months, I got started on this work.
    > > > > Early feedback is the best, so here is the first stab at it.

    > >
    > > > >http://www.youtube.com/watch?v=EJHyjwy6-sc

    > >
    > > > > The idea is that when you mouse over a step, you get indicators as to how you can attach the step to other steps.
    > > > > In the sample, the "CSV Input" step accepts input (filenames) and produces output (rows of CSV data).

    > >
    > > > > You can click on the "output" icon to connect to another step.
    > > > > So you no longer need to SHIFT-click drag or drag the middle mouse button.
    > > > > In fact you no longer need to drag the mouse anymore either (even though you still can).

    > >
    > > > > Next on the agenda is exposing the type of input/output that is possible for each step.
    > > > > Please keep in mind that the positioning of the mini-icons, the icons themselves are all up for discussion.
    > > > > I just put these in to get a feel for the API that is needed to get this show on the road.

    > >
    > > > > For the info steps I'm thinking about showing the possibilities at the top of the step.
    > > > > The target steps are harder since they are always driven by the step itself.
    > > > > Take for example the "Switch/Case" step. That one allows N target output steps.
    > > > > There is a different step-specific condition for each target. Unless the conditions are lined up, you can't really allow the user to pick a target.

    > >
    > > > > Please let us know what you think and help us come up with a cool solution for these things.
    > > > > I'll make sure it gets in 4.0 somewhere this week. ;-)

    > >
    > > > > Cheers,
    > > > > Matt

    > >
    > > > > Matt Casters <mcast... (AT) pentaho (DOT) org>
    > > > > Chief Data Integration
    > > > > Fonteinstraat 70, 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
    > > > > Pentaho : The Commercial Open Source Alternative for Business Intelligence

    > >

    >
    >



    --~--~---------~--~----~------------~-------~--~----~
    You received this message because you are subscribed to the Google Groups "kettle-developers" group.
    To post to this group, send email to kettle-developers (AT) googlegroups (DOT) com
    To unsubscribe from this group, send email to kettle-developers+unsubscribe (AT) googlegroups (DOT) com
    For more options, visit this group at http://groups.google.com/group/kettle-developers?hl=en
    -~----------~----~----~----~------~----~------~--~---

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.