Hitachi Vantara Pentaho Community Forums

Search Forums:

Type: Posts; User: Sparkles; Keyword(s):

Page 1 of 6 1 2 3 4

Search Forums: Search took 0.03 seconds.

  1. Replies
    1
    Views
    325

    Just a side note: If you usenotepad++ you can...

    Just a side note: If you usenotepad++ you can search text in files and folders (more relevant for file repos, git etc).
  2. Is this the only consequence? I'm tempted to...

    Is this the only consequence? I'm tempted to delete this big plugin because I don't need it. I also don't need repository or any button for it, as I use git for version history.
  3. Replies
    6
    Views
    712

    Thanks for reply! Version used is 5.4.0.1-130....

    Thanks for reply! Version used is 5.4.0.1-130. The transformation is executed by kitchen.sh which is scheduled and executed by a service, but it could just as well have been a cron job. This is not a...
  4. Replies
    6
    Views
    712

    Turns out setOutputDone() did NOT stop the Memory...

    Turns out setOutputDone() did NOT stop the Memory Group By error. It still happens a couple of times per day randomly. It's still useful for immediately disabling steps. The Memory Group By error...
  5. Replies
    6
    Views
    712

    Thanks! Always useful. Anyway, so far it seems...

    Thanks! Always useful.

    Anyway, so far it seems like the combo of return true and setOutputDone() solved my problem. The bug hasn't happened again, I will keep monitor to see if it's gone for good....
  6. Replies
    6
    Views
    712

    Group By / Return Value cannot be found

    I have a unique situation which is quite complicated, but let's give it a try. This applies to a transformation I made which reads fact rows from files, applies various filters and calculations, then...
  7. Replies
    2
    Views
    357

    If you want to reference a variable in Spoon, you...

    If you want to reference a variable in Spoon, you must use:

    ${variable-name}

    Variables inside a transformation only has a single value, they cannot change value while the transformation is...
  8. Replies
    3
    Views
    430

    Using a log table is new to me. The built-in log...

    Using a log table is new to me. The built-in log system is way too verbose and messy for my taste. Logging is critical to my use, I need to know exactly what has been done and when. I also need to...
  9. Replies
    4
    Views
    485

    Thanks for sharing your thoughts. It did cross my...

    Thanks for sharing your thoughts. It did cross my mind. But as you point out, the problem is code duplication and could lead to high amount of transformation files for multiple combinations.
  10. Replies
    4
    Views
    485

    I think you're right. That should most probably...

    I think you're right. That should most probably work without affecting performance. It still bothers me that a sub-stream can't be shut off immediately. Your solution looks like middleground when it...
  11. Replies
    4
    Views
    485

    Disabling a branch in a transformation

    After experimenting with some "improved" stream/branch disable functionality, my big data transformation now suffers from congestion and comes to a halt! The transformation reads a single file and...
  12. Replies
    6
    Views
    512

    Thanks a bunch! With some useful tips from you...

    Thanks a bunch!

    With some useful tips from you guys I managed to go around the problem, either checking for <null> rows or setting a variable if no rows were copied to result. Now I am aware of...
  13. Replies
    6
    Views
    512

    I made a ticket for jira here:...

    I made a ticket for jira here:
    https://jira.pentaho.com/browse/PDI-17926

    Got a reply saying that the issue was not reproduced. I added an example, but I'm currently using an older version...
  14. The "problem" with Spoon is variables can...

    The "problem" with Spoon is variables can sometimes live between executions and transfer between different jobs/transformations. If you want to test in Spoon, you need to close and re-open Spoon...
  15. Where are the variables originating from? You...

    Where are the variables originating from? You need to log the variables step by step. Start with where it's created and move your way up to the destination.
  16. There have been other discussions pointing out...

    There have been other discussions pointing out some bugs in 7.1 which makes transformation start slowly:

    https://forums.pentaho.com/threads/229619-Spoon-takes-insanely-long-time-to-start/
  17. Did you see this discussion? ...

    Did you see this discussion?

    https://forums.pentaho.com/threads/219905-Connect-Repository-Button-is-not-visble-in-my-Pentaho-Community-Edition/
  18. Sorry I don't have anything useful to come with...

    Sorry I don't have anything useful to come with :p I'll have a look around if I find that earlier discussion on this topic.
  19. Yes I had that article in mind, there's also been...

    Yes I had that article in mind, there's also been other discussions on the topic I think.
  20. So the problem is slow execution / startup...

    So the problem is slow execution / startup regardless of what version control is used?
  21. I don't understand exactly, are you executing...

    I don't understand exactly, are you executing code located on the same computer as kettle, or are you executing code externally from a git repository?
  22. That looks good.

    That looks good.
  23. This is the sort of thing which is easy and fun...

    This is the sort of thing which is easy and fun to do in java (and much faster than Cartesian join which must wait for all rows before producing output).

    Example:


    Just remember the imports:
    ...
  24. You downgraded from 8.2 to 8.1 because of a bug?...

    You downgraded from 8.2 to 8.1 because of a bug? I've heard others upgrade from 8.1 to 8.2 because of bugs...

    I'm still on 5.4, not sure if there's anything to gain by updating.
  25. Replies
    6
    Views
    512

    That sounds good exactly how you describe it...

    That sounds good exactly how you describe it there. I created a jira ticket.

    It may very well not be intented for these sorts of things, but it is VERY common to have some sort of list/array in...
  26. Replies
    6
    Views
    512

    I have never found any satisfying documentation...

    I have never found any satisfying documentation for the dirty details of result rows. It seems like nobody knows exactly how this works. Only way to find out is test yourself. That's what I did. What...
  27. Replies
    2
    Views
    281

    It could be potentially bad for performance, each...

    It could be potentially bad for performance, each step should be more "work". But removing fields could give great boost to some steps also. It can often help readability and easier debugging to...
  28. Hasn't there been several discussions about bugs...

    Hasn't there been several discussions about bugs in version 8? I would try the latest version before looking for differences or bugs.
  29. Thread: Longest match

    by Sparkles
    Replies
    4
    Views
    374

    I like how you think! I had one attempt where I...

    I like how you think! I had one attempt where I compared all possibilities (in this case, using a max string length of 15 characters). So first compare any incoming data with any info-list of length...
  30. Thread: Longest match

    by Sparkles
    Replies
    4
    Views
    374

    Longest match

    Trying to process a stream of data by comparing a field to a info-list, then returning an ID from this info-list (if a match is found). So far so good, a fairly simply task. However, this lookup must...
  31. Replies
    6
    Views
    420

    I don't remember exactly cause I had to discard...

    I don't remember exactly cause I had to discard it, but it was some simple mathematical expression with rounding. A Calculator step was not useful here because it was the sort of calculation which...
  32. Replies
    6
    Views
    420

    A little side-note: I found the Formula step to...

    A little side-note: I found the Formula step to give poor performance on big data. Not sure if this was because I'm using an older version or perhaps the formula I used was long/complicated.
  33. Replies
    1
    Views
    261

    What method are you trying to pass the two byte...

    What method are you trying to pass the two byte arguments to? Is this method declared with 2 parameters?
  34. Replies
    1
    Views
    249

    Yes I use external jar in udjc (user defined java...

    Yes I use external jar in udjc (user defined java class) step (on version 5.4.0.1-130).

    As far as I remember, I only needed to:
    1) put .jar in /data-integration/lib folder
    2) declare correct...
  35. That's very interesting, thanks! Positive:...

    That's very interesting, thanks!

    Positive:
    -Don't need dummy logic
    -More intuitive execution with rows in stream triggering sql (as opposed to the info steps)

    Negative:
    -Large SQL query...
  36. Before doing anything with emails, you have to...

    Before doing anything with emails, you have to understand how variables are working!


    In a job, create 2 transformations:

    Transformation nr. 1: Create the variable with the value you want. Use...
  37. Yes, good clarifications. I must insist that...

    Yes, good clarifications.

    I must insist that before trying to use variables in email step/job (either in job or transformation), first do a simple demo where the variable is created and printed to...
  38. Yes, in normal situations. But this case is...

    Yes, in normal situations. But this case is actually a sub-transformation (the parent is also a transformation). I used "Simple Mapping" and noticed the sub-transformation would always be executed.
  39. Sorry I have no experience with email or...

    Sorry I have no experience with email or attaching images. I can only help with variables and changing file names.
  40. Yes, thanks for reply. I don't want something...

    Yes, thanks for reply. I don't want something ugly as you mention, but to have something of the type you describe at the end would be good. I'm not in a position to contribute in the source code at...
  41. Sorry, I was not accurate in how I described it....

    Sorry, I was not accurate in how I described it. What I meant was, I often have to do these simple hacks in order to get the behavior I want. I realize as you point out some of these steps are aimed...
  42. You can certainly create a variable with the...

    You can certainly create a variable with the value equal to relative date, no problem! I suggest creating this variable in the parent job before starting your main transformation where this variable...
  43. Trigger a Table Input without the use of dummy parameters?

    In this example a transformation starts with a Table Input step. However, I only want to execute this select query in certain situations. So I create a step before that, doing some logic and sending...
  44. You need to know the difference between a field...

    You need to know the difference between a field and a variable. A field is a value that is moved between steps in a transformation and does NOT use the ${NAME} syntax. A variable however is set...
  45. It works if I disable "Never stop generating...

    It works if I disable "Never stop generating rows". But if I have 100000 rows, I want row nr 1 to be dispatched immediately, not wait for row nr 100000.

    It seems to me that Merge Join step is even...
  46. I tried the "Merge Join" step. I started with 2...

    I tried the "Merge Join" step. I started with 2 "Generate Rows" steps. One of them only write one row. The other step continously writes a row every second.

    But the Merge Join step didn't output...
  47. How to join constant columns from another step?

    Is it possible to do something like a "Stream Value Lookup" which adds a handful of fields to a stream, but without looking up keys?

    Basically I can achieve this behavior by adding a dummy "key"...
  48. This is a heavy topic! I actually did many...

    This is a heavy topic! I actually did many Process commands against SSAS using Kettle/Spoon. It required massive amounts of customizations. You need http access, which is pretty much only achievable...
  49. Replies
    3
    Views
    1,194

    Wow this was a nice gem! Really easy and fun to...

    Wow this was a nice gem! Really easy and fun to use.
  50. Instead of executing kettle pentaho, could you...

    Instead of executing kettle pentaho, could you try to remotely execute a simple bat script which just writes a simple log file as output? I would suggest trying to make this work first, then move on...
Results 1 to 50 of 290
Page 1 of 6 1 2 3 4
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.