Hitachi Vantara Pentaho Community Forums

Search Forums:

Type: Posts; User: Isha Lamboo; Keyword(s):

Page 1 of 3 1 2 3

Search Forums: Search took 0.02 seconds.

  1. Pentaho marks those lines as ERROR lines simply...

    Pentaho marks those lines as ERROR lines simply because they come from stderr, which is meant for errors.
    The Osmosis executable apparently returns extra information on stderr instead of stdout,...
  2. If you know at start time which stream(s) you...

    If you know at start time which stream(s) you want, why not make separate transformations for the common cases and launch the correct one either by name or from a parent job?

    Performance wise,...
  3. Does the active transformation exist as Java object?

    Does the active/running transformation exist as an object that can be used in java/javascript code?

    I have an existing set of template-style jobs and transformations (made by someone else) that...
  4. Replies
    6
    Views
    1,270

    You can use Filter Rows to do this most easily....

    You can use Filter Rows to do this most easily. It has a test "is null" and even allows you to test multiple fields with AND/OR. You can then direct True output to one Text File Output and False...
  5. I suspect the dummy has nothing to do with your...

    I suspect the dummy has nothing to do with your performance problems. Instead, your lookups are causing deadlocks because the third one is probably preventing the shared Table Input step...
  6. Use a Calculator step to calculate the difference...

    Use a Calculator step to calculate the difference in days between date 1 and 2, then add a Filter Rows step to filter any whose date is not in the range 0-30 days.
  7. Replies
    5
    Views
    4,040

    There is a very useful example included in the...

    There is a very useful example included in the installation: Look under samples/transformations for "General - Annotated SOAP Web Service call - dialog.ktr"

    This has become the basis for most SOAP...
  8. Replies
    7
    Views
    1,221

    If the number of customer IDs in your csv file is...

    If the number of customer IDs in your csv file is small relative to the total in the lookup table, you should use the Database lookup step instead. It will do what you were trying to do: Lookup each...
  9. Replies
    7
    Views
    1,221

    Try leaving out the "AND CustID = ?". The...

    Try leaving out the "AND CustID = ?". The Database join already does lookups using the key fields that you specify, so you don't have to put them in the query.
  10. Replies
    4
    Views
    879

    As far as I know Spoon logs to the Logging tab,...

    As far as I know Spoon logs to the Logging tab, which is probably just redirected console output.

    You can configure database logging in job settings, but I haven't found any way to configure a log...
  11. I don't think there's a built-in option. The...

    I don't think there's a built-in option.

    The best I can think of is to use the Data Validator step to check the lengths beforehand and log some warnings, then truncate them all and send to MYSQL....
  12. Replies
    4
    Views
    879

    If you're running from the command line, kitchen...

    If you're running from the command line, kitchen supports the /log parameter:

    /opt/pentaho/data-integration/kitchen.sh /rep:myrepo /dir:/ /job:myjob_main /log:/var/log/pentaho/myjob_main-$(date...
  13. You can use the step "Strings Cut" to truncate...

    You can use the step "Strings Cut" to truncate the fields to the length of the table field.
  14. Replies
    16
    Views
    1,585

    I mean that you had a data-integration folder...

    I mean that you had a data-integration folder with version 4.4 or 7.0 or something and then unzipped your current version (5.0.1) into that same folder.

    I like to still call it installing, even...
  15. Replies
    16
    Views
    1,585

    Have you perhaps upgraded the installation by...

    Have you perhaps upgraded the installation by unzipping a new version over an older one?
  16. Replies
    16
    Views
    1,585

    It's the folder where you unzip pdi, normally...

    It's the folder where you unzip pdi, normally named "data-integration".

    If this specific jar keeps disappearing, check your anti-virus solution. It may have quarantined them for some weird reason.
  17. I'm not sure I understand your remark on CPU...

    I'm not sure I understand your remark on CPU usage. Are you seeing 100% CPU already use when you run the job or are you worried you will get that if you introduce parallelism?

    Creating an index...
  18. Replies
    5
    Views
    1,011

    I see nice green checkmarks on all steps! ...

    I see nice green checkmarks on all steps!

    Really, you will need to tell us what is happening and what you want to happen before we can help you.
  19. Sounds like a case of optimizing the table design...

    Sounds like a case of optimizing the table design and throughput.

    I would start by creating a copy of the destination table without any indexes, triggers or foreign keys and connect your...
  20. Replies
    6
    Views
    1,491

    PDI is mostly the data engine, the GUI is just a...

    PDI is mostly the data engine, the GUI is just a small part of it. All of them are launched in roughly the same way with the same jars. You could strip out some files and folders, but you save a few...
  21. Replies
    6
    Views
    1,491

    In my experience so far, new installations are...

    In my experience so far, new installations are fairly rare, so it's always been half-automated:


    Manually verify prerequisites (Java and DB installations) on the server
    Manually install the...
  22. Replies
    8
    Views
    896

    Is that the real input file? If so, I don't envy...

    Is that the real input file? If so, I don't envy you.

    The file you provide is not tab-separated but fixed width, except it's broken at the field ExposureID where the first entry is missing a...
  23. Replies
    8
    Views
    896

    The filters tab of the Text File input allows you...

    The filters tab of the Text File input allows you to remove the junk lines. Something like "Date Generated" at position 0 and positive match = N will discard any lines that start with the string...
  24. Replies
    4
    Views
    950

    Does the macro do more than just email the file?...

    Does the macro do more than just email the file? If not, you can use the Email Job step in PDI to email the file after the transformation completes and have it all in one place.
  25. Ah, I see now. I should have said "previous row",...

    Ah, I see now. I should have said "previous row", not last row. I fixed that in my other post.

    First and Last rows can be done with Group By in a similar way, by not specifying a group and...
  26. 18195 This transform works for me in 5.2, 6.1,...

    18195

    This transform works for me in 5.2, 6.1, 7.0 and 7.1.
  27. No need to get rude,...

    No need to get rude, ocremedios-with-a-new-account.

    We're trying to help you and if that's not satisfactory, you can always get paid support at Pentaho.

    Sequences are not a generic feature that...
  28. The modified javascript value step already runs...

    The modified javascript value step already runs for each rows that passes through it, while keeping variables in scope for the runtime of the transformation.

    You can define script tabs in it of 3...
  29. Much better :-) Give each field its own name...

    Much better :-)

    Give each field its own name as the Type (Get Fields makes that easy) and set the new field to "Equipment" or "CO2" for all.
    Each of those fields needs to end up in its own row...
  30. It's almost impossible to read the text in the...

    It's almost impossible to read the text in the screenshot after the forum has squished it, can you make a second one of only the step configuration?

    If I'm not mistaken, you have configured the...
  31. Then just use Sort Rows with pass unique rows...

    Then just use Sort Rows with pass unique rows only.

    Do note that for each unique key, you will get a random record, whichever one the database coughs up first for each key.
  32. Are the fields you are sorting by enough to...

    Are the fields you are sorting by enough to determine uniqueness or do you want to check ALL fields?

    Using Sort Rows by itself seems the most efficient option, but I don't know what the step will...
  33. Replies
    3
    Views
    2,811

    If you want it to be a transformation, use the...

    If you want it to be a transformation, use the Calculator step instead of Add sequence. In the calculator, first set a temp field to constant value 1, then use Field A + B with your incoming field...
  34. What kind of database are you using and what type...

    What kind of database are you using and what type of operations are you doing in parallel?
  35. As you show, the step was unable to get the...

    As you show, the step was unable to get the Metadata for the row. Your first step is a Table Input, which means the metadata comes from the database.

    Can it be that the table is altered, dropped...
  36. Replies
    2
    Views
    957

    "Integrate Stock Feed" really sounds like there...

    "Integrate Stock Feed" really sounds like there might be a webservice available, something you can use directly with an authentication token in a single request.

    Assuming the login page is the...
  37. Replies
    3
    Views
    1,112

    No environment variable is going to work without...

    No environment variable is going to work without a proper way to access the linux server. This is no different from Windows machines: You can't just copy to C:\tmp on another Windows PC without...
  38. Replies
    3
    Views
    1,112

    If you are running the job on your local Windows...

    If you are running the job on your local Windows machine (as shown by the file ending up in c:\tmp), how are you accessing the /tmp folder on the linux server when you manually check access?

    If...
  39. I think in the Enterprise Edition you can get...

    I think in the Enterprise Edition you can get these variables in any job because the server keeps this information in its database. In the Community Edition, you will need to set up a job admin and...
  40. Why are you not using the Oracle DB connection?...

    Why are you not using the Oracle DB connection? It's likely the generic one doesn't understand Oracle Sequences.
  41. Replies
    5
    Views
    1,130

    Yes, just make it inside plugins. You'll notice...

    Yes, just make it inside plugins. You'll notice on the github that the files are listed under steps/ckan-datastore-plugin. That's how it needs to be.
  42. Replies
    5
    Views
    1,130

    It probably needs to be in the steps folder if it...

    It probably needs to be in the steps folder if it isn't there currently.
  43. Replies
    2
    Views
    739

    Use the Generate Rows step with Add Sequence and...

    Use the Generate Rows step with Add Sequence and Table output.

    Configure the number of rows you want and the field Description with your default value in Generate Rows. Add a Sequence only if your...
  44. So you have a list of 62 valid offer IDs, rather...

    So you have a list of 62 valid offer IDs, rather than generic validation rules? In that case, Stream Lookup is probably more suitable.

    If your big input file is sorted on the lookup key already,...
  45. I would go with Filter Rows. It operates on a...

    I would go with Filter Rows. It operates on a row-level and will be as quick as your text file is read.

    Stream Lookup will read the ENTIRE file into memory and then you can look up a few rows,...
  46. In the Fixed File you are giving the line width...

    In the Fixed File you are giving the line width in bytes (104) and that seems to break when you hit a multi-byte character.
    Every line with an Ë has an extra byte and then a trailing space gets...
  47. If you do sorting in the source databases, you...

    If you do sorting in the source databases, you need to make sure the collations are identical. It's likely your second schema is set to something different than German, so you get errors. If you set...
  48. Replies
    7
    Views
    2,139

    You can use a second Text File Output connected...

    You can use a second Text File Output connected to the Detect Empty Stream step and configure your default filename there:

    18044

    Your main path can have the formula for the filename based on...
  49. Properties files work well for this. If your...

    Properties files work well for this. If your variable applies to most or all jobs, you can put it in kettle.properties in your user's .kettle directory.

    To use your own files, use the Property...
  50. You don't mention which database you are updating...

    You don't mention which database you are updating on, but just about all of them should use matching indexes automatically. If you force the use of an index it's not using by itself, you are likely...
Results 1 to 50 of 109
Page 1 of 3 1 2 3
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.