Hitachi Vantara Pentaho Community Forums

Search Forums:

Type: Posts; User: gutlez; Keyword(s):

Page 1 of 96 1 2 3 4

Search Forums: Search took 0.07 seconds; generated 21 minute(s) ago.

  1. Replies
    2
    Views
    89

    REST API Chunking is a common problem, and one...

    REST API Chunking is a common problem, and one that I haven't seen solved cleanly here yet.
    Part of the problem is that you're not really releasing the row sets until you hit the end of the REST...
  2. Also a security note: FTP is not secure. You...

    Also a security note: FTP is not secure. You can compromise the entire machine by leaving an FTP port open on the internet.

    This forum is not really about system architecture, but about PDI and...
  3. Copying files from one Remote Windows machine to...

    Copying files from one Remote Windows machine to another Remote Windows machine (relative to PDI) is not really a good use of PDI.

    Note: You would need to have an FTP or SFTP service installed an...
  4. Replies
    4
    Views
    187

    If you absolutely have to use this RegEx (Bad...

    If you absolutely have to use this RegEx (Bad Idea!)
    Try:
    .*?([a-zA-Z0-9._-\+]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+).*?

    Since you know your line contains "EMail:" you can redirect your row using a...
  5. Replies
    4
    Views
    187

    It's becuase PDI adds an implicit ^ and $ to...

    It's becuase PDI adds an implicit ^ and $ to either end of the RegEx
    so your RegEx that is actually being run is ^([a-zA-Z0-9._-]+@[a-zA-Z0-9._-]+\.[a-zA-Z0-9_-]+)$

    Try that in your Online...
  6. Replies
    6
    Views
    315

    Not that it's a lot of help, but......

    Not that it's a lot of help, but...
    https://github.com/pentaho/pentaho-kettle/blob/master/engine/src/main/java/org/pentaho/di/trans/step/BaseStep.java#L2569
  7. Replies
    3
    Views
    263

    Much like the PRD can help you convert a DB Table...

    Much like the PRD can help you convert a DB Table into a formatted PDF, Mondrian can help you convert a structured DB to Statistical "Slice&Dice"

    Basically:
    OLTP(DB) -> PDI -> OLAP(DB) ->...
  8. Replies
    1
    Views
    168

    Use Metadata Injection and Row Denormalizer. ...

    Use Metadata Injection and Row Denormalizer.

    Row Denormalizer needs to know how many columns to make. The Metadata Injection can figure that out and send it to the template transformation.
  9. Replies
    1
    Views
    161

    The tools are telling you that one of the steps...

    The tools are telling you that one of the steps gives 4 columns.
    The other step only gives one column.

    This is not allowed.

    From the logs you've posted, we can see that each step is giving...
  10. I feel like you have asked this before. The...

    I feel like you have asked this before.

    The hint is in the expansion of the acronym: PDF
    PDF is Page Description Format
    It doesn't have standard columns and rows, so very few tools will actually...
  11. Because all variables in PDI are strings. ...

    Because all variables in PDI are strings.

    setVariable does an implicit .toString() on anything that is not a string. Parse it into a date when you are declaring your sixty_second_start...
  12. Replies
    1
    Views
    152

    You can use the generic DB connection to be able...

    You can use the generic DB connection to be able to specify a JDBC driver, but you will lose some PDI functionality since it will be falling back to lowest-common-denominator SQL.
    PDI will not know...
  13. Replies
    4
    Views
    275

    The upstream steps will always send the data. ...

    The upstream steps will always send the data.

    What if you refactor a little bit?



    public boolean processRow(StepMetaInterface smi, StepDataInterface sdi) throws KettleException
    {
    if...
  14. It's hard to tell what the product direction is....

    It's hard to tell what the product direction is.

    The changes that you linked to were made as part of a fix for https://jira.pentaho.com/browse/BISERVER-12433
    But then it seems like there was...
  15. Thread: Transpose data

    by gutlez
    Replies
    4
    Views
    941

    Because your expected output is re-denormalized....

    Because your expected output is re-denormalized.

    So.
    Remove the Art from the Normalize step.
    That gives you output like:



    typefield
    Art
  16. Probably has to do with CompanyRole and...

    Probably has to do with CompanyRole and CompanyType being adjacent branches, rather than being Parent/Child.

    You will likely need to chain a few Get Data from XML steps together to get a standard...
  17. Replies
    4
    Views
    184

    You have to convert your JSON to standard rows....

    You have to convert your JSON to standard rows. (JSON Input Step) - You may need to link a few of these together to get standardized rows.
    Then you can filter out the rows that don't belong (Filter...
  18. Replies
    1
    Views
    167

    The problem is in the code... ...

    The problem is in the code...

    https://github.com/pentaho/pentaho-kettle/blob/master/core/src/main/java/org/pentaho/di/core/database/VerticaDatabaseMeta.java#L262

    and...
  19. Replies
    4
    Views
    184

    Convert your JSON to standard PDI rows Then...

    Convert your JSON to standard PDI rows
    Then filter your rows by Type=Ext to allow external rows to continue through
    Write your output as JSON
  20. Replies
    2
    Views
    247

    If you consider this in the streaming nature of...

    If you consider this in the streaming nature of PDI, you'll see why this is not an easy ask.

    1) Read Line from file_import.csv
    2) Perform transformations on line
    3) Write Line to file
    3.1) If...
  21. This is similar to what marabu posted, but...

    This is similar to what marabu posted, but recreated for this thread.

    KTR File:

    <?xml version="1.0" encoding="UTF-8"?><transformation>
    <info>
    <name>Demo</name>
    <description/>
    ...
  22. Nope... Not that one. That is how to join two...

    Nope... Not that one.
    That is how to join two lines into one.
  23. I know that there's an example on this forum...

    I know that there's an example on this forum somewhere by a poster by the name of marabu.

    It uses a trinary operator looking for a line start flag, and sets a column value to 1 or 0 based on that...
  24. Replies
    6
    Views
    323

    Hint: Detect Empty Rows step will send a row of...

    Hint: Detect Empty Rows step will send a row of all nulls if no rows reach it.
    It would probably be a good feature ask ( http://jira.pentaho.com ) to have an optional "Clear Result Rows at...
  25. Replies
    2
    Views
    173

    From what I recall, rebuilding the row is an...

    From what I recall, rebuilding the row is an expensive step.
    So your best bet would be to do all your joins, and then before sending the data to the dashboard, do one Select Values to rebuild the...
  26. Thread: Longest match

    by gutlez
    Replies
    4
    Views
    226

    I feel like tree string splitting this with...

    I feel like tree string splitting this with string structures would be more CPU efficient (and also time efficient), and potentially a way to hard-code it, but I really hate hard-coding.

    EG. you...
  27. Thread: Longest match

    by gutlez
    Replies
    4
    Views
    226

    So... I'm a dork. Two streams: Incoming Row...

    So... I'm a dork.

    Two streams:
    Incoming Row
    LookupStream

    Add a field (Key Length) to the lookup row.

    Do a cartesian Join on Incoming Row.KeyField Starts With LookupStream.KeyValue
    Sort...
  28. Thread: Longest match

    by gutlez
    Replies
    4
    Views
    226

    Perhaps: Break Strings to Character Lines, and...

    Perhaps:
    Break Strings to Character Lines, and do a comparison between them.
    Then do a running total of matching and total characters
    Filter for max value where Ematch = Etotal

    But that would...
  29. Replies
    6
    Views
    254

    Which sense of big data do you mean? Formula can...

    Which sense of big data do you mean?
    Formula can be pretty slow if you're doing string searches in a very long field, but as far as I've experienced (note: I only really go up to the 100K rows...
  30. Replies
    6
    Views
    254

    So... What if some creative type enters W@TCH as...

    So... What if some creative type enters W@TCH as the first 5 characters?
  31. Replies
    6
    Views
    254

    How many lines are you hoping to push through it...

    How many lines are you hoping to push through it in a run?
    How often will you be running the data?

    How often will the characters that you're testing change?
    How often will the number of...
  32. Remember to change your spreadsheet type before...

    Remember to change your spreadsheet type before you try to find the files (don't worry... I make this mistake *ALL* the time)
    If you change it from "Excel 97 - 2003 XLS (JXL)" to "Excel 2007 XLSX...
  33. Replies
    1
    Views
    440

    If you read the other posts on this subject, you...

    If you read the other posts on this subject, you will find that you would need to have X running in order to use Spoon on a Linux machine
    This is not recommended.

    Do your design work on a...
  34. Did you do a web-search first? Your error is...

    Did you do a web-search first?

    Your error is complaining about not understanding the JSON column type.

    JSON is not a column type that PDI natively speaks. Even if you were were to upgrade to...
  35. That looks like it should work, but you haven't...

    That looks like it should work, but you haven't mentioned your version... Someone else noted that there were issues with variables in 8.1
    If you post the actual transformations and job on Drive,...
  36. Yeah, I have the same issue sometimes...

    Yeah, I have the same issue sometimes...
  37. None that I'm aware of. Table Inputs are...

    None that I'm aware of.
    Table Inputs are designed to run independently as soon as possible. You would need to have some sort of flag into the SQL to indicate if it's supposed to run... otherwise,...
  38. Considering that Merge Join is supposed to be...

    Considering that Merge Join is supposed to be able to do a number of different match joins, you would have to hold all rows in memory the whole time...
    You can't release any rows until you have all...
  39. There is a way to set up the "Join Rows" step for...

    There is a way to set up the "Join Rows" step for this... But I can't remember the right way.

    Since one data set has only one row, you want to join all the other rows against it. (This will...
  40. Probably because... Microsoft wrote the XML/A...

    Probably because...
    Microsoft wrote the XML/A standard, and promptly failed to adhere to it.

    It might take some time to get it working, but it should work -- unfortunately, I don't have SSAS...
  41. You mean like the OLAP Input step?

    You mean like the OLAP Input step?
  42. Ok... So, what about: Job: 1) Start 2)...

    Ok...

    So, what about:
    Job:
    1) Start
    2) Transformation
    2.1) Get File Names (Note: Only requires Directory and RegEx to run)
    2.2) Set Variable (Filename)
    3) Simple Evaluation (Variable...
  43. What about using a transformation to: 1) Get...

    What about using a transformation to:
    1) Get File Name
    2) Set Variable

    Then connect that to a "Wait for File" using the variable that you just set...
  44. Replies
    2
    Views
    264

    Make sure you post your solution!

    Make sure you post your solution!
  45. The error codes come back from a web search as...

    The error codes come back from a web search as "Invalid credentials"
    This suggests that the password is not correct for that user.

    Since this isn't directly related to Pentaho, you're likely not...
  46. Replies
    2
    Views
    350

    Try a PDI path of ./transformation.ktr instead of...

    Try a PDI path of ./transformation.ktr instead of c:\pdi\workspace\some_job\transformation.ktr
    The limitation then is that the PRPT and the KTR need to be in the same directory.

    You should also...
  47. Replies
    2
    Views
    288

    So you want to do a stream lookup?

    So you want to do a stream lookup?
  48. This looks like a regression bug. You should...

    This looks like a regression bug.
    You should file a report at http://jira.pentaho.com
  49. You could sort rows on ID and State1. Then feed...

    You could sort rows on ID and State1.
    Then feed it to a Group By step calculating the Max value of the group, returning all rows.

    If you do this, you would have:
    ID | S1 | S2 | MS2
    1 | 1 | 0 |...
  50. Replies
    3
    Views
    584

    I seem to recall that you have a bit of a...

    I seem to recall that you have a bit of a programming background Gosforth.
    As such, you are likely thinking of Timestamp as a Unix Epoch value (either in seconds or milliseconds)

    In PDI, when...
Results 1 to 50 of 4762
Page 1 of 96 1 2 3 4
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.