Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Getting last row as variable

  1. #1

    Default Getting last row as variable

    lately, i've been pondering about error handling

    is there a way to get last row read as a variable to retry the job instead using loop to reread whole database from start?

    something like this

    transformation ---x--->get variable (last row read)------>set variable
    ^++++++++++++++++++++++++++++++++++++|
    | ++++++++++++++++++++++++++++++++++++|
    |------------------------------------------------------------------|

  2. #2
    Join Date
    Nov 2009
    Posts
    688

    Default

    With the step "Top/Bottom/First/Last filter" you can get the last row in a stream. This step is a plugin

  3. #3
    Join Date
    Aug 2016
    Posts
    290

    Default

    Alternatively: Identify Last Row In Stream step.

  4. #4

    Default

    Quote Originally Posted by johanhammink View Post
    With the step "Top/Bottom/First/Last filter" you can get the last row in a stream. This step is a plugin
    thank you for your answer johanhammink, it works and your information is truly mind opening as i''ve just know that there's dlc plugin for pdi

  5. #5

    Default

    Quote Originally Posted by Sparkles View Post
    Alternatively: Identify Last Row In Stream step.
    hi sparkles, thanks for the alternative.

    i've tried Identify Last Row In Stream step with filter row and it works too
    and faster

  6. #6

    Default

    now i got the last row, but how do i convert it to a variable?

    is there a way to extract only the value of id and convert it to variable?
    or is there a way to make table input recognize the last row we set and only take row after it?

    thank you

  7. #7
    Join Date
    Nov 2009
    Posts
    688

    Default

    With the "Set Variable" step you can declare a variable with a certain value

  8. #8

    Default

    Quote Originally Posted by johanhammink View Post
    With the "Set Variable" step you can declare a variable with a certain value
    yes, i can use it to set the variable

    but as far as i know, sub select and limit over is not usable in pentaho table input

    or is there alternative way to do it?

  9. #9
    Join Date
    Nov 2009
    Posts
    688

    Default

    I am lost in what you are trying to achieve. Can you give an example of that
    You can use a subquery or a limit or a variable within a table input step.

  10. #10

    Default

    Quote Originally Posted by johanhammink View Post
    I am lost in what you are trying to achieve. Can you give an example of that
    You can use a subquery or a limit or a variable within a table input step.
    i'm sorry, i found out that i was wrong
    subquery or a limit or a variable within a table input step is indeed usable
    just that rank and row number is not ( CMIIW) from here
    https://forums.pentaho.com/showthrea...Rank-in-Kettle
    and somehow in a specific postgresql connection i cannot make use of offset, which is weird

    anyhow
    i've got my solution, which is
    table input --x-->table output (count) --v--> set variable (count as nr)--x-->table input (limit offset ${nr}) something like that

    thank you very much johanhammink

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.