Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Adding rows to result from JavaScript Step

  1. #1
    Join Date
    Jun 2009
    Posts
    23

    Default Adding rows to result from JavaScript Step

    Hi,
    I'm getting a Java Heap Space error when I run my transformation on my whole dataset so I was thinking of creating a job with 2 transformations. The first would simply fetch the first and last id and then with those insert the ids in a table so that the 2nd transformation can look at those values and only process those ids per iteration:

    e.g. TR1: - in table input step: select min(id), max(id) from table (e.g. min = 1000, max = 3678)
    - in JavaScript step, set up a loop so that I can output a 'table' with all the intervals. (e.g output like : row1=1000,1100; row2= 1101,1200, ..., row27= 3601,3700) ( e.g. JS code like : ... while (cur< max) add_output row (cur, cur+100) ... )
    - copy rows to result step

    TR2: - get rows from result step,
    - call my query from my data: select * from table where id >= start and id < end
    - do rest of processing for transform (i.e. output to XML file).

    My problem is I have no idea how I can output the temp 'start, end' ids to a temporary location (I could probably just create a temp table in the DB but I'd rather not do that since that means adding a table in the clients db and I suspect Kettle has a way of doing this anyway).

    Thanks,
    Mike

    p.s. sorry if there's a trivial solution to this: I'm a newb and didn't see anything in the docs / searching.

  2. #2
    Join Date
    Dec 2009
    Posts
    22

    Default

    Within the first transformation add a 'Copy rows to result' step (this step is under 'Job' category in transformation steps in left navigation).
    Within the 2nd transformation, let the first node be 'Get rows from result', where you specify the names of fields written out by the 1st transform.

    See if this works.

    Also see the usage of sub-transformations (aka mappings): http://wiki.pentaho.com/display/EAI/Mapping


    Last edited by sam; 01-08-2010 at 10:07 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.