Hitachi Vantara Pentaho Community Forums
Results 1 to 10 of 10

Thread: Commit Not happening in Table Output Step

  1. #1
    Join Date
    Sep 2007
    Posts
    12

    Default Commit Not happening in Table Output Step

    Hi Guys,

    I have a problem in my Etl where Commit not happening...I am using 3.1.0.Also tried with pdi-ce-3.2.0-M1 version.

    Let me explain my issue clearly..

    1) I have a transformation with Table input connected to table Output
    2) Table input fetches 1000 rows and table output has commit size with '100'
    3)it means that commit should happen on target for every 100 rows.
    4) i got an error at 761th row in table input.so i think 700 rows should be loaded into target because of the commit.but count in target table shows zero.
    5)means it rolledback all the transactions in this step.So commit is not happening.

    your input is really appreciated...


    Thanks,
    Pad

  2. #2
    Join Date
    Feb 2009
    Posts
    296

    Default

    Check the 'number of rows in rowset' option of your transformation (CTRL+T in spoon) and you will see that only after 10k or 50k (can't remember the default) number of lines the rowset is handed from one step to the next.

    So your input step has not sent ANY rows to the next step yet. You should be able to verify this with the tab right next to the logging-tab in spoon. It will show you how many rows are in which step.
    Fabian,
    doing ETL with his hands bound on his back

  3. #3
    Join Date
    Sep 2007
    Posts
    834

    Default

    Check the parameter "Make the transformation database transactional"
    (before version 3.2: Use unique connections)

    If it is checked, any error causes the roll back of the DB operations.

  4. #4
    Join Date
    Sep 2007
    Posts
    12

    Default

    Hi Maria,

    Thanks for your Reply..

    Yes, u r Right.if i unchecked the Unique connection check box,roll back is not happening when error occurs.But also commit is not happening at the commit size which we give in Table Output step.Due to this my Transactional logs are getting full...

    Again answer to this is really appreciated...


    Thanks,
    Pad

  5. #5
    Join Date
    Feb 2009
    Posts
    296

    Default

    Quote Originally Posted by fabianS View Post
    Check the 'number of rows in rowset' option of your transformation (CTRL+T in spoon) and you will see that only after 10k or 50k (can't remember the default) number of lines the rowset is handed from one step to the next.

    So your input step has not sent ANY rows to the next step yet. You should be able to verify this with the tab right next to the logging-tab in spoon. It will show you how many rows are in which step.
    Please read my comment again. It should shed light on your problem.
    Fabian,
    doing ETL with his hands bound on his back

  6. #6
    Join Date
    Apr 2009
    Posts
    28

    Default

    hi fabian,

    Your comment!

    "So your input step has not sent ANY rows to the next step yet. You should be able to verify this with the tab right next to the logging-tab in spoon. It will show you how many rows are in which step"

    What exactly this mean

    Number of rows in Rowset: 1000 it is declared in Transformation settings.

    unique connections is unchecked, still data is not commting when error occurs as padmakar posted.


    Regards
    ----------
    Narendar Vishwanatham

  7. #7
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Also tried with pdi-ce-3.2.0-M1 version.
    What part of "Development version" do you not understand?
    3.2 stable has been out for months.

  8. #8
    Join Date
    Apr 2009
    Posts
    28

    Default

    hi matt,

    we are facing problem with commit here i tell u the issue clearly

    in process of ETL with commit size of 100 ....

    as padmakar posted below 700 records got successful and at next phase of running at 61th record got disturbed so the transaction got rolled back out of which all 700 records got rolled back..(why this happened instead of delcaring commit size 100 it is rolling back ,den what is the use of commit field to be declared in settings)

    PLEASE LET ME KNOW THE REASON

  9. #9
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    How would I know? Look around yourself, search the code, browse JIRA.
    Alternatively, get a support contract.

    All I'm saying it's not wise to use development versions.

  10. #10
    Join Date
    Feb 2009
    Posts
    296

    Default

    Hi viswanathamn.

    Quote Originally Posted by viswanathamn View Post
    "So your input step has not sent ANY rows to the next step yet. You should be able to verify this with the tab right next to the logging-tab in spoon. It will show you how many rows are in which step"

    What exactly this mean

    Number of rows in Rowset: 1000 it is declared in Transformation settings.

    unique connections is unchecked, still data is not commting when error occurs as padmakar posted.
    Please read my comment again:
    Check the 'number of rows in rowset' option of your transformation (CTRL+T in spoon)
    THIS is the number you might want to change.
    However... you're not handling the real problem - WHY is it failing? Work on those errors and don't try to get 100 rows commit. Try to get everything commited.
    Fabian,
    doing ETL with his hands bound on his back

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.