Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Recover From Job Failure

  1. #1
    Join Date
    May 2007
    Posts
    22

    Default Recover From Job Failure

    Hi,
    (1) If a kettle job fails (hangs, db crashes, network goes down) how do we handle this? I think the kettle keeps information in the db repository. But how do we know from which record in the data source we have to start processing from?

    (2) Is it advisable to use database transactions in ETL. For example we only want to read committed data. Or We want to roll back a batch update because of a failure? Please let me know your views

    Thanks,
    Gavin

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    (1) for databases that's functionality you have to provide yourself. E.g. by reading the maximum value which was processed and then using that value as lower bound in the input query.

    (2) you will always use database transactions. If you're reading from database I would assume the input has to remain static while you extract data in the ETL (if it isn't you get into all kinds of weird problems e.g. when using Oracle). For the output side the choice is yours.

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.