Hello,

I need some advice about structuring my processes using Pentaho data integration. I'd like to build an ODS from an ERP data running under Oracle. Extracting the data is quite slow and I'm not yet sure why (ERP is on a different continent). I have to extract data from many tables and a bunch of them don't have a "Last Modified Date" field therefore I will have to extract the complete table and proceed with a delta locally.
What I'm planning to do is to extract all the table in a local "Staging" environment and proceed with the loads afterwards. My concern is that in case of failure during extraction of a table (let's presume that all table are going to be extracted in a sequential way to avoid overloading the ERP) I don't want to start all over again. I used to work with Microsoft SSIS in the past and we had a way to use some kind of Checkpoint in case of failure so we can restart the whole thing at exactly where it failed.
So my question is, is there some kind of similar feature in PDI or I have to "simulate" a checkpoint by let's say store a flag somewhere after a table has been successfully extracted?

Any advice would be appreciated on how you guys are proceeding.
Thank you.

Best regards,
Martin