Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Daily Batch updates

  1. #1
    Join Date
    Jul 2007

    Default Daily Batch updates

    Hi All,

    I am new to Kettle....

    I am trying to build a daily batch update process. I receive a text file from the Dataentry application on daily basis that will have Insert/Delete/Update records to header and child tables. I have a field in the text file that gives the information about the record type, that is whether the record is Insert/Delete/Update to Header table or perticular child table.

    Now looking at the record type I need to take the appropriate action agianst the perticular table. And if my operation (Insert/Delete/Update) against the table fails then that record needs to go to a text output file with certain message.

    Currently I am using MYSQL, but eventually I will be using DB2.

    Please let me know how can I handle this...


  2. #2
    Join Date
    May 2006


    If on DB2 use DPROPR ... outside of Kettle.

    In Kettle you could go a long way, but it can't be covered 100%...

    - Run the query to get your rows with I/U/D
    - Have 3 filters filter on I/U/D
    - Use Delete/Update/Insert-Update step to do the actions (use insert-update with don't do updates option)
    - Define error handling on the 3 above steps.

    Problem case will be when several rows appear on the same key right after each other. Each step in Kettle runs concurrently so you can't assume 1 row is processed before the other.


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.