Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Daily Batch updates

  1. #1
    Join Date
    Jul 2007
    Posts
    1

    Default Daily Batch updates

    Hi All,

    I am new to Kettle....

    I am trying to build a daily batch update process. I receive a text file from the Dataentry application on daily basis that will have Insert/Delete/Update records to header and child tables. I have a field in the text file that gives the information about the record type, that is whether the record is Insert/Delete/Update to Header table or perticular child table.

    Now looking at the record type I need to take the appropriate action agianst the perticular table. And if my operation (Insert/Delete/Update) against the table fails then that record needs to go to a text output file with certain message.

    Currently I am using MYSQL, but eventually I will be using DB2.

    Please let me know how can I handle this...

    Thanks,
    Somu

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    If on DB2 use DPROPR ... outside of Kettle.

    In Kettle you could go a long way, but it can't be covered 100%...

    - Run the query to get your rows with I/U/D
    - Have 3 filters filter on I/U/D
    - Use Delete/Update/Insert-Update step to do the actions (use insert-update with don't do updates option)
    - Define error handling on the 3 above steps.

    Problem case will be when several rows appear on the same key right after each other. Each step in Kettle runs concurrently so you can't assume 1 row is processed before the other.

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.