Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: Migrating Mission-Critical Data With PDI?

  1. #1

    Default Migrating Mission-Critical Data With PDI?

    I'm currently planning a large-scale data migration project and I am
    wondering if Kettle is the right tool for the job?

    I'm considering the migration of a DB2 system (holding almost a terrabyte) to
    Oracle. The data model on the Oracle DB is much different, thus, a lot of
    mapping and filtering is necessary (which can probably be done with PDI).
    Furthermore, the system is highly dynamic (more than 100000 transactions a
    day), most data is mission-critical and the downtime should be minimal (I don't know if Kettle provides any help for such problems?)

    My current plan looks as follows:

    1) the source system should not be touched (but fallback-secure)
    Therefore, I will replicate the data on the DB2 system to an
    "intermediate" copy in an Oracle DB (my "staging area"). For this purpose,
    I will initially fill my Oracle Copy using a dump of the DB2 data and keep
    the copy in sync with the original DB2 data using IBM's Data Propagator.

    2) the transformation process should happen on-the-fly: after the first
    copy of the source is mapped to the target system, I want to map only the
    changes which the Data Propagator recorded in the meantime (delta data).
    My plan is, that the delta data gets smaller and smaller (maybe I need to
    stop some processes to achieve this) until its mapping fits into a planned
    maintenance break.

    Since both systems (DB2-Copy and target) are Oracle DBs, I was planning to
    handle the mapping with PL/SQL. Would you rather recommend Kettle? What is
    the best tutorial to get started?

    Your estimation is greatly appreciated.
    Thanks in advance.

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    It all depends.... a tool is just a tool, if you're in an Oracle shop where all people are PL-SQL developers, for your particular case PL-SQL may not be that bad

    Do a small "Proof Of Concept", take an easy and a hard table qua mappings and try them with both PDI and PL-SQL.

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.