Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Can Update step be configured dynamically?

  1. #1
    Join Date
    Apr 2008
    Posts
    29

    Default Can Update step be configured dynamically?

    Hello,
    I am using Pentaho/Kettle version 3.2.3 (I am currently restricted using this version with my project)

    I have a single transformation that is called from a job to update a set of tables(100+) in the target from the source. While I can "configure" Insert and Delete steps using the kettle variables dynamically in the calling job, for the Update step I have to specify the fields to update explicitely in the step for the data to be updated. Currently I have taken an approach of not using the update step, and instead generating a script file with update statements which I execute after the execution of this job and transformation set finishes. I was wondering if there was a better alternative to that. (I also briefly explored combining delete/insert steps for the updates, but it is less efficient way of modifying the db).

    After looking around more, I found out about Metadata Injection step, but it is experimental and is only available in 4.0+ versions of Kettle. Would that be a general approach to a problem like this if I was using a newer version of Kettle? If it is, are there ways of doing something similar using 3.2.3 version of Kettle?

    Any suggestions would be greatly appreciated.

    Thank you

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    You can do similar things if you would use the Java API to generate the transformation that updates, as a lot of folks actually do.

  3. #3
    Join Date
    Apr 2008
    Posts
    29

    Default

    In case someone else has to do this or anything similar.
    I am only somewhat familiar with java and could not find quickly enough what methods/objects I should use to make/cause the necessary updates.
    Instead I took a different route and created a transformation which modifies the transformation which moves the data. By placing it before the transformation step that moves the data I ensure that the update step is updated with the list of update fields for each of the tables to be moved dynamically. I used add xml column step and xml join step to generate a "custom" ktr for each table dynamically as job to move entire db's data is running. The Update step was overwritten with the proper update fields list from metadata.
    There are multiple ways to get the required result.
    Is there a link or reliable place to check and see what Pentaho objects and methods are exposed preferably with the description of what they do/how they can be used? I would like to try the suggested method of using java API if I could find more information on it.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.