Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Dynamic job name into Job log table

  1. #1
    Join Date
    Jan 2016
    Posts
    4

    Question Dynamic job name into Job log table

    Hi guys!
    I want to load many tables from source to target
    I'v made dynamic job and transformations for loading multiple tables one proccess consequentially

    "get params" gets table names, intervals, sql and so on.
    "loading" just loads table by gotten params
    "runner" is parent job witch runs child jobs, root job

    Also, I want to use Job log table for get "startdate" and "enddate" params for CDC loading.
    I want to run one and the same job many times under different jobnames

    My log table looks like this:



    I tried to use "Set variables" but it didn't halped me, i've tried everything without successful

    Cuold you help my, please?

    Thanks
    I

  2. #2
    Join Date
    Aug 2011
    Posts
    360

    Default

    Hi,

    I dont think you can change what is logged into database. The data logged is enforced by the database logging plugin.
    Moreover, you should really implement your own CDC system for tracking CDC dates instead of using the pdi logging system.
    Here is my arguments for that:
    - the pdi system is quit limiting. You will soon have needs for more logic than the one in pdi.
    - with this system, your CDC date tracking is tied to the implementation of your loading. For example, suppose you rename your job....bam your dates stsrts again at 1900-01-01!
    - a CDC date tracking system should be tracking logical source systems, i.e. "What is the last date i've loaded the Xxxx oltp database", not "what is the last date i'v executed the job that load oltp database"

    So, I really advise you to build some dedicated tables for CDC trackig with logical names, while puting in there the batch id and the channel of the job that eas used to do the load

  3. #3
    Join Date
    Jan 2016
    Posts
    4

    Default

    Mathias,
    thank you for your answer.
    Might be do you now another, ready to use CDC system compatible with Pentaho ???

    Thanks

  4. #4
    Join Date
    Aug 2011
    Posts
    360

    Default

    Hi,

    No i dont know such system, but i havent search for it neither!
    I think this could be hard to build a cdc framework for every cases, as it is really database and source system dependent.
    But you should find on the web what are the goood building blocks for a cdc system

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.