Hitachi Vantara Pentaho Community Forums
Results 1 to 6 of 6

Thread: Real time Scheduling

  1. #1

    Default Real time Scheduling

    Hi,
    Can any one help me to schedule mappings in real time?
    Meaning -- when ever there is an Insert,Update on a source table I want the workflows to execute.

    We are using community version.

    Thanks
    SK

  2. #2

    Default

    Have a look at Change Data Capturing basics under the READ ME FIRST topic in this forum...

  3. #3

    Default

    mblondinde,
    Thanks for the info.We did design the CDC but the issue is with real time scheduling.The above thread does talk about CDC but I did not find any information on the real time scheduling.

    Can you please help.

  4. #4
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    The book Pentaho Kettle Solutions has a chapter on Real-Time Data Integration - quote from the summary:

    The chapter discussed:

    • How a transformation is ideally suited for handling streams of data, and the attention points with respect to database time-outs, memory consumption, and logging
    • A practical example in the form of a step that continuously reads Twitter messages
    • How to debug a long-running transformation
    • Integration possibilities with third-party real-time integration software
    • How to send and receive messages using Java Message Services
    The Twitter sample should be in your Kettle samples folder, too.
    So long, and thanks for all the fish.

  5. #5
    Join Date
    Apr 2009
    Posts
    337

  6. #6
    Join Date
    Apr 2009
    Posts
    337

    Default

    Dear SK,

    1. Schedule to run your parent job after every 5 minutes

    2.A quick dirty solution is to use the "Wait for SQL" to step perform some querying, and based on its success, the transformation will be called.. but this parent job has to run in a never ending manner 24X7 mode

    Other options are listed in the CDC link which I posted earlier..
    Regards,
    Madhu

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.