Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Job scheduling and kitchen

  1. #1
    Join Date
    Jul 2007
    Posts
    2,498

    Question Job scheduling and kitchen

    After all this forum spam from a new user (me), I promise that I will submit the result of my work on the form of a tech tip or job sample, in appreciation for all the help


    1. First of all, am I correct in assuming that there is no long a program called chef and it's not merged in spoon? Documentation always refers to chef, but it's not in the distribution file


    2. The documentation of kitchen says that for job execution we should use at/cron. Taken that, what's that Start job entry for, that allows us to specify when to run the job?


    3. If I'm not mistaken, we can only run transformations from pentaho, not jobs, is that correct? Will we be able to do that in the future?


    4. Is there any advantage in using a repository for small transformations? Seems easier to maintain a file-based solution than a repository one (eg: "mr client, can you please drag and drop that file to that dir and your problem is solved? Thank you")


    5. This (like many other things) is probably on the documentation, but when a transformation is run, is it enclosed in a db transaction? If thinks go wrong, do you get a rollback, or when getting table indexes can we be secure that no concurrent process will change them?


    Pedro Alves
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    1. First of all, am I correct in assuming that there is no long a program called chef and it's not merged in spoon? Documentation always refers to chef, but it's not in the distribution file
    There once was chef, but no more... Chef was the current job graph in spoon (more or less)

    2. The documentation of kitchen says that for job execution we should use at/cron. Taken that, what's that Start job entry for, that allows us to specify when to run the job?
    ouch... you're right.

    3. If I'm not mistaken, we can only run transformations from pentaho, not jobs, is that correct? Will we be able to do that in the future?
    Jobs are in there... carte doesn't support jobs yet

    4. Is there any advantage in using a repository for small transformations? Seems easier to maintain a file-based solution than a repository one (eg: "mr client, can you please drag and drop that file to that dir and your problem is solved? Thank you")
    For productive use I would use XML based jobs/transformations. But I'm a little bit on the conservative side.

    5. This (like many other things) is probably on the documentation, but when a transformation is run, is it enclosed in a db transaction? If thinks go wrong, do you get a rollback, or when getting table indexes can we be secure that no concurrent process will change them?
    Default is that every step has its own connection and corresponding transaction, and most database steps allow to specify a commit size. You can also put connetions to "unique" and then you have 1 connection per transaction.

    Regards,
    Sven

  3. #3
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    Thanks for the answers Sven, but regarding 2, what should we do? run from cron/at, or can kitchen be run like a "daemon" mode, listening to the choices made in the 'Start' job entry.

    Not sure if I made myself clear, but 3. reported to the pentaho BI xaction for ELT. Can it run jobs?


    Thanks
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    Thanks for the answers Sven, but regarding 2, what should we do? run from cron/at, or can kitchen be run like a "daemon" mode, listening to the choices made in the 'Start' job entry.
    For the moment from cron/at short term... long term there's going to be something "servery-type"... I need something by the end of the year

    Not sure if I made myself clear, but 3. reported to the pentaho BI xaction for ELT. Can it run jobs?
    I'm not the expert, I run PDI more standalone... but from what I've seen xaction is able to run jobs.

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.