Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: job flow control

  1. #1
    Join Date
    Sep 2007
    Posts
    4

    Default job flow control

    Hello,

    I have a job with 3 transformation entries: a, b, c.
    a and b starts in parallel (they're connected to the start entry by dashed lines), c is connected to a and b with simple green line.
    I'm expeting that c doesn't start before both a and b finished, but it seem like it starts when one of the two finished..

    In fact if I connected the 3 entry sequentially a->b->c the execution finishes with success, while if I use parallelism for a e b there are errors on queries on repository.

    Is there any way to tell an entries to wait for the execution of 2 or more entries?

    Sorry for the silly question.
    Thanks for every suggestion.

    D.Clementi

  2. #2
    Join Date
    May 2006
    Posts
    4,882

    Default

    If you use parallelism in a job the first time you get this nice little warning dialog to take care that all subsequent job entries will start in parallel. You can raise a JIRA for it, but it will be hard to implement what you see as intuitive

    The only way out for the moment is make a subjob and put a and b in the subjob (running in parallel) and then make a master job running that subjob in sequence with c.

    By the way, which errors do you get on queries on the repository? That should probably not happen.

    Regards,
    Sven
    Last edited by sboden; 08-19-2008 at 07:37 AM.

  3. #3
    Join Date
    Sep 2007
    Posts
    4

    Default

    Thank you very much Sven. I didn't consider to subdivide the entries in different jobs for more control on execution..

    For the errors in query I can post here the stack of errors. I verify that when I ran job with transformations in sequence nothing similar will be logged.. I don't understand why... (consider that when I spoke of a, b, c I was simplifing the job!)

    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - Loading transformation from repository [1_normalizza settori - sezioni] in directory [/procedure spot/Ateco 2007]
    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : A database error occured reading a transformation from the repository
    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : org.pentaho.di.core.exception.KettleDatabaseException:
    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : An error occurred executing SQL:
    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : SELECT ID_TRANSFORMATION FROM R_TRANSFORMATION WHERE "name" = ? AND ID_DIRECTORY = ?
    2008/08/19 14:17:37 - 1_normalizza settori - sezioni - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Questo statement è stato chiuso.
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : org.pentaho.di.core.exception.KettleException:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Unexpected error during transformation metadata load
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : A database error occured reading a transformation from the repository
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : An error occurred executing SQL:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : SELECT ID_TRANSFORMATION FROM R_TRANSFORMATION WHERE "name" = ? AND ID_DIRECTORY = ?
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Questo statement è stato chiuso.
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:781)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.entries.trans.JobEntryTrans.execute(JobEntryTrans.java:432)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.Job.execute(Job.java:333)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.Job.access$000(Job.java:58)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.Job$1.run(Job.java:409)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at java.lang.Thread.run(Unknown Source)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Caused by: org.pentaho.di.core.exception.KettleException:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : A database error occured reading a transformation from the repository
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : An error occurred executing SQL:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : SELECT ID_TRANSFORMATION FROM R_TRANSFORMATION WHERE "name" = ? AND ID_DIRECTORY = ?
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Questo statement è stato chiuso.
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2414)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2229)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.job.entries.trans.JobEntryTrans.getTransMeta(JobEntryTrans.java:761)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : ... 5 more
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Caused by: org.pentaho.di.core.exception.KettleDatabaseException:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : An error occurred executing SQL:
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : SELECT ID_TRANSFORMATION FROM R_TRANSFORMATION WHERE "name" = ? AND ID_DIRECTORY = ?
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Questo statement è stato chiuso.
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) :
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.core.database.Database.openQuery(Database.java:1736)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.core.database.Database.openQuery(Database.java:1660)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.core.database.Database.openQuery(Database.java:1655)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.core.database.Database.getOneRow(Database.java:3344)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.repository.Repository.getIDWithValue(Repository.java:1064)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.repository.Repository.getTransformationID(Repository.java:923)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2282)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : ... 7 more
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : Caused by: org.postgresql.util.PSQLException: Questo statement è stato chiuso.
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement.checkClosed(AbstractJdbc2Statement.java:2444)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement.getMaxRows(AbstractJdbc2Statement.java:543)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc3.Jdbc3Statement.createResultSet(Jdbc3Statement.java:36)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement$StatementResultHandler.handleResultRows(AbstractJdbc2Statement.java:193)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1278)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:191)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:452)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:351)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:255)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : at org.pentaho.di.core.database.Database.openQuery(Database.java:1697)
    2008/08/19 14:17:37 - org.pentaho.di.job.Job$1@1f9aeda - ERROR (version 3.0.4, build 53 from 2008/06/24 11:14:00) : ... 13 more
    2008/08/19 14:17:37 - Spoon - Job has ended.

  4. #4
    Join Date
    May 2006
    Posts
    4,882

    Default

    Postgres may not support multiple open SQL statements on 1 connection. You can try raising a JIRA for it, but for the moment I don't see a quick fix solution for it, besides not running a repository on postgres.

    Regards,
    Sven

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.