Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Error message when running nested transformations from command line

  1. #1

    Question Error message when running nested transformations from command line

    Hi community,

    I created a job that executes multiple jobs, each of which runs multiple transformations. When I run the job in PDI (i. e. the GUI), all works fine and it finishes within a couple of minutes successfully.

    Name:  JobRunsSuccessfully.jpg
Views: 341
Size:  10.4 KB

    However, I need to run this job as a script (for automation purposes). I'm fairly new to this, but I'm able to run a simple test job (which just sends an email) via command line, so I believe I understood the basics.

    I'm running PDI 8.2 (Community Edition), and I use a file repository. The jobs and transformations all sit in the same folder, and I try to execute this command:

    Code:
    kitchen.bat /file:C:\Users\RFS\PentahoRepository\BI\00_00_00_Update_BI.kjb /level:Basic
    But then, I'm getting errors like "Unexpected error during job metadata load" (I'll paste the full output at the end). It tries to execute the various jobs, but the execution of the transformations within the jobs fails. Between every job I included a "send email" job, and those are executed properly.

    What would I need to do so the transformations within the jobs run properly?


    Thanks a lot for any help!

    Here's some of the output (and no further text at the end)

    C:\Program Files\Pentaho\data-integration\8.2>kitchen.bat /file:C:\Users\RFS\PentahoRepository\BI\00_00_00_Update_BI.kjb /level:Basic
    DEBUG: Using PENTAHO_JAVA_HOME
    DEBUG: _PENTAHO_JAVA_HOME=C:\Program Files\Java\jre1.8.0_211\
    DEBUG: _PENTAHO_JAVA=C:\Program Files\Java\jre1.8.0_211\\bin\java.exe


    C:\Program Files\Pentaho\data-integration\8.2>"C:\Program Files\Java\jre1.8.0_211\\bin\java.exe" "-Xms4g" "-Xmx28g" "-XX:MaxPermSize=512m" "-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2" "-Djava.library.path=libswt\win64" "-DKETTLE_HOME=" "-DKETTLE_REPOSITORY=" "-DKETTLE_USER=" "-DKETTLE_PASSWORD=" "-DKETTLE_PLUGIN_PACKAGES=" "-DKETTLE_LOG_SIZE_LIMIT=" "-DKETTLE_JNDI_ROOT=" -jar launcher\launcher.jar -lib ..\libswt\win64 -main org.pentaho.di.kitchen.Kitchen -initialDir "C:\Program Files\Pentaho\data-integration\8.2"\ /file:C:\Users\RFS\PentahoRepository\BI\00_00_00_Update_BI.kjb /level:Basic
    Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0
    14:06:50,030 INFO [KarafBoot] Checking to see if org.pentaho.clean.karaf.cache is enabled
    14:06:54,269 INFO [KarafInstance]
    *******************************************************************************
    *** Karaf Instance Number: 2 at C:\Program Files\Pentaho\data-integration\8 ***
    *** .2\.\system\karaf\caches\kitchen\data-1 ***
    *** FastBin Provider Port:52902 ***
    *** Karaf Port:8803 ***
    *** OSGI Service Port:9052 ***
    *******************************************************************************
    Jul 13, 2019 2:06:55 PM org.apache.karaf.main.Main$KarafLockCallback lockAquired
    INFO: Lock acquired. Setting startlevel to 100
    2019/07/13 14:06:55 - Kitchen - Logging is at level : Basic
    2019/07/13 14:06:55 - Kitchen - Start of run.
    2019-07-13 14:06:58.487:INFOejs.Server:jetty-8.1.15.v20140411
    2019-07-13 14:06:58.519:INFOejs.AbstractConnector:Started NIOSocketConnectorWrapper@0.0.0.0:9052
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/core
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/beans
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/parameterized-types
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/configuration/security
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/wsdl/
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://www.w3.org/2005/08/addressing
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2004/08/addressing
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-management (195) [org.apache.cxf.management.InstrumentationManager]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-wsdl (198) [org.apache.cxf.wsdl.WSDLManager]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-xml (200) [org.apache.cxf.binding.xml.XMLBindingFactory, org.apache.cxf.binding.xml.wsdl11.XMLWSDLExtensionLoader]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-soap (201) [org.apache.cxf.binding.soap.SoapBindingFactory, org.apache.cxf.binding.soap.SoapTransportFactory]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/bindings/soap
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-policy (220) [org.apache.cxf.ws.policy.PolicyEngine, org.apache.cxf.policy.PolicyDataEngine, org.apache.cxf.ws.policy.AssertionBuilderRegistry, org.apache.cxf.ws.policy.PolicyInterceptorProviderRegistry, org.apache.cxf.ws.policy.PolicyBuilder, org.apache.cxf.ws.policy.PolicyAnnotationListener, org.apache.cxf.ws.policy.attachment.ServiceModelPolicyProvider, org.apache.cxf.ws.policy.attachment.external.DomainExpressionBuilderRegistry, org.apache.cxf.ws.policy.attachment.external.EndpointReferenceDomainExpressionBuilder, org.apache.cxf.ws.policy.attachment.external.URIDomainExpressionBuilder, org.apache.cxf.ws.policy.attachment.wsdl11.Wsdl11AttachmentPolicyProvider, org.apache.cxf.ws.policy.mtom.MTOMAssertionBuilder, org.apache.cxf.ws.policy.mtom.MTOMPolicyInterceptorProvider]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-transports-http (202) [org.apache.cxf.transport.http.HTTPTransportFactory, org.apache.cxf.transport.http.HTTPWSDLExtensionLoader, org.apache.cxf.transport.http.policy.HTTPClientAssertionBuilder, org.apache.cxf.transport.http.policy.HTTPServerAssertionBuilder, org.apache.cxf.transport.http.policy.NoOpPolicyInterceptorProvider]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/transports/http/configuration
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/simple
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-frontend-jaxws (204) [org.apache.cxf.jaxws.context.WebServiceContextResourceResolver]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxws
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxrs
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/jaxrs-client
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/binding/coloc
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-transports-local (216) [org.apache.cxf.transport.local.LocalTransportFactory]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-bindings-object (217) [org.apache.cxf.binding.object.ObjectBindingFactory]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/blueprint/binding/object
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/policy
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://www.w3.org/ns/ws-policy
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://www.w3.org/2006/07/ws-policy
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2004/09/policy
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/wss/2004/...secext-1.0.xsd
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/wss/2004/...tility-1.0.xsd
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://www.w3.org/2000/09/xmldsig#
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://docs.oasis-open.org/ws-sx/ws-...ypolicy/200702
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-addr (237) [org.apache.cxf.ws.addressing.policy.AddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.UsingAddressingAssertionBuilder, org.apache.cxf.ws.addressing.policy.AddressingPolicyInterceptorProvider, org.apache.cxf.ws.addressing.impl.AddressingWSDLExtensionLoader, org.apache.cxf.ws.addressing.WSAddressingFeature$WSAddressingFeatureApplier, org.apache.cxf.ws.addressing.MAPAggregator$MAPAggregatorLoader]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/ws/addressing
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-security (239) [org.apache.cxf.ws.security.policy.WSSecurityPolicyLoader, org.apache.cxf.ws.security.cache.CacheCleanupListener]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-ws-rm (241) [org.apache.cxf.ws.rm.RMManager, org.apache.cxf.ws.rm.policy.RMPolicyInterceptorProvider, org.apache.cxf.ws.rm.policy.RM10AssertionBuilder, org.apache.cxf.ws.rm.policy.RM12AssertionBuilder, org.apache.cxf.ws.rm.policy.WSRMP12PolicyLoader, org.apache.cxf.ws.rm.policy.MC11PolicyLoader, org.apache.cxf.ws.rm.policy.RSPPolicyLoader]
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://cxf.apache.org/ws/rm/manager
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.blueprint.NamespaceHandlerRegisterer register
    INFO: Registered blueprint namespace handler for http://schemas.xmlsoap.org/ws/2005/02/rm/policy
    Jul 13, 2019 2:06:59 PM org.apache.cxf.bus.osgi.CXFExtensionBundleListener addExtensions
    INFO: Adding the extensions from bundle org.apache.cxf.cxf-rt-javascript (242) [org.apache.cxf.javascript.JavascriptServerListener]
    Jul 13, 2019 2:06:59 PM org.pentaho.caching.impl.PentahoCacheManagerFactory$RegistrationHandler$1 onSuccess
    INFO: New Caching Service registered
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/C:/Program%20Files/Pentaho/data-integration/8.2/launcher/../lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/C:/Program Files/Pentaho/data-integration/8.2/plugins/pentaho-big-data-plugin/lib/slf4j-log4j12-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
    2019/07/13 14:07:01 - 00_00_00_Update_BI - Start of job execution
    2019/07/13 14:07:01 - 00_00_00_Update_BI - Starting entry [Mail: Update BI started]
    Jul 13, 2019 2:07:02 PM org.apache.cxf.endpoint.ServerImpl initDestination
    INFO: Setting the server's publish address to be /lineage
    2019/07/13 14:07:02 - 00_00_00_Update_BI - Starting entry [Deletions]
    2019/07/13 14:07:02 - Deletions - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Error running job entry 'job' :
    2019/07/13 14:07:02 - Deletions - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException:
    2019/07/13 14:07:02 - Deletions - Unexpected error during job metadata load
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.run (Job.java:384)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute (Job.java:497)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute (Job.java:680)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.execute (JobEntryJob.java:667)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1345)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1381)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository (JobEntryJob.java:1353)
    2019/07/13 14:07:02 - Deletions -
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1421)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1345)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.execute(JobEntryJob.java:667)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute(Job.java:680)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.execute(Job.java:497)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.Job.run(Job.java:384)
    2019/07/13 14:07:02 - Deletions - Caused by: java.lang.NullPointerException
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository(JobEntryJob.java:1353)
    2019/07/13 14:07:02 - Deletions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1381)
    2019/07/13 14:07:02 - Deletions - ... 7 more
    2019/07/13 14:07:02 - 00_00_00_Update_BI - Starting entry [Deletions -> Dimensions]
    Jul 13, 2019 2:07:02 PM org.apache.cxf.endpoint.ServerImpl initDestination
    INFO: Setting the server's publish address to be /i18n
    2019/07/13 14:07:02 - 00_00_00_Update_BI - Starting entry [Update Dimensions]
    2019/07/13 14:07:02 - Update Dimensions - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : Error running job entry 'job' :
    2019/07/13 14:07:02 - Update Dimensions - ERROR (version 8.2.0.0-342, build 8.2.0.0-342 from 2018-11-14 10.30.55 by buildguy) : org.pentaho.di.core.exception.KettleException:
    2019/07/13 14:07:02 - Update Dimensions - Unexpected error during job metadata load
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.run (Job.java:384)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:497)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute (Job.java:680)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.execute (JobEntryJob.java:667)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1345)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta (JobEntryJob.java:1381)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository (JobEntryJob.java:1353)
    2019/07/13 14:07:02 - Update Dimensions -
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1421)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1345)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.execute(JobEntryJob.java:667)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:680)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:821)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.execute(Job.java:497)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.Job.run(Job.java:384)
    2019/07/13 14:07:02 - Update Dimensions - Caused by: java.lang.NullPointerException
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMetaFromRepository(JobEntryJob.java:1353)
    2019/07/13 14:07:02 - Update Dimensions - at org.pentaho.di.job.entries.job.JobEntryJob.getJobMeta(JobEntryJob.java:1381)
    2019/07/13 14:07:02 - Update Dimensions - ... 9 more

    2019/07/13 14:07:03 - Kitchen - Processing ended after 8 seconds.
    Last edited by MacPhotoBiker; 07-13-2019 at 02:22 PM.

  2. #2
    Join Date
    May 2016
    Posts
    282

    Default

    The log isn't stating a specific error, but I would begin testing if PDI in script mode is having trouble to find the subjob files.
    1) Build a simple job calling another job that just doing some dummy operation, or just sending a test mail.
    2) Try to run that simple job by command line, upping the logging level if necessary.

    A comment on your job image, I would add a Success step to finish the job, and, if I don't want the job to continue if one of the subjobs fails, I would put conditional transitions and even add an abort step if one the subjobs fail. But maybe in your case you want to continue, so it's all well. Just a suggestion in case you didn't know.
    Name:  20190715_001.jpg
Views: 133
Size:  14.5 KB

    Regards
    OS: Ubuntu 16.04 64 bits
    Java: Openjdk 1.8.0_131
    Pentaho 6.1 CE

  3. #3
    Join Date
    Jul 2009
    Posts
    476

    Default

    What path do you specify for the job entries? I use ${Internal.Job.Filename.Directory} as the directory name, which specifies that the sub-jobs and/or transformations are in the same filesystem directory as the job calling them. This screenshot is from an older version of PDI than 8.2, but it probably works the same way. Here, the "receipt_type.ktr" file is in the same directory as the job calling it:

    Name:  Screen Shot 2019-07-15 at 8.09.54 AM.jpg
Views: 133
Size:  12.3 KB

  4. #4

    Default

    Hi robj,

    Thank you very much for pointing that out. I reviewed my settings for all transformations, the format how they were referenced was not consistent. I selected them via the browse button, and this is the syntax I get:

    ${Internal.Entry.Current.Directory}/00_10_00_Deletions

    Your values within the brackets are different, and (maybe even more importantly?), I see that you make reference to the file (with extension ktr), while I'm only referencing the transformation name (without the file extension).

    Is there a setting that determines how this string gets created?

    Anyway, I guess my best bet is combining your advice with Ana's, and I'll create a less complex job. I'll post my progress here. Thanks again for your help.

  5. #5

    Default

    Hi Ana,

    Thank you very much for your reply. Indeed, it seems that the problem is that I'm not referencing the transformations properly from the various jobs. I'll follow your advice and create a simpler job. Thank you also for pointing out the success/ no success conditions. I've never really looked into that, but it's certainly a good idea to introduce a more solid error handling.

    Much appreciated!

    Gert

  6. #6
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by MacPhotoBiker View Post
    Is there a setting that determines how this string gets created?
    I believe that this comes from using a file repository.
    The file repository is a bit of a pain... You have to use it consistently or not at all.

    When you run the job from spoon, you are using the repository.
    When you run the job from kitchen, you are not using the repository.

    Therein lies the problem.

    Imagine for a moment that your repository was in a DB rather than in files... You would have to tell kitchen to use that DB repository instead of trying to use files. Same thing applies with the file repository. However, if not everyone that is using the repository have it set up as a resository (ie. they are using the raw files), then you get really odd behaviour.

    Hope this helps!

  7. #7
    Join Date
    May 2016
    Posts
    282

    Default

    I don't use Pentaho repository, I use a git repository external to Pentaho, so from Pentaho point of view my transformations and jobs are in a external file system and need to be referenced with a full path (or variables to reference part of the path, as the ${Internal.Job.Filename.Directory} @robj has mentioned), with their extensions.
    If you don't plan to use the Enterprise Edition of Pentaho, and even if you plan to use it, unless there's some specific feature of the Pentaho repository you plan to use, and you prefer using a file system repository (that I think it's not oficially supported in the Community Edition, although I might be wrong), I think it's better to use a more standard version control system and repository, that integrates with the repositories of your other tools. There's even a plugin developed for the community to integrate Pentaho with Git, google gitspoon for more details, although it's not necessary, a normal filesystem will work.
    The official documentation encourages the use of the Pentaho repository, but I don't see the advantages unless you plan to use the Carte Server. Transformations and Jobs are XML files perfectly compatible with the popular version control systems, using a separated version control system for them, when you use other systems for other parts of your development (such as DB Objects and the like), just complicates things.
    Just as a guide (you'll have to work out your own system) I'll tell you what I use:
    In the kettle.properties file, that it's in your home directory in a hiden directory named .kettle, I define a variable to point to my root git local repository, for example:
    PATH_REPOSITORY=/home/ana/gitrepository/pdi
    The kettle.properties file gets read each time spoon, pan or kitchen are called, so the variables are available for the session, I order my transformations and jobs inside subdirectories, so each time I reference a sub-job or transformation in a parent job, I need to edit the path to use the variable (which is a pain): ${PATH_REPOSITORY}/my_subdirectory/my_job.kjb
    This way, moving from development to production (another machine), (or working on a different project/branch in local) is just cloning the git repository and editing the kettle.properties file before starting spoon/pan/kitchen, to point to the correct path.
    I use variables in kettle.properties to keep database connection information too, so moving the development to production is painless and I don't need to edit the information inside the jobs and transformations, and other general information common for the development (paths where I look for files and such).
    If you use this approach, once you have the gist of it, take a look to the Enviroment plugin Matt Caster has developed for the community, instead of editing the kettle.properties, you create different enviroments and you choose which one you want to use the moment you start spoon or maitre (maitre is another thing developed by Matt to unify pan and kitchen, adding other things Matt has developed such as the use of enviroments)
    Regards
    OS: Ubuntu 16.04 64 bits
    Java: Openjdk 1.8.0_131
    Pentaho 6.1 CE

  8. #8
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by Ana GH View Post
    If ... you prefer using a file system repository (that I think it's not oficially supported in the Community Edition, although I might be wrong)
    The original poster said they were using File Repository. It is listed as an option that users can use.

    Using a bunch of files in a directory is NOT the same as using File Repository.
    My point was that File Repository created more problems than it solved.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.