Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: help with Hadoop Job Executor

  1. #1
    Join Date
    Sep 2011
    Posts
    9

    Default help with Hadoop Job Executor

    hi, i am very new to this environment, but I am also trying out a very simple hadoop job by running the hadoop job executor, I don't see any error but apparently nothing has been done ( i didnt see the job being submitted to the hadoop cluster), the following is the logs, how do i debug it?

    2011/11/01 16:07:06 - Spoon - Starting job...
    2011/11/01 16:07:06 - hadoop test - Start of job execution
    2011/11/01 16:07:06 - hadoop test - exec(0, 0, START.0)
    2011/11/01 16:07:06 - START - Starting job entry
    2011/11/01 16:07:06 - hadoop test - Starting entry [Hadoop Job Executor]
    2011/11/01 16:07:06 - hadoop test - exec(1, 0, Hadoop Job Executor.0)
    2011/11/01 16:07:06 - Hadoop Job Executor - Starting job entry
    2011/11/01 16:07:06 - Hadoop Job Executor - Using jar path: file:/home/hadoop/hadoop_src/test/wordcount/wordcount.jar
    2011/11/01 16:07:06 - Hadoop Job Executor - Running Hadoop Job in Simple Mode
    2011/11/01 16:07:06 - hadoop test - Starting entry [Success]
    2011/11/01 16:07:06 - hadoop test - exec(2, 0, Success.0)
    2011/11/01 16:07:06 - Success - Starting job entry
    2011/11/01 16:07:06 - hadoop test - Finished job entry [Success] (result=[true])
    2011/11/01 16:07:06 - hadoop test - Finished job entry [Hadoop Job Executor] (result=[true])
    2011/11/01 16:07:06 - hadoop test - Job execution finished
    2011/11/01 16:07:06 - Spoon - Job has ended.
    2011/11/01 16:07:38 - Spoon - Spoon

  2. #2

    Default

    Hi Kenneth,

    The thing is when you use the simple version of the job executor the pointers to your hadoop cluster have to be in the jar itself. You have to do some settings for the conf object in java and recompile it.

    Its different from when you use the advanced tab, were you can enter the connection details of your Hadoop cluster.

    I recognize what you say about the job apparently finishing without problems but in the meantime nothing happens on the Hadoop side. Its frustrating. There is no relevant error catching for this.

    Greetz Jasper

  3. #3
    Join Date
    Sep 2011
    Posts
    9

    Default

    yup! that was it. Thank you so much for the tips.



    Quote Originally Posted by Jasper View Post
    Hi Kenneth,

    The thing is when you use the simple version of the job executor the pointers to your hadoop cluster have to be in the jar itself. You have to do some settings for the conf object in java and recompile it.

    Its different from when you use the advanced tab, were you can enter the connection details of your Hadoop cluster.

    I recognize what you say about the job apparently finishing without problems but in the meantime nothing happens on the Hadoop side. Its frustrating. There is no relevant error catching for this.

    Greetz Jasper

  4. #4
    Join Date
    Jul 2012
    Posts
    1

    Default

    Hi, I am new to pentaho. I am having a similar issue. I have a simple wordcount.jar that works fine outside of spoon. But not working with Hadoop job extractor in simple mode. I have updated the hadoop configuration with host and tasktracker info. Whatelse am I missing here? Can you tell me how you resolved the issue.
    My env: windows

    2012/07/21 10:16:34 - kettlesamplehadoop - Starting entry [WordCount - Simple]
    2012/07/21 10:16:34 - kettlesamplehadoop - exec(2, 0, WordCount - Simple.0)
    2012/07/21 10:16:34 - WordCount - Simple - Starting job entry
    2012/07/21 10:16:34 - WordCount - Simple - Using jar path: file:/wordCountSample/wordcount.jar
    2012/07/21 10:16:34 - WordCount - Simple - Running Hadoop Job in Simple Mode
    2012/07/21 10:16:34 - kettlesamplehadoop - Starting entry [Success]
    2012/07/21 10:16:34 - kettlesamplehadoop - exec(3, 0, Success.0)
    2012/07/21 10:16:34 - Success - Starting job entry
    2012/07/21 10:16:34 - kettlesamplehadoop - Finished job entry [Success] (result=[true])
    2012/07/21 10:16:34 - kettlesamplehadoop - Finished job entry [WordCount - Simple] (result=[true])
    2012/07/21 10:16:34 - kettlesamplehadoop - Finished job entry [Clean Output] (result=[true])
    2012/07/21 10:16:34 - kettlesamplehadoop - Job execution finished
    2012/07/21 10:16:34 - Spoon - Job has ended.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.