Pentaho MapReduce ClassNotFoundException: KettleException

[joniloky]

Hi guys,
We are putting together a Job that uses a Pentaho MapReduce Transformation. We are trying to run it using Spoon but when it reaches the MapReduce step we get the following stacktrace:

2012/08/13 20:03:35 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : [FAILED] -- Task: 0  Attempt: 0  Event: 1 
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Error: java.lang.ClassNotFoundException: org.pentaho.di.core.exception.KettleException
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.security.AccessController.doPrivileged(Native Method)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName0(Native Method)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName(Class.java:247)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:790)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
2012/08/13 20:03:35 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.Child.main(Child.java:170)
2012/08/13 20:03:41 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : [FAILED] -- Task: 0  Attempt: 1  Event: 2 
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Error: java.lang.ClassNotFoundException: org.pentaho.di.core.exception.KettleException
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.security.AccessController.doPrivileged(Native Method)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName0(Native Method)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName(Class.java:247)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:790)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
2012/08/13 20:03:41 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.Child.main(Child.java:170)
2012/08/13 20:03:47 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : [FAILED] -- Task: 0  Attempt: 2  Event: 3 
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) : Error: java.lang.ClassNotFoundException: org.pentaho.di.core.exception.KettleException
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.security.AccessController.doPrivileged(Native Method)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName0(Native Method)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at java.lang.Class.forName(Class.java:247)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:762)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:807)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.JobConf.getMapRunnerClass(JobConf.java:790)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
2012/08/13 20:03:47 - Pentaho MapReduce - ERROR (version 4.3.0-stable, build 16786 from 2012-04-24 14.11.32 by buildguy) :     at org.apache.hadoop.mapred.Child.main(Child.java:170)

Any thoughts?

Thanks!

[joniloky]

Here's more info.

This is what my mapred-site.xml looks like:

<configuration>
  <property>
    <name>mapred.job.tracker</name>
    <value>localhost:9001</value>
    <description>The host and port that the MapReduce job tracker runsat.</description>
  </property>
  <property>
    <name>pentaho.kettle.home</name>
    <value>/opt/pentaho/mapreduce/4.3.0</value>
  </property>
  <property>
    <name>pentaho.kettle.plugins.dir</name>
    <value>/opt/pentaho/mapreduce/4.3.0/plugins</value>
  </property>
</configuration>

I've also added the following to my hadoop-env.sh

export HADOOP_CLASSPATH=/opt/pentaho/mapreduce/4.3.0/lib/*:$HADOOP_CLASSPATH

Any thoughts?

I will reeeeaaaaally appreciate your help guys!
Cheers!

[MattCasters]

I can recommend searching the Big Data forum or posting there.