PDA

View Full Version : Unable to load class for step/plugin with id [HadoopEnterPlugin]



gao1738
03-26-2012, 12:54 AM
Hi all,

I am a newer in kettle, and I am going to try the intergration of kettle and hadoop.
I set up the hadoop as pseudo-distributed mode. And it started correctly:

gaoxiaoxin@gaoxiaoxin-Lenovo-G470:~/kettle/hadoop/hadoop-0.20.2$ jps
6113 DataNode
6426 JobTracker
6647 TaskTracker
5881 NameNode
6761 Jps
6343 SecondaryNameNode

And I install the PHD in my machine as follow:

22 <property>
23 <name>pentaho.kettle.home</name>
24 <value>/home/gaoxiaoxin</value>
25 </property>
26
27 <property>
28 <name>pentaho.kettle.plugins.dir</name>
29 <value>/home/gaoxiaoxin/plugins</value>
30 </property>
(I put the lib and plugin floders of PHD in my home folder, because the .kettle folder is there)

then I follow the guid: http://wiki.pentaho.com/display/BAD/Using+Pentaho+MapReduce+to+Parse+Weblog+Date

when I start to run the job, I get the following exceptions: (sorry , there are some Chinese words, I write the related english version beside...)


------------------------------------------------------------------------
INFO 26-03 11:30:01,508 - Spoon - Spoon
INFO 26-03 11:30:41,956 - Spoon - Spoon
INFO 26-03 11:31:47,731 - Spoon - Spoon
INFO 26-03 11:32:11,300 - Spoon - 正在开始任务... (start the mission)
INFO 26-03 11:32:11,300 - weblog_parse_mr - 开始执行任务 (begin to exec the mission)
INFO 26-03 11:32:11,305 - weblog_parse_mr - 开始项[Pentaho MapReduce]
WARN 26-03 11:32:11,342 - could not parse [vertical] as Align value
INFO 26-03 11:32:11,435 - Pentaho MapReduce - Configuring for Hadoop distribution: Cloudera
WARN 26-03 11:32:11,636 - could not parse [vertical] as Align value
INFO 26-03 11:32:11,917 - Pentaho MapReduce - Cleaning output path: hdfs://localhost:9000/user/pdi/weblogs/parse
WARN 26-03 11:32:12,029 - Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
INFO 26-03 11:32:12,305 - Total input paths to process : 1
INFO 26-03 11:32:13,069 - Pentaho MapReduce - Setup Complete: 0.0 Mapper Completion: 0.0 Reducer Completion: 0.0
INFO 26-03 11:32:23,077 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
INFO 26-03 11:32:33,097 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
ERROR 26-03 11:32:33,106 - Pentaho MapReduce - [FAILED] -- Task: 0 Attempt: 0 Event: 1
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error loading transformation
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:180)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:150)
... 10 more
Caused by: org.pentaho.di.core.exception.KettleXMLException:
错误从XML文件读取转换 (error during reading transform from the xml file )

Unable to load step info from XML step nodeorg.pentaho.di.core.exception.KettleStepLoaderException:
Unable to load class for step/plugin with id [HadoopEnterPlugin]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution.


Unable to load class for step/plugin with id [HadoopEnterPlugin]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution.

at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2880)
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2429)
at org.pentaho.di.trans.TransMeta.loadXML(TransMeta.java:2415)
at org.pentaho.di.trans.TransMeta.<init>(TransMeta.java:2402)
at org.pentaho.di.trans.TransConfiguration.<init>(TransConfiguration.java:69)
at org.pentaho.di.trans.TransConfiguration.fromXML(TransConfiguration.java:76)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:76)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:178)
... 11 more
Caused by: org.pentaho.di.core.exception.KettleXMLException:
Unable to load step info from XML step nodeorg.pentaho.di.core.exception.KettleStepLoaderException:
Unable to load class for step/plugin with id [HadoopEnterPlugin]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution.


Unable to load class for step/plugin with id [HadoopEnterPlugin]. Check if the plugin is available in the plugins subdirectory of the Kettle distribution.
......

Is there anyone know what is the problem?
Thanks in advance!!!

cdeptula
03-26-2012, 11:48 AM
Hi Gao,

Off the top of my head this may be caused by a permissions issue with using the home directory. Make sure the user who is running the task and job tracker daemons has read and execute access to the subdirectories of /home/gaoxiaoxin/plugins.

Also, make sure the kettle-gpload-plugin and pentaho-big-data-plugin directories exist in your plugins folder.

Chris

gao1738
03-27-2012, 08:25 AM
Thanks a lot , Chris. I guess you are correct.
When I change the "pentaho.kettle.home" to the "/opt/pentaho/..." as the guide and create a new repository, I works.
Thanks for your kindly reply again!