PDA

View Full Version : Hadoop Transformation Job Executor: Base class does not have a trans



Jasper
12-08-2010, 09:33 AM
Hi,

When I run the out-of-the-box "Hadoop Trans Job Executor - weblogs.kjb" sample job I now get this error when it arrives at the Hadoop Transformation Job Executor;

----------------------------------
2010/12/08 14:16:29 - Hadoop Transformation Job Executor - Setup Complete: 0.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2010/12/08 14:16:34 - Hadoop Transformation Job Executor - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2010/12/08 14:16:34 - Hadoop Transformation Job Executor - [SUCCEEDED] -- Task: 5 Attempt: 0 Event: 0
2010/12/08 14:16:39 - Hadoop Transformation Job Executor - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2010/12/08 14:16:44 - Hadoop Transformation Job Executor - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2010/12/08 14:16:44 - Hadoop Transformation Job Executor - ERROR (version 4.1.0-GA, build 14380 from 2010-11-09 17.25.17 by buildguy) : [FAILED] -- Task: 0 Attempt: 0 Event: 1
2010/12/08 14:16:44 - Hadoop Transformation Job Executor - ERROR (version 4.1.0-GA, build 14380 from 2010-11-09 17.25.17 by buildguy) : java.io.IOException: java.lang.RuntimeException: Base class does not have a trans.
at org.pentaho.hadoop.mapreduce.GenericTransMap.map(SourceFile:188)
at org.pentaho.hadoop.mapreduce.GenericTransMap.map(SourceFile:22)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.RuntimeException: Base class does not have a trans.
at org.pentaho.hadoop.mapreduce.GenericTransMap.map(SourceFile:35)
... 5 more

------------------------------------

I don't get it because the job used to run fine before.

There is an Jira issue [#PDI-4718] about this which should have been fixed in 4.1.0 GA
so I hope its not back again.

Jasper
12-09-2010, 05:27 PM
Could be that the "Base class does not have a trans" error is raised when there is something wrong with the mapper sub.ktr which is translated for execution in Hadoop.

Can somebody confirm or is there something else?

jganoff
01-04-2011, 10:39 AM
Generally this is a configuration issue. In regards to the nondescript error, this will be be fixed in PDI 4.1.1. See http://jira.pentaho.com/browse/PDI-5062 for more information.

Jasper
01-05-2011, 05:41 AM
Hi,

Could it be that the Hadoop Pentaho Job Executor can not take ktr files that are not from a repository ?

jganoff
01-05-2011, 09:59 AM
You should be able to specify files from the three classification of locations: local, repository by name, and repository by reference.

acimi
01-05-2011, 12:11 PM
Generally this is a configuration issue. In regards to the nondescript error, this will be be fixed in PDI 4.1.1. See http://jira.pentaho.com/browse/PDI-5062 for more information.

What is the work-around for this configuration issue?
Thanks.

jganoff
01-05-2011, 02:36 PM
What is the work-around for this configuration issue?
Thanks.

In general, there are a few common areas to check when trying to debug the "Base class does not have a trans" error in PHD 4.1.0-GA. Try these:

1) Verify that you can run the mapper/reducer transformation locally. You can simply create an input step to replace the Injector to simulate key/value pairs coming in to the transformation from Hadoop.
2) Verify that any plugins and required java libraries you're using are correctly installed on each node.
2a) The plugins need to be placed in $USER_HOME/.kettle/plugins due to a bug in PDI 4.1.0-GA noted in PDI-5062 (http://jira.pentaho.com/browse/PDI-5062).
2b) Additional libraries (e.g. database drivers) should be placed in $HADOOP_HOME/lib.

- Jordan

Jasper
01-05-2011, 06:53 PM
Hi Jordan,
You just made my day!

Doing 2a. on all my nodes gave me a more precise error message at first. I had not renamed my key/value pairs to the required names outKey and outValue in the mapper and reducer ktr's.

Now after almost a month it is finally doing the job again, .

Thanks a lot!

Jasper
01-05-2011, 07:24 PM
Immediately followed by the next problem.

See http://forums.pentaho.com/showthread.php?80056-Hadoop-Transformation-Job-Executor-Can-not-get-the-relative-path&p=249958#post249958