Hadoop Transformation Job Executor: Can not get the relative path

[Jasper]

Hi,

Almost at the end of my Hadoop Transformation Job Executor, I suddenly get this error:


---------------------------------------------------------------------------------------------------------
2011/01/05 23:56:31 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 98.76033 Reducer Completion: 0.0
2011/01/05 23:56:31 - Hadoop Count Unique Impressions Job Executor - [SUCCEEDED] -- Task: 0 Attempt: 0 Event: 1
2011/01/05 23:56:36 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 0.0
2011/01/05 23:56:36 - Hadoop Count Unique Impressions Job Executor - [SUCCEEDED] -- Task: 1 Attempt: 0 Event: 2
2011/01/05 23:56:41 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 0.0
2011/01/05 23:56:46 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 16.666668
2011/01/05 23:56:51 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 16.666668
2011/01/05 23:56:56 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 16.666668
2011/01/05 23:57:01 - Hadoop Count Unique Impressions Job Executor - Setup Complete: 100.0 Mapper Completion: 100.0 Reducer Completion: 0.0
2011/01/05 23:57:01 - Hadoop Count Unique Impressions Job Executor - ERROR (version 4.1.0-GA, build 14380 from 2010-11-09 17.25.17 by buildguy) : [FAILED] -- Task: 0 Attempt: 0 Event: 3
2011/01/05 23:57:01 - Hadoop Count Unique Impressions Job Executor - ERROR (version 4.1.0-GA, build 14380 from 2010-11-09 17.25.17 by buildguy) : java.io.IOException: Can not get the relative path: base = hdfs://rhhadoop1/user/pentaho/output/_temporary/_attempt_201101052011_0007_r_000000_0 child = hdfs://RHHadoop1/user/pentaho/output/_temporary/_attempt_201101052011_0007_r_000000_0/part-00000
at org.apache.hadoop.mapred.FileOutputCommitter.getFinalPath(FileOutputCommitter.java:150)
at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:106)
at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:126)
at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:86)
at org.apache.hadoop.mapred.OutputCommitter.commitTask(OutputCommitter.java:171)
at org.apache.hadoop.mapred.Task.commit(Task.java:779)
at org.apache.hadoop.mapred.Task.done(Task.java:691)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
at org.apache.hadoop.mapred.Child.main(Child.java:170)

--------------------------------------------------------------------------------------

I see in the error "Can not get the relative path: base = hdfs://rhhadoop1/user/pentaho/output/_temporary/....... " that it has forgotten all about the 8020 port. Because it should be hdfs://rhhadoop1:8020/user/pentaho/.........

I don't get it..:(

[jganoff]

It looks like a subtle configuration issue with the way you're defining/using the canonical address of the NameNode ("rhhadoop1"). The message of the IOException shows both "rhhadoop1" and "RHHadoop1". I assume one resolves and the other does not.

[Jasper]

Quite right, it turns out that this issue occurs when you use uppercase for the hadoop network addresses (hdfs & mapred) on the cluster tab of the hadoop trans job executor. So don't use uppercase there, best to do that not at all.