I have set up a transformation to load file to HDFS, it failed with error like this,
2011/09/28 17:17:58 - Hadoop File Output.0 - We can not find parent folder [hdfs://root:111111@wangdjvm/data/hadoop_data]!
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : Couldn't open file hdfs://root:111111@wangdjvm/data/hadoop_data/ss
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : org.pentaho.di.core.exception.KettleException:
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : Error opening new file : org.pentaho.di.core.exception.KettleException:
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : We can not find folder [hdfs://root:111111@wangdjvm/data/hadoop_data]! You need to create it before generating file [hdfs://root:111111@wangdjvm/data/hadoop_data/ss].
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) :
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) :
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.openNewFile(TextFileOutput.java:683)
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : at org.pentaho.di.trans.steps.textfileoutput.TextFileOutput.init(TextFileOutput.java:776)
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : at org.pentaho.di.trans.step.StepInitThread.run(StepInitThread.java:52)
2011/09/28 17:17:58 - Hadoop File Output.0 - ERROR (version 4.2.0-stable, build 15748 from 2011-09-08 13.11.42 by buildguy) : at java.lang.Thread.run(Thread.java:662)

I run PDI on the server as datanode, here is the information about this sever,
--- the folder actually exists.
[@wangdjvm bin]# ./hadoop dfs -ls /data/hadoop_data
Found 1 items
-rw-r--r-- 1 root supergroup 54 2011-09-27 16:42 /data/hadoop_data/s

--- here is HDFS conf
[@wangdjvm bin]# cat /data/hadoop-0.20.203.0/conf/core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://wangdjvm:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/data/hadoop-0.20.203.0/data/tmp</value>
</property>
</configuration>

--- Here is /etc/hosts.
[@wangdjvm bin]# cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
192.168.8.28 wangdjvm

I am very confused by this issue. Could anyone help me out? Thanks in advance!


Best wishes,
Satine