PDA

View Full Version : Hadoop Copy Files - How to set HDFS User?



RainerWahnsinn
02-22-2013, 01:56 PM
Hello all,

I am in the process of evaluating Pentaho Data Integration for my customer. We have set up a Hadoop Cluster and I can copy files to the HDFS via command line.

If I try to Copy a file with the Hadoop Copy Files step, I get an exception in den Hadoop log. Something like




17:10:47.926 Uhr
WARN
org.apache.hadoop.security.UserGroupInformation

No groups available for user Rainer Wahnsinn



17:10:47.987 Uhr
WARN
org.apache.hadoop.security.ShellBasedUnixGroupsMapping

got exception trying to get groups for user Rainer Wahnsinn
org.apache.hadoop.util.Shell$ExitCodeException: id: extra operand `Wahnsinn'
Try `id --help' for more information.



I can browse the HDFS with the Hadoop Copy Files component fine. My connection string looks like hdfs://hdfs@<server>:8020/tmp

I thought this way, the file is copied with the hdfs user. But somehow Pentaho is using my Windows Account or something...

The Error in Pentaho is just "Connection refused"

I would really appreciate if someone could help me.


Thanks in advance,
RW