PDA

View Full Version : Pentaho with IDH shim - Unable to connec to Intel hadoop cluster



naren_bigInfo
10-17-2013, 08:44 AM
Hello,
I am using Intel distribution of apache hadoop and I properly set up the single node cluster.

I am Using pentaho and I am trying text input to hbase output. I am referring intel hadoop's hbase-site.xml.

I have downloaded bigdata plugin 1.3.3.1 and copied under plugins/pentaho-big-data-plugin/
under hadoop-configuration I have placed idh23 folder too which is specific for intel.I have updated plugin.properties to point to idh23 folder.

Inside pentaho under idh23 folder hbase-site.xml is also available and it has the following entry:
<!--Thu May 30 10:40:32 2013-->
<configuration>
<property>
<name>hbase.rpc.engine</name>
<value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>
</property>
</configuration>

Getting the following Issue in intel hbase log:
2013-10-17 18:07:05,833 WARN org.apache.hadoop.ipc.SecureServer: Incorrect header or version mismatch from 192.168.1.53:40026 got version 3 expected version 4

Pentaho got stuck and unable get table . Getting the following issue in pentaho log:
java.io.IOException: Call to localhost/192.168.1.53:60000 failed on local exception: java.io.EOFException

I think its more on secure rpc engine issue. May be this property not getting set up in pentaho. Kindly help me to resolve this issue.

Regards,
Naren.

Manoj S
10-18-2013, 06:24 AM
Naren,

||2013-10-17 18:07:05,833 WARN org.apache.hadoop.ipc.SecureServer: Incorrect header or version mismatch from 192.168.1.53:40026 got version 3 expected version 4

This error has come up because you are using an older version of the API to connect. More details about this error and how to resolve is available at https://intelhadoop.zendesk.com/entries/23838313-HBase-client-failure-Scenario-2

Also, please note that a java.io.IOException occours when the socket mentioned in the logs is in-accessible. (192.168.1.53:60000 in your case)

I came across another post on Pentaho forums that might be of help to you. http://forums.pentaho.com/showthread.php?149592-compatibility-issues-pdi-ce-4-4-0-stable-pentaho-big-data-shimtastic-1-3-3-1-version

Regards,
Manoj

naren_bigInfo
12-06-2013, 04:35 AM
Actually through Pentaho client only we are connecting to intel hadoop.

My Pentaho kettle referring hbase 0.94.1 only and also my IDH using hbase 0.94.1. In this case where the mismatch is coming.

How to set the java classpath entries for hbase-site.xml, hdfs-site.xml, core-site.xml.