Hitachi Vantara Pentaho Community Forums
Results 1 to 2 of 2

Thread: MapReduce On PDI

  1. #1
    Join Date
    Apr 2013
    Posts
    2

    Default MapReduce On PDI

    Hello,

    this is my first post on this forum, so i'm sorry if this topic is already posted.

    So this is my problem :
    I would like to connect my hdfs cluster for a MapReduce Job. But I have this error in log file when I try to throw it :

    Code:
    ERROR 12-04 12:18:08,728 - PriviledgedActionException as:aloutfi (auth:SIMPLE) cause:java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "LIN00024/127.0.1.1"; destination host is: "slave1":50030; java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "LIN00024/127.0.1.1"; destination host is: "slave1":50030; 
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:765)
        at org.apache.hadoop.ipc.Client.call(Client.java:1165)
        at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
        at org.apache.hadoop.mapred.$Proxy62.getStagingAreaDir(Unknown Source)
        at org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1223)
        at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:818)
        at org.pentaho.hadoop.shim.common.CommonHadoopShim.submitJob(CommonHadoopShim.java:201)
        at org.pentaho.di.job.entries.hadooptransjobexecutor.JobEntryHadoopTransJobExecutor.execute(JobEntryHadoopTransJobExecutor.java:806)
        at org.pentaho.di.job.Job.execute(Job.java:589)
        at org.pentaho.di.job.Job.execute(Job.java:728)
        at org.pentaho.di.job.Job.execute(Job.java:443)
        at org.pentaho.di.job.Job.run(Job.java:363)
    Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
        at com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
        at com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
        at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
        at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:282)
        at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
        at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
        at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
        at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:850)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
    ERROR 12-04 12:18:08,729 - Pentaho MapReduce - Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "LIN00024/127.0.1.1"; destination host is: "slave1":50030; 
    ERROR 12-04 12:18:08,730 - Pentaho MapReduce - java.io.IOException: Failed on local exception: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.; Host Details : local host is: "LIN00024/127.0.1.1"; destination host is: "slave1":50030; 
        at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:765)
        at org.apache.hadoop.ipc.Client.call(Client.java:1165)
        at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:224)
        at org.apache.hadoop.mapred.$Proxy62.getStagingAreaDir(Unknown Source)
        at org.apache.hadoop.mapred.JobClient.getStagingAreaDir(JobClient.java:1223)
        at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:102)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:844)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Unknown Source)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:844)
        at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:818)
        at org.pentaho.hadoop.shim.common.CommonHadoopShim.submitJob(CommonHadoopShim.java:201)
        at org.pentaho.di.job.entries.hadooptransjobexecutor.JobEntryHadoopTransJobExecutor.execute(JobEntryHadoopTransJobExecutor.java:806)
        at org.pentaho.di.job.Job.execute(Job.java:589)
        at org.pentaho.di.job.Job.execute(Job.java:728)
        at org.pentaho.di.job.Job.execute(Job.java:443)
        at org.pentaho.di.job.Job.run(Job.java:363)
    Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag.
        at com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:73)
        at com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:124)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:213)
        at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:746)
        at com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:238)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:282)
        at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:760)
        at com.google.protobuf.AbstractMessageLite$Builder.mergeDelimitedFrom(AbstractMessageLite.java:288)
        at com.google.protobuf.AbstractMessage$Builder.mergeDelimitedFrom(AbstractMessage.java:752)
        at org.apache.hadoop.ipc.protobuf.RpcPayloadHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcPayloadHeaderProtos.java:985)
        at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:850)
        at org.apache.hadoop.ipc.Client$Connection.run(Client.java:781)
    Can anyone could explain me why I have this error and how I can resolve it ?

    I worke with PDI 4.4.0 and Cloudera cdh4.


    Regards,

    Abdel H.

  2. #2
    Join Date
    Sep 2012
    Posts
    71

    Default

    What exact version of Cloudera CDH4 are you using? There are differences between various 4.x.x versions that may prevent you from using the "cdh4" configuration directly. Usually this can be remedied manually by copying certain JARs into your cdh4 configuration under PDI. Also, is the"active.hadoop.configuration" property in /data-integration/plugins/pentaho-big-data-plugin/plugin.properties set to cdh4?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.