Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: fail to run sqoop export/import via kettle

  1. #1

    Default fail to run sqoop export/import via kettle

    Hi all

    In the big data part, I found two compoents, Sqoop Export/Sqoop Import.
    When running sqoop import or export via kettle, I always met some errors.
    The error for import:
    2013/08/08 16:27:41 - Sqoop Import - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : 2013/08/08 16:27:41 - java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.sqoop.mapreduce.RawKeyTextOutputFormat not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1587)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:227)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
    Caused by: java.lang.ClassNotFoundException: Class org.apache.sqoop.mapreduce.RawKeyTextOutputFormat not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1493)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1585)
    ... 8 more

    The error for export:
    2013/08/08 16:17:35 - Sqoop Export - ERROR (version 4.4.0-stable, build 17588 from 2012-11-21 16.02.21 by buildguy) : 2013/08/08 16:17:35 - java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.sqoop.mapreduce.ExportOutputFormat not found
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1587)
    at org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:227)
    at org.apache.hadoop.mapred.Task.initialize(Task.java:515)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:314)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
    at org.apache.hadoop.mapred.Child.main(Child.java:262)
    Caused by: java.lang.ClassNotFoundException: Class org.apache.sqoop.mapreduce.ExportOutputFormat not found
    at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1493)
    at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1585)
    ... 8 more

    Is there anything that I need to configure?
    Thanks.
    Maria

  2. #2

    Default

    Did anyone meet this issue before?
    Is there any solution?

  3. #3
    Join Date
    Sep 2013
    Posts
    1

    Default fail to run sqoop export/import via kettle

    I have the problem too,did someone meet this or had used sqoop import/export successfully?
    thanks!

  4. #4

    Default

    I have the same problem in windows, and I checked the code and the job.xml file in the Tasktractor.


    Finally I found it caused by the classpath in job.xml:


    <property><name>mapred.job.classpath.archives</name><value>/home/tekken/hadoop-1.0.3/tmp/mapred/staging/tekken/.staging/job_201310100157_0013/libjars/sqoop-1.4.1-incubating-hadoop-20.jar;/home/tekken/hadoop-1.0.3/tmp/mapred/staging/tekken/.staging/job_201310100157_0013/libjars/mysql-connector-java-3.1.14-bin.jar</value></property>


    the seperator in classpath is ";" not ":"


    I am not sure whether it is a bug.
    But I fixed the code, Added the following statement in the method executeJobEnry of AbstractSqoopJobEntry class


    System.setProperty("path.separator",":");


    I attached two classes as a patch of this problem. (tested on kettle 4.4 windows)


    you can download the two classes, and replace the same class files in the pentaho-big-data-plugin-TRUNK-SNAPSHOT.jar


    I think It will works!
    Last edited by jianjunchu; 10-11-2013 at 02:25 AM.
    Jason chu
    Beijing Aofei BI Consulting.
    Web: http://www.pentahochina.com
    Email: jianjunchu@gmail.com

  5. #5

    Default

    patch file
    Attached Files Attached Files
    Jason chu
    Beijing Aofei BI Consulting.
    Web: http://www.pentahochina.com
    Email: jianjunchu@gmail.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.