Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: How do kettle write hadoop job xml

  1. #1
    Join Date
    May 2012
    Posts
    6

    Default How do kettle write hadoop job xml

    Hi everyone:
    when I use kettle to develop hadoop mapReduce job. I encounter a issue. I have configurated mapred.local.dir in mapred-site.xml file(both namenode and datanode). when I just run pig (no use kettle to run pig), mapred.local.dir of job xml match which I configurated. but when I use kettle to submit pig job or mapReduce job. The mapred.local.dir are not what I configurated. They are always ${hadoop.tmp.dir}/mapred/local. I so confuse about it. What should I do to make job xml(kettle submit) like what I want.
    Thanks for everyone give me help and advice.

  2. #2
    Join Date
    Aug 2010
    Posts
    87

    Default

    You can set any custom properties under the User Defined tab of the Pentaho MapReduce or Hadoop Job Executor steps. You can also make sure the hadoop configuration in mapred-site.xml is on the Kettle class path (any directory under $KETTLE/libext should work for you). We include a mapred-site.xml at $KETTLE/libext/bigdata/pigConf/mapred-site.xml so you can edit that one if you'd rather not set the properties for every Hadoop job.

  3. #3
    Join Date
    May 2012
    Posts
    6

    Default

    Thanks for your help jganoff. That's very helpful.
    Last edited by lyllbl; 05-29-2012 at 02:27 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.