Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: Resource Manager and Job History server not showing Map Reduce jobs

  1. #1
    Join Date
    Oct 2015
    Posts
    2

    Default Resource Manager and Job History server not showing Map Reduce jobs

    I have AWS EMR 3.4 setup. I use Pentaho version is 5.0
    I am trying to do copy data from my local to HDFS. I assume it in turn runs as MR jobs. If so I am not seeing any Job ID in the Resource Manager.
    Also I'm running some Pig scripts on top of data in HDFS(which sits on the EMR cluster). Resource Manager and Job History server not showing Map Reduce jobs in the list of jobs.. but it is showing the jobs which I run in pig grunt shell.
    I am running both jobs as hadoop user.
    Please help.

  2. #2
    Join Date
    Sep 2012
    Posts
    71

    Default

    Our Amazon EMR shim (which supports steps/entries like "Hadoop Copy Files") is not supported in Pentaho 5.0, our first product to support this is Pentaho 5.4. As a side note, the Copy Files step/entry does not create a MapReduce job, it uses an HDFS client to do a copy from your local area to AWS (which is S3 not HDFS?)

  3. #3
    Join Date
    Oct 2015
    Posts
    2

    Default

    Thank you very much..!
    I am sorry we are using Pentaho 5.4 only.
    Then how about the Mapreduce jobs not showing in Resource Manager and Job History server when I run PDI jobs with pig script..?
    I can see mapreduce job if I run the same pig script through grunt shell.

    thanks mattb_pdi

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.