Hitachi Vantara Pentaho Community Forums
Results 1 to 8 of 8

Thread: Hadoop with PDI CE 4.2.1

  1. #1
    ronans Guest

    Default Hadoop with PDI CE 4.2.1

    Hi all

    is use of the hadoop job executor step supported in the PDI 4.2.1 community edition without installation of the enterprise components (PHD EE) on the hadoop nodes ? or is PHD stil needed for pure jar based jobs.

    If so, is there a community edition version that matches 4.2.1 and what is the link to it

    Ronan S.

  2. #2
    Join Date
    Sep 2009
    Posts
    810

    Default

    Ronan,

    beginning with 4.3. pre-release CE, kettle includes all big data components. The release is linked on the big data community home:
    http://wiki.pentaho.com/display/BAD/...Community+Home

    Please check out the child pages of that page, there is a growing number of updated docs and how-to's to get things up and running with the CE version

    Cheers
    Slawo

  3. #3
    ronans Guest

    Default

    Thanks Slawo,

    However in the meantime, can 4.2.1 community edition be used for executing jar based jobs or does it require the PHD ?

    Ronan

  4. #4
    Join Date
    Sep 2009
    Posts
    810

    Default

    for 4.2.1 you'd have to get PHD, as far as I know

  5. #5
    dmoran Guest

    Default

    PHD is required in 4.2.1 and 4.3 Pre release. We have a solution that gets rid of PHD that we will try to get in for 4.3 RC and definitely by 4.3 GA

  6. #6
    Join Date
    Aug 2010
    Posts
    87

    Default

    Hi Ronan,

    You can use 4.2.1 CE to submit jobs using a jar. That functionality is provided through the "Hadoop Job Executor" step. With this you can use a custom MR job with Java-code or even Hadoop streaming. It's a thin UI wrapper around the JobClient interface for submitting Hadoop jobs.

    Hope this helps,
    Jordan

  7. #7
    ronans Guest

    Default

    To clear up any misunderstanding, this was for to execute pure java jar based jobs (not pentaho map reduce transformations), and I was able to confirm that they can execute under 4.2.1 without installing PHD on the hadoop nodes.

  8. #8
    Join Date
    Aug 2010
    Posts
    87

    Default

    Happy to hear that, ronans!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.