I am running PDI Pentaho MapReduce job on CDH5.8 (Cloudera Quickstart VM). I always get either "java.lang.OutOfMemoryError: GC overhead limit exceeded" or "java.lang.OutOfMemoryError: Java heap space" I did the following the adjust the heap size and GC overhead limit. Can someone suggest me what other thinkg I should do to overcome this problem.

1) I have increased memory in Spoon.bat and Spoon.sh (in Spoon.bat file, I have set
"%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms2048m" "-Xmx2048m" "-XX:MaxPermSize=2048m")
2) In hadoop-env.sh (/etc/hadoop/conf in CDH5.8), I have added "export HADOOP_CLIENT_OPTS="-XX:-UseGCOverheadLimit -Xmx4096m"
3) I have added the following two properties in mapred-site.xml (/etc/hadoop/conf in CDH5.8)
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx2048m -XX:-UseGCOverheadLimit</value>
</property>
<property>
<name>mapred.job.reuse.jvm.num.tasks</name>
<value>1</value>
</property>
4) I have set - mapreduce.job.heap.memory-mb.ratio to 0.8

I am attaching the Pentaho PDI job log as below.
2016/10/25 14:40:11 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000001_0 Attempt: attempt_1477386149872_0001_m_000001_0 Event: 0
2016/10/25 14:40:11 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.HashMap.inflateTable(HashMap.java:316)
at java.util.HashMap.put(HashMap.java:488)
at java.util.jar.Attributes.put(Attributes.java:148)
at java.util.jar.Attributes.putValue(Attributes.java:168)
at java.util.jar.Attributes.read(Attributes.java:421)
at java.util.jar.Manifest.read(Manifest.java:251)
at java.util.jar.Manifest.<init>(Manifest.java:69)
at java.util.jar.JarFile.getManifestFromReference(JarFile.java:181)
at java.util.jar.JarFile.getManifest(JarFile.java:167)
at sun.net.http://www.protocol.jar.URLJarFile.i...File.java:174)
at sun.net.http://www.protocol.jar.URLJarFile.g...File.java:142)
at java.util.jar.JarFile.maybeInstantiateVerifier(JarFile.java:309)
at java.util.jar.JarFile.getInputStream(JarFile.java:410)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2016/10/25 14:40:16 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2016/10/25 14:40:16 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000000_0 Attempt: attempt_1477386149872_0001_m_000000_0 Event: 1
2016/10/25 14:40:16 - Pentaho MapReduce - Exception from container-launch.
Container id: container_1477386149872_0001_01_000002
Exit code: 255
Stack trace: ExitCodeException exitCode=255:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:578)
at org.apache.hadoop.util.Shell.run(Shell.java:481)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:763)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:213)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)




Container exited with a non-zero exit code 255

2016/10/25 14:40:51 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000001_1 Attempt: attempt_1477386149872_0001_m_000001_1 Event: 2
2016/10/25 14:40:51 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringCoding$StringDecoder.decode(StringCoding.java:149)
at java.lang.StringCoding.decode(StringCoding.java:193)
at java.lang.String.<init>(String.java:416)
at java.util.jar.Attributes.read(Attributes.java:418)
at java.util.jar.Manifest.read(Manifest.java:251)
at java.util.jar.Manifest.<init>(Manifest.java:69)
at java.util.jar.JarFile.getManifestFromReference(JarFile.java:181)
at java.util.jar.JarFile.getManifest(JarFile.java:167)
at sun.net.http://www.protocol.jar.URLJarFile.i...File.java:174)
at sun.net.http://www.protocol.jar.URLJarFile.g...File.java:142)
at java.util.jar.JarFile.maybeInstantiateVerifier(JarFile.java:309)
at java.util.jar.JarFile.getInputStream(JarFile.java:410)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
2016/10/25 14:40:56 - Pentaho MapReduce - Setup Complete: 100.0 Mapper Completion: 0.0 Reducer Completion: 0.0
2016/10/25 14:40:56 - Pentaho MapReduce - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : [FAILED] -- Task: attempt_1477386149872_0001_m_000000_1 Attempt: attempt_1477386149872_0001_m_000000_1 Event: 3
2016/10/25 14:40:56 - Pentaho MapReduce - Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:75)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:449)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
... 9 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.String.substring(String.java:1913)
at sun.security.x509.X509AttributeName.<init>(X509AttributeName.java:53)
at sun.security.x509.X509CertInfo.get(X509CertInfo.java:572)
at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1784)
at sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:196)
at sun.security.provider.X509Factory.engineGenerateCertificate(X509Factory.java:97)
at java.security.cert.CertificateFactory.generateCertificate(CertificateFactory.java:339)
at sun.security.pkcs.PKCS7.parseSignedData(PKCS7.java:299)
at sun.security.pkcs.PKCS7.parse(PKCS7.java:158)
at sun.security.pkcs.PKCS7.parse(PKCS7.java:126)
at sun.security.pkcs.PKCS7.<init>(PKCS7.java:108)
at sun.security.util.SignatureFileVerifier.<init>(SignatureFileVerifier.java:96)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:296)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:348)
at java.util.jar.JarFile.getInputStream(JarFile.java:415)
at sun.net.http://www.protocol.jar.JarURLConnec...tion.java:162)
at java.net.URL.openStream(URL.java:1037)
at org.apache.karaf.main.KarafActivatorManager.startKarafActivators(KarafActivatorManager.java:61)
at org.apache.karaf.main.Main.launch(Main.java:253)
at org.pentaho.platform.osgi.KarafBoot$3.run(KarafBoot.java:209)
at java.lang.Thread.run(Thread.java:745)
at org.pentaho.platform.osgi.KarafBoot.startup(KarafBoot.java:217)
at org.pentaho.di.osgi.registryExtension.OSGIPluginRegistryExtension.init(OSGIPluginRegistryExtension.java:105)
at org.pentaho.di.core.plugins.PluginRegistry.init(PluginRegistry.java:557)
at org.pentaho.di.core.KettleClientEnvironment.init(KettleClientEnvironment.java:101)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:97)
at org.pentaho.di.core.KettleEnvironment.init(KettleEnvironment.java:75)
at org.pentaho.hadoop.mapreduce.MRUtil.initKettleEnvironment(MRUtil.java:101)
at org.pentaho.hadoop.mapreduce.MRUtil.getTrans(MRUtil.java:59)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.createTrans(PentahoMapRunnable.java:252)
at org.pentaho.hadoop.mapreduce.PentahoMapRunnable.configure(PentahoMapRunnable.java:180)