Hitachi Vantara Pentaho Community Forums
Results 1 to 18 of 18

Thread: MongoDB Version Upgrade

  1. #1
    Join Date
    Dec 2015
    Posts
    11

    Default MongoDB Version Upgrade

    We just upgraded our MongoDB to version 3.0.7. This has broken myauthentication to MongoDB in Pentaho PDI. I really need to be able to access MongoDB again. Thanks for any help.

    See error below -

    An unexpected error occurred in Spoon: javax/crypto/spec/PBEKeySpec
    java.lang.NoClassDefFoundError: javax/crypto/spec/PBEKeySpec
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.Hi(DBPort.java:822)
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.computeClientFinalMessage(DBPort.java:747)
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.evaluateChallenge(DBPort.java:681)
    at com.mongodb.DBPort$SaslAuthenticator.authenticate(DBPort.java:904)
    at com.mongodb.DBPort.authenticate(DBPort.java:432)
    at com.mongodb.DBPort.checkAuth(DBPort.java:443)
    at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:289)
    at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:269)
    at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)
    at com.mongodb.DB.command(DB.java:320)
    at com.mongodb.DB.command(DB.java:299)
    at com.mongodb.DB.command(DB.java:388)
    at com.mongodb.DBApiLayer.getCollectionNames(DBApiLayer.java:152)
    at org.pentaho.mongo.wrapper.NoAuthMongoClientWrapper.getCollectionsNames(NoAuthMongoClientWrapper.java:197)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.setupCollectionNamesForDB(MongoDbInputDialog.java:1532)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.access$600(MongoDbInputDialog.java:82)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog$6.widgetSelected(MongoDbInputDialog.java:494)
    at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
    at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
    at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1339)
    at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7939)
    at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9214)
    at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:653)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)

  2. #2
    Join Date
    Dec 2015
    Posts
    11

    Default

    Anybody have any ideas here? It is causing me some headache at the moment.

  3. #3
    Join Date
    Dec 2015
    Posts
    1

    Default

    I have exactly the same problem. Any answer please ?

  4. #4
    Join Date
    Dec 2015
    Posts
    1

    Default

    javax/crypto/spec/PBEKeySpec is in the jce.jar file which is bundled with the JRE. It is probably due to PDI environment not finding your JRE or Java installation and is simply getting java from the path. Try setting either the JRE_HOME or JAVA_HOME environment variables to point to either your JRE or JDK. Pentaho set-pentaho-env.bat or set-pentaho-env.sh script should then be able to set up your environment correctly. NOTE: you can also use a Pentaho specific JRE that is different from your main OS java installation by setting the PENTAHO_JAVA_HOME environment variable.

    Hope that helps to get you back on the right track!

  5. #5
    Join Date
    Jan 2016
    Posts
    4

    Default

    Hello:

    I am having the exact same problem. I verified that jce.jar is located in the following folders:

    C:\Program Files\Java\jre1.8.0_65\lib
    C:\Program Files\Java\jdk1.8.0_65\jre\lib

    The environment is being set correctly since I get the following output when running Spoon.bat:

    DEBUG: Using PENTAHO_JAVA_HOME
    DEBUG: _PENTAHO_JAVA_HOME=C:\Program Files\Java\jre1.8.0_65
    DEBUG: _PENTAHO_JAVA=C:\Program Files\Java\jre1.8.0_65\bin\javaw.exe


    start "Spoon" "C:\Program Files\Java\jre1.8.0_65\bin\javaw.exe" "-Xms1024m" "-Xmx2048m" "-XX:MaxPermSize=256m" "-Dhttps.protocols=TLSv1,TLSv1.1,TLSv1.2" "-Djava.library.path=libswt\win64" "-DKETTLE_HOME=" "-DKETTLE_REPOSITORY=" "-DKETTLE_USER=" "-DKETTLE_PASSWORD=" "-DKETTLE_PLUGIN_PACKAGES=" "-DKETTLE_LOG_SIZE_LIMIT=" "-DKETTLE_JNDI_ROOT=" -jar launcher\pentaho-application-launcher-6.0.0.0-353.jar -lib ..\libswt\win64


    I tried setting PENTAHO_JAVA_HOME to C:\Program Files\Java\jdk1.8.0_65 but got the exact same result.

    Does anyone know of anything else I could try? I have been looking at this for a while but cannot find other references to similar issues online or on this forum. Any other information will be greatly appreciated.

    Thanks,
    Camuyano

  6. #6
    Join Date
    Jan 2016
    Posts
    4

    Default

    Any thoughts?

  7. #7
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    The two paths that you posted are not in your LIBPATH, so they won't be found.

    Try copying the JAR into %PENTAHO_DIRECTORY%\lib or %PENTAHO_DIRECTORY%\libswt\win64

  8. #8
    Join Date
    Jan 2016
    Posts
    4

    Default

    Gutlez,

    Thanks for the kind reply and sorry for the delay in answering. I did try to copy the file into those two directories but got the same result. I also updated launcher.properties and added the directory containing the file to both the classpath and libraries properties per this blog post.

    At this point I have almost given up in trying to debug the issue and started writing my own connector using the MongoDB Java driver. However, this issue is really bugging me. I see a lot of potential in this tool but have had a hard time debugging issues like this. I would appreciate any other thoughts anyone has.

    BTW: I cannot import my own .jar files inside UDJC either. I can instantiate the external class but cannot call public methods from inside the UDJC!

    Sincerely,
    Camuyano

  9. #9
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Try putting any jar file that you want to use in the %PENTAHO_INSTALL_DIR%/lib directory.

  10. #10
    Join Date
    Jan 2016
    Posts
    4

    Default

    Gutlez,

    Thanks again for the reply. I did put the .jar file for the MySQL driver there and it works without any issue. Also, I managed to call methods from classes contained in my own .jar files from within a UDJC. I also confirmed that jce.jar file is being loaded since I cannot move or delete it while Spoon is running. This is very puzzling and annoying.

    -Camuyano

  11. #11
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by Camuyano View Post
    This is very puzzling and annoying.
    What is puzzling and annoying?
    I made a mistake in my first response - putting jar files in /lib is the right place.

  12. #12
    Join Date
    Jan 2016
    Posts
    2

    Default

    Is there a solution to this problem. I tried DI - 6.0.1.0-386 with Mongo 2.6.9. I get same ClassNotFound Exception

    javax/crypto/spec/PBEKeySpec
    java.lang.NoClassDefFoundError: javax/crypto/spec/PBEKeySpec
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.Hi(DBPort.java:822)
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.computeClientFinalMessage(DBPort.java:747)
    at com.mongodb.DBPort$ScramSha1Authenticator$ScramSha1SaslClient.evaluateChallenge(DBPort.java:681)
    at com.mongodb.DBPort$SaslAuthenticator.authenticate(DBPort.java:904)
    at com.mongodb.DBPort.authenticate(DBPort.java:432)
    at com.mongodb.DBPort.checkAuth(DBPort.java:443)
    at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:289)
    at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:269)
    at com.mongodb.DBCollectionImpl.find(DBCollectionImpl.java:84)
    at com.mongodb.DB.command(DB.java:320)
    at com.mongodb.DB.command(DB.java:299)
    at com.mongodb.DB.command(DB.java:388)
    at com.mongodb.DBApiLayer.getCollectionNames(DBApiLayer.java:152)
    at org.pentaho.mongo.wrapper.NoAuthMongoClientWrapper.getCollectionsNames(NoAuthMongoClientWrapper.java:197)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.setupCollectionNamesForDB(MongoDbInputDialog.java:1532)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog.access$600(MongoDbInputDialog.java:82)
    at org.pentaho.di.ui.trans.steps.mongodbinput.MongoDbInputDialog$6.widgetSelected(MongoDbInputDialog.java:494)
    at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
    at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
    at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
    at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
    at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1339)
    at org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:7939)
    at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9214)
    at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:653)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)


    Seems like mongo plugin is working inside karaf container and for some reason JCE is not enabled even when it is put inside %PENTAHO_INSTALLATION%/lib folder. I have even tried putting jce jar with mongo-driver jar and it does not work.

  13. #13
    Join Date
    Jan 2016
    Posts
    2

    Default

    At the same time, DI 5.4.0.1 - 130 works fine. Need to check what we might be loosing with using this version of DI.

    Regards,
    Nitin

  14. #14
    Join Date
    Apr 2015
    Posts
    7

    Default

    I'm also getting the same error with 6.0.1.0-386. The previous version I was using (5.4.0.1-130) did not have this issue and was able to use the Mongo steps without a problem.

    Does anybody have a workaround for the issue? It sounds like it's not being seen as a bug; eg. there's a bug report on Jira which was just closed as "not a bug", with a message directing the user to these forums for support.

  15. #15
    Join Date
    May 2014
    Posts
    1

    Default

    I have a workaround for proof-of-concept and test sites with non-sensitive data. I can write output to a local test database by removing username and password values on the "Configure Connection" tab of the MongoDB Output step. Probably not applicable to your situation, but it might help someone just trying to get kettle to write to mongoDB.

  16. #16
    Join Date
    May 2016
    Posts
    1

    Default

    I'm currently using DI 6.1 and I had the same problem when mongodb requires authentication.

    In a older version (DI 5.4), there is a plugin to mongodb (pentaho-mongodb-plugin) in plugins folder.

    If you copy the entire plugin folder from DI 5.4 to DI 6.1, the mongodb connection with authentication works.

  17. #17
    Join Date
    Oct 2014
    Posts
    1

    Default

    Karaf is not making the class available to the Mongo plugins. It is not enough to have the jce.jar file available, the bundle in Karaf needs to be granted access to it.
    Edit the following file (windows version)
    (your pdi home)\data-integration\system\karaf\etc\config.properties

    There should be a org.osgi.framework.bootdelegation= property defined
    add the following to the end: ,javax.crypto,javax.crypto.*

    What this property does (from my meager understanding) is list the classes to make available to all bundles.

  18. #18

    Default

    thanks a lot gorgar...that solved my problem! I have been thinking how to solve this since last Fri afternoon.
    Honestly, I found it puzzling that a fix for this issue was not deployed since this reported as fixed through http://jira.pentaho.com/browse/PDI-15038 however it will be deployed with v7.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.