Hitachi Vantara Pentaho Community Forums
Results 1 to 4 of 4

Thread: Schema (+jcr files) publish automatization

  1. #1

    Default Schema (+jcr files) publish automatization

    Hi there,

    Some questions related to automatization over Pentaho 5.1.

    We need to define a procedure (over Jenkins), in order to push updates of schemas, dashboards and reports into a remote Pentaho.

    In previous releases we just scp the files it into the pentaho filesystem and forced a restart. Worked like charms.
    But now with jackrabbit just don't know where schemas are stored, or even how to explore the jcr fs.

    1. Is there any filesystem explorer like tool to do this?
    Just found the RepositorySyncronizer but it has given us some headaches so we prefer to avoid it.

    2. Is there any service exposed to publish schemas?
    Put wireshark sniffing the Schema Workbench > publish action and got a put operation but would prefer a simpler method.
    Code:
    PUT /pentaho/plugin/data-access/api/mondrian/putSchema HTTP/1.1
    Also found the 'MondrianCatalogPublisher' servlet, but have no found information about it. Could try to understand parameters (publishPath, jndiName, EnableXmla, ..)
    http://grepcode.com/file/repository....Publisher.java

    3. Same for other type files stored in the JCR.
    Have no workarounds yet

    Any ideas I could explore?

  2. #2
    Join Date
    Apr 2008
    Posts
    146

    Default

    First, you would enjoy reading http://help.pentaho.com/Documentatio...70/010/0B0/0I0
    Which is the REST API for dealing with Schema's. You'll find almost everything from that point.

    I use something called DHC in Google Chrome which allows me to build up and rapidly experiment with REST API calls in general.
    There is also in PDI 5.2 a marketplace where you will find some BA Server Plugins created by Webdetails that can call to the BA server and present you with a list of REST calls that are available. As a step in PDI, you can definitely automate things using transforms.

    Great news... You can keep your current way of doing things if you are up for a twist. There is a magic directory where you can drop a zip file of all the content you want to appear, even if it's in Pentaho's magic lock box. Just put a zip file with the relevant directory structure into biserver-(whatever)/pentaho-solutions/system/default-content.

    Pentaho wanted some locked box approach to a directory /etc hidden away in the repository. That's actually where all the good stuff is. As far as I can tell, all REST interfaces are programmatically prohibited from altering the contents of the /etc directory unless Pentaho exposed an API for some privileged content in there.

    Some words of caution.
    1. If you manage to sneak some PDI ktr files into the server, datasources are imported as JDBC connections automatically, even if they are not used.
    2. If you manage to sneak them into the /etc directory, there is no way to obliterate them except for dropping the jackrabbit schema and letting the server repopulate itself.
    3. Referential Integrity is something to watch out for related to datasources and whatever uses them. Example: KTR uses one, it gets imported, so no deleting it, without first deleting the file.

    I really want to hear back from you when you experiment with either of the methods. I want to know what happens when you upload by REST or zip file to content that uses existing connections. Hopefully it works. If you figure out how to delete things in /etc, let me know. Pentaho themselves didn't see at least one problem looming. If you are trying their EE product and you improved upon the Operations Mart and want to replace it, you can't actually delete the whole thing because of both the lock-box orientation of the repository and the referential integrity that prevents you from deleting the JDBC connection belonging to that stuff. You'd have to zap the entire jackrabbit repo and remove Pentaho's zip files from that 'system/default-content' directory i mentioned before. I've got a support case in about what I just mentioned. I think Pentaho should open that directory up to us. If we break our system, it's on us.

  3. #3

    Default

    w0w, thx for all those resources and ideas.

    - Didn't even know a Marketplace was available at PDI!
    Found the BA Server plug-in really interesting, but gonna need time to digest and find the usage of all those endpoints of all those modules. Is there any documentation available which gathers all the module information?

    - The /default-content directory could be helpful. Wonder how does it manage zips (new ones, not the renamed) when it's content is already loaded. Will it override things? Will it delete files within root folder before loading them? Will play with it.

    - Regarding the /etc directory, we're actually not sneaking so much as what you seem to be, but quite understand your frustration regarding locking things up. Guess it has its reasons and hope they're working on Jackrabit file management tools.
    __

    Regarding my first post, I ended up solving the problems using:

    1. REST API for /putSchema
    Code:
    curl  -u admin:password -X PUT -F 'uploadAnalysis=@myschema.xml' -F  'catalogName=MyCatalog' -F 'Datasource=mydbresource' -F 'overwrite=true'  -F 'xmlaEnabledFlag=true' -F 'parameters=Datasource=mydbresource'   http://localhost:8080/pentaho/plugin/data-access/api/mondrian/putSchema
    2. bi-server/import-export.sh to upload/update content

    It serves us well for the moment, but will give a try to your ideas.

  4. #4
    Join Date
    Apr 2008
    Posts
    146

    Default

    I just figured everything out:

    1. Ivy wrote a blog on the basic approach to the API calls:
    http://ivy-is.blogspot.com/2013/11/p...vices-101.html

    2. I got the source code to the API calls themselves;
    Interface is here for FULL reference
    https://github.com/pentaho/pentaho-p...eResource.java
    What it actually does when called is here:
    https://github.com/pentaho/pentaho-p...leService.java

    If you want a fully uncrippled API, just "git clone" that repo, install Apache ant, Java 7 JDK, go into platform/extensions and run "ant subfloor.resolve". After it get's done, you can comment out 3 lines ( 978-980) in https://github.com/pentaho/pentaho-p...leService.java
    The function will return "true" no matter what the path. That activates REST calls for everything in there.

    After the mod. run "ant jar" and after successful compile, you'll find a replacement for pentaho-extension-blah-version.jar in the ./dist directory.
    Stop the server, replace with your modified one and start it again. Everything should work perfectly.

    If you know your way around CURL then you can fully automate anything you want at this point.

    We are actually not sneaking things into /etc. I just wanted my files as I've always had them. Secondly, I was trying to diagnose another issue on my 5.2 server and wanted to remove all noise like operations mart and so on. XMLA was not very performant in 5.1.2 against Mondrian. We have PDI ktrs used as CDA datasources in a few dashboards that do some forecasting and benefitted greatly from MDX query flexibility. They were really fast on 4.8.3.2, but slower than a snail in 5.X. It might be fixed now.
    My hopes are that the developers could very well put the Pentaho server in "Child Friendly" mode out of the box, but really leave some advanced switches for people who really want to do advanced and creative things.
    Last edited by Smoodo; 10-17-2014 at 05:38 PM.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.