Hitachi Vantara Pentaho Community Forums
Page 1 of 3 123 LastLast
Results 1 to 10 of 22

Thread: HOWTO: My (very big) pentaho install and setup tutorial

  1. #1
    Join Date
    Jul 2007
    Posts
    2,498

    Post HOWTO: My (very big) pentaho install and setup tutorial

    Like many others, I had a hard time installing and setting up a development environment for pentaho. I still have no idea how to create a simple report in pentaho, but I think I managed to get a good startup point. This were my premises:

    - I needed to know how to build a demo solution that connects to an arbitrary database and shows simple pentaho abilities in the least possible time (eg: have a demo set in 2 hours in a client's database)

    - I need to switch configuration on the fly so that I can switch from different scenarios (eg: from client A to client B)

    - I don't want to change any original files that could get overwritten in a upgrade

    - the platform upgrade must be easy to do and not break the rest of the setup

    - Debug is a "must have"

    - Must support all kinds of different customization in different projects, from using a different databases (I'm using postgresql , thanks elassad) to different security types.


    I decided to use not the pre-compiled binaries but the sources from svn, as described here: http://wiki.pentaho.org/display/Pent...o+with+Eclipse


    Project pentaho already has a good dev_build.xml ant script, but it wasn't enough for my needs, since I needed to change the svn sources in order to configure a solution. So I built my own structure, and its like this:

    Code:
    MyProjectDir
    |-- build.properties
    |-- build.xml
    |-- pentaho
    |-- pentaho-data
    |-- pentaho-preconfiguredinstall
    |-- pentaho-solutions
    |-- project-client
    |   |-- patches
    |   |   |-- pentaho
    |   |   `-- target-preconfiguredinstall
    |   `-- solution
    |-- target-preconfiguredinstall
    |-- target-solutions
    `-- updateall-svn.sh
    pentaho* directories are cheched out from svn, and for me are read-only. Feel free to use any svn client you want; I'm currently in windows but I like to use cygwin, so I use the original client. I checked out the sources with:

    Code:
    $ svn co http://source.pentaho.org/svnroot/pentaho/trunk pentaho
    
    $ svn co http://source.pentaho.org/svnroot/pentaho-preconfiguredinstall/trunk pentaho-preconfiguredinstall
    
    $ svn co http://source.pentaho.org/svnroot/pentaho-solutions/trunk pentaho-solutions
    
    $ svn co http://source.pentaho.org/svnroot/pentaho-data/trunk pentaho-data
    I had to use http:// because I'm under a proxy, guess most of you can use svn:// .

    target-* directories are the ones generated by the build script. I don't use target-solutions at all (honestly, I'm not sure what this are for, cause they seem a copy of pentaho-solutions. Guess the point is to pick target* and copy it, w/o any other dependencies, to the desired machine).


    This leaves us with the "project-client" directory. As I mentioned before, I don't want to write anything under pentaho* directories, so all my changes go to this directory. Here is the structure in more detail:

    Code:
    project-client/
    |-- patches
    |   |-- pentaho
    |   |   `-- override.properties
    |   `-- target-preconfiguredinstall
    |       `-- server
    |           `-- default
    |               |-- conf
    |               |   |-- jboss-minimal.xml
    |               |   `-- jboss-service.xml
    |               |-- deploy
    |               |   |-- PentahoHibernate-ds.xml
    |               |   |-- client-ds.xml
    |               |   |-- datasource1-ds.xml
    |               |   |-- datasource2-ds.xml
    |               |   |-- datasource3-ds.xml
    |               |   |-- datasource4-ds.xml
    |               |   |-- datasource5-ds.xml
    |               |   |-- pentaho.war
    |               |   |   `-- WEB-INF
    |               |   |       |-- applicationContext-acegi-security-jdbc.xml
    |               |   |       |-- applicationContext-pentaho-security-jdbc.xml
    |               |   |       |-- classes
    |               |   |       |   `-- hibernate.cfg.xml
    |               |   |       |-- jboss-web.xml
    |               |   |       `-- web.xml
    |               |   |-- quartz-ds.xml
    |               |   |-- sampledata-ds.xml
    |               |   |-- sampledata_admin-ds.xml
    |               |   |-- shark-ds.xml
    |               |   |-- solution1-ds.xml
    |               |   |-- solution2-ds.xml
    |               |   |-- solution3-ds.xml
    |               |   |-- solution4-ds.xml
    |               |   `-- solution5-ds.xml
    |               `-- lib
    |                   `-- postgresql-8.2-505.jdbc3.jar
    `-- solution
        |-- Portal.properties
        |-- Portal.url
        |-- Portal_pt.properties
        |-- admin
        .... etc
    The idea is very simple: All changes that I would normally do under pentaho* I put them under "patches" directory. My top level ant script just picks them and copies the files to the top level directory. It's recommended to patch pentaho* as least as possible because we are actually overwriting the source files, but the file override.properties is meant to be changed. All other changes are made by patching the final directory, target-preconfiguredinstall.

    For my first scenario, those were the files I had to change. I had to change some jboss's config files just because I had to change a port that was unavailable; added my datasource (cliente-ds.xml) and changed the jboss-web.xml and web.xml to register it; changed all other ds's cause I'm using postgresql instead of hsql; added the jdbc lib; changed the security system from memory to jdbc.

    This is scalable. Just change files there and they will be copied to the destination target. My ant file does not allow (yet) to remove files from target to destination, but I also don't think thats a relevant issue.

    The solution is under project-client/solution, and override.properties points to this dir. For now, I just copied pentaho-solutions to here, and will start from there by adding new solutions and playing a bit.


    This is the top level build.xml file. Very simple, indeed:

    Code:
    <!-- ======================================================================
         description: main build file for pentaho  . .                                                    
         ====================================================================== -->
    <project name="pentaho-build" default="all" >
            <description>Pentaho BI Platform build helper..</description>
    
            <property file="build.properties" />
    
    
            <target name="init">
                    <echo>Building project ${project}</echo>
    
            </target>
    
    
            <target name="copy-init" depends="init" description="Copy project files">
                    <!-- apply patches -->
                    <copy todir="." failonerror="true" verbose="true" >
                            <fileset dir="project-${project}/patches/" />
                    </copy>
            </target>
    
    
            <target name="dev-setup" description="Compiles and builds the entire project" depends="copy-init" >
                    <ant dir="${pentaho.dir}" antfile="dev_build.xml" target="dev-setup" inheritAll="false"/>
            </target>
    
            <target name="copy-finish" depends="dev-setup" description="Copy target files">
                    <!-- apply patches - Not sure if I need this -->
                    <!--copy todir="." failonerror="true" verbose="true" >
                            <fileset dir="project-${project}/patches/"/>
                    </copy-->
            </target>
    
    
            <target name="all" description="Compiles and builds the entire project" depends="copy-finish" />
    
    
            <!-- Clean targets -->
    
            <target name="clean" description="Cleans the solution" depends="init" >
                    <ant dir="${pentaho.dir}" antfile="dev_build.xml" target="clean" inheritAll="false"/>
            </target>
    
    </project>
    The build.properties is also a very extensive file

    Code:
    project = client
    pentaho.dir = pentaho/
    How to build this? Simple: 'ant clean' to delete all compiled classes and delete target* directories, and 'ant all' to build everything.

    If we had another project in a directory called 'project-anotherclient', all we had to do was to simply change build.properties file and change project entry from client to another client and with a 'ant clean all' we would have a clean solution with totally different configuration


    There is an extra file there, called 'updateall-svn.sh'. Its just a simple bash script to update all sources from svn. It will only work on some systems:

    Code:
    for i in `ls -d p*`; do cd $i && svn update && cd .. ; done |less
    At this point, all that was missing was the design studio and knowing how to debug the platform. I'm a netbeans user, so eclipse was totally new for me and I was a bit lost. Since I usually work with tomcat, debugging jboss was also a new area.

    I tried, and tried, and tried once again to build designstudio from sources, but I could not do it. there are 3 projects on the svn root, but failed miserable in all of them. So I went to the precompiled plug in; There is a tutorial here that shows how to use jbossIDE to do the debug jboss, but I found that a bit complicated for 2 reasons: 1 - I could not get designstudio plugin to work in the same eclipse instance as jbosside plugin; 2 - I'm used to do debugging just by attaching a debugger to a socket, so using a >100 Mb plugin just for debugging seemed to me a great overhead. So I opted by *not* using jbosside plugin. This is what I did:

    1 - Downloaded and installed (unzipped) eclipse (the version recommended for the design studio plugin, version 3.2.1_M20060921-0945

    2 - downloaded design studio pre-compiled plug in and unzipped it to the eclipse directory

    3 - copied file pentaho-preconfiguredinstall/bin/run.bat to pentaho-preconfiguredinstall/bin/debug.bat (the same for .sh files in linux/unix), and uncomment the line that says:

    Code:
    set JAVA_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n %JAVA_OPTS%
    4 - run eclipse, added a project named project-client, and a java project (with existing ant script) for pentaho/dev_script.xml.

    And that was it. If I want to work on my xactions I just use "project client", if I need to debug jboss I just need to start it with 'debug.(bat|sh)' and attach the eclipse debugger to it (click debug, choose 'debug remote application' and point to the correct host, port 8787 as the above example). Breakpoints will work as magic


    __________________

    This is my 2 cents in appreciation for the great effort pentaho team puts in this project, and their courage to go for an open-source approach.

    Comments and suggestions appreciated. I'm sure the build.xml file can have great improvements. If the admins think its appropriate, feel free to copy-paste this to the wiki.

    Now I can finally "start to work" in pentaho. Does anyone know how can I build a simple report?

    Thread sponsored by irc channel ##pentaho in freenode


    Thanks

  2. #2
    Join Date
    Jun 2006
    Posts
    102

    Default

    hi,

    Thank you for this great intro, i will be testing it today evening

    greetings
    & Thanks
    Last edited by elassad; 07-10-2007 at 08:11 AM. Reason: spelling

  3. #3
    dmoran Guest

    Thumbs up

    This is a great article. I think it would make an excellent tech tip! If you are interested in moving this to the wiki, I will setup a page for you.

    Thanks,
    Spanky

  4. #4
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    Thanks, and sure, just tell me how

  5. #5
    dmoran Guest

    Default

    I created a stub page for you to use, give it a title and have fun!

    http://wiki.pentaho.org/display/COM/Blank+Tech+Tip

    You should have permission to edit it. Just select "Edit"under "Page Operations"

    Let me know if you have any trouble. For examples of other tech tips, check out:

    http://wiki.pentaho.org/display/COM/...ution+Articles

    Thanks,
    Spanky

  6. #6

    Default

    Hello thanks for turrials. when we can see the updates jbosws for error free solutions. This releases cant run while we are not connected internet also with proxy... I m still using 1.2.534 for this purposes.

    regards

  7. #7
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    I registered with the same username as the forum but I don't have permissions to edit the page

  8. #8
    dmoran Guest

    Default

    Sorry about that - you are all set now. I changed the permissions.

  9. #9
    Join Date
    Jul 2007
    Posts
    2,498

    Default

    Done, thank you. Check http://tinyurl.com/38lnye
    Pedro Alves
    Meet us on ##pentaho, a FreeNode irc channel

  10. #10

    Default

    I followed all the directions, and everything worked really smoothly except for one thing:

    When I run jboss, I get the following errors:

    17:27:27,163 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.AdhocWebService
    17:27:27,188 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.ShowTestResult
    17:27:27,189 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.GetImage
    17:27:27,191 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.UIServlet
    17:27:27,192 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.SubscriptionUserCleanup
    17:27:27,193 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.RepositoryFilePublisher
    17:27:27,194 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.GetContent
    17:27:27,196 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.HttpWebService
    17:27:27,209 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.GetMondrianModel
    17:27:27,210 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.ViewAction
    17:27:27,211 WARN [DeployerInterceptorJSE] Cannot load servlet class: org.pentaho.ui.servlet.GetResource
    17:27:27,674 ERROR [[/pentaho]] Error configuring application listener of class org.pentaho.core.session.PentahoHttpSessionListener
    java.lang.ClassNotFoundException: org.pentaho.core.session.PentahoHttpSessionListener
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1352)
    at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1198)
    at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:3677)

    I've barely started digging into the problem, but thought I'd post in case anyone else has encountered this.

    Two things worth noting: when following the directions exactly, the dev_build.xml file was replacing the solution-path in the web.xml with an empty string. I had to define target.solutions.dir and target.solutions.path in order to get a valid value in that node in the web.xml. solutions.dir needs to be relative to the pentaho source, and solutions.path should be relative to the PARENT of the pentaho source, I believe. AT least, that's what I have to do to get it to look like the web.xml that comes in pentaho-demo. My override.properties file looks like the following:

    solutions.dir=../project-adconion/solution
    target.solutions.dir=../target-solutions
    target.solutions.path=target-solutions

    It looks like there must just be a jar file or two missing from the webapp classpath, since it is failing to find pentaho classes, but perhaps it goes deeper than that?

    --sam

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.