Hitachi Vantara Pentaho Community Forums

View Poll Results: Do you use file based, or database based repository?

Voters
32. You may not vote on this poll
  • File

    19 59.38%
  • Database

    13 40.63%
Results 1 to 14 of 14

Thread: Who uses database repository?

  1. #1
    Join Date
    Apr 2007
    Posts
    2,010

    Default Who uses database repository?

    Just curious.. How many people use a database repository vs a file based approach?

    At both my previous places we always used file based. But at my new place we use a database based solution.

    If you use database based - what db? mysql? Oracle? Other?

  2. #2
    Join Date
    Nov 2008
    Posts
    777

    Default

    I have used both but I currently use a file-based approach. I switched so I could use a Subversion repository to handle version control.
    pdi-ce-4.4.0-stable
    Java 1.7 (64 bit)
    MySQL 5.6 (64 bit)
    Windows 7 (64 bit)

  3. #3
    Join Date
    Nov 2008
    Posts
    271

    Default

    some months ago asked the same on twitter

    Anyway, same as darrell. I like the fllesystem+svn solution more than the db
    Andrea Torre
    twitter: @andtorg

    join the community on ##pentaho - a freenode irc channel

  4. #4
    Join Date
    Jan 2006
    Posts
    245

    Default

    Always filesystem + version control

    S.
    Follow Me on Twitter: sramazzina
    My Skype account: sramazzina
    My Blog
    View my profile on LinkedIn: http://www.linkedin.com/in/sramazzina
    Author of Pentaho Data Integration Kitchen How-To and Pentaho Business Analytics Cookbook

    Join us on IRC server Freenode.net, channel ##pentaho ##saiku

  5. #5

    Default

    We switched to file based to easily use Kettle Cookbook. We're not using SVN yet, but curious if anyone has used GIT for version control rather than Subversion...

    DB

  6. #6

    Default database repository vs files

    Hi. I have been using MySQL database repository for test and production, and files + SVN for development. The idea being to ease deployment from TEST system to PROD system. The expected deployment benefit has not been as great as expected, and startup of a job is slow with MySQL on VmWare. So I am considering changing to all file-based deployment. Also because it requires an additional effort to deploy a file-based development system in the TEST system.

  7. #7
    Join Date
    Apr 2008
    Posts
    1,771

    Default

    Hi.
    I use files because I find it easier to move from test environment to full production environment machines.
    In addition, I tend to develop a job/transformation on a laptop and then moving it to a server.
    Since I use MS Windows, it's easy for me to replicate folders paths and then copy ktr ktj files across.

    Mick

  8. #8

    Default

    I use file based approach for the SVN support, but I think that with some effort it's possible to have a solution with db repository, SVN, and automated deployment into different environments using repository too (I like the db repository idea).

    When worked on db repository it was Oracle (the same used for theDWH).

    Ciao.

  9. #9
    Join Date
    Oct 2010
    Posts
    13

    Default

    We worked with development , test and production environments + kettle db repositories for a year..and now we just switched over to filebased. In the 4.1.0 4.1.2 range there where some bugs in kettle with storing connections and parameters in the repo. After that time we had a "not so integer" repo. I do not know if this is cause and effect. Everything was oke as long as nothing was changed in the stored connections. But when it was needed to get a job or transformation from test to production, connections got lost and everything needed to be checked and reconfigured.

    And as mentioned above in previous responses, we want to make a move for better versioning and we want to go for git.
    But we still need to figure out how we should implement it in our procedures.

  10. #10
    Join Date
    Dec 2009
    Posts
    609

    Default

    Hi,

    so far my implementation stats look like:
    DB-based: 5 projects
    File-based: 2 projects
    I think, the fact that most people tend to see a database as a reliable, save and backuped system, the prefer the database to hold this data.
    However, I see advantages in the file-based approach like: Easy upgrade/patching transformations by copying the transformation from STAGE to PRODUCTION system which seems easier on FS than on DB repository.

    Cheers,

    Tom

  11. #11
    Join Date
    Apr 2008
    Posts
    146

    Default

    We currently use a file based system along with GIT for versions. We are new at the GIT part, but it seems to be working well so far.

    We tried the database backed solution, but it was too much trouble to manage the files (move, rename, group copy, cut paste), and their links.
    The file based metaphor is a great one with the kind of content and configuration required in the Pentaho Suite in general. Hopefully the wave of the future will be something like Dropbox where there is a folder on whatever computer you access and that Pentaho tools will transparently deal with syncing their 'database' or 'JCR' etc with what is in that folder.

  12. #12
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    That is indeed the way of the future, as in version 4.3 of PDI if all goes as planned (and it should).
    I don't really believe in automatic synchronization as I believe it's optimal to have metadata stored in a single location.
    However, it should be possible to migrate "artifacts" from one environment to another with interactive and automated tools.
    Last edited by MattCasters; 10-11-2011 at 05:48 PM.

  13. #13
    Join Date
    Apr 2007
    Posts
    2,010

    Default

    Only in EE though... (right?)

  14. #14
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    Yes, but it's not a black and white story. A lot of the core libs and environment switching code will be open source while the "enterprisy" lifecycle management tools and so forth will be EE.
    So all in all we'll continue to release more code as open source.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.