Hitachi Vantara Pentaho Community Forums
Results 1 to 11 of 11

Thread: Metadata

  1. #1
    Join Date
    Jul 2005
    Posts
    6

    Default Metadata

    Hi,

    I've had a skim over the roadmap and everything, and am currently using some of the tools that are targeted for this project.

    My only concern is, I havent seen any common metadata layer ?

    It's pertenant in my mind, as I'm currently maintaning a SAS Metadata server, Webfocus metadata and Cognos metadata and it made me realise that there arent any good open source metadata servers out there.

    Certainly nothing that covers business and technical metadata, or anything that has the scope for an whole-of-enterprise setup ... well, I should say, not that I know of.

    With all the tools we're looking at in Pentaho, a metadata layer would be essential in tying it all together.

    Food for thought.

    Tim.

  2. #2
    Join Date
    Jul 2005
    Posts
    10

    Default Re:Metadata

    This is interesting and I have been thinking of it too. Maybe in a slightly different way.

    I think the reason that there is no OSS Metadata layer is because there was no need. There was nothing like Pentaho. Parts and pieces but no combined tool.

    In the reporting world (BIRT, JasperReports and such) mapping could be done via a domain layer (JDO, Hibernate, EJBs, etc). This is also done with applications (ie OLTP) It would be nice if this could be reused in the other areas such as ETL and Analysis. I had suggested this in the Mondrian forums but only got negative responses.

    I've seen a little of what COGNOS has but I think it is very data centric and probably isn't for everything in the enterprise (I could be wrong - these are just my thoughts).

    I would like to see this worked out so there is little duplication of effort but works well for the different "stacks" (ie ETL, Reporting, OLTP, ...).

    I definitely think this is something that would be useful in Pentaho. I would love to hear others' thoughts .

  3. #3
    Join Date
    Jun 2005
    Posts
    144

    Default Re:Metadata

    There is already a good "base" for this in the Common Warehouse Metamodel. I believe Mondrian uses parts of this as well, however I've never looked that closely.

    I think that CWM is "broad" enough that there is much ambiguity in using it between products. The OSS community could really bring the standard into usefulness by using it as the basis for it's BI/DW/ETL metadata repository...

    Anyhow... something to mull... I believe the OMG also has Java classes that implement the model, but I'm not sure about that either...

    Nick

  4. #4
    Join Date
    Jul 2005
    Posts
    10

    Default Re:Metadata

    Sounds like a good idea.

    I had not seen that in Mondrian. I'll have to dig for it.

    Did a quick google and it looks like JOLAP "uses the CWM OLAP metamodel". On the down side, JOLAP (sadly) seems to be going no where. I've not heard anything new about it lately. Anything I find is old news. Mondrian does support JOLAP, somewhat, and that might be the connection to CWM.

  5. #5

    Default Re:Metadata

    Mondrian does use CWM as part of its JOLAP implementation - I think that is part of the JOLAP spec. Given that JOLAP does appear to be going nowhere, the JOLAP API in Mondrian has not been used by the Mondrian community.


    Sherman

  6. #6
    Join Date
    Jul 2005
    Posts
    10

    Default Re:Metadata

    Maybe this project will give more exposure to Mondrian/JPivot and thus get the big vendors to start moving on JOLAP again.

    That or look for IBM or Oracle to buy the main company behind Mondrian and/or JPivot.

  7. #7
    Join Date
    Jul 2005
    Posts
    6

    Default Re:Metadata

    Keep in mind, that metadata should go ALOT further than OLAP, Id really love to see Octopus extended to support a metadata layer in ETL, and Jasper utilizing this also.

    Maybe I'm a dreamer, but when I think of a integrated platform, I really think of common threads between naming conventions, or business terms for things. In our business we have different business units that would give you a different definition of a "Standard Claim", this is where the power of a COMMON metadata layer is.

    Even if JOLAP were ramped up, I dont think this will support a common metadata layer/repository for all the reporting components in the pentaho platform.

    (I should note, I have not used JOLAP, so am speculating about its use).

  8. #8
    Join Date
    Jun 2005
    Posts
    144

    Default Re:Metadata

    tfindlay wrote:
    Keep in mind, that metadata should go ALOT further than OLAP, Id really love to see Octopus extended to support a metadata layer in ETL, and Jasper utilizing this also.
    CWM has provisions for ETL mappings (flows), transformations, etc etc. However, as previously mentioned it's not gained much traction. There's a book on it (not very useful, just a guided view of the standard) as well which has been collecting dust on my shelf...

    Nick

  9. #9
    jdixon Guest

    Default Re:Metadata

    We agree that metadata should go much further than OLAP. Our intention is to integrate metadata into

    • Report queries
    • Report drilling
    • Dashboard queries
    • Dashboard drilling
    • Pivot table queries
    • Pivot table drilling
    • ETL


    It is true that meta-data initiatives have not gained much traction in products designed for SMEs (small to medium sized enterprises) and Open Source projects. I think largely because its hard.

    There are a collection of standards that are all ultimately based on OMG's Meta-Object Facility (MOF).
    • Common Warehouse Metamodel (CWM) is based on MOF.
    • Sun's Java Metadata Interface (JMI - JSR40) is based on MOF and can map CWM to Java interfaces.
    • Java OLAP Interface (JOLAP - JSR69) uses the CWM OLAP metamodel and leverages JMI
    • Java Data Mining API (JDMAPI - JSR73) is based on the CWM Data Mining metamodel


    Between them these standards are backed by HP, IBM, Oracle, Unisys, Hyperion, SAS, Sun, SPSS, SAP. Getting these groups to cooperate and agree on anything is a major achievement.

    It is certainly not trying re-create or compete with these standards.
    Our initial research will be to attempt a integration of reporting, dashboards, pivot views and ETL based on these standards.

  10. #10
    Join Date
    Jul 2005
    Posts
    6

    Default Re:Metadata

    Sounds great!

    I think you've hit the nail on the head (so to speak). The biggest problem is understanding the standards that are out there. I'll be the first to admit I dont fully understand them. I think with better knowledge about how they work and how to use them, we'll see bigger compliance. I think the corporates that work in this space will be far more likely to comply with the standards if Open Source led the way.

    IMO, as long as noone complies with the standard, big companies will happily detour from it as a form of pseudo lock-in, it's only when someone follows the standard that the corporate players are held to it. Not to mention, it makes it easier for them to implement it when someone shows them how.

    I guess that leads to the obvious question - How can Pentaho encourage these standards in projects like Octopus and some of the others that are lagging in this area ?

    ... I guess the obvious answer is, how prepared are developers to get on board and make it happen! As per your point however, I'd get on board myself and start implementing some support for CWM or OMG's MOF, if I understood it properly!

    I wonder if I can convince my boss to send on a fancy training course ? haha! (not likely).

    Prehaps if OMG & CWM were to write up some simpler, clearer docs about their standards and how to implement them, with better education it'd be easier to promote them in Open Source projects, and get people interested. I know I would be!

  11. #11
    Join Date
    Nov 1999
    Posts
    9,729

    Default Re:Metadata

    Well Tim, you have been heard.
    Check out "Pentaho metadata" & if you have questions join us on the forum with the same name.

    Matt

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.