PDA

View Full Version : [Mondrian] Architectural notes: Mondrian and unique names



Anton Nikitin
05-29-2007, 08:30 AM
Good day!

I would like to share some of my thoughts about Mondrian architecture which
appeared during 4-years of product usage and development.

We are using Mondrian engine in our software product. We are very interested
in reusing base MDX engine logic, but at the same time we would like to have
an option to extend and to override some basic behavior, but not to lose the
ability to merge our code with latest Mondrian sources.

I do understand that there is a lot of work related to business features,
stability, performance, etc. But maybe some of my ideas will be also useful
to other Pentaho/Mondrian customers...

Actually my thoughts are about OLAP elements (metadata) unique names and all
related logic such as creation, resolution, etc.

The following points in current architecture (Mondrian 2.3.2.8944) seem to
me not very well-designed.

1. The format of unique names is currently similar to MSAS 2000. It is
hardcoded at the level of base classes (mondrian.olap package) and could not
be effectively changed without changing some of these classes. At the same
time there can be situations when one needs more flexible behavior for
assigning unique names (for example, SSAS 2005 has several different
mechanisms for assigning member unique names depending on the structure of
the hierarchy: it affects performance). There also can be other reasons why
one wants to redefine this logic (for example, someone could want to have
hierarchy unique names as in SSAS 2005).

2. The name resolution logic is distributed across multiple classes. There
are many lookupXXX() methods in different classes.

3. Lookup methods usage is rather intricate. Sometimes they accept unique
names, sometimes simple names, sometimes exploded unique names (!);
sometimes they have options to throw an exception in case of failure,
sometimes not, etc.

My suggestion is abstracting/encapsulating unique-names-related logic into
set of classes/interfaces and providing corresponding extension points. It
seems that something similar was initially intended: there is
mondrian.olap.NameResolver interface that was never (!) used.

- Introducing interface UniqueName, creating base implementation which will
also contain all this explode/implode/quote stuff from mondrian.olap.Util.
- Introducing interface for unique names creation
- Introducing interface for resolving unique names into metadata objects
- Ensuring that there is no code in mondrian.olap package (which doesn't
belong to base unique names implementation) which relies on some format of
unique names: only usage of interfaces is allowed.

Actually, the absence of such flexibility is currently the only reason why
we cannot use mondrian.olap.*Base classes for our metadata and need to make
some changes in core MDX Engine classes. It really makes merging with latest
Mondrian code rather complicated.

I saw some related point in roadmap (3.10), maybe it should be extended.

I would very appreciate any comments about this.

Kind regards,
Anton Nikitin




_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Julian Hyde
05-29-2007, 03:00 PM
Anton,

I totally agree on all points. This stuff 'just grew', and we knew we
could refactor it when necessary. That's not a bad thing - we added a
lot of functionality, and more importantly tests to ensure that
functionality - and I don't think it's difficult to rearchitect this
area.

If you are motivated to clean up & refactor this area, I would be
delighted, and will help you as much as I can.

If it would help, we could schedule some time to discuss your
architecture, and how/where it extends mondrian. I am on vacation from
today until Thu 7th, so it would have to be after that.

Turns out that unique names are as important to us as they are to
MSAS/SSAS. I think they need them as keys for their repository - but our
repository is in-memory, so you can use any name which resolves to the
right object.

Some other points:

1. Note the related thread 'Looking up members by key'. That problem
could be solved before or after a refactoring.

2. One of the problems was a lack of a public API - people tended to put
code wherever it was most convenient. Olap4j will help with this. When
olap4j is finalized, it will eventually become mondrian's public API,
which should give us more freedom to refactor what is behind it.

3. I would like to preserve the SchemaReader, or something which has a
similar intent. It's great to be able to browse the schema from a
particular access control perspective.

Note that if you browse a schema through the olap4j api, you will be
seeing only the objects you have access to. My reference implementation
of olap4j on mondrian does that by storing a SchemaReader pointer in
every metadata object, which is ok because olap4j metadata objects are
throwaway shells around the real metadata object.s

4. I have been thinking of getting rid of references to Member (and
RolapMember) in the lower parts of the code, and introducing a more
lightweight interface. And besides a lot of things we are dealing with
are tuples of relational columns. I don't know whether this will impact
your intitiative, just FYI.

Julian



> -----Original Message-----
> From: mondrian-bounces (AT) pentaho (DOT) org
> [mailto:mondrian-bounces (AT) pentaho (DOT) org] On Behalf Of Anton Nikitin
> Sent: Tuesday, May 29, 2007 5:25 AM
> To: 'Mondrian developer mailing list'
> Subject: [Mondrian] Architectural notes: Mondrian and unique names
>
> Good day!
>
> I would like to share some of my thoughts about Mondrian
> architecture which
> appeared during 4-years of product usage and development.
>
> We are using Mondrian engine in our software product. We are
> very interested
> in reusing base MDX engine logic, but at the same time we
> would like to have
> an option to extend and to override some basic behavior, but
> not to lose the
> ability to merge our code with latest Mondrian sources.
>
> I do understand that there is a lot of work related to
> business features,
> stability, performance, etc. But maybe some of my ideas will
> be also useful
> to other Pentaho/Mondrian customers...
>
> Actually my thoughts are about OLAP elements (metadata)
> unique names and all
> related logic such as creation, resolution, etc.
>
> The following points in current architecture (Mondrian
> 2.3.2.8944) seem to
> me not very well-designed.
>
> 1. The format of unique names is currently similar to MSAS 2000. It is
> hardcoded at the level of base classes (mondrian.olap
> package) and could not
> be effectively changed without changing some of these
> classes. At the same
> time there can be situations when one needs more flexible behavior for
> assigning unique names (for example, SSAS 2005 has several different
> mechanisms for assigning member unique names depending on the
> structure of
> the hierarchy: it affects performance). There also can be
> other reasons why
> one wants to redefine this logic (for example, someone could
> want to have
> hierarchy unique names as in SSAS 2005).
>
> 2. The name resolution logic is distributed across multiple
> classes. There
> are many lookupXXX() methods in different classes.
>
> 3. Lookup methods usage is rather intricate. Sometimes they
> accept unique
> names, sometimes simple names, sometimes exploded unique names (!);
> sometimes they have options to throw an exception in case of failure,
> sometimes not, etc.
>
> My suggestion is abstracting/encapsulating
> unique-names-related logic into
> set of classes/interfaces and providing corresponding
> extension points. It
> seems that something similar was initially intended: there is
> mondrian.olap.NameResolver interface that was never (!) used.
>
> - Introducing interface UniqueName, creating base
> implementation which will
> also contain all this explode/implode/quote stuff from
> mondrian.olap.Util.
> - Introducing interface for unique names creation
> - Introducing interface for resolving unique names into
> metadata objects
> - Ensuring that there is no code in mondrian.olap package
> (which doesn't
> belong to base unique names implementation) which relies on
> some format of
> unique names: only usage of interfaces is allowed.
>
> Actually, the absence of such flexibility is currently the
> only reason why
> we cannot use mondrian.olap.*Base classes for our metadata
> and need to make
> some changes in core MDX Engine classes. It really makes
> merging with latest
> Mondrian code rather complicated.
>
> I saw some related point in roadmap (3.10), maybe it should
> be extended.
>
> I would very appreciate any comments about this.
>
> Kind regards,
> Anton Nikitin
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian (AT) pentaho (DOT) org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>

_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Anton Nikitin
05-30-2007, 07:01 AM
Julian,

thank you for such quick and comprehensive answer.

Unfortunately I'm afraid that I won't be able to participate in redesigning
of this code in the near future. The most important task for our project
(after business features development) is merging with Mondrian 2.3.2 (we are
currently based on 2.0.1, so this is non-trivial task, taking into account
the amount of our own changes). However we will review Mondrian sources
deeply and maybe will find some bugs or will propose some enhancements...
Maybe after that I will be able to come back to this problem.

About olap4j - it is really a great initiative. We will definitely try it
for our server as soon as some stable spec/reference implementation is
available.

Thanks again.

Anton

-----Original Message-----
From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org] On
Behalf Of Julian Hyde
Sent: Tuesday, May 29, 2007 10:59 PM
To: 'Mondrian developer mailing list'
Subject: RE: [Mondrian] Architectural notes: Mondrian and unique names

Anton,

I totally agree on all points. This stuff 'just grew', and we knew we
could refactor it when necessary. That's not a bad thing - we added a
lot of functionality, and more importantly tests to ensure that
functionality - and I don't think it's difficult to rearchitect this
area.

If you are motivated to clean up & refactor this area, I would be
delighted, and will help you as much as I can.

If it would help, we could schedule some time to discuss your
architecture, and how/where it extends mondrian. I am on vacation from
today until Thu 7th, so it would have to be after that.

Turns out that unique names are as important to us as they are to
MSAS/SSAS. I think they need them as keys for their repository - but our
repository is in-memory, so you can use any name which resolves to the
right object.

Some other points:

1. Note the related thread 'Looking up members by key'. That problem
could be solved before or after a refactoring.

2. One of the problems was a lack of a public API - people tended to put
code wherever it was most convenient. Olap4j will help with this. When
olap4j is finalized, it will eventually become mondrian's public API,
which should give us more freedom to refactor what is behind it.

3. I would like to preserve the SchemaReader, or something which has a
similar intent. It's great to be able to browse the schema from a
particular access control perspective.

Note that if you browse a schema through the olap4j api, you will be
seeing only the objects you have access to. My reference implementation
of olap4j on mondrian does that by storing a SchemaReader pointer in
every metadata object, which is ok because olap4j metadata objects are
throwaway shells around the real metadata object.s

4. I have been thinking of getting rid of references to Member (and
RolapMember) in the lower parts of the code, and introducing a more
lightweight interface. And besides a lot of things we are dealing with
are tuples of relational columns. I don't know whether this will impact
your intitiative, just FYI.

Julian



> -----Original Message-----
> From: mondrian-bounces (AT) pentaho (DOT) org
> [mailto:mondrian-bounces (AT) pentaho (DOT) org] On Behalf Of Anton Nikitin
> Sent: Tuesday, May 29, 2007 5:25 AM
> To: 'Mondrian developer mailing list'
> Subject: [Mondrian] Architectural notes: Mondrian and unique names
>
> Good day!
>
> I would like to share some of my thoughts about Mondrian
> architecture which
> appeared during 4-years of product usage and development.
>
> We are using Mondrian engine in our software product. We are
> very interested
> in reusing base MDX engine logic, but at the same time we
> would like to have
> an option to extend and to override some basic behavior, but
> not to lose the
> ability to merge our code with latest Mondrian sources.
>
> I do understand that there is a lot of work related to
> business features,
> stability, performance, etc. But maybe some of my ideas will
> be also useful
> to other Pentaho/Mondrian customers...
>
> Actually my thoughts are about OLAP elements (metadata)
> unique names and all
> related logic such as creation, resolution, etc.
>
> The following points in current architecture (Mondrian
> 2.3.2.8944) seem to
> me not very well-designed.
>
> 1. The format of unique names is currently similar to MSAS 2000. It is
> hardcoded at the level of base classes (mondrian.olap
> package) and could not
> be effectively changed without changing some of these
> classes. At the same
> time there can be situations when one needs more flexible behavior for
> assigning unique names (for example, SSAS 2005 has several different
> mechanisms for assigning member unique names depending on the
> structure of
> the hierarchy: it affects performance). There also can be
> other reasons why
> one wants to redefine this logic (for example, someone could
> want to have
> hierarchy unique names as in SSAS 2005).
>
> 2. The name resolution logic is distributed across multiple
> classes. There
> are many lookupXXX() methods in different classes.
>
> 3. Lookup methods usage is rather intricate. Sometimes they
> accept unique
> names, sometimes simple names, sometimes exploded unique names (!);
> sometimes they have options to throw an exception in case of failure,
> sometimes not, etc.
>
> My suggestion is abstracting/encapsulating
> unique-names-related logic into
> set of classes/interfaces and providing corresponding
> extension points. It
> seems that something similar was initially intended: there is
> mondrian.olap.NameResolver interface that was never (!) used.
>
> - Introducing interface UniqueName, creating base
> implementation which will
> also contain all this explode/implode/quote stuff from
> mondrian.olap.Util.
> - Introducing interface for unique names creation
> - Introducing interface for resolving unique names into
> metadata objects
> - Ensuring that there is no code in mondrian.olap package
> (which doesn't
> belong to base unique names implementation) which relies on
> some format of
> unique names: only usage of interfaces is allowed.
>
> Actually, the absence of such flexibility is currently the
> only reason why
> we cannot use mondrian.olap.*Base classes for our metadata
> and need to make
> some changes in core MDX Engine classes. It really makes
> merging with latest
> Mondrian code rather complicated.
>
> I saw some related point in roadmap (3.10), maybe it should
> be extended.
>
> I would very appreciate any comments about this.
>
> Kind regards,
> Anton Nikitin
>
>
>
>
> _______________________________________________
> Mondrian mailing list
> Mondrian (AT) pentaho (DOT) org
> http://lists.pentaho.org/mailman/listinfo/mondrian
>

_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian