PDA

View Full Version : [Mondrian] checkin 8525 ITERABLE



Richard Emberson
01-15-2007, 07:14 PM
With checkin 8525, ITERABLE has been added to the ExpCompiler's
ResultStyle enums. While it is true that all Lists are Iterable,
not all Iterables are Lists - basically all you can do with an
Iterable is a copy of an Iterator and iterate across it - you can
not get its size. The IterCalc and AbstractIterCalc classes were
added.

Note that which ResultStyle should be returned by a compile call
is a negotiation between the client and producer. The client
says which type so ResultStyle it can accept and the the producer
either returns one or throws a ResultStyleException. During
development a ResultStyleException was never thrown, it its there
just in case.

Also, as compilation proceeds, if a client Calc does not set
a new set of ResultStyle's in the compiler for its producer to get,
the current set of ResultStyle's is propagated down. This way,
if at the top, one wants a Calc of type LIST, then unless some
function explitily overrides this, all Calc's see the same
ResultStyle during compilation.

I then modified the function FilterFunDef to request
in order of preference: ITERABLE, LIST (immutable), and the MUTABLE_LIST.
In the case of an Calc of type ITERABLE being returned during
the compilation stage, a Calc is returned that during the
evaluator stage filters the objects returned by the underlying
Iterable's Iterator. For the LIST Calc, a new list is created during
evaluation and, lastly, for MUTABLE_LIST Calc, the list is
filtered (modified) and return during evaluation. (Maybe, a MUTABLE_LIST
should be in the second position in the ResultStyle array rather
than the LIST).
A number of other functions had to be modified for FilterFunDef
to work and pass tests (e.g., AbstractAggregateFunDef, CountFunDef,
etc.).

Then I modified CrossJoinFunDef such that based upon what its client
requests, the attempts to return the right Calc type. This involves
CrossJoinFunDef requesting different Calc types from is own
producers, for each list which it will produce the crossjoin from.
Needless to say, there are a lot of combinations.

The Query class was modified so that one can change whether it
attempts to return an ITERABLE or LIST (or MUTABLE_LIST) as the
top Calc. The default is to return an ITERABLE. To pass the
BasicQueryTest.testQueryIterationLimit requires that that test
set the Query's ResultStyle to be of type LIST
(query.setResultStyle(mondrian.calc.ExpCompiler.ResultStyle.LIST)).

The RolapNative code respects the ResultStyle request by does not
return a true Iterable, rather a List is returned. This could
be an area of future work. In addition, everywhere the code
uses a RolapNative for evaluation, whether or not to use the
RolapNative is determined at evaluation-time rather than compile-time
which is something that also should be looked at.

Lastly, the RolapAxis now has versions of the Axis interface that
that consume an Iterable (which is not a List) as well as the
previous implementations that use Lists. The RolapAxis's which
are base on Iterable have the feature that if one of its list
methods, 'size' or 'get', are invoked, the Iterable is materialized
into a List. For clients that only iterate across Postions and their
Members, the Iterable version of RolapAxis has the lightest memory
foot print. Thoses that require a List can still use the RolapAxis
which is base upon an Iterable, it simply (lazily) materializes
in place.

I had to add:
evaluator= evaluator.push();
to the beginning of the FunUtil evaluateMembers method, otherwise
the CrossJoinFunDef assert on line 272:
assert oldEval.equals(evaluator) : "calc2 changed context";
was triggered. The evaluateMembers method certainly changes the
evaluation object so how this worked before is not known.

The ITERABLE is new and it will take some use to figure out
whats best.


Richard

--
This email message is for the sole use of the intended recipient(s) and
may contain confidential information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all
copies of the original message.
_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian