PDA

View Full Version : [Mondrian] RE: FunctionTest.testFilterCompound



Pappyn Bart
02-06-2007, 05:31 AM
Richard, Julian,

First of all, your changes with Iterable would also break without my
changes.
Since cube.clearCachedAggregations() was already there in the past. It
would
clear all cache in case mondrian.rolap.star.disableCaching=true was set.
Even
the cache build by other concurrently running threads.

You have changed the behavior of RolapResult, now, results can be
materialized
after RolapResult has been constructed. This is not compatible with my
changes
for the moment. But it is not compatible without my changes either, in
case
mondrian.rolap.star.disableCaching=true was set or a cube had cache
turned off.

RolapBase.close() is never called for the moment. Since it would be up
to
the end user to call close(), I don't think it is a very good idea to
leave
the cache management up to the end user.

My changes do the following :

* In case mondrian.rolap.star.disableCaching=true is set or a cube has
cache turned off,
cache is kept in local cache, to prevent that clearing of the cache
will break
other concurrently running threads.

Problem --> The clearing of the cache was already there, only the scope
has changed.
The problem with this feature is that the behavior of
RolapResult has changed.
The timing when the cache should be cleared must be changed.

Possible solution --> Clear the local cache before a query is run. This
way mondrian is responsible
for managing the cache. If the thread is
terminated, cache would be
cleared also, since it is using ThreadLocal.

* Loading of the aggregate cache is now thread based in a local cache.
To prevent breaking other
concurrently running threads. After a query has finished, the results
are checked into global cache,
for all other threads to use. This is done in the finally part of
RolapResult constructor, calling
pushAggregateModificationsToGlobalCache().

Question --> This feature depends on the fact that all aggregates are
loaded before RolapResult constructor
has finished. While I do not see any problems yet, can you
check if this is the case,
since the behavior of RolapResult has changed. If this is
not the case, then I would say
mondrian is broken and it will be very difficult in the
future to implement transactions
and control cache integrity. Since the lifetime cannot be
predicted.

If my solution is ok by you, I will implement it.

Bart

-----Original Message-----
From: Richard Emberson [mailto:remberson (AT) edgedynamics (DOT) com]
Sent: dinsdag 6 februari 2007 0:39
To: Pappyn Bart
Cc: Julian Hyde
Subject: FunctionTest.testFilterCompound


Pappyn,

I looked into the the FunctionTest.testFilterCompound test failure. It
turns out that with the new Iterable code, results are not necessarily
materialized until the mondrian client request the data. What this means
is that having the call cube.clearCachedAggregations(); in the
RolapResult executeBody finally clause, clears the data before the
client can (lazily) access it.
RolapCube clearCachedAggregations method calls RolapStar
clearCachedAggregations which in turn calls
localAggregations.get().clear();

Julian suggested that you take a look at this and that maybe the fix is
to move the cache clearing call from the RolapResult executeBody method
to the ResultBase close method.


Richard


Richard Emberson wrote:
> Setting
> mondrian.rolap.star.disableCaching=true
> causes it to fail with me.
>
> Richard
>
>
> Julian Hyde wrote:
>>> The FunctionTest.testFilterCompound works for me.
>>> Is there anything special about the test run in which it is failing?
>> Richard,
>>
>> I'm not surprised that it worked for you - it ran successfully in all

>> but one of the runs. You can identify which set of conditions caused
>> the failure, because the parameters are printed in the log file
>> before the test run starts:
>>
>> Mon Feb 5 05:58:13 PST 2007
>> Running test with JDK=jdk1.5 database=oracle props={
>> mondrian.rolap.star.disableCaching=true}
>>
>> As always, these tests are run on Linux 2.6 AMD dual-core, 2GB
memory.
>>
>> You and Bart have both been diligent in running the test suite before

>> you check in. I appreciate that, and I hate to push test failures
>> onto you. But the alternative is for me to spend hours fixing
>> problems probably caused by features you have put into the product --

>> and I don't have the time or the patience for that.
>>
>> The unfortunate truth is that mondrian is becoming a complex system
>> and when you put in a complex feature -- as you and Bart have done in

>> this release -- you have to expect to spend as much time testing it
as writing the code.
>>
>> Julian
>>
>>
>>
>
>


--
Quis custodiet ipsos custodes:
This email message is for the sole use of the intended recipient(s) and
may contain confidential information. Any unauthorized review, use,
disclosure or distribution is prohibited. If you are not the intended
recipient, please contact the sender by reply email and destroy all
copies of the original message.

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________
_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian