PDA

View Full Version : [Mondrian] Multi-user access to mondrian fails under high stress



Pappyn Bart
01-03-2007, 11:40 AM
Hi,

Can someone explain me why mondrian fails to execute queries when a
lot of simultaneous queries are executed ?

When my system is under a lot of stress, then queries usually start to
fail : maxEvalDepth is reached and an error is thrown.

Executing the same queries on a system that is not running under high
stress, then all queries run fine.

The place where the failure occurs is in RolapResult.executeBody();

Thanks,
Bart

_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Pappyn Bart
01-04-2007, 05:22 AM
Hi,

After a lot of debugging. I think I found that the problem is caused by
the fact I use cubes that
have aggregate cache turned off.

In RolapResult.executeBody(), there is a loop that evaluates all
expressions and tries to load the needed
aggregations afterwards. This loop will run until maxEvalDepth is
reached and then throw an exception.

But if there is multi-user access, it is possible that another thread
flushed the aggregate cache of the cube, while another thread is still
trying to load all aggregates. In most cases this will add more loops
and thus eventually the loop will reach maxEvalDepth and throw an
exception.

I am not sure if anyone has a suggestion how to do things better ?
Could the aggregate cache be flushed in a better way ?

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: woensdag 3 januari 2007 16:34
To: mondrian (AT) pentaho (DOT) org
Subject: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

Can someone explain me why mondrian fails to execute queries when a
lot of simultaneous queries are executed ?

When my system is under a lot of stress, then queries usually start to
fail : maxEvalDepth is reached and an error is thrown.

Executing the same queries on a system that is not running under high
stress, then all queries run fine.

The place where the failure occurs is in RolapResult.executeBody();

Thanks,
Bart

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________


_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Pappyn Bart
01-04-2007, 05:30 AM
Only an idea. Should aggregate cache not be maintained per rolapresult
in case the cube maintains no cache?

Not sure if this is do-able though...

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: donderdag 4 januari 2007 10:21
To: Mondrian developer mailing list
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

After a lot of debugging. I think I found that the problem is caused by
the fact I use cubes that
have aggregate cache turned off.

In RolapResult.executeBody(), there is a loop that evaluates all
expressions and tries to load the needed
aggregations afterwards. This loop will run until maxEvalDepth is
reached and then throw an exception.

But if there is multi-user access, it is possible that another thread
flushed the aggregate cache of the cube, while another thread is still
trying to load all aggregates. In most cases this will add more loops
and thus eventually the loop will reach maxEvalDepth and throw an
exception.

I am not sure if anyone has a suggestion how to do things better ?
Could the aggregate cache be flushed in a better way ?

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: woensdag 3 januari 2007 16:34
To: mondrian (AT) pentaho (DOT) org
Subject: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

Can someone explain me why mondrian fails to execute queries when a
lot of simultaneous queries are executed ?

When my system is under a lot of stress, then queries usually start to
fail : maxEvalDepth is reached and an error is thrown.

Executing the same queries on a system that is not running under high
stress, then all queries run fine.

The place where the failure occurs is in RolapResult.executeBody();

Thanks,
Bart

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________


_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Pappyn Bart
01-04-2007, 09:30 AM
I have a proposal to make clearing of the aggregation cache thread safe.

A) In case of stars that will not maintain a cache, store the
aggregations per thread,
only cleanup the aggregations made by this thread, not by others.

This could be done by either storing the thread identity in the
aggregation cache of
in the function RolapStar.lookupOrCreateAggregation. Or using a
ThreadLocal variable
in this case.

Has anyone experience in this matter ? I would prefer the first
option, using the threads
unique name, since I have no experience with ThreadLocal. Also stars
with cache turned
on could use the same datamember, but with the thread name left to
blank.

B) When clearing the cache after a query has run, only the cache of the
current thread should
be cleared.

C) When explicitly clearing all the aggregation cache (of cubes
maintaining cache), a request should be stored,
the aggregation cache should not be cleared immediately when queries are
running. Not sure if mondrian has a
way to check if queries are running? Would MondrianServer be a good
place to record all running
query threads ? When this list is empty when a query registers a
request, all aggregation cache may be emptied.

A) is something I can do myself - if I get a go to do these changes -.

Bart

________________________________

From: Pappyn Bart
Sent: donderdag 4 januari 2007 10:28
To: 'Mondrian developer mailing list'
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


Only an idea. Should aggregate cache not be maintained per rolapresult
in case the cube maintains no cache?

Not sure if this is do-able though...

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: donderdag 4 januari 2007 10:21
To: Mondrian developer mailing list
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

After a lot of debugging. I think I found that the problem is caused by
the fact I use cubes that
have aggregate cache turned off.

In RolapResult.executeBody(), there is a loop that evaluates all
expressions and tries to load the needed
aggregations afterwards. This loop will run until maxEvalDepth is
reached and then throw an exception.

But if there is multi-user access, it is possible that another thread
flushed the aggregate cache of the cube, while another thread is still
trying to load all aggregates. In most cases this will add more loops
and thus eventually the loop will reach maxEvalDepth and throw an
exception.

I am not sure if anyone has a suggestion how to do things better ?
Could the aggregate cache be flushed in a better way ?

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: woensdag 3 januari 2007 16:34
To: mondrian (AT) pentaho (DOT) org
Subject: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

Can someone explain me why mondrian fails to execute queries when a
lot of simultaneous queries are executed ?

When my system is under a lot of stress, then queries usually start to
fail : maxEvalDepth is reached and an error is thrown.

Executing the same queries on a system that is not running under high
stress, then all queries run fine.

The place where the failure occurs is in RolapResult.executeBody();

Thanks,
Bart

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________


_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Pappyn Bart
01-08-2007, 09:51 AM
Hi,

I solved this one in change 8481.

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: donderdag 4 januari 2007 14:26
To: Mondrian developer mailing list
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


I have a proposal to make clearing of the aggregation cache thread safe.

A) In case of stars that will not maintain a cache, store the
aggregations per thread,
only cleanup the aggregations made by this thread, not by others.

This could be done by either storing the thread identity in the
aggregation cache of
in the function RolapStar.lookupOrCreateAggregation. Or using a
ThreadLocal variable
in this case.

Has anyone experience in this matter ? I would prefer the first
option, using the threads
unique name, since I have no experience with ThreadLocal. Also stars
with cache turned
on could use the same datamember, but with the thread name left to
blank.

B) When clearing the cache after a query has run, only the cache of the
current thread should
be cleared.

C) When explicitly clearing all the aggregation cache (of cubes
maintaining cache), a request should be stored,
the aggregation cache should not be cleared immediately when queries are
running. Not sure if mondrian has a
way to check if queries are running? Would MondrianServer be a good
place to record all running
query threads ? When this list is empty when a query registers a
request, all aggregation cache may be emptied.

A) is something I can do myself - if I get a go to do these changes -.

Bart

________________________________

From: Pappyn Bart
Sent: donderdag 4 januari 2007 10:28
To: 'Mondrian developer mailing list'
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


Only an idea. Should aggregate cache not be maintained per rolapresult
in case the cube maintains no cache?

Not sure if this is do-able though...

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: donderdag 4 januari 2007 10:21
To: Mondrian developer mailing list
Subject: RE: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

After a lot of debugging. I think I found that the problem is caused by
the fact I use cubes that
have aggregate cache turned off.

In RolapResult.executeBody(), there is a loop that evaluates all
expressions and tries to load the needed
aggregations afterwards. This loop will run until maxEvalDepth is
reached and then throw an exception.

But if there is multi-user access, it is possible that another thread
flushed the aggregate cache of the cube, while another thread is still
trying to load all aggregates. In most cases this will add more loops
and thus eventually the loop will reach maxEvalDepth and throw an
exception.

I am not sure if anyone has a suggestion how to do things better ?
Could the aggregate cache be flushed in a better way ?

Bart

________________________________

From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
On Behalf Of Pappyn Bart
Sent: woensdag 3 januari 2007 16:34
To: mondrian (AT) pentaho (DOT) org
Subject: [Mondrian] Multi-user access to mondrian fails under high
stress


Hi,

Can someone explain me why mondrian fails to execute queries when a
lot of simultaneous queries are executed ?

When my system is under a lot of stress, then queries usually start to
fail : maxEvalDepth is reached and an error is thrown.

Executing the same queries on a system that is not running under high
stress, then all queries run fine.

The place where the failure occurs is in RolapResult.executeBody();

Thanks,
Bart

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________

______________________________________________________________________
This email has been scanned by the Email Security System.
______________________________________________________________________


_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian