PDA

View Full Version : Re : [Mondrian] Multi-threading SQL execution



michael bienstein
02-13-2007, 06:10 PM
Matt,

The reason that threads shouldn't be created in Servlets is conceptually simple. Servlets run in a managed environment. The control over the usage of system resources should be handled by the environment without interference from applications being managed. Threads are one of those resources. To give an idea of this, there is actually no restriction on the number of Threads you create in Java. However if you go over 8 threads per CPU you get serious performance degradation because of the switching the CPU has to do between the threads (at least on Windows Sun JDK 1.4). So it's better if you set up your application server and configure how many threads get used for all the applications in the same JVM and no application changes this.

More generally, Java has very bad semantics for enforcing this sort of control over system resources. Consider memory. I didn't know that Java 5 allowed some control over this as Richard has shown us recently.. Look at the problems though: Derby runs in the same JVM as Mondrian and all of a sudden we get lots of failures because we are told that we're running out of memory. How can you limit how much memory a WAR can use? You can't. With File access you probably could do it with a custom SecurityManager. With Threads that's probably true too, but no major app server does it.

The second reason has nothing to do with Mondrian. EJB 1 and 2 focussed on being able to call business logic without worrying about where that logic was implemented. It could be in a different JVM or even on another computer. Using threads (or static variables) is therefore forbidden even though if you know you are only running locally then it will still all work.

Lastly, I know commercial BI software that *breaks* this rule and launches a thread in a Servlet init() method. It is truly astounding how many bad things pop up when the language or framework doesn't constrain you.

Michael

PS: Congrats on 2.3 feature complete. Really happy to see cache flushing in there. But did Bart's bug on virtual cube get fixed? I'm really curious as to what causes it.








___________________________________________________________________________
D

John V. Sichi
02-13-2007, 08:28 PM
michael bienstein wrote:
> PS: Congrats on 2.3 feature complete. Really happy to see cache
> flushing in there. But did Bart's bug on virtual cube get fixed? I'm
> really curious as to what causes it.

I just sync'd the latest, and am running through on the 4-way against
Derby; so far I see 12 E's in the test output...

JVS
_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

John V. Sichi
02-13-2007, 08:56 PM
John V. Sichi wrote:
> michael bienstein wrote:
>> PS: Congrats on 2.3 feature complete. Really happy to see cache
>> flushing in there. But did Bart's bug on virtual cube get fixed? I'm
>> really curious as to what causes it.
>
> I just sync'd the latest, and am running through on the 4-way against
> Derby; so far I see 12 E's in the test output...

After completion, there were no failures in VirtualCubeTest. Could be
Julian fixed it, or could just be back in hiding (since the conclusion
was that it was there all along and Bart's changes just exposed it).

Looks like most or all of the errors are due to NPE at line 292 of
SqlConstraintUtils.addMemberConstraint.

JVS "Doubling as CruiseControl for the day"
_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian

Julian Hyde
02-13-2007, 10:05 PM
> After completion, there were no failures in VirtualCubeTest.
> Could be
> Julian fixed it, or could just be back in hiding (since the
> conclusion
> was that it was there all along and Bart's changes just exposed it).

The two failures which have been showing up in the test suite for the last
month or two are still there. They only appear on certain DBMS/JDK
combinations.

>
> Looks like most or all of the errors are due to NPE at line 292 of
> SqlConstraintUtils.addMemberConstraint.

I introduced a bug with an edit late in the game. I am going to check in a
fix as soon as my test suite finishes.

Julian

_______________________________________________
Mondrian mailing list
Mondrian (AT) pentaho (DOT) org
http://lists.pentaho.org/mailman/listinfo/mondrian