Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: [Mondrian] problems with concurrency (shepherd) execution

  1. #1
    Paul Stoellberger Guest

    Default [Mondrian] problems with concurrency (shepherd) execution

    Hi,

    some users have report the below pasted bug when using saiku (mondrian 3.3-SNAPSHOT). it doesn't always appear, just sometimes.... and in some environments... i couldnt find a clear path to reproduce the problem...

    My assumption is, since statement.end() is called in RolapConnection:642 (see stacktrace) that in case of an error StatementImpl throws an error because the execution hasn't finished

    What confuses me as well in MondrianOlap4jStatement is that this.execution is never set. only in start() and in there i cant see how that would ever work by just using that

    public synchronized void start(Execution execution) {
    if (this.execution != null) {
    throw new AssertionError();
    }
    ....
    this.execution = execution;



    the stacktrace of the bug:
    Caused by: mondrian.olap.MondrianException: Mondrian Error:java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: mondrian.olap4j.FactoryJdbc4Impl$MondrianOlap4jCellSetJdbc4@13ea6bf != null
    at mondrian.rolap.RolapResultShepherd.shepherdExecution(RolapResultShepherd.java:171)
    at mondrian.rolap.RolapConnection.execute(RolapConnection.java:580)
    at mondrian.olap4j.MondrianOlap4jCellSet.execute(MondrianOlap4jCellSet.java:88)
    at mondrian.olap4j.MondrianOlap4jStatement.executeOlapQueryInternal(MondrianOlap4jStatement.java:408)
    at mondrian.olap4j.MondrianOlap4jStatement.executeOlapQuery(MondrianOlap4jStatement.java:340)
    at mondrian.olap4j.MondrianOlap4jStatement.executeOlapQuery(MondrianOlap4jStatement.java:426)
    at org.olap4j.query.Query.execute(Query.java:292)
    at org.saiku.olap.query.OlapQuery.execute(OlapQuery.java:175)
    at org.saiku.service.olap.OlapQueryService.execute(OlapQueryService.java:164)
    ... 91 more
    Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: mondrian.olap4j.FactoryJdbc4Impl$MondrianOlap4jCellSetJdbc4@13ea6bf != null
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
    at java.util.concurrent.FutureTask.get(FutureTask.java:83)
    at mondrian.rolap.RolapResultShepherd.shepherdExecution(RolapResultShepherd.java:145)
    ... 99 more
    Caused by: java.lang.IllegalArgumentException: mondrian.olap4j.FactoryJdbc4Impl$MondrianOlap4jCellSetJdbc4@13ea6bf != null
    at mondrian.server.StatementImpl.end(StatementImpl.java:94)
    at mondrian.rolap.RolapConnection.executeInternal(RolapConnection.java:642)
    at mondrian.rolap.RolapConnection.access$000(RolapConnection.java:52)
    at mondrian.rolap.RolapConnection$1.call(RolapConnection.java:586)
    at mondrian.rolap.RolapConnection$1.call(RolapConnection.java:585)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
    at java.util.concurrent.FutureTask.run(FutureTask.java:138)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    ... 1 more


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  2. #2
    Julian Hyde Guest

    Default Re: [Mondrian] problems with concurrency (shepherd) execution

    On Oct 17, 2011, at 6:01 AM, Paul Stoellberger wrote:

    > Hi,
    >
    > some users have report the below pasted bug when using saiku (mondrian 3.3-SNAPSHOT). it doesn't always appear, just sometimes.... and in some environments... i couldnt find a clear path to reproduce the problem...


    I think I've seen a similar problem -- so even though it's not consistently reproducible, I'm sure there's a bug here. Can you log it please.

    I might have fixed it in my current set of changes for monitoring. (I had to do quite a bit of work on connection/statement/execution lifecycle.) But it's still good to have it logged.

    > My assumption is, since statement.end() is called in RolapConnection:642 (see stacktrace) that in case of an error StatementImpl throws an error because the execution hasn't finished
    >
    > What confuses me as well in MondrianOlap4jStatement is that this.execution is never set. only in start() and in there i cant see how that would ever work by just using that


    I think start() is OK. It first ensures that the statement is not already executing, then marks it as executing by setting the 'execution' field.

    But I can't think what might be cause right now.

    Julian
    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  3. #3
    Paul Stoellberger Guest

    Default Re: [Mondrian] problems with concurrency (shepherd) execution

    I filed it as http://jira.pentaho.com/browse/MONDRIAN-1014

    Is the monitoring refactoring part of 3.3?

    -Paul

    On Oct 18, 2011, at 9:02 AM, Julian Hyde wrote:

    >
    > On Oct 17, 2011, at 6:01 AM, Paul Stoellberger wrote:
    >
    >> Hi,
    >>
    >> some users have report the below pasted bug when using saiku (mondrian 3.3-SNAPSHOT). it doesn't always appear, just sometimes.... and in some environments... i couldnt find a clear path to reproduce the problem...

    >
    > I think I've seen a similar problem -- so even though it's not consistently reproducible, I'm sure there's a bug here. Can you log it please.
    >
    > I might have fixed it in my current set of changes for monitoring. (I had to do quite a bit of work on connection/statement/execution lifecycle.) But it's still good to have it logged.
    >
    >> My assumption is, since statement.end() is called in RolapConnection:642 (see stacktrace) that in case of an error StatementImpl throws an error because the execution hasn't finished
    >>
    >> What confuses me as well in MondrianOlap4jStatement is that this.execution is never set. only in start() and in there i cant see how that would ever work by just using that

    >
    > I think start() is OK. It first ensures that the statement is not already executing, then marks it as executing by setting the 'execution' field.
    >
    > But I can't think what might be cause right now.
    >
    > Julian
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian



    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  4. #4
    Luc Boudreau Guest

    Default Re: [Mondrian] problems with concurrency (shepherd) execution

    Paul,

    I've reproduced your issue and fixed it on our side. Seems like the
    Saiku code is executing the same OlapStatement twice concurrently (or
    the olap4j CellSet). I added a check in our code that cancels the
    previous execution prior to running it again, but you might want to
    look into your server code and make sure that you do not share
    statements across your HTTP threads.

    Luc


    On Tue, Oct 18, 2011 at 5:32 AM, Paul Stoellberger
    <p.stoellberger (AT) gmail (DOT) com> wrote:
    > I filed it as http://jira.pentaho.com/browse/MONDRIAN-1014
    > Is the monitoring refactoring part of 3.3?
    > -Paul
    > On Oct 18, 2011, at 9:02 AM, Julian Hyde wrote:
    >
    > On Oct 17, 2011, at 6:01 AM, Paul Stoellberger wrote:
    >
    > Hi,
    > some users have report the below pasted bug when using saiku (mondrian
    > 3.3-SNAPSHOT). it doesn't always appear, just sometimes.... and in some
    > environments... i couldnt find a clear path to reproduce the problem...
    >
    > I think I've seen a similar problem -- so even though it's not consistently
    > reproducible, I'm sure there's a bug here. Can you log it please.
    > I might have fixed it in my current set of changes for monitoring. (I had to
    > do quite a bit of work on connection/statement/execution lifecycle.) But
    > it's still good to have it logged.
    >
    > My assumption is, since statement.end() is called in RolapConnection:642
    > (see stacktrace) that in case of an error StatementImpl throws an error
    > because the execution hasn't finished
    > What confuses me as well in MondrianOlap4jStatement is that this.execution
    > is never set. only in start() and in there i cant see how that would ever
    > work by just using that
    >
    > I think start() is OK. It first ensures that the statement is not already
    > executing, then marks it as executing by setting the 'execution' field.
    > But I can't think what might be cause right now.
    > Julian
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian
    >
    >
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian
    >
    >

    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  5. #5
    Julian Hyde Guest

    Default Re: [Mondrian] [Olap4j-devel] problems with concurrency (shepherd)execution

    On Oct 18, 2011, at 2:32 AM, Paul Stoellberger wrote:

    > I filed it as http://jira.pentaho.com/browse/MONDRIAN-1014


    Thanks.

    >
    > Is the monitoring refactoring part of 3.3?


    No. Too dangerous, at this late stage.

    Julian
    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  6. #6
    Julian Hyde Guest

    Default Re: [Mondrian] [Olap4j-devel] problems with concurrency (shepherd)execution

    On Oct 18, 2011, at 9:01 AM, Luc Boudreau wrote:

    > I've reproduced your issue and fixed it on our side. Seems like the
    > Saiku code is executing the same OlapStatement twice concurrently (or
    > the olap4j CellSet). I added a check in our code that cancels the
    > previous execution prior to running it again, but you might want to
    > look into your server code and make sure that you do not share
    > statements across your HTTP threads.


    If that's what's happening -- running two queries on the same statement -- the best behavior would be to fail, but give a descriptive error message. Tough love, as it were.

    Julian
    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  7. #7
    Paul Stoellberger Guest

    Default Re: [Mondrian] [Olap4j-devel] problems with concurrency (shepherd)execution

    i had a quick look into our code but i couldn't find anything in there that would match it
    is executeInternal only called when an actual query is fired?
    harris has tested the new fixed version but seems to get the same error as previously.

    i'll have another look tomorrow but i wouldn't see how our code executes the same statement twice in our context.
    will let you know what/if i find anything

    -paul

    On Oct 19, 2011, at 1:07 AM, Julian Hyde wrote:

    >
    > On Oct 18, 2011, at 9:01 AM, Luc Boudreau wrote:
    >
    >> I've reproduced your issue and fixed it on our side. Seems like the
    >> Saiku code is executing the same OlapStatement twice concurrently (or
    >> the olap4j CellSet). I added a check in our code that cancels the
    >> previous execution prior to running it again, but you might want to
    >> look into your server code and make sure that you do not share
    >> statements across your HTTP threads.

    >
    > If that's what's happening -- running two queries on the same statement -- the best behavior would be to fail, but give a descriptive error message. Tough love, as it were.
    >
    > Julian
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  8. #8
    Luc Boudreau Guest

    Default Re: [Mondrian] [Olap4j-devel] problems with concurrency (shepherd)execution

    The Mondrian job on the CI server is turned off until we finish the 3.3
    release. Unless you compile it yourself, you won't be able to test my
    changes. I'll ping you as soon as the SNAPSHOT builds are published again.

    Luc
    On Oct 18, 2011 9:22 PM, "Paul Stoellberger" <p.stoellberger (AT) gmail (DOT) com>
    wrote:

    > i had a quick look into our code but i couldn't find anything in there that
    > would match it
    > is executeInternal only called when an actual query is fired?
    > harris has tested the new fixed version but seems to get the same error as
    > previously.
    >
    > i'll have another look tomorrow but i wouldn't see how our code executes
    > the same statement twice in our context.
    > will let you know what/if i find anything
    >
    > -paul
    >
    > On Oct 19, 2011, at 1:07 AM, Julian Hyde wrote:
    >
    > >
    > > On Oct 18, 2011, at 9:01 AM, Luc Boudreau wrote:
    > >
    > >> I've reproduced your issue and fixed it on our side. Seems like the
    > >> Saiku code is executing the same OlapStatement twice concurrently (or
    > >> the olap4j CellSet). I added a check in our code that cancels the
    > >> previous execution prior to running it again, but you might want to
    > >> look into your server code and make sure that you do not share
    > >> statements across your HTTP threads.

    > >
    > > If that's what's happening -- running two queries on the same statement

    > -- the best behavior would be to fail, but give a descriptive error message.
    > Tough love, as it were.
    > >
    > > Julian
    > > _______________________________________________
    > > Mondrian mailing list
    > > Mondrian (AT) pentaho (DOT) org
    > > http://lists.pentaho.org/mailman/listinfo/mondrian

    >
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian
    >


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  9. #9
    Paul Stoellberger Guest

    Default Re: [Mondrian] [Olap4j-devel] problems with concurrency (shepherd)execution

    Will pointed me to:
    http://repository.pentaho.org/artifa...%2F3.3.0.14699

    i assumed in this build your changes would be already in, but harris tested it and had the same issues.
    i dont share statements across http threads and i double checked that we execute it only once... i dont see how we could cause that, but i cant debug either at the moment.

    It would be nice if I could test and fix this before you release 3.3. even if it means i have to spend some time in front of the debugger.
    Then maybe we can, for once, use the same mondrian instance in the biserver instead of depending on a separate SNAPSHOT, which would be quite nice!

    -Paul



    On Oct 19, 2011, at 3:57 AM, Luc Boudreau wrote:

    > The Mondrian job on the CI server is turned off until we finish the 3.3 release. Unless you compile it yourself, you won't be able to test my changes. I'll ping you as soon as the SNAPSHOT builds are published again.
    >
    > Luc
    >
    > On Oct 18, 2011 9:22 PM, "Paul Stoellberger" <p.stoellberger (AT) gmail (DOT) com> wrote:
    > i had a quick look into our code but i couldn't find anything in there that would match it
    > is executeInternal only called when an actual query is fired?
    > harris has tested the new fixed version but seems to get the same error as previously.
    >
    > i'll have another look tomorrow but i wouldn't see how our code executes the same statement twice in our context.
    > will let you know what/if i find anything
    >
    > -paul
    >
    > On Oct 19, 2011, at 1:07 AM, Julian Hyde wrote:
    >
    > >
    > > On Oct 18, 2011, at 9:01 AM, Luc Boudreau wrote:
    > >
    > >> I've reproduced your issue and fixed it on our side. Seems like the
    > >> Saiku code is executing the same OlapStatement twice concurrently (or
    > >> the olap4j CellSet). I added a check in our code that cancels the
    > >> previous execution prior to running it again, but you might want to
    > >> look into your server code and make sure that you do not share
    > >> statements across your HTTP threads.

    > >
    > > If that's what's happening -- running two queries on the same statement -- the best behavior would be to fail, but give a descriptive error message. Tough love, as it were.
    > >
    > > Julian
    > > _______________________________________________
    > > Mondrian mailing list
    > > Mondrian (AT) pentaho (DOT) org
    > > http://lists.pentaho.org/mailman/listinfo/mondrian

    >
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian



    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2017 Pentaho Corporation. All Rights Reserved.