Hitachi Vantara Pentaho Community Forums
Results 1 to 11 of 11

Thread: [Mondrian] Mondrian Query Timeout

  1. #1
    Luc Boudreau Guest

    Default [Mondrian] Mondrian Query Timeout

    Julian,



    Following your commit on the statements refactoring, I have started to think
    about how we would now enforce a proper query timeout. (MONDRIAN-415).



    What I had in mind is a simple system where, when creating RolapResult
    objects (in RolapConnection.execute()) we fork the execution to a second
    thread (let’s call it the execution thread). The first thread (let’s call it
    the user thread) will wait a predetermined amount of time (the actual
    timeout) and if the execution thread has not completed, the user thread will
    perform the following.

    - Stop all SQL statements currently associated to this Execution.
    - Do house cleaning.
    - Bake a cake. (optional)
    - Throw back a timeout exception.

    This has several advantages. First off, we could remove all checks for query
    timeout in the evaluator and centralize everything in a single place. Second,
    as it stands now, there is no way to do a “hard stop” of any SQL statements
    running for a given Execution as we sit in the same thread. As long as the
    SQL Statement has not returned, we have no way to check for a timeout. As of
    now, the best we can do is to set an arbitrary JDBC timeout, but since there
    is no telling how many SQL statements will be issued, nor their priority, we
    cannot determine the proper JDBC timeout value to apply. By having the user
    thread overview the execution process, it becomes much simpler to manage and
    cleanup all the resources used by a query execution.


    Should I try this approach and submit a first POC?

    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  2. #2
    Julian Hyde Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    However we solve this, we need a data structure that records all SQL
    statements that are running in the service of a particular MDX statement.
    The Execution class that I added on Friday (see
    <http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10>
    http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10 ) is perfect for this.

    I couldn't tell whether you were suggesting this, but just in case, let me
    state for the record: I think that killing Java threads is a bad idea. It is
    very likely to make a mess. I think that Java code under mondrian's control
    should continue to poll the 'canceled' flag. (Which on Friday moved from
    Query to Execution.)

    For SQL statements, mondrian should definitely call cancel. If the JDBC
    driver of the database concerned is well-implemented, it will do something
    sensible with the cancel request. (Probably three things: cancel the
    synchronous call fairly quicky, throw a SQLException indicating that the
    statement is canceled, and send a message to the deeper parts of the
    infrastructure to cancel asynchronous processing, slower but more
    exhaustively.)

    Since Mondrian is itself an JDBC server (by virtue of olap4j being an
    extension to JDBC), we should do something similar. It's worth asking: what
    MondrianOlap4jStatement.executeOlapQuery() should do when canceled? I expect
    that Execute.cancel will be polled every ~millisecond, so cancel the
    statement should come back fairly quickly. We could make it come back
    quicker still if executeOlapQuery were making a call to another thread. Then
    we could interrupt that call without damaging either of the threads.

    Let me be clear that this is a "nice to have". And there are some
    implementation details I haven't solved:

    (a) I am speculating that there is an 'cancelable future' in JDK 1.6
    off-the-shelf. I mean: create a Future, have another thread (from an
    ExecutionService presumably) do the work, and allow any 3rd thread to cancel
    that call. Is this available in the JDK?

    (b) If the calls you need are not available in JDK 1.4 or JDK 1.5, it's OK
    if they have inferior behavior. But it still needs to build & run in those
    JDKs.

    (c) MondrianOlap4jStatement.executeOlapQuery is not necessarily the right
    place in the architecture to make the asynchronous 'slice'. Find the ideal
    place, maybe a little ways below it. But note that Locus.peek() etc. uses a
    thread-local.

    Julian


    _____

    From: Luc Boudreau [mailto:lucboudreau (AT) gmail (DOT) com]
    Sent: Monday, July 11, 2011 8:57 AM
    To: Julian Hyde
    Cc: Mondrian developer mailing list
    Subject: Mondrian Query Timeout



    Julian,



    Following your commit on the statements refactoring, I have started to think
    about how we would now enforce a proper query timeout. (MONDRIAN-415).



    What I had in mind is a simple system where, when creating RolapResult
    objects (in RolapConnection.execute()) we fork the execution to a second
    thread (let's call it the execution thread). The first thread (let's call it
    the user thread) will wait a predetermined amount of time (the actual
    timeout) and if the execution thread has not completed, the user thread will
    perform the following.

    * Stop all SQL statements currently associated to this Execution.

    * Do house cleaning.

    * Bake a cake. (optional)


    * Throw back a timeout exception.

    This has several advantages. First off, we could remove all checks for query
    timeout in the evaluator and centralize everything in a single place.
    Second, as it stands now, there is no way to do a "hard stop" of any SQL
    statements running for a given Execution as we sit in the same thread. As
    long as the SQL Statement has not returned, we have no way to check for a
    timeout. As of now, the best we can do is to set an arbitrary JDBC timeout,
    but since there is no telling how many SQL statements will be issued, nor
    their priority, we cannot determine the proper JDBC timeout value to apply.
    By having the user thread overview the execution process, it becomes much
    simpler to manage and cleanup all the resources used by a query execution.




    Should I try this approach and submit a first POC?



    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  3. #3
    Luc Boudreau Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    Good. That's exactly what I had in mind.

    As for b), we are using backport-util-concurrent so I think we're good.I'll
    make sure to test thoroughly.

    Luc



    On Mon, Jul 11, 2011 at 6:21 PM, Julian Hyde <jhyde (AT) pentaho (DOT) com> wrote:

    > **
    > However we solve this, we need a data structure that records all SQL
    > statements that are running in the service of a particular MDX statement.
    > The Execution class that I added on Friday (see *
    > http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10*<http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10> )
    > is perfect for this.
    >
    > I couldn't tell whether you were suggesting this, but just in case, let me
    > state for the record: I think that killing Java threads is a bad idea. It is
    > very likely to make a mess. I think that Java code under mondrian's control
    > should continue to poll the 'canceled' flag. (Which on Friday moved from
    > Query to Execution.)
    >
    > For SQL statements, mondrian should definitely call cancel. If the JDBC
    > driver of the database concerned is well-implemented, it will do something
    > sensible with the cancel request. (Probably three things: cancel the
    > synchronous call fairly quicky, throw a SQLException indicating that the
    > statement is canceled, and send a message to the deeper parts of the
    > infrastructure to cancel asynchronous processing, slower but more
    > exhaustively.)
    >
    > Since Mondrian is itself an JDBC server (by virtue of olap4j being an
    > extension to JDBC), we should do something similar. It's worth asking: what
    > MondrianOlap4jStatement.executeOlapQuery() should do when canceled? I expect
    > that Execute.cancel will be polled every ~millisecond, so cancel the
    > statement should come back fairly quickly. We could make it come back
    > quicker still if executeOlapQuery were making a call to another thread. Then
    > we could interrupt that call without damaging either of the threads.
    >
    > Let me be clear that this is a "nice to have". And there are some
    > implementation details I haven't solved:
    >
    > (a) I am speculating that there is an 'cancelable future' in JDK 1.6
    > off-the-shelf. I mean: create a Future, have another thread (from an
    > ExecutionService presumably) do the work, and allow any 3rd thread to cancel
    > that call. Is this available in the JDK?
    >
    > (b) If the calls you need are not available in JDK 1.4 or JDK 1.5, it's OK
    > if they have inferior behavior. But it still needs to build & run in those
    > JDKs.
    >
    > (c) MondrianOlap4jStatement.executeOlapQuery is not necessarily the right
    > place in the architecture to make the asynchronous 'slice'. Find the ideal
    > place, maybe a little ways below it. But note that Locus.peek() etc. uses a
    > thread-local.
    >
    > Julian
    >
    > ------------------------------
    > *From:* Luc Boudreau [mailto:lucboudreau (AT) gmail (DOT) com]
    > *Sent:* Monday, July 11, 2011 8:57 AM
    > *To:* Julian Hyde
    > *Cc:* Mondrian developer mailing list
    > *Subject:* Mondrian Query Timeout
    >
    > Julian,
    >
    >
    >
    > Following your commit on the statements refactoring, I have started to
    > think about how we would now enforce a proper query timeout. (MONDRIAN-415).
    >
    >
    >
    >
    > What I had in mind is a simple system where, when creating RolapResult
    > objects (in RolapConnection.execute()) we fork the execution to a second
    > thread (let’s call it the execution thread). The first thread (let’s call it
    > the user thread) will wait a predetermined amount of time (the actual
    > timeout) and if the execution thread has not completed, the user thread will
    > perform the following.
    >
    > - Stop all SQL statements currently associated to this Execution.
    > - Do house cleaning.
    > - Bake a cake. (optional)
    > - Throw back a timeout exception.
    >
    > This has several advantages. First off, we could remove all checks for
    > query timeout in the evaluator and centralize everything in a single place.
    > Second, as it stands now, there is no way to do a “hard stop” of any SQL
    > statements running for a given Execution as we sit in the same thread. As
    > long as the SQL Statement has not returned, we have no way to check for a
    > timeout. As of now, the best we can do is to set an arbitrary JDBC timeout,
    > but since there is no telling how many SQL statements will be issued, nor
    > their priority, we cannot determine the proper JDBC timeout value to apply. By
    > having the user thread overview the execution process, it becomes much
    > simpler to manage and cleanup all the resources used by a query execution..
    >
    >
    > Should I try this approach and submit a first POC?
    >
    >


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  4. #4
    Joe Barnett Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    One potential concern I have with this is that we have a UDF that
    essentially relies upon ThreadLocals that are set up by our
    application to control routing to the appropriate database. Actually,
    the DataSources we pass to mondrian rely on those ThreadLocals as
    well, so not actually UDF specific. It sounds like this approach
    would have mondrian run in a separate thread context, and hence not
    have the appropriate ThreadLocal values set.

    Are you planning on adding any hooks to the thread fork where we might
    be able to set up the thread appropriately? We'd also probably have
    to adjust some things (connection pools, etc) to expect an additional
    thread per thread-that-calls-into-mondrian, but would need to study
    the exact execution patterns a little more closely to see if that's
    really necessary.

    Do you have a sense of the differences in response time between
    cancel()ing the request without this extra thread vs/ with it? Seems
    like if the code is able to poll often enough (and
    interrupt()/cancel() any underlying IO), the difference should be
    negligible enough? But not sure exactly where all the
    cancel-checking-hooks are.

    Other thoughts?

    -Joe

    On Tue, Jul 12, 2011 at 7:35 AM, Luc Boudreau <lucboudreau (AT) gmail (DOT) com> wrote:[color=blue]
    >
    > Good. That's exactly what I had in mind.
    >
    > As for b), we are using backport-util-concurrent so I think we're good.I'll
    > make sure to test thoroughly.
    >
    > Luc
    >
    >
    >
    > On Mon, Jul 11, 2011 at 6:21 PM, Julian Hyde <jhyde (AT) pentaho (DOT) com> wrote:[color=green]
    >>
    >> However we solve this, we need a data structure that records all SQL
    >> statements that are running in the service of a particular MDX statement.
    >> The Execution class that I added on Friday (see
    >> [url]http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10

  5. #5
    Julian Hyde Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    Take a look at Execution.peek(). I want to reduce the use of thread-locals
    in mondrian because they make it more difficult to have a parallel execution
    model. As much state as possible should be stored in Execution. However, I
    know I can't eradicate thread-locals overnight.

    Execution.peek() will allow you to retrieve the current Execution object.
    Under the covers it uses -- you guessed it -- a thread-local. (So you don't
    have to.)

    Use Execution.peek() when your thread starts work, and store the Execution
    object in its local state.

    Execution isn't a supported API yet, but we can evolve it into one once
    we've proved the approach is viable.

    Julian

    [color=blue]
    > -----Original Message-----
    > From: mondrian-bounces (AT) pentaho (DOT) org
    > [mailto:mondrian-bounces (AT) pentaho (DOT) org] On Behalf Of Joe Barnett
    > Sent: Wednesday, July 13, 2011 3:25 PM
    > To: Mondrian developer mailing list
    > Subject: Re: [Mondrian] Mondrian Query Timeout
    >
    > One potential concern I have with this is that we have a UDF that
    > essentially relies upon ThreadLocals that are set up by our
    > application to control routing to the appropriate database. Actually,
    > the DataSources we pass to mondrian rely on those ThreadLocals as
    > well, so not actually UDF specific. It sounds like this approach
    > would have mondrian run in a separate thread context, and hence not
    > have the appropriate ThreadLocal values set.
    >
    > Are you planning on adding any hooks to the thread fork where we might
    > be able to set up the thread appropriately? We'd also probably have
    > to adjust some things (connection pools, etc) to expect an additional
    > thread per thread-that-calls-into-mondrian, but would need to study
    > the exact execution patterns a little more closely to see if that's
    > really necessary.
    >
    > Do you have a sense of the differences in response time between
    > cancel()ing the request without this extra thread vs/ with it? Seems
    > like if the code is able to poll often enough (and
    > interrupt()/cancel() any underlying IO), the difference should be
    > negligible enough? But not sure exactly where all the
    > cancel-checking-hooks are.
    >
    > Other thoughts?
    >
    > -Joe
    >
    > On Tue, Jul 12, 2011 at 7:35 AM, Luc Boudreau
    > <lucboudreau (AT) gmail (DOT) com> wrote:
    > >
    > > Good. That's exactly what I had in mind.
    > >
    > > As for b), we are using backport-util-concurrent so I think

    > we're good.I'll
    > > make sure to test thoroughly.
    > >
    > > Luc
    > >
    > >
    > >
    > > On Mon, Jul 11, 2011 at 6:21 PM, Julian Hyde

    > <jhyde (AT) pentaho (DOT) com> wrote:
    > >>
    > >> However we solve this, we need a data structure that

    > records all SQL
    > >> statements that are running in the service of a particular

    > MDX statement.[color=green][color=darkred]
    > >> The Execution class that I added on Friday (see
    > >> [url]http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10

  6. #6
    Luc Boudreau Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    > Do you have a sense of the differences in response time between
    > cancel()ing the request without this extra thread vs/ with it? Seems
    > like if the code is able to poll often enough
    >


    Our experience has shown that there are cases where the cancellation cannot
    occur as expected. When Mondrian is issuing a long running SQL query, there
    was no way to make the MDX request timeout correctly. As we want to avoid
    doing a hard kill of threads (we'd rather have the JDBC resources freed
    properly in the background so as not to hose the RDBMS and whatever other
    symptoms), we needed to split the user thread from the execution thread.

    Julian mentioned Execution.peek(). He should have mentioned Locus.peek()
    instead. Using this static accessor, your UDF will be capable of accessing
    the Execution instance of your particular thread by doing
    "Locus.peek().execution". Let's work together to figure out which
    information the Execution state should contain.

    Luc


    On Wed, Jul 13, 2011 at 6:35 PM, Julian Hyde <jhyde (AT) pentaho (DOT) com> wrote:

    > Take a look at Execution.peek(). I want to reduce the use of thread-locals
    > in mondrian because they make it more difficult to have a parallel
    > execution
    > model. As much state as possible should be stored in Execution. However, I
    > know I can't eradicate thread-locals overnight.
    >
    > Execution.peek() will allow you to retrieve the current Execution object.
    > Under the covers it uses -- you guessed it -- a thread-local. (So you don't
    > have to.)
    >
    > Use Execution.peek() when your thread starts work, and store the Execution
    > object in its local state.
    >
    > Execution isn't a supported API yet, but we can evolve it into one once
    > we've proved the approach is viable.
    >
    > Julian
    >
    >
    > > -----Original Message-----
    > > From: mondrian-bounces (AT) pentaho (DOT) org
    > > [mailto:mondrian-bounces (AT) pentaho (DOT) org] On Behalf Of Joe Barnett
    > > Sent: Wednesday, July 13, 2011 3:25 PM
    > > To: Mondrian developer mailing list
    > > Subject: Re: [Mondrian] Mondrian Query Timeout
    > >
    > > One potential concern I have with this is that we have a UDF that
    > > essentially relies upon ThreadLocals that are set up by our
    > > application to control routing to the appropriate database. Actually,
    > > the DataSources we pass to mondrian rely on those ThreadLocals as
    > > well, so not actually UDF specific. It sounds like this approach
    > > would have mondrian run in a separate thread context, and hence not
    > > have the appropriate ThreadLocal values set.
    > >
    > > Are you planning on adding any hooks to the thread fork where we might
    > > be able to set up the thread appropriately? We'd also probably have
    > > to adjust some things (connection pools, etc) to expect an additional
    > > thread per thread-that-calls-into-mondrian, but would need to study
    > > the exact execution patterns a little more closely to see if that's
    > > really necessary.
    > >
    > > Do you have a sense of the differences in response time between
    > > cancel()ing the request without this extra thread vs/ with it? Seems
    > > like if the code is able to poll often enough (and
    > > interrupt()/cancel() any underlying IO), the difference should be
    > > negligible enough? But not sure exactly where all the
    > > cancel-checking-hooks are.
    > >
    > > Other thoughts?
    > >
    > > -Joe
    > >
    > > On Tue, Jul 12, 2011 at 7:35 AM, Luc Boudreau
    > > <lucboudreau (AT) gmail (DOT) com> wrote:
    > > >
    > > > Good. That's exactly what I had in mind.
    > > >
    > > > As for b), we are using backport-util-concurrent so I think

    > > we're good.I'll
    > > > make sure to test thoroughly.
    > > >
    > > > Luc
    > > >
    > > >
    > > >
    > > > On Mon, Jul 11, 2011 at 6:21 PM, Julian Hyde

    > > <jhyde (AT) pentaho (DOT) com> wrote:
    > > >>
    > > >> However we solve this, we need a data structure that

    > > records all SQL
    > > >> statements that are running in the service of a particular

    > > MDX statement.
    > > >> The Execution class that I added on Friday (see
    > > >> http://p4web.eigenbase.org/@md=d&c=6PU@/14436?ac=10 ) is

    > > perfect for this.
    > > >>
    > > >> I couldn't tell whether you were suggesting this, but just

    > > in case, let me
    > > >> state for the record: I think that killing Java threads is

    > > a bad idea. It is
    > > >> very likely to make a mess. I think that Java code under

    > > mondrian's control
    > > >> should continue to poll the 'canceled' flag. (Which on

    > > Friday moved from
    > > >> Query to Execution.)
    > > >>
    > > >> For SQL statements, mondrian should definitely call

    > > cancel. If the JDBC
    > > >> driver of the database concerned is well-implemented, it

    > > will do something
    > > >> sensible with the cancel request. (Probably three things:

    > > cancel the
    > > >> synchronous call fairly quicky, throw a SQLException

    > > indicating that the
    > > >> statement is canceled, and send a message to the deeper

    > > parts of the
    > > >> infrastructure to cancel asynchronous processing, slower but more
    > > >> exhaustively.)
    > > >>
    > > >> Since Mondrian is itself an JDBC server (by virtue of

    > > olap4j being an
    > > >> extension to JDBC), we should do something similar. It's

    > > worth asking: what
    > > >> MondrianOlap4jStatement.executeOlapQuery() should do when

    > > canceled? I expect
    > > >> that Execute.cancel will be polled every ~millisecond, so

    > > cancel the
    > > >> statement should come back fairly quickly. We could make

    > > it come back
    > > >> quicker still if executeOlapQuery were making a call to

    > > another thread. Then
    > > >> we could interrupt that call without damaging either of

    > > the threads.
    > > >>
    > > >> Let me be clear that this is a "nice to have". And there are some
    > > >> implementation details I haven't solved:
    > > >>
    > > >> (a) I am speculating that there is an 'cancelable future'

    > > in JDK 1.6
    > > >> off-the-shelf. I mean: create a Future, have another

    > > thread (from an
    > > >> ExecutionService presumably) do the work, and allow any

    > > 3rd thread to cancel
    > > >> that call. Is this available in the JDK?
    > > >>
    > > >> (b) If the calls you need are not available in JDK 1.4 or

    > > JDK 1.5, it's OK
    > > >> if they have inferior behavior. But it still needs to

    > > build & run in those
    > > >> JDKs.
    > > >>
    > > >> (c) MondrianOlap4jStatement.executeOlapQuery is not

    > > necessarily the right
    > > >> place in the architecture to make the asynchronous

    > > 'slice'. Find the ideal
    > > >> place, maybe a little ways below it. But note that

    > > Locus.peek() etc. uses a
    > > >> thread-local.
    > > >>
    > > >> Julian
    > > >>
    > > >> ________________________________
    > > >> From: Luc Boudreau [mailto:lucboudreau (AT) gmail (DOT) com]
    > > >> Sent: Monday, July 11, 2011 8:57 AM
    > > >> To: Julian Hyde
    > > >> Cc: Mondrian developer mailing list
    > > >> Subject: Mondrian Query Timeout
    > > >>
    > > >> Julian,
    > > >>
    > > >>
    > > >>
    > > >> Following your commit on the statements refactoring, I

    > > have started to
    > > >> think about how we would now enforce a proper query

    > > timeout. (MONDRIAN-415).
    > > >>
    > > >>
    > > >>
    > > >> What I had in mind is a simple system where, when creating

    > > RolapResult
    > > >> objects (in RolapConnection.execute()) we fork the

    > > execution to a second
    > > >> thread (let’s call it the execution thread). The first

    > > thread (let’s call it
    > > >> the user thread) will wait a predetermined amount of time

    > > (the actual
    > > >> timeout) and if the execution thread has not completed,

    > > the user thread will
    > > >> perform the following.
    > > >>
    > > >> Stop all SQL statements currently associated to this Execution.
    > > >> Do house cleaning.
    > > >> Bake a cake. (optional)
    > > >> Throw back a timeout exception.
    > > >>
    > > >> This has several advantages. First off, we could remove

    > > all checks for
    > > >> query timeout in the evaluator and centralize everything

    > > in a single place.
    > > >> Second, as it stands now, there is no way to do a “hard

    > > stop” of any SQL
    > > >> statements running for a given Execution as we sit in the

    > > same thread. As
    > > >> long as the SQL Statement has not returned, we have no way

    > > to check for a
    > > >> timeout. As of now, the best we can do is to set an

    > > arbitrary JDBC timeout,
    > > >> but since there is no telling how many SQL statements will

    > > be issued, nor
    > > >> their priority, we cannot determine the proper JDBC

    > > timeout value to apply.
    > > >> By having the user thread overview the execution process,

    > > it becomes much
    > > >> simpler to manage and cleanup all the resources used by a

    > > query execution.
    > > >>
    > > >> Should I try this approach and submit a first POC?
    > > >
    > > >
    > > > _______________________________________________
    > > > Mondrian mailing list
    > > > Mondrian (AT) pentaho (DOT) org
    > > > http://lists.pentaho.org/mailman/listinfo/mondrian
    > > >
    > > >

    > > _______________________________________________
    > > Mondrian mailing list
    > > Mondrian (AT) pentaho (DOT) org
    > > http://lists.pentaho.org/mailman/listinfo/mondrian
    > >

    >
    > _______________________________________________
    > Mondrian mailing list
    > Mondrian (AT) pentaho (DOT) org
    > http://lists.pentaho.org/mailman/listinfo/mondrian
    >


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  7. #7
    Joe Barnett Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    On Thu, Jul 14, 2011 at 7:56 AM, Luc Boudreau <lucboudreau (AT) gmail (DOT) com> wrote:[color=blue]
    >[color=green]
    >> Do you have a sense of the differences in response time between
    >> cancel()ing the request without this extra thread vs/ with it?

  8. #8
    Joe Barnett Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    Ok, I've had a little bit of time to play with a trunk snapshot (up to
    perforce changelist 14501 right now). Seems like a configurable
    ThreadFactory is, in fact, what we need, so I've attached a quick
    patch to allow this. Let me know if you'd rather it be in JIRA, or if
    we need more discussion first (here or in JIRA) instead.

    A couple of questions I've come across not directly related to the
    shepherd thread:

    1) We're still using mondrian.olap.* rather than the olap4j API, and
    are starting to get some deprecation warnings with 3.3 code. Fine for
    now, but looking into moving over to olap4j instead, I don't see an
    obvious replacement for m.o.DriverManager#getConnection(PropertyList,
    CatalogLocator, DataSource). We basically want to get an
    OlapConnection and pass it an existing, configured JDBC DataSource to
    use for db access.

    2) What's the best way to get the QueryTiming information after a
    result has been returned? Right now, I'm casting the
    mondrian.olap.Result into a RolapResult and calling
    RolapResult#getExecution().getQueryTiming(), but seems like there
    might be a more elegant way of accessing it without needing a cast?

    Thanks,
    -Joe

    On Fri, Jul 22, 2011 at 12:10 PM, Joe Barnett <thejoe (AT) gmail (DOT) com> wrote:[color=blue]
    > On Thu, Jul 14, 2011 at 7:56 AM, Luc Boudreau <lucboudreau (AT) gmail (DOT) com> wrote:[color=green]
    >>[color=darkred]
    >>> Do you have a sense of the differences in response time between
    >>> cancel()ing the request without this extra thread vs/ with it?

  9. #9
    Joe Barnett Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    ugh, attached the wrong patch, so just filed a JIRA with the right one:

    http://jira.pentaho.com/browse/MONDRIAN-976

    -Joe

    On Mon, Aug 8, 2011 at 2:04 PM, Joe Barnett <thejoe (AT) gmail (DOT) com> wrote:[color=blue]
    > Ok, I've had a little bit of time to play with a trunk snapshot (up to
    > perforce changelist 14501 right now).

  10. #10
    Julian Hyde Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    > On Mon, Aug 8, 2011 at 2:04 PM, Joe Barnett <thejoe (AT) gmail (DOT) com> wrote:
    >> Ok, I've had a little bit of time to play with a trunk snapshot (up to
    >> perforce changelist 14501 right now). Seems like a configurable
    >> ThreadFactory is, in fact, what we need, so I've attached a quick
    >> patch to allow this. Let me know if you'd rather it be in JIRA, or if
    >> we need more discussion first (here or in JIRA) instead.



    I'm trying to get away from thread-locals. (They're the new global variables, you know!)

    So I'd rather find a solution to this problem that doesn't involve creating new threads. That's why I created the new classes Execution and Locus -- contexts that can be passed around, regardless of what thread is using them.

    RolapConnection.execute(Execution) ensures that the Execution is passed to executeInternal. I think that Execution object is sufficient context.

    If you still want to weave associations between threads, you should be able to use Execution.id as key to a hashmap.

    >>
    >> A couple of questions I've come across not directly related to the
    >> shepherd thread:
    >>
    >> 1) We're still using mondrian.olap.* rather than the olap4j API, and
    >> are starting to get some deprecation warnings with 3.3 code. Fine for
    >> now, but looking into moving over to olap4j instead, I don't see an
    >> obvious replacement for m.o.DriverManager#getConnection(PropertyList,
    >> CatalogLocator, DataSource). We basically want to get an
    >> OlapConnection and pass it an existing, configured JDBC DataSource to
    >> use for db access.


    Pentaho BI server had the same problem. See how RolapConnection.createDataSource creates a javax.sql.DataSource from a name. It is usually the name of a class, but not necessarily. You can store a DataSource instance in a LockBox, pass its moniker (a string) as one of the properties of the olap4j connection, then resolve the object inside mondrian.

    Also note the interface DataSourceResolver, and its implementation JndiDataSourceResolver. Rather than using LockBox, you could stash your data source object in JNDI.

    Sorry it's so hairy. But it's JDBC's fault -- it only allows string-valued properties.

    >>
    >> 2) What's the best way to get the QueryTiming information after a
    >> result has been returned? Right now, I'm casting the
    >> mondrian.olap.Result into a RolapResult and calling
    >> RolapResult#getExecution().getQueryTiming(), but seems like there
    >> might be a more elegant way of accessing it without needing a cast?


    Call MondrianOlap4jStatement.enableProfiling(ProfileHandler). The ProfileHandler will be called back after the statement has completed.

    Julian
    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  11. #11
    Joe Barnett Guest

    Default Re: [Mondrian] Mondrian Query Timeout

    On Mon, Aug 8, 2011 at 11:27 PM, Julian Hyde <jhyde (AT) pentaho (DOT) com> wrote:[color=blue]
    >[color=green]
    >> On Mon, Aug 8, 2011 at 2:04 PM, Joe Barnett <thejoe (AT) gmail (DOT) com> wrote:[color=darkred]
    >>> Ok, I've had a little bit of time to play with a trunk snapshot (up to
    >>> perforce changelist 14501 right now).

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.