Hitachi Vantara Pentaho Community Forums
Results 1 to 5 of 5

Thread: [Mondrian] RE: Eigenbase perforce change 13367 for review

  1. #1
    Julian Hyde Guest

    Default [Mondrian] RE: Eigenbase perforce change 13367 for review

    This is a smart fix, and I'm sure that it helps for really large schemas,
    but is it always a win? We need RolapEvaluator.push() to be efficient, and
    copying a treemap isn't that cheap: it involves an non-trivial iterator, and
    lots of memory allocations. A map from integer to members can obviously be
    implemented as an array. I want to be sure that we haven't made a critical
    piece of code much more expensive in the usual just to improve algorithmic
    complexity.

    Julian

    > -----Original Message-----
    > From: Matt Campbell [mailto:Matthew.Campbell (AT) thomson (DOT) com]
    > Sent: Wednesday, February 03, 2010 1:28 PM
    > To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
    > Sichi; Will Gorman
    > Subject: Eigenbase perforce change 13367 for review
    >
    > http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
    >
    > Change 13367 by mkambol@guest_AA-5501 on 2010/02/03 13:26:50
    >
    > MONDRIAN: performance enhancment. RolapEvaluator now
    > has a map that contains all members in the context minus any
    > "all" members in context. The new list is used in
    > getProperty and other places to significantly trim down the
    > number of iterations needed for each cell. The amount of
    > performance gain is proportional to the number of cells in
    > the results times the number of dimensions in the schema.
    >
    > Affected files ...
    >
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > ava#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 edit
    > ...
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 edit
    >
    > Differences ...
    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > ava#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > 27c27
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > 227a228,232
    > > * Returns an array of the non-All members which make

    > up the current context
    > > */
    > > Member[] getNonAllMembers();
    > >
    > > /**

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 $
    > 32c32
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 $
    > 98c98
    > < final Member[] currentMembers = evaluator.getMembers();
    > ---
    > > final Member[] currentMembers =

    > evaluator.getNonAllMembers();
    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 $
    > 45c45
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 $
    > 55a56
    > > private TreeMap<Integer, RolapMember> nonAllMemberMap;

    > 112a114,115
    > > nonAllMemberMap =
    > > (TreeMap<Integer, RolapMember>)

    > parent.nonAllMemberMap.clone();
    > 140a144,145
    > > nonAllMemberMap =
    > > (TreeMap<Integer, RolapMember>)

    > root.nonAllDefaultMembers.clone();
    > 223a229,233
    > > public final Member[] getNonAllMembers() {
    > > final Collection<RolapMember> members =

    > nonAllMemberMap.values();
    > > return members.toArray(new Member[members.size()]);
    > > }
    > >

    > 383a394,398
    > > if (m.isAll()) {
    > >

    > nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
    > > } else {
    > >

    > nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
    > > }

    > 600,602c615,617
    > < for (int i = 0; i < currentMembers.length; i++) {
    > < final Member member = currentMembers[i];
    > <
    > ---
    > > int i = -1;
    > > for (RolapMember member : nonAllMemberMap.values()) {
    > > i++;

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 $
    > 28c28
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 $
    > 47a48
    > > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;

    > 66a68
    > > nonAllDefaultMembers = new TreeMap<Integer, RolapMember>();

    > 91a94,97
    > > if (!defaultMember.isAll()) {
    > > nonAllDefaultMembers.put(
    > > hierarchy.getOrdinalInCube(), defaultMember);
    > > }

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 $
    > 32c32
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 $
    > 57c57
    > < Member[] members = evaluator.getMembers();
    > ---
    > > Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 $
    > 39c39
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 $
    > 688c688
    > < final Member[] members = evaluator.getMembers();
    > ---
    > > final Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 (ktext) ====
    >
    > 65c65
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 $
    > 93c93
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 $
    > 1132c1132
    > < final Member[] members = evaluator.getMembers();
    > ---
    > > final Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 $
    > 22c22
    > < * @version $Id:
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 $
    > 214a215,354
    > > /***
    > > * Tests performance of a larger schema with a large

    > number of result cells
    > > * Runs in 12.2 seconds when RolapEvaluator.getProperty

    > uses currentMemmber
    > > * Runs in 7.5 seconds when RolapEvaluator.getProperty

    > uses nonAllMemberMap
    > > * The performance boost gets more significant as the

    > schema size grows
    > > */
    > > public void testBigResultsWithBigSchemaPerforms() {
    > > TestContext testContext =

    > TestContext.createSubstitutingCube(
    > > "Sales",
    > > "<Dimension name=\"Gender2\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender3\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender4\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender5\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender6\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender7\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender8\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender9\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender10\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender11\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender12\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender13\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender14\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender15\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender16\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender17\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender18\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender19\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender20\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>",
    > > null);
    > > String mdx = "with "
    > > + " member [Measures].[one] as '1'"
    > > + " member [Measures].[two] as '2'"
    > > + " member [Measures].[three] as '3'"
    > > + " member [Measures].[four] as '4'"
    > > + " member [Measures].[five] as '5'"
    > > + " select "
    > > +

    > "{[Measures].[one],[Measures].[two],[Measures].[three],[Measur
    > es].[four],[Measures].[five]}"
    > > + " on 0, "
    > > +

    > "Crossjoin([Customers].[name].members,[Store].[Store Name].members)"
    > > + " on 1 from sales";
    > > long start = System.currentTimeMillis();
    > > testContext.executeQuery(mdx);
    > > printDuration("getProperty taking a long time", start);
    > > }
    > >

    >


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  2. #2
    Guest

    Default RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Hi Julian,
    I implemented nonAllMembers as a Map because the keys will be very
    sparse compared to possible values. If it was an array, it would have
    to be the full length of the currentMembers, and we'd have to filter out
    all the nulls whenever we wanted to use it. Do you see another way?

    Since TreeMap.clone does a shallow copy, and I expect nonAllMembersMap
    usually has a very small number of members in it, I figured it would be
    fast. I'll do some additional profiling to see what kind of time we are
    spending in the push. My previous profiling runs did not surface it as
    a hot spot, but I also wasn't specifically looking at that method.

    Kurt

    -----Original Message-----
    From: mondrian-bounces (AT) pentaho (DOT) org [mailto:mondrian-bounces (AT) pentaho (DOT) org]
    On Behalf Of Julian Hyde
    Sent: Wednesday, February 24, 2010 5:52 AM
    To: Campbell, Matthew (Healthcare USA)
    Cc: 'Mondrian developer mailing list'
    Subject: [Mondrian] RE: Eigenbase perforce change 13367 for review

    This is a smart fix, and I'm sure that it helps for really large
    schemas,
    but is it always a win? We need RolapEvaluator.push() to be efficient,
    and
    copying a treemap isn't that cheap: it involves an non-trivial iterator,
    and
    lots of memory allocations. A map from integer to members can obviously
    be
    implemented as an array. I want to be sure that we haven't made a
    critical
    piece of code much more expensive in the usual just to improve
    algorithmic
    complexity.

    Julian

    > -----Original Message-----
    > From: Matt Campbell [mailto:Matthew.Campbell (AT) thomson (DOT) com]
    > Sent: Wednesday, February 03, 2010 1:28 PM
    > To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
    > Sichi; Will Gorman
    > Subject: Eigenbase perforce change 13367 for review
    >
    > http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
    >
    > Change 13367 by mkambol@guest_AA-5501 on 2010/02/03 13:26:50
    >
    > MONDRIAN: performance enhancment. RolapEvaluator now
    > has a map that contains all members in the context minus any
    > "all" members in context. The new list is used in
    > getProperty and other places to significantly trim down the
    > number of iterations needed for each cell. The amount of
    > performance gain is proportional to the number of cells in
    > the results times the number of dimensions in the schema.
    >
    > Affected files ...
    >
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > ava#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 edit
    > ...
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 edit
    > ...
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 edit
    >
    > Differences ...
    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > ava#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > 27c27
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > 227a228,232
    > > * Returns an array of the non-All members which make

    > up the current context
    > > */
    > > Member[] getNonAllMembers();
    > >
    > > /**

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 $
    > 32c32
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > gationManager.java#2 $
    > 98c98
    > < final Member[] currentMembers = evaluator.getMembers();
    > ---
    > > final Member[] currentMembers =

    > evaluator.getNonAllMembers();
    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 $
    > 45c45
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > ator.java#2 $
    > 55a56
    > > private TreeMap<Integer, RolapMember> nonAllMemberMap;

    > 112a114,115
    > > nonAllMemberMap =
    > > (TreeMap<Integer, RolapMember>)

    > parent.nonAllMemberMap.clone();
    > 140a144,145
    > > nonAllMemberMap =
    > > (TreeMap<Integer, RolapMember>)

    > root.nonAllDefaultMembers.clone();
    > 223a229,233
    > > public final Member[] getNonAllMembers() {
    > > final Collection<RolapMember> members =

    > nonAllMemberMap.values();
    > > return members.toArray(new Member[members.size()]);
    > > }
    > >

    > 383a394,398
    > > if (m.isAll()) {
    > >

    > nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
    > > } else {
    > >

    > nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
    > > }

    > 600,602c615,617
    > < for (int i = 0; i < currentMembers.length; i++) {
    > < final Member member = currentMembers[i];
    > <
    > ---
    > > int i = -1;
    > > for (RolapMember member : nonAllMemberMap.values()) {
    > > i++;

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 $
    > 28c28
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > atorRoot.java#2 $
    > 47a48
    > > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;

    > 66a68
    > > nonAllDefaultMembers = new TreeMap<Integer, RolapMember>();

    > 91a94,97
    > > if (!defaultMember.isAll()) {
    > > nonAllDefaultMembers.put(
    > > hierarchy.getOrdinalInCube(), defaultMember);
    > > }

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 $
    > 32c32
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > intUtils.java#2 $
    > 57c57
    > < Member[] members = evaluator.getMembers();
    > ---
    > > Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 $
    > 39c39
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > ource.java#2 $
    > 688c688
    > < final Member[] members = evaluator.getMembers();
    > ---
    > > final Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 (ktext) ====
    >
    > 65c65
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 $
    > 93c93
    > < * @version $Id:
    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > ader.java#2 $
    > 1132c1132
    > < final Member[] members = evaluator.getMembers();
    > ---
    > > final Member[] members = evaluator.getNonAllMembers();

    >
    > ====
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 (ktext) ====
    >
    > 2c2
    > < // $Id:
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#1 $
    > ---
    > > // $Id:

    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 $
    > 22c22
    > < * @version $Id:
    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#1 $
    > ---
    > > * @version $Id:

    > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > anceTest.java#2 $
    > 214a215,354
    > > /***
    > > * Tests performance of a larger schema with a large

    > number of result cells
    > > * Runs in 12.2 seconds when RolapEvaluator.getProperty

    > uses currentMemmber
    > > * Runs in 7.5 seconds when RolapEvaluator.getProperty

    > uses nonAllMemberMap
    > > * The performance boost gets more significant as the

    > schema size grows
    > > */
    > > public void testBigResultsWithBigSchemaPerforms() {
    > > TestContext testContext =

    > TestContext.createSubstitutingCube(
    > > "Sales",
    > > "<Dimension name=\"Gender2\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender3\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender4\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender5\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender6\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender7\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender8\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender9\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender10\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender11\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender12\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender13\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender14\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender15\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender16\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender17\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender18\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender19\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>"
    > > + "<Dimension name=\"Gender20\"

    > foreignKey=\"customer_id\">\n"
    > > + "<Hierarchy hasAll=\"true\"

    > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > + " <Table name=\"customer\"/>\n"
    > > + " <Level name=\"Gender\"

    > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > + " </Hierarchy>"
    > > + "</Dimension>",
    > > null);
    > > String mdx = "with "
    > > + " member [Measures].[one] as '1'"
    > > + " member [Measures].[two] as '2'"
    > > + " member [Measures].[three] as '3'"
    > > + " member [Measures].[four] as '4'"
    > > + " member [Measures].[five] as '5'"
    > > + " select "
    > > +

    > "{[Measures].[one],[Measures].[two],[Measures].[three],[Measur
    > es].[four],[Measures].[five]}"
    > > + " on 0, "
    > > +

    > "Crossjoin([Customers].[name].members,[Store].[Store Name].members)"
    > > + " on 1 from sales";
    > > long start = System.currentTimeMillis();
    > > testContext.executeQuery(mdx);
    > > printDuration("getProperty taking a long time", start);
    > > }
    > >

    >


    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

    _______________________________________________
    Mondrian mailing list
    Mondrian (AT) pentaho (DOT) org
    http://lists.pentaho.org/mailman/listinfo/mondrian

  3. #3
    Julian Hyde Guest

    Default RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Kurtis,

    I've added PerformanceTest.testInMemoryCalc to illustrate the problem. Your
    change 13367 took the running time from 574s to 663s on my machine -- a 15%
    slowdown. My change 13397 used tunings in different areas to take the
    performance back to 614s, but that doesn't get you off the hook -- I need
    your change to be performance-neutral for this test.

    The map might typically have half a dozen to a dozen elements in it, and for
    a treemap each of those is a memory allocation. Pushing a new evaluator
    needs to be a very cheap operation, and since memory allocations have shown
    up a major factor when we profile, I have a preference for data structures
    that do fewer memory allocations. It's difficult to beat array-based data
    structure for that, and also because all elements tend to be in the same L1
    cache block.

    So I would be thinking about creating an implementation of Map based on an
    array. (If the map is sparse, you can simulate a sparse data structure in
    the array by using int offsets as 'pointers'.)

    Also worth finding a way to reduce the number of collections that are cloned
    each time an evaluator is pushed.

    Can you give me an idea of the timescale that you are going to work on this
    problem? If it's going to be more than a couple of weeks, I'd prefer to back
    out your change and wait for the real fix.

    Julian

    > -----Original Message-----
    > From: Kurtis.Walker (AT) thomsonreuters (DOT) com
    > [mailto:Kurtis.Walker (AT) thomsonreuters (DOT) com]
    > Sent: Wednesday, February 24, 2010 6:41 AM
    > To: jhyde (AT) pentaho (DOT) com; mondrian (AT) pentaho (DOT) org;
    > matthew.campbell (AT) thomsonreuters (DOT) com
    > Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > Hi Julian,
    > I implemented nonAllMembers as a Map because the keys will be very
    > sparse compared to possible values. If it was an array, it would have
    > to be the full length of the currentMembers, and we'd have to
    > filter out
    > all the nulls whenever we wanted to use it. Do you see another way?
    >
    > Since TreeMap.clone does a shallow copy, and I expect nonAllMembersMap
    > usually has a very small number of members in it, I figured
    > it would be
    > fast. I'll do some additional profiling to see what kind of
    > time we are
    > spending in the push. My previous profiling runs did not
    > surface it as
    > a hot spot, but I also wasn't specifically looking at that method.
    >
    > Kurt
    >
    > -----Original Message-----
    > From: mondrian-bounces (AT) pentaho (DOT) org
    > [mailto:mondrian-bounces (AT) pentaho (DOT) org]
    > On Behalf Of Julian Hyde
    > Sent: Wednesday, February 24, 2010 5:52 AM
    > To: Campbell, Matthew (Healthcare USA)
    > Cc: 'Mondrian developer mailing list'
    > Subject: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > This is a smart fix, and I'm sure that it helps for really large
    > schemas,
    > but is it always a win? We need RolapEvaluator.push() to be efficient,
    > and
    > copying a treemap isn't that cheap: it involves an
    > non-trivial iterator,
    > and
    > lots of memory allocations. A map from integer to members can
    > obviously
    > be
    > implemented as an array. I want to be sure that we haven't made a
    > critical
    > piece of code much more expensive in the usual just to improve
    > algorithmic
    > complexity.
    >
    > Julian
    >
    > > -----Original Message-----
    > > From: Matt Campbell [mailto:Matthew.Campbell (AT) thomson (DOT) com]
    > > Sent: Wednesday, February 03, 2010 1:28 PM
    > > To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
    > > Sichi; Will Gorman
    > > Subject: Eigenbase perforce change 13367 for review
    > >
    > > http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
    > >
    > > Change 13367 by mkambol@guest_AA-5501 on 2010/02/03 13:26:50
    > >
    > > MONDRIAN: performance enhancment. RolapEvaluator now
    > > has a map that contains all members in the context minus any
    > > "all" members in context. The new list is used in
    > > getProperty and other places to significantly trim down the
    > > number of iterations needed for each cell. The amount of
    > > performance gain is proportional to the number of cells in
    > > the results times the number of dimensions in the schema.
    > >
    > > Affected files ...
    > >
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 edit
    > >
    > > Differences ...
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > // $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 27c27
    > > < * @version $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > * @version $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 227a228,232
    > > > * Returns an array of the non-All members which make

    > > up the current context
    > > > */
    > > > Member[] getNonAllMembers();
    > > >
    > > > /**

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 98c98
    > > < final Member[] currentMembers = evaluator.getMembers();
    > > ---
    > > > final Member[] currentMembers =

    > > evaluator.getNonAllMembers();
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 45c45
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 55a56
    > > > private TreeMap<Integer, RolapMember> nonAllMemberMap;

    > > 112a114,115
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > parent.nonAllMemberMap.clone();
    > > 140a144,145
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > root.nonAllDefaultMembers.clone();
    > > 223a229,233
    > > > public final Member[] getNonAllMembers() {
    > > > final Collection<RolapMember> members =

    > > nonAllMemberMap.values();
    > > > return members.toArray(new Member[members.size()]);
    > > > }
    > > >

    > > 383a394,398
    > > > if (m.isAll()) {
    > > >

    > > nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
    > > > } else {
    > > >

    > > nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
    > > > }

    > > 600,602c615,617
    > > < for (int i = 0; i < currentMembers.length; i++) {
    > > < final Member member = currentMembers[i];
    > > <
    > > ---
    > > > int i = -1;
    > > > for (RolapMember member : nonAllMemberMap.values()) {
    > > > i++;

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 28c28
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 47a48
    > > > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;

    > > 66a68
    > > > nonAllDefaultMembers = new TreeMap<Integer,

    > RolapMember>();
    > > 91a94,97
    > > > if (!defaultMember.isAll()) {
    > > > nonAllDefaultMembers.put(
    > > > hierarchy.getOrdinalInCube(), defaultMember);
    > > > }

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 57c57
    > > < Member[] members = evaluator.getMembers();
    > > ---
    > > > Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 39c39
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 688c688
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 (ktext) ====
    > >
    > > 65c65
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 93c93
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 1132c1132
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 22c22
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 214a215,354
    > > > /***
    > > > * Tests performance of a larger schema with a large

    > > number of result cells
    > > > * Runs in 12.2 seconds when RolapEvaluator.getProperty

    > > uses currentMemmber
    > > > * Runs in 7.5 seconds when RolapEvaluator.getProperty

    > > uses nonAllMemberMap
    > > > * The performance boost gets more significant as the

    > > schema size grows
    > > > */
    > > > public void testBigResultsWithBigSchemaPerforms() {
    > > > TestContext testContext =

    > > TestContext.createSubstitutingCube(
    > > > "Sales",
    > > > "<Dimension name=\"Gender2\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender3\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender4\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender5\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender6\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender7\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender8\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender9\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender10\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender11\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender12\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender13\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender14\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender15\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender16\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender17\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender18\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender19\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n...Message truncated due to length...

  4. #4
    Guest

    Default RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Thanks for writing the test that demonstrates the degradation. I'll try
    some experiments and get back to you by end of day tomorrow to let you
    know when I'll be able to get in a complete fix. Thanks.

    Kurt

    -----Original Message-----
    From: Julian Hyde [mailto:jhyde (AT) pentaho (DOT) com]
    Sent: Saturday, March 13, 2010 3:36 PM
    To: Walker, Kurtis (Healthcare USA); mondrian (AT) pentaho (DOT) org; Campbell,
    Matthew (Healthcare USA)
    Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Kurtis,

    I've added PerformanceTest.testInMemoryCalc to illustrate the problem.
    Your
    change 13367 took the running time from 574s to 663s on my machine -- a
    15%
    slowdown. My change 13397 used tunings in different areas to take the
    performance back to 614s, but that doesn't get you off the hook -- I
    need
    your change to be performance-neutral for this test.

    The map might typically have half a dozen to a dozen elements in it, and
    for
    a treemap each of those is a memory allocation. Pushing a new evaluator
    needs to be a very cheap operation, and since memory allocations have
    shown
    up a major factor when we profile, I have a preference for data
    structures
    that do fewer memory allocations. It's difficult to beat array-based
    data
    structure for that, and also because all elements tend to be in the same
    L1
    cache block.

    So I would be thinking about creating an implementation of Map based on
    an
    array. (If the map is sparse, you can simulate a sparse data structure
    in
    the array by using int offsets as 'pointers'.)

    Also worth finding a way to reduce the number of collections that are
    cloned
    each time an evaluator is pushed.

    Can you give me an idea of the timescale that you are going to work on
    this
    problem? If it's going to be more than a couple of weeks, I'd prefer to
    back
    out your change and wait for the real fix.

    Julian

    > -----Original Message-----
    > From: Kurtis.Walker (AT) thomsonreuters (DOT) com
    > [mailto:Kurtis.Walker (AT) thomsonreuters (DOT) com]
    > Sent: Wednesday, February 24, 2010 6:41 AM
    > To: jhyde (AT) pentaho (DOT) com; mondrian (AT) pentaho (DOT) org;
    > matthew.campbell (AT) thomsonreuters (DOT) com
    > Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > Hi Julian,
    > I implemented nonAllMembers as a Map because the keys will be very
    > sparse compared to possible values. If it was an array, it would have
    > to be the full length of the currentMembers, and we'd have to
    > filter out
    > all the nulls whenever we wanted to use it. Do you see another way?
    >
    > Since TreeMap.clone does a shallow copy, and I expect nonAllMembersMap
    > usually has a very small number of members in it, I figured
    > it would be
    > fast. I'll do some additional profiling to see what kind of
    > time we are
    > spending in the push. My previous profiling runs did not
    > surface it as
    > a hot spot, but I also wasn't specifically looking at that method.
    >
    > Kurt
    >
    > -----Original Message-----
    > From: mondrian-bounces (AT) pentaho (DOT) org
    > [mailto:mondrian-bounces (AT) pentaho (DOT) org]
    > On Behalf Of Julian Hyde
    > Sent: Wednesday, February 24, 2010 5:52 AM
    > To: Campbell, Matthew (Healthcare USA)
    > Cc: 'Mondrian developer mailing list'
    > Subject: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > This is a smart fix, and I'm sure that it helps for really large
    > schemas,
    > but is it always a win? We need RolapEvaluator.push() to be efficient,
    > and
    > copying a treemap isn't that cheap: it involves an
    > non-trivial iterator,
    > and
    > lots of memory allocations. A map from integer to members can
    > obviously
    > be
    > implemented as an array. I want to be sure that we haven't made a
    > critical
    > piece of code much more expensive in the usual just to improve
    > algorithmic
    > complexity.
    >
    > Julian
    >
    > > -----Original Message-----
    > > From: Matt Campbell [mailto:Matthew.Campbell (AT) thomson (DOT) com]
    > > Sent: Wednesday, February 03, 2010 1:28 PM
    > > To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
    > > Sichi; Will Gorman
    > > Subject: Eigenbase perforce change 13367 for review
    > >
    > > http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
    > >
    > > Change 13367 by mkambol@guest_AA-5501 on 2010/02/03 13:26:50
    > >
    > > MONDRIAN: performance enhancment. RolapEvaluator now
    > > has a map that contains all members in the context minus any
    > > "all" members in context. The new list is used in
    > > getProperty and other places to significantly trim down the
    > > number of iterations needed for each cell. The amount of
    > > performance gain is proportional to the number of cells in
    > > the results times the number of dimensions in the schema.
    > >
    > > Affected files ...
    > >
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 edit
    > >
    > > Differences ...
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > // $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 27c27
    > > < * @version $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > * @version $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 227a228,232
    > > > * Returns an array of the non-All members which make

    > > up the current context
    > > > */
    > > > Member[] getNonAllMembers();
    > > >
    > > > /**

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 98c98
    > > < final Member[] currentMembers = evaluator.getMembers();
    > > ---
    > > > final Member[] currentMembers =

    > > evaluator.getNonAllMembers();
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 45c45
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 55a56
    > > > private TreeMap<Integer, RolapMember> nonAllMemberMap;

    > > 112a114,115
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > parent.nonAllMemberMap.clone();
    > > 140a144,145
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > root.nonAllDefaultMembers.clone();
    > > 223a229,233
    > > > public final Member[] getNonAllMembers() {
    > > > final Collection<RolapMember> members =

    > > nonAllMemberMap.values();
    > > > return members.toArray(new Member[members.size()]);
    > > > }
    > > >

    > > 383a394,398
    > > > if (m.isAll()) {
    > > >

    > > nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
    > > > } else {
    > > >

    > > nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
    > > > }

    > > 600,602c615,617
    > > < for (int i = 0; i < currentMembers.length; i++) {
    > > < final Member member = currentMembers[i];
    > > <
    > > ---
    > > > int i = -1;
    > > > for (RolapMember member : nonAllMemberMap.values()) {
    > > > i++;

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 28c28
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 47a48
    > > > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;

    > > 66a68
    > > > nonAllDefaultMembers = new TreeMap<Integer,

    > RolapMember>();
    > > 91a94,97
    > > > if (!defaultMember.isAll()) {
    > > > nonAllDefaultMembers.put(
    > > > hierarchy.getOrdinalInCube(), defaultMember);
    > > > }

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 57c57
    > > < Member[] members = evaluator.getMembers();
    > > ---
    > > > Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 39c39
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 688c688
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 (ktext) ====
    > >
    > > 65c65
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 93c93
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 1132c1132
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 22c22
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 214a215,354
    > > > /***
    > > > * Tests performance of a larger schema with a large

    > > number of result cells
    > > > * Runs in 12.2 seconds when RolapEvaluator.getProperty

    > > uses currentMemmber
    > > > * Runs in 7.5 seconds when RolapEvaluator.getProperty

    > > uses nonAllMemberMap
    > > > * The performance boost gets more significant as the

    > > schema size grows
    > > > */
    > > > public void testBigResultsWithBigSchemaPerforms() {
    > > > TestContext testContext =

    > > TestContext.createSubstitutingCube(
    > > > "Sales",
    > > > "<Dimension name=\"Gender2\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender3\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender4\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender5\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender6\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender7\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender8\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender9\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender10\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender11\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender12\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender13\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender14\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender15\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender16\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender17\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender18\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll\n...Message truncated due to length...

  5. #5
    Guest

    Default RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    I'll back out the Map from RolapEvaluator, but leave the method
    getNonAllMembers and do all the work in that method. This gives us
    worse performance, but it's better than before we created the Map.
    Eventually I'll work on finding a way to do the Map thing more
    efficiently.

    Kurt

    -----Original Message-----
    From: Walker, Kurtis (Healthcare USA)
    Sent: Monday, March 15, 2010 12:18 PM
    To: 'jhyde (AT) pentaho (DOT) com'; mondrian (AT) pentaho (DOT) org; Campbell, Matthew
    (Healthcare USA)
    Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Thanks for writing the test that demonstrates the degradation. I'll try
    some experiments and get back to you by end of day tomorrow to let you
    know when I'll be able to get in a complete fix. Thanks.

    Kurt

    -----Original Message-----
    From: Julian Hyde [mailto:jhyde (AT) pentaho (DOT) com]
    Sent: Saturday, March 13, 2010 3:36 PM
    To: Walker, Kurtis (Healthcare USA); mondrian (AT) pentaho (DOT) org; Campbell,
    Matthew (Healthcare USA)
    Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review

    Kurtis,

    I've added PerformanceTest.testInMemoryCalc to illustrate the problem.
    Your
    change 13367 took the running time from 574s to 663s on my machine -- a
    15%
    slowdown. My change 13397 used tunings in different areas to take the
    performance back to 614s, but that doesn't get you off the hook -- I
    need
    your change to be performance-neutral for this test.

    The map might typically have half a dozen to a dozen elements in it, and
    for
    a treemap each of those is a memory allocation. Pushing a new evaluator
    needs to be a very cheap operation, and since memory allocations have
    shown
    up a major factor when we profile, I have a preference for data
    structures
    that do fewer memory allocations. It's difficult to beat array-based
    data
    structure for that, and also because all elements tend to be in the same
    L1
    cache block.

    So I would be thinking about creating an implementation of Map based on
    an
    array. (If the map is sparse, you can simulate a sparse data structure
    in
    the array by using int offsets as 'pointers'.)

    Also worth finding a way to reduce the number of collections that are
    cloned
    each time an evaluator is pushed.

    Can you give me an idea of the timescale that you are going to work on
    this
    problem? If it's going to be more than a couple of weeks, I'd prefer to
    back
    out your change and wait for the real fix.

    Julian

    > -----Original Message-----
    > From: Kurtis.Walker (AT) thomsonreuters (DOT) com
    > [mailto:Kurtis.Walker (AT) thomsonreuters (DOT) com]
    > Sent: Wednesday, February 24, 2010 6:41 AM
    > To: jhyde (AT) pentaho (DOT) com; mondrian (AT) pentaho (DOT) org;
    > matthew.campbell (AT) thomsonreuters (DOT) com
    > Subject: RE: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > Hi Julian,
    > I implemented nonAllMembers as a Map because the keys will be very
    > sparse compared to possible values. If it was an array, it would have
    > to be the full length of the currentMembers, and we'd have to
    > filter out
    > all the nulls whenever we wanted to use it. Do you see another way?
    >
    > Since TreeMap.clone does a shallow copy, and I expect nonAllMembersMap
    > usually has a very small number of members in it, I figured
    > it would be
    > fast. I'll do some additional profiling to see what kind of
    > time we are
    > spending in the push. My previous profiling runs did not
    > surface it as
    > a hot spot, but I also wasn't specifically looking at that method.
    >
    > Kurt
    >
    > -----Original Message-----
    > From: mondrian-bounces (AT) pentaho (DOT) org
    > [mailto:mondrian-bounces (AT) pentaho (DOT) org]
    > On Behalf Of Julian Hyde
    > Sent: Wednesday, February 24, 2010 5:52 AM
    > To: Campbell, Matthew (Healthcare USA)
    > Cc: 'Mondrian developer mailing list'
    > Subject: [Mondrian] RE: Eigenbase perforce change 13367 for review
    >
    > This is a smart fix, and I'm sure that it helps for really large
    > schemas,
    > but is it always a win? We need RolapEvaluator.push() to be efficient,
    > and
    > copying a treemap isn't that cheap: it involves an
    > non-trivial iterator,
    > and
    > lots of memory allocations. A map from integer to members can
    > obviously
    > be
    > implemented as an array. I want to be sure that we haven't made a
    > critical
    > piece of code much more expensive in the usual just to improve
    > algorithmic
    > complexity.
    >
    > Julian
    >
    > > -----Original Message-----
    > > From: Matt Campbell [mailto:Matthew.Campbell (AT) thomson (DOT) com]
    > > Sent: Wednesday, February 03, 2010 1:28 PM
    > > To: Aaron Phillips; Ezequiel Cuellar; Julian Hyde; John V.
    > > Sichi; Will Gorman
    > > Subject: Eigenbase perforce change 13367 for review
    > >
    > > http://p4web.eigenbase.org/@md=d&c=6PU@//13367?ac=10
    > >
    > > Change 13367 by mkambol@guest_AA-5501 on 2010/02/03 13:26:50
    > >
    > > MONDRIAN: performance enhancment. RolapEvaluator now
    > > has a map that contains all members in the context minus any
    > > "all" members in context. The new list is used in
    > > getProperty and other places to significantly trim down the
    > > number of iterations needed for each cell. The amount of
    > > performance gain is proportional to the number of cells in
    > > the results times the number of dimensions in the schema.
    > >
    > > Affected files ...
    > >
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 edit
    > > ...
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 edit
    > >
    > > Differences ...
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.j
    > > ava#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > // $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 27c27
    > > < * @version $Id:
    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#1 $
    > > ---
    > > > * @version $Id:

    > >

    > //open/mondrian-release/3.2/src/main/mondrian/olap/Evaluator.java#2 $
    > > 227a228,232
    > > > * Returns an array of the non-All members which make

    > > up the current context
    > > > */
    > > > Member[] getNonAllMembers();
    > > >
    > > > /**

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapAggre
    > > gationManager.java#2 $
    > > 98c98
    > > < final Member[] currentMembers = evaluator.getMembers();
    > > ---
    > > > final Member[] currentMembers =

    > > evaluator.getNonAllMembers();
    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 45c45
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > ator.java#2 $
    > > 55a56
    > > > private TreeMap<Integer, RolapMember> nonAllMemberMap;

    > > 112a114,115
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > parent.nonAllMemberMap.clone();
    > > 140a144,145
    > > > nonAllMemberMap =
    > > > (TreeMap<Integer, RolapMember>)

    > > root.nonAllDefaultMembers.clone();
    > > 223a229,233
    > > > public final Member[] getNonAllMembers() {
    > > > final Collection<RolapMember> members =

    > > nonAllMemberMap.values();
    > > > return members.toArray(new Member[members.size()]);
    > > > }
    > > >

    > > 383a394,398
    > > > if (m.isAll()) {
    > > >

    > > nonAllMemberMap.remove(m.getHierarchy().getOrdinalInCube());
    > > > } else {
    > > >

    > > nonAllMemberMap.put(m.getHierarchy().getOrdinalInCube(), m);
    > > > }

    > > 600,602c615,617
    > > < for (int i = 0; i < currentMembers.length; i++) {
    > > < final Member member = currentMembers[i];
    > > <
    > > ---
    > > > int i = -1;
    > > > for (RolapMember member : nonAllMemberMap.values()) {
    > > > i++;

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 28c28
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/RolapEvalu
    > > atorRoot.java#2 $
    > > 47a48
    > > > final TreeMap<Integer, RolapMember> nonAllDefaultMembers;

    > > 66a68
    > > > nonAllDefaultMembers = new TreeMap<Integer,

    > RolapMember>();
    > > 91a94,97
    > > > if (!defaultMember.isAll()) {
    > > > nonAllDefaultMembers.put(
    > > > hierarchy.getOrdinalInCube(), defaultMember);
    > > > }

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 32c32
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlConstra
    > > intUtils.java#2 $
    > > 57c57
    > > < Member[] members = evaluator.getMembers();
    > > ---
    > > > Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 39c39
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlMemberS
    > > ource.java#2 $
    > > 688c688
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 (ktext) ====
    > >
    > > 65c65
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 93c93
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/src/main/mondrian/rolap/SqlTupleRe
    > > ader.java#2 $
    > > 1132c1132
    > > < final Member[] members = evaluator.getMembers();
    > > ---
    > > > final Member[] members = evaluator.getNonAllMembers();

    > >
    > > ====
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 (ktext) ====
    > >
    > > 2c2
    > > < // $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > // $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 22c22
    > > < * @version $Id:
    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#1 $
    > > ---
    > > > * @version $Id:

    > > //open/mondrian-release/3.2/testsrc/main/mondrian/test/Perform
    > > anceTest.java#2 $
    > > 214a215,354
    > > > /***
    > > > * Tests performance of a larger schema with a large

    > > number of result cells
    > > > * Runs in 12.2 seconds when RolapEvaluator.getProperty

    > > uses currentMemmber
    > > > * Runs in 7.5 seconds when RolapEvaluator.getProperty

    > > uses nonAllMemberMap
    > > > * The performance boost gets more significant as the

    > > schema size grows
    > > > */
    > > > public void testBigResultsWithBigSchemaPerforms() {
    > > > TestContext testContext =

    > > TestContext.createSubstitutingCube(
    > > > "Sales",
    > > > "<Dimension name=\"Gender2\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender3\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender4\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender5\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender6\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender7\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender8\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender9\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender10\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender11\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender12\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender13\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender14\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender15\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"
    > > > + "<Dimension name=\"Gender16\"

    > > foreignKey=\"customer_id\">\n"
    > > > + "<Hierarchy hasAll=\"true\"

    > > allMemberName=\"All Gender\" primaryKey=\"customer_id\">\n"
    > > > + " <Table name=\"customer\"/>\n"
    > > > + " <Level name=\"Gender\"

    > > column=\"gender\" uniqueMembers=\"true\"/>\n"
    > > > + " </Hierarchy>"
    > > > + "</Dimension>"

    \n...Message truncated due to length...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.