michael bienstein

03-21-2007, 08:41 AM

Bart and Julian,

I have read with interest your latest exchanges on testing and think it is all going in the right direction.

I have only two points to add about testing a dynamic database.

1) A situation in which we launch a process/thread that sends updates to a database at the same time as we perform MDX queries on it can best be tested by ensuring that we test invariant statistics that can be perturbed in transactions change. Some examples:

a) Always add transactions with a fact in one fact table A and its opposite in fact table B. Do statistics on the sum of these two tables.

b) Find the ratio of two sums over two measures and add in paired but separate transactions facts to the two tables. E.g. add one row to table A with amount X and two rows to table B with each amount X. Compare that the sums are in a ratio of 2:1.

c) We can pre-calculate the possible consistent values in a separate static table. E.g. if we stream facts to a table X then we can know the sum of the facts and the count of the facts that together can be correct. Run this as MDX and compare if it is a valid pair.

d) Alternatively we can obtain a timestamp or other transaction ID as Bart seems to already do in his DataSourcePlugIn and use this for a lookup of expected values.

2) The second idea is that we can have threads signal each other so that we do actually get deterministic behaviour. This is another reason for my design in the TaskSystem/Tasks. We can make a debugging TaskSystem that checks it got the right Tasks and then check deterministically by running different orders and interleaving deterministically external events.

Michael

___________________________________________________________________________

D

I have read with interest your latest exchanges on testing and think it is all going in the right direction.

I have only two points to add about testing a dynamic database.

1) A situation in which we launch a process/thread that sends updates to a database at the same time as we perform MDX queries on it can best be tested by ensuring that we test invariant statistics that can be perturbed in transactions change. Some examples:

a) Always add transactions with a fact in one fact table A and its opposite in fact table B. Do statistics on the sum of these two tables.

b) Find the ratio of two sums over two measures and add in paired but separate transactions facts to the two tables. E.g. add one row to table A with amount X and two rows to table B with each amount X. Compare that the sums are in a ratio of 2:1.

c) We can pre-calculate the possible consistent values in a separate static table. E.g. if we stream facts to a table X then we can know the sum of the facts and the count of the facts that together can be correct. Run this as MDX and compare if it is a valid pair.

d) Alternatively we can obtain a timestamp or other transaction ID as Bart seems to already do in his DataSourcePlugIn and use this for a lookup of expected values.

2) The second idea is that we can have threads signal each other so that we do actually get deterministic behaviour. This is another reason for my design in the TaskSystem/Tasks. We can make a debugging TaskSystem that checks it got the right Tasks and then check deterministically by running different orders and interleaving deterministically external events.

Michael

___________________________________________________________________________

D