PDA

View Full Version : Performance problems saving to Mysql DB



begunrom
10-24-2007, 08:12 AM
I use Metadataeditor 1.6.0.GA.162

I clean delete all tables form the metadata database, in order to get new tables when metadataeditor starts up.
I save and give it a name.
I create a connection and load 100 tables. It takes 50 seconds to load it from an AS/400 DB2/400 database. Not fast but ...
When i save again it takes over 5 minutes to save the whole to the database.
Does anyone have better performance using mysql5?


repository.properties is


# Class name of MDR storage plugin factory
org.netbeans.mdr.storagemodel.StorageFactoryClassName=org.netbeans.mdr.persistence.jdbcimpl.JdbcStorageFactory
# JDBC driver for MySQL
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.driverClassName=org.gjt.mm.mysql.Driver
# URL for database storage
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.url=jdbc:mysql://bewymysql:3306/cwm_repo_test?defaultFetchSize=500&useCursorFetch=true
# user name
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.userName=cwm_repo_test
# password
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.password=xxxxxxxxx
# schema name (used as a table prefix)
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.schemaName=cwm
# turn on debugging!
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.debugPrint=false
# data type mappings...
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.datatype.mofid=BIGINT
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.datatype.streamable=BLOB
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.datatype.string=VARCHAR(750)
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.datatype.int=INT
# Query for duplicates to prevent transaction aborts
# This can be disabled for some future PostgreSQL version with real subtransaction support
MDRStorageProperty.org.netbeans.mdr.persistence.jdbcimpl.queryDuplicates=true

MattCasters
10-24-2007, 11:28 AM
The problem is in the Sun MDR logic. It uses tables to run the Metadata Object Facility (MOF).
Tables are dynamically created and destroyed to do so. That's causing the slowdown.
Personally, I would run on the local machine (from file) until I really had to push it into a MySQL repository.
Even safer is to export to XMI after everything is OK and then import it into the MySQL repository.

That way you can edit you models at ... euh... more reasonable speed.
BTW, this performance drop is the price we pay for going with the CWM model.
We are investigating alternatives, but it's a "nobody ever got fired for chosing CWM" situation ;-)

Take care,

Matt

marx
10-31-2007, 06:41 AM
Hi,
Could you elaborate a bit on "going with the CWM model" part. To my best understanding CWM is just a framework
(or a metamodel, if you wish) so I'd like to understand what do you mean by 'CWM model' here. Do you have any
technical details of your metadata implementation available? A whitepaper or an ER diagram perhaps?

Best regards,
marx

MattCasters
10-31-2007, 04:06 PM
CWM is indeed a metamodel. However, to persist/store a metamodel, you need to store it in a (Level 3) Metadata Object Facility, MOF.

These are the levels you can discern:

Level 0 : Data
Level 1 : Metadata (describing data)
Level 2 : Metamodel (describing metadata)
Level 3 : MOF (describing metamodels)

Typical examples of metamodels that can be stored in a MOF are the CWM and UML.

Note: There is no level 4 since a MOF can be stored in another MOF.

The MOF used here is the MetaData Repository (MDR) from Sun Microsystems Netbeans project.
--> http://mdr.netbeans.org/

HTH,

Matt