PDA

View Full Version : Bulk Update



Murali Doss
11-20-2008, 02:24 PM
I have 3 million of records and it's takes 1 days to update only 100000 records.I like to know whether we can any concept of bulk update.

There is no issue with database because insert for 3 million takes only 3 hours to complete.Please adive me any good solution

tdidomenico
11-20-2008, 02:33 PM
First of all, you should try to improve your forum selection skills. The "Documentation" forum doesn't seem like the place for this thread :P

I'm assuming your question is related to Kettle, even though you don't say.

There's a lot of information that would be needed to be able to make informed suggestions: what db engine are you using? What kind of step for updating? Do you have indexes? Kettle version? And what could be a rather large etc...

Cheers!

Murali Doss
11-20-2008, 02:44 PM
Thanks for your reply.Sorry for posting the queries in documentation forum.

what db engine are you using -

What kind of step for updating -

Do you have indexes
Kettle version

Murali Doss
11-20-2008, 02:46 PM
Thanks for your reply.Sorry for posting the queries in documentation forum.

what db engine are you using - PostgreSQL 8.2.6 on x86_64-redhat-linux-gnu, compiled by GCC gcc (GCC) 4.1.2 20070925 (Red Hat 4.1.2-27).I am not sure whether i provided enough information

What kind of step for updating - update step in output design

Do you have indexes - 4 indexes

Kettle version - Kettle Spoon 3.1.0.-RC1

tdidomenico
11-20-2008, 03:04 PM
I'd recommend downloading version 3.1.0GA. It won't necessarily solve your problem but it's always a good idea to use the GA release.

Second, by searching the forums for "slow update" I fould these thread. Take a look at them and see if you can get some ideas:

http://forums.pentaho.org/showthread.php?t=65936
http://forums.pentaho.org/showthread.php?t=63126
http://forums.pentaho.org/showthread.php?t=58609

Cheers!