Running mulitple instances of carte in parallel

[agrohe]

Has anyone run multiple instances of Carte on RHEL? It would seem possible using different ports, but if others have had mixed results, I would not want to go this route.

This is really for running multiple independent jobs or transformations in parallel.

[MattCasters]

You can **obviously** start as many as you like on different ports.

 This is really for running multiple independent jobs or transformations in parallel.

I know.

[agrohe]

Matt,

I understand how to do launch multiple instances. My question was more along the lines of "Are there technical reasons why one instance of carte might stomp on another instance?" Are some files shared, etc..

Is carte certified to be multi-thread safe?

If I have 8 processors and 16GB of RAM, I can run several instances of carte in parallel or as a cluster, rather than stuffing everything through on single-threaded path.

It is really a question of best practices of scalability across multiple transformations. Your previous posts on slave servers and clusters address single transformation scalability options.

[MattCasters]

The main purpose of Carte is to allow you to run transformations in a cluster.
It is typical and recommended to run multiple instances of carte on 4, 8 or 16 CPU boxes.
That is usually because it's sometimes hard to make a single JVM use all CPUs in a single machine.

If I have 8 processors and 16GB of RAM, I can run several instances of carte in parallel or as a cluster, rather than stuffing everything through on single-threaded path.

You can. That being said, there are no single-threaded transformations. They all run with 1 thread per step copy and you can launch multiple copies per step. (standard functionality, no clustering/carte-slaves involved)

Is carte certified to be multi-thread safe?

Since each Carte instance runs in a separate JVM, the answer to that question is most certainly "yes". :-)

Matt

[agrohe]

Thanks Matt.

This is what I was looking for. "It is typical and recommended".

[MattCasters]

Remember, YMMV. I saw a nice little transformation a few weeks ago running on all 8 CPUs of a box.
Nothing special was required. That was on Linux 64-bit. Now, on other occasions I've had trouble making a transformation use more than 1 and a half CPUs.
In those cases, you can go for clustering. In the first case, sticking with a standard transformation is better.

Matt