Hello Pentaho Community,
I've gone through many posts in the past few days, however was unable to find a suitable solution for my challenge. I hope some one has done something similar in the past and would be willing to share...
I have a requirement to get hundreds of tables residing on different databases flattened and shared with other system. Instead of creating a separate transformation for each table i need to export, i created one job that loops through a driver file that contains all parameters including SQL, output file, etc for all tables; that works just fine. The part that i'm not sure on how to implement is how to create new/select pre-defined dynamic connection from same driver file or list of existing shared connections. When the process runs, it goes though each record on the driver table and create a text file. I'm looking for a way to pass either all components of a connection as a variable to create that connection at run time or select pre-defined connection based on driver entry. I'm using Table Input control to pull the data from a SQL passed via a variable and need to be able to create/select a connection that will correspond to source table residing either on SQL Server/Oracle or Sybase databases. We have shared kettle.properties file on each environment to maintain corresponding connection properties. I think, it would be a simpler task if the data came from one database, but i cannot think of a way to create a connection based on database type.
PDI 5.2
Appreciate your help in advance!