Hitachi Vantara Pentaho Community Forums
Results 1 to 3 of 3

Thread: how to modify all transformations/jobs of a repository (for ex. edit all log config)

  1. #1
    Join Date
    Aug 2011
    Posts
    360

    Default how to modify all transformations/jobs of a repository (for ex. edit all log config)

    Hi folks,

    We're migrating from pdi 3.2 CE to pdi 5.4 EE.
    We have imported everything in repository, and everything's fine.
    However, know we need to configure the logging into database for all jobs and transformations.
    Doing it by hand is cumbersum and time consumming, haiving thousand of objects to edut.

    So the question is: is there a mean to edit some configuration of all transformation/jobs?

    I though I can export repository to xml, write XSLT to tweak some part like the logging configuration,
    And then import the result into repository.
    I could write prrof of concept XSLT, but have two problems:
    1. The numerique entity references are translated jn the output. I mean, stuff like $ arr converted
    Into the real caractere after the xslt transform. Which can cause problème with quotes or content
    Of scripts!

    2. The repository XML is too big (hundreds of Mo), getting Java heap space.

    Do you have some ideas to adress theses problems, or another method to do it?
    Was also thibking of using the java API to load the TransMeta objectd, tweeak them with API,
    then save into repo. But sounds risky, while more powerfull!

  2. #2
    Join Date
    Jun 2012
    Posts
    5,534

    Default

    I'm sure I would find me an UPDATE statement to adjust the logging right in the repository database, but why don't you ask Pentaho for advice? They sure won't turn a deaf ear to a paying customer.
    So long, and thanks for all the fish.

  3. #3
    Join Date
    Dec 2009
    Posts
    332

    Default

    Someone else may have a much better idea, but since I just performed this on a few hundred of our jobs/transforms ... (PDI 6.0EE/java7/linux)

    We do use a file based repository rather than the enterprise repository which makes this sort of thing easier.

    In case this helps: The XML escaped stuff like "$" appears to be optional to some extent - at least on linux it runs the same way in plain text or that escaped format. So while it can be a pain to xslt/sed/awk your way through it, the output does not appear to require the escaped format (again at least in linux). While it will be poorly formed XML, neither kitchen or spoon seem to encounter any problems with it. Try it out on one job/transform in your implentation before going wild though.

    We use kettle variables for everything which means we can change all the connection details (except type database type) on the fly. If your logging change is changing connection information, it would be best to change the static entry to a variable so it will be easier next time.

    I have also used pentaho to edit pentaho. Ran the transformation files through as xml files and parsed them that way.

    Good hunting.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.