Hitachi Vantara Pentaho Community Forums
Results 1 to 25 of 25

Thread: JSON Input Step Error "We MUST have the same number of values for all paths"

  1. #1
    Join Date
    Oct 2011
    Posts
    5

    Default JSON Input Step Error "We MUST have the same number of values for all paths"

    Hi,
    I have a JSON input step in my transformation. The json object can have optional fields. So I have checked the option "Ignore Missing Path". But still I get the error "The data structure is not the same inside the resource! We found 46 values for json path [$.results[*].doc.event.credentialID], which is different that the number retourned for path [$.results[*].doc.event.location] (51 values). We MUST have the same number of values for all paths."

    Could you please let me know if I am missing something.

    Thanks
    Satyaprakash J

  2. #2
    Join Date
    Dec 2011
    Posts
    2

    Default

    Hi,

    I too have the same problem, i doubt whether this option works "Ignore Missing Path", is this a bug or do we have to work around to avoid additional fields in JSON object.

    Please let us know what this option "Ignore Missing Path" does.

    Thanks.

  3. #3
    Join Date
    Aug 2011
    Posts
    236

    Default

    Hi,

    I had this issue and got around it be looking for a json parser - I found one and just code the field parsing using modified javascript component.

    https://github.com/douglascrockford/...aster/json2.js

    Not pretty but the only way I found to work around the bug.

    HTH.

  4. #4

    Default

    Sorry to beat a long-gone horse, but in short, this is due to querying sub-arrays of a JSON document.

    If multiple Paths are queried in a JSON Input step, each of the results of the Path query need to return the same number of rows. If multiple arrays are being used, it's better to break out each JSON array into a field, and then use multiple JSON Input steps downstream, one for each array that needs to be exploded/unwound.

    The ignore missing path only applies if 0 results are found for a Path query, it doesn't apply if different numbers of results are found across Path queries in the step.

  5. #5
    Join Date
    Aug 2015
    Posts
    313

    Default

    Hi ,

    I am facing issue with uneven data for JSON file, could you please post your sample transformation( I'm new to json implementation).

    Error: - which is different that the number returned for path [$.items..entityName] (2 values). We MUST have the same number of values for all paths.

    Attached the my uneven data json file.

    Thank you
    Attached Files Attached Files
    Last edited by santhi; 08-26-2015 at 07:25 AM.

  6. #6
    Join Date
    Aug 2015
    Posts
    313

    Default

    JSON ENTITY_SERVICE_STATUS.ktr

    finally got succeeded . Thank you, attached the sample ktr

  7. #7
    Join Date
    Oct 2015
    Posts
    8

    Default

    samplejson1.txtsamplejson1.txtHi everyone,

    I am getting same error "The data structure is not the same inside the resource! We found 9 values for json path [$..countryLocations[*]], which is different that the number returned for path [$..country_name] (8 values). We MUST have the same number of values for all paths."

    I have attached the files.
    MongoDBTransformation2.ktr
    Last edited by swathi4sap; 10-05-2015 at 01:25 PM. Reason: added json file

  8. #8
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Short answer: To avoid getting this error, do not use the .. notation in the JSON input step.

  9. #9
    Join Date
    Oct 2015
    Posts
    8

    Default

    If i remove .. null values are displaying, I need Country Locations array

  10. #10
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    I couldn't get it to work in a single JSON Input step.

    I ended up using 4 of them.
    1) New Column "Countries" path: $.regionCountries.*
    2) New Column "CountryName" JSON in column "Countries" path: $.country_name
    3) New Column "Locations" JSON in column "Countries" path: $.countryLocations.*
    --- Filter rows to deal with Countries with no Locations ---
    4) New Columns "Province","PostalCode","StreetAddress","LocationID","City" ... Details as an exercise.
    Last edited by gutlez; 10-05-2015 at 03:27 PM.

  11. #11
    Join Date
    Oct 2015
    Posts
    8

    Default

    i tried using $.regionCountries[*].country_name and $.regionCountries[*].countryLocations[*]

    getting blank values


  12. #12
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    See edited post.

  13. #13
    Join Date
    Oct 2015
    Posts
    8

    Default

    MongoDBTransformation2.ktr


    i have attached the latest ktr file.but still some errors
    Last edited by swathi4sap; 10-05-2015 at 03:42 PM.

  14. #14
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Your data flow doesn't make a lot of sense, but I don't know what you're trying to do.

    Try going MongoDB -> JSON (1) -> JSON (2) -> JSON (3) -> Filter Rows -> JSON (4)

    Using the JSON Input configurations I provided earlier.

  15. #15
    Join Date
    Oct 2015
    Posts
    8

    Default

    NO, This will not work.

  16. #16
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by swathi4sap View Post
    NO, This will not work.
    What will not work, and why not?

  17. #17
    Join Date
    Oct 2015
    Posts
    8

    Default

    Is this working for you?If it is working can you send the ktr of this.
    I need to convert total JSON into table format.that is my requirement.

    This is a bug from pentaho side, they are working on this issue.

  18. #18
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Yes, I was able to get it to work, exactly as I described in my posts. I don't agree that it's a bug - it's working exactly as they said it would.
    Unfortunately, I am not currently able to post files to the forum (limitations from my side, not the forum side)

  19. #19
    Join Date
    Oct 2015
    Posts
    8

    Default

    I dont know why, iam not able to do.

    if you send the file that will be gr8.
    Last edited by gutlez; 10-06-2015 at 01:59 PM.

  20. #20
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    I ended up using 4 of JSON Inputs.
    1) New Column "Countries" path: $.regionCountries.*
    2) New Column "CountryName" JSON in column "Countries" path: $.country_name
    3) New Column "Locations" JSON in column "Countries" path: $.countryLocations.*
    --- Filter rows to deal with Countries with no Locations ---
    4) New Columns "Province","PostalCode","StreetAddress","LocationID","City" ... Details as an exercise.

  21. #21
    Join Date
    Apr 2011
    Posts
    159

    Default

    Quote Originally Posted by gutlez View Post
    Short answer: To avoid getting this error, do not use the .. notation in the JSON input step.
    + on this avoid using the recursive ".."

    Thank you!

  22. #22
    Join Date
    Aug 2016
    Posts
    1

    Default

    Hi,
    i can not open this .ktr file. I am looking same solution but still searching..

  23. #23
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Vinoths,

    The solution is usually very simple. DO NOT USE .. in your JSON path.

  24. #24

    Default

    Quote Originally Posted by gutlez View Post
    Vinoths,

    The solution is usually very simple. DO NOT USE .. in your JSON path.
    Vinoths



    Can you give me a bit of a hint here.. I am doing my first Json script and am using the following
    $.result[*].location and getting the same error


    Greatly appreciated

  25. #25
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by meyerder View Post
    $.result[*].location and getting the same error
    You will get this error if you have different numbers of values for your keys.
    If you have:
    $.result[*].location
    and also
    $.result[*].sitename[*]
    Then your JSON is not balanced, and it will complain.
    Post a sample of your JSON and the JSONPath values that you are using, and we'll help you dig through it.

    If you have .. *ANYWHERE* in your JSONPath values, it is highly likely to cause issues.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.