Hitachi Vantara Pentaho Community Forums
Results 1 to 9 of 9

Thread: LDAP Input with WindowsAD reads 1000 rows

  1. #1
    Join Date
    May 2009
    Posts
    10

    Unhappy LDAP Input with WindowsAD reads 1000 rows

    I am trying to get the user list from our Windows Active Directory to a database. However LDAP Input reads only first 1000 rows. If anyone knows how to fix it, please let me know. I am using 3.2 ga PDI version.

  2. #2
    Join Date
    Nov 1999
    Posts
    9,729

    Default

    It seems to be an AD "feature" :

    From http://forums.sun.com/thread.jspa?threadID=581444 :

    One problem with large group memberships is that a feature of the Active Directory which is designed to ensure performance and limit Denial of Service attacks is that it limits the number of results that may be returned from a multi-valued attribute to 1000 values. Therefore the above code such would only retrieve 1000 members from a group even though it may contain more. (ed.)
    File a bug report so we can create a work-around.

  3. #3
    Join Date
    May 2009
    Posts
    10

    Default

    Is there anything I can do to avoid this feature from Windows AD?

  4. #4
    Join Date
    Jul 2007
    Posts
    1,013

  5. #5
    Join Date
    Apr 2009
    Posts
    12

    Default SimplePageSize or Use Script

    There is an LDAP parameter, simplepagesize, to tell the server to provide the results back in smaller chunks, such as 1000. Apparently, the Kettle LDAP client does not support this.

    One option is to get a stand-alone client (Netware, Oracle, Sun, et. al) that does support simplepagesize and build a script around it. Have the script output the results to a file then input the file with LDIF Input instead of LDAP input.

  6. #6

    Default

    Quote Originally Posted by Anupam View Post
    Is there anything I can do to avoid this feature from Windows AD?
    I know that this is really old, but I'm running 4.4. In the "Advanced" tab of the LDAP Input, turn on Paging, and set it to some number less than 1000.
    I set mine to 500 and was able to get all 9,734 from my AD in one go.

  7. #7
    Join Date
    Jan 2017
    Posts
    2

    Default

    Oh, old thread but really helped..

    Thanks, the "Pagin" worked for me

  8. #8

    Default

    This issue still exists. Any resolution?

  9. #9
    Join Date
    Apr 2008
    Posts
    4,696

    Default

    Quote Originally Posted by hemanth.vaddi View Post
    This issue still exists. Any resolution?
    The issue is on the Windows AD side, not the Pentaho side. Therefore Pentaho cannot fix it.
    There is a work-around in the thread if you read it.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Privacy Policy | Legal Notices | Safe Harbor Privacy Policy

Copyright © 2005 - 2019 Hitachi Vantara Corporation. All Rights Reserved.