3rd-party-integrations/github-team-sync

Add pagination for lookups with more than 1000 entries

primetheus opened this issue · 1 comments

When looking up groups in AD, if there are more than 1000 users that are returned it will err out with the following message:

Traceback (most recent call last):
  File "ad_user.py", line 71, in <module>
    print('|--------------------------------------------------------------------------------|')
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 852, in search_s
    return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 846, in search_ext_s
    return self.result(msgid,all=1,timeout=timeout)[1]
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 738, in result
    resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 742, in result2
    resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 749, in result3
    resp_ctrl_classes=resp_ctrl_classes
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 756, in result4
    ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 329, in _ldap_call
    reraise(exc_type, exc_value, exc_traceback)
  File "/usr/local/lib/python2.7/site-packages/ldap/ldapobject.py", line 313, in _ldap_call
    result = func(*args,**kwargs)
ldap.SIZELIMIT_EXCEEDED: {'desc': u'Size limit exceeded'}

The size limit is configured on the server side, and in some cases cannot be changed. So we'll need to implement pagination to be able to gather all user accounts in this case

Solved with #7