script 10-openldap fails if 79 character line wrap occurs

[Anganthier]

Summary

The slapd server won't start if the replhosts_sanity can't be resolved to an ip address. This happens, because the fqdn of the hostname behind the provider label is wrapped to the next line.

Steps to reproduce

I configured replication and used a long fqdn (Kubernetes StatefulSet Service) which will be wrapped by the slapd automatically.

What is the expected correct behavior?

Start of the slapd without an error

Relevant logs and/or screenshots

+ var_true FALSE
+ '[' FALSE = TRUE ']'
+ '[' FALSE = true ']'
+ '[' FALSE = YES ']'
+ '[' FALSE = yes ']'
+ '[' -e /etc/openldap/slapd.d/docker-openldap-was-started-with-replication ']'
+ set +e
++ grep -o ++ awk -h -F '[//]' '{ print $3 }'
'provider=ldap.*//.*.' '/etc/openldap/slapd.d/cn=config/olcDatabase={-1}frontend.ldif' '/etc/openldap/slapd.d/cn=config/olcDatabase={0}config.ldif' '/etc/openldap/slapd.d/cn=config/olcDatabase={1}mdb.ldif' '/etc/openldap/slapd.d/cn=config/olcDatabase={2}monitor.ldif'
++ awk -F binddn '{ print $1 }'
++ awk '!a[$0]++'
++ sed 's/  / /g'
++ ++ tr '\n' ' '
awk '!a[$0]++'
+ replhosts_sanity='openldap-0.openldap-headless.startse openldap-1.openldap-headless.startse '
+ '[' '!' -z 'openldap-0.openldap-headless.startse openldap-1.openldap-headless.startse ' ']'
+ for sanity_host in $replhosts_sanity
+ [[ openldap-0.openldap-headless.startse =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]
++ getent hosts openldap-0.openldap-headless.startse
++ awk '{ print $1 }'
+ sanity_ip=
+ '[' -z '' ']'
+ exit 1
[cont-init.d] 10-openldap: exited 1.

Environment

• Image version / tag: 7.0.3
• Host OS: ubuntu 18.04 / kubernetes 1.19.10

Example ldif file in the /etc/openldap/slap.d/cn=config directory:

olcDbIndex: entryCSN eq
olcDbIndex: entryUUID eq
olcDbIndex: objectClass eq
olcAccess: {0}to attrs=userPassword,shadowLastChange by self =xw by dn="cn=a
 dmin,dc=xxxxxxxxx,dc=xx,dc=com" write by anonymous auth by * none
olcAccess: {1}to * by self write by dn="cn=admin,dc=xxxxxxxxx,dc=xx,dc=com" 
 write by * read
olcAccess: {2}to * by self read by dn="cn=admin,dc=xxxxxxxxx,dc=xx,dc=com" w
 rite by dn="cn=reader,dc=xxxxxxxxx,dc=xx,dc=com" read by * none
olcSyncrepl: {0}rid=101 provider=ldap://openldap-0.openldap-headless.startse
 ite.svc.cluster.local binddn="cn=admin,dc=xxxxxxxxx,dc=xx,dc=com" bindmetho
 d=simple credentials=PASSWORD1 searchbase="dc=xxxxxxxxx,dc=xx,dc=com" type=
 refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 tls_reqcert=n
 ever
olcSyncrepl: {1}rid=102 provider=ldap://openldap-1.openldap-headless.startse
 ite.svc.cluster.local binddn="cn=admin,dc=xxxxxxxxx,dc=xx,dc=com" bindmetho
 d=simple credentials=PASSWORD1 searchbase="dc=xxxxxxxxx,dc=xx,dc=com" type=
 refreshAndPersist interval=00:00:00:10 retry="60 +" timeout=1 tls_reqcert=n
 ever
olcMirrorMode: TRUE
entryCSN: 20210507110835.079923Z#000000#002#000000
modifiersName: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth

Possible fixes

The lines have to be joined before trying to grep the content or the regex has to be extended so that multiple lines are joined correctly.

I found the following as a description how openldap actually splits the lines automatically:

OpenLDAP deletes the first white space of a continuation line and joins all lines.

[Anganthier]

Ok, looks like it is the wrong project. The script is inside the tiredofit/openldap container image.

[tiredofit]

Moved it over to the proper repo not to worry.
So just to be clear, it's not the LDIF that is writing/updating wrong, but it is the host checker function that is failing to determine if it can be solved because it is wrapping? I haven't run into this personally, my hosts list is about 3x the size of 179char.

I can look into this though, that function for checking hosts was written about 4 years ago in a haze - it deserves a review :)

[Anganthier]

I looked at the code and tried myself with a multine enabled grep statement but stopped. It got messy.

So just googled it and found https://richmegginson.livejournal.com/18726.html

The interresting part is here:

Here is an example:
$ attrname=svattrname
$ attrval=`ldapsearch ... '(somefilter)' $attrname | sed -n '/^'$attrname':/,/^$/ { /^'$attrname':/ { s/^'$attrname': *// ; h ; $ !d}; /^ / { H; $ !d}; /^ /! { x; s/\n //g; p; q}; $ { x; s/\n //g; p; q} }'`

I tried it with my file:

attrname=olcSyncrepl
sed -n '/^'$attrname':/,/^$/ { /^'$attrname':/ { s/^'$attrname': *// ; h ; $ !d}; /^ / { H; $ !d}; /^ /! { x; s/\n //g; p; q}; $ { x; s/\n //g; p; q} }'  test.ldif

The result was:

{1}rid=002 provider=ldap://openldap-1.openldap-headless.xxxxxx.svc.cluster.local binddn="cn=config" bindmethod=simple credentials=PASSWORD1 searchbase="cn=config" type=refreshAndPersist retry="5 5 60 +" timeout=1 filter="(!(objectclass=olcGlobal))" tls_reqcert=never

So the sed script successfully unwrapped the line. The only problem is that there are actually two or event more attributes with the same name in that file regarding replication and the example always find the last attribute only.

[Anganthier]

Moved it over to the proper repo not to worry.
So just to be clear, it's not the LDIF that is writing/updating wrong, but it is the host checker function that is failing to determine if it can be solved because it is wrapping? I haven't run into this personally, my hosts list is about 3x the size of 179char.

I can look into this though, that function for checking hosts was written about 4 years ago in a haze - it deserves a review :)

It seems to happen if you modify the values through ldapmodify just once. If slapd is rewriting those files it will wrap the lines. You can force that if you manually change that file using your editor of choice without recalculating the crc32.

I think it stopps in the 10-openldap script in part "Replication Sanity Tester" somewhere below line 370.

[tiredofit]

tiredofit/openldap:7.1.16 and tiredofit/openldap-fusiondirectory:7.0.4 will have a new variable for the time being REPLICATION_SAFETY_CHECK defaulted as TRUE. If switching it to FALSE it will skip that check while we get to the bottom of this.

Looking at that code I vaguely remember my reasoning for it, and it deserves a rewrite. I'm going to add it to the list to look into here on a rainy day, Let's hope in the interim you can still use the images with this patch.

[Anganthier]

I'll test as soon as the image is available in the repository,

[Anganthier]

Yes, the workaround works.

Thank you.