Earlier today I got an alert that the LDAP service on my FreeIPA server was down. This was not long after I had received another alert that the drive space on the /var partition was critical. I logged on, freed up some drive space, and tried to start the service:
$ sudo ipactl start
Starting Directory Service
Failed to start Directory Service: Command '/bin/systemctl start dirsrv@FREEIPA-OSRIC-NET.service' returned non-zero exit status 1I tried running systemctl directly to see the error message:
$ sudo systemctl start dirsrv@FREEIPA-OSRIC-NET.service
It produced an error again, as expected, so I examined the status message:
$ systemctl -l status dirsrv@FREEIPA-OSRIC-NET.service
● dirsrv@FREEIPA-OSRIC-NET.service - 389 Directory Server FREEIPA-OSRIC-NET.
Loaded: loaded (/usr/lib/systemd/system/dirsrv@.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2018-10-30 10:41:41 CDT; 27s ago
Process: 23515 ExecStart=/usr/sbin/ns-slapd -D /etc/dirsrv/slapd-%i -i /var/run/dirsrv/slapd-%i.pid (code=exited, status=1/FAILURE)
Process: 23510 ExecStartPre=/usr/sbin/ds_systemd_ask_password_acl /etc/dirsrv/slapd-%i/dse.ldif (code=exited, status=0/SUCCESS)
Main PID: 23515 (code=exited, status=1/FAILURE)
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.555157141 -0500] - ERR - plugin_dependency_startall - object plugin Roles Plugin is not started
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.556541498 -0500] - ERR - plugin_dependency_startall - preoperation plugin sudorule name uniqueness is not started
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.557822184 -0500] - ERR - plugin_dependency_startall - preoperation plugin uid uniqueness is not started
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.559024594 -0500] - ERR - plugin_dependency_startall - object plugin USN is not started
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.560834787 -0500] - ERR - plugin_dependency_startall - object plugin Views is not started
Oct 30 10:41:41 freeipa.osric.net ns-slapd[23515]: [30/Oct/2018:10:41:41.564809195 -0500] - ERR - plugin_dependency_startall - extendedop plugin whoami is not started
Oct 30 10:41:41 freeipa.osric.net systemd[1]: dirsrv@FREEIPA-OSRIC-NET.service: main process exited, code=exited, status=1/FAILURE
Oct 30 10:41:41 freeipa.osric.net systemd[1]: Failed to start 389 Directory Server FREEIPA-OSRIC-NET..
Oct 30 10:41:41 freeipa.osric.net systemd[1]: Unit dirsrv@FREEIPA-OSRIC-NET.service entered failed state.
Oct 30 10:41:41 freeipa.osric.net systemd[1]: dirsrv@FREEIPA-OSRIC-NET.service failed.Several dependencies the directory server expects to find aren’t running! I assumed they crashed when the drive ran out of space, so I tried rebooting.
Once the system restarted, I confirmed that the directory server and other FreeIPA services started as expected:
$ sudo ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa: INFO: The ipactl command was successfulThat fixed it! Sometimes the most basic troubleshooting steps are enough.