lots of daemons dead but pid file exists

[Bill Campbell]

On Wed, Jul 18, 2007, kumarlimbu wrote:
>
>Hi everyone,
>
>We recently had problem with our linux server and had to (hard) restart it.
>During the restart we found that a lots of services were not able to start
>properly like ssh, apache, mysql, xfs etc. which caused the machine not
>being able to go to runlevel 5 (no GUI). So we switched to runlevel 3. We
>found that when we looked at the status for the usual services we found the
>following error:
><service> dead but pid file exists
>
>What could cause an error like this.
>
>I searched through the forums and found that most of the problem with
>xserver starting properly has to do with DISK full etc. but nothing was
>quite similar to what we had.

The proper way to use pid files is to test them with something like:

kill -0 `cat $pidfile`
if $? = 0
then
	echo it is running
else
	rm $pidfile
	echo it is not running
fi

Unfortunately some programs simply test for the existence of the
pid file, but don't test to see if there's still a process
running with that pid.  There are a few programs that create pid
files that have more than one line, or more than a simple pid
string so require something more than `cat $pidfile'' to get it.

The clamav program can be problematic as it doesn't create its
pid file until after it's read all the data files which can take
several minutes after an update so a program can check for a pid
file, not find it, then start muliple copies of the program.

Bill
--
INTERNET:   bill at celestial.com  Bill Campbell; Celestial Software LLC
URL: http://www.celestial.com/  PO Box 820; 6641 E. Mercer Way
FAX:            (206) 232-9186  Mercer Island, WA 98040-0820; (206) 236-1676

My reading of history convinces me that most bad government results
from too much government. --Thomas Jefferson.