Reliability issues...

1 min read

[@121]
To 12.42.232.132:

If you think I'm dumb enough to allow javascript in my guestbook, you got another thing coming.

[@109]
Reliability issues…

I'm having some major reliability issues with this site. Spam is the problem. I'm using SpamAssassin's spamd/spamc which have a nasty tendancy to spike up the load.

If the load stays too high for too long, everything else starts dying. Changing the priority on the spamd deamon has had very little effect so far.

I emailed Matt over the weekend and asked him what he was using to monitor/restart Tomcat. He sent me some nifty scripts his ISP wrote for him. They gave me a pretty good idea as to what to look for.

I'm using monit, to monitor and restart services. Here's what I have for Tomcat 5 in my /etc/monitrc:

check process tomcat-apache with pidfile /var/run/tomcat5.pid
  start program = "/etc/rc.d/init.d/tomcat5 start"
  stop program = "/etc/rc.d/init.d/tomcat5 stop"
  if failed host www.thauvin.net port 80
    send "GET /blog/monit.jsp HTTP/1.0"
    expect "HTTP/[0-9.]{3} 200 .*"
    with timeout 15 seconds
  then restart
  every 3 cycles
  alert erik@localhost
  depends on tomcat
  group web

check process tomcat with pidfile /var/run/tomcat5.pid
  start program = "/etc/rc.d/init.d/tomcat5 start"
  stop program = "/etc/rc.d/init.d/tomcat5 stop"
  if failed port 8009 then restart
  if cpu > 80% for 8 cycles then restart
  if loadavg(15min) > 10 for 8 cycles then restart
  if 3 restarts within 5 cycles then timeout
  alert erik@localhost
  group web

So far it has been working like a charm.

Of course it isn't a permanent solution. I need to address the real problem. In the next few days I'll be installing RBLs directly into sendmail which I'm hoping will deal with it once and for all.

Where did the links go? Visit my new Linkblog.