Server issues, again

April 5, 2005 | 15 comments
By

We just had a technical mini-crisis. I think it’s past.

The crisis went like this — server load shot up to 80, memory used was at 80%, and cppop, imap, and ftpd all failed. (This is _exactly_ what happened to us at our old new server, and got that account suspended.)

I temporarily disabled a few plugins. It didn’t seem to help.

The thought occurred to me that blacklist might be causing the problem. I took blacklist and flood control off for a few minutes. It lookied like things were getting better.

Then we got hit with a huge resource spike that bumped server use to 160 (our green-light amount is 4!), and knocked Apache into “failed.” At that point, I disabled the comment script altogether. (If that killed your comment-in-progress, I apologize).

That brought things back to earth very quickly. Apache eventually went green again (whew!).

I believe that the problem is that we’re getting hit with a flood of spam. This is triggering repeated use of moderation and blacklist, which is straining server resources. (Each time blacklist runs, it checks each comment against a list of over 3000 banned sites).

I could be wrong. Anyway, we’re continuing to see if we can figure out what’s going on.

Be the first to like.

15 Responses to Server issues, again

  1. Kaimi on April 5, 2005 at 3:35 pm

    Test comment

  2. Kaimi on April 5, 2005 at 3:38 pm

    Woo-hoo! Green lights all across, except for server load which is at a mere 6.36 and dropping.

  3. Kaimi on April 5, 2005 at 3:41 pm

    Damn it! I thought the crisis was over. We even had all green, briefly. Now server load is going up again. Not good.

  4. Kaimi on April 5, 2005 at 3:58 pm

    TEST.

  5. Kaimi on April 5, 2005 at 4:16 pm

    TEST AGAIN.

  6. Eric James Stone on April 5, 2005 at 4:27 pm

    As far as I know, the “die” option ends the script immediately, which helps cut the server load.

  7. Dave on April 5, 2005 at 4:52 pm

    Yes, last night and this morning I noticed spam in the comment queue. If you start charging for comments, the problem should go away. ;-) Make it a per line charge, and the comment bloat problem will go away too.

  8. Kaimi on April 5, 2005 at 4:52 pm

    Thanks for the information, Eric.

  9. Kaimi on April 5, 2005 at 5:34 pm

    TEST TEST

  10. a random John on April 6, 2005 at 10:50 am

    does the blacklist search through each site in the list one by one? A hashtable would be a more efficient structure. Still, it seems that even 3,000 sites isn’t that many.

  11. Kaimi on April 7, 2005 at 11:18 am

    TEST COMMENT.

    TEST TEST TEST.

  12. danithew on April 7, 2005 at 12:41 pm

    When trying to arrive at T&S I find that about 50% of the time I’m getting an error page of some sort. Is anyone/everyone else having this problem?

  13. danithew on April 7, 2005 at 1:06 pm

    This is the url I’m hitting almost constantly when trying to access T&S:

    http://baby-insight.com/suspended.page/

  14. Kaimi on April 7, 2005 at 1:12 pm

    Ack. DNS errors, again.

  15. danithew on April 7, 2005 at 7:59 pm

    By the way … nice favicon.

WELCOME

Times and Seasons is a place to gather and discuss ideas of interest to faithful Latter-day Saints.