All the Perl that's Practical to Extract and Report

perl.org: now more absorbent

Robrt on 2004-02-06T04:50:18

Yesterday we started absorbing mydoom instead of bouncing them. In around 24 hours, our main MX server for perl.org/cpan.org absorbed 1.3 GB of mydoom. That's 41083 messages. Or, 1710 an hour. Or, 29 a minute. Or, about one Mydoom every 2 seconds. Ouch!

In addition, we absorbed 354 MB of bounces, in 17394 messages. (One bounce every 5 seconds or so.)

The good news is that things seem to have stabilized. We have the machines configured such that they are no longer pegged to the wall. Occasionally a burst of mail will make them lean a little, but they're handling just fine. Mail is moving at full speed, transiting our system in seconds, instead of minutes or hours.

I hope the flow of mail slows soon - even though we are putting plans in place for more servers - I miss the old days, where you could just run sendmail and have it work.

2000 mydoom an hour, 2500 bounces an hour

merlyn on 2004-02-06T12:30:17

At the peak, the machine running stonehenge.com/geekcruises.com was getting 2000 mydoom per hour. I think it's slowed a bit, but the virus notifications are the worst.

configuration changes

WebDragon on 2004-02-06T16:08:28

not knowing a whole lot about mailserver configuration, I'd be very interested in knowing specifically what changes you had to make to the system in order to detect-and-absorb the mydoom stuff without also getting false-positives in the process. I mean, even spamassassin misses a few, or is overzealous on rare occasions.

And, if you've managed a 100% solution, a config file diff might also go a long way towards helping other admins (who also are not mailserver gurus :) do the same and thus further reduce the traffic.

Re:configuration changes

Elian on 2004-02-06T16:37:44

Dunno about perl.org specifically, but ClamAV seems to detect MyDoom and its variants. It's what I'm running on my linux-based mailserver as the virus scanner and while there's a bit of lag with new viruses (like any other system, as you need to wait for the signatures to be updated) it's as reliable as anything else I've found. And free, which is nice.

Re:configuration changes

krellis on 2004-02-06T18:20:12

Actually, in the case of MyDoom, ClamAV had signatures several hours before ANY of the commercial AV vendors. We've been blocking up to a peak of just over 5000 copies per hour on our mail servers using ClamAV. It's awesome.

Re:configuration changes

Robrt on 2004-02-06T18:27:06

We're using a hacked up qpsmtpd (soon to be rolled back into the main dist). Basically, we have a rule that matches mydoom, and directs it to a folder.

The rule is overly liberal right now - getting hit with 2 of these a second, spamassassin and clamav would grind us to a halt. But that's ok.

Of the 25000 emails caught in my mydoom filter in the past 13 hours, there are 1322 unique (case insensitive) subject lines. None of them look like anything someone will miss.