I loathe crackers. Not just the salty kind. Not the pejorative slang word for white people. The crackers I hate are the punks that contribute nothing positive or creative for the Internet community. Thugs with electronic bricks. I've never found computer security all that engrosing. I've never felt urge to comprimize other people's machines and set up IRC bots so that I can "get channel ops." Such a waste of my time it is to clean up after them. Here's what I've pieced together and what I've done to prevent this from happening again. Perhaps someone else will find this useful.
Last Thursday or Friday, I changed over my home's net connection from DSL to Comcast's cable modem service. The change-over went smoothly. I had no trouble adjusting my Linksys firewall for the new connection. Recall that one difference between DSL service and cable modem service is that cable modem users are on the same network segment. This will become omniously relevant later.
Last night, as most people were watching the "Stupor Bowl," I was watching 2001: A Space Odyssey. And drinking a fine bottle of Bunnahabhain 12 year-old scotch. Little did I know that at 6:56PM, some turd-burger from 80.185.242.59 (an IP apparently associated with a german ISP) waltzed in uninvited to my main linux box (RedHat 7.3) using ssh. Yes, I knew about the root exploit for the version of ssh on that box. However, it doesn't appear that this kind of exploit was used. Note the this portion of my last log:
jjohn pts/6 80.185.242.59 Sun Feb 1 22:51 - 22:56 (00:04) jjohn pts/2 80.185.242.59 Sun Feb 1 19:25 - 19:26 (00:01) jjohn pts/2 80.185.242.59 Sun Feb 1 18:56 - 18:59 (00:02)
My best guess is that the loser used my username and password against me, in a kind Internet jujitsu maneuver. Oddly, I found no record of commands issued by the interloper in .bash_history (my shell). A casual vetting of netstat resulted in the discovery of a new server process called 'join,' started from my home directory. I found no additional cron jobs or user accounts. The checksum of /bin/login matched that of an uncomprimized host. How could this have possibly happened? I (unfairly) blame the Erics Raymond and Allman.
I like command line mail programs. mail. pine. mutt. It's all good. I trace this fondness back to my inculcation into computers at UMass/Boston, whose CompSci program started students on an ol' skool VAX system. I like piping the output of a program to mail. I like grepping through mboxes. I enjoying laughing at the impotency of messages containing Windows-afflicting viruses. To this end I collect email from my various accounts with Raymond's fetchmail and good old insecure POP3. fetchmail sends login credentials across the wire in plain 7-bit ASCII with life-affirming machismo. Ashewing crypto, the use of the POP3 protocol in today's terror-alert world defiantly proclaims to all "I'm living like it's 1982, biznatch!"
To make matters worse, I've recently configured sendmail on my public relay host (aliensaliensaliens.com) to use the SASL authentication layer. This is a good thing (because it prevents naughty spammers from using A3 like a cheap harlot), but I didn't get around to setting up TLS. This means that, you guessed it, my credentials for A3 were also getting sent in plain text across the wire.
In my defense I must invoke the computer Barbie defense.
«Secure POP is hard! Configuring sendmail with STARTTLS/SASL is hard!»
Let me also jump ahead of my story to defend my system adminstrating virility and state that I have now successfully frobbed sendmail into using STARTTLS and created secure SSH2 tunnels for POP3 to travel through.
Even otherwise burly admin types I know will, when cornered and plastered with liquor, admit to their angst whenever they have to battle sendmail. The trouble doesn't squarely lie with sendmail, but rather with sendmail's intregingly opaque documentation. It's a nasty piece of work. It's not that the docs are wrong, slim or poorly organized. In fact, much of the sendmail documention is clear, professional and to the point. However, and I can't stress this point enough, it was written by Martians, for Martians. By this I mean that the docs do not have a task-oriented organization that can be readily consumed by a human sys admin. Precisely what the guiding principal is to the layout of the sendmail docs I cannot say, not being Martian.
Quick: what are three venerable and popular unix programs come with copious, but unusable documentation? Feel free to write in your answers, but here are mine: Sendmail, BIND and Emacs.
Anyways, back to the break-in.
A careful look at the login history log reproduced above will show that the trespasser logged in as 'jjohn,' which is my preferred account name. Typically software exploits that produce 'root' shells don't show up in wtmp logs. Also, why use the 'jjohn' account? I believe the answer is that name would have been observed frequently by anyone sniffing the traffic leaving my cable modem. Recall that whenever I sent or received email, I "broadcast" my credentials (yes, I tend to use the same username and password for less-than-critical systems)
So if someone could sniff the traffic leaving my modem, they could easily comprimise my system (which forwarded the publically accessible port 22 to my internal linux box). But how? I ran with this same mail configuration for four years on my DSL. How could I get hacked within four days on switching to the cable modem? The answer is that my network segment was probably already comprised when I joined it. I know that cable modem networks are attactive targets for intruders. Not only are machines connected via cable modems on a fast pipe, but also their owners are typically naive (guilty), slothful (guilty), and stupid (no comment). More seductively, the shared network segment of cable modems allows one comprimised machine to spy on the traffic of its neighboring PCs.
To put this in less technical terms, I moved from the relative monogamy of DSL service to the Paris Hilton-like fealty of cable modem life. And I made this switch without first buying technological rubbers. Sigh.
The resulting veneral infection manifested as an IRC bot, called emech. Here are some of the configuration files, which I publish here in the fond hope that someone will recognize this fucktard and slap him (and yes, I believe it's a him) repeated about the head and neck.
# file: mech.set ENTITY l0ol LINKPASS 12345 LINKPORT 1235 AUTOLINK NICK `johy LOGIN john IRCNAME john USERFILE `1.users MODES +i-ws CMDCHAR - TOG SPY 1 TOG CC 1 TOG CLOAK 1 SET OPMODES 6 SET BANMODES 6 CHANNEL #gheorg SERVER stockholm.se.eu.undernet.org 6667 SERVER flanders.be.eu.undernet.org 6667 SERVER eu.undernet.org 6667
# mech.session
entity l0ol
linkpass 12345
linkport 1235
autolink
nick lotus}{
login john
ircname john
modes i
userfile `1.users
set BANMODES 6
set OPMODES 6
tog SPY 1
channel #Erlangen
channel #robby
channel #caras-severin
channel #carasseverin
channel #skorpion
server stockholm.se.eu.undernet.org 6667
server flanders.be.eu.undernet.org 6667
server eu.undernet.org 6667
# `1.users <-- note the 133t backtick handle Skorpion mask *!*@Eifel65.users.undernet.org prot 4 aop channel * access 100
# `johy.seen <-- whoa! There it is again! This guy rulz!
^Bogdan^ ^Bogdan^!~bogdan@bobo007.users.undernet.org none 1075726402 2 Quit: I'm just an illusion!!!
{|___E|} {|___E|}!~E@bittet.um.eine.tafel.e-schokola.de none 1075711128 2 *.net *.split
ZeeN` ZeeN`!Elite@what.a-wanna.be none 1075723606 3 ZeeN
vio45 vio45!~vio_45@hamerock.nextra.ro none 1075714606 1 #skorpion
Templeton Templeton!~luci@203.149.49.90 none 1075698114 2 *.net *.split
Torero Torero!~lolex@dialin-145-254-118-150.arcor-ip.net none 1075719060 2 Quit: 9,1mIRC Script RO v2.0 8,1download---> 11,12www.drunkenkillerz.go.ro
Tr|v|a` Tr|v|a`!~Tr|v|a@excitatu.users.undernet.org none 1075710920 2 Read error: Connection reset by peer
TCM TCM!~atractiv@Atractiv.users.undernet.org none 1075723654 2 Quit
TACO^ TACO^!~Statia8@81.18.72.10 none 1075724628 1 #caras-severin
Skorpion Skorpion!~Eifel65@server.tc.ut.ee none 1075710242 2 Registered
S-R-I S-R-I!iulius@SR1.Biz none 1075708216 3 SRI
Stylee Stylee!~Style@Cornelia.users.undernet.org none 1075705120 2 Quit
SRI SRI!iulius@SR1.Biz none 1075708340 2 Read error: EOF from client
Statia8 Statia8!~Statia8@81.18.72.10 none 1075716631 1 #caras-severin
sharpe`` sharpe``!~sharpe@MrSharpe.users.undernet.org none 1075718615 2 Quit: pa
Statia5 Statia5!~Statia5@81.18.72.7 none 1075724411 3 My}{ALL`
Statia1` Statia1`!~Statia1@81.18.72.1 none 1075720455 3 Ady15
Statia4 Statia4!~Statia4@81.18.72.4 none 1075721178 3 groparuu
Soapte Soapte!~Lam3rz.Ro@ZmeuFairfax.users.undernet.org none 1075721898 1 #caras-severin
Statia4` Statia4`!~Statia4@81.18.72.4 none 1075722317 3 enk
SirQ SirQ!~maciucara@82.77.71.151 none 1075723137 2 Quit: Be Right Back !!!
samira17^ samira17^!samira@PowerPopGirl.users.undernet.org none 1075724222 2 Read error: Connection reset by peer
scumpykul scumpykul!~scumpykul@godzillla.users.undernet.org none 1075724481 3 _ImhoteP_
sandyGIRL sandyGIRL!~sandy@SANDYgirl.users.undernet.org none 1075727326 2 Quit: byebye
redshadow redshadow!~redshadow@195.223.166.68 none 1075724279 2 Registered
Ramyrez Ramyrez!~ramona@35.8.29.105 none 1075698114 2 *.net *.split
R2-D2 R2-D2!Connex@R2D2.users.undernet.org none 1075711055 2 Read error: EOF from client
Raf`y Raf`y!~Rep@195.119.130.218 none 1075696177 2 Read error: Connection reset by peer
Robby^}{^ Robby^}{^!~Robby@80.96.144.104 none 1075701818 2 Registered
Robby`}{` Robby`}{`!~Robby@80.96.144.104 none 1075720179 2 Read error: EOF from client
pisuka pisuka!~pisuka@80.250.7.18 none 1075719930 2 Read error: Connection reset by peer
pentru`98 pentru`98!sfdfdsa@82.77.71.161 none 1075711137 1 #caras-severin
Protect_1 Protect_1!~atractiv@Secrete.users.undernet.org none 1075714898 2 Read error: Connection reset by peer
portos portos!TeBlestem@portos2004.users.undernet.org none 1075715428 1 #skorpion
papalapte papalapte!666@81.89.13.119 none 1075719315 3 disel
parazituu parazituu!~Statia4@81.18.72.4 none 1075727252 2 Quit
ozu ozu!baltoc@195.190.159.197 none 1075700128 2 Quit: 2I'm using 7-=mIRC-GOLD=- 2By Kaptein (http://welcome.to/mirc-gold
Ozzy_Fly Ozzy_Fly!cd@81.180.32.5 none 1075715097 1 #caras-severin
Nicola`` Nicola``!romania@ich.bin.co0l.de none 1075722781 1 #carasseverin
november` november`!~dfsd@80.97.160.24 none 1075713738 2 Signed off
manu_BB manu_BB!identNou@195.47.194.200 none 1075722369 1 #skorpion
mirciu mirciu!~Statia5@81.18.72.7 none 1075722359 2 Read error: EOF from client
MADD MADD!DASuser@213.233.111.61 X!cservice@undernet.org 1075721880 4 (Atractiv) out
My}{ALL` My}{ALL`!~Statia5@Atractiv.users.undernet.org none 1075727018 2 Quit: brb
LongDonG LongDonG!~stxx@st3.3f.net4you.ro none 1075696122 2 Quit
L|tleBlak L|tleBlak!~sile@flood.attaq.net none 1075721628 2 Registered
ls361 ls361!~skorpion1@Skorpion1.users.undernet.org none 1075698114 2 *.net *.split
LitleBig LitleBig!~aaaaaa@80.97.160.150 none 1075715472 2 Quit: auzisi grasule te ia draq daca numi zici maine ce joci :) o zi buna
Laddy_Z Laddy_Z!~Dihania@81.18.72.13 none 1075722121 2 Quit
Lock^^6 Lock^^6!LittleBoy@courier.users.undernet.org none 1075723557 1 #carasseverin
Johnnyd Johnnyd!~JohnnyD@cblmdm63-166-33-8.buckeye-express.com none 1075724374 2 Read error: Connection reset by peer
J0HNNY377 J0HNNY377!~JohnnyD@c66.169.193.115.ts46v-19.ftwrth.tx.charter.com none 1075698114 3 J0HNNYD
jejejejej jejejejej!~io@80.96.144.104 none 1075699413 2 Read error: EOF from client
IIIusion IIIusion!romania@port3.ts04.dialup.hannover.mops.net none 1075706062 2 Read error: Operation timed out
Inocenta` Inocenta`!ef@80.97.160.226 none 1075722480 2 Quit: Uf ce mam saturat de tot de ce mai sufera omu`? de ce sa mai crezi in dragoste daca ea doare?.....mdea` cred ca a inceput sa imi placa durerea si sa o vreau in viata mea ..Doamne unde esti? te caut dar nu te gasesc in lumea asta rea unde esti tu cel care ma va iubi pana la capat si promit ca iti voi da totul!
IRemenber IRemenber!~Lam3rz.Ro@ZmeuFairfax.users.undernet.org A_Dany!fg3ea@danebunu.users.undernet.org 1075723717 4 Fain vorbeshti bha....ia cauta-ne pe status..!!
Hairstyle Hairstyle!romania@Hairstyle.users.undernet.org none 1075711128 2 *.net *.split
Hyper`Pen Hyper`Pen!~Karakter@62.231.108.142 none 1075707849 2 Ping timeout
harivena harivena!~Statia5@81.18.72.7 none 1075715091 1 #caras-severin
gruia gruia!~logojan_g@217.73.175.226 none 1075726343 1 #skorpion
freak_ freak_!paulica@shell10.powershells.de none 1075710890 3 freak
freak freak!paulica@shell10.powershells.de none 1075711080 2 Killed (*.undernet.org (older nick overruled))
FlasHack FlasHack!south@FlashHack.users.undernet.org none 1075726196 2 Quit: eu numai pot deja ma duc la baie
FellOwes FellOwes!~bigvasa@195.78.42.123 none 1075724805 2 Quit
enk enk!~Statia4@81.18.72.4 none 1075725474 3 parazituu
DARxIDE DARxIDE!~dxd@darxid3.users.undernet.org none 1075708201 3 DARKSIDE-
DesKjet DesKjet!~Karakter@62.231.108.142 none 1075707867 2 Ping timeout
Delfinu` Delfinu`!~Delfinu@81.18.72.11 none 1075714898 2 Read error: Connection reset by peer
dadycool dadycool!~dady@196.41.3.246 none 1075717476 2 Registered
DADDYzZz_ DADDYzZz_!~aiurea@www.adler-law.com none 1075712081 1 #carasseverin
Delfinu`` Delfinu``!~Delfinu@81.18.72.11 none 1075718016 2 Signed off
Diana}{^ Diana}{^!south@DianaLove.users.undernet.org none 1075718835 1 #caras-severin
disel disel!666@81.89.13.119 none 1075719928 2 Read error: Connection reset by peer
dream1703 dream1703!seanjohn4l@tntwc01-3-236.idx.com.au none 1075723646 1 #carasseverin
DaYaNa__s DaYaNa__s!Standasads@com3.combatnet.tveurosat.ro none 1075724470 2 Signed off
Dihania Dihania!~Statia8@81.18.72.10 none 1075724622 3 TACO^
doggyy doggyy!~ovidiurot@80.96.72.14 none 1075726107 2 Read error: EOF from client
Delu^ Delu^!levente_@217.156.38.87 none 1075726171 2 Read error: EOF from client
candito candito!candito@208.251.137.98 none 1075700644 1 #carasseverin
ChimpMama ChimpMama!~chimp@cm108.gamma54.maxonline.com.sg none 1075711404 1 #carasseverin
Contele_D Contele_D!~user@81.196.122.34 none 1075715290 2 Quit
cychi cychi!cyber14@80.97.160.239 none 1075717953 1 #caras-severin
c0sti c0sti!~cagulici@ToSexY4u.users.undernet.org A_Dany!fg3ea@danebunu.users.undernet.org 1075724524 4 Reclamele pe status...OUT
Babi Babi!romania@tserv04.mops.net none 1075723035 2 EOF from client
BrUne|a BrUne|a!jackson@will.dein.e-schokola.de none 1075711128 2 *.net *.split
BAD-B0Y BAD-B0Y!admin@carolrudi.users.undernet.org none 1075707603 2 Quit
Best-girl Best-girl!media@81.18.72.12 none 1075716359 2 Signed off
bobo1 bobo1!S6@194.105.21.72.ec.deva.rdsnet.ro none 1075723111 2 Read error: EOF from client
besinoasa besinoasa!~5THG5T@80.97.160.24 X!cservice@undernet.org 1075724204 4 (FlashHack) out fa
bla_r bla_r!6rtyr@81.196.246.158 none 1075724938 2 Ping timeout
BigVasa BigVasa!~bigvasa@sarutok.users.undernet.org none 1075726786 2 Read error: Operation timed out
A-T-B A-T-B!romania@loves.the.quake-net.org none 1075711128 2 *.net *.split
abgar abgar!candgj@home-326185.galati.astral.ro none 1075710101 1 #Erlangen
Ady15 Ady15!~Statia1@81.18.72.1 none 1075724832 1 #caras-severin
Adi}{ Adi}{!~st34@aladin18.users.undernet.org none 1075723337 1 #caras-severin
ALBERT}{ ALBERT}{!cyber14@80.97.160.239 none 1075723381 1 #caras-severin
aghiutza_ aghiutza_!aghiutza_@82.208.146.120 none 1075725905 2 Quit
`johy `johy!~john@h00045a218cc4.ne.client2.attbi.com none 1075694253 3 lotus}{
_Raphaell _Raphaell!~DjScorpio@pose.adultplanet.ca none 1075698139 3 Raphaello
___Yo___ ___Yo___!Yo-klq@NBA17.users.undernet.org none 1075723091 2 Quit: bye
_ImhoteP_ _ImhoteP_!~scumpykul@godzillla.users.undernet.org none 1075724834 2 Quit: byeeee
On a positive note, here is the perl script I wrote to set up local SSH2 tunnels for POP3 or any other service that would otherwise pass credentials in plain text. Note that you will need to set up a SSH2 private/public key for your local system. This is well documented elsewhere.
#!/usr/bin/perl -- -*-cperl-*-
#
# create ssh tunnels to send and receive mail securely
# loop and sleep to ensure connections
use strict;
use POSIX qw(setsid);
use File::Basename;
our $VERBOSE = ($ARGV[0] eq '-v');
our $DEBUG = 0;
our $pidfile= '/tmp/ssh-tunnels.pid';
our $piddir = '/tmp/ssh-tunnels';
our $ssh = '/usr/local/bin/ssh';
$SIG{CHLD} = sub {wait()};
# port => realhost
my %forwards = (2110 => 'aliensaliensaliens.com:110',
2111 => 'name.withheld.com:110',
2112 => 'shameful.network.provider.com:110',
9080 => 'use.perl.org:80',
);
my $action = pop @ARGV;
my %dispatch = ( stop => \&stop,
status => \&status,
);
if (defined $dispatch{$action}) {
$dispatch{$action}->();
exit;
}
# otherwise, attempt to start
if (getstatus() eq 'running') {
warn "Can't start. Already running\n";
exit 1;
}
unless ($DEBUG) {
open STDERR, ">>/tmp/ssh-tunnels.log";
}
warn "started: " . localtime() . "\n";
daemonize() unless $DEBUG;
while (1) {
while (my ($lport, $host) = each %forwards) {
if (-e "$piddir/$lport.pid") {
my $pid = getpid($lport);
if (-e "/proc/$pid/status") {
next;
}
}
start_tunnel($lport, $host);
}
sleep 120;
}
#-------------------
# subs
#-------------------
sub getpid {
my ($port) = @_;
my $file = "$piddir/$port.pid";
return unless -r $file;
open my $in, $file or die "can't open file: $file\n";
my $pid = <$in>;
close $in;
($pid) = ($pid =~ /^(\d+)/);
return $pid;
}
sub setpid {
my ($port, $pid) = @_;
my $file = "$piddir/$port.pid";
mkdir $piddir unless -d $piddir;
open my $out, ">$file" or die "Can't write '$file': $!";
print $out $pid;
close $out;
}
sub getserverpid {
return unless -r $pidfile;
open my $in, $pidfile or die "can't open pidfile: $pidfile\n";
my $pid = <$in>;
close $in;
($pid) = ($pid =~ /^(\d+)/);
return $pid;
}
sub setserverpid {
my ($pid) = @_;
open my $out, ">$pidfile" or die "Can't write '$pidfile': $!";
print $out $pid;
close $out;
}
sub start_tunnel {
my ($lport, $host) = @_;
my $cmd = "$ssh -2 -N -L$lport:$host jjohn\@localhost";
if ($VERBOSE) {
warn ("starting: '$cmd'\n");
}
unless (fork()) {
setpid($lport, $$);
exec $cmd;
}
sleep 2; # parent sleeps, kid never sees this line
}
sub daemonize {
close STDIN;
close STDOUT;
if (fork()) {
warn("$$: exiting\n");
exit;
}
setsid();
open STDIN, ">/dev/null";
open STDOUT, "/dev/null";
setserverpid($$);
}
sub stop {
my $pid = getserverpid();
mykill($pid);
print "Killed server: $pid\n";
# stop the tunnels too
while (my ($lport, $host) = each %forwards) {
$pid = getpid($lport);
mykill($pid);
print "Killed tunnel: $pid\n";
}
}
sub mykill {
my ($pid) = @_ or return;
while (-e "/proc/$pid") {
kill -9, $pid;
sleep 1;
}
}
sub status {
my $pid = getserverpid();
my $status = getstatus($pid);
my $name = basename($0);
print "$name is $status\n";
}
# returns a one-word status for service
# 'running', 'stopped'
sub getstatus {
my ($pid) = @_;
# should grep through status to ensure
# this is the right process
if (-e "/proc/$pid/status") {
return "running";
}
return "stopped";
}
Please note that although I forward to use.perl.com:80, I'm so far unable to use the SOAP interface to the journals through this tunnel. If anyone was some pointers on doing this, spill the beans in the comment section. One last bit, I've turned off all port forwarding on my firewall.