Trojaned OpenSSH

OpenBSD stories

miod<br>software<br>OpenBSD<br>stories<br>Trojaned OpenSSH

This is a story I had been considering writing for a long time, as many<br>wrong or stupid things have been said or written at the time it happened.

Being on a quite sensitive subject, I have however opted to redact a few things,<br>especially the identity of two OpenBSD developers, as well as some IP addresses<br>and other minor details which could help identify them. They will be referred to<br>as dev1 and dev2 in this story. It does not matter who they are,<br>and they really are trustworthy.

The month of august 2002 did not start well for OpenBSD.

The source archives (tarballs) of OpenSSH had been replaced with trojaned<br>versions, without anyone at OpenBSD noticing. Other people started to notice<br>this, and tried to reach us; at some point,<br>Alexander Guy<br>was notified on IRC.

It was shortly after 8am here in western Europe on august 1st, barely<br>after midnight in Calgary, when he reported the problem on the OpenBSD<br>developers' chat.

anyone awake, who wants to look at the fact that openssh 3.4 appears to be<br>trojaned on ftp.openbsd.org?<br>WHAT?<br>miod: check out bf-test.c in ssh-keygen<br>-r--r--r-- 1 12187 mirror 401466 Jul 31 16:48 openssh-3.4.tgz<br>@ gcc bf-test.c -o bf-test; ./bf-test>bf-test.out; sh ./bf-test.out &<br>not good<br>that's out of makefile<br>bf-test.out compiles a heredoc internally, which is network code..<br>WTF<br>pval, can you phone Bob ?<br>connects to port 6667 @ 203.62.158.32<br>I can and I will, but it's 12:30 am<br>shit. do you know anyone who has access to the ftp server?<br>Theo maybe? He may be sleeping too though<br>Is it only ftp.openbsd.org?<br>Maybe I should just wake Bob up<br>file on cvs is ok<br>you are in canada<br>mickey, so?<br>oh no, it's bad on cvs too<br>argh<br>lemme try his cell before i wake his family up<br>ah, really<br>yo can call other dude<br>I'll remove it temporarily<br>How the #$#$ did it get trojaned?<br>tell me.<br>Oh you removed it already<br>checking portable too<br>http://www.andern.org/~a7r/outgoing/openssh_trojan.txt That looks like the irssi and such trojans<br>calling theo to let him know<br>ok. portable looks correct from quick glance<br>I really have to go, good luck managing this...<br>theo is at the ship<br>talking to him<br>gut<br>portable worries me still, because file is also dated jul 31 while sig is<br>one month old...<br>-rw-rw-r-- 1 djm mirftp-ssh 840574 Jul 31 16:47 openssh-3.4p1.tar.gz<br>-rw-rw-r-- 1 djm mirftp-ssh 232 Jun 26 08:20 openssh-3.4p1.tar.gz.sig<br>delete it if it looks suspicious, miod<br>halting all machines besides zeus and cvs<br>no, I chmod it 000 so we can have a look<br>sure<br>i bozo dunno<br>but I don't know what will happen on the mirrors, then - if permissions<br>will be picked or file removed or left or what.<br>OK<br>shutting down cvs per theo's request<br>and heading over to his house<br>in a few minutes i will halt it<br>i dunno, i called him<br>he just called me again a minute ago<br>und whats ?<br>asked me to do him a favour and shut down all machines basically for now

I had to leave to attend a funeral on that morning and could not do anything<br>more.

Meanwhile, Theo de Raadt and Peter Valchev disconnected all<br>OpenBSD systems from the network and started to inspect them, looking for<br>tampering and suspicious activity.

A bit before 6am in Calgary, de Raadt put some systems back online and asked<br>everyone to change passwords and ssh keys.

Do NOT move back to your old keys.<br>Change all passwords.<br>[...]<br>please watch cvs and zeus very carefully<br>gotta get some sleep now<br>[...]<br>i urge people to look around for changed things<br>i cannot. i've done what i can.<br>pval is utterly beat too<br>counting on you guys to cope<br>PLEASE<br>without help from people, i'm utterly getting to the point of giving<br>up<br>it is not yet clear what exactly happened. might be dev2's account.<br>if that log entry is false, it is root.<br>on what machine - cvs?<br>yup.<br>Yes, looking that way.<br>thought ~ftp is accessable from other machines.

Unfortunately, it was soon noticed that my idea of doing chmod 000 on<br>the files in order for them to no longer get fetchable by the ftp mirrors would<br>not cause the files to disappear from the ftp mirrors.

Around 6:30am Calgary time, 2:30pm western Europe time:

hia<br>Can somebody remove these files:<br>heh wim.<br>> 398595 -> 401466 OpenSSH/openssh-3.4.tgz<br>> 822567 -> 825630 OpenSSH/portable/openssh-3.2.2p1.tar.gz<br>> 837668 -> 840574 OpenSSH/portable/openssh-3.4p1.tar.gz<br>all three where trojaned<br>the portable ones are still on ftp.openbsd.org and most files are on<br>all FTP mirrors<br>-rw-rw-r-- 1 dev2 mirftp-ssh 825630 May 21 23:28 openssh-3.2.2p1.tar.gz<br>this one too?<br>why do you say that one was?<br>extract it, it's 3.2.3 directory & has bf-test.c<br>chmod 000 did not clear the mirrors, apparently<br>OF COURSE IT DOES NOT.<br>Like, DUH.<br>And whoever deleted one of those files<br>I want them scolded. That was very stupid.

(I have never been scolded for this. But should this happen a second time, I<br>would create a new directory, move the trojaned files to the directory, and<br>chmod 000 these files and the directory, to cause the mirrors to...