[110975] in North American Network Operators' Group
Re: isprime DOS in progress
daemon@ATHENA.MIT.EDU (David Andersen)
Sun Jan 25 12:23:40 2009
From: David Andersen <dga@cs.cmu.edu>
To: Phil Rosenthal <pr@isprime.com>
In-Reply-To: <90932AF0-4D17-4D1D-B8BD-ABC2DEF8A27E@isprime.com>
Date: Sun, 25 Jan 2009 12:23:27 -0500
Cc: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-69--953546751
Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
I'm not sure you're entirely out of the water yet:
17:13:45.680944 76.9.16.171.53868 > XXXXXXXX.53: 58451+ NS? . (17)
17:13:45.681251 XXXXXXXX.53 > 76.9.16.171.53868: 58451 Refused- 0/0/0 =20=
(17)
CIDR: 76.9.0.0/19
NetName: ISPRIME-ARIN-3
In addition to the one that Brian Keefer mentioned a few days ago =20
(206.71.158.30).
But on that subject, I figured I'd toss in a (sad) anecdote about =20
security and upgrades. I'd upgraded this nameserver to bind-9 some =20
time ago, during a bit of a security panic. And in the process, I =20
screwed it up - I'd updated the machine itself, but had failed to =20
propagate the changes to the master that sends updates to all of the =20
servers. The obvious thing happened: after a while, this nameserver =20=
pulled its updates from the master, and downgraded to bind-8 again, =20
which we didn't notice until I saw it spitting full cached NS =20
responses to isprime hosts. Human error strikes again. Apologies for =20=
letting my host be an amplifier.
-Dave
On Jan 23, 2009, at 1:11 PM, Phil Rosenthal wrote:
> Just a friendly notice, the attack against =20
> 66.230.128.15/66.230.160.1 seems to have stopped for now.
>
> -Phil
> On Jan 22, 2009, at 6:01 AM, Bj=F8rn Mork wrote:
>
>> Graeme Fowler <graeme@graemef.net> writes:
>>
>>> I've been seeing a lot of noise from the latter two addresses after
>>> switching on query logging (and finishing an application of Team =20
>>> Cymru's
>>> excellent template) so I decided to DROP traffic from the addresses
>>> (with source port !=3D 53) at the hosts in question.
>>>
>>> Well, blow me down if they didn't completely stop talking to me. =20
>>> Four
>>> dropped packets each, and they've gone away.
>>>
>>> Something smells "not quite right" here - if the traffic is =20
>>> spoofed, and
>>> my "Refused" responses have been flying right back to the *real* IP
>>> addresses, how are the spoofing hosts to know that I'm dropping the
>>> traffic?
>>
>> Did you filter *only* 66.230.128.15/66.230.160.1, or are you dropping
>> traffic from other sources too? Looks like some of the other source
>> addresses are controlled by the DOSers. Possibly used to detect =20
>> filters?
>>
>> These clients may look similar to the DOS attack, but there are =20
>> subtle
>> differences:
>>
>> Jan 18 05:08:33 canardo named[32046]: client 211.72.249.201#29656: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 05:08:33 canardo named[32046]: client 211.72.249.201#29656: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 05:08:34 canardo named[32046]: client 211.72.249.201#29656: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 05:47:00 canardo named[32046]: client 211.72.249.201#29662: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 05:47:01 canardo named[32046]: client 211.72.249.201#29662: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 05:47:01 canardo named[32046]: client 211.72.249.201#29662: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 06:25:22 canardo named[32046]: client 211.72.249.201#29664: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 06:25:22 canardo named[32046]: client 211.72.249.201#29664: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 06:25:23 canardo named[32046]: client 211.72.249.201#29664: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:03:41 canardo named[32046]: client 211.72.249.201#29667: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:03:41 canardo named[32046]: client 211.72.249.201#29667: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:03:42 canardo named[32046]: client 211.72.249.201#29667: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:42:08 canardo named[32046]: client 211.72.249.201#29670: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:42:09 canardo named[32046]: client 211.72.249.201#29670: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 07:42:09 canardo named[32046]: client 211.72.249.201#29670: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:20:29 canardo named[32046]: client 211.72.249.201#29673: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:20:29 canardo named[32046]: client 211.72.249.201#29673: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:20:30 canardo named[32046]: client 211.72.249.201#29673: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:58:50 canardo named[32046]: client 211.72.249.201#29678: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:58:51 canardo named[32046]: client 211.72.249.201#29678: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 08:58:51 canardo named[32046]: client 211.72.249.201#29678: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 09:37:12 canardo named[32046]: client 211.72.249.201#29679: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 09:37:12 canardo named[32046]: client 211.72.249.201#29679: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 18 09:37:13 canardo named[32046]: client 211.72.249.201#29679: =20=
>> view external: query (cache) './NS/IN' denied
>>
>> Jan 20 07:02:51 canardo named[32046]: client 213.61.92.192#46716: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 07:02:51 canardo named[32046]: client 213.61.92.192#46716: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 07:02:51 canardo named[32046]: client 213.61.92.192#46716: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 07:41:21 canardo named[32046]: client 213.61.92.192#46752: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 07:41:21 canardo named[32046]: client 213.61.92.192#46752: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 07:41:21 canardo named[32046]: client 213.61.92.192#46752: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:19:46 canardo named[32046]: client 213.61.92.192#46785: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:19:46 canardo named[32046]: client 213.61.92.192#46785: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:19:46 canardo named[32046]: client 213.61.92.192#46785: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:58:12 canardo named[32046]: client 213.61.92.192#46808: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:58:12 canardo named[32046]: client 213.61.92.192#46808: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 08:58:12 canardo named[32046]: client 213.61.92.192#46808: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 09:36:34 canardo named[32046]: client 213.61.92.192#46833: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 09:36:34 canardo named[32046]: client 213.61.92.192#46833: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 09:36:34 canardo named[32046]: client 213.61.92.192#46833: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 10:14:58 canardo named[32046]: client 213.61.92.192#46858: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 10:14:58 canardo named[32046]: client 213.61.92.192#46858: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 20 10:14:58 canardo named[32046]: client 213.61.92.192#46858: =20
>> view external: query (cache) './NS/IN' denied
>>
>> Jan 22 06:27:28 canardo named[32046]: client 66.238.93.161#34373: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 06:27:28 canardo named[32046]: client 66.238.93.161#34373: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 06:27:28 canardo named[32046]: client 66.238.93.161#34373: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:05:55 canardo named[32046]: client 66.238.93.161#34420: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:05:55 canardo named[32046]: client 66.238.93.161#34420: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:05:55 canardo named[32046]: client 66.238.93.161#34420: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:44:20 canardo named[32046]: client 66.238.93.161#34473: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:44:20 canardo named[32046]: client 66.238.93.161#34473: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 07:44:21 canardo named[32046]: client 66.238.93.161#34473: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 08:22:38 canardo named[32046]: client 66.238.93.161#34503: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 08:22:38 canardo named[32046]: client 66.238.93.161#34503: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 08:22:38 canardo named[32046]: client 66.238.93.161#34503: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:00:56 canardo named[32046]: client 66.238.93.161#34540: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:00:56 canardo named[32046]: client 66.238.93.161#34540: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:00:56 canardo named[32046]: client 66.238.93.161#34540: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:39:20 canardo named[32046]: client 66.238.93.161#34574: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:39:21 canardo named[32046]: client 66.238.93.161#34574: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 22 09:39:21 canardo named[32046]: client 66.238.93.161#34574: =20
>> view external: query (cache) './NS/IN' denied
>>
>>
>> Notice the pattern:
>> 3 probes every 38 minutes
>> Each probe from the same source port
>> Source port increases slowly and steadily
>>
>> This looks like some application actually waiting for a response. =20=
>> The
>> slow source port change is probably an indication that this client =20=
>> only
>> tests a small number of DNS servers. I guess that this client is =20
>> either
>> one of the many bots used to send the spoofed requests, or maybe a =20=
>> bot
>> not allowed to spoof its source and therefore used for other
>> purposes. In any case, I assume that other DNS servers may see such
>> control sessions coming from other addresses.
>>
>> These 3 clients started probing my DNS server almost simultaneously =20=
>> on January 8th:
>>
>>
>> Jan 8 19:33:52 canardo named[26496]: client 213.61.92.192#31195: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:33:52 canardo named[26496]: client 213.61.92.192#31195: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:33:52 canardo named[26496]: client 213.61.92.192#31195: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:36:29 canardo named[26496]: client 66.238.93.161#11299: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:36:29 canardo named[26496]: client 66.238.93.161#11299: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:36:30 canardo named[26496]: client 66.238.93.161#11299: =20
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:37:47 canardo named[26496]: client 211.72.249.201#29112: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:37:47 canardo named[26496]: client 211.72.249.201#29112: =20=
>> view external: query (cache) './NS/IN' denied
>> Jan 8 19:37:47 canardo named[26496]: client 211.72.249.201#29112: =20=
>> view external: query (cache) './NS/IN' denied
>>
>> Maybe preparing for the attack on ISPrime? I didn't start receiving
>> spoofed requests from 66.230.128.15/66.230.160.1 before January 20th
>>
>>
>> I just tried filtering the probing addresses. This made the probing
>> stop immediately after dropping a set of 3 probes. But the spoofed
>> requests continuted at the same rate as before, so this does not =20
>> support
>> my theory.
>>
>> However, I believe it would be too much of a coincidence if there =20
>> isn't
>> some connection between the probing and the DOS attack. It would be
>> interesting to hear if others see similar probing.
>>
>>
>>
>> Bj=F8rn
>>
>
>
>
--Apple-Mail-69--953546751
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)
iEYEARECAAYFAkl8oA8ACgkQLZyskygNjpyVdACfZaMZ/wU/+zDx2EQdoLEeinuX
tUoAnjUMXkQSltHMaqdwGgrYvNZgqXTw
=VVwN
-----END PGP SIGNATURE-----
--Apple-Mail-69--953546751--
