[184433] in North American Network Operators' Group
Mikrotik in the DFZ (Was Re: AW: AW: /27 the new /24)
daemon@ATHENA.MIT.EDU (William Waites)
Sat Oct 3 04:25:21 2015
X-Original-To: nanog@nanog.org
Date: Sat, 03 Oct 2015 09:23:49 +0100 (BST)
To: jj@anexia.at
From: William Waites <wwaites@tardis.ed.ac.uk>
In-Reply-To: <133ff0fed3394045add071349c00fcef@anx-i-dag02.anx.local>
Cc: nanog@nanog.org
Errors-To: nanog-bounces@nanog.org
----Security_Multipart(Sat_Oct__3_09_23_49_2015_231)--
Content-Type: Text/Plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable
On Fri, 2 Oct 2015 23:11:47 +0000, J=FCrgen Jaritsch <jj@anexia.at> sai=
d:
> Regarding the words "I have a small router which handles
> multiple full tables ...": push and pull a few full tables at
> the same time and you'll see what's happening: the CCRs are
> SLOW. And why? Because the software is not as good as it could
> be: the BGP daemon uses only one core of a 36(?) core CPU.
To expand on this, the problem is worse than being single-threaded. I
had one of these in the lab and fed it 2x full tables. Sure it wasn't
the fastest at accepting them but then I noticed that even in steady
state one of the CPUs was pegged. What was happening -- and this was
confirmed by Mikrotik -- was that it was recalculating the *entire*
FIB for each update. The general background noise of announce /
withdraw messages means it is doing this all the time. Any churn and
it would have a very hard time.
There are other serious bugs such as not doing recursive next hop
lookup for IPv6 (it does for IPv4). This makes them unuseable as BGP
routers even for partial tables with most non-trivial iBGP
topologies. All of which may be fixed one day in version 7 of their
operating system, which will inevitably have many bugs as any software
project .0 release will, so we'll have to wait for 7.x for it to be
reasonably safe to use.
That said, we use a lot of Mikrotik kit for our rural
networks. They're weird and quirky but you can't beat them on price,
port density and power consumption. With 16 ports and 36 cores surely =
they should be capable of pushing several Gbps of traffic with a few
full tables.
I wish it were possible today to run different software on their
larger boxes. If some like-minded small providers wanted to get
together with us to fund a FreeBSD port to the CCR routers that would
be great. Please contact me off-list if you are interested in this,
I'll coordinate.
As it is we don't let them anywhere near the DFZ, that's done with PCs
running FreeBSD and BIRD which can easily do the job but is still an
order of magnitude more expensive (and an order of magnitude less
expensive than what you need if you want 10s of Gbps).
-w
--
William Waites <wwaites@tardis.ed.ac.uk> | School of Informatics
http://tardis.ed.ac.uk/~wwaites/ | University of Edinburgh
https://hubs.net.uk/ | HUBS AS60241
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.
----Security_Multipart(Sat_Oct__3_09_23_49_2015_231)--
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
iQIcBAABCgAGBQJWD5CVAAoJEHhNnKzjwx5/SeUP/AvLpPLryVneUjBbuPb0jRfC
0HnMXZ5OQj4uvjQUxbMDe5JRIWWOABfw80EgbomN4/ahHMqnUJ2MR/oSWtD4Ihj6
XZ9aC7ogs2xq0h2SiFFYFpYac22u9DOR7Zz76q9gVMFz7Wbqsf9KsOl6yd6JZCG0
E+p+izhXidRziC1EEtP5qH1rJ34nNCbwrKubBTUNGbF5ngFE23i0Nq5xtFL5ruel
tdW/ah/3DWD4+C5O3yQ+Cth2pvRgu2u3E3O21IeO2e9sHqiBobL6WajszbtNK4ru
U8kkaqa6JNbkI2LCLfulqaouLfEiX1kzu2bMpbf2UUcVQFmkXrCUA+ryQkKkiCZd
oqN4UY4rS5V9lNJS22gbivPFViKV3yq2rIy5JHYOEa/pOPhEnqnExJB6Oqr7fPOc
Jg/AJlN1vFXH4vUDksAgtCgqZ/8ECubL73q/WB1/dc0ZhVqg8uA1yI9MUT8TD2iE
WB8s22JppPUgjYDyeWWl5MkrNSCBQkq6Ac8XAUUSmWPCcvzGGpS2tCeH3j5UNzSD
irTBUS05DyblaneOG8s6Ig9FzgCVqnrtov+FQJzsocN0LdFCsUJKSgMMy3w1M5n6
aDfpQZB9qAbPL2fHVBNB7whbjaPXId/YwtQoaFoShl8idAhNK4u28MOVzA4YeFWp
f37aPCxHikE9l7V3El7b
=pP/9
-----END PGP SIGNATURE-----
----Security_Multipart(Sat_Oct__3_09_23_49_2015_231)----
