The world cheapest 40GbE switch CRS326-24S+2Q+RM arrived!
MikroTik CRS326-24S+2Q+RM just arrived at home! It is the world cheapest 40GbE capable Ethernet switch out of legally distributed ones as of 9/4/2019, I guess. That's our MikroTik! But then, I got their product for the first time.
I suppose the guys come to see this page are aware of it, but CRS326-24S+2Q+RM has 10GbE capable 24 SFP+ ports and additionally two 40GbE QSFP+ ports, and its price is only 499 USD. What a crazy stuff it is! Though the price as worth is pretty expensive if I though with a calm mind, it is a unprecedented price-slashing considering its performance. Moreover, it is sold at 384 USD in EuroDK. I can't believe it.
That's why I couldn't help buying it. I had ordered it on 22nd August (JST), then it delivered on 26th quickly like 40GbE, but I couldn't receive it until the weekend. A total costs was approximate 50,000 yen including shipping, taxes and etc.
I tentatively test a 40GbE connection below environment in figure because of some reasons. It marked 28Gbps by iperf3 with MTU=1500.
| Server | Client | |
|---|---|---|
| OS | FreeBSD 11.2-RELEASE (x64) | Windows 10 Pro. (x64) |
| M/B | Supermicro X10DRi | MSI X370 SLI PLUS |
| CPU | Xeon E5-2648L v3 @1.8GHz | Ryzen 7 1700 @3.0GHz |
| RAM | Registered DDR4-2400 16GB×4 (2133MHz driven) | DDR4-2666 16GB×4 |
| NIC | ConnectX-3 Pro EN (PCIe 3.0x8) | |
It seems a switching chip is Marvell 98DX8332. This switch might also support 2.5GbE and 5GbE although MikroTik doesn't clearly say, because the chip support them and there are their check boxes on the switch's config screen…
The switch supports dual boot of RouterOS and SwOS, so it can do L3 processing to some extent. Having said that, the processes which aren't handled by the chip are processed by CPU, and what is worse, an internal bandwidth between them is only 1Gbps. Therefore, we had better use it as plain L2 switch, or should keep lightweight L3 usage such as connecting to WAN with simple packet filtering. Otherwise its valuable performance will be completely dead. Fortunately it has enough 10GbE ports, so I think fit to pass on the L3 processes to a dedicated router with a LAG. AT-x510-28GTX which is a L3 switch I got some time ago will have tears of joy for this application.
Following table shows rough power consumptions:
| Situation | Power Consumption |
|---|---|
| Idle（No SFP modules) | 13W |
| Idle（Two 40GBASE-SR4 modules link up） | 15～16W |
| Run iperf with 40GBASE-SR4 interfaces | 17W |
Cooling fans were pretty noisy in factory default RouterOS version. I updated it to version 6.45.4, now they get quieter than before. The OS aggressively controls their speed between zero to full blast depending on the internal temperature. I think these specs are very kindly for common home use. Thank you, MikroTik. Well, when the fans is running, they cause a certain level of noise. It seems themselves' operating noise is louder rather than wind noise. It is acceptable for me, but not for someone, I guess.