But do we really need 25G — or just want it?
The need for speed has always been one of mankind's greatest loves. From the start of time, the fastest hunters caught the best prey, the fastest developing nations dominated world trade and markets. The need for speed pushed mankind to achieve greater and greater things. Speed records have been sought and broken by using the latest technology to strive for the seemingly unachievable.
People will always look at a record and wonder how it can be broken, but there is another aspect. Record breaking is driven by a want to be the best, the biggest, the first, the strongest or the fastest for our own ego — but do we always need it?
Well, maybe not. The land speed record is currently 763.035 mph - but attaching a rocket to every car is incredibly inefficient, dangerous and pointless. You need to look at the costs - as well as the benefits - in order to make a sensible decision around whether you really need to move from 10G to 25G.
Why on earth would we need more than 10G?
When I look around, I see people talking to their phone, and saying things like, “OK Google — where’s the closest coffee shop?”. We expect everything to be available all the time, to everyone, at the same time.
If you’re reading this blog I’m sure you already understand. Many trends over the last decade have led to an exponential increase in both the data we store, and the speed we need to access it — even the simplest applications you are responsible for, often need an enormous ability to scale.
The initial 10G SFP+ transition was fairly painless
Network cards have evolved rapidly in recent years, moving quickly from 1Gbps to 10Gbps first with switch uplinks then servers. The move from 1Gbps (SFP) to 10Gbps (SFP+) for switch uplinks was fairly seamless from a space and power perspective. The interfaces are the same size and can therefore provide the same density per rack, so you could just swap switches out one at a time.
Then with desktops - we came to love 10G over copper RJ-45
Slightly more latency and power consumption, but so easy to roll out and understand. Again the same physical form factor as SPF but completely backwards-compatible with expensive physical wiring in offices. In fact, we loved it so much, most servers moved back from fiber to copper instead.
Meanwhile, the data center backplane went with QSFP
When we started using 40G in the data center I don’t think we ever dreamed we would need more than 10G to application servers. We didn’t think that it's bigger form factor, different power requirements, cables and huge price would ever be an issue. And it keeps getting better with speeds of up to 400G now.
Surprisingly quickly, we found applications that did need 40G to the server
I’ve had a lot of experience with 40G deployed in environments such as Object Storage networks. And that's just for small-to-medium sized deployments. Larger deployments need 100G, or preferably multiple clusters behind GSLB.
But QSFP application clusters still give us two annoying problems...
Firstly, QSFP switches are expensive, with lower port density and higher power requirements. And secondly how do you link them to your existing infrastructure?
I’ve seen load balancers with 8 x 10G ports bonded together — cross-linked and up-linked to a pair of 40G switches in order to take the load for an object storage cluster. How nuts is that? At Loadbalancer.org we were quick to help our partners with 40G and 100G connectivity, depending on the size of the deployment. So you can put them in the same switch as your application servers.
In fact with our hardware flexibility promise you can change your cards on our hardware appliances whenever you want — we've talked about this in detail before.
Luckily, it looks like 25G’s coming to the rescue
The 25Gbps (SFP28) in the same size format, provides a clear upgrade path without the need for additional space, as it's the same (form factor) as the 1G and 10G forerunners. With fully-backward compatibility with the 1G SFP and 10G SFP+ modules, this means new SFP28 units can be purchased today, ready for future network upgrades, while providing backward compatibility with older (current) network switches and infrastructure.
As the number of 25Gbps ports shipped for data centers in 2021 is expected to exceed the number of 10Gbps ports, the upgrade path is becoming clear — the cost of moving to 25G is no longer inhibitive.
What are vendors in object storage doing?
I’ve been talking to a lot of our Object Storage partners regarding future connectivity requirements, and most are now defaulting to 25G where they have the choice; some don't even offer a 40G option any more. Why bother when you can get 4 x 25G per node more cost-effectively?
So, just how flexible and compatible is a 25G card?
I’ve recently tested the Mellanox 25G (MCX512A-ACAT) card used in our load balancer, to see just how flexible it can be. I was concerned about backwards compatibility for our customers who have been using Intel 10G cards and optics.
So I had a rummage through our new test lab, and found a variety of 10G SFP+ and 25G SFP28 modules including Mellanox, Cisco and Intel. I tested with both a new 25G switch and an old 10G switch (optical and copper) to double-check compatibility... and the short answer is to question above is 'very compatible'! Every connector I tried in the Mellanox 25G card worked - first time.
This is a big improvement — I tried non-branded optics in 10G cards with little success.
So which optics should I use for a 25G switch?
The compatibility of brands has massively improved, and I recently had a chat with the very helpful technical support team at FS.com. They confirmed the software on optics is identical, they just quickly flash the optic with the manufacturer's unique tag.
So that makes it nice and easy for me to recommend three simple choices for customers using our load balancer with a 25G Mellanox card:
| For 25G switches | Backwards compatible with 10G switches - RJ45 |
Backwards compatible with 10G switches - Fiber |
|---|---|---|
 |
 |
 |
For full compatibility, please refer to Melanox's web site, or alternatively FS.com.
When you buy hardware, you need to consider both the past and the future
History teaches us that network speed requirements grow faster than you think. So, given the fact that 25G cards are almost as cheap as 10G cards, and they are also completely backwards compatible:
- Why do you want to use 10G?
- When you clearly need to use 25G!
If you think I’m talking rubbish, please leave your comments below — we love a good argument.
P.S. Loadbalancer.org is happy to help our customers change existing hardware to 10G / 25G / 40G / 50G or 100G options at low cost. In fact, with our hardware flexibility promise you can change your cards on our hardware appliances whenever you want - with no charge from us.