The 15 year story behind Loadbalancer.org....

The 15 year story behind Loadbalancer.org....

Comparisons Published on 7 mins Last updated

I was part of the generation that grew up without computers. We played Cowboys and Indians and dreamed of going into space, or flying planes. When I was about 11 my friends started getting Sinclair Spectrums and Comodore 64s. Unfortunately, my parents were middle class and bought me an Amstrad :-(.

My first piece of code was an animated fish tank (because I wasn't allowed a real one). I soon started dreaming of being a software developer and started learning machine code. At 13, I wrote my friends A Level project (like an AP but much harder), which involved an animation of a teacher asking questions in a quiz (with a real digitised voice) all in 32K of useable RAM. I recall that he only paid me with 6 cans of coke for 6 months work!

Luckily at 15 I discovered girls, and sold all my computer stuff to take them on dates....

So fast forward to 2002, I'm happily married, mortgaged to the hilt, working 24/7 fire-fighting at an e-commerce start up — and my wife Karen is heavily pregnant. What a great time to start a business!

So why did I create Loadbalancer.org?

I was a classic engineer scratching an itch. Frustrated with the cost and complexity of the CISCO LocalDirector — I decided to build a better solution, one that wouldn't keep me awake at night thinking what happens when the single CISCO bridge breaks?

"I hate bridge topology - and no, we will NEVER support it"

I've always loved open source and had lots of experience of using Linux for firewalls. So I knew it should be possible to build something with the help of the open source community. And yes, they helped me build something much better than the CISCO in a very short period of time.

I made a few stupid mistakes when I started - but thanks to the patience of Wensong Zhang, Simon Horman, Julian Anastasov and others I eventually figured out how to build a solid combination of LVS, Ldirectord & HA-Linux + a simple web interface with one-click maintenance and graphical reporting.

Lesson 1

Open Source is the most scaleable and secure way to build any great product.

Commitment
Loadbalancer.org is totally committed to contributing to the open source community both financially and with time.

For the first year, I custom built the hardware and did free on-site installs. This was a great way to see the problems that customers were dealing with. I set up a call center for the support line and one of the first calls was a customer who had a site down. A hard reset got him back online, unfortunately before I could diagnose the problem he had another freeze (site down). My on-site diagnosis showed that it was problem with a memory leak in an SSL health check, a work around was implemeneted followed by a fix. He was happy with the fix but understandably he lost faith and moved to F5 Networks.

Lesson 2

Customers are under a lot of stress if a site goes down. High-availability must be maintained at all costs. The first question developers ask before adding a feature or release is, "Is it secure?", followed by, "Will it break?".

Commitment
Loadbalancer.org staff will always give a damn about your situation and are fully committed to helping someone first — then justifying any actions and costs to management later.

We have always used quality branded solid state storage in our appliances but suffered from a bad batch that made our failure rate go from less than 1% to greater than 6%. The load balancer would keep running in memory but a re-boot would cause a failure.

When a new employee discovered the problem he remembered the company policy to fix customer problems first and deal with the consequences after. So he immediately shipped all of the affected customers replacement storage devices, so that they could replace them in a convenient maintenance window. He then spent the next 24 hours wondering whether he'd get sacked for costing his new employers a fair bit of cash. He was pleasantly surprised to receive a bonus for his actions instead.

Cancer definitely sucks....

Just before my son's 1st birthday, I was diagnosed with Leukaemia (ALL p+) and immediately hospitalised. Long story short but I was in hospital for almost a year. Eventually I received a life saving stem cell transplant from an amazing person, thank you Silke!

Transplant

"If you are still smiling on day 4 of a transplant — that's good news."

As you can imagine Karen had a lot of problems to deal with. And building servers in the garage was not her priority — so within 2 weeks she had arranged a deal to outsource production, distribution and support for Loadbalancer.org.

Lesson 3

Life is more important than work and outsourcing is a life saver.

Commitment
All staff at Loadbalancer.org are treated as part of the familly, we commit to always helping each other when life throws us lemons. Non-core services such as system builders and distribution are outsourced.

Garage

The garage where Karen built load balancers while I was in hospital...

A year later we both worked together to build the business and I did the first full re-write of the product (v5) — which is solid and still in use on some sites...

Stock
"Look the chair does fit in the study..."

IMG_0246
"If you don't get a proper office - then we get a divorce!"

When I fully recovered, we moved to a small village in the country - we got two dogs and re-designed our web site which helped sales take off. We then finally opened an office (the staff who worked from our house will remember how nice that was!).

Shortly after opening the office I finally added layer 7 load balancing support to v5 of the product. We use the awesome HAProxy — so a massive thank you to Willy Tarreau, you rock!

I also handed over development to someone who actually knew what they were doing... I am a terrible programmer so the product rapidly improved from that point onwards :-).

Number 2 of the 4, ever bigger offices we enjoyed at the Technopole Portsmouth, UK.

24/7 support is hard....

As a small company a lot of the staff had to go above and beyond in the first few years. Trying to solve a complicated problem when you are woken by a phone call at 2am is not easy.

As the support load increased we had 3 options:

  1. Outsource it and/or use resellers and distributors
  2. Work a full 24/7 shift pattern with the support staff
  3. Send fully trained support staff to the other side of the world

We decided that option 3 was the best long term solution, however when we asked the staff — we weren't quite prepared for everyone wanting to leave the country...!

So over the years we've had staff in Hong Kong, Australia, Tokyo, New York, San Jose, Germany, Romania... but Vancouver, Canada has gradually become our second home.

Lesson 4

Great 24/7 Support — 365 days of the year is hard work and requires years of effort, training, planning and building.

Commitment
Loadbalancer.org is committed to building the best support team for your needs in your time zone. We have a higher ratio of support staff per customer (more than any other application delivery vendor) and they truly give a damn about your problems.

One customer had a pair of our load balancers for a mission critical system with 200,000 users. They had a power surge that destoyed a lot of kit including the 2 load balancers, and they were desperate — so rather than wait for replacement kit — one of our engineers suggested a different plan. He said he could use iLo remote access into two spare HP servers. Then he could install a fresh bare metal ISO. It took him a couple of hours because he needed to compile a new kernel to support the HP SCSI driver. An unusual approach but most importantly — problem solved.

NB. We don't normally support bare metal installations precisely because of driver and testing issues.

Compass-House

Our current UK Head Office with some of the team...

We're happy to use any excuse for a party...

To celebrate our 15 year journey, we decided to throw a birthday bash — and as expected, it may have gone a little bit crazy.

Cake-small
But thanks very much to Annu for the first ever load balancer cake!

So, where are we going in the next 15 years?

One thing that I have learned in business is that things change fast. Most of the time we are actively trying to manage our growth, so that we can fully train new support staff well before the increase in customer demand.

New and exciting opportunities come up all of the time but if they don't fit with our core priorities we let them go. The first 15 years got us a great product, with great staff and loyal customers.

Some amazing things are happening with artificial intelligence and cloud computing. Who know's what will change in the next 15 years. But I'm committed to ensuring that you get the high-availability tools you need, from experts that you can trust.

So my question is: "What will YOU be building in 15 years time?"

To see what's in the pipeline for our next release, take a look at our roadmap blog.