Rethinking cloud native load balancing

Public cloud can be inflexible and the costs difficult to manage. Although for some, the convenience and scalability of public cloud trumps cost.

In this blog we put public cloud provider AWS under the lens, and highlight some important considerations for those weighing up their cloud options. We also explain the pros and cons of a public v. platform-agnostic load balancer.  

Cloud migration

A cloud-first application strategy will involve moving some or all of your applications to the cloud. This can be achieved in a number of different ways:

1. Rehosting (lift and shift your existing local applications to cloud)
2. Replatforming (refactoring your application to work in cloud environments)
3. Rebuilding (developing cloud native applications)

Amazon Web Services (AWS) was an early leader in public cloud computing and provides the platform of choice for many hundreds of thousands of organizations globally.

In order to leverage the benefits of AWS infrastructure and services (such as scalable computer resources, high availability architectures and the ability to provide access to a larger user base from anywhere) load balancing will form a necessary part of your cloud application stack.

What do I need from a cloud load balancer?

Load balancing facilitates high availability and performance at scale, however there are a number of considerations:

• What features and intelligence do you need that deliver tangible results?
• How does this fit with my hybrid/multi-cloud strategy?
• How do I manage my project budget without a reliable estimation of costs over time?
• What level of support do I receive that provides access to guidance and expertise?

Below we compare public v. platform-agnostic load balancers in the cloud.

Public: AWS Elastic load balancing

AWS provides four cloud load balancing options natively:

1. Application Load Balancer — Used for load balancing HTTP and HTTPS traffic, operates at Layer 7. The Application load balancer is designed for modern web applications and routes traffic to Amazon VPC targets, based upon the content of the request.
2. Network load balancer — Used for TCP, UDP and TLS traffic, operates at Layer 4. The network load balancer is designed for high performance and routes traffic to targets within Amazon VPC at low latency.
3. Classic load balancer — Basic load balancing for applications that were built within the EC2-Classic network, operates at both layers 4 and 7. Classic load balancer is considered a legacy service.
4. Gateway Load balancer — A transparent gateway for third party network appliances.

Considerations when choosing cloud load balancing

The benefits of cloud native load balancing are also met with a number of drawbacks.

Drawback One: Inflexibility

The load balancers are designed to provide essential, out-of-the-box load balancing capabilities within AWS. Each load balancer performs specific functions and multiple different load balancer instances may be required to provide your required application functionality. For example, Application load balancer supports only HTTP/HTTPS requests.

Drawback Two: Cost management

Classic load balancer pricing was historically simple, metering uptime and traffic. Application and Network load balancer pricing is more complicated. As well as metered uptime (Application/Network Load Balancer-hour) Amazon introduced the concept of load balancer capacity units (LCU-hour).

Application load balancer

An LCU measures the dimensions on which the Application Load Balancer processes your traffic (averaged over an hour). The four dimensions measured are:

1. New connections: Number of newly established connections per second. Typically, many requests are sent per connection.
2. Active connections: Number of active connections per minute.
3. Processed bytes: Number of bytes processed by the load balancer in Gigabytes (GB) for HTTP(S) requests and responses.
4. Rule evaluations: Product of the number of rules processed by your load balancer and the request rate.

Network load balancer

An NLCU measures the dimensions on which the Network Load Balancer processes your traffic (averaged over an hour). The three dimensions measured are:

1. New connections or flows: Number of newly established connections/flows per second. Many technologies (HTTP, WebSockets etc.) reuse Transmission Control Protocol (TCP) connections for efficiency. The number of new connections is typically lower than your request or message count.
2. Active connections or flows: Peak concurrent connections/flows, sampled minutely.
3. Processed bytes: The number of bytes processed by the load balancer in Gigabytes (GB).

Resource and cost management is a crucial part of cloud strategy. Bill shock is real, and perhaps the most difficult task is estimating what your peak traffic volume will be (based upon the metrics used to calculate billing).

AWS provides tools to calculate the monthly costs which at full capacity can quickly become an expensive component of your application.

The two most dominant metrics that impact costs are data processed (throughput) and new connections/requests per second (each new connection generating several requests)

It’s worth pointing out that you are only charged only for the metric you use the most; here are some examples:

Throughput
100 GB per hour / 1 GB processed bytes per hour per LCU for EC2 instances and IP addresses as targets = 100 processed bytes LCUs for EC2 instances and IP addresses as targets
100 new connections per second / 25 new connections per second per LCU = 4 new connections LCUs
100 new connections per second x 5 seconds = 500 active connections
500 active connections / 3000 connections per LCU = 0.16666666666666666 active connections LCUs
10 rules per request - 10 free rules = 0 paid rules per request after 10 free rules
Max (0 USD, 0 USD) = 0.00 paid rules per request
Max (100 processed bytes LCUs, 4 new connections LCUs, 0.16666666666666666 active connections LCUs, 0 rule evaluation LCUs) = 100 maximum LCUs
1 load balancers x 100 LCUs x 0.008 LCU price per hour x 730 hours per month = 584.00 USD

Application Load Balancer LCU usage charges: 584.00 USD/month

Connections
1 GB per hour / 1 GB processed bytes per hour per LCU for EC2 instances and IP addresses as targets = 1 processed bytes LCUs for EC2 instances and IP addresses as targets
10,000 new connections per second / 25 new connections per second per LCU = 400 new connections LCUs
10,000 new connections per second x 3 seconds = 30,000 active connections
30,000 active connections / 3000 connections per LCU = 10 active connections LCUs
3 rules per request - 10 free rules = -7 paid rules per request after 10 free rules
Max (-7 USD, 0 USD) = 0.00 paid rules per request
Max (1 processed bytes LCUs, 400 new connections LCUs, 10 active connections LCUs, 0 rule evaluation LCUs) = 400 maximum LCUs
1 load balancers x 400 LCUs x 0.008 LCU price per hour x 730 hours per month = 2,336.00 USD

Application Load Balancer LCU usage charges: 2,336.00 USD/month


Here are a couple of examples of AWS customers with 100,000s of requests per second:

https://pinchito.es/2016/nginx-balancer.html
https://segment.com/blog/the-10m-engineering-problem/

Cloud-native load balancing can be cost-effective, but costs creep at scale. With Cloud spending becoming a dominant KPI, reducing the weight of cloud costs on organizations is key.

Drawback Three: Lack of support for hybrid cloud and multi-cloud infrastructures

It's also worth noting that as many adopt the hybrid and multi-cloud approach, managing these workloads optimally across different environments becomes increasingly challenging and the benefits of standardization on a single platform are lost.

Thorough due diligence on short, medium and long term return on investment is therefore essential.

Platform-agnostic: Loadbalancer.org in AWS

Using a platform agnostic load balancer in the cloud, such as Loadbalancer.org, can have a number of advantages.

Technical advantages:

• A feature rich Application Delivery Controller (ADC) - Enables the load balancing of virtually any protocol. Advanced Layer 4/7 load balancing with integrated Auto Scaling, multi-AZ resilience, SSL/TLS offloading.
• Granular load balancing - Choose your algorithm and a range of application level health checks, including support for custom health check scripts.
• Global Server Load balancing (GSLB) - Location affinity load balancing and DNS based routing to multiple data center locations.
• Simple by design - Ease of use with simple management interface, simplified and accelerated cloud migration.
• Secure your web application - With an OWASP top 10 compliant Web Application Firewall (WAF).
• Automation support - Full API support enables integration with automation platforms and tie-in to AWS Lambda functions.
• Freedom license - Our no lock-in guarantee and Freedom License offers our customers complete freedom to move to any platform, de-risking your future.

Cost advantages:

Optimizing your load balancer costs at scale can provide significant savings.

Loadbalancer.org costs are calculated simply, with the option of fixed hourly/annual billing or Bring-your-own-License (BYOL).

Hourly billing

The fixed running costs consist of:

• Software/hr - hourly billing for the active load balancer
• EC2/hr - EC2 running costs for the active running EC2 instance
• Annual billing represents a saving of 15% at the time of writing

BYOL

The BYOL option allows the purchase or migration of licenses to your load balancer in AWS without the software/hour billing. Industry leading enterprise support is included in the pricing.

Support advantages:

Cloud-native load balancers such as AWS do not include support. Support is billed as an additional cost in AWS so you need to be very confident that you're not going to run into issues.

The reverse is true at Loadbalancer.org. Our team works with our customers directly to provide consultative support that enables you to get the solution you need up and running as quickly and smoothly as possible, with experts on hand to answer any queries you may have. All at no extra cost.

Our tierless, hands-on support team are customer service obsessed and care about your application performance (and when we say 'obsessed', we mean dancing-around-the-office-shouting-about-an-exciting-new-idea obsessed....).

So we're proud to say that they are way more than just a helpdesk. We work directly with our customers to underpin their success with:

• Unlimited features and updates
• Unlimited security updates
• Priority case management.
• 24/7 or business hours support.
• Multi-channel support
• Infrastructure and 3rd party assistance
• Remote assistance

In fact, we love our job and what we do so much we offer frank advice with no oversell - so we can sleep at night and you can move on to the next thing on your to-do list.

Stop. Load balance. Go.