Load balancing Dell EMC ECS
Useful resources
About Dell EMC ECS
Dell EMC ECS is an object storage solution developed by Dell EMC. It uses hardware ‘nodes’ to provide storage and is designed to be flexible, resilient, and simple to deploy. Dell recommends the use of load balancing in an ECS deployment, in order to distribute the inbound workload across all ECS nodes in an effort to maximize performance.
One of Dell EMC’s approved and documented solutions for load balancing ECS is the free and open source HAProxy load balancer. HAProxy is a key component of the Loadbalancer.org appliance, making it a great fit for load balancing ECS deployments.
Key benefits of load balancing
Here are a few key benefits:
- Ensures data is protected
- Helps make sure data is accessible at all times
- Enables businesses to meet growing data demands through scalability
How to load balance Dell EMC ECS
Persistence (Aka Server Affinity)
Persistence is only recommended for NFS connections when load balancing a Dell EMC ECS deployment. This is due to the fact that caching occurs on the ECS servers when the NFS protocol is used. To maximize efficiency, a given NFS client should continue connecting to the same ECS server, so as to continue re-using the established cache.
Virtual Service (VIP) Requirements
To provide load balancing and HA for Dell EMC ECS, the following VIPs are usually required:
- S3 (for object access via the S3 protocol)
- Atmos (for object access via the Atmos protocol)
- Swift (for object access via the Swift protocol)
- NFS (for providing highly available NFS services)
Optionally, additional VIPs may be required as follows:
- ECS Combined Service (for scenario 2, where only a single IP address is client-facing)
- TLS/SSL termination service (for scenario 2, where HTTPS traffic must be decrypted for inspection)
Port Requirements
The following table shows the ports that are load balanced:
Port | Protocols | Use |
80 | TCP/HTTP | Object access via HTTP calls |
443 | TCP/HTTPS | Object access via HTTP calls (encrypted HTTPS) |
9020 | TCP/HTTP | Object access via the S3 protocol (HTTP) |
9021 | TCP/HTTPS | Object access via the S3 protocol (HTTPS) |
9022 | TCP/HTTP | Object access via the Atmos protocol (HTTP) |
9023 | TCP/HTTPS | Object access via the Atmos protocol (HTTPS) |
9024 | TCP/HTTP | Object access via the Swift protocol (HTTPS) |
2049 | TCP/NFS | NFS service (mountd and nfsd) |
111 | TCP/ONC RPC | Port mapper service |
10000 | TCP/lockd | lockd NFS service |
TLS/SSL Termination
Terminating TLS/SSL connections on the load balancer is not recommended, due to the significant computational overhead, this introduces on the load balancer. Termination and decryption should continue to occur at the ECS servers, which are designed and best placed to perform this function. It may be necessary to terminate and decrypt traffic at the load balancer, so that it may then be read as plaintext and sorted. This is required if sorting incoming traffic by protocol is not possible by using different ports or IP addresses.
Health Checks
S3, Atmos, and Swift Virtual Services
The S3 and Swift virtual services use protocol-specific health checks to query the readiness of a given ECS server to accept connections for those protocols. The Atmos virtual service uses a standard ‘connect to port’ check, which examines whether the Atmos port is open on a given ECS server to determine whether the server is ready to accept connections using the Atmos protocol.
NFS Virtual Service
The NFS virtual service uses a standard ‘connect to port’ check by default, which examines whether the NFS port (2049) is open on a given ECS server to determine whether the server is ready to accept NFS connections. It is possible to configure a custom health check for the NFS service, which will check the availability of all three ports related to NFS operation (111, 2049, and 10000) on the real servers. Only if all three ports are available will a real server be considered ‘healthy’ and ready to accept NFS connections.
There are two deployment scenarios when using Loadbalancer.org appliances as part of a Dell EMC ECS deployment.
Scenario 1 – Virtual Services for each protocol
Note: VIPs = Virtual IP Addresses
Scenario 2 – Single Client-Facing Virtual Service
Note: VIPs = Virtual IP Addresses
deployment guide
Dell EMC ECS Deployment Guide
Read deployment guidemanual
Administration manual v8
Read manualcase study
Dell EMC ECS - Providing a highly available, high throughput object-based storage solution
Read case studyblogs
Things to keep in mind while choosing a load balancer for your object storage system
Read blogNAS vs Object Storage: what's best for unstructured data?
Read blogHow load balancing helps to store and protect petabytes of data
Read blogLoad balancing: The driving force behind successful object storage
Read blogwhite paper
The essential ingredient in successful object storage solutions
Read white paper