Handling large requests with a Web Application Firewall (WAF) while avoiding Denial of Service (DoS) attacks

A web application firewall (WAF) secures web applications by monitoring, filtering, and blocking HTTP messages. It's a topic I've covered extensively in other blog posts, so if you'd like to read more about the general subject then take a look at my other WAF top tips and how-tos.

Sometimes, we need to pass unusually large HTTP requests through our WAF stack. This is something we've come across a number of times over the years. One recent example: I worked with a colleague whose customer had a print-related web application that inserted 11 kB of authentication data into HTTP cookies. This resulted in a massive cookie request header field: because it exceeded our default limit of 8 kB it was rejected by our WAF stack with a 400 Bad Request response status code. I discussed the situation with the customer, explained the risks of increasing the header field size limit and put a working solution in place.

But what are the risks and how do we mitigate them? How can we safely deal with very large HTTP requests when working with a WAF?

Why does a WAF limit the size of what it accepts?

A WAF engine, when paired with a set of rules to provide defence, is a complex piece of machinery with many moving parts: lots of computation takes place under the hood. WAFs can be susceptible to simple denial-of-service attacks. Bombarding a WAF with certain types of HTTP requests can overload it and cause serious problems:

• Requests with large bodies (excluding file uploads) consume a significant amount of CPU time while the WAF rules are executed against the body data.
• Requests with large bodies consume a large amount of memory while they're buffered for inspection.
• Requests with very large headers consume a significant amount of CPU time while the WAF rules are executed against the header data.
• Large file uploads consume a large amount of disk space while they're buffered during a transaction.

We can start to see how a WAF can be attacked from multiple angles: attempting to exhaust its CPU, memory, and disk space. Because of this, it's imperative to have controls in place to limit the resources that a single HTTP transaction can consume. This mitigates the effect of denial-of-service attacks and hardens the WAF for real-world use on the public internet.

Per-transaction resource limits

Default limits

To protect against DoS attacks, all WAF gateways on a Loadbalancer.org appliance have the following default per-transaction resource limits in place:

• Maximum request body size (including file uploads): 13 MB (13107200 bytes)
• Maximum request body size (excluding file uploads): 131 kB (131072 bytes)
• Maximum amount of request body to store in memory before switching to disk: 131 kB (131072 bytes)
• Maximum size of a single HTTP request header field: 8 kB (8190 bytes)

Tip: The default limits for the maximum non-file request body size and the maximum amount of memory to use to buffer a request body are equal. This means that non-file request bodies under the size limit will always be buffered in memory: memory is fast but there's not much of it. Only request bodies of file uploads exceeding 131 kB will be streamed to disk, which is much slower but plentiful.

Example 1

As an example, under the default limits, it's possible to send an HTTP request with up to 131 kB of non-file upload data, e.g. data from filling in form fields:

A request that exceeds the limit will:

• receive a 400 Bad Request response status code if the WAF engine is enabled
• pass through the WAF gateway without issue if the WAF engine is not enabled, i.e. is in detection only mode

Example 2

As another example, under the default limits, it's possible to send an HTTP request with up to 13 MB of file upload data, either as one large file or multiple smaller files:

A request that exceeds the limit will:

• receive a 500 Internal Server Error response status code if the WAF engine is enabled
• pass through the WAF gateway without issue if the WAF engine is not enabled, i.e. is in detection only mode

Why the distinction between file upload data and non-file data?

When handling an HTTP message body, file upload data and non-file data is handled differently.

File upload data

Uploading a file over HTTP usually takes the form of a multipart/form-data request. By default, our WAF engine (ModSecurity) doesn't spend any CPU time inspecting uploaded files (a WAF isn't antivirus software!). With this in mind, it's generally safe to accept megabytes worth of data per request: the only resource being used is disk space while the request is being processed.

Non-file data

An HTTP request body will often contain non-file related data. There are situations where it's useful to send data in an HTTP request where the situation isn't related to file uploads. Consider the example of submitting a username and password on a login page, or providing your address and credit card details to buy something online.

Non-file data is inspected by the WAF engine. Because of this, it's very important to make sure that a sensible limit is in place to control how much data will be accepted for inspection. Allowing huge amounts of data to be inspected means a lot of hard work for the CPU as dozens of time-consuming (and potentially very complex) regular expressions and other operators are evaluated against the data. This opens the door to a simple yet devastating DoS attack.

How to handle large HTTP requests

When working with some web applications, it might be necessary to handle HTTP requests that exceed the default size limits described above.

Large file uploads

The maximum request body size, including file uploads, should be as large as the largest file upload that the web application should accept. If the web application should accept file uploads that are larger than the default 13 MB limit then the limit should be increased.

As an example, to increase the limit to 20 MB for a given WAF gateway, add the following directive to the WAF gateway's manual configuration:

# Buffer request bodies of up to 20 MB in size.
# Allows for requests to contain up to 20 MB of file upload data.
SecRequestBodyLimit 20000000

For additional granularity, a Location directive can be used to increase the limit for certain locations only. For example:

# For the admin portal upload page only, buffer request bodies of up to 500 MB
# in size. Allows for requests to contain up to 500 MB of file upload data.
<Location "/admin-portal/upload.php">
    SecRequestBodyLimit 500000000
</Location>

Note: Request body handling has a hard upper limit of 1 GB.

Requests containing large amounts of non-file upload data

The maximum request body size, excluding file uploads, should be as small as is practical. An example of non-file upload data would be data from filling in form fields. Unlike file upload data, non-file upload data is inspected: the WAF rules are executed against the data. For large chunks of data, rule execution can use a significant amount of CPU time.

If a web application should accept request bodies containing more than the default limit of 131 kB of non-file upload data then the limit should be increased. Some web applications transfer state data or small files in this way, which can often exceed the 131 kB limit where this technique is used.

Warning: Increasing the non-file upload request body size limit makes the WAF more susceptible to denial-of-service attacks. This limit should only be increased if necessary and should still be kept as small as possible. Take this warning seriously!

As an example, to increase the limit to 200 kB for a given WAF gateway, add the following directive to the WAF gateway's manual configuration:

# Buffer request bodies, excluding file uploads, of up to 200 kB in size.
# Allows for requests to contain up to 200 kB of non-file upload data.
SecRequestBodyNoFilesLimit 200000

Because increasing this limit makes a WAF more susceptible to denial-of-service attacks, it's a very good idea to increase the limit only where it's needed. For additional granularity, a Location directive can be used to increase the limit for certain locations only. For example:

# For the user profile avatar upload functionality only, buffer request bodies
# containing up to 200 kB of non-file upload data. Allows users to upload
# avatars up to 200 kB in size.
<Location "/user/profile/avatar-upload.php">
    SecRequestBodyNoFilesLimit 200000
</Location>

Note: The limit for the maximum amount of memory we're prepared to use to buffer a request body can be set using the SecRequestBodyInMemoryLimit directive. The default values of the SecRequestBodyNoFilesLimit and SecRequestBodyInMemoryLimit directives are equal.

Requests with massive header fields

Very occasionally, we encounter a web application that requires the use of massive HTTP request header fields. If any single header field will exceed the default size limit of 8 kB then the limit should be increased.

As an example, to increase the limit to 13 kB for a given WAF gateway, which should be large enough to let through many known large authentication headers which are seen very occasionally, add the following directive to the WAF gateway's manual configuration:

# Allow HTTP request header fields of up to 13 kB in size.
# Should be large enough to let through many known large authentication headers.
LimitRequestFieldSize 13000

In this scenario, the LimitRequestFieldSize directive can't be defined inside a Location directive, so being granular with this setting isn't possible.

Note: To handle HTTP request header fields above 15 kB in size it's necessary to increase HAProxy's Request buffer length. This can be found under Cluster Configuration > Layer 7 - Advanced Configuration. Making this change is required as HAProxy becomes the limiting factor, not Apache.

Time to deploy!

With this knowledge to hand, you can now confidently put a WAF in front of even the strangest web applications featuring unexpectedly large request bodies and massive header fields. And you should, too: any web service exposed to the public internet should have at least a baseline level of WAF protection in place.

So, give it a go, see how you get on, and if you need some extra help, remember that we offer full WAF consultancy and training to get you where you need to be. Whatever your web application security aims are, we're ready to help you achieve your goals.