Examples of how to upload gzip-compressed HTTP requests to a web server and process these requests on the server.
There's a lot of misleading snippets on the Internet about this topic, as we learned when we tried to implement the ability to gzip data uploaded from our clients to the cloud.
While there are clear standards on how to gzip data in an HTTP response (such as when compressing HTML and CSS to send to a web browser) there doesn't appear to be a standard approach to gzip the data in an HTTP request.
We developed a convention internally of using the
Content-Encoding: header on the HTTP request to
gzip (which mirrors
Accept-Encoding: on browser requests and the resulting
Content-Encoding: gzip on the response that follows). This also makes the process transparent to underlying
Content-Type: headers. Our payloads were JSON but this could easily be used for XML or any other text-like data.
In testing it also became apparent that very small requests (like small responses) grow when gzip compressed. It's probably best to put a minimum bound on content lengths that you will compress -- 100-150 bytes is reasonable.
Advantages to using Content-Encoding instead of a custom Content-Type header is that this will be generally transparent to web frameworks, allowing you to continue using any out-of-the-box parsers/helpers/etc.
Use At Your Own Risk
Using this bare-minimum code, there's a huge potential for a denial of service attack since your application server will continue to accept gzip content until it runs out of memory.
This can potentially be mitigated by configuring the load balancer or reverse proxy (which will almost inevitably be processing traffic for a production application server) to limit the HTTP message size, e.g. Nginx's
Show Me The Code
- A reference python client
- A reference C++ client
- A minimal Django project
- A minimal Flask application
Launch one of the servers (instructions in the each directory). This will start a listener on localhost:9000. Now you can run either of the clients, which expect to connect to localhost:9000.