Squid3 Patch – Normalize Accept-Encoding Header

If you don’t strip Accept-Encoding, you might want to normalize this important Http Header to make sure your cache doesn’t get flooded with multiple identical copies.

Why? Squid takes the Accept-Encoding header serious. Especially, if the backend sets a Vary header like Vary: Accept-Encoding.  This is fine if two requestors send headers which cause the Backend System to respond differently, like “deflate” and “gzip”.

In this case, you want Squid to keep the two variations of Responses within its Cache. What you do not want is to keep identical responses cached multiple times like for “gzip,deflate” and “gzip, deflate” (keep an eye on the space after the comma). By default Squid does treat such variants as separate encodings and therefore store an identical copy of your Backend’s Response in its Cache.

With this problem being the major impact, there is another downside: Your backend has to serve all this variations which is totally unnecessary!

This code change allows Squid 3 to normalize the Accept-Encoding Header and keeps a maximum variation of 3 in its store (no encoding, gzip and deflate).

File: client_side_request.cc of Squid version 3.2.0.6

static void
clientInterpretRequestHeaders(ClientHttpRequest * http)
{
    ...

    // *** NORMALIZE ACCEPT-ENCODING HEADER ***

    if (req_hdr->has(HDR_ACCEPT_ENCODING)) {
        String aeh = req_hdr->getStrOrList(HDR_ACCEPT_ENCODING);
        req_hdr->delById(HDR_ACCEPT_ENCODING);

        if (aeh.find("gzip") != String::npos) {
            req_hdr->addEntry(new HttpHeaderEntry(HDR_ACCEPT_ENCODING, NULL, "gzip"));
        } else if (aeh.find("deflate") != String::npos) {
            req_hdr->addEntry(new HttpHeaderEntry(HDR_ACCEPT_ENCODING, NULL, "deflate"));
        }

        aeh.clean();
    }

    // *** NORMALIZE ACCEPT-ENCODING HEADER ***

 if (req_hdr->has(HDR_AUTHORIZATION))
        request->flags.auth = 1;
 ...

Continue to this blog-post in case you are interested in patching Squid Version 2.7: http://letsgetdugg.com/2009/12/06/squid-headers-normalization-patch/

Happy Caching! ;-)