requests: disable auto decoding

Question

Can you disable the auto decoding feature in requests version 1.2.3?

I've looked through the documentation and couldn't find anything, I'm currently experiencing a gzip decode error and want to manually debug the data coming through the request.

Answer 1

You can access the raw response as bytes like this:

resp = requests.get(url, stream=True)
resp.raw.read()

In order to use raw you need to set stream=True for the original request.

Note that raw is a file-like object and reading from response.content will affect its seek cursor, so if you read response.content then try to response.raw.read(), it will return an empty bytes object.

See FAQ: Encoded Data? and Quickstart: Raw Response Content in the Requests documentation.

Answer 2

import requests

r = requests.get(url, stream=True)
with open(local_filename, 'wb') as f:
    for chunk in r.raw.stream(1024, decode_content=False):
        if chunk:
            f.write(chunk)

This way, you will avoid automatic decompress of gzip-encoded response, and still write it to file chunk by chunk (useful for getting big files)