Using Python Requests Module In Google App Engine

January 16, 2018

Python Requests seems to work fine on production, but you will bump into the following error on development server

File “/*/requests/adapters.py”, line 490, in send raise ConnectionError(err, request=request) ConnectionError: (‘Connection aborted.’, error(13, ‘Permission denied’))

To fix it, import requests-toolbelt in your equivelant of main.py.

import requests
import requests_toolbelt.adapters.appengine

# Use the App Engine Requests adapter. This makes sure that Requests uses URLFetch.
requests_toolbelt.adapters.appengine.monkeypatch()

In production, you will still bump into the following error (which is actually a warning).

AppEnginePlatformWarning: urllib3 is using URLFetch on Google App Engine sandbox instead of sockets. To use sockets directly instead of URLFetch see https://urllib3.readthedocs.io/en/latest/reference/urllib3.contrib.html.

You can ignore the above error/warning, or you can enable making httplib use sockets.

Edit app.yaml and add the following.

env_variables:
   GAE_USE_SOCKETS_HTTPLIB : 'anyvalue'

NOTE: Enable GAE_USE_SOCKETS_HTTPLIB will make http request using Socket instead of urlfetch. Refer to UrlFetch quota vs Sockets quota.

NOTE: I stick with UrlFetch at the moment and didn’t try the Sockets solution.

References:

This work is licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License.