Task Queue is usually used to execute background task, where you can even setup Cron Jobs.
But if you have a small task which you intent to run in background after each request, do consider using deferred. Sample use cases are:
- Send email or sms after each request (avoid blocking the request)
- Post for FB/Twitter after 5 minutes delay (allow user window to make edits)
The following code will execute defer_post in background after the request complete.
import loggingfrom google.appengine.ext import ndb, deferreddef defer_post(item_key=None): try: item = item_key.get() # do something except Exception as e: # this to avoid logging.info('item_key: %s', item_key.urlsafe()) logging.exception(e) raise deferred.PermanentTaskFailure()@app.route('/test_refer/<item_key>')def test_defer(item_key): item_key = ndb.Key(urlsafe=item_key) deferred.defer(defer_post, item_key=item_key) return "defer_post queued"NOTE: implementing deferred.PermanentTaskFailure is optional, where it will keep on retrying after specific intervals.
NOTE: deferred function must be a module function, not inner function or anonymous function.
You can delay the execution using _countdown parameter.
# wait 60s then run in backgrounddeferred.defer(defer_post, item_key=item_key, _countdown=60)If you setup a Push Queue to specify the processing rate, you can specify the queue name.
NOTE: Queue name must match pattern ^[a-zA-Z0-9-]{1,100}$
# wait 60s then run in backgrounddeferred.defer(defer_post, item_key=item_key, _countdown=60, _queue="my-defer-queue")There are a few gotchas using deferred, mainly:
- Don't pass entities as parameter (the entity might be stale by the time background task execute), use entity key.
- Avoid permanent repeated failure by raising
deferred.PermanentTaskFailure. - If you are using modules imported using
sys.path.append, make sure you include them in the current module. - All argument must be pickable.
- Don't pass in nested function, methods of nested classes, lambda function, static method. Deferred function must be a module function, not inner function or anonymous function.
- Use this for small task only.
References: