Cloud Functions Listen to Firestore Triggers (detect create, update, delete) - Python

June 25, 2019

Google Cloud Firestore Triggers

  • Still in Beta as of 25 Jun 2019
  • Listen to event such as create, update, delete or write (create + update + delete)
  • Can monitor specific document (users/marie) or all documents in collection (users/{username})
  • Cannot monitor specific fields
  • Language support: Node.js8/10, Python, Go
  • It can take more than 5 seconds for a functions to respond to Cloud Firestore changes.
  • Proper function invocation is not yet currently guaranteed. Currently, events might not be delivered at all or delivered more than once. Eventually, we plan to guarantee “at least once” delivery. To avoid depending on “exactly-once” mechanics, write your functions to be idempotent.
  • Ordering is not guaranteed. Rapid changes can trigger function invocations in an unexpected order.
  • Firestore in Native mode only.

Test

Write a test function to monitor all documents in specific collection.

log = logging.getLogger(__name__)

# https://cloud.google.com/functions/docs/writing/background#function_parameters
def observe_user_activity(data, context):
    # context.timestamp
    # log.info(f'context.event_id={context.event_id}')
    log.info(f'context.resource={context.resource}')
    log.info(f'context.event_type={context.event_type}')

    log.info(data)

Deploy: monitor all documents in specific collection for write (create + update + delete) events.

gcloud functions deploy observe_user_activity \
  --project PROJECT_ID \
  --runtime python37 \
  --trigger-event providers/cloud.firestore/eventTypes/document.write \
  --trigger-resource "projects/PROJECT_ID/databases/(default)/documents/users/{username}"

Test:

  • Goto Firebase Console -> Develop -> Database.
  • Add collection users and create document.
  • Goto Google Cloud Console -> [Cloud Functions](https://console.cloud.google.com/functions/list) -> Select Function -> View Logs

Observation

Create:

  • oldValue is empty.
context.resource=projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp
context.event_type=providers/cloud.firestore/eventTypes/document.write
{
  "oldValue":{

  },
  "updateMask":{

  },
  "value":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "name":{
        "stringValue":"Desmond"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:08:34.989295Z"
  }
}

Add new field: age=40

  • Entry in updateMask: age
  • age exist in value but not oldValue
{
  "oldValue":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "name":{
        "stringValue":"Desmond"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:11:22.662447Z"
  },
  "updateMask":{
    "fieldPaths":[
      "age"
    ]
  },
  "value":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "age":{
        "integerValue":"40"
      },
      "name":{
        "stringValue":"Desmond"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:11:22.662447Z"
  }
}

Modify field: name=Jack

  • Entry in updateMask: name
  • name exist in value and oldValue, but different value
{
  "oldValue":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "age":{
        "integerValue":"40"
      },
      "name":{
        "stringValue":"Desmond"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:22:36.045007Z"
  },
  "updateMask":{
    "fieldPaths":[
      "name"
    ]
  },
  "value":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "age":{
        "integerValue":"40"
      },
      "name":{
        "stringValue":"Jack"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:22:36.045007Z"
  }
}

Delete field: age

  • Entry in updateMask: age
  • age exist in oldValue but not in value
{
  "oldValue":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "age":{
        "integerValue":"40"
      },
      "name":{
        "stringValue":"Jack"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:26:02.196047Z"
  },
  "updateMask":{
    "fieldPaths":[
      "age"
    ]
  },
  "value":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "name":{
        "stringValue":"Jack"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:26:02.196047Z"
  }
}

Delete document

  • value is empty, but oldValue exist.
{
  "oldValue":{
    "createTime":"2019-06-25T09:08:34.989295Z",
    "fields":{
      "name":{
        "stringValue":"Jack"
      }
    },
    "name":"projects/nuanxindan/databases/(default)/documents/users/Ej0xMlWPab56Wjol7aFp",
    "updateTime":"2019-06-25T09:28:07.021103Z"
  },
  "updateMask":{

  },
  "value":{

  }
}

Detect create, update, delete

import logging
import json
from google.cloud import firestore

log = logging.getLogger(__name__)
firestore_client = firestore.Client()

def observe_user_activity(data, context):
    path_parts = context.resource.split('/documents/')[1].split('/')
    collection_path = path_parts[0]
    document_path = '/'.join(path_parts[1:])

    log.info(f'collection={collection_path}, document={document_path}')
    doc_ref = firestore_client.collection(collection_path).document(document_path)
    # do something with doc_ref

    if not data['oldValue'] and data['value']:
        log.info('create document')
    elif data['oldValue'] and not data['value']:
        log.info('delete document')
    elif data['updateMask']:
        for _field in data['updateMask']['fieldPaths']:
            if _field not in data['oldValue']['fields'] and _field in data['value']['fields']:
                log.info(f'new field: {_field}')
            elif _field in data['oldValue']['fields'] and _field not in data['value']['fields']:
                log.info(f'delete field: {_field}')
            else:
                log.info(f'modified field: {_field}')

    log.info(data)

NOTE: If one of the field is map or list, the behaviour might differ. Refer to Listen to Firestore Map Field Changes.

References:

This work is licensed under a
Creative Commons Attribution-NonCommercial 4.0 International License.