Feature - ability to export declared schema to a file and detect deletions

[codekiln]

At PR time, we'd like our CI system to be able to detect any field or model deletions, and put a warning on the PR or require additional approvals if they are detected.

To do this we'd like the ability to export a flat file of the whole schema modeled in mongoengine, so we can diff it against the deployed schema to find any deletions.

Ideally, a CI workflow would use a cli tool, something like this:

git checkout <previous, deployed release branch> mongo_schema.json
mv mongo_schema.json mongo_schema_previous.json
# generate the current schema and save to file
mongoengine_schema . mongo_schema_current.json
# perform a diff of the schema changes and output only the deletions 
mongoengine_schema diff mongo_schema_previous.json mongo_schema_current.json --deletions > new_deletions.txt 
# take action if new_deletions.txt length is greater than 0

I'm picturing such a schema might be available by searching the python classpath at the given location for all subclasses of Document, and then using the field definitions to generate a schema.

[bagerard]

We don't have anything built-in but you should be able to build something quite easily with the following:

• Once all your modules containing Document classes are imported, all classes gets registered in
  _document_registry importable here from mongoengine.base import _document_registry
• A Document class exposes all its fields under YourDocu._fields as a dict

This is not necessarly bulletproof, but you should be able to build something that would detect easily new/deleted fields quite easily which are probably what you are aiming for.

Here s a snippet that should help

from mongoengine import *
from mongoengine.base import _document_registry

connect()

class Child(Document):
    p = StringField()

for doc_cls_name, doc_cls in _document_registry.items():
    for doc_cls_field_name, doc_cls_field_cls in doc_cls._fields.items():
        print(doc_cls_name, doc_cls_field_name, doc_cls_field_cls.__class__.__name__)

Which prints

Child p StringField
Child id ObjectIdField

Shouldn't be too difficult to build a json out of those information and detect straightforward changes