/depot

Toolkit for storing files and attachments in web applications

Primary LanguagePythonMIT LicenseMIT

https://raw.github.com/amol-/depot/master/docs/_static/logo.png

DEPOT - File Storage Made Easy

https://coveralls.io/repos/amol-/depot/badge.png?branch=master

DEPOT is a framework for easily storing and serving files in web applications on Python2.6+ and Python3.2+.

DEPOT supports storing files in multiple backends, like:

  • Local Disk
  • In Memory (for tests)
  • On GridFS
  • On Amazon S3 (or compatible services)
  • On Google Cloud Storage

and integrates with database by providing files attached to your SQLAlchemy or Ming/MongoDB models with respect to transactions behaviours (files are rolled back too).

Installing

Installing DEPOT can be done from PyPi itself by installing the filedepot distribution:

$ pip install filedepot

Getting Started

To start using Depot refer to Documentation

DEPOT was presented at PyConUK and PyConFR in 2014

standalone

Here is a simple example of using depot standalone to store files on MongoDB:

from depot.manager import DepotManager

# Configure a *default* depot to store files on MongoDB GridFS
DepotManager.configure('default', {
    'depot.backend': 'depot.io.gridfs.GridFSStorage',
    'depot.mongouri': 'mongodb://localhost/db'
})

depot = DepotManager.get()

# Save the file and get the fileid
fileid = depot.create(open('/tmp/file.png'))

# Get the file back
stored_file = depot.get(fileid)
print stored_file.filename
print stored_file.content_type

models

Or you can use depot with SQLAlchemy to store attachments:

from depot.fields.sqlalchemy import UploadedFileField
from depot.fields.specialized.image import UploadedImageWithThumb


class Document(Base):
    __tablename__ = 'document'

    uid = Column(Integer, autoincrement=True, primary_key=True)
    name = Column(Unicode(16), unique=True)
    content = Column('content_col', UploadedFileField)  # plain attached file

    # photo field will automatically generate thumbnail
    photo = Column(UploadedFileField(upload_type=UploadedImageWithThumb))


# Store documents with attached files, the source can be a file or bytes
doc = Document(name=u'Foo',
            content=b'TEXT CONTENT STORED AS FILE',
            photo=open('/tmp/file.png'))
DBSession.add(doc)
DBSession.flush()

# DEPOT is session aware, commit/rollback to keep or delete the stored files.
DBSession.commit()

ChangeLog

0.11.0

  • Officially support Python 3.12
  • Addressed deprecation of Image.ANTIALIAS in Pillow, Image.LANCZOS is used instead
  • TurboGears2 is no longer needed to run tests
  • Depot is now compatible with multipart module or other replacements of cgi.FieldStorage
  • Fixed an open file leak in UploadedImageWithThumb
  • Fixed an open file leak in WithThumbnailFilter

0.10.0

  • Added support for Google Cloud Storage
  • Fixed ACL issues with S3
  • Deprecated boto2 backend, use boto3
  • Added support for performing backups by copying to another storage

0.9.0

  • Support for SQLAlchemy 1.4 and 2.0
  • Support for SQLAlchemy objects deleted with .delete(synchronize_session="fetch")
  • Tests migrated to unittest

0.8.0

  • Replaced unidecode dependency with anyascii to better cope with MIT License.

0.7.1

  • Fix a bug in AWS-S3 support for unicode filenames.

0.7.0

  • Support for storage_class option in depot.io.boto3.S3Storage backend. Detaults to STANDARD

0.6.0

  • Officially support Python 3.7
  • Fix DEPOT wrongly serving requests for any url that starts with the mountpoint. (IE: /depotsomething was wrongly served for /depot mountpoint)
  • In SQLAlchemy properly handle deletion of objects deleted through Relationship.remove (IE: parent.children.remove(X))
  • In SQLAlchemy properly handle entities deleted through cascade='delete-orphan'

0.5.2

  • Fixed an start_response called a second time without providing exc_info error with storages supporting plublic urls

0.5.1

  • URLs generated by DepotMiddleware are now guaranteed to be plain ascii
  • [Breaking change]: Bucket existance with S3 storages should now be more reliable when the bucket didn't already exist, but it requires an additional AWS policy: s3:ListAllMyBuckets that wasn't required on 0.5.0

0.5.0

  • depot.io.boto3.S3Storage now provides support for accessing S3 with boto3. The previously existing depot.io.awss3.S3Storage can still be used to store files on S3 using boto.
  • SQLAlchemy integration now handles deletion of files on rollback when session is not flushed. Previously flushing the session was required before a rollback too.
  • It is now possible to run tests through tox and build docs through tox -e docs
  • DEPOT is now tested against Python 3.6

0.4.1

  • Fixed installation error on non-UTF8 systems
  • Improved support for polymorphic subtypes in SQLAlchemy

0.4.0

  • Support for Python 3.5
  • Fixed Content-Disposition header for filenames including a comma

0.3.2

  • MemoryFileStorage now accepts any option, for easier testing configuration

0.3.1

  • Fixed Content-Disposition header when serving from S3 directly
  • Fixed size of SQLAlchemy field on Oracle (was bigger than the allowed maximum)

0.3.0

  • MemoryFileStorage provides in memory storage for files. This is meant to provide a convenient way to speed up test suites and avoid fixture clean up issues.
  • S3Storage can now generate public urls for private files (expire in 1 year)
  • Files created from plain bytes are now named "unnamed" instead of missing a filename.

0.2.1

  • S3Storage now supports the prefix option to store files in a subpath

0.2.0

  • Storages now provide a list method to list files available on the store (This is not meant to be used to retrieve files uploaded by depot as it lists all the files).
  • DepotExtension for Ming is now properly documented

0.1.2

  • It is now possible to use multiple WithThumbnailFilter to generate multiple thumbnails with different resolutions.
  • Better documentation for MongoDB UploadedFileProperty

0.1.1

  • Fixed a bug with Ming support when acessing UploadedFileProperty as a class property
  • Improved support for DEPOT inside TurboGears admin when using MongoDB

0.1.0

  • Added DepotManager.alias to configure aliases to storage. This allows easy migration from one storage to another by switching where the alias points.
  • Now UploadedFileField permits to specify upload_storage to link a Model Column to a specific storage.
  • Added policy and encrypt_key options to S3Storage to upload private and encrypted files.

0.0.6

  • Added host option to S3Storage to allow using providers different from AWS.

0.0.5

  • Added FileIntent to explicitly provide content_type and filename to uploaded content.

0.0.4

  • Added Content-Disposition header with original filename in WSGI middleware

0.0.3

  • Work-Around for issue with wsgi.file_wrapper provided by Waitress WSGI Server

0.0.2

  • Official Support for AWS S3 on Python3