This system is a POC of a document management tool that will initiate acquisition document packages, and allow the acquisition work force to manage them as data. Handling some of the manual work of keeping the document up to date from an intuitive user interface.
I opted to use relational database to allow rapid development using Django, and PostgreSQL. Data model is available https://dbdiagram.io/d/5db0a60602e6e93440f28f98
-
Files will be kept inside a PostgreSQL HStore
-
Documents will be represented in Markdown. TODO: selected markdown editor and link here.
-
Sheets will be represented in a JSON string defined by the javascript sheet editor. TODO: Select a sheet editor and link here.
-
Other document will be saves as binary blobs and will not be editable.
-
Each file can belong to one folder
-
We will use HTML comment notation to insert and update properties in all editable files like so for a property named 'name':
<!--PROPERTY:name-->Value of name goes here<!--/PROPERTY:name-->
-
-
A Folder object contains files.
- Folders can have have other folders as parents.
- If folder dose not have a parent folder it will be considered a package.
- Packages are folders that wil have the following attribute enabled:
- The project URL of the trello board.
- The properties JSON array stored in HSstore that will include all configured properties for this package.
-
Access control is managed vua the ACL table.
- An ACL item connects a User or Group to a document or a Folder with a defined access level.
- Access should be propagated down, that is to say, if a User has access to a folder they should also have access to all sub folders and document in those folders.
- A user will be presented with all items that he has any access to upon login.
TODO:
TODO:
https://trello.com/b/SL7jTaz3/oia-acquisition-tracker
- Django Easy Audit: Audit trail for all authentication and model change events.
- Django Admin Plus: Adding custom views to the django admin.
- Django Extension: Django Extensions is a collection of custom extensions for the Django Framework. These include management commands, additional database fields, admin extensions and much more.
- Django Model Utils: Django model mix-ins and utilities.
- Django Reversion: an extension to the Django web framework that provides version control for model instances (like documents :) ).
- Django Admin Hstore Widget A nice Hstore editors widget for properties in the django admin.
- Django Environ: Take sensitive project settings out of git and save them in
.env
file or the local env for deployment. - Django Guardian: Object level permissions in django.
The easiest way to get rolling is to use Docker:
# Build the application and database images
docker-compose build
# Run database migrations
docker-compose run app python manage.py migrate
# Load fixtures
docker-compose run app python manage.py loaddata data_driven_acquisition/fixtures/*
# Create a superuser
docker-compose run app python manage.py createsuperuser
Then, to start the application and its database:
docker-compose up
Use CTRL+C to stop.
Alternatively, follow these setup steps:
-
Database Setup
-
install PostgreSQL.
-
Create database
-
Configure your Create a new DB and role. e.g.:
$ psql CREATE DATABASE myproject; CREATE USER myprojectuser WITH PASSWORD 'password'; ALTER ROLE myprojectuser SET client_encoding TO 'utf8'; ALTER ROLE myprojectuser SET default_transaction_isolation TO 'read committed'; ALTER ROLE myprojectuser SET timezone TO 'UTC'; GRANT ALL PRIVILEGES ON DATABASE myproject TO myprojectuser; \q
-
Create the hsstore extention in the database by running:
$ psql myproject CREATE EXTENSION IF NOT EXISTS hstore;
-
-
Project set up
-
Create project folder
mkdir data-driven-acquisition
-
Create virtual environment (using virtual env wrapper in this example).
mkvirtualenv data-driven-acquisition --python=python37
-
Install the requirements
pip install -r requirements.txt
-
Optional: Install the development and testing requirements if needed.
pip install -r requirements_dev.txt pip install -r requirements_test.txt
-
Create a .env file looking like so
# Set debug settings globally to the app, makes logging very verbose. (on/off) DEBUG=on # Used for all encryption operations. Use something like this to generate. # python -c "import random; ''.join([random.SystemRandom().choice('abcdefghijklmnopqrstuvwxyz0123456789!@#$%^&*(-_=+)') for i in range(50)])" SECRET_KEY=your-secret-key DATABASE_URL=psql://DB_USER:DB_PASS@DB_IP/URL:DB_PORT/DB_NAME ALLOWED_HOSTS=127.0.0.1,0.0.0.0 # Prod IP goes here # GitHub Config # https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line GITHUB_ACCESS_KEY=your-key-goes-here # MAke sure your github user has access to repo GITHUB_TEMPLATE_REPO=github-repo-containing-templates # if you want ot use trello #Do we use trello, def to no USER_TRELLO=yes # Set trello API Key, required if use trello is yes #https://developers.trello.com/reference#api-key-tokens TRELLO_APP_KEY=trello-api-key TRELLO_APP_SECRET=trello-oauth-secret # https://developers.trello.com/page/authorization # Get token by running and visiting the URL # trello_auth_url = ''.join([ # 'https://trello.com/1/authorize?expiration=never', # '&name=DataDrivenAcquisition&scope=read,write&response_type=token', # f'&key={settings.TRELLO["APP_KEY"]}']) TRELLO_TOKEN=token-goes-here # Get the board ID from the URL of your trello board TRELLO_BORED_ID=board-id
-
Collect static and run
python manage.py collectstatic python manage.py migrate python manage.py loaddata data_driven_acquisition/fixtures/*
-
```
cf login -a api.fr.cloud.gov --sso
# Follow instructions to get key
cf target -o sandbox-gsa -s adam.grandt
cf push data_driven_acquisition
# login to web UI to:
# * Create the same env vars described in the .env section above.
# * Create and bind shared postgres service
cf restart data_driven_acquisition
cf shell
# in the cf shell run :
/tmp/lifecycle/shell
python manage.py createsuperuser
# Follow instructions
# Visit the URL configured in the manifest file
```
- TODO
Does the code actually work?
```shell
(myenv) $ pip install tox (or -r requirements_test.txt)
(myenv) $ tox
```