Source code for a database for open fishery data
by RJ
fishBase is a public database for storing and retrieving fishery data.
- It is likely that the best solution for problem statement 2 involves not only establishing a protocol for standardizing data but also supplying an open database to store fishery data collected by participating applications. As a result, the vision for fishBase is to provide a service similar to CrunchBase does for financial data. It should also be noted that the objects presented in the problem statement are not the ideal way of organizing such a database. For example, it is important that time-series data (generated by things like operation location) are not included in account type data (such as vessel details) for optimal indexing. So, further deconstruction of the suggested objects should be done before a database is constructed.
- Web interface
- API endpoints for model schema
- Reorganization of object models (TO DO)
- Creation of database (TO DO)
- User accounts (TO DO)
- Methods for get, put, delete and update data (TO DO)
- Angular-based UI (TO DO)
- Auto-validation (TO DO)
- Alpine Edge (OS)
- Python 3.5.1 (Environment)
- Gunicorn 19.4.5 (Server)
- Flask 0.10.1 (Framework)
- Bootstrap (CSS Template)
- awsDB (Data Model Map)
- Docker (Provisioning)
- GitHub (Version Control)
- LocalTunnel (Secure Tunnel)
- AWS (Web Services)
-- EC2 (Webhosting)
-- S3 (Records Database)
-- Lambda (Scheduled Task Trigger) - PyCharm (IDE)
- Google Drive(Sync, Backup)
- Python 3.5
- Regex
- Shell Script
- Javascript
- Flask in a Container
- Local Credential Controls
- Lean Footprint
- Tunnel Ready for Testing
- EC2 Ready for Deployment
- Install Docker Toolbox on Local Device
- Install Git on Local Device
- Clone/Fork Repository from Version Control service
- Create a /cred Folder in Root to Store Tokens
- [Optional] Create a New Private Remote Repository
-- assets/ (sub-folder for non-python code and project resources)
-- data/ (sub-folder for ephemeral data stored on image)
-- methods/ (sub-folder for application specific python classes)
-- models/ (sub-folder for data object model declarations)
-- static/ (sub-folder for public accessible application content)
-- templates/ (sub-folder for html templates)
startServer.sh
Creates container with required volumes and starts flask on a gunicorn server
Requires:
- Container Alias
- Container Ports
- Mapped Volumes
- Initial Command
- Container Root Folder Name (if AWS EC2 deployment with awsDocker module)
- Virtualbox Name (if Windows or Mac localhost)
rebuildDocker.sh
Initiates an automated build command on Docker to update base image
Requires:
- Container Alias
- Token from Docker Build Settings
- Environment Variable File (in cred/)
tunnelMe.sh
Initiates a secure tunnel from local device to endpoint on localtunnel.me
Requires:
- Container Alias
The Git and Docker repos contain all the configuration for deployment to AWS.
Google Drive can be used to synchronize local copies across multiple dev devices.
Use AWS IAM to assign user permissions and create keys for each collaborator.
Collaborators are required to install dependencies on their local device.
Repo should be FORKED by collaborators so reality is controlled by one admin.
New dependencies should be added to the Dockerfile, NOT to the repo files.
Collaborators should test changes to Dockerfile locally before merging to remote:
docker build -t test-image .
.gitignore and .dockerignore have already been installed in key locations.
To prevent unintended file proliferation through version control & provisioning,
add/edit .gitignore and .dockerignore to include all new:
- local environments folders
- localhost dependencies
- configuration files with credentials and local variables