/python3-dev-env

A container-based Python back-end development environment featuring flask, mysql, ngrok, make, invoke, pip env and {dev, staging, prod} docker-compose override files.

Primary LanguagePython

python3-dev-env

A container-based Python back-end environment featuring:

  • Flask web application using MySQL for data storage
  • ngrok to publicly expose the app
  • A dev environment container with LunarVim, Python dev tools, Node.js LTS, Git, Jupyter, etc.
  • Makefile for quick, common operations
  • Invoke (tasks.py) for Pythonic command-running
  • docker-compose override files for development, production, and staging
  • pytest for tests
  • pipenv for optional dependency management

Project Structure

python3-dev-env/
├── docker-compose.yml                # Base Docker Compose file
├── docker-compose.dev.yml            # Development overrides
├── docker-compose.prod.yml           # Production overrides
├── docker-compose.staging.yml        # Staging overrides
├── Makefile                          # Make-based command shortcuts
├── Pipfile                           # pipenv dependencies (optional)
├── Pipfile.lock                      # Generated pipenv lock file
├── README.md                         # This README
├── tasks.py                          # Invoke tasks
├── flask_app/
│   ├── app.py                        # Flask app code
│   ├── requirements.txt              # Traditional requirements file (if not using pipenv)
│   └── Dockerfile                    # Docker build for Flask
├── dev_env/
│   └── Dockerfile                    # Docker build for dev environment
├── ngrok/
│   └── Dockerfile                    # Docker build for ngrok
└── tests/
    └── test_app.py                   # Sample pytest tests

Quick Start

  1. Clone this repository:

    git clone https://github.com/grahamg/python3-dev-env.git
cd python3-dev-env

  2. (Optional) Install pipenv if you want advanced Python dependency management:

    pip install pipenv

    or rely on the standard requirements.txt.

  3. Build and run in development mode:

    docker-compose -f docker-compose.yml -f docker-compose.dev.yml up --build

    Alternatively, you can use the Makefile or Invoke tasks (see below).

  4. View the Flask App:

  5. Check ngrok:

    • The public URL can be found in the container logs or the local ngrok dashboard at http://localhost:4040.

  6. Stop the environment:

    docker-compose -f docker-compose.yml -f docker-compose.dev.yml down

    or use the Makefile / Invoke commands.

Docker Compose Overrides

We use multiple files to separate configuration:

  • docker-compose.yml: Base configuration (Flask, MySQL, dev_env, ngrok).
  • docker-compose.dev.yml: Development overrides (e.g., volume mounts, environment variables, etc.).
  • docker-compose.prod.yml: Production overrides (remove local volumes, set FLASK_ENV=production, etc.).
  • docker-compose.staging.yml: Staging overrides (similar to production, but with staging environment vars).

Usage Examples

  • Development: 
    docker-compose -f docker-compose.yml -f docker-compose.dev.yml up --build
  • Production: 
    docker-compose -f docker-compose.yml -f docker-compose.prod.yml up --build
  • Staging: 
    docker-compose -f docker-compose.yml -f docker-compose.staging.yml up --build

Makefile Usage

A Makefile provides convenient shortcuts for common operations. Example targets:

  • make build – Builds all containers.
  • make up – Brings up all services (in dev mode).
  • make down – Stops all services (in dev mode).
  • make test – Runs pytest inside the dev container.
  • make shell – Opens a bash shell in the dev container.

Example

# Build all images
make build

# Run in dev mode (detached)
make up

# Run tests
make test

# Stop everything
make down

You can customize the Makefile rules as needed.

Invoke (tasks.py)

We also include a Python-based task runner using Invoke. See tasks.py for definitions like:

  • inv build
  • inv up [--env=dev|prod|staging]
  • inv down [--env=dev|prod|staging]
  • inv test
  • inv shell

Example

# Install invoke if needed
pip install invoke

# Build the images
inv build

# Bring up dev environment
inv up

# Bring down dev environment
inv down

# Run tests in dev_env container
inv test

By default, env=dev. You can override with --env=prod or --env=staging.

The Dev Environment Container

  • Based on ubuntu:22.04
  • Includes:
    • Python 3, pip, pipenv, build-essential
    • Node.js + npm
    • Jupyter Notebook
    • LunarVim, Emacs, nano, tig, netcat
    • Git, etc.

To develop inside this container:

  1. Ensure containers are running: 
    docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d dev_env
  2. Shell in: 
    docker exec -it dev_env /bin/bash
  3. Your project files are typically in /workspace (based on the docker-compose.yml volume mount).

Inside, you can run:

  • lvim or emacs or nano to edit files.
  • pipenv install or pip install to manage dependencies.
  • pytest to run tests (or make test / inv test from the host).

The Flask App

Located in flask_app/.

  • app.py: Basic Flask to-do application.
  • Connects to MySQL using environment variables:
    • MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DATABASE
  • Exposes HTTP on port 8003.

Example Endpoints

  • GET / – Show the to-do list (HTML).
  • POST /add – Add a new task (form-data key: task).
  • POST /complete/<int:task_id> – Mark a task as completed.

The MySQL Container

  • Using official mysql:8.0 image.
  • Root password = root (development only!).
  • Database = todo_db.
  • Mapped port is 3306 on the host (if you need direct DB access).

ngrok Container

  • Based on a minimal Alpine image.
  • Exposes the Flask app publicly at a random subdomain.
  • Internal command: ngrok http flask_app:8003.
  • Access the ngrok web interface at http://localhost:4040.

Testing with pytest

We have a tests/ directory containing sample tests (e.g., test_app.py).

  • Install pytest (already in Pipfile or requirements.txt).
  • Ensure containers are running (especially flask_app).
  • Run from the host machine via dev_env: 
    docker exec -it dev_env pytest
    or with the Makefile: 
    make test
    or with Invoke: 
    inv test
    or directly from inside dev_env: 
    pytest

pipenv

If you prefer pipenv over requirements.txt:

  1. Check Pipfile in the project root, containing dependencies like Flask, mysql-connector-python, pytest, invoke, etc.
  2. In the dev container: 
    cd /workspace
pipenv install
  3. Run your Flask app under pipenv: 
    pipenv run python flask_app/app.py
  4. Or just rely on requirements.txt within the flask_app Docker build.

(Both approaches are valid; choose whichever you prefer.)

Contributing

  • Issues and pull requests are welcome.
  • For major changes, please open an issue first.
  • Adhere to coding standards and test thoroughly.

License

This project is provided under an open-source license (e.g., MIT).
Feel free to modify, distribute, or adapt as needed.

TODO

  1. Linting/Formatting: Tools like Black, flake8, or isort should be used to keep code formatting consistent.
  2. Continuous Integration: Set up a CI pipeline to run pytest on every git push.
  3. Secrets Management: For production deployments use Docker secrets, environment variable management, or a vault solution.
  4. Production Hardening: Instead of the built-in Flask server, use something like gunicorn or uwsgi behind a reverse proxy.
  5. Scaling: For heavier usage, adopt container orchestration (Kubernetes, ECS, etc.) or at least scale MySQL with persistent volumes.