Ae-Mc/fastapi-alembic-sqlmodel-async

This is a project template which uses FastAPI, Alembic and async SQLModel as ORM. It shows a complete async CRUD using authentication and role base access control.

Async configuration for FastAPI and SQLModel

This is a project template which uses FastAPI, Alembic and async SQLModel as ORM. It shows a complete async CRUD template using authentication. Our implementation utilizes the newest version of FastAPI and incorporates typing hints that are fully compatible with Python 3.10 and later versions.

Set environment variables

Create an .env file on root folder and copy the content from .env.example. Feel free to change it according to your own configuration.

Run the project using Docker containers and forcing build containers

Using docker compose command

docker compose -f docker-compose-dev.yml up --build

Using Makefile command

make run-dev-build

Run project using Docker containers

Using docker compose command

docker compose -f docker-compose-dev.yml up

Using Makefile command

make run-dev

Setup database with initial data

This creates sample users on database.

Using docker compose command

docker compose -f docker-compose-dev.yml exec fastapi_server python app/initial_data.py

Using Makefile command

make init-db

Any of the above commands creates three users with the following passwords:

• Admin credentials -> username: admin@admin.com and password: admin
• Manager credentials -> username: manager@example.com and password: admin
• User credentials -> username: user@example.com and password: admin

You can connect to the Database using pgAdmin4 and use the credentials from .env file. Database port on local machine has been configured to 5454 on docker-compose-dev.yml file

(Optional) If you prefer you can run pgAdmin4 on a docker container using the following commands, they should executed on different terminals:

Starts pgadmin

make run-pgadmin

Load server configuration (It is required just the first time)

make load-server-pgadmin

This starts pgamin in http://localhost:15432.

ERD Database model

Containers architecture

As this project uses Caddy as a reverse proxy, which uses namespaces routing, you can access the documentation with the following path http://fastapi.localhost/docs

Preview

Static files

All files on static folder will be served by Caddy container as static files. You can check it with this link http://static.localhost

Minio server

This template allows users can upload their photos. The images are stored using the open source Object Storage Service (OSS) minio, which provides storage of images using buckets in a secure way through presigned URLs.

• Minio credentials -> username: minioadmin and password: minioadmin

Celery

Celery is a distributed task queue that allows developers to run asynchronous tasks in their applications. It is particularly useful for tasks that are time-consuming, require heavy computation or access external services, and can be run independently of the main application. It also offers features such as task scheduling, task prioritization, and retries in case of failure.

Celery Beat is an additional component of Celery that allows developers to schedule periodic tasks and intervals for their Celery workers. It provides an easy-to-use interface for defining task schedules and supports several scheduling options such as crontab, interval, and relative.

You can see the architecture used in this project which uses Redis as celery broker and the current postgres database as celery backend. It also uses celery-sqlalchemy-scheduler to store celery beats task into database so they can mutated.

Within the natural_language endpoints, you can access a sample application that demonstrates not only synchronous prediction of machine learning models but also batch prediction. Additionally, there are examples of how to schedule periodic tasks using Celery Beat in the periodic_tasks endpoints.

Run Alembic migrations (Only if you change the DB model)

Using docker compose command

docker compose -f docker-compose-dev.yml exec fastapi_server alembic revision --autogenerate
docker compose -f docker-compose-dev.yml exec fastapi_server alembic upgrade head

Using Makefile command

make add-dev-migration

Production Deployment

Remember to use a persistant PostgreSQL database, update the new credentials on .env file and use this command to run the project in a production environment. For testing this configuration on localhost you can uncomment the database container and depends_on of fastapi container otherwise it will not work on a local environment.

Using docker compose command

docker compose up --build

Database unique IDs

Generating and using unique IDs is a really important desicion when starting a new project and its most common use is as primary keys for database tables. This project uses a custom UUID7 Draft04 implementation to make it simple to use and take advantage of UUID type of PostgreSQL. UUID7 combines timestamp with random data in order to help to convert data into time-stamped sequencially. If you are looking for another alternatives for tables IDs like Snowflakes, ULID, KSUID, pushID, xid among others you can check these references.

Code Style

To ensure a standardized code style this project uses black and ruff. If you want to change the config rules you can edit both ruff and black rules in the pyproject.toml file.

To reformat files execute the following command

make formatter

To run lint, you can run the following command:

make lint

To run lint in watch mode, you can run the following command:

make lint-watch

To run lint and try to fix the errors, you can run the following command:

make lint-fix

SonarQube static analysis

SonarQube is an automatic code review tool that detects bugs, vulnerabilities, and code smells in a project. You can read this post in order to have a better understanding about what SonarQube can do.

The following steps can help you to run a local static code analysis

1. Start SonarQube container

make run-sonarqube

The above code starts SonarQube at localhost:9000. You can login using this credentials -> username: admin and password: admin, after that it should requiere you change your password.

2. Add new project

5. Copy projectKey and login and replace on backend/sonar-project.properties file.

backend/sonar-project.properties file

# Organization and project keys are displayed in the right sidebar of the project homepage
sonar.organization=my_organization
sonar.projectKey=fastapi-alembic-sqlmodel-async
sonar.host.url=http://host.docker.internal:9000
sonar.login=157cc42f5b2702f470af3466610eebf38551fdd7

# --- optional properties ---

# defaults to project key
sonar.projectName=fastapi-alembic-sqlmodel-async
# defaults to 'not provided'
sonar.projectVersion=1.0

# Path is relative to the sonar-project.properties file. Defaults to .
sonar.sources=app

# Encoding of the source code. Default is default system encoding
sonar.sourceEncoding=UTF-8

6. Run the following command to execute a new code scan

make run-sonar-scanner

When the build is successful, you can see the SonarQube screen automatically refreshed with the analysis. If you want to export a report, you can check this this post.

Testing

Testing in FastAPI with pytest involves creating test functions that simulate HTTP requests to the API endpoints and verifying the responses. This approach allows us to conduct both unit tests for individual functions and integration tests for the entire application.

To perform tests in this project, we utilize two essential libraries: pytest and pytest-asyncio.

However, when testing FastAPI endpoints that utilize async connections with the database and a pool strategy, there is a trick to be aware of. The recommended approach is to create an isolated testing environment that connects to the database using the "poolclass": NullPool parameter on the engine. This helps to avoid potential issues related to tasks being attached to different loops. For more details on this, you can refer to the following references: Fastapi testing RuntimeError: Task attached to a different loop and Connection Pooling.

To execute the tests, follow these steps:

1. Start the testing environment using the command:

make run-test

2. Once the testing environment is up and running, open another terminal and run the tests with the following command:

make pytest

Inspiration and References

• full-stack-fastapi-postgresql.
• fastapi-sqlmodel-alembic.
• sqlmodel-tutorial.
• asyncer-tutorial.
• fastapi-pagination.
• fastapi-cache.
• fastapi-keycloak.
• fastapi-async-sqlalchemy.
• fastapi-minio.
• fastapi-best-practices.
• pgadmin Makefile.
• Styling and makefiles.
• awesome-fastapi.
• Serving ML Models in Production with FastAPI and Celery
• Database detup
• Dispatch

TODO List:

• Add Custom Response model

• Create sample one to many relationship

• Create sample many to many relationship

• Add JWT authentication

• Add Pagination

• Add User birthday field with timezone

• Add static server

• Add basic RBAC (Role base access control)

• Add sample heroes, teams and groups on init db

• Add cache configuration using fastapi-cache2 and redis

• Create a global database pool of sessions to avoid to pass the session as dependency injection on each handle

• Refactor tablename to Pascal case

• Add one to one relationship sample

• Add sample to upload images and store them using minio

• Invalidate access and refresh tokens when the password is changed using Redis

• Add shortcuts using a Makefile

• Add sample async, sync and concurrent functions using asyncer

• Add Black formatter and flake8 lint (Rasa as reference)

• Add static code analysis using SonarQube

• Function return type annotations to declare the response_model (fastapi > 0.89.0)

• Add export report api in csv/xlsx files using StreamingResponse

• Add production deployment orchestation using terraform + Elastic Beanstalk - AWS

• Add Github actions automation for deploy on Elastic Beanstalk - AWS

• Database query optimization. Many-Many use "selectin" and One-One and One-Many use "joined" issue

• Add Enum sample column

• Add docstrings

• Install pg_trgm by code and add a query for smart search of users by name

• Upgrade typing (Compatible just with python > 3.10)

• Add sample transformers NLP models and use them globally

• Add Celery samples for tasks, and schedule tasks

• Migrate from traefik reverse proxy to Caddy reverse proxy for automatic ssl

• Add fastapi limiter to natural language endpoints

• Add websocket conneting with chatgpt

• Setup testing configuracion

• Add sample composition using pydantic

• Add a nextjs sample frontend

• Add testing

• Add jsonb field on table sample

• Make that celery-sqlalchemy-scheduler works async

• Add AuthZ using oso

• Add SSL to reverse proxy on prod

• Add instructions on doc for production deployment using github actions and dockerhub (CI/CD)

• Convert repo into template using cookiecutter

PR are welcome ❤️

License

• This project is licensed under the terms of the MIT license