Application in this repository is developed using the FastStream
framework. Below, you'll find a guide on how to get started, develop new features or bug fixes, and ensure the quality of your code through testing and linting, run the FastStream
application locally, and view AsyncAPI
documentation.
To set up your development environment, follow these steps:
-
Clone this repository to your local machine:
git clone https://github.com/<username>/<repo-name>.git cd <repo-name>
NOTE: Replace
<username>
with your GitHub username and<repo-name>
with the name of your repository. -
Install all development requirements using pip:
pip install -e ".[dev]"
The application code is located in the app/
directory. You can add new features or fix bugs in this directory. However, remember that code changes must be accompanied by corresponding updates to the tests located in the tests/
directory.
Once you have updated tests, you can execute the tests using pytest
:
pytest
To run the FastStream
application locally, follow these steps:
-
Start the Kafka Docker container locally using the provided script:
./scripts/start_kafka_broker_locally.sh
-
Start the
FastStream
application with the following command:faststream run app.application:app --workers 1
-
You can now send messages to the Kafka topic and can test the application. Optionally, if you want to view messages in a topic, you can subscribe to it using the provided script:
./scripts/subscribe_to_kafka_broker_locally.sh <topic_name>
-
To stop the
FastStream
application, pressCtrl+C
. -
Finally, stop the Kafka Docker container by running the script:
./scripts/stop_kafka_broker_locally.sh
If you'd like to build and test the Docker
image locally, follow these steps:
-
Run the provided script to build the
Docker
image locally. Use the following command:./scripts/build_docker.sh <username> <repo-name>
This script will build the
Docker
image locally with the same name as the one built inCI
. -
Before starting the
Docker
container, ensure that a KafkaDocker
container is running locally. You can start it using the provided script:./scripts/start_kafka_broker_locally.sh
-
Once Kafka is up and running, you can start the local
Docker
container using the following command:docker run --rm --name faststream-app --net=host ghcr.io/<username>/<repo-name>:latest
-
--rm
: This flag removes the container once it stops running, ensuring that it doesn't clutter your system with unused containers. -
--name faststream-app
: Assigns a name to the running container, in this case, "faststream-app". -
--net=host
: This flag allows theDocker
container to share the host's network namespace.
-
-
To stop the local
Docker
container, simply pressCtrl+C
in your terminal. -
Finally, stop the Kafka
Docker
container by running the provided script:./scripts/stop_kafka_broker_locally.sh
NOTE: Replace
<username>
with your GitHub username and<repo-name>
with the name of your repository in the above commands.
After making changes to the code, it's essential to ensure it adheres to coding standards. We provide a script to help you with code formatting and linting. Run the following script to automatically fix linting issues:
./scripts/lint.sh
Static analysis tools mypy
and bandit
can help identify potential issues in your code. To run static analysis, use the following script:
./scripts/static-analysis.sh
If there are any static analysis errors, resolve them in your code and rerun the script until it passes successfully.
FastStream
framework supports AsyncAPI
documentation. To ensure that your changes are reflected in the AsyncAPI
documentation, follow these steps:
-
Run the following command to view the
AsyncAPI
documentation:faststream docs serve app.application:app
This command builds the
AsyncAPI
specification file, generatesAsyncAPI
documentation based on the specification, and serves it atlocalhost:8000
. -
Open your web browser and navigate to http://localhost:8000 to view the
AsyncAPI
documentation reflecting your changes. -
To stop the
AsyncAPI
documentation server, pressCtrl+C
.
Once you have successfully completed all the above steps, you are ready to contribute your changes:
-
Add and commit your changes:
git add . git commit -m "Your commit message"
-
Push your changes to GitHub:
git push origin your-branch
-
Create a merge request on GitHub.
This repository is equipped with GitHub Actions that automate static analysis and pytest in the CI pipeline. Even if you forget to perform any of the required steps, CI will catch any issues before merging your changes.
This repository has three workflows, each triggered when code is pushed:
-
Tests Workflow: This workflow is named "Tests" and consists of two jobs. The first job runs static analysis tools
mypy
andbandit
to identify potential issues in the codebase. The second job runs tests usingpytest
to ensure the functionality of the application. Both jobs run simultaneously to expedite theCI
process. -
Build Docker Image Workflow: This workflow is named "Build Docker Image" and has one job. In this job, a
Docker
image is built based on the provided Dockerfile. The built image is then pushed to the GitHub Container Registry, making it available for deployment or other purposes. -
Deploy FastStream AsyncAPI Docs Workflow: The final workflow is named "Deploy FastStream AsyncAPI Docs" and also consists of a single job. In this job, the
AsyncAPI
documentation is built from the specification, and the resulting documentation is deployed to GitHub Pages. This allows for easy access and sharing of theAsyncAPI
documentation with the project's stakeholders.
After the Deploy FastStream AsyncAPI Docs workflow in CI
has been successfully completed, the AsyncAPI
documentation is automatically deployed to GitHub Pages. This provides a convenient way to access and share the documentation with project stakeholders.
To view the deployed AsyncAPI
documentation, open your web browser and navigate to the following URL:
https://<username>.github.io/<repo-name>/
NOTE: Replace
<username>
with your GitHub username and<repo-name>
with the name of your repository.
You will be directed to the GitHub Pages site where your AsyncAPI
documentation is hosted. This hosted documentation allows you to easily share your AsyncAPI
specifications with others and provides a centralized location for reviewing the AsyncAPI
documentation.
Once the Build Docker Image workflow in CI
has successfully completed, the built Docker
image is pushed to the GitHub Container Registry. You can then deploy this image on your server by following these steps:
-
Pull the
Docker
image from the GitHub Container Registry to your server using the following command:docker pull ghcr.io/<username>/<repo-name>:latest
NOTE: Replace
<username>
with your GitHub username and<repo-name>
with the name of your repository. -
After successfully pulling the image, start the
Docker
container using the following command:docker run --rm --name faststream-app --env-file /path/to/env-file ghcr.io/<username>/<repo-name>:latest
-
--rm
: This flag removes the container once it stops running, ensuring that it doesn't clutter your system with unused containers. -
--name faststream-app
: Assigns a name to the running container, in this case, "faststream-app". -
--env-file /path/to/env-file
: Specifies the path to an environment file (commonly a.env
file) that contains environment variables required by yourFastStream
application. Storing secrets and configuration in an environment file is a secure and best practice for handling sensitive information such as Kafka host, port, and authentication details.
-
By following these steps, you can easily deploy your FastStream
application as a Docker
container on your server. Remember to customize the env-file
and other environment variables as needed to suit your specific application requirements.
Happy coding with FastStream
Application! If you have any questions or encounter any problems, feel free to reach out to us. We appreciate your contributions and commitment to maintaining code quality.