This project provides an implementation of a backend for an Instant Messaging (IM) system, developed in the Rust programming language, with a microservice architecture design. Its primary goal is to offer developers an efficient, stable, and scalable IM solution. The focus of this project is to explore and demonstrate the potential and practices of Rust in building high-performance backend services.
- Microservice Architecture: The system is split into multiple independent service units, each responsible for a portion of the core business logic and communication with other services.
- Containerized Deployment: All services can be packaged with Docker, facilitating deployment and management.
- Asynchronous Processing: Utilizes Rust's asynchronous programming capabilities to handle concurrent workloads, enhancing performance and throughput.
- Data Storage: Uses PostgreSQL and MongoDB for storing messages permanently and for inbox functionalities, respectively.
- Message Queue: Leverages Kafka as a message queue to support high concurrency message pushing and processing.
-
Service Layer
-
Authentication Service: Handles user registration, login, and verification.
-
Message Service: Responsible for message sending, receiving, and forwarding.
-
Friend Service: Manages the user's friends list and status.
-
Group Service: Takes care of group creation, message broadcasting, and member management.
-
-
Data Storage Layer
- PostgreSQL: Storing user information, friendship relations, and message history, along with automated archival through scheduled tasks.
- MongoDB: Acts as a message inbox, handling offline message storage and retrieval.
-
Middleware Layer
- Kafka: Provides a high-throughput message queue to decouple services.
- Redis: Implements caching and maintains message status to optimize database load.
-
Infrastructure Layer
- Docker and Docker-Compose: Containers for building and deploying services.
- Synapse: For service registration and discovery.
- MinIO: An object storage solution for handling file uploads and downloads.
The project is designed with high performance and horizontal scalability in mind. Through asynchronous processing and a microservice architecture, the system is capable of scaling effectively by increasing the number of service instances in response to the growing load. Additionally, the project adopts a modular design philosophy that allows developers to customize or replace modules as needed.
- Integrating Member ID Retrieval from Cache into DB Service: Whether the method for retrieving member IDs from the cache should be integrated into the DB service is under consideration.
- Friendship Redesign: The current design for representing friendships is inadequate and requires a thorough redesign. --rebuilding
- Conversation Feature: There is currently no implementation of conversations on the server-side, as it exists only client-side.
- Partition Table for Messages (Mongodb) Not Implemented: The strategy for implementing partitioned tables for messages has not been realized yet.
- User Table Should Add Login Device Field: There should be consideration to add a field for the login device to the user table, which is used to check if clients need to sync the friend list.
- Friendship Read Status: we should delete the Friendship related message after user read it.
- need to handle friendship/group operations while user desktop and mobile are both online
- knock off desk from the mobile
- delete minio file by period
- support matrix protocol so that we can import some robot
- maybe we should combine the query send_seq and incr recv_seq into one operation with lua
-
install
librdkafka
Ubuntu:
apt install librdkafka-dev
Windows:
# install vcpkg git clone https://github.com/microsoft/vcpkg cd vcpkg .\bootstrap-vcpkg.bat # Install librdkafka vcpkg install librdkafka .\vcpkg integrate install
-
run docker compose
docker-compose up -d
important: make sure all the third service are running in docker.
-
install sqlx-cli and init the database
cargo install sqlx-cli sqlx migrate run
-
clone the project
git clone https://github.com/Xu-Mj/sandcat-backend.git cd sandcat-backend
-
build
-
Linux: use the static feature
cargo build --release --features=static
-
Windows: use the dynamic feature
cargo build --release --features=dynamic
-
-
copy the binary file to root path
cp target/release/cmd ./sandcat
-
run
./sandcat
if you need adjust some configuration, please modify the
config.yml
important: Given that our working environment may differ, should you encounter any errors during your deployment, please do let me know. Together, we'll work towards finding a solution.