Thesis: Information retrieval web application of MOOCs (mass open online courses) for IT specialties
Table of Contents
TechPath is a web platform developed to simplify searching for educational materials to study IT disciplines. Every person may find himself drowning in a vast variery of topics, paths and learning resources.
That is why this app may come in handy, helping to clear one's mind by suggesting hand-picked materials and roadmaps to learn a topic or even a whole profession.
The repo consists of 5 packages (apps):
- Client - react web app
- Server - primary endpoint for a client app
- Parser - web crawler that parses data from different resources (e.g., Coursera, Stepik) and stores the unprocessed data in a RabbitMQ message queue
- DataProcessor - python script that processes courses data from the message queue - performs data cleanup, determines languages and tags
- Shared - components (e.g., utils, types, constants)
System Architecture Diagram | GraphQL Schema Diagram |
---|---|
![]() |
![]() |
Blue color on architecture diagram describes a component that is to be added, red - a components that is not planned yet.
TypeScript
React
Mantine
GraphQL
Apollo
Vite
TypeScript
Nest.js
Apollo
Crawlee
Neo4j
RabbitMQ
Python
Pika
(RabbitMQ adapter)Neo4j
Fasttext
Transformers
Nx
(monorepo tool)Docker
Docker compose
A list of required software:
npm
/yarn
docker
python3
- Clone the repo
git clone git@github.com:FloydanTheBeast/diploma.git
- Install dependencies
npm i
## OR
yarn
- Copy
env.example
config file and fill it with desired values - Create and start docker containers
docker-compose up -d
- Serve NX package - server, client or parser
nx serve <server|client|parser>
- Generate GraphQL types based on a schema introspection for client or server
yarn gql-codegen:<client|server>
- Role-based JWT auth
- Parse and process courses data from different vendors
- Courses languages and tags classification
- GraphQL API with CRUD operations for every entity
- Data filtering, sorting, pagination and aggregation
- Global fuzzy search for courses and roadmaps
- Basic recommendation system
- Roadmaps serialization and deserealization to the format supported by the client
- Train own model to improve classification accuracy
- User lists for saved courses/roadmaps
- Personalized courses/roadmaps compilations
- Tag clustering by specializations
- Personalized collaborative recommendation system, user data collection
- Testing system for users to determine their preferences
- CI/CD deployment pipeline
- Basic JWT auth
- Role-based routing
- View courses, roadmaps and course vendors
- Table pagination, search, sort and filter
- Specific course page
- Global search
- Bookmarks for courses and roadmaps
- In-app reviews for courses and roadmaps
- View courses recommendations
- Visual roadmap building tool
- Improve roadmaps visually
- Combine several roadmaps into a series
- Use roadmaps as template
- Stats dashboard for admins
- View user profiles
- Guest access to the app
- Internationalization
- Adaptation for mobile devices
- CI/CD deployment pipeline
2022-2023 Graduate Work (Diploma) by Shadi Abdelsalam
Higher School of Economics, Faculty of Computer Science, Software Engineering Department