Completed the backend heavy assignment and completed the requirement given by sendx.
- Requirements:
- Pages at times may not be available so you may have to retry The application may have paying customers and non paying customers. So your server needs to always give priority for crawling to paying customers. For simplicity you can differentiate between paying and non paying customers via query parameter being passed to the backend from the frontend API call.Using go colly in golang the project will crawl the url and fetch the required details.
- Ensuring that sometimes Pages may not be available so it will retry 3 times or for paying customers 5 times. Priority is given to the paying customers is done with the help of priority_queue which will put the paying customers to the top in the queue.
To run the web app on your local computer, clone this repository-
1.Open the terminal in the client directory and run the following command :
go mod tidy
go run main.go