Fetcher
This service allows to execute HTTP requests to other resources in the worker pool and store results.
API Endpoints:
POST /task/fetch allow the service to perform an HTTP request to a specific resource.
For example:
curl -X POST \
http://localhost:8080/task/fetch \
-H 'Content-Type: application/json' \
-d '{
"method": "GET",
"address": "http://ozon.ru/context/detail/id/200",
"headers": {
"Content-Type": "text/html;charset=windows-1251",
"Allow": "GET",
"Content-Length": "1984"
},
"body": "some text"
}'
All requests are performed by a special pool worker in the goroutines. Pool worker settings (number of workers, buffer size) located in pkg/services/pool/fetch_pool.go.
GET /task/{id} returns task by id.
DELETE /task/{id} removes task by id.
GET /tasks returns all tasks from storage.
GET /tasks/page/{id} returns tasks by pages
The value of the count of tasks per one page is stored in the TasksPerPageCount in pkg/settings/task.go.