DFK crawler fetches web pages and extracts structured data from them, according to some conditions. So that it can be stored in a database and manipulated as desired.
It can be used in many ways for data mining or information processing.
The actual use case can be grabbing list of products on several pages and follow each product’s main page to retrieve additional information. The application returns information as a JSON, XML or CSV data.
