Data retrieval pipeline
Closed this issue · 4 comments
Madhour commented
- Generate Bing Query using advanced search operators (e.g.,
filetype:pdf site:apple.com
)- Implement Browsing through search results (limit to first n pages)
- Download PDFs inside links (results)
- Run downloaded PDFs through PDF-Miner
Madhour commented
- Created Class for enumerating files + links (3fc8c1d)
- Currently search is limited to first 50 pages max
BennerLukas commented
OWDSC commented
done