Issues
- 6
Is it possible to extract broken links from the crawl?
#175 opened by metsis - 3
Already crawled URL attempted as % encoded
#172 opened by apsaltis - 1
Running with decentralized feature
#171 opened by zmedelis - 7
Is it possible to dynamicall add links to crawl?
#170 opened by oiwn - 1
Chrome flag chrome_intercept page hang.
#168 opened by j-mendez - 17
- 11
Some pages have 0 bytes from scraped page. After rerunning, different pages have 0 bytes
#165 opened by esemeniuc - 4
Support ignoring SSL errors
#162 opened by superkelvint - 8
Extracting all urls on a page
#160 opened by apsaltis - 11
- 2
Scraping timeout Issue
#158 opened by virajk31 - 4
- 3
- 2
`with_on_link_find_callback` doesn't exist
#145 opened by SamuelMarks - 1
Extract text from Html
#141 opened by MihirModi1421 - 4
only let me spider one url
#138 opened by sebs - 1
cli parameters
#139 opened by sebs - 2
cli tutorial store crawls result as json
#134 opened by sebs - 4
Getting URL after redirect
#127 opened by joksas - 1
error[E0061]: this function takes 2 arguments but 1 argument was supplied
#136 opened by roniemartinez - 1
Add the ability to download not only html, but also all site assets: css, js, imgs, etc
#132 opened by namen3645 - 6
full-resource feature seems to be missing Javascript
#130 opened by Byter09 - 2
Blacklist regex for CLI does not seem to work
#129 opened by Byter09 - 6
Change API to builder pattern
#115 opened by roniemartinez - 2
[Feature request] Sitemap
#114 opened by j-mendez - 5
- 3
- 11
- 7
[Feature request] URL Globbing
#111 opened by roniemartinez - 18
- 1
- 2
Should make the media selector more configurable?
#104 opened by zishon - 2
Async runtime
#21 opened by j-mendez - 2
- 2
- 3
Get a list of images and their alt text
#79 opened by mgifford - 3
- 8
Add better documentation to get started
#78 opened by mgifford - 2
Trailing slash appended on link visited
#26 opened by j-mendez - 2
Allow subdomain crawling
#48 opened by j-mendez - 1
CLI -d flag is duplicated
#58 opened by thesurlydev - 1
Use version from Cargot.toml for user agent
#11 opened by madeindjs - 3
Blacklist entire url tree?
#32 opened by quietlychris - 1
Remove `.DS_Store` and add it to `.gitignore`
#35 opened by madeindjs - 5
Add --help option to spider_cli
#33 opened by quietlychris - 3
spider_cli doesn't install using cargo
#28 opened by quietlychris - 3
crawling nested anchors are not found
#2 opened by j-mendez - 2