Pinned issues
Issues
- 0
--parserobots doesn't seem to work
#93 opened by varna9000 - 0
Client-side rendered SPAs don't work
#91 opened by Garrett-R - 0
iframe contents ignored
#90 opened by Garrett-R - 1
Alphabetize URLs
#85 opened by Garrett-R - 2
Option to specify bot for robots.txt
#87 opened by Garrett-R - 0
How to exclude noindex page
#86 opened by damarkuzz - 0
crawling depth setting
#84 opened by styxiik - 2
How to add hreflang tags
#83 opened by crossmania337 - 2
Windows and/or Python 3.7.2?
#82 opened by VVilku - 2
urls not saved to sitemap.xml
#80 opened by crossmania337 - 5
Add package to PyPI
#64 opened by Garrett-R - 0
URL UnicodeEncodeError
#79 opened by wkingnet - 4
- 0
Exclude canonicalized pages
#78 opened by Spidle - 5
Python 3.9.6 support? SyntaxError
#76 opened by PeterWoelfel - 4
Handling more than 50,000 URLs
#59 opened by jswilson - 0
- 6
- 7
No URLs found
#57 opened by exportio - 1
Stop and continue
#58 opened by ishandutta2007 - 1
sintax error
#53 opened by francisco-baptista - 2
Change name project
#54 opened by cavazquez - 5
Limit search to path instead of domain?
#50 opened by 1kastner - 0
Image Licence
#29 opened by c4software - 1
Working with Angular sites?
#44 opened by tspicer - 3
Error: No space left on device
#46 opened by FranciscoPaixao - 2
Relative URLs are parsed incorrectly
#48 opened - 2
Please move the project away from GitHub
#47 opened - 2
Video sitemap
#45 opened by FranciscoPaixao - 3
Online interface for the script
#39 opened - 1
Adding trailing '/' to all URLs
#43 opened by dgursh - 0
- 3
Double entry if 2 slashes in the url
#38 opened - 1
- 0
- 1
Endless loop fix
#34 opened - 0
Crawl fails to find one page
#36 opened - 0
- 4
- 3
Duplicate entry
#32 opened - 0
Improvement proposal: video support
#31 opened - 1
IMG Data URI and image license
#26 opened - 0
Read the image title / alt
#30 opened by c4software - 2
Tracker images are included
#27 opened - 3
Add options to pretty print the output XML
#28 opened by c4software - 3
Slash missing in URL
#25 opened - 22
Image Sitemap?
#22 opened by wernerb90 - 2
HTTPS urls
#23 opened by wernerb90 - 3
Crawler.py giving error
#19 opened by KKS161994 - 4
Question about Sitemap
#15 opened by cocojambo89