MarginaliaSearch/MarginaliaSearch

(crawler) Rare NPE in sniffRootDocument

Closed this issue · 1 comments

ERROR CrawlerRetreiver- Error configuring link filter
java.lang.NullPointerException: Cannot invoke "nu.marginalia.bigstring.BigString.decode()" because "sample.documentBody" is null
        at nu.marginalia.crawl.retreival.CrawlerRetreiver.sniffRootDocument(CrawlerRetreiver.java:245) ~[crawling-process.jar:?]
        at nu.marginalia.crawl.retreival.CrawlerRetreiver.crawlDomain(CrawlerRetreiver.java:144) ~[crawling-process.jar:?]
        at nu.marginalia.crawl.retreival.CrawlerRetreiver.fetch(CrawlerRetreiver.java:99) ~[crawling-process.jar:?]
        at nu.marginalia.crawl.CrawlerMain.fetchDomain(CrawlerMain.java:121) ~[crawling-process.jar:?]
        at nu.marginalia.crawl.CrawlerMain.lambda$startCrawlTask$1(CrawlerMain.java:103) ~[crawling-process.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

Already fixed on master, ran old code.