[FEATURE] Exclude Page from crawling

[tomasnorre]

Semilar to exclude from search, a page setting "exclude from crawling" could be added.
This would ease the setup of what needs to be crawled.

Today you would have to add it to a crawler configuration.

Some things to consider here, is that Page A, might be included for Crawler configuration "default", but not for "extended".