allinurl/goaccess

Differentiate the Fediverse from Crawlers

Opened this issue · 0 comments

I run mastodon.radio, it's a mastodon server and around half of the traffic goAccess identifies as "Crawlers".

However, a lot of that traffic is the Fediverse doing its thing, fetching posts from the server for their users, or apps connecting to the server, etc.

It would be great to have a new category to differentiate traditional crawlers from legitimate fediverse traffic.

ActivityPub, HomeTown, and Lemmy, alongside Googlebot