Lemmy Explorer (https://lemmyverse.net/)

This project provides a simple way to explore Lemmy Instances and Communities.

The project consists of three modules:

Crawler (NodeJS, Redis)
Frontend (ReactJS, MUI Joy, TanStack)
Deploy (Amazon CDK v2)

FAQ

How does discovery work?

It uses a seed list of communities and scans the equivalent of the /instances federation lists, and then creates jobs to scan each of those servers.

How long till my instance shows up?

How long it takes to discover a new instance can vary depending on if you post content that's picked up by one of these servers.

Since the crawler looks at lists of federated instances, we can't discover instances that aren't on those lists.

Additionally, the lists are cached for 24 hours, so it can take up to 24 hours for an instance to show up after it's been discovered till it shows up.

Can I use your data in my app/website/project?

I do not own any of the data retrieved by the cralwer, it is available from public endpoints on the source instances.

Please don't hotlink the files on the public website, I'd prefer if you don't pull data from there directly in scripts.

Currently, I upload a Redis dump generated by the crawler each night to s3, GitLab builds the JSON dump from that.

Data is available from the artifacts of this Action. You can also download Latest ZIP (using nightly.link)

dist-json-bundle.zip file contains the data in JSON format:

communities.full.json - list of all communities

instances.full.json - list of all instances

overview.json - metadata and counts

Crawler

Crawler README

2. Frontend

Frontend README

Deploy

The deploy is an Amazon CDK v2 project that deploys the crawler and frontend to AWS.

config.example.json has the configuration for the deploy.

then run cdk deploy --all to deploy the frontend to AWS.

Similar Sites

Lemmy Stats Pages

Thanks / Related Lemmy Tools

Credits

Logo made by Andy Cuccaro (@andycuccaro) under the CC-BY-SA 4.0 license.

Poolitzer/lemmy-explorer