Feature: Upload artifacts to Github Pages
Closed this issue · 10 comments
What do you want to see?
In your GitHub action, serve all the json artifacts via GitHub pages instead of just as zip artifacts in the actions.
Additional context
Since you understandably don't want people to hotlink the files from your public website, I'm making my own workflow that just downloads your artifacts and reuploads them to GitHub pages, so they can then be accessed freely. It would be even better if this could be done directly in your workflow, so that both the action artifact and GitHub page are updated simultaneously. I can open a PR for this if you're interested.
damn that's a good idea. do you have an example workflow? i have them building to assets already so I can copy them into pages I guess.
@tgxn I actually ended up just stealing your crawler code and using only the specific part I needed, and now I'm running the whole thing from within a GitHub workflow. I didn't feel like setting up my own server 😅
Here you can see me running the script with node and then publishing the output to GitHub Pages: https://github.com/Raicuparta/lemmy-community-crawler/blob/main/.github/workflows/deploy.yml
Which results in this: lemmy.raicuparta.com/communities.json
I am gonna give it a go!
I could just host the whole dang site on GH pages 🤔
Yep, you totally could. It's not the most efficient though. I guess as long as the whole crawler takes less than 6 hours to finish (free workflow limit), it should be ok.
I don't think I'll move the whole thing, I'll just add data. for the data...
https://data.lemmyverse.net/data/community.full.json
also made a simple browse thing
https://data.lemmyverse.net/
Beautiful. I guess you can consider this one resolved then :)
soon.
just building a simple ui .... make it nicer
@tgxn btw just thought you should know, the reason I wanted to use this was for a browser extension I'm making: https://github.com/Raicuparta/lemmy-go
I was initially using browser.feddit.de, but that one seemed either very outdated or missing a lot of large communities for some reason. I was about to make my own crawler from scratch, but I'm glad I found yours, saved me some precious time. So thanks!
@tgxn btw just thought you should know, the reason I wanted to use this was for a browser extension I'm making: https://github.com/Raicuparta/lemmy-go
I was initially using browser.feddit.de, but that one seemed either very outdated or missing a lot of large communities for some reason. I was about to make my own crawler from scratch, but I'm glad I found yours, saved me some precious time. So thanks!
Good to hear, maybe we should compile a list of apps that use our data.