BackEnd?
flxwu opened this issue · 40 comments
Subsequently to #19 and the things we talked about in 30-seconds-of-code, I am thinking about adding a BackEnd. I mean, we don't have to give it full functionality, but we could for sure just set it up and maybe use it to easily try out things like voting. Maybe you guys already discussed it, but What do you think?
We would, of course, thus have to put it on i.e. heroku, but that's not a big deal.
As I mentioned the whole syncing things in #19 - There're going to be some problems.
Obvious points:
- We will need some kind of unique id to store each question.
- The whole API is a joke, 2 routes are enough and it will do the job.
The following is going to be a problem:
- When someone PR's a new question that question that question goes into
questionsfolder. The Travis automatically buildsreadme.mdfile and website. Somehow the server would need to pick up the changes from thequestionsfolder and add a new instance to our mini-database.
As far as I understood, we would like to have the server as the separate project?
This is definitely doable and completely easy peasy once we figure out how to identify questions and sync them with our database when someone makes a new PR.
NOTE: Also someone mentioned that is so mainstream. Everyone does it and it is not so fancy anymore. @atomiks had a fancy idea I discussed yesterday with all of you but no one was really really interested in doing that. @Chalarangelo has a point too, that was a good comment.
Any ideas? I don't really know what to with this really 😟
As far as uniquely identifying snippets, we should use hashing on either the name of the question, the whole question or something similar. Filenames as unique identifiers could work, too. I mean all of our scripting is based on them being unique anyways.
fair enough there with the file names. Plus if we use another entities OAuth system and just log identities we could make it easily stored in a NoSQL data base with a few tables: One by question with the number of upvotes, and one by user with what questions they've voted on?
If that's the case this is a perfect use case for server less deployment
there is also another way to do it: user identification is used to reference what they've voted on, and we keep a single flat file for the votes. Basically we authenticate the votes, but keep a single version around so as to not need multiple tables and instead can get away with file storage costs instead of lots of reads should we get a lot of questions... Plus this way would allow us to namespace things and make it easier to use the same backend and authentication for OTHER 30* projects and voting
cough https://aws.amazon.com/free/
No I don't like AWS at all... what are you talking about...
the big problem with a flat file though is hosting costs of the file. While it'd probably be cheap( ((identifier+number+:+("2)) + , )#ofQuestions + { + } ) it does mean there'd be some real world costs... I'm curious if there would be a way for us to use other free services they offer to hold hte file? Time to dive.
ALSO other back end suggestions would be perfectly awesome too, I'm just drawing on my prior experience
Heroku for server / hosting
Express/Node back-end
MongoDB / mLab for vote storage
Use IP to judge uniqueness
I've done this before for a project and it works fine.
I sadly work in an environment where IP would not work as it'd be X votes for around 75 people. This especially limits it from dynamic external IPs(common practice) and public libraries
Also I'm not certain we'd want express since this is something considered a "low traffic" application. At that point it would be better to use a serverless where we run on invocation
We could allow up to 3 votes on a single item per IP. I doubt more than that in the same building is going to be using/voting on the site.
I suggested express/heroku because it's super easy to set up and free while it's low traffic so no problem.
To be honest, I agree more with @atomiks on this one. This should be as simple as possible.
I doubt more than that in the same building is going to be using/voting on the site.
This is totally true. We're not building social media platform to target 2B people
Heroku and express seem pretty easy to set up. We could go with that setup and be ready by the end of the week.
What do you think?
I sort of disagree with the assumption that multiple people in the same building will not vote. If a page like this is popular, especially in its early days when traffic is really high, people will share it with co-workers and there is a high chance people could vote from the same IP. Could we use some way to differentiate them, maybe a cookie or somesuch? Even if it's exploitable, that would still be better than binding votes to IPs.
NAT defeats the IP based idea in concept if this would gain any sort of traction
Why not bind it to mac addresses? 🤔
@atomiks As you are mentioning MongoDB for data storage: What is your opinion on implementing realtime-voting (once we've clarified the other things)? So that every client is automatically served by any updates to the database. I have worked with pusher and socket.io before, and they're both really easy and nice to use.
I'd recomend staying away form Websockets since we don't know if they'll be a supported technology in the future. We should use Server Side Events if we want to do any sort of subscription model since then we only have a one way push and don't have to worry about ingestion logic competing with subscription logic(in my testing I was able to support 582 people a thread in 100 MB of memory using Express as the request handler)
it does mean duplexing is achieved through POST requests to specific topic routes, but oh no... we have extremely clear cut application logic that isn't confusing or trying to do to much... Darn
And it makes multiplexing it easier since then all we need is each server to connect to a message bus
I'm really enjoying Hapi right now. The swagger plugin and joi validation are pretty sweet.
@imcodingideas I just found out about Hapi a few days ago. It seems really cool and interesting (and kinda less complex than Express). Might be worth looking into.
@Chalarangelo express is complex? I always thought express was so un-opinionated that if you didn't know what you were doing you'd quickly go down in flames. I'm not such a fan of Hapi though due to it's really confusing changing returns and awkward handler registration.
The difference is that Hapi is more opinionated which means a lot of things will work, but only work if you do it the way Hapi says they should be done.
Express is more a raw request handler, letting you do whatever you want, but you have to do it really hard, almost on a per route basis
you have to do it really hard, almost on a per route basis
@skatcat31 you just described my experience with Express. Which is why I think hapi can be easier to start with (or anything a bit more opinionated). Working on something as barebones as Express is cool, but it takes a lot of work if you are not super familiar with it - I'm not.
@Chalarangelo an example of my express autoloader
I have a LOT of experience with Express and scaffolding it in such ways that makes it extremely extendable, and easy, composable, and contained.
As an example, this is what my app.js files usually look like:
const app = require('simpleexpress')(folder);This crawls the folder and looks for three sub folders:
- routes
- plugins
- middleware
It then loads them into the application. I do need to document it better but I haven't had a chance too...
To be fair it could use an update so that it will use either the arguments form the commandline, or environment variables...
And to be fairer saying I don't have time is basically me not really remembering too
Yeah, both could be merged somehow.
Are we still looking at this?
A week ago DigitalOcean agreed to be a sponsor of our project. We got one droplet that will be used to host backend.
So yes, this is still open. I wish we can start implementing this these days but time is tough.
we can all agree on node/Express right?
we can all agree on node/Express right?
yes I can agree on that
Me too. Seems like a really good option. 👍
Yep, I agree as it's competent and I am familiar with that stack.
the question is how do we footprint each individual user so as to keep votes as non game-able as possible?
Two viable options from where I stand:
- IP + browser data to approximate users and use a cookie to store returning visitors, make for a more inclusive system but can be gamed under circumstances.
- Use a 3rd party authorization system, preferrably GitHub, less game-able, but less inclusive, too.
A combination of the two would also be possible, but we will have to weigh votes and build a system to detect fraud.
however let's just start and deal with that later
@flxwu A simple IP+token based implementation sounds like a good start. After all, all of the 30s projects are based on trial and error and we have goofed in a lot of places before. This won't cause any irreversible damage even if the system is abused to death, as we can always wipe the slate clean and start over with the voting. So let's get started on that.
IP won't work with so much NAT in the real world. As a point in case yesterday my IP address was in the 20.XXX.YYY block and today it's in the 29, and it hasn't actually been 24 hours.
@skatcat31 That's a decent point, my IP changes a lot plus I work in a building with 100+ computers all under the same IP. But we cannot access MAC address in any way or anything else that feels worthwhile, so the only alternative would be to use GitHub's authorization system and only accept verified users. Not a terrible choice, but might cause vote counts to be lower than what we might want. Although, I kinda like this idea, as it will be better for community-building in the long run.
we could fingerprint: https://arstechnica.com/information-technology/2017/02/now-sites-can-fingerprint-you-online-even-when-you-use-multiple-browsers/
however I like the idea of GitHub integration so that only verified users can vote. This will give us a repeatable and verifiable way to keep track with an already secure login system 😉
I like the Github idea very much. That way only real developers could vote 😄
NOTE: I got a small surprise comming very soon to this project. Expect new category, 50 questions.
I am closing this due to inactivity. If we ever think about backend again, feel free to reopen this issue.
Also to mention we now support React questions as well 👍