user discord online status is not checking everyone

[climu]

When we scrape, we update discord users online status:

openstudyroom/league/views.py

Lines 88 to 96 in d1979e1

	#finally discord
	#first we set everyone to offline
	DiscordUser.objects.all().update(status='offline')
	discord_users = requests.get('https://discordapp.com/api/guilds/287487891003932672/widget.json').json()['members']
	for user in discord_users:
	discord_user = DiscordUser.objects.filter(uid=user['id']).first()
	if discord_user is not None:
	discord_user.status = user['status']
	discord_user.save()

Then we display it like that:

I just realised that this url only show first 100 users: https://discordapp.com/api/guilds/287487891003932672/widget.json

We should find another way to check who is online.

[climu]

So I read this: https://discord.com/developers/docs/resources/guild#guild-object-guild-structure.
I try:
curl -H 'Authorization: Bot MYBOTTOKEN' https://discordapp.com/api/guilds/287487891003932672/members?/
but it only returns one user.
I try
curl -H 'Authorization: Bot MYBOTTOKEN' https://discordapp.com/api/guilds/287487891003932672/presences?/
and get a 404 :(

[raylu]

the question mark after the field means it's optional. those are fields in the responses to /api/guilds/:id, not the member list. I think it's optional because the websocket gateway won't send it to you unless you opt your bot into the presence privileged intent

[climu]

Thanks for clarification. I still couldn't manage to get the wanted information from discord api.

[lime-green]

@climu @raylu

OK I looked into this today:

• There's no HTTP endpoint for this discord/discord-api-docs#666 (although they created a websocket endpoint, which isn't that useful)
• There's a PRESENCE_UPDATE event that gets sent that updates the member's status as expected

so I propose:

1. Create a new API endpoint /api/discord/online_statuses/ or similar that saves a user's status if they exist in the DB
2. In the discord bot, create a cog (https://discordpy.readthedocs.io/en/latest/ext/tasks/) that runs the following task every ~5 minutes:

• gathers all the members in bot: bot.get_guild(guild_id).members
• builds a status payload of the form: [{user_id: "...", status: "online|offline|etc."}] using member.status
• POST to the endpoint

EDIT:
We could also listen to the event and only send users who’s statuses have changed as a performance improvement

[climu]

Thanks for the investigation!
I think this can work.

I have one security wonder: The API endpoint would change our database and I am reluctant to open that to everyone.

Maybe we could do a trick: The bot read a text file /etc/discord_bot_password containing a random string and pass it to the POST request. Then in the view we can open the very same file and check that it matches.

Is my security concern not relevant? Is the suggested security check ok?

[raylu]

the bot and the db are on the same server. no need to post

I think this is pretty overkill and I would personally settle for the discord widget, though

[climu]

the bot and the db are on the same server. no need to post

You mean we could connect the bot to OSR db directly? Isn't it tricky without django ORM?

I think this is pretty overkill and I would personally settle for the discord widget, though

Well discord widget would look like : https://discord.com/widget?id=287487891003932672&theme=light

It just show a global activity count and we have this working allready:

The idea here is to show users online status on users links. Is it really needed? it's not clear indeed.

[raylu]

the bot and the db are on the same server. no need to post

You mean we could connect the bot to OSR db directly? Isn't it tricky without django ORM?

just install psycopg2 in the bot's venv