Update all users

[ovv]

Add arguments to update all users and filter deleted users in the user store

async def all(self, fetch=False, deleted=False)

Query from db if not fetch else query slack api with await self._client.get_users() and update the db.

It should also add a deleted boolean in the user table of the database to permit easy query of non deleted user.

[ovv]

pyslackers/sirbot-pythondev#37

[nik849]

@ovv, I'm working on this, will querying from the db and the slack api return the same format? I was thinking something like this:
`async def all(self, fetch=False, deleted=False):
db = self._facades.get('database')
if fetch:
data = await self._client.get_users()

    else:
        data = await database.__dict__[db.type].user.get_all(db)

    return [
        User(
            id_=raw_data['id'],
            raw=raw_data['raw'],
            last_update=raw_data['last_update'],
            dm_id=raw_data['dm_id']
            deleted=raw_data['deleted']
        ) for raw_data in data
    ]`

And then under if fetch: add update the database. Does this make sense?

[ovv]

the raw column (raw_data['raw']) of the db should be exactly what slack replied.

[nik849]

great, so I could do deleted=raw['deleted']

[ovv]

For information response from SlackHQ about a deleted parameters for users.list

Hi Quentin,

Thanks for writing to us! There are no current plans to add this sort of filtering to the users.list method but I see how it would make getting data from Slack simpler. I will pass the suggestion to the product team, appreciate you taking the time to share your thoughts.

Wish I had a better answer for you. If you have other questions, feel free to ask.

Best,
Wojtek.

[nik849]

@ovv, I'm not quite clear on updating the db - should I check if the users fetched are all in the db, and if not use _add? Seems not that effiecient, but I can't think of another way right now.

[ovv]

That's the idea.

But we should create an add_multiple method for the db. I don't think sqlite provide a bulk update functionality so we would need to loop over the add.

It might not be super efficient with SQLite for big teams but we will see

[nik849]

Ok, thats answered that - i'll create a new method for multiple user addition and call it from all We might have to refresh the db at somepoint as previously, it won't have the deleted parameter.

[ovv]

There is a way to update the database: here and we would also need to add the new column at initialization of the db: here

[nik849]

@ovv, this is where I am - not sure if I can compare the two dbs like this?
`async def all(self, fetch=False, deleted=False):

        db = self._facades.get('database')
        db_data = await database.__dict__[db.type].user.get_all(db)
        if fetch:
            fetched_data = await self._client.get_users()
            users_to_add = []
            for user in fetched_data:
               if user['id'] not in db_data.keys():
                     users_to_add.append(user)
            self._add_multiple(users_to_add)
            data = fetched_data
        else:
            data = db_data
        return [
            User(
                id_=raw_data['id'],
                raw=raw_data['raw'],
                last_update=raw_data['last_update'],
                dm_id=raw_data['dm_id']
                deleted=raw['deleted']
            ) for raw_data in data
        ]`

[ovv]

A few points

• I think it would be best to update them regardless if they are in or not. They can have the same ID but not the same profile info for example.
• self._add_multiple(users_to_add) should be in the database (sqlite) directory as others db could provide bulk update
• The add method in the db should take a User object and not a dict
• We will also need to update the User object for the deleted key https://github.com/pyslackers/sirbot-slack/blob/master/sirbot/slack/store/user.py#L11

Anyways thank for hanging in there @nik849 :)