Don't fetch full details for channels user is not a member of

[danvratil]

It will massively speed up joining large teams.