bug: 401 Unauthorized when using search query parameter
snavarro-factorial opened this issue · 7 comments
Describe the bug
Whenever I try to sync with with a search query I get this error:
Error: cannot sync groups and their members: error doing the first sync: error getting groups from the SCIM service: scim: error listing groups: aws checkHTTPResponse: error code: 401 Unauthorized, body:
To Reproduce
Steps to reproduce the behavior:
- Create a group called "SysAdmin" in Google Admin.
- Add some users there.
- Run
./idpscim [...] -q 'name:SysAdmin* email:sysadmin*'(I've tried too withname=SysAdminand many others as this doc says https://developers.google.com/admin-sdk/directory/v1/guides/search-groups) - That error appears.
Expected behavior
It actually syncs that group.
Additional context
If I run ./idpscim without the query parameter it actually starts reading some emails but then occurs the same error as in issue #49 so I can't find how to sync groups.
Also my Google account has full admin permissions, don't know if I'm missing something on the service account (which has domain-wide delegation permissions applied too and they work, since removing them leads to another error).
Thanks!
Hi @snavarro-factorial could you validate your Google Service Account created?, because looks like the Google email account associated with the Google Service Account doesn't have sufficient access.
that is why you are receiving ... error code: 401 Unauthorized ... as a part of your error message.
Please confirm that to close this issue
That was my first option, but I've checked and:
- I tested both with my email (superuser) and the Google Workspace owner email (and with owner permissions on the Google Service Account).
- I tried removing Domain-Wide Delegation, the error changes into this:
Error: cannot sync groups and their members: error getting groups from the identity provider: idp: error listing groups: Get "https://admin.googleapis.com/admin/directory/v1/groups?alt=json&customer=my_customer&fields=groups%28id%2Cname%2Cemail%2Cetag%29&prettyPrint=false&query=name%3DSysAdmin": oauth2: cannot fetch token: 401 Unauthorized
Response: {
"error": "unauthorized_client",
"error_description": "Client is unauthorized to retrieve access tokens using this method, or client not authorized for any of the scopes requested."
}
- As soon as I create the Domain-Wide Delegation again, I get the same 401 error.
My roles in Google Workspace are:
- Super Admin
- Groups Admin
- User Management Admin
- Help Desk Admin
- Services Admin
- Groups Reader
- Groups Editor
All without any restricted condition, so I suppose the problem is not with the Google Workspace account, but with the Google Service Account?
I did the steps that were needed for the previous (abandoned) project --> https://github.com/awslabs/ssosync#google
Just in case that helps somehow :/
Thanks a lot!
hi @snavarro-factorial , thank you for the details in your issue.
Important things here, the problem is with the Google Workspace Service Account, if you see that last part of your error message ...oauth2: cannot fetch token: 401 Unauthorized ...
I would like to help you because looks like issue #49 are the same problem, which is the configuration of credentials for Google Workspace.
To help both of you, could you try the following?
Instead of using idpscim let's try with idpscimcli and let me know your output whithout sensible data
idpscimcli --help
This is a Command-Line Interfaced (CLI) to help you validate and check your source and target Single Sing-On endpoints.
Check your AWS Single Sign-On (SSO) / Google Workspace Groups users and groups and validate your filters over Google Workspace users and groups.
Usage:
idpscimcli [command]
Available Commands:
aws AWS SSO SCIM commands
completion Generate the autocompletion script for the specified shell
gws Google Workspace commands
help Help about any command
Flags:
-c, --config-file string configuration file (default ".idpscim.yaml")
-d, --debug enable log debug level
-h, --help help for idpscimcli
-f, --log-format string set the log format (default "text")
-l, --log-level string set the log level (default "info")
--output-format string output format (json|yaml) (default "json")
--timeout duration requests timeout (default 10s)
-v, --version version for idpscimcli
Use "idpscimcli [command] --help" for more information about a command.Get Google Workspace-> users list
idpscimcli gws users list \
--gws-service-account-file <location of your google workspace service account json file> \
--gws-user-email <the google workspace email associate the to service account> Get Google Workspace-> groups list
idpscimcli gws users list \
--gws-service-account-file <location of your google workspace service account json file> \
--gws-user-email <the google workspace email associate the to service account> In both option you also can use filters:
...
--gws-users-filter 'email:ch*'
...
# https://developers.google.com/admin-sdk/directory/v1/guides/search-users
or
...
--gws-groups-filter
...
# https://developers.google.com/admin-sdk/directory/v1/guides/search-groupsAlso, validate you are following this instruction to create your Service Account authorizing Directory API action ``
Using OAuth 2.0 for Server to Server Applications
The permission needs by Service Account are:
Hi @snavarro-factorial see the latest comments on issue #49, for sure this is the error that is affecting you too.
this is a duplicate issue of #49
Thanks for the help! I'll reply on this issue since the other is closed.
(Delegation permissions are set up as you said, everything correct from that side)
I cloned the repo with all of the changes and compile it to test with lastest code, just in case, and this was the result:
$ ./idpscimcli gws groups list \
--gws-service-account-file credentials.json \
--gws-user-email xxxxxxxx@factorial.co \
--gws-groups-filter 'name=SysAdmin'
INFO[0000] 1 groups found
INFO[0000] [
{
"email": "xxxxxx@factorial.co",
"etag": "xxxxxxxx",
"id": "xxxxxxxxxxx",
"name": "SysAdmin"
}
]
That actually works, and if I remove the filter, it actually shows all groups. gws users list does also work, so from that side everything is correct!
But when I try the sync command, it fails:
INFO[0000] starting sync groups codeVersion=main
INFO[0000] getting Identity Provider data group_filter="[name=SysAdmin]"
INFO[0002] getting state data
WARN[0002] no state file found in the state repository, creating this
WARN[0002] syncing from scim service, first time syncing
WARN[0002] reconciling the SCIM data with the Identity Provider data
INFO[0002] getting SCIM Groups
WARN[0003] aws checkHTTPResponse: body: status="401 Unauthorized" statusCode=401
Error: cannot sync groups and their members: error doing the first sync: error getting groups from the SCIM service: scim: error listing groups: statusCode: 401, errCode: 401 Unauthorized, errMsg:
I also tried to do what you said on this comment:
#49 (comment)
But:
INFO[0000] starting sync groups codeVersion=main
INFO[0000] getting Identity Provider data group_filter="[name=SysAdmin]"
WARN[0001] google: not including xxx@factorial.co to group xxxxxxxxx members due to incorrect status (not ACTIVE)
WARN[0001] google: not including xxx@factorial.co to group xxxxxxxxx members due to incorrect status (not ACTIVE)
WARN[0001] google: not including xxx@factorial.co to group xxxxxxxxx members due to incorrect status (not ACTIVE)
WARN[0001] google: not including xxx@factorial.co to group xxxxxxxxx members due to incorrect status (not ACTIVE)
WARN[0001] google: not including xxx@factorial.co to group xxxxxxxxx members due to incorrect status (not ACTIVE)
WARN[0001] there are no users in the identity provider
INFO[0001] getting state data
WARN[0001] no state file found in the state repository, creating this
WARN[0001] syncing from scim service, first time syncing
WARN[0001] reconciling the SCIM data with the Identity Provider data
INFO[0001] getting SCIM Groups
WARN[0001] aws checkHTTPResponse: body: status="401 Unauthorized" statusCode=401
Error: cannot sync groups and their members: error doing the first sync: error getting groups from the SCIM service: scim: error listing groups: statusCode: 401, errCode: 401 Unauthorized, errMsg:
The emails I ofuscated are all the members from that group, and after errMsg: I just get the help message. Also between incorrect status and (not ACTIVE) goes the actual status, like SUSPENDED as far I could've seen, but in this case is empty.
I tried both syncs with this filter --gws-groups-filter 'name=SysAdmin'.
If I try the master branch without changes and without filters, I get a lot of warnings like these (I suppose they're okay):
WARN[0013] google: not including xxx@xxx.com to group xxxxx members due to incorrect status (not ACTIVE)
WARN[0013] skipping member because is a group, but group members will be included email=xxxxx@factorial.co id=xxxx
WARN[0013] google: not including xxx@factorial.co to group xxxxx members due to incorrect status SUSPENDED (not ACTIVE)
The ones that shows the SUSPENDED are actually from the company, and those who don't get any status, are external users (as opposite from previous log where I was using the filter and no status appeared; those emails were actuall from the company).
And after a long wait, I get this again:
INFO[0597] getting state data
WARN[0597] no state file found in the state repository, creating this
WARN[0597] syncing from scim service, first time syncing
WARN[0597] reconciling the SCIM data with the Identity Provider data
INFO[0597] getting SCIM Groups
INFO[0597] [DEBUG] GET https://scim.eu-central-1.amazonaws.com/xxxxxxxx/scim/v2/Groups
WARN[0597] aws checkHTTPResponse: body: status="401 Unauthorized" statusCode=401
Error: cannot sync groups and their members: error doing the first sync: error getting groups from the SCIM service: scim: error listing groups: statusCode: 401, errCode: 401 Unauthorized, errMsg:
I don't know what else could be happening :/
Thanks a lot!
Oooooh... facepalm
I've been putting the credentials.json info into the service account access key parameter for a week -.-'
Now it works... Sorry a lot.