When an additional signing key is added to the operator, users on accounts issued by it observe Authorization Violation
Closed this issue · 4 comments
kimjarvis commented
Following the documented Signing Keys example we add a new signing key to the operator. We then create an account and sign it with the operator signing key. We then create a user and sign it with the account key. We generate credentials for the user. We push the accounts to the NATS server. Everything works as described in the documentation but the user is not authorized to publish messages.
root@serverj1:~# nsc list keys
+------------------------------------------------------------------------------------------+
| Keys |
+--------+----------------------------------------------------------+-------------+--------+
| Entity | Key | Signing Key | Stored |
+--------+----------------------------------------------------------+-------------+--------+
| SO3 | OA263OMTDTQGQJ44XWV4KZKEOP2IC447CDBBEVOD5NSHVQXNHBC5OZIA | | * |
| SO3 | ODBPZV7BODFXRKNKXFOU3BDSO3TZ3M2DHFA2HN6MLGH54GHGF7JWUC6N | * | * |
| SYS | ACMCERILIWN6MX57P27NZHC7HIM23YHPSLEBPFB7NQ5NIIMQIQTYL6AH | | * |
| SYS | AAFAGPTPMW2CS46KCSODW34OSST3ITAG7QFR3C2MDYN6ZUYOO3V5OWG3 | * | * |
| sys | UBCQ72B2IUCY3BEGZ7R7H2NPIE6Q7QDXYRQCJFSPTESWTTUMPSIBS44O | | * |
+--------+----------------------------------------------------------+-------------+--------+
root@serverj1:~# nsc generate nkey -o --store
OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3
operator key stored /root/.local/share/nats/nsc/keys/keys/O/BN/OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3.nk
root@serverj1:~# nsc edit operator --sk OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3
[ OK ] added signing key "OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3"
[ OK ] edited operator "SO3"
root@serverj1:~# nsc add account SA3 -K /root/.local/share/nats/nsc/keys/keys/O/BN/OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3.nk
[ OK ] generated and stored account key "AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N"
[ OK ] added account "SA3"
root@serverj1:~# nsc generate nkey -a --store
AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T
account key stored /root/.local/share/nats/nsc/keys/keys/A/AY/AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T.nk
root@serverj1:~# nsc edit account SA3 --sk AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T
[ OK ] added signing key "AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T"
[ OK ] edited account "SA3"
root@serverj1:~# nsc add user -a SA3 -n SU3
[ OK ] generated and stored user key "UC6UO3K45S46MJSZUXK7DFHKOCNFFNG3F547WICNTOQUO4JM3KJM7P7Y"
[ OK ] generated user creds file `~/.local/share/nats/nsc/keys/creds/SO3/SA3/SU3.creds`
[ OK ] added user "SU3" to account "SA3"
root@serverj1:~# nsc generate creds --account SA3 --name SU3 --output-file SU3.creds
[ OK ] wrote credentials to `SU3.creds`
Success!! - generated `SU3.creds`
root@serverj1:~# nsc push -A -u nats://10.20.193.22
[ OK ] push to nats-server "nats://10.20.193.22" using system account "SYS":
[ OK ] push SA3 to nats-server with nats account resolver:
[ OK ] pushed "SA3" to nats-server NCHF5NIYH5FXRRNQ67I2MTVBLSA52S4UEBOVRTLHTGCTPOSPLZGNMRA7: jwt updated
[ OK ] pushed to a total of 1 nats-server
[ OK ] push SYS to nats-server with nats account resolver:
[ OK ] pushed "SYS" to nats-server NCHF5NIYH5FXRRNQ67I2MTVBLSA52S4UEBOVRTLHTGCTPOSPLZGNMRA7: jwt updated
[ OK ] pushed to a total of 1 nats-server
root@serverj1:~# nats --creds SU3.creds -s nats://10.20.193.22 pub hello world
nats: error: nats: Authorization ViolationThe relationships between the user, account and operator issuers and the signing keys appear similar to those in the example.
root@serverj1:~# nsc list keys
+------------------------------------------------------------------------------------------+
| Keys |
+--------+----------------------------------------------------------+-------------+--------+
| Entity | Key | Signing Key | Stored |
+--------+----------------------------------------------------------+-------------+--------+
| SO3 | OA263OMTDTQGQJ44XWV4KZKEOP2IC447CDBBEVOD5NSHVQXNHBC5OZIA | | * |
| SO3 | ODBPZV7BODFXRKNKXFOU3BDSO3TZ3M2DHFA2HN6MLGH54GHGF7JWUC6N | * | * |
| SO3 | OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3 | * | * |
| SA3 | AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N | | * |
| SA3 | AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T | * | * |
| SU3 | UC6UO3K45S46MJSZUXK7DFHKOCNFFNG3F547WICNTOQUO4JM3KJM7P7Y | | * |
+--------+----------------------------------------------------------+-------------+--------+
root@serverj1:~# nsc describe operator SO3
+----------------------------------------------------------------------------------------+
| Operator Details |
+-----------------------+----------------------------------------------------------------+
| Name | SO3 |
| Operator ID | OA263OMTDTQGQJ44XWV4KZKEOP2IC447CDBBEVOD5NSHVQXNHBC5OZIA |
| Issuer ID | ODBPZV7BODFXRKNKXFOU3BDSO3TZ3M2DHFA2HN6MLGH54GHGF7JWUC6N |
| Issued | 2023-03-25 12:10:12 UTC |
| Expires | |
| Account JWT Server | nats://10.20.193.22:4222 |
| Operator Service URLs | nats://10.20.193.22:4222 |
| System Account | ACMCERILIWN6MX57P27NZHC7HIM23YHPSLEBPFB7NQ5NIIMQIQTYL6AH / SYS |
| Require Signing Keys | true |
+-----------------------+----------------------------------------------------------------+
| Signing Keys | ODBPZV7BODFXRKNKXFOU3BDSO3TZ3M2DHFA2HN6MLGH54GHGF7JWUC6N |
| | OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3 |
+-----------------------+----------------------------------------------------------------+
root@serverj1:~# nsc describe account SA3
+--------------------------------------------------------------------------------------+
| Account Details |
+---------------------------+----------------------------------------------------------+
| Name | SA3 |
| Account ID | AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N |
| Issuer ID | OBNS74JCHUYMPO7V7364ZYELJVPYL2R345Z6LZLVF3E3C3HZV423UQP3 |
| Issued | 2023-03-25 12:11:28 UTC |
| Expires | |
+---------------------------+----------------------------------------------------------+
| Signing Keys | AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T |
+---------------------------+----------------------------------------------------------+
| Max Connections | Unlimited |
| Max Leaf Node Connections | Unlimited |
| Max Data | Unlimited |
| Max Exports | Unlimited |
| Max Imports | Unlimited |
| Max Msg Payload | Unlimited |
| Max Subscriptions | Unlimited |
| Exports Allows Wildcards | True |
| Disallow Bearer Token | False |
| Response Permissions | Not Set |
+---------------------------+----------------------------------------------------------+
| Jetstream | Disabled |
+---------------------------+----------------------------------------------------------+
| Imports | None |
| Exports | None |
+---------------------------+----------------------------------------------------------+
root@serverj1:~# nsc describe user SU3
+---------------------------------------------------------------------------------+
| User |
+----------------------+----------------------------------------------------------+
| Name | SU3 |
| User ID | UC6UO3K45S46MJSZUXK7DFHKOCNFFNG3F547WICNTOQUO4JM3KJM7P7Y |
| Issuer ID | AAYKCJFQHOT6NUDVPYGW3F232K2JLADALKYDDF2IPZZUBZCCADCNR66T |
| Issuer Account | AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N |
| Issued | 2023-03-25 12:11:42 UTC |
| Expires | |
| Bearer Token | No |
| Response Permissions | Not Set |
+----------------------+----------------------------------------------------------+
| Max Msg Payload | Unlimited |
| Max Data | Unlimited |
| Max Subs | Unlimited |
| Network Src | Any |
| Time | Any |
+----------------------+----------------------------------------------------------+The error is reported in the log.
[18825] 2023/03/25 05:13:38.037095 [DBG] Account [AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N] fetch took 143.847µs
[18825] 2023/03/25 05:13:38.037642 [DBG] 9.20.193.83:50002 - cid:7 - Account JWT lookup error: account validation failed
The problem appears to be that the NATS server does not know about the new signing key added to the operator and consequently does not trust accounts issued by it.
aricart commented
Note the operator was not pushed to the server - you may need to do a nsc push -A for that (it may be that it is not supported since typically the operator JWTs are part of the server configuration).
kimjarvis commented
The ns push -A (shown above) reports pushing both accounts SYS and SA3. The debug information indicates that the accounts were both pushed.
[18825] 2023/03/25 13:26:16.809582 [DBG] jwt updated - AAC75Z5AOBVYOKVFGSNXZBBZE7ISIGVR2WPFOKYEFRXZE4WPWYQBTG2N
[18825] 2023/03/25 13:26:17.810495 [DBG] jwt updated - ACMCERILIWN6MX57P27NZHC7HIM23YHPSLEBPFB7NQ5NIIMQIQTYL6AH
Pushing operator changes using the NATS server does not appear to be supported.
- There is no indication that the operator changes are being pushed to the NATS server.
- There is no mention of pushing operator changes in the source
cmd/push.go.
This ticket is for documentation changes.
The Signing Keys example in the documentation describes how to set up an environment that is not useful. That is, users observe Authorization Violation. The signing keys documentation should instead describe how to add multiple signing keys to an account.
The In Depth JWT Guide should be edited to clarify the role of operator signing keys.
The Import Operator - Self Service Deployment Modes (steps 1-4) describe how to generate an operator signing key. We found that the environment that these steps set up is not useful. That is, users observe Authorization Violation.
aricart commented
Yes the missing piece here is exactly that and that needs to be made clear
aricart commented
Fixed by nats-io/nats.docs#584