Sign.createSignatureData failing with AWS KMS signature
Closed this issue · 5 comments
Implementing a HSMRequestProcessor requires to return the Sign.SignatureData which can be retrieved from the signature via Sign.createSignatureData(ECDSASignature sig, BigInteger publicKey, byte[] messageHash).
When implementing such a processor using KMS to sign the message, I experience a "Could not construct a recoverable key. Are your credentials valid?" (Line 89, Sign.class) or occassionally a "Invalid point compression" further down the stack. The reason for why the error changes without change of code is not known.
As can be checked in the below code, the signature is being verified in a separate KMS request. Additionally the public key is also retrieved from KMS to make sure it is the same one as in HSMPass
`
override fun callHSM(dataToSign : ByteArray, hsmPass: HSMPass): Sign.SignatureData? {
val dataHash: ByteArray = MessageDigest.getInstance("SHA-256").digest(dataToSign)
// Create the SignRequest for AWS KMS
val signRequest = SignRequest()
.withKeyId(kmsKeyId)
.withMessage(ByteBuffer.wrap(dataHash))
.withMessageType(MessageType.DIGEST)
.withSigningAlgorithm(SigningAlgorithmSpec.ECDSA_SHA_256)
val getPublicKeyRequest : GetPublicKeyRequest = GetPublicKeyRequest().withKeyId(kmsKeyId)
val publicKeyResponse = kmsClient.getPublicKey(getPublicKeyRequest)
val publicKeyBuffer = publicKeyResponse.publicKey
val keyBytes = ByteArray(publicKeyBuffer.remaining())
publicKeyBuffer.get(keyBytes)
val publicKeyAsInt = BigInteger(keyBytes)
//checking for sanity
if(publicKeyAsInt != hsmPass.publicKey){
throw RuntimeException("Public keys are different")
}
// Sign the data using AWS KMS
val signResult: SignResult = kmsClient.sign(signRequest)
val signatureBuffer: ByteBuffer = signResult.signature
// Convert the signature to byte array
val signBytes = ByteArray(signatureBuffer.remaining())
signatureBuffer.get(signBytes)
//verifying signature on kms for sanity
val verifyRequest : VerifyRequest = VerifyRequest()
.withKeyId(kmsKeyId)
.withMessage(ByteBuffer.wrap(dataHash))
.withMessageType(MessageType.DIGEST)
.withSigningAlgorithm(SigningAlgorithmSpec.ECDSA_SHA_256)
.withSignature(ByteBuffer.wrap(signBytes))
val verifyRequestResult: VerifyResult = kmsClient.verify(verifyRequest)
if(!verifyRequestResult.isSignatureValid){
throw RuntimeException("Signature not valid (KMS)")
}
val signature = CryptoUtils.fromDerFormat(signBytes)
return Sign.createSignatureData(signature, hsmPass.publicKey, dataHash) //error happens here
}
`
The service calling this processor is identical to the one implemented by Web3j (TxHSMSignService), although I did comment the line
byte[] messageHash = Hash.sha3(encodedTransaction);
as I am manually digesting the message using sh256 in the processor.
I'm suspecting the issue could be related to the format or wrong manipulation of the keys.
I was surprised to see k in
BigInteger k = recoverFromSignature(headerByte, sig, messageHash);
to be quite smaller in size than the PublicKey, which results in the recId == -1.
If anyone knows of a working solution using KMS to sign transaction I'd appreciate the input.
For your information, the KeySpec I am using is ECC_SECG_P256K1 with Signing Algo ECDSA_SHA_256
@philipbeaucamp I look in to it today, don't have a AWS env to test it but I checked some snippets with aws sdk which worked in the past for it. Web3j does provide support for KMS, or at least it did for the previous SDK versions.
-
Please try to replace SignResponse and run a test. From what I see on the SignRequest has signature().asByteArray() method support, with this there is no need for the conversion, also this is what we used in the past.
-
From what I see in the documentation ByteBuffer has the https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#array() method, try to use this too and let me know.
Please try to replace SignResponse and run a test. From what I see on the SignRequest has signature().asByteArray() method support, with this there is no need for the conversion, also this is what we used in the past.
I dont quite understand, the SignRequests doesn't hold a signature(). The SignResult has, which I did convert to a ByteArray using your below comment.
From what I see in the documentation ByteBuffer has the https://docs.oracle.com/javase/7/docs/api/java/nio/ByteBuffer.html#array() method, try to use this too and let me know.
Tried this but didnt change anything.
Since I am successfully verifying the signature with the VerifyRequest to KMS, I dont think the convertion of the signature itself is the issue.
@philipbeaucamp, sorry it was SignResponse, this is an old code snip with SignResponse:
public class AWSECKeyPair extends ECKeyPair {
private static final KmsClient CLIENT = KmsClient.create();
private final String keyId; //AWS KMS KeyId;
private final BigInteger publicKey;
public AWSECKeyPair(String keyId) {
super(null, null);
this.keyId = keyId;
byte[] derPublicKey = CLIENT
.getPublicKey((var builder) -> {
builder.keyId(keyId);
})
.publicKey()
.asByteArray();
byte[] publicKey = SubjectPublicKeyInfo
.getInstance(derPublicKey)
.getPublicKeyData()
.getBytes();
this.publicKey = new BigInteger(1, Arrays.copyOfRange(publicKey, 1, publicKey.length));
}
@Override
public BigInteger getPrivateKey() {
throw new UnsupportedOperationException("Private key is not accessible when using AWS KMS");
}
@Override
public BigInteger getPublicKey() {
return publicKey;
}
@Override
public ECDSASignature sign(byte[] transactionHash) {
SignResponse sign = CLIENT.sign((var builder) -> {
builder.keyId(keyId)
.messageType(MessageType.DIGEST)
.message(SdkBytes.fromByteArray(transactionHash))
.signingAlgorithm(SigningAlgorithmSpec.ECDSA_SHA_256);
});
ASN1Sequence instance = ASN1Sequence.getInstance(sign.signature().asByteArray());
ASN1Integer r = (ASN1Integer) instance.getObjectAt(0);
ASN1Integer s = (ASN1Integer) instance.getObjectAt(1);
return new ECDSASignature(r.getValue(), s.getValue()).toCanonicalised();
}
}
@gtebrean I am pretty sure at this point the issue is with the formatting of the KMS keys.
AWS states regarding the return type of the public key:
The value is a DER-encoded X.509 public key, also known as SubjectPublicKeyInfo (SPKI), as defined in RFC 5280.
Since the key seems to have some additional meta data, it looks like some manipulation is necessary. Just using "publicKey" (see code below) will actually result in an error saying that the public key (BigInteger) is too long (182 characters), but less than or equals 128 are expected. I solved this issue with the approach below but it still fails at the same place, trying the compare the publickey retrieved from the signature with the public key from KMSKeyPair.
For your information , regarding the private key, AWS KMS Documenation states
When used with the ECDSA_SHA_256, ECDSA_SHA_384, or ECDSA_SHA_512 signing algorithms, this value is a DER-encoded object as defined by ANSI X9.62–2005 and RFC 3279 Section 2.2.3
`
class KMSKeyPair(
val kmsClient: AWSKMS,
val kmsKeyId: String
) : ECKeyPair(null, null) {
private val publicKey: BigInteger
init {
val getPublicKeyRequest : GetPublicKeyRequest = GetPublicKeyRequest().withKeyId(kmsKeyId)
val publicKeyResponse = kmsClient.getPublicKey(getPublicKeyRequest)
val publicKeyBuffer = publicKeyResponse.publicKey
val publicKey = publicKeyBuffer.array()
val keyFactory = KeyFactory.getInstance("EC")
val keySpec = X509EncodedKeySpec(publicKey)
val pubKey = keyFactory.generatePublic(keySpec) as ECPublicKey
// Get the elliptic curve point (X and Y coordinates)
val ecPoint = pubKey.w
val x = ecPoint.affineX.toByteArray() // X coordinate
val y = ecPoint.affineY.toByteArray() // Y coordinate
// Ensure X and Y are 32 bytes each (padded if needed)
val xPadded = x.copyOfRange(maxOf(0, x.size - 32), x.size) // Ensure 32 bytes
val yPadded = y.copyOfRange(maxOf(0, y.size - 32), y.size) // Ensure 32 bytes
this.publicKey = BigInteger(xPadded + yPadded)
}
override fun getPrivateKey(): BigInteger {
throw UnsupportedOperationException("Private key is not accessible when using AWS KMS")
}
override fun getPublicKey(): BigInteger {
return publicKey
}
override fun sign(transactionHash: ByteArray): ECDSASignature {
// Create the SignRequest for AWS KMS
val signRequest = SignRequest()
.withKeyId(kmsKeyId)
.withMessage(ByteBuffer.wrap(transactionHash))
.withMessageType(MessageType.DIGEST)
.withSigningAlgorithm(SigningAlgorithmSpec.ECDSA_SHA_256)
val signResult: SignResult = kmsClient.sign(signRequest)
val signatureBuffer: ByteBuffer = signResult.signature
// Convert the signature to byte array
val signBytes = signatureBuffer.array()
val signature = CryptoUtils.fromDerFormat(signBytes)
return signature.toCanonicalised()
// val instance = ASN1Sequence.getInstance(signBytes)
// val r = instance.getObjectAt(0) as ASN1Integer
// val s = instance.getObjectAt(1) as ASN1Integer
// val signature2 = ECDSASignature(r.value, s.value).toCanonicalised()
// return signature2 //r is the same as signature, s is different ?
}
}
`
Edit: For your information, the publicKey created above matches 1:1 with your approach:
val publicKey = SubjectPublicKeyInfo.getInstance(derPublicKey).publicKeyData.bytes val cleanPublicKey = BigInteger(1, Arrays.copyOfRange(publicKey,1,publicKey.size))
Found the issue(s)!
For anyone in the future stumbling across this, make sure to
- Set KeySpec to ECC_SECG_P256K1 (I was using multiple keys, one of which was ECC_NIST_P256)
- Decode the public key received from KMS by using
val publicKey = SubjectPublicKeyInfo.getInstance(derPublicKey).publicKeyData.bytes
and
val publicKeyBigInt= BigInteger(1, Arrays.copyOfRange(publicKey,1,publicKey.size))
or by using X509EncodedKeySpec as I did in my previous comment.
It was both these issues together that led to the signature not being able to align with the public key provided by AWS.
Apart from that it was interesting to see the ECKeyPair approach posted by @gtebrean as I had been using the HSMRequestProcessor before. Both approaches work almost identically (the requests to KMS are essentially the same code), but I think I prefer the ECKeyPair approach for simplicity, as most people are probably used to working with KeyPairs.