iOS RSA Decryption Fails with IllegalStateException: Requested tag 'SEQUENCE', received: 'INTEGER'

[emoiroux]

I'm experiencing a runtime exception when trying to decrypt data using RSA (SHA1, OAEP)on iOS, while the exact same implementation works perfectly fine on Android. The decryption process fails with an IllegalStateException

Library Versions:

implementation("dev.whyoleg.cryptography:cryptography-core:0.5.0")
implementation("dev.whyoleg.cryptography:cryptography-provider-optimal:0.5.0")
Kotlin : 2.2.0

Here is my code :

val encryptionAlgorithm: CryptographyAlgorithmId =
CryptographyProvider.Default
.get(RSA.OAEP)
.privateKeyDecoder(SHA1)
.decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.DER, privateKeyPem.decodeToString().pemToDer())
.decryptor()
.decryptBlocking(ciphertext)

The application crashes with the following exception only on iOS:

kotlin.IllegalStateException: Requested tag 'SEQUENCE', received: 'INTEGER'
kfun:dev.whyoleg.cryptography.serialization.asn1.internal.readRequestedTag#internal + 411
kfun:dev.whyoleg.cryptography.serialization.asn1.internal.DerDecoder#beginStructure(kotlinx.serialization.descriptors.SerialDescriptor){}kotlinx.serialization.encoding.CompositeDecoder + 711
kfun:dev.whyoleg.cryptography.serialization.asn1.modules.AlgorithmIdentifierSerializer#deserialize(kotlinx.serialization.encoding.Decoder){}1:0 + 291
kfun:dev.whyoleg.cryptography.serialization.asn1.internal.DerDecoder#decodeSerializableValue(kotlinx.serialization.DeserializationStrategy<0:0>){0§<kotlin.Any?>}0:0 + 3163
kfun:kotlinx.serialization.ContextualSerializer#deserialize(kotlinx.serialization.encoding.Decoder){}1:0 + 559

[whyoleg]

Hey! It's a bit hard to understand why it happens, without the PEM file, but I understand that it's not possible to share it :)
Still, could you share the header of the PEM file? Is it PRIVATE KEY or RSA PRIVATE KEY`?
Also, could you try the following and share the results:

1. decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.PEM,privateKeyPem)
2. decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.PEM.PKCS1,privateKeyPem)

[emoiroux]

Hey,
Maybe it's something with implementation("dev.whyoleg.cryptography:cryptography-provider-optimal"), If I add the iOS provider implementation("dev.whyoleg.cryptography:cryptography-provider-openssl3-prebuilt") and remove optimal then it work fine.

PEM file header is PRIVATE key
-----BEGIN PRIVATE KEY-----
.......=
-----END PRIVATE KEY-----

decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.PEM,privateKeyPem)

kotlin.IllegalStateException: Requested tag 'SEQUENCE', received: 'INTEGER'
kfun:dev.whyoleg.cryptography.serialization.asn1.internal.readRequestedTag#internal + 411
kfun:dev.whyoleg.cryptography.serialization.asn1.internal.DerDecoder#beginStructure(kotlinx.serialization.descriptors.SerialDescriptor){}kotlinx.serialization.encoding.CompositeDecoder + 711

decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.PEM.PKCS1,privateKeyPem)

kotlin.IllegalStateException: Wrong PEM label, expected PemLabel(representation=RSA PRIVATE KEY), actual PemLabel(representation=PRIVATE KEY)
kfun:dev.whyoleg.cryptography.serialization.pem#ensurePemLabel__at__dev.whyoleg.cryptography.serialization.pem.PemContent(dev.whyoleg.cryptography.serialization.pem.PemLabel){}dev.whyoleg.cryptography.serialization.pem.PemContent + 391

[whyoleg]

Maybe it's something with implementation("dev.whyoleg.cryptography:cryptography-provider-optimal")

Yeah, looks like this is an issue inside of apple provider, that's why replacing it with openssl provider solves the issue.

PEM file header is PRIVATE key

Hmm, that's interesting. I feel like, inside the PRIVATE KEY fenced PEM document, the RsaPrivateKey structure is stored, while it should contain PrivateKeyInfo. It could happen, just because of some issue in a third-party tool, or something like that.

Could you try one more thing: replace PRIVATE KEY with RSA PRIVATE KEY in the PEM file and use decodeFromByteArrayBlocking(format = RSA.PrivateKey.Format.PEM.PKCS1,privateKeyPem)?

[emoiroux]

Seems like this is related to PEM. I’m on DER now, so the issue might be with my key ??. Not totally sure, but thanks a lot for the support!

[JesusMcCloud]

maybe PKCS#1 vs PKCS#8 encoding?